February 17, 2008

CS3 ACE Exam Bewilderment

I've never really cared much for the Adobe Certified Expert exam itself. I mean, I totally understand that there should be some qualifier for those who choose to make their profession through the use of Adobe software.

I have long lamented that while doctors and lawyers have always been able to maintain some sense of "ownership" of their field -- meaning entering the field requires a large investment of both time and money, there are numerous exams and boards in place, and on some levels, a degree of apprenticeship is required. Those professions even have their own language. The design field requires not much more than a computer and a business card printed on a micro-perfed sheet of paper. Even the language we once owned (font, point size, kerning, etc) has become known by all.

Because of this, if you are either in a position where you are looking to be hired by a professional design agency or production house, or if you are looking to employ a designer or operator, how does one go about demonstrating their knowledge of the necessary products? How does one trust that a potential hire actually knows what they are doing? In my own profession of training or consulting, how do potential clients sleep at night, wondering if the person they are hiring to train their staff knows what they are talking about. In fact, I myself only take the exams because form a pure marketing perspective, my clients feel more comfortable hiring a certified trainer, rather than an uncertified one.

Case in point: Recently, Adobe (yes, the company that actually makes the software), hired a training company to provide some training for their own employees. Which if you ask me, is pretty smart of Adobe. I mean, just because someone works at a company, doesn't mean those people know how to use the software. For example, not everyone who works at American Airlines knows how to fly a plane, or how a plane actually flies. But when employees understand their own products better, you have to assume that they will probably do a better job in improving their own products and marketing it. While I haven't spoken with Guy Kawasaki about this point, I gotta assume he'd think it should be the MO of a company to do this on some level.

In any case, during some video training, the hired trainer made mention of a video format known as H.263. One of the Adobe employees raised a hand to correct the trainer -- H.264 is obviously the right format. But the trainer insisted that his notes were correct and that the format was H.263. Nice, eh? I guess Adobe is embracing a new standard.

But getting back to my original point, Adobe publishes these exams and has something called ACE certification -- which stands for Adobe Certified Expert. However, with the CS3 exams that have recently been made available, I can argue that the term ACE really stands for Absolute Crap Exams. Maybe I'm being harsh, but I found that taking the Illustrator CS3 test (which I did last night) was a tremendous insult to my own intelligence. For the record, I scored an 81% and passed. The majority of the questions on the Illustrator test made absolutely no sense, and only few presented clear answers. The exam was a "recertification" exam, so there were only about 30 or so questions, and the intention was that I'd be tested on features new to CS3. Well, obviously, that was the biggest joke ever. What are the new features in Illustrator CS3? Let's see... Live Color? Not a single question on that subject. Nothing about Flash integration or symbol support, nor anything at all about the new align or selection features. The majority of the questions I had dealt with sorting files in Bridge, setting the right transparency flattener settings to work with Level 2 RIPs (puh-leez), and working with spot colors. And I really REALLY don't mean to sound like I'm a know-it-all on any level, but I do like to think that someone like me should be able to score higher than an 81% on an Illustrator exam.

And if you think it's just Illustrator, read about what both Anne Marie Concepcion and Michael Murphy have said about their experience with the InDesign CS3 exam.

What bothers me most is that Adobe charges people a heck of a lot of money ($150 per exam, $50 per "upgrade" test) for these exams. And in my eyes, these exams are totally worthless. Both to trainers, employees, and employers.


Well, you know me -- I'm not one to complain much -- my mentor at Adobe, Dave Burkett, used to tell me, "If you don't have a solution, you're part of the problem", meaning, don't criticize unless you also make a suggestion on how to fix things (not to be political in any way, but if only our public figures would heed such advice).

Back when I was doing art production, I would actually create a document and have potential employees try to recreate that document within a specified period of time. Even analyzing the potential candidate during the exam would reveal their true knowledge of the product, by watching which methods they were using to complete the task. In my opinion, that exam has some value. Not only because it tests real use of the product, but because life isn't a multiple-choice exam. And anyone who uses Adobe's products know that there's always a plethora of ways to accomplish a task. And memorization of features (which is necessary for the current exam) shows nothing of a person's ability to understand when and how to use those features. Nor their ability to understand a real-world workflow.

If you're familiar with the CompTIA CTT+ certification, you'll know that TWO tests are required for certification: a written test, and a video test. That's right -- you have to submit a video of yourself giving a class. Your video is reviewed and only upon passing both exams are you certified as a technical trainer. After all, just because someone is a whiz at understanding technology doesn't mean they also know how to clearly present that material for others to understand.

I know that there are numerous training materials out there -- books, videos, and blogs for example. These are wonderful, but it's rare that you'd put a note on your resume that you've got a dog-eared copy of Real World Illustrator CS3 on your desk and that you read my the Real World Illustrator blog religiously. There are also amazing conferences and seminars. While I'm a bit biased here because I regularly teach at these conferences, I also attribute a large part of my success to actually being at these events. I've networked with amazing people, I've learned things I never would have even had the time to explore before, and there's really no better way to learn than also having fun doing it at the same time.

So the problem is, there's a disconnect. A disconnect between really good training, and a really good certification standard. Because you can spend money to get great training, but there's no great way to advertise that you've mastered that training. And you can spend money on getting a recognized "Adobe" authorized training certificate, but that in reality has no training value. Most people probably have to shell out cash for both -- books, videos, conferences and seminars to really learn the stuff, and meaningless exams to "prove" your expertise to the world. Do you see something wrong with this? I do.

So I find myself in a position where maybe I could find the resources to make a change. I already create books, as do many of my friends. I already record videos, as do many of my friends. I already participate heavily in conferences and seminars, and have begun building a membership community that I hope will continue to grow. Just like 3rd-party training is a valuable and credible resource, could a 3rd party endorsement or certification ALSO prove its value?

Let me ask you. What do you think? If I could somehow find a way to truly qualify the expertise of those who use Adobe software, would that be something valuable -- both from an employee and employer perspective? I'd love to hear your thoughts on this matter.


Rene said...

Qualifying expertise would be a good idea in principle, but in reality it would be hard to implement. For some applications it would be easier, but nearly impossible for Illustrator/Photoshop.

For both apps, there are so many distinct fields of expertise. In my case, I love color correction and color management, and consider myself very proficient with PS in this area. However, ask me to design something and, well, I suck.

My point is that you can be a genuine expert in PS or AI but only in a specific area. And no test can account for all these areas -- and even if it did, very people would pass a "real" test. You could do a test per area of expertise, but that is a can of worms that can quickly get out of hand.

Mordy Golding said...

That's a great point Rene. Even I have my "comfort zones" within the applications I use. I can also imagine that package designers who work with spot colors and ink coverage all day could care less about the web-specific features in AI. Such a person might fail a test, but that isn't indicative of his high level of expertise in his field. So are we just left with a mediocre middle road?

Your comment about testing per expertise intrigues me though. Maybe such a test would cover Adobe Creative Suite in general and not specific apps (who uses just ONE app these days anyway?). You might be able to have a basic layer of coverage across the apps, but offer questions specific to certain workflows on another layer. Kind of the way college credits work -- you need pre-requisites, and then you choose an area to focus your strengths on...

So a test might give you a certification that states "solid knowledge on Adobe Creative Suite: Web Focus" for example... you might not cover all types of work, but maybe enough of the main ones to be successful.

khiltd said...

I just took the Photoshop CS3 recert and found it to be even more ridiculous than that. I'd say over half of the questions on it pertained to the 3D and video editing features which are only available in the extra-special super-deluxe mega-expensive version of Photoshop for science people with government budgets. If Adobe considers any of that feature bloating nonsense important enough to test me on, then they should consider it important enough to give to everyone who buys a copy of Photoshop. Mine has none of those things in it, so I obviously had no idea what they were even talking about.

And of course there were plenty of questions about the stupid cell phone previewing app which I tried to use once but never looked at again because it was impossible to create profiles for devices Adobe hadn't decided to support. Is any of this really more relevant to one's expertise in Photoshop than the new features of the Camera Raw plugin, or does Adobe just want to make sure that we know this worthless app is on our disk taking up space?

There was one question about "which tool would you use to make a bunch of planes in the sky?" that made no sense to me, but my absolute favorite was something along the lines of "If you were going to print a magazine, which color mode would you use?" One would think that a company such as Adobe, being on the forefront of semi-adequate color management implementations, would know that there is no correct answer to such a ludicrous question, but nope. There it was.

I picked "CMYK" because that would have been the answer 16 years ago when I first started using Photoshop. Who knows if it was right. I passed and that's all that really matters. To whom, I'm not sure. It is genuinely difficult to test anyone on anything other than their ability to memorize trivial information from a product manual, but I don't think they're even trying anymore. It was hands down the worst ACE test I've ever seen, and the certificate they sent me had some rather embarrassing typesetting sins to boot.

Gary Spedding, Ph.D. said...

I post this coment knowing that I will never take the ACE exams (on any app)but also with the certainty that I have and do know how to set, grade and take exams.

I tried to post a similar comment at Michael Murphy's site but it did not go through. It seems (I know several experts out there) that 80% is a typical score for most. That is not at issue and is a good passing grade.

However, it is clear that Adobe have absolutely no idea at all about setting examinations - nor have they a clue when it comes to assessing a person's abilities. An ACE needs to know enough about the programs to be able to teach others (sure the little quirks that might never bug anyone are good to know by the experts and Adobe should point out these features in training for the ACE exams)and to have teaching skills.

It is absolutely clear to me (and others) that Adobe needs to let you experts set the exams and leave well enough alone their own efforts to alienate the industry and potential new customers of their products.

One can only hope that David, Anne-Marie, Jeff Witchel, Michael Murphy, Mordy and other well known and respected experts have made abundantly clear the very serious shortcoming in Adobe's education/teaching abilities.

Adobe - stick to making the software and listening to your ACE's and clients when in comes to improving the software. They are the ones who use it and the ones who need to know what is important and what is not on a day to day use basis.

Rudi Spitzers said...

Good point Mordy.
I've taken these exams for many years now. I'm an ACE in InDesign, Illustrator and Photoshop.
Big deal. You know the trick, just study the Help files and you most likely will pass. I know a lot of ACE's who know every little button and option in Photoshop. But don't ask them to make a usable mask, path or alpha channel. They know it's in the application, they know which buttons to push. But that's not what it's all about, is it? It's not about choosing an option and knowing what it does it's about knowing when and why to use it.
I have been an Adobe presenter for years and I'm an instructor in Adobe applications since Photoshop 2.5 and Illustrator 88 and a graphic designer. But I still have to do these exams to prove that I know what I'm talking about.
I was planning to do the new exams but now I think I won't.
I think the level of exams should be much higher. I also think that you should do some kind of test to show that you can really use the application.
Teaching or presenting is something very different. It's about working with people not just software. That's something which is really difficult to test. But why would you? If you're a bad teacher you won't get any assignment and no one will hire you, will they?

khiltd said...

From my experience, the worse you are at teaching the better your chances of getting a teaching gig. It's not what you know, but who, etc, etc, etc. If you're really good at shaking hands at the right parties, you'll never have trouble getting gigantic corporate gigs where the students are too disinterested to care whether or not you know what you're talking about. I remember when I sat through Apple's official week-long Cocoa class which my department paid nearly $2,000 to send me to the instructor was some old NeXT expat who had never actually seen OS X before.

As for the value of certifications in general, it's tough for me to believe that anybody anywhere takes them all that seriously. To quote myself: "Certifications are a marketing tool, same as any other. They allow a small company without a Superbowl sized ad budget to suckle a split second of legitimacy from the bloodstream of a much larger company. Like the badge on a mall cop, they're there to inspire a reflexive reaction--keeping that mental door open just long enough to wedge a foot into in the hopes that someone will listen to you when you tell them to stop flicking Dippin' Dots at the girl behind the Cinnabon cart. But, also like the badge on a mall cop, they are in no way indicative of genuine authority."

Ben said...

You all know why it's important to get certify or else you wouldn't have taken it.

Adobe's logo is a visible label of you being a professional using their products. Sure, it's a marketing tool and not a true measure of your knowledge nor expertise. The same argument can be made of having a college degree or not, but wouldn't you have more trust in someone with one?

khiltd said...

Considering the number of CS graduates I've supervised who didn't know the first thing about programming as well as the volume of alumni for whom basic 6th grade English grammar is a pipe dream, I personally would not trust anyone on the basis of a college degree, no.

I have no delusions as to why I keep paying for the chance to be assigned a numeric rating by an unattended computer at a trade school learning annex, I just think that these tests should be free of charge if all they're going to do is covertly market Adobe's less successful products to us.

The CS1 test was full of irrelevant questions about ImageReady; another ailing product which got the axe shortly thereafter. Bridge continues to be ignored by everyone despite the fact that they put launcher buttons for it in every other application, so they make sure and throw a lot of questions about that in there, too. Now they're pimping the cell phone thing nobody cares about and the answer to the question nobody was asking that is "Photoshop Extended." What's in store for CS4? I think there might eventually be a question like this:

Given the following choices, which option would result in the greatest customer satisfaction and make you happy all the time?

A. Clicking the FedEx Kinko's button to send your Adobe Creative Suite document to the FedEx Kinko's store of your choice.

B. Taking your Adobe Creative Suite documents directly to your nearest FedEx Kinko's location where one of our knowledgeable sales associates will listen carefully to your needs and provide you with the excellent customer service you've come to expect from FedEx Kinko's.

C. Asking about the special promotional pricing available to Adobe customers at FedEx Kinko's.

D. Signing up for the new Adobe Gold Rewards Card which offers members 2% cash back on all Adobe Certified Expert exams as well as additional discounts on the already low everyday prices available at any FedEx Kinko's store.

Chad Chelius said...

Hello all. I just took and passed the InDesign exam about two weeks ago and found it just as horrible as what Mordy described in this post. I'll be taking the Illustrator exam next week and recently downloaded the prep guide for Illustrator. Is it me, or are the topics for the areas of study totally off-base? It actually looks like they took the Photoshop prep guide and filled in new content without actually changing the headings. For instance, topic 2 is Painting and Retouching. Retouching? In Illustrator?

Aongus Collins said...

Great post! I passed the ACE exam in Illustrator 10 and never bothered to recertify since my business doesn't require it. It should be called Unreal World Illustrator, since it assesses primarily the candidate's ability to memorise and regurgitate chunks of the manual and even the contents of preference screens or dialogue boxes.

Yet I got a lot out of the exercise. It spurred me to study good books and training videos, exploring aspects of the software I'd previously skipped over.

It's highly ironic that the exam is based on the least impressive training aid available, i.e. the Adobe manual.

Any chance that Adobe might outsource its exams and training to professionals working in that field?

Cheryl said...

Thanks, Mordy — you just saved me 150 bucks!

Xpectro said...

Hey Mordy, greetings from Colombia... maybe you remember me when we were at Adobe last year...

This subject has always puzzled me, since I have been teaching, using and showing the tools for so long and everyday I'm more amazed at how people use them and get interesting results.

I think there is a huuuge difference between lawyers and designers: you have to know the law to apply them. But in our business chance is really important. Experimentation leads to amazing results... and that is something you can't show in tests like this.

I think portfolios and reels are more important for us. To try to regulate how good we are is like trying to get Picasso a seal of approval based on his use of brushes.

I really think the tests are confusing if you use many tools (not to mention previous versions of the same tool) and irrelevant if they don't test your skills on solving things.

THAT would be an interesting way to test. Not only memory but skills (imagine results based on solutions, that measure your speed and creativity using the tool. That would be really useful!)

Xpectro said...

PS: Nice time stamp for my comment! 3:00am, 03.03

Sorry... couldn't resist it... ;-)

siotha said...

My last experience with the Adobe ACE exam left me with the - gees who's using that feature. i think the test should be geared towards the type user. Adobe obviously knows who and what they are, but why have an ACE (especially since they've put out a new - i know how to use Adobe products exam) - why not have photoshop for print specialist, photoshop for photo retouching, photoshop for animation & 3-D (as it sound to me from your post those features are exclusive the the PS Extended app.).. InDesign for Book creation, InDesign for PDF interactivity, etc... Go for the areas that people work in the application in!! Not jsut off the latest feature list! Come on! And my other complaint is - they just now got the exams? I would say - have the exams within a month after launch! waiting this long is like an eternity in the software world.
But Alas, I love their products... and their makers are only human.... And I will forever be a huge fan!!!

Ben said...

Any test/examine are supposed to encourage end user to study the subject matter before hand. The purpose isn't to test your memory skills, even though it seems like that's the main goal. Tests such as ACE isn't suppose to test your skills nor resourcefulness. You develop those skills on your own.

A designer with a 4-year design degree only translates to "he/she was train to THINK creatively" but does not mean such person actually IS.

Going back to my analogy of having a college degree, the degree itself is more symbolic than actually representations of one's knowledge. Likewise, it's the same here.

Anonymous said...

I have a question... my colleagues and I are going to be taking the recert exam for CS3 shortly (we are all ACE for CS2). My question is if anyone fails, is there a waiting period before they can re-take the test? I ask because I think we only have until 4/30 to get the simpler upgrade test. Thank you.

Andreas said...

Very interesting.

I just passed the Photoshop cs3 resertification and I agree with khiltd that it was ridiculous. Some questions was just too subjective that the answer they wanted would in a specific situation be completely wrong. Photoshop is such a creative program that there is never only one way of doing it.

I do, though, belive that the ACE exams are good but only if everyone knows what it messures. The ACE exams messures how diverse your knowledge is for the program is. Not how innovative nor how creative you are and not how good you are at using the different workflows you can use to create or recreate an image. But if everyone was aware that the ACE exam only messures your diversity of the program knowledge then everything would be fine.
I would like to see more specific exams where you get a problem and then you solve it. Like Recreate this image in 1 hour. Or: You have a project where you need to produce a flyer that needs to go to both a printing press and for an advert on the internet. What is your workflow.
This kind of question would force you to know what the difference is between the color management for internet and printing presses. Also for how you should plan the work so you optimize the workflow for quality and security.

I took the ace exam just to market myself with it and I get the feel that that is the only reason other people also take the aces for.


Ammar Midani said...

Hi Everybody,

great ideas and thoughts, I really support the whole CS exam concept.
would like to make some input thogh..

While back in the day when I thought that passing the exam was the final step, it turned out to be the first if not the basic step for a comprehensive learning of the application.
For me the significance of the exam, comes from the preparing for it, to know some little things, and is more important, is using those in a real world environment.
I think it was meant to fast forward the workflow. As if Adobe is trying to pull you up to a certain standard.
Are ACEs better designers? Absolutely not. You can study for the exam and pass it at 100% score without any design talent at all! Are they better teachers, yes they are, and then are ACIs better instructors with as you said two certificates needed, I don’t think so..
As for the questions I always had the feeling that they are random, I have took the exam for more than 9 times and sometimes as you said they just won’t ask about a particular category! Or concentrate on promoting issues such as the File Browser/ Bridge or even Device manager..for me its always good to know between through an exam or a book.
Using the Adobe logo must be the great attraction on taking the exam, people love Adobe and being related to it in somehow, while on the other hand you find employers who aren’t aware of the certificate itself!
Have heard so many stories about the InDesign exam, from badly worded to complete crap, they said Adobe fixed it.

Anonymous said...

You'll want to add a facebook button to your blog. I just bookmarked this article, although I had to complete it manually. Simply my $.02 :)

- Robson