Tag Archives: GMAT questions

That official GMAT question might cost $3000

 

I spend a lot of time telling GMAT students that no GMAT practice problems can ever be as good as real, retired questions from official GMAT tests and publications. And if you read a bunch of GMAT blog posts written by other GMAT test-prep “experts”, you’ll find that many of them seem to disagree with me – since they’re trying to sell you a nice bundle of non-official GMAT questions and exams.

Sometimes, even my own GMAT students object when I tell them to avoid using knockoff GMAT tests from, say, Manhattan GMAT or Veritas: “How terrible could those materials be? There are smart people working at those GMAT test-prep companies, right?”

Yes, of course there are smart people working at GMAT test-prep companies – though some test-prep firms clearly employ smarter GMAT question-writers than others. But even at their best, no GMAT test-prep company can possibly compete with the precision of real, official GMAT questions. Part of the reason is that copying the style of an intricate standardized test is inherently challenging, but the bigger reason is basic economics.

If you’re reading a GMAT blog, you’re probably a business-savvy future MBA student, and you probably have some intuition for the business models of companies like Kaplan or Manhattan GMAT. So take a guess: how much do you think those test-prep companies spend developing each individual question on their GMAT practice tests or in their GMAT books? Go ahead and think of a number.

OK, got something in mind? Great. Personally, I would imagine that GMAT test-prep companies spend something in the range of $5-30 per “knockoff” GMAT question if they actually want to make a profit, but I could be wrong.

Now let’s ask Dr. Lawrence Rudner, former GMAT Chief Psychometrician, how much the GMAT’s developers spend on each practice question:

Test items are costly to develop, often in the range of US$1,500 to US$2,500 per item.

This comes straight from a paper that Rudner presented at the GMAC Conference on Computer Adaptive Testing way back in 2007. If question development costs have kept pace with inflation, then each GMAT question would cost somewhere between $1,700 to $3,000 per question in 2016, though that figure could be even higher if shortages of trained psychometricians continue to inflate GMAT development costs.

(Incidentally, I got halfway through a PhD in education policy and psychometrics – the statistical science behind standardized testing – before I came to my senses and realized that I’m much happier as a GMAT tutor than as an academic or psychometrician. If you’re interested in a career in psychometrics, send me an email, and I’ll do my best to help out.)

So yeah: that GMAT question you saw on the latest version of the exam? It might have cost $3000 or more to develop – perhaps 100 times more than a GMAT test-prep company could reasonably spend on each question.

And if that sounds batsh*t crazy – or at least wildly inefficient – it probably isn’t. Actual GMAT test items go through a painstaking cycle of editing, testing (often as “experimental questions” on actual GMAT tests), re-editing, and re-testing before they actually become part of a real GMAT exam. For every question that actually makes its way onto the GMAT, countless others are discarded for any number of reasons. For example, there might be evidence that the discarded GMAT questions were phrased badly, that they don’t seem to capture the skills the GMAT promises to measure, or that they (accidentally) discriminate against GMAT test-takers from certain demographic groups. Much of this testing requires substantial skill and training – and employees with these skills certainly aren’t inexpensive.

You probably know that I’m not particularly impressed by the GMAT’s ability to identify the most talented business leaders, but the GMAT’s creators deserve credit for producing subtle and meticulously phrased questions. How could any GMAT test-prep company possibly produce anything of comparable quality or precision, given test-prep firms’ economic limitations?

So whenever you work through practice tests from major GMAT test-prep companies – or GMAT practice worksheets from an obscure little GMAT tutor dude in Colorado – remember to take your results with a huge grain of salt. Sure, our “knockoff” GMAT questions can probably help you build skills, but none of us can ever give you a truly accurate GMAT practice test – simply because none of us will ever be able to afford to do so if we actually want to keep our doors open.

 

Hootie and the (very old) GMAT Paper Tests

Q: What are the GMAT Paper Tests, and why don’t I hear about them very often?

A: Back in the Dark Ages before Steve Jobs and Bill Gates conquered the world, the GMAT was a paper-based test. The GMAT Paper Tests are basically old, retired exams from way back in the day. Each test contains approximately 110 questions, divided into one data sufficiency section, two quant problem solving sections, one sentence correction section, one critical reasoning section, and one reading comprehension section.

You don’t hear about them terribly often because they aren’t very popular. GMAC has sold the GMAT Paper Tests on their website for a long time, but I don’t get the impression that they sell many copies, largely because test-takers have newer and sexier options available, including the GMAT Official Guide, the GMAT Quant Review guide and Verbal Review Guide, the GMATPrep Question Pack, and tons of materials from test-prep companies.

Considering all of the other options out there, the old GMAT Paper Tests—somewhat understandably—don’t seem to register on most test-takers’ radar screens.

Q: How old are the GMAT Paper Tests?

A: Pretty old. Most of them have a publication date of 1995, and a few were initially published even earlier.

In case you don’t remember 1995 very well: back then, a BlackBerry was just a type of fruit, Google didn’t exist yet, and hits by Coolio, Real McCoy (!!), Alanis Morissette (ouch!), Seal, and Hootie & The Blowfish topped the U.S. music charts. Yup, Hootie & The Blowfish.

Back in 1995, the average GMAT score for entering students at Wharton was 650. The median score is 720 now. In 1995, the GMAT was written and administered by ETS; today, the questions are written by the wise Iowans at ACT, and the test is administered by Pearson VUE, which loves palm vein scanners.

In 1995, palm vein scanners didn’t exist.

So yeah: the GMAT Paper Tests are old.

Q: Wait a minute… what the heck is Hootie & The Blowfish?

A: If you have to ask, you probably don’t want to know. Can we get back to the GMAT now, please?

Q: OK, fine. So the GMAT Paper Tests are old. But have GMAT questions changed that much since the 1990s?

A: GMAT questions haven’t actually changed all that much since the 1990s. The old GMAT Paper Tests still include problem solving, data sufficiency, sentence correction, critical reasoning, and reading comprehension, just like the current version of the GMAT. The only major difference is that the GMAT now includes Integrated Reasoning, which may or may not be very important to your MBA goals.

So if you need practice with the core GMAT quant and verbal question types, there’s no reason why you couldn’t use the GMAT Paper Tests. Sure, these questions are old, but they’re not radically different from, say, current GMAT Official Guide questions. And for whatever it’s worth, many of the questions in the GMAT Official Guide are from the 1980s and 1990s, and have appeared in several consecutive editions of the book.

The GMAT is constantly evolving in subtle ways, it would be ridiculous to suggest that 20-year-old GMAT questions are exactly the same as new ones. But if you’re just looking for some extra practice materials, the age of the GMAT Paper Test questions isn’t really a huge problem. They aren’t all that different from the questions you’ll find in the GMAT Official Guide.

Q: If I buy the GMAT Paper Tests, will I recognize some of the questions from the GMAT Official Guide or other GMAT resources?

A: This is where things get a little bit tricky. There is substantial overlap between the GMAT Paper Tests and other GMAT resources, and my best guess is that somewhere around 40% of the GMAT Paper Test questions have appeared elsewhere at some point in the last 20 years.

But “elsewhere” is a funny place when we’re talking about GMAT questions. Since 1995, GMAC has released six different editions of the GMAT Official Guide, and some of the GMAT Paper Test questions have appeared in each edition. GMAT Paper Test questions also pop up in both editions of the Quant Review Guide and both editions of the Verbal Review Guide, as well as the GMATPrep tests and the now-obsolete GMAT PowerPrep tests. So yes, the GMAT Paper Test questions are scattered all over the place.

Even if 40% of the GMAT Paper Test questions appear in other resources, you probably won’t recognize nearly that many, since many of questions appeared only in older editions of the Official Guide or in the thoroughly ancient GMAT PowerPrep tests. My best guess is that only 15-20% of the GMAT Paper Test questions appear in the newest versions of the GMAT Official Guide, GMAT quant and verbal supplements, and the GMATPrep exams.

So the bottom line is that you’ll see an overwhelming number of repeats if you’re a connoisseur of obsolete GMAT resources, but you’ll see only a moderate number of repeats if you’re sticking with the newest versions of the official GMAT books and software.

Q: Is the scoring on the GMAT Paper Tests accurate?

A: The GMAT Paper Tests were very accurate back in 1995, when the test was actually administered on paper. But in the 21st century, the scoring on the GMAT Paper Tests is pretty much irrelevant.

Think about it this way: one of the biggest challenges of the modern GMAT is time management. You can’t go back to review the questions you already answered on the GMAT, so you constantly face a difficult choice: do I keep grappling with a particular question, or should I guess and move on? I would argue that the GMAT timing conundrum is one of the single biggest determinants of your score on the exam.

But on the GMAT Paper Tests, you can move around within each section as much as you want. If you want to change an answer, great. If you want to skip a question and come back to it later, that’s no problem at all.

So the testing experience is completely different. Yes, the GMAT Paper Tests will generate scores on the familiar 200-800 scale, but they don’t mean very much. Despite the name, it’s best to think of the GMAT Paper Tests as practice questions, not as practice exams.

Q: Are the GMAT Paper Tests hard?

A: Because the test wasn’t adaptive back in 1995, the GMAT Paper Tests include an incredibly broad cross-section of questions, ranging from insanely easy to insanely difficult. In theory, the GMAT Paper Tests are much like the GMAT Official Guide: they contain a little bit of everything, and many of the questions will be of limited value to you if you’re either a very weak GMAT student or a very strong one.

If you’re scoring above a 650 on the GMAT, you’ll probably be bored by the majority of GMAT Paper Test questions. Sure, some of the questions will offer you a good challenge, but plenty of them won’t. That doesn’t mean that the GMAT Paper Tests aren’t useful; it just means that high-scoring students will get a thorough workout from perhaps 20-40% of the questions. And that’s probably better than nothing, though you could easily argue that it’s not worth spending the money on the GMAT Paper Tests if you’re a relatively advanced GMAT student.

Speaking of which…

Q: Are the GMAT Paper Tests worth the money?

A: Well, I suppose it depends on your skill level and how much you value $29.99. For that price, GMAC will send you a set of three GMAT Paper Tests, each of which contains roughly 110 questions, give or take a few. So you’re paying a little bit less than a dime per question.

That’s pretty cheap compared to GMAT Focus (which costs more than $1 per question), but a little bit more expensive than the GMATPrep Question Pack (roughly six cents per question), and much more expensive than the GMAT Official Guide and Quant Review Guide (around three or four cents per question, depending on the exact price you pay for the books).

So if we’re talking about an either/or proposition, the GMAT Paper Tests pale in comparison to the newer, cheaper GMAT Official Guides or the GMATPrep Question Pack.

But if you simply need more practice questions—either because you’ve already burned through the newer resources, or because you want to save them for later—then the GMAT Paper Tests aren’t a terrible option. Advanced GMAT students might get less mileage out of the tests, since so many of the questions are relatively easy, but you’ll still get something out of the GMAT Paper Tests.

I don’t know whether you’ll get $29.99 worth of GMAT awesomeness from the tests, but official GMAT questions are never exactly bad for you. So if you have the money and the time to do the GMAT Paper Tests, go for it. But keep in mind that they’re not a particularly good substitute for the Official Guide or the quant/verbal supplements or the GMATPrep Question Pack.