Tag Archives: GMAT test prep

That official GMAT question might cost $3000

 

I spend a lot of time telling GMAT students that no GMAT practice problems can ever be as good as real, retired questions from official GMAT tests and publications. And if you read a bunch of GMAT blog posts written by other GMAT test-prep “experts”, you’ll find that many of them seem to disagree with me – since they’re trying to sell you a nice bundle of non-official GMAT questions and exams.

Sometimes, even my own GMAT students object when I tell them to avoid using knockoff GMAT tests from, say, Manhattan GMAT or Veritas: “How terrible could those materials be? There are smart people working at those GMAT test-prep companies, right?”

Yes, of course there are smart people working at GMAT test-prep companies – though some test-prep firms clearly employ smarter GMAT question-writers than others. But even at their best, no GMAT test-prep company can possibly compete with the precision of real, official GMAT questions. Part of the reason is that copying the style of an intricate standardized test is inherently challenging, but the bigger reason is basic economics.

If you’re reading a GMAT blog, you’re probably a business-savvy future MBA student, and you probably have some intuition for the business models of companies like Kaplan or Manhattan GMAT. So take a guess: how much do you think those test-prep companies spend developing each individual question on their GMAT practice tests or in their GMAT books? Go ahead and think of a number.

OK, got something in mind? Great. Personally, I would imagine that GMAT test-prep companies spend something in the range of $5-30 per “knockoff” GMAT question if they actually want to make a profit, but I could be wrong.

Now let’s ask Dr. Lawrence Rudner, former GMAT Chief Psychometrician, how much the GMAT’s developers spend on each practice question:

Test items are costly to develop, often in the range of US$1,500 to US$2,500 per item.

This comes straight from a paper that Rudner presented at the GMAC Conference on Computer Adaptive Testing way back in 2007. If question development costs have kept pace with inflation, then each GMAT question would cost somewhere between $1,700 to $3,000 per question in 2016, though that figure could be even higher if shortages of trained psychometricians continue to inflate GMAT development costs.

(Incidentally, I got halfway through a PhD in education policy and psychometrics – the statistical science behind standardized testing – before I came to my senses and realized that I’m much happier as a GMAT tutor than as an academic or psychometrician. If you’re interested in a career in psychometrics, send me an email, and I’ll do my best to help out.)

So yeah: that GMAT question you saw on the latest version of the exam? It might have cost $3000 or more to develop – perhaps 100 times more than a GMAT test-prep company could reasonably spend on each question.

And if that sounds batsh*t crazy – or at least wildly inefficient – it probably isn’t. Actual GMAT test items go through a painstaking cycle of editing, testing (often as “experimental questions” on actual GMAT tests), re-editing, and re-testing before they actually become part of a real GMAT exam. For every question that actually makes its way onto the GMAT, countless others are discarded for any number of reasons. For example, there might be evidence that the discarded GMAT questions were phrased badly, that they don’t seem to capture the skills the GMAT promises to measure, or that they (accidentally) discriminate against GMAT test-takers from certain demographic groups. Much of this testing requires substantial skill and training – and employees with these skills certainly aren’t inexpensive.

You probably know that I’m not particularly impressed by the GMAT’s ability to identify the most talented business leaders, but the GMAT’s creators deserve credit for producing subtle and meticulously phrased questions. How could any GMAT test-prep company possibly produce anything of comparable quality or precision, given test-prep firms’ economic limitations?

So whenever you work through practice tests from major GMAT test-prep companies – or GMAT practice worksheets from an obscure little GMAT tutor dude in Colorado – remember to take your results with a huge grain of salt. Sure, our “knockoff” GMAT questions can probably help you build skills, but none of us can ever give you a truly accurate GMAT practice test – simply because none of us will ever be able to afford to do so if we actually want to keep our doors open.

 

GMAT verbal underperformance

I recently received an email from a lovely mother of three who has done an absolutely heroic job of raising her GMAT quantitative score from 18 to 36 in just a few months. That’s an amazing jump, and her math tutor deserves a cookie. Unfortunately, the poor woman has seen her GMAT verbal score move in the opposite direction in the past few months: she’s gone from a 33 to a 26 to a 24.

If you were her, you’d probably be pretty upset, too.

Believe it or not, she’s absolutely not alone. I think that about 20-30% of the people I meet have experienced some sort of inexplicable verbal calamity on the GMAT. By “calamity,” I just mean that their official GMAT scores don’t match their practice test scores—and unfortunately, the scores sometimes aren’t even close. This seems to happen much more often on the verbal than on the math section, and it took me quite a few years to figure out why that might be the case.

First of all, I really don’t think that the GMAT official guides necessarily give you a good sense of what “real” verbal questions feel like. Very few of the reading comprehension and critical reasoning questions in the official guide seem to be much tougher than, say, 650-level questions. I’ve seen plenty of actual GMAT passages that are nearly incomprehensible, but there are very few such passages in the official guides. (Many of my students–especially those who scored above a 35 on the verbal section–seem to share my opinion on this.) The bottom line is that students who rely primarily on the GMAT OGs might be thrown off by the surprisingly difficult critical reasoning and reading comprehension passages on the real test.

Worse yet, some students rely far too heavily on “knockoff” test-prep material, and you know how I feel about that. It’s outrageously difficult for GMAT test-prep companies to accurately copy the nuanced style of the verbal section of the GMAT, and non-official practice questions are inevitably very different from the real thing. In a lot of cases, using these materials can actually do more harm than good, unfortunately. Students grow accustomed to the question-writing style of their chosen test-prep firm, and then suffer mightily when they take the actual GMAT exam. The best test-prep firms offer some good techniques for tackling the verbal, but I think it’s a mistake to rely too heavily on their practice tests. There is no replacement for official GMAT and LSAT materials, and most students will do best if they use “knockoff” tests sparingly.

But I think the most important reason for verbal underperformance has nothing to do with study habits: many test-takers are simply way too “amped” when they start the verbal section. For pretty much everybody, the quant section of GMAT is an intense experience–you’re racing against the clock, the adrenaline starts to flow, and you push yourself as hard as you possibly can. You take your little eight-minute break, and then you go back in to the testing room, foaming at the mouth, ready to rip the verbal to shreds.

And in your haste to power your way through the verbal, you get a little bit too intense, and maybe you skip a word here or there. You finish with a few minutes to spare, or maybe more. And then your score sucks. Why? If you’re not reading very, very carefully, you’ll get GMATed by every verbal question type. Even if you miss just one key word on every other question, you’ll do massive, massive damage to your overall GMAT score.

Case in point: an unusually brilliant JD/MBA applicant hired me for two weeks of fast-paced, balls-to-the-wall GMAT lessons. This guy is at an Ivy League law school, and he’s ridiculously smart, even when compared to other Ivy League lawyer types. He hired a GMAT tutor just to help him to shake off some math rust, and he needed basically no help on verbal. On the GMATPrep test that he took before his first tutoring session with me, Mr. Ivy League Law scored a 37 on math, and somewhere in the mid-to-high 40s on verbal. So we focused on math, and brought his actual GMAT quant score up to a 47 after just a couple of weeks.

Awesome, right? But hold on: Mr. Ivy League Law was so amped after the math that he raced through the GMAT verbal section at breakneck speed, and finished with 20 minutes (!!) to spare. His verbal score dropped to a 41. That’s still enough for a 710 composite, but if he’d taken a few more deep breaths, he could have easily gotten a 750. (Not that it really matters–a 710 is high enough, and Mr. Ivy League Law will soon be Mr. Ivy League JD/MBA. But he’s a great illustration of how an overaggressive approach to verbal can sabotage your score.)

So I’m convinced that stress, exhaustion, and haste are the biggest culprit for almost everybody whose real GMAT verbal score doesn’t seem to match their practice test scores. If you have a tendency to race too much on the verbal, taking a few deep breaths might be more important than any studying you could possibly do. When you finish the quant, use your eight-minute break to re-orient yourself. Grab a snack or a drink or a smoke or whatever makes you happy, and relax a little bit. Focus on being precise and thorough and alert on the verbal. Even if you’re a slow reader, you’ll gain much more from being calm, focused, and accurate than you’ll lose from having to guess on a small handful of the 41 verbal questions.

a ridiculous, costly GMAT myth

Before I make the main point of this blog post, a few disclaimers: 1) I have absolutely nothing against GMAT prep classes; on the contrary, I think that they’re more cost-effective than a private tutor for many students. 2) I have absolutely nothing against any of the major GMAT test-prep companies. In fact, I had an absolutely wonderful experience working for one in the early 2000s. 3) The following is meant to be informative, not bitter or critical.

OK. Just wanted to make sure that I made it clear that I’m not trying to be a jerk here.

But I had to share this with anybody who might be interested. I just started working with a new student here in NYC, and she’s using her GMAT tutor as a supplement to a test-prep course. Her classroom teacher apparently said the following: “If you’re debating between A and D on a GMAT question, the answer is probably A.”

Whoa. Really? This deserves a special place in the Crappy Test Advice Hall of Fame, alongside the old adage about picking C whenever you need to guess.

I would be shocked if the GMAT doesn’t randomize its answer choices. Randomizing might be a little bit tricky on, say, data sufficiency, but I’m pretty certain that A and D are both correct about 20% of the time on GMAT. It’s an awfully sophisticated test; why would they do something stupid like making A the answer most of the time?

As soon as my student said this to me, I probably got a crazed, skeptical look on my face. I started searching for any conceivable explanation… was it on sentence correction, and the teacher just meant to say that people are often overly reluctant to choose A? Was it on data sufficiency, where the same might be true? Nope. My student insisted that they were discussing a plain old math problem solving question.

Again, (here comes disclaimer #4), this is second-hand information, and I’m wondering whether something got lost in translation from teacher to GMAT student to GMAT tutor. But if a major test-prep company is making blanket statements about A (or D or C or any other letter) being a fundamentally better guess than other choices, that’s pretty scary stuff.