Monthly Archives: December 2014

How lucky was Mr. Fat Pants?

If any of you are expecting a practical, useful article about GMAT tutoring, I apologize in advance:  this is going to be a pretty useless, vaguely technocratic article about standardized testing.  Consider yourself warned.

A couple of years ago, I wrote a blog post about a fellow named Mr. FP (Mr. “Fat Pants”), who once wrote a brilliant MBA application essay about his lunchtime festivities at the office.  Mr. FP deserves to be a legend simply because he actually used the phrase “fat pants” in his MBA admissions essays, but he’s also notorious (in my little GMAT world, anyway) for springing a huge test-day surprise on both himself and his GMAT tutor:  he scored a 680 and then a 640 on his only two full-length GMATPrep practice tests, then nailed a 720 on the actual GMAT—just a few days after the 640.  Mr. FP is a great guy who kicked all sorts of ass at a top-10 business school… but we agree that he got a little bit lucky.

But how lucky did he get?  And can we somehow quantify his luck?

The quick answer is that Mr. FP was pretty darned lucky.  If you dive deep into the bowels of your GMAT score report (sorry for the image; do you really want to dive “deep into the bowels” of anything?), you’ll see some mumbo-jumbo about something called the standard error of measurement, and the score report will tell you that the standard error of measurement is about 30 points on the GMAT.

To understand the idea of a standard error of measurement, imagine for a moment that you have something called a “true” GMAT score.  There are two ways to think about a “true” GMAT score.  First, you could imagine that it’s the score you would receive if the GMAT always did a 100% perfect, 100% consistent job of measuring your verbal and quantitative reasoning skills (or whatever it is that the GMAT actually measures).  If that doesn’t seem intuitive, imagine this:  you’ll take the GMAT, I’ll wipe out your memory of the exam using the “flashy thingy” from the Men in Black movies (so that you don’t remember any of the questions), then I’ll send you back to the very same GMAT test appointment using a time machine from the Back to the Future movies.  I’ll do this an infinite number of times.  Assuming that you don’t get brain damage, your average score on your infinite number of GMAT exams would be your “true score.”

Of course, you wouldn’t always get exactly the same score on every single test appointment, even if we conjure some Hollywood magic to eliminate all day-to-day variations in your test-taking behavior.  The truth is that GMAT scores are a little bit random.  Or maybe a lot random.  Even if you behaved exactly the same way every single time you took the test, you’d still get slightly different GMAT scores from one day to the next.  That’s an inevitable part of standardized testing, especially on an adaptive exam that spits out somewhat randomized questions.

Back to the technical crap:  the standard error of measurement is basically a measure of that random variation in your scores from one GMAT exam to the next, even if we assume that you’re a robotic test-taker who always behaves the same way.  If you’ve taken a few statistics courses, you could think of the standard error of measurement as the rough equivalent of a standard deviation for your GMAT score:  since the GMAT’s standard error of measurement is around 30 points, roughly 2/3 of all test-takers will score within 30 points of their “true” GMAT score, and roughly 95% of test-takers will score within 60 points of their “true” GMAT score.

In other words, your GMAT score could plausibly fluctuate by around 30 points from day to day, but you aren’t terribly likely to gain or lose 60 points simply by random chance.

With this in mind, Mr. FP seemed to be a very lucky man.  We’ll never know Mr. FP’s “true” GMAT score, but based on his homework and GMATPrep test results, I would have wagered that he was in line for a 660 or so—halfway between his two GMATPrep scores.  If we pretend that 660 really was his “true” GMAT score, then Mr. FP scored 60 points—or two standard errors of measurement—above his “true” score.

If you’re a statistics geek, you’re welcome to bust out your favorite z-chart and do the calculations yourself, but the bottom line is that there was roughly a 1 in 40 (2.5%) chance that Mr. FP would receive a random gift of 60 or more points from the GMAT gods.  And keep in mind that any random, luck-based deviation from your “true” GMAT score could work in your favor—but it’s equally likely that it will work against you:  if 1 out of every 40 test-takers will earn at least 60 “measurement error points” that they don’t “deserve”, then 1 out of every 40 test-takers will lose at least 60 points on their GMAT score, just from bad stinking luck.  The gods of random chance are sometimes very kind, but they’re also randomly very cruel.

So if you’re still 60 or so points away from your ideal GMAT score and you want to roll the dice, go for it.  But your odds of a Mr. FP-esque stroke of good luck aren’t all that good, and it’s probably better to hit the GMAT books hard and make your own luck.