Thursday, August 20, 2009

LSAT Statistics Week - Scale Scores and the Bell Curve

Welcome to the third post about LSAT statistics. Today we discuss the LSAT scale scores in depth, especially how they interact with the number of missed questions, their relatively stable distribution over time, and how they fit along a standard bell curve.

Below is a graph of a standard bell curve; however, we added several layers of LSAT-specific statistics. Say that three times fast. All of the data was pulled from reports published by LSACNet, the research division of the Law School Admissions Council.
The first thing you should notice about the bell curve: it's turned sideways from how you normally see it. This is because we aligned the curve to each one of the LSAT scale scores on the June 2007 LSAT. This particular scale score distribution is fairly common, but we'll have a post next week that covers the irregularities that do occur. Those are rather small, however, as you can see from the thick black lines roping along the standard bell curve. Each of those points represents the scale score distribution for all test-takers on a real LSAT between 1997 and 2004.

When you zoom in on the graph, you'll see how each scale score from the conversion chart maps on to the bell curve.
We've focused in on the scores between 158 and 165 because those scores are the first and second standard deviations, respectively, above the average score of 151. As you read from left to right starting at the scale score, you see the lowest and highest raw score (the number of correctly answered questions), then the approximate number of
missed questions, the cumulative amount of test-takers who scored above the scale score, and finally the number of standard deviations.

As you continue looking left to right into the gradient portion of the bell curve, you see that approximately 13.59% of all test-takers fall between the scale scores 158 and 165. The cluster of data points outside--in the white space--of the standard bell curve near 165 show that the LSAT is slightly more generous within this restricted range than strictly called for by statistics. Conversely, the network of data points within the standard bell curve near 158 show that the LSAT scale score distribution near the first standard deviation is
less than would be expected.

Within the graph itself there are

several important pieces of information. The first is how this graph interacts with our first post about how many people score a perfect 180 each year on the LSAT. On the right you see a close-up of the graph with 0-4% along the y-axis. This does not represent the scale score ranking, but rather the percentage of test-takers who scored that specific scale score. For example, at 165, the LSAT bell curve intersects the percentage axis near 1.4%. This does not mean that someone who scored a 165 is in the top 1.4% of all test-takers, it means that around 1.4% of the 151,400 people who took an LSAT in 2009--or 2,150 people--received an LSAT score report with 165.

The second piece of

confusing percentage use is the cumulative percentage. The number of each test-taker who scored above a given scale score is often compared to the overall number of test-takers, in references like "the 99th percentile." What this actually means is that only 1% of the test-takers scored at or above that scale score; on the LSAT the cutoff is usually around scale score 172. The cumulative percentage is the aggregate ratio of scores above that point on the curve to the scores below.