What Makes a Good Test?

It Measures What It Purports to Measure

This should go without saying. However, tests like the SAT prove that this isn't always the case. For many decades, the test was the "Scholastic Aptitude Test," and it purported to measure intelligence. We helped push the College Board to admit that the SAT did no such thing and the test was renamed in 1994 (fittingly, the acronym "SAT" now stands for nothing at all). The College Board also promoted the idea that the SAT measures high school studies in a way that eliminated differences in grades across schools and classes; The Princeton Review helped show that the test measures almost nothing taught in high school. Finally, the test's advocates claim it predicts college success. In fact, the correlation between college performance and SAT scores is weaker than that indicated by high school transcripts and not much better than family income and other purely socioeconomic indicators.

It's Unbiased

High–stakes tests should be unbiased. That doesn't mean that every demographic group should score equally well; it simply means that similar students should achieve similar scores. Women score 40 to 50 points lower on the SAT, for example, than do men, though they have better grades in both high school and college. Since SAT scores determine both admissions and scholarship/financial aid awards, women are doubly penalized.

At the same time tests can help highlight unequal outcomes, as with the provision in No Child Left Behind that requires states to report scores separately for each subgroup of students (boys and girls, rich and poor, white and non–white, etc.). This disaggregated data helps expose systems that are failing the kids most in need.

It's Fair, Open, and Has Reasonable Policies

More than 20 years ago, New York State Senator Ken LaValle promulgated the first "Truth–in–Testing" laws. They required that any admissions test given in New York be subject to independent review of its content and scoring, that test items and scoring be publicly released, and that there be due process for any student accused of cheating or other irregularities.

Although the Educational Testing Service (the designers of the SAT) lobbied hard against the laws, claiming they would raise the cost and difficulty of administering the test so greatly that they would no longer be able to give them in New York, no such calamities took place. In fact, the testing companies have since stated that Truth–in–Testing has made their tests better. We believe that all high–stakes tests, including those used for K-12 accountability, should be subject to similar rules.

It Promotes Good Education

High–stakes tests are more than snapshots or benchmarks. They are powerful motivators of whatever behavior–good or bad– will lead most directly to higher scores. You can see this demonstrated when some schools drop recess in favor of narrow drill–and–kill practice sessions, when others provide their teachers with high–quality professional development designed to improve teaching and learning, or by the hard work kids do in our test prep courses to learn skills that are useful only on tests.

Psychometricians tend to focus on a test's accuracy, precision, and reliability. More important than those, though, are the teaching and learning behaviors that the test promotes. The best way to prepare for an essay test is to write a lot; not surprisingly, teachers in states giving essay tests have their students write more. Though it may be no better than a multiple–choice test at assessing writing skills (and somewhat more expensive) it has the inherent advantage of actually getting teachers to get kids to write.

Test Preparation Should Be Stress–Relieving, Not Stress–Inducing

If you give high–stakes tests, people will prepare for them. The question is whether that preparation is efficient, equitable, and effective and does not distort or disrupt the larger learning context. Our twenty–five years of test prep experience has taught us that, above all, there is a time and a place for it, ideally one that is as minimally intrusive and time–consuming as possible. When we work with schools, we show teachers how to avoid test prep and instead focus on the real work of education that keeps their classrooms from becoming dominated by deadening exercises intended primarily to raise test scores. Assessment should be the tail, not the dog.

From the outset, we've made sure that the fruits of our research and development have been widely available through inexpensive books, free distance learning, and courses underwritten by The Princeton Review Foundation. And we work with thousands of schools at every socioeconomic level to help them deal with testing and admissions issues, almost always at a cost savings relative to their existing approaches, and with better outcomes.

Finally, we measurably improve performance. We hire outside firms to assess our results, and participate in every third–party study proposed to us. We encourage our customers to ask us–as well as our competitors–for documentation of any performance claims

Through all of these approaches, and with a consistently honest voice about testing, we've tried to relieve, rather than stimulate, anxiety.