Mind reader: do personality tests pick out bad apples
by Tim Beardsley
For years, employers have given job applicants paper-and-pencil tests to assess basic skills such as reading and arithmetic. These days, candidates may be confronted with a different type of exam as well: a personality test that asks about their attitudes toward a variety of situations. Answers are often run through a computer to produce a profile that rates the applicant on scales, or "personality constructs," as they are called in the jargon of the trade, with names like "gregarious" and "tough-minded."
But widespread screening worries some psychologists and personality researchers. The tests, marketed in the U.S. by such companies as the Institute for Personality and Ability Testing in Champaign, Ill., Consulting Psychologists Press in Palo Alto, calif., and the Psychological Corporation in San Antonio, Tex., are being used by employers to exclude those not suited for sensitive jobs, in police departments and nuclear power plants, for example. They are also a means of selecting candidates with desirable traits, such as extroversion, for marketing positions.
Rodney L. Lowman, a psychologist at Duke University Medical center and author of a book about testing, offers some words of caution: "There is far more practice than there is research literature to support the proactive use of these tests." Lowman believes that "mistakes are being made by screening services that may be overly aggressive at weeding people out." And the desirable characteristics used for screening applicants "often read like a Boy Scout list of virtues, and the specific job relevancy has yet to be demonstrated." Lowman points out that one commonly used test--the Minnesota Multiphasic Personality Inventory, published by the University of Minnesota Press-was designed for clinical patients rather than job applicants.
In addition, few commercial personality tests have been validated in published studies. The confirming studies that have appeared, charge Steve Blinkhorn and Charles Johnson, industrial psychologists at Psychometric Research and Development Ltd. in England, are so full of statistical errors that it is doubtful whether most of the constructs predict anything. The two fired a broadside at preemployment personality testing in the December 20-27, 1990, issue of Nature, where they accused psychologists of adopting "an approach to correlation that would have left its inventor Karl Pearson gasping."
Usually, the British critics suggests, developers of tests have identified apparently significant associations by sifting through thousands of combinations of response patterns and job performance measures. But such trawls throw up spurious correlation by blind chance, a complication that test marketers tend to overlook. Nor have the test services often checked their claimed correlations against new data. Blinkhorn and Johnson found that for three well-known commercial tests, most of the supposed correlations between] scoores and job performance were likely to be the result of pure chance.
Other critics are concerned about tests that purport to measure honesty or integrity, which have become increasingly popular since polygraph testing by businesses was banned in 1988. More than five million people in the U.S. take honesty tests each year, according to one estimate. These tests attempt to flag job candidates who are likely to steal property or company time by asking about their past behavior and their attitudes to various types of theft.
Despite the obvious possibilities for cheating, some sellers claim the tests can predict subsequent inventory disappearances or detect previous criminal behavior. But validating such tests is difficult because employee-thieves are seldom caught. In addition, publishers of honesty tests "intentionally err on the side of lenience" to avoid making false accusations, contends Richard E. Clingenpeel, who runs Personnel Selection International, a job agency in Milford, Mich.
According to a draft report on honesty tests by the American Psychological Association, "a few firms have made public a number of reports of studies having to do with the reliability and validity of their instruments, but most firms have produced nothing of this sort." The congressional Office of Technology Assessment concluded recently that published validations of integrity tests are flawed and inadequate.
Still, basic personality tests have their proponents, who argue that they can be a useful screening tool. "I agree there's plenty of very bad research using personality tests, but I wouldn't characterize the whole field that way," says Paul R. Sackett, a psychologist at the University of Minnesota.
Since the mid-1980s a consensus has emerged that there are five or six robust personality dimensions out of the many more that have been proposed. Michael K. Mount, an organizational psychologist at the University of Iowa, groups personality constructs into one or other of what he calls the Big Five categories (extroversion, emotional stability, agreeableness, conscientousness and openness to experience).
Mount and some other psychologists say they are finding modest correlations between such personality traits and job performance. Using the emerging technique of meta-analysis, which rigorously compares data collected in different studies, Mount and his colleague Murray R. Barrick surveyed 117 studies of personality traits and job performance for a paper that will be published shortly in Personnel Psychology. "In a nutshell, personality indicators were not good predictors," Mount says. One measure, conscientiousness, did show a persistent correlation with job performance. But the correlation is about half as strong as that achieved by mental ability tests.
The U.S. Army has for some years been conducting an exercise, known cryptically as Project A, to develop personality tests. Long-term validation studies with large numbers of subjects are under way. Leaetta M. Hought of the Personnel Decisions Research Institute in Minneapolis, the principal contractor for Project A, has also used meta-analysis to show that some aggregated personality constructs may predict something about job performance. One of these, called dependability, is similar to Mount's conscientiousness construct.
Ivan Robertson, a psychologist at the University of Manchester in England, agrees that meta-analysis shows that "there's a definite effect of personality--but it's small." Properly constructed personality tests seem to give unique information that could be useful to employers if used with other forms of screening, Robertson says.
So will meta-analysis bring validated personality tests that are the answer to a corporate recruiter's prayer? Blinkhorn doubts it because of the difficulty in training testers. Moreover, he points out that if Hough's results are typical, a test would have to be used very stringently to hire noticeably better workers. If only the best-scoring 10 percent of candidates were hired, the proportion of above-average workers in a group would rise from 50 percent to only 59 percent, and many good workers would be wrongly rejected.
The suspicion that personality tests unjustly reject some applicants is probably why they are unpopular, admits Scott Martin, a psychologist at London House, a test publisher in Park Ridge, Ill. "Once you make it objective, it gives people something to criticize." But Martin argues that even a poor test will erroneously turn away fewer good candidates than selecting at random.
Martin's argument may be irrelevant in the real world. Although some pre-employment test publishers say their tests should accompany other forms of assessment, such as an interview, in practice personality and honesty tests are often used instead of other forms of assessment. So they could be a step backward. Lou Maltby of the American Civil Liberties Union fears that tests may create an unemployable group who "test dishonest" or otherwise prove unsuitable. If Maltby is right, pseudopsychology will hurt employers as well as employees.