Questions to Ask When Choosing
a Personality Assessment
Choosing the right personality assessment for employee selection and leadership development is challenging. There are so many choices and many of them not good for selection. Some of the most “popular” personality tests, like the MBTI and DISC are NOT recommended for use in selection as they are not considered a high enough standard of reliability for that more rigorous testing purpose. They usually find their way in to an organization through training and development where the simplistic nature of the personality typology can be useful. You don’t have to be a professional or certified test administrator to deliver and interpret many of these test. People in training like the aspect that these simple and unsophisticated tests can show that there are different types of people in the world. For managers, that is useful learning. They have to learn to treat people differently. However, the Personality types that the DISC or MBTI demonstrate are not stable. It is estimated that up to 50% of people will have a different profile in 6 weeks. These tests are not predictive and therefore do not meet the Uniform Guidelines standards for use in selection. So, to help identify quality assessments with real world implications we decided to address what questions to ask when choosing a personality assessment?
1. Is the Personality test a Big 5 Personality test. Does it reliably measure all 5 of the Big 5 aspects of Personality at work? Is the questionnaire of sufficient length (at least 160 items, somewhere between 200 and 250 is better).
AAI’s Work Behavior Inventory is designed around the Big 5, or OCEAN model of personal (Openness to Experience, Conscientiousness, Emotional Stability, Agreeableness, and Neuroticism). These personality factors are the standard for personality tests used in selection and conform to the Uniform Standards. For most tests of the Big 5, the Personality domain for Neuroticism is labeled as Extraversion which is the opposite of Neuroticism. As the scales are bi-polar, it is fine to use the opposite end of the scale to define the scale title. Under Extraversion, we seek the scales, Sociability, Leadership, Influence and Energy.
2. What are the personality assessments designed to do relative to the needs/goals of the customer?
The AAI suite of tests measures 21 scales of of work behavior personality across the Big 5. Beyond Extraversion, we have Agreeableness which is made up of Collaboration, Concern for Others and Diplomacy
A test has to determine if the person fits the job Some tests want to make it easy and say Extraverts are needed for sales positions and Introverts are needed for long distance truck drivers. There is more to fit than the scales of introversion and extraversion. The AAI suite of personality tests measures 21 scales that are closely associated with common work competencies. Further, the AAI scales are mapped to a number of competency models demonstrating they measure important characteristics that predict individual performance in individual jobs.
The output of the selection process should provide a solid basis for coaching around career development.
The AAI suite of tests is built off the WBI, which is the premier Big 5 Personality test in the market today. It has the highest reliabilities in the industry. The Selling Styles Inventory measures personality traits identified through research with sales position success. We have validation studies on the SSI, WBI, CSI and Transportation Services Inventory to prove that the tests not only measure traits important to success in the job, the tests do a superior job of predicting candidate success, or failure.
3. Is the personality assessment provider a member of the American Psychological Association (APA), Society of Industrial/Organizational Psychology (SIOP), or other professional organization such as the International Association of Applied Psychology (IAAP) that mandates ethical and statistical guidelines for creating personality assessments?
Dr. Ron Page, Tom Payne, Nathan Page, Paweena Payne along with senior consultants authorized to work with the AAI suite of products, Dr. Larry Clark, Dr. William Mobley, Dr. David Campbell, Dr. Rich Arvey are all respected members of both the Society of Industrial Organizational Psychology (SIOP) and the International Association of Applied Psychology (IAAP). AAI routinely hosts, chairs and participates in research panels at all the conferences of both SIOP and IAAP. Research and studies on the WBI have been accepted and presented at both of these renowned international psychological conferences.
4. Have the personality tests been reviewed in the Buros Mental Measurement Yearbook?
The Buros book of Mental Measurement is considered primary resource for critical reviews of psychometric assessment tools. The Buros testing center at the University of Nebraska publishes reviews of all submitted psychometric assessments. AAI has submitted it’s work and research to Buros on the WBI and SSI and is in the process of submitting the research findings for 6 more tests. Buros is considered the best source for information on the attributes of psychometric assessments and any serious psychometric test will be reviewed in Buros.
5. Is each personality test supported by a test manual that is organized according to the standards outlined in the Uniform Guidelines on Employee Selection Procedures?
AAI maintains test manuals for all it’s assessments and has reliability and validity research available for professional scrutiny. In addition, AAI supports research on psychometric assessment with universities around the world and provides tests for research purposes for research that will be professionally published in scientific journals
6. What standardized validation process is followed before the personality assessment provider implements a selection test in an organization?
AAI follows the Uniform Guidelines in conducting both reliability and validity studies. We make sure that the inclusion of any of the AAI suite of tests is adding predictive validity to the selection process. Validity depends on Reliability. If a test does not constructed in a way that allows it to accurately measure the domains it claims to measure, it cannot be valid. The most important reliability is item reliability, which is making sure that the individual items on the test or questionnaire, have high correlations with the scales they purport to measure. For a Personality test, reliabilities of scales should be +.70. The WBI scales all show reliabilities in the .82 or above and reliabilities are verified in each language.
7. Does the personality assessment provider supply technical reports containing competent validity studies (as defined by the Uniform Guidelines) using the personality tests in real organizations?
AAI produces research for publication on the use and application of its suite of tests in many countries around the world. We are proud of our work in this area and want to raise the standards for use and application of psychometric tests used in selection around the world.
8. Can the personality assessment provider produce a summary of validation results for jobs similar to the one under consideration?
AAI routinely does research with clients, other consultants, universities on the validity of their tests in use in selection situations. Every selection situation is unique and just because a personality assessment has been used in one sales organization, or one managerial position, does not make it valid for all such jobs with the same title. AAI will work with clients to assure that the AAI test used is appropriate for use in that particular selection situation.
9. Does AAI use cutoff scores and how are cutoff scores established for selection purposes?
AAI conducts validation studies to prove that it’s scales scores are relevant to predicting hi and lo performance of individuals with consistent accuracy in individual jobs. However, as with all prediction, it is a matter of
improving the odds of making a correct selection decision. We strive to improve the percentage of correct selection decisions. Our goal is to help clients build a selection process with reliable and valid assessment elements that each contribute to a higher overall validity or performance prediction. Generally we don’t encourage cut off scores unless it is used as a way of screening in candidates to be interviewed in high volume applicant selection situations. Not all jobs lend themselves to a single cut-off score. With a test as professional and valid as the WBI, there are 21 scales and anywhere from 6 to 15 of these will be typically predictive of successful candidates for a given job. A cut off score approach will produce a general criteria that allows comparison of similarly qualified candidates for priority in interviewing. Tests are one element of the selection process and although they add value in the decision making process, they should never be thought of in a way of Pass/Fail for candidates. The role of the test is to add greater insight into the fit of the candidate and direction for onboarding for those that are hired.
10. In Assessment tools used in Selection it is important to have established norms. Does this test have and publish its research on establishing norms?
The AAI suite of assessment tools is built off the Big 5 platform using what is called a Likert scoring scale and this form of assessment is ideal for establishing norms against which all candidates can be compared on similar scales. In the AAI suite of assessments, not all high scores are good and not all low scores are bad. It depends on the requirements of jobs. By carefully establishing norms and then doing a job analysis to set the requirements for candidate selection, cut off scores can be established or better yet ranges for each scale depending on the specific requirements of a job can be established to compare and prioritize candidates. Many other forms of tests are not suitable for establishing norms and are therefore not valid for comparing candidates against each other.
11. How does the assessment provider systematically evaluate the performance of their tests?
AAI recommends a holistic approach to validating the selection process which the personality test is an important part of. The selection process should be well considered, should include a selection of the most job relevant competencies, professional knowledge and education of the candidate as selection criteria. The process can then choose to add certain job relevant assessment tools such as personality and cognitive ability. The desired end of a selection process is to hire better people, with greater rates of retention and greater longevity of those hired. Partnering with AAI typically reduces fist year turnover by 50%.
12. Does the personality assessment provider maintain research files that can be accessed to confirm the results of validity studies?
AAI tests are being evaluated constantly for reliabilities, validities and translations. AAI makes studies and statistics of it’s tests as well as in language versions.
13. What is the personality assessment provider’s policy for supporting customers in the event of a legal challenge to the use of a test?
AAI follows the Uniform Guidelines on selection testing, is a supporter of the use and application of O-Net (the US Dept of Labor) website on job requirements and recommended selection process steps, including testing standards. O-Net is available on line and is free to everyone. It can be a great resource to look up job descriptions, competencies, psychometric assessment guidelines, etc. You can get to O-Net by the following website: http://www.onetonline.org/help/online/
14. Has the personality assessment provider been involved in any legal challenges of a test, and if the outcome?
AAI.s suite of personality and cognitive ability along with talent management assessments has not been challenged as being discriminatory or biased. AAI works very hard to assure that it’s tests are #1 reliable (they measure what they are supposed to measure) #2 they are valid predictors of job performance of candidates.
