Let’s apply science and common sense to a business challenge. Imagine that you advertised a pizza delivery job and received 10 online job applications each accompanied by a resume. The resumes seem equally impressive. The applicants are all high school graduates. They like to interact with people, have valid driver’s permits, and are over 18. Knowing that your budget allows you to hire only one person at this time, what would you do to select the best employee for your business?
Let’s say that you decide to use an interview as a part of your process.
The interview
- Should every applicant get the same set of questions? Why or why not?
- Are there benefits to creating a set of questions before you start interviewing?
- During the interview, will you ask probing questions to follow-up on the job applicant’s responses? Why is this a bad idea?
Shall we apply science and common sense to answer some of these questions?
Should every applicant get the same set of questions? Why or why not?
If you want to compare job applicants in a fair manner, you need to use the same “yardstick.” Thus, your interview must be made up of the same set of questions. Anything other than the same set of questions prevents a fair comparison.
Are there benefits to creating a set of questions before you start interviewing?
When you need to consistently ask each job applicant the same set of questions, it is practical and useful to prepare your list of questions in advance of the interview. Thus equipped, you can ask each job seeker the same set of questions in the same order. This helps you to keep the interview administration conditions consistent.
As it pertains to interviews, what does the term “structured” mean?
In the field of personnel selection, interviews are a type of assessment. Interviews can be “structured” vs. “unstructured.” In “structured” interviews, the same questions are consistently asked in the same order with no probing. Thus, each job applicant gets the same exact assessment.
In this way, the interview’s structure facilitates “comparability” of the results or scores because each job seeker has equal opportunity to answer the same set of questions. Structured interviews confer a sense of fairness to your process.
Apart from consistently asking the same set of questions, what else is needed to create scores that can be genuinely compared?
We need to score job applicants’ responses in a consistent manner. To do this well, you need a well-constructed rubric or scoring guide to score the job applicants’ responses. A proper scoring guide contains sample responses and the number of points you will assign for each part of a response. A good scoring guide reduces subjectivity in the scoring process and lets you score responses in a consistent manner. Thus, a well-designed scoring guide enables you to score responses of different job applicants reliably and objectively.
The Goal
When you use an interview as a part of your selection process, the scores must assist you in making accurate personnel selection decisions. This occurs when candidates who score higher on the interview also perform better on the job. The psychometric term for this property of assessments is “predictive validity.” Check out our blog articles that specifically address predictive validity.
Summary
In interviews, when job applicants are consistently asked the same questions in the same order with no probing, the interview is said to be “structured”. Such structure gives each job applicant an equal opportunity to respond. This means that you are essentially using the same yardstick with each candidate.
To ensure that the scores from your interview can be compared we highly recommend that you employ a scientific process to develop the set of questions and the scoring guide. Prepare a structured interview questionnaire. Then, administer the interview and use the scoring guide in a consistent manner.
Scoring responses to the set of questions that comprise the structured interview, using a well-constructed rubric, produces results that are comparable. There is an additional benefit. A well-constructed rubric confers objectivity to the scoring process. This means that responses scored using a well-constructed rubric, are less influenced by the subjective judgments of different evaluators or hiring managers.
At the end of your process, scores from the assessment must assist you in making valid personnel selection decisions. This occurs when candidates who score higher on the interview also perform better on the job. The psychometric term for this property of assessments is “predictive validity.”
Note: Predictive validity answers the question: Do people who score higher on the interview outperform those who score lower? To ensure your structured interviews and psychometric assessments have predictive validity, we recommend that you employ a scientific process to develop the assessment.
We hope you found this blog article helpful. For more information or assistance, please contact us at info@psychometric-solutions.com
Related Blog Articles
- How do I accurately and objectively measure an Individual’s Job Performance?
- What data do I need to be able to calculate predictive validity myself?
- When is the best time to calculate predictive validity?
About the Author
Dr. Dennison Bhola has more than 25 years of experience working in the field of psychometrics creating validated instruments such as assessments, structured interviews, and inventories. With his teammates, he has developed more than 1000 assessments for use by organizations around the globe. His expertise includes evaluating new hire quality, assessment effectiveness, source channel efficiency, time to hire, cost per hire, and other such metrics that are of critical importance to Human Resource Professionals. He also specializes in fairness analyses and auditing compliance with Equal Employment Opportunity guidelines, such as the 4/5ths rule.