What is computer adaptive testing?
Before we get into the pros and cons, let’s start with a definition.
Computer adaptive testing is a new form of testing that varies the difficulty, level, and order of questions that get asked depending on your performance within the test. This is in contrast to static testing, where the questions are predefined and nothing changes during the exam.
The idea is to narrow down into the level of proficiency to the greatest granularity. If you’re struggling with a certain type of question, computer adaptive testing provides easier questions. If you’re excelling in a subject, the test will automatically provide harder questions.
Why is computer adaptive testing quickly gaining attention? And does this new form of testing actually help measure abilities better?
When measuring skills with computer adaptive testing, you’re essentially trying to find the coordinate of a dot on a 2×2 grid by trying out different latitude (horizontal line) and longitude (vertical line) combinations. It’s similar to the Hot & Cold game where you make multiple guesses and receive feedback to get close to the target.
The advantage of computer adaptive testing is that it can consistently move that horizontal line to get a more granular level of understanding about one’s skill level.
The downside of computer adaptive testing is that they can be stressful on the test taker. Traditionally, tests are already taken in stressful situations like education systems or for getting a job.
In a job context, if the questions all of a sudden start getting easier (or the perception is they’re getting easier) then the test taker may become nervous that they’re failing. It can create a psychological barrier because the nerves overwhelm the test taker and result in a poor performance (worse than one would have typically done). This will go against the goal of assessing skills accurately.
In the recruiting context, the focus should be on clearing a certain bar, not pinpointing a candidate’s exact, specific skill-level. If too much focus is put on their exact skill level, the test taker could become too stressed to deliver results. Overall, this kind of testing can be detrimental in recruiting and the cons outweigh the pros.
Computer adaptive testing can be incredible in educational contexts, where you’re attempting to measure specific skills to see what needs to be taught next. It can allow for personalization of education and going over previous material to get the student to have a deeper understanding.
This blog post is based on the twenty-ninth episode of the data-driven recruiting podcast hosted by CodeSignal co-founders Sophia Baik and Tigran Sloyan. You can find and listen to this specific episode here or check out the video version embedded above.