Why test design matters

This blog post is based on the fourth episode of the data-driven recruiting podcast hosted by CodeSignal co-founders Sophia Baik and Tigran Sloyan. You can find and listen to this specific episode here or check out the video version embedded below.

Test design. What does that actually mean and how important is it in the process of going beyond resumes? It’s not just about the assessment because having a bunch of shiny new tests isn’t the goal. You can have all the tests in the world, but if you end up with candidates that can’t do the job or aren’t a good fit for your organization, you haven’t gained anything. 

The assessment is just a tool in the tool belt, and the goal is measuring the right skills as a way to get to the right candidates. So, let’s uncover what ingredients go into the secret sauce. 

Like a good friend, a good assessment is defined by its characteristics. What skills does it need to test for and what level of expertise does it need to measure? It should also consider the experience of the test taker. It may be easy to forget, but there’s a human on the other end of any assessment. Designing what you consider to be a test extraordinaire doesn’t really matter if you’re torturing the subject that’s taking it (spoiler alert – even if they pass they probably won’t want to work for you). 

Anything over an hour and people will be turned off. After all, they may want to see their family and friends instead of taking the weekend to hover over a laptop and answer rapid-fire questions. What’s the best advice to keep it brief? Step back and determine what really matters. There may be questions you need answered or skills you want to measure but take the time to evaluate if you have to get those answers at THIS stage of the game. 

Another common mistake is assuming the steps to creating an assessment are simple. But buying a product off the shelf and testing it with a few in-house engineers works about as well as buying a piece of artwork from eBay and hoping it’s a real Van Gogh. The value isn’t in having any piece of art; it’s in having a piece of art that was created by an expert. The same is true with assessments. Use experts to ensure you’re implementing something that will be effective.

If you notice any of the following, you have some red flags to look out for:

– Candidates that are passing your assessment but not doing well in the rest of the process. 

– Candidates that upon further review would have made a great fit but didn’t pass the test.

– Candidates that are unhappy about the process. 

Take the time to determine what exact skills you are measuring, at what depth, and how you can package all of that into a great candidate experience. With those things in mind, assessments can make a huge difference in hiring the right candidate.