In the tech industry and beyond, more and more employers are using skill assessments as a tool to identify qualified candidates and reduce bias in the hiring process. In summer 2020, nearly one in four US companies was using a pre-hiring assessment—a number that some predict will rise to 70 percent by 2025.
Developing an effective skill assessment, however, is no small task. Poorly-designed tests can deter qualified candidates, provide a poor signal of candidates’ skills, and, in the worst cases, do both. That’s why we here at CodeSignal highly recommend that companies think carefully about whether they need to design their own assessments, or if they would be better served by using a standardized coding assessment (we’re partial to CodeSignal’s certified assessments ????).
For those hiring teams who find it necessary to develop their own assessments, CodeSignal’s in-house test design experts have advice. Here are 3 key numbers to keep in mind when designing a specialized or senior-level technical assessment.
One
One represents the ideal number of skills you should be assessing for in an assessment. Yes, just one! (Or—if you must—just a few core skills.)
Why? Focusing on one key skill at a time allows for more targeted and cleaner measurement. Not only does this provide a simpler user experience—it also allows you to more accurately identify your strongest predictors of performance. Once you know candidates can do the basics, then you can measure more advanced skills at later stages of the hiring process.
Two
Next, we find that initial assessments should ideally consist of just 2 coding tasks—and no more than 4. When deciding which tasks to select, think about the base level of skills that are required for the job (instead of going into very specific or difficult skill sets, for instance).
Why? In our experience, candidates can comfortably complete 2-3 coding tasks in an hour if they have the skills needed for basic proficiency at the task. And compared to assigning just one coding task, using multiple tasks provides a stronger signal of a candidate’s abilities—so you can be confident in the measurement of that skill.
Sixty
The last number you need to know? Sixty—as in, a test that takes a candidate about an hour to complete.
Why? Here at CodeSignal, we have found 60-70 minutes to be the optimal test length to capture an accurate signal from your candidates (both breadth and depth of knowledge)—but without deterring candidates from taking or completing the assessment.
More than just numbers
Is “1-2-60” all you need to know to develop an effective technical assessment? Of course not. As we explored in our previous article on assessment design, it is crucial that your assessment is job-relevant. This requires conducting a job analysis and performing content validation. And from the candidate experience perspective, it’s important to design an assessment that leaves your candidates with a positive impression of your company—so the assessment should also be comprehensible, coherent, and enjoyable to take.
Finally, another key component of good assessment design is how easy it is to cheat or fake. Does your team regularly monitor sites like Stack Overflow to see if your questions have been leaked? If anyone can just copy and paste a solution they find online, the assessment is no longer valid for identifying who has the skill you’re trying to measure.
Conclusion
Good test design requires more than just subject matter expertise—it also requires expertise in best practices around how to objectively measure skills. (This is why our team at CodeSignal includes Industrial-Organizational Psychologists! ????)
For technical hiring teams getting started with custom-built assessments for specialized or senior-level roles, the 3 metrics we describe above are a good place to start. A well-designed test may take time and effort to build, but the reward—identifying and hiring the best talent—is well worth the challenge.
Learn More
Want to learn more about creating validated and job-relevant technical assessment for your organization? CodeSignal’s team of assessment experts would love to talk with you. Reach us at [email protected].