Organizations invest substantial amounts of time, effort, and resources into university recruiting every year, but they’re often left wondering how students actually feel about the process—especially when it comes to students belonging to underrepresented groups. To shed light on student experiences with university recruiting, IO Psychologists from CodeSignal’s Talent Science team held more than a dozen focus group sessions in partnership with ColorStack, a nonprofit organization invested in career development for Black and Latinx computer science students.
A total of 40 Black and Latinx software engineering students who’ve had real-world experience with pre-hire technical assessments, a critical part of the university recruiting process, participated in these in-depth focus group conversations to help us get a pulse on students’ tech recruiting experiences.
What do students have to say about pre-hire technical assessments?
When we asked Black and Latinx software engineering students about their expectations and experiences with pre-hire technical assessments broadly, these themes came up again and again:
They’re an opportunity to get noticed for your skills
In addition to the pressure students feel taking pre-hire technical assessments, students were equally likely to express that they value the opportunity to demonstrate their skills and knowledge. They made statements like, “it makes you feel as if you got noticed from your application.”
When developed to be job-relevant and fair, students stated that they actually tend to prefer when companies have an assessment that “you can complete on your own and are timed.” To many students, being able to complete a skills-based assessment made it more likely for them to feel like they had agency in the hiring process and weren’t going to have the experience they do with companies “who don’t have assessments that you never hear from again.”
The questions tend to be inconsistent and unclear
Students emphasized that, though most employers require technical assessments, it’s difficult to know what to expect when you start one. The lack of consistency across organizations’ pre-hire technical assessments led students to express feelings of confusion and inability to effectively prepare. There were expectations that an “assessment should align with job descriptions,” but those were often not met.
Many students pointed out that questions asked often seem unclear in what they are asking. Student’s found themselves thinking things like, “Why are they even asking me this?”
It’s hard to see how the questions relate to the role
A topic that came up again and again was the expectation that technical assessments should align with the technical skills and knowledge organizations want to see on the job. Students stated that their pre-hire technical assessments should “assess technical abilities necessary to do the job,” act as “a barometer to gauge the skills of candidates,” and assess “basic, core skills,” each time expressing disappointment that those expectations were often unmet.
Some pre-hire technical assessments met the mark, but many left students asking, “What’s the point?”
They’re just part of the process
Many students felt that pre-hire technical assessments are “just part of the process.” They considered pre-hire technical assessments to simply be a step in their path towards receiving a job offer, “one of the first steps you take before even talking to a person.” Though technical assessments can be challenging, students generally felt that they were a standard part of landing an engineering position and, thus, worthwhile to complete.
Even though pre-hire technical assessments were normalized in students’ minds, the importance of performing well came through clearly, with students emphasizing that “they can make or break you.”
A good deal of stress and anxiety comes with them
Given the importance of receiving an offer from your dream employer, it’s no surprise that technical assessments bring out students’ nerves. Many students emphasized that they feel a great deal of pressure and stress associated with attempting technical evaluations, which they described as “difficult” and “daunting.” This was particularly true for students describing taking a technical assessment for the first time.
Pre-hire technical assessments were openly referred to as “high pressure situations.” Because of that pressure, students reported beliefs that “you need to be on your A game” to stand out in a sea of candidates.
Preparation is key
Many students were surprised by the lack of clarity and transparency offered by companies on how to effectively prepare for pre-hire technical assessments. They consistently mentioned the need for study materials, practice resources, and tips and strategies to help them best showcase their skills and knowledge. When thinking about specific knowledge and skills required, students made statements like, “you don’t often know what it is the company is looking for.”
Students were excited to discuss ways companies could help them maximize their potential to get noticed in the pre-hire assessment stage of the hiring process.
Tips for developing an undergrad recruiting process that empowers students to do their best
Below are some meaningful ways for companies to apply these focus group takeaways to improve experiences for undergraduate students:
1. Adopt a scientific approach
The best way to ensure technical assessments are job-relevant, clear, and fair while providing an accurate and reliable measurement of candidate technical skills, is to use the science of personnel selection to inform the pre-hire assessment development process. This means developing a process to prioritize validity, reliability, job-relevance, fairness, and promoting a positive candidate experience—infusing this kind of rigor provides a strong foundation for a positive experience where students feel the assessment is fair and relevant.
2. Make sure the assessment maps onto the role
Students were quick to say that they appreciate when questions are truly based on the knowledge and skills required on the job. Many students agreed that when “questions have a lot of context and tell a story” it helps them think through what they need to do to solve them.
Providing realistic scenarios can simultaneously provide a sense of job-relevance and fairness for candidates, and it gives students a preview of what types of tasks they can expect to encounter at work.
3. Avoid information overload
At the same time, students emphasized that they prefer questions not to be “too wordy” so they can focus their time on answering them. One student described an experience where they were reading through three paragraphs of context that had nothing to do with the question before getting to the meat of the question. Students agreed that oftentimes, a good deal of the time they are given is wasted in cases like this. One student stated that “sometimes the context is good, but sometimes the context is overwhelming.”
Pre-hire technical assessments should be carefully designed to balance needs for context, relevance, and brevity.
4. Set clear expectations and offer practice opportunities
Offering a job-relevant, familiar experience for students and other candidates can help keep them motivated and engaged in the hiring process. Students in our focus groups shared a sense of self efficacy, or believing they could succeed, when they were given questions that were challenging but not completely unexpected.
Clear communication about what to expect in taking a pre-hire assessment can help set students up for success. In addition to making sure the questions are clearly job-relevant, it’s important to give students some idea of what skills they’ll be tested on in a pre-hire assessment. One candidate said when they opened an assessment “and saw there was a practice question, it relieved a little bit of the stress of what to expect.”
Offering opportunities to practice and providing a clear outline of what the assessment will be evaluating can give students confidence to work through the assessment and avoid stress and anxiety that can come up when students are shocked by the questions they’re given or are uncomfortable using the platform.
5. Provide skills-based developmental feedback
Students in our focus groups consistently said they wished they had more feedback on their performance on pre-hire technical assessments to give them a sense of agency in the hiring process. One student who had taken a CodeSignal evaluation in the past stated, “It was cool because I learned my coding skills were slower based on the coding report, but I also learned my creativity was high. It helped me learn what I needed to develop, including how to pace myself in these assessments.”
Students are especially receptive to feedback, due to the fact that they’re transitioning from a learning environment to the workplace.
How our Talent Science team helps companies level up their undergrad recruiting efforts
Here are some ways CodeSignal’s Talent Science team builds a strong candidate experience into our skills-based evaluations:
- Putting job-relevance first in a rigorous skills-based assessment development process involving job analysis and interviewing subject matter experts (SMEs)
- Creating opportunities by putting skills first and expanding talent pools to include all candidates, regardless of background or pedigree
- Setting up candidates for success by providing opportunities to practice and familiarize themselves with the platform, and by offering resources to help set clear expectations going into our skills-based evaluations
- Saving candidate time by thoughtfully developing technical assessments to accurately and reliably tap into the most job-relevant candidate skills in as little time as possible, and by offering candidates opportunities to share one evaluation result with multiple employers
- Providing an engaging candidate experience by simulating job-relevant tasks in our evaluations and monitoring, evaluating, and making adjustments to maximize candidate reactions on an ongoing basis
- Prioritizing fairness and diversity in a rigorous assessment development process that reduces biases and allows all candidates an equal opportunity to demonstrate their skills
- Promoting transparency by providing candidates with quick feedback that is directly tied to the skills we evaluate in our evaluations, giving candidates a sense of agency and fairness in the hiring processes of organizations who use our evaluations
CodeSignal’s Talent Science team conducts research like this to get a pulse on candidate experiences in technical hiring processes and apply findings to optimize fairness and rigor in our assessment development processes.
Learn more
Want to connect with our Talent Science team to see how you can improve the tech recruiting processes at your organization? Schedule a call with one of our experts today.