Objectively Measuring Code Quality

objectively measuring code quality

“All of the software engineering candidates seem equally qualified. Which one should we hire?”

For technical recruiters like you, it’s very frustrating to hear these words from your engineering department. After all, you’ve worked hard to fill the funnel with highly qualified candidates, host interviews, coordinate technical assessments, and schedule on-site visits. Surely there’s enough data to make an informed decision — right?

If indecision and gut reaction is plaguing your hiring process, perhaps it’s time to revisit your company’s technical assessment methodology.

The Problems with “Traditional” Coding Assessments

Let’s start by reviewing some common assessment-related challenges that many tech companies struggle with. Do any of these sound familiar?

  1. Too Generic to Be Useful – Your company has very specific requirements when it comes to recruiting technical talent. Unfortunately, most off-the-shelf coding templates fail to assess a developer’s understanding and proficiency with their daily responsibilities. This forces your company to rely on tests that can only measure generic topics, such as algorithms and data structure knowledge, and this typically results in many candidates passing the initial technical screen but failing the onsite interview.
  2. Subjective by Nature – You can’t fit a coding assessment into the confines of a multiple choice test. Each assessment must be manually administered and scored, most likely, by someone with in-depth technical knowledge. To complicate matters, reviewing hundreds of lines of code isn’t exactly a straightforward exercise, and the logic behind the way the code was written is also important. It’s human nature to see situations subjectively and this could foster more internal indecision and debate.
  3. Internal Bottlenecks – There’s no question that technical assessments can create bottlenecks at your company. Manually scheduling, preparing, administering, and evaluating assessments often requires cross-departmental coordination. All of this consumes time, which slows down the recruiting process. Slow recruiting causes developers to lose interest, and that’s not ideal in today’s competitive market.
  4. Plagiarism – With distributed workforces, many assessments are administered virtually. How can you confirm that candidates are actually submitting their own work and not something that has been plagiarized? Without the right tools, you may unknowingly give the upper hand to candidates who cheat.

Cultivating a More Objective Assessment Process

If any of the aforementioned problems sound all too familiar, then our CodeSignal Recruiter platform might be a useful resource. We built CodeSignal Recruiter specifically for the needs of technical recruiters like you, delivering the following solutions:

Language-Specific Assessments: Our IDE supports 40+ programming languages and offers a library of curated tasks, making it easier to build assessments that actually help you measure competency.

Custom-Calibrated Solutions: Our testing experts will work with you to create two custom assessments at no extra charge.  These assessments can align with your company’s mission or the daily responsibilities of the position.  They are calibrated with your onsite interview questions and are far less subject to any attempts of plagiarism.

Consistent & Collaborative Interface: As candidates submit their work, assessment scores become instantly available within the CodeSignal interface. For developers who progress further in the pipeline, the interview coding environment delivers additional real-time insights into skills and abilities.

ATS Integration: Already using an ATS? CodeSignal Recruiter integrates with several of the most popular applicant tracking systems.

Native Plagiarism Checking: CodeSignal also tests each code submission for plagiarism issues, helping you to feel more confident about the integrity of your assessment data.

Sign up for a quick demo to learn how CodeSignal is helping recruiters create more objective technical assessments.

The Bottlenecks that Bog Down Your Recruiting Process

Tired of feeling bogged down by recruiting bottlenecks? Maybe it’s time you stopped working in your pipeline and started working on it. Start by objectively evaluating your workflow and then seek solutions that will yield greater efficiency.

It’s easy to feel overworked and underappreciated as a technical recruiter.

You put in long hours sifting through countless profiles on job boards, only to be consistently disappointed by the handful of developers who actually apply. To make matters worse, your engineering team isn’t interested in hearing excuses. They need technical talent — and lots of it. Unfortunately, so do your competitors. Simply put, it’s a developer’s market.

Although you can’t change the demand for high-caliber engineering talent, you can take proactive steps to identify and streamline your recruiting process.

In this article, we’ll discuss best practices for overcoming what’s holding you back.

Common Bottlenecks Facing Technical Recruiters

Let’s look at some common bottlenecks facing many technical recruiters these days.

Do any of these sound familiar?

Labor-Intensive Outreach Process: Developer outreach consumes way too much time and effort for the results that it yields. There are literally thousands of qualified developers to engage through platforms like LinkedIn.com — and that’s precisely the problem. Where do you begin? Using filters can be a logical starting point, but even then you’re still dealing with a list that’s far too lengthy. To complicate the issue, most developers are happily employed or not seeking employment opportunities. That’s a big reason why so few developers reply. It’s also why you find yourself spending so much time on outreach. After all, recruiting is a numbers game — or is it?

Clicks, Clicks, & More Clicks: Stop and think about how much time you spend jumping between browser windows. For example, let’s say that you view 250 developer profiles on LinkedIn.com and reach out to 50 of them (requiring several clicks each). You then note each interaction in your candidate tracking spreadsheet (another tab and series of clicks). With each click taking at least a second of your time, you’ve spent at least ten minutes just clicking stuff. That may not seem like much, but over the course of a 260-day work year, it translates into 40+ hours of clicking.

Unintentional Bias: Not every developer is concerned with how his or her LinkedIn profile looks. Some may not even have LinkedIn accounts. So, by isolating the search to only those developers in the PHP Developer group, for example, you could be overlooking an entire crop of technical talent. Even within the group’s membership, some developers may have failed to upload a professional photo or provide updated work histories. Should this disqualify them as candidates? Certainly not, but what other information do you have to go on?

Poor Communication from Internal Stakeholders: Your engineering team is looking for results, but that doesn’t necessarily make them more willing to invest in the process. They’re busy people with an aggressive product roadmap to manage. As a result, you might struggle to get their feedback on candidate qualifications, interview performance, and technical assessment scores.

Too Many Calendar Reminders: Using your calendar to track candidate follow-ups and interviews is certainly better than using handwritten sticky notes. Here’s the big problem: your brain can only handle so many pop-ups before becoming fatigued. Calendar fatigue can cause important things to slip through the cracks.

Objectively Evaluating Your Recruiting Process

If you find yourself constantly battling the same bottlenecks, perhaps it’s time to take a step back and look objectively at your recruiting process as a whole. The bottlenecks you’re experiencing could be symptoms of deeper issues.

Consider these questions:

What is your outreach engagement rate? Your time has a tangible cost. Spending a substantial amount of time on fruitless engagement could represent a negative value proposition. Would a better mix utilize less manual effort and more automation? By monitoring your outreach engagement rate, you’ll feel empowered to make more informed allocations of time and resources.

How long does it take to fill a position? Time-to-fill metrics can vary widely in the technical recruiting world, particularly when it comes to hard-to-find skills. Be that as it may, establishing a baseline should be a priority for your company. Once the baseline has been established, you can then dive deeper into each phase and strategically implement programs that increase throughput.

Do you know your onsite-to-offer rate? Of candidates who get an onsite interview (or, for remote companies, a final virtual interview), what percentage will receive offers? If you’re not presently tracking this metric, it should be relatively easy to re-engineer for recent job postings. As a point of reference, the industry average is about 20-25%. Anything lower than that might indicate problems with your pre-interview and technical assessment processes.

How many qualified candidates do you need in your pipeline? Metrics such as onsite-to-offer and outreach engagement are also useful when estimating your pipeline volume requirements. Rather than aiming for “as many candidates as possible,” a measured approach relies on actual data to find a more realistic balance.

Leveraging Technology to Overcome Bottlenecks

Having developed an objective understanding of your organization’s recruiting challenges, it may be time to shift your focus toward some viable solutions.

One such solution is our CodeSignal Recruiter platform. CodeSignal Recruiter is an all-in-one talent and candidate management system built for technical recruiters like you. With more than 1 million vetted developers, CodeSignal offers one of the industry’s largest pools of technical talent. Many of the developers in the CodeSignal community are actively seeking employment opportunities. Our system’s proprietary matching algorithm combined with the human touch of our Talent Success Managers deliver personalized, curated candidate recommendations that align with your needs.

By harnessing the power of machine learning and AI, CodeSignal keeps you on track — without overwhelming you.

Sign up for a personalized demo today.

Guest Post: AI Will Dominate Recruiting – So Prepare For Major Changes In These Areas

AI Will Dominate Recruiting – So Prepare For Major Changes In These Areas

Most recruiters are busy with their day-to-day work. So, some fail to realize that many recruiting processes and tools currently in use will soon improve significantly by the continual learning provided by Artificial Intelligence (AI). In addition, not only will AI and its advanced cousin Machine Learning (ML) make recruiting processes faster and cheaper, soon and in many cases are already adding significant new capabilities that were simply not possible with legacy systems. However, relax, this isn’t a job security issue, it’s an opportunity to improve performance with little effort on the recruiter’s part.

It’s quite common these days for the CEO’s from Amazon, Google, MS, Facebook and Apple to expound on how artificial intelligence and machine learning will dominate their businesses over the next few years. Even Vladimir Putin stated, “The country that leads in artificial intelligence will lead the world.” It’s also important to realize that in addition to contributing to the most visible product areas, like digital assistants and driverless cars, “Machine learning and AI are a horizontal enabling layer” says, Jeff Bezos of Amazon, meaning that AI will impact and improve every major function and its processes and decisions. Recruiting leaders shouldn’t be surprised that I predict that “machine learning will soon begin to dominate every major aspect of recruiting.” Just as previous technologies like ATS’s and CRM’s have already transformed recruiting. It’s important for recruiters to be aware that there is an upcoming wave of mostly vendor developed recruiting applications that assist in producing extraordinary hiring results because they include machine learning capabilities.

The goal of this article is to highlight the upcoming AI/ML and technology changes that are likely to occur in each of the major areas of recruiting.

 

The Top 15 Recruiting Areas That Will Be Most Impacted By AI And Machine Learning

The areas of skills-based recruiting and job/candidate matching that will be impacted are below. Note that they are listed so that the initial items in the recruiting process appear first.

Recruiting areas related to finding and attracting prospects

  • Advertising placement and content – Machine learning will continually improve your placement process for branding materials and job postings rather than relying on costly trial and error approach to advertising. This is critical because accurate placement is essential if you expect to get the right kind and number of applicants. Systems will continually learn by analyzing visitor cookies and response rates so that you place your highly targeted materials in front of the right people at the right time. Also, machine learning technology can help you continually refine your content so that it gets the highest response from your recruiting targets.
  • Your own website and social media – continually improve by firms using machine learning on their web and social media pages to better attract and continually engage your target audience. Software bolstered with machine learning will also be able to monitor and make you aware of both positive and negative comments that others make about your firm and jobs on the Internet and social media.
  • Finding individual prospects – during sourcing will become much more automated and accurate when augmented with machine learning capabilities. Automated sourcing programs will be able to find many more and better matches, based on the continually updated target profile that you develop as a result of feedback. There are already vendor packages that allow you to identify currently employed individuals (e., passives) that are likely to quit soon and prospects that are likely to be diverse.
  • Enhancing prospect profiles – can make the existing candidate profiles found on sites (like LinkedIn) more complete by supplementing them with additional information that a machine learning program will find on the Internet. Machine learning driven programs can sort through a prospects search histories, cookies and social media sharing. The additional information on a prospects interest, capabilities and behaviors might indicate that a candidate can do things that they haven’t done in the past. Once they apply, chatbots can contact an applicant directly to clarify unclear elements in their resume or profile.
  • Improving job descriptions and postings Recent research data has revealed that job descriptions and job postings can be dramatically improved so that the content better attracts your target audience. So, rewriting them can reduce terms that create a bias. Software can now help you reduce those biases and add content that draws initial attention and that attracts more qualified applicants.
  • Responding to questions – from potential or actual applicants is immensely time-consuming for recruiters. So many firms are already utilizing chatbot’s to answer questions quickly 24/7. The U.S. Army, for example, has been using its Sgt. Star chatbot for over ten years to answer its extremely high volume of questions. Chatbots can also periodically update a candidate status, once again saving recruiters time.
  • Personalize selling – Machine learning uses big data to identify the attraction factors and the elements of the firm’s employee value proposition that best engage certain personas (e., types of individuals). Rather than a “one-size-fits-all” approach, this allows you to make your attraction, marketing messages and personal communications more effective because they are highly personalized to the individual.

Recruiting areas after candidates apply

  • Resume sorting – with machine learning software uses the resumes of successful hires at your firm to find patterns and then it can use these past success patterns as a basis for predicting which resumes and candidates are most likely also to be successful when hired. If programmed correctly, resume sorting software can also help to eliminate a great deal of unconscious bias in resume screening and candidate slate selection. Machine learning assisted search programs can also help you find hidden or lost talent within your ATS database.
  • Matching people and jobs – Using matching programs supplemented by machine learning can help a firm determine if there are any, less obvious, jobs that an applicant would also qualify. Matching people with jobs will also be improved by looking not just at an applicant’s past job titles and degrees, but also at their skills and capabilities.
  • Interview scheduling – is time-consuming and dramatically reduces your speed of hiring. Fortunately, there is existing software that allows a candidate to self-schedule their own interviews depending on their availability.
  • Interviews – can be time-consuming, so it makes sense to automate the initial ones with a chatbot that provides personalized questions based on your job profile. Also, there already exists technology that allows the use of neuroscience tools like voice and facial recognition to assess aspects of video recorded interviews that no humans could detect. There are even voice modulation programs that can help you obscure the voice of telephone interviewees so that it’s harder to identify their gender and national origin.
  • Supplemental candidate assessment – in addition to traditional interviews. Natural language processing can check language skills and online technical tests and challenges can help to assess the skills of applicants. There are automated programs that can more consistently determine cultural fit. Eventually, virtual reality simulations will be able to supplement interviews by giving candidates actual problems from the job to solve.
  • Offer acceptance – based on the candidate’s persona and profile. Recruiters can put together offers that are more likely to be accepted while at the same time treating all genders equally when it comes to compensation.
  • Learning from hiring failures – By definition, machine learning processes continually identify mistakes and errors. Recruiting will have an ongoing failure analysis process that continually and automatically finds hiring and bias errors and their root causes, allowing recruiting processes to improve at a much faster rate.
  • Other technologies – in addition to AI/ML technologies. Block Chain may eventually make checking educational and employment credentials easier and more accurate. Skype and video technologies already make it much easier to interview remote candidates without requiring them to travel. Machine learning will make predictive analytics in the area of projecting the future trajectory of finalists (in the areas of performance, retention and promotions) much more accurate.

Final Thoughts

Although most firms don’t track it, the average failure rate of new-hires at all job levels hovers around 50%. For example, Leadership IQ found that when “they tracked 20,000 new hires, 46% of them failed within 18 months”. Former Harvard Professor and author Michael Watkins reveals that “58% of the highest-priority hires, new executives hired from the outside, failing in their new position within 18 months”. Part of this broad failure results from overworked recruiters, normal human errors and unconscious biases throughout the recruiting process. Fortunately, the machine learning technologies highlighted above will soon minimize those problems through automation and continuous improvement. The results will be hiring faster, lower cost and more importantly hires that perform better on the job (i.e., quality of hire), that are more diverse and with fewer hiring failures. Recruiters should also take note that as more recruiting transactions are automated, it will allow current recruiters to “raise the bar” and to move into the more strategic Talent Advisor role.

Finally, recruiters should also be aware that they will soon be recruiting many more individuals into machine learning roles. The share of jobs requiring AI skills has grown 4.5 times since 2013 (Source: Stanford).

Want to see how machine learning can help you find better technical matches for your open roles?  Check out CodeSignal Recruiter or attend an upcoming webinar.

About the Author:

Dr. John Sullivan is an internationally known HR thought-leader from the Silicon Valley. Specializing in strategic Talent Management solution. He is a prolific author with over 900 articles and 10 books covering all areas of Talent Management. Fast Company called him the “Michael Jordan of Hiring”, Staffing.org called him “the father of HR metrics” and SHRM called him “One of the industries most respected strategists”. He was selected among HR’s “Top 10 Leading Thinkers” and was ranked #8 among the top 25 online influencers in Talent Management. Dr. Sullivan is currently a Professor of Management at San Francisco State

If this article stimulated your thinking and provided you with an accurate picture of the future of technology in recruiting, please take a minute to follow or connect with Dr. Sullivan on LinkedIn.

© Dr. John Sullivan 5/2/18 for Codefights

Diamonds not Dirt

Diamonds not Dirt

Your product team just informed you that they need to hire five Full Stack developers – all by month’s end.

As the request sinks in, you begin to think through the monumental effort that will be required to pull this off. Hiring a single Full Stack developer is hard enough. But five all at once? Seriously?

Most recruiters in your shoes would commence the arduous process of posting ads and combing countless Linkedin developer groups and profiles. Part of you is tempted to do exactly that, but past experience tells you that there’s got to be a more efficient way to recruit talent.

So, what should you do?

Let’s weigh your options.

Does Quantity Ensure Quality? Probably Not.

You’ve probably heard it said at least a dozen times: “Recruiting is a numbers game.” And, if you’re relying on the traditional recruiting model, there’s probably some truth to that statement. After all, if you reach out to 100 developers on LinkedIn.com, you might receive a reply from two or three. Out of those who reply, only a handful will be interested in a career move. In other words, to build a viable pool of candidates, you’ll need to engage with hundreds of people.

This approach to developer recruitment presents a number of issues. Specifically:

Upfront Bottlenecks: Recruiting developers isn’t exactly the easiest thing to do. For starters, there are literally hundreds of programming languages, each of which has its own nuances and complexities. Before your recruiting team can effectively promote any job opportunity, you must become somewhat conversant with the subject matter. This may involve several conversations with in-house experts and existing development staff, who can be notoriously difficult to track down. All of this must happen before any job postings or outreach campaigns can go live.

Cost of Outreach: Once the details have been ironed out internally, your focus typically shifts to an aggressive outreach program. If there’s a budget for it, this may include the use of LinkedIn Jobs ads. With budgets as tight as they are, your team is usually forced to rely on manual tactics. However, filtering candidates, copying, pasting, and sending outreach messages, and updating an ATS (or spreadsheet) comes with a tangible cost. There’s also an opportunity cost worth considering. In a perfect world, your team would be engaging candidates – not spamming them.

Friction with Qualified Engineers: Speaking of spam, let’s be honest: Quality candidates are tired of being spammed by recruiters. As a result, your well-crafted message is probably never even being read, much less responded to. Hence the low response rates.

Few Diamonds, Lots of Dirt: Perhaps the biggest flaw of a “quantity over quality” approach is that it presupposes more dirt than diamonds. Going into it, you know that you’ll need to reach out to hundreds of developers just to get a few responses. Instead of yielding a bucket of dirt to sift through later, wouldn’t a better process deliver a shovelful of 1-carat gemstones? Yes, yes it would.

Focusing on Quality, Not Quantity

It’s clear that quantity does not guarantee quality recruits. But, what other option do you realistically have?

Due to the inefficiencies and challenges of manual, resume-based recruitment, an increasing number of technical recruiters are turning to skills-based recruiting. What is skills-based recruiting? In short, skills-based recruiting prioritizes the engineer’s skills over his/her resume, or LinkedIn profile. As you know all too well, resume-based recruiting is driven by “quantity over quality” because you can’t measure the skills on a resume. So, as a recruiter, you need to have a high number of candidates at the top of the recruiting funnel just to get a few interviews. By contrast, skills-based recruiting moves testing and assessment to the top of the recruiting funnel, providing an accurate measurement of the candidate’s skill level before you even reach out. It’s certainly an outside-the-box style of recruiting, but it’s one that is gaining popularity because of the results that it delivers.

 

Our CodeSignal Recruiter platform is the perfect example of skills-based recruiting in action. We start with more than a million pre-screened software developers. Many of them are actively looking for new positions, and we use our matching algorithm (along with a human touch) to match these engineers to open roles. This means you’re able to source higher caliber talent in a shorter time.

Not surprisingly, companies that have moved to a skills-based recruitment model are experiencing better response rates. Thanks to our proven matching technology, the response rate from candidates is 5 times greater than traditional recruiting channels used to target passive candidates. When the initial response rate is from candidates is higher, all your funnel metrics improve.

Replace Quality for Quantity

Many companies still fall into the trap of making recruitment into a numbers game — just because that’s the recruiting model they’ve used for years — but it doesn’t have to be that way. As you prepare to hire your programmers, give skills-based recruiting a try.

If you’re still feeling skeptical, here’s some additional good news: You don’t have to replace your traditional resume-based recruiting efforts on day one. Try augmenting your traditional process with skills-based recruiting and compare the results.  Check out CodeSignal Recruiter.

CodeSignal Customer Stories: Wizeline

Customer Wizeline uses CodeFights to source, assess, and interview engineers

“You would be wrong to assume that CodeSignal is just like any other other tool to automate technical recruiting on the markets. CodeSignal should be your weapon when it comes to attracting technical talent.” – Vidal Gonzalez, Wizeline CTO

Wizeline builds scalable software solutions (like AI-driven chatbots) for its customers. Since they create software for other companies, they need to make sure that their engineering team is full of innovative, talented engineers who can deliver amazing products quickly.

Wizeline is a truly international company, with offices all over the world. When they started growing their Guadalajara office, the talent acquisition team needed a way to scale up their engineering team quickly. But they didn’t want to hire just anybody – they wanted to hire strong, innovative engineers. This meant that they needed a strong recruitment system.  They researched recruiting tools that could help them reach their hiring goals. Ceci Salazar, the Talent Acquisition Manager at Wizeline, heard about CodeSignal Recruiter from a coworker.

Ceci and her team tried CodeSignal Recruiter’s sourcing service. The Wizeline engineering team tried out the Test and Interview tools. Together, they agreed that CodeSignal Recruiter would be a great way to streamline their technical recruiting process. Vidal Gonzalez, Wizeline’s CTO, says that CodeSignal’ tools are a great fit for Wizeline’s engineering-driven company culture.

Talented engineering candidates

Wizeline uses CodeSignal Recruiter Source to get access to pre-qualified engineers from the CodeSignal community. Candidates they get from CodeSignal, like Ziad Mohamed, tend to have higher conversion rates. The Wizeline talent acquisition team loves knowing that the candidates they get through CodeSignal are driven and challenge-oriented, and that they love to keep learning. The team’s recruitment process has been streamlined and simplified with CodeSignal Recruiter.

Skills-based recruiting

It’s simple for their recruiting team to send out coding tests since so much of the process is automated. This means the recruiting team can immediately assess inbound applicants and prospects who have been sourced from online job boards. Since they are testing candidate skills at the very top of the recruitment funnel, this eliminates unqualified people right away. Ultimately, this saves the engineering team a lot of time, since they don’t need to spend time screening or interviewing candidates who don’t have the necessary skills.

Data-driven decisions

Vidal loves how much data CodeSignal Recruiter gives to his hiring team. While the candidate information is useful, he also appreciates the level of insight that data from the hiring team gives him into their interviewing process. So if the statistics from an individual interviewer looks abnormal, he and his team can dig into the interview data to see what’s going on. The standardization and consistency that they get using the Test and Interview tools means that he feels confident in the decisions that his team makes about candidates.

Uncovering diverse talent

CodeSignal Recruiter lets Wizeline to focus on skills instead of educational or work credentials. So it’s gone a long way towards eliminating hidden biases in the recruiting process! As a result, their engineering team is extremely diverse, which supports one of Wizeline’s core hiring goals. (Interested in working for Wizeline? Take a look at their Careers page!)

A partner in recruiting

Wizeline considers CodeSignal to be an valuable partner in their recruitment process. They appreciate the close relationship they have with their CodeSignal Account Executive, who’s always available to answer questions and help out. Ceci says, “The relationship we have built with CodeSignal is really valuable. We feel like they really know our business, so they have become partners for our growth.”


Since they started using CodeSignal Recruiter, Wizeline has sent out 348 coding assessments with the Test tool. They’ve also conducted 152 technical interviews using the Interview tool. Their recruiting and engineering teams have saved time and energy. CodeSignal Recruiter has helped them streamline their entire technical recruiting process!

Ready to see how CodeSignal can help you grow your engineering team too? Sign up for a free demo. Our team will walk you through the ways that CodeSignal Recruiter can streamline your recruiting process too. CodeSignal Recruiter makes it easy to hire the talented engineers you need to grow your company.

The Culture Fit Trap

Don't Hire Engineers for Culture Fit - Hire for Skills

Companies – especially companies in Silicon Valley – have long prized the idea of culture fit. Culture fit is, as a concept, a little bit nebulous, but in general it refers to the idea that a candidate “fits in” with the existing company and team dynamics. On the surface, this seems like a great idea. After all, why wouldn’t you want to hire someone who seems like a good match for your engineering team? The thinking is that they’ll integrate into the team more easily and have easier interactions with their fellow team members.

So what’s the problem?

The issue is that recruiting and hiring based on culture fit has an unintended consequence: it leads to homogenous engineering teams. This can negatively impact your engineering team, your company culture as a whole, and your bottom line.

Hiring for culture fit is based on the idea that people who have similar viewpoints and ways of working will form a cohesive team. But this premise, if left unchecked, leads to teams full of people who have similar backgrounds, viewpoints, and working styles – and a glaring lack of diversity. (And it’s not just the kinds of diversity that usually get talked about, like gender and race, that suffer. It also homogenizes things like experience, age, working style, problem-solving methods, and more.) If interviewers prioritize finding people they think they’d get along well (and easily) with, then they’re deprioritizing other, more important factors like technical skills. Remember, they’re trying to hire engineers, not friends!

Prioritizing culture fit creates engineering teams full of people who are very similar to each other, in an industry that already suffers from a noticeable lack of diversity. Culture fit often simply codifies a hiring team’s unconscious biases. After all, people tend to want to hire people like themselves! As Tigran Sloyan, the CEO of CodeSignal, put it:

The biggest problem with diversity in tech is that humans are too involved in the skill evaluation process. We tend to like people who have a similar background to ours, which creates a self-reinforcing cycle.

In an effort to find candidates who “fit in” well with your company’s engineering team, odds are you’ll end up recruiting people who come from very similar backgrounds. Maybe this means that they went to the same few schools, or worked at the same handful of companies. And relying on employee referrals, a common practice in many companies, can exacerbate this problem. Employees often refer friends or people from their social circles – another form of culture fit.

Culture fit moves from being fairly innocuous (“Is this person’s working style similar to mine?”) to being problematic (“Does this person come from a background like mine?”) easily, and often unnoticed.

Who gets left out?

Think about who gets omitted when you recruit based on culture fit: People who went to the wrong school – or no school at all. Developers who’ve spent their career in a different industry. Candidates who just don’t “look like an engineer” or “act like a software developer.” People who have different working styles or needs. The list goes on and on. Hiring for culture fit tends to reduce your candidate pool down to a small, homogenous group. And it does little towards increasing diversity at your company.

The benefits of diversity

We often think about diversity in terms of race and gender. But the term also covers age, background, experience, points of view, working and communication styles, and talents. A truly diverse team won’t be homogenous on any of these points, and everyone will bring unique perspectives and ideas to the table.

This means that a team composed of people who have different backgrounds and experiences will generate more – and more interesting ideas – simply because of the fact that they have different points of view. Innovation will blossom. And once your company truly commits to supporting diversity, a new form of self-reinforcing cycle will start. But it will be a good one this time! A team that is already diverse is seen as more welcoming of diversity. So the kind of diverse talent that you want to attract will be more interested in joining your company.

The benefits of diversity can be fiscal as well. According to several studies, diverse companies are often both more successful and more profitable than non-diverse companies.

5 steps to avoid the culture fit trap

So how can your company avoid falling in to the culture fit trap in your recruiting and hiring processes?

1. Focus on skills

The number one thing to do to avoid the culture fit trap? Prioritize skills instead. This is the primary goal of skills-based recruiting, of course. By putting skill assessment right at the top of the recruiting funnel, you ensure that only qualified candidates make it to the interview stage. Coding assessments that use machine learning to quantify technical skills, like the ones the CodeSignal Recruiter Test application supports, are great for this. No one can argue with numbers!

2. Make sure they’re the right skills

However, make sure that your company’s coding assessments don’t reinforce existing biases. You don’t want a situation where the engineers who set up the coding assessments bring their own beliefs into the mix! If their unconscious biases skew towards believing that only computer science graduates from top schools can do a job well, their assessment will reflect that. But if a company is honest and objectively thinks about the skills the job actually requires, they can create a coding assessment that will filter for people with the right on-the-job skills, regardless of race, age, gender, orientation, or background. (Learn more about how to craft a coding assessment that will test for the right skills.)

3. Be specific with feedback

Don’t let hiring teams rely on culture fit when they’re deciding whether to move on to the next stage with a candidate. When they’re reviewing a candidate, they need to provide specific feedback that relies on objective data from the screening or interview. This keeps the focus on the candidate’s skills, instead of allowing imprecise “gut feelings” to determine whether they get hired or not.

4. Don’t mistake soft skills for technical skills

Don’t go overboard on letting a candidate’s soft skills influence your decisions. One reason that the culture fit trap is so insidious is that we’re hardwired to want to hire someone who’s likeable. But just because someone communicates well and gets along with the team doesn’t mean they have the necessary technical skills. That’s why it’s important to assess skills objectively!

5. Think about value add

Of course, intangibles and soft skills are important too when you’re recruiting engineers. If a candidate is a good programmer but was rude during an interview, you probably wouldn’t want to hire them. But how can you avoid relying on culture fit when you’re considering a candidate’s soft skills? By thinking in terms of value add instead. What will this person add to your team? If you look at recruiting and hiring this way, it’s easy to see how bringing in people with diverse backgrounds and experiences will be beneficial to your engineering team.


Remember: Focus on a candidate’s skills, but make sure you’re measuring the right skills. Make your hiring team give specific feedback that relies on data. Don’t be swayed by a candidate’s soft skills. But do consider what their value add to the team will be! By following these five steps, you can avoid the culture fit trap in your company’s recruiting and hiring processes. This will create an environment in which engineers from diverse backgrounds will be excited to join your team!

CodeSignal Recruiter is a skills-based recruiting tool for modern hiring teams that helps companies source, test, and measure technical talent. We’re on a mission to make sure that you’re only talking to the best candidates at every part of your hiring process.

CodeSignal Recruiter gives your hiring team the tools you need to stop hiring for culture fit, and start hiring based on skills. Interested in seeing what CodeSignal Recruiter can do for you? Sign up today for a free demo!

Test the Right Skills With Your Coding Assessments

Creating coding tests to assess developers

The initial coding assessment is a crucial component of any technical recruiting process. It allows you to weed out unqualified candidates at the top of the funnel. In turn, this gives you and your team more bandwidth to concentrate on the qualified ones. It also lets your candidates learn a little more about the role and your company. This leads to a better candidate experience, which is a key component of keeping top talent engaged in your process.

But it can be surprisingly tricky to put together an initial assessment that actually tests the skills necessary for the role at hand – without creating an undue time burden for your engineering team. There are four major factors to consider when you’re creating a coding assessment: format, content, length, and ease of management.

Here’s how to use these four considerations to create a coding assessment that tests for the skills that actually matter for the role.

Formatting the assessment

When it comes to the format of the initial assessment, there are a few common options:

  • Phone screens: Many companies do a technical phone screen as their initial coding assessment. This works, but it takes a lot of time and energy away from the engineering team in 30 to 60 minute increments! And since this is very close to the top of the recruiting funnel, the possibility that these candidates won’t meet the necessary technical bar is very high. Another issue is that it’s difficult to standardize the phone screen process. Most engineers have a preferred way of asking questions, and many have their own off-book questions that they like to ask as well.
  • Take-home projects: Some companies head straight to sending candidates a take-home project. Though take-home projects require less direct candidate interaction, they take as much time and energy from your engineering team as phone screens – if not more! Consider the time involved in managing and scoring these candidate projects. Take-home tests are great, but it makes much more sense to send them later in the process, when the candidate pool is much smaller.
  • Coding tests: For sheer ability to weed out unqualified applicants with minimal hands-on management, the ideal initial technical assessment is a take-home coding test. An engineer can set up the test initially, then recruiters can send them out to applicants at scale. This saves both recruiter and engineer time, while still testing for the skills necessary to succeed on the job. For most roles, this should take the form of solving a coding task, debugging some existing code, or both.

Creating the assessment

When you’re screening candidates, it can be tempting to test whether they can reverse a linked list and leave it at that. But is someone in the job you’re hiring for ever going to need to reverse a linked list? For many roles (think front-end, database, DevOps, and more), the skills that the role requires might be miles away from the ones you’re currently screening for. This means you could be missing out on some amazing candidates.

In order to create a meaningful coding assessment, the obvious (but often overlooked) first step is to figure out what you should be testing candidates on! Align with the rest of the hiring team on what the role actually entails. What skills does a candidate need to have in order to be successful in this role? Be careful not to get bogged down in nice-to-haves – focus on the fundamental need-to-haves instead.

Once you’ve got a solid list of these skills, think about how to test for them. If you’re sending out a coding test, which is our recommended initial assessment step, then you should identify 2-4 coding tasks that directly correlate with the necessary skills. Your aim here is to establish a baseline level of skills that a candidate must exceed in order to move to the next step. So your assessment doesn’t have to test for every single skill that you listed earlier – just the core ones. You’re creating a threshold that will weed out the unqualified candidates while allowing the qualified ones through.

Assessment length

There’s a common idea in the tech world that long or involved assessments will automatically weed out applicants who aren’t serious about the role or your company. But the reality is that these people are busy! They’re probably working full time at a job, or are already working on multiple assessments for other companies. Or both! By creating assessments that take forever to complete, all you’re really doing is driving potentially great candidates to other companies that have less onerous screening processes. Keep it short and sweet. For the initial screening, create a test that will only take the applicant about 30-60 minutes to complete.

Ease of management

Think about how you’ll send, receive, and score these assessments. Ideally, you won’t have to spend a lot of time managing them. Some companies ask their candidates to email them code snippets or to upload them to GitHub, but this typically requires a lot of oversight and management. Then, of course, there’s the time that someone on the engineering team will spend looking the tests over!

For maximum ease of use, send out tests that get automatically scored when the candidate completes them. Ideally, the testing system would also show solution replays and flag instances of potential plagiarism. (Oh, wait, there’s an application that does all these things already – CodeSignal Recruiter Test!)

Next steps

Sending a coding test as the initial assessment is an amazing time-saver for both your company’s recruiting and engineering teams. Since you’ll have already weeded out people who don’t exceed the technical bar for the role, you won’t have to worry about passing unqualified candidates along to your engineering team. Different companies will handle the subsequent steps differently, but in general the next step should be to either get the candidate on a phone screen with an engineer or send them a take-home project. Both of these methods have pros and cons, with the cons chiefly being the time necessary to manage them.

The future of technical assessments

Luckily, we’re extremely close to a future in which any of these intermediate steps will be unnecessary. Once skill verification scores like the ones provided by CodeSignal become more familiar to recruiters and hiring teams, companies will feel comfortable sending candidates who pass their coding assessment straight to onsite interviews. Since they’ll know that the candidates have the skills they need, at the level they need, they won’t need to bother with any intermediate skill verification steps like technical phone screens or take-home projects. This will save everyone in the company time and will make candidate experience much faster and simpler.

CodeSignal Recruiter is a skills-based recruiting tool for modern hiring teams that helps companies source, test, and measure technical talent. Founded in 2014 and based in San Francisco, the CodeSignal mission is to make sure that you’re only talking to the best candidates at every part of the recruiting funnel.

CodeSignal Recruiter gives your hiring team the tools you need to create, send, and manage coding assessments quickly and easily. Interested in seeing what CodeSignal Recruiter can do for your company? Sign up here for a free demo!

Introducing the CodeSignal Sourcing Assistant

We’re very excited to announce the launch of our new CodeSignal Sourcing Assistant!

This new sourcing tool predicts whether a prospect on LinkedIn will be a good match for your open engineering roles. The Sourcing Assistant is powered by machine learning to make recruiting faster, easier, and more effective.

How the Sourcing Assistant works

The CodeSignal Sourcing Assistant is a Chrome extension that’s available only for CodeSignal Recruiter customers. In the background, the Assistant analyzes the necessary skills for a company’s open role. Then when a recruiter sources potential candidates on LinkedIn, the AI-driven Sourcing Assistant automatically scans their profile pages. It instantly identifies the skills that match those needed for the open developer roles! Then it uses this information to generate a matching score for each candidate.

Since recruiters and sourcers don’t need to manually read or scan each profile, they’re able to find more qualified engineering candidates, faster. The tool frees up huge amounts of time for the recruiting team!

CodeSignal Sourcing Assistant matching score

Use the data you’re already collecting

The CodeSignal Sourcing Assistant also goes beyond matching skills listed in job descriptions with skills listed in LinkedIn profiles. It’s actually using data from the technical assessments a company has already done in CodeSignal Recruiter to go deeper! If the engineering team says it’s looking for certain skills, but the skills they’re testing for in their assessments are different, the assistant accounts for this. This means that the engineers a recruiter sources from LinkedIn using the assistant are more likely to be liked (and hired!) by the engineering team.

The Sourcing Assistant lets companies use CodeSignal Recruiter data to streamline the sourcing process by extending the use of this data to LinkedIn. By identifying promising candidates, and flagging those that aren’t a good fit, CodeSignal empowers companies to calibrate their hiring process between the recruiting and engineering teams.

Reduce bias

The Sourcing Assistant also helps recruiters and hiring managers reduce bias and increase diversity in their engineering teams. The reality is that unconscious biases creep into all decision-making processes. A recruiter might prioritize people who went to a certain school. Or a hiring manager may only want to interview people who worked at a certain company. In contrast, the Sourcing Assistant only looks at a prospect’s skills. This surfaces “hidden gem” candidates that a hiring team might not otherwise look at because they don’t fit the usual profile.

Bridge the gap between recruiting and engineering

The CodeSignal Sourcing Assistant has a great side benefit: improving the relationship between a company’s recruiting and engineering teams!

CodeSignal CEO Tigran Sloyan says, “There’s a huge gap between the recruiting and the engineering teams in most companies, and it can seem like they are speaking different languages. The Sourcing Assistant bridges that gap. Since the assistant is using data to generate its match scores, recruiters can rest easy knowing that they are prioritizing the things that matter most to the engineering team for each role.” 

Learn more!

The CodeSignal Sourcing Assistant is available for CodeSignal Recruiter customers, since it relies on data from the CodeSignal Recruiter platform. Click here to get a free demonstration of CodeSignal Recruiter!

Stop Focusing on Degrees – Recruit for Skills Instead

Recruit for skills, not for credentials or pedigree

The tech industry’s talent shortage is no secret, but let’s go over the numbers again just for the heck of it.

According to data collected by Code.org, there are over 500,000 unfilled technology-related jobs right now. But only about 49,000 people graduated into the workforce from computer science programs in 2017. The numbers are clear: There just aren’t enough computer science students graduating each year to fill all the available roles.

And it doesn’t look like things will change anytime soon. The U.S. Bureau of Labor Statistics has predicted that in 2020, there will be 1.4 million open technology jobs, but only 400,000 people will graduate from computer science programs.

So how can companies find enough qualified people to fill their open engineering roles if there aren’t enough students graduating from computer science programs to go around?

By looking at candidates who don’t have a traditional computer science background.

It can be nerve-wracking for recruiters to reach out to candidates who don’t have a computer science degree from a 4 year program. But it’s absolutely worth it! The past 10 years have seen a revolution in the way that people learn technical skills. Whether they’re learning computer science fundamentals or mastering in-depth topics, there are a plethora of new ways for people to get the skills they need. And this new educational model is democratizing computer science.

The new educational landscape

In the last decade, online educational resources have grown exponentially, both in quantity and quality. Platforms like Udacity, Coursera, and edX offer free online courses from big-name schools like MIT. These services also offer students the option to get a professional certification when they finish one of these online courses. Khan Academy, FreeCodeCamp, and Treehouse abandon the online classroom format in favor of more interactive learning experiences. And YouTube has a massive amount of free content. (Including educational videos from CodeSignal!) For the self-motivated learner, ways to learn online for free or for fairly nominal fees abound. And online and onsite coding bootcamps offer a more hands-on approach, for a fraction of the cost of a four-year computer science degree.

No matter what platform they choose, these learners have a wealth of information at their fingertips. But what they don’t have are the traditional learning credentials. These degrees or school names are what recruiters often look for when they’re sourcing prospects or looking at applicant resumes.

The case for non-traditional candidates

Programmers with non-traditional backgrounds don’t have the educational qualifications that recruiters usually look for. Their resumes and their LinkedIn profiles will reflect this, often placing much more emphasis on personal or open-source projects than on educational or work experiences. But these candidates can be just as skilled as ones who have the “right” markers! This means they can be the solution to the tech talent shortage facing the industry today.

If companies only consider candidates with traditional pedigree markers to fill their open roles, then their pool of available prospects will be fairly small. And the competition for these pedigreed candidates is fierce. Of course, none of this is to say that people who do have these credentials aren’t great candidates! But when companies limit themselves to just these people, they miss out on amazing “hidden gem” candidates.

Recruiters need to be able to reframe how they think about finding prospects and what to look for when when they’re considering candidates. The best way to do this? Focus on skills, not on credentials.

People can (and do) list any old skill on their resumes and LinkedIn profiles, so you can’t rely on them to tell you the full story. It’s crucial to be able to verify these skills before moving forward with a prospect. Phone screens, take-home projects, or interview tasks can all help verify skills. But the best, and most efficient, way of verifying skills is at the very top of the funnel, even before a phone screening. A coding test that is emailed to prospects and delivers automatic results back to the company, like those sent from CodeSignal Recruiter Test, can streamline the recruiting process because recruiters are able to verify skills right away.

Education has changed, and recruiting has to change as well. It’s time to stop prioritizing educational credentials. Start measuring people by what they can do in a data-driven, skills-based way. You’ll uncover a treasure trove of amazing candidates!

CodeSignal Recruiter is a skills-based recruiting tool for modern hiring teams that helps companies source, test, and measure technical talent. Founded in 2014 and based in San Francisco, the CodeSignal mission is to make sure that you’re only talking to the best candidates at every part of the recruiting funnel.

If you’re ready to discover how candidates with non-traditional educational backgrounds can contribute to your company, CodeSignal Recruiter can help. Sign up for a free demo and find out how!

Find Engineers Your Hiring Team Will Love

Skill Verification Will Revolutionize Your Tech Recruiting

Technical recruiters, does this scenario sound familiar?

You work hard sourcing candidates, looking through resumes, and reaching out to high-potential prospects in order to find engineers to fill your company’s crucial open roles. But sometimes you feel like your efforts are for nothing: The engineering team rejects most of the likely candidates you find for them. Maybe they reject them right away, just based on their LinkedIn profile or resume. Or maybe it’s later on, after a phone screen or interview reveals the candidate doesn’t have a key skill. You start feeling under-appreciated and underutilized. The hiring manager and engineering team start feeling frustrated. The relationship between your teams starts becoming a little… strained.

How can you repair this relationship and find engineering candidates that the team actually likes (and hopefully wants to hire)?

To find a solution, it helps to dig into the two main underlying problems:

Problem 1: You’re spending a lot of time sourcing candidates, but to your engineering team, they don’t look “right” – wrong school, no school at all, or an unconventional background. There might be some unconscious biases at play here, or even some very conscious ones. (How often have you heard a hiring manager say that they only want to interview candidates who graduated from a top 20 engineering program, for example?)

Problem 2: You’ve put a lot of time and energy into getting candidates to the phone screen or interview stage, only to have them choke because they don’t pass the technical assessment. Of course, they should be bounced out of the process if they don’t meet the technical bar. The problem is that too many of your candidates aren’t making it, and the engineering team is getting frustrated because they’re wasting valuable time interviewing unqualified candidates.

If one or both of these are happening to you, your recruiting process slows down to a crawl – possibly jeopardizing your recruiting stats and preventing you from meeting your goals. So what’s the solution?

Add skill verification to your recruiting toolbox.

Skills speak for themselves. If the hiring manager has data right away that a candidate has the experiences and skills they need, then they’ll stop worrying about surface-level things like credentials and stop rejecting your candidates at the top of the funnel. And with skills-based hiring, you won’t have to worry that your candidates aren’t meeting the engineering team’s technical bar because you’ll know for sure that they do, even before the phone screen or interview.

[bctt tweet=”With skills-based hiring, you don’t have to worry that your candidates aren’t meeting the engineering team’s technical bar – because you know for sure that they do.” username=”CodeSignal”]

Sourcing candidates

Ideally, you have skill verification at the very top of your recruiting funnel, at the sourcing stage. This solves a lot of problems immediately by removing questions about skill. This is especially important to hiring managers when they’re considering candidates with non-traditional backgrounds. To do this effectively, you’ll probably need to subscribe to a service that gives you access to pre-screened candidates.

Pre-screened is the key idea here. Many sourcing products on the market give you candidate access. But there’s not a skill verification process to ensure that the candidates they send you actually have the skills they say they have. And the same problem crops up with recruiting agencies. Their candidate recommendations are based on the same techniques you’re trying to get away from (sourcing candidates based strictly on self-reported and unverified declarations of skill).

The Source application on the CodeSignal Recruiter platform solves this by having a huge pool of diverse, skilled users to pull from, then building an additional verification step on top of that. If you use a tool like this, you can be confident that the candidates you submit to the hiring manager have already proven their technical skills, allaying any worries they might have about them.

Assessing candidates

But maybe you’re not using a service like CodeSignal Recruiter Source that provides you with pre-vetted candidates. If not, it’s important to confirm their skills before engineers ever have to interact with a candidate. As a recruiter, you might not be technical yourself. So you need a tool that will both let you send assessments and interpret the results. This way, you can weed out unqualified candidates early, saving your engineering team’s time and energy.

Again, there are some services on the market that can help recruiters send out technical assessments. But the ability to select coding tasks from a huge pool of professionally-written questions, interpret candidate results, automatically screen for plagiarism, and send coding replays to your colleagues aren’t as common. CodeSignal Recruiter gives you access to all of these features with the Test application. CodeSignal Recruiter Test integrates with online applicant tracking systems like Greenhouse and Lever. This allows you to send and manage technical assessments right from the platform you’re already using.

Skill verification changes everything

When you can get to the point where you know that your candidates will meet your engineering team’s technical bar, you can feel more comfortable in knowing that your team will like them. The results will speak for themselves. Your phone screen to onsite rate will go up, as will your onsite to offer rate. (On average, CodeSignal Recruiter customers see a screen to onsite rate of 71% and an onsite to offer rate of 25%. Some customers have onsite to offer rates of over 40%!)

The hiring manager and engineering team will start trusting your recommendations. You’ll save time. The engineering team will save time and energy. Your company will make more quality hires. Everyone will be happier!

CodeSignal Recruiter helps you hire more qualified engineers with much less effort. It’s a win-win for both recruiters and hiring managers!

CodeSignal Recruiter is a skills-based recruiting tool for modern hiring teams that helps companies source, test, and measure technical talent. Founded in 2014 and based in San Francisco, the CodeSignal mission is to make sure that you’re only talking to the best candidates at every part of the recruiting funnel.

By supporting skills-based recruiting best practices, CodeSignal Recruiter gives your hiring team the tools you need to find the right developers for your company’s open roles. Interested in seeing what CodeSignal Recruiter can do for your recruiting process? Sign up here for a free demo!