Skip to content Skip to footer

How to run a structured interview with CodeSignal: A guide for technical interviewers

Structured interviews are interviews that use a consistent set of questions, phases, and evaluation metrics across candidates. Research has shown that structured interviews allow employers to successfully predict how candidates will perform on the job. Even more, they help reduce bias in hiring by making the interview process more objective and consistent

In a recent blog post, we described the building blocks of running an effective structured interview: the pre-brief, the interview session, and the debrief. CodeSignal Interview fits seamlessly into this process by empowering hiring teams to conduct structured and collaborative technical interviews in a cloud-based platform.

In this practical guide, we describe how to use templates in CodeSignal to conduct an exceptional structured interview, from pre-brief to debrief.

What is CodeSignal Interview?

CodeSignal Interview is a comprehensive cloud interview solution that enables your team to conduct all technical interviews remotely, without compromising quality or candidate experience.

Other technical interviewing solutions are more limited in scope and tend to be “free-form” coding platforms, which can make it challenging to conduct a structured interview. CodeSignal Interview is the only assessment solution on the market designed to bring structure, consistency, and scale to the technical interviewing process.

Preparing for the interview

Before diving into the interview session, we recommend that the interviewer (or interviewers) log in to CodeSignal and get comfortable with the coding environment. For engineers, the CodeSignal IDE – powered by Monaco Editor – should be familiar. It’s very similar to Visual Studio Code, including all the bells and whistles like autocomplete and syntax highlighting.

Once the interviewing team is comfortable with the tool, they’ll need to align on which tasks they’ll ask candidates to complete during the interview. CodeSignal Interview supports a wide variety of task types for both general and highly specific technical interviews – from algorithmic, to front-end, to mobile development, and more. You can choose tasks from CodeSignal’s extensive task library or create your own custom tasks that exactly fit your use case.

Whenever you run a structured interview, you need to be sure to ask the same questions of each candidate, in the same order every time. That’s why we created templates for CodeSignal. From your Interview Dashboard, you can easily create a template that consists of the tasks you want to use during the interview, in your desired order. 

Collaborating during the interview

Going into a structured interview, your team needs to know what questions your team will ask, what order you’ll ask them in, and who is responsible for which parts of the interview. (If you hold a pre-interview meeting with your interviewing team, that’s a great time to get these details sorted out.)

To conduct a structured technical interview in CodeSignal, just start the interview session with your candidate (CodeSignal accommodates up to three interviewers) and launch the appropriate template. That’s it!

When you’re ready to move on to the next question, the template will guide you to the next task in your list. 

One of the keys to a successful structured interview is staying human: you want the candidate to feel comfortable and confident coding during the interview, just like they do at home. So we built powerful tools into CodeSignal to help you make your structured interview feel natural, and not like a robotic testing environment. Here are a few of the many ways you can collaborate with candidates in CodeSignal Interview:

Communicating face-to-face with built-in video calling

While technical interviews are first and foremost tools to assess candidates’ coding skills, they also serve other key purposes. For interviewers, they provide an opportunity to understand how a candidate communicates, collaborates, and problem-solves in real time. And for candidates, they offer a glimpse of your company and team culture that will help them decide whether the opportunity they are interviewing for is a good fit.

A cloud-based IDE with built-in video calling allows candidates and interviewers to not just code together, but to communicate face-to-face at the same time. For candidates, seeing a smiling face in the corner of their screen creates a more personal and human interviewing experience. It’s also a chance for candidates to (literally) see who they’d be working with if they get the job!

Pair programming in a terminal window

Now, onto the coding part of the interview. An advanced IDE is a key component of an effective technical interviewing tool, but an IDE alone can’t simulate a real software development experience. You also need a terminal.

CodeSignal Interview gives candidates and interviewers access to a shared, collaborative terminal that pairs with our IDE, enabling hiring teams to evaluate complex and real-world software development skills. Our terminal allows you to run a command-line shell (REPL), install software, run code and debuggers, start services, write to files, and more, making it significantly more powerful than the basic REPL shells supported by other interviewing tools.

Interacting live with a mobile device emulator

If you’re interviewing mobile developers for your team, you’ll want to know that candidates can develop an app that actually works. CodeSignal Interview is the first and only technical interviewing tool on the market to allow for collaborative development of a fully-functional mobile app within a web browser.

The mobile device emulator on the right-hand side of the window is a virtual mobile device that can do almost everything a real mobile device can. The emulator runs the mobile application that the candidate and interviewer(s) will collaborate on.

When a candidate or interviewer makes changes to the code, those changes can be seen in real-time in the mobile device emulator. Even better, both parties can interact with the app to ensure it behaves the way it’s supposed to. 

Visualizing data with Jupyter Notebook

For all their powerful capabilities, one thing even the most advanced IDE cannot do is create data visualizations—a key skill to evaluate if you’re hiring data scientists. Most technical interviewing tools only support text output, making it difficult to conduct effective technical interviews with data scientists (this is also a problem if you’re interviewing frontend engineers).

CodeSignal Interview, however, supports Jupyter Notebook, the preferred tool among data scientists for writing code and visualizing data. Just click the Open Jupyter button within the interview session and start collaborating with the candidate in a Jupyter Notebook in the web browser.

Regardless of the specialized features you use, the entire CodeSignal interview session comes with keystroke recording to allow for later review by the interview team.

Interview debrief

Just as important as running a structured interview session is giving feedback in a structured way during the debrief. What good is asking the same questions of each candidate in a consistent order, after all, if the debriefs for each candidate are inconsistent? 

Two key practices can keep your debriefs focused and structured: use rubrics for feedback, and be as objective as possible. Rubrics keep your hiring team focused on elements of the interview relevant to the candidate’s qualifications for the job, minimizing the potential for bias in their feedback.

Guidelines for creating a technical interview rubric

Every role and every engineering team is different, so there is no “one size fits all” technical rubric. There are, however, some universal principles that should guide your approach for developing a rubric for any kind of technical interview.

  1. Use a numerical system. Numbers allow you to calculate a final score. Without numbers, you risk ending up in the nebulous territory of “pretty good” and “okay.”
  2. Spell out what each score means. If you’re using a range of 1-4 for each item in your rubric, for instance, give a clear description of what counts as a “3” coding syntax score versus a “4.”
  3. Include both technical and communication skills. Collaboration and communication may be “softer” skills, but that doesn’t mean they shouldn’t be scored on the rubric. If they matter for your decision-making, include them on the rubric.
  4. Calibrate your rubric. We recommend having all interviewers on your team score the same interview independently. Then they’ll compare scores, discuss differences, and (if you have time) repeat. This process will help ensure that everyone is using the rubric in the same way.

Looking for an example? We like this one for technical interviews.

Staying objective

Staying as objective as possible means referring back to what actually happened in the interview. With CodeSignal, the team can easily share and play back the interview session, and even step through and run the code that the candidate wrote. Rather than relying on notes, which can be partial and subjective, you can easily replay relevant parts of an interview session to help make a decision.

CodeSignal also provides comprehensive data on how your entire candidate pool has performed across various tasks in your database. This can be useful when you want to calibrate a candidate’s score. Even if a candidate failed a few test cases, they might have actually performed well compared to the average person on a particularly hard task.

Exceptional structured interviews bring consistency, objectivity, and fairness to the interviewing process. With CodeSignal Interview, structuring a technical interview is simple and intuitive. This frees up your hiring team to focus on what matters most: finding the best candidate for the job.

Learn more

Want to learn more about how you can build a winning organization through data-driven recruiting? Visit CodeSignal to find out how you can measure technical skills effectively and objectively with automated assessment and live interview solutions.