When building a platform where candidates write code in an interview, there are several ways you can design the architecture. Perhaps the most straightforward approach would be to build an IDE interface and run the code in a separate microservice. You might also include a lightweight shell for executing small functions in-line.
By and large, this is the way that most interview platforms are designed today. CodeSignal started off on this path as well. We built a full-featured IDE and powerful distributed services for running code. But we thought there was still more to do in order to enable the most realistic development capabilities in a remote interview. So, last year, we deployed a brand new architecture using persistent sessions.
The idea of persistent sessions is that instead of sending the code off to run on a microservice, we run the entire interview in its own Docker container and provide full access to configure the environment via a terminal. Persistent sessions are currently used in CodeSignal’s free-coding tasks, and we’re actively extending this architecture to more contexts and building additional features on top of it (stay tuned!). In this article, we’ll explain the motivation behind persistent sessions and how they work.
The demand for more realistic interviews
There’s a growing demand for remote interviews that go deeper to resemble the actual everyday work of programming. In recent years, companies have been rethinking their algorithmic brainteasers and asking more practical challenges related to on-the-job skills. This is a trend in the right direction for reducing bias and improving the experience for candidates, especially more senior developers.
The COVID-19 pandemic has also changed the interview landscape. In the past, many companies (especially startups) relied heavily on onsite interviews where candidates would sit down at a desk in the office and essentially join the team for a half-day. They’d write code on a local machine and maybe pair-program with the interviewer to implement a realistic service or mini-application. Now, as hiring has diversified and increasingly gone remote, every team has been trying to figure out “virtual onsites.”
As we looked at the existing solutions out there for a “virtual onsite,” we saw that they fell into a few categories:
- Using screen-sharing with the candidate’s computer. This didn’t enable live collaboration, required the candidate and interviewers to manage multiple windows, and was prone to glitches.
- Asking the candidate to do a take-home project, usually by pulling something from the company’s Git repo. This wasn’t collaborative, could be a logistical and security nightmare, and tended to be a major time commitment (so it wasn’t great for candidates with families and existing jobs).
- Falling back on whiteboarding or algorithmic questions. This left candidates feeling like they didn’t get a chance to show their real skills, and didn’t provide a great signal for interviewers.
The idea of persistent sessions
As we thought about how we could create a better interview experience and solve the “virtual onsite” problem, we had the idea of adding a terminal feature that would let developers fully customize their environment.
After all, we built CodeSignal to give companies a reliable way to measure all kinds of developer skills. And there are some skills that you can only capture by handing the developer a terminal. For example, is the person comfortable with UNIX? Can they start a service and communicate with it?
We also want candidates to shine whenever they use the CodeSignal platform. And many developers are simply more at ease when they’re working with the command line. Asking them to code with an IDE and no terminal can feel like asking them to tie their shoes with one hand behind their back.
To make the terminal work, we built persistent sessions. And this architecture has many more applications as well. For example, with persistent sessions we can let candidates develop mobile applications in an interview without having to wait a long time for the app to build between changes.
How persistent sessions work
When working on a free-coding task in CodeSignal, the interview participants are in a completely isolated environment that they are free to modify however they like, using the command line. It’s a remote environment that will look the same for both the interviewer and candidate as they collaborate live.
Essentially, we start a Docker image inside a server and leave it up until the interview session ends. All the client-server communication happens through a WebSocket for super-fast performance. The great thing about persistent sessions is that we can set up the environment specifically for the interview language and task. For example, if the language is JavaScript, the environment will have many npm packages preinstalled, which can be previewed by running npm list -g. And if you don’t see a package that you need, you can easily install it. Practically anything you can do on your local machine, you can do here. You can even use a text editor like vim from the command line.
Another advantage of persistent sessions is that the architecture is automatically scalable. The parts of our platform that use persistent sessions can easily handle high loads because every session is independent and doesn’t use shared resources.
Conclusion
Without an architecture like persistent sessions, remote interviews will always be somewhat restricted for developers, no matter how many features they have. Persistent sessions allow interview participants to completely customize their environment using a terminal. Furthermore, participants have a shared view of the terminal and a recording is even saved for the team’s reference when making a hiring decision.
We’re very excited about what else we can do with persistent sessions. We believe that this functionality is the key to building not just the next-generation remote interview platform, but even the next-generation IDE. We envision an IDE that feels like programming locally, but with all the collaborative power of working in the cloud.
If you’re interested in helping build the future of remote technical hiring, take a look at our careers page.
Albert Sahakyan is a Senior Software Engineer at CodeSignal. His primary focus is making CodeSignal’s IDE great—which includes not only the editor but also components like our terminal and filesystem.