When creating any kind of standardized test, one of the main goals is to avoid any potential bias in the results; the questions should be consistent and fair, with a concentrated focus on the subject matter. To help achieve this goal, the CodeSignal team has conducted extensive research on test design, and developed a framework for creating new tasks.
As a result of this research, the team can make confident assertions regarding the length and content of a technical assessment. For example, it’s been shown that the ideal test length is around 60 minutes (if the test is shorter, it can limit the quality and accuracy of the signal it’s providing, while a longer test can lead to candidate dropoff).
Furthermore, questions can be categorized according to meaningful distinctions in their complexity; for example, there should be tasks that simply assess language familiarity, without asking the candidate to implement any complicated algorithms or data structures. There’s also a distinction between implementation tasks (which describe the exact instructions to follow) vs problem solving tasks, which describe the given scenario but leave it up to the candidate how to solve the problem.