In this episode we revisited the same file system task that we saw last time, but with a focus on how the tests work. The main distinguishing feature of these tasks is that the tests are stored within the project’s file tree, similar to how a software engineer would write tests. Furthermore, the tests can be written to have any kind of interaction with the app, rather than just providing a set of inputs and expected outputs for a single function.
On a technical level, the tests are executed by zipping the whole file system, and sending it to a coderunner server which then runs the tests and returns a report on the results. The candidate is scored on how many of the tests they passed, similar to the way most other types of tasks are scored.
From the candidate’s perspective, the feedback is presented in the same format as a standard algorithmic task, with familiar error reporting in the console. It’s also possible for some of the tests to be hidden, as they can be stored in different files with different user permissions. This allows the tests to contain edge cases that the user would need to consider in order to pass all the hidden tests.