This post will give a high level overview of a few core features and the tech that was used to build it.
As with any MVP, the goal was to always fail fast and avoid any pre-optimisations.
Features that enable collaborative programmingCodePusher currently ships with five features that are required to create a collaborative coding environment:Collaborative text editingCode executionDirect messagingConference callsSearchCollaborative text editingEnabling collaborative code editing within a session was the first technical challenge.
There needed to be a feature that allowed all users in the same session to make changes to the same document.
To achieve this, CodePusher leverages a technique called operational transformation (OT).
This is a similar approach used to implement collaborative features in applications like Google docs.
The theory behind OT can be quite challenging to understand and certainly non-trivial to implement from scratch.
For anyone interested in computer science and it’s applications I would definitely recommend doing a deeper dive into this area if possible.
However, in our case, it was clear that there was a lack of time and resources to develop a completely robust OT system that could rival Google docs.
Fortunately, and thanks to the open source community, there was no need to reinvent the wheel.
ShareDB was the perfect solution that provided an abstracted interface for getting quickly setup with collaborative text editing without having to worry too much about the low level details of OT.
Demo of CodePusher’s collaborative code editor powered by ShareDB.
Code executionHowever, collaborative code editing on it’s own isn’t enough.
As users work through a programming challenge on CodePusher there is also a need to run the code against a set of test cases to see whether it passed or failed.
In order to achieve this the frontend uses the terminal-in-react library to simulate a terminal with the built-in command push.
During a session, when a code is submitted it gets sent to a backend where it is executed and asserted against a set of test cases.
At runtime, all logs from the code will also be pushed back to each user’s terminal.
When execution finishes, the session is then notified whether or not their code passed or failed.
The challenging part about this feature was creating a system for running untrusted and user submitted code.
The aim here is to isolate it as much as possible from our core backend processes.
This is necessary to protect ourselves from code that can be intentionally, or unintentionally, malicious.
A simple example is submitted code that ends up running in an infinite loop.
But we also want to protect ourselves from more serious security flaws such as code being able to access private data from environment variables or scripts that can crash the entire server at will.
To solve this problem we can split our code execution system into two components which we can call a runner and a launcher.
The runner is responsible for executing code in an isolated sandboxed container that is on a seperate process to our core backend.
The launcher is then used to setup and manage the lifecycle of a runner.
In practice this can mean setting it up with the correct language, code, and tests along with relaying all logs back to the session during runtime.
Demo of CodePusher’s terminal and code execution.
Direct messagingImplementing direct messaging was relatively simple compared to the other collaboration features.
It relies mostly on firebase real-time database.
During a session, when a user sends a message it gets pushed into a session specific path on the database.
The message is then broadcasted to all clients within the session.
Easy!Demo of direct messaging in CodePusher.
Conference callsBuilding a conference call feature was pretty tricky.
There are a few different approaches you can take depending on use case, time, and resource constraints.
It took three iterations before landing on the most viable option for CodePusher’s MVP.
The first iteration was an attempt to build out the feature using WebRTC through the simple-peer library.
However this gets challenging when your use case requires connecting to more then one user.
The easiest solution to this is by creating a full mesh topology where each user is sending data to every other connected user.
Obviously it’s not a very scalable solution and would still take some time to get right when taking into account edge cases like dropped connections.
The next attempt was to try and use twillio.
The service has a tonne of great primitives that can be used to create all sorts of communication features, with conference calls being just one of them.
However, there was still a time restriction that had to be considered.
Since CodePusher is in a pre-launch stage, I couldn’t justify the time required to learn an entire platform and integrate it into the app just for a single feature.
Again, this would also take into account building a supporting UI and handling all edge cases.
This lead to attempt number three which is tokbox.
The key feature here was the ability to directly embed a conference call widget into any web app via an iframe element.
The value added along with the time saved made this the best solution for our current case.
Demo of a conference call in CodePusher powered by Tokbox.
SearchThe last feature we needed was the ability for users to search through repositories of different programming challenges that are currently available.
To achieve this CodePusher leverages the full-text search capabilities provided by algolia.
Overall algolia was very easy to integrate with which was also a huge bonus.
Each time a repository of challenges is created or updated in the database we also update the index on algolia in such a way that any user can simply search for a challenge by name, repository, or author.
Demo of CodePusher’s search function powered by algolia.
SummaryIn the end, using a combination of very interesting technologies, I’ve managed to put together an MVP for a code collaboration software in a time frame of two months.
To recap, this is a list of technologies used to build the core features for CodePusher:Collaborative text editing: ShareDB and Operational TransformationSandboxed code execution: Docker containersDirect messaging: Firebase realtime databaseConference calls: Tokbox video chat embedsSearching through repositories and challenges: Algolia full-text searchThe key takeaway for me throughout the building process was really driving home the point that technology, no matter what it is, is just a tool for solving a problem.
As software engineers it can be really easy for us to want to apply the latest hype to our products (queue the blockchain and A.
I buzzwords).
So much so that we end up biasing a solution to fit a certain tool, or spend an unjustified amount of time pre-optimising the smallest details and losing the big picture view.
For an early stage product, I think the best thing we can do is fail fast and keep iterating until we build something that users will actually want.
This was the final part to the series where I document what it takes to build an MVP for a real-time code collaboration and learning platform.
If you’re interested in trying out CodePusher, drop your email on the landing page to be notified on launch!.