A new Python kernel for JupyterMartin RenouBlockedUnblockFollowFollowingJan 9Project Jupyter aims at providing a consistent set of tools for interactive computing workflows across multiple programming languages.
Jupyter projects are popular at all stages of a research project from the exploration phase to the communication of results and teaching.
The flagship project of Jupyter, the Notebook, and its modernized version, JupyterLab are web applications allowing the creation of documents including prose, executable code, and interactive visualizations.
The kernel is the part of the backend responsible for executing code written by the user in the web application.
For example, in the case of a Python notebook, execution of the code is typically handled by ipykernel, the reference implementation.
HistoryInitially, the IPython project included everything from the notebook web application to the kernel and server implementation.
Later on, the language-agnostic parts (the notebook format, messaging protocol, qtconsole, notebook web application) were split into separate projects with a clearer scope, together forming Project Jupyter.
The Big Split(TM) of IPythonSplitting IPython into multiple packages was a good decision in that it brought a clear separation of concerns between the projects.
The ipykernel project was elected as the reference implementation of the Jupyter kernel protocol.
This came at a cost since ipykernel brought all the history and technical debt of IPython with it.
Besides, Python is not a natural language for a base implementation.
As a result, R and Julia kernels don’t use the reference implementation but use their own implementation of the protocol.
Furthermore, the standard implementation of the Jupyter Interactive Widgets lies in the Python package ipywidgets.
That means that once you, as a kernel developer, implemented the Jupyter kernel protocol, you would still need to make your own backend for the interactive widgets if you want widgets support (and you should want it, honestly…).
To prevent this useless duplication of effort, which harms sustainability, we set ourselves to implement a solid reusable implementation of the Jupyter kernel protocol, xeus.
C++ implementation of the kernel protocolXeus is a C++ implementation of the Jupyter kernel protocol.
It is not a kernel by itself but a library that helps kernel authoring.
Xeus is very helpful when developing a kernel for a language that has a C or a C++API (Python, Lua, SQL, etc.
).
It takes the cumbersome task of implementing the Jupyter messaging protocol for you so you just can focus on the core interpreter tasks: executing code, inspecting, completing, etc.
C++ is a good choice for a standard implementation of the protocol, it is a common denominator of most of the languages out there, it has a massive developer community and is widely adopted in the industry for performance middleware applications.
With xeus, it is also very easy to reimplement core logic (server, kernel, history management etc.
), if need be, by simply inheriting from xeus library classes.
Xeus is already known for being used as a base for the C++ kernel xeus-cling.
A xeus-based C++ KernelXeus-cling is a xeus-based C++ kernel for the Jupyter Notebook.
It’s very useful for teaching or learning C++.
C++ has never been this interactive, thanks to the combined power of Jupyter, xeus, and cling.
And even more impressive than executing interpreted C++, it supports interactive widgets, thanks to the xwidgets project.
Just like xeus is a C++ implementation of the Jupyter kernel protocol, xwidgets is a C++ implementation of the Jupyter Interactive Widgets protocol, and it can be used as a base for other implementations.
You can try it right now with binder, without the need of installing anything on your computer, just by clicking on this link.
Interactive C++ in the Jupyter NotebookA xeus-based Python kernelToday, I am pleased to announce a new Python kernel based on xeus: xeus-python.
Xeus-python supports error formatting, rich display, interactive widgets, input requests, code completion, code inspection, etc.
Thanks to xeus and pybind11, it was very straightforward to have a first simple version of a Python kernel written in C++.
It took a couple of months to have the features listed above, with a relatively small amount of code (< 3000 lines of C++) and a good coverage of ipykernel’s features.
Simple code executionError formattingCode completionIpywidgets supportJust like with xeus-cling, you can try it online without the need of installing anything on your computer following this link.
Why another Python kernel?Obviously, xeus-python does not cover 100% of the features of ipykernel.
For examples, IPython magics are not supported yet by xeus-python.
However:xeus-python is a lot lighter than ipykernel and IPython combined, which makes it a lot easier to implement new features on top of it.
Our next goal is to augment the protocol to implement a Python debugger in JupyterLab.
xeus-based kernels are more versatile in that one can overload e.
g.
the concurrency model.
This is something that Kitware’s SlicerJupyter project takes advantage of to integrate with the Qt event loop of their Qt-based desktop application.
AcknowledgmentsThe development of xeus, xwidgets, xeus-python, and related packages are led by QuantStack.
This development is sponsored by Bloomberg.
About the AuthorMartin Renou is a Scientific Software Engineer at QuantStack.
Before joining QuantStack, he studied at the French Aerospace Engineering School SUPAERO.
He also worked at Logilab in Paris and Enthought in Cambridge.
As an open source developer at QuantStack, Martin worked on a variety of projects, from xsimd, xtensor, and xframe in C++ to ipyleaflet and ipywebrtc in Python and JavaScript.
.