Tips, Tricks, Hacks, and Magic: How to Effortlessly Optimize Your Jupyter Notebook

That’s incredibly easy and you can switch it up any time you want to.

First, go to your terminal and install Jupyterthemes withpip install jupyterthemesNow you can install the super popular dark theme withjt -t chesterishRestore the main theme any time withjt -rClick here to find the Jupyterthemes GitHub repo.

Basic commandsYou can quickly access keyboard shortcuts with the command palette.

just type Ctrl + Shift + P or Cmd + Shift + P to access a dialog box that’s a lot like Spotlight Search on a Mac.

It can help you run any command by name, which is great if you don’t know the keyboard shortcut.

Shift + Enter lets you run the current cellEsc takes you into command mode.

Now you can navigate around your notebook with your arrow keys!In command mode, useA to insert a new cell above your current cellB to insert a new cell below your current cellM to change the current cell to MarkdownY to change back to a code cellD + D to delete the current cell (press the key twice)Enter takes you from command mode back into edit modeAlsoShift + Tab will show you the documentation for the object you just typed into a code cell.

(You can keep pressing this to cycle through a few modes.

)Esc + F helps you find and replace info in your code (not in the outputs)Esc + 0 Toggles cell outputShift + J or Shift + Down selects the next cell in a downwards direction.

Shift + K or Shift + Up selects cells in an upwards direction.

Once your cells are selected, you can delete/ copy/cut/paste/run them as a batch.

That’s awesome when you need to move parts of a notebook!Shift + M lets you merge multiple cells.

(If you just try to click on the cells you want to work you’ll have trouble.

Hold down the shift key and click the cells you want to merge.

Then, while you’re still holding down the shift key, press M.

)Also, you can run bash commands in a notebook if you put an exclamation point at the beginning.

For example: !pip install numpyYou can suppress the output of the function on a final line of code any time by adding a semicolon at the end.

Commenting and uncommenting a block of codeYou might want to add new lines of code and comment out the old lines while you’re working.

This is great if you’re improving the performance of your code or trying to debug it.

First, select all the lines you want to comment out.

Next hit cmd + / to comment out the highlighted code!LaTexYou can write LaTex in a Markdown cell any time and it will be rendered as a formula.

That changes this$P(A mid B) = frac{P(B mid A)P(A)}{P(B)}$into thisPretty Print all cell outputsNormally only the last output in the cell will be printed.

For everything else, you have to manually add print(), which is fine but not super convenient.

You can change that by adding this at the top of the notebook:from IPython.

core.

interactiveshell import InteractiveShellInteractiveShell.

ast_node_interactivity = "all"This means that, while normally you’d only get one output printedNow you’ll see both outputs!Any time you want to go back to the original setting, just runfrom IPython.

core.

interactiveshell import InteractiveShellInteractiveShell.

ast_node_interactivity = "last_expr"Just be aware that you have to run the setting change in a separate cell for it to take effect for the next cell run.

ExtensionsBecause it’s an open source web app, a ton of extensions have been developed for Jupyter Notebooks.

You can find the official iPython extension list here.

This is another popular bundle of extensions.

You can install Nbextensions any time from your command line like thiswith pippip install jupyter_contrib_nbextensionsjupyter contrib nbextension install –useror with Anacondaconda install -c conda-forge jupyter_contrib_nbextensionsconda install -c conda-forge jupyter_nbextensions_configuratorjupyter contrib nbextension install –userOnce they’re installed, you’ll see an Nbextensions tab.

Explore away!Head on over here to read more about the extensions and how to enable them, disable them, and more.

I won’t go into too much detail about adding and enabling extensions and how to use them because it’s so incredibly well explained right in your Jupyter Notebook!.Just click on “Nbextensions” at the top of the screen, click on the extension you’re interested in, and then scroll down for the information you need and a GIF of the extension in action!Popular extensionsScratchpad — This is awesome.

It allows you to create a temporary cell to do quick calculations without creating a new cell in your workbook.

This is a huge time saver!Hinterland — This enables a code autocompletion menu for every keypress in a code cell instead of just with tabSnippets — Adds a drop-down menu to insert snippet cells into the current notebook.

Autopep8 — This is a tool that automatically formats Python code to conform to the PEP 8 style guide.

It’s so handy!.Make sure you have run pip install autopep8 –user on your local machine.

This will make sure you’re following the correct python coding conventions.

Split Cells Notebook — Enables split cells in Jupyter notebooks.

Enter command mode and use Shift + S to toggle the current cell to either a split cell or full width.

Table of Contents — This extension enables you to collect all running headers and display them in a floating window, as a sidebar, or with a navigation menu.

A Code Prettifier — Cleans up, formats, and indents your code, so you don’t have to.

Notify — This displays a desktop notification when your kernel becomes idle.

This is awesome when you’re running code that takes more than a couple of seconds to complete.

Code Folding — While in edit mode, a triangle appears in the gutter to fold your code.

Good when you have large functions you want to hide for readability.

Zen mode — Makes things a bit less cluttered.

Make sure to turn off the backgrounds in the settings.

MagicMagics are handy commands that make life easier when you want to perform particular tasks.

They often look like unix commands, but they’re all implemented in Python.

There are a ton of magics out there!There are two kinds of magic: line magic (use this on one line) and cell magic (this applies to the whole cell).

Line magics start with a percent character %, and cell magics start with two, %%.

To see the available magics, run:%lsmagicSet EnvironmentYou can easily manage the environment variables of your notebook without restarting anything with %env.

If you run it without any variables, it will list all of your environment variables.

Insert codeYou can insert code from an external script with %load.

(More on this below, but it’s awesome, so I’m adding it up here) For example:%load basic_imports.

pywill grab the basic_imports.

py file and load it in your notebook!Export the contents of a cellThis is unbelievably helpful.

With almost no effort, you can export the contents of a cell any time with %%writefile.

For example%%writefile thiscode.

pyyou'd write some cool code or function in here that you'd want to export and maybe even use later!Do you find yourself running the same imports in every notebook or adding the same function all the time?.Now you can write it once and use it everywhere!You can write a file basic_imports.

py containing the following code:%writefile basic_imports.

pyimport pandas as pdimport numpy as npimport matplotlib.

pyplot as pltThat will create a .

py file that contains your basic imports.

You can load this any time by writing:%load basic_imports.

pyExecuting this replaces the cell contents with the loaded file.

# %load imports.

pyimport pandas as pdimport numpy as npimport matplotlib.

pyplot as pltNow we can run the cell again to import all our modules and we’re ready to go.

Store and reuse code: %macro MagicLike most people, you probably find yourself writing the same few tasks over and over again.

Maybe there are a few equations that you find yourself computing repeatedly or some lines of code that you’ve produced countless times.

Jupyter lets you save code snippets as executable macros!.Macros are just code, so they can contain variables that will have to be defined before execution.

Let’s define one!Let’s sayname = 'Kitten'Now, to define a macro we need some code to use.

We can save pretty much anything, from a string to a function, or whatever you need.

print('Hello, %s!' % name)Hello, Kitten!We use the %macro and %load magics to set up a reusable macro.

It’s pretty common to begin macro names with a double underscore to distinguish them from other variables.

%macro -q __hello_you 32The %macro magic takes a name and a cell number (or numbers), and we also passed -q to make it less verbose.

%store allows us to save any variable for use in other sessions.

Here we passed the name of the macro we created so we can use it again after the kernel is shut down or in other notebooks.

To load the macro, we just run:%load __hello_youAnd to execute it, we can just run a cell that contains the macro name.

__hello_youHello, Kitten!Image by Ilona Ilyés from PixabaySuccess!Let’s modify the variable we used in the macro.

name = 'Muffins'When we run the macro now, our modified value is picked up.

__hello_youHello, Muffins!This works because macros execute the saved code in the scope of the cell.

If name was undefined we’d get an error.

Image by Helga Kattinger from PixabayWant to use that same macro across all of your notebooks?Store Magic%store lets you store your macro and use it across all of your Jupyter Notebooks.

Now you can open a new notebook and try it out with %store -r __hello_you.

Load that baby up and you’re ready to go!%store -r __hello_youname = 'Rambo'%load __hello_youHello, Rambo!Image by Winsker from PixabayRun magic%run magic will execute your code and display any output, including Matplotlib plots.

You cou;d even execute entire notebooks this way.

%run can execute python code from .

py files.

It can also execute other Jupyter notebooks.

Pycat magicYou can use %pycat any time to show the contents of a script if you aren’t sure what’s in there.

%pycat basic_imports.

pyAutosaveThe %autosave magic lets you change how often your notebook will auto save to its checkpoint file.

%autosave 60That will set you to autosave every 60 seconds.

Display plot images%matplotlib inlineYou probably already know this one, but %matplotlib inline will display your Matplotlib plot images right in your cell outputs.

That means that you can include Matplotlib charts and graphs right in your notebooks.

It makes sense to run this at the beginning of your notebook, right in the first cell.

TimingThere are two IPython Magic commands that are useful for timing — %%time and %timeit.

These are seriously useful when you have some slow code and you’re trying to identify where the issue is.

They both have line and cell modes.

The main difference between %timeit and %time is that %timeit runs the specified code many times and computes an average.

%%time will give you information about a single run of the code in your cell.

%%timeit uses the Python timeit module which runs a statement a ton of times and then provides the mean of the results.

You can specify the number of runs with the -n option, -r to specify the number of repeats, and more.

Run code from a different kernelYou can also execute a cell using the specified language.

There are extensions available for several languages.

You have options like%%bash%%HTML%%python%%python2%%python3%%ruby%%perl%%capture%%javascript%%js%%latex%%markdown%%pypyTo render HTML in your notebook, for example, you’d run:%%HTMLThis is <em>really</em> neat!You can also use LaTeX directly any time you want to with%%latexThis is an equation: $E = mc^2$Who magicThe %who command without any arguments will list all variables that existing in the global scope.

Passing a parameter like str will list only variables of that type.

So if you type something like%who strin our notebook, you’d seePrun Magic%prun shows how much time your program spent in each function.

Using %prun statement_name gives you an ordered table showing the number of times each internal function was called within the statement, the time each call took, and the cumulative time of all runs of the function.

Python Debugger MagicJupyter has own interface for The Python Debugger.

This makes it possible to go inside the function and take a look at what happens there.

You can turn that on by running %pdb at the top of the cell.

High-Resolution PlotsOne simple line of IPython magic will give you double resolution plot output for Retina screens.

Just be aware that this won’t render on non-retina screens.

%config InlineBackend.

figure_format ='retina'To Skip A Cell From RunningJust add %%script false at the top of the cell%%script falseyou'd put some long code here that you don't want to run right nowAlert meThis is actually a Python trick, but you might want it for when you’re running code that’s taking forever.

If you don’t want to be staring at your code all day, but you need to know when it’s done, you can make your code sound an “alarm” when it’s finished!On Linux (and Mac)import osduration = 1 # secondfreq = 440 # Hzos.

system('play –no-show-progress –null –channels 1 synth %s sine %f' % (duration, freq))On Windowsimport winsoundduration = 1000 # millisecondfreq = 440 # Hzwinsound.

Beep(freq, duration)In order to use this, you need to install sox, which you should be able to do withbrew install sox…assuming you have homebrew installed.

If you haven’t taken any time to customize and improve your terminal, you might want to check out this article!Trick Out Your Terminal in 10 Minutes or LessHow to make a better, faster, stronger, and sexier terminal in mere minutestowardsdatascience.

comNow have fun!That should be enough to get you started!.If you know any tips and tricks that you think could help other beginners, let everyone know about them in the comments below.

Your options here are endless!If you really want to take things to the next level, you might want to check out this post:Optimizing Jupyter Notebooks — A Comprehensive GuideFinding bottlenecks and increasing your speed performance by magnitudes with some tips I came along over the past year.

towardsdatascience.

comAnd if you’re interested in building interactive dashboards, check out this one:Building Interactive Dashboards with JupyterWelcome to Part II of "Advanced Jupyter Notebook Tricks.

" In Part I, I described magics, and how to calculate notebooks…blog.

dominodatalab.

comAnd as always…Thanks for reading!.. More details

Leave a Reply