How to use Docker to Deploy Jupyter with Nginx

Become a Datascience Pro – Host your own Jupyter Notebooks

β€œTo say goodbye is to die a little.” ― Raymond Chandler, The Long Goodbye

1. Introduction

Docker is pretty cool when it comes to deployment of software. It isolates the deployed software from the host system. It does this by running a lighter version of an OS inside a container and deploying the software on top of this OS.

In this article, we show you how to build your own Jupyter deployment system inside docker with Nginx managing the front end. Jupyter is a very useful python runtime system which can be used from the browser to run python code on the server. It is especially useful in datascience projects because you can use render beautiful graphics in the browser using packages like matplotlib and seaborn. You can also use it to run data analysis using pandas and numpy packages.

Before continuing, you may want to review this article which discusses setting up Nginx inside a Docker container with SSL-enabled.

2. Pre-Requisites

We assume that you have a server where you want to deploy Nginx with a Jupyter notebook capability. You need to install docker on the server. This is the only dependency needed on the server.

apt-get install docker.io

You may also need to grant appropriate permissions to the user using docker. Assuming this user is named aurora, run these commands to grant the permissions.

usermod -aG docker aurora

3. The Dockerfile

Let us begin by describing the Dockerfile. We use the stock Ubuntu Server 16.04 distribution as the container OS. On booting the OS, we update the server and install the needed software: net-tools for netstat, nginx and python-pip.

FROM ubuntu:16.04
MAINTAINER Jay Sridhar "jay.sridhar@gmail.com"
RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
    DEBIAN_FRONTEND=noninteractive apt-get -y dist-upgrade && \
    DEBIAN_FRONTEND=noninteractive apt-get -yq install net-tools nginx python-pip

Let us now install the jupyter software package.

RUN pip install --upgrade pip && \
    pip install jupyter

Create a user named aurora in the container OS as a non-privileged user. We also cleanup some unneeded files from the Nginx distribution.

RUN useradd -ms /bin/bash aurora && \
    rm -f /etc/nginx/fastcgi.conf /etc/nginx/fastcgi_params && \
    rm -f /etc/nginx/snippets/fastcgi-php.conf /etc/nginx/snippets/snakeoil.conf

We need ports 80 and 443 open, so expose them.

EXPOSE 80
EXPOSE 443

Adjust Nginx configuration for SSL. We also make some changes to Nginx configuration for handling Jupyter which we discuss below.

COPY nginx/ssl /etc/nginx/ssl
COPY nginx/snippets /etc/nginx/snippets
COPY nginx/sites-available /etc/nginx/sites-available

Next we copy configuration for Jupyter and any notebooks we want in the installation. The notebooks directory is mounted as a read-write volume into the container, so changes made to notebooks from the browser persist even after the container is removed.

COPY .jupyter/jupyter_notebook_config.py /home/aurora/.jupyter/jupyter_notebook_config.py

We also have a startup script which starts the jupyter server and Nginx. This startup script is the ENTRYPOINT for docker.

COPY etc/startup.sh /etc/startup.sh
RUN chown -R aurora:aurora /home/aurora/.jupyter
ENTRYPOINT ["/bin/bash", "-c", "/etc/startup.sh"]

4. Nginx Configuration Changes

In addition to the configuration changes needed for SSL (discussed here), we need some more changes to setup the jupyter server as a reverse proxy for Nginx. This means URLs starting with the prefix /jupyter/ will be redirected to Jupyter and the response routed back to the user’s browser.

Here are the relevant changes. It tells Nginx to route requests starting with the prefix /jupyter/ to the server listening on port 8888 on the same docker container. The other settings are needed for Jupyter to work properly.

This is the setting you need to change if you plan to host the Jupyter notebooks on another server.

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    ...

    location /jupyter {
        proxy_pass http://localhost:8888;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header Host $http_host;
        proxy_http_version 1.1;
        proxy_redirect off;
        proxy_buffering off;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_read_timeout 86400;
    }

    ...
}

5. Jupyter Configuration

Jupyter looks for its configuration at startup in the location: $HOME/.jupyter/jupyter_notebook_config.py, where $HOME is the home directory of the user starting the server. It contains bunch of settings of which we only adjust the following:

c.NotebookApp.base_url = '/jupyter'
c.NotebookApp.notebook_dir = u'notebooks'
c.NotebookApp.open_browser = False
c.NotebookApp.port = 8888

6. Startup Script

A startup script is needed since we are starting two services inside the docker container – a jupyter notebook server and Nginx.

The first command starts the jupyter server as the non-privileged user we added earlier. The second command (which does not exit) starts Nginx.

su -l -c 'jupyter notebook &' aurora
/usr/sbin/nginx -g 'daemon off;'

7. Launch the Bundle: Nginx and the Notebook Server

These are all the changes needed to start the server. Built the docker container using the Dockefile as follows:

docker build --rm=true --force-rm=true -t myjupyter:latest .

Once the build process completes, launch the server as follows. We mount the notebooks directory into the container to be able to access them across containers.

docker run --rm=true -p 80:80 -p 443:443 -v ~/server/docker/jupyter/src/notebooks:/home/aurora/notebooks myjupyter:latest

Once this command completes, you should have Nginx listening on port 80 and 443, with jupyter accessible at /jupyter/ on the server web site.

At this point, if you need to host a web site, you can mount the website root also. See this post for details.

Check out source code on github.

Review

We have presented the details of how to host your own Jupyter notebooks using Nginx as the web server, and running the whole show inside a docker container. Running it inside docker allows you a lot of flexibility. And having Jupyter at your fingertips makes for a very effective learning and development environment.

Leave a Reply

Your email address will not be published. Required fields are marked *