Contributing to BinderHub#

Welcome! As a Jupyter project, we follow the Jupyter contributor guide.

Depending on what you want to develop, you can setup BinderHub in different ways.

This document also contains information on how to run tests and common maintainer tasks.

Develop documentation#

You are assumed to have a modern version of Python.

  1. Clone the BinderHub repository to your local computer and cd into it.

    git clone https://github.com/jupyterhub/binderhub
cd binderhub

  2. Install BinderHub and the documentation tools:

    python3 -m pip install -r docs/requirements.txt
python3 -m pip install sphinx-autobuild

  3. To build the documentation run:

    sphinx-autobuild docs/source docs/_build/html

    Changes to the source code will automatically trigger a re-build of the documentation.

  4. Open http://127.0.0.1:8000 in your browser.

Develop user interface#

Developing BinderHub’s user interface can be done both without Kubernetes and JupyterHub by mocking that interaction. You are assumed to have a modern version of Python and node / npm installed.

  1. Clone the BinderHub git repository to your local computer and cd into it.

    git clone https://github.com/jupyterhub/binderhub
cd binderhub

  2. Install BinderHub, the Python package.

    python3 -m pip install -e .

  3. Install the NodeJS dependencies from package.json.

    npm install

  4. Create the JS and CSS bundles BinderHub serves as a webserver to visitors.

    npm run webpack:watch

  5. Start the BinderHub webserver locally.

    python3 -m binderhub -f testing/local-binder-mocked-hub/binderhub_config.py

  6. Visit the BinderHub webserver at http://localhost:8585.

Building and launching repositories will not work. You can still work on the user interface of those parts as BinderHub is configured to fake those actions. You can tell you are using the fake builder and launcher from the fact that the build will never complete.

We use eslint to catch errors in our JS, and you can run it locally with npm run lint.

To learn how to set yourself with a BinderHub development environment that lets you modify the builder and launcher refer to Develop Kubernetes integration or Develop Helm chart.

Develop Kubernetes integration#

Important

This requires you create the JS and CSS bundles BinderHub serves to visitors.

  1. Install the NodeJS dependencies from package.json.

    npm install

  2. Create the JS and CSS bundles.

    npm run webpack

Important

This requires helm and a functional Kubernetes cluster. Please do preliminary Kubernetes setup if you haven’t already before continuing here.

  1. Verify that you have access to a Kubernetes cluster running.

    kubectl version

  2. Locally install BinderHub as a Python package and its development requirements locally.

    python3 -m pip install -e ".[pycurl]" -r dev-requirements.txt

  3. Install the JupyterHub Helm chart by itself into your Kubernetes cluster current namespace.

    # Append --auth here if you want to develop against a non-public BinderHub
# that relies on JupyterHub's configured Authenticator to decide if the users
# are allowed access or not.
./testing/local-binder-k8s-hub/install-jupyterhub-chart

  4. Export the location of your JupyterHub to the environment variable LOCAL_BINDER_JUPYTERHUB_IP.

    If using minikube,

    export LOCAL_BINDER_JUPYTERHUB_IP=$(minikube ip)

    If using Docker Desktop,

    export LOCAL_BINDER_JUPYTERHUB_IP='kubernetes.docker.internal'

  5. Configure docker using environment variables to use the same Docker daemon as your local Kubernetes cluster. This means images you build are directly available to the cluster.

    If using minikube,

    eval $(minikube docker-env)

    If using Docker Desktop,

    docker context use desktop-linux

  6. Start BinderHub with the testing config file.

    python3 -m binderhub -f testing/local-binder-k8s-hub/binderhub_config.py

  7. Visit http://localhost:8585

Congratulations, you can now make changes and see how they influence the deployment. You may be required to restart the BinderHub depending on what you change. You can also start running pytest tests to verify the Deployment functions as it should.

Cleanup resources#

  1. Cleanup the JupyterHub Helm chart you have installed in Kubernetes.

    helm delete binderhub-test

  2. Restore the context used by docker.

    docker context use default

  3. Stop running the Kubernetes cluster.

    If using minikube,

    minikube stop

Develop Helm chart#

This requires helm and a functional Kubernetes cluster. Please do preliminary Kubernetes setup if you haven’t already before continuing here.

With a Kubernetes cluster running, as you verify with kubectl version, you can continue.

  1. Install development requirements, including pytest and chartpress.

    python3 -m pip install -r dev-requirements.txt

  2. Configure docker using environment variables to use the same Docker daemon as your local Kubernetes cluster. This means images you build are directly available to the cluster. If using minikube,

    eval $(minikube docker-env)

    If using Docker Desktop,

    docker context use desktop-linux

  3. Build the docker images referenced by the Helm chart and update its default values to reference these images.

    (python3 -m build . && cd helm-chart && chartpress)

  4. Get the chart dependencies (for example JupyterHub)

    cd helm-chart/binderhub
helm dependency update

  5. Validate, and then install the Helm chart defined in helm-chart/binderhub.

    This validation step is not making any modification to your Kubernetes cluster, but will use it to validate if the Helm chart’s rendered resources are valid Kubernetes resources according to the Kubernetes cluster.

    helm template --validate binderhub-test helm-chart/binderhub \
   --values testing/k8s-binder-k8s-hub/binderhub-chart-config.yaml \
   --set config.BinderHub.hub_url=http://$(minikube ip):30902 \
   --set config.GitHubRepoProvider.access_token=$GITHUB_ACCESS_TOKEN

    If the validation succeeds, go ahead and upgrade/install the Helm chart.

    Note that this will do the installation in the current namespace.

    helm upgrade --install binderhub-test helm-chart/binderhub \
   --values testing/k8s-binder-k8s-hub/binderhub-chart-config.yaml \
   --set config.BinderHub.hub_url=http://$(minikube ip):30902 \
   --set config.GitHubRepoProvider.access_token=$GITHUB_ACCESS_TOKEN

echo "BinderHub inside the Minikube based Kubernetes cluster is starting up at http://$(minikube ip):30901"

  6. Congratulations, you can now make changes and run the step above again to see how they influence the deployment. You can also start running pytest tests to verify the Deployment functions as it should.

Cleanup resources#

  1. To cleanup resources you have installed and start fresh…

    helm delete binderhub-test

  2. To stop running the Kubernetes cluster…

    minikube stop

Kubernetes setup#

A fully functional BinderHub needs to have access to a Kubernetes cluster with a JupyterHub installed on it. You can either run BinderHub as a Python webserver locally and install JupyterHub on its own in the Kubernetes cluster, or install the entire BinderHub Helm chart which installs the JupyterHub Helm chart as a dependency.

All the steps are given as command-line commands for Ubuntu systems. They are used as a common denominator that can be translated into the correct commands on your local system.

You are assumed to have a modern version of Python, node / npm installed, and the command line tool curl.

  1. Install Minikube.

  2. Start a minikube Kubernetes cluster inside a virtual machine (virtualbox, xhyve, or KVM2).

    minikube start

  3. Install kubectl - the CLI to interact with a Kubernetes cluster.

    curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/bin/

    Here are the official installation instructions.

  4. Install helm - the Kubernetes package manager.

    curl -sf https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3

    Here are the official installation instructions.

  5. Let helm know about the official JupyterHub Helm chart repository.

    helm repo add --force-update jupyterhub https://jupyterhub.github.io/helm-chart/
helm repo update

  6. Clone the binderhub git repository to your local computer and cd into it.

    git clone https://github.com/jupyterhub/binderhub
cd binderhub

Tip: Use local repo2docker version#

BinderHub runs repo2docker in a container. For testing the combination of an unreleased repo2docker feature with BinderHub, you can use a locally build repo2docker image. You can configure the image in the file testing/local-binder-k8s-hub/binderhub_config.py with the following line:

c.BinderHub.build_image = 'jupyter-repo2docker:my_image_tag'

Important: the image must be build using the same Docker daemon as the minikube cluster, otherwise you get an error “Failed to pull image […] repository does not exist or may require ‘docker login’”.

Tip: Increase your GitHub API limit#

By default, GitHub has a limit of 60 API requests per hour. We recommend using a GitHub API token before running tests in order to avoid hitting your API limit. Steps to do so are outlined in the BinderHub documentation.

Tip: Start Minikube with more memory#

By default, minikube start allocates 2GiB of main memory to the underlying VM, which might be too low to run the builder successfully.

You may run minikube start --memory 8192 to start Minikube with a 8GiB VM underneath.

Running tests#

This git repository contains pytest based tests that you can run locally. Depending on your development setup, you may want to run different kind of tests. You can get some hints on what tests to run and how by inspecting .github.

Environment variables influencing tests#

  • BINDER_URL: An address of an already running BinderHub as reachable from the tests. If this is set, the test suite will not start temporary local BinderHub servers but instead interact with the remote BinderHub.

  • GITHUB_ACCESS_TOKEN: A GitHub access token to help avoid quota limitations for anonymous users. It is used to enable certain tests making many calls to GitHub API.

Pytest marks labelling tests#

  • remote: Tests for when BinderHub is already running somewhere.

  • github_api: Tests that communicate with the GitHub API a lot.

  • auth: Tests related to BinderHub’s usage of JupyterHub as an OAuth2 Identity Provider (IdP) for non public access.

Common maintainer tasks#

These are tasks that BinderHub maintainers perform.

Bumping the JupyterHub Helm chart version#

The BinderHub Helm chart depends on the JupyterHub Helm chart, and its version is pinned within helm-chart/binderhub/Chart.yaml. It is straightforward to update it with another version from the JupyterHub Helm chart repository.

Use the JupyterHub Helm chart’s changelog to prepare for breaking changes associated with the version bump.

Releasing#

BinderHub Python package release checklist#

  • update/close the CHANGES.md for this release (see below)

  • create a git tag for the release

  • pip install build twine

  • python -mbuild .

  • twine check dist/* to check the README parses on PyPI

  • edit $HOME/.pypirc to use the binder team account

  • twine upload dist/*

  • create a new release on https://github.com/jupyterhub/binderhub/releases

  • add a new section at the top of the change log for future releases

For more details, see this guide on uploading package to PyPI.

BinderHub NPM package release checklist#

  • update the version number in js/packages/binderhub-client/package.json and js/packages/binderhub-react-components/package.json

  • update/close the CHANGES.md for this release (see below)

  • create a git tag for the release

  • login into NPM

  • run

    npm publish -w js/packages/binderhub-client/ --access public

    and follow the instructions for authentication.

  • run

    npm publish -w js/packages/binderhub-react-components/ --access public

    and follow the instructions for authentication.

  • create a new release on https://github.com/jupyterhub/binderhub/releases

  • add a new section at the top of the change log for future releases

Updating the changelog#

As BinderHub does not have a typical semver release schedule, we try to update the changelog in CHANGES.md every three months. A useful tool for this can be found here. If you choose to use this tool, the command that generated current sections in the changelog is below:

github-activity jupyterhub/binderhub -s <START-DATE> -u <END-DATE> --tags enhancement,bug --strip-brackets

Copy and paste the output of this command into a new section in CHANGES.md.