From e2438127453d9ff58887e6c42c4f34c55ccadc83 Mon Sep 17 00:00:00 2001 From: Eric Dill Date: Wed, 12 Jun 2019 07:34:31 -0400 Subject: [PATCH 001/214] Some suggestions from reading through the docs Thanks to @willingc, @minrk and @manics for the review so far. This commit is the PR so far rebased onto master and squashed. --- README.md | 2 +- docs/source/contributing/docs.rst | 4 +++- docs/source/contributing/setup.rst | 9 +++++--- docs/source/contributing/tests.rst | 2 +- .../authenticators-users-basics.md | 3 +++ docs/source/getting-started/config-basics.md | 2 +- .../getting-started/networking-basics.md | 8 +++---- .../getting-started/security-basics.rst | 6 +++--- .../source/getting-started/services-basics.md | 2 +- docs/source/quickstart-docker.rst | 2 +- docs/source/reference/authenticators.md | 13 ++++++------ docs/source/reference/config-proxy.md | 10 ++++----- docs/source/reference/config-sudo.md | 11 ++++++---- docs/source/reference/config-user-env.md | 10 ++++++--- docs/source/reference/proxy.md | 6 +++--- docs/source/reference/separate-proxy.md | 4 ++-- docs/source/reference/services.md | 20 +++++++++++++----- docs/source/reference/spawners.md | 21 ++++++++++--------- docs/source/reference/urls.md | 9 ++------ 19 files changed, 83 insertions(+), 61 deletions(-) diff --git a/README.md b/README.md index 2fc37039..9e9c504f 100644 --- a/README.md +++ b/README.md @@ -188,7 +188,7 @@ this a good choice for **testing JupyterHub on your desktop or laptop**. If you want to run docker on a computer that has a public IP then you should (as in MUST) **secure it with ssl** by adding ssl options to your docker -configuration or by using a ssl enabled proxy. +configuration or by using an ssl enabled proxy. [Mounting volumes](https://docs.docker.com/engine/admin/volumes/volumes/) will allow you to **store data outside the docker image (host system) so it will be persistent**, even when you start diff --git a/docs/source/contributing/docs.rst b/docs/source/contributing/docs.rst index 6b6b647f..3e954224 100644 --- a/docs/source/contributing/docs.rst +++ b/docs/source/contributing/docs.rst @@ -27,7 +27,7 @@ change renders correctly, it is good practice to test it locally. python3 -m pip install -r docs/requirements.txt #. Build the html version of the docs. This is the most commonly used - output format, so verifying it renders as you should is usually good + output format, so verifying it renders correctly is usually good enough. .. code-block:: bash @@ -46,6 +46,8 @@ change renders correctly, it is good practice to test it locally. On macOS, you can open a file from the terminal with ``open ``. On Linux, you can do the same with ``xdg-open ``. + After opening index.html in your browser you can just refresh the page whenever + you rebuild the docs via ``make html`` .. _contributing/docs/conventions: diff --git a/docs/source/contributing/setup.rst b/docs/source/contributing/setup.rst index aa0f0595..4ef70b08 100644 --- a/docs/source/contributing/setup.rst +++ b/docs/source/contributing/setup.rst @@ -30,6 +30,9 @@ JupyterHub, is written in Javascript to run on `NodeJS recommend installing it in the ``miniconda`` environment you set up for Python. You can do so with ``conda install nodejs``. +Many in the Jupyter community use [``nvm``](https://github.com/nvm-sh/nvm) to +managing node dependencies. + Install git ----------- @@ -75,9 +78,9 @@ happen. npm install -g configurable-http-proxy - If you get an error that says ``Error: EACCES: permission denied``, - you might need to prefix the command with ``sudo``. If you do not - have access to sudo, you may instead run the following commands: + If you get an error that says ``Error: EACCES: permission denied``, you might need to prefix the command with ``sudo``. + ``sudo`` may be required to perform a system-wide install. + If you do not have access to sudo, you may instead run the following commands: .. code:: bash diff --git a/docs/source/contributing/tests.rst b/docs/source/contributing/tests.rst index a59487c0..27022156 100644 --- a/docs/source/contributing/tests.rst +++ b/docs/source/contributing/tests.rst @@ -4,7 +4,7 @@ Testing JupyterHub ================== -Unit test help validate that JupyterHub works the way we think it does, +Unit testing helps to validate that JupyterHub works the way we think it does, and continues to do so when changes occur. They also help communicate precisely what we expect our code to do. diff --git a/docs/source/getting-started/authenticators-users-basics.md b/docs/source/getting-started/authenticators-users-basics.md index b954f88d..f4b9dff4 100644 --- a/docs/source/getting-started/authenticators-users-basics.md +++ b/docs/source/getting-started/authenticators-users-basics.md @@ -101,6 +101,9 @@ popular services: - Okpy - OpenShift +NOTE: Open issue asking for more details on this generic implementation. +It's not clear if this is a different implementation or if the JupyterHub OAuth +_is_ the generic implementation. A generic implementation, which you can use for OAuth authentication with any provider, is also available. diff --git a/docs/source/getting-started/config-basics.md b/docs/source/getting-started/config-basics.md index 47850ac9..f4ae9e4d 100644 --- a/docs/source/getting-started/config-basics.md +++ b/docs/source/getting-started/config-basics.md @@ -93,7 +93,7 @@ use a custom proxy (e.g. Traefik), this also not needed. Connections to user servers go through the proxy, and *not* the hub itself. If the proxy stays running when the hub restarts (for -maintenance, re-configuration, etc.), then use connections are not +maintenance, re-configuration, etc.), then user connections are not interrupted. For simplicity, by default the hub starts the proxy automatically, so if the hub restarts, the proxy restarts, and user connections are interrupted. It is easy to run the proxy separately, diff --git a/docs/source/getting-started/networking-basics.md b/docs/source/getting-started/networking-basics.md index 86c80dcc..0c86e181 100644 --- a/docs/source/getting-started/networking-basics.md +++ b/docs/source/getting-started/networking-basics.md @@ -12,7 +12,7 @@ This section will help you with basic proxy and network configuration to: The Proxy's main IP address setting determines where JupyterHub is available to users. By default, JupyterHub is configured to be available on all network interfaces (`''`) on port 8000. *Note*: Use of `'*'` is discouraged for IP configuration; -instead, use of `'0.0.0.0'` is preferred. +instead, use of `'0.0.0.0'` is preferred. Changing the Proxy's main IP address and port can be done with the following JupyterHub **command line options**: @@ -41,9 +41,9 @@ port. ## Set the Proxy's REST API communication URL (optional) -By default, this REST API listens on port 8081 of `localhost` only. -The Hub service talks to the proxy via a REST API on a secondary port. The -API URL can be configured separately and override the default settings. +By default, the proxy's REST API listens on port 8081 of `localhost` only. +The Hub service talks to the proxy via a REST API on a secondary port. +The REST API URL (hostname and port) can be configured separately and override the default settings. ### Set api_url diff --git a/docs/source/getting-started/security-basics.rst b/docs/source/getting-started/security-basics.rst index 80996555..9223c362 100644 --- a/docs/source/getting-started/security-basics.rst +++ b/docs/source/getting-started/security-basics.rst @@ -76,9 +76,9 @@ In certain cases, for example if the hub is running behind a reverse proxy, and `SSL termination is being provided by NGINX `_, it is reasonable to run the hub without SSL. -To achieve this, simply omit the configuration settings -``c.JupyterHub.ssl_key`` and ``c.JupyterHub.ssl_cert`` -(setting them to ``None`` does not have the same effect, and is an error). +To achieve this, remove ``c.JupyterHub.ssl_key`` and ``c.JupyterHub.ssl_cert`` +from your configuration (setting them to ``None`` or an empty string does not +have the same effect, and will result in an error). .. _cookie-secret: diff --git a/docs/source/getting-started/services-basics.md b/docs/source/getting-started/services-basics.md index 9014fb9b..e6562520 100644 --- a/docs/source/getting-started/services-basics.md +++ b/docs/source/getting-started/services-basics.md @@ -67,7 +67,7 @@ c.JupyterHub.services = [ Upon restarting JupyterHub, you should see a message like below in the logs: -``` +```none Adding API token for ``` diff --git a/docs/source/quickstart-docker.rst b/docs/source/quickstart-docker.rst index d4153a14..bded4ac7 100644 --- a/docs/source/quickstart-docker.rst +++ b/docs/source/quickstart-docker.rst @@ -35,7 +35,7 @@ this a good choice for **testing JupyterHub on your desktop or laptop**. If you want to run docker on a computer that has a public IP then you should (as in MUST) **secure it with ssl** by adding ssl options to your docker -configuration or using a ssl enabled proxy. +configuration or using an ssl enabled proxy. `Mounting volumes `_ will allow you to store data outside the docker image (host system) so it will diff --git a/docs/source/reference/authenticators.md b/docs/source/reference/authenticators.md index 0775809f..c7016d0d 100644 --- a/docs/source/reference/authenticators.md +++ b/docs/source/reference/authenticators.md @@ -31,8 +31,7 @@ popular services: - Okpy - OpenShift -A generic implementation, which you can use for OAuth authentication -with any provider, is also available. +A [generic implementation](https://github.com/jupyterhub/oauthenticator/blob/master/oauthenticator/generic.py), which you can use for OAuth authentication with any provider, is also available. ## The Dummy Authenticator @@ -113,7 +112,9 @@ to uid to username), which is useful in case you use some external service that allows multiple usernames mapping to the same user (such as ActiveDirectory, yes, this really happens). When `pam_normalize_username` is on, usernames are *not* normalized to -lowercase. +lowercase. +NOTE: Earlier it says that usernames are normalized using PAM. +I guess that doesn't normalize them? #### Validate usernames @@ -169,7 +170,7 @@ setup( ``` If you have added this metadata to your package, -users can select your authenticator with the configuration: +admins can select your authenticator with the configuration: ```python c.JupyterHub.authenticator_class = 'myservice' @@ -259,8 +260,8 @@ class MyAuthenticator(Authenticator): ## pre_spawn_start and post_spawn_stop hooks Authenticators uses two hooks, [pre_spawn_start(user, spawner)][] and -[post_spawn_stop(user, spawner)][] to add pass additional state information -between the authenticator and a spawner. These hooks are typically used auth-related +[post_spawn_stop(user, spawner)][] to pass additional state information +between the authenticator and a spawner. These hooks are typically used for auth-related startup, i.e. opening a PAM session, and auth-related cleanup, i.e. closing a PAM session. diff --git a/docs/source/reference/config-proxy.md b/docs/source/reference/config-proxy.md index 564f2efc..26ec8546 100644 --- a/docs/source/reference/config-proxy.md +++ b/docs/source/reference/config-proxy.md @@ -207,8 +207,8 @@ httpd.conf amendments: ``` jupyterhub_config.py amendments: - ```bash - --The public facing URL of the whole JupyterHub application. - --This is the address on which the proxy will bind. Sets protocol, ip, base_url - c.JupyterHub.bind_url = 'http://127.0.0.1:8000/jhub/' - ``` +```python +# The public facing URL of the whole JupyterHub application. +# This is the address on which the proxy will bind. Sets protocol, ip, base_url +c.JupyterHub.bind_url = 'http://127.0.0.1:8000/jhub/' +``` diff --git a/docs/source/reference/config-sudo.md b/docs/source/reference/config-sudo.md index fc0d4865..b2f32e82 100644 --- a/docs/source/reference/config-sudo.md +++ b/docs/source/reference/config-sudo.md @@ -70,7 +70,8 @@ Cmnd_Alias JUPYTER_CMD = /usr/local/bin/sudospawner rhea ALL=(JUPYTER_USERS) NOPASSWD:JUPYTER_CMD ``` -It might be useful to modify `secure_path` to add commands in path. +It might be useful to modify `secure_path` to add commands in path. (Search for +`secure_path` in the [sudo docs](https://www.sudo.ws/man/1.8.14/sudoers.man.html) As an alternative to adding every user to the `/etc/sudoers` file, you can use a group in the last line above, instead of `JUPYTER_USERS`: @@ -152,7 +153,8 @@ then you will need to give `node` permission to do so: ```bash sudo setcap 'cap_net_bind_service=+ep' /usr/bin/node ``` -However, you may want to further understand the consequences of this. +However, you may want to further understand the consequences of this. +([Further reading](http://man7.org/linux/man-pages/man7/capabilities.7.html)) You may also be interested in limiting the amount of CPU any process can use on your server. `cpulimit` is a useful tool that is available for many Linux @@ -163,7 +165,8 @@ instructions](http://ubuntuforums.org/showthread.php?t=992706). ### Shadow group (FreeBSD) -**NOTE:** This has not been tested and may not work as expected. +**NOTE:** This has not been tested on FreeBSD and may not work as expected on +the FreeBSD platform. *Do not use in production without verifying that it works properly!* ```bash $ ls -l /etc/spwd.db /etc/master.passwd @@ -249,6 +252,6 @@ $ semodule -i sudo_exec_selinux.pp ## Troubleshooting: PAM session errors If the PAM authentication doesn't work and you see errors for -`login:session-auth`, or similar, considering updating to a more recent version +`login:session-auth`, or similar, consider updating to a more recent version of jupyterhub and disabling the opening of PAM sessions with `c.PAMAuthenticator.open_sessions=False`. diff --git a/docs/source/reference/config-user-env.md b/docs/source/reference/config-user-env.md index c085b106..27fd9b30 100644 --- a/docs/source/reference/config-user-env.md +++ b/docs/source/reference/config-user-env.md @@ -7,8 +7,8 @@ environment in some way. Since the `jupyterhub-singleuser` server extends the standard Jupyter notebook server, most configuration and documentation that applies to Jupyter Notebook applies to the single-user environments. Configuration of user environments -typically does not occur through JupyterHub itself, but rather through system- -wide configuration of Jupyter, which is inherited by `jupyterhub-singleuser`. +typically does not occur through JupyterHub itself, but rather through system-wide +configuration of Jupyter, which is inherited by `jupyterhub-singleuser`. **Tip:** When searching for configuration tips for JupyterHub user environments, try removing JupyterHub from your search because there are a lot @@ -43,9 +43,13 @@ sudo python3 -m pip install numpy to install the numpy package in the default system Python 3 environment (typically `/usr/local`). +TODO: Get a link from the conda team for a description of what "appropriate permissions for users" is + You may also use conda to install packages. If you do, you should make sure that the conda environment has appropriate permissions for users to be able to -run Python code in the env. +run Python code in the env. The env must be *readable and executable* by all +users. Additionally it must be *writeable* if you want users to install +additional packages. ## Configuring Jupyter and IPython diff --git a/docs/source/reference/proxy.md b/docs/source/reference/proxy.md index fd58816a..620073b6 100644 --- a/docs/source/reference/proxy.md +++ b/docs/source/reference/proxy.md @@ -138,7 +138,7 @@ async def delete_route(self, routespec): For retrieval, you only *need* to implement a single method that retrieves all routes. The return value for this function should be a dictionary, keyed by -`routespect`, of dicts whose keys are the same three arguments passed to +`routespec`, of dicts whose keys are the same three arguments passed to `add_route` (`routespec`, `target`, `data`) ```python @@ -204,7 +204,7 @@ setup( ``` If you have added this metadata to your package, -users can select your proxy with the configuration: +admins can select your authenticator with the configuration: ```python c.JupyterHub.proxy_class = 'mything' @@ -216,7 +216,7 @@ instead of the full c.JupyterHub.proxy_class = 'mypackage:MyProxy' ``` -previously required. +as previously required. Additionally, configurable attributes for your proxy will appear in jupyterhub help output and auto-generated configuration files via `jupyterhub --generate-config`. diff --git a/docs/source/reference/separate-proxy.md b/docs/source/reference/separate-proxy.md index 92c19ef4..56f3dd2b 100644 --- a/docs/source/reference/separate-proxy.md +++ b/docs/source/reference/separate-proxy.md @@ -14,8 +14,8 @@ because the proxy is automatically managed by the hub. This is great for getting started and even most use, but everytime you restart the hub, all user connections also get restarted. But it's also simple to run the proxy as a service separate from the hub, so that you are free -to reconfigure the hub while only interrupting users who are currently -actively starting the hub. +to reconfigure the hub while only interrupting users who are waiting for their notebook server to start. +starting their notebook server. The default JupyterHub proxy is [configurable-http-proxy](https://github.com/jupyterhub/configurable-http-proxy), diff --git a/docs/source/reference/services.md b/docs/source/reference/services.md index 24de3bdd..3020fa69 100644 --- a/docs/source/reference/services.md +++ b/docs/source/reference/services.md @@ -65,7 +65,7 @@ If a service is also to be managed by the Hub, it has a few extra options: A **Hub-Managed Service** is started by the Hub, and the Hub is responsible for the Service's actions. A Hub-Managed Service can only be a local subprocess of the Hub. The Hub will take care of starting the process and -restarts it if it stops. +restart the service if the service stops. While Hub-Managed Services share some similarities with notebook Spawners, there are no plans for Hub-Managed Services to support the same spawning @@ -201,8 +201,18 @@ below. The reference, or base, implementation is the [`HubAuth`][HubAuth] class, which implements the requests to the Hub. -To use HubAuth, you must set the `.api_token`, either programmatically when constructing the class, -or via the `JUPYTERHUB_API_TOKEN` environment variable. +To use HubAuth, you must set the `.api_token` instance variable. This can be +done either programmatically when constructing the class, or via the +`JUPYTERHUB_API_TOKEN` environment variable. A number of the examples in the +root of the jupyterhub git repository set the `JUPYTERHUB_API_TOKEN` variable +so consider having a look at those for futher reading +([cull-idle](https://github.com/jupyterhub/jupyterhub/tree/master/examples/cull-idle), +[external-oauth](https://github.com/jupyterhub/jupyterhub/tree/master/examples/external-oauth), +[service-notebook](https://github.com/jupyterhub/jupyterhub/tree/master/examples/service-notebook) +and [service-whoiami](https://github.com/jupyterhub/jupyterhub/tree/master/examples/service-whoami)) + + +(TODO: Where is this API TOKen set?) Most of the logic for authentication implementation is found in the [`HubAuth.user_for_cookie`][HubAuth.user_for_cookie] @@ -353,13 +363,13 @@ and taking note of the following process: ```json { "name": "inara", - "groups": ["serenity", "guild"], + "groups": ["serenity", "guild"] } ``` An example of using an Externally-Managed Service and authentication is -in [nbviewer README][nbviewer example] section on securing the notebook viewer, +in the [nbviewer README][nbviewer example] section on securing the notebook viewer, and an example of its configuration is found [here](https://github.com/jupyter/nbviewer/blob/master/nbviewer/providers/base.py#L94). nbviewer can also be run as a Hub-Managed Service as described [nbviewer README][nbviewer example] section on securing the notebook viewer. diff --git a/docs/source/reference/spawners.md b/docs/source/reference/spawners.md index 37937c72..cc3b7a65 100644 --- a/docs/source/reference/spawners.md +++ b/docs/source/reference/spawners.md @@ -16,9 +16,9 @@ Some examples include: - [DockerSpawner](https://github.com/jupyterhub/dockerspawner) for spawning user servers in Docker containers * `dockerspawner.DockerSpawner` for spawning identical Docker containers for - each users + each user * `dockerspawner.SystemUserSpawner` for spawning Docker containers with an - environment and home directory for each users + environment and home directory for each user * both `DockerSpawner` and `SystemUserSpawner` also work with Docker Swarm for launching containers on remote machines - [SudoSpawner](https://github.com/jupyterhub/sudospawner) enables JupyterHub to @@ -222,8 +222,8 @@ discover these resource limits and guarantees, such as for memory and CPU. For the limits and guarantees to be useful, **the spawner must implement support for them**. For example, LocalProcessSpawner, the default spawner, does not support limits and guarantees. One of the spawners -that supports limits and guarantees is the `systemdspawner`. - +that supports limits and guarantees is the +[`systemdspawner`](https://github.com/jupyterhub/systemdspawner). ### Memory Limits & Guarantees @@ -283,9 +283,10 @@ container `ip` prior to starting and pass that to `.create_certs` (TODO: edit). In general though, this method will not need to be changed and the default `ip`/`dns` (localhost) info will suffice. -When `.create_certs` is run, it will `.create_certs` in a default, central -location specified by `c.JupyterHub.internal_certs_location`. For `Spawners` -that need access to these certs elsewhere (i.e. on another host altogether), -the `.move_certs` method can be overridden to move the certs appropriately. -Again, using `DockerSpawner` as an example, this would entail moving certs -to a directory that will get mounted into the container this spawner starts. +When `.create_certs` is run, it will create the certificates in a default, +central location specified by `c.JupyterHub.internal_certs_location`. For +`Spawners` that need access to these certs elsewhere (i.e. on another host +altogether), the `.move_certs` method can be overridden to move the certs +appropriately. Again, using `DockerSpawner` as an example, this would entail +moving certs to a directory that will get mounted into the container this +spawner starts. diff --git a/docs/source/reference/urls.md b/docs/source/reference/urls.md index 6497fcb1..53aefe4b 100644 --- a/docs/source/reference/urls.md +++ b/docs/source/reference/urls.md @@ -119,14 +119,9 @@ This URL indicates a request for a user server that is not running (because `/user/...` would have been handled by the notebook server if the specified server were running). -Handling this URL is the most complicated condition in JupyterHub, -because there can be many states: +Handling this URL depends on two conditions: whether a requested user is found +as a match and the state of the requested user's notebook server. -1. server is not active - a. user matches - b. user doesn't match -2. server is ready -3. server is pending, but not ready If the server is pending spawn, the browser will be redirected to `/hub/spawn-pending/:username/:servername` From 09c4fee780dd6ed03d295865f671cb14f69afb7b Mon Sep 17 00:00:00 2001 From: mouse1203 Date: Fri, 2 Sep 2022 13:12:31 +0200 Subject: [PATCH 002/214] add selenium tests --- .github/workflows/test.yml | 9 + dev-requirements.txt | 1 + jupyterhub/tests/selenium/__init__.py | 0 jupyterhub/tests/selenium/conftest.py | 10 + jupyterhub/tests/selenium/locators.py | 93 +++++ jupyterhub/tests/selenium/test_browser.py | 417 ++++++++++++++++++++++ 6 files changed, 530 insertions(+) create mode 100644 jupyterhub/tests/selenium/__init__.py create mode 100644 jupyterhub/tests/selenium/conftest.py create mode 100644 jupyterhub/tests/selenium/locators.py create mode 100644 jupyterhub/tests/selenium/test_browser.py diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 18cf6cae..8ec37393 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -214,6 +214,15 @@ jobs: DB=postgres bash ci/docker-db.sh DB=postgres bash ci/init-db.sh fi + - name: Setup Firefox + if: matrix.os == 'ubuntu-latest' + uses: browser-actions/setup-firefox@latest + with: + firefox-version: latest + + - name: Setup Geckodriver + if: matrix.os == 'ubuntu-latest' + uses: browser-actions/setup-geckodriver@latest - name: Run pytest run: | diff --git a/dev-requirements.txt b/dev-requirements.txt index a7bdaefe..e97dfb6c 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -18,6 +18,7 @@ pytest>=3.3 pytest-asyncio>=0.17 pytest-cov requests-mock +selenium tbump # blacklist urllib3 releases affected by https://github.com/urllib3/urllib3/issues/1683 # I *think* this should only affect testing, not production diff --git a/jupyterhub/tests/selenium/__init__.py b/jupyterhub/tests/selenium/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/jupyterhub/tests/selenium/conftest.py b/jupyterhub/tests/selenium/conftest.py new file mode 100644 index 00000000..a6352a4d --- /dev/null +++ b/jupyterhub/tests/selenium/conftest.py @@ -0,0 +1,10 @@ +import pytest +from selenium import webdriver + + +@pytest.fixture() +def browser(): + driver = webdriver.Firefox() + yield driver + driver.close() + driver.quit() diff --git a/jupyterhub/tests/selenium/locators.py b/jupyterhub/tests/selenium/locators.py new file mode 100644 index 00000000..07d3e67e --- /dev/null +++ b/jupyterhub/tests/selenium/locators.py @@ -0,0 +1,93 @@ +from selenium.webdriver.common.by import By + + +class LoginPageLocators: + + FORM_LOGIN = (By.XPATH, '//*[@id="login-main"]/form') + """/hub/login?next=%2Fhub%2F" , "/hub/login?next=""" + + SIGN_IN = (By.CLASS_NAME, 'auth-form-header') + ACCOUNT = (By.ID, "username_input") + PASSWORD = (By.ID, "password_input") + LOGIN_BUTTON = (By.ID, "login_submit") + """ + """ + LOGO = (By.ID, "jupyterhub-logo") + LOGO_LINK = (By.XPATH, '//*[@id="jupyterhub-logo"]/a') + LOGO_TITLE = (By.XPATH, '//*[@id="jupyterhub-logo"]/a/img') + + ERROR_INVALID_CREDANTIALS = (By.CSS_SELECTOR, "p.login_error") + + PAGE_TITLE = 'JupyterHub' + ERROR_MESSAGES_LOGIN = "Invalid username or password" + + ERROR_403 = (By.CLASS_NAME, "error") + ERROR_MESSAGES_403 = ( + "Action is not authorized with current scopes; requires any of [admin-ui]" + ) + + +class HomePageLocators: + """http://127.0.0.1:8000/hub/home""" + + """Home""" + LINK_HOME = (By.CSS_SELECTOR, "a[href*='/hub/home']") + + """Token""" + LINK_TOKEN = (By.CSS_SELECTOR, "a[href*='/hub/token']") + + """

username

""" + USER_NAME = (By.XPATH, "to be done") + + """ Logout + """ + BUTTON_LOGOUT = (By.ID, "logout") + + """ + Start + My Server + """ + """has 2 names My server and Start My server""" + + BUTTON_START_SERVER = (By.ID, "start") + BUTTON_STOP_SERVER = (By.ID, "stop") + + """""" + BUTTON_REVOKE = () + + +class TokenPageLocators: + """""" + + BUTTON_API_REQ = (By.XPATH, '//*[@id="request-token-form"]/div[1]/button') + + """""" + INPUT_TOKEN = (By.ID, "token-note") + + """ """ + + LIST_EXP_TOKEN_FIELD = (By.ID, "token-expiration-seconds") + LIST_EXP_TOKEN_OPT = (By.XPATH, '//option') + + NEVER_EXP = (By.ID, "Never") + DAY1 = (By.ID, "3600") + + """
+ Your new API Token +
""" + + """ style= "display: none;" when it is hidden""" + PANEL_AREA = (By.ID, 'token-area') + PANEL_TOKEN = (By.CLASS_NAME, 'panel-heading') + + """7226c895c101455889a82ca908251c68""" + RESULT_TOKEN = (By.ID, 'token-result') + TEXT = "Copy this token. You won't be able to see it again, but you can always come back here to get a new one." diff --git a/jupyterhub/tests/selenium/test_browser.py b/jupyterhub/tests/selenium/test_browser.py new file mode 100644 index 00000000..5a43a526 --- /dev/null +++ b/jupyterhub/tests/selenium/test_browser.py @@ -0,0 +1,417 @@ +import asyncio +import pickle +import pprint +import time +from functools import partial +from operator import contains +from pprint import pprint +from termios import TABDLY +from traceback import format_stack +from urllib.parse import urlencode, urlparse + +import pytest +from selenium import webdriver +from selenium.common.exceptions import NoSuchElementException, TimeoutException +from selenium.webdriver.common.by import By +from selenium.webdriver.support import expected_conditions as EC +from selenium.webdriver.support.ui import WebDriverWait +from tornado.escape import url_escape +from tornado.httputil import url_concat + +from jupyterhub.tests.conftest import admin_user +from jupyterhub.tests.selenium.locators import ( + HomePageLocators, + LoginPageLocators, + TokenPageLocators, +) +from jupyterhub.utils import exponential_backoff + +from ...utils import url_escape_path, url_path_join +from ..utils import async_requests, get_page, public_host, public_url, ujoin + + +async def webdriver_wait(driver, condition, timeout=30): + return await exponential_backoff( + partial(condition, driver), + timeout=timeout, + fail_message=f"WebDriver condition not met: {condition}", + ) + + +def in_thread(f, *args, **kwargs): + """Run a function in a background thread + + via current event loop's run_in_executor + + Returns asyncio.Future + """ + + return asyncio.get_event_loop().run_in_executor(None, partial(f, *args, **kwargs)) + + +async def open_url(app, browser): + + url = url_path_join(public_host(app), app.hub.base_url, "login") + await in_thread(browser.get, url) + return url + + +def click(browser, by_locator): + WebDriverWait(browser, 10).until( + EC.visibility_of_element_located(by_locator) + ).click() + + +def is_displayed(browser, by_locator): + return ( + WebDriverWait(browser, 10) + .until(EC.visibility_of_element_located(by_locator)) + .is_displayed() + ) + + +def send_text(browser, by_locator, text): + return ( + WebDriverWait(browser, 10) + .until(EC.presence_of_element_located(by_locator)) + .send_keys(text) + ) + + +def clear(browser, by_locator): + return ( + WebDriverWait(browser, 10) + .until(EC.presence_of_element_located(by_locator)) + .clear() + ) + + +def element(browser, by_locator): + WebDriverWait(browser, 10).until(EC.visibility_of_element_located(by_locator)) + + +def delete_cookies(browser, domains=None): + + if domains is not None: + cookies = browser.get_cookies() + original_len = len(cookies) + for cookie in cookies: + if str(cookie["domain"]) in domains: + cookies.remove(cookie) + if len(cookies) < original_len: # if cookies changed, we will update them + # deleting everything and adding the modified cookie object + browser.delete_all_cookies() + for cookie in cookies: + browser.add_cookie(cookie) + else: + browser.delete_all_cookies() + + +# LOGIN PAGE +async def test_elements_of_login_page(app, browser): + await open_url(app, browser) + logo = is_displayed(browser, LoginPageLocators.LOGO) + logo_text = browser.find_element(*LoginPageLocators.LOGO).get_attribute("innerHTML") + """TBD""" + assert logo == True + + +async def login(browser, user, pass_w): + # fill in username field + send_text(browser, LoginPageLocators.ACCOUNT, user) + # fill in password field + send_text(browser, LoginPageLocators.PASSWORD, pass_w) + # click submit button + click(browser, LoginPageLocators.LOGIN_BUTTON) + await webdriver_wait(browser, EC.url_changes(browser.current_url)) + + +async def test_submit_login_form(app, browser): + user = "test_user" + pass_w = "test_user" + + await open_url(app, browser) + redirected_url = ujoin(public_url(app), f"/user/{user}/") + await login(browser, user, pass_w) + # verify url contains username + if f"/user/{user}/" not in browser.current_url: + webdriver_wait(browser, EC.url_to_be(redirected_url)) + else: + pass + assert browser.current_url == redirected_url + + +@pytest.mark.parametrize( + 'url, params, redirected_url, form_action', + [ + ( + # spawn?param=value + # will encode given parameters for an unauthenticated URL in the next url + # the next parameter will contain the app base URL (replaces BASE_URL in tests) + 'spawn', + [('param', 'value')], + '/hub/login?next={{BASE_URL}}hub%2Fspawn%3Fparam%3Dvalue', + '/hub/login?next={{BASE_URL}}hub%2Fspawn%3Fparam%3Dvalue', + ), + ( + # login?param=fromlogin&next=encoded(/hub/spawn?param=value) + # will drop parameters given to the login page, passing only the next url + 'login', + [('param', 'fromlogin'), ('next', '/hub/spawn?param=value')], + '/hub/login?param=fromlogin&next=%2Fhub%2Fspawn%3Fparam%3Dvalue', + '/hub/login?next=%2Fhub%2Fspawn%3Fparam%3Dvalue', + ), + ( + # login?param=value&anotherparam=anothervalue + # will drop parameters given to the login page, and use an empty next url + 'login', + [('param', 'value'), ('anotherparam', 'anothervalue')], + '/hub/login?param=value&anotherparam=anothervalue', + '/hub/login?next=', + ), + ( + # login + # simplest case, accessing the login URL, gives an empty next url + 'login', + [], + '/hub/login', + '/hub/login?next=', + ), + ], +) +async def test_open_url_login(app, browser, url, params, redirected_url, form_action): + url = url_path_join(public_host(app), app.hub.base_url, url) + url_new = url_concat(url, params) + await in_thread(browser.get, url_new) + redirected_url = redirected_url.replace('{{BASE_URL}}', url_escape(app.base_url)) + form_action = form_action.replace('{{BASE_URL}}', url_escape(app.base_url)) + form = browser.find_element(*LoginPageLocators.FORM_LOGIN).get_attribute('action') + + print("Link in form: " + form) + # verify title / url + assert browser.title == LoginPageLocators.PAGE_TITLE + assert form.endswith(form_action) + print("Current Link in form: " + browser.current_url) + # login in with params + await login(browser, user='test_user', pass_w='test_user') + # verify next url + params + user = 'test_user' + next_url = browser.current_url + if url_escape(app.base_url) in form_action: + print("Current Link 2: " + browser.current_url) + assert next_url.endswith("param=value") + elif "next=%2Fhub" in form_action: + assert next_url.endswith("spawn?param=value") + assert f"user/{user}/" not in next_url + print("Current Link 2: " + browser.current_url) + else: + if next_url.endswith(f"/user/{user}/") == False: + webdriver_wait( + browser, EC.url_to_be(ujoin(public_url(app), f"/user/{user}/")) + ) + assert next_url.endswith(f"/user/{user}/") + print("Current Link 2: " + browser.current_url) + + +"""TBD""" + + +@pytest.mark.parametrize( + 'running, next_url, location, params', + [ + # default URL if next not specified, for both running and not + (True, '', '', None), + (False, '', '', None), + # next_url is respected + (False, '/hub/admin', '/hub/admin', None), + (False, '/user/other', '/hub/user/other', None), + (False, '/absolute', '/absolute', None), + (False, '/has?query#andhash', '/has?query#andhash', None), + # :// in query string or fragment + (False, '/has?repo=https/host.git', '/has?repo=https/host.git', None), + (False, '/has?repo=https://host.git', '/has?repo=https://host.git', None), + (False, '/has#repo=https://host.git', '/has#repo=https://host.git', None), + # next_url outside is not allowed + (False, 'relative/path', '', None), + (False, 'https://other.domain', '', None), + (False, 'ftp://other.domain', '', None), + (False, '//other.domain', '', None), + (False, '///other.domain/triple', '', None), + (False, '\\\\other.domain/backslashes', '', None), + # params are handled correctly (ignored if ?next= specified) + ( + True, + '/hub/admin?left=1&right=2', + 'hub/admin?left=1&right=2', + {"left": "abc"}, + ), + (False, '/hub/admin', 'hub/admin', [('left', 1), ('right', 2)]), + (True, '', '', {"keep": "yes"}), + (False, '', '', {"keep": "yes"}), + ], +) +async def test_login_redirect(app, browser, running, next_url, location, params, user): + + if location: + location = ujoin(app.base_url, location) + print("if location is:" + location) + elif running: + # location not specified, + location = user.url + if params: + location = url_concat(location, params) + else: + # use default url + location = ujoin(app.base_url, 'hub/spawn') + if params: + location = url_concat(location, params) + + url = 'login' + if params: + url = url_concat(url, params) + if next_url: + if next_url.startswith('/') and not ( + next_url.startswith("//") or urlparse(next_url).netloc + ): + next_url = ujoin(app.base_url, next_url, '') + url = url_concat(url, dict(next=next_url)) + + if running and not user.active: + # ensure running + await user.spawn() + elif user.active and not running: + # ensure not running + await user.stop() + time.sleep(5) + # open_url(app,browser) + await in_thread(browser.get, (ujoin(public_url(app), url))) + await login(browser, user='test_user', pass_w='test_user') + + # await get_page(url, app, allow_redirects=False) + time.sleep(50) + + if user != 'admin' and next_url == '/hub/admin': # user!=user.admin + assert browser.current_url.endswith('hub/admin/') + elm = browser.find_element( + LoginPageLocators.ERROR_403 + ) # .get_attribute('action') + print(elm) + assert elm == LoginPageLocators.ERROR_MESSAGES_403 + if params: + assert browser.current_url.endswith('hub/admin/') + elm = browser.find_element( + *LoginPageLocators.ERROR_403 + ) # .get_attribute('action') + print(elm) + assert elm == LoginPageLocators.ERROR_MESSAGES_403 + + elif next_url == '/has?repo=https': + # print(app.users['test_user'].keys()) + print("else if") + else: + print("else") + + """ +
+ + +
+ 0. 1., 9-14, 17-18 /space word/user/test_user/ + 1. /space word/user/test_user/ + 2. - 15,16 - space word/hub/admin

+ 403 : Forbidden +

:

+

+ Action is not authorized with current scopes; requires any of [admin-ui] +

+ + 3. space word/hub/user/other

+ 404 : Not Found +

Jupyter has lots of moons, but this is not one...

+ 4. - 8. the same as 3 + """ + + """ + print("url ---" +browser.current_url) + + #print("user ()" +f"{user}") + print("user (type)" +str(user)) + assert location in browser.current_url """ + + +@pytest.mark.parametrize( + "user, pass_w", + [ + (" ", ""), + ("user", ""), + (" ", "password"), + ("user", "password"), + ], +) +async def test_invalid_credantials(app, browser, user, pass_w): + + await open_url(app, browser) + await login(browser, user, pass_w) + time.sleep(5) + error = browser.find_element(*LoginPageLocators.ERROR_INVALID_CREDANTIALS).text + + # verify error message and url + assert LoginPageLocators.ERROR_MESSAGES_LOGIN == error + assert 'hub/login' in browser.current_url + + """"on page http://localhost:8000/user/username/lab + check only page title for now""" + + +async def test_logout(app, browser): + if is_displayed(browser, HomePageLocators.BUTTON_LOGOUT) == True: + click(browser, HomePageLocators.BUTTON_LOGOUT) + else: + print("TBD") + await async_requests.get(public_host(app)) + + assert 'hub/login' in browser.current_url + + +# HOME PAGE + + +def navigate_bar(): + print("TBD") + + +# TOKEN PAGE +@pytest.mark.parametrize( + "user, pass_w", + [ + ("user", "user"), + ], +) +async def test_elements_of_token_page(app, browser, user, pass_w): + await open_url(app, browser) + await login(browser, user, pass_w) + + cookies = await app.login_user(user) + await get_page("token", app, cookies=cookies) + # await get_page("token", app) + + buttonAPI = is_displayed(browser, TokenPageLocators.BUTTON_API_REQ) + token_note = is_displayed(browser, TokenPageLocators.INPUT_TOKEN) + token_dropdown = is_displayed(browser, TokenPageLocators.LIST_EXP_TOKEN_FIELD) + + options = browser.find_elements(TokenPageLocators.LIST_EXP_TOKEN_OPT) + for option in options: + is_selected = option.is_selected() + print(str(is_selected)) + + """TBD""" + assert buttonAPI == True + assert token_note == True + assert token_dropdown == True + assert str(is_selected) == "Never" + + +async def test_request_token(app, browser, user, pass_w): + await login(app, browser, user, pass_w) + cookies = await app.login_user(user) + await get_page("token", app, cookies=cookies) From b87561d396b6b183d56dd3f8bbb1a75d00d23193 Mon Sep 17 00:00:00 2001 From: mouse1203 Date: Fri, 2 Sep 2022 13:32:11 +0200 Subject: [PATCH 003/214] remove conditional for FireFox removing conditionals for FF --- .github/workflows/test.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 8ec37393..8f702092 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -215,13 +215,11 @@ jobs: DB=postgres bash ci/init-db.sh fi - name: Setup Firefox - if: matrix.os == 'ubuntu-latest' uses: browser-actions/setup-firefox@latest with: firefox-version: latest - name: Setup Geckodriver - if: matrix.os == 'ubuntu-latest' uses: browser-actions/setup-geckodriver@latest - name: Run pytest From 5885f88d798556606d3f6d4daadb335b66667812 Mon Sep 17 00:00:00 2001 From: mouse1203 <34538183+mouse1203@users.noreply.github.com> Date: Fri, 2 Sep 2022 13:43:20 +0200 Subject: [PATCH 004/214] Update jupyterhub/tests/selenium/conftest.py Co-authored-by: Min RK --- jupyterhub/tests/selenium/conftest.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/jupyterhub/tests/selenium/conftest.py b/jupyterhub/tests/selenium/conftest.py index a6352a4d..9454e0f3 100644 --- a/jupyterhub/tests/selenium/conftest.py +++ b/jupyterhub/tests/selenium/conftest.py @@ -4,7 +4,9 @@ from selenium import webdriver @pytest.fixture() def browser(): - driver = webdriver.Firefox() + options = webdriver.FirefoxOptions() + options.set_headless() + driver = webdriver.Firefox(firefox_options=options) yield driver driver.close() driver.quit() From b90200667fe1b3adaa80494971e3ad38fc7d3a3c Mon Sep 17 00:00:00 2001 From: Min RK Date: Fri, 2 Sep 2022 13:55:48 +0200 Subject: [PATCH 005/214] fix headless firefox option --- jupyterhub/tests/selenium/conftest.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/jupyterhub/tests/selenium/conftest.py b/jupyterhub/tests/selenium/conftest.py index 9454e0f3..a68ce6e3 100644 --- a/jupyterhub/tests/selenium/conftest.py +++ b/jupyterhub/tests/selenium/conftest.py @@ -5,8 +5,8 @@ from selenium import webdriver @pytest.fixture() def browser(): options = webdriver.FirefoxOptions() - options.set_headless() - driver = webdriver.Firefox(firefox_options=options) + options.headless = True + driver = webdriver.Firefox(options=options) yield driver driver.close() driver.quit() From 485190e5af141f6d432a69fb24fdd91f9dd7f246 Mon Sep 17 00:00:00 2001 From: Min RK Date: Fri, 2 Sep 2022 13:53:16 +0200 Subject: [PATCH 006/214] make selenium tests opt-in avoids running them over and over again --- .github/workflows/test.yml | 9 +++++++++ jupyterhub/tests/selenium/test_browser.py | 4 +++- pytest.ini | 3 ++- 3 files changed, 14 insertions(+), 2 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 8f702092..e9e25d28 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -87,6 +87,8 @@ jobs: subdomain: subdomain - python: "3.10" ssl: ssl + - python: "3.10" + selenium: selenium - python: "3.11.0-rc.1" - python: "3.10" main_dependencies: main_dependencies @@ -215,16 +217,23 @@ jobs: DB=postgres bash ci/init-db.sh fi - name: Setup Firefox + if: matrix.selenium uses: browser-actions/setup-firefox@latest with: firefox-version: latest - name: Setup Geckodriver + if: matrix.selenium uses: browser-actions/setup-geckodriver@latest + - name: Configure selenium tests + if: matrix.selenium + run: echo "PYTEST_ADDOPTS=$PYTEST_ADDOPTS -m selenium" >> "${GITHUB_ENV}" + - name: Run pytest run: | pytest --maxfail=2 --cov=jupyterhub jupyterhub/tests + - name: Submit codecov report run: | codecov diff --git a/jupyterhub/tests/selenium/test_browser.py b/jupyterhub/tests/selenium/test_browser.py index 5a43a526..067b259e 100644 --- a/jupyterhub/tests/selenium/test_browser.py +++ b/jupyterhub/tests/selenium/test_browser.py @@ -29,6 +29,8 @@ from jupyterhub.utils import exponential_backoff from ...utils import url_escape_path, url_path_join from ..utils import async_requests, get_page, public_host, public_url, ujoin +pytestmark = pytest.mark.selenium + async def webdriver_wait(driver, condition, timeout=30): return await exponential_backoff( @@ -333,7 +335,7 @@ async def test_login_redirect(app, browser, running, next_url, location, params, """ print("url ---" +browser.current_url) - + #print("user ()" +f"{user}") print("user (type)" +str(user)) assert location in browser.current_url """ diff --git a/pytest.ini b/pytest.ini index fb03e4da..82c19dcd 100644 --- a/pytest.ini +++ b/pytest.ini @@ -7,7 +7,7 @@ asyncio_mode = auto # jupyter_server plugin is incompatible with notebook imports -addopts = -p no:jupyter_server +addopts = -p no:jupyter_server -m 'not selenium' python_files = test_*.py markers = @@ -17,3 +17,4 @@ markers = user: mark as a test for a user slow: mark a test as slow role: mark as a test for roles + selenium: web tests that run with selenium From 9d90496549f1c1e573e4ed3b60f922d931a261ae Mon Sep 17 00:00:00 2001 From: mouse1203 Date: Mon, 19 Sep 2022 15:18:07 +0200 Subject: [PATCH 007/214] updating regarding to the review removed time.sleep(), removed inrelevant comments, removed unused packages, added few explanations for some functions (in progress), documenting in progress, added to conftest.py package with Options class --- jupyterhub/tests/selenium/conftest.py | 1 + jupyterhub/tests/selenium/locators.py | 54 +---- jupyterhub/tests/selenium/test_browser.py | 237 ++-------------------- 3 files changed, 27 insertions(+), 265 deletions(-) diff --git a/jupyterhub/tests/selenium/conftest.py b/jupyterhub/tests/selenium/conftest.py index a68ce6e3..06de2029 100644 --- a/jupyterhub/tests/selenium/conftest.py +++ b/jupyterhub/tests/selenium/conftest.py @@ -1,5 +1,6 @@ import pytest from selenium import webdriver +from selenium.webdriver.firefox.options import Options as FirefoxOptions @pytest.fixture() diff --git a/jupyterhub/tests/selenium/locators.py b/jupyterhub/tests/selenium/locators.py index 07d3e67e..9a5657f7 100644 --- a/jupyterhub/tests/selenium/locators.py +++ b/jupyterhub/tests/selenium/locators.py @@ -2,25 +2,19 @@ from selenium.webdriver.common.by import By class LoginPageLocators: + """class for handling the login page locators""" FORM_LOGIN = (By.XPATH, '//*[@id="login-main"]/form') - """/hub/login?next=%2Fhub%2F" , "/hub/login?next=""" - SIGN_IN = (By.CLASS_NAME, 'auth-form-header') ACCOUNT = (By.ID, "username_input") PASSWORD = (By.ID, "password_input") LOGIN_BUTTON = (By.ID, "login_submit") - """ - """ LOGO = (By.ID, "jupyterhub-logo") LOGO_LINK = (By.XPATH, '//*[@id="jupyterhub-logo"]/a') LOGO_TITLE = (By.XPATH, '//*[@id="jupyterhub-logo"]/a/img') - ERROR_INVALID_CREDANTIALS = (By.CSS_SELECTOR, "p.login_error") - PAGE_TITLE = 'JupyterHub' ERROR_MESSAGES_LOGIN = "Invalid username or password" - ERROR_403 = (By.CLASS_NAME, "error") ERROR_MESSAGES_403 = ( "Action is not authorized with current scopes; requires any of [admin-ui]" @@ -28,66 +22,26 @@ class LoginPageLocators: class HomePageLocators: - """http://127.0.0.1:8000/hub/home""" + """class for handling the home page locators""" - """Home""" + LINK_HOME_BAR = (By.CSS_SELECTOR, "div.container-fluid a") LINK_HOME = (By.CSS_SELECTOR, "a[href*='/hub/home']") - - """Token""" LINK_TOKEN = (By.CSS_SELECTOR, "a[href*='/hub/token']") - - """

username

""" - USER_NAME = (By.XPATH, "to be done") - - """ Logout - """ BUTTON_LOGOUT = (By.ID, "logout") - - """ - Start - My Server - """ - """has 2 names My server and Start My server""" - BUTTON_START_SERVER = (By.ID, "start") BUTTON_STOP_SERVER = (By.ID, "stop") - """""" - BUTTON_REVOKE = () - class TokenPageLocators: - """""" + """class for handling the Token page locators""" BUTTON_API_REQ = (By.XPATH, '//*[@id="request-token-form"]/div[1]/button') - - """""" INPUT_TOKEN = (By.ID, "token-note") - - """ """ - LIST_EXP_TOKEN_FIELD = (By.ID, "token-expiration-seconds") LIST_EXP_TOKEN_OPT = (By.XPATH, '//option') - NEVER_EXP = (By.ID, "Never") DAY1 = (By.ID, "3600") - - """
- Your new API Token -
""" - - """ style= "display: none;" when it is hidden""" PANEL_AREA = (By.ID, 'token-area') PANEL_TOKEN = (By.CLASS_NAME, 'panel-heading') - - """7226c895c101455889a82ca908251c68""" RESULT_TOKEN = (By.ID, 'token-result') TEXT = "Copy this token. You won't be able to see it again, but you can always come back here to get a new one." diff --git a/jupyterhub/tests/selenium/test_browser.py b/jupyterhub/tests/selenium/test_browser.py index 067b259e..8b0a04e9 100644 --- a/jupyterhub/tests/selenium/test_browser.py +++ b/jupyterhub/tests/selenium/test_browser.py @@ -1,13 +1,6 @@ import asyncio -import pickle -import pprint import time from functools import partial -from operator import contains -from pprint import pprint -from termios import TABDLY -from traceback import format_stack -from urllib.parse import urlencode, urlparse import pytest from selenium import webdriver @@ -18,7 +11,6 @@ from selenium.webdriver.support.ui import WebDriverWait from tornado.escape import url_escape from tornado.httputil import url_concat -from jupyterhub.tests.conftest import admin_user from jupyterhub.tests.selenium.locators import ( HomePageLocators, LoginPageLocators, @@ -26,8 +18,8 @@ from jupyterhub.tests.selenium.locators import ( ) from jupyterhub.utils import exponential_backoff -from ...utils import url_escape_path, url_path_join -from ..utils import async_requests, get_page, public_host, public_url, ujoin +from ...utils import url_path_join +from ..utils import public_host, public_url, ujoin pytestmark = pytest.mark.selenium @@ -52,7 +44,7 @@ def in_thread(f, *args, **kwargs): async def open_url(app, browser): - + """initiating open the login page in the browser""" url = url_path_join(public_host(app), app.hub.base_url, "login") await in_thread(browser.get, url) return url @@ -92,29 +84,11 @@ def element(browser, by_locator): WebDriverWait(browser, 10).until(EC.visibility_of_element_located(by_locator)) -def delete_cookies(browser, domains=None): - - if domains is not None: - cookies = browser.get_cookies() - original_len = len(cookies) - for cookie in cookies: - if str(cookie["domain"]) in domains: - cookies.remove(cookie) - if len(cookies) < original_len: # if cookies changed, we will update them - # deleting everything and adding the modified cookie object - browser.delete_all_cookies() - for cookie in cookies: - browser.add_cookie(cookie) - else: - browser.delete_all_cookies() - - # LOGIN PAGE async def test_elements_of_login_page(app, browser): await open_url(app, browser) logo = is_displayed(browser, LoginPageLocators.LOGO) logo_text = browser.find_element(*LoginPageLocators.LOGO).get_attribute("innerHTML") - """TBD""" assert logo == True @@ -189,156 +163,25 @@ async def test_open_url_login(app, browser, url, params, redirected_url, form_ac form_action = form_action.replace('{{BASE_URL}}', url_escape(app.base_url)) form = browser.find_element(*LoginPageLocators.FORM_LOGIN).get_attribute('action') - print("Link in form: " + form) # verify title / url assert browser.title == LoginPageLocators.PAGE_TITLE assert form.endswith(form_action) - print("Current Link in form: " + browser.current_url) # login in with params await login(browser, user='test_user', pass_w='test_user') # verify next url + params user = 'test_user' next_url = browser.current_url if url_escape(app.base_url) in form_action: - print("Current Link 2: " + browser.current_url) assert next_url.endswith("param=value") elif "next=%2Fhub" in form_action: assert next_url.endswith("spawn?param=value") assert f"user/{user}/" not in next_url - print("Current Link 2: " + browser.current_url) else: if next_url.endswith(f"/user/{user}/") == False: webdriver_wait( browser, EC.url_to_be(ujoin(public_url(app), f"/user/{user}/")) ) assert next_url.endswith(f"/user/{user}/") - print("Current Link 2: " + browser.current_url) - - -"""TBD""" - - -@pytest.mark.parametrize( - 'running, next_url, location, params', - [ - # default URL if next not specified, for both running and not - (True, '', '', None), - (False, '', '', None), - # next_url is respected - (False, '/hub/admin', '/hub/admin', None), - (False, '/user/other', '/hub/user/other', None), - (False, '/absolute', '/absolute', None), - (False, '/has?query#andhash', '/has?query#andhash', None), - # :// in query string or fragment - (False, '/has?repo=https/host.git', '/has?repo=https/host.git', None), - (False, '/has?repo=https://host.git', '/has?repo=https://host.git', None), - (False, '/has#repo=https://host.git', '/has#repo=https://host.git', None), - # next_url outside is not allowed - (False, 'relative/path', '', None), - (False, 'https://other.domain', '', None), - (False, 'ftp://other.domain', '', None), - (False, '//other.domain', '', None), - (False, '///other.domain/triple', '', None), - (False, '\\\\other.domain/backslashes', '', None), - # params are handled correctly (ignored if ?next= specified) - ( - True, - '/hub/admin?left=1&right=2', - 'hub/admin?left=1&right=2', - {"left": "abc"}, - ), - (False, '/hub/admin', 'hub/admin', [('left', 1), ('right', 2)]), - (True, '', '', {"keep": "yes"}), - (False, '', '', {"keep": "yes"}), - ], -) -async def test_login_redirect(app, browser, running, next_url, location, params, user): - - if location: - location = ujoin(app.base_url, location) - print("if location is:" + location) - elif running: - # location not specified, - location = user.url - if params: - location = url_concat(location, params) - else: - # use default url - location = ujoin(app.base_url, 'hub/spawn') - if params: - location = url_concat(location, params) - - url = 'login' - if params: - url = url_concat(url, params) - if next_url: - if next_url.startswith('/') and not ( - next_url.startswith("//") or urlparse(next_url).netloc - ): - next_url = ujoin(app.base_url, next_url, '') - url = url_concat(url, dict(next=next_url)) - - if running and not user.active: - # ensure running - await user.spawn() - elif user.active and not running: - # ensure not running - await user.stop() - time.sleep(5) - # open_url(app,browser) - await in_thread(browser.get, (ujoin(public_url(app), url))) - await login(browser, user='test_user', pass_w='test_user') - - # await get_page(url, app, allow_redirects=False) - time.sleep(50) - - if user != 'admin' and next_url == '/hub/admin': # user!=user.admin - assert browser.current_url.endswith('hub/admin/') - elm = browser.find_element( - LoginPageLocators.ERROR_403 - ) # .get_attribute('action') - print(elm) - assert elm == LoginPageLocators.ERROR_MESSAGES_403 - if params: - assert browser.current_url.endswith('hub/admin/') - elm = browser.find_element( - *LoginPageLocators.ERROR_403 - ) # .get_attribute('action') - print(elm) - assert elm == LoginPageLocators.ERROR_MESSAGES_403 - - elif next_url == '/has?repo=https': - # print(app.users['test_user'].keys()) - print("else if") - else: - print("else") - - """ -
- - -
- 0. 1., 9-14, 17-18 /space word/user/test_user/ - 1. /space word/user/test_user/ - 2. - 15,16 - space word/hub/admin

- 403 : Forbidden -

:

-

- Action is not authorized with current scopes; requires any of [admin-ui] -

- - 3. space word/hub/user/other

- 404 : Not Found -

Jupyter has lots of moons, but this is not one...

- 4. - 8. the same as 3 - """ - - """ - print("url ---" +browser.current_url) - - #print("user ()" +f"{user}") - print("user (type)" +str(user)) - assert location in browser.current_url """ @pytest.mark.parametrize( @@ -351,69 +194,33 @@ async def test_login_redirect(app, browser, running, next_url, location, params, ], ) async def test_invalid_credantials(app, browser, user, pass_w): - await open_url(app, browser) await login(browser, user, pass_w) - time.sleep(5) - error = browser.find_element(*LoginPageLocators.ERROR_INVALID_CREDANTIALS).text - - # verify error message and url - assert LoginPageLocators.ERROR_MESSAGES_LOGIN == error - assert 'hub/login' in browser.current_url - - """"on page http://localhost:8000/user/username/lab - check only page title for now""" - - -async def test_logout(app, browser): - if is_displayed(browser, HomePageLocators.BUTTON_LOGOUT) == True: - click(browser, HomePageLocators.BUTTON_LOGOUT) - else: - print("TBD") - await async_requests.get(public_host(app)) + try: + error = browser.find_element(*LoginPageLocators.ERROR_INVALID_CREDANTIALS) + await webdriver_wait(browser, EC.visibility_of(error)) + except NoSuchElementException: + error = None + # verify error message and url still eguals to the login page + assert LoginPageLocators.ERROR_MESSAGES_LOGIN == error.text assert 'hub/login' in browser.current_url # HOME PAGE - - -def navigate_bar(): - print("TBD") +async def open_home_page(app, browser, user="test_user", pass_w="test_user"): + url = url_path_join(public_host(app), app.hub.base_url, "/login?next=/hub/home") + await in_thread(browser.get, url) + redirected_url = url_path_join(public_host(app), app.base_url, '/hub/home') + await login(browser, user, pass_w) + await in_thread(browser.get, redirected_url) # TOKEN PAGE -@pytest.mark.parametrize( - "user, pass_w", - [ - ("user", "user"), - ], -) -async def test_elements_of_token_page(app, browser, user, pass_w): - await open_url(app, browser) +async def open_token_page(app, browser, user="test_user", pass_w="test_user"): + + url = url_path_join(public_host(app), app.hub.base_url, "/login?next=/hub/token") + await in_thread(browser.get, url) + redirected_url = url_path_join(public_host(app), app.base_url, '/hub/token') await login(browser, user, pass_w) - - cookies = await app.login_user(user) - await get_page("token", app, cookies=cookies) - # await get_page("token", app) - - buttonAPI = is_displayed(browser, TokenPageLocators.BUTTON_API_REQ) - token_note = is_displayed(browser, TokenPageLocators.INPUT_TOKEN) - token_dropdown = is_displayed(browser, TokenPageLocators.LIST_EXP_TOKEN_FIELD) - - options = browser.find_elements(TokenPageLocators.LIST_EXP_TOKEN_OPT) - for option in options: - is_selected = option.is_selected() - print(str(is_selected)) - - """TBD""" - assert buttonAPI == True - assert token_note == True - assert token_dropdown == True - assert str(is_selected) == "Never" - - -async def test_request_token(app, browser, user, pass_w): - await login(app, browser, user, pass_w) - cookies = await app.login_user(user) - await get_page("token", app, cookies=cookies) + await in_thread(browser.get, redirected_url) From 08b71e7f562c9314e6b8c15892eefcee59199044 Mon Sep 17 00:00:00 2001 From: mouse1203 Date: Mon, 19 Sep 2022 16:28:33 +0200 Subject: [PATCH 008/214] adding asyncio.sleep into test_invalid_credantials added asyncio.sleep into test_invalid_credantials --- jupyterhub/tests/selenium/test_browser.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/jupyterhub/tests/selenium/test_browser.py b/jupyterhub/tests/selenium/test_browser.py index 8b0a04e9..67c8a4cc 100644 --- a/jupyterhub/tests/selenium/test_browser.py +++ b/jupyterhub/tests/selenium/test_browser.py @@ -196,6 +196,8 @@ async def test_open_url_login(app, browser, url, params, redirected_url, form_ac async def test_invalid_credantials(app, browser, user, pass_w): await open_url(app, browser) await login(browser, user, pass_w) + await asyncio.sleep(0.1) + """adding for a catching of the reflected error""" try: error = browser.find_element(*LoginPageLocators.ERROR_INVALID_CREDANTIALS) await webdriver_wait(browser, EC.visibility_of(error)) From f0c825cc1ebb81d6b1713ead5b337479277550fc Mon Sep 17 00:00:00 2001 From: Shloka Date: Sat, 8 Oct 2022 22:40:22 +0530 Subject: [PATCH 009/214] Modification in community channels --- docs/source/contributing/community.rst | 26 +++++++++----------------- 1 file changed, 9 insertions(+), 17 deletions(-) diff --git a/docs/source/contributing/community.rst b/docs/source/contributing/community.rst index 359d2472..638b3667 100644 --- a/docs/source/contributing/community.rst +++ b/docs/source/contributing/community.rst @@ -4,27 +4,19 @@ Community communication channels ================================ -We use `Discourse ` for online discussion. -Everyone in the Jupyter community is welcome to bring ideas and questions there. -In addition, we use `Gitter `_ for online, real-time text chat, -a place for more ephemeral discussions. -The primary Gitter channel for JupyterHub is `jupyterhub/jupyterhub `_. -Gitter isn't archived or searchable, so we recommend going to discourse first -to make sure that discussions are most useful and accessible to the community. -Remember that our community is distributed across the world in various -timezones, so be patient if you do not get an answer immediately! +There is a place for everyone in the Jupyter community to share thoughts and ask questions. We use `Discourse `_ , `Gitter `_ , and GitHub issues for online discussion. -GitHub issues are used for most long-form project discussions, bug reports -and feature requests. Issues related to a specific authenticator or -spawner should be directed to the appropriate repository for the -authenticator or spawner. If you are using a specific JupyterHub +JupyterHub's main Gitter channel is `jupyterhub/jupyterhub `_. +However, to ensure that discussions are the most useful and accessible to the community, we advise using Discourse first as Gitter doesn't have an archive or search function +Please be patient if you do not recieve a response right away; keep in mind that our comunity is spread out over the globe in different timezones. + +Most lengthy project conversations, bug reports, and feature requests take place via GitHub issues. If you have an issue with a certain authenticator or spawner, you should report them to the relevant repository. If you are using a specific JupyterHub distribution (such as `Zero to JupyterHub on Kubernetes `_ or `The Littlest JupyterHub `_), -you should open issues directly in their repository. If you can not +you should open issues directly in the respective repository. If you can not find a repository to open your issue in, do not worry! Create it in the `main JupyterHub repository `_ and our community will help you figure it out. -A `mailing list `_ for all -of Project Jupyter exists, along with one for `teaching with Jupyter -`_. +There is a `mailing list `_ for `teaching with Jupyter +`_ as well as one for the entire of Project Jupyter. \ No newline at end of file From 204bfaf8f4521a8ac3fd1f08ce59e0364bd40788 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sat, 8 Oct 2022 17:13:40 +0000 Subject: [PATCH 010/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/contributing/community.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/community.rst b/docs/source/contributing/community.rst index 638b3667..87a615bb 100644 --- a/docs/source/contributing/community.rst +++ b/docs/source/contributing/community.rst @@ -19,4 +19,4 @@ JupyterHub repository `_ and our community will help you figure it out. There is a `mailing list `_ for `teaching with Jupyter -`_ as well as one for the entire of Project Jupyter. \ No newline at end of file +`_ as well as one for the entire of Project Jupyter. From 395b1a56812d0eb4e5b53be4815ba3687fc849f5 Mon Sep 17 00:00:00 2001 From: Shloka Date: Sun, 9 Oct 2022 19:07:20 +0530 Subject: [PATCH 011/214] Modifications is testing docs --- docs/source/contributing/tests.rst | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/docs/source/contributing/tests.rst b/docs/source/contributing/tests.rst index 044aa7ca..8fbf17aa 100644 --- a/docs/source/contributing/tests.rst +++ b/docs/source/contributing/tests.rst @@ -4,19 +4,16 @@ Testing JupyterHub and linting code =================================== -Unit test help validate that JupyterHub works the way we think it does, -and continues to do so when changes occur. They also help communicate -precisely what we expect our code to do. +Unit tests help confirming that JupyterHub works as intended and does so even when modifications are made. Additionally, they help in clarifying our expectations for our code. -JupyterHub uses `pytest `_ for all our tests. You -can find them under ``jupyterhub/tests`` directory in the git repository. +JupyterHub uses `pytest `_ for all the tests. You +can find them under the `jupyterhub/tests `_ directory in the git repository. Running the tests ================== -#. Make sure you have completed :ref:`contributing/setup`. You should be able - to start ``jupyterhub`` from the commandline & access it from your - web browser. This ensures that the dev environment is properly set +#. Make sure you have completed :ref:`contributing/setup`. Once completed, you should be able + to start ``jupyterhub`` through a web browser as well as the command line. By doing this, it is ensured that the dev environment is properly set up for tests to run. #. You can run all tests in JupyterHub @@ -57,7 +54,7 @@ Running the tests pytest -v jupyterhub/tests/test_api.py::test_shutdown - See the `pytest usage documentation `_ for more details. + For more information, refer to `pytest usage documentation `_. Test organisation ================= @@ -98,8 +95,7 @@ And fixtures to add functionality or spawning behavior: - ``bad_spawn``: enables the BadSpawner (a spawner that fails immediately) - ``slow_bad_spawn``: enables the SlowBadSpawner (a spawner that fails after a short delay) -See the `pytest fixtures documentation `_ -for how to use the existing fixtures, and how to create new ones. +For information on using the existing fixtures and creating new ones, refer to `pytest fixtures documentation `_ Troubleshooting Test Failures @@ -109,7 +105,7 @@ All the tests are failing ------------------------- Make sure you have completed all the steps in :ref:`contributing/setup` successfully, and -can launch ``jupyterhub`` from the terminal. +can launch ``jupyterhub`` from the terminal as well as the web browser. Code formatting and linting @@ -117,13 +113,13 @@ Code formatting and linting JupyterHub has adopted automatic code formatting and linting. As long as your code is valid, the pre-commit hook should take care of how it should look. -You can invoke the pre-commit hook by hand at any time with: +You can invoke the pre-commit hook manually at any time with: .. code:: bash pre-commit run -which should run any autoformatting on your code and tell you about any errors it couldn't fix automatically. +This should run any auto formatting on your code and tell you about any errors it couldn't fix automatically. You may also install `black integration `_ into your text editor to format code automatically. From 3425269cb2a67d59eaf9fd8ae9e1abb4cdce5feb Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Mon, 10 Oct 2022 12:34:09 +0500 Subject: [PATCH 012/214] Update index.rst --- docs/source/index.rst | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index a7b2c0c8..23b93a54 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -2,20 +2,17 @@ JupyterHub ========== -`JupyterHub`_ is the best way to serve `Jupyter notebook`_ for multiple users. +`JupyterHub`_ is the best way to serve `Jupyter notebook`_ for multiple users. Because of it's distributed nature, It can be used in a class of students, a corporate data science group or scientific research group. It is a multi-user **Hub** that spawns, manages, and proxies multiple instances of the single-user `Jupyter notebook`_ server. -To make life easier, JupyterHub has distributions. Be sure to +JupyterHub offers distributions for different use cases. Be sure to take a look at them before continuing with the configuration of the broad -original system of `JupyterHub`_. Today, you can find two main cases: +original system of `JupyterHub`_. As of now, you can find two main cases: -1. If you need a simple case for a small amount of users (0-100) and single server - take a look at - `The Littlest JupyterHub `__ distribution. -2. If you need to allow for even more users, a dynamic amount of servers can be used on a cloud, - take a look at the `Zero to JupyterHub with Kubernetes `__ . +1. `The Littlest JupyterHub `__ distribution is suitable if you need small amount of users (0-100) and single server with a simple environment. +2. `Zero to JupyterHub with Kubernetes `__ allows you to deploy dynamic servers on cloud if you need even more users. Four subsystems make up JupyterHub: @@ -26,7 +23,7 @@ Four subsystems make up JupyterHub: * an **authentication class** that manages how users can access the system -Besides these central pieces, you can add optional configurations through a `config.py` file and manage users kernels on an admin panel. A simplification of the whole system can be seen in the figure below: +Besides these central pieces, you can add optional configurations through a `config.py` file and manage users' kernels through admin panel. A simplification of the whole system can be seen in the figure below: .. image:: images/jhub-fluxogram.jpeg :alt: JupyterHub subsystems From 915fab2d26094d15ad679ecf4c3a891fc4d1d109 Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Mon, 10 Oct 2022 12:55:16 +0500 Subject: [PATCH 013/214] Update index.rst --- docs/source/index.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index 23b93a54..9b8c56b2 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -3,7 +3,7 @@ JupyterHub ========== `JupyterHub`_ is the best way to serve `Jupyter notebook`_ for multiple users. Because of it's distributed nature, -It can be used in a class of students, a corporate data science group or scientific +it can be used in a class of students, a corporate data science group, or a scientific research group. It is a multi-user **Hub** that spawns, manages, and proxies multiple instances of the single-user `Jupyter notebook`_ server. @@ -11,14 +11,14 @@ JupyterHub offers distributions for different use cases. Be sure to take a look at them before continuing with the configuration of the broad original system of `JupyterHub`_. As of now, you can find two main cases: -1. `The Littlest JupyterHub `__ distribution is suitable if you need small amount of users (0-100) and single server with a simple environment. -2. `Zero to JupyterHub with Kubernetes `__ allows you to deploy dynamic servers on cloud if you need even more users. +1. `The Littlest JupyterHub `__ distribution is suitable if you need a small number of users (0-100) and a single server with a simple environment. +2. `Zero to JupyterHub with Kubernetes `__ allows you to deploy dynamic servers on the cloud if you need even more users. Four subsystems make up JupyterHub: * a **Hub** (tornado process) that is the heart of JupyterHub -* a **configurable http proxy** (node-http-proxy) that receives the requests from the client's browser +* a **configurable HTTP proxy** (node-http-proxy) that receives the requests from the client's browser * multiple **single-user Jupyter notebook servers** (Python/IPython/tornado) that are monitored by Spawners * an **authentication class** that manages how users can access the system From aa101a7aff888bffe52778e95aca29345bbd7c68 Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Mon, 10 Oct 2022 15:36:10 +0500 Subject: [PATCH 014/214] Update index.rst --- docs/source/index.rst | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index 9b8c56b2..96b56485 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -2,8 +2,8 @@ JupyterHub ========== -`JupyterHub`_ is the best way to serve `Jupyter notebook`_ for multiple users. Because of it's distributed nature, -it can be used in a class of students, a corporate data science group, or a scientific +`JupyterHub`_ is the best way to serve `Jupyter notebook`_ for multiple users. The distributed Jupyter Notebook environment allows +it to be used in a class of students, a corporate data science group, or a scientific research group. It is a multi-user **Hub** that spawns, manages, and proxies multiple instances of the single-user `Jupyter notebook`_ server. @@ -11,7 +11,7 @@ JupyterHub offers distributions for different use cases. Be sure to take a look at them before continuing with the configuration of the broad original system of `JupyterHub`_. As of now, you can find two main cases: -1. `The Littlest JupyterHub `__ distribution is suitable if you need a small number of users (0-100) and a single server with a simple environment. +1. `The Littlest JupyterHub `__ distribution is suitable if you need a small number of users (1-100) and a single server with a simple environment. 2. `Zero to JupyterHub with Kubernetes `__ allows you to deploy dynamic servers on the cloud if you need even more users. @@ -23,7 +23,7 @@ Four subsystems make up JupyterHub: * an **authentication class** that manages how users can access the system -Besides these central pieces, you can add optional configurations through a `config.py` file and manage users' kernels through admin panel. A simplification of the whole system can be seen in the figure below: +Besides these central pieces, you can add optional configurations through a `config.py` file and manage users' environments through an admin panel. A simplification of the whole system can be seen in the figure below: .. image:: images/jhub-fluxogram.jpeg :alt: JupyterHub subsystems @@ -53,17 +53,17 @@ Contents Distributions ------------- -A JupyterHub **distribution** is tailored towards a particular set of +Each JupyterHub **distribution** is tailored toward a particular set of use cases. These are generally easier to set up than setting up JupyterHub from scratch, assuming they fit your use case. The two popular ones are: -* `Zero to JupyterHub on Kubernetes `_, for - running JupyterHub on top of `Kubernetes `_. This - can scale to large number of machines & users. * `The Littlest JupyterHub `_, for an easy to set up & run JupyterHub supporting 1-100 users on a single machine. +* `Zero to JupyterHub on Kubernetes `_, for + running JupyterHub on top of `Kubernetes `_. This + can scale to a large number of machines & users. Installation Guide ------------------ @@ -116,8 +116,8 @@ RBAC Reference Contributing ------------ -We want you to contribute to JupyterHub in ways that are most exciting -& useful to you. We value documentation, testing, bug reporting & code equally, +We welcome you to contribute to JupyterHub in ways that are most exciting +& useful to you. We value documentation, testing, bug reporting & code equally and are glad to have your contributions in whatever form you wish :) Our `Code of Conduct `_ From b87b8c52d35a43d3852673106b8c6278a112d991 Mon Sep 17 00:00:00 2001 From: Arafat Abdussalam Date: Tue, 11 Oct 2022 07:21:19 +0100 Subject: [PATCH 015/214] updated websecirity.md I have the grammatical errors written on the page such as improving the abbreviated words with apostrophes and other typos --- docs/source/reference/websecurity.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/source/reference/websecurity.md b/docs/source/reference/websecurity.md index 8473c3e9..d9a9a397 100644 --- a/docs/source/reference/websecurity.md +++ b/docs/source/reference/websecurity.md @@ -5,18 +5,18 @@ The **Security Overview** section helps you learn about: - the design of JupyterHub with respect to web security - the semi-trusted user - the available mitigations to protect untrusted users from each other -- the value of periodic security audits. +- the value of periodic security audits This overview also helps you obtain a deeper understanding of how JupyterHub works. -## Semi-trusted and untrusted users +## Semi-trusted and Untrusted Users JupyterHub is designed to be a _simple multi-user server for modestly sized groups_ of **semi-trusted** users. While the design reflects serving semi-trusted users, JupyterHub is not necessarily unsuitable for serving **untrusted** users. -Using JupyterHub with **untrusted** users does mean more work by the +Using JupyterHub with **untrusted** users entails more work by the administrator. Much care is required to secure a Hub, with extra caution on protecting users from each other as the Hub is serving untrusted users. @@ -32,7 +32,7 @@ servers) as a single website (i.e. single domain). To protect users from each other, a user must **never** be able to write arbitrary HTML and serve it to another user on the Hub's domain. JupyterHub's -authentication setup prevents a user writing arbitrary HTML and serving it to +authentication setup prevents a user from writing arbitrary HTML and serving it to another user because only the owner of a given single-user notebook server is allowed to view user-authored pages served by the given single-user notebook server. @@ -41,7 +41,7 @@ To protect all users from each other, JupyterHub administrators must ensure that: - A user **does not have permission** to modify their single-user notebook server, - including: + as well as: - A user **may not** install new packages in the Python environment that runs their single-user server. - If the `PATH` is used to resolve the single-user executable (instead of @@ -101,8 +101,8 @@ pose additional risk to the web application's security. ### Encrypt internal connections with SSL/TLS -By default, all communication on the server, between the proxy, hub, and single --user notebooks is performed unencrypted. Setting the `internal_ssl` flag in +By default, all communications on the server, between the proxy, hub, and single +-user notebooks are performed unencrypted. Setting the `internal_ssl` flag in `jupyterhub_config.py` secures the aforementioned routes. Turning this feature on does require that the enabled `Spawner` can use the certificates generated by the `Hub` (the default `LocalProcessSpawner` can, for instance). @@ -118,7 +118,7 @@ extend to securing the `tcp` sockets as well. ## Security audits -We recommend that you do periodic reviews of your deployment's security. It's +We recommend that you do periodic reviews of your deployment's security. It is good practice to keep JupyterHub, configurable-http-proxy, and nodejs versions up to date. @@ -129,7 +129,7 @@ A handy website for testing your deployment is ## Vulnerability reporting -If you believe you’ve found a security vulnerability in JupyterHub, or any +If you believe you have found a security vulnerability in JupyterHub, or any Jupyter project, please report it to [security@ipython.org](mailto:security@ipython.org). If you prefer to encrypt your security reports, you can use [this PGP public From f2059c0d039f571480b2974c6d17e32b5800389a Mon Sep 17 00:00:00 2001 From: Goodness Chris-Ugari Date: Tue, 11 Oct 2022 09:37:54 +0100 Subject: [PATCH 016/214] Update troubleshooting.md This PR improves the troubleshooting doc and is part of [issue 41](https://github.com/jupyterhub/outreachy/issues/41) --- docs/source/troubleshooting.md | 171 +++++++++++++++++---------------- 1 file changed, 88 insertions(+), 83 deletions(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index 0cecb6d9..b3ac424c 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -1,32 +1,37 @@ # Troubleshooting When troubleshooting, you may see unexpected behaviors or receive an error -message. This section provide links for identifying the cause of the +message. This section provides links for identifying the cause of the problem and how to resolve it. [_Behavior_](#behavior) -- JupyterHub proxy fails to start -- sudospawner fails to run -- What is the default behavior when none of the lists (admin, allowed, - allowed groups) are set? -- JupyterHub Docker container not accessible at localhost +- [JupyterHub proxy fails to start](#jupyterhub-proxy-fails-to-start) +- [sudospawner fails to run](#sudospawner-fails-to-run) +- [What is the default behavior when none of the lists (admin, allowed, + allowed groups) are set?](#what-is-the-default-behavior-when-none-of-the-lists-admin-allowed-allowed-groups-are-set) +- [JupyterHub Docker container not accessible at localhost](#jupyterhub-docker-container-not-accessible-at-localhost) +- [How can I kill ports from JupyterHub-managed services that have been orphaned?](#how-can-i-kill-ports-from-jupyterhub-managed-services-that-have-been-orphaned) +- [Why am I getting a Spawn failed error message?](#why-am-i-getting-a-spawn-failed-error-message) +- [How can I run JupyterHub with sudo but use my current env vars and virtualenv location?](#how-can-i-run-jupyterhub-with-sudo-but-use-my-current-env-vars-and-virtualenv-location) [_Errors_](#errors) -- 500 error after spawning my single-user server +- [Error 500 after spawning my single-user server](#error-500-after-spawning-my-single-user-server) +- [Launching Jupyter Notebooks to run as an externally managed JupyterHub service with the `jupyterhub-singleuser` command returns a `JUPYTERHUB_API_TOKEN` error](#launching-jupyter-notebooks-to-run-as-an-externally-managed-jupyterhub-service-with-the-jupyterhub-singleuser-command-returns-a-jupyterhub-api-token-error) [_How do I...?_](#how-do-i) -- Use a chained SSL certificate -- Install JupyterHub without a network connection -- I want access to the whole filesystem, but still default users to their home directory -- How do I increase the number of pySpark executors on YARN? -- How do I use JupyterLab's prerelease version with JupyterHub? -- How do I set up JupyterHub for a workshop (when users are not known ahead of time)? -- How do I set up rotating daily logs? -- Toree integration with HDFS rack awareness script -- Where do I find Docker images and Dockerfiles related to JupyterHub? +- [Use a chained SSL certificate](#use-a-chained-ssl-certificate) +- [Install JupyterHub without a network connection](#install-jupyterhub-without-a-network-connection) +- [I want access to the whole filesystem and still default users to their home directory](#i-want-access-to-the-whole-filesystem-and-still-default-users-to-their-home-directory) +- [How do I increase the number of pySpark executors on YARN?](#how-do-i-increase-the-number-of-pyspark-executors-on-yarn) +- [How do I use JupyterLab's prerelease version with JupyterHub?](#how-do-i-use-jupyterlab-s-prerelease-version-with-jupyterhub) +- [How do I set up JupyterHub for a workshop (when users are not known ahead of time)?](#how-do-i-set-up-jupyterhub-for-a-workshop-when-users-are-not-known-ahead-of-time) +- [How do I set up rotating daily logs?](#how-do-i-set-up-rotating-daily-logs) +- [Toree integration with HDFS rack awareness script](#toree-integration-with-hdfs-rack-awareness-script) +- [Where do I find Docker images and Dockerfiles related to JupyterHub?](#where-do-i-find-docker-images-and-dockerfiles-related-to-jupyterhub) +- [How can I view the logs for JupyterHub or the user's Notebook servers when using the DockerSpawner?](#how-can-i-view-the-logs-for-jupyterhub-or-the-user-s-notebook-servers-when-using-the-dockerspawner) [_Troubleshooting commands_](#troubleshooting-commands) @@ -40,9 +45,9 @@ If you have tried to start the JupyterHub proxy and it fails to start: `c.JupyterHub.ip = '*'`; if it is, try `c.JupyterHub.ip = ''` - Try starting with `jupyterhub --ip=0.0.0.0` -**Note**: If this occurs on Ubuntu/Debian, check that the you are using a -recent version of node. Some versions of Ubuntu/Debian come with a version -of node that is very old, and it is necessary to update node. +**Note**: If this occurs on Ubuntu/Debian, check that you are using a +recent version of [Node](https://nodejs.org). Some versions of Ubuntu/Debian come with a version +of Node that is very old, and it is necessary to update Node. ### sudospawner fails to run @@ -61,24 +66,24 @@ to the config file, `jupyterhub_config.py`. ### What is the default behavior when none of the lists (admin, allowed, allowed groups) are set? When nothing is given for these lists, there will be no admins, and all users -who can authenticate on the system (i.e. all the unix users on the server with +who can authenticate on the system (i.e. all the Unix users on the server with a password) will be allowed to start a server. The allowed username set lets you limit this to a particular set of users, and admin_users lets you specify who among them may use the admin interface (not necessary, unless you need to do -things like inspect other users' servers, or modify the user list at runtime). +things like inspect other users' servers or modify the user list at runtime). ### JupyterHub Docker container not accessible at localhost Even though the command to start your Docker container exposes port 8000 (`docker run -p 8000:8000 -d --name jupyterhub jupyterhub/jupyterhub jupyterhub`), -it is possible that the IP address itself is not accessible/visible. As a result +it is possible that the IP address itself is not accessible/visible. As a result, when you try http://localhost:8000 in your browser, you are unable to connect even though the container is running properly. One workaround is to explicitly tell Jupyterhub to start at `0.0.0.0` which is visible to everyone. Try this command: `docker run -p 8000:8000 -d --name jupyterhub jupyterhub/jupyterhub jupyterhub --ip 0.0.0.0 --port 8000` -### How can I kill ports from JupyterHub managed services that have been orphaned? +### How can I kill ports from JupyterHub-managed services that have been orphaned? I started JupyterHub + nbgrader on the same host without containers. When I try to restart JupyterHub + nbgrader with this configuration, errors appear that the service accounts cannot start because the ports are being used. @@ -92,7 +97,7 @@ Where `` is the port used by the nbgrader course service. This con ### Why am I getting a Spawn failed error message? -After successfully logging in to JupyterHub with a compatible authenticators, I get a 'Spawn failed' error message in the browser. The JupyterHub logs have `jupyterhub KeyError: "getpwnam(): name not found: `. +After successfully logging in to JupyterHub with a compatible authenticator, I get a 'Spawn failed' error message in the browser. The JupyterHub logs have `jupyterhub KeyError: "getpwnam(): name not found: `. This issue occurs when the authenticator requires a local system user to exist. In these cases, you need to use a spawner that does not require an existing system user account, such as `DockerSpawner` or `KubeSpawner`. @@ -109,23 +114,9 @@ sudo MY_ENV=abc123 \ /srv/jupyterhub/jupyterhub ``` -### How can I view the logs for JupyterHub or the user's Notebook servers when using the DockerSpawner? - -Use `docker logs ` where `` is the container name defined within `docker-compose.yml`. For example, to view the logs of the JupyterHub container use: - - docker logs hub - -By default, the user's notebook server is named `jupyter-` where `username` is the user's username within JupyterHub's db. So if you wanted to see the logs for user `foo` you would use: - - docker logs jupyter-foo - -You can also tail logs to view them in real time using the `-f` option: - - docker logs -f hub - ## Errors -### 500 error after spawning my single-user server +### Error 500 after spawning my single-user server You receive a 500 error when accessing the URL `/user//...`. This is often seen when your single-user server cannot verify your user cookie @@ -185,10 +176,10 @@ If you receive a 403 error, the API token for the single-user server is likely invalid. Commonly, the 403 error is caused by resetting the JupyterHub database (either removing jupyterhub.sqlite or some other action) while leaving single-user servers running. This happens most frequently when using -DockerSpawner, because Docker's default behavior is to stop/start containers -which resets the JupyterHub database, rather than destroying and recreating +DockerSpawner because Docker's default behavior is to stop/start containers +that reset the JupyterHub database, rather than destroying and recreating the container every time. This means that the same API token is used by the -server for its whole life, until the container is rebuilt. +server for its whole life until the container is rebuilt. The fix for this Docker case is to remove any Docker containers seeing this issue (typically all containers created before a certain point in time): @@ -201,14 +192,14 @@ your server again. ##### Proxy settings (403 GET) -When your whole JupyterHub sits behind a organization proxy (_not_ a reverse proxy like NGINX as part of your setup and _not_ the configurable-http-proxy) the environment variables `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy` and `https_proxy` might be set. This confuses the jupyterhub-singleuser servers: When connecting to the Hub for authorization they connect via the proxy instead of directly connecting to the Hub on localhost. The proxy might deny the request (403 GET). This results in the singleuser server thinking it has a wrong auth token. To circumvent this you should add `,,localhost,127.0.0.1` to the environment variables `NO_PROXY` and `no_proxy`. +When your whole JupyterHub sits behind an organization proxy (_not_ a reverse proxy like NGINX as part of your setup and _not_ the configurable-http-proxy) the environment variables `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy`, and `https_proxy` might be set. This confuses the Jupyterhub single-user servers: When connecting to the Hub for authorization they connect via the proxy instead of directly connecting to the Hub on localhost. The proxy might deny the request (403 GET). This results in the single-user server thinking it has the wrong auth token. To circumvent this you should add `,,localhost,127.0.0.1` to the environment variables `NO_PROXY` and `no_proxy`. ### Launching Jupyter Notebooks to run as an externally managed JupyterHub service with the `jupyterhub-singleuser` command returns a `JUPYTERHUB_API_TOKEN` error [JupyterHub services](https://jupyterhub.readthedocs.io/en/stable/reference/services.html) allow processes to interact with JupyterHub's REST API. Example use-cases include: - **Secure Testing**: provide a canonical Jupyter Notebook for testing production data to reduce the number of entry points into production systems. -- **Grading Assignments**: provide access to shared Jupyter Notebooks that may be used for management tasks such grading assignments. +- **Grading Assignments**: provide access to shared Jupyter Notebooks that may be used for management tasks such as grading assignments. - **Private Dashboards**: share dashboards with certain group members. If possible, try to run the Jupyter Notebook as an externally managed service with one of the provided [jupyter/docker-stacks](https://github.com/jupyter/docker-stacks). @@ -222,7 +213,7 @@ If you launch a Jupyter Notebook with the `jupyterhub-singleuser` command direct Did you launch it manually? ``` -If you plan on testing `jupyterhub-singleuser` independently from JupyterHub, then you can set the api token environment variable. For example, if were to run the single-user Jupyter Notebook on the host, then: +If you plan on testing `jupyterhub-singleuser` independently from JupyterHub, then you can set the API token environment variable. For example, if you were to run the single-user Jupyter Notebook on the host, then: export JUPYTERHUB_API_TOKEN=my_secret_token jupyterhub-singleuser @@ -256,7 +247,7 @@ You would then set in your `jupyterhub_config.py` file the `ssl_key` and #### Example Your certificate provider gives you the following files: `example_host.crt`, -`Entrust_L1Kroot.txt` and `Entrust_Root.txt`. +`Entrust_L1Kroot.txt`, and `Entrust_Root.txt`. Concatenate the files appending the chain cert and root cert to your host cert: @@ -289,7 +280,7 @@ with npmbox: python3 -m pip wheel jupyterhub npmbox configurable-http-proxy -### I want access to the whole filesystem, but still default users to their home directory +### I want access to the whole filesystem and still default users to their home directory Setting the following in `jupyterhub_config.py` will configure access to the entire filesystem and set the default to the user's home directory. @@ -321,7 +312,7 @@ For instance: python3 -m pip install jupyterlab jupyter serverextension enable --py jupyterlab --sys-prefix -The important thing is that jupyterlab is installed and enabled in the +The important thing is that Jupyterlab is installed and enabled in the single-user notebook server environment. For system users, this means system-wide, as indicated above. For Docker containers, it means inside the single-user docker image, etc. @@ -334,14 +325,14 @@ notebook servers to default to JupyterLab: ### How do I set up JupyterHub for a workshop (when users are not known ahead of time)? 1. Set up JupyterHub using OAuthenticator for GitHub authentication -2. Configure admin list to have workshop leaders be listed with administrator privileges. +2. Configure the admin list to have workshop leaders be listed with administrator privileges. -Users will need a GitHub account to login and be authenticated by the Hub. +Users will need a GitHub account to log in and be authenticated by the Hub. ### How do I set up rotating daily logs? You can do this with [logrotate](https://linux.die.net/man/8/logrotate), -or pipe to `logger` to use syslog instead of directly to a file. +or pipe to `logger` to use Syslog instead of directly to a file. For example, with this logrotate config file: @@ -361,6 +352,52 @@ logrotate /path/to/above-config Or use syslog: jupyterhub | logger -t jupyterhub + +### Toree integration with HDFS rack awareness script + +The Apache Toree kernel will have an issue when running with JupyterHub if the standard HDFS +rack awareness script is used. This will materialize in the logs as a repeated WARN: + +```bash +16/11/29 16:24:20 WARN ScriptBasedMapping: Exception running /etc/hadoop/conf/topology_script.py some.ip.address +ExitCodeException exitCode=1: File "/etc/hadoop/conf/topology_script.py", line 63 + print rack + ^ +SyntaxError: Missing parentheses in call to 'print' + + at `org.apache.hadoop.util.Shell.runCommand(Shell.java:576)` +``` + +In order to resolve this issue, there are two potential options. + +1. Update HDFS core-site.xml, so the parameter "net.topology.script.file.name" points to a custom + script (e.g. /etc/hadoop/conf/custom_topology_script.py). Copy the original script and change the first line point + to a python two installation (e.g. /usr/bin/python). +2. In spark-env.sh add a Python 2 installation to your path (e.g. export PATH=/opt/anaconda2/bin:$PATH). + +### Where do I find Docker images and Dockerfiles related to JupyterHub? + +Docker images can be found at the [JupyterHub organization on DockerHub](https://hub.docker.com/u/jupyterhub/). +The Docker image [jupyterhub/singleuser](https://hub.docker.com/r/jupyterhub/singleuser/) +provides an example single-user notebook server for use with DockerSpawner. + +Additional single-user notebook server images can be found at the [Jupyter +organization on DockerHub](https://hub.docker.com/r/jupyter/) and information +about each image at the [jupyter/docker-stacks repo](https://github.com/jupyter/docker-stacks). + +### How can I view the logs for JupyterHub or the user's Notebook servers when using the DockerSpawner? + +Use `docker logs ` where `` is the container name defined within `docker-compose.yml`. For example, to view the logs of the JupyterHub container use: + + docker logs hub + +By default, the user's notebook server is named `jupyter-` where `username` is the user's username within JupyterHub's db. So if you wanted to see the logs for user `foo` you would use: + + docker logs jupyter-foo + +You can also tail logs to view them in real-time using the `-f` option: + + docker logs -f hub ## Troubleshooting commands @@ -385,35 +422,3 @@ jupyter kernelspec list ```bash jupyterhub --debug ``` - -### Toree integration with HDFS rack awareness script - -The Apache Toree kernel will an issue, when running with JupyterHub, if the standard HDFS -rack awareness script is used. This will materialize in the logs as a repeated WARN: - -```bash -16/11/29 16:24:20 WARN ScriptBasedMapping: Exception running /etc/hadoop/conf/topology_script.py some.ip.address -ExitCodeException exitCode=1: File "/etc/hadoop/conf/topology_script.py", line 63 - print rack - ^ -SyntaxError: Missing parentheses in call to 'print' - - at `org.apache.hadoop.util.Shell.runCommand(Shell.java:576)` -``` - -In order to resolve this issue, there are two potential options. - -1. Update HDFS core-site.xml, so the parameter "net.topology.script.file.name" points to a custom - script (e.g. /etc/hadoop/conf/custom_topology_script.py). Copy the original script and change the first line point - to a python two installation (e.g. /usr/bin/python). -2. In spark-env.sh add a Python 2 installation to your path (e.g. export PATH=/opt/anaconda2/bin:$PATH). - -### Where do I find Docker images and Dockerfiles related to JupyterHub? - -Docker images can be found at the [JupyterHub organization on DockerHub](https://hub.docker.com/u/jupyterhub/). -The Docker image [jupyterhub/singleuser](https://hub.docker.com/r/jupyterhub/singleuser/) -provides an example single user notebook server for use with DockerSpawner. - -Additional single user notebook server images can be found at the [Jupyter -organization on DockerHub](https://hub.docker.com/r/jupyter/) and information -about each image at the [jupyter/docker-stacks repo](https://github.com/jupyter/docker-stacks). From 5d5a424aeac8e99c07e601d6f02c629743c20337 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue, 11 Oct 2022 08:40:39 +0000 Subject: [PATCH 017/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/troubleshooting.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index b3ac424c..5c9bcbcc 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -76,7 +76,7 @@ things like inspect other users' servers or modify the user list at runtime). Even though the command to start your Docker container exposes port 8000 (`docker run -p 8000:8000 -d --name jupyterhub jupyterhub/jupyterhub jupyterhub`), -it is possible that the IP address itself is not accessible/visible. As a result, +it is possible that the IP address itself is not accessible/visible. As a result, when you try http://localhost:8000 in your browser, you are unable to connect even though the container is running properly. One workaround is to explicitly tell Jupyterhub to start at `0.0.0.0` which is visible to everyone. Try this @@ -352,7 +352,7 @@ logrotate /path/to/above-config Or use syslog: jupyterhub | logger -t jupyterhub - + ### Toree integration with HDFS rack awareness script The Apache Toree kernel will have an issue when running with JupyterHub if the standard HDFS @@ -384,7 +384,7 @@ provides an example single-user notebook server for use with DockerSpawner. Additional single-user notebook server images can be found at the [Jupyter organization on DockerHub](https://hub.docker.com/r/jupyter/) and information about each image at the [jupyter/docker-stacks repo](https://github.com/jupyter/docker-stacks). - + ### How can I view the logs for JupyterHub or the user's Notebook servers when using the DockerSpawner? Use `docker logs ` where `` is the container name defined within `docker-compose.yml`. For example, to view the logs of the JupyterHub container use: From 941e8c928aead3f37234199ebca248a0268eb8e1 Mon Sep 17 00:00:00 2001 From: Arafat Abdussalam Date: Tue, 11 Oct 2022 10:14:03 +0100 Subject: [PATCH 018/214] Updated log messages I improved the log message subsection of the documentation by making the page more user friendly --- docs/source/admin/log-messages.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/docs/source/admin/log-messages.md b/docs/source/admin/log-messages.md index fd672c6a..31edb683 100644 --- a/docs/source/admin/log-messages.md +++ b/docs/source/admin/log-messages.md @@ -1,22 +1,20 @@ # Interpreting common log messages -When debugging errors and outages, looking at the logs emitted by -JupyterHub is very helpful. This document tries to document some common -log messages, and what they mean. +In this subsection of the documentation, you will get to understand the meaning of common log messages and how to resolve them. When debugging errors and outages, it is very helpful to look at the logs emitted by JupyterHub. ## Failing suspected API request to not-running server ### Example -Your logs might be littered with lines that might look slightly scary +When your screen displays the log message as shown below, you need not to get scared. ``` [W 2022-03-10 17:25:19.774 JupyterHub base:1349] Failing suspected API request to not-running server: /hub/user//api/metrics/v1 ``` -### Most likely cause +### Cause -This likely means is that the user's server has stopped running but they +This likely cause is that the user's server has stopped running but they still have a browser tab open. For example, you might have 3 tabs open, and shut your server down via one. Or you closed your laptop, your server was culled for inactivity, and then you reopen your laptop again! The From 3ee21cc967d8ef91ffbeffd3fc717705aa9e53e8 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue, 11 Oct 2022 09:17:45 +0000 Subject: [PATCH 019/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/admin/log-messages.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/admin/log-messages.md b/docs/source/admin/log-messages.md index 31edb683..c2974260 100644 --- a/docs/source/admin/log-messages.md +++ b/docs/source/admin/log-messages.md @@ -1,6 +1,6 @@ # Interpreting common log messages -In this subsection of the documentation, you will get to understand the meaning of common log messages and how to resolve them. When debugging errors and outages, it is very helpful to look at the logs emitted by JupyterHub. +In this subsection of the documentation, you will get to understand the meaning of common log messages and how to resolve them. When debugging errors and outages, it is very helpful to look at the logs emitted by JupyterHub. ## Failing suspected API request to not-running server From 15cf30156d7f63408f771854934f6ab1b9d56931 Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Tue, 11 Oct 2022 19:12:03 +0500 Subject: [PATCH 020/214] update upgrading.rst --- docs/source/admin/upgrading.rst | 65 ++++++++++++++++----------------- 1 file changed, 32 insertions(+), 33 deletions(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index efe98c27..96aee1c8 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -5,20 +5,20 @@ Upgrading JupyterHub JupyterHub offers easy upgrade pathways between minor versions. This document describes how to do these upgrades. -If you are using :ref:`a JupyterHub distribution `, you +If you are using [distributions](index/distributions), you should consult the distribution's documentation on how to upgrade. This -document is if you have set up your own JupyterHub without using a +document is useful if you have set up your own JupyterHub without using a distribution. -It is long because is pretty detailed! Most likely, upgrading -JupyterHub is painless, quick and with minimal user interruption. +The steps are discussed in detail. Most likely, +upgrading JupyterHub is painless, quick and with minimal user interruption. Read the Changelog ================== -The `changelog <../changelog.html>`_ contains information on what has -changed with the new JupyterHub release, and any deprecation warnings. -Read these notes to familiarize yourself with the coming changes. There +The `changelog <../changelog.md>`_ contains information on what has +changed with the new JupyterHub release and any deprecation warnings. +Read these notes to familiarize yourself with the upcoming changes. There might be new releases of authenticators & spawners you are using, so read the changelogs for those too! @@ -27,13 +27,13 @@ Notify your users If you are using the default configuration where ``configurable-http-proxy`` is managed by JupyterHub, your users will see service disruption during -the upgrade process. You should notify them, and pick a time to do the +the upgrade process. You will need notify them, and pick a time to do the upgrade where they will be least disrupted. -If you are using a different proxy, or running ``configurable-http-proxy`` -independent of JupyterHub, your users will be able to continue using notebook +If you are using a different proxy or running ``configurable-http-proxy`` +independent of JupyterHub, your users will be able to continue using a notebook servers they had already launched, but will not be able to launch new servers -nor sign in. +or sign in. Backup database & config @@ -41,22 +41,21 @@ Backup database & config Before doing an upgrade, it is critical to back up: -#. Your JupyterHub database (sqlite by default, or MySQL / Postgres - if you used those). If you are using sqlite (the default), you - should backup the ``jupyterhub.sqlite`` file. +#. Your JupyterHub database (SQLite by default, or MySQL / Postgres + if you used those). If you are using SQLite (the default), you + should back up the ``jupyterhub.sqlite`` file. #. Your ``jupyterhub_config.py`` file. -#. Your user's home directories. This is unlikely to be affected directly by - a JupyterHub upgrade, but we recommend a backup since user data is very - critical. +#. Your users' home directories. This is unlikely to be affected directly by + a JupyterHub upgrade, but we recommend a backup since user data is critical. -Shutdown JupyterHub +Shut down JupyterHub =================== -Shutdown the JupyterHub process. This would vary depending on how you +Shut down the JupyterHub process. This would vary depending on how you have set up JupyterHub to run. Most likely, it is using a process supervisor of some sort (``systemd`` or ``supervisord`` or even ``docker``). -Use the supervisor specific command to stop the JupyterHub process. +Use the supervisor-specific command to stop the JupyterHub process. Upgrade JupyterHub packages =========================== @@ -64,14 +63,14 @@ Upgrade JupyterHub packages There are two environments where the ``jupyterhub`` package is installed: #. The *hub environment*, which is where the JupyterHub server process - runs. This is started with the ``jupyterhub`` command, and is what - people generally think of as JupyterHub. + runs. This is started with the ``jupyterhub`` command and is what + people generally think of it as JupyterHub. -#. The *notebook user environments*. This is where the user notebook - servers are launched from, and is probably custom to your own +#. The *notebook user environments*. This is where the user's notebook + servers are launched from and are probably custom to your own installation. This could be just one environment (different from the hub environment) that is shared by all users, one environment - per user, or same environment as the hub environment. The hub + per user, or the same environment as the hub environment. The hub launched the ``jupyterhub-singleuser`` command in this environment, which in turn starts the notebook server. @@ -95,7 +94,7 @@ with: Where ```` is the version of JupyterHub you are upgrading to. You should also check for new releases of the authenticator & spawner you -are using. You might wish to upgrade those packages too along with JupyterHub, +are using. You might wish to upgrade those packages too along with JupyterHub or upgrade them separately. Upgrade JupyterHub database @@ -109,7 +108,7 @@ database. From the hub environment, in the same directory as your jupyterhub upgrade-db -This should find the location of your database, and run necessary upgrades +This should find the location of your database, and run the necessary upgrades for it. SQLite database disadvantages @@ -118,11 +117,11 @@ SQLite database disadvantages SQLite has some disadvantages when it comes to upgrading JupyterHub. These are: -- ``upgrade-db`` may not work, and you may need delete your database +- ``upgrade-db`` may not work, and you may need to delete your database and start with a fresh one. - ``downgrade-db`` **will not** work if you want to rollback to an earlier version, so backup the ``jupyterhub.sqlite`` file before - upgrading + upgrading. What happens if I delete my database? ------------------------------------- @@ -137,10 +136,10 @@ resides only in the Hub database includes: If the following conditions are true, you should be fine clearing the Hub database and starting over: -- users specified in config file, or login using an external +- users specified in the config file, or login using an external authentication provider (Google, GitHub, LDAP, etc) -- user servers are stopped during upgrade -- don't mind causing users to login again after upgrade +- user servers are stopped during the upgrade +- don't mind causing users to log in again after the upgrade Start JupyterHub ================ @@ -148,7 +147,7 @@ Start JupyterHub Once the database upgrade is completed, start the ``jupyterhub`` process again. -#. Log-in and start the server to make sure things work as +#. Log in and start the server to make sure things work as expected. #. Check the logs for any errors or deprecation warnings. You might have to update your ``jupyterhub_config.py`` file to From 6b4c5e4bcea032b1ffcbe32e9d423b00df350e4f Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Tue, 11 Oct 2022 19:36:04 +0500 Subject: [PATCH 021/214] update upgrading.rst --- docs/source/admin/upgrading.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index 96aee1c8..e5d77634 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -5,7 +5,7 @@ Upgrading JupyterHub JupyterHub offers easy upgrade pathways between minor versions. This document describes how to do these upgrades. -If you are using [distributions](index/distributions), you +If you are using [distributions](../index.rst#distributions), you should consult the distribution's documentation on how to upgrade. This document is useful if you have set up your own JupyterHub without using a distribution. From d5790ce386b0383f0734e6479eff9eb3746594ab Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Tue, 11 Oct 2022 19:38:36 +0500 Subject: [PATCH 022/214] update upgrading.rst --- docs/source/admin/upgrading.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index e5d77634..35b8c47e 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -5,7 +5,7 @@ Upgrading JupyterHub JupyterHub offers easy upgrade pathways between minor versions. This document describes how to do these upgrades. -If you are using [distributions](../index.rst#distributions), you +If you are using [distributions](https://github.com/ToobaJamal/jupyterhub/blob/main/docs/source/index.rst#distributions), you should consult the distribution's documentation on how to upgrade. This document is useful if you have set up your own JupyterHub without using a distribution. From 8f30f4afd96d1627bd0ad70f7a79a3b11f156c66 Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Tue, 11 Oct 2022 19:40:15 +0500 Subject: [PATCH 023/214] update upgrading.rst --- docs/source/admin/upgrading.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index 35b8c47e..5cc9430f 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -5,7 +5,7 @@ Upgrading JupyterHub JupyterHub offers easy upgrade pathways between minor versions. This document describes how to do these upgrades. -If you are using [distributions](https://github.com/ToobaJamal/jupyterhub/blob/main/docs/source/index.rst#distributions), you +If you are using `distributions `__, you should consult the distribution's documentation on how to upgrade. This document is useful if you have set up your own JupyterHub without using a distribution. From 916a83a9549ed85358f9c3d443f2091ec2dff33b Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Tue, 11 Oct 2022 19:03:21 +0100 Subject: [PATCH 024/214] Update tech-implementation.md Fix typos and grammatical errors. Improve sentence structure. Use appropriate technical terms. --- docs/source/rbac/tech-implementation.md | 35 ++++++++++++------------- 1 file changed, 17 insertions(+), 18 deletions(-) diff --git a/docs/source/rbac/tech-implementation.md b/docs/source/rbac/tech-implementation.md index 3156dbf1..b3084ed5 100644 --- a/docs/source/rbac/tech-implementation.md +++ b/docs/source/rbac/tech-implementation.md @@ -1,8 +1,8 @@ # Technical Implementation -Roles are stored in the database, where they are associated with users, services, etc., and can be added or modified as explained in {ref}`define-role-target` section. Users, services, groups, and tokens can gain, change, and lose roles. This is currently achieved via `jupyterhub_config.py` (see {ref}`define-role-target`) and will be made available via API in future. The latter will allow for changing a token's role, and thereby its permissions, without the need to issue a new token. +[Roles](https://jupyterhub.readthedocs.io/en/stable/rbac/roles.html#roles) are stored in the database, where they are associated with users, services, groups and tokens. Roles can be added or modified as explained in the {ref}`define-role-target` section. Users, services, groups, and tokens can gain, change, and lose roles. This is currently achieved via `jupyterhub_config.py` (see {ref}`define-role-target`) and will be made available via API in the future. The latter will allow for changing a token's role, and thereby its permissions, without the need to issue a new token. -Roles and scopes utilities can be found in `roles.py` and `scopes.py` modules. Scope variables take on five different formats which is reflected throughout the utilities via specific nomenclature: +Roles and scopes utilities can be found in `roles.py` and `scopes.py` modules. Scope variables take on five different formats that are reflected throughout the utilities via specific nomenclature: ```{admonition} **Scope variable nomenclature** :class: tip @@ -11,22 +11,22 @@ Roles and scopes utilities can be found in `roles.py` and `scopes.py` modules. S - _expanded scopes_ \ Set of fully expanded scopes without abbreviations (i.e., resolved metascopes, filters, and subscopes). E.g., `{"users:activity!user=charlie", "read:users:activity!user=charlie"}`. - _parsed scopes_ \ - Dictionary represenation of expanded scopes. E.g., `{"users:activity": {"user": ["charlie"]}, "read:users:activity": {"users": ["charlie"]}}`. + Dictionary representation of expanded scopes. E.g., `{"users:activity": {"user": ["charlie"]}, "read:users:activity": {"users": ["charlie"]}}`. - _intersection_ \ Set of expanded scopes as intersection of 2 expanded scope sets. - _identify scopes_ \ - Set of expanded scopes needed for identify (whoami) endpoints. + Set of expanded scopes needed for identity (whoami) endpoints. ``` (resolving-roles-scopes-target)= ## Resolving roles and scopes -**Resolving roles** refers to determining which roles a user, service, or group has, extracting the list of scopes from each role and combining them into a single set of scopes. +**Resolving roles** involves determining which roles a user, service, or group has, extracting the list of scopes from each role and combining them into a single set of scopes. -**Resolving scopes** involves expanding scopes into all their possible subscopes (_expanded scopes_), parsing them into format used for access evaluation (_parsed scopes_) and, if applicable, comparing two sets of scopes (_intersection_). All procedures take into account the scope hierarchy, {ref}`vertical ` and {ref}`horizontal filtering `, limiting or elevated permissions (`read:` or `admin:`, respectively), and metascopes. +**Resolving scopes** involves expanding scopes into all their possible subscopes (_expanded scopes_), parsing them into the format used for access evaluation (_parsed scopes_) and, if applicable, comparing two sets of scopes (_intersection_). All procedures take into account the scope hierarchy, {ref}`vertical ` and {ref}`horizontal filtering `, limiting or elevated permissions (`read:` or `admin:`, respectively), and metascopes. -Roles and scopes are resolved on several occasions, for example when requesting an API token with specific scopes or making an API request. The following sections provide more details. +Roles and scopes are resolved on several occasions, for example when requesting an API token with specific scopes or when making an API request. The following sections provide more details. (requesting-api-token-target)= @@ -43,26 +43,25 @@ Prior to 3.0, tokens stored _roles_, which meant their scopes were resolved on each request. ::: -API tokens grant access to JupyterHub's APIs. The RBAC framework allows for requesting tokens with specific permissions. +API tokens grant access to JupyterHub's APIs. The Role Based Access Control (RBAC) framework allows for the requesting of tokens with specific permissions. RBAC is involved in several stages of the OAuth token flow. When requesting a token via the tokens API (`/users/:name/tokens`), or the token page (`/hub/token`), if no scopes are requested, the token is issued with the permissions stored on the default `token` role -(providing the requester is allowed to create the token). +(provided the requester is allowed to create the token). OAuth tokens are also requested via OAuth flow If the token is requested with any scopes, the permissions of requesting entity are checked against the requested permissions to ensure the token would not grant its owner additional privileges. -If, due to modifications of permissions of the token or token owner, -at API request time a token has any scopes that its owner does not, -those scopes are removed. +If a token has any scopes that its owner does not possess +at the time of making the API request, those scopes are removed. The API request is resolved without additional errors using the scope _intersection_; the Hub logs a warning in this case (see {ref}`Figure 2 `). -Resolving a token's scope (yellow box in {ref}`Figure 1 `) corresponds to resolving all the token's owner roles (including the roles associated with their groups) and the token's own scopes into a set of scopes. The two sets are compared (Resolve the scopes box in orange in {ref}`Figure 1 `), taking into account the scope hierarchy. -If the token's scopes are a subset of the token owner's scopes, the token is issued with the requested scopes; if not, JupyterHub will raise an error. +Resolving a token's scope (yellow box in {ref}`Figure 1 `) corresponds to resolving all the roles of the token's owner (including the roles associated with their groups) and the token's own scopes into a set of scopes. The two sets are compared (Resolve the scopes box in orange in {ref}`Figure 1 `), taking into account the scope hierarchy. +If the token's scopes are a subset of the token owner's scopes, the token is issued with the requested scopes; if not, JupyterHub will throw an error. {ref}`Figure 1 ` below illustrates the steps involved. The orange rectangles highlight where in the process the roles and scopes are resolved. @@ -75,10 +74,10 @@ Figure 1. Resolving roles and scopes during API token request ### Making an API request -With the RBAC framework, each authenticated JupyterHub API request is guarded by a scope decorator that specifies which scopes are required to gain the access to the API. +With the RBAC framework, each authenticated JupyterHub API request is guarded by a scope decorator that specifies which scopes are required in order to gain the access to the API. -When an API request is performed, the requesting API token's scopes are again intersected with its owner's (yellow box in {ref}`Figure 2 `) to ensure the token does not grant more permissions than its owner has at the request time (e.g., due to changing/losing roles). -If the owner's roles do not include some scopes of the token's scopes, only the _intersection_ of the token's and owner's scopes will be used. For example, using a token with scope `users` whose owner's role scope is `read:users:name` will result in only the `read:users:name` scope being passed on. In the case of no _intersection_, an empty set of scopes will be used. +When an API request is made, the requesting API token's scopes are again intersected with its owner's (yellow box in {ref}`Figure 2 `) to ensure that the token does not grant more permissions than its owner has at the request time (e.g., due to changing/losing roles). +If the owner's roles do not include some scopes of the token, only the _intersection_ of the token's and owner's scopes will be used. For example, using a token with scope `users` whose owner's role scope is `read:users:name` will result in only the `read:users:name` scope being passed on. In the case of no _intersection_, an empty set of scopes will be used. The passed scopes are compared to the scopes required to access the API as follows: @@ -86,7 +85,7 @@ The passed scopes are compared to the scopes required to access the API as follo - if that is not the case, another check is utilized to determine if subscopes of the required API scopes can be found in the passed scope set: - - if found, the RBAC framework employs the {ref}`filtering ` procedures to refine the API response to access only resource attributes corresponding to the passed scopes. For example, providing a scope `read:users:activity!group=class-C` for the _GET /users_ API will return a list of user models from group `class-C` containing only the `last_activity` attribute for each user model + - if found, the RBAC framework employs the {ref}`filtering ` procedures to refine the API response to access only resource attributes corresponding to the passed scopes. For example, providing a scope `read:users:activity!group=class-C` for the `GET /users` API will return a list of user models from group `class-C` containing only the `last_activity` attribute for each user model - if not found, the access to API is denied From a697c80475bc932fe26c1d55916e44628f4d891f Mon Sep 17 00:00:00 2001 From: Joel-Ando Date: Wed, 12 Oct 2022 01:10:09 +0100 Subject: [PATCH 025/214] Update tech-implementation.md --- docs/source/rbac/tech-implementation.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/source/rbac/tech-implementation.md b/docs/source/rbac/tech-implementation.md index 3156dbf1..d5565d8f 100644 --- a/docs/source/rbac/tech-implementation.md +++ b/docs/source/rbac/tech-implementation.md @@ -11,7 +11,7 @@ Roles and scopes utilities can be found in `roles.py` and `scopes.py` modules. S - _expanded scopes_ \ Set of fully expanded scopes without abbreviations (i.e., resolved metascopes, filters, and subscopes). E.g., `{"users:activity!user=charlie", "read:users:activity!user=charlie"}`. - _parsed scopes_ \ - Dictionary represenation of expanded scopes. E.g., `{"users:activity": {"user": ["charlie"]}, "read:users:activity": {"users": ["charlie"]}}`. + Dictionary representation of expanded scopes. E.g., `{"users:activity": {"user": ["charlie"]}, "read:users:activity": {"users": ["charlie"]}}`. - _intersection_ \ Set of expanded scopes as intersection of 2 expanded scope sets. - _identify scopes_ \ @@ -43,7 +43,7 @@ Prior to 3.0, tokens stored _roles_, which meant their scopes were resolved on each request. ::: -API tokens grant access to JupyterHub's APIs. The RBAC framework allows for requesting tokens with specific permissions. +API tokens grant access to JupyterHub's APIs. The [RBAC framework](https://auth0.com/docs/manage-users/access-control/rbac) allows for requesting tokens with specific permissions. RBAC is involved in several stages of the OAuth token flow. @@ -78,15 +78,15 @@ Figure 1. Resolving roles and scopes during API token request With the RBAC framework, each authenticated JupyterHub API request is guarded by a scope decorator that specifies which scopes are required to gain the access to the API. When an API request is performed, the requesting API token's scopes are again intersected with its owner's (yellow box in {ref}`Figure 2 `) to ensure the token does not grant more permissions than its owner has at the request time (e.g., due to changing/losing roles). -If the owner's roles do not include some scopes of the token's scopes, only the _intersection_ of the token's and owner's scopes will be used. For example, using a token with scope `users` whose owner's role scope is `read:users:name` will result in only the `read:users:name` scope being passed on. In the case of no _intersection_, an empty set of scopes will be used. +If the owner's roles do not include some scopes of the token's scopes, only the _intersection_ of the token's and owner's scopes will be used. For example, using a token with scope `users` whose owner's role scope is `read:users:name` will result in only the `read:users:name` scope being parsed on. In the case of no _intersection_, an empty set of scopes will be used. -The passed scopes are compared to the scopes required to access the API as follows: +The parsed scopes are compared to the scopes required to access the API as follows: -- if the API scopes are present within the set of passed scopes, the access is granted and the API returns its "full" response +- if the API scopes are present within the set of parsed scopes, the access is granted and the API returns its "full" response -- if that is not the case, another check is utilized to determine if subscopes of the required API scopes can be found in the passed scope set: +- if that is not the case, another check is utilized to determine if subscopes of the required API scopes can be found in the parsed scope set: - - if found, the RBAC framework employs the {ref}`filtering ` procedures to refine the API response to access only resource attributes corresponding to the passed scopes. For example, providing a scope `read:users:activity!group=class-C` for the _GET /users_ API will return a list of user models from group `class-C` containing only the `last_activity` attribute for each user model + - if found, the RBAC framework employs the {ref}`filtering ` procedures to refine the API response to access only resource attributes corresponding to the parsed scopes. For example, providing a scope `read:users:activity!group=class-C` for the _GET /users_ API will return a list of user models from group `class-C` containing only the `last_activity` attribute for each user model - if not found, the access to API is denied From 6ef8120f94dbdc4bd0031f34ff09df01e96c58e3 Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Wed, 12 Oct 2022 10:58:13 +0500 Subject: [PATCH 026/214] fix grammatical error --- docs/source/admin/upgrading.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index 5cc9430f..8b786143 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -27,11 +27,11 @@ Notify your users If you are using the default configuration where ``configurable-http-proxy`` is managed by JupyterHub, your users will see service disruption during -the upgrade process. You will need notify them, and pick a time to do the +the upgrade process. You will need to notify them, and pick a time to do the upgrade where they will be least disrupted. If you are using a different proxy or running ``configurable-http-proxy`` -independent of JupyterHub, your users will be able to continue using a notebook +independent of JupyterHub, your users will be able to continue using notebook servers they had already launched, but will not be able to launch new servers or sign in. From dcd4e689aa5587dd0ab6dececcd5322087491c5c Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Wed, 12 Oct 2022 12:09:57 +0500 Subject: [PATCH 027/214] update index.rst --- docs/source/events/index.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/events/index.rst b/docs/source/events/index.rst index bc086ba1..b53c8448 100644 --- a/docs/source/events/index.rst +++ b/docs/source/events/index.rst @@ -1,7 +1,7 @@ -Eventlogging and Telemetry +Event logging and Telemetry ========================== -JupyterHub can be configured to record structured events from a running server using Jupyter's `Telemetry System`_. The types of events that JupyterHub emits are defined by `JSON schemas`_ listed at the bottom of this page_. +JupyterHub can be configured to record structured events from a running server using Jupyter's `Telemetry System`_. The types of events that JupyterHub emits are defined by `JSON schemas`_ listed at the bottom of this `page `__. .. _logging: https://docs.python.org/3/library/logging.html .. _`Telemetry System`: https://github.com/jupyter/telemetry @@ -15,8 +15,8 @@ Event logging is handled by its ``Eventlog`` object. This leverages Python's sta To begin recording events, you'll need to set two configurations: - 1. ``handlers``: tells the EventLog *where* to route your events. This trait is a list of Python logging handlers that route events to - 2. ``allows_schemas``: tells the EventLog *which* events should be recorded. No events are emitted by default; all recorded events must be listed here. + 1. ``handlers``: tells the EventLog *where* to route your events; this trait is a list of Python logging handlers that route events to the destination + 2. ``allows_schemas``: tells the EventLog *which* events should be recorded; no events are emitted by default; all recorded events must be listed here Here's a basic example: From 1050dadda4fe0a40221987af5bd7becbac1451ff Mon Sep 17 00:00:00 2001 From: Temidayo Date: Wed, 12 Oct 2022 12:27:57 +0100 Subject: [PATCH 028/214] modified announcement README and config.py --- examples/service-announcement/README.md | 19 ++++++++++++++++++- .../service-announcement/jupyterhub_config.py | 2 ++ 2 files changed, 20 insertions(+), 1 deletion(-) diff --git a/examples/service-announcement/README.md b/examples/service-announcement/README.md index 0582facc..6e31e8e5 100644 --- a/examples/service-announcement/README.md +++ b/examples/service-announcement/README.md @@ -6,12 +6,29 @@ that appear when JupyterHub renders pages. To run the service as a hub-managed service simply include in your JupyterHub configuration file something like: +:notebook:**Info**: You can run the announcement service example from the `examples` + directory, using one of the several services provided by JupyterHub. + + ```python + +import sys + +from pathlib import Path +# absolute path to announcement.py +announcement_py = str(Path(__file__).parent.joinpath("announcement.py").resolve()) + +#ensure get_config() is added to in + c = get_config() + +... +.. + c.JupyterHub.services = [ { 'name': 'announcement', 'url': 'http://127.0.0.1:8888', - 'command': [sys.executable, "-m", "announcement", "--port", "8888"], + 'command': [sys.executable, announcement_py, "--port", "8888"], } ] ``` diff --git a/examples/service-announcement/jupyterhub_config.py b/examples/service-announcement/jupyterhub_config.py index 9bca9809..07a9fe76 100644 --- a/examples/service-announcement/jupyterhub_config.py +++ b/examples/service-announcement/jupyterhub_config.py @@ -1,5 +1,7 @@ import sys +c = get_config() + # To run the announcement service managed by the hub, add this. port = 9999 From 08a125489d21983909822277cbdfd0a6aa9d925c Mon Sep 17 00:00:00 2001 From: Temidayo Date: Wed, 12 Oct 2022 12:29:07 +0100 Subject: [PATCH 029/214] fixed a typo --- examples/service-announcement/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/service-announcement/README.md b/examples/service-announcement/README.md index 6e31e8e5..f6dbb7ed 100644 --- a/examples/service-announcement/README.md +++ b/examples/service-announcement/README.md @@ -18,7 +18,7 @@ from pathlib import Path # absolute path to announcement.py announcement_py = str(Path(__file__).parent.joinpath("announcement.py").resolve()) -#ensure get_config() is added to in +#ensure get_config() is added in c = get_config() ... From 67bca186b44b6c0b79d41883275428a421ff32ba Mon Sep 17 00:00:00 2001 From: Goodness Chris-Ugari Date: Wed, 12 Oct 2022 12:31:44 +0100 Subject: [PATCH 030/214] Update config-user-env.md This PR improves the 'Configuring user environments' doc and is part of [issue 41](https://github.com/jupyterhub/outreachy/issues/41) --- docs/source/reference/config-user-env.md | 32 ++++++++++++------------ 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/source/reference/config-user-env.md b/docs/source/reference/config-user-env.md index 503d2661..4a2be932 100644 --- a/docs/source/reference/config-user-env.md +++ b/docs/source/reference/config-user-env.md @@ -7,8 +7,8 @@ environment in some way. Since the `jupyterhub-singleuser` server extends the standard Jupyter notebook server, most configuration and documentation that applies to Jupyter Notebook applies to the single-user environments. Configuration of user environments -typically does not occur through JupyterHub itself, but rather through system- -wide configuration of Jupyter, which is inherited by `jupyterhub-singleuser`. +typically does not occur through JupyterHub itself, but rather through the system-wide +configuration of Jupyter, which is inherited by `jupyterhub-singleuser`. **Tip:** When searching for configuration tips for JupyterHub user environments, try removing JupyterHub from your search because there are a lot @@ -17,10 +17,10 @@ configuration is the same. This section will focus on user environments, including: -- Installing packages -- Configuring Jupyter and IPython -- Installing kernelspecs -- Using containers vs. multi-user hosts +- [Installing packages](#installing-packages) +- [Configuring Jupyter and IPython](#configuring-jupyter-and-ipython) +- [Installing kernelspecs](#installing-kernelspecs) +- [Using containers vs. multi-user hosts](#multi-user-hosts-vs-containers) ## Installing packages @@ -30,7 +30,7 @@ system-wide or in a shared environment. This installation location should always be in the same environment that `jupyterhub-singleuser` itself is installed in, and must be _readable and executable_ by your users. If you want users to be able to install additional -packages, it must also be _writable_ by your users. +packages, they must also be _writable_ by your users. If you are using a standard system Python install, you would use: @@ -109,7 +109,7 @@ they are available to all of your users. This means installing kernelspecs either system-wide (e.g. in /usr/local/) or in the `sys.prefix` of JupyterHub itself. -Jupyter kernelspec installation is system wide by default, but some kernels +Jupyter kernelspec installation is system-wide by default, but some kernels may default to installing kernelspecs in your home directory. These will need to be moved system-wide to ensure that they are accessible. @@ -143,12 +143,12 @@ depending on what Spawner you are using. The first category is a **shared system (multi-user host)** where each user has a JupyterHub account and a home directory as well as being a real system user. In this example, shared configuration and installation -must be in a 'system-wide' location, such as `/etc/` or `/usr/local` +must be in a 'system-wide' location, such as `/etc/`, or `/usr/local` or a custom prefix such as `/opt/conda`. When JupyterHub uses **container-based** Spawners (e.g. KubeSpawner or DockerSpawner), the 'system-wide' environment is really the container image -which you are using for users. +that you are using for users. In both cases, you want to _avoid putting configuration in user home directories_ because users can change those configuration settings. Also, @@ -157,7 +157,7 @@ difficult for admins to update later. ## Named servers -By default, in a JupyterHub deployment each user has exactly one server. +By default, in a JupyterHub deployment, each user has exactly one server. JupyterHub can, however, have multiple servers per user. This is most useful in deployments where users can configure the environment @@ -181,7 +181,7 @@ as well as the admin page: ![named servers on the admin page](../images/named-servers-admin.png) Named servers can be accessed, created, started, stopped, and deleted -from these pages. Activity tracking is now per-server as well. +from these pages. Activity tracking is now per server as well. The number of named servers per user can be limited by setting a constant value: @@ -207,9 +207,9 @@ If `named_server_limit_per_user` is set to `0`, no limit is enforced. (classic-notebook-ui)= -## Switching back to classic notebook +## Switching back to the classic notebook -By default the single-user server launches JupyterLab, +By default, the single-user server launches JupyterLab, which is based on [Jupyter Server][]. This is the default server when running JupyterHub ≥ 2.0. You can switch to using the legacy Jupyter Notebook server by setting the `JUPYTERHUB_SINGLEUSER_APP` environment variable @@ -223,11 +223,11 @@ export JUPYTERHUB_SINGLEUSER_APP='notebook.notebookapp.NotebookApp' [jupyter notebook]: https://jupyter-notebook.readthedocs.io :::{versionchanged} 2.0 -JupyterLab is now the default singleuser UI, if available, +JupyterLab is now the default single-user UI, if available, which is based on the [Jupyter Server][], no longer the legacy [Jupyter Notebook][] server. JupyterHub prior to 2.0 launched the legacy notebook server (`jupyter notebook`), -and Jupyter server could be selected by specifying +and the Jupyter server could be selected by specifying ```python # jupyterhub_config.py From af95d643b14faef952155eee90a1d653bf4e59e0 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 12 Oct 2022 11:33:53 +0000 Subject: [PATCH 031/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/reference/config-user-env.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/reference/config-user-env.md b/docs/source/reference/config-user-env.md index 4a2be932..251fee0c 100644 --- a/docs/source/reference/config-user-env.md +++ b/docs/source/reference/config-user-env.md @@ -7,7 +7,7 @@ environment in some way. Since the `jupyterhub-singleuser` server extends the standard Jupyter notebook server, most configuration and documentation that applies to Jupyter Notebook applies to the single-user environments. Configuration of user environments -typically does not occur through JupyterHub itself, but rather through the system-wide +typically does not occur through JupyterHub itself, but rather through the system-wide configuration of Jupyter, which is inherited by `jupyterhub-singleuser`. **Tip:** When searching for configuration tips for JupyterHub user From ef7545fc758d1f081a5eb5cdd40eea5925e76615 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 12 Oct 2022 11:34:18 +0000 Subject: [PATCH 032/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- examples/service-announcement/README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/examples/service-announcement/README.md b/examples/service-announcement/README.md index f6dbb7ed..637f25f5 100644 --- a/examples/service-announcement/README.md +++ b/examples/service-announcement/README.md @@ -7,8 +7,7 @@ To run the service as a hub-managed service simply include in your JupyterHub configuration file something like: :notebook:**Info**: You can run the announcement service example from the `examples` - directory, using one of the several services provided by JupyterHub. - +directory, using one of the several services provided by JupyterHub. ```python From 71ce37b83467c8fbe8534f6c7e93ba105e7a2c04 Mon Sep 17 00:00:00 2001 From: Goodness Chris-Ugari Date: Wed, 12 Oct 2022 13:02:12 +0100 Subject: [PATCH 033/214] Update troubleshooting.md Implement suggested changes. --- docs/source/troubleshooting.md | 42 +++++++++++++++++----------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index 5c9bcbcc..1f051859 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -6,32 +6,32 @@ problem and how to resolve it. [_Behavior_](#behavior) -- [JupyterHub proxy fails to start](#jupyterhub-proxy-fails-to-start) -- [sudospawner fails to run](#sudospawner-fails-to-run) -- [What is the default behavior when none of the lists (admin, allowed, - allowed groups) are set?](#what-is-the-default-behavior-when-none-of-the-lists-admin-allowed-allowed-groups-are-set) -- [JupyterHub Docker container not accessible at localhost](#jupyterhub-docker-container-not-accessible-at-localhost) -- [How can I kill ports from JupyterHub-managed services that have been orphaned?](#how-can-i-kill-ports-from-jupyterhub-managed-services-that-have-been-orphaned) -- [Why am I getting a Spawn failed error message?](#why-am-i-getting-a-spawn-failed-error-message) -- [How can I run JupyterHub with sudo but use my current env vars and virtualenv location?](#how-can-i-run-jupyterhub-with-sudo-but-use-my-current-env-vars-and-virtualenv-location) +- JupyterHub proxy fails to start +- sudospawner fails to run] +- What is the default behavior when none of the lists (admin, allowed, + allowed groups) are set? +- JupyterHub Docker container not accessible at localhost +- How can I kill ports from JupyterHub-managed services that have been orphaned? +- Why am I getting a Spawn failed error message? +- How can I run JupyterHub with sudo but use my current env vars and virtualenv location? [_Errors_](#errors) -- [Error 500 after spawning my single-user server](#error-500-after-spawning-my-single-user-server) -- [Launching Jupyter Notebooks to run as an externally managed JupyterHub service with the `jupyterhub-singleuser` command returns a `JUPYTERHUB_API_TOKEN` error](#launching-jupyter-notebooks-to-run-as-an-externally-managed-jupyterhub-service-with-the-jupyterhub-singleuser-command-returns-a-jupyterhub-api-token-error) +- Error 500 after spawning my single-user server +- Launching Jupyter Notebooks to run as an externally managed JupyterHub service with the `jupyterhub-singleuser` command returns a `JUPYTERHUB_API_TOKEN` error [_How do I...?_](#how-do-i) -- [Use a chained SSL certificate](#use-a-chained-ssl-certificate) -- [Install JupyterHub without a network connection](#install-jupyterhub-without-a-network-connection) -- [I want access to the whole filesystem and still default users to their home directory](#i-want-access-to-the-whole-filesystem-and-still-default-users-to-their-home-directory) -- [How do I increase the number of pySpark executors on YARN?](#how-do-i-increase-the-number-of-pyspark-executors-on-yarn) -- [How do I use JupyterLab's prerelease version with JupyterHub?](#how-do-i-use-jupyterlab-s-prerelease-version-with-jupyterhub) -- [How do I set up JupyterHub for a workshop (when users are not known ahead of time)?](#how-do-i-set-up-jupyterhub-for-a-workshop-when-users-are-not-known-ahead-of-time) -- [How do I set up rotating daily logs?](#how-do-i-set-up-rotating-daily-logs) -- [Toree integration with HDFS rack awareness script](#toree-integration-with-hdfs-rack-awareness-script) -- [Where do I find Docker images and Dockerfiles related to JupyterHub?](#where-do-i-find-docker-images-and-dockerfiles-related-to-jupyterhub) -- [How can I view the logs for JupyterHub or the user's Notebook servers when using the DockerSpawner?](#how-can-i-view-the-logs-for-jupyterhub-or-the-user-s-notebook-servers-when-using-the-dockerspawner) +- Use a chained SSL certificate +- Install JupyterHub without a network connection +- I want access to the whole filesystem and still default users to their home directory +- How do I increase the number of pySpark executors on YARN? +- How do I use JupyterLab's prerelease version with JupyterHub? +- How do I set up JupyterHub for a workshop (when users are not known ahead of time)? +- How do I set up rotating daily logs? +- Toree integration with HDFS rack awareness script +- Where do I find Docker images and Dockerfiles related to JupyterHub? +- How can I view the logs for JupyterHub or the user's Notebook servers when using the DockerSpawner? [_Troubleshooting commands_](#troubleshooting-commands) @@ -312,7 +312,7 @@ For instance: python3 -m pip install jupyterlab jupyter serverextension enable --py jupyterlab --sys-prefix -The important thing is that Jupyterlab is installed and enabled in the +The important thing is that JupyterLab is installed and enabled in the single-user notebook server environment. For system users, this means system-wide, as indicated above. For Docker containers, it means inside the single-user docker image, etc. From f386da1b7ac81e29d81c31c95f20d8d9bd35de2b Mon Sep 17 00:00:00 2001 From: Ojoachele Onuh Date: Wed, 12 Oct 2022 13:12:08 +0100 Subject: [PATCH 034/214] fixed some typos and technical terms --- docs/source/rbac/tech-implementation.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/source/rbac/tech-implementation.md b/docs/source/rbac/tech-implementation.md index 3156dbf1..eccb2d0b 100644 --- a/docs/source/rbac/tech-implementation.md +++ b/docs/source/rbac/tech-implementation.md @@ -11,7 +11,7 @@ Roles and scopes utilities can be found in `roles.py` and `scopes.py` modules. S - _expanded scopes_ \ Set of fully expanded scopes without abbreviations (i.e., resolved metascopes, filters, and subscopes). E.g., `{"users:activity!user=charlie", "read:users:activity!user=charlie"}`. - _parsed scopes_ \ - Dictionary represenation of expanded scopes. E.g., `{"users:activity": {"user": ["charlie"]}, "read:users:activity": {"users": ["charlie"]}}`. + Dictionary representation of expanded scopes. E.g., `{"users:activity": {"user": ["charlie"]}, "read:users:activity": {"users": ["charlie"]}}`. - _intersection_ \ Set of expanded scopes as intersection of 2 expanded scope sets. - _identify scopes_ \ @@ -78,15 +78,15 @@ Figure 1. Resolving roles and scopes during API token request With the RBAC framework, each authenticated JupyterHub API request is guarded by a scope decorator that specifies which scopes are required to gain the access to the API. When an API request is performed, the requesting API token's scopes are again intersected with its owner's (yellow box in {ref}`Figure 2 `) to ensure the token does not grant more permissions than its owner has at the request time (e.g., due to changing/losing roles). -If the owner's roles do not include some scopes of the token's scopes, only the _intersection_ of the token's and owner's scopes will be used. For example, using a token with scope `users` whose owner's role scope is `read:users:name` will result in only the `read:users:name` scope being passed on. In the case of no _intersection_, an empty set of scopes will be used. +If the owner's roles do not include some scopes of the token's scopes, only the _intersection_ of the token's and owner's scopes will be used. For example, using a token with scope `users` whose owner's role scope is `read:users:name` will result in only the `read:users:name` scope being parsed on. In the case of no _intersection_, an empty set of scopes will be used. -The passed scopes are compared to the scopes required to access the API as follows: +The parsed scopes are compared to the scopes required to access the API as follows: -- if the API scopes are present within the set of passed scopes, the access is granted and the API returns its "full" response +- if the API scopes are present within the set of parsed scopes, the access is granted and the API returns its "full" response -- if that is not the case, another check is utilized to determine if subscopes of the required API scopes can be found in the passed scope set: +- if that is not the case, another check is utilized to determine if subscopes of the required API scopes can be found in the parsed scope set: - - if found, the RBAC framework employs the {ref}`filtering ` procedures to refine the API response to access only resource attributes corresponding to the passed scopes. For example, providing a scope `read:users:activity!group=class-C` for the _GET /users_ API will return a list of user models from group `class-C` containing only the `last_activity` attribute for each user model + - if found, the RBAC framework employs the {ref}`filtering ` procedures to refine the API response to access only resource attributes corresponding to the parsed scopes. For example, providing a scope `read:users:activity!group=class-C` for the _GET /users_ API will return a list of user models from group `class-C` containing only the `last_activity` attribute for each user model - if not found, the access to API is denied From 0820f4cfa10ccb260327cf6f0a59121dc9943195 Mon Sep 17 00:00:00 2001 From: Shloka Date: Wed, 12 Oct 2022 18:52:58 +0530 Subject: [PATCH 035/214] Modifications --- docs/source/contributing/community.rst | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/docs/source/contributing/community.rst b/docs/source/contributing/community.rst index 638b3667..9ab8a54d 100644 --- a/docs/source/contributing/community.rst +++ b/docs/source/contributing/community.rst @@ -4,13 +4,8 @@ Community communication channels ================================ -There is a place for everyone in the Jupyter community to share thoughts and ask questions. We use `Discourse `_ , `Gitter `_ , and GitHub issues for online discussion. - -JupyterHub's main Gitter channel is `jupyterhub/jupyterhub `_. -However, to ensure that discussions are the most useful and accessible to the community, we advise using Discourse first as Gitter doesn't have an archive or search function -Please be patient if you do not recieve a response right away; keep in mind that our comunity is spread out over the globe in different timezones. - -Most lengthy project conversations, bug reports, and feature requests take place via GitHub issues. If you have an issue with a certain authenticator or spawner, you should report them to the relevant repository. If you are using a specific JupyterHub +`Discourse `_ is primarily used by the Jupyter community for discussions and support. Additionally, we use JupyterHub's primary Gitter channel, `jupyterhub/jupyterhub `_, for quick discussions, and GitHub issues for detailed discussions relating to modifying a repository's content. +If you have an issue with a certain authenticator or spawner, please report them to the relevant repository. If you are using a specific JupyterHub distribution (such as `Zero to JupyterHub on Kubernetes `_ or `The Littlest JupyterHub `_), you should open issues directly in the respective repository. If you can not @@ -18,5 +13,4 @@ find a repository to open your issue in, do not worry! Create it in the `main JupyterHub repository `_ and our community will help you figure it out. -There is a `mailing list `_ for `teaching with Jupyter -`_ as well as one for the entire of Project Jupyter. \ No newline at end of file +Please be patient if you do not recieve a response right away; keep in mind that our comunity is spread out over the globe in different timezones. \ No newline at end of file From a87872e7aa87bbccb0b56039c4543f7dfed3fd60 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 12 Oct 2022 13:27:59 +0000 Subject: [PATCH 036/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/contributing/community.rst | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/source/contributing/community.rst b/docs/source/contributing/community.rst index b0dc6be2..fe88e40f 100644 --- a/docs/source/contributing/community.rst +++ b/docs/source/contributing/community.rst @@ -14,4 +14,3 @@ JupyterHub repository `_ and our community will help you figure it out. Please be patient if you do not recieve a response right away; keep in mind that our comunity is spread out over the globe in different timezones. - From 733e018bdc7550309f04b033c7d3a0944c867891 Mon Sep 17 00:00:00 2001 From: Shloka Date: Wed, 12 Oct 2022 19:07:43 +0530 Subject: [PATCH 037/214] Update docs/source/contributing/tests.rst Co-authored-by: Simon Li --- docs/source/contributing/tests.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/tests.rst b/docs/source/contributing/tests.rst index 8fbf17aa..faca41b0 100644 --- a/docs/source/contributing/tests.rst +++ b/docs/source/contributing/tests.rst @@ -4,7 +4,7 @@ Testing JupyterHub and linting code =================================== -Unit tests help confirming that JupyterHub works as intended and does so even when modifications are made. Additionally, they help in clarifying our expectations for our code. +Unit tests help confirm that JupyterHub works as intended, including after modifications are made. Additionally, they help in clarifying our expectations for our code. JupyterHub uses `pytest `_ for all the tests. You can find them under the `jupyterhub/tests `_ directory in the git repository. From 7a915533a6eb51a6c14b522ab4cb5b6e8422d204 Mon Sep 17 00:00:00 2001 From: Shloka Date: Wed, 12 Oct 2022 19:07:50 +0530 Subject: [PATCH 038/214] Update docs/source/contributing/tests.rst Co-authored-by: Simon Li --- docs/source/contributing/tests.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/tests.rst b/docs/source/contributing/tests.rst index faca41b0..2fbfa6f1 100644 --- a/docs/source/contributing/tests.rst +++ b/docs/source/contributing/tests.rst @@ -54,7 +54,7 @@ Running the tests pytest -v jupyterhub/tests/test_api.py::test_shutdown - For more information, refer to `pytest usage documentation `_. + For more information, refer to the `pytest usage documentation `_. Test organisation ================= From b0f90a0f4b576b5e581ed56f7adbad2eb59e617e Mon Sep 17 00:00:00 2001 From: Shloka Date: Wed, 12 Oct 2022 19:08:01 +0530 Subject: [PATCH 039/214] Update docs/source/contributing/tests.rst Co-authored-by: Simon Li --- docs/source/contributing/tests.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/tests.rst b/docs/source/contributing/tests.rst index 2fbfa6f1..35561e37 100644 --- a/docs/source/contributing/tests.rst +++ b/docs/source/contributing/tests.rst @@ -95,7 +95,7 @@ And fixtures to add functionality or spawning behavior: - ``bad_spawn``: enables the BadSpawner (a spawner that fails immediately) - ``slow_bad_spawn``: enables the SlowBadSpawner (a spawner that fails after a short delay) -For information on using the existing fixtures and creating new ones, refer to `pytest fixtures documentation `_ +For information on using the existing fixtures and creating new ones, refer to the `pytest fixtures documentation `_ Troubleshooting Test Failures From da302f5206c83a292db0aefd17221aceb4387567 Mon Sep 17 00:00:00 2001 From: Goodness Chris-Ugari Date: Wed, 12 Oct 2022 14:51:22 +0100 Subject: [PATCH 040/214] Update config-proxy.md This PR improves the proxy configuration doc and is part of [issue 41](https://github.com/jupyterhub/outreachy/issues/41) --- docs/source/reference/config-proxy.md | 52 +++++++++++++-------------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/docs/source/reference/config-proxy.md b/docs/source/reference/config-proxy.md index 4f05a8c7..90e56b06 100644 --- a/docs/source/reference/config-proxy.md +++ b/docs/source/reference/config-proxy.md @@ -14,7 +14,7 @@ satisfy the following: - After testing, the server in question should be able to score at least an A on the Qualys SSL Labs [SSL Server Test](https://www.ssllabs.com/ssltest/) -Let's start out with needed JupyterHub configuration in `jupyterhub_config.py`: +Let's start out with the needed JupyterHub configuration in `jupyterhub_config.py`: ```python # Force the proxy to only listen to connections to 127.0.0.1 (on port 8000) @@ -30,15 +30,15 @@ This can take a few minutes: openssl dhparam -out /etc/ssl/certs/dhparam.pem 4096 ``` -## nginx +## Nginx This **`nginx` config file** is fairly standard fare except for the two `location` blocks within the main section for HUB.DOMAIN.tld. -To create a new site for jupyterhub in your nginx config, make a new file +To create a new site for jupyterhub in your Nginx config, make a new file in `sites.enabled`, e.g. `/etc/nginx/sites.enabled/jupyterhub.conf`: ```bash -# top-level http config for websocket headers +# Top-level HTTP config for WebSocket headers # If Upgrade is defined, Connection = upgrade # If Upgrade is empty, Connection = close map $http_upgrade $connection_upgrade { @@ -51,7 +51,7 @@ server { listen 80; server_name HUB.DOMAIN.TLD; - # Tell all requests to port 80 to be 302 redirected to HTTPS + # Send all requests to port 80 to 302 and redirect to HTTPS return 302 https://$host$request_uri; } @@ -75,7 +75,7 @@ server { ssl_stapling_verify on; add_header Strict-Transport-Security max-age=15768000; - # Managing literal requests to the JupyterHub front end + # Managing literal requests to the JupyterHub frontend location / { proxy_pass http://127.0.0.1:8000; proxy_set_header X-Real-IP $remote_addr; @@ -101,10 +101,10 @@ server { If `nginx` is not running on port 443, substitute `$http_host` for `$host` on the lines setting the `Host` header. -`nginx` will now be the front facing element of JupyterHub on `443` which means +`nginx` will now be the front-facing element of JupyterHub on `443` which means it is also free to bind other servers, like `NO_HUB.DOMAIN.TLD` to the same port on the same machine and network interface. In fact, one can simply use the same -server blocks as above for `NO_HUB` and simply add line for the root directory +server blocks as above for `NO_HUB` and simply add a line for the root directory of the site as well as the applicable location call: ```bash @@ -112,7 +112,7 @@ server { listen 80; server_name NO_HUB.DOMAIN.TLD; - # Tell all requests to port 80 to be 302 redirected to HTTPS + # Send all requests to port 80 to 302 and redirect to HTTPS return 302 https://$host$request_uri; } @@ -143,12 +143,12 @@ Now restart `nginx`, restart the JupyterHub, and enjoy accessing `https://HUB.DOMAIN.TLD` while serving other content securely on `https://NO_HUB.DOMAIN.TLD`. -### SELinux permissions for nginx +### SELinux permissions for Nginx On distributions with SELinux enabled (e.g. Fedora), one may encounter permission errors -when the nginx service is started. +when the Nginx service is started. -We need to allow nginx to perform network relay and connect to the jupyterhub port. The +We need to allow Nginx to perform network relay and connect to the JupyterHub port. The following commands do that: ```bash @@ -157,26 +157,26 @@ setsebool -P httpd_can_network_relay 1 setsebool -P httpd_can_network_connect 1 ``` -Replace 8000 with the port the jupyterhub server is running from. +Replace 8000 with the port the JupyterHub server is running from. ## Apache -As with nginx above, you can use [Apache](https://httpd.apache.org) as the reverse proxy. -First, we will need to enable the apache modules that we are going to need: +As with Nginx above, you can use [Apache](https://httpd.apache.org) as the reverse proxy. +First, we will need to enable the Apache modules that we are going to need: ```bash a2enmod ssl rewrite proxy headers proxy_http proxy_wstunnel ``` -Our Apache configuration is equivalent to the nginx configuration above: +Our Apache configuration is equivalent to the Nginx configuration above: - Redirect HTTP to HTTPS - Good SSL Configuration -- Support for websockets on any proxied URL +- Support for WebSocket on any proxied URL - JupyterHub is running locally at http://127.0.0.1:8000 ```bash -# redirect HTTP to HTTPS +# Redirect HTTP to HTTPS Listen 80 ServerName HUB.DOMAIN.TLD @@ -188,26 +188,26 @@ Listen 443 ServerName HUB.DOMAIN.TLD - # enable HTTP/2, if available + # Enable HTTP/2, if available Protocols h2 http/1.1 # HTTP Strict Transport Security (mod_headers is required) (63072000 seconds) Header always set Strict-Transport-Security "max-age=63072000" - # configure SSL + # Configure SSL SSLEngine on SSLCertificateFile /etc/letsencrypt/live/HUB.DOMAIN.TLD/fullchain.pem SSLCertificateKeyFile /etc/letsencrypt/live/HUB.DOMAIN.TLD/privkey.pem SSLOpenSSLConfCmd DHParameters /etc/ssl/certs/dhparam.pem - # intermediate configuration from ssl-config.mozilla.org (2022-03-03) - # Please note, that this configuration might be out-dated - please update it accordingly using https://ssl-config.mozilla.org/ + # Intermediate configuration from SSL-config.mozilla.org (2022-03-03) + # Please note, that this configuration might be outdated - please update it accordingly using https://ssl-config.mozilla.org/ SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1 SSLCipherSuite ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384 SSLHonorCipherOrder off SSLSessionTickets off - # Use RewriteEngine to handle websocket connection upgrades + # Use RewriteEngine to handle WebSocket connection upgrades RewriteEngine On RewriteCond %{HTTP:Connection} Upgrade [NC] RewriteCond %{HTTP:Upgrade} websocket [NC] @@ -224,7 +224,7 @@ Listen 443 ``` -In case of the need to run the jupyterhub under /jhub/ or other location please use the below configurations: +In case of the need to run JupyterHub under /jhub/ or another location please use the below configurations: - JupyterHub running locally at http://127.0.0.1:8000/jhub/ or other location @@ -241,7 +241,7 @@ httpd.conf amendments: jupyterhub_config.py amendments: ```bash - --The public facing URL of the whole JupyterHub application. - --This is the address on which the proxy will bind. Sets protocol, ip, base_url + --The public-facing URL of the whole JupyterHub application. + --This is the address on which the proxy will bind. Sets protocol, IP, base_url c.JupyterHub.bind_url = 'http://127.0.0.1:8000/jhub/' ``` From 0f5f0f0df9b02e75f46c906838a23b720432a898 Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Wed, 12 Oct 2022 18:58:28 +0500 Subject: [PATCH 041/214] update index.rst Made the requested changes --- docs/source/index.rst | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index 96b56485..4f98070e 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -2,7 +2,8 @@ JupyterHub ========== -`JupyterHub`_ is the best way to serve `Jupyter notebook`_ for multiple users. The distributed Jupyter Notebook environment allows +`JupyterHub`_ is the best way to serve `Jupyter notebook`_ for multiple users. +Because JupyterHub manages a separate Jupyter environment for each user, it to be used in a class of students, a corporate data science group, or a scientific research group. It is a multi-user **Hub** that spawns, manages, and proxies multiple instances of the single-user `Jupyter notebook`_ server. @@ -18,9 +19,9 @@ original system of `JupyterHub`_. As of now, you can find two main cases: Four subsystems make up JupyterHub: * a **Hub** (tornado process) that is the heart of JupyterHub -* a **configurable HTTP proxy** (node-http-proxy) that receives the requests from the client's browser -* multiple **single-user Jupyter notebook servers** (Python/IPython/tornado) that are monitored by Spawners -* an **authentication class** that manages how users can access the system +* a **Configurable HTTP Proxy** (node-http-proxy) that receives the requests from the client's browser +* multiple **Single-User Jupyter Notebook Servers** (Python/IPython/tornado) that are monitored by Spawners +* an **Authentication Class** that manages how users can access the system Besides these central pieces, you can add optional configurations through a `config.py` file and manage users' environments through an admin panel. A simplification of the whole system can be seen in the figure below: From 2e6949c6e19487aabd19413327e679dc2e50abf9 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Wed, 12 Oct 2022 14:59:31 +0100 Subject: [PATCH 042/214] add reference for github oauth config with jupyter add link to the reference documentation for github oauth configuration with jupyterhub. Fix typos --- docs/source/reference/config-ghoauth.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/source/reference/config-ghoauth.md b/docs/source/reference/config-ghoauth.md index f6aa5ca4..588b1f93 100644 --- a/docs/source/reference/config-ghoauth.md +++ b/docs/source/reference/config-ghoauth.md @@ -5,15 +5,15 @@ deployment with the following assumptions: - Running JupyterHub on a single cloud server - Using SSL on the standard HTTPS port 443 -- Using GitHub OAuth (using oauthenticator) for login +- Using GitHub OAuth (using OAuthenticator) for login - Using the default spawner (to configure other spawners, uncomment and edit `spawner_class` as well as follow the instructions for your desired spawner) - Users exist locally on the server - Users' notebooks to be served from `~/assignments` to allow users to browse for notebooks within other users' home directories - You want the landing page for each user to be a `Welcome.ipynb` notebook in - their assignments directory. -- All runtime files are put into `/srv/jupyterhub` and log files in `/var/log`. + their assignments directory +- All runtime files are put into `/srv/jupyterhub` and log files in `/var/log` The `jupyterhub_config.py` file would have these settings: @@ -69,7 +69,7 @@ c.Spawner.args = ['--NotebookApp.default_url=/notebooks/Welcome.ipynb'] ``` Using the GitHub Authenticator requires a few additional -environment variable to be set prior to launching JupyterHub: +environment variables to be set prior to launching JupyterHub: ```bash export GITHUB_CLIENT_ID=github_id @@ -79,3 +79,5 @@ export CONFIGPROXY_AUTH_TOKEN=super-secret # append log output to log file /var/log/jupyterhub.log jupyterhub -f /etc/jupyterhub/jupyterhub_config.py &>> /var/log/jupyterhub.log ``` + +Visit the [Github OAuthenticator reference](https://oauthenticator.readthedocs.io/en/latest/api/gen/oauthenticator.github.html) to see the full list of options for configuring Github OAuth with JupyterHub. From 99255b04ac3de7328c14ed117e4713c010b12963 Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Wed, 12 Oct 2022 18:59:56 +0500 Subject: [PATCH 043/214] fix typo --- docs/source/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index 4f98070e..98eaefb2 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -4,7 +4,7 @@ JupyterHub `JupyterHub`_ is the best way to serve `Jupyter notebook`_ for multiple users. Because JupyterHub manages a separate Jupyter environment for each user, -it to be used in a class of students, a corporate data science group, or a scientific +it can be used in a class of students, a corporate data science group, or a scientific research group. It is a multi-user **Hub** that spawns, manages, and proxies multiple instances of the single-user `Jupyter notebook`_ server. From 5018c13b8194a22a818599636d7d29c3283cd582 Mon Sep 17 00:00:00 2001 From: Shloka Date: Wed, 12 Oct 2022 19:31:04 +0530 Subject: [PATCH 044/214] Fixes --- docs/source/contributing/tests.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/source/contributing/tests.rst b/docs/source/contributing/tests.rst index 35561e37..bc3fc7a5 100644 --- a/docs/source/contributing/tests.rst +++ b/docs/source/contributing/tests.rst @@ -13,7 +13,7 @@ Running the tests ================== #. Make sure you have completed :ref:`contributing/setup`. Once completed, you should be able - to start ``jupyterhub`` through a web browser as well as the command line. By doing this, it is ensured that the dev environment is properly set + to run ``jupyterhub`` on your command line and access JupyterHub from your browser at http://localhost:8000. By doing this, it is ensured that the dev environment is properly set up for tests to run. #. You can run all tests in JupyterHub @@ -104,8 +104,7 @@ Troubleshooting Test Failures All the tests are failing ------------------------- -Make sure you have completed all the steps in :ref:`contributing/setup` successfully, and -can launch ``jupyterhub`` from the terminal as well as the web browser. +Make sure you have completed all the steps in :ref:`contributing/setup` successfully, and are able to can access JupyterHub from your browser at http://localhost:8000 after starting ``jupyterhub`` in your command line. Code formatting and linting From 8060003fd6a951c19fc197999c1df976c344896e Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Wed, 12 Oct 2022 20:12:20 +0500 Subject: [PATCH 045/214] update spawners-basics.md --- docs/source/getting-started/spawners-basics.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/getting-started/spawners-basics.md b/docs/source/getting-started/spawners-basics.md index 9988c2f8..92d1a895 100644 --- a/docs/source/getting-started/spawners-basics.md +++ b/docs/source/getting-started/spawners-basics.md @@ -1,12 +1,12 @@ # Spawners and single-user notebook servers -Since the single-user server is an instance of `jupyter notebook`, an entire separate -multi-process application, there are many aspects of that server that can be configured, and a lot +Since the single-user server is an instance of `jupyter notebook`, an entirely separate +multi-process application. There are many aspects of that server that can be configured and a lot of ways to express that configuration. At the JupyterHub level, you can set some values on the Spawner. The simplest of these is `Spawner.notebook_dir`, which lets you set the root directory for a user's server. This root -notebook directory is the highest level directory users will be able to access in the notebook +notebook directory is the highest-level directory users will be able to access in the notebook dashboard. In this example, the root notebook directory is set to `~/notebooks`, where `~` is expanded to the user's home directory. @@ -20,7 +20,7 @@ You can also specify extra command line arguments to the notebook server with: c.Spawner.args = ['--debug', '--profile=PHYS131'] ``` -This could be used to set the users default page for the single user server: +This could be used to set the user's default page for the single-user server: ```python c.Spawner.args = ['--NotebookApp.default_url=/notebooks/Welcome.ipynb'] @@ -30,4 +30,4 @@ Since the single-user server extends the notebook server application, it still loads configuration from the `jupyter_notebook_config.py` config file. Each user may have one of these files in `$HOME/.jupyter/`. Jupyter also supports loading system-wide config files from `/etc/jupyter/`, -which is the place to put configuration that you want to affect all of your users. +which is the place to put the configuration that you want to affect all of your users. From c2d1a21d3213da6311f4c22754419375664dc329 Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Wed, 12 Oct 2022 20:18:18 +0500 Subject: [PATCH 046/214] update spawners-basics.md --- docs/source/getting-started/spawners-basics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/getting-started/spawners-basics.md b/docs/source/getting-started/spawners-basics.md index 92d1a895..2b3dc78e 100644 --- a/docs/source/getting-started/spawners-basics.md +++ b/docs/source/getting-started/spawners-basics.md @@ -1,7 +1,7 @@ # Spawners and single-user notebook servers Since the single-user server is an instance of `jupyter notebook`, an entirely separate -multi-process application. There are many aspects of that server that can be configured and a lot +multi-process application, there are many aspects of that server that can be configured and a lot of ways to express that configuration. At the JupyterHub level, you can set some values on the Spawner. The simplest of these is From a10879f493410531c87885a3b1b4247bcac38eeb Mon Sep 17 00:00:00 2001 From: Chidinma Udo <97461848+Mackenzie-OO7@users.noreply.github.com> Date: Wed, 12 Oct 2022 16:20:18 +0100 Subject: [PATCH 047/214] Modifications to URLs docs --- docs/source/reference/urls.md | 86 +++++++++++++++++------------------ 1 file changed, 42 insertions(+), 44 deletions(-) diff --git a/docs/source/reference/urls.md b/docs/source/reference/urls.md index d71feaf2..8e4ec0f3 100644 --- a/docs/source/reference/urls.md +++ b/docs/source/reference/urls.md @@ -2,13 +2,13 @@ This document describes how JupyterHub routes requests. -This does not include the [REST API](./rest.md) urls. +This does not include the [REST API](./rest.md) URLs. In general, all URLs can be prefixed with `c.JupyterHub.base_url` to run the whole JupyterHub application on a prefix. -All authenticated handlers redirect to `/hub/login` to login users -prior to being redirected back to the originating page. +All authenticated handlers redirect to `/hub/login` to log-in users +before being redirected back to the originating page. The returned request should preserve all query parameters. ## `/` @@ -25,12 +25,12 @@ This is an authenticated URL. This handler redirects users to the default URL of the application, which defaults to the user's default server. -That is, it redirects to `/hub/spawn` if the user's server is not running, -or the server itself (`/user/:name`) if the server is running. +That is, the handler redirects to `/hub/spawn` if the user's server is not running, +or to the server itself (`/user/:name`) if the server is running. -This default url behavior can be customized in two ways: +This default URL behavior can be customized in two ways: -To redirect users to the JupyterHub home page (`/hub/home`) +First, to redirect users to the JupyterHub home page (`/hub/home`) instead of spawning their server, set `redirect_to_server` to False: @@ -40,7 +40,7 @@ c.JupyterHub.redirect_to_server = False This might be useful if you have a Hub where you expect users to be managing multiple server configurations -and automatic spawning is not desirable. +but automatic spawning is not desirable. Second, you can customise the landing page to any page you like, such as a custom service you have deployed e.g. with course information: @@ -57,7 +57,7 @@ By default, the Hub home page has just one or two buttons for starting and stopping the user's server. If named servers are enabled, there will be some additional -tools for management of named servers. +tools for management of the named servers. _Version added: 1.0_ named server UI is new in 1.0. @@ -65,34 +65,34 @@ _Version added: 1.0_ named server UI is new in 1.0. This is the JupyterHub login page. If you have a form-based username+password login, -such as the default PAMAuthenticator, +such as the default [PAMAuthenticator](https://en.wikipedia.org/wiki/Pluggable_authentication_module), this page will render the login form. ![A login form](../images/login-form.png) If login is handled by an external service, e.g. with OAuth, this page will have a button, -declaring "Login with ..." which users can click -to login with the chosen service. +declaring "Log in with ..." which users can click +to log in with the chosen service. ![A login redirect button](../images/login-button.png) -If you want to skip the user-interaction to initiate logging in -via the button, you can set +If you want to skip the user interaction and initiate login +via the button, you can set: ```python c.Authenticator.auto_login = True ``` -This can be useful when the user is "already logged in" via some mechanism, -but a handshake via redirects is necessary to complete the authentication with JupyterHub. +This can be useful when the user is "already logged in" via some mechanism. +However, a handshake via `redirects` is necessary to complete the authentication with JupyterHub. ## `/hub/logout` -Visiting `/hub/logout` clears cookies from the current browser. +Visiting `/hub/logout` clears [cookies](https://en.wikipedia.org/wiki/HTTP_cookie) from the current browser. Note that **logging out does not stop a user's server(s)** by default. -If you would like to shutdown user servers on logout, +If you would like to shut down user servers on logout, you can enable this behavior with: ```python @@ -105,8 +105,8 @@ does not mean the user is no longer actively using their server from another mac ## `/user/:username[/:servername]` If a user's server is running, this URL is handled by the user's given server, -not the Hub. -The username is the first part and, if using named servers, +not by the Hub. +The username is the first part, and if using named servers, the server name is the second part. If the user's server is _not_ running, this will be redirected to `/hub/user/:username/...` @@ -120,11 +120,11 @@ if the specified server were running). Handling this URL is the most complicated condition in JupyterHub, because there can be many states: -1. server is not active +1. the server is not active a. user matches b. user doesn't match -2. server is ready -3. server is pending, but not ready +2. the server is ready +3. the server is pending, but not ready If the server is pending spawn, the browser will be redirected to `/hub/spawn-pending/:username/:servername` @@ -140,39 +140,37 @@ Some checks are performed and a delay is added before redirecting back to `/user If something is really wrong, this can result in a redirect loop. Visiting this page will never result in triggering the spawn of servers -without additional user action (i.e. clicking the link on the page) +without additional user action (i.e. clicking the link on the page). ![Visiting a URL for a server that's not running](../images/not-running.png) _Version changed: 1.0_ -Prior to 1.0, this URL itself was responsible for spawning servers, -and served the progress page if it was pending, -redirected to running servers, and -This was useful because it made sure that requested servers were restarted after they stopped, -but could also be harmful because unused servers would continuously be restarted if e.g. -an idle JupyterLab frontend were open pointed at it, -which constantly makes polling requests. +Prior to 1.0, this URL itself was responsible for spawning servers. +If the progress page was pending, the URL redirected it to running servers. +This was useful because it made sure that the requested servers were restarted after they stopped. +However, it could also be harmful because unused servers would continuously be restarted if e.g. +an idle JupyterLab frontend that constantly makes polling requests was openly pointed at it. ### Special handling of API requests Requests to `/user/:username[/:servername]/api/...` are assumed to be from applications connected to stopped servers. -These are failed with 503 and an informative JSON error message -indicating how to spawn the server. -This is meant to help applications such as JupyterLab +These requests fail with a `503` status code and an informative JSON error message +that indicates how to spawn the server. +This is meant to help applications such as JupyterLab, that are connected to a server that has stopped. _Version changed: 1.0_ -JupyterHub 0.9 failed these API requests with status 404, -but 1.0 uses 503. +JupyterHub version 0.9 failed these API requests with status `404`, +but version 1.0 uses 503. ## `/user-redirect/...` -This URL is for sharing a URL that will redirect a user +The `/user-redirect/...` URL is for sharing a URL that will redirect a user to a path on their own default server. -This is useful when users have the same file at the same URL on their servers, +This is useful when different users have the same file at the same URL on their servers, and you want a single link to give to any user that will open that file on their server. e.g. a link to `/user-redirect/notebooks/Index.ipynb` @@ -194,7 +192,7 @@ that is intended to make it possible. ### `/hub/spawn[/:username[/:servername]]` Requesting `/hub/spawn` will spawn the default server for the current user. -If `username` and optionally `servername` are specified, +If the `username` and optionally `servername` are specified, then the specified server for the specified user will be spawned. Once spawn has been requested, the browser is redirected to `/hub/spawn-pending/...`. @@ -207,7 +205,7 @@ and a POST request will trigger the actual spawn and redirect. _Version added: 1.0_ -1.0 adds the ability to specify username and servername. +1.0 adds the ability to specify `username` and `servername`. Prior to 1.0, only `/hub/spawn` was recognized for the default server. _Version changed: 1.0_ @@ -247,7 +245,7 @@ against the [JupyterHub REST API](./rest.md). Administrators can take various administrative actions from this page: -1. add/remove users -2. grant admin privileges -3. start/stop user servers -4. shutdown JupyterHub itself +- add/remove users +- grant admin privileges +- start/stop user servers +- shutdown JupyterHub itself From b15d56432dbc9cc2106418038d2310fb863fe435 Mon Sep 17 00:00:00 2001 From: Deborah Udoh Date: Wed, 12 Oct 2022 16:21:07 +0100 Subject: [PATCH 048/214] Proofread and improve security-basics.rst --- .../getting-started/security-basics.rst | 71 +++++++++---------- 1 file changed, 35 insertions(+), 36 deletions(-) diff --git a/docs/source/getting-started/security-basics.rst b/docs/source/getting-started/security-basics.rst index 661b3832..b87df593 100644 --- a/docs/source/getting-started/security-basics.rst +++ b/docs/source/getting-started/security-basics.rst @@ -1,12 +1,12 @@ -Security settings +Security Settings ================= .. important:: You should not run JupyterHub without SSL encryption on a public network. -Security is the most important aspect of configuring Jupyter. Three -configuration settings are the main aspects of security configuration: +Security is the most important aspect of configuring Jupyter. +Three (3) configuration settings are the main aspects of security configuration: 1. :ref:`SSL encryption ` (to enable HTTPS) 2. :ref:`Cookie secret ` (a key for encrypting browser cookies) @@ -15,23 +15,23 @@ configuration settings are the main aspects of security configuration: The Hub hashes all secrets (e.g., auth tokens) before storing them in its database. A loss of control over read-access to the database should have -minimal impact on your deployment; if your database has been compromised, it +minimal impact on your deployment. If your database has been compromised, it is still a good idea to revoke existing tokens. .. _ssl-encryption: -Enabling SSL encryption +Enabling SSL Encryption ----------------------- Since JupyterHub includes authentication and allows arbitrary code execution, you should not run it without SSL (HTTPS). -Using an SSL certificate +Using an SSL Certificate ~~~~~~~~~~~~~~~~~~~~~~~~ This will require you to obtain an official, trusted SSL certificate or create a self-signed certificate. Once you have obtained and installed a key and -certificate you need to specify their locations in the ``jupyterhub_config.py`` +certificate, you need to specify their locations in the ``jupyterhub_config.py`` configuration file as follows: .. code-block:: python @@ -44,7 +44,7 @@ Some cert files also contain the key, in which case only the cert is needed. It is important that these files be put in a secure location on your server, where they are not readable by regular users. -If you are using a **chain certificate**, see also chained certificate for SSL +If you are using a **chain certificate**, see also, chained certificate for SSL in the JupyterHub `Troubleshooting FAQ <../troubleshooting.html>`_. Using letsencrypt @@ -72,17 +72,17 @@ would be the needed configuration: If SSL termination happens outside of the Hub ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -In certain cases, for example if the hub is running behind a reverse proxy, and +In certain cases, for example, if the hub is running behind a reverse proxy, and `SSL termination is being provided by NGINX `_, it is reasonable to run the hub without SSL. To achieve this, simply omit the configuration settings ``c.JupyterHub.ssl_key`` and ``c.JupyterHub.ssl_cert`` -(setting them to ``None`` does not have the same effect, and is an error). +(setting them to ``None`` does not have the same effect, but results in an error). .. _authentication-token: -Proxy authentication token +Proxy Authentication Token -------------------------- The Hub authenticates its requests to the Proxy using a secret token that @@ -92,9 +92,9 @@ use an auth token. The value of this token should be a random string (for example, generated by ``openssl rand -hex 32``). You can store it in the configuration file or an -environment variable +environment variable. -Generating and storing token in the configuration file +Generating and Storing Token in the Configuration File ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can set the value in the configuration file, ``jupyterhub_config.py``: @@ -103,7 +103,7 @@ You can set the value in the configuration file, ``jupyterhub_config.py``: c.ConfigurableHTTPProxy.api_token = 'abc123...' # any random string -Generating and storing as an environment variable +Generating and Storing as an Environment Variable ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can pass this value of the proxy authentication token to the Hub and Proxy @@ -115,29 +115,29 @@ using the ``CONFIGPROXY_AUTH_TOKEN`` environment variable: This environment variable needs to be visible to the Hub and Proxy. -Default if token is not set +Default if Token is Not Set ~~~~~~~~~~~~~~~~~~~~~~~~~~~ If you don't set the Proxy authentication token, the Hub will generate a random -key itself, which means that any time you restart the Hub you **must also +key itself. This means that anytime you restart the Hub, you **must also restart the Proxy**. If the proxy is a subprocess of the Hub, this should happen automatically (this is the default configuration). .. _cookie-secret: -Cookie secret +Cookie Secret ------------- -The cookie secret is an encryption key, used to encrypt the browser cookies +The cookie secret is an encryption key, used to encrypt the browser cookies, which are used for authentication. Three common methods are described for generating and configuring the cookie secret. -Generating and storing as a cookie secret file +Generating and Storing as a Cookie Secret File ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The cookie secret should be 32 random bytes, encoded as hex, and is typically -stored in a ``jupyterhub_cookie_secret`` file. An example command to generate the -``jupyterhub_cookie_secret`` file is: +stored in a ``jupyterhub_cookie_secret`` file. Below, is an example of a command to generate the +``jupyterhub_cookie_secret`` file: .. code-block:: bash @@ -155,10 +155,10 @@ The location of the ``jupyterhub_cookie_secret`` file can be specified in the If the cookie secret file doesn't exist when the Hub starts, a new cookie secret is generated and stored in the file. The file must not be readable by -``group`` or ``other`` or the server won't start. The recommended permissions +``group`` or ``other``, otherwise, the server won't start. The recommended permissions for the cookie secret file are ``600`` (owner-only rw). -Generating and storing as an environment variable +Generating and Storing as an Environment Variable ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If you would like to avoid the need for files, the value can be loaded in the @@ -173,11 +173,11 @@ For security reasons, this environment variable should only be visible to the Hub. If you set it dynamically as above, all users will be logged out each time the Hub starts. -Generating and storing as a binary string +Generating and Storing as a Binary String ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -You can also set the cookie secret in the configuration file -itself, ``jupyterhub_config.py``, as a binary string: +You can also set the cookie secret, as a binary string, +in the configuration file (``jupyterhub_config.py``) itself: .. code-block:: python @@ -185,7 +185,7 @@ itself, ``jupyterhub_config.py``, as a binary string: .. _cookies: -Cookies used by JupyterHub authentication +Cookies Used by JupyterHub Authentication ----------------------------------------- The following cookies are used by the Hub for handling user authentication. @@ -198,7 +198,7 @@ jupyterhub-hub-login ~~~~~~~~~~~~~~~~~~~~ This is the login token used when visiting Hub-served pages that are -protected by authentication such as the main home, the spawn form, etc. +protected by authentication, such as the main home, the spawn form, etc. If this cookie is set, then the user is logged in. Resetting the Hub cookie secret effectively revokes this cookie. @@ -209,7 +209,7 @@ jupyterhub-user- ~~~~~~~~~~~~~~~~~~~~~~~~~~ This is the cookie used for authenticating with a single-user server. -It is set by the single-user server after OAuth with the Hub. +It is set by the single-user server, after OAuth with the Hub. Effectively the same as ``jupyterhub-hub-login``, but for the single-user server instead of the Hub. It contains an OAuth access token, @@ -218,14 +218,13 @@ which is checked with the Hub to authenticate the browser. Each OAuth access token is associated with a session id (see ``jupyterhub-session-id`` section below). -To avoid hitting the Hub on every request, the authentication response -is cached. And to avoid a stale cache the cache key is comprised of both -the token and session id. +To avoid hitting the Hub on every request, the authentication response is cached. +The cache key is comprised of both the token and session id, to avoid a stale cache. Resetting the Hub cookie secret effectively revokes this cookie. -This cookie is restricted to the path ``/user/``, so that -only the user’s server receives it. +This cookie is restricted to the path ``/user/``, +to ensure that only the user’s server receives it. jupyterhub-session-id ~~~~~~~~~~~~~~~~~~~~~ @@ -235,7 +234,7 @@ shared by the Hub and single-user servers. Its sole purpose is to coordinate logout of the multiple OAuth cookies. -This cookie is set to ``/`` so all endpoints can receive it, or clear it, etc. +This cookie is set to ``/`` so all endpoints can receive or clear it, etc. jupyterhub-user--oauth-state ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -245,7 +244,7 @@ It is only set while OAuth between the single-user server and the Hub is processing. If you use your browser development tools, you should see this cookie -for a very brief moment before your are logged in, +for a very brief moment before you are logged in, with an expiration date shorter than ``jupyterhub-hub-login`` or ``jupyterhub-user-``. From f9dbfd7275062147ca988ab693678d6889aa4f15 Mon Sep 17 00:00:00 2001 From: Allan Wasega Date: Thu, 13 Oct 2022 08:58:42 +0300 Subject: [PATCH 049/214] Edited and restructured server-api file --- docs/source/reference/server-api.md | 163 ++++++++++------------------ 1 file changed, 57 insertions(+), 106 deletions(-) diff --git a/docs/source/reference/server-api.md b/docs/source/reference/server-api.md index 2ee9bfcf..3fbc8e9f 100644 --- a/docs/source/reference/server-api.md +++ b/docs/source/reference/server-api.md @@ -1,37 +1,29 @@ # Starting servers with the JupyterHub API -JupyterHub's [REST API][] allows launching servers on behalf of users -without ever interacting with the JupyterHub UI. -This allows you to build services launching Jupyter-based services for users -without relying on the JupyterHub UI at all, -enabling a variety of user/launch/lifecycle patterns not natively supported by JupyterHub, -without needing to develop all the server management features of JupyterHub Spawners and/or Authenticators. -[BinderHub][] is an example of such an application. +Sometimes, such as when working with applications such as [BinderHub](https://binderhub.readthedocs.io), it may be necessary to launch Jupyter-based services on behalf of your users. Doing so can be achieved through JupyterHub's [REST API](../reference/rest.md), which allows one to launch and manage servers on behalf of users through API calls instead of through the JupyterHub UI. In doing so, you can take advantage of other user/launch/lifecycle patterns that are not natively supported by JupyterHub, all without the need to develop the server management features of JupyterHub Spawners and/or Authenticators. -[binderhub]: https://binderhub.readthedocs.io -[rest api]: ../reference/rest.md +This tutorial goes through the processes involved while working with the JupyterHub API to manage servers for users. In particular, it covers how to: -This document provides an example of working with the JupyterHub API to -manage servers for users. -In particular, we will cover how to: +1. [Check the status of servers](checking) +2. [Start servers](starting) +3. [Wait for servers to be ready](waiting) +4. [Communicate with servers](communicating) +5. [Stop servers](stopping) -1. [check status of servers](checking) -2. [start servers](starting) -3. [wait for servers to be ready](waiting) -4. [communicate with servers](communicating) -5. [stop servers](stopping) +In the end, we also provide a sample Python code that can be used to implement these steps. (checking)= ## Checking server status -Requesting information about a user includes a `servers` field, -which is a dictionary. +First, request information about a particular user using a GET request as below. ``` GET /hub/api/users/:username ``` +The response you get will include a `servers` field, which is a dictionary, as shown in the JSON-formatted response below. + **Required scope: `read:servers`** ```json @@ -49,13 +41,9 @@ GET /hub/api/users/:username } ``` -If the `servers` dict is empty, the user has no running servers. -The keys of the `servers` dict are server names as strings. -Many JupyterHub deployments only use the 'default' server, -which has the empty string `''` for a name. -In this case, the servers dict will always have either zero or one elements. +Many JupyterHub deployments only use a 'default' server, represented as an empty string `''` for a name. An investigation of the `servers` field can yield one of two results. First, it can be empty as in the sample JSON response above. In such a case, the user has no running servers. -This is the servers dict when the user's default server is fully running and ready: +However, should the user have running servers, then the returned dict should contain various information, as shown in the response below. ```json "servers": { @@ -75,34 +63,28 @@ This is the servers dict when the user's default server is fully running and rea Key properties of a server: name -: the server's name. Always the same as the key in `servers` +: the server's name. Always the same as the key in `servers`. ready : boolean. If true, the server can be expected to respond to requests at `url`. pending : `null` or a string indicating a transitional state (such as `start` or `stop`). -Will always be `null` if `ready` is true, -and will always be a string if `ready` is false. +Will always be `null` if `ready` is true or a string if false. url -: The server's url (just the path, e.g. `/users/:name/:servername/`) -where the server can be accessed if `ready` is true. +: The server's url path (e.g. `/users/:name/:servername/`) where the server can be accessed if `ready` is true. progress_url -: The API url path (starting with `/hub/api`) -where the progress API can be used to wait for the server to be ready. -See below for more details on the progress API. +: The API URL path (starting with `/hub/api`) where the progress API can be used to wait for the server to be ready. last_activity -: ISO8601 timestamp indicating when activity was last observed on the server +: ISO8601 timestamp indicating when activity was last observed on the server. started -: ISO801 timestamp indicating when the server was last started +: ISO801 timestamp indicating when the server was last started. -We've seen the `servers` model with no servers and with one `ready` server. -Here is what it looks like immediately after requesting a server launch, -while the server is not ready yet: +The two responses above are from a user with no servers and another with one `ready` server. The sample below is a response likely to be received when one requests a server launch while the server is not yet ready: ```json "servers": { @@ -119,11 +101,7 @@ while the server is not ready yet: } ``` -Note that `ready` is false and `pending` is `spawn`. -This means that the server is not ready -(attempting to access it may not work) -because it isn't finished spawning yet. -We'll get more into that below in [waiting for a server][]. +Note that `ready` is assigned `false` and `pending` is assigned `spawn`, meaning that the server is not ready and attempting to access it may not work as it is still in the process of spawning. [waiting for a server]: waiting @@ -131,7 +109,7 @@ We'll get more into that below in [waiting for a server][]. ## Starting servers -To start a server, make the request +To start a server, make the following API request: ``` POST /hub/api/users/:username/servers/[:servername] @@ -139,47 +117,34 @@ POST /hub/api/users/:username/servers/[:servername] **Required scope: `servers`** -(omit servername for the default server) - -Assuming the request was valid, -there are two possible responses: +Assuming the request was valid, there are two possible responses: 201 Created -: This status code means the launch completed and the server is ready. -It should be available at the server's URL immediately. +: This status code means the launch completed and the server is ready and is available at the server's URL immediately. 202 Accepted -: This is the more likely response, -and means that the server has begun launching, -but isn't immediately ready. -The server has `pending: 'spawn'` at this point. - -_Aside: how quickly JupyterHub responds with `202 Accepted` is governed by the `slow_spawn_timeout` tornado setting._ +: This is the more likely response, and means that the server has begun launching, +but is not immediately ready. As a result, the server shows `pending: 'spawn'` at this point and you should wait for it to start. (waiting)= -## Waiting for a server +## Waiting for a server to start -If you are starting a server via the API, -there's a good change you want to know when it's ready. -There are two ways to do with: +After receiving a `202 Accepted` response, you have to wait for it to start. Two approaches can be applied to establish when the server is ready: 1. {ref}`Polling the server model ` -2. the {ref}`progress API ` +2. {ref}`Using the Progress API ` (polling)= ### Polling the server model -The simplest way to check if a server is ready -is to request the user model. +The simplest way to check if a server is ready is to programmatically query the server model until the following two conditions are true: -If: +: 1. The server name is contained in the `servers` response, and +: 2. `servers['servername']['ready']` is true -1. the server name is in the user's `servers` model, and -2. `servers['servername']['ready']` is true - -A Python example, checking if a server is ready: +The Python code snippet below can be used to check if a server is ready: ```python def server_ready(hub_url, user, server_name="", token): @@ -202,18 +167,15 @@ def server_ready(hub_url, user, server_name="", token): return False ``` -You can keep making this check until `ready` is true. +The function polls the server until `ready` is true. (progress)= -### Progress API +### Using the Progress API -The most _efficient_ way to wait for a server to start is the progress API. +The most _efficient_ way to wait for a server to start is by using the Progress API. The progress URL is available in the server model under `progress_url` and has the form `/hub/api/users/:user/servers/:servername/progress`. -The progress URL is available in the server model under `progress_url`, -and has the form `/hub/api/users/:user/servers/:servername/progress`. - -_the default server progress can be accessed at `:user/servers//progress` or `:user/server/progress`_ +The default server progress can be accessed at `:user/servers//progress` or `:user/server/progress` as demonstrated in the following GET request: ``` GET /hub/api/users/:user/servers/:servername/progress @@ -221,8 +183,7 @@ GET /hub/api/users/:user/servers/:servername/progress **Required scope: `read:servers`** -This is an [EventStream][] API. -In an event stream, messages are _streamed_ and delivered on lines of the form: +The Progress API is an example of a [EventStream][] API. Therefore, as in a typical event stream, messages are _streamed_ and delivered in the form: ``` data: {"progress": 10, "message": "...", ...} @@ -233,7 +194,7 @@ Lines that do not start with `data:` should be ignored. [eventstream]: https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#examples -progress events have the form: +Progress events have the form: ```python { @@ -254,11 +215,9 @@ ready : present and true only for the last event when the server is ready url -: only present if `ready` is true; will be the server's url +: only present if `ready` is true; will be the server's URL -the progress API can be used even with fully ready servers. -If the server is ready, -there will only be one event that looks like: +The Progress API can be used even with fully ready servers. In such a situation, there will only be one event response of the form: ```json { @@ -270,9 +229,9 @@ there will only be one event that looks like: } ``` -where `ready` and `url` are the same as in the server model (`ready` will always be true). +In this case, `ready` and `url` are the same as in the server model, and `ready` will always be true. -A typical complete stream from the event-stream API: +A significant advantage of the Progress API is that it shows the status of the server through a stream of messages. An example of a typical complete stream from the API is as shown below: ``` @@ -300,23 +259,19 @@ Servers can be stopped with a DELETE request: DELETE /hub/api/users/:user/servers/[:servername] ``` -**Required scope: `servers`** - -Like start, delete may not complete immediately. -The DELETE request has two possible response codes: +Similar to when starting a server, issuing the DELETE request above might not stop the server immediately. Instead, the DELETE request has two possible response codes: 204 Deleted : This status code means the delete completed and the server is fully stopped. It will now be absent from the user `servers` model. 202 Accepted -: Like start, `202` means your request was accepted, -but is not yet complete. +: This code means your request was accepted, but is not yet completely processed. The server has `pending: 'stop'` at this point. -Unlike start, there is no progress API for stop. -To wait for stop to finish, you must poll the user model -and wait for the server to disappear from the user `servers` model. +There is no Progress API for checking when a server actually stops. Thus, the only available alternative is to poll the server and wait for it to disappear from the user `servers` model. + +The Python code snippet below can be used to check if a server stops: ```{literalinclude} ../../../examples/server-api/start-stop-server.py :language: python @@ -327,9 +282,7 @@ and wait for the server to disappear from the user `servers` model. ## Communicating with servers -JupyterHub tokens with the the `access:servers` scope -can be used to communicate with servers themselves. -This can be the same token you used to launch your service. +JupyterHub tokens with the `access:servers` scope can be used to communicate with servers themselves. The tokens can be the same as those you used to launch your service. ```{note} Access scopes are new in JupyterHub 2.0. @@ -338,29 +291,27 @@ a token must be owned by the same user as the server, *or* be an admin token if admin_access is enabled. ``` -The URL returned from a server model is the url path suffix, +The URL returned from a server model is the URL path suffix, e.g. `/user/:name/` to append to the jupyterhub base URL. For instance, `{hub_url}{server_url}`, -where `hub_url` would be e.g. `http://127.0.0.1:8000` by default, -and `server_url` `/user/myname`, -for a full url of `http://127.0.0.1:8000/user/myname`. +where `hub_url` would be such as `http://127.0.0.1:8000` by default, +and `server_url` is `/user/myname.` When combined, the two give a full URL of `http://127.0.0.1:8000/user/myname`. ## Python example The JupyterHub repo includes a complete example in {file}`examples/server-api` -tying all this together. +tying all the above mentioned steps together. -To summarize the steps: +In summary, the steps involved while managing servers on behalf of users are: -1. get user info from `/user/:name` +1. get user information from `/user/:name` 2. the server model includes a `ready` state to tell you if it's ready 3. if it's not ready, you can follow up with `progress_url` to wait for it 4. if it is ready, you can use the `url` field to link directly to the running server -The example demonstrates starting and stopping servers via the JupyterHub API, -including waiting for them to start via the progress API, -as well as waiting for them to stop via polling the user model. +The example below demonstrates starting and stopping servers via the JupyterHub API, +including waiting for them to start via the Progress API and waiting for them to stop via polling the user model. ```{literalinclude} ../../../examples/server-api/start-stop-server.py :language: python From 67b62db52455da6b23966d37fcca2bd1ebe466f2 Mon Sep 17 00:00:00 2001 From: Esther Christopher Date: Thu, 13 Oct 2022 13:04:42 +0100 Subject: [PATCH 050/214] Reviewed the documentation 1. Edited the topic of the documentation to Pascal Case. 2. Completed a sentence for clarity --- docs/source/events/index.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/events/index.rst b/docs/source/events/index.rst index bc086ba1..b6303263 100644 --- a/docs/source/events/index.rst +++ b/docs/source/events/index.rst @@ -1,4 +1,4 @@ -Eventlogging and Telemetry +Event Logging and Telemetry ========================== JupyterHub can be configured to record structured events from a running server using Jupyter's `Telemetry System`_. The types of events that JupyterHub emits are defined by `JSON schemas`_ listed at the bottom of this page_. @@ -15,7 +15,7 @@ Event logging is handled by its ``Eventlog`` object. This leverages Python's sta To begin recording events, you'll need to set two configurations: - 1. ``handlers``: tells the EventLog *where* to route your events. This trait is a list of Python logging handlers that route events to + 1. ``handlers``: tells the EventLog *where* to route your events. This trait is a list of Python logging handlers that route events to the event log file. 2. ``allows_schemas``: tells the EventLog *which* events should be recorded. No events are emitted by default; all recorded events must be listed here. Here's a basic example: From 88906c2e1be6b05305d3fc418bb7505c5ae3891d Mon Sep 17 00:00:00 2001 From: Zeelyha <73789529+zeelyha@users.noreply.github.com> Date: Thu, 13 Oct 2022 15:13:30 +0100 Subject: [PATCH 051/214] Update templates.md Fixed a typo --- docs/source/reference/templates.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/reference/templates.md b/docs/source/reference/templates.md index 1509e0f2..f29708b9 100644 --- a/docs/source/reference/templates.md +++ b/docs/source/reference/templates.md @@ -84,5 +84,5 @@ template (for example, `login.html`) with: ``` Extending `page.html` puts the message on all pages, but note that -extending `page.html` take precedence over an extension of a specific +extending `page.html` takes precedence over an extension of a specific page (unlike the variable-based approach above). From 72096b51667f7d06e82bef7e62c0f2abe34ec8b3 Mon Sep 17 00:00:00 2001 From: Esther Christopher Date: Thu, 13 Oct 2022 17:00:49 +0100 Subject: [PATCH 052/214] Edited for Clarity Edited for sentence restructuring, sentence clarity, change of grammar, grammatical punctuation errors, and grammatical correctness. --- docs/source/admin/upgrading.rst | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index efe98c27..24a8ee5a 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -5,12 +5,12 @@ Upgrading JupyterHub JupyterHub offers easy upgrade pathways between minor versions. This document describes how to do these upgrades. -If you are using :ref:`a JupyterHub distribution `, you +If you use :ref:`a JupyterHub distribution `, you should consult the distribution's documentation on how to upgrade. This -document is if you have set up your own JupyterHub without using a +document is applicable if you have set up your own JupyterHub without using a distribution. -It is long because is pretty detailed! Most likely, upgrading +This documentation is lengthy because it is quite detailed! Most likely, upgrading JupyterHub is painless, quick and with minimal user interruption. Read the Changelog @@ -19,21 +19,20 @@ Read the Changelog The `changelog <../changelog.html>`_ contains information on what has changed with the new JupyterHub release, and any deprecation warnings. Read these notes to familiarize yourself with the coming changes. There -might be new releases of authenticators & spawners you are using, so +might be new releases for the authenticators & spawners you use, so read the changelogs for those too! Notify your users ================= -If you are using the default configuration where ``configurable-http-proxy`` +If you use the default configuration where ``configurable-http-proxy`` is managed by JupyterHub, your users will see service disruption during the upgrade process. You should notify them, and pick a time to do the upgrade where they will be least disrupted. -If you are using a different proxy, or running ``configurable-http-proxy`` +If you use a different proxy, or running ``configurable-http-proxy`` independent of JupyterHub, your users will be able to continue using notebook -servers they had already launched, but will not be able to launch new servers -nor sign in. +servers they had already launched, but will not be able to launch new servers or sign in. Backup database & config @@ -41,11 +40,11 @@ Backup database & config Before doing an upgrade, it is critical to back up: -#. Your JupyterHub database (sqlite by default, or MySQL / Postgres - if you used those). If you are using sqlite (the default), you +#. Your JupyterHub database (SQLite by default, or MySQL / Postgres + if you used those). If you use SQLite (the default), you should backup the ``jupyterhub.sqlite`` file. #. Your ``jupyterhub_config.py`` file. -#. Your user's home directories. This is unlikely to be affected directly by +#. Your users' home directories. This is unlikely to be affected directly by a JupyterHub upgrade, but we recommend a backup since user data is very critical. @@ -54,7 +53,7 @@ Shutdown JupyterHub =================== Shutdown the JupyterHub process. This would vary depending on how you -have set up JupyterHub to run. Most likely, it is using a process +have set up JupyterHub to run. It is likely using a process supervisor of some sort (``systemd`` or ``supervisord`` or even ``docker``). Use the supervisor specific command to stop the JupyterHub process. @@ -118,7 +117,7 @@ SQLite database disadvantages SQLite has some disadvantages when it comes to upgrading JupyterHub. These are: -- ``upgrade-db`` may not work, and you may need delete your database +- ``upgrade-db`` may not work, and you may need to delete your database and start with a fresh one. - ``downgrade-db`` **will not** work if you want to rollback to an earlier version, so backup the ``jupyterhub.sqlite`` file before From 9f62c76a8e7dd8032204b7eb390eae51be905063 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 13 Oct 2022 22:41:04 +0000 Subject: [PATCH 053/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/reference/config-user-env.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/source/reference/config-user-env.md b/docs/source/reference/config-user-env.md index 814dd365..be0424bf 100644 --- a/docs/source/reference/config-user-env.md +++ b/docs/source/reference/config-user-env.md @@ -1,6 +1,5 @@ ![jupyterhub](https://jupyter.org/assets/homepage/hublogo.svg) - # Configuring user environments To deploy JupyterHub means you are providing Jupyter notebook environments for @@ -8,7 +7,7 @@ multiple users. Often, this includes a desire to configure the user environment in a custom way. Since the `jupyterhub-singleuser` server extends the standard Jupyter notebook -server, most Jupyter notebook configuration and documentation also applies to single-user environments. +server, most Jupyter notebook configuration and documentation also applies to single-user environments. Configuration of user environments typically does not occur through JupyterHub itself, but rather through system- wide Jupyter's configuration, which is inherited by `jupyterhub-singleuser`. From 35d26e75f417f7d8bdefad4a219e5dba828b2a91 Mon Sep 17 00:00:00 2001 From: Alexander Chosen okon Date: Thu, 13 Oct 2022 23:20:10 +0000 Subject: [PATCH 054/214] Documentation reviewed, made concise & image added --- docs/source/reference/config-user-env.md | 80 ++++++++++++------------ 1 file changed, 41 insertions(+), 39 deletions(-) diff --git a/docs/source/reference/config-user-env.md b/docs/source/reference/config-user-env.md index 503d2661..814dd365 100644 --- a/docs/source/reference/config-user-env.md +++ b/docs/source/reference/config-user-env.md @@ -1,21 +1,23 @@ +![jupyterhub](https://jupyter.org/assets/homepage/hublogo.svg) + + # Configuring user environments -Deploying JupyterHub means you are providing Jupyter notebook environments for +To deploy JupyterHub means you are providing Jupyter notebook environments for multiple users. Often, this includes a desire to configure the user -environment in some way. +environment in a custom way. Since the `jupyterhub-singleuser` server extends the standard Jupyter notebook -server, most configuration and documentation that applies to Jupyter Notebook -applies to the single-user environments. Configuration of user environments -typically does not occur through JupyterHub itself, but rather through system- -wide configuration of Jupyter, which is inherited by `jupyterhub-singleuser`. +server, most Jupyter notebook configuration and documentation also applies to single-user environments. +Configuration of user environments typically does not occur through JupyterHub itself, but rather through system- +wide Jupyter's configuration, which is inherited by `jupyterhub-singleuser`. **Tip:** When searching for configuration tips for JupyterHub user -environments, try removing JupyterHub from your search because there are a lot +environments, you might want to remove JupyterHub from your search because there are a lot more people out there configuring Jupyter than JupyterHub and the configuration is the same. -This section will focus on user environments, including: +This section will focus on user environments, which includes the following: - Installing packages - Configuring Jupyter and IPython @@ -24,26 +26,26 @@ This section will focus on user environments, including: ## Installing packages -To make packages available to users, you generally will install packages +To make packages available to users, you will typically install packages system-wide or in a shared environment. -This installation location should always be in the same environment that -`jupyterhub-singleuser` itself is installed in, and must be _readable and +This installation location should always be in the same environment where +`jupyterhub-singleuser` itself is installed, and must be _readable and executable_ by your users. If you want users to be able to install additional packages, it must also be _writable_ by your users. -If you are using a standard system Python install, you would use: +If you are using a standard Python installation on your system, use the following command: ```bash sudo python3 -m pip install numpy ``` -to install the numpy package in the default system Python 3 environment +to install the numpy package in the default Python 3 environment on your system (typically `/usr/local`). -You may also use conda to install packages. If you do, you should make sure -that the conda environment has appropriate permissions for users to be able to -run Python code in the env. +Alternatively, You may also use conda to install packages. To do this, ensure +that the conda environment has appropriate users permissions needed to +run Python code in the environment. ## Configuring Jupyter and IPython @@ -52,13 +54,13 @@ and [IPython](https://ipython.readthedocs.io/en/stable/development/config.html) have their own configuration systems. As a JupyterHub administrator, you will typically want to install and configure -environments for all JupyterHub users. For example, you wish for each student in +environments for all JupyterHub users. For example, let's say you wish for each student in a class to have the same user environment configuration. Jupyter and IPython support **"system-wide"** locations for configuration, which is the logical place to put global configuration that you want to affect all users. It's generally more efficient to configure user environments "system-wide", -and it's a good idea to avoid creating files in users' home directories. +and it's a good practice to avoid creating files in the users' home directories. The typical locations for these config files are: @@ -84,8 +86,8 @@ If you are using the classing Jupyter Notebook server, the same things should work, with the following substitutions: -- Where you see `jupyter_server_config`, use `jupyter_notebook_config` -- Where you see `NotebookApp`, use `ServerApp` +- Search for `jupyter_server_config`, and replace with `jupyter_notebook_config` +- Search for `NotebookApp`, and replace with `ServerApp` ::: @@ -113,7 +115,7 @@ Jupyter kernelspec installation is system wide by default, but some kernels may default to installing kernelspecs in your home directory. These will need to be moved system-wide to ensure that they are accessible. -You can see where your kernelspecs are with: +To see where your kernelspecs are, you can use the following command: ```bash jupyter kernelspec list @@ -121,8 +123,8 @@ jupyter kernelspec list ### Example: Installing kernels system-wide -Assuming I have a Python 2 and Python 3 environment that I want to make -sure are available, I can install their specs system-wide (in /usr/local) with: +Let's assume that I have a Python 2 and Python 3 environment that I want to make +sure are available, I can install their specs **system-wide** (in /usr/local) using the following command: ```bash /path/to/python3 -m ipykernel install --prefix=/usr/local @@ -141,31 +143,31 @@ How you configure user environments for each category can differ a bit depending on what Spawner you are using. The first category is a **shared system (multi-user host)** where -each user has a JupyterHub account and a home directory as well as being +each user has a JupyterHub account, a home directory as well as being a real system user. In this example, shared configuration and installation -must be in a 'system-wide' location, such as `/etc/` or `/usr/local` +must be in a **system-wide** location, such as `/etc/` or `/usr/local` or a custom prefix such as `/opt/conda`. When JupyterHub uses **container-based** Spawners (e.g. KubeSpawner or -DockerSpawner), the 'system-wide' environment is really the container image -which you are using for users. +DockerSpawner), the 'system-wide' environment is the container image +which is used for users. In both cases, you want to _avoid putting configuration in user home directories_ because users can change those configuration settings. Also, -home directories typically persist once they are created, so they are +home directories typically persist once they are created, thereby making it difficult for admins to update later. ## Named servers -By default, in a JupyterHub deployment each user has exactly one server. +By default, in a JupyterHub deployment, each user has only one server. JupyterHub can, however, have multiple servers per user. -This is most useful in deployments where users can configure the environment +This is mostly useful in deployments where users can configure the environment in which their server will start (e.g. resource requests on an HPC cluster), so that a given user can have multiple configurations running at the same time, -without having to stop and restart their one server. +without having to stop and restart their own server. -To allow named servers: +To allow named servers, use the following command: ```python c.JupyterHub.allow_named_servers = True @@ -183,13 +185,13 @@ as well as the admin page: Named servers can be accessed, created, started, stopped, and deleted from these pages. Activity tracking is now per-server as well. -The number of named servers per user can be limited by setting a constant value: +To limit the number of **named server** per user by setting a constant value, use the following command: ```python c.JupyterHub.named_server_limit_per_user = 5 ``` -or a callable/awaitable based on the handler object: +Alternatively, to use a callable/awaitable based on the handler object, use the following command: ```python def named_server_limit_per_user_fn(handler): @@ -209,10 +211,10 @@ If `named_server_limit_per_user` is set to `0`, no limit is enforced. ## Switching back to classic notebook -By default the single-user server launches JupyterLab, +By default, the single-user server launches JupyterLab, which is based on [Jupyter Server][]. This is the default server when running JupyterHub ≥ 2.0. -You can switch to using the legacy Jupyter Notebook server by setting the `JUPYTERHUB_SINGLEUSER_APP` environment variable +To switch to using the legacy Jupyter Notebook server, you can set the `JUPYTERHUB_SINGLEUSER_APP` environment variable (in the single-user environment) to: ```bash @@ -227,15 +229,15 @@ JupyterLab is now the default singleuser UI, if available, which is based on the [Jupyter Server][], no longer the legacy [Jupyter Notebook][] server. JupyterHub prior to 2.0 launched the legacy notebook server (`jupyter notebook`), -and Jupyter server could be selected by specifying +and Jupyter server could be selected by specifying the following command: ```python # jupyterhub_config.py c.Spawner.cmd = ["jupyter-labhub"] ``` -or for an otherwise customized Jupyter Server app, -set the environment variable: +Alternatively, for an otherwise customized Jupyter Server app, +set the environment variable using the following command: ```bash export JUPYTERHUB_SINGLEUSER_APP='jupyter_server.serverapp.ServerApp' From 066745158456bf5fb352e32dd71079d31c5eea10 Mon Sep 17 00:00:00 2001 From: Eshy10 Date: Fri, 14 Oct 2022 02:55:27 +0100 Subject: [PATCH 055/214] chore: remove duplicate version statement and add color to environments title --- docs/source/admin/upgrading.rst | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index efe98c27..d7651d2d 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -63,11 +63,11 @@ Upgrade JupyterHub packages There are two environments where the ``jupyterhub`` package is installed: -#. The *hub environment*, which is where the JupyterHub server process +#. The *hub environment*: which is where the JupyterHub server process runs. This is started with the ``jupyterhub`` command, and is what people generally think of as JupyterHub. -#. The *notebook user environments*. This is where the user notebook +#. The *notebook user environments*: This is where the user notebook servers are launched from, and is probably custom to your own installation. This could be just one environment (different from the hub environment) that is shared by all users, one environment @@ -92,8 +92,6 @@ with: conda install -c conda-forge jupyterhub== -Where ```` is the version of JupyterHub you are upgrading to. - You should also check for new releases of the authenticator & spawner you are using. You might wish to upgrade those packages too along with JupyterHub, or upgrade them separately. From e6e1e903865cc6f4de5c232b0362eeaca4be7629 Mon Sep 17 00:00:00 2001 From: Eshy10 Date: Fri, 14 Oct 2022 03:04:08 +0100 Subject: [PATCH 056/214] chore: captitalize the first letter on explation of hub environment --- docs/source/admin/upgrading.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index d7651d2d..84e30cbf 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -63,7 +63,7 @@ Upgrade JupyterHub packages There are two environments where the ``jupyterhub`` package is installed: -#. The *hub environment*: which is where the JupyterHub server process +#. The *hub environment*: Which is where the JupyterHub server process runs. This is started with the ``jupyterhub`` command, and is what people generally think of as JupyterHub. From de3210536fb22f5151fffadce0d255a65ab6114b Mon Sep 17 00:00:00 2001 From: Ngobiri Falyne Date: Fri, 14 Oct 2022 04:53:09 +0100 Subject: [PATCH 057/214] Update troubleshooting.md Adjusted the Heading sudospawner - Sudospawner to suite sentence case Under proxy Settings, i adjusted the sentence "a organization to an organization" also singleuser to single-user under Toree intergration, the sentence is not clear so I adjusted to Toree kernel will raise and issue when running with jupyterHub, --- docs/source/troubleshooting.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index 0cecb6d9..abc6cde9 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -44,7 +44,7 @@ If you have tried to start the JupyterHub proxy and it fails to start: recent version of node. Some versions of Ubuntu/Debian come with a version of node that is very old, and it is necessary to update node. -### sudospawner fails to run +### Sudospawner fails to run If the sudospawner script is not found in the path, sudospawner will not run. To avoid this, specify sudospawner's absolute path. For example, start @@ -201,7 +201,7 @@ your server again. ##### Proxy settings (403 GET) -When your whole JupyterHub sits behind a organization proxy (_not_ a reverse proxy like NGINX as part of your setup and _not_ the configurable-http-proxy) the environment variables `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy` and `https_proxy` might be set. This confuses the jupyterhub-singleuser servers: When connecting to the Hub for authorization they connect via the proxy instead of directly connecting to the Hub on localhost. The proxy might deny the request (403 GET). This results in the singleuser server thinking it has a wrong auth token. To circumvent this you should add `,,localhost,127.0.0.1` to the environment variables `NO_PROXY` and `no_proxy`. +When your whole JupyterHub sits behind an organization proxy (_not_ a reverse proxy like NGINX as part of your setup and _not_ the configurable-http-proxy) the environment variables `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy` and `https_proxy` might be set. This confuses the jupyterhub-singleuser servers: When connecting to the Hub for authorization they connect via the proxy instead of directly connecting to the Hub on localhost. The proxy might deny the request (403 GET). This results in the single-user server thinking it has a wrong auth token. To circumvent this you should add `,,localhost,127.0.0.1` to the environment variables `NO_PROXY` and `no_proxy`. ### Launching Jupyter Notebooks to run as an externally managed JupyterHub service with the `jupyterhub-singleuser` command returns a `JUPYTERHUB_API_TOKEN` error @@ -388,7 +388,7 @@ jupyterhub --debug ### Toree integration with HDFS rack awareness script -The Apache Toree kernel will an issue, when running with JupyterHub, if the standard HDFS +The Apache Toree kernel will raise an issue when running with JupyterHub, if the standard HDFS rack awareness script is used. This will materialize in the logs as a repeated WARN: ```bash @@ -412,8 +412,8 @@ In order to resolve this issue, there are two potential options. Docker images can be found at the [JupyterHub organization on DockerHub](https://hub.docker.com/u/jupyterhub/). The Docker image [jupyterhub/singleuser](https://hub.docker.com/r/jupyterhub/singleuser/) -provides an example single user notebook server for use with DockerSpawner. +provides an example single-user notebook server for use with DockerSpawner. -Additional single user notebook server images can be found at the [Jupyter +Additional single-user notebook server images can be found at the [Jupyter organization on DockerHub](https://hub.docker.com/r/jupyter/) and information about each image at the [jupyter/docker-stacks repo](https://github.com/jupyter/docker-stacks). From 714b5925f68ef9452154828328c54a1537624887 Mon Sep 17 00:00:00 2001 From: Arafat Abdussalam Date: Fri, 14 Oct 2022 02:51:40 -0700 Subject: [PATCH 058/214] Correction of previous commit I made a correction to my previous commit as suggested. I will def try to avoid such mistakes. --- docs/source/admin/log-messages.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/admin/log-messages.md b/docs/source/admin/log-messages.md index c2974260..057692b6 100644 --- a/docs/source/admin/log-messages.md +++ b/docs/source/admin/log-messages.md @@ -1,12 +1,12 @@ # Interpreting common log messages -In this subsection of the documentation, you will get to understand the meaning of common log messages and how to resolve them. When debugging errors and outages, it is very helpful to look at the logs emitted by JupyterHub. +When debugging errors and outages, looking at the logs emitted by JupyterHub is very helpful This page documents some common log messages and what they mean. ## Failing suspected API request to not-running server ### Example -When your screen displays the log message as shown below, you need not to get scared. +Your logs might be littered with lines that might look scary ``` [W 2022-03-10 17:25:19.774 JupyterHub base:1349] Failing suspected API request to not-running server: /hub/user//api/metrics/v1 @@ -14,7 +14,7 @@ When your screen displays the log message as shown below, you need not to get sc ### Cause -This likely cause is that the user's server has stopped running but they +This likely means that the user's server has stopped running but they still have a browser tab open. For example, you might have 3 tabs open, and shut your server down via one. Or you closed your laptop, your server was culled for inactivity, and then you reopen your laptop again! The From a785b8d38a0110e34d59e838794973c4b37570b9 Mon Sep 17 00:00:00 2001 From: Arafat Abdussalam Date: Fri, 14 Oct 2022 02:59:34 -0700 Subject: [PATCH 059/214] I made the correction to the PR I resolved my previous commit as suggested --- docs/source/reference/websecurity.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/reference/websecurity.md b/docs/source/reference/websecurity.md index d9a9a397..9de4b38b 100644 --- a/docs/source/reference/websecurity.md +++ b/docs/source/reference/websecurity.md @@ -10,13 +10,13 @@ The **Security Overview** section helps you learn about: This overview also helps you obtain a deeper understanding of how JupyterHub works. -## Semi-trusted and Untrusted Users +## Semi-trusted and untrusted users JupyterHub is designed to be a _simple multi-user server for modestly sized groups_ of **semi-trusted** users. While the design reflects serving semi-trusted users, JupyterHub is not necessarily unsuitable for serving **untrusted** users. -Using JupyterHub with **untrusted** users entails more work by the +Using JupyterHub with **untrusted** users does mean more work by the administrator. Much care is required to secure a Hub, with extra caution on protecting users from each other as the Hub is serving untrusted users. @@ -41,7 +41,7 @@ To protect all users from each other, JupyterHub administrators must ensure that: - A user **does not have permission** to modify their single-user notebook server, - as well as: + including: - A user **may not** install new packages in the Python environment that runs their single-user server. - If the `PATH` is used to resolve the single-user executable (instead of From 7462cfa6fd96cbee892ef4965016b3337e215267 Mon Sep 17 00:00:00 2001 From: AdrianaHelga <58259099+AdrianaHelga@users.noreply.github.com> Date: Sun, 16 Oct 2022 11:01:21 +0300 Subject: [PATCH 060/214] Update services-basics.md --- docs/source/getting-started/services-basics.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/docs/source/getting-started/services-basics.md b/docs/source/getting-started/services-basics.md index f32aefe2..b8c1eefb 100644 --- a/docs/source/getting-started/services-basics.md +++ b/docs/source/getting-started/services-basics.md @@ -24,7 +24,7 @@ Hub via the REST API. ## API Token basics -### Create an API token +### Step 1: Generate an API token To run such an external service, an API token must be created and provided to the service. @@ -43,12 +43,12 @@ generating an API token is available from the JupyterHub user interface: ![API TOKEN success page](../images/token-request-success.png) -### Pass environment variable with token to the Hub +### Step 2: Pass environment variable with token to the Hub In the case of `cull_idle_servers`, it is passed as the environment variable called `JUPYTERHUB_API_TOKEN`. -### Use API tokens for services and tasks that require external access +### Step 3: Use API tokens for services and tasks that require external access While API tokens are often associated with a specific user, API tokens can be used by services that require external access for activities @@ -62,7 +62,7 @@ c.JupyterHub.services = [ ] ``` -### Restart JupyterHub +### Step 4: Restart JupyterHub Upon restarting JupyterHub, you should see a message like below in the logs: @@ -78,16 +78,15 @@ single-user servers, and only cookies can be used for authentication. 0.8 supports using JupyterHub API tokens to authenticate to single-user servers. -## Configure the idle culler to run as a Hub-Managed Service +## How to configure the idle culler to run as a Hub-Managed Service -Install the idle culler: +### Step 1: Install the idle culler: ``` pip install jupyterhub-idle-culler ``` -In `jupyterhub_config.py`, add the following dictionary for the -`idle-culler` Service to the `c.JupyterHub.services` list: +### Step 2: In `jupyterhub_config.py`, add the following dictionary for the `idle-culler` Service to the `c.JupyterHub.services` list: ```python c.JupyterHub.services = [ @@ -127,7 +126,7 @@ It now needs the scopes: - `admin:servers` to start/stop servers ``` -## Run `cull-idle` manually as a standalone script +## How to run `cull-idle` manually as a standalone script Now you can run your script by providing it the API token and it will authenticate through the REST API to From bcaaaa2d35e9e0597178742bbeec92c293d8acd9 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 17 Oct 2022 05:20:46 +0000 Subject: [PATCH 061/214] Bump docker/setup-qemu-action from 2.0.0 to 2.1.0 Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 2.0.0 to 2.1.0. - [Release notes](https://github.com/docker/setup-qemu-action/releases) - [Commits](https://github.com/docker/setup-qemu-action/compare/8b122486cedac8393e77aa9734c3528886e4a1a8...e81a89b1732b9c48d79cd809d8d81d79c4647a18) --- updated-dependencies: - dependency-name: docker/setup-qemu-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] --- .github/workflows/release.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index e6e2f020..7834a43d 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -108,7 +108,7 @@ jobs: # https://github.com/docker/build-push-action/tree/v2.4.0#usage # https://github.com/docker/build-push-action/blob/v2.4.0/docs/advanced/multi-platform.md - name: Set up QEMU (for docker buildx) - uses: docker/setup-qemu-action@8b122486cedac8393e77aa9734c3528886e4a1a8 # associated tag: v1.0.2 + uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # associated tag: v1.0.2 - name: Set up Docker Buildx (for multi-arch builds) uses: docker/setup-buildx-action@dc7b9719a96d48369863986a06765841d7ea23f6 # associated tag: v1.1.2 From cf1dcd6f3a49f3b770720c25436ac1a83801863f Mon Sep 17 00:00:00 2001 From: Goodness Chris-Ugari Date: Mon, 17 Oct 2022 10:59:20 +0100 Subject: [PATCH 062/214] Update config-user-env.md update --- docs/source/reference/config-user-env.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/reference/config-user-env.md b/docs/source/reference/config-user-env.md index 251fee0c..134fc268 100644 --- a/docs/source/reference/config-user-env.md +++ b/docs/source/reference/config-user-env.md @@ -30,7 +30,7 @@ system-wide or in a shared environment. This installation location should always be in the same environment that `jupyterhub-singleuser` itself is installed in, and must be _readable and executable_ by your users. If you want users to be able to install additional -packages, they must also be _writable_ by your users. +packages, the installation location must also be _writable_ by your users. If you are using a standard system Python install, you would use: From c7f14eec141b895b297e19443b0b62949865b9b7 Mon Sep 17 00:00:00 2001 From: Goodness Chris-Ugari Date: Mon, 17 Oct 2022 11:07:39 +0100 Subject: [PATCH 063/214] Update troubleshooting.md Removed static header items --- docs/source/troubleshooting.md | 31 ------------------------------- 1 file changed, 31 deletions(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index 1f051859..fcbb345d 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -4,37 +4,6 @@ When troubleshooting, you may see unexpected behaviors or receive an error message. This section provides links for identifying the cause of the problem and how to resolve it. -[_Behavior_](#behavior) - -- JupyterHub proxy fails to start -- sudospawner fails to run] -- What is the default behavior when none of the lists (admin, allowed, - allowed groups) are set? -- JupyterHub Docker container not accessible at localhost -- How can I kill ports from JupyterHub-managed services that have been orphaned? -- Why am I getting a Spawn failed error message? -- How can I run JupyterHub with sudo but use my current env vars and virtualenv location? - -[_Errors_](#errors) - -- Error 500 after spawning my single-user server -- Launching Jupyter Notebooks to run as an externally managed JupyterHub service with the `jupyterhub-singleuser` command returns a `JUPYTERHUB_API_TOKEN` error - -[_How do I...?_](#how-do-i) - -- Use a chained SSL certificate -- Install JupyterHub without a network connection -- I want access to the whole filesystem and still default users to their home directory -- How do I increase the number of pySpark executors on YARN? -- How do I use JupyterLab's prerelease version with JupyterHub? -- How do I set up JupyterHub for a workshop (when users are not known ahead of time)? -- How do I set up rotating daily logs? -- Toree integration with HDFS rack awareness script -- Where do I find Docker images and Dockerfiles related to JupyterHub? -- How can I view the logs for JupyterHub or the user's Notebook servers when using the DockerSpawner? - -[_Troubleshooting commands_](#troubleshooting-commands) - ## Behavior ### JupyterHub proxy fails to start From ea88427b7f4804fc35885708e331cf5dd53fbc4d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 17 Oct 2022 10:53:28 +0000 Subject: [PATCH 064/214] Bump docker/build-push-action from 3.1.1 to 3.2.0 Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 3.1.1 to 3.2.0. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](https://github.com/docker/build-push-action/compare/c84f38281176d4c9cdb1626ffafcd6b3911b5d94...c56af957549030174b10d6867f20e78cfd7debc5) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] --- .github/workflows/release.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 7834a43d..721d221c 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -149,7 +149,7 @@ jobs: branchRegex: ^\w[\w-.]*$ - name: Build and push jupyterhub - uses: docker/build-push-action@c84f38281176d4c9cdb1626ffafcd6b3911b5d94 + uses: docker/build-push-action@c56af957549030174b10d6867f20e78cfd7debc5 with: context: . platforms: linux/amd64,linux/arm64 @@ -170,7 +170,7 @@ jobs: branchRegex: ^\w[\w-.]*$ - name: Build and push jupyterhub-onbuild - uses: docker/build-push-action@c84f38281176d4c9cdb1626ffafcd6b3911b5d94 + uses: docker/build-push-action@c56af957549030174b10d6867f20e78cfd7debc5 with: build-args: | BASE_IMAGE=${{ fromJson(steps.jupyterhubtags.outputs.tags)[0] }} @@ -191,7 +191,7 @@ jobs: branchRegex: ^\w[\w-.]*$ - name: Build and push jupyterhub-demo - uses: docker/build-push-action@c84f38281176d4c9cdb1626ffafcd6b3911b5d94 + uses: docker/build-push-action@c56af957549030174b10d6867f20e78cfd7debc5 with: build-args: | BASE_IMAGE=${{ fromJson(steps.onbuildtags.outputs.tags)[0] }} @@ -215,7 +215,7 @@ jobs: branchRegex: ^\w[\w-.]*$ - name: Build and push jupyterhub/singleuser - uses: docker/build-push-action@c84f38281176d4c9cdb1626ffafcd6b3911b5d94 + uses: docker/build-push-action@c56af957549030174b10d6867f20e78cfd7debc5 with: build-args: | JUPYTERHUB_VERSION=${{ github.ref_type == 'tag' && github.ref_name || format('git:{0}', github.sha) }} From 9d630add9a2906fbc5517daae8455ffbfbd16e4e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 17 Oct 2022 10:53:28 +0000 Subject: [PATCH 065/214] Bump docker/setup-buildx-action from 2.0.0 to 2.1.0 Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2.0.0 to 2.1.0. - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](https://github.com/docker/setup-buildx-action/compare/dc7b9719a96d48369863986a06765841d7ea23f6...95cb08cb2672c73d4ffd2f422e6d11953d2a9c70) --- updated-dependencies: - dependency-name: docker/setup-buildx-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] --- .github/workflows/release.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 7834a43d..b61337f1 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -111,7 +111,7 @@ jobs: uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # associated tag: v1.0.2 - name: Set up Docker Buildx (for multi-arch builds) - uses: docker/setup-buildx-action@dc7b9719a96d48369863986a06765841d7ea23f6 # associated tag: v1.1.2 + uses: docker/setup-buildx-action@95cb08cb2672c73d4ffd2f422e6d11953d2a9c70 # associated tag: v1.1.2 with: # Allows pushing to registry on localhost:5000 driver-opts: network=host From fc6d93bbe37209f796bc15a344cfb62a078400c8 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Mon, 17 Oct 2022 13:20:02 +0100 Subject: [PATCH 066/214] Update docs/source/rbac/tech-implementation.md Replace URLs with internal markdown link Co-authored-by: Min RK --- docs/source/rbac/tech-implementation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/rbac/tech-implementation.md b/docs/source/rbac/tech-implementation.md index b3084ed5..2a8561ca 100644 --- a/docs/source/rbac/tech-implementation.md +++ b/docs/source/rbac/tech-implementation.md @@ -1,6 +1,6 @@ # Technical Implementation -[Roles](https://jupyterhub.readthedocs.io/en/stable/rbac/roles.html#roles) are stored in the database, where they are associated with users, services, groups and tokens. Roles can be added or modified as explained in the {ref}`define-role-target` section. Users, services, groups, and tokens can gain, change, and lose roles. This is currently achieved via `jupyterhub_config.py` (see {ref}`define-role-target`) and will be made available via API in the future. The latter will allow for changing a token's role, and thereby its permissions, without the need to issue a new token. +[Roles](roles) are stored in the database, where they are associated with users, services, and groups. Roles can be added or modified as explained in the {ref}`define-role-target` section. Users, services, groups, and tokens can gain, change, and lose roles. This is currently achieved via `jupyterhub_config.py` (see {ref}`define-role-target`) and will be made available via API in the future. The latter will allow for changing a user's role, and thereby its permissions, without the need to restart JupyterHub. Roles and scopes utilities can be found in `roles.py` and `scopes.py` modules. Scope variables take on five different formats that are reflected throughout the utilities via specific nomenclature: From e7a325ed248796eab72874d0a3789bfaaa7b1aa7 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Mon, 17 Oct 2022 13:20:37 +0100 Subject: [PATCH 067/214] Update docs/source/rbac/tech-implementation.md update text Co-authored-by: Min RK --- docs/source/rbac/tech-implementation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/rbac/tech-implementation.md b/docs/source/rbac/tech-implementation.md index 2a8561ca..93558791 100644 --- a/docs/source/rbac/tech-implementation.md +++ b/docs/source/rbac/tech-implementation.md @@ -61,7 +61,7 @@ The API request is resolved without additional errors using the scope _intersect the Hub logs a warning in this case (see {ref}`Figure 2 `). Resolving a token's scope (yellow box in {ref}`Figure 1 `) corresponds to resolving all the roles of the token's owner (including the roles associated with their groups) and the token's own scopes into a set of scopes. The two sets are compared (Resolve the scopes box in orange in {ref}`Figure 1 `), taking into account the scope hierarchy. -If the token's scopes are a subset of the token owner's scopes, the token is issued with the requested scopes; if not, JupyterHub will throw an error. +If the token's scopes are a subset of the token owner's scopes, the token is issued with the requested scopes; if not, JupyterHub will raise an error. {ref}`Figure 1 ` below illustrates the steps involved. The orange rectangles highlight where in the process the roles and scopes are resolved. From 8d2a987c819c85b38030f6c5238d73efacc53db8 Mon Sep 17 00:00:00 2001 From: Simon Li Date: Mon, 17 Oct 2022 13:29:25 +0100 Subject: [PATCH 068/214] remove out of date commented tag version --- .github/workflows/release.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index b61337f1..be80d54a 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -111,7 +111,7 @@ jobs: uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # associated tag: v1.0.2 - name: Set up Docker Buildx (for multi-arch builds) - uses: docker/setup-buildx-action@95cb08cb2672c73d4ffd2f422e6d11953d2a9c70 # associated tag: v1.1.2 + uses: docker/setup-buildx-action@95cb08cb2672c73d4ffd2f422e6d11953d2a9c70 with: # Allows pushing to registry on localhost:5000 driver-opts: network=host From 6ca5c1a276e661ee3eae717c2985994878c58e25 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Tue, 11 Oct 2022 01:10:11 +0100 Subject: [PATCH 069/214] Update documentation Fix structure of documentation to aid readability and flow. --- docs/source/contributing/community.md | 23 +++++++++++++++++++++++ docs/source/contributing/community.rst | 16 ---------------- 2 files changed, 23 insertions(+), 16 deletions(-) create mode 100644 docs/source/contributing/community.md delete mode 100644 docs/source/contributing/community.rst diff --git a/docs/source/contributing/community.md b/docs/source/contributing/community.md new file mode 100644 index 00000000..dd1fb38e --- /dev/null +++ b/docs/source/contributing/community.md @@ -0,0 +1,23 @@ +# Community communication channels + +We use different channels of communication for different purposes. Whichever one you use will depend on what kind of communication you want to engage in. + +## Discourse +We use [Discourse](https://discourse.jupyter.org) for online discussions. Everyone in the Jupyter community is welcome to bring ideas and questions there. + +All our past and current discussions on Discourse are archived and searchable. This is why we recommend you first go to Discourse, so that discussions remain useful and accessible to the whole community. + +## Gitter +We use [our Gitter channel](https://gitter.im/jupyterhub/jupyterhub) for online, real-time text chat; a place for more ephemeral discussions. When you're not on Discourse, you can stop here to have other discussions on the fly. + +## Github Issues +Github issues are used for most long-form project discussions, bug reports and feature requests. + +Issues related to a specific authenticator or spawner should be opened in the appropriate repository for the authenticator or spawner. If you are using a specific JupyterHub distribution (such as [Zero to JupyterHub on Kubernetes](http://github.com/jupyterhub/zero-to-jupyterhub-k8s) or [The Littlest JupyterHub](http://github.com/jupyterhub/the-littlest-jupyterhub/)), you should open issues directly in their repository. + +If you cannot find a repository to open your issue in, do not worry! Open the issue in the [main JupyterHub repository](https://github.com/jupyterhub/jupyterhub/) and our community will help you figure it out. + +## Mailing List +A [mailing list](https://groups.google.com/forum/#!forum/jupyter) for all of Project Jupyter exists, along with one for [teaching with Jupyter Notebooks](https://groups.google.com/forum/#!forum/jupyter-education). + +**NOTE**: Our community is distributed across the world in various timezones, so please be patient if you do not get a response immediately! diff --git a/docs/source/contributing/community.rst b/docs/source/contributing/community.rst deleted file mode 100644 index fe88e40f..00000000 --- a/docs/source/contributing/community.rst +++ /dev/null @@ -1,16 +0,0 @@ -.. _contributing/community: - -================================ -Community communication channels -================================ - -`Discourse `_ is primarily used by the Jupyter community for discussions and support. Additionally, we use JupyterHub's primary Gitter channel, `jupyterhub/jupyterhub `_, for quick discussions, and GitHub issues for detailed discussions relating to modifying a repository's content. -If you have an issue with a certain authenticator or spawner, please report them to the relevant repository. If you are using a specific JupyterHub -distribution (such as `Zero to JupyterHub on Kubernetes `_ -or `The Littlest JupyterHub `_), -you should open issues directly in the respective repository. If you can not -find a repository to open your issue in, do not worry! Create it in the `main -JupyterHub repository `_ and our -community will help you figure it out. - -Please be patient if you do not recieve a response right away; keep in mind that our comunity is spread out over the globe in different timezones. From 838719e7aba20e6bff38e920ebb364502fd2c0ab Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue, 11 Oct 2022 00:18:15 +0000 Subject: [PATCH 070/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/contributing/community.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/source/contributing/community.md b/docs/source/contributing/community.md index dd1fb38e..870e4309 100644 --- a/docs/source/contributing/community.md +++ b/docs/source/contributing/community.md @@ -3,21 +3,25 @@ We use different channels of communication for different purposes. Whichever one you use will depend on what kind of communication you want to engage in. ## Discourse + We use [Discourse](https://discourse.jupyter.org) for online discussions. Everyone in the Jupyter community is welcome to bring ideas and questions there. -All our past and current discussions on Discourse are archived and searchable. This is why we recommend you first go to Discourse, so that discussions remain useful and accessible to the whole community. +All our past and current discussions on Discourse are archived and searchable. This is why we recommend you first go to Discourse, so that discussions remain useful and accessible to the whole community. ## Gitter + We use [our Gitter channel](https://gitter.im/jupyterhub/jupyterhub) for online, real-time text chat; a place for more ephemeral discussions. When you're not on Discourse, you can stop here to have other discussions on the fly. ## Github Issues -Github issues are used for most long-form project discussions, bug reports and feature requests. + +Github issues are used for most long-form project discussions, bug reports and feature requests. Issues related to a specific authenticator or spawner should be opened in the appropriate repository for the authenticator or spawner. If you are using a specific JupyterHub distribution (such as [Zero to JupyterHub on Kubernetes](http://github.com/jupyterhub/zero-to-jupyterhub-k8s) or [The Littlest JupyterHub](http://github.com/jupyterhub/the-littlest-jupyterhub/)), you should open issues directly in their repository. If you cannot find a repository to open your issue in, do not worry! Open the issue in the [main JupyterHub repository](https://github.com/jupyterhub/jupyterhub/) and our community will help you figure it out. ## Mailing List + A [mailing list](https://groups.google.com/forum/#!forum/jupyter) for all of Project Jupyter exists, along with one for [teaching with Jupyter Notebooks](https://groups.google.com/forum/#!forum/jupyter-education). **NOTE**: Our community is distributed across the world in various timezones, so please be patient if you do not get a response immediately! From deaccdc6684e71ab41592091eb27290a852d56c8 Mon Sep 17 00:00:00 2001 From: Ojoachele Onuh Date: Mon, 17 Oct 2022 17:27:09 +0100 Subject: [PATCH 071/214] Resolved and updated corrections from previous pull request --- docs/source/rbac/tech-implementation.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/rbac/tech-implementation.md b/docs/source/rbac/tech-implementation.md index eccb2d0b..1d479de6 100644 --- a/docs/source/rbac/tech-implementation.md +++ b/docs/source/rbac/tech-implementation.md @@ -78,15 +78,15 @@ Figure 1. Resolving roles and scopes during API token request With the RBAC framework, each authenticated JupyterHub API request is guarded by a scope decorator that specifies which scopes are required to gain the access to the API. When an API request is performed, the requesting API token's scopes are again intersected with its owner's (yellow box in {ref}`Figure 2 `) to ensure the token does not grant more permissions than its owner has at the request time (e.g., due to changing/losing roles). -If the owner's roles do not include some scopes of the token's scopes, only the _intersection_ of the token's and owner's scopes will be used. For example, using a token with scope `users` whose owner's role scope is `read:users:name` will result in only the `read:users:name` scope being parsed on. In the case of no _intersection_, an empty set of scopes will be used. +If the owner's roles do not include some scopes of the token's scopes, only the _intersection_ of the token's and owner's scopes will be used. For example, using a token with scope `users` whose owner's role scope is `read:users:name` will result in only the `read:users:name` scope being passed on. In the case of no _intersection_, an empty set of scopes will be used. -The parsed scopes are compared to the scopes required to access the API as follows: +The passed scopes are compared to the scopes required to access the API as follows: -- if the API scopes are present within the set of parsed scopes, the access is granted and the API returns its "full" response +- if the API scopes are present within the set of passed scopes, the access is granted and the API returns its "full" response -- if that is not the case, another check is utilized to determine if subscopes of the required API scopes can be found in the parsed scope set: +- if that is not the case, another check is utilized to determine if subscopes of the required API scopes can be found in the passed scope set: - - if found, the RBAC framework employs the {ref}`filtering ` procedures to refine the API response to access only resource attributes corresponding to the parsed scopes. For example, providing a scope `read:users:activity!group=class-C` for the _GET /users_ API will return a list of user models from group `class-C` containing only the `last_activity` attribute for each user model + - if found, the RBAC framework employs the {ref}`filtering ` procedures to refine the API response to access only resource attributes corresponding to the passed scopes. For example, providing a scope `read:users:activity!group=class-C` for the _GET /users_ API will return a list of user models from group `class-C` containing only the `last_activity` attribute for each user model - if not found, the access to API is denied From 79ea4038e552b362be8def8f1b5d27072dd84f82 Mon Sep 17 00:00:00 2001 From: Shloka Date: Mon, 17 Oct 2022 22:30:47 +0530 Subject: [PATCH 072/214] Update docs/source/contributing/tests.rst Co-authored-by: Min RK --- docs/source/contributing/tests.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/tests.rst b/docs/source/contributing/tests.rst index bc3fc7a5..0dee7e9a 100644 --- a/docs/source/contributing/tests.rst +++ b/docs/source/contributing/tests.rst @@ -104,7 +104,7 @@ Troubleshooting Test Failures All the tests are failing ------------------------- -Make sure you have completed all the steps in :ref:`contributing/setup` successfully, and are able to can access JupyterHub from your browser at http://localhost:8000 after starting ``jupyterhub`` in your command line. +Make sure you have completed all the steps in :ref:`contributing/setup` successfully, and are able to access JupyterHub from your browser at http://localhost:8000 after starting ``jupyterhub`` in your command line. Code formatting and linting From 15b88577283c92585bbde197363a0eabffe28cf8 Mon Sep 17 00:00:00 2001 From: Shloka Date: Mon, 17 Oct 2022 22:30:55 +0530 Subject: [PATCH 073/214] Update docs/source/contributing/tests.rst Co-authored-by: Min RK --- docs/source/contributing/tests.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/tests.rst b/docs/source/contributing/tests.rst index 0dee7e9a..8904b8c1 100644 --- a/docs/source/contributing/tests.rst +++ b/docs/source/contributing/tests.rst @@ -13,7 +13,7 @@ Running the tests ================== #. Make sure you have completed :ref:`contributing/setup`. Once completed, you should be able - to run ``jupyterhub`` on your command line and access JupyterHub from your browser at http://localhost:8000. By doing this, it is ensured that the dev environment is properly set + to run ``jupyterhub`` on your command line and access JupyterHub from your browser at http://localhost:8000. Being able to run and access `jupyterhub` should mean that the dev environment is properly set up for tests to run. #. You can run all tests in JupyterHub From ae833d4a51b1133a9b3e65de62605f2443f0fe85 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Mon, 17 Oct 2022 18:09:49 +0100 Subject: [PATCH 074/214] Add link to gh oauth Co-authored-by: Georgiana --- docs/source/reference/config-ghoauth.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/reference/config-ghoauth.md b/docs/source/reference/config-ghoauth.md index 588b1f93..bd8e290b 100644 --- a/docs/source/reference/config-ghoauth.md +++ b/docs/source/reference/config-ghoauth.md @@ -5,7 +5,7 @@ deployment with the following assumptions: - Running JupyterHub on a single cloud server - Using SSL on the standard HTTPS port 443 -- Using GitHub OAuth (using OAuthenticator) for login +- Using GitHub OAuth (using [OAuthenticator](https://oauthenticator.readthedocs.io/en/latest)) for login - Using the default spawner (to configure other spawners, uncomment and edit `spawner_class` as well as follow the instructions for your desired spawner) - Users exist locally on the server From 97c72dd779af98910988b931b61717e990c0af4b Mon Sep 17 00:00:00 2001 From: PoorvajaRayas <106921794+PoorvajaRayas@users.noreply.github.com> Date: Tue, 18 Oct 2022 02:01:50 +0530 Subject: [PATCH 075/214] Update troubleshooting.md Improved the documentation to make it clearer --- docs/source/troubleshooting.md | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index fcbb345d..0edfac41 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -1,22 +1,23 @@ # Troubleshooting -When troubleshooting, you may see unexpected behaviors or receive an error +While troubleshooting, you may see unexpected behaviors or receive an error message. This section provides links for identifying the cause of the problem and how to resolve it. + ## Behavior ### JupyterHub proxy fails to start -If you have tried to start the JupyterHub proxy and it fails to start: +If you have tried to start the JupyterHub proxy and it fails to run: - check if the JupyterHub IP configuration setting is `c.JupyterHub.ip = '*'`; if it is, try `c.JupyterHub.ip = ''` - Try starting with `jupyterhub --ip=0.0.0.0` **Note**: If this occurs on Ubuntu/Debian, check that you are using a -recent version of [Node](https://nodejs.org). Some versions of Ubuntu/Debian come with a version -of Node that is very old, and it is necessary to update Node. +recent version of [Node](https://nodejs.org). Some versions of Ubuntu/Debian come with an older version +of Node and it is necessary to update Node. ### sudospawner fails to run @@ -37,11 +38,11 @@ to the config file, `jupyterhub_config.py`. When nothing is given for these lists, there will be no admins, and all users who can authenticate on the system (i.e. all the Unix users on the server with a password) will be allowed to start a server. The allowed username set lets you limit -this to a particular set of users, and admin_users lets you specify who +this to a particular set of users, and admin_users let you specify who among them may use the admin interface (not necessary, unless you need to do things like inspect other users' servers or modify the user list at runtime). -### JupyterHub Docker container not accessible at localhost +### What to do when JupyterHub Docker container is not accessible at localhost? Even though the command to start your Docker container exposes port 8000 (`docker run -p 8000:8000 -d --name jupyterhub jupyterhub/jupyterhub jupyterhub`), @@ -71,7 +72,7 @@ After successfully logging in to JupyterHub with a compatible authenticator, I g This issue occurs when the authenticator requires a local system user to exist. In these cases, you need to use a spawner that does not require an existing system user account, such as `DockerSpawner` or `KubeSpawner`. -### How can I run JupyterHub with sudo but use my current env vars and virtualenv location? +### How can I run JupyterHub with sudo but use my current environment variables and virtualenv location? When launching JupyterHub with `sudo jupyterhub` I get import errors and my environment variables don't work. @@ -87,7 +88,7 @@ sudo MY_ENV=abc123 \ ### Error 500 after spawning my single-user server -You receive a 500 error when accessing the URL `/user//...`. +You receive an Error 500 while accessing the URL `/user//...`. This is often seen when your single-user server cannot verify your user cookie with the Hub. @@ -100,7 +101,7 @@ There are two likely reasons for this: #### Symptoms The main symptom is a failure to load _any_ page served by the single-user -server, met with a 500 error. This is typically the first page at `/user/` +server, met with an Error 500. This is typically the first page at `/user/` after logging in or clicking "Start my server". When a single-user notebook server receives a request, the notebook server makes an API request to the Hub to check if the cookie corresponds to the right user. This request is logged. @@ -167,7 +168,7 @@ When your whole JupyterHub sits behind an organization proxy (_not_ a reverse pr [JupyterHub services](https://jupyterhub.readthedocs.io/en/stable/reference/services.html) allow processes to interact with JupyterHub's REST API. Example use-cases include: -- **Secure Testing**: provide a canonical Jupyter Notebook for testing production data to reduce the number of entry points into production systems. +- **Secure Testing**: provides a canonical Jupyter Notebook for testing production data to reduce the number of entry points into production systems. - **Grading Assignments**: provide access to shared Jupyter Notebooks that may be used for management tasks such as grading assignments. - **Private Dashboards**: share dashboards with certain group members. @@ -194,7 +195,7 @@ With a docker container, pass in the environment variable with the run command: -e JUPYTERHUB_API_TOKEN=my_secret_token \ jupyter/datascience-notebook:latest -[This example](https://github.com/jupyterhub/jupyterhub/tree/HEAD/examples/service-notebook/external) demonstrates how to combine the use of the `jupyterhub-singleuser` environment variables when launching a Notebook as an externally managed service. +[This example](https://github.com/jupyterhub/jupyterhub/tree/HEAD/examples/service-notebook/external) demonstrates how to combine the use of the `jupyterhub-singleuser` environment variables when launching a Notebook as an externally managed service. ## How do I...? @@ -268,7 +269,7 @@ similar to this one: provides additional information. The [pySpark configuration documentation](https://spark.apache.org/docs/0.9.0/configuration.html) is also helpful for programmatic configuration examples. -### How do I use JupyterLab's prerelease version with JupyterHub? +### How do I use JupyterLab's pre-release version with JupyterHub? While JupyterLab is still under active development, we have had users ask about how to try out JupyterLab with JupyterHub. @@ -294,7 +295,7 @@ notebook servers to default to JupyterLab: ### How do I set up JupyterHub for a workshop (when users are not known ahead of time)? 1. Set up JupyterHub using OAuthenticator for GitHub authentication -2. Configure the admin list to have workshop leaders be listed with administrator privileges. +2. Configure the admin list to have workshop leaders listed with administrator privileges. Users will need a GitHub account to log in and be authenticated by the Hub. @@ -324,7 +325,7 @@ Or use syslog: ### Toree integration with HDFS rack awareness script -The Apache Toree kernel will have an issue when running with JupyterHub if the standard HDFS +The Apache Toree kernel will have an issue while running with JupyterHub if the standard HDFS rack awareness script is used. This will materialize in the logs as a repeated WARN: ```bash @@ -371,7 +372,7 @@ You can also tail logs to view them in real-time using the `-f` option: ## Troubleshooting commands The following commands provide additional detail about installed packages, -versions, and system information that may be helpful when troubleshooting +versions, and system information that may be helpful while troubleshooting a JupyterHub deployment. The commands are: - System and deployment information From b890a7486da28cce6a3a82a29b7bc2f43017c6d3 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 17 Oct 2022 20:33:17 +0000 Subject: [PATCH 076/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/troubleshooting.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index 0edfac41..6950264a 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -4,7 +4,6 @@ While troubleshooting, you may see unexpected behaviors or receive an error message. This section provides links for identifying the cause of the problem and how to resolve it. - ## Behavior ### JupyterHub proxy fails to start @@ -195,7 +194,7 @@ With a docker container, pass in the environment variable with the run command: -e JUPYTERHUB_API_TOKEN=my_secret_token \ jupyter/datascience-notebook:latest -[This example](https://github.com/jupyterhub/jupyterhub/tree/HEAD/examples/service-notebook/external) demonstrates how to combine the use of the `jupyterhub-singleuser` environment variables when launching a Notebook as an externally managed service. +[This example](https://github.com/jupyterhub/jupyterhub/tree/HEAD/examples/service-notebook/external) demonstrates how to combine the use of the `jupyterhub-singleuser` environment variables when launching a Notebook as an externally managed service. ## How do I...? From 10c54353da723ef005d9f1143ca713d582d670ff Mon Sep 17 00:00:00 2001 From: lumenCodes Date: Tue, 18 Oct 2022 02:19:33 +0100 Subject: [PATCH 077/214] Rest Api doc update --- docs/source/reference/rest.md | 46 +++++++++++++++++++---------------- 1 file changed, 25 insertions(+), 21 deletions(-) diff --git a/docs/source/reference/rest.md b/docs/source/reference/rest.md index 12f8f44e..cbf45016 100644 --- a/docs/source/reference/rest.md +++ b/docs/source/reference/rest.md @@ -4,15 +4,20 @@ This section will give you information on: -- what you can do with the API -- create an API token -- add API tokens to the config files -- make an API request programmatically using the requests library -- learn more about JupyterHub's API +- [what you can do with the API](#what-you-can-do-with-the-api) +- [how to create an API token](#create-an-api-token) +- [how to add API tokens to the config files](#) +- [how to make an API request programmatically using the requests library](#make-an-api-request) +- [where to learn more about JupyterHub's API](#learn-more-about-the-api) + + +Before we discuss about JupyterHub's REST API, you can learn about [REST APIs here](https://en.wikipedia.org/wiki/Representational_state_transfer). A REST +API provides a standard way for users to get and send information to the +Hub. ## What you can do with the API -Using the [JupyterHub REST API][], you can perform actions on the Hub, +Using the [JupyterHub REST API](https://jupyterhub.readthedocs.io/en/stable/reference/rest-api.html#/default/delete_groups__name_), you can perform actions on the Hub, such as: - checking which users are active @@ -21,16 +26,14 @@ such as: - authenticating services - communicating with an individual Jupyter server's REST API -A [REST](https://en.wikipedia.org/wiki/Representational_state_transfer) -API provides a standard way for users to get and send information to the -Hub. + ## Create an API token To send requests using JupyterHub API, you must pass an API token with -the request. +the request. -The preferred way of generating an API token is: +The preferred way of generating an API token is by running: ```bash openssl rand -hex 32 @@ -41,7 +44,7 @@ added to JupyterHub using `.api_tokens` configuration setting in `jupyterhub_config.py`. Alternatively, use the `jupyterhub token` command to generate a token -for a specific hub user by passing the 'username': +for a specific hub user by passing the " **username**": ```bash jupyterhub token @@ -123,9 +126,9 @@ c.JupyterHub.load_roles = [ ``` The token will have the permissions listed in the role -(see [scopes][] for a list of available permissions), +(see [scopes](https://jupyterhub.readthedocs.io/en/stable/rbac/scopes.html#scopes-in-jupyterhub) for a list of available permissions), but there will no longer be a user account created to house it. -The main noticeable difference is that there will be no notebook server associated with the account +The main noticeable difference between a configuration and a service is that there will be no notebook server associated with the account and the service will not show up in the various user list pages and APIs. ## Make an API request @@ -136,8 +139,8 @@ Authorization header. ### Use requests Using the popular Python [requests](https://docs.python-requests.org) -library, here's example code to make an API request for the users of a JupyterHub -deployment. An API GET request is made, and the request sends an API token for +library, (here's example code to make an API request for the users of a JupyterHub +deployment) an API GET request is made, and the request sends an API token for authorization. The response contains information about the users: ```python @@ -175,10 +178,11 @@ r.raise_for_status() r.json() ``` -The same API token can also authorize access to the [Jupyter Notebook REST API][] -provided by notebook servers managed by JupyterHub if it has the necessary `access:users:servers` scope: +The same API token can also authorize access to the [Jupyter Notebook REST API](https://jupyterhub.readthedocs.io/en/stable/reference/rest-api.html#/) +provided by notebook servers managed by JupyterHub if it has the necessary `access:users:servers` scope. +

 

+ -(api-pagination)= ## Paginating API requests @@ -275,7 +279,7 @@ First you must enable named-servers by including the following setting in the `j `c.JupyterHub.allow_named_servers = True` -If using the [zero-to-jupyterhub-k8s](https://github.com/jupyterhub/zero-to-jupyterhub-k8s) set-up to run JupyterHub, +If you are using the [zero-to-jupyterhub-k8s](https://github.com/jupyterhub/zero-to-jupyterhub-k8s) set-up to run JupyterHub, then instead of editing the `jupyterhub_config.py` file directly, you could pass the following as part of the `config.yaml` file, as per the [tutorial](https://zero-to-jupyterhub.readthedocs.io/en/latest/): @@ -303,7 +307,7 @@ or kubernetes pods. ## Learn more about the API -You can see the full [JupyterHub REST API][] for details. +You can see the full [JupyterHub REST API](https://jupyterhub.readthedocs.io/en/stable/reference/rest-api.html#/) for details. [openapi initiative]: https://www.openapis.org/ [jupyterhub rest api]: ./rest-api From 28de35facd347d902e5de3bc4416e96ea4d89cbe Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue, 18 Oct 2022 01:25:27 +0000 Subject: [PATCH 078/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/reference/rest.md | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/docs/source/reference/rest.md b/docs/source/reference/rest.md index cbf45016..42bfc6eb 100644 --- a/docs/source/reference/rest.md +++ b/docs/source/reference/rest.md @@ -10,7 +10,6 @@ This section will give you information on: - [how to make an API request programmatically using the requests library](#make-an-api-request) - [where to learn more about JupyterHub's API](#learn-more-about-the-api) - Before we discuss about JupyterHub's REST API, you can learn about [REST APIs here](https://en.wikipedia.org/wiki/Representational_state_transfer). A REST API provides a standard way for users to get and send information to the Hub. @@ -26,12 +25,10 @@ such as: - authenticating services - communicating with an individual Jupyter server's REST API - - ## Create an API token To send requests using JupyterHub API, you must pass an API token with -the request. +the request. The preferred way of generating an API token is by running: @@ -179,11 +176,10 @@ r.json() ``` The same API token can also authorize access to the [Jupyter Notebook REST API](https://jupyterhub.readthedocs.io/en/stable/reference/rest-api.html#/) -provided by notebook servers managed by JupyterHub if it has the necessary `access:users:servers` scope. +provided by notebook servers managed by JupyterHub if it has the necessary `access:users:servers` scope. +

 

- - ## Paginating API requests ```{versionadded} 2.0 From 1041bc53b12bfddce396ff8369e2e200c6b85cb7 Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Tue, 18 Oct 2022 11:21:50 +0500 Subject: [PATCH 079/214] fix typo --- docs/source/admin/upgrading.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index 8b786143..65481265 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -5,12 +5,12 @@ Upgrading JupyterHub JupyterHub offers easy upgrade pathways between minor versions. This document describes how to do these upgrades. -If you are using `distributions `__, you +If you are using :ref:`a JupyterHub distribution `, you should consult the distribution's documentation on how to upgrade. This document is useful if you have set up your own JupyterHub without using a distribution. -The steps are discussed in detail. Most likely, +The steps are discussed in detail, so if you get stuck at any step you can always refer to this guide. Most likely, upgrading JupyterHub is painless, quick and with minimal user interruption. Read the Changelog @@ -27,7 +27,7 @@ Notify your users If you are using the default configuration where ``configurable-http-proxy`` is managed by JupyterHub, your users will see service disruption during -the upgrade process. You will need to notify them, and pick a time to do the +the upgrade process. You should notify them, and pick a time to do the upgrade where they will be least disrupted. If you are using a different proxy or running ``configurable-http-proxy`` @@ -50,7 +50,7 @@ Before doing an upgrade, it is critical to back up: Shut down JupyterHub -=================== +==================== Shut down the JupyterHub process. This would vary depending on how you have set up JupyterHub to run. Most likely, it is using a process @@ -64,7 +64,7 @@ There are two environments where the ``jupyterhub`` package is installed: #. The *hub environment*, which is where the JupyterHub server process runs. This is started with the ``jupyterhub`` command and is what - people generally think of it as JupyterHub. + people generally think of as JupyterHub. #. The *notebook user environments*. This is where the user's notebook servers are launched from and are probably custom to your own @@ -94,7 +94,7 @@ with: Where ```` is the version of JupyterHub you are upgrading to. You should also check for new releases of the authenticator & spawner you -are using. You might wish to upgrade those packages too along with JupyterHub +are using. You might wish to upgrade those packages, too, along with JupyterHub or upgrade them separately. Upgrade JupyterHub database From d1ad045335498c6105f072ff01a1e4e96642fb3f Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Tue, 18 Oct 2022 11:55:48 +0500 Subject: [PATCH 080/214] update index.rst --- docs/source/events/index.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/events/index.rst b/docs/source/events/index.rst index b53c8448..f94bd9fc 100644 --- a/docs/source/events/index.rst +++ b/docs/source/events/index.rst @@ -1,7 +1,7 @@ Event logging and Telemetry -========================== +=========================== -JupyterHub can be configured to record structured events from a running server using Jupyter's `Telemetry System`_. The types of events that JupyterHub emits are defined by `JSON schemas`_ listed at the bottom of this `page `__. +JupyterHub can be configured to record structured events from a running server using Jupyter's `Telemetry System`_. The types of events that JupyterHub emits are defined by `JSON schemas`_ listed at the bottom of this page_. .. _logging: https://docs.python.org/3/library/logging.html .. _`Telemetry System`: https://github.com/jupyter/telemetry From 895d713370df32782e600a4d2178a2f36a32622d Mon Sep 17 00:00:00 2001 From: Tooba Jamal <52610124+ToobaJamal@users.noreply.github.com> Date: Tue, 18 Oct 2022 12:01:13 +0500 Subject: [PATCH 081/214] update spawners-basics.md --- docs/source/getting-started/spawners-basics.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/source/getting-started/spawners-basics.md b/docs/source/getting-started/spawners-basics.md index 9988c2f8..b264c043 100644 --- a/docs/source/getting-started/spawners-basics.md +++ b/docs/source/getting-started/spawners-basics.md @@ -1,12 +1,12 @@ # Spawners and single-user notebook servers -Since the single-user server is an instance of `jupyter notebook`, an entire separate -multi-process application, there are many aspects of that server that can be configured, and a lot +Since the single-user server is an instance of `jupyter notebook`, an entirely separate +multi-process application, there are many aspects of that server that can be configured and a lot of ways to express that configuration. At the JupyterHub level, you can set some values on the Spawner. The simplest of these is `Spawner.notebook_dir`, which lets you set the root directory for a user's server. This root -notebook directory is the highest level directory users will be able to access in the notebook +notebook directory is the highest-level directory users will be able to access in the notebook dashboard. In this example, the root notebook directory is set to `~/notebooks`, where `~` is expanded to the user's home directory. @@ -14,13 +14,13 @@ expanded to the user's home directory. c.Spawner.notebook_dir = '~/notebooks' ``` -You can also specify extra command line arguments to the notebook server with: +You can also specify extra command line arguments to the notebook server with the following: ```python c.Spawner.args = ['--debug', '--profile=PHYS131'] ``` -This could be used to set the users default page for the single user server: +This could be used to set the user's default page for the single-user server: ```python c.Spawner.args = ['--NotebookApp.default_url=/notebooks/Welcome.ipynb'] @@ -30,4 +30,4 @@ Since the single-user server extends the notebook server application, it still loads configuration from the `jupyter_notebook_config.py` config file. Each user may have one of these files in `$HOME/.jupyter/`. Jupyter also supports loading system-wide config files from `/etc/jupyter/`, -which is the place to put configuration that you want to affect all of your users. +which is the place to put the configuration that you want to affect all of your users. From 504ebe90127fa1a6ec0a462973d73a01462fd23f Mon Sep 17 00:00:00 2001 From: Teniola Olowookere <90247181+Teniola-theDev@users.noreply.github.com> Date: Tue, 18 Oct 2022 10:09:31 +0100 Subject: [PATCH 082/214] capitalized cli and added y to jupterhub --- docs/source/reference/spawners.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/reference/spawners.md b/docs/source/reference/spawners.md index 7c183018..afb58744 100644 --- a/docs/source/reference/spawners.md +++ b/docs/source/reference/spawners.md @@ -267,8 +267,8 @@ Spawners mainly do one thing: launch a command in an environment. The command-line is constructed from user configuration: -- Spawner.cmd (default: `['jupterhub-singleuser']`) -- Spawner.args (cli args to pass to the cmd, default: empty) +- Spawner.cmd (default: `['jupyterhub-singleuser']`) +- Spawner.args (CLI args to pass to the cmd, default: empty) where the configuration: @@ -319,12 +319,12 @@ Optional environment variables, depending on configuration: - JUPYTERHUB_ROOT_DIR - the root directory of the server (notebook directory), when Spawner.notebook_dir is defined (new in 2.0) - JUPYTERHUB_DEFAULT_URL - the default URL for the server (for redirects from /user/:name/), if Spawner.default_url is defined - (new in 2.0, previously passed via cli) + (new in 2.0, previously passed via CLI) - JUPYTERHUB_DEBUG=1 - generic debug flag, sets maximum log level when Spawner.debug is True - (new in 2.0, previously passed via cli) + (new in 2.0, previously passed via CLI) - JUPYTERHUB_DISABLE_USER_CONFIG=1 - disable loading user config, sets maximum log level when Spawner.debug is True (new in 2.0, - previously passed via cli) + previously passed via CLI) - JUPYTERHUB*[MEM|CPU]*[LIMIT_GUARANTEE] - the values of cpu and memory limits and guarantees. These are not expected to be enforced by the process, From 853f8accf5edf65e843939447eb4d274b84b8322 Mon Sep 17 00:00:00 2001 From: Goodness Chris-Ugari Date: Tue, 18 Oct 2022 15:09:44 +0100 Subject: [PATCH 083/214] Update config-proxy.md update --- docs/source/reference/config-proxy.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/reference/config-proxy.md b/docs/source/reference/config-proxy.md index 90e56b06..3c0e1866 100644 --- a/docs/source/reference/config-proxy.md +++ b/docs/source/reference/config-proxy.md @@ -51,7 +51,7 @@ server { listen 80; server_name HUB.DOMAIN.TLD; - # Send all requests to port 80 to 302 and redirect to HTTPS + # Send a redirect request using the $request_uri that was requested but via HTTPS return 302 https://$host$request_uri; } @@ -112,7 +112,7 @@ server { listen 80; server_name NO_HUB.DOMAIN.TLD; - # Send all requests to port 80 to 302 and redirect to HTTPS + # Send a redirect request using the $request_uri that was requested but via HTTPS return 302 https://$host$request_uri; } From 390efc1c5a203f4e5e53b06e3dbee41ea76b1b0f Mon Sep 17 00:00:00 2001 From: Alexander Chosen okon Date: Tue, 18 Oct 2022 23:24:53 +0000 Subject: [PATCH 084/214] removed the logo image and resolved some conflicts --- docs/source/reference/config-user-env.md | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/docs/source/reference/config-user-env.md b/docs/source/reference/config-user-env.md index 814dd365..facf263f 100644 --- a/docs/source/reference/config-user-env.md +++ b/docs/source/reference/config-user-env.md @@ -1,14 +1,10 @@ -![jupyterhub](https://jupyter.org/assets/homepage/hublogo.svg) - - # Configuring user environments To deploy JupyterHub means you are providing Jupyter notebook environments for multiple users. Often, this includes a desire to configure the user environment in a custom way. -Since the `jupyterhub-singleuser` server extends the standard Jupyter notebook -server, most Jupyter notebook configuration and documentation also applies to single-user environments. +Since the `jupyterhub-singleuser` server extends the standard Jupyter notebook server, most configuration and documentation that applies to Jupyter Notebook applies to the single-user environments. Configuration of user environments typically does not occur through JupyterHub itself, but rather through system- wide Jupyter's configuration, which is inherited by `jupyterhub-singleuser`. @@ -145,12 +141,12 @@ depending on what Spawner you are using. The first category is a **shared system (multi-user host)** where each user has a JupyterHub account, a home directory as well as being a real system user. In this example, shared configuration and installation -must be in a **system-wide** location, such as `/etc/` or `/usr/local` +must be in a 'system-wide' location, such as `/etc/` or `/usr/local` or a custom prefix such as `/opt/conda`. When JupyterHub uses **container-based** Spawners (e.g. KubeSpawner or -DockerSpawner), the 'system-wide' environment is the container image -which is used for users. +DockerSpawner), the 'system-wide' environment is really the container image +used for users. In both cases, you want to _avoid putting configuration in user home directories_ because users can change those configuration settings. Also, @@ -159,7 +155,7 @@ difficult for admins to update later. ## Named servers -By default, in a JupyterHub deployment, each user has only one server. +By default, in a JupyterHub deployment, each user has one server only. JupyterHub can, however, have multiple servers per user. This is mostly useful in deployments where users can configure the environment @@ -229,7 +225,7 @@ JupyterLab is now the default singleuser UI, if available, which is based on the [Jupyter Server][], no longer the legacy [Jupyter Notebook][] server. JupyterHub prior to 2.0 launched the legacy notebook server (`jupyter notebook`), -and Jupyter server could be selected by specifying the following command: +and Jupyter server could be selected by specifying the following: ```python # jupyterhub_config.py From eaeef65560ab4efd52d57111700a71300e921a9b Mon Sep 17 00:00:00 2001 From: Goodness Chris-Ugari Date: Wed, 19 Oct 2022 09:34:42 +0100 Subject: [PATCH 085/214] Update config-proxy.md Update comment --- docs/source/reference/config-proxy.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/reference/config-proxy.md b/docs/source/reference/config-proxy.md index 3c0e1866..09494ea0 100644 --- a/docs/source/reference/config-proxy.md +++ b/docs/source/reference/config-proxy.md @@ -51,7 +51,7 @@ server { listen 80; server_name HUB.DOMAIN.TLD; - # Send a redirect request using the $request_uri that was requested but via HTTPS + # Redirect the request to HTTPS return 302 https://$host$request_uri; } @@ -112,7 +112,7 @@ server { listen 80; server_name NO_HUB.DOMAIN.TLD; - # Send a redirect request using the $request_uri that was requested but via HTTPS + # Redirect the request to HTTPS return 302 https://$host$request_uri; } From 313623256f92eed8ae117be1d0dbae746e6b1312 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Wed, 19 Oct 2022 10:44:15 +0100 Subject: [PATCH 086/214] Update docs/source/contributing/community.md Co-authored-by: Georgiana --- docs/source/contributing/community.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/community.md b/docs/source/contributing/community.md index 870e4309..1c8d79d3 100644 --- a/docs/source/contributing/community.md +++ b/docs/source/contributing/community.md @@ -2,7 +2,7 @@ We use different channels of communication for different purposes. Whichever one you use will depend on what kind of communication you want to engage in. -## Discourse +## Discourse (recommended) We use [Discourse](https://discourse.jupyter.org) for online discussions. Everyone in the Jupyter community is welcome to bring ideas and questions there. From c8aae0ea1c86076d55ebff508a0b84dc90cbd9a8 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Wed, 19 Oct 2022 10:44:41 +0100 Subject: [PATCH 087/214] Update docs/source/contributing/community.md Co-authored-by: Georgiana --- docs/source/contributing/community.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/community.md b/docs/source/contributing/community.md index 1c8d79d3..13d6db19 100644 --- a/docs/source/contributing/community.md +++ b/docs/source/contributing/community.md @@ -4,7 +4,7 @@ We use different channels of communication for different purposes. Whichever one ## Discourse (recommended) -We use [Discourse](https://discourse.jupyter.org) for online discussions. Everyone in the Jupyter community is welcome to bring ideas and questions there. +We use [Discourse](https://discourse.jupyter.org) for online discussions and support questions. Everyone in the Jupyter community is welcome to bring ideas and questions there. All our past and current discussions on Discourse are archived and searchable. This is why we recommend you first go to Discourse, so that discussions remain useful and accessible to the whole community. From 84e1216dda71e0cbf6f473f913501290c6a26f51 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Wed, 19 Oct 2022 10:45:49 +0100 Subject: [PATCH 088/214] Update docs/source/contributing/community.md Co-authored-by: Georgiana --- docs/source/contributing/community.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/docs/source/contributing/community.md b/docs/source/contributing/community.md index 13d6db19..a1aecfa8 100644 --- a/docs/source/contributing/community.md +++ b/docs/source/contributing/community.md @@ -20,8 +20,5 @@ Issues related to a specific authenticator or spawner should be opened in the ap If you cannot find a repository to open your issue in, do not worry! Open the issue in the [main JupyterHub repository](https://github.com/jupyterhub/jupyterhub/) and our community will help you figure it out. -## Mailing List - -A [mailing list](https://groups.google.com/forum/#!forum/jupyter) for all of Project Jupyter exists, along with one for [teaching with Jupyter Notebooks](https://groups.google.com/forum/#!forum/jupyter-education). **NOTE**: Our community is distributed across the world in various timezones, so please be patient if you do not get a response immediately! From d407c96ee89aee5fce87c30ac10ad6783918e63d Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 19 Oct 2022 09:47:26 +0000 Subject: [PATCH 089/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/contributing/community.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/source/contributing/community.md b/docs/source/contributing/community.md index a1aecfa8..4687bf98 100644 --- a/docs/source/contributing/community.md +++ b/docs/source/contributing/community.md @@ -20,5 +20,4 @@ Issues related to a specific authenticator or spawner should be opened in the ap If you cannot find a repository to open your issue in, do not worry! Open the issue in the [main JupyterHub repository](https://github.com/jupyterhub/jupyterhub/) and our community will help you figure it out. - **NOTE**: Our community is distributed across the world in various timezones, so please be patient if you do not get a response immediately! From 7d73d5774ee5b7aae5917b7d11a366238a24dc49 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Wed, 19 Oct 2022 12:37:13 +0100 Subject: [PATCH 090/214] update websecurity.md - fix typos/edit text - add important links --- docs/source/reference/websecurity.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/docs/source/reference/websecurity.md b/docs/source/reference/websecurity.md index 8473c3e9..b5ab2680 100644 --- a/docs/source/reference/websecurity.md +++ b/docs/source/reference/websecurity.md @@ -16,9 +16,9 @@ JupyterHub is designed to be a _simple multi-user server for modestly sized groups_ of **semi-trusted** users. While the design reflects serving semi-trusted users, JupyterHub is not necessarily unsuitable for serving **untrusted** users. -Using JupyterHub with **untrusted** users does mean more work by the +Using JupyterHub with **untrusted** users does mean more work for the administrator. Much care is required to secure a Hub, with extra caution on -protecting users from each other as the Hub is serving untrusted users. +protecting users from each other, since the Hub serves untrusted users. One aspect of JupyterHub's _design simplicity_ for **semi-trusted** users is that the Hub and single-user servers are placed in a _single domain_, behind a @@ -47,7 +47,7 @@ ensure that: - If the `PATH` is used to resolve the single-user executable (instead of using an absolute path), a user **may not** create new files in any `PATH` directory that precedes the directory containing `jupyterhub-singleuser`. - - A user may not modify environment variables (e.g. PATH, PYTHONPATH) for + - A user may not modify environment variables (e.g. `PATH`, `PYTHONPATH`) for their single-user server. - A user **may not** modify the configuration of the notebook server (the `~/.jupyter` or `JUPYTER_CONFIG_DIR` directory). @@ -58,7 +58,7 @@ If any additional services are run on the same domain as the Hub, the services ## Mitigate security issues -Several approaches to mitigating these issues with configuration +The several approaches to mitigating security issues with configuration options provided by JupyterHub include: ### Enable subdomains @@ -76,10 +76,10 @@ resolves the cross-site issues. ### Disable user config -If subdomains are not available or not desirable, JupyterHub provides a +If subdomains are unavailable or undesirable, JupyterHub provides a configuration option `Spawner.disable_user_config`, which can be set to prevent the user-owned configuration files from being loaded. After implementing this -option, PATHs and package installation and PATHs are the other things that the +option, `PATH`s and package installation are the other things that the admin must enforce. ### Prevent spawners from evaluating shell configuration files @@ -119,14 +119,12 @@ extend to securing the `tcp` sockets as well. ## Security audits We recommend that you do periodic reviews of your deployment's security. It's -good practice to keep JupyterHub, configurable-http-proxy, and nodejs -versions up to date. +good practice to keep [JupyterHub](https://readthedocs.org/projects/jupyterhub/), [configurable-http-proxy](https://github.com/jupyterhub/configurable-http-proxy), and [nodejs +versions](https://github.com/nodejs/Release) up to date. A handy website for testing your deployment is [Qualsys' SSL analyzer tool](https://www.ssllabs.com/ssltest/analyze.html). -[configurable-http-proxy]: https://github.com/jupyterhub/configurable-http-proxy - ## Vulnerability reporting If you believe you’ve found a security vulnerability in JupyterHub, or any From 6f25abac2eab5fe4d90b87b88c16cb356bcd23a2 Mon Sep 17 00:00:00 2001 From: Alexander Chosen Okon Date: Wed, 19 Oct 2022 11:49:55 +0000 Subject: [PATCH 091/214] Update config-user-env.md I made a few changes to the spacing used. --- docs/source/reference/config-user-env.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/source/reference/config-user-env.md b/docs/source/reference/config-user-env.md index 0cd315be..10430c12 100644 --- a/docs/source/reference/config-user-env.md +++ b/docs/source/reference/config-user-env.md @@ -18,8 +18,7 @@ This section will focus on user environments, which includes the following: ## Installing packages -To make packages available to users, you will typically install packages -system-wide or in a shared environment. +To make packages available to users, you will typically install packages system-wide or in a shared environment. This installation location should always be in the same environment where `jupyterhub-singleuser` itself is installed, and must be _readable and executable_ by your users. If you want users to be able to install additional packages, it must also be _writable_ by your users. @@ -40,8 +39,7 @@ Alternatively, You may also use conda to install packages. To do this, ensure th and [IPython](https://ipython.readthedocs.io/en/stable/development/config.html) have their own configuration systems. -As a JupyterHub administrator, you will typically want to install and configure environments for all JupyterHub users. For example, let's say you wish for each student in -a class to have the same user environment configuration. +As a JupyterHub administrator, you will typically want to install and configure environments for all JupyterHub users. For example, let's say you wish for each student in a class to have the same user environment configuration. Jupyter and IPython support **"system-wide"** locations for configuration, which is the logical place to put global configuration that you want to affect all users. It's generally more efficient to configure user environments "system-wide",and it's a good practice to avoid creating files in the users' home directories. The typical locations for these config files are: From ecf486d678d15b0a0968a81fa58406a674a522b7 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 19 Oct 2022 11:59:09 +0000 Subject: [PATCH 092/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/reference/websecurity.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/source/reference/websecurity.md b/docs/source/reference/websecurity.md index 6cdf67d9..2e77f176 100644 --- a/docs/source/reference/websecurity.md +++ b/docs/source/reference/websecurity.md @@ -126,7 +126,8 @@ versions](https://github.com/nodejs/Release) up to date. We recommend that you do periodic reviews of your deployment's security. It is good practice to keep JupyterHub, configurable-http-proxy, and nodejs versions up to date. ->>>>>>> main + +> > > > > > > main A handy website for testing your deployment is [Qualsys' SSL analyzer tool](https://www.ssllabs.com/ssltest/analyze.html). From 9c044e863ac73c6d0d37eba06c931528a8ac1a23 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 19 Oct 2022 13:33:48 +0000 Subject: [PATCH 093/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/reference/config-user-env.md | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/docs/source/reference/config-user-env.md b/docs/source/reference/config-user-env.md index d32f4d0a..cbf8abd5 100644 --- a/docs/source/reference/config-user-env.md +++ b/docs/source/reference/config-user-env.md @@ -23,13 +23,11 @@ This section will focus on user environments, which includes the following: To make packages available to users, you will typically install packages system-wide or in a shared environment. - This installation location should always be in the same environment where `jupyterhub-singleuser` itself is installed in, and must be _readable and executable_ by your users. If you want your users to be able to install additional packages, the installation location must also be _writable_ by your users. - If you are using a standard Python installation on your system, use the following command: ```bash @@ -78,6 +76,7 @@ with the following substitutions: ::: To enable Jupyter notebook's internal idle-shutdown behavior (requires notebook ≥ 5.4), set the following in the `/etc/jupyter/jupyter_server_config.py` file: + ```python # shutdown the server after no activity for an hour c.ServerApp.shutdown_no_activity_timeout = 60 * 60 @@ -92,12 +91,10 @@ c.MappingKernelManager.cull_interval = 2 * 60 You may have multiple Jupyter kernels installed and want to make sure that they are available to all of your users. This means installing kernelspecs either system-wide (e.g. in /usr/local/) or in the `sys.prefix` of JupyterHub itself. - Jupyter kernelspec installation is system-wide by default, but some kernels may default to installing kernelspecs in your home directory. These will need to be moved system-wide to ensure that they are accessible. - To see where your kernelspecs are, you can use the following command: ```bash @@ -133,16 +130,13 @@ or a custom prefix such as `/opt/conda`. When JupyterHub uses **container-based** Spawners (e.g. KubeSpawner or DockerSpawner), the 'system-wide' environment is really the container image used for users. - In both cases, you want to _avoid putting configuration in user home directories_ because users can change those configuration settings. Also, home directories typically persist once they are created, thereby making it difficult for admins to update later. ## Named servers - By default, in a JupyterHub deployment, each user has one server only. - JupyterHub can, however, have multiple servers per user. This is mostly useful in deployments where users can configure the environment in which their server will start (e.g. resource requests on an HPC cluster), so that a given user can have multiple configurations running at the same time, without having to stop and restart their own server. @@ -161,11 +155,9 @@ as well as the admin page: ![named servers on the admin page](../images/named-servers-admin.png) - Named servers can be accessed, created, started, stopped, and deleted from these pages. Activity tracking is now per server as well. - To limit the number of **named server** per user by setting a constant value, use this: ```python @@ -192,7 +184,6 @@ If `named_server_limit_per_user` is set to `0`, no limit is enforced. ## Switching back to the classic notebook - By default, the single-user server launches JupyterLab, which is based on [Jupyter Server][]. @@ -215,7 +206,6 @@ no longer the legacy [Jupyter Notebook][] server. JupyterHub prior to 2.0 launched the legacy notebook server (`jupyter notebook`), and the Jupyter server could be selected by specifying the following: - ```python # jupyterhub_config.py c.Spawner.cmd = ["jupyter-labhub"] From c14d8e3446d4a0201371978ecef5eaeb41a1094a Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 20 Oct 2022 03:28:19 +0000 Subject: [PATCH 094/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/rbac/tech-implementation.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/docs/source/rbac/tech-implementation.md b/docs/source/rbac/tech-implementation.md index 2a4c170a..ea6231f7 100644 --- a/docs/source/rbac/tech-implementation.md +++ b/docs/source/rbac/tech-implementation.md @@ -43,10 +43,8 @@ Prior to 3.0, tokens stored _roles_, which meant their scopes were resolved on each request. ::: - API tokens grant access to JupyterHub's APIs. The [RBAC framework](https://auth0.com/docs/manage-users/access-control/rbac) allows for requesting tokens with specific permissions. - RBAC is involved in several stages of the OAuth token flow. When requesting a token via the tokens API (`/users/:name/tokens`), or the token page (`/hub/token`), @@ -78,21 +76,17 @@ Figure 1. Resolving roles and scopes during API token request With the RBAC framework, each authenticated JupyterHub API request is guarded by a scope decorator that specifies which scopes are required in order to gain the access to the API. - When an API request is made, the requesting API token's scopes are again intersected with its owner's (yellow box in {ref}`Figure 2 `) to ensure that the token does not grant more permissions than its owner has at the request time (e.g., due to changing/losing roles). If the owner's roles do not include some scopes of the token, only the _intersection_ of the token's and owner's scopes will be used. For example, using a token with scope `users` whose owner's role scope is `read:users:name` will result in only the `read:users:name` scope being passed on. In the case of no _intersection_, an empty set of scopes will be used. - The parsed scopes are compared to the scopes required to access the API as follows: - if the API scopes are present within the set of parsed scopes, the access is granted and the API returns its "full" response - if that is not the case, another check is utilized to determine if subscopes of the required API scopes can be found in the parsed scope set: - - if found, the RBAC framework employs the {ref}`filtering ` procedures to refine the API response to access only resource attributes corresponding to the passed scopes. For example, providing a scope `read:users:activity!group=class-C` for the `GET /users` API will return a list of user models from group `class-C` containing only the `last_activity` attribute for each user model - - if not found, the access to API is denied {ref}`Figure 2 ` illustrates this process highlighting the steps where the role and scope resolutions as well as filtering occur in orange. From 281658ccce2935c1b070232c5b8c511946fb315a Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Thu, 20 Oct 2022 07:48:52 +0100 Subject: [PATCH 095/214] update websecurity.md Delete erroneous text from merge conflict --- docs/source/reference/websecurity.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/docs/source/reference/websecurity.md b/docs/source/reference/websecurity.md index 2e77f176..4b82c8d2 100644 --- a/docs/source/reference/websecurity.md +++ b/docs/source/reference/websecurity.md @@ -118,16 +118,9 @@ extend to securing the `tcp` sockets as well. ## Security audits -<<<<<<< HEAD We recommend that you do periodic reviews of your deployment's security. It's good practice to keep [JupyterHub](https://readthedocs.org/projects/jupyterhub/), [configurable-http-proxy](https://github.com/jupyterhub/configurable-http-proxy), and [nodejs versions](https://github.com/nodejs/Release) up to date. -======= -We recommend that you do periodic reviews of your deployment's security. It is -good practice to keep JupyterHub, configurable-http-proxy, and nodejs -versions up to date. - -> > > > > > > main A handy website for testing your deployment is [Qualsys' SSL analyzer tool](https://www.ssllabs.com/ssltest/analyze.html). From 1ea557c999d78e6c8129f590ad1df6497e4296b7 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 20 Oct 2022 08:44:58 +0000 Subject: [PATCH 096/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/troubleshooting.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index 60de6613..8e81c7b3 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -161,10 +161,8 @@ your server again. ##### Proxy settings (403 GET) - When your whole JupyterHub sits behind an organization proxy (_not_ a reverse proxy like NGINX as part of your setup and _not_ the configurable-http-proxy) the environment variables `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy` and `https_proxy` might be set. This confuses the jupyterhub single-user servers: When connecting to the Hub for authorization they connect via the proxy instead of directly connecting to the Hub on localhost. The proxy might deny the request (403 GET). This results in the single-user server thinking it has a wrong auth token. To circumvent this you should add `,,localhost,127.0.0.1` to the environment variables `NO_PROXY` and `no_proxy`. - ### Launching Jupyter Notebooks to run as an externally managed JupyterHub service with the `jupyterhub-singleuser` command returns a `JUPYTERHUB_API_TOKEN` error [JupyterHub services](https://jupyterhub.readthedocs.io/en/stable/reference/services.html) allow processes to interact with JupyterHub's REST API. Example use-cases include: @@ -326,7 +324,6 @@ Or use syslog: ### Toree integration with HDFS rack awareness script - The Apache Toree kernel will have an issue when running with JupyterHub if the standard HDFS rack awareness script is used. This will materialize in the logs as a repeated WARN: ```bash From 439b96978e51666431f96e9bca2b2466234b33f8 Mon Sep 17 00:00:00 2001 From: Ngobiri Falyne Date: Thu, 20 Oct 2022 09:58:37 +0100 Subject: [PATCH 097/214] Update troubleshooting.md --- docs/source/troubleshooting.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index 8e81c7b3..29030c71 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -160,7 +160,6 @@ new container. If this was the underlying cause of the issue, you should see your server again. ##### Proxy settings (403 GET) - When your whole JupyterHub sits behind an organization proxy (_not_ a reverse proxy like NGINX as part of your setup and _not_ the configurable-http-proxy) the environment variables `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy` and `https_proxy` might be set. This confuses the jupyterhub single-user servers: When connecting to the Hub for authorization they connect via the proxy instead of directly connecting to the Hub on localhost. The proxy might deny the request (403 GET). This results in the single-user server thinking it has a wrong auth token. To circumvent this you should add `,,localhost,127.0.0.1` to the environment variables `NO_PROXY` and `no_proxy`. ### Launching Jupyter Notebooks to run as an externally managed JupyterHub service with the `jupyterhub-singleuser` command returns a `JUPYTERHUB_API_TOKEN` error @@ -323,7 +322,6 @@ Or use syslog: jupyterhub | logger -t jupyterhub ### Toree integration with HDFS rack awareness script - The Apache Toree kernel will have an issue when running with JupyterHub if the standard HDFS rack awareness script is used. This will materialize in the logs as a repeated WARN: ```bash From c3da12c19513e5e3b079fc65cb4b627a22ed1c91 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 20 Oct 2022 09:00:33 +0000 Subject: [PATCH 098/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/troubleshooting.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index 29030c71..8e81c7b3 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -160,6 +160,7 @@ new container. If this was the underlying cause of the issue, you should see your server again. ##### Proxy settings (403 GET) + When your whole JupyterHub sits behind an organization proxy (_not_ a reverse proxy like NGINX as part of your setup and _not_ the configurable-http-proxy) the environment variables `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy` and `https_proxy` might be set. This confuses the jupyterhub single-user servers: When connecting to the Hub for authorization they connect via the proxy instead of directly connecting to the Hub on localhost. The proxy might deny the request (403 GET). This results in the single-user server thinking it has a wrong auth token. To circumvent this you should add `,,localhost,127.0.0.1` to the environment variables `NO_PROXY` and `no_proxy`. ### Launching Jupyter Notebooks to run as an externally managed JupyterHub service with the `jupyterhub-singleuser` command returns a `JUPYTERHUB_API_TOKEN` error @@ -322,6 +323,7 @@ Or use syslog: jupyterhub | logger -t jupyterhub ### Toree integration with HDFS rack awareness script + The Apache Toree kernel will have an issue when running with JupyterHub if the standard HDFS rack awareness script is used. This will materialize in the logs as a repeated WARN: ```bash From 141f5ea2b4d3def677bf4897b6253040c8fa5b7f Mon Sep 17 00:00:00 2001 From: Ngobiri Falyne Date: Thu, 20 Oct 2022 10:01:44 +0100 Subject: [PATCH 099/214] Update troubleshooting.md --- docs/source/troubleshooting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index 8e81c7b3..66a7512b 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -18,7 +18,7 @@ If you have tried to start the JupyterHub proxy and it fails to start: recent version of [Node](https://nodejs.org). Some versions of Ubuntu/Debian come with a version of Node that is very old, and it is necessary to update Node. -### Sudospawner fails to run +### sudospawner fails to run If the sudospawner script is not found in the path, sudospawner will not run. To avoid this, specify sudospawner's absolute path. For example, start From 145ccfbd4f4820dfa3d94ae36bbe29466d63f60f Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Thu, 20 Oct 2022 10:36:59 +0100 Subject: [PATCH 100/214] update websecurity.md --- docs/source/reference/websecurity.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/source/reference/websecurity.md b/docs/source/reference/websecurity.md index 4b82c8d2..67d9e8f2 100644 --- a/docs/source/reference/websecurity.md +++ b/docs/source/reference/websecurity.md @@ -119,12 +119,14 @@ extend to securing the `tcp` sockets as well. ## Security audits We recommend that you do periodic reviews of your deployment's security. It's -good practice to keep [JupyterHub](https://readthedocs.org/projects/jupyterhub/), [configurable-http-proxy](https://github.com/jupyterhub/configurable-http-proxy), and [nodejs +good practice to keep [JupyterHub](https://readthedocs.org/projects/jupyterhub/), [configurable-http-proxy][], and [nodejs versions](https://github.com/nodejs/Release) up to date. A handy website for testing your deployment is [Qualsys' SSL analyzer tool](https://www.ssllabs.com/ssltest/analyze.html). +[configurable-http-proxy]: https://github.com/jupyterhub/configurable-http-proxy + ## Vulnerability reporting If you believe you have found a security vulnerability in JupyterHub, or any From 91f06f49e0c7fd40b6c1e7fc0c3690194f408801 Mon Sep 17 00:00:00 2001 From: lumenCodes Date: Thu, 20 Oct 2022 12:54:42 +0100 Subject: [PATCH 101/214] update to the spawner basic file --- docs/source/getting-started/spawners-basics.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/getting-started/spawners-basics.md b/docs/source/getting-started/spawners-basics.md index b264c043..9cc02631 100644 --- a/docs/source/getting-started/spawners-basics.md +++ b/docs/source/getting-started/spawners-basics.md @@ -1,7 +1,7 @@ # Spawners and single-user notebook servers -Since the single-user server is an instance of `jupyter notebook`, an entirely separate -multi-process application, there are many aspects of that server that can be configured and a lot +A Spawner starts each single-user notebook server. Since the single-user server is an instance of `jupyter notebook`, an entire separate +multi-process application, many aspects of that server can be configured and there are a lot of ways to express that configuration. At the JupyterHub level, you can set some values on the Spawner. The simplest of these is @@ -14,7 +14,7 @@ expanded to the user's home directory. c.Spawner.notebook_dir = '~/notebooks' ``` -You can also specify extra command line arguments to the notebook server with the following: +You can also specify extra command line arguments to the notebook server with: ```python c.Spawner.args = ['--debug', '--profile=PHYS131'] @@ -30,4 +30,4 @@ Since the single-user server extends the notebook server application, it still loads configuration from the `jupyter_notebook_config.py` config file. Each user may have one of these files in `$HOME/.jupyter/`. Jupyter also supports loading system-wide config files from `/etc/jupyter/`, -which is the place to put the configuration that you want to affect all of your users. +which is the place to put configuration that you want to affect all of your users. From cd3f37f6a6e237d6a195d5a3bfddb7a8cb72409d Mon Sep 17 00:00:00 2001 From: PoorvajaRayas <106921794+PoorvajaRayas@users.noreply.github.com> Date: Thu, 20 Oct 2022 17:24:55 +0530 Subject: [PATCH 102/214] Update troubleshooting.md --- docs/source/troubleshooting.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index 6950264a..5363e295 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -1,6 +1,6 @@ # Troubleshooting -While troubleshooting, you may see unexpected behaviors or receive an error +When troubleshooting, you may see unexpected behaviors or receive an error message. This section provides links for identifying the cause of the problem and how to resolve it. @@ -8,14 +8,14 @@ problem and how to resolve it. ### JupyterHub proxy fails to start -If you have tried to start the JupyterHub proxy and it fails to run: +If you have tried to start the JupyterHub proxy and it fails to start: - check if the JupyterHub IP configuration setting is `c.JupyterHub.ip = '*'`; if it is, try `c.JupyterHub.ip = ''` - Try starting with `jupyterhub --ip=0.0.0.0` **Note**: If this occurs on Ubuntu/Debian, check that you are using a -recent version of [Node](https://nodejs.org). Some versions of Ubuntu/Debian come with an older version +recent version of [Node](https://nodejs.org). Some versions of Ubuntu/Debian come with a very old version of Node and it is necessary to update Node. ### sudospawner fails to run @@ -37,7 +37,7 @@ to the config file, `jupyterhub_config.py`. When nothing is given for these lists, there will be no admins, and all users who can authenticate on the system (i.e. all the Unix users on the server with a password) will be allowed to start a server. The allowed username set lets you limit -this to a particular set of users, and admin_users let you specify who +this to a particular set of users, and admin_users lets you specify who among them may use the admin interface (not necessary, unless you need to do things like inspect other users' servers or modify the user list at runtime). @@ -87,7 +87,7 @@ sudo MY_ENV=abc123 \ ### Error 500 after spawning my single-user server -You receive an Error 500 while accessing the URL `/user//...`. +You receive a 500 error while accessing the URL `/user//...`. This is often seen when your single-user server cannot verify your user cookie with the Hub. @@ -100,7 +100,7 @@ There are two likely reasons for this: #### Symptoms The main symptom is a failure to load _any_ page served by the single-user -server, met with an Error 500. This is typically the first page at `/user/` +server, met with a 500 error. This is typically the first page at `/user/` after logging in or clicking "Start my server". When a single-user notebook server receives a request, the notebook server makes an API request to the Hub to check if the cookie corresponds to the right user. This request is logged. @@ -167,7 +167,7 @@ When your whole JupyterHub sits behind an organization proxy (_not_ a reverse pr [JupyterHub services](https://jupyterhub.readthedocs.io/en/stable/reference/services.html) allow processes to interact with JupyterHub's REST API. Example use-cases include: -- **Secure Testing**: provides a canonical Jupyter Notebook for testing production data to reduce the number of entry points into production systems. +- **Secure Testing**: provide a canonical Jupyter Notebook for testing production data to reduce the number of entry points into production systems. - **Grading Assignments**: provide access to shared Jupyter Notebooks that may be used for management tasks such as grading assignments. - **Private Dashboards**: share dashboards with certain group members. @@ -324,7 +324,7 @@ Or use syslog: ### Toree integration with HDFS rack awareness script -The Apache Toree kernel will have an issue while running with JupyterHub if the standard HDFS +The Apache Toree kernel will have an issue when running with JupyterHub if the standard HDFS rack awareness script is used. This will materialize in the logs as a repeated WARN: ```bash @@ -371,7 +371,7 @@ You can also tail logs to view them in real-time using the `-f` option: ## Troubleshooting commands The following commands provide additional detail about installed packages, -versions, and system information that may be helpful while troubleshooting +versions, and system information that may be helpful when troubleshooting a JupyterHub deployment. The commands are: - System and deployment information From 9fcaf8df52e17b37493330cbdd67268ba0b5f9c8 Mon Sep 17 00:00:00 2001 From: Alexander Chosen Okon Date: Thu, 20 Oct 2022 11:55:23 +0000 Subject: [PATCH 103/214] further simplified the words in the document I made changes to some of the words used by simplifying them to ensure that all users can easily understand the document. --- docs/source/reference/config-user-env.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/reference/config-user-env.md b/docs/source/reference/config-user-env.md index cbf8abd5..3850dcbe 100644 --- a/docs/source/reference/config-user-env.md +++ b/docs/source/reference/config-user-env.md @@ -47,7 +47,7 @@ have their own configuration systems. As a JupyterHub administrator, you will typically want to install and configure environments for all JupyterHub users. For example, let's say you wish for each student in a class to have the same user environment configuration. -Jupyter and IPython support **"system-wide"** locations for configuration, which is the logical place to put global configuration that you want to affect all users. It's generally more efficient to configure user environments "system-wide",and it's a good practice to avoid creating files in the users' home directories. +Jupyter and IPython support **"system-wide"** locations for configuration, which is the logical place to put global configuration that you want to affect all users. It's generally more efficient to configure user environments "system-wide", and it's a good practice to avoid creating files in the users' home directories. The typical locations for these config files are: - **system-wide** in `/etc/{jupyter|ipython}` @@ -140,7 +140,7 @@ By default, in a JupyterHub deployment, each user has one server only. JupyterHub can, however, have multiple servers per user. This is mostly useful in deployments where users can configure the environment in which their server will start (e.g. resource requests on an HPC cluster), so that a given user can have multiple configurations running at the same time, without having to stop and restart their own server. -To allow named servers, use this: +To allow named servers, include this code snippet in your config file: ```python c.JupyterHub.allow_named_servers = True @@ -158,13 +158,13 @@ as well as the admin page: Named servers can be accessed, created, started, stopped, and deleted from these pages. Activity tracking is now per server as well. -To limit the number of **named server** per user by setting a constant value, use this: +To limit the number of **named server** per user by setting a constant value, include this code snippet in your config file: ```python c.JupyterHub.named_server_limit_per_user = 5 ``` -Alternatively, to use a callable/awaitable based on the handler object, use this: +Alternatively, to use a callable/awaitable based on the handler object, include this code snippet in your config file: ```python def named_server_limit_per_user_fn(handler): From 677895c3eb5597a87bebcf7ffb2450403b1f5765 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 20 Oct 2022 11:55:28 +0000 Subject: [PATCH 104/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/troubleshooting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index 5363e295..cbbf20c1 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -15,7 +15,7 @@ If you have tried to start the JupyterHub proxy and it fails to start: - Try starting with `jupyterhub --ip=0.0.0.0` **Note**: If this occurs on Ubuntu/Debian, check that you are using a -recent version of [Node](https://nodejs.org). Some versions of Ubuntu/Debian come with a very old version +recent version of [Node](https://nodejs.org). Some versions of Ubuntu/Debian come with a very old version of Node and it is necessary to update Node. ### sudospawner fails to run From dd8259fb469b0a967009bb804cab0773ac128796 Mon Sep 17 00:00:00 2001 From: PoorvajaRayas <106921794+PoorvajaRayas@users.noreply.github.com> Date: Thu, 20 Oct 2022 17:47:13 +0530 Subject: [PATCH 105/214] Updated the document by making necessary changes --- docs/source/troubleshooting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index cbbf20c1..e27fe48e 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -41,7 +41,7 @@ this to a particular set of users, and admin_users lets you specify who among them may use the admin interface (not necessary, unless you need to do things like inspect other users' servers or modify the user list at runtime). -### What to do when JupyterHub Docker container is not accessible at localhost? +### JupyterHub Docker container is not accessible at localhost Even though the command to start your Docker container exposes port 8000 (`docker run -p 8000:8000 -d --name jupyterhub jupyterhub/jupyterhub jupyterhub`), From f71388633fc282050cde814f59fe4153684df81f Mon Sep 17 00:00:00 2001 From: Kelvin Obidozie Date: Thu, 20 Oct 2022 13:38:12 +0100 Subject: [PATCH 106/214] Added text to documentation for more readability --- docs/source/getting-started/config-basics.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/getting-started/config-basics.md b/docs/source/getting-started/config-basics.md index 8b4babe6..03495a62 100644 --- a/docs/source/getting-started/config-basics.md +++ b/docs/source/getting-started/config-basics.md @@ -1,6 +1,6 @@ # Configuration Basics -The section contains basic information about configuring settings for a JupyterHub +This section contains basic information about configuring settings for a JupyterHub deployment. The [Technical Reference](../reference/index) documentation provides additional details. @@ -49,7 +49,7 @@ that Jupyter uses. ## Configure using command line options -To display all command line options that are available for configuration: +To display all command line options that are available for configuration run the following command: ```bash jupyterhub --help-all @@ -81,7 +81,7 @@ specific [authenticators](./authenticators-users-basics) and [spawners](./spawners-basics) can be set in the configuration file. This enables JupyterHub to be used with a variety of authentication methods or process control and deployment environments. [Some examples](../reference/config-examples), -meant as illustration, are: +meant as illustrations, are: - Using GitHub OAuth instead of PAM with [OAuthenticator](https://github.com/jupyterhub/oauthenticator) - Spawning single-user servers with Docker, using the [DockerSpawner](https://github.com/jupyterhub/dockerspawner) From 248bf8ef835212ee828a43fc5e97a089e6faaa4c Mon Sep 17 00:00:00 2001 From: Eshy10 Date: Thu, 20 Oct 2022 14:25:36 +0100 Subject: [PATCH 107/214] chore: remove capitalize word and "This is" after the colon for environments --- docs/source/admin/upgrading.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index 84e30cbf..7e541726 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -63,11 +63,11 @@ Upgrade JupyterHub packages There are two environments where the ``jupyterhub`` package is installed: -#. The *hub environment*: Which is where the JupyterHub server process +#. The *hub environment*: where the JupyterHub server process runs. This is started with the ``jupyterhub`` command, and is what people generally think of as JupyterHub. -#. The *notebook user environments*: This is where the user notebook +#. The *notebook user environments*: where the user notebook servers are launched from, and is probably custom to your own installation. This could be just one environment (different from the hub environment) that is shared by all users, one environment From 8e111665cd8fd28e9ed91b838699a5ff097818ed Mon Sep 17 00:00:00 2001 From: Toyibat Adele Date: Thu, 20 Oct 2022 17:44:56 +0100 Subject: [PATCH 108/214] Added Punctuations and Capitalized words where necessary. --- docs/source/reference/oauth.md | 54 +++++++++++++++++----------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/docs/source/reference/oauth.md b/docs/source/reference/oauth.md index a658eb90..7611e9be 100644 --- a/docs/source/reference/oauth.md +++ b/docs/source/reference/oauth.md @@ -30,19 +30,19 @@ Some relevant points: Here are some key definitions to keep in mind when we are talking about OAuth. You can also read more detail [here](https://www.oauth.com/oauth2-servers/definitions/). -- **provider** the entity responsible for managing identity and authorization, +- **provider**: The entity responsible for managing identity and authorization, always a web server. JupyterHub is _always_ an oauth provider for JupyterHub's components. When OAuthenticator is used, an external service, such as GitHub or KeyCloak, is also an oauth provider. -- **client** An entity that requests OAuth **tokens** on a user's behalf, +- **client**: An entity that requests OAuth **tokens** on a user's behalf, generally a web server of some kind. OAuth **clients** are services that _delegate_ authentication and/or authorization to an OAuth **provider**. JupyterHub _services_ or single-user _servers_ are OAuth **clients** of the JupyterHub **provider**. When OAuthenticator is used, JupyterHub is itself _also_ an OAuth **client** for the external oauth **provider**, e.g. GitHub. -- **browser** A user's web browser, which makes requests and stores things like cookies -- **token** The secret value used to represent a user's authorization. This is the final product of the OAuth process. -- **code** A short-lived temporary secret that the **client** exchanges +- **browser**: A user's web browser, which makes requests and stores things like cookies. +- **token**: The secret value used to represent a user's authorization. This is the final product of the OAuth process. +- **code**: A short-lived temporary secret that the **client** exchanges for a **token** at the conclusion of oauth, in what's generally called the "oauth callback handler." @@ -56,8 +56,8 @@ A single oauth flow generally goes like this: 1. A **browser** makes an HTTP request to an oauth **client**. 2. There are no credentials, so the client _redirects_ the browser to an "authorize" page on the oauth **provider** with some extra information: - - the oauth **client id** of the client itself - - the **redirect uri** to be redirected back to after completion + - the oauth **client id** of the client itself. + - the **redirect uri** to be redirected back to after completion. - the **scopes** requested, which the user should be presented with to confirm. This is the "X would like to be able to Y on your behalf. Allow this?" page you see on all the "Login with ..." pages around the Internet. 3. During this authorize step, @@ -77,8 +77,8 @@ That's the end of the requests made between the **browser** and the **provider** At this point: -- The browser is authenticated with the _provider_ -- The user's authorized permissions are recorded in an _oauth code_ +- The browser is authenticated with the _provider_. +- The user's authorized permissions are recorded in an _oauth code_. - The _provider_ knows that the given oauth client's requested permissions have been granted, but the client doesn't know this yet. - All requests so far have been made directly by the browser. No requests have originated at the client or provider. @@ -86,8 +86,8 @@ At this point: ### OAuth Client Handles Callback Request Now we get to finish the OAuth process. -Let's dig into what the oauth client does when it handles -the oauth callback request with the +Let's dig into what the OAuth client does when it handles +the OAuth callback request. - The OAuth client receives the _code_ and makes an API request to the _provider_ to exchange the code for a real _token_. This is the first direct request between the OAuth _client_ and the _provider_. @@ -113,24 +113,24 @@ So that's _one_ OAuth process. ## Full sequence of OAuth in JupyterHub -Let's go through the above oauth process in JupyterHub, +Let's go through the above OAuth process in JupyterHub, with specific examples of each HTTP request and what information is contained. -For bonus points, we are using the double-oauth example of JupyterHub configured with GitHubOAuthenticator. +For bonus points, we are using the double-OAuth example of JupyterHub configured with GitHubOAuthenticator. -To disambiguate, we will call the OAuth process where JupyterHub is the **provider** "internal oauth," -and the one with JupyterHub as a **client** "external oauth." +To disambiguate, we will call the OAuth process where JupyterHub is the **provider** "internal OAuth," +and the one with JupyterHub as a **client** "external OAuth." Our starting point: - a user's single-user server is running. Let's call them `danez` -- jupyterhub is running with GitHub as an oauth provider (this means two full instances of oauth), -- Danez has a fresh browser session with no cookies yet +- Jupyterhub is running with GitHub as an OAuth provider (this means two full instances of OAuth), +- Danez has a fresh browser session with no cookies yet. First request: - browser->single-user server running JupyterLab or Jupyter Classic - `GET /user/danez/notebooks/mynotebook.ipynb` -- no credentials, so single-user server (as an oauth **client**) starts internal oauth process with JupyterHub (the **provider**) +- no credentials, so single-user server (as an OAuth **client**) starts internal OAuth process with JupyterHub (the **provider**) - response: 302 redirect -> `/hub/api/oauth2/authorize` with: - client-id=`jupyterhub-user-danez` @@ -138,9 +138,9 @@ First request: Second request, following redirect: -- browser->jupyterhub +- browser->JupyterHub - `GET /hub/api/oauth2/authorize` -- no credentials, so jupyterhub starts external oauth process _with GitHub_ +- no credentials, so JupyterHub starts external OAuth process _with GitHub_ - response: 302 redirect -> `https://github.com/login/oauth/authorize` with: - client-id=`jupyterhub-client-uuid` @@ -154,8 +154,8 @@ c.JupyterHub.authenticator_class = 'github' ``` That means authenticating a request to the Hub itself starts -a _second_, external oauth process with GitHub as a provider. -This external oauth process is optional, though. +a _second_, external OAuth process with GitHub as a provider. +This external OAuth process is optional, though. If you were using the default username+password PAMAuthenticator, this redirect would have been to `/hub/login` instead, to present the user with a login form. @@ -171,7 +171,7 @@ Here, GitHub prompts for login and asks for confirmation of authorization After successful authorization (either by looking up a pre-existing authorization, or recording it via form submission) -GitHub issues an **oauth code** and redirects to `/hub/oauth_callback?code=github-code` +GitHub issues an **OAuth code** and redirects to `/hub/oauth_callback?code=github-code` Next request: @@ -194,9 +194,9 @@ The second: - request made with access **token** in the `Authorization` header - response is the user model, including username, email, etc. -Now the external oauth callback request completes with: +Now the external OAuth callback request completes with: -- set cookie on `/hub/` path, recording jupyterhub authentication so we don't need to do external oauth with GitHub again for a while +- set cookie on `/hub/` path, recording jupyterhub authentication so we don't need to do external OAuth with GitHub again for a while - redirect -> `/hub/api/oauth2/authorize` 🎉 At this point, we have completed our first OAuth flow! 🎉 @@ -211,14 +211,14 @@ Now, we get our first repeated request: 2. automatically accepts authorization (shortcut taken when a user is visiting their own server) - redirect -> `/user/danez/oauth_callback?code=jupyterhub-code` -Here, we start the same oauth callback process as before, but at Danez's single-user server for the _internal_ oauth +Here, we start the same OAuth callback process as before, but at Danez's single-user server for the _internal_ OAuth. - browser->single-user server - `GET /user/danez/oauth_callback` (in handler) -Inside the internal oauth callback handler, +Inside the internal OAuth callback handler, Danez's server makes two API requests to JupyterHub: The first: From fedcd22b0cfd3c7dde87e0693f3f1d8211a5ffd3 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Fri, 21 Oct 2022 01:29:31 +0100 Subject: [PATCH 109/214] update spawners.md --- docs/source/reference/spawners.md | 36 +++++++++++++++---------------- 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/source/reference/spawners.md b/docs/source/reference/spawners.md index afb58744..921ed202 100644 --- a/docs/source/reference/spawners.md +++ b/docs/source/reference/spawners.md @@ -4,9 +4,9 @@ A [Spawner][] starts each single-user notebook server. The Spawner represents an abstract interface to a process, and a custom Spawner needs to be able to take three actions: -- start the process -- poll whether the process is still running -- stop the process +- start a process +- poll whether a process is still running +- stop a process ## Examples @@ -15,9 +15,9 @@ Some examples include: - [DockerSpawner](https://github.com/jupyterhub/dockerspawner) for spawning user servers in Docker containers - `dockerspawner.DockerSpawner` for spawning identical Docker containers for - each users + each user - `dockerspawner.SystemUserSpawner` for spawning Docker containers with an - environment and home directory for each users + environment and home directory for each user - both `DockerSpawner` and `SystemUserSpawner` also work with Docker Swarm for launching containers on remote machines - [SudoSpawner](https://github.com/jupyterhub/sudospawner) enables JupyterHub to @@ -34,7 +34,7 @@ Some examples include: ### Spawner.start -`Spawner.start` should start the single-user server for a single user. +`Spawner.start` starts a single-user server for a single user. Information about the user can be retrieved from `self.user`, an object encapsulating the user's name, authentication, and server info. @@ -69,13 +69,13 @@ via relaxing the `Spawner.start_timeout` config value. #### Note on IPs and ports -`Spawner.ip` and `Spawner.port` attributes set the _bind_ url, +`Spawner.ip` and `Spawner.port` attributes set the _bind_ URL, which the single-user server should listen on (passed to the single-user process via the `JUPYTERHUB_SERVICE_URL` environment variable). -The _return_ value is the ip and port (or full url) the Hub should _connect to_. +The _return_ value is the `ip` and `port` (or full URL) the Hub should _connect to_. These are not necessarily the same, and usually won't be in any Spawner that works with remote resources or containers. -The default for Spawner.ip, and Spawner.port is `127.0.0.1:{random}`, +The default for `Spawner.ip`, and `Spawner.port` is `127.0.0.1:{random}`, which is appropriate for Spawners that launch local processes, where everything is on localhost and each server needs its own port. For remote or container Spawners, it will often make sense to use a different value, @@ -111,7 +111,7 @@ class MySpawner(Spawner): #### Exception handling -When `Spawner.start` raises an Exception, a message can be passed on to the user via the exception via a `.jupyterhub_html_message` or `.jupyterhub_message` attribute. +When `Spawner.start` raises an Exception, a message can be passed on to the user via the exception using a `.jupyterhub_html_message` or `.jupyterhub_message` attribute. When the Exception has a `.jupyterhub_html_message` attribute, it will be rendered as HTML to the user. @@ -121,11 +121,11 @@ If both attributes are not present, the Exception will be shown to the user as u ### Spawner.poll -`Spawner.poll` should check if the spawner is still running. +`Spawner.poll` checks if the spawner is still running. It should return `None` if it is still running, and an integer exit status, otherwise. -For the local process case, `Spawner.poll` uses `os.kill(PID, 0)` +In the case of local processes, `Spawner.poll` uses `os.kill(PID, 0)` to check if the local process is still running. On Windows, it uses `psutil.pid_exists`. ### Spawner.stop @@ -141,7 +141,7 @@ A JSON-able dictionary of state can be used to store persisted information. Unlike start, stop, and poll methods, the state methods must not be coroutines. -For the single-process case, the Spawner state is only the process ID of the server: +In the case of single processes, the Spawner state is only the process ID of the server: ```python def get_state(self): @@ -297,18 +297,18 @@ Additional variables can be specified via the `Spawner.environment` configuratio The process environment is returned by `Spawner.get_env`, which specifies the following environment variables: -- JUPYTERHUB*SERVICE_URL - the \_bind* url where the server should launch its http server (`http://127.0.0.1:12345`). +- JUPYTERHUB*SERVICE_URL - the \_bind* URL where the server should launch its http server (`http://127.0.0.1:12345`). This includes Spawner.ip and Spawner.port; _new in 2.0, prior to 2.0 ip,port were on the command-line and only if specified_ - JUPYTERHUB_SERVICE_PREFIX - the URL prefix the service will run on (e.g. `/user/name/`) - JUPYTERHUB_USER - the JupyterHub user's username - JUPYTERHUB_SERVER_NAME - the server's name, if using named servers (default server has an empty name) -- JUPYTERHUB_API_URL - the full url for the JupyterHub API (http://17.0.0.1:8001/hub/api) -- JUPYTERHUB_BASE_URL - the base url of the whole jupyterhub deployment, i.e. the bit before `hub/` or `user/`, +- JUPYTERHUB_API_URL - the full URL for the JupyterHub API (http://17.0.0.1:8001/hub/api) +- JUPYTERHUB_BASE_URL - the base URL of the whole jupyterhub deployment, i.e. the bit before `hub/` or `user/`, as set by c.JupyterHub.base_url (default: `/`) - JUPYTERHUB_API_TOKEN - the API token the server can use to make requests to the Hub. This is also the OAuth client secret. - JUPYTERHUB_CLIENT_ID - the OAuth client ID for authenticating visitors. -- JUPYTERHUB_OAUTH_CALLBACK_URL - the callback URL to use in oauth, typically `/user/:name/oauth_callback` +- JUPYTERHUB_OAUTH_CALLBACK_URL - the callback URL to use in OAuth, typically `/user/:name/oauth_callback` - JUPYTERHUB_OAUTH_ACCESS_SCOPES - the scopes required to access the server (called JUPYTERHUB_OAUTH_SCOPES prior to 3.0) - JUPYTERHUB_OAUTH_CLIENT_ALLOWED_SCOPES - the scopes the service is allowed to request. If no scopes are requested explicitly, these scopes will be requested. @@ -326,7 +326,7 @@ Optional environment variables, depending on configuration: sets maximum log level when Spawner.debug is True (new in 2.0, previously passed via CLI) -- JUPYTERHUB*[MEM|CPU]*[LIMIT_GUARANTEE] - the values of cpu and memory limits and guarantees. +- JUPYTERHUB*[MEM|CPU]*[LIMIT_GUARANTEE] - the values of CPU and memory limits and guarantees. These are not expected to be enforced by the process, but are made available as a hint, e.g. for resource monitoring extensions. From 69869ee8230bbbfe37a9bef381f88da740963e19 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Fri, 21 Oct 2022 01:49:36 +0100 Subject: [PATCH 110/214] update spawners.md --- docs/source/reference/spawners.md | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/docs/source/reference/spawners.md b/docs/source/reference/spawners.md index 921ed202..5d774a2d 100644 --- a/docs/source/reference/spawners.md +++ b/docs/source/reference/spawners.md @@ -34,7 +34,7 @@ Some examples include: ### Spawner.start -`Spawner.start` starts a single-user server for a single user. +`Spawner.start` should start a single-user server for a single user. Information about the user can be retrieved from `self.user`, an object encapsulating the user's name, authentication, and server info. @@ -283,7 +283,7 @@ would result in spawning the command: my-singleuser-wrapper --debug --flag ``` -The `Spawner.get_args()` method is how Spawner.args is accessed, +The `Spawner.get_args()` method is how `Spawner.args` is accessed, and can be used by Spawners to customize/extend user-provided arguments. Prior to 2.0, JupyterHub unconditionally added certain options _if specified_ to the command-line, @@ -297,14 +297,14 @@ Additional variables can be specified via the `Spawner.environment` configuratio The process environment is returned by `Spawner.get_env`, which specifies the following environment variables: -- JUPYTERHUB*SERVICE_URL - the \_bind* URL where the server should launch its http server (`http://127.0.0.1:12345`). - This includes Spawner.ip and Spawner.port; _new in 2.0, prior to 2.0 ip,port were on the command-line and only if specified_ +- JUPYTERHUB*SERVICE_URL - the \_bind* URL where the server should launch its HTTP server (`http://127.0.0.1:12345`). + This includes `Spawner.ip` and `Spawner.port`; _new in 2.0, prior to 2.0 `ip`, `port` were on the command-line and only if specified_ - JUPYTERHUB_SERVICE_PREFIX - the URL prefix the service will run on (e.g. `/user/name/`) - JUPYTERHUB_USER - the JupyterHub user's username - JUPYTERHUB_SERVER_NAME - the server's name, if using named servers (default server has an empty name) - JUPYTERHUB_API_URL - the full URL for the JupyterHub API (http://17.0.0.1:8001/hub/api) - JUPYTERHUB_BASE_URL - the base URL of the whole jupyterhub deployment, i.e. the bit before `hub/` or `user/`, - as set by c.JupyterHub.base_url (default: `/`) + as set by `c.JupyterHub.base_url` (default: `/`) - JUPYTERHUB_API_TOKEN - the API token the server can use to make requests to the Hub. This is also the OAuth client secret. - JUPYTERHUB_CLIENT_ID - the OAuth client ID for authenticating visitors. @@ -315,15 +315,15 @@ The process environment is returned by `Spawner.get_env`, which specifies the fo Optional environment variables, depending on configuration: -- JUPYTERHUB*SSL*[KEYFILE|CERTFILE|CLIENT_CI] - SSL configuration, when internal_ssl is enabled -- JUPYTERHUB_ROOT_DIR - the root directory of the server (notebook directory), when Spawner.notebook_dir is defined (new in 2.0) -- JUPYTERHUB_DEFAULT_URL - the default URL for the server (for redirects from /user/:name/), - if Spawner.default_url is defined +- JUPYTERHUB*SSL*[KEYFILE|CERTFILE|CLIENT_CI] - SSL configuration, when `internal_ssl` is enabled +- JUPYTERHUB_ROOT_DIR - the root directory of the server (notebook directory), when `Spawner.notebook_dir` is defined (new in 2.0) +- JUPYTERHUB_DEFAULT_URL - the default URL for the server (for redirects from `/user/:name/`), + if `Spawner.default_url` is defined (new in 2.0, previously passed via CLI) -- JUPYTERHUB_DEBUG=1 - generic debug flag, sets maximum log level when Spawner.debug is True +- JUPYTERHUB_DEBUG=1 - generic debug flag, sets maximum log level when `Spawner.debug` is True (new in 2.0, previously passed via CLI) - JUPYTERHUB_DISABLE_USER_CONFIG=1 - disable loading user config, - sets maximum log level when Spawner.debug is True (new in 2.0, + sets maximum log level when `Spawner.debug` is True (new in 2.0, previously passed via CLI) - JUPYTERHUB*[MEM|CPU]*[LIMIT_GUARANTEE] - the values of CPU and memory limits and guarantees. @@ -338,7 +338,7 @@ guarantees on resources, such as CPU and memory. To provide a consistent experience for sysadmins and users, we provide a standard way to set and discover these resource limits and guarantees, such as for memory and CPU. For the limits and guarantees to be useful, **the spawner must implement -support for them**. For example, LocalProcessSpawner, the default +support for them**. For example, `LocalProcessSpawner`, the default spawner, does not support limits and guarantees. One of the spawners that supports limits and guarantees is the `systemdspawner`. @@ -367,7 +367,7 @@ limits or guarantees are provided, and no environment values are set. `c.Spawner.cpu_limit`: In supported spawners, you can set `c.Spawner.cpu_limit` to limit the total number of cpu-cores that a single-user notebook server can use. These can be fractional - `0.5` means 50% -of one CPU core, `4.0` is 4 cpu-cores, etc. This value is also set in the +of one CPU core, `4.0` is 4 CPU-cores, etc. This value is also set in the single-user notebook server's environment variable `CPU_LIMIT`. The limit does not claim that you will be able to use all the CPU up to your limit as other higher priority applications might be taking up CPU. From a71f6be00176dd1ae5e4da434ff2bfb1f67ce097 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Fri, 21 Oct 2022 12:04:45 +0100 Subject: [PATCH 111/214] Update docs/source/reference/spawners.md Co-authored-by: Sarah Gibson <44771837+sgibson91@users.noreply.github.com> --- docs/source/reference/spawners.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/reference/spawners.md b/docs/source/reference/spawners.md index 5d774a2d..bd6bd5c0 100644 --- a/docs/source/reference/spawners.md +++ b/docs/source/reference/spawners.md @@ -72,7 +72,7 @@ via relaxing the `Spawner.start_timeout` config value. `Spawner.ip` and `Spawner.port` attributes set the _bind_ URL, which the single-user server should listen on (passed to the single-user process via the `JUPYTERHUB_SERVICE_URL` environment variable). -The _return_ value is the `ip` and `port` (or full URL) the Hub should _connect to_. +The _return_ value is the IP and port (or full URL) the Hub should _connect to_. These are not necessarily the same, and usually won't be in any Spawner that works with remote resources or containers. The default for `Spawner.ip`, and `Spawner.port` is `127.0.0.1:{random}`, From 37b82f7c2ae45f3433326d052109e1ba3e36f1aa Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Fri, 21 Oct 2022 12:05:12 +0100 Subject: [PATCH 112/214] Update docs/source/reference/spawners.md Co-authored-by: Sarah Gibson <44771837+sgibson91@users.noreply.github.com> --- docs/source/reference/spawners.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/reference/spawners.md b/docs/source/reference/spawners.md index bd6bd5c0..80b2bccd 100644 --- a/docs/source/reference/spawners.md +++ b/docs/source/reference/spawners.md @@ -298,7 +298,7 @@ Additional variables can be specified via the `Spawner.environment` configuratio The process environment is returned by `Spawner.get_env`, which specifies the following environment variables: - JUPYTERHUB*SERVICE_URL - the \_bind* URL where the server should launch its HTTP server (`http://127.0.0.1:12345`). - This includes `Spawner.ip` and `Spawner.port`; _new in 2.0, prior to 2.0 `ip`, `port` were on the command-line and only if specified_ + This includes `Spawner.ip` and `Spawner.port`; _new in 2.0, prior to 2.0 IP, port were on the command-line and only if specified_ - JUPYTERHUB_SERVICE_PREFIX - the URL prefix the service will run on (e.g. `/user/name/`) - JUPYTERHUB_USER - the JupyterHub user's username - JUPYTERHUB_SERVER_NAME - the server's name, if using named servers (default server has an empty name) From 69e973d53a10ca07b04199c6ee55850a4069ef28 Mon Sep 17 00:00:00 2001 From: Emmanuella Orioma Date: Fri, 21 Oct 2022 13:21:20 +0100 Subject: [PATCH 113/214] Update use-case.md Worked on the documentation page (jupyterhub/docs/source/rbac/use-case.md) Added a wikipedia reference to [RBCA framework] and also emphasized on the solution under the *service to cull idle servers* --- docs/source/rbac/use-cases.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/rbac/use-cases.md b/docs/source/rbac/use-cases.md index d0858ae3..4169f055 100644 --- a/docs/source/rbac/use-cases.md +++ b/docs/source/rbac/use-cases.md @@ -9,12 +9,12 @@ To determine which scopes a role should have, one can follow these steps: 5. Customize the scopes with filters if needed 6. Define the role with required scopes and assign to users/services/groups/tokens -Below, different use cases are presented on how to use the RBAC framework. +Below, different use cases are presented on how to use the [RBAC framework](https://en.wikipedia.org/wiki/Role-based_access_control). ## Service to cull idle servers Finding and shutting down idle servers can save a lot of computational resources. -We can make use of [jupyterhub-idle-culler](https://github.com/jupyterhub/jupyterhub-idle-culler) to manage this for us. +**We can make use of [jupyterhub-idle-culler](https://github.com/jupyterhub/jupyterhub-idle-culler) to manage this for us.** Below follows a short tutorial on how to add a cull-idle service in the RBAC system. 1. Install the cull-idle server script with `pip install jupyterhub-idle-culler`. From 4e45b89ed1d5b105ef8f302ceaa65a04d13eb1b6 Mon Sep 17 00:00:00 2001 From: Teniola Olowookere <90247181+Teniola-theDev@users.noreply.github.com> Date: Fri, 21 Oct 2022 14:40:24 +0100 Subject: [PATCH 114/214] Capitalised some words --- .../authenticators-users-basics.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/docs/source/getting-started/authenticators-users-basics.md b/docs/source/getting-started/authenticators-users-basics.md index d0d7d41d..63c16e22 100644 --- a/docs/source/getting-started/authenticators-users-basics.md +++ b/docs/source/getting-started/authenticators-users-basics.md @@ -1,6 +1,6 @@ # Authentication and User Basics -The default Authenticator uses [PAM][] to authenticate system users with +The default Authenticator uses [PAM][] (Pluggable Authentication Module) to authenticate system users with their username and password. With the default Authenticator, any user with an account and password on the system will be allowed to login. @@ -25,9 +25,10 @@ If this configuration value is not set, then **all authenticated users will be a ```{note} As of JupyterHub 2.0, the full permissions of `admin_users` should not be required. -Instead, you can assign [roles](https://jupyterhub.readthedocs.io/en/stable/rbac/roles.html#define-role-target) to users or groups +Instead, you can assign **roles** to users or groups with only the scopes they require. ``` +**[roles](https://jupyterhub.readthedocs.io/en/stable/rbac/roles.html#define-role-target)** Admin users of JupyterHub, `admin_users`, can add and remove users from the user `allowed_users` set. `admin_users` can take actions on other users' @@ -42,7 +43,7 @@ c.Authenticator.admin_users = {'mal', 'zoe'} Users in the admin set are automatically added to the user `allowed_users` set, if they are not already present. -Each authenticator may have different ways of determining whether a user is an +Each Authenticator may have different ways of determining whether a user is an administrator. By default, JupyterHub uses the PAMAuthenticator which provides the `admin_groups` option and can set administrator status based on a user group. For example, we can let any user in the `wheel` group be an admin: @@ -72,11 +73,11 @@ After starting the Hub once, it is not sufficient to **remove** a user from the allowed users set in your config file. You must also remove the user from the Hub's database, either by deleting the user from JupyterHub's admin page, or you can clear the `jupyterhub.sqlite` database and start -fresh. +afresh. ## Use LocalAuthenticator to create system users -The `LocalAuthenticator` is a special kind of authenticator that has +The `LocalAuthenticator` is a special kind of Authenticator that has the ability to manage users on the local system. When you try to add a new user to the Hub, a `LocalAuthenticator` will check if the user already exists. If you set the configuration value, `create_system_users`, @@ -118,8 +119,8 @@ with any provider, is also available. ## Use DummyAuthenticator for testing -The `DummyAuthenticator` is a simple authenticator that -allows for any username/password unless a global password has been set. If +The `DummyAuthenticator` is a simple Authenticator that +allows for any username or password unless a global password has been set. If set, it will allow for any username as long as the correct password is provided. To set a global password, add this to the config file: @@ -127,5 +128,5 @@ To set a global password, add this to the config file: c.DummyAuthenticator.password = "some_password" ``` -[pam]: https://en.wikipedia.org/wiki/Pluggable_authentication_module +[PAM]: https://en.wikipedia.org/wiki/Pluggable_authentication_module [oauthenticator]: https://github.com/jupyterhub/oauthenticator From 794a1aa70ff2128b7ebc40757abe29db7e1f9bb0 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 21 Oct 2022 13:44:45 +0000 Subject: [PATCH 115/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/getting-started/authenticators-users-basics.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/source/getting-started/authenticators-users-basics.md b/docs/source/getting-started/authenticators-users-basics.md index 63c16e22..405d0208 100644 --- a/docs/source/getting-started/authenticators-users-basics.md +++ b/docs/source/getting-started/authenticators-users-basics.md @@ -28,6 +28,7 @@ should not be required. Instead, you can assign **roles** to users or groups with only the scopes they require. ``` + **[roles](https://jupyterhub.readthedocs.io/en/stable/rbac/roles.html#define-role-target)** Admin users of JupyterHub, `admin_users`, can add and remove users from @@ -128,5 +129,5 @@ To set a global password, add this to the config file: c.DummyAuthenticator.password = "some_password" ``` -[PAM]: https://en.wikipedia.org/wiki/Pluggable_authentication_module +[pam]: https://en.wikipedia.org/wiki/Pluggable_authentication_module [oauthenticator]: https://github.com/jupyterhub/oauthenticator From aebe33b62bba7e4f1a85fd4629366b4d6c7d251c Mon Sep 17 00:00:00 2001 From: Min RK Date: Fri, 21 Oct 2022 15:59:49 +0200 Subject: [PATCH 116/214] Remove redundant ref target for roles it's already addressable at that same target name, having this here results in ambiguous ref targets in MyST --- docs/source/rbac/roles.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/source/rbac/roles.md b/docs/source/rbac/roles.md index 4e7e4a10..7828c8cd 100644 --- a/docs/source/rbac/roles.md +++ b/docs/source/rbac/roles.md @@ -1,5 +1,3 @@ -(roles)= - # Roles JupyterHub provides four (4) roles that are available by default: From b741a30dc3863d8bef24d393cd71ccb56bac914b Mon Sep 17 00:00:00 2001 From: Teniola Olowookere <90247181+Teniola-theDev@users.noreply.github.com> Date: Fri, 21 Oct 2022 20:25:21 +0100 Subject: [PATCH 117/214] Update docs/source/getting-started/authenticators-users-basics.md I committed your suggestion about changing the target for the "roles". Thank you! Co-authored-by: Min RK --- docs/source/getting-started/authenticators-users-basics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/getting-started/authenticators-users-basics.md b/docs/source/getting-started/authenticators-users-basics.md index 405d0208..5b169eb2 100644 --- a/docs/source/getting-started/authenticators-users-basics.md +++ b/docs/source/getting-started/authenticators-users-basics.md @@ -25,7 +25,7 @@ If this configuration value is not set, then **all authenticated users will be a ```{note} As of JupyterHub 2.0, the full permissions of `admin_users` should not be required. -Instead, you can assign **roles** to users or groups +Instead, you can assign [roles](define-role-target) to users or groups with only the scopes they require. ``` From 5a16d770da30a388efd10001d61293e78c06179a Mon Sep 17 00:00:00 2001 From: Allan Wasega Date: Fri, 21 Oct 2022 23:41:06 +0300 Subject: [PATCH 118/214] Updated the server-api file as per review comments --- docs/source/reference/server-api.md | 55 +++++++++++++++-------------- 1 file changed, 28 insertions(+), 27 deletions(-) diff --git a/docs/source/reference/server-api.md b/docs/source/reference/server-api.md index 3fbc8e9f..769a5a13 100644 --- a/docs/source/reference/server-api.md +++ b/docs/source/reference/server-api.md @@ -1,6 +1,6 @@ # Starting servers with the JupyterHub API -Sometimes, such as when working with applications such as [BinderHub](https://binderhub.readthedocs.io), it may be necessary to launch Jupyter-based services on behalf of your users. Doing so can be achieved through JupyterHub's [REST API](../reference/rest.md), which allows one to launch and manage servers on behalf of users through API calls instead of through the JupyterHub UI. In doing so, you can take advantage of other user/launch/lifecycle patterns that are not natively supported by JupyterHub, all without the need to develop the server management features of JupyterHub Spawners and/or Authenticators. +Sometimes, when working with applications such as [BinderHub](https://binderhub.readthedocs.io), it may be necessary to launch Jupyter-based services on behalf of your users. Doing so can be achieved through JupyterHub's [REST API](../reference/rest.md), which allows one to launch and manage servers on behalf of users through API calls instead of through the JupyterHub UI. In doing so, you can take advantage of other user/launch/lifecycle patterns that are not natively supported by JupyterHub, all without the need to develop the server management features of JupyterHub Spawners and/or Authenticators. This tutorial goes through the processes involved while working with the JupyterHub API to manage servers for users. In particular, it covers how to: @@ -10,19 +10,19 @@ This tutorial goes through the processes involved while working with the Jupyter 4. [Communicate with servers](communicating) 5. [Stop servers](stopping) -In the end, we also provide a sample Python code that can be used to implement these steps. +In the end, we also provide sample Python code that can be used to implement these steps. (checking)= ## Checking server status -First, request information about a particular user using a GET request as below. +First, request information about a particular user using a GET request: ``` GET /hub/api/users/:username ``` -The response you get will include a `servers` field, which is a dictionary, as shown in the JSON-formatted response below. +The response you get will include a `servers` field, which is a dictionary, as shown in this JSON-formatted response: **Required scope: `read:servers`** @@ -43,7 +43,7 @@ The response you get will include a `servers` field, which is a dictionary, as s Many JupyterHub deployments only use a 'default' server, represented as an empty string `''` for a name. An investigation of the `servers` field can yield one of two results. First, it can be empty as in the sample JSON response above. In such a case, the user has no running servers. -However, should the user have running servers, then the returned dict should contain various information, as shown in the response below. +However, should the user have running servers, then the returned dict should contain various information, as shown in this response: ```json "servers": { @@ -101,7 +101,7 @@ The two responses above are from a user with no servers and another with one `re } ``` -Note that `ready` is assigned `false` and `pending` is assigned `spawn`, meaning that the server is not ready and attempting to access it may not work as it is still in the process of spawning. +Note that `ready` is `false` and `pending` has the value `spawn`, meaning that the server is not ready and attempting to access it may not work as it is still in the process of spawning. We'll get more into this below in [waiting for a server][]. [waiting for a server]: waiting @@ -109,7 +109,7 @@ Note that `ready` is assigned `false` and `pending` is assigned `spawn`, meaning ## Starting servers -To start a server, make the following API request: +To start a server, make this API request: ``` POST /hub/api/users/:username/servers/[:servername] @@ -130,19 +130,19 @@ but is not immediately ready. As a result, the server shows `pending: 'spawn'` a ## Waiting for a server to start -After receiving a `202 Accepted` response, you have to wait for it to start. Two approaches can be applied to establish when the server is ready: +After receiving a `202 Accepted` response, you have to wait for the server to start. +Two approaches can be applied to establish when the server is ready: 1. {ref}`Polling the server model ` -2. {ref}`Using the Progress API ` +2. {ref}`Using the progress API ` (polling)= ### Polling the server model -The simplest way to check if a server is ready is to programmatically query the server model until the following two conditions are true: - -: 1. The server name is contained in the `servers` response, and -: 2. `servers['servername']['ready']` is true +The simplest way to check if a server is ready is to programmatically query the server model until two conditions are true: +1. The server name is contained in the `servers` response, and +2. `servers['servername']['ready']` is true. The Python code snippet below can be used to check if a server is ready: @@ -167,13 +167,13 @@ def server_ready(hub_url, user, server_name="", token): return False ``` -The function polls the server until `ready` is true. +You can keep making this check until `ready` is true. (progress)= -### Using the Progress API +### Using the progress API -The most _efficient_ way to wait for a server to start is by using the Progress API. The progress URL is available in the server model under `progress_url` and has the form `/hub/api/users/:user/servers/:servername/progress`. +The most _efficient_ way to wait for a server to start is by using the progress API. The progress URL is available in the server model under `progress_url` and has the form `/hub/api/users/:user/servers/:servername/progress`. The default server progress can be accessed at `:user/servers//progress` or `:user/server/progress` as demonstrated in the following GET request: @@ -183,7 +183,7 @@ GET /hub/api/users/:user/servers/:servername/progress **Required scope: `read:servers`** -The Progress API is an example of a [EventStream][] API. Therefore, as in a typical event stream, messages are _streamed_ and delivered in the form: +The progress API is an example of an [EventStream][] API. Therefore, messages are _streamed_ and delivered in the form: ``` data: {"progress": 10, "message": "...", ...} @@ -217,7 +217,7 @@ ready url : only present if `ready` is true; will be the server's URL -The Progress API can be used even with fully ready servers. In such a situation, there will only be one event response of the form: +The progress API can be used even with fully ready servers. In such a situation, there will only be one event response of the form: ```json { @@ -231,7 +231,7 @@ The Progress API can be used even with fully ready servers. In such a situation, In this case, `ready` and `url` are the same as in the server model, and `ready` will always be true. -A significant advantage of the Progress API is that it shows the status of the server through a stream of messages. An example of a typical complete stream from the API is as shown below: +A significant advantage of the progress API is that it shows the status of the server through a stream of messages. Below is an example of a typical complete stream from the API: ``` @@ -269,9 +269,9 @@ It will now be absent from the user `servers` model. : This code means your request was accepted, but is not yet completely processed. The server has `pending: 'stop'` at this point. -There is no Progress API for checking when a server actually stops. Thus, the only available alternative is to poll the server and wait for it to disappear from the user `servers` model. +There is no progress API for checking when a server actually stops. Thus, the only available alternative is to poll the server and wait for it to disappear from the user `servers` model. -The Python code snippet below can be used to check if a server stops: +This Python code snippet can be used to check if a server stops: ```{literalinclude} ../../../examples/server-api/start-stop-server.py :language: python @@ -282,7 +282,8 @@ The Python code snippet below can be used to check if a server stops: ## Communicating with servers -JupyterHub tokens with the `access:servers` scope can be used to communicate with servers themselves. The tokens can be the same as those you used to launch your service. +JupyterHub tokens with the `access:servers` scope can be used to communicate with servers themselves. +The tokens can be the same as those you used to launch your service. ```{note} Access scopes are new in JupyterHub 2.0. @@ -305,13 +306,13 @@ tying all the above mentioned steps together. In summary, the steps involved while managing servers on behalf of users are: -1. get user information from `/user/:name` -2. the server model includes a `ready` state to tell you if it's ready -3. if it's not ready, you can follow up with `progress_url` to wait for it -4. if it is ready, you can use the `url` field to link directly to the running server +1. Get user information from `/user/:name`. +2. The server model includes a `ready` state to tell you if it's ready. +3. If it's not ready, you can follow up with `progress_url` to wait for it. +4. If it is ready, you can use the `url` field to link directly to the running server. The example below demonstrates starting and stopping servers via the JupyterHub API, -including waiting for them to start via the Progress API and waiting for them to stop via polling the user model. +including waiting for them to start via the progress API and waiting for them to stop by polling the user model. ```{literalinclude} ../../../examples/server-api/start-stop-server.py :language: python From 7ce4b1399832f2f7edc627c16a09090a279319bc Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 21 Oct 2022 20:41:35 +0000 Subject: [PATCH 119/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/reference/server-api.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/source/reference/server-api.md b/docs/source/reference/server-api.md index 769a5a13..fa8ac461 100644 --- a/docs/source/reference/server-api.md +++ b/docs/source/reference/server-api.md @@ -130,7 +130,7 @@ but is not immediately ready. As a result, the server shows `pending: 'spawn'` a ## Waiting for a server to start -After receiving a `202 Accepted` response, you have to wait for the server to start. +After receiving a `202 Accepted` response, you have to wait for the server to start. Two approaches can be applied to establish when the server is ready: 1. {ref}`Polling the server model ` @@ -141,6 +141,7 @@ Two approaches can be applied to establish when the server is ready: ### Polling the server model The simplest way to check if a server is ready is to programmatically query the server model until two conditions are true: + 1. The server name is contained in the `servers` response, and 2. `servers['servername']['ready']` is true. @@ -282,7 +283,7 @@ This Python code snippet can be used to check if a server stops: ## Communicating with servers -JupyterHub tokens with the `access:servers` scope can be used to communicate with servers themselves. +JupyterHub tokens with the `access:servers` scope can be used to communicate with servers themselves. The tokens can be the same as those you used to launch your service. ```{note} From 7a0b98d9cee8b9825300b106da76455b7862acb6 Mon Sep 17 00:00:00 2001 From: Allan Wasega Date: Sat, 22 Oct 2022 00:04:37 +0300 Subject: [PATCH 120/214] Updated the server-api file with all comments --- docs/source/reference/server-api.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/source/reference/server-api.md b/docs/source/reference/server-api.md index fa8ac461..7b2634f7 100644 --- a/docs/source/reference/server-api.md +++ b/docs/source/reference/server-api.md @@ -260,6 +260,8 @@ Servers can be stopped with a DELETE request: DELETE /hub/api/users/:user/servers/[:servername] ``` +**Required scope: `servers`** + Similar to when starting a server, issuing the DELETE request above might not stop the server immediately. Instead, the DELETE request has two possible response codes: 204 Deleted @@ -270,7 +272,8 @@ It will now be absent from the user `servers` model. : This code means your request was accepted, but is not yet completely processed. The server has `pending: 'stop'` at this point. -There is no progress API for checking when a server actually stops. Thus, the only available alternative is to poll the server and wait for it to disappear from the user `servers` model. +There is no progress API for checking when a server actually stops. +Thus, the only available alternative is to poll the server and wait for it to disappear from the user `servers` model. This Python code snippet can be used to check if a server stops: From 28867760a4d72b9e95d2a16610c5dae6908a9c8c Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 21 Oct 2022 21:05:16 +0000 Subject: [PATCH 121/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/reference/server-api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/reference/server-api.md b/docs/source/reference/server-api.md index 7b2634f7..6bfff7fc 100644 --- a/docs/source/reference/server-api.md +++ b/docs/source/reference/server-api.md @@ -272,7 +272,7 @@ It will now be absent from the user `servers` model. : This code means your request was accepted, but is not yet completely processed. The server has `pending: 'stop'` at this point. -There is no progress API for checking when a server actually stops. +There is no progress API for checking when a server actually stops. Thus, the only available alternative is to poll the server and wait for it to disappear from the user `servers` model. This Python code snippet can be used to check if a server stops: From 74b0ebefe974a66aac67cebea41df0262bf60888 Mon Sep 17 00:00:00 2001 From: Allan Wasega Date: Sat, 22 Oct 2022 00:34:23 +0300 Subject: [PATCH 122/214] Fixed minor issues in community communication channels file --- docs/source/contributing/community.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/docs/source/contributing/community.md b/docs/source/contributing/community.md index 4687bf98..be7e0bf5 100644 --- a/docs/source/contributing/community.md +++ b/docs/source/contributing/community.md @@ -6,7 +6,7 @@ We use different channels of communication for different purposes. Whichever one We use [Discourse](https://discourse.jupyter.org) for online discussions and support questions. Everyone in the Jupyter community is welcome to bring ideas and questions there. -All our past and current discussions on Discourse are archived and searchable. This is why we recommend you first go to Discourse, so that discussions remain useful and accessible to the whole community. +We recommend that you first use our Discourse as all our past and current discussions on it are archived and searchable. Thus, all discussions remain useful and accessible to the whole community. ## Gitter @@ -14,10 +14,12 @@ We use [our Gitter channel](https://gitter.im/jupyterhub/jupyterhub) for online, ## Github Issues -Github issues are used for most long-form project discussions, bug reports and feature requests. +[Github issues](https://docs.github.com/en/issues/tracking-your-work-with-issues/about-issues) are used for most long-form project discussions, bug reports and feature requests. -Issues related to a specific authenticator or spawner should be opened in the appropriate repository for the authenticator or spawner. If you are using a specific JupyterHub distribution (such as [Zero to JupyterHub on Kubernetes](http://github.com/jupyterhub/zero-to-jupyterhub-k8s) or [The Littlest JupyterHub](http://github.com/jupyterhub/the-littlest-jupyterhub/)), you should open issues directly in their repository. +- Issues related to a specific authenticator or spawner should be opened in the appropriate repository for the authenticator or spawner. +- If you are using a specific JupyterHub distribution (such as [Zero to JupyterHub on Kubernetes](http://github.com/jupyterhub/zero-to-jupyterhub-k8s) or [The Littlest JupyterHub](http://github.com/jupyterhub/the-littlest-jupyterhub/)), you should open issues directly in their repository. +- If you cannot find a repository to open your issue in, do not worry! Open the issue in the [main JupyterHub repository](https://github.com/jupyterhub/jupyterhub/) and our community will help you figure it out. -If you cannot find a repository to open your issue in, do not worry! Open the issue in the [main JupyterHub repository](https://github.com/jupyterhub/jupyterhub/) and our community will help you figure it out. - -**NOTE**: Our community is distributed across the world in various timezones, so please be patient if you do not get a response immediately! +```{note} +Our community is distributed across the world in various timezones, so please be patient if you do not get a response immediately! +``` From 39eb1f055fcafb30df5704f538ea10c3006728be Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 21 Oct 2022 21:40:41 +0000 Subject: [PATCH 123/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/contributing/community.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/community.md b/docs/source/contributing/community.md index be7e0bf5..ea19216b 100644 --- a/docs/source/contributing/community.md +++ b/docs/source/contributing/community.md @@ -16,7 +16,7 @@ We use [our Gitter channel](https://gitter.im/jupyterhub/jupyterhub) for online, [Github issues](https://docs.github.com/en/issues/tracking-your-work-with-issues/about-issues) are used for most long-form project discussions, bug reports and feature requests. -- Issues related to a specific authenticator or spawner should be opened in the appropriate repository for the authenticator or spawner. +- Issues related to a specific authenticator or spawner should be opened in the appropriate repository for the authenticator or spawner. - If you are using a specific JupyterHub distribution (such as [Zero to JupyterHub on Kubernetes](http://github.com/jupyterhub/zero-to-jupyterhub-k8s) or [The Littlest JupyterHub](http://github.com/jupyterhub/the-littlest-jupyterhub/)), you should open issues directly in their repository. - If you cannot find a repository to open your issue in, do not worry! Open the issue in the [main JupyterHub repository](https://github.com/jupyterhub/jupyterhub/) and our community will help you figure it out. From 9d93df6baf0a98c47746689269d6ab7b44f86d37 Mon Sep 17 00:00:00 2001 From: Allan Wasega Date: Sat, 22 Oct 2022 00:57:04 +0300 Subject: [PATCH 124/214] Fixed a minor error --- docs/source/contributing/community.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/community.md b/docs/source/contributing/community.md index ea19216b..2c78fb79 100644 --- a/docs/source/contributing/community.md +++ b/docs/source/contributing/community.md @@ -6,7 +6,7 @@ We use different channels of communication for different purposes. Whichever one We use [Discourse](https://discourse.jupyter.org) for online discussions and support questions. Everyone in the Jupyter community is welcome to bring ideas and questions there. -We recommend that you first use our Discourse as all our past and current discussions on it are archived and searchable. Thus, all discussions remain useful and accessible to the whole community. +We recommend that you first use our Discourse as all past and current discussions on it are archived and searchable. Thus, all discussions remain useful and accessible to the whole community. ## Gitter From 9f5f19cb264bcaac334d828367c501abb5e94f8e Mon Sep 17 00:00:00 2001 From: Allan Wasega Date: Sat, 22 Oct 2022 01:06:56 +0300 Subject: [PATCH 125/214] Fixed minor errors --- docs/source/reference/server-api.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/reference/server-api.md b/docs/source/reference/server-api.md index 6bfff7fc..6e49f081 100644 --- a/docs/source/reference/server-api.md +++ b/docs/source/reference/server-api.md @@ -1,6 +1,6 @@ # Starting servers with the JupyterHub API -Sometimes, when working with applications such as [BinderHub](https://binderhub.readthedocs.io), it may be necessary to launch Jupyter-based services on behalf of your users. Doing so can be achieved through JupyterHub's [REST API](../reference/rest.md), which allows one to launch and manage servers on behalf of users through API calls instead of through the JupyterHub UI. In doing so, you can take advantage of other user/launch/lifecycle patterns that are not natively supported by JupyterHub, all without the need to develop the server management features of JupyterHub Spawners and/or Authenticators. +Sometimes, when working with applications such as [BinderHub](https://binderhub.readthedocs.io), it may be necessary to launch Jupyter-based services on behalf of your users. Doing so can be achieved through JupyterHub's [REST API](../reference/rest.md), which allows one to launch and manage servers on behalf of users through API calls instead of the JupyterHub UI. In doing so, you can take advantage of other user/launch/lifecycle patterns that are not natively supported by the JupyterHub UI, all without the need to develop the server management features of JupyterHub Spawners and/or Authenticators. This tutorial goes through the processes involved while working with the JupyterHub API to manage servers for users. In particular, it covers how to: @@ -10,7 +10,7 @@ This tutorial goes through the processes involved while working with the Jupyter 4. [Communicate with servers](communicating) 5. [Stop servers](stopping) -In the end, we also provide sample Python code that can be used to implement these steps. +At the end, we also provide sample Python code that can be used to implement these steps. (checking)= From 411189e54c9d7ee64199bae86c2bb19c7eeaaa9d Mon Sep 17 00:00:00 2001 From: Esther Christopher Date: Sat, 22 Oct 2022 00:32:18 +0100 Subject: [PATCH 126/214] Update upgrading.rst --- docs/source/admin/upgrading.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index 5156c957..0007485f 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -38,7 +38,7 @@ the upgrade process. You should notify them, and pick a time to do the upgrade where they will be least disrupted. patch-2 -If you use a different proxy, or running ``configurable-http-proxy`` +If you use a different proxy, or run ``configurable-http-proxy`` independent of JupyterHub, your users will be able to continue using notebook servers they had already launched, but will not be able to launch new servers or sign in. main @@ -55,7 +55,7 @@ patch-2 should backup the ``jupyterhub.sqlite`` file. #. Your ``jupyterhub_config.py`` file. #. Your users' home directories. This is unlikely to be affected directly by - a JupyterHub upgrade, but we recommend a backup since user data is significant. + a JupyterHub upgrade, but we recommend a backup since user data is critical. main From 02f33073add642c6c5c0eae776f10f8d62fceb41 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Sat, 22 Oct 2022 00:37:08 +0100 Subject: [PATCH 127/214] update oauth.md --- docs/source/reference/oauth.md | 98 +++++++++++++++++----------------- 1 file changed, 49 insertions(+), 49 deletions(-) diff --git a/docs/source/reference/oauth.md b/docs/source/reference/oauth.md index 7611e9be..69bffb2f 100644 --- a/docs/source/reference/oauth.md +++ b/docs/source/reference/oauth.md @@ -1,26 +1,26 @@ # JupyterHub and OAuth -JupyterHub uses OAuth 2 internally as a mechanism for authenticating users. +JupyterHub uses OAuth 2 as an internal mechanism for authenticating users. As such, JupyterHub itself always functions as an OAuth **provider**. -More on what that means [below](oauth-terms). +You can find out more about what that means [below](oauth-terms). -Additionally, JupyterHub is _often_ deployed with [oauthenticator](https://oauthenticator.readthedocs.io), +Additionally, JupyterHub is _often_ deployed with [OAuthenticator](https://oauthenticator.readthedocs.io), where an external identity provider, such as GitHub or KeyCloak, is used to authenticate users. -When this is the case, there are _two_ nested oauth flows: -an _internal_ oauth flow where JupyterHub is the **provider**, -and and _external_ oauth flow, where JupyterHub is a **client**. +When this is the case, there are _two_ nested OAuth flows: +an _internal_ OAuth flow where JupyterHub is the **provider**, +and an _external_ OAuth flow, where JupyterHub is the **client**. This means that when you are using JupyterHub, there is always _at least one_ and often two layers of OAuth involved in a user logging in and accessing their server. -Some relevant points: +The following points are noteworthy: - Single-user servers _never_ need to communicate with or be aware of the upstream provider configured in your Authenticator. - As far as they are concerned, only JupyterHub is an OAuth provider, + As far as the servers are concerned, only JupyterHub is an OAuth provider, and how users authenticate with the Hub itself is irrelevant. -- When talking to a single-user server, +- When interacting with a single-user server, there are ~always two tokens: - a token issued to the server itself to communicate with the Hub API, - and a second per-user token in the browser to represent the completed login process and authorized permissions. + first, a token issued to the server itself to communicate with the Hub API, + and second, a per-user token in the browser to represent the completed login process and authorized permissions. More on this [later](two-tokens). (oauth-terms)= @@ -28,64 +28,64 @@ Some relevant points: ## Key OAuth terms Here are some key definitions to keep in mind when we are talking about OAuth. -You can also read more detail [here](https://www.oauth.com/oauth2-servers/definitions/). +You can also read more in detail [here](https://www.oauth.com/oauth2-servers/definitions/). -- **provider**: The entity responsible for managing identity and authorization, +- **provider**: The entity responsible for managing identity and authorization; always a web server. - JupyterHub is _always_ an oauth provider for JupyterHub's components. - When OAuthenticator is used, an external service, such as GitHub or KeyCloak, is also an oauth provider. -- **client**: An entity that requests OAuth **tokens** on a user's behalf, + JupyterHub is _always_ an OAuth provider for JupyterHub's components. + When OAuthenticator is used, an external service, such as GitHub or KeyCloak, is also an OAuth provider. +- **client**: An entity that requests OAuth **tokens** on a user's behalf; generally a web server of some kind. OAuth **clients** are services that _delegate_ authentication and/or authorization to an OAuth **provider**. JupyterHub _services_ or single-user _servers_ are OAuth **clients** of the JupyterHub **provider**. - When OAuthenticator is used, JupyterHub is itself _also_ an OAuth **client** for the external oauth **provider**, e.g. GitHub. + When OAuthenticator is used, JupyterHub is itself _also_ an OAuth **client** for the external OAuth **provider**, e.g. GitHub. - **browser**: A user's web browser, which makes requests and stores things like cookies. - **token**: The secret value used to represent a user's authorization. This is the final product of the OAuth process. - **code**: A short-lived temporary secret that the **client** exchanges - for a **token** at the conclusion of oauth, - in what's generally called the "oauth callback handler." + for a **token** at the conclusion of OAuth, + in what's generally called the "OAuth callback handler." ## One oauth flow -OAuth **flow** is what we call the sequence of HTTP requests involved in authenticating a user and issuing a token, ultimately used for authorized access to a service or single-user server. +OAuth **flow** is what we call the sequence of HTTP requests involved in authenticating a user and issuing a token, ultimately used for authorizing access to a service or single-user server. -A single oauth flow generally goes like this: +A single OAuth flow typically goes like this: ### OAuth request and redirect -1. A **browser** makes an HTTP request to an oauth **client**. -2. There are no credentials, so the client _redirects_ the browser to an "authorize" page on the oauth **provider** with some extra information: - - the oauth **client id** of the client itself. - - the **redirect uri** to be redirected back to after completion. +1. A **browser** makes an HTTP request to an OAuth **client**. +2. There are no credentials, so the client _redirects_ the browser to an "authorize" page on the OAuth **provider** with some extra information: + - the OAuth **client ID** of the client itself. + - the **redirect URI** to be redirected back to after completion. - the **scopes** requested, which the user should be presented with to confirm. This is the "X would like to be able to Y on your behalf. Allow this?" page you see on all the "Login with ..." pages around the Internet. 3. During this authorize step, the browser must be _authenticated_ with the provider. This is often already stored in a cookie, but if not the provider webapp must begin its _own_ authentication process before serving the authorization page. - This _may_ even begin another oauth flow! + This _may_ even begin another OAuth flow! 4. After the user tells the provider that they want to proceed with the authorization, - the provider records this authorization in a short-lived record called an **oauth code**. -5. Finally, the oauth provider redirects the browser _back_ to the oauth client's "redirect uri" - (or "oauth callback uri"), - with the oauth code in a url parameter. + the provider records this authorization in a short-lived record called an **OAuth code**. +5. Finally, the oauth provider redirects the browser _back_ to the oauth client's "redirect URI" + (or "OAuth callback URI"), + with the OAuth code in a URL parameter. -That's the end of the requests made between the **browser** and the **provider**. +That marks the end of the requests made between the **browser** and the **provider**. ### State after redirect At this point: - The browser is authenticated with the _provider_. -- The user's authorized permissions are recorded in an _oauth code_. -- The _provider_ knows that the given oauth client's requested permissions have been granted, but the client doesn't know this yet. -- All requests so far have been made directly by the browser. - No requests have originated at the client or provider. +- The user's authorized permissions are recorded in an _OAuth code_. +- The _provider_ knows that the permissions requested by the OAuth client have been granted, but the client doesn't know this yet. +- All the requests so far have been made directly by the browser. + No requests have originated from the client or provider. ### OAuth Client Handles Callback Request -Now we get to finish the OAuth process. +At this stage, we get to finish the OAuth process. Let's dig into what the OAuth client does when it handles the OAuth callback request. @@ -95,12 +95,12 @@ the OAuth callback request. makes a second API request to the _provider_ to retrieve information about the owner of the token (the user). This is the step where behavior diverges for different OAuth providers. - Up to this point, all oauth providers are the same, following the oauth specification. - However, oauth does not define a standard for exchanging tokens for information about their owner or permissions ([OpenID Connect](https://openid.net/connect/) does that), + Up to this point, all OAuth providers are the same, following the OAuth specification. + However, OAuth does not define a standard for issuing tokens in exchange for information about their owner or permissions ([OpenID Connect](https://openid.net/connect/) does that), so this step may be different for each OAuth provider. -- Finally, the oauth client stores its own record that the user is authorized in a cookie. +- Finally, the OAuth client stores its own record that the user is authorized in a cookie. This could be the token itself, or any other appropriate representation of successful authentication. -- Last of all, now that credentials have been established, +- Now that credentials have been established, the browser can be redirected to the _original_ URL where it started, to try the request again. If the client wasn't able to keep track of the original URL all this time @@ -114,7 +114,7 @@ So that's _one_ OAuth process. ## Full sequence of OAuth in JupyterHub Let's go through the above OAuth process in JupyterHub, -with specific examples of each HTTP request and what information is contained. +with specific examples of each HTTP request and what information it contains. For bonus points, we are using the double-OAuth example of JupyterHub configured with GitHubOAuthenticator. To disambiguate, we will call the OAuth process where JupyterHub is the **provider** "internal OAuth," @@ -184,7 +184,7 @@ The first: - JupyterHub->GitHub - `POST https://github.com/login/oauth/access_token` -- request made with oauth **code** from url parameter +- request made with OAuth **code** from URL parameter - response includes an access **token** The second: @@ -271,15 +271,15 @@ To handle this, OAuth tokens and the various places they are stored can _expire_ which should have the same effect as no credentials, and trigger the authorization process again. -In JupyterHub's internal oauth, we have these layers of information that can go stale: +In JupyterHub's internal OAuth, we have these layers of information that can go stale: -- The oauth client has a **cache** of Hub responses for tokens, +- The OAuth client has a **cache** of Hub responses for tokens, so it doesn't need to make API requests to the Hub for every request it receives. This cache has an expiry of five minutes by default, and is governed by the configuration `HubAuth.cache_max_age` in the single-user server. -- The internal oauth token is stored in a cookie, which has its own expiry (default: 14 days), +- The internal OAuth token is stored in a cookie, which has its own expiry (default: 14 days), governed by `JupyterHub.cookie_max_age_days`. -- The internal oauth token can also itself expire, +- The internal OAuth token itself can also expire, which is by default the same as the cookie expiry, since it makes sense for the token itself and the place it is stored to expire at the same time. This is governed by `JupyterHub.cookie_max_age_days` first, @@ -317,9 +317,9 @@ triggering the external login process anew before letting a user proceed. - If the token has expired, but is still in the cookie: when the token response cache expires, the next time the server asks the hub about the token, - no user will be identified and the internal oauth process begins again. + no user will be identified and the internal OAuth process begins again. - If the token _cookie_ expires, the next browser request will be made with no credentials, - and the internal oauth process will begin again. + and the internal OAuth process will begin again. This will usually have the form of a transparent redirect browsers won't notice. However, if this occurs on an API request in a long-lived page visit such as a JupyterLab session, the API request may fail and require @@ -352,7 +352,7 @@ Logging out of JupyterHub means clearing and revoking many of these credentials: ### A tale of two tokens **TODO**: discuss API token issued to server at startup ($JUPYTERHUB_API_TOKEN) -and oauth-issued token in the cookie, +and OAuth-issued token in the cookie, and some details of how JupyterLab currently deals with that. They are different, and JupyterLab should be making requests using the token from the cookie, not the token from the server, From 70bfdd6d003907e5b145450bf5ae663d9ef1dcba Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Sat, 22 Oct 2022 00:51:38 +0100 Subject: [PATCH 128/214] update oauth.md --- docs/source/reference/oauth.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/reference/oauth.md b/docs/source/reference/oauth.md index 69bffb2f..ab0aeb86 100644 --- a/docs/source/reference/oauth.md +++ b/docs/source/reference/oauth.md @@ -1,6 +1,6 @@ # JupyterHub and OAuth -JupyterHub uses OAuth 2 as an internal mechanism for authenticating users. +JupyterHub uses [OAuth 2](https://oauth.net/2/) as an internal mechanism for authenticating users. As such, JupyterHub itself always functions as an OAuth **provider**. You can find out more about what that means [below](oauth-terms). From fc5f55caf9d8b0111c6660462c2ba2ccf0761281 Mon Sep 17 00:00:00 2001 From: Ngobiri Falyne Date: Sat, 22 Oct 2022 06:53:16 +0100 Subject: [PATCH 129/214] Update troubleshooting.md I replaced the comma, adjusted jupyterhub to JuputerHub and the others as adiced --- docs/source/troubleshooting.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/troubleshooting.md b/docs/source/troubleshooting.md index 66a7512b..532c88d3 100644 --- a/docs/source/troubleshooting.md +++ b/docs/source/troubleshooting.md @@ -161,7 +161,7 @@ your server again. ##### Proxy settings (403 GET) -When your whole JupyterHub sits behind an organization proxy (_not_ a reverse proxy like NGINX as part of your setup and _not_ the configurable-http-proxy) the environment variables `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy` and `https_proxy` might be set. This confuses the jupyterhub single-user servers: When connecting to the Hub for authorization they connect via the proxy instead of directly connecting to the Hub on localhost. The proxy might deny the request (403 GET). This results in the single-user server thinking it has a wrong auth token. To circumvent this you should add `,,localhost,127.0.0.1` to the environment variables `NO_PROXY` and `no_proxy`. +When your whole JupyterHub sits behind an organization proxy (_not_ a reverse proxy like NGINX as part of your setup and _not_ the configurable-http-proxy) the environment variables `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy`, and `https_proxy` might be set. This confuses the JupyterHub single-user servers: When connecting to the Hub for authorization they connect via the proxy instead of directly connecting to the Hub on localhost. The proxy might deny the request (403 GET). This results in the single-user server thinking it has the wrong auth token. To circumvent this you should add `,,localhost,127.0.0.1` to the environment variables `NO_PROXY` and `no_proxy`. ### Launching Jupyter Notebooks to run as an externally managed JupyterHub service with the `jupyterhub-singleuser` command returns a `JUPYTERHUB_API_TOKEN` error From eb8f338186c3092ea2bb7a6d22834a86bb87e773 Mon Sep 17 00:00:00 2001 From: Uzochukwu Precious Date: Sat, 22 Oct 2022 10:02:39 +0100 Subject: [PATCH 130/214] Updated JupyterHub's REST API doc --- docs/source/reference/rest.md | 80 +++++++++++++++++++++-------------- 1 file changed, 49 insertions(+), 31 deletions(-) diff --git a/docs/source/reference/rest.md b/docs/source/reference/rest.md index 12f8f44e..13bec97b 100644 --- a/docs/source/reference/rest.md +++ b/docs/source/reference/rest.md @@ -4,22 +4,25 @@ This section will give you information on: -- what you can do with the API -- create an API token -- add API tokens to the config files -- make an API request programmatically using the requests library -- learn more about JupyterHub's API +- [What you can do with the API](#what-you-can-do-with-the-api) +- [How to create an API token](#create-an-api-token) +- [Assigning permissions to a token](#assigning-permissions-to-a-token) +- [Updating to admin services](#updating-to-admin-services) +- [Making an API request programmatically using the requests library](#make-an-api-request) +- [Paginating API requests](#paginating-api-requests) +- [Enabling users to spawn multiple named-servers via the API](#enabling-users-to-spawn-multiple-named-servers-via-the-api) +- [Learn more about JupyterHub's API](#learn-more-about-the-api) ## What you can do with the API Using the [JupyterHub REST API][], you can perform actions on the Hub, such as: -- checking which users are active -- adding or removing users -- stopping or starting single user notebook servers -- authenticating services -- communicating with an individual Jupyter server's REST API +- Checking which users are active +- Adding or removing users +- Stopping or starting single user notebook servers +- Authenticating services +- Communicating with an individual Jupyter server's REST API A [REST](https://en.wikipedia.org/wiki/Representational_state_transfer) API provides a standard way for users to get and send information to the @@ -27,21 +30,25 @@ Hub. ## Create an API token -To send requests using JupyterHub API, you must pass an API token with +To send requests using the JupyterHub API, you must pass an API token with the request. -The preferred way of generating an API token is: +The preferred way of generating an API token is by running: ```bash openssl rand -hex 32 ``` -This `openssl` command generates a potential token that can then be +The `openssl` command generates a potential token that can then be added to JupyterHub using `.api_tokens` configuration setting in `jupyterhub_config.py`. -Alternatively, use the `jupyterhub token` command to generate a token -for a specific hub user by passing the 'username': +```{note} +The api_tokens configuration has been softly deprecated since the introduction of services. +``` + +Alternatively, you can use the `jupyterhub token` command to generate a token +for a specific hub user by passing the **_username_**: ```bash jupyterhub token @@ -53,9 +60,19 @@ it for the given user with the Hub's database. In [version 0.8.0](../changelog.md), a token request page for generating an API token is available from the JupyterHub user interface: -![Request API token page](../images/token-request.png) +```{figure} ../images/token-request.png +--- +name: token-request +--- +JupyterHub's API token page +``` -![API token success page](../images/token-request-success.png) +```{figure} ../images/token-request-success.png +--- +name: token-request-success +--- +JupyterHub's API token success page +``` ## Assigning permissions to a token @@ -67,25 +84,26 @@ Prior to JupyterHub 2.0, there were two levels of permissions: where a token would always have full permissions to do whatever its owner could do. In JupyterHub 2.0, -specific permissions are now defined as 'scopes', +specific permissions are now defined as '**scopes**', and can be assigned both at the user/service level, and at the individual token level. This allows e.g. a user with full admin permissions to request a token with limited permissions. -### Updating to admin services +## Updating to admin services +```{note} The `api_tokens` configuration has been softly deprecated since the introduction of services. We have no plans to remove it, but deployments are encouraged to use service configuration instead. +``` If you have been using `api_tokens` to create an admin user -and a token for that user to perform some automations, -the services mechanism may be a better fit. -If you have the following configuration: +and the token for that user to perform some automations, then +the services' mechanism may be a better fit if you have the following configuration: ```python -c.JupyterHub.admin_users = {"service-admin",} +c.JupyterHub.admin_users = {"service-admin"} c.JupyterHub.api_tokens = { "secret-token": "service-admin", } @@ -103,9 +121,8 @@ c.JupyterHub.services = [ }, ] -# roles are new in JupyterHub 2.0 -# prior to 2.0, only 'admin': True or False -# was available +# roles were introduced in JupyterHub 2.0 +# prior to 2.0, only "admin": True or False was available c.JupyterHub.load_roles = [ { @@ -176,7 +193,7 @@ r.json() ``` The same API token can also authorize access to the [Jupyter Notebook REST API][] -provided by notebook servers managed by JupyterHub if it has the necessary `access:users:servers` scope: +provided by notebook servers managed by JupyterHub if it has the necessary `access:users:servers` scope. (api-pagination)= @@ -245,7 +262,7 @@ with your request, in which case a response will look like: where the list results (same as pre-2.0) will be in `items`, and pagination info will be in `_pagination`. -The `next` field will include the offset, limit, and URL for requesting the next page. +The `next` field will include the `offset`, `limit`, and `url` for requesting the next page. `next` will be `null` if there is no next page. Pagination is governed by two configuration options: @@ -259,7 +276,7 @@ Pagination is enabled on the `GET /users`, `GET /groups`, and `GET /proxy` REST ## Enabling users to spawn multiple named-servers via the API -With JupyterHub version 0.8, support for multiple servers per user has landed. +Support for multiple servers per user was introduced in JupyterHub [version 0.8.](../changelog.md) Prior to that, each user could only launch a single default server via the API like this: @@ -275,7 +292,7 @@ First you must enable named-servers by including the following setting in the `j `c.JupyterHub.allow_named_servers = True` -If using the [zero-to-jupyterhub-k8s](https://github.com/jupyterhub/zero-to-jupyterhub-k8s) set-up to run JupyterHub, +If you are using the [zero-to-jupyterhub-k8s](https://github.com/jupyterhub/zero-to-jupyterhub-k8s) set-up to run JupyterHub, then instead of editing the `jupyterhub_config.py` file directly, you could pass the following as part of the `config.yaml` file, as per the [tutorial](https://zero-to-jupyterhub.readthedocs.io/en/latest/): @@ -303,8 +320,9 @@ or kubernetes pods. ## Learn more about the API -You can see the full [JupyterHub REST API][] for details. +You can see the full [JupyterHub REST API][] for more details. [openapi initiative]: https://www.openapis.org/ [jupyterhub rest api]: ./rest-api +[scopes]: https://jupyterhub.readthedocs.io/en/stable/rbac/scopes.html [jupyter notebook rest api]: https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/jupyter/notebook/HEAD/notebook/services/api/api.yaml From 29ba669c73ef715a9c5e5150b875b3493a69cdbf Mon Sep 17 00:00:00 2001 From: Deborah Udoh Date: Sat, 22 Oct 2022 10:32:16 +0100 Subject: [PATCH 131/214] Update security-basics.rst --- .../getting-started/security-basics.rst | 32 +++++++++---------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/source/getting-started/security-basics.rst b/docs/source/getting-started/security-basics.rst index b87df593..ab48cdfe 100644 --- a/docs/source/getting-started/security-basics.rst +++ b/docs/source/getting-started/security-basics.rst @@ -1,4 +1,4 @@ -Security Settings +Security settings ================= .. important:: @@ -20,13 +20,13 @@ is still a good idea to revoke existing tokens. .. _ssl-encryption: -Enabling SSL Encryption +Enabling SSL encryption ----------------------- Since JupyterHub includes authentication and allows arbitrary code execution, you should not run it without SSL (HTTPS). -Using an SSL Certificate +Using an SSL certificate ~~~~~~~~~~~~~~~~~~~~~~~~ This will require you to obtain an official, trusted SSL certificate or create a @@ -44,7 +44,7 @@ Some cert files also contain the key, in which case only the cert is needed. It is important that these files be put in a secure location on your server, where they are not readable by regular users. -If you are using a **chain certificate**, see also, chained certificate for SSL +If you are using a **chain certificate**, see also chained certificate for SSL in the JupyterHub `Troubleshooting FAQ <../troubleshooting.html>`_. Using letsencrypt @@ -82,7 +82,7 @@ To achieve this, simply omit the configuration settings .. _authentication-token: -Proxy Authentication Token +Proxy authentication token -------------------------- The Hub authenticates its requests to the Proxy using a secret token that @@ -94,7 +94,7 @@ The value of this token should be a random string (for example, generated by ``openssl rand -hex 32``). You can store it in the configuration file or an environment variable. -Generating and Storing Token in the Configuration File +Generating and storing token in the configuration file ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can set the value in the configuration file, ``jupyterhub_config.py``: @@ -103,7 +103,7 @@ You can set the value in the configuration file, ``jupyterhub_config.py``: c.ConfigurableHTTPProxy.api_token = 'abc123...' # any random string -Generating and Storing as an Environment Variable +Generating and storing as an environment variable ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can pass this value of the proxy authentication token to the Hub and Proxy @@ -115,28 +115,28 @@ using the ``CONFIGPROXY_AUTH_TOKEN`` environment variable: This environment variable needs to be visible to the Hub and Proxy. -Default if Token is Not Set +Default if token is not set ~~~~~~~~~~~~~~~~~~~~~~~~~~~ If you don't set the Proxy authentication token, the Hub will generate a random -key itself. This means that anytime you restart the Hub, you **must also +key itself. This means that any time you restart the Hub, you **must also restart the Proxy**. If the proxy is a subprocess of the Hub, this should happen automatically (this is the default configuration). .. _cookie-secret: -Cookie Secret +Cookie secret ------------- The cookie secret is an encryption key, used to encrypt the browser cookies, which are used for authentication. Three common methods are described for generating and configuring the cookie secret. -Generating and Storing as a Cookie Secret File +Generating and storing as a cookie secret file ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The cookie secret should be 32 random bytes, encoded as hex, and is typically -stored in a ``jupyterhub_cookie_secret`` file. Below, is an example of a command to generate the +stored in a ``jupyterhub_cookie_secret`` file. Below, is an example command to generate the ``jupyterhub_cookie_secret`` file: .. code-block:: bash @@ -155,10 +155,10 @@ The location of the ``jupyterhub_cookie_secret`` file can be specified in the If the cookie secret file doesn't exist when the Hub starts, a new cookie secret is generated and stored in the file. The file must not be readable by -``group`` or ``other``, otherwise, the server won't start. The recommended permissions +``group`` or ``other`` otherwise, the server won't start. The recommended permissions for the cookie secret file are ``600`` (owner-only rw). -Generating and Storing as an Environment Variable +Generating and storing as an environment variable ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If you would like to avoid the need for files, the value can be loaded in the @@ -173,7 +173,7 @@ For security reasons, this environment variable should only be visible to the Hub. If you set it dynamically as above, all users will be logged out each time the Hub starts. -Generating and Storing as a Binary String +Generating and storing as a binary string ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can also set the cookie secret, as a binary string, @@ -185,7 +185,7 @@ in the configuration file (``jupyterhub_config.py``) itself: .. _cookies: -Cookies Used by JupyterHub Authentication +Cookies used by JupyterHub authentication ----------------------------------------- The following cookies are used by the Hub for handling user authentication. From 23fc2f42d04c3709968fbef8870c09f95b41dcaa Mon Sep 17 00:00:00 2001 From: Deborah Udoh Date: Sat, 22 Oct 2022 10:41:10 +0100 Subject: [PATCH 132/214] Update security-basics.rst Co-authored-by: Min RK --- docs/source/getting-started/security-basics.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/getting-started/security-basics.rst b/docs/source/getting-started/security-basics.rst index ab48cdfe..210c7d27 100644 --- a/docs/source/getting-started/security-basics.rst +++ b/docs/source/getting-started/security-basics.rst @@ -234,7 +234,7 @@ shared by the Hub and single-user servers. Its sole purpose is to coordinate logout of the multiple OAuth cookies. -This cookie is set to ``/`` so all endpoints can receive or clear it, etc. +This cookie is set to ``/`` so all endpoints can receive it, clear it, etc. jupyterhub-user--oauth-state ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 750d36a8f7a7bec4173b3708020855235bc70662 Mon Sep 17 00:00:00 2001 From: Deborah Udoh Date: Sat, 22 Oct 2022 11:19:27 +0100 Subject: [PATCH 133/214] Update security-basics.rst --- docs/source/getting-started/security-basics.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/getting-started/security-basics.rst b/docs/source/getting-started/security-basics.rst index 210c7d27..c3649ee3 100644 --- a/docs/source/getting-started/security-basics.rst +++ b/docs/source/getting-started/security-basics.rst @@ -155,7 +155,7 @@ The location of the ``jupyterhub_cookie_secret`` file can be specified in the If the cookie secret file doesn't exist when the Hub starts, a new cookie secret is generated and stored in the file. The file must not be readable by -``group`` or ``other`` otherwise, the server won't start. The recommended permissions +``group`` or ``other``, otherwise the server won't start. The recommended permissions for the cookie secret file are ``600`` (owner-only rw). Generating and storing as an environment variable From a17b4c5801dd05c0344e25859f745ac7fe8e1e6a Mon Sep 17 00:00:00 2001 From: Emmanuella Orioma Date: Sat, 22 Oct 2022 22:23:26 +0300 Subject: [PATCH 134/214] Updated use-cases.md Added the RBAC jupyter documentation link --- docs/source/rbac/use-cases.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/rbac/use-cases.md b/docs/source/rbac/use-cases.md index 4169f055..a6fb34ff 100644 --- a/docs/source/rbac/use-cases.md +++ b/docs/source/rbac/use-cases.md @@ -9,7 +9,7 @@ To determine which scopes a role should have, one can follow these steps: 5. Customize the scopes with filters if needed 6. Define the role with required scopes and assign to users/services/groups/tokens -Below, different use cases are presented on how to use the [RBAC framework](https://en.wikipedia.org/wiki/Role-based_access_control). +Below, different use cases are presented on how to use the [RBAC framework](https://jupyterhub.readthedocs.io/en/stable/rbac/index.html). ## Service to cull idle servers From 61369ea5da78bbb1a45ffef57708a632ab56c5ea Mon Sep 17 00:00:00 2001 From: Lili Yao <103026397+liliyao2022@users.noreply.github.com> Date: Sun, 23 Oct 2022 13:53:02 +1100 Subject: [PATCH 135/214] Update README.md Modified some spelling and grammar errors. --- jsx/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/jsx/README.md b/jsx/README.md index 89249f93..99d635af 100644 --- a/jsx/README.md +++ b/jsx/README.md @@ -23,7 +23,7 @@ This app is written in JSX, and then transpiled into an ES5 bundle with Babel an #### Centralized state and data management with Redux: -The app use Redux throughout the components via the `useSelector` and `useDispatch` hooks to store and update user and group data from the API. With Redux, this data is available to any connected component. This means that if one component recieves new data, they all do. +The app uses Redux throughout the components via the `useSelector` and `useDispatch` hooks to store and update user and group data from the API. With Redux, this data is available to any connected component. This means that if one component receives new data, they all do. #### API functions @@ -31,7 +31,7 @@ All API functions used by the front end are packaged as a library of props withi #### Pagination -Indicies of paginated user and group data is stored in a `page` variable in the query string, as well as the `user_page` / `group_page` state variables in Redux. This allows the app to maintain two sources of truth, as well as protect the admin user's place in the collection on page reload. Limit is constant at this point and is held in the Redux state. +Indicies of paginated user and group data are stored in a `page` variable in the query string, as well as the `user_page` / `group_page` state variables in Redux. This allows the app to maintain two sources of truth, as well as protect the admin user's place in the collection on page reload. The limit is constant at this point and is held in the Redux state. On updates to the paginated data, the app can respond in one of two ways. If a user/group record is either added or deleted, the pagination will reset and data will be pulled back with no offset. Alternatively, if a record is modified, the offset will remain and the change will be shown. @@ -55,7 +55,7 @@ startServer().then(() => { .then((data) => dispatchPageChange(data, page)); }); -// Alternatively, a new user was added, user data is being refreshed from offset 0. +// Alternatively, a new user was added, and user data is being refreshed from offset 0. addUser().then(() => { updateUsers(0, limit) // After data is fetched, the Redux store is updated with the data and asserts page 0. From ad25ef8f87ac6e540726795b53e7dd5b0227ad74 Mon Sep 17 00:00:00 2001 From: Lili Yao <103026397+liliyao2022@users.noreply.github.com> Date: Sun, 23 Oct 2022 15:42:57 +1100 Subject: [PATCH 136/214] Update roadmap.md It seems that we don't need an "a" here, or should we change it to "an" instead? --- docs/source/contributing/roadmap.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/roadmap.md b/docs/source/contributing/roadmap.md index 0bbde407..5f2bc49c 100644 --- a/docs/source/contributing/roadmap.md +++ b/docs/source/contributing/roadmap.md @@ -4,7 +4,7 @@ This roadmap collects "next steps" for JupyterHub. It is about creating a shared understanding of the project's vision and direction amongst the community of users, contributors, and maintainers. The goal is to communicate priorities and upcoming release plans. -It is not a aimed at limiting contributions to what is listed here. +It is not aimed at limiting contributions to what is listed here. ## Using the roadmap From 6b62fe794ec52d1106c2304683163a95b73ef201 Mon Sep 17 00:00:00 2001 From: Joel-Ando Date: Sun, 23 Oct 2022 16:13:20 +0100 Subject: [PATCH 137/214] Update tech-implementation.md did the necessary changes --- docs/source/rbac/tech-implementation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/rbac/tech-implementation.md b/docs/source/rbac/tech-implementation.md index ea6231f7..8779c1bc 100644 --- a/docs/source/rbac/tech-implementation.md +++ b/docs/source/rbac/tech-implementation.md @@ -43,7 +43,7 @@ Prior to 3.0, tokens stored _roles_, which meant their scopes were resolved on each request. ::: -API tokens grant access to JupyterHub's APIs. The [RBAC framework](https://auth0.com/docs/manage-users/access-control/rbac) allows for requesting tokens with specific permissions. +API tokens grant access to JupyterHub's APIs. The [RBAC framework](https://jupyterhub.readthedocs.io/en/stable/rbac/index.html) allows for requesting tokens with specific permissions. RBAC is involved in several stages of the OAuth token flow. From 18b049e3c942d380b0e2c421849c4b7e0d6efb00 Mon Sep 17 00:00:00 2001 From: Joel-Ando Date: Sun, 23 Oct 2022 16:23:35 +0100 Subject: [PATCH 138/214] Update tech-implementation.md --- docs/source/rbac/tech-implementation.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/source/rbac/tech-implementation.md b/docs/source/rbac/tech-implementation.md index 8779c1bc..496d4b7f 100644 --- a/docs/source/rbac/tech-implementation.md +++ b/docs/source/rbac/tech-implementation.md @@ -79,11 +79,11 @@ With the RBAC framework, each authenticated JupyterHub API request is guarded by When an API request is made, the requesting API token's scopes are again intersected with its owner's (yellow box in {ref}`Figure 2 `) to ensure that the token does not grant more permissions than its owner has at the request time (e.g., due to changing/losing roles). If the owner's roles do not include some scopes of the token, only the _intersection_ of the token's and owner's scopes will be used. For example, using a token with scope `users` whose owner's role scope is `read:users:name` will result in only the `read:users:name` scope being passed on. In the case of no _intersection_, an empty set of scopes will be used. -The parsed scopes are compared to the scopes required to access the API as follows: +The passed scopes are compared to the scopes required to access the API as follows: -- if the API scopes are present within the set of parsed scopes, the access is granted and the API returns its "full" response +- if the API scopes are present within the set of passed scopes, the access is granted and the API returns its "full" response -- if that is not the case, another check is utilized to determine if subscopes of the required API scopes can be found in the parsed scope set: +- if that is not the case, another check is utilized to determine if subscopes of the required API scopes can be found in the passed scope set: - if found, the RBAC framework employs the {ref}`filtering ` procedures to refine the API response to access only resource attributes corresponding to the passed scopes. For example, providing a scope `read:users:activity!group=class-C` for the `GET /users` API will return a list of user models from group `class-C` containing only the `last_activity` attribute for each user model From 35b06481e2235d433748c8a4343a98a5f2aa0461 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Sun, 23 Oct 2022 19:03:32 +0100 Subject: [PATCH 139/214] Update config-sudo.md --- docs/source/reference/config-sudo.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/reference/config-sudo.md b/docs/source/reference/config-sudo.md index 33e1ad00..35a59bce 100644 --- a/docs/source/reference/config-sudo.md +++ b/docs/source/reference/config-sudo.md @@ -6,8 +6,8 @@ Only do this if you are very sure you must. ## Overview -There are many Authenticators and Spawners available for JupyterHub. Some, such -as DockerSpawner or OAuthenticator, do not need any elevated permissions. This +There are many [Authenticators](./authenticators-users-basics) and [Spawners](./spawners-basics) available for JupyterHub. Some, such +as [DockerSpawner](https://github.com/jupyterhub/dockerspawner) or [OAuthenticator](https://github.com/jupyterhub/oauthenticator), do not need any elevated permissions. This document describes how to get the full default behavior of JupyterHub while running notebook servers as real system users on a shared system without running the Hub itself as root. @@ -46,7 +46,7 @@ We want to confine these permissions to only what we really need. ## Edit `/etc/sudoers` -To do this we add to `/etc/sudoers` (use `visudo` for safe editing of sudoers): +To do this we write to `/etc/sudoers` (use `visudo` for safe editing of sudoers): - specify the list of users `JUPYTER_USERS` for whom `rhea` can spawn servers - set the command `JUPYTER_CMD` that `rhea` can execute on behalf of users @@ -150,7 +150,7 @@ We want our new user to be able to read the shadow passwords, so add it to the s $ sudo usermod -a -G shadow rhea ``` -If you want jupyterhub to serve pages on a restricted port (such as port 80 for http), +If you want jupyterhub to serve pages on a restricted port (such as port 80 for HTTP), then you will need to give `node` permission to do so: ```bash From 7b636b6f9c96f4e2174a34952c474db92cd98af5 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Sun, 23 Oct 2022 19:26:24 +0100 Subject: [PATCH 140/214] Update config-sudo.md --- docs/source/reference/config-sudo.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/source/reference/config-sudo.md b/docs/source/reference/config-sudo.md index 35a59bce..55ad2fb3 100644 --- a/docs/source/reference/config-sudo.md +++ b/docs/source/reference/config-sudo.md @@ -9,7 +9,7 @@ Only do this if you are very sure you must. There are many [Authenticators](./authenticators-users-basics) and [Spawners](./spawners-basics) available for JupyterHub. Some, such as [DockerSpawner](https://github.com/jupyterhub/dockerspawner) or [OAuthenticator](https://github.com/jupyterhub/oauthenticator), do not need any elevated permissions. This document describes how to get the full default behavior of JupyterHub while -running notebook servers as real system users on a shared system without +running notebook servers as real system users on a shared system, without running the Hub itself as root. Since JupyterHub needs to spawn processes as other users, the simplest way @@ -90,7 +90,7 @@ $ adduser -G jupyterhub newuser Test that the new user doesn't need to enter a password to run the sudospawner command. -This should prompt for your password to switch to rhea, but _not_ prompt for +This should prompt for your password to switch to `rhea`, but _not_ prompt for any password for the second switch. It should show some help output about logging options: @@ -119,7 +119,7 @@ the shadow password database. ### Shadow group (Linux) -**Note:** On Fedora based distributions there is no clear way to configure +**Note:** On [Fedora based distributions](https://fedoraproject.org/wiki/List_of_Fedora_remixes) there is no clear way to configure the PAM database to allow sufficient access for authenticating with the target user's password from JupyterHub. As a workaround we recommend use an [alternative authentication method](https://github.com/jupyterhub/jupyterhub/wiki/Authenticators). @@ -226,7 +226,7 @@ And try logging in. ## Troubleshooting: SELinux If you still get a generic `Permission denied` `PermissionError`, it's possible SELinux is blocking you. -Here's how you can make a module to allow this. +Here's how you can make a module to resolve this. First, put this in a file named `sudo_exec_selinux.te`: ```bash @@ -253,6 +253,6 @@ $ semodule -i sudo_exec_selinux.pp ## Troubleshooting: PAM session errors If the PAM authentication doesn't work and you see errors for -`login:session-auth`, or similar, considering updating to a more recent version -of jupyterhub and disabling the opening of PAM sessions with +`login:session-auth`, or similar, consider updating to a more recent [version +of jupyterhub](https://readthedocs.org/projects/jupyterhub/) and disabling the opening of PAM sessions with `c.PAMAuthenticator.open_sessions=False`. From 36b15d7ce01afae4039933248257ad74eb8803e7 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Sun, 23 Oct 2022 20:11:36 +0100 Subject: [PATCH 141/214] Update separate-proxy.md --- docs/source/reference/separate-proxy.md | 27 ++++++++++++++----------- 1 file changed, 15 insertions(+), 12 deletions(-) diff --git a/docs/source/reference/separate-proxy.md b/docs/source/reference/separate-proxy.md index db370c9c..fb054bb1 100644 --- a/docs/source/reference/separate-proxy.md +++ b/docs/source/reference/separate-proxy.md @@ -2,7 +2,7 @@ ## Background -The thing which users directly connect to is the proxy, by default +The thing which users directly connect to is the proxy, which by default is `configurable-http-proxy`. The proxy either redirects users to the hub (for login and managing servers), or to their own single-user servers. Thus, as long as the proxy stays running, access to existing @@ -10,16 +10,15 @@ servers continues, even if the hub itself restarts or goes down. When you first configure the hub, you may not even realize this because the proxy is automatically managed by the hub. This is great -for getting started and even most use, but everytime you restart the -hub, all user connections also get restarted. But it's also simple to +for getting started and even most use-cases, although, everytime you restart the +hub, all user connections are also restarted. However, it is also simple to run the proxy as a service separate from the hub, so that you are free to reconfigure the hub while only interrupting users who are currently actively starting the hub. The default JupyterHub proxy is -[configurable-http-proxy](https://github.com/jupyterhub/configurable-http-proxy), -and that page has some docs. If you are using a different proxy, such -as Traefik, these instructions are probably not relevant to you. +[configurable-http-proxy](https://github.com/jupyterhub/configurable-http-proxy). If you are using a different proxy, such +as [Traefik](https://github.com/traefik/traefik), these instructions are probably not relevant to you. ## Configuration options @@ -40,9 +39,13 @@ set to the URL which the hub uses to connect _to the proxy's API_. ## Proxy configuration You need to configure a service to start the proxy. An example -command line for this is `configurable-http-proxy --ip=127.0.0.1 --port=8000 --api-ip=127.0.0.1 --api-port=8001 --default-target=http://localhost:8081 --error-target=http://localhost:8081/hub/error`. (Details for how to -do this is out of scope for this tutorial - for example it might be a -systemd service on within another docker cotainer). The proxy has no +command line argument for this is: + +```bash +$ configurable-http-proxy --ip=127.0.0.1 --port=8000 --api-ip=127.0.0.1 --api-port=8001 --default-target=http://localhost:8081 --error-target=http://localhost:8081/hub/error` +``` +(Details on how to do this is out of the scope of this tutorial. For example, it might be a +systemd service configured within another docker container). The proxy has no configuration files, all configuration is via the command line and environment variables. @@ -57,9 +60,9 @@ match the token given to `c.ConfigurableHTTPProxy.auth_token`. You should check the [configurable-http-proxy options](https://github.com/jupyterhub/configurable-http-proxy) to see -what other options are needed, for example SSL options. Note that -these are configured in the hub if the hub is starting the proxy - you -need to move the options to here. +what other options are needed, for example, SSL options. Note that +these options are configured in the hub if the hub is starting the proxy, so you +need to configure the options there. ## Docker image From 5ce930324cc10ab9abe5991c8bc27b3c414475b5 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Sun, 23 Oct 2022 20:16:46 +0100 Subject: [PATCH 142/214] Update separate-proxy.md --- docs/source/reference/separate-proxy.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/source/reference/separate-proxy.md b/docs/source/reference/separate-proxy.md index fb054bb1..057e2f5a 100644 --- a/docs/source/reference/separate-proxy.md +++ b/docs/source/reference/separate-proxy.md @@ -40,9 +40,8 @@ set to the URL which the hub uses to connect _to the proxy's API_. You need to configure a service to start the proxy. An example command line argument for this is: - ```bash -$ configurable-http-proxy --ip=127.0.0.1 --port=8000 --api-ip=127.0.0.1 --api-port=8001 --default-target=http://localhost:8081 --error-target=http://localhost:8081/hub/error` +$ configurable-http-proxy --ip=127.0.0.1 --port=8000 --api-ip=127.0.0.1 --api-port=8001 --default-target=http://localhost:8081 --error-target=http://localhost:8081/hub/error ``` (Details on how to do this is out of the scope of this tutorial. For example, it might be a systemd service configured within another docker container). The proxy has no From ced408c205763dad40812f560726d168f0bae5a5 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Sun, 23 Oct 2022 19:18:11 +0000 Subject: [PATCH 143/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/reference/separate-proxy.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/reference/separate-proxy.md b/docs/source/reference/separate-proxy.md index 057e2f5a..5ac29356 100644 --- a/docs/source/reference/separate-proxy.md +++ b/docs/source/reference/separate-proxy.md @@ -40,9 +40,11 @@ set to the URL which the hub uses to connect _to the proxy's API_. You need to configure a service to start the proxy. An example command line argument for this is: + ```bash $ configurable-http-proxy --ip=127.0.0.1 --port=8000 --api-ip=127.0.0.1 --api-port=8001 --default-target=http://localhost:8081 --error-target=http://localhost:8081/hub/error ``` + (Details on how to do this is out of the scope of this tutorial. For example, it might be a systemd service configured within another docker container). The proxy has no configuration files, all configuration is via the command line and From c4d8be22b607a885053c9552f0e445a072dd7bf4 Mon Sep 17 00:00:00 2001 From: Lili Yao <103026397+liliyao2022@users.noreply.github.com> Date: Mon, 24 Oct 2022 12:51:48 +1100 Subject: [PATCH 144/214] Update institutional-faq.md Corrected a spelling error. --- docs/source/getting-started/institutional-faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/getting-started/institutional-faq.md b/docs/source/getting-started/institutional-faq.md index d7f1a5dd..678184dd 100644 --- a/docs/source/getting-started/institutional-faq.md +++ b/docs/source/getting-started/institutional-faq.md @@ -78,7 +78,7 @@ gives administrators more control over their setup and hardware. Because JupyterHub is an open-source, community-driven tool, it can be extended and modified to fit an institution's needs. It plays nicely with the open source data science -stack, and can serve a variety of computing enviroments, user interfaces, and +stack, and can serve a variety of computing environments, user interfaces, and computational hardware. It can also be deployed anywhere - on enterprise cloud infrastructure, on High-Performance-Computing machines, on local hardware, or even on a single laptop, which is not possible with most other tools for shared interactive computing. From 37be9b4a5bdd2c517e80b86ca1fed3fffd2ad48d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 24 Oct 2022 05:27:46 +0000 Subject: [PATCH 145/214] Bump docker/setup-buildx-action from 2.1.0 to 2.2.1 Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2.1.0 to 2.2.1. - [Release notes](https://github.com/docker/setup-buildx-action/releases) - [Commits](https://github.com/docker/setup-buildx-action/compare/95cb08cb2672c73d4ffd2f422e6d11953d2a9c70...8c0edbc76e98fa90f69d9a2c020dcb50019dc325) --- updated-dependencies: - dependency-name: docker/setup-buildx-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] --- .github/workflows/release.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 180fa746..aa3a6f5e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -111,7 +111,7 @@ jobs: uses: docker/setup-qemu-action@e81a89b1732b9c48d79cd809d8d81d79c4647a18 # associated tag: v1.0.2 - name: Set up Docker Buildx (for multi-arch builds) - uses: docker/setup-buildx-action@95cb08cb2672c73d4ffd2f422e6d11953d2a9c70 + uses: docker/setup-buildx-action@8c0edbc76e98fa90f69d9a2c020dcb50019dc325 with: # Allows pushing to registry on localhost:5000 driver-opts: network=host From e76e9099c2bba2d472b02a2a09741e79beb27431 Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Mon, 24 Oct 2022 11:33:48 +0100 Subject: [PATCH 146/214] Update docs/source/reference/config-sudo.md Co-authored-by: Min RK --- docs/source/reference/config-sudo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/reference/config-sudo.md b/docs/source/reference/config-sudo.md index 55ad2fb3..1db337f1 100644 --- a/docs/source/reference/config-sudo.md +++ b/docs/source/reference/config-sudo.md @@ -46,7 +46,7 @@ We want to confine these permissions to only what we really need. ## Edit `/etc/sudoers` -To do this we write to `/etc/sudoers` (use `visudo` for safe editing of sudoers): +To do this we add to `/etc/sudoers` (use `visudo` for safe editing of sudoers): - specify the list of users `JUPYTER_USERS` for whom `rhea` can spawn servers - set the command `JUPYTER_CMD` that `rhea` can execute on behalf of users From 1c95d94b96c4e3d890b2cbc144d3c514ee44459c Mon Sep 17 00:00:00 2001 From: Christian Dike Date: Mon, 24 Oct 2022 11:38:58 +0100 Subject: [PATCH 147/214] Update config-sudo.md --- docs/source/reference/config-sudo.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/reference/config-sudo.md b/docs/source/reference/config-sudo.md index 1db337f1..2962ae97 100644 --- a/docs/source/reference/config-sudo.md +++ b/docs/source/reference/config-sudo.md @@ -253,6 +253,6 @@ $ semodule -i sudo_exec_selinux.pp ## Troubleshooting: PAM session errors If the PAM authentication doesn't work and you see errors for -`login:session-auth`, or similar, consider updating to a more recent [version -of jupyterhub](https://readthedocs.org/projects/jupyterhub/) and disabling the opening of PAM sessions with +`login:session-auth`, or similar, consider updating to a more recent version +of jupyterhub and disabling the opening of PAM sessions with `c.PAMAuthenticator.open_sessions=False`. From 9b7a77c36f52519dcb2cd45d0907815b424866ce Mon Sep 17 00:00:00 2001 From: mouse1203 Date: Mon, 24 Oct 2022 13:09:46 +0200 Subject: [PATCH 148/214] selenium: added docstrings added docstrings, parameter for open_url function, removed element function (unused) --- jupyterhub/tests/selenium/test_browser.py | 40 ++++++++++++++++------- 1 file changed, 28 insertions(+), 12 deletions(-) diff --git a/jupyterhub/tests/selenium/test_browser.py b/jupyterhub/tests/selenium/test_browser.py index 67c8a4cc..4e7ba00c 100644 --- a/jupyterhub/tests/selenium/test_browser.py +++ b/jupyterhub/tests/selenium/test_browser.py @@ -25,6 +25,9 @@ pytestmark = pytest.mark.selenium async def webdriver_wait(driver, condition, timeout=30): + """an async wrapper for selenium's wait function, + a condition is something from selenium's expected_conditions""" + return await exponential_backoff( partial(condition, driver), timeout=timeout, @@ -43,20 +46,25 @@ def in_thread(f, *args, **kwargs): return asyncio.get_event_loop().run_in_executor(None, partial(f, *args, **kwargs)) -async def open_url(app, browser): +async def open_url(app, browser, url="login"): """initiating open the login page in the browser""" - url = url_path_join(public_host(app), app.hub.base_url, "login") + + url = url_path_join(public_host(app), app.hub.base_url, url) await in_thread(browser.get, url) return url def click(browser, by_locator): + """wait for element to be visible, then click on it""" + WebDriverWait(browser, 10).until( EC.visibility_of_element_located(by_locator) ).click() def is_displayed(browser, by_locator): + """Whether the element is visible or not""" + return ( WebDriverWait(browser, 10) .until(EC.visibility_of_element_located(by_locator)) @@ -65,6 +73,8 @@ def is_displayed(browser, by_locator): def send_text(browser, by_locator, text): + """wait for element to be presented, then put the text in it""" + return ( WebDriverWait(browser, 10) .until(EC.presence_of_element_located(by_locator)) @@ -73,6 +83,8 @@ def send_text(browser, by_locator, text): def clear(browser, by_locator): + """wait for element to be presented, then clear the text in it""" + return ( WebDriverWait(browser, 10) .until(EC.presence_of_element_located(by_locator)) @@ -80,10 +92,6 @@ def clear(browser, by_locator): ) -def element(browser, by_locator): - WebDriverWait(browser, 10).until(EC.visibility_of_element_located(by_locator)) - - # LOGIN PAGE async def test_elements_of_login_page(app, browser): await open_url(app, browser) @@ -106,12 +114,12 @@ async def test_submit_login_form(app, browser): user = "test_user" pass_w = "test_user" - await open_url(app, browser) + await open_url(app, browser, url="login") redirected_url = ujoin(public_url(app), f"/user/{user}/") await login(browser, user, pass_w) # verify url contains username if f"/user/{user}/" not in browser.current_url: - webdriver_wait(browser, EC.url_to_be(redirected_url)) + await webdriver_wait(browser, EC.url_to_be(redirected_url)) else: pass assert browser.current_url == redirected_url @@ -155,7 +163,16 @@ async def test_submit_login_form(app, browser): ), ], ) -async def test_open_url_login(app, browser, url, params, redirected_url, form_action): +async def test_open_url_login( + app, + browser, + url, + params, + redirected_url, + form_action, + user='test_user', + pass_w='test_user', +): url = url_path_join(public_host(app), app.hub.base_url, url) url_new = url_concat(url, params) await in_thread(browser.get, url_new) @@ -167,9 +184,8 @@ async def test_open_url_login(app, browser, url, params, redirected_url, form_ac assert browser.title == LoginPageLocators.PAGE_TITLE assert form.endswith(form_action) # login in with params - await login(browser, user='test_user', pass_w='test_user') + await login(browser, user, pass_w) # verify next url + params - user = 'test_user' next_url = browser.current_url if url_escape(app.base_url) in form_action: assert next_url.endswith("param=value") @@ -178,7 +194,7 @@ async def test_open_url_login(app, browser, url, params, redirected_url, form_ac assert f"user/{user}/" not in next_url else: if next_url.endswith(f"/user/{user}/") == False: - webdriver_wait( + await webdriver_wait( browser, EC.url_to_be(ujoin(public_url(app), f"/user/{user}/")) ) assert next_url.endswith(f"/user/{user}/") From a2e59e686759057ced30fc49bfcbe39d1bf73ef7 Mon Sep 17 00:00:00 2001 From: Teniola Olowookere <90247181+Teniola-theDev@users.noreply.github.com> Date: Mon, 24 Oct 2022 13:03:04 +0100 Subject: [PATCH 149/214] Update docs/source/getting-started/authenticators-users-basics.md Co-authored-by: Min RK --- docs/source/getting-started/authenticators-users-basics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/getting-started/authenticators-users-basics.md b/docs/source/getting-started/authenticators-users-basics.md index 5b169eb2..344bccf2 100644 --- a/docs/source/getting-started/authenticators-users-basics.md +++ b/docs/source/getting-started/authenticators-users-basics.md @@ -74,7 +74,7 @@ After starting the Hub once, it is not sufficient to **remove** a user from the allowed users set in your config file. You must also remove the user from the Hub's database, either by deleting the user from JupyterHub's admin page, or you can clear the `jupyterhub.sqlite` database and start -afresh. +fresh. ## Use LocalAuthenticator to create system users From ec0cb70f355f60b4d6c46fe5472ae1a973b62a63 Mon Sep 17 00:00:00 2001 From: Teniola Olowookere <90247181+Teniola-theDev@users.noreply.github.com> Date: Mon, 24 Oct 2022 13:08:52 +0100 Subject: [PATCH 150/214] removed duplicate link --- docs/source/getting-started/authenticators-users-basics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/getting-started/authenticators-users-basics.md b/docs/source/getting-started/authenticators-users-basics.md index 344bccf2..51843b57 100644 --- a/docs/source/getting-started/authenticators-users-basics.md +++ b/docs/source/getting-started/authenticators-users-basics.md @@ -29,7 +29,7 @@ Instead, you can assign [roles](define-role-target) to users or groups with only the scopes they require. ``` -**[roles](https://jupyterhub.readthedocs.io/en/stable/rbac/roles.html#define-role-target)** +roles Admin users of JupyterHub, `admin_users`, can add and remove users from the user `allowed_users` set. `admin_users` can take actions on other users' From ce9feb5139c641e02f4a018aa6817e462db611f0 Mon Sep 17 00:00:00 2001 From: Emmanuella Orioma Date: Mon, 24 Oct 2022 15:09:29 +0300 Subject: [PATCH 151/214] Update use-case.md changed RBAC documentation url to the internal rbac jupyter documentation --- docs/source/rbac/use-cases.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/rbac/use-cases.md b/docs/source/rbac/use-cases.md index a6fb34ff..874c772d 100644 --- a/docs/source/rbac/use-cases.md +++ b/docs/source/rbac/use-cases.md @@ -9,7 +9,7 @@ To determine which scopes a role should have, one can follow these steps: 5. Customize the scopes with filters if needed 6. Define the role with required scopes and assign to users/services/groups/tokens -Below, different use cases are presented on how to use the [RBAC framework](https://jupyterhub.readthedocs.io/en/stable/rbac/index.html). +Below, different use cases are presented on how to use the [RBAC framework](./index.md) ## Service to cull idle servers From 48fc74b9b40353722a5f823f490d9934a845d01e Mon Sep 17 00:00:00 2001 From: Teniola Olowookere <90247181+Teniola-theDev@users.noreply.github.com> Date: Mon, 24 Oct 2022 13:17:52 +0100 Subject: [PATCH 152/214] removed extra text --- docs/source/getting-started/authenticators-users-basics.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/source/getting-started/authenticators-users-basics.md b/docs/source/getting-started/authenticators-users-basics.md index 51843b57..460579c9 100644 --- a/docs/source/getting-started/authenticators-users-basics.md +++ b/docs/source/getting-started/authenticators-users-basics.md @@ -29,8 +29,6 @@ Instead, you can assign [roles](define-role-target) to users or groups with only the scopes they require. ``` -roles - Admin users of JupyterHub, `admin_users`, can add and remove users from the user `allowed_users` set. `admin_users` can take actions on other users' behalf, such as stopping and restarting their servers. From d0a48c0655ad8003bdeeaa39aa0b7ec3b573e3df Mon Sep 17 00:00:00 2001 From: Joel-Ando Date: Mon, 24 Oct 2022 13:28:28 +0100 Subject: [PATCH 153/214] Update tech-implementation.md --- docs/source/rbac/tech-implementation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/rbac/tech-implementation.md b/docs/source/rbac/tech-implementation.md index 496d4b7f..d4ff4cc0 100644 --- a/docs/source/rbac/tech-implementation.md +++ b/docs/source/rbac/tech-implementation.md @@ -43,7 +43,7 @@ Prior to 3.0, tokens stored _roles_, which meant their scopes were resolved on each request. ::: -API tokens grant access to JupyterHub's APIs. The [RBAC framework](https://jupyterhub.readthedocs.io/en/stable/rbac/index.html) allows for requesting tokens with specific permissions. +API tokens grant access to JupyterHub's APIs. The [RBAC framework](https://jupyterhub.readthedocs.io/en/stable/rbac/upgrade.html) allows for requesting tokens with specific permissions. RBAC is involved in several stages of the OAuth token flow. From 3e4cc0b86957e0f66d8ffeb12bc9d80e154df299 Mon Sep 17 00:00:00 2001 From: Joel-Ando Date: Mon, 24 Oct 2022 14:14:48 +0100 Subject: [PATCH 154/214] Update tech-implementation.md --- docs/source/rbac/tech-implementation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/rbac/tech-implementation.md b/docs/source/rbac/tech-implementation.md index d4ff4cc0..7db16472 100644 --- a/docs/source/rbac/tech-implementation.md +++ b/docs/source/rbac/tech-implementation.md @@ -43,7 +43,7 @@ Prior to 3.0, tokens stored _roles_, which meant their scopes were resolved on each request. ::: -API tokens grant access to JupyterHub's APIs. The [RBAC framework](https://jupyterhub.readthedocs.io/en/stable/rbac/upgrade.html) allows for requesting tokens with specific permissions. +API tokens grant access to JupyterHub's APIs. The [RBAC framework](https://github.com/jupyterhub/jupyterhub/blob/main/docs/source/rbac/index.md) allows for requesting tokens with specific permissions. RBAC is involved in several stages of the OAuth token flow. From 3da4dc66c9848375df41513c97e459032b80b661 Mon Sep 17 00:00:00 2001 From: Allan Wasega Date: Mon, 24 Oct 2022 17:44:57 +0300 Subject: [PATCH 155/214] Incorporated suggested edits --- docs/source/reference/server-api.md | 41 +++++++++++++++++------------ 1 file changed, 24 insertions(+), 17 deletions(-) diff --git a/docs/source/reference/server-api.md b/docs/source/reference/server-api.md index 6e49f081..69753cb8 100644 --- a/docs/source/reference/server-api.md +++ b/docs/source/reference/server-api.md @@ -1,8 +1,11 @@ # Starting servers with the JupyterHub API -Sometimes, when working with applications such as [BinderHub](https://binderhub.readthedocs.io), it may be necessary to launch Jupyter-based services on behalf of your users. Doing so can be achieved through JupyterHub's [REST API](../reference/rest.md), which allows one to launch and manage servers on behalf of users through API calls instead of the JupyterHub UI. In doing so, you can take advantage of other user/launch/lifecycle patterns that are not natively supported by the JupyterHub UI, all without the need to develop the server management features of JupyterHub Spawners and/or Authenticators. +Sometimes, when working with applications such as [BinderHub](https://binderhub.readthedocs.io), it may be necessary to launch Jupyter-based services on behalf of your users. +Doing so can be achieved through JupyterHub's [REST API](../reference/rest.md), which allows one to launch and manage servers on behalf of users through API calls instead of the JupyterHub UI. +This way, you can take advantage of other user/launch/lifecycle patterns that are not natively supported by the JupyterHub UI, all without the need to develop the server management features of JupyterHub Spawners and/or Authenticators. -This tutorial goes through the processes involved while working with the JupyterHub API to manage servers for users. In particular, it covers how to: +This tutorial goes through working with the JupyterHub API to manage servers for users. +In particular, it covers how to: 1. [Check the status of servers](checking) 2. [Start servers](starting) @@ -174,7 +177,8 @@ You can keep making this check until `ready` is true. ### Using the progress API -The most _efficient_ way to wait for a server to start is by using the progress API. The progress URL is available in the server model under `progress_url` and has the form `/hub/api/users/:user/servers/:servername/progress`. +The most _efficient_ way to wait for a server to start is by using the progress API. +The progress URL is available in the server model under `progress_url` and has the form `/hub/api/users/:user/servers/:servername/progress`. The default server progress can be accessed at `:user/servers//progress` or `:user/server/progress` as demonstrated in the following GET request: @@ -184,7 +188,8 @@ GET /hub/api/users/:user/servers/:servername/progress **Required scope: `read:servers`** -The progress API is an example of an [EventStream][] API. Therefore, messages are _streamed_ and delivered in the form: +The progress API is an example of an [EventStream][] API. +Messages are _streamed_ and delivered in the form: ``` data: {"progress": 10, "message": "...", ...} @@ -218,7 +223,8 @@ ready url : only present if `ready` is true; will be the server's URL -The progress API can be used even with fully ready servers. In such a situation, there will only be one event response of the form: +The progress API can be used even with fully ready servers. +If the server is ready, there will only be one event, which will look like: ```json { @@ -230,9 +236,10 @@ The progress API can be used even with fully ready servers. In such a situation, } ``` -In this case, `ready` and `url` are the same as in the server model, and `ready` will always be true. +where `ready` and `url` are the same as in the server model, and `ready` will always be true. -A significant advantage of the progress API is that it shows the status of the server through a stream of messages. Below is an example of a typical complete stream from the API: +A significant advantage of the progress API is that it shows the status of the server through a stream of messages. +Below is an example of a typical complete stream from the API: ``` @@ -262,20 +269,21 @@ DELETE /hub/api/users/:user/servers/[:servername] **Required scope: `servers`** -Similar to when starting a server, issuing the DELETE request above might not stop the server immediately. Instead, the DELETE request has two possible response codes: +Similar to when starting a server, issuing the DELETE request above might not stop the server immediately. +Instead, the DELETE request has two possible response codes: 204 Deleted : This status code means the delete completed and the server is fully stopped. It will now be absent from the user `servers` model. 202 Accepted -: This code means your request was accepted, but is not yet completely processed. +: This code means your request was accepted but is not yet completely processed. The server has `pending: 'stop'` at this point. There is no progress API for checking when a server actually stops. -Thus, the only available alternative is to poll the server and wait for it to disappear from the user `servers` model. +The only way to wait for a server to stop is to poll it and wait for the server to disappear from the user `servers` model. -This Python code snippet can be used to check if a server stops: +This Python code snippet can be used to stop a server and the wait for the process to complete: ```{literalinclude} ../../../examples/server-api/start-stop-server.py :language: python @@ -298,17 +306,16 @@ a token must be owned by the same user as the server, The URL returned from a server model is the URL path suffix, e.g. `/user/:name/` to append to the jupyterhub base URL. - -For instance, `{hub_url}{server_url}`, -where `hub_url` would be such as `http://127.0.0.1:8000` by default, -and `server_url` is `/user/myname.` When combined, the two give a full URL of `http://127.0.0.1:8000/user/myname`. +The returned URL is of the form `{hub_url}{server_url}`, +where `hub_url` would be `http://127.0.0.1:8000` by default and `server_url` is `/user/myname`. +When combined, the two give a full URL of `http://127.0.0.1:8000/user/myname`. ## Python example The JupyterHub repo includes a complete example in {file}`examples/server-api` -tying all the above mentioned steps together. +that ties all theses steps together. -In summary, the steps involved while managing servers on behalf of users are: +In summary, the processes involved in managing servers on behalf of users are: 1. Get user information from `/user/:name`. 2. The server model includes a `ready` state to tell you if it's ready. From eb72067ab3dff03f74dcca9e3155e44630ab9564 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 24 Oct 2022 14:46:41 +0000 Subject: [PATCH 156/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/reference/server-api.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/source/reference/server-api.md b/docs/source/reference/server-api.md index 69753cb8..03369dab 100644 --- a/docs/source/reference/server-api.md +++ b/docs/source/reference/server-api.md @@ -1,10 +1,10 @@ # Starting servers with the JupyterHub API -Sometimes, when working with applications such as [BinderHub](https://binderhub.readthedocs.io), it may be necessary to launch Jupyter-based services on behalf of your users. -Doing so can be achieved through JupyterHub's [REST API](../reference/rest.md), which allows one to launch and manage servers on behalf of users through API calls instead of the JupyterHub UI. +Sometimes, when working with applications such as [BinderHub](https://binderhub.readthedocs.io), it may be necessary to launch Jupyter-based services on behalf of your users. +Doing so can be achieved through JupyterHub's [REST API](../reference/rest.md), which allows one to launch and manage servers on behalf of users through API calls instead of the JupyterHub UI. This way, you can take advantage of other user/launch/lifecycle patterns that are not natively supported by the JupyterHub UI, all without the need to develop the server management features of JupyterHub Spawners and/or Authenticators. -This tutorial goes through working with the JupyterHub API to manage servers for users. +This tutorial goes through working with the JupyterHub API to manage servers for users. In particular, it covers how to: 1. [Check the status of servers](checking) @@ -177,7 +177,7 @@ You can keep making this check until `ready` is true. ### Using the progress API -The most _efficient_ way to wait for a server to start is by using the progress API. +The most _efficient_ way to wait for a server to start is by using the progress API. The progress URL is available in the server model under `progress_url` and has the form `/hub/api/users/:user/servers/:servername/progress`. The default server progress can be accessed at `:user/servers//progress` or `:user/server/progress` as demonstrated in the following GET request: @@ -188,7 +188,7 @@ GET /hub/api/users/:user/servers/:servername/progress **Required scope: `read:servers`** -The progress API is an example of an [EventStream][] API. +The progress API is an example of an [EventStream][] API. Messages are _streamed_ and delivered in the form: ``` @@ -223,7 +223,7 @@ ready url : only present if `ready` is true; will be the server's URL -The progress API can be used even with fully ready servers. +The progress API can be used even with fully ready servers. If the server is ready, there will only be one event, which will look like: ```json @@ -238,7 +238,7 @@ If the server is ready, there will only be one event, which will look like: where `ready` and `url` are the same as in the server model, and `ready` will always be true. -A significant advantage of the progress API is that it shows the status of the server through a stream of messages. +A significant advantage of the progress API is that it shows the status of the server through a stream of messages. Below is an example of a typical complete stream from the API: ``` @@ -269,7 +269,7 @@ DELETE /hub/api/users/:user/servers/[:servername] **Required scope: `servers`** -Similar to when starting a server, issuing the DELETE request above might not stop the server immediately. +Similar to when starting a server, issuing the DELETE request above might not stop the server immediately. Instead, the DELETE request has two possible response codes: 204 Deleted @@ -307,7 +307,7 @@ a token must be owned by the same user as the server, The URL returned from a server model is the URL path suffix, e.g. `/user/:name/` to append to the jupyterhub base URL. The returned URL is of the form `{hub_url}{server_url}`, -where `hub_url` would be `http://127.0.0.1:8000` by default and `server_url` is `/user/myname`. +where `hub_url` would be `http://127.0.0.1:8000` by default and `server_url` is `/user/myname`. When combined, the two give a full URL of `http://127.0.0.1:8000/user/myname`. ## Python example From 7a22a2cf9302d0323eff05cff8072385461cb59a Mon Sep 17 00:00:00 2001 From: lumenCodes Date: Mon, 24 Oct 2022 17:36:38 +0100 Subject: [PATCH 157/214] Addittion to the contributing .md --- docs/source/contributing/community.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/community.md b/docs/source/contributing/community.md index 2c78fb79..25cbd930 100644 --- a/docs/source/contributing/community.md +++ b/docs/source/contributing/community.md @@ -4,7 +4,7 @@ We use different channels of communication for different purposes. Whichever one ## Discourse (recommended) -We use [Discourse](https://discourse.jupyter.org) for online discussions and support questions. Everyone in the Jupyter community is welcome to bring ideas and questions there. +We use [Discourse](https://discourse.jupyter.org) for online discussions and support questions. You can ask questions here if you are a first-time contributor to the jupyterhub project. Everyone in the Jupyter community is welcome to bring ideas and questions there. We recommend that you first use our Discourse as all past and current discussions on it are archived and searchable. Thus, all discussions remain useful and accessible to the whole community. From 5cfa16dbfdedde07e12cda3ec2ddc3b6fab971c6 Mon Sep 17 00:00:00 2001 From: Uzochukwu Precious Date: Mon, 24 Oct 2022 19:21:43 +0100 Subject: [PATCH 158/214] review update --- docs/source/reference/rest.md | 46 +++++++++++++++++------------------ 1 file changed, 23 insertions(+), 23 deletions(-) diff --git a/docs/source/reference/rest.md b/docs/source/reference/rest.md index 13bec97b..440095a8 100644 --- a/docs/source/reference/rest.md +++ b/docs/source/reference/rest.md @@ -4,14 +4,14 @@ This section will give you information on: -- [What you can do with the API](#what-you-can-do-with-the-api) -- [How to create an API token](#create-an-api-token) -- [Assigning permissions to a token](#assigning-permissions-to-a-token) -- [Updating to admin services](#updating-to-admin-services) -- [Making an API request programmatically using the requests library](#make-an-api-request) -- [Paginating API requests](#paginating-api-requests) -- [Enabling users to spawn multiple named-servers via the API](#enabling-users-to-spawn-multiple-named-servers-via-the-api) -- [Learn more about JupyterHub's API](#learn-more-about-the-api) +- What you can do with the API +- How to create an API token +- Assigning permissions to a token +- Updating to admin services +- Making an API request programmatically using the requests library +- Paginating API requests +- Enabling users to spawn multiple named-servers via the API +- Learn more about JupyterHub's API ## What you can do with the API @@ -39,7 +39,7 @@ The preferred way of generating an API token is by running: openssl rand -hex 32 ``` -The `openssl` command generates a potential token that can then be +This `openssl` command generates a potential token that can then be added to JupyterHub using `.api_tokens` configuration setting in `jupyterhub_config.py`. @@ -48,7 +48,7 @@ The api_tokens configuration has been softly deprecated since the introduction o ``` Alternatively, you can use the `jupyterhub token` command to generate a token -for a specific hub user by passing the **_username_**: +for a specific hub user by passing the **username**: ```bash jupyterhub token @@ -60,19 +60,19 @@ it for the given user with the Hub's database. In [version 0.8.0](../changelog.md), a token request page for generating an API token is available from the JupyterHub user interface: -```{figure} ../images/token-request.png ---- -name: token-request ---- -JupyterHub's API token page -``` +:::{figure-md} -```{figure} ../images/token-request-success.png ---- -name: token-request-success ---- -JupyterHub's API token success page -``` +![token request page](../images/token-request.png) + +JupyterHub's API token page +::: + +:::{figure-md} +![token-request-success](../images/token-request-success.png) + +JupyterHub's token page after successfully requesting a token. + +::: ## Assigning permissions to a token @@ -324,5 +324,5 @@ You can see the full [JupyterHub REST API][] for more details. [openapi initiative]: https://www.openapis.org/ [jupyterhub rest api]: ./rest-api -[scopes]: https://jupyterhub.readthedocs.io/en/stable/rbac/scopes.html +[scopes]: ../rbac/scopes.md [jupyter notebook rest api]: https://petstore3.swagger.io/?url=https://raw.githubusercontent.com/jupyter/notebook/HEAD/notebook/services/api/api.yaml From 3b4c40e5e0f4f02c10a5cdb6b3a5f6591e8ed14f Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 24 Oct 2022 18:22:25 +0000 Subject: [PATCH 159/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/reference/rest.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/reference/rest.md b/docs/source/reference/rest.md index 440095a8..9741f45e 100644 --- a/docs/source/reference/rest.md +++ b/docs/source/reference/rest.md @@ -67,7 +67,7 @@ generating an API token is available from the JupyterHub user interface: JupyterHub's API token page ::: -:::{figure-md} +:::{figure-md} ![token-request-success](../images/token-request-success.png) JupyterHub's token page after successfully requesting a token. From d5beda293b206e04bda92bba5352bba1fe7d6338 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 24 Oct 2022 23:08:59 +0000 Subject: [PATCH 160/214] [pre-commit.ci] pre-commit autoupdate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/asottile/pyupgrade: v2.37.3 → v3.1.0](https://github.com/asottile/pyupgrade/compare/v2.37.3...v3.1.0) - [github.com/psf/black: 22.6.0 → 22.10.0](https://github.com/psf/black/compare/22.6.0...22.10.0) - [github.com/pre-commit/mirrors-prettier: v2.7.1 → v3.0.0-alpha.3](https://github.com/pre-commit/mirrors-prettier/compare/v2.7.1...v3.0.0-alpha.3) --- .pre-commit-config.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index e3689228..582e2610 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -11,7 +11,7 @@ repos: # Autoformat: Python code, syntax patterns are modernized - repo: https://github.com/asottile/pyupgrade - rev: v2.37.3 + rev: v3.1.0 hooks: - id: pyupgrade args: @@ -25,13 +25,13 @@ repos: # Autoformat: Python code - repo: https://github.com/psf/black - rev: 22.6.0 + rev: 22.10.0 hooks: - id: black # Autoformat: markdown, yaml, javascript (see the file .prettierignore) - repo: https://github.com/pre-commit/mirrors-prettier - rev: v2.7.1 + rev: v3.0.0-alpha.3 hooks: - id: prettier From 7c92902e48985028cc60648990f34c376be8ce1d Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Mon, 24 Oct 2022 23:10:46 +0000 Subject: [PATCH 161/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- jsx/src/components/AddUser/AddUser.jsx | 6 +++--- jsx/src/components/AddUser/AddUser.test.js | 2 +- .../components/CreateGroup/CreateGroup.jsx | 4 ++-- .../CreateGroup/CreateGroup.test.js | 2 +- jsx/src/components/EditUser/EditUser.jsx | 8 ++++---- jsx/src/components/GroupEdit/GroupEdit.jsx | 6 +++--- .../components/GroupSelect/GroupSelect.jsx | 2 +- jsx/src/components/Groups/Groups.jsx | 2 +- .../ServerDashboard/ServerDashboard.jsx | 20 +++++++++---------- .../ServerDashboard/ServerDashboard.test.js | 14 ++++++------- jsx/src/util/withAPI.js | 4 ++-- jsx/webpack.config.js | 4 ++-- share/jupyterhub/static/js/admin.js | 10 +++++----- share/jupyterhub/static/js/jhapi.js | 10 +++++----- share/jupyterhub/static/js/token.js | 2 +- share/jupyterhub/static/js/utils.js | 2 +- 16 files changed, 49 insertions(+), 49 deletions(-) diff --git a/jsx/src/components/AddUser/AddUser.jsx b/jsx/src/components/AddUser/AddUser.jsx index 3826ad2c..b56279bd 100644 --- a/jsx/src/components/AddUser/AddUser.jsx +++ b/jsx/src/components/AddUser/AddUser.jsx @@ -98,13 +98,13 @@ const AddUser = (props) => { .then((data) => dispatchPageChange(data, 0)) .then(() => history.push("/")) .catch(() => - setErrorAlert(`Failed to update users.`) + setErrorAlert(`Failed to update users.`), ) : setErrorAlert( `Failed to create user. ${ data.status == 409 ? "User already exists." : "" - }` - ) + }`, + ), ) .catch(() => setErrorAlert(`Failed to create user.`)); }} diff --git a/jsx/src/components/AddUser/AddUser.test.js b/jsx/src/components/AddUser/AddUser.test.js index e3965f15..ee91fdbd 100644 --- a/jsx/src/components/AddUser/AddUser.test.js +++ b/jsx/src/components/AddUser/AddUser.test.js @@ -131,7 +131,7 @@ test("Shows a more specific UI error dialogue when user creation returns an impr }); let errorDialog = screen.getByText( - "Failed to create user. User already exists." + "Failed to create user. User already exists.", ); expect(errorDialog).toBeVisible(); diff --git a/jsx/src/components/CreateGroup/CreateGroup.jsx b/jsx/src/components/CreateGroup/CreateGroup.jsx index 66b8b9d8..37deb4ed 100644 --- a/jsx/src/components/CreateGroup/CreateGroup.jsx +++ b/jsx/src/components/CreateGroup/CreateGroup.jsx @@ -81,14 +81,14 @@ const CreateGroup = (props) => { .then((data) => dispatchPageUpdate(data, 0)) .then(() => history.push("/groups")) .catch(() => - setErrorAlert(`Could not update groups list.`) + setErrorAlert(`Could not update groups list.`), ) : setErrorAlert( `Failed to create group. ${ data.status == 409 ? "Group already exists." : "" - }` + }`, ); }) .catch(() => setErrorAlert(`Failed to create group.`)); diff --git a/jsx/src/components/CreateGroup/CreateGroup.test.js b/jsx/src/components/CreateGroup/CreateGroup.test.js index c4b44148..cfa20128 100644 --- a/jsx/src/components/CreateGroup/CreateGroup.test.js +++ b/jsx/src/components/CreateGroup/CreateGroup.test.js @@ -107,7 +107,7 @@ test("Shows a more specific UI error dialogue when user creation returns an impr }); let errorDialog = screen.getByText( - "Failed to create group. Group already exists." + "Failed to create group. Group already exists.", ); expect(errorDialog).toBeVisible(); diff --git a/jsx/src/components/EditUser/EditUser.jsx b/jsx/src/components/EditUser/EditUser.jsx index 211b9c46..2575b808 100644 --- a/jsx/src/components/EditUser/EditUser.jsx +++ b/jsx/src/components/EditUser/EditUser.jsx @@ -96,8 +96,8 @@ const EditUser = (props) => { .then(() => history.push("/")) .catch(() => setErrorAlert( - `Could not update users list.` - ) + `Could not update users list.`, + ), ) : setErrorAlert(`Failed to edit user.`); }) @@ -129,7 +129,7 @@ const EditUser = (props) => { editUser( username, updatedUsername != "" ? updatedUsername : username, - admin + admin, ) .then((data) => { data.status < 300 @@ -137,7 +137,7 @@ const EditUser = (props) => { .then((data) => dispatchPageChange(data, 0)) .then(() => history.push("/")) .catch(() => - setErrorAlert(`Could not update users list.`) + setErrorAlert(`Could not update users list.`), ) : setErrorAlert(`Failed to edit user.`); }) diff --git a/jsx/src/components/GroupEdit/GroupEdit.jsx b/jsx/src/components/GroupEdit/GroupEdit.jsx index 26225beb..21ac69b3 100644 --- a/jsx/src/components/GroupEdit/GroupEdit.jsx +++ b/jsx/src/components/GroupEdit/GroupEdit.jsx @@ -94,10 +94,10 @@ const GroupEdit = (props) => { } let new_users = selected.filter( - (e) => !group_data.users.includes(e) + (e) => !group_data.users.includes(e), ); let removed_users = group_data.users.filter( - (e) => !selected.includes(e) + (e) => !selected.includes(e), ); let promiseQueue = []; @@ -105,7 +105,7 @@ const GroupEdit = (props) => { promiseQueue.push(addToGroup(new_users, group_data.name)); if (removed_users.length > 0) promiseQueue.push( - removeFromGroup(removed_users, group_data.name) + removeFromGroup(removed_users, group_data.name), ); Promise.all(promiseQueue) diff --git a/jsx/src/components/GroupSelect/GroupSelect.jsx b/jsx/src/components/GroupSelect/GroupSelect.jsx index aa75e528..2f48bf3e 100644 --- a/jsx/src/components/GroupSelect/GroupSelect.jsx +++ b/jsx/src/components/GroupSelect/GroupSelect.jsx @@ -90,7 +90,7 @@ const GroupSelect = (props) => { > {e} - ) + ), )} diff --git a/jsx/src/components/Groups/Groups.jsx b/jsx/src/components/Groups/Groups.jsx index 634888d8..42e987dd 100644 --- a/jsx/src/components/Groups/Groups.jsx +++ b/jsx/src/components/Groups/Groups.jsx @@ -37,7 +37,7 @@ const Groups = (props) => { useEffect(() => { updateGroups(offset, limit).then((data) => - dispatchPageUpdate(data.items, data._pagination) + dispatchPageUpdate(data.items, data._pagination), ); }, [offset, limit]); diff --git a/jsx/src/components/ServerDashboard/ServerDashboard.jsx b/jsx/src/components/ServerDashboard/ServerDashboard.jsx index 0ae4275d..0ec29fd7 100644 --- a/jsx/src/components/ServerDashboard/ServerDashboard.jsx +++ b/jsx/src/components/ServerDashboard/ServerDashboard.jsx @@ -38,11 +38,11 @@ const ServerDashboard = (props) => { adminAsc = (e) => e.sort((a) => (a.admin ? 1 : -1)), dateDesc = (e) => e.sort((a, b) => - new Date(a.last_activity) - new Date(b.last_activity) > 0 ? -1 : 1 + new Date(a.last_activity) - new Date(b.last_activity) > 0 ? -1 : 1, ), dateAsc = (e) => e.sort((a, b) => - new Date(a.last_activity) - new Date(b.last_activity) > 0 ? 1 : -1 + new Date(a.last_activity) - new Date(b.last_activity) > 0 ? 1 : -1, ), runningAsc = (e) => e.sort((a) => (a.server == null ? -1 : 1)), runningDesc = (e) => e.sort((a) => (a.server == null ? 1 : -1)); @@ -136,7 +136,7 @@ const ServerDashboard = (props) => { dispatchPageUpdate( data.items, data._pagination, - name_filter + name_filter, ); }) .catch(() => { @@ -176,7 +176,7 @@ const ServerDashboard = (props) => { dispatchPageUpdate( data.items, data._pagination, - name_filter + name_filter, ); }) .catch(() => { @@ -471,7 +471,7 @@ const ServerDashboard = (props) => { failedServers.length > 1 ? "servers" : "server" }. ${ failedServers.length > 1 ? "Are they " : "Is it " - } already running?` + } already running?`, ); } return res; @@ -482,11 +482,11 @@ const ServerDashboard = (props) => { dispatchPageUpdate( data.items, data._pagination, - name_filter + name_filter, ); }) .catch(() => - setErrorAlert(`Failed to update users list.`) + setErrorAlert(`Failed to update users list.`), ); return res; }) @@ -511,7 +511,7 @@ const ServerDashboard = (props) => { failedServers.length > 1 ? "servers" : "server" }. ${ failedServers.length > 1 ? "Are they " : "Is it " - } already stopped?` + } already stopped?`, ); } return res; @@ -522,11 +522,11 @@ const ServerDashboard = (props) => { dispatchPageUpdate( data.items, data._pagination, - name_filter + name_filter, ); }) .catch(() => - setErrorAlert(`Failed to update users list.`) + setErrorAlert(`Failed to update users list.`), ); return res; }) diff --git a/jsx/src/components/ServerDashboard/ServerDashboard.test.js b/jsx/src/components/ServerDashboard/ServerDashboard.test.js index 7213a081..901badae 100644 --- a/jsx/src/components/ServerDashboard/ServerDashboard.test.js +++ b/jsx/src/components/ServerDashboard/ServerDashboard.test.js @@ -369,7 +369,7 @@ test("Shows a UI error dialogue when start all servers fails", async () => { /> - + , ); }); @@ -403,7 +403,7 @@ test("Shows a UI error dialogue when stop all servers fails", async () => { /> - + , ); }); @@ -437,7 +437,7 @@ test("Shows a UI error dialogue when start user server fails", async () => { /> - + , ); }); @@ -471,7 +471,7 @@ test("Shows a UI error dialogue when start user server returns an improper statu /> - + , ); }); @@ -505,7 +505,7 @@ test("Shows a UI error dialogue when stop user servers fails", async () => { /> - + , ); }); @@ -539,7 +539,7 @@ test("Shows a UI error dialogue when stop user server returns an improper status /> - + , ); }); @@ -585,7 +585,7 @@ test("Search for user calls updateUsers with name filter", async () => { /> - + , ); }); diff --git a/jsx/src/util/withAPI.js b/jsx/src/util/withAPI.js index 198d3387..f17f7f25 100644 --- a/jsx/src/util/withAPI.js +++ b/jsx/src/util/withAPI.js @@ -7,11 +7,11 @@ const withAPI = withProps(() => ({ `/users?include_stopped_servers&offset=${offset}&limit=${limit}&name_filter=${ name_filter || "" }`, - "GET" + "GET", ).then((data) => data.json()), updateGroups: (offset, limit) => jhapiRequest(`/groups?offset=${offset}&limit=${limit}`, "GET").then( - (data) => data.json() + (data) => data.json(), ), shutdownHub: () => jhapiRequest("/shutdown", "POST"), startServer: (name, serverName = "") => diff --git a/jsx/webpack.config.js b/jsx/webpack.config.js index ee877af7..426066fc 100644 --- a/jsx/webpack.config.js +++ b/jsx/webpack.config.js @@ -41,10 +41,10 @@ module.exports = { const app = devServer.app; var user_data = JSON.parse( - '[{"kind":"user","name":"foo","admin":true,"groups":[],"server":"/user/foo/","pending":null,"created":"2020-12-07T18:46:27.112695Z","last_activity":"2020-12-07T21:00:33.336354Z","servers":{"":{"name":"","last_activity":"2020-12-07T20:58:02.437408Z","started":"2020-12-07T20:58:01.508266Z","pending":null,"ready":true,"state":{"pid":28085},"url":"/user/foo/","user_options":{},"progress_url":"/hub/api/users/foo/server/progress"}}},{"kind":"user","name":"bar","admin":false,"groups":[],"server":null,"pending":null,"created":"2020-12-07T18:46:27.115528Z","last_activity":"2020-12-07T20:43:51.013613Z","servers":{}}]' + '[{"kind":"user","name":"foo","admin":true,"groups":[],"server":"/user/foo/","pending":null,"created":"2020-12-07T18:46:27.112695Z","last_activity":"2020-12-07T21:00:33.336354Z","servers":{"":{"name":"","last_activity":"2020-12-07T20:58:02.437408Z","started":"2020-12-07T20:58:01.508266Z","pending":null,"ready":true,"state":{"pid":28085},"url":"/user/foo/","user_options":{},"progress_url":"/hub/api/users/foo/server/progress"}}},{"kind":"user","name":"bar","admin":false,"groups":[],"server":null,"pending":null,"created":"2020-12-07T18:46:27.115528Z","last_activity":"2020-12-07T20:43:51.013613Z","servers":{}}]', ); var group_data = JSON.parse( - '[{"kind":"group","name":"testgroup","users":[]}, {"kind":"group","name":"testgroup2","users":["foo", "bar"]}]' + '[{"kind":"group","name":"testgroup","users":[]}, {"kind":"group","name":"testgroup2","users":["foo", "bar"]}]', ); // get user_data diff --git a/share/jupyterhub/static/js/admin.js b/share/jupyterhub/static/js/admin.js index d167853f..89af21ac 100644 --- a/share/jupyterhub/static/js/admin.js +++ b/share/jupyterhub/static/js/admin.js @@ -5,7 +5,7 @@ require(["jquery", "moment", "jhapi", "utils"], function ( $, moment, JHAPI, - utils + utils, ) { "use strict"; @@ -113,7 +113,7 @@ require(["jquery", "moment", "jhapi", "utils"], function ( var serverName = row.data("server-name"); el.attr( "href", - utils.url_path_join(prefix, "user", user, serverName) + "/" + utils.url_path_join(prefix, "user", user, serverName) + "/", ); }); @@ -127,7 +127,7 @@ require(["jquery", "moment", "jhapi", "utils"], function ( var serverName = row.data("server-name"); el.attr( "href", - utils.url_path_join(prefix, "hub/spawn", user, serverName) + utils.url_path_join(prefix, "hub/spawn", user, serverName), ); }); // cannot start all servers in this case @@ -187,7 +187,7 @@ require(["jquery", "moment", "jhapi", "utils"], function ( success: function () { window.location.reload(); }, - } + }, ); }); @@ -241,7 +241,7 @@ require(["jquery", "moment", "jhapi", "utils"], function ( success: function () { window.location.reload(); }, - } + }, ); }); diff --git a/share/jupyterhub/static/js/jhapi.js b/share/jupyterhub/static/js/jhapi.js index 169cc143..ba3c7a53 100644 --- a/share/jupyterhub/static/js/jhapi.js +++ b/share/jupyterhub/static/js/jhapi.js @@ -38,7 +38,7 @@ define(["jquery", "utils"], function ($, utils) { var url = utils.url_path_join( this.base_url, "api", - utils.encode_uri_components(path) + utils.encode_uri_components(path), ); $.ajax(url, options); }; @@ -54,7 +54,7 @@ define(["jquery", "utils"], function ($, utils) { options = update(options, { type: "POST", dataType: null }); this.api_request( utils.url_path_join("users", user, "servers", server_name), - options + options, ); }; @@ -69,7 +69,7 @@ define(["jquery", "utils"], function ($, utils) { options = update(options, { type: "DELETE", dataType: null }); this.api_request( utils.url_path_join("users", user, "servers", server_name), - options + options, ); }; @@ -119,7 +119,7 @@ define(["jquery", "utils"], function ($, utils) { this.api_request( utils.url_path_join("users", user, "admin-access"), - options + options, ); }; @@ -143,7 +143,7 @@ define(["jquery", "utils"], function ($, utils) { options = update(options, { type: "DELETE" }); this.api_request( utils.url_path_join("users", user, "tokens", token_id), - options + options, ); }; diff --git a/share/jupyterhub/static/js/token.js b/share/jupyterhub/static/js/token.js index 3022a3f2..96ac559d 100644 --- a/share/jupyterhub/static/js/token.js +++ b/share/jupyterhub/static/js/token.js @@ -33,7 +33,7 @@ require(["jquery", "jhapi", "moment"], function ($, JHAPI, moment) { $("#token-result").text(reply.token); $("#token-area").show(); }, - } + }, ); return false; }); diff --git a/share/jupyterhub/static/js/utils.js b/share/jupyterhub/static/js/utils.js index 96b3ee91..b57c72a0 100644 --- a/share/jupyterhub/static/js/utils.js +++ b/share/jupyterhub/static/js/utils.js @@ -73,7 +73,7 @@ define(["jquery"], function ($) { ua = navigator.userAgent, tem; var M = ua.match( - /(opera|chrome|safari|firefox|msie)\/?\s*(\.?\d+(\.\d+)*)/i + /(opera|chrome|safari|firefox|msie)\/?\s*(\.?\d+(\.\d+)*)/i, ); if (M && (tem = ua.match(/version\/([\.\d]+)/i)) !== null) M[2] = tem[1]; M = M ? [M[1], M[2]] : [N, navigator.appVersion, "-?"]; From 5f32abeebae6795471fb03d4a962e6d0c838def6 Mon Sep 17 00:00:00 2001 From: Joel-Ando Date: Tue, 25 Oct 2022 14:55:53 +0100 Subject: [PATCH 162/214] Update tech-implementation.md --- docs/source/rbac/tech-implementation.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/rbac/tech-implementation.md b/docs/source/rbac/tech-implementation.md index 7db16472..bc765020 100644 --- a/docs/source/rbac/tech-implementation.md +++ b/docs/source/rbac/tech-implementation.md @@ -43,7 +43,7 @@ Prior to 3.0, tokens stored _roles_, which meant their scopes were resolved on each request. ::: -API tokens grant access to JupyterHub's APIs. The [RBAC framework](https://github.com/jupyterhub/jupyterhub/blob/main/docs/source/rbac/index.md) allows for requesting tokens with specific permissions. +API tokens grant access to JupyterHub's APIs. The [RBAC framework](./index.md) allows for requesting tokens with specific permissions. RBAC is involved in several stages of the OAuth token flow. From cde1b652525199bb9381e095f1163554a8191bd4 Mon Sep 17 00:00:00 2001 From: Chinwendu Date: Tue, 25 Oct 2022 16:11:52 +0100 Subject: [PATCH 163/214] Update docs/source/contributing/community.md Co-authored-by: Min RK --- docs/source/contributing/community.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/source/contributing/community.md b/docs/source/contributing/community.md index 25cbd930..f7273ad1 100644 --- a/docs/source/contributing/community.md +++ b/docs/source/contributing/community.md @@ -4,7 +4,9 @@ We use different channels of communication for different purposes. Whichever one ## Discourse (recommended) -We use [Discourse](https://discourse.jupyter.org) for online discussions and support questions. You can ask questions here if you are a first-time contributor to the jupyterhub project. Everyone in the Jupyter community is welcome to bring ideas and questions there. +We use [Discourse](https://discourse.jupyter.org) for online discussions and support questions. +You can ask questions here if you are a first-time contributor to the JupyterHub project. +Everyone in the Jupyter community is welcome to bring ideas and questions there. We recommend that you first use our Discourse as all past and current discussions on it are archived and searchable. Thus, all discussions remain useful and accessible to the whole community. From 488331e0331587a81754ca7885acd338dda2b56d Mon Sep 17 00:00:00 2001 From: Emmanuella Orioma Date: Tue, 25 Oct 2022 23:26:16 +0300 Subject: [PATCH 164/214] Update docs.rst I corrected the spelling error in line 21 --- docs/source/contributing/docs.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/docs.rst b/docs/source/contributing/docs.rst index 398b7324..ca3000e7 100644 --- a/docs/source/contributing/docs.rst +++ b/docs/source/contributing/docs.rst @@ -18,7 +18,7 @@ stored under the ``docs/source`` directory) and converts it into various formats for people to read. To make sure the documentation you write or change renders correctly, it is good practice to test it locally. -#. Make sure you have successfuly completed :ref:`contributing/setup`. +#. Make sure you have successfully completed :ref:`contributing/setup`. #. Install the packages required to build the docs. From d215248d8a954326bddc09bd767a782e654c8c98 Mon Sep 17 00:00:00 2001 From: Kalu Chibuikem Victor <67438668+KaluBuikem@users.noreply.github.com> Date: Wed, 26 Oct 2022 01:50:28 +0100 Subject: [PATCH 165/214] Updated gallery-jhub-deployments.md I embedded links into the text and updated minor text errors --- docs/source/gallery-jhub-deployments.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/docs/source/gallery-jhub-deployments.md b/docs/source/gallery-jhub-deployments.md index 992816f4..6d98f214 100644 --- a/docs/source/gallery-jhub-deployments.md +++ b/docs/source/gallery-jhub-deployments.md @@ -97,7 +97,7 @@ easy to do with RStudio too. ### University of Illinois -- https://datascience.business.illinois.edu (currently down; checked 04/26/19) +- https://datascience.business.illinois.edu (currently down; checked 10/26/22) ### IllustrisTNG Simulation Project @@ -126,7 +126,7 @@ easy to do with RStudio too. ### Penn State University -- [Press release](https://news.psu.edu/story/523093/2018/05/24/new-open-source-web-apps-available-students-and-faculty): "New open-source web apps available for students and faculty" (but Hub is currently down; checked 04/26/19) +- [Press release](https://news.psu.edu/story/523093/2018/05/24/new-open-source-web-apps-available-students-and-faculty): "New open-source web apps available for students and faculty" ### University of Rochester CIRC @@ -156,13 +156,13 @@ easy to do with RStudio too. ### Elucidata - What's new in Jupyter Notebooks @[Elucidata](https://elucidata.io/): - - Using Jupyter Notebooks with Jupyterhub on GCP, managed by GKE - https://medium.com/elucidata/why-you-should-be-using-a-jupyter-notebook-8385a4ccd93d + - [Using Jupyter Notebooks with Jupyterhub on GCP, managed by GKE](https://medium.com/elucidata/why-you-should-be-using-a-jupyter-notebook-8385a4ccd93d) ## Service Providers ### AWS -- [running-jupyter-notebook-and-jupyterhub-on-amazon-emr](https://aws.amazon.com/blogs/big-data/running-jupyter-notebook-and-jupyterhub-on-amazon-emr/) +- [Run Jupyter Notebook and JupyterHub on Amazon EMR](https://aws.amazon.com/blogs/big-data/running-jupyter-notebook-and-jupyterhub-on-amazon-emr/) ### Google Cloud Platform @@ -175,12 +175,12 @@ easy to do with RStudio too. ### Microsoft Azure -- https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-data-science-linux-dsvm-intro +- [Azure Data Science Virtual Machine release notes](https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-data-science-linux-dsvm-intro) ### Rackspace Carina - https://getcarina.com/blog/learning-how-to-whale/ -- http://carolynvanslyck.com/talk/carina/jupyterhub/#/ +- http://carolynvanslyck.com/talk/carina/jupyterhub/#/ (but carolynvanslyck is currently down; checked 10/26/22) ### Hadoop @@ -189,13 +189,14 @@ easy to do with RStudio too. ## Miscellaneous - https://medium.com/@ybarraud/setting-up-jupyterhub-with-sudospawner-and-anaconda-844628c0dbee#.rm3yt87e1 -- https://groups.google.com/forum/#!topic/jupyter/nkPSEeMr8c0 Mailing list UT deployment -- JupyterHub setup on Centos https://gist.github.com/johnrc/604971f7d41ebf12370bf5729bf3e0a4 -- Deploy JupyterHub to Docker Swarm https://jupyterhub.surge.sh/#/welcome +- [Mailing list UT deployment](https://groups.google.com/forum/#!topic/jupyter/nkPSEeMr8c0) +- [JupyterHub setup on Centos](https://gist.github.com/johnrc/604971f7d41ebf12370bf5729bf3e0a4) +- [Deploy JupyterHub to Docker Swarm](https://jupyterhub.surge.sh/#/welcome) - http://www.laketide.com/building-your-lab-part-3/ - http://estrellita.hatenablog.com/entry/2015/07/31/083202 - http://www.walkingrandomly.com/?p=5734 - https://wrdrd.com/docs/consulting/education-technology - https://bitbucket.org/jackhale/fenics-jupyter - [LinuxCluster blog](https://linuxcluster.wordpress.com/category/application/jupyterhub/) -- [Network Technology](https://arnesund.com/tag/jupyterhub/) [Spark Cluster on OpenStack with Multi-User Jupyter Notebook](https://arnesund.com/2015/09/21/spark-cluster-on-openstack-with-multi-user-jupyter-notebook/) +- [Network Technology](https://arnesund.com/tag/jupyterhub/) +- [Spark Cluster on OpenStack with Multi-User Jupyter Notebook](https://arnesund.com/2015/09/21/spark-cluster-on-openstack-with-multi-user-jupyter-notebook/) From 22d2bbe1aeb938d1c2b315dc0a244bffb70dd15d Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 26 Oct 2022 00:51:26 +0000 Subject: [PATCH 166/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/gallery-jhub-deployments.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/gallery-jhub-deployments.md b/docs/source/gallery-jhub-deployments.md index 6d98f214..89780e65 100644 --- a/docs/source/gallery-jhub-deployments.md +++ b/docs/source/gallery-jhub-deployments.md @@ -198,5 +198,5 @@ easy to do with RStudio too. - https://wrdrd.com/docs/consulting/education-technology - https://bitbucket.org/jackhale/fenics-jupyter - [LinuxCluster blog](https://linuxcluster.wordpress.com/category/application/jupyterhub/) -- [Network Technology](https://arnesund.com/tag/jupyterhub/) +- [Network Technology](https://arnesund.com/tag/jupyterhub/) - [Spark Cluster on OpenStack with Multi-User Jupyter Notebook](https://arnesund.com/2015/09/21/spark-cluster-on-openstack-with-multi-user-jupyter-notebook/) From 76f7ff472130b2c1e4522e06e306f0ac3acf3f90 Mon Sep 17 00:00:00 2001 From: Lili Yao <103026397+liliyao2022@users.noreply.github.com> Date: Wed, 26 Oct 2022 15:44:52 +1100 Subject: [PATCH 167/214] Update README.md Corrected some spelling errors. --- examples/bootstrap-script/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/examples/bootstrap-script/README.md b/examples/bootstrap-script/README.md index 9fa85b9f..50e15993 100644 --- a/examples/bootstrap-script/README.md +++ b/examples/bootstrap-script/README.md @@ -9,7 +9,7 @@ _Providing writeable storage for LDAP users_ Your Jupyterhub is configured to use the LDAPAuthenticator and DockerSpawer. -- The user has no file directory on the host since your are using LDAP. +- The user has no file directory on the host since you are using LDAP. - When a user has no directory and DockerSpawner wants to mount a volume, the spawner will use docker to create a directory. Since the docker daemon is running as root, the generated directory for the volume @@ -23,7 +23,7 @@ Another use would be to copy initial content, such as tutorial files or referenc material, into the user's space when a notebook server is newly spawned. You can define your own bootstrap process by implementing a `pre_spawn_hook` on any spawner. -The Spawner itself is passed as parameter to your hook and you can easily get the contextual information out of the spawning process. +The Spawner itself is passed as a parameter to your hook and you can easily get the contextual information out of the spawning process. Similarly, there may be cases where you would like to clean up after a spawner stops. You may implement a `post_stop_hook` that is always executed after the spawner stops. From cb5cc8c1b468ccab469db4107b1772460e86121e Mon Sep 17 00:00:00 2001 From: Lili Yao <103026397+liliyao2022@users.noreply.github.com> Date: Wed, 26 Oct 2022 17:58:26 +1100 Subject: [PATCH 168/214] Update Groups.jsx "useState" was not used in this file. --- jsx/src/components/Groups/Groups.jsx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jsx/src/components/Groups/Groups.jsx b/jsx/src/components/Groups/Groups.jsx index 42e987dd..ef3b4dae 100644 --- a/jsx/src/components/Groups/Groups.jsx +++ b/jsx/src/components/Groups/Groups.jsx @@ -1,4 +1,4 @@ -import React, { useEffect, useState } from "react"; +import React, { useEffect } from "react"; import { useSelector, useDispatch } from "react-redux"; import PropTypes from "prop-types"; From 2019bd27971baa1b81bb9a148d8e97dedf48f2de Mon Sep 17 00:00:00 2001 From: Esther Christopher Date: Wed, 26 Oct 2022 11:24:53 +0100 Subject: [PATCH 169/214] Update upgrading.rst Hi @minrk, I went through the reference files you sent and made the necessary adjustments. I think everything should be fine now. I also added a new correction. Please review and revert. --- docs/source/admin/upgrading.rst | 21 +++++---------------- 1 file changed, 5 insertions(+), 16 deletions(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index 0007485f..0de8f16d 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -7,26 +7,21 @@ document describes how to do these upgrades. If you use :ref:`a JupyterHub distribution `, you should consult the distribution's documentation on how to upgrade. This -patch-2 document is applicable if you have set up your own JupyterHub without using a distribution. -This documentation is lengthy because it is quite detailed! Most likely, upgrading +This documentation is lengthy because it is quite detailed. Most likely, upgrading JupyterHub is painless, quick and with minimal user interruption. -The steps are discussed in detail, so if you get stuck at any step you can always refer to this guide. Most likely, -upgrading JupyterHub is painless, quick and with minimal user interruption. -main +The steps are discussed in detail, so if you get stuck at any step you can always refer to this guide. Read the Changelog ================== -patch-2 The `changelog <../changelog.html>`_ contains information on what has changed with the new JupyterHub release, and any deprecation warnings. Read these notes to familiarize yourself with the coming changes. There might be new releases for the authenticators & spawners you use, so -main read the changelogs for those too! Notify your users @@ -37,11 +32,9 @@ is managed by JupyterHub, your users will see service disruption during the upgrade process. You should notify them, and pick a time to do the upgrade where they will be least disrupted. -patch-2 If you use a different proxy, or run ``configurable-http-proxy`` independent of JupyterHub, your users will be able to continue using notebook servers they had already launched, but will not be able to launch new servers or sign in. -main Backup database & config @@ -49,23 +42,19 @@ Backup database & config Before doing an upgrade, it is critical to back up: -#. Your JupyterHub database (SQLite by default, or MySQL / Postgres -patch-2 - if you used those). If you use SQLite (the default), you +#. Your JupyterHub database (SQLite by default, or MySQL / Postgres if you used those). +If you use SQLite (the default), you should backup the ``jupyterhub.sqlite`` file. #. Your ``jupyterhub_config.py`` file. #. Your users' home directories. This is unlikely to be affected directly by a JupyterHub upgrade, but we recommend a backup since user data is critical. -main Shut down JupyterHub ==================== -patch-2 -Shutdown the JupyterHub process. This would vary depending on how you +Shut down the JupyterHub process. This would vary depending on how you have set up JupyterHub to run. It is most likely using a process -main supervisor of some sort (``systemd`` or ``supervisord`` or even ``docker``). Use the supervisor-specific command to stop the JupyterHub process. From 8ffa07e82d5a767ae1ae2f5d5a0726d7e077ae9c Mon Sep 17 00:00:00 2001 From: lumenCodes Date: Mon, 24 Oct 2022 17:47:49 +0100 Subject: [PATCH 170/214] Update to the services.md --- docs/source/reference/services.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/reference/services.md b/docs/source/reference/services.md index aaa417f7..34da6a7f 100644 --- a/docs/source/reference/services.md +++ b/docs/source/reference/services.md @@ -146,8 +146,8 @@ See the GitHub repo for additional information about the [jupyterhub_idle_culler ## Externally-Managed Services -You may prefer to use your own service management tools, such as Docker or -systemd, to manage a JupyterHub Service. These **Externally-Managed +You may prefer to use your service management tools, such as `Docker` or +`systemd`, to manage a JupyterHub Service. These **Externally-Managed Services**, unlike Hub-Managed Services, are not subprocesses of the Hub. You must tell JupyterHub which API token the Externally-Managed Service is using to perform its API requests. Each Externally-Managed Service will need a @@ -186,7 +186,7 @@ information to the Service via the environment variables described above. A flexible Service, whether managed by the Hub or not, can make use of these same environment variables. -When you run a service that has a url, it will be accessible under a +When you run a service that has a `url`, it will be accessible under a `/services/` prefix, such as `https://myhub.horse/services/my-service/`. For your service to route proxied requests properly, it must take `JUPYTERHUB_SERVICE_PREFIX` into account when routing requests. For example, a @@ -257,7 +257,7 @@ which makes a request of the Hub, and returns: You are then free to use the returned user information to take appropriate action. -HubAuth also caches the Hub's response for a number of seconds, +HubAuth also caches the Hub's response for some seconds, configurable by the `cookie_cache_max_age` setting (default: five minutes). If your service would like to make further requests _on behalf of users_, @@ -268,7 +268,7 @@ you can access the token authenticating the current request with {meth}`.HubAuth :::{versionchanged} 2.2 {meth}`.HubAuth.get_token` adds support for retrieving -tokens stored in tornado cookies after completion of OAuth. +tokens stored in tornado cookies after the completion of OAuth. Previously, it only retrieved tokens from URL parameters or the Authorization header. Passing `get_token(handler, in_cookie=False)` preserves this behavior. ::: From 23506a25e552e4547860e32767b2013835a35637 Mon Sep 17 00:00:00 2001 From: lumenCodes Date: Tue, 25 Oct 2022 18:49:42 +0100 Subject: [PATCH 171/214] applied PR change suggestions --- docs/source/reference/services.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/reference/services.md b/docs/source/reference/services.md index 34da6a7f..2880fa19 100644 --- a/docs/source/reference/services.md +++ b/docs/source/reference/services.md @@ -146,8 +146,8 @@ See the GitHub repo for additional information about the [jupyterhub_idle_culler ## Externally-Managed Services -You may prefer to use your service management tools, such as `Docker` or -`systemd`, to manage a JupyterHub Service. These **Externally-Managed +You may prefer to use your own service management tools, such as Docker or +systemd, to manage a JupyterHub Service. These **Externally-Managed Services**, unlike Hub-Managed Services, are not subprocesses of the Hub. You must tell JupyterHub which API token the Externally-Managed Service is using to perform its API requests. Each Externally-Managed Service will need a @@ -186,7 +186,7 @@ information to the Service via the environment variables described above. A flexible Service, whether managed by the Hub or not, can make use of these same environment variables. -When you run a service that has a `url`, it will be accessible under a +When you run a service that has a URL, it will be accessible under a `/services/` prefix, such as `https://myhub.horse/services/my-service/`. For your service to route proxied requests properly, it must take `JUPYTERHUB_SERVICE_PREFIX` into account when routing requests. For example, a @@ -257,7 +257,7 @@ which makes a request of the Hub, and returns: You are then free to use the returned user information to take appropriate action. -HubAuth also caches the Hub's response for some seconds, +HubAuth also caches the Hub's response for a number of seconds, configurable by the `cookie_cache_max_age` setting (default: five minutes). If your service would like to make further requests _on behalf of users_, From 12de64828cc0f13630a8b3bb1e399c2561938cd5 Mon Sep 17 00:00:00 2001 From: lumenCodes Date: Mon, 24 Oct 2022 17:56:24 +0100 Subject: [PATCH 172/214] update to the proxy file. --- docs/source/reference/proxy.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/source/reference/proxy.md b/docs/source/reference/proxy.md index 6723d88e..d830ecb0 100644 --- a/docs/source/reference/proxy.md +++ b/docs/source/reference/proxy.md @@ -7,9 +7,11 @@ Hub manages by default as a subprocess (it can be run externally, as well, and typically is in production deployments). The upside to CHP, and why we use it by default, is that it's easy to install -and run (if you have nodejs, you are set!). The downsides are that it's a -single process and does not support any persistence of the routing table. So -if the proxy process dies, your whole JupyterHub instance is inaccessible +and run (if you have nodejs, you are set!). The downsides are that +- it's a single process and +- does not support any persistence of the routing table. + +So if the proxy process dies, your whole JupyterHub instance is inaccessible until the Hub notices, restarts the proxy, and restores the routing table. For deployments that want to avoid such a single point of failure, or leverage existing proxy infrastructure in their chosen deployment (such as Kubernetes From 6a4078f977b6efff53a556c6169a992328047b58 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 26 Oct 2022 17:17:19 +0000 Subject: [PATCH 173/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/reference/proxy.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/source/reference/proxy.md b/docs/source/reference/proxy.md index d830ecb0..240993bd 100644 --- a/docs/source/reference/proxy.md +++ b/docs/source/reference/proxy.md @@ -7,9 +7,10 @@ Hub manages by default as a subprocess (it can be run externally, as well, and typically is in production deployments). The upside to CHP, and why we use it by default, is that it's easy to install -and run (if you have nodejs, you are set!). The downsides are that -- it's a single process and -- does not support any persistence of the routing table. +and run (if you have nodejs, you are set!). The downsides are that + +- it's a single process and +- does not support any persistence of the routing table. So if the proxy process dies, your whole JupyterHub instance is inaccessible until the Hub notices, restarts the proxy, and restores the routing table. For From cf2ec324b78bc2b4e8f8148b8cdf020dcaf195ca Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 26 Oct 2022 17:23:18 +0000 Subject: [PATCH 174/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/reference/rest.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/source/reference/rest.md b/docs/source/reference/rest.md index 1df93524..526727c0 100644 --- a/docs/source/reference/rest.md +++ b/docs/source/reference/rest.md @@ -4,7 +4,6 @@ This section will give you information on: - - What you can do with the API - How to create an API token - Assigning permissions to a token @@ -16,7 +15,7 @@ This section will give you information on: Before we discuss about JupyterHub's REST API, you can learn about [REST APIs here](https://en.wikipedia.org/wiki/Representational_state_transfer). A REST API provides a standard way for users to get and send information to the -Hub. +Hub. ## What you can do with the API @@ -197,7 +196,6 @@ The same API token can also authorize access to the [Jupyter Notebook REST API][ provided by notebook servers managed by JupyterHub if it has the necessary `access:users:servers` scope. - ## Paginating API requests ```{versionadded} 2.0 @@ -323,7 +321,6 @@ or kubernetes pods. You can see the full [JupyterHub REST API][] for more details. - [openapi initiative]: https://www.openapis.org/ [jupyterhub rest api]: ./rest-api [scopes]: ../rbac/scopes.md From 438b285670f0cd8b2d592c7a73a9f2a6fb932b83 Mon Sep 17 00:00:00 2001 From: lumenCodes Date: Wed, 26 Oct 2022 18:44:51 +0100 Subject: [PATCH 175/214] I effected PR requested changes --- docs/source/reference/rest.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/source/reference/rest.md b/docs/source/reference/rest.md index 526727c0..36db7271 100644 --- a/docs/source/reference/rest.md +++ b/docs/source/reference/rest.md @@ -19,7 +19,7 @@ Hub. ## What you can do with the API -Using the [JupyterHub REST API](https://jupyterhub.readthedocs.io/en/stable/reference/rest-api.html#/default/delete_groups__name_), you can perform actions on the Hub, +Using the [JupyterHub REST API][], you can perform actions on the Hub, such as: - Checking which users are active @@ -140,9 +140,9 @@ c.JupyterHub.load_roles = [ ``` The token will have the permissions listed in the role -(see [scopes](https://jupyterhub.readthedocs.io/en/stable/rbac/scopes.html#scopes-in-jupyterhub) for a list of available permissions), +(see [scopes][] for a list of available permissions), but there will no longer be a user account created to house it. -The main noticeable difference between a configuration and a service is that there will be no notebook server associated with the account +The main noticeable difference between a user and a service is that there will be no notebook server associated with the account and the service will not show up in the various user list pages and APIs. ## Make an API request @@ -153,9 +153,8 @@ Authorization header. ### Use requests Using the popular Python [requests](https://docs.python-requests.org) -library, (here's example code to make an API request for the users of a JupyterHub -deployment) an API GET request is made, and the request sends an API token for -authorization. The response contains information about the users: +library, an API GET request is made, and the request sends an API token for +authorization. The response contains information about the users, here's example code to make an API request for the users of a JupyterHub deployment ```python import requests @@ -196,6 +195,8 @@ The same API token can also authorize access to the [Jupyter Notebook REST API][ provided by notebook servers managed by JupyterHub if it has the necessary `access:users:servers` scope. +(api-pagination)= + ## Paginating API requests ```{versionadded} 2.0 From f0957ad247b1ca925522ec932a00f45ae8b522a2 Mon Sep 17 00:00:00 2001 From: Esther Christopher Date: Sat, 22 Oct 2022 00:07:12 +0100 Subject: [PATCH 176/214] Resolve conflicts in event doc --- docs/source/events/index.rst | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/docs/source/events/index.rst b/docs/source/events/index.rst index 232e01ad..bbbf680f 100644 --- a/docs/source/events/index.rst +++ b/docs/source/events/index.rst @@ -1,6 +1,5 @@ -patch-1 -Event Logging and Telemetry -main +Event logging and telemetry +=========================== JupyterHub can be configured to record structured events from a running server using Jupyter's `Telemetry System`_. The types of events that JupyterHub emits are defined by `JSON schemas`_ listed at the bottom of this page_. @@ -16,10 +15,8 @@ Event logging is handled by its ``Eventlog`` object. This leverages Python's sta To begin recording events, you'll need to set two configurations: -patch-1 1. ``handlers``: tells the EventLog *where* to route your events. This trait is a list of Python logging handlers that route events to the event log file. 2. ``allows_schemas``: tells the EventLog *which* events should be recorded. No events are emitted by default; all recorded events must be listed here. - main Here's a basic example: From c19b3b540a72b912260ec48d8270016461d3ecff Mon Sep 17 00:00:00 2001 From: Esther Christopher Date: Fri, 28 Oct 2022 13:50:02 +0100 Subject: [PATCH 177/214] Update upgrading.rst Hopefully, it's all sorted now and gets merged. Sorry for the back and forth. --- docs/source/admin/upgrading.rst | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index 0de8f16d..6e3ddf4c 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -18,10 +18,10 @@ The steps are discussed in detail, so if you get stuck at any step you can alway Read the Changelog ================== -The `changelog <../changelog.html>`_ contains information on what has -changed with the new JupyterHub release, and any deprecation warnings. +The `changelog <../changelog.md>`_ contains information on what has +changed with the new JupyterHub release and any deprecation warnings. Read these notes to familiarize yourself with the coming changes. There -might be new releases for the authenticators & spawners you use, so +might be new releases of the authenticators & spawners you use, so read the changelogs for those too! Notify your users @@ -32,7 +32,7 @@ is managed by JupyterHub, your users will see service disruption during the upgrade process. You should notify them, and pick a time to do the upgrade where they will be least disrupted. -If you use a different proxy, or run ``configurable-http-proxy`` +If you use a different proxy or run ``configurable-http-proxy`` independent of JupyterHub, your users will be able to continue using notebook servers they had already launched, but will not be able to launch new servers or sign in. @@ -43,8 +43,7 @@ Backup database & config Before doing an upgrade, it is critical to back up: #. Your JupyterHub database (SQLite by default, or MySQL / Postgres if you used those). -If you use SQLite (the default), you - should backup the ``jupyterhub.sqlite`` file. +If you use SQLite (the default), you should backup the ``jupyterhub.sqlite`` file. #. Your ``jupyterhub_config.py`` file. #. Your users' home directories. This is unlikely to be affected directly by a JupyterHub upgrade, but we recommend a backup since user data is critical. From b83a6250baa794e64cab4d7365a1888f1dad944d Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Mon, 31 Oct 2022 19:36:37 +0100 Subject: [PATCH 178/214] ci: use non-deprecated codecov uploader --- .github/workflows/test.yml | 4 +--- dev-requirements.txt | 1 - 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index e9e25d28..52b2a736 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -234,9 +234,7 @@ jobs: run: | pytest --maxfail=2 --cov=jupyterhub jupyterhub/tests - - name: Submit codecov report - run: | - codecov + - uses: codecov/codecov-action@v3 docker-build: runs-on: ubuntu-20.04 diff --git a/dev-requirements.txt b/dev-requirements.txt index e97dfb6c..2f9cb22d 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -3,7 +3,6 @@ # seems to be a pip bug attrs>=17.4.0 beautifulsoup4 -codecov coverage cryptography html5lib # needed for beautifulsoup From 54a5d2c1522691d947609d221941c24f136286d5 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue, 1 Nov 2022 00:03:21 +0000 Subject: [PATCH 179/214] [pre-commit.ci] pre-commit autoupdate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/asottile/pyupgrade: v3.1.0 → v3.2.0](https://github.com/asottile/pyupgrade/compare/v3.1.0...v3.2.0) - [github.com/pre-commit/mirrors-prettier: v3.0.0-alpha.3 → v3.0.0-alpha.4](https://github.com/pre-commit/mirrors-prettier/compare/v3.0.0-alpha.3...v3.0.0-alpha.4) --- .pre-commit-config.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 582e2610..6401614b 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -11,7 +11,7 @@ repos: # Autoformat: Python code, syntax patterns are modernized - repo: https://github.com/asottile/pyupgrade - rev: v3.1.0 + rev: v3.2.0 hooks: - id: pyupgrade args: @@ -31,7 +31,7 @@ repos: # Autoformat: markdown, yaml, javascript (see the file .prettierignore) - repo: https://github.com/pre-commit/mirrors-prettier - rev: v3.0.0-alpha.3 + rev: v3.0.0-alpha.4 hooks: - id: prettier From 59f14ad7c021c9fc85b2b9d64056a8b5312dfecb Mon Sep 17 00:00:00 2001 From: Joel-Ando Date: Tue, 11 Oct 2022 22:30:49 +0100 Subject: [PATCH 180/214] Update quickstart-docker.rst --- docs/source/quickstart-docker.rst | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/docs/source/quickstart-docker.rst b/docs/source/quickstart-docker.rst index 6ecacf3b..2876d6e0 100644 --- a/docs/source/quickstart-docker.rst +++ b/docs/source/quickstart-docker.rst @@ -1,10 +1,16 @@ Install JupyterHub with Docker ============================== + The JupyterHub `docker image `_ is the fastest way to set up Jupyterhub in your local development environment. -The ``JupyterHub`` docker image runs the Hub service only. It does not provide other Jupyter components such as Notebook installation, which are needed by the single-user servers. -To run the single-user servers, which may be on the same system as the Hub or not, Jupyter Notebook or Jupyter Lab must be installed. +.. note:: + This ``jupyterhub/jupyterhub`` docker image is only an image for running + the Hub service itself. It does not provide the other Jupyter components, + such as Notebook installation, which are needed by the single-user servers. + To run the single-user servers, which may be on the same system as the Hub or + not, `JupyterLab `_ or Jupyter Notebook must be installed. + .. important:: We strongly recommend that you follow the `Zero to JupyterHub`_ tutorial to @@ -24,6 +30,7 @@ To pull the latest JupyterHub image and start the `jupyterhub` container, run th docker run -d -p 8000:8000 --name jupyterhub jupyterhub/jupyterhub jupyterhub + This command exposes the Jupyter container on port:8000. Navigate to `http://localhost:8000` in a web browser to access the JupyterHub console. You can stop and resume the container by running `docker stop` and `docker start` respectively. From 2ed84b0de17e11b99ee4675df638b69b283ccb54 Mon Sep 17 00:00:00 2001 From: Min RK Date: Fri, 11 Nov 2022 09:23:23 +0100 Subject: [PATCH 181/214] typo in access:servers scope it's not access:users:servers --- docs/source/reference/rest.md | 2 +- docs/source/reference/services.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/reference/rest.md b/docs/source/reference/rest.md index 36db7271..3ac0f423 100644 --- a/docs/source/reference/rest.md +++ b/docs/source/reference/rest.md @@ -193,7 +193,7 @@ r.json() The same API token can also authorize access to the [Jupyter Notebook REST API][] -provided by notebook servers managed by JupyterHub if it has the necessary `access:users:servers` scope. +provided by notebook servers managed by JupyterHub if it has the necessary `access:servers` scope. (api-pagination)= diff --git a/docs/source/reference/services.md b/docs/source/reference/services.md index 2880fa19..360ca1e3 100644 --- a/docs/source/reference/services.md +++ b/docs/source/reference/services.md @@ -249,7 +249,7 @@ which makes a request of the Hub, and returns: "name": "username", "groups": ["list", "of", "groups"], "scopes": [ - "access:users:servers!server=username/", + "access:servers!server=username/", ], } ``` From 32cddfbdfe87d50c7829ae6c41c838269af72ebb Mon Sep 17 00:00:00 2001 From: Arafat Abdussalam Date: Mon, 14 Nov 2022 10:30:56 +0100 Subject: [PATCH 182/214] Update docs.rst --- docs/source/contributing/docs.rst | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/source/contributing/docs.rst b/docs/source/contributing/docs.rst index ca3000e7..b988e157 100644 --- a/docs/source/contributing/docs.rst +++ b/docs/source/contributing/docs.rst @@ -5,7 +5,7 @@ Contributing Documentation ========================== Documentation is often more important than code. This page helps -you get set up on how to contribute documentation to JupyterHub. +you get set up on how to contribute to JupyterHub's documentation. Building documentation locally ============================== @@ -44,9 +44,11 @@ change renders correctly, it is good practice to test it locally. .. tip:: - On Windows, you can open a file from the terminal with ``start ``. - On macOS, you can do the same with ``open ``. - On Linux, you can do the same with ``xdg-open ``. + **On Windows**, you can open a file from the terminal with ``start ``. + + **On macOS**, you can do the same with ``open ``. + + **On Linux**, you can do the same with ``xdg-open ``. .. _contributing/docs/conventions: From e6e890b46ca3d4e0886430c614a2562857351919 Mon Sep 17 00:00:00 2001 From: Arafat Abdussalam Date: Mon, 14 Nov 2022 11:22:12 +0100 Subject: [PATCH 183/214] Update security-basics.rst --- docs/source/getting-started/security-basics.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/getting-started/security-basics.rst b/docs/source/getting-started/security-basics.rst index c3649ee3..0e403132 100644 --- a/docs/source/getting-started/security-basics.rst +++ b/docs/source/getting-started/security-basics.rst @@ -13,7 +13,7 @@ Three (3) configuration settings are the main aspects of security configuration: 3. Proxy :ref:`authentication token ` (used for the Hub and other services to authenticate to the Proxy) -The Hub hashes all secrets (e.g., auth tokens) before storing them in its +The Hub hashes all secrets (e.g. auth tokens) before storing them in its database. A loss of control over read-access to the database should have minimal impact on your deployment. If your database has been compromised, it is still a good idea to revoke existing tokens. @@ -118,7 +118,7 @@ This environment variable needs to be visible to the Hub and Proxy. Default if token is not set ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -If you don't set the Proxy authentication token, the Hub will generate a random +If you do not set the Proxy authentication token, the Hub will generate a random key itself. This means that any time you restart the Hub, you **must also restart the Proxy**. If the proxy is a subprocess of the Hub, this should happen automatically (this is the default configuration). From c993163f0be3a2927b752f2667a2b8af36d2287a Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue, 15 Nov 2022 00:12:51 +0000 Subject: [PATCH 184/214] [pre-commit.ci] pre-commit autoupdate MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit updates: - [github.com/asottile/pyupgrade: v3.2.0 → v3.2.2](https://github.com/asottile/pyupgrade/compare/v3.2.0...v3.2.2) --- .pre-commit-config.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 6401614b..0681ae84 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -11,7 +11,7 @@ repos: # Autoformat: Python code, syntax patterns are modernized - repo: https://github.com/asottile/pyupgrade - rev: v3.2.0 + rev: v3.2.2 hooks: - id: pyupgrade args: From 24dfa5d228aba755f44d8c8cb62f3fc29ab2b0a2 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Wed, 16 Nov 2022 08:05:46 +0100 Subject: [PATCH 185/214] ci: flake8, cleanup unused/redundant config - E### warnings were already ignored by E - Sorting of import warnings won't matter as we have black and isort - D400: First line should end with a period This is not flake8 configuration, but related to `pydocstyle`, which isn't used. --- .flake8 | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/.flake8 b/.flake8 index 58538de2..a3bdb0cb 100644 --- a/.flake8 +++ b/.flake8 @@ -7,10 +7,7 @@ # F403: import * # F811: redefinition of unused `name` from line `N` # F841: local variable assigned but never used -# E402: module level import not at top of file -# I100: Import statements are in the wrong order -# I101: Imported names are in the wrong order. Should be -ignore = E, C, W, F401, F403, F811, F841, E402, I100, I101, D400 +ignore = E, C, W, F401, F403, F811, F841 builtins = c, get_config exclude = .cache, From b77e9cbf0864c10bac468abd28eeebea2becf10c Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Wed, 16 Nov 2022 08:54:23 +0100 Subject: [PATCH 186/214] ci: flake8, add back general ignore of pydocstyle warnings --- .flake8 | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.flake8 b/.flake8 index a3bdb0cb..3c729960 100644 --- a/.flake8 +++ b/.flake8 @@ -3,11 +3,12 @@ # E: style errors # W: style warnings # C: complexity +# D: docstring warnings (unused pydocstyle extension) # F401: module imported but unused # F403: import * # F811: redefinition of unused `name` from line `N` # F841: local variable assigned but never used -ignore = E, C, W, F401, F403, F811, F841 +ignore = E, C, W, D, F401, F403, F811, F841 builtins = c, get_config exclude = .cache, From eaa096152a175bc2cda34c488e13ad241939d91e Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Wed, 16 Nov 2022 10:45:51 +0100 Subject: [PATCH 187/214] flake8: check F403 about import * --- .flake8 | 3 +-- jupyterhub/apihandlers/__init__.py | 2 +- jupyterhub/handlers/__init__.py | 4 ++-- 3 files changed, 4 insertions(+), 5 deletions(-) diff --git a/.flake8 b/.flake8 index 3c729960..a016c4dc 100644 --- a/.flake8 +++ b/.flake8 @@ -5,10 +5,9 @@ # C: complexity # D: docstring warnings (unused pydocstyle extension) # F401: module imported but unused -# F403: import * # F811: redefinition of unused `name` from line `N` # F841: local variable assigned but never used -ignore = E, C, W, D, F401, F403, F811, F841 +ignore = E, C, W, D, F401, F811, F841 builtins = c, get_config exclude = .cache, diff --git a/jupyterhub/apihandlers/__init__.py b/jupyterhub/apihandlers/__init__.py index 39733829..f72887eb 100644 --- a/jupyterhub/apihandlers/__init__.py +++ b/jupyterhub/apihandlers/__init__.py @@ -1,5 +1,5 @@ from . import auth, groups, hub, proxy, services, users -from .base import * +from .base import * # noqa default_handlers = [] for mod in (auth, hub, proxy, users, groups, services): diff --git a/jupyterhub/handlers/__init__.py b/jupyterhub/handlers/__init__.py index e39e5730..79f3ea87 100644 --- a/jupyterhub/handlers/__init__.py +++ b/jupyterhub/handlers/__init__.py @@ -1,6 +1,6 @@ from . import base, login, metrics, pages -from .base import * -from .login import * +from .base import * # noqa +from .login import * # noqa default_handlers = [] for mod in (base, pages, login, metrics): From 957fd9cc20adf3010636d620a5228d5ba7f67323 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Wed, 16 Nov 2022 10:48:01 +0100 Subject: [PATCH 188/214] flake8: check F811 about redefinition of unused name --- .flake8 | 3 +-- examples/service-fastapi/app/service.py | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/.flake8 b/.flake8 index a016c4dc..77095d0c 100644 --- a/.flake8 +++ b/.flake8 @@ -5,9 +5,8 @@ # C: complexity # D: docstring warnings (unused pydocstyle extension) # F401: module imported but unused -# F811: redefinition of unused `name` from line `N` # F841: local variable assigned but never used -ignore = E, C, W, D, F401, F811, F841 +ignore = E, C, W, D, F401, F841 builtins = c, get_config exclude = .cache, diff --git a/examples/service-fastapi/app/service.py b/examples/service-fastapi/app/service.py index 81d7b87d..8aaeef94 100644 --- a/examples/service-fastapi/app/service.py +++ b/examples/service-fastapi/app/service.py @@ -51,7 +51,7 @@ async def me(user: User = Depends(get_current_user)): @router.get("/debug") -async def index(request: Request, user: User = Depends(get_current_user)): +async def debug(request: Request, user: User = Depends(get_current_user)): """ Authenticated function that returns a few pieces of debug * Environ of the service process From 7a9491c323709d6b9e7f900c3c83eb3c3047e7d1 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Wed, 16 Nov 2022 11:01:07 +0100 Subject: [PATCH 189/214] flake8: check F408 about unused imports, manual fixes only --- .flake8 | 3 +-- docs/source/conf.py | 2 +- examples/service-fastapi/app/__init__.py | 2 +- jupyterhub/__init__.py | 2 +- .../alembic/versions/19c0846f6344_base_revision_for_0_5.py | 1 - jupyterhub/singleuser/__init__.py | 4 ++-- 6 files changed, 6 insertions(+), 8 deletions(-) diff --git a/.flake8 b/.flake8 index 77095d0c..41852dd2 100644 --- a/.flake8 +++ b/.flake8 @@ -4,9 +4,8 @@ # W: style warnings # C: complexity # D: docstring warnings (unused pydocstyle extension) -# F401: module imported but unused # F841: local variable assigned but never used -ignore = E, C, W, D, F401, F841 +ignore = E, C, W, D, F841 builtins = c, get_config exclude = .cache, diff --git a/docs/source/conf.py b/docs/source/conf.py index 63d61624..5905787e 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -250,7 +250,7 @@ if on_rtd: # -- Spell checking ------------------------------------------------------- try: - import sphinxcontrib.spelling + import sphinxcontrib.spelling # noqa except ImportError: pass else: diff --git a/examples/service-fastapi/app/__init__.py b/examples/service-fastapi/app/__init__.py index c07c4599..a4576905 100644 --- a/examples/service-fastapi/app/__init__.py +++ b/examples/service-fastapi/app/__init__.py @@ -1 +1 @@ -from .app import app +from .app import app # noqa diff --git a/jupyterhub/__init__.py b/jupyterhub/__init__.py index 948f47a6..6bb39770 100644 --- a/jupyterhub/__init__.py +++ b/jupyterhub/__init__.py @@ -1 +1 @@ -from ._version import __version__, version_info +from ._version import __version__, version_info # noqa diff --git a/jupyterhub/alembic/versions/19c0846f6344_base_revision_for_0_5.py b/jupyterhub/alembic/versions/19c0846f6344_base_revision_for_0_5.py index 9ee5a24a..7734993a 100644 --- a/jupyterhub/alembic/versions/19c0846f6344_base_revision_for_0_5.py +++ b/jupyterhub/alembic/versions/19c0846f6344_base_revision_for_0_5.py @@ -14,7 +14,6 @@ depends_on = None import sqlalchemy as sa from alembic import op - def upgrade(): pass diff --git a/jupyterhub/singleuser/__init__.py b/jupyterhub/singleuser/__init__.py index 60159406..be6c08f8 100644 --- a/jupyterhub/singleuser/__init__.py +++ b/jupyterhub/singleuser/__init__.py @@ -2,8 +2,8 @@ Contains default notebook-app subclass and mixins """ -from .app import SingleUserNotebookApp, main -from .mixins import HubAuthenticatedHandler, make_singleuser_app +from .app import SingleUserNotebookApp, main # noqa +from .mixins import HubAuthenticatedHandler, make_singleuser_app # noqa # backward-compatibility JupyterHubLoginHandler = SingleUserNotebookApp.login_handler_class From 3b5b42e6202d3b9f8714c979e035369c81b82826 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Wed, 16 Nov 2022 14:59:55 +0100 Subject: [PATCH 190/214] flake8: manual F408 fix, avoid indirect import of public_host / public_url --- jupyterhub/tests/mocking.py | 2 +- jupyterhub/tests/test_api.py | 11 +++++++++-- jupyterhub/tests/test_named_servers.py | 4 ++-- 3 files changed, 12 insertions(+), 5 deletions(-) diff --git a/jupyterhub/tests/mocking.py b/jupyterhub/tests/mocking.py index 5066ddde..561f672c 100644 --- a/jupyterhub/tests/mocking.py +++ b/jupyterhub/tests/mocking.py @@ -45,7 +45,7 @@ from ..auth import PAMAuthenticator from ..singleuser import SingleUserNotebookApp from ..spawner import SimpleLocalProcessSpawner from ..utils import random_port, utcnow -from .utils import async_requests, public_host, public_url, ssl_setup +from .utils import async_requests, public_url, ssl_setup def mock_authenticate(username, password, service, encoding): diff --git a/jupyterhub/tests/test_api.py b/jupyterhub/tests/test_api.py index fa96b22d..2bd8c811 100644 --- a/jupyterhub/tests/test_api.py +++ b/jupyterhub/tests/test_api.py @@ -19,8 +19,15 @@ from ..objects import Server from ..utils import url_path_join as ujoin from ..utils import utcnow from .conftest import new_username -from .mocking import public_host, public_url -from .utils import add_user, api_request, async_requests, auth_header, find_user +from .utils import ( + add_user, + api_request, + async_requests, + auth_header, + find_user, + public_host, + public_url, +) # -------------------- # Authentication tests diff --git a/jupyterhub/tests/test_named_servers.py b/jupyterhub/tests/test_named_servers.py index 0c1c9d69..d5f03054 100644 --- a/jupyterhub/tests/test_named_servers.py +++ b/jupyterhub/tests/test_named_servers.py @@ -11,9 +11,9 @@ from tornado.httputil import url_concat from .. import orm from ..utils import url_escape_path, url_path_join -from .mocking import FormSpawner, public_url +from .mocking import FormSpawner from .test_api import TIMESTAMP, add_user, api_request, fill_user, normalize_user -from .utils import async_requests, get_page +from .utils import async_requests, get_page, public_url @pytest.fixture From b9596b2deef71d4784c6b4cac34ced0bbc2b702a Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Wed, 16 Nov 2022 13:05:23 +0100 Subject: [PATCH 191/214] pre-commit: add autoflake --- .pre-commit-config.yaml | 9 +++++++++ pyproject.toml | 11 +++++++++++ 2 files changed, 20 insertions(+) diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 6401614b..41a0a3dd 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -17,6 +17,15 @@ repos: args: - --py36-plus + # Autoformat: Python code + - repo: https://github.com/PyCQA/autoflake + rev: v1.7.7 + hooks: + - id: autoflake + # args ref: https://github.com/PyCQA/autoflake#advanced-usage + args: + - --in-place + # Autoformat: Python code - repo: https://github.com/pycqa/isort rev: 5.10.1 diff --git a/pyproject.toml b/pyproject.toml index b018fc66..553da4e1 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,3 +1,14 @@ +# autoflake is used for autoformatting Python code +# +# ref: https://github.com/PyCQA/autoflake#readme +# +[tool.autoflake] +ignore-init-module-imports = true +remove-all-unused-imports = true +remove-duplicate-keys = true +#remove-unused-variables = true + + [tool.isort] profile = "black" From 5a5e0118b8ee0211eaa829ac7f40e0d0ec62525b Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Wed, 16 Nov 2022 15:38:18 +0100 Subject: [PATCH 192/214] docs: comment about isort, black, tbump in pyproject.toml --- pyproject.toml | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/pyproject.toml b/pyproject.toml index 553da4e1..054c6f1c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -9,9 +9,18 @@ remove-duplicate-keys = true #remove-unused-variables = true +# isort is used for autoformatting Python code +# +# ref: https://pycqa.github.io/isort/ +# [tool.isort] profile = "black" + +# black is used for autoformatting Python code +# +# ref: https://black.readthedocs.io/en/stable/ +# [tool.black] skip-string-normalization = true # target-version should be all supported versions, see @@ -23,6 +32,12 @@ target_version = [ "py310", ] + +# tbump is used to simplify and standardize the release process when updating +# the version, making a git commit and tag, and pushing changes. +# +# ref: https://github.com/your-tools/tbump#readme +# [tool.tbump] # Uncomment this if your project is hosted on GitHub: github_url = "https://github.com/jupyterhub/jupyterhub" From 865d5f764657e207c562a83dba0abcfce1302944 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed, 16 Nov 2022 14:53:11 +0000 Subject: [PATCH 193/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- ci/check_sdist.py | 1 - docs/generate-metrics.py | 1 - .../versions/19c0846f6344_base_revision_for_0_5.py | 2 -- jupyterhub/apihandlers/hub.py | 1 - jupyterhub/auth.py | 2 -- jupyterhub/oauth/provider.py | 1 - jupyterhub/orm.py | 2 -- jupyterhub/proxy.py | 4 ---- jupyterhub/services/auth.py | 1 - jupyterhub/singleuser/mixins.py | 1 - jupyterhub/spawner.py | 2 -- jupyterhub/tests/conftest.py | 2 -- jupyterhub/tests/mocking.py | 1 - jupyterhub/tests/populate_db.py | 1 - jupyterhub/tests/selenium/conftest.py | 1 - jupyterhub/tests/selenium/test_browser.py | 11 ++--------- jupyterhub/tests/test_auth_expiry.py | 1 - jupyterhub/tests/test_eventlog.py | 2 -- jupyterhub/tests/test_orm.py | 1 - jupyterhub/tests/test_services.py | 2 -- jupyterhub/tests/test_singleuser.py | 2 +- jupyterhub/utils.py | 1 - 22 files changed, 3 insertions(+), 40 deletions(-) diff --git a/ci/check_sdist.py b/ci/check_sdist.py index 5fd008f3..b3ff18de 100755 --- a/ci/check_sdist.py +++ b/ci/check_sdist.py @@ -3,7 +3,6 @@ import sys import tarfile -from tarfile import TarFile expected_files = [ "docs/requirements.txt", diff --git a/docs/generate-metrics.py b/docs/generate-metrics.py index 81304bef..05cc6c35 100644 --- a/docs/generate-metrics.py +++ b/docs/generate-metrics.py @@ -1,5 +1,4 @@ import os -from os.path import join from pytablewriter import RstSimpleTableWriter from pytablewriter.style import Style diff --git a/jupyterhub/alembic/versions/19c0846f6344_base_revision_for_0_5.py b/jupyterhub/alembic/versions/19c0846f6344_base_revision_for_0_5.py index 7734993a..c8421da4 100644 --- a/jupyterhub/alembic/versions/19c0846f6344_base_revision_for_0_5.py +++ b/jupyterhub/alembic/versions/19c0846f6344_base_revision_for_0_5.py @@ -11,8 +11,6 @@ down_revision = None branch_labels = None depends_on = None -import sqlalchemy as sa -from alembic import op def upgrade(): pass diff --git a/jupyterhub/apihandlers/hub.py b/jupyterhub/apihandlers/hub.py index 47a4a6e5..4474c821 100644 --- a/jupyterhub/apihandlers/hub.py +++ b/jupyterhub/apihandlers/hub.py @@ -5,7 +5,6 @@ import json import sys from tornado import web -from tornado.ioloop import IOLoop from .._version import __version__ from ..scopes import needs_scope diff --git a/jupyterhub/auth.py b/jupyterhub/auth.py index e31cd395..2795a781 100644 --- a/jupyterhub/auth.py +++ b/jupyterhub/auth.py @@ -776,8 +776,6 @@ def _deprecated_method(old_name, new_name, version): return deprecated -import types - # deprecate white/blacklist method names for _old_name, _new_name, _version in [ ("check_whitelist", "check_allowed", "1.2"), diff --git a/jupyterhub/oauth/provider.py b/jupyterhub/oauth/provider.py index d3e17c46..bb72e338 100644 --- a/jupyterhub/oauth/provider.py +++ b/jupyterhub/oauth/provider.py @@ -13,7 +13,6 @@ from ..scopes import ( _check_scopes_exist, _resolve_requested_scopes, access_scopes, - expand_scopes, identify_scopes, ) from ..utils import compare_token, hash_token diff --git a/jupyterhub/orm.py b/jupyterhub/orm.py index ea8f54d8..0aa1e1ca 100644 --- a/jupyterhub/orm.py +++ b/jupyterhub/orm.py @@ -35,7 +35,6 @@ from sqlalchemy.orm import ( sessionmaker, ) from sqlalchemy.pool import StaticPool -from sqlalchemy.sql.expression import bindparam from sqlalchemy.types import LargeBinary, Text, TypeDecorator from tornado.log import app_log @@ -996,7 +995,6 @@ def check_db_revision(engine): ).first()[0] if alembic_revision == head: app_log.debug("database schema version found: %s", alembic_revision) - pass else: raise DatabaseSchemaMismatch( "Found database schema version {found} != {head}. " diff --git a/jupyterhub/proxy.py b/jupyterhub/proxy.py index dbe0fcdb..87b47a6c 100644 --- a/jupyterhub/proxy.py +++ b/jupyterhub/proxy.py @@ -250,14 +250,12 @@ class Proxy(LoggingConfigurable): The proxy implementation should also have a way to associate the fact that a route came from JupyterHub. """ - pass async def delete_route(self, routespec): """Delete a route with a given routespec if it exists. **Subclasses must define this method** """ - pass async def get_all_routes(self): """Fetch and return all the routes associated by JupyterHub from the @@ -274,7 +272,6 @@ class Proxy(LoggingConfigurable): 'data': the attached data dict for this route (as specified in add_route) } """ - pass async def get_route(self, routespec): """Return the route info for a given routespec. @@ -683,7 +680,6 @@ class ConfigurableHTTPProxy(Proxy): os.remove(self.pid_file) except FileNotFoundError: self.log.debug("PID file %s already removed", self.pid_file) - pass def _get_ssl_options(self): """List of cmd proxy options to use internal SSL""" diff --git a/jupyterhub/services/auth.py b/jupyterhub/services/auth.py index d61239e5..24106ee2 100644 --- a/jupyterhub/services/auth.py +++ b/jupyterhub/services/auth.py @@ -35,7 +35,6 @@ import string import time import uuid import warnings -from functools import partial from http import HTTPStatus from unittest import mock from urllib.parse import urlencode diff --git a/jupyterhub/singleuser/mixins.py b/jupyterhub/singleuser/mixins.py index 953d3d27..dfad82ca 100755 --- a/jupyterhub/singleuser/mixins.py +++ b/jupyterhub/singleuser/mixins.py @@ -14,7 +14,6 @@ import logging import os import random import secrets -import ssl import sys import warnings from datetime import timezone diff --git a/jupyterhub/spawner.py b/jupyterhub/spawner.py index 39837b08..756fefa7 100644 --- a/jupyterhub/spawner.py +++ b/jupyterhub/spawner.py @@ -896,7 +896,6 @@ class Spawner(LoggingConfigurable): Override in subclasses to restore any extra state that is needed to track the single-user server for that user. Subclasses should call super(). """ - pass def get_state(self): """Save state of spawner into database. @@ -1341,7 +1340,6 @@ class Spawner(LoggingConfigurable): Stopping a server does *not* call this method. """ - pass def add_poll_callback(self, callback, *args, **kwargs): """Add a callback to fire when the single-user server stops""" diff --git a/jupyterhub/tests/conftest.py b/jupyterhub/tests/conftest.py index b200bb44..b880017b 100644 --- a/jupyterhub/tests/conftest.py +++ b/jupyterhub/tests/conftest.py @@ -27,7 +27,6 @@ Fixtures to add functionality or spawning behavior # Distributed under the terms of the Modified BSD License. import asyncio import copy -import inspect import os import sys from functools import partial @@ -36,7 +35,6 @@ from subprocess import TimeoutExpired from unittest import mock from pytest import fixture, raises -from tornado import ioloop from tornado.httpclient import HTTPError from tornado.platform.asyncio import AsyncIOMainLoop diff --git a/jupyterhub/tests/mocking.py b/jupyterhub/tests/mocking.py index 561f672c..8aed2bfb 100644 --- a/jupyterhub/tests/mocking.py +++ b/jupyterhub/tests/mocking.py @@ -36,7 +36,6 @@ from unittest import mock from urllib.parse import urlparse from pamela import PAMError -from tornado.ioloop import IOLoop from traitlets import Bool, Dict, default from .. import metrics, orm, roles diff --git a/jupyterhub/tests/populate_db.py b/jupyterhub/tests/populate_db.py index c85e8c57..33b5247b 100644 --- a/jupyterhub/tests/populate_db.py +++ b/jupyterhub/tests/populate_db.py @@ -4,7 +4,6 @@ Run with old versions of jupyterhub to test upgrade/downgrade used in test_db.py """ -import os from datetime import datetime from functools import partial diff --git a/jupyterhub/tests/selenium/conftest.py b/jupyterhub/tests/selenium/conftest.py index 06de2029..a68ce6e3 100644 --- a/jupyterhub/tests/selenium/conftest.py +++ b/jupyterhub/tests/selenium/conftest.py @@ -1,6 +1,5 @@ import pytest from selenium import webdriver -from selenium.webdriver.firefox.options import Options as FirefoxOptions @pytest.fixture() diff --git a/jupyterhub/tests/selenium/test_browser.py b/jupyterhub/tests/selenium/test_browser.py index 4e7ba00c..9c52878c 100644 --- a/jupyterhub/tests/selenium/test_browser.py +++ b/jupyterhub/tests/selenium/test_browser.py @@ -1,21 +1,14 @@ import asyncio -import time from functools import partial import pytest -from selenium import webdriver -from selenium.common.exceptions import NoSuchElementException, TimeoutException -from selenium.webdriver.common.by import By +from selenium.common.exceptions import NoSuchElementException from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.support.ui import WebDriverWait from tornado.escape import url_escape from tornado.httputil import url_concat -from jupyterhub.tests.selenium.locators import ( - HomePageLocators, - LoginPageLocators, - TokenPageLocators, -) +from jupyterhub.tests.selenium.locators import LoginPageLocators from jupyterhub.utils import exponential_backoff from ...utils import url_path_join diff --git a/jupyterhub/tests/test_auth_expiry.py b/jupyterhub/tests/test_auth_expiry.py index 5b750e79..876f85b8 100644 --- a/jupyterhub/tests/test_auth_expiry.py +++ b/jupyterhub/tests/test_auth_expiry.py @@ -7,7 +7,6 @@ authentication can expire in a number of ways: - doesn't need refresh - needs refresh and cannot be refreshed without new login """ -from contextlib import contextmanager from unittest import mock from urllib.parse import parse_qs, urlparse diff --git a/jupyterhub/tests/test_eventlog.py b/jupyterhub/tests/test_eventlog.py index d8dcd285..5b6649cd 100644 --- a/jupyterhub/tests/test_eventlog.py +++ b/jupyterhub/tests/test_eventlog.py @@ -14,8 +14,6 @@ import jsonschema import pytest from traitlets.config import Config -from .mocking import MockHub - # To test new schemas, add them to the `valid_events` # and `invalid_events` dictionary below. diff --git a/jupyterhub/tests/test_orm.py b/jupyterhub/tests/test_orm.py index 7a7f0310..3009a11d 100644 --- a/jupyterhub/tests/test_orm.py +++ b/jupyterhub/tests/test_orm.py @@ -7,7 +7,6 @@ from datetime import datetime, timedelta from unittest import mock import pytest -from tornado import gen from .. import crypto, objects, orm, roles from ..emptyclass import EmptyClass diff --git a/jupyterhub/tests/test_services.py b/jupyterhub/tests/test_services.py index a5d50571..b1e9fe9f 100644 --- a/jupyterhub/tests/test_services.py +++ b/jupyterhub/tests/test_services.py @@ -6,8 +6,6 @@ from subprocess import Popen from async_generator import asynccontextmanager -from .. import orm -from ..roles import roles_to_scopes from ..utils import ( exponential_backoff, maybe_future, diff --git a/jupyterhub/tests/test_singleuser.py b/jupyterhub/tests/test_singleuser.py index c45ade95..e139b41f 100644 --- a/jupyterhub/tests/test_singleuser.py +++ b/jupyterhub/tests/test_singleuser.py @@ -1,7 +1,7 @@ """Tests for jupyterhub.singleuser""" import os import sys -from contextlib import contextmanager, nullcontext +from contextlib import nullcontext from subprocess import CalledProcessError, check_output from unittest import mock from urllib.parse import urlencode, urlparse diff --git a/jupyterhub/utils.py b/jupyterhub/utils.py index 61fc6c7c..291c412b 100644 --- a/jupyterhub/utils.py +++ b/jupyterhub/utils.py @@ -527,7 +527,6 @@ def print_stacks(file=sys.stderr): """ # local imports because these will not be used often, # no need to add them to startup - import asyncio import traceback from .log import coroutine_frames From 5d7383278ff2cdbc1558ce05aa1aaaf3d483a648 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 17 Nov 2022 04:52:44 +0000 Subject: [PATCH 194/214] Bump loader-utils from 2.0.2 to 2.0.4 in /jsx Bumps [loader-utils](https://github.com/webpack/loader-utils) from 2.0.2 to 2.0.4. - [Release notes](https://github.com/webpack/loader-utils/releases) - [Changelog](https://github.com/webpack/loader-utils/blob/v2.0.4/CHANGELOG.md) - [Commits](https://github.com/webpack/loader-utils/compare/v2.0.2...v2.0.4) --- updated-dependencies: - dependency-name: loader-utils dependency-type: indirect ... Signed-off-by: dependabot[bot] --- jsx/yarn.lock | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/jsx/yarn.lock b/jsx/yarn.lock index 1d7afb1e..115940e7 100644 --- a/jsx/yarn.lock +++ b/jsx/yarn.lock @@ -5046,9 +5046,9 @@ loader-runner@^4.2.0: integrity sha512-3R/1M+yS3j5ou80Me59j7F9IMs4PXs3VqRrm0TU3AbKPxlmpoY1TNscJV/oGJXo8qCatFGTfDbY6W6ipGOYXfg== loader-utils@^2.0.0: - version "2.0.2" - resolved "https://registry.yarnpkg.com/loader-utils/-/loader-utils-2.0.2.tgz#d6e3b4fb81870721ae4e0868ab11dd638368c129" - integrity sha512-TM57VeHptv569d/GKh6TAYdzKblwDNiumOdkFnejjD0XwTH87K90w3O7AiJRqdQoXygvi1VQTJTLGhJl7WqA7A== + version "2.0.4" + resolved "https://registry.yarnpkg.com/loader-utils/-/loader-utils-2.0.4.tgz#8b5cb38b5c34a9a018ee1fc0e6a066d1dfcc528c" + integrity sha512-xXqpXoINfFhgua9xiqD8fPFHgkoq1mmmpE92WlDbm9rNRd/EbRb+Gqf908T2DMfuHjjJlksiK2RbHVOdD/MqSw== dependencies: big.js "^5.2.2" emojis-list "^3.0.0" From 90811196d7da6b4d7166885f5d8fb1fb3c983679 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Thu, 17 Nov 2022 13:00:22 +0100 Subject: [PATCH 195/214] docs: remove unused alabaster_jupyterhub requirement --- docs/requirements.txt | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 54c6f363..03dd2da8 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,6 +1,5 @@ -r ../requirements.txt -alabaster_jupyterhub autodoc-traits myst-parser pre-commit From 26e5efeec4f69e451aae47afffd0c8402864b4d8 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Thu, 17 Nov 2022 13:08:17 +0100 Subject: [PATCH 196/214] docs: cleanup unused config for htmlhelp, latex, manual --- docs/source/conf.py | 72 ++------------------------------------------- 1 file changed, 2 insertions(+), 70 deletions(-) diff --git a/docs/source/conf.py b/docs/source/conf.py index 63d61624..39a84fc8 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -7,9 +7,6 @@ sys.path.insert(0, os.path.abspath('.')) # -- General configuration ------------------------------------------------ -# Minimal Sphinx version -needs_sphinx = '1.4' - # Sphinx extension modules extensions = [ 'sphinx.ext.autodoc', @@ -27,7 +24,7 @@ myst_enable_extensions = [ 'deflist', ] # The master toctree document. -master_doc = 'index' +root_doc = master_doc = 'index' # General information about the project. project = 'JupyterHub' @@ -118,11 +115,10 @@ def setup(app): source_suffix = ['.rst', '.md'] -# source_encoding = 'utf-8-sig' # -- Options for HTML output ---------------------------------------------- -# The theme to use for HTML and HTML Help pages. +# The theme to use for HTML html_theme = 'pydata_sphinx_theme' html_logo = '_static/images/logo/logo.png' @@ -131,8 +127,6 @@ html_favicon = '_static/images/logo/favicon.ico' # Paths that contain custom static files (such as style sheets) html_static_path = ['_static'] -htmlhelp_basename = 'JupyterHubdoc' - html_theme_options = { "icon_links": [ { @@ -157,68 +151,6 @@ html_context = { "doc_path": "docs", } -# -- Options for LaTeX output --------------------------------------------- - -latex_elements = { - # 'papersize': 'letterpaper', - # 'pointsize': '10pt', - # 'preamble': '', - # 'figure_align': 'htbp', -} - -# Grouping the document tree into LaTeX files. List of tuples -# (source start file, target name, title, -# author, documentclass [howto, manual, or own class]). -latex_documents = [ - ( - master_doc, - 'JupyterHub.tex', - 'JupyterHub Documentation', - 'Project Jupyter team', - 'manual', - ) -] - -# latex_logo = None -# latex_use_parts = False -# latex_show_pagerefs = False -# latex_show_urls = False -# latex_appendices = [] -# latex_domain_indices = True - - -# -- manual page output ------------------------------------------------- - -# One entry per manual page. List of tuples -# (source start file, name, description, authors, manual section). -man_pages = [(master_doc, 'jupyterhub', 'JupyterHub Documentation', [author], 1)] - -# man_show_urls = False - - -# -- Texinfo output ----------------------------------------------------- - -# Grouping the document tree into Texinfo files. List of tuples -# (source start file, target name, title, author, -# dir menu entry, description, category) -texinfo_documents = [ - ( - master_doc, - 'JupyterHub', - 'JupyterHub Documentation', - author, - 'JupyterHub', - 'One line description of project.', - 'Miscellaneous', - ) -] - -# texinfo_appendices = [] -# texinfo_domain_indices = True -# texinfo_show_urls = 'footnote' -# texinfo_no_detailmenu = False - - # -- Epub output -------------------------------------------------------- # Bibliographic Dublin Core info. From 1c9499e91ec67105b9e75515ef68b259665e2c5e Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Thu, 17 Nov 2022 13:09:06 +0100 Subject: [PATCH 197/214] docs: remove epub documentation build --- .readthedocs.yaml | 9 ++++++++- docs/source/conf.py | 11 ----------- 2 files changed, 8 insertions(+), 12 deletions(-) diff --git a/.readthedocs.yaml b/.readthedocs.yaml index c1e09b1a..31ff7a72 100644 --- a/.readthedocs.yaml +++ b/.readthedocs.yaml @@ -1,3 +1,7 @@ +# Configuration on how ReadTheDocs (RTD) builds our documentation +# ref: https://readthedocs.org/projects/jupyterhub/ +# ref: https://docs.readthedocs.io/en/stable/config-file/v2.html +# version: 2 sphinx: @@ -16,5 +20,8 @@ python: - requirements: docs/requirements.txt formats: + # Adding htmlzip enables a Downloads section in the rendered website's RTD + # menu where the html build can be downloaded. This doesn't require any + # additional configuration in docs/source/conf.py. + # - htmlzip - - epub diff --git a/docs/source/conf.py b/docs/source/conf.py index 39a84fc8..03044de1 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -151,17 +151,6 @@ html_context = { "doc_path": "docs", } -# -- Epub output -------------------------------------------------------- - -# Bibliographic Dublin Core info. -epub_title = project -epub_author = author -epub_publisher = author -epub_copyright = copyright - -# A list of files that should not be packed into the epub file. -epub_exclude_files = ['search.html'] - # -- Intersphinx ---------------------------------------------------------- intersphinx_mapping = { From 7f3fd7e3cc87aaab3255a3c73f538ada678859cf Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Thu, 17 Nov 2022 13:38:25 +0100 Subject: [PATCH 198/214] docs: fix broken links and formatting errors --- docs/source/admin/upgrading.rst | 4 +++- docs/source/getting-started/config-basics.md | 4 ++-- docs/source/reference/config-sudo.md | 2 +- 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/source/admin/upgrading.rst b/docs/source/admin/upgrading.rst index fb76a94a..fe793915 100644 --- a/docs/source/admin/upgrading.rst +++ b/docs/source/admin/upgrading.rst @@ -43,8 +43,10 @@ Backup database & config Before doing an upgrade, it is critical to back up: #. Your JupyterHub database (SQLite by default, or MySQL / Postgres if you used those). -If you use SQLite (the default), you should backup the ``jupyterhub.sqlite`` file. + If you use SQLite (the default), you should backup the ``jupyterhub.sqlite`` file. + #. Your ``jupyterhub_config.py`` file. + #. Your users' home directories. This is unlikely to be affected directly by a JupyterHub upgrade, but we recommend a backup since user data is critical. diff --git a/docs/source/getting-started/config-basics.md b/docs/source/getting-started/config-basics.md index 03495a62..7f838871 100644 --- a/docs/source/getting-started/config-basics.md +++ b/docs/source/getting-started/config-basics.md @@ -77,8 +77,8 @@ jupyterhub --Spawner.notebook_dir='~/assignments' ## Configure for various deployment environments The default authentication and process spawning mechanisms can be replaced, and -specific [authenticators](./authenticators-users-basics) and -[spawners](./spawners-basics) can be set in the configuration file. +specific [authenticators](authenticators-users-basics) and +[spawners](spawners-basics) can be set in the configuration file. This enables JupyterHub to be used with a variety of authentication methods or process control and deployment environments. [Some examples](../reference/config-examples), meant as illustrations, are: diff --git a/docs/source/reference/config-sudo.md b/docs/source/reference/config-sudo.md index 2962ae97..57d6a385 100644 --- a/docs/source/reference/config-sudo.md +++ b/docs/source/reference/config-sudo.md @@ -6,7 +6,7 @@ Only do this if you are very sure you must. ## Overview -There are many [Authenticators](./authenticators-users-basics) and [Spawners](./spawners-basics) available for JupyterHub. Some, such +There are many [Authenticators](../getting-started/authenticators-users-basics) and [Spawners](../getting-started/spawners-basics) available for JupyterHub. Some, such as [DockerSpawner](https://github.com/jupyterhub/dockerspawner) or [OAuthenticator](https://github.com/jupyterhub/oauthenticator), do not need any elevated permissions. This document describes how to get the full default behavior of JupyterHub while running notebook servers as real system users on a shared system, without From 12594631e01c69cc2fcbf9beb1c2148fae1eeb42 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Fri, 18 Nov 2022 16:42:43 +0100 Subject: [PATCH 199/214] maint: use __all__ statements where its reasonable to do Co-authored-by: Simon Li --- examples/service-fastapi/app/__init__.py | 4 +++- jupyterhub/__init__.py | 4 +++- jupyterhub/singleuser/__init__.py | 6 ++++-- 3 files changed, 10 insertions(+), 4 deletions(-) diff --git a/examples/service-fastapi/app/__init__.py b/examples/service-fastapi/app/__init__.py index a4576905..34f275ed 100644 --- a/examples/service-fastapi/app/__init__.py +++ b/examples/service-fastapi/app/__init__.py @@ -1 +1,3 @@ -from .app import app # noqa +from .app import app + +__all__ = ["app"] diff --git a/jupyterhub/__init__.py b/jupyterhub/__init__.py index 6bb39770..4b758c5d 100644 --- a/jupyterhub/__init__.py +++ b/jupyterhub/__init__.py @@ -1 +1,3 @@ -from ._version import __version__, version_info # noqa +from ._version import __version__, version_info + +__all__ = ["__version__", "version_info"] diff --git a/jupyterhub/singleuser/__init__.py b/jupyterhub/singleuser/__init__.py index be6c08f8..dcd8c332 100644 --- a/jupyterhub/singleuser/__init__.py +++ b/jupyterhub/singleuser/__init__.py @@ -2,8 +2,10 @@ Contains default notebook-app subclass and mixins """ -from .app import SingleUserNotebookApp, main # noqa -from .mixins import HubAuthenticatedHandler, make_singleuser_app # noqa +from .app import SingleUserNotebookApp, main +from .mixins import HubAuthenticatedHandler, make_singleuser_app + +__all__ = ["SingleUserNotebookApp", "main", "HubAuthenticatedHandler", "make_singleuser_app"] # backward-compatibility JupyterHubLoginHandler = SingleUserNotebookApp.login_handler_class From 9d592fff31768f256fa96883e66f872a0e4249f0 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Fri, 18 Nov 2022 15:43:09 +0000 Subject: [PATCH 200/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- jupyterhub/singleuser/__init__.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/jupyterhub/singleuser/__init__.py b/jupyterhub/singleuser/__init__.py index dcd8c332..f3188bc7 100644 --- a/jupyterhub/singleuser/__init__.py +++ b/jupyterhub/singleuser/__init__.py @@ -5,7 +5,12 @@ Contains default notebook-app subclass and mixins from .app import SingleUserNotebookApp, main from .mixins import HubAuthenticatedHandler, make_singleuser_app -__all__ = ["SingleUserNotebookApp", "main", "HubAuthenticatedHandler", "make_singleuser_app"] +__all__ = [ + "SingleUserNotebookApp", + "main", + "HubAuthenticatedHandler", + "make_singleuser_app", +] # backward-compatibility JupyterHubLoginHandler = SingleUserNotebookApp.login_handler_class From 29d84f41921c9bdefcc2bea1c39c5de735996f8f Mon Sep 17 00:00:00 2001 From: Min RK Date: Tue, 22 Nov 2022 09:41:24 +0100 Subject: [PATCH 201/214] selenium: update next_url after waiting for it to change --- jupyterhub/tests/selenium/test_browser.py | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/jupyterhub/tests/selenium/test_browser.py b/jupyterhub/tests/selenium/test_browser.py index 9c52878c..d02348cb 100644 --- a/jupyterhub/tests/selenium/test_browser.py +++ b/jupyterhub/tests/selenium/test_browser.py @@ -88,9 +88,8 @@ def clear(browser, by_locator): # LOGIN PAGE async def test_elements_of_login_page(app, browser): await open_url(app, browser) - logo = is_displayed(browser, LoginPageLocators.LOGO) + assert is_displayed(browser, LoginPageLocators.LOGO) logo_text = browser.find_element(*LoginPageLocators.LOGO).get_attribute("innerHTML") - assert logo == True async def login(browser, user, pass_w): @@ -186,10 +185,11 @@ async def test_open_url_login( assert next_url.endswith("spawn?param=value") assert f"user/{user}/" not in next_url else: - if next_url.endswith(f"/user/{user}/") == False: + if not next_url.endswith(f"/user/{user}/"): await webdriver_wait( browser, EC.url_to_be(ujoin(public_url(app), f"/user/{user}/")) ) + next_url = browser.current_url assert next_url.endswith(f"/user/{user}/") From e6e84eabb38082a3b767cfabc179c638f70a3fa0 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Fri, 18 Nov 2022 15:51:29 +0100 Subject: [PATCH 202/214] ci: focus tests towards py311 over py310 --- .github/workflows/test.yml | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 52b2a736..9d1ef764 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -83,14 +83,13 @@ jobs: db: mysql - python: "3.10" db: postgres - - python: "3.10" + - python: "3.11" subdomain: subdomain - - python: "3.10" + - python: "3.11" ssl: ssl - - python: "3.10" + - python: "3.11" selenium: selenium - - python: "3.11.0-rc.1" - - python: "3.10" + - python: "3.11" main_dependencies: main_dependencies steps: From f26b43f2096d7a4d3a26b5ebbb2fd61edec58490 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Fri, 18 Nov 2022 15:37:29 +0100 Subject: [PATCH 203/214] maint: remove no longer needed py311 greenlet workaround --- .github/workflows/test.yml | 9 --------- ci/mock-greenlet/greenlet.py | 3 --- ci/mock-greenlet/pyproject.toml | 13 ------------- 3 files changed, 25 deletions(-) delete mode 100644 ci/mock-greenlet/greenlet.py delete mode 100644 ci/mock-greenlet/pyproject.toml diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 9d1ef764..94847e23 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -140,15 +140,6 @@ jobs: - name: Install Python dependencies run: | pip install --upgrade pip - - if [[ "${{ matrix.python }}" == "3.11"* ]]; then - # greenlet is not actually required, - # but is an install dependency of sqlalchemy. - # It does not yet install on 3.11 - # see: see https://github.com/gevent/gevent/issues/1867 - pip install ./ci/mock-greenlet - fi - pip install --upgrade . -r dev-requirements.txt if [ "${{ matrix.oldest_dependencies }}" != "" ]; then diff --git a/ci/mock-greenlet/greenlet.py b/ci/mock-greenlet/greenlet.py deleted file mode 100644 index c1066e6e..00000000 --- a/ci/mock-greenlet/greenlet.py +++ /dev/null @@ -1,3 +0,0 @@ -__version__ = "22.0.0.dev0" - -raise ImportError("Don't actually have greenlet") diff --git a/ci/mock-greenlet/pyproject.toml b/ci/mock-greenlet/pyproject.toml deleted file mode 100644 index 3a1c23e8..00000000 --- a/ci/mock-greenlet/pyproject.toml +++ /dev/null @@ -1,13 +0,0 @@ -[build-system] -requires = ["hatchling"] -build-backend = "hatchling.build" - -[project] -name = "greenlet" -description = 'Mock greenlet to allow install on 3.11' -requires-python = ">=3.7" -dynamic = ["version"] - - -[tool.hatch.version] -path = "greenlet.py" From 87ce2a9b2f61fe03df488ed390138450c65a0f2f Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Fri, 18 Nov 2022 14:41:43 +0100 Subject: [PATCH 204/214] maint: add jsonschema to test requirements and cleanup attrs constraint --- dev-requirements.txt | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/dev-requirements.txt b/dev-requirements.txt index 2f9cb22d..7e031535 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -1,12 +1,9 @@ --r requirements.txt -# temporary pin of attrs for jsonschema 0.3.0a1 -# seems to be a pip bug -attrs>=17.4.0 beautifulsoup4 coverage cryptography html5lib # needed for beautifulsoup -jupyterlab >=3 +jsonschema +jupyterlab>=3 mock # nbclassic provides the '/tree/' handler, which we use in tests # it is a transitive dependency via jupyterlab, From 0cdaa833c4ac35d6594e7323fb61de069246cf50 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Fri, 18 Nov 2022 14:54:44 +0100 Subject: [PATCH 205/214] maint: remove unused urllib3 from test requirements --- dev-requirements.txt | 3 --- 1 file changed, 3 deletions(-) diff --git a/dev-requirements.txt b/dev-requirements.txt index 7e031535..c5186122 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -16,7 +16,4 @@ pytest-cov requests-mock selenium tbump -# blacklist urllib3 releases affected by https://github.com/urllib3/urllib3/issues/1683 -# I *think* this should only affect testing, not production -urllib3!=1.25.4,!=1.25.5 virtualenv From 08ed8443f277518cfbf686f91a7bd76ea10977db Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Fri, 18 Nov 2022 14:58:29 +0100 Subject: [PATCH 206/214] maint: install html5lib via beautifulsoup4's extra_requires --- dev-requirements.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/dev-requirements.txt b/dev-requirements.txt index c5186122..65747e80 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -1,7 +1,6 @@ -beautifulsoup4 +beautifulsoup4[html5lib] coverage cryptography -html5lib # needed for beautifulsoup jsonschema jupyterlab>=3 mock From 1ac9c443b56465fe1ee83139a059bcd337a763ec Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Fri, 18 Nov 2022 15:11:44 +0100 Subject: [PATCH 207/214] maint: add test to extras_require, remove dev-requirements.txt --- .github/workflows/test.yml | 2 +- RELEASE.md | 49 ++++++++++++++++-------------- dev-requirements.txt | 18 ----------- docs/source/contributing/setup.rst | 19 ++++-------- docs/source/contributing/tests.rst | 30 ++++++++++-------- setup.py | 21 +++++++++++++ 6 files changed, 73 insertions(+), 66 deletions(-) delete mode 100644 dev-requirements.txt diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 94847e23..78bb9b8f 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -140,7 +140,7 @@ jobs: - name: Install Python dependencies run: | pip install --upgrade pip - pip install --upgrade . -r dev-requirements.txt + pip install ".[test]" if [ "${{ matrix.oldest_dependencies }}" != "" ]; then # take any dependencies in requirements.txt such as tornado>=5.0 diff --git a/RELEASE.md b/RELEASE.md index 1bdbb3d4..b00a089e 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,39 +1,42 @@ # How to make a release -`jupyterhub` is a package [available on -PyPI](https://pypi.org/project/jupyterhub/) and -[conda-forge](https://conda-forge.org/). -These are instructions on how to make a release on PyPI. -The PyPI release is done automatically by CI when a tag is pushed. +`jupyterhub` is a package available on [PyPI][] and [conda-forge][]. +These are instructions on how to make a release. -For you to follow along according to these instructions, you need: +## Pre-requisites -- To have push rights to the [jupyterhub GitHub - repository](https://github.com/jupyterhub/jupyterhub). +- Push rights to [jupyterhub/jupyterhub][] +- Push rights to [conda-forge/jupyterhub-feedstock][] ## Steps to make a release +1. Create a PR updating `docs/source/changelog.md` with [github-activity][] and + continue only when its merged. + + ```shell + pip install github-activity + + github-activity --heading-level=3 jupyterhub/jupyterhub + ``` + 1. Checkout main and make sure it is up to date. ```shell - ORIGIN=${ORIGIN:-origin} # set to the canonical remote, e.g. 'upstream' if 'origin' is not the official repo git checkout main - git fetch $ORIGIN main - git reset --hard $ORIGIN/main + git fetch origin main + git reset --hard origin/main ``` -1. Make sure `docs/source/changelog.md` is up-to-date. - [github-activity][] can help with this. - -1. Update the version with `tbump`. - You can see what will happen without making any changes with `tbump --dry-run ${VERSION}` +1. Update the version, make commits, and push a git tag with `tbump`. ```shell + pip install tbump + tbump --dry-run ${VERSION} + tbump ${VERSION} ``` - This will tag and publish a release, - which will be finished on CI. + Following this, the [CI system][] will build and publish a release. 1. Reset the version back to dev, e.g. `2.1.0.dev` after releasing `2.0.0` @@ -42,9 +45,11 @@ For you to follow along according to these instructions, you need: ``` 1. Following the release to PyPI, an automated PR should arrive to - [conda-forge/jupyterhub-feedstock][], - check for the tests to succeed on this PR and then merge it to successfully - update the package for `conda` on the conda-forge channel. + [conda-forge/jupyterhub-feedstock][] with instructions. -[github-activity]: https://github.com/choldgraf/github-activity +[pypi]: https://pypi.org/project/jupyterhub/ +[conda-forge]: https://anaconda.org/conda-forge/jupyterhub +[jupyterhub/jupyterhub]: https://github.com/jupyterhub/jupyterhub [conda-forge/jupyterhub-feedstock]: https://github.com/conda-forge/jupyterhub-feedstock +[github-activity]: https://github.com/executablebooks/github-activity +[ci system]: https://github.com/jupyterhub/jupyterhub/actions/workflows/release.yml diff --git a/dev-requirements.txt b/dev-requirements.txt deleted file mode 100644 index 65747e80..00000000 --- a/dev-requirements.txt +++ /dev/null @@ -1,18 +0,0 @@ -beautifulsoup4[html5lib] -coverage -cryptography -jsonschema -jupyterlab>=3 -mock -# nbclassic provides the '/tree/' handler, which we use in tests -# it is a transitive dependency via jupyterlab, -# but depend on it directly -nbclassic -pre-commit -pytest>=3.3 -pytest-asyncio>=0.17 -pytest-cov -requests-mock -selenium -tbump -virtualenv diff --git a/docs/source/contributing/setup.rst b/docs/source/contributing/setup.rst index aa60f3b7..1fabe6f7 100644 --- a/docs/source/contributing/setup.rst +++ b/docs/source/contributing/setup.rst @@ -94,12 +94,13 @@ When developing JupyterHub, you would need to make changes and be able to instan conda install configurable-http-proxy yarn -4. Install the python packages required for JupyterHub development. +4. Install an editable version of JupyterHub and its requirements for + development and testing. This lets you edit JupyterHub code in a text editor + & restart the JupyterHub process to see your code changes immediately. .. code:: bash - python3 -m pip install -r dev-requirements.txt - python3 -m pip install -r requirements.txt + python3 -m pip install --editable ".[test]" 5. Set up a database. @@ -108,21 +109,13 @@ When developing JupyterHub, you would need to make changes and be able to instan available via `Python `__. See :doc:`/reference/database` for details on other supported databases. -6. Install the development version of JupyterHub. This lets you edit - JupyterHub code in a text editor & restart the JupyterHub process to - see your code changes immediately. - - .. code:: bash - - python3 -m pip install --editable . - -7. You are now ready to start JupyterHub! +6. You are now ready to start JupyterHub! .. code:: bash jupyterhub -8. You can access JupyterHub from your browser at +7. You can access JupyterHub from your browser at ``http://localhost:8000`` now. Happy developing! diff --git a/docs/source/contributing/tests.rst b/docs/source/contributing/tests.rst index 8904b8c1..55bded61 100644 --- a/docs/source/contributing/tests.rst +++ b/docs/source/contributing/tests.rst @@ -110,22 +110,28 @@ Make sure you have completed all the steps in :ref:`contributing/setup` successf Code formatting and linting =========================== -JupyterHub has adopted automatic code formatting and linting. -As long as your code is valid, the pre-commit hook should take care of how it should look. -You can invoke the pre-commit hook manually at any time with: +JupyterHub automatically enforces code formatting. This means that pull requests +with changes breaking this formatting will receive a commit from pre-commit.ci +automatically. + +To automatically format code locally, you can install pre-commit and register a +_git hook_ to automatically check with pre-commit before you make a commit if +the formatting is okay. .. code:: bash + pip install pre-commit + pre-commit install --install-hooks + +To run pre-commit manually you would do: + +.. code:: bash + + # check for changes to code not yet committed pre-commit run -This should run any auto formatting on your code and tell you about any errors it couldn't fix automatically. -You may also install `black integration `_ -into your text editor to format code automatically. - -If you have already committed files before running pre-commit you can fix everything using: - -.. code:: bash - + # check for changes also in already committed code pre-commit run --all-files -And committing the changes. +You may also install `black integration `_ +into your text editor to format code automatically. diff --git a/setup.py b/setup.py index eff3cbf3..9f06e496 100755 --- a/setup.py +++ b/setup.py @@ -121,6 +121,27 @@ setup_args = dict( 'Source': 'https://github.com/jupyterhub/jupyterhub/', 'Tracker': 'https://github.com/jupyterhub/jupyterhub/issues', }, + extras_require={ + "test": [ + "beautifulsoup4[html5lib]", + "coverage", + # cryptography is an optional dependency for jupyterhub that we test + # against by default + "cryptography", + "jsonschema", + "jupyterlab>=3", + "mock", + # nbclassic provides the '/tree/' handler that we tests against in + # the test test_nbclassic_control_panel. + "nbclassic", + "pytest>=3.3", + "pytest-asyncio>=0.17", + "pytest-cov", + "requests-mock", + "selenium", + "virtualenv", + ], + } ) From 0575022186a0d17305cb97b7fe22ccb79eec48ea Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Mon, 21 Nov 2022 15:32:23 +0100 Subject: [PATCH 208/214] docs: install jupyterhub package and remove path setup steps When we install the jupyterhub package, we don't have to put it on the path explicitly. --- .github/workflows/test-docs.yml | 2 +- .readthedocs.yaml | 2 -- docs/requirements.txt | 10 +++++++++- docs/source/conf.py | 12 ++---------- 4 files changed, 12 insertions(+), 14 deletions(-) diff --git a/.github/workflows/test-docs.yml b/.github/workflows/test-docs.yml index 18ce9174..0e96a2ec 100644 --- a/.github/workflows/test-docs.yml +++ b/.github/workflows/test-docs.yml @@ -55,7 +55,7 @@ jobs: - name: Install requirements run: | - pip install -r docs/requirements.txt pytest -e . + pip install -r docs/requirements.txt pytest - name: pytest docs/ run: | diff --git a/.readthedocs.yaml b/.readthedocs.yaml index 31ff7a72..d2d85361 100644 --- a/.readthedocs.yaml +++ b/.readthedocs.yaml @@ -15,8 +15,6 @@ build: python: install: - - method: pip - path: . - requirements: docs/requirements.txt formats: diff --git a/docs/requirements.txt b/docs/requirements.txt index 03dd2da8..697d50cb 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,4 +1,12 @@ --r ../requirements.txt +# We install the jupyterhub package to help autodoc-traits inspect it and +# generate documentation. +# +# FIXME: If there is a way for this requirements.txt file to pass a flag that +# the build system can intercept to not build the javascript artifacts, +# then do so so. That would mean that installing the documentation can +# avoid needing node/npm installed. +# +--editable . autodoc-traits myst-parser diff --git a/docs/source/conf.py b/docs/source/conf.py index 1a7ae015..5ac0b9ad 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -1,9 +1,5 @@ # import os -import sys - -# Set paths -sys.path.insert(0, os.path.abspath('.')) # -- General configuration ------------------------------------------------ @@ -32,12 +28,6 @@ copyright = '2016, Project Jupyter team' author = 'Project Jupyter team' # Autopopulate version -from os.path import dirname - -docs = dirname(dirname(__file__)) -root = dirname(docs) -sys.path.insert(0, root) - import jupyterhub # The short X.Y version. @@ -164,8 +154,10 @@ on_rtd = os.environ.get('READTHEDOCS', None) == 'True' if on_rtd: # readthedocs.org uses their theme by default, so no need to specify it # build both metrics and rest-api, since RTD doesn't run make + from os.path import dirname from subprocess import check_call as sh + docs = dirname(dirname(__file__)) sh(['make', 'metrics', 'scopes'], cwd=docs) # -- Spell checking ------------------------------------------------------- From 8ac3a8e4e66432a3be0db46d5825bb5fee9c1a04 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Tue, 22 Nov 2022 12:53:05 +0000 Subject: [PATCH 209/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.py b/setup.py index 9f06e496..93c6fec7 100755 --- a/setup.py +++ b/setup.py @@ -141,7 +141,7 @@ setup_args = dict( "selenium", "virtualenv", ], - } + }, ) From 96d4486ae5947c49e8ed836ada300216371748f6 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Tue, 22 Nov 2022 15:26:55 +0100 Subject: [PATCH 210/214] docs: refresh conf.py for readability --- docs/Makefile | 3 + docs/requirements.txt | 2 +- docs/source/conf.py | 210 +++++++++++++++++++++++------------------- 3 files changed, 120 insertions(+), 95 deletions(-) diff --git a/docs/Makefile b/docs/Makefile index 64f7d449..5aa434a9 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -63,6 +63,9 @@ scopes: source/rbac/scope-table.md source/rbac/scope-table.md: source/rbac/generate-scope-table.py python3 source/rbac/generate-scope-table.py +# If the pre-requisites for the html target is updated, also update the Read The +# Docs section in docs/source/conf.py. +# html: metrics scopes $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html @echo diff --git a/docs/requirements.txt b/docs/requirements.txt index 697d50cb..315a926f 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -14,6 +14,6 @@ pre-commit pydata-sphinx-theme pytablewriter>=0.56 ruamel.yaml -sphinx>=1.7 +sphinx>=4 sphinx-copybutton sphinx-jsonschema diff --git a/docs/source/conf.py b/docs/source/conf.py index 5ac0b9ad..98aeb64b 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -1,59 +1,68 @@ +# Configuration file for Sphinx to build our documentation to HTML. # +# Configuration reference: https://www.sphinx-doc.org/en/master/usage/configuration.html +# +import contextlib +import datetime +import io import os - -# -- General configuration ------------------------------------------------ - -# Sphinx extension modules -extensions = [ - 'sphinx.ext.autodoc', - 'sphinx.ext.intersphinx', - 'sphinx.ext.napoleon', - 'autodoc_traits', - 'sphinx_copybutton', - 'sphinx-jsonschema', - 'myst_parser', -] - -myst_heading_anchors = 2 -myst_enable_extensions = [ - 'colon_fence', - 'deflist', -] -# The master toctree document. -root_doc = master_doc = 'index' - -# General information about the project. -project = 'JupyterHub' -copyright = '2016, Project Jupyter team' -author = 'Project Jupyter team' - -# Autopopulate version -import jupyterhub - -# The short X.Y version. -version = '%i.%i' % jupyterhub.version_info[:2] -# The full version, including alpha/beta/rc tags. -release = jupyterhub.__version__ - -language = "en" -exclude_patterns = [] -pygments_style = 'sphinx' -todo_include_todos = False - -# Set the default role so we can use `foo` instead of ``foo`` -default_role = 'literal' - -from contextlib import redirect_stdout -from io import StringIO +import subprocess from docutils import nodes from sphinx.directives.other import SphinxDirective -# -- Config ------------------------------------------------------------- +import jupyterhub from jupyterhub.app import JupyterHub -# create a temp instance of JupyterHub just to get the output of the generate-config -# and help --all commands. +# -- Project information ----------------------------------------------------- +# ref: https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information +# +project = "JupyterHub" +author = "Project Jupyter Contributors" +copyright = f"{datetime.date.today().year}, {author}" +version = "%i.%i" % jupyterhub.version_info[:2] +release = jupyterhub.__version__ + + +# -- General Sphinx configuration -------------------------------------------- +# ref: https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration +# +extensions = [ + "sphinx.ext.autodoc", + "sphinx.ext.intersphinx", + "sphinx.ext.napoleon", + "autodoc_traits", + "sphinx_copybutton", + "sphinx-jsonschema", + "myst_parser", +] +root_doc = "index" +source_suffix = [".md", ".rst"] +# default_role let's use use `foo` instead of ``foo`` in rST +default_role = "literal" + + +# -- MyST configuration ------------------------------------------------------ +# ref: https://myst-parser.readthedocs.io/en/latest/configuration.html +# +myst_heading_anchors = 2 +myst_enable_extensions = [ + "colon_fence", + "deflist", +] + + +# -- Custom directives to generate documentation ----------------------------- +# ref: https://myst-parser.readthedocs.io/en/latest/syntax/roles-and-directives.html +# +# We define custom directives to help us generate documentation using Python on +# demand when referenced from our documentation files. +# + +# Create a temp instance of JupyterHub for use by two separate directive classes +# to get the output from using the "--generate-config" and "--help-all" CLI +# flags respectively. +# jupyterhub_app = JupyterHub() @@ -70,8 +79,8 @@ class ConfigDirective(SphinxDirective): # The generated configuration file for this version generated_config = jupyterhub_app.generate_config_file() # post-process output - home_dir = os.environ['HOME'] - generated_config = generated_config.replace(home_dir, '$HOME', 1) + home_dir = os.environ["HOME"] + generated_config = generated_config.replace(home_dir, "$HOME", 1) par = nodes.literal_block(text=generated_config) return [par] @@ -87,36 +96,55 @@ class HelpAllDirective(SphinxDirective): def run(self): # The output of the help command for this version - buffer = StringIO() - with redirect_stdout(buffer): - jupyterhub_app.print_help('--help-all') + buffer = io.StringIO() + with contextlib.redirect_stdout(buffer): + jupyterhub_app.print_help("--help-all") all_help = buffer.getvalue() # post-process output - home_dir = os.environ['HOME'] - all_help = all_help.replace(home_dir, '$HOME', 1) + home_dir = os.environ["HOME"] + all_help = all_help.replace(home_dir, "$HOME", 1) par = nodes.literal_block(text=all_help) return [par] def setup(app): - app.add_css_file('custom.css') - app.add_directive('jupyterhub-generate-config', ConfigDirective) - app.add_directive('jupyterhub-help-all', HelpAllDirective) + app.add_css_file("custom.css") + app.add_directive("jupyterhub-generate-config", ConfigDirective) + app.add_directive("jupyterhub-help-all", HelpAllDirective) -source_suffix = ['.rst', '.md'] +# -- Read The Docs ----------------------------------------------------------- +# +# Since RTD runs sphinx-build directly without running "make html", we run the +# pre-requisite steps for "make html" from here if needed. +# +if os.environ.get("READTHEDOCS"): + docs = os.path.dirname(os.path.dirname(__file__)) + subprocess.check_call(["make", "metrics", "scopes"], cwd=docs) -# -- Options for HTML output ---------------------------------------------- -# The theme to use for HTML -html_theme = 'pydata_sphinx_theme' +# -- Spell checking ---------------------------------------------------------- +# ref: https://sphinxcontrib-spelling.readthedocs.io/en/latest/customize.html#configuration-options +# +# The "sphinxcontrib.spelling" extension is optionally enabled if its available. +# +try: + import sphinxcontrib.spelling # noqa +except ImportError: + pass +else: + extensions.append("sphinxcontrib.spelling") +spelling_word_list_filename = "spelling_wordlist.txt" -html_logo = '_static/images/logo/logo.png' -html_favicon = '_static/images/logo/favicon.ico' -# Paths that contain custom static files (such as style sheets) -html_static_path = ['_static'] +# -- Options for HTML output ------------------------------------------------- +# ref: https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output +# +html_logo = "_static/images/logo/logo.png" +html_favicon = "_static/images/logo/favicon.ico" +html_static_path = ["_static"] +html_theme = "pydata_sphinx_theme" html_theme_options = { "icon_links": [ { @@ -133,40 +161,34 @@ html_theme_options = { "use_edit_page_button": True, "navbar_align": "left", } - html_context = { "github_user": "jupyterhub", "github_repo": "jupyterhub", "github_version": "main", - "doc_path": "docs", + "doc_path": "docs/source", } -# -- Intersphinx ---------------------------------------------------------- +# -- Options for linkcheck builder ------------------------------------------- +# ref: https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-the-linkcheck-builder +# +linkcheck_ignore = [ + r"(.*)github\.com(.*)#", # javascript based anchors + r"(.*)/#%21(.*)/(.*)", # /#!forum/jupyter - encoded anchor edge case + r"https://github.com/[^/]*$", # too many github usernames / searches in changelog + "https://github.com/jupyterhub/jupyterhub/pull/", # too many PRs in changelog + "https://github.com/jupyterhub/jupyterhub/compare/", # too many comparisons in changelog +] +linkcheck_anchors_ignore = [ + "/#!", + "/#%21", +] + + +# -- Intersphinx ------------------------------------------------------------- +# ref: https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html#configuration +# intersphinx_mapping = { - 'python': ('https://docs.python.org/3/', None), - 'tornado': ('https://www.tornadoweb.org/en/stable/', None), + "python": ("https://docs.python.org/3/", None), + "tornado": ("https://www.tornadoweb.org/en/stable/", None), } - -# -- Read The Docs -------------------------------------------------------- - -on_rtd = os.environ.get('READTHEDOCS', None) == 'True' -if on_rtd: - # readthedocs.org uses their theme by default, so no need to specify it - # build both metrics and rest-api, since RTD doesn't run make - from os.path import dirname - from subprocess import check_call as sh - - docs = dirname(dirname(__file__)) - sh(['make', 'metrics', 'scopes'], cwd=docs) - -# -- Spell checking ------------------------------------------------------- - -try: - import sphinxcontrib.spelling # noqa -except ImportError: - pass -else: - extensions.append("sphinxcontrib.spelling") - -spelling_word_list_filename = 'spelling_wordlist.txt' From d23e09510631241cd76d3eb6e5416064650ba703 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Tue, 22 Nov 2022 17:16:08 +0100 Subject: [PATCH 211/214] docs: add opengraph sphinx extension This extension helps add metadata etc to the rendered HTML documentation to help for example links on discourse.jupyter.org look good. --- docs/requirements.txt | 1 + docs/source/conf.py | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/docs/requirements.txt b/docs/requirements.txt index 315a926f..bd71f28e 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -17,3 +17,4 @@ ruamel.yaml sphinx>=4 sphinx-copybutton sphinx-jsonschema +sphinxext-opengraph diff --git a/docs/source/conf.py b/docs/source/conf.py index 98aeb64b..f36304e6 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -34,6 +34,7 @@ extensions = [ "autodoc_traits", "sphinx_copybutton", "sphinx-jsonschema", + "sphinxext.opengraph", "myst_parser", ] root_doc = "index" @@ -192,3 +193,9 @@ intersphinx_mapping = { "python": ("https://docs.python.org/3/", None), "tornado": ("https://www.tornadoweb.org/en/stable/", None), } +# -- Options for the opengraph extension ------------------------------------- +# ref: https://github.com/wpilibsuite/sphinxext-opengraph#options +# +# ogp_site_url is set automatically by RTD +ogp_image = "_static/logo.png" +ogp_use_first_image = True From bfe0186ad2c9b537ece80f51152ebb265c681418 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Tue, 22 Nov 2022 17:17:38 +0100 Subject: [PATCH 212/214] docs: add rediraffe sphinx extension This extension helps us restructure our documentation without creating dead links. It requires us to explicitly declare what should be redirected where though. It seems better to have it in place ahead of time than to be something we ask a contributor add just in time when its needed. --- docs/requirements.txt | 1 + docs/source/conf.py | 14 ++++++++++++++ 2 files changed, 15 insertions(+) diff --git a/docs/requirements.txt b/docs/requirements.txt index bd71f28e..6748578b 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -18,3 +18,4 @@ sphinx>=4 sphinx-copybutton sphinx-jsonschema sphinxext-opengraph +sphinxext-rediraffe diff --git a/docs/source/conf.py b/docs/source/conf.py index f36304e6..736a0acb 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -35,6 +35,7 @@ extensions = [ "sphinx_copybutton", "sphinx-jsonschema", "sphinxext.opengraph", + "sphinxext.rediraffe", "myst_parser", ] root_doc = "index" @@ -199,3 +200,16 @@ intersphinx_mapping = { # ogp_site_url is set automatically by RTD ogp_image = "_static/logo.png" ogp_use_first_image = True + + +# -- Options for the rediraffe extension ------------------------------------- +# ref: https://github.com/wpilibsuite/sphinxext-rediraffe#readme +# +# This extensions help us relocated content without breaking links. If a +# document is moved internally, a redirect like should be configured below to +# help us not break links. +# +rediraffe_branch = "main" +rediraffe_redirects = { + # "old-file": "new-folder/new-file-name", +} From f877588e52212a81e014fa9bd4026d8deec2a9d5 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Tue, 22 Nov 2022 17:22:39 +0100 Subject: [PATCH 213/214] docs: fix rST syntax error for italics --- docs/source/contributing/tests.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/contributing/tests.rst b/docs/source/contributing/tests.rst index 55bded61..023635d1 100644 --- a/docs/source/contributing/tests.rst +++ b/docs/source/contributing/tests.rst @@ -115,7 +115,7 @@ with changes breaking this formatting will receive a commit from pre-commit.ci automatically. To automatically format code locally, you can install pre-commit and register a -_git hook_ to automatically check with pre-commit before you make a commit if +*git hook* to automatically check with pre-commit before you make a commit if the formatting is okay. .. code:: bash From 33fa9b953bf4708280fdec86a31869db8a397971 Mon Sep 17 00:00:00 2001 From: "pre-commit-ci[bot]" <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu, 24 Nov 2022 08:41:12 +0000 Subject: [PATCH 214/214] [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --- docs/source/getting-started/config-basics.md | 6 +++--- docs/source/getting-started/networking-basics.md | 2 +- docs/source/reference/authenticators.md | 4 ++-- docs/source/reference/config-sudo.md | 6 +++--- docs/source/reference/config-user-env.md | 4 ++-- docs/source/reference/services.md | 3 +-- docs/source/reference/spawners.md | 2 +- 7 files changed, 13 insertions(+), 14 deletions(-) diff --git a/docs/source/getting-started/config-basics.md b/docs/source/getting-started/config-basics.md index 853dc3a7..7f838871 100644 --- a/docs/source/getting-started/config-basics.md +++ b/docs/source/getting-started/config-basics.md @@ -91,10 +91,10 @@ meant as illustrations, are: This is _not_ strictly necessary, but useful in many cases. If you use a custom proxy (e.g. Traefik), this is also not needed. -Connections to user servers go through the proxy, and *not* the hub -itself. If the proxy stays running when the hub restarts (for +Connections to user servers go through the proxy, and _not_ the hub +itself. If the proxy stays running when the hub restarts (for maintenance, re-configuration, etc.), then user connections are not -interrupted. For simplicity, by default the hub starts the proxy +interrupted. For simplicity, by default the hub starts the proxy automatically, so if the hub restarts, the proxy restarts, and user connections are interrupted. It is easy to run the proxy separately, for information see [the separate proxy page](../reference/separate-proxy). diff --git a/docs/source/getting-started/networking-basics.md b/docs/source/getting-started/networking-basics.md index 1439400c..f924cd60 100644 --- a/docs/source/getting-started/networking-basics.md +++ b/docs/source/getting-started/networking-basics.md @@ -42,7 +42,7 @@ port. ## Set the Proxy's REST API communication URL (optional) By default, the proxy's REST API listens on port 8081 of `localhost` only. -The Hub service talks to the proxy via a REST API on a secondary port. +The Hub service talks to the proxy via a REST API on a secondary port. The REST API URL (hostname and port) can be configured separately and override the default settings. ### Set api_url diff --git a/docs/source/reference/authenticators.md b/docs/source/reference/authenticators.md index ee8227c8..a81be9f0 100644 --- a/docs/source/reference/authenticators.md +++ b/docs/source/reference/authenticators.md @@ -109,9 +109,9 @@ When using `PAMAuthenticator`, you can set normalize usernames using PAM (basically round-tripping them: username to uid to username), which is useful in case you use some external service that allows multiple usernames mapping to the same user (such -as ActiveDirectory, yes, this really happens). When +as ActiveDirectory, yes, this really happens). When `pam_normalize_username` is on, usernames are _not_ normalized to -lowercase. +lowercase. #### Validate usernames diff --git a/docs/source/reference/config-sudo.md b/docs/source/reference/config-sudo.md index 9cbfe642..12940665 100644 --- a/docs/source/reference/config-sudo.md +++ b/docs/source/reference/config-sudo.md @@ -70,7 +70,7 @@ rhea ALL=(JUPYTER_USERS) NOPASSWD:JUPYTER_CMD ``` It might be useful to modify `secure_path` to add commands in path. (Search for -`secure_path` in the [sudo docs](https://www.sudo.ws/man/1.8.14/sudoers.man.html) +`secure_path` in the [sudo docs](https://www.sudo.ws/man/1.8.14/sudoers.man.html) As an alternative to adding every user to the `/etc/sudoers` file, you can use a group in the last line above, instead of `JUPYTER_USERS`: @@ -158,7 +158,7 @@ then you will need to give `node` permission to do so: sudo setcap 'cap_net_bind_service=+ep' /usr/bin/node ``` -However, you may want to further understand the consequences of this. +However, you may want to further understand the consequences of this. ([Further reading](http://man7.org/linux/man-pages/man7/capabilities.7.html)) You may also be interested in limiting the amount of CPU any process can use @@ -170,7 +170,7 @@ instructions](http://ubuntuforums.org/showthread.php?t=992706). ### Shadow group (FreeBSD) **NOTE:** This has not been tested on FreeBSD and may not work as expected on -the FreeBSD platform. *Do not use in production without verifying that it works properly!* +the FreeBSD platform. _Do not use in production without verifying that it works properly!_ ```bash $ ls -l /etc/spwd.db /etc/master.passwd diff --git a/docs/source/reference/config-user-env.md b/docs/source/reference/config-user-env.md index 204c062d..31bc2d8d 100644 --- a/docs/source/reference/config-user-env.md +++ b/docs/source/reference/config-user-env.md @@ -39,8 +39,8 @@ to install the numpy package in the default Python 3 environment on your system You may also use conda to install packages. If you do, you should make sure that the conda environment has appropriate permissions for users to be able to -run Python code in the env. The env must be *readable and executable* by all -users. Additionally it must be *writeable* if you want users to install +run Python code in the env. The env must be _readable and executable_ by all +users. Additionally it must be _writeable_ if you want users to install additional packages. ## Configuring Jupyter and IPython diff --git a/docs/source/reference/services.md b/docs/source/reference/services.md index 32229f2c..90d755ab 100644 --- a/docs/source/reference/services.md +++ b/docs/source/reference/services.md @@ -239,12 +239,11 @@ done either programmatically when constructing the class, or via the `JUPYTERHUB_API_TOKEN` environment variable. A number of the examples in the root of the jupyterhub git repository set the `JUPYTERHUB_API_TOKEN` variable so consider having a look at those for futher reading -([cull-idle](https://github.com/jupyterhub/jupyterhub/tree/master/examples/cull-idle), +([cull-idle](https://github.com/jupyterhub/jupyterhub/tree/master/examples/cull-idle), [external-oauth](https://github.com/jupyterhub/jupyterhub/tree/master/examples/external-oauth), [service-notebook](https://github.com/jupyterhub/jupyterhub/tree/master/examples/service-notebook) and [service-whoiami](https://github.com/jupyterhub/jupyterhub/tree/master/examples/service-whoami)) - (TODO: Where is this API TOKen set?) Most of the logic for authentication implementation is found in the diff --git a/docs/source/reference/spawners.md b/docs/source/reference/spawners.md index a8b7c522..a98e7deb 100644 --- a/docs/source/reference/spawners.md +++ b/docs/source/reference/spawners.md @@ -405,6 +405,6 @@ When `.create_certs` is run, it will create the certificates in a default, central location specified by `c.JupyterHub.internal_certs_location`. For `Spawners` that need access to these certs elsewhere (i.e. on another host altogether), the `.move_certs` method can be overridden to move the certs -appropriately. Again, using `DockerSpawner` as an example, this would entail +appropriately. Again, using `DockerSpawner` as an example, this would entail moving certs to a directory that will get mounted into the container this spawner starts.