From e21737399b1af46d76bdd7118835605686ca4ab9 Mon Sep 17 00:00:00 2001 From: Carol Willing Date: Mon, 24 Jul 2017 07:53:20 -0700 Subject: [PATCH 1/8] Edit config basics doc and examples --- docs/source/config-basics.md | 101 +++++++++++++++++++-------------- docs/source/config-examples.md | 73 ++++++++++++++---------- 2 files changed, 102 insertions(+), 72 deletions(-) diff --git a/docs/source/config-basics.md b/docs/source/config-basics.md index e1b798d4..4bcd4382 100644 --- a/docs/source/config-basics.md +++ b/docs/source/config-basics.md @@ -1,69 +1,86 @@ # Configuration Basics -The [getting started document](docs/source/getting-started.md) contains -general information about configuring a JupyterHub deployment and the -[configuration reference](docs/source/configuration-guide.md) provides more -comprehensive detail. +The section contains basic information about configuring settings for a JupyterHub +deployment. The [configuration reference](docs/source/configuration-guide.rst) +provides additional detail. -## JupyterHub configuration +This section will help you learn how to: -Configuration parameters may be set by: -- a configuration file `jupyterhub_config.py`, or -- as options from the command line. +- generate a default configuration file, `jupyterhub_config.py` +- start with a specific configuration file +- configure JupyterHub using command line options +- find information and examples for some common deployments -### Generate a default config file +## Generate a default config file On startup, JupyterHub will look by default for a configuration file named `jupyterhub_config.py` in the current working directory. -To generate a default config file `jupyterhub_config.py`: +To generate a default config file, `jupyterhub_config.py`: ```bash jupyterhub --generate-config ``` This default `jupyterhub_config.py` file contains comments and guidance for all -configuration variables and their default values. +configuration variables and their default values. We recommend storing +configuration files in the standard UNIX filesystem location, i.e. +`/etc/jupyterhub`. -### Configure using command line options +## Start with a specific config file -To display all command line options that are available for configuration: - - jupyterhub --help-all - -Configuration using the command line options is done when launching JupyterHub. -For example, to start JupyterHub on ``10.0.1.2:443`` with **https**, you -would enter: - - jupyterhub --ip 10.0.1.2 --port 443 --ssl-key my_ssl.key --ssl-cert my_ssl.cert - -All configurable options are technically configurable on the command-line, -even if some are really inconvenient to type. Just replace the desired option, -`c.Class.trait`, with `--Class.trait`. For example, to configure the -`c.Spawner.notebook_dir` trait from the command-line, use the -`--Spawner.notebook_dir` option: - -```bash -jupyterhub --Spawner.notebook_dir='~/assignments' -``` - -### Load a specific config file - -You can load a specific config file with: +You can load a specific config file and start JupyterHub using: ```bash jupyterhub -f /path/to/jupyterhub_config.py ``` -See also: [general docs](http://ipython.org/ipython-doc/dev/development/config.html) -on the config system Jupyter uses. +If you have stored your configuration file in the recommended UNIX filesystem +location, `/etc/jupyterhub`, the following command will start JupyterHub using +the configuration file: -### Configuration for different deployment environments +```bash +jupyterhub -f /etc/jupyterhub/jupyterhub_config.py +``` -The default authentication and process spawning mechanisms can be replaced, -which allows plugging into a variety of authentication methods or process -control and deployment environments. Some examples, meant as illustration and -testing of this concept, are: +The IPython documentation provides additional information on the +[config system](http://ipython.readthedocs.io/en/stable/development/config.html) +that Jupyter uses. + +## Configure using command line options + +To display all command line options that are available for configuration: + +```bash + jupyterhub --help-all +``` + +Configuration using the command line options is done when launching JupyterHub. +For example, to start JupyterHub on ``10.0.1.2:443`` with **https**, you +would enter: + +```bash + jupyterhub --ip 10.0.1.2 --port 443 --ssl-key my_ssl.key --ssl-cert my_ssl.cert +``` + +All configurable options are technically configurable on the command-line, +though some are inconvenient to type. To set a particular configuration +parameter, `c.Class.trait`, use the command line option, `--Class.trait`, when +starting JupyterHub. For example, configure the `c.Spawner.notebook_dir` +trait from the command-line, use `--Spawner.notebook_dir`: + +```bash +jupyterhub --Spawner.notebook_dir='~/assignments' +``` + +## Configure for various deployment environments + +The default authentication and process spawning mechanisms can be replaced, and +specific [authenticators](docs/source/authenticators-users-basics.md) and +[spawners](docs/source/spawners-basics.md) can be set in the configuration file. +This enables JupyterHub to be used in a variety of authentication methods or process +control and deployment environments. [Some examples](docs/source/config-examples.md), +meant as illustration, are: - Using GitHub OAuth instead of PAM with [OAuthenticator](https://github.com/jupyterhub/oauthenticator) - Spawning single-user servers with Docker, using the [DockerSpawner](https://github.com/jupyterhub/dockerspawner) diff --git a/docs/source/config-examples.md b/docs/source/config-examples.md index 39eda0f7..684f0d00 100644 --- a/docs/source/config-examples.md +++ b/docs/source/config-examples.md @@ -1,30 +1,30 @@ # Configuration examples -This section provides configuration files and tips for the following -configurations: +This section provides examples, including configuration files and tips, for the +following configurations: -- Example with GitHub OAuth -- Example with nginx reverse proxy -- JupyterHub deployment on AWS with NGINX +- Using GitHub OAuth +- Using nginx reverse proxy +## Using GitHub OAuth -## Example with GitHub OAuth +In this example, we show a configuration file for a fairly standard JupyterHub +deployment with the following assumptions: -In the following example, we show a configuration files for a fairly standard JupyterHub deployment with the following assumptions: - -* JupyterHub is running on a single cloud server +* Running JupyterHub on a single cloud server * Using SSL on the standard HTTPS port 443 -* You want to use GitHub OAuth (using oauthenticator) for login -* You need the users to exist locally on the server -* You want users' notebooks to be served from `~/assignments` to allow users to browse for notebooks within - other users home directories -* You want the landing page for each user to be a Welcome.ipynb notebook in their assignments directory. +* Using GitHub OAuth (using oauthenticator) for login +* Users exist locally on the server +* Users' notebooks to be served from `~/assignments` to allow users to browse + for notebooks within other users home directories +* You want the landing page for each user to be a `Welcome.ipynb` notebook in + their assignments directory. * All runtime files are put into `/srv/jupyterhub` and log files in `/var/log`. -Let's start out with `jupyterhub_config.py`: +The `jupyterhub_config.py` file would have these settings: ```python -# jupyterhub_config.py +# jupyterhub_config.py file c = get_config() import os @@ -35,7 +35,6 @@ ssl_dir = pjoin(runtime_dir, 'ssl') if not os.path.exists(ssl_dir): os.makedirs(ssl_dir) - # Allows multiple single-server per user c.JupyterHub.allow_named_servers = True @@ -54,9 +53,9 @@ c.JupyterHub.db_url = pjoin(runtime_dir, 'jupyterhub.sqlite') c.JupyterHub.extra_log_file = '/var/log/jupyterhub.log' # use GitHub OAuthenticator for local users - c.JupyterHub.authenticator_class = 'oauthenticator.LocalGitHubOAuthenticator' c.GitHubOAuthenticator.oauth_callback_url = os.environ['OAUTH_CALLBACK_URL'] + # create system users that don't exist yet c.LocalAuthenticator.create_system_users = True @@ -72,34 +71,42 @@ c.Spawner.notebook_dir = '~/assignments' c.Spawner.args = ['--NotebookApp.default_url=/notebooks/Welcome.ipynb'] ``` -Using the GitHub Authenticator [requires a few additional env variables][oauth-setup], -which we will need to set when we launch the server: +In addition, using the GitHub Authenticator requires a few additional +environment variable to be set prior to launching JupyterHub: ```bash export GITHUB_CLIENT_ID=github_id export GITHUB_CLIENT_SECRET=github_secret export OAUTH_CALLBACK_URL=https://example.com/hub/oauth_callback export CONFIGPROXY_AUTH_TOKEN=super-secret -jupyterhub -f /path/to/aboveconfig.py +jupyterhub -f /etc/jupyterhub/jupyterhub_config.py ``` -## Example with nginx reverse proxy +## Using nginx reverse proxy -In the following example, we show configuration files for a JupyterHub server running locally on port `8000` but accessible from the outside on the standard SSL port `443`. This could be useful if the JupyterHub server machine is also hosting other domains or content on `443`. The goal here is to have the following be true: +In the following example, we show configuration files for a JupyterHub server +running locally on port `8000` but accessible from the outside on the standard +SSL port `443`. This could be useful if the JupyterHub server machine is also +hosting other domains or content on `443`. The goal in this example is to +satisfy the following: * JupyterHub is running on a server, accessed *only* via `HUB.DOMAIN.TLD:443` -* On the same machine, `NO_HUB.DOMAIN.TLD` strictly serves different content, also on port `443` -* `nginx` is used to manage the web servers / reverse proxy (which means that only nginx will be able to bind two servers to `443`) -* After testing, the server in question should be able to score an A+ on the Qualys SSL Labs [SSL Server Test](https://www.ssllabs.com/ssltest/) +* On the same machine, `NO_HUB.DOMAIN.TLD` strictly serves different content, + also on port `443` +* `nginx` is used to manage the web servers / reverse proxy (which means that + only nginx will be able to bind two servers to `443`) +* After testing, the server in question should be able to score an A+ on the + Qualys SSL Labs [SSL Server Test](https://www.ssllabs.com/ssltest/) -Let's start out with `jupyterhub_config.py`: +Let's start out with needed JupyterHub configuration in `jupyterhub_config.py`: ```python # Force the proxy to only listen to connections to 127.0.0.1 c.JupyterHub.ip = '127.0.0.1' ``` -The `nginx` server config files are fairly standard fare except for the two `location` blocks within the `HUB.DOMAIN.TLD` config file: +The **`nginx` server config file** is fairly standard fare except for the two +`location` blocks within the `HUB.DOMAIN.TLD` config file: ```bash # HTTP server to redirect all 80 traffic to SSL/HTTPS @@ -162,7 +169,11 @@ server { } ``` -`nginx` will now be the front facing element of JupyterHub on `443` which means it is also free to bind other servers, like `NO_HUB.DOMAIN.TLD` to the same port on the same machine and network interface. In fact, one can simply use the same server blocks as above for `NO_HUB` and simply add line for the root directory of the site as well as the applicable location call: +`nginx` will now be the front facing element of JupyterHub on `443` which means +it is also free to bind other servers, like `NO_HUB.DOMAIN.TLD` to the same port +on the same machine and network interface. In fact, one can simply use the same +server blocks as above for `NO_HUB` and simply add line for the root directory +of the site as well as the applicable location call: ```bash server { @@ -195,4 +206,6 @@ server { } ``` -Now just restart `nginx`, restart the JupyterHub, and enjoy accessing https://HUB.DOMAIN.TLD while serving other content securely on https://NO_HUB.DOMAIN.TLD. +Now just restart `nginx`, restart the JupyterHub, and enjoy accessing +https://HUB.DOMAIN.TLD while serving other content securely on +https://NO_HUB.DOMAIN.TLD. From 265479496899da3bc0b0a439d540e44f377c5d58 Mon Sep 17 00:00:00 2001 From: Min RK Date: Mon, 24 Jul 2017 14:40:29 +0200 Subject: [PATCH 2/8] don't salt & extra hash uuids They have enough entropy on their own, so use just the hash and no salt. ref: https://security.stackexchange.com/a/151262/155114 --- jupyterhub/orm.py | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/jupyterhub/orm.py b/jupyterhub/orm.py index 7ddc4f91..58fe5857 100644 --- a/jupyterhub/orm.py +++ b/jupyterhub/orm.py @@ -268,7 +268,19 @@ class Hashed(object): def token(self, token): """Store the hashed value and prefix for a token""" self.prefix = token[:self.prefix_length] - self.hashed = hash_token(token, rounds=self.rounds, salt=self.salt_bytes, algorithm=self.algorithm) + if len(token) >= 32: + # Tokens are generally UUIDs, which have sufficient entropy on their own + # and don't need salt & hash rounds. + # ref: https://security.stackexchange.com/a/151262/155114 + rounds = 1 + salt_bytes = b'' + else: + # users can still specify API tokens in a few ways, + # so trigger salt & hash rounds if they provide a short token + app_log.warning("Applying salt & hash rounds to %sB token" % len(token)) + rounds = self.rounds + salt_bytes = self.salt_bytes + self.hashed = hash_token(token, rounds=rounds, salt=salt_bytes, algorithm=self.algorithm) def match(self, token): """Is this my token?""" From 639ccf5582553e2d22a5ea6f5e4ded7822e37869 Mon Sep 17 00:00:00 2001 From: Min RK Date: Mon, 24 Jul 2017 17:27:19 +0200 Subject: [PATCH 3/8] index prefix column --- jupyterhub/orm.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jupyterhub/orm.py b/jupyterhub/orm.py index 58fe5857..61461e68 100644 --- a/jupyterhub/orm.py +++ b/jupyterhub/orm.py @@ -427,7 +427,7 @@ class OAuthAccessToken(Hashed, Base): # from Hashed hashed = Column(Unicode(64)) - prefix = Column(Unicode(16)) + prefix = Column(Unicode(16), index=True) def __repr__(self): return "<{cls}('{prefix}...', user='{user}'>".format( From 2da115f5c4976f8dc025fe557f2041fba7d6623a Mon Sep 17 00:00:00 2001 From: Carol Willing Date: Mon, 24 Jul 2017 08:32:12 -0700 Subject: [PATCH 4/8] Reformat docstring causing Sphinx error --- jupyterhub/spawner.py | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/jupyterhub/spawner.py b/jupyterhub/spawner.py index 6bb3cdb5..5f4ccd91 100644 --- a/jupyterhub/spawner.py +++ b/jupyterhub/spawner.py @@ -129,7 +129,7 @@ class Spawner(LoggingConfigurable): The surrounding `
` element and the submit button are already provided. For example: - + .. code:: html Set your key: @@ -353,18 +353,18 @@ class Spawner(LoggingConfigurable): help=""" An optional hook function that you can implement to do some bootstrapping work before the spawner starts. For example, create a directory for your user or load initial content. - - This can be set independent of any concrete spawner implementation. - - Example: - - from subprocess import check_call - def my_hook(spawner): - username = spawner.user.name - check_call(['./examples/bootstrap-script/bootstrap.sh', username]) - c.Spawner.pre_spawn_hook = my_hook - + This can be set independent of any concrete spawner implementation. + + Example:: + + from subprocess import check_call + def my_hook(spawner): + username = spawner.user.name + check_call(['./examples/bootstrap-script/bootstrap.sh', username]) + + c.Spawner.pre_spawn_hook = my_hook + """ ).tag(config=True) @@ -452,7 +452,7 @@ class Spawner(LoggingConfigurable): env['JUPYTERHUB_HOST'] = self.hub.public_host env['JUPYTERHUB_OAUTH_CALLBACK_URL'] = \ url_path_join(self.user.url, 'oauth_callback') - + # Info previously passed on args env['JUPYTERHUB_USER'] = self.user.name env['JUPYTERHUB_API_URL'] = self.hub.api_url @@ -870,7 +870,7 @@ class LocalProcessSpawner(Spawner): cmd = self.shell_cmd + [' '.join(pipes.quote(s) for s in cmd)] self.log.info("Spawning %s", ' '.join(pipes.quote(s) for s in cmd)) - + popen_kwargs = dict( preexec_fn=self.make_preexec_fn(self.user.name), start_new_session=True, # don't forward signals From c0b482e68c996e30b4c406d1b8951a667a0a1da9 Mon Sep 17 00:00:00 2001 From: Carol Willing Date: Mon, 24 Jul 2017 08:57:25 -0700 Subject: [PATCH 5/8] Fix typos --- docs/source/config-basics.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/docs/source/config-basics.md b/docs/source/config-basics.md index 4bcd4382..9b6b9401 100644 --- a/docs/source/config-basics.md +++ b/docs/source/config-basics.md @@ -13,8 +13,8 @@ This section will help you learn how to: ## Generate a default config file -On startup, JupyterHub will look by default for a configuration file named -`jupyterhub_config.py` in the current working directory. +On startup, JupyterHub will look by default for a configuration file, +`jupyterhub_config.py`, in the current working directory. To generate a default config file, `jupyterhub_config.py`: @@ -56,18 +56,19 @@ To display all command line options that are available for configuration: ``` Configuration using the command line options is done when launching JupyterHub. -For example, to start JupyterHub on ``10.0.1.2:443`` with **https**, you +For example, to start JupyterHub on ``10.0.1.2:443`` with https, you would enter: ```bash jupyterhub --ip 10.0.1.2 --port 443 --ssl-key my_ssl.key --ssl-cert my_ssl.cert ``` -All configurable options are technically configurable on the command-line, +All configurable options may technically be set on the command-line, though some are inconvenient to type. To set a particular configuration -parameter, `c.Class.trait`, use the command line option, `--Class.trait`, when -starting JupyterHub. For example, configure the `c.Spawner.notebook_dir` -trait from the command-line, use `--Spawner.notebook_dir`: +parameter, `c.Class.trait`, you would use the command line option, +`--Class.trait`, when starting JupyterHub. For example, to configure the +`c.Spawner.notebook_dir` trait from the command-line, use the +`--Spawner.notebook_dir` option: ```bash jupyterhub --Spawner.notebook_dir='~/assignments' @@ -78,8 +79,8 @@ jupyterhub --Spawner.notebook_dir='~/assignments' The default authentication and process spawning mechanisms can be replaced, and specific [authenticators](docs/source/authenticators-users-basics.md) and [spawners](docs/source/spawners-basics.md) can be set in the configuration file. -This enables JupyterHub to be used in a variety of authentication methods or process -control and deployment environments. [Some examples](docs/source/config-examples.md), +This enables JupyterHub to be used with a variety of authentication methods or +process control and deployment environments. [Some examples](docs/source/config-examples.md), meant as illustration, are: - Using GitHub OAuth instead of PAM with [OAuthenticator](https://github.com/jupyterhub/oauthenticator) From f37243169a9d212b3943785f16c4bcd600f2655a Mon Sep 17 00:00:00 2001 From: Carol Willing Date: Mon, 24 Jul 2017 09:07:14 -0700 Subject: [PATCH 6/8] Update links --- docs/source/config-basics.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/source/config-basics.md b/docs/source/config-basics.md index 9b6b9401..3ba24860 100644 --- a/docs/source/config-basics.md +++ b/docs/source/config-basics.md @@ -1,7 +1,7 @@ # Configuration Basics The section contains basic information about configuring settings for a JupyterHub -deployment. The [configuration reference](docs/source/configuration-guide.rst) +deployment. The [configuration reference](./configuration-guide.html) provides additional detail. This section will help you learn how to: @@ -77,10 +77,10 @@ jupyterhub --Spawner.notebook_dir='~/assignments' ## Configure for various deployment environments The default authentication and process spawning mechanisms can be replaced, and -specific [authenticators](docs/source/authenticators-users-basics.md) and -[spawners](docs/source/spawners-basics.md) can be set in the configuration file. +specific [authenticators](./authenticators-users-basics.html) and +[spawners](./spawners-basics.html) can be set in the configuration file. This enables JupyterHub to be used with a variety of authentication methods or -process control and deployment environments. [Some examples](docs/source/config-examples.md), +process control and deployment environments. [Some examples](./config-examples.html), meant as illustration, are: - Using GitHub OAuth instead of PAM with [OAuthenticator](https://github.com/jupyterhub/oauthenticator) From bcdcf4351d2460e0d684ec92c6992c1c710c40e7 Mon Sep 17 00:00:00 2001 From: Carol Willing Date: Tue, 25 Jul 2017 07:04:47 -0700 Subject: [PATCH 7/8] Fix link to direct to docs not source --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2e330d77..2a31f24d 100644 --- a/README.md +++ b/README.md @@ -99,7 +99,7 @@ more configuration of the system. ## Configuration -The [Getting Started](docs/source/getting-started.md) section of the +The [Getting Started](http://jupyterhub.readthedocs.io/en/latest/getting-started.html) section of the documentation explains the common steps in setting up JupyterHub. The [**JupyterHub tutorial**](https://github.com/jupyterhub/jupyterhub-tutorial) From dee55df94aad178f2d5651d001b0ccc96ad3d34c Mon Sep 17 00:00:00 2001 From: Carol Willing Date: Tue, 25 Jul 2017 07:43:48 -0700 Subject: [PATCH 8/8] Edits per @minrk review --- docs/source/config-examples.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/config-examples.md b/docs/source/config-examples.md index 684f0d00..cf05c66b 100644 --- a/docs/source/config-examples.md +++ b/docs/source/config-examples.md @@ -16,7 +16,7 @@ deployment with the following assumptions: * Using GitHub OAuth (using oauthenticator) for login * Users exist locally on the server * Users' notebooks to be served from `~/assignments` to allow users to browse - for notebooks within other users home directories + for notebooks within other users' home directories * You want the landing page for each user to be a `Welcome.ipynb` notebook in their assignments directory. * All runtime files are put into `/srv/jupyterhub` and log files in `/var/log`. @@ -71,7 +71,7 @@ c.Spawner.notebook_dir = '~/assignments' c.Spawner.args = ['--NotebookApp.default_url=/notebooks/Welcome.ipynb'] ``` -In addition, using the GitHub Authenticator requires a few additional +Using the GitHub Authenticator requires a few additional environment variable to be set prior to launching JupyterHub: ```bash