Merge branch 'master' into remove-unused-imports

This commit is contained in:
Erik Sundell
2020-10-01 12:07:37 +02:00
90 changed files with 2005 additions and 1110 deletions

View File

@@ -1,106 +1,4 @@
div#helm-chart-schema h2,
div#helm-chart-schema h3,
div#helm-chart-schema h4,
div#helm-chart-schema h5,
div#helm-chart-schema h6 {
font-family: courier new;
}
h3, h3 ~ * {
margin-left: 3% !important;
}
h4, h4 ~ * {
margin-left: 6% !important;
}
h5, h5 ~ * {
margin-left: 9% !important;
}
h6, h6 ~ * {
margin-left: 12% !important;
}
h7, h7 ~ * {
margin-left: 15% !important;
}
img.logo {
width:100%
}
.right-next {
float: right;
max-width: 45%;
overflow: auto;
text-overflow: ellipsis;
white-space: nowrap;
}
.right-next::after{
content: ' »';
}
.left-prev {
float: left;
max-width: 45%;
overflow: auto;
text-overflow: ellipsis;
white-space: nowrap;
}
.left-prev::before{
content: '« ';
}
.prev-next-bottom {
margin-top: 3em;
}
.prev-next-top {
margin-bottom: 1em;
}
/* Sidebar TOC and headers */
div.sphinxsidebarwrapper div {
margin-bottom: .8em;
}
div.sphinxsidebar h3 {
font-size: 1.3em;
padding-top: 0px;
font-weight: 800;
margin-left: 0px !important;
}
div.sphinxsidebar p.caption {
font-size: 1.2em;
margin-bottom: 0px;
margin-left: 0px !important;
font-weight: 900;
color: #767676;
}
div.sphinxsidebar ul {
font-size: .8em;
margin-top: 0px;
padding-left: 3%;
margin-left: 0px !important;
}
div.relations ul {
font-size: 1em;
margin-left: 0px !important;
}
div#searchbox form {
margin-left: 0px !important;
}
/* body elements */
.toctree-wrapper span.caption-text {
color: #767676;
font-style: italic;
font-weight: 300;
/* Added to avoid logo being too squeezed */
.navbar-brand {
height: 4rem !important;
}

View File

@@ -1,16 +0,0 @@
{# Custom template for navigation.html
alabaster theme does not provide blocks for titles to
be overridden so this custom theme handles title and
toctree for sidebar
#}
<h3>{{ _('Table of Contents') }}</h3>
{{ toctree(includehidden=theme_sidebar_includehidden, collapse=theme_sidebar_collapse) }}
{% if theme_extra_nav_links %}
<hr />
<ul>
{% for text, uri in theme_extra_nav_links.items() %}
<li class="toctree-l1"><a href="{{ uri }}">{{ text }}</a></li>
{% endfor %}
</ul>
{% endif %}

View File

@@ -1,17 +0,0 @@
{# Custom template for relations.html
alabaster theme does not provide previous/next page by default
#}
<div class="relations">
<h3>Navigation</h3>
<ul>
<li><a href="{{ pathto(master_doc) }}">Documentation Home</a><ul>
{%- if prev %}
<li><a href="{{ prev.link|e }}" title="Previous">Previous topic</a></li>
{%- endif %}
{%- if next %}
<li><a href="{{ next.link|e }}" title="Next">Next topic</a></li>
{%- endif %}
</ul>
</ul>
</div>

View File

@@ -7,6 +7,8 @@ command line for details.
## [Unreleased]
## 1.1
### [1.1.0] 2020-01-17
@@ -116,7 +118,7 @@ Thanks to everyone who has contributed to this release!
- Log JupyterHub version on startup [#2752](https://github.com/jupyterhub/jupyterhub/pull/2752) ([@consideRatio](https://github.com/consideRatio))
- Reduce verbosity for "Failing suspected API request to not-running server" (new) [#2751](https://github.com/jupyterhub/jupyterhub/pull/2751) ([@rkdarst](https://github.com/rkdarst))
- Add missing package for json schema doc build [#2744](https://github.com/jupyterhub/jupyterhub/pull/2744) ([@willingc](https://github.com/willingc))
- blacklist urllib3 versions with encoding bug [#2743](https://github.com/jupyterhub/jupyterhub/pull/2743) ([@minrk](https://github.com/minrk))
- block urllib3 versions with encoding bug [#2743](https://github.com/jupyterhub/jupyterhub/pull/2743) ([@minrk](https://github.com/minrk))
- Remove tornado deprecated/unnecessary AsyncIOMainLoop().install() call [#2740](https://github.com/jupyterhub/jupyterhub/pull/2740) ([@kinow](https://github.com/kinow))
- Fix deprecated call [#2739](https://github.com/jupyterhub/jupyterhub/pull/2739) ([@kinow](https://github.com/kinow))
- Remove duplicate hub and authenticator traitlets from Spawner [#2736](https://github.com/jupyterhub/jupyterhub/pull/2736) ([@eslavich](https://github.com/eslavich))
@@ -231,8 +233,8 @@ whether it was through discussion, testing, documentation, or development.
This hook may transform the return value of `Authenticator.authenticate()`
and return a new authentication dictionary,
e.g. specifying admin privileges, group membership,
or custom white/blacklisting logic.
This hook is called *after* existing normalization and whitelist checking.
or custom allowed/blocked logic.
This hook is called *after* existing normalization and allowed-username checking.
- `Spawner.options_from_form` may now be async
- Added `JupyterHub.shutdown_on_logout` option to trigger shutdown of a user's
servers when they log out.
@@ -418,7 +420,7 @@ and tornado < 5.0.
launching an IPython session connected to your JupyterHub database.
- Include `User.auth_state` in user model on single-user REST endpoints for admins only.
- Include `Server.state` in server model on REST endpoints for admins only.
- Add `Authenticator.blacklist` for blacklisting users instead of whitelisting.
- Add `Authenticator.blacklist` for blocking users instead of allowing.
- Pass `c.JupyterHub.tornado_settings['cookie_options']` down to Spawners
so that cookie options (e.g. `expires_days`) can be set globally for the whole application.
- SIGINFO (`ctrl-t`) handler showing the current status of all running threads,

View File

@@ -19,10 +19,9 @@ extensions = [
'autodoc_traits',
'sphinx_copybutton',
'sphinx-jsonschema',
'recommonmark',
]
templates_path = ['_templates']
# The master toctree document.
master_doc = 'index'
@@ -58,15 +57,67 @@ default_role = 'literal'
import recommonmark
from recommonmark.transform import AutoStructify
# -- Config -------------------------------------------------------------
from jupyterhub.app import JupyterHub
from docutils import nodes
from sphinx.directives.other import SphinxDirective
from contextlib import redirect_stdout
from io import StringIO
# create a temp instance of JupyterHub just to get the output of the generate-config
# and help --all commands.
jupyterhub_app = JupyterHub()
class ConfigDirective(SphinxDirective):
"""Generate the configuration file output for use in the documentation."""
has_content = False
required_arguments = 0
optional_arguments = 0
final_argument_whitespace = False
option_spec = {}
def run(self):
# The generated configuration file for this version
generated_config = jupyterhub_app.generate_config_file()
# post-process output
home_dir = os.environ['HOME']
generated_config = generated_config.replace(home_dir, '$HOME', 1)
par = nodes.literal_block(text=generated_config)
return [par]
class HelpAllDirective(SphinxDirective):
"""Print the output of jupyterhub help --all for use in the documentation."""
has_content = False
required_arguments = 0
optional_arguments = 0
final_argument_whitespace = False
option_spec = {}
def run(self):
# The output of the help command for this version
buffer = StringIO()
with redirect_stdout(buffer):
jupyterhub_app.print_help('--help-all')
all_help = buffer.getvalue()
# post-process output
home_dir = os.environ['HOME']
all_help = all_help.replace(home_dir, '$HOME', 1)
par = nodes.literal_block(text=all_help)
return [par]
def setup(app):
app.add_config_value('recommonmark_config', {'enable_eval_rst': True}, True)
app.add_stylesheet('custom.css')
app.add_css_file('custom.css')
app.add_transform(AutoStructify)
app.add_directive('jupyterhub-generate-config', ConfigDirective)
app.add_directive('jupyterhub-help-all', HelpAllDirective)
source_parsers = {'.md': 'recommonmark.parser.CommonMarkParser'}
source_suffix = ['.rst', '.md']
# source_encoding = 'utf-8-sig'
@@ -165,10 +216,10 @@ intersphinx_mapping = {'https://docs.python.org/3/': None}
on_rtd = os.environ.get('READTHEDOCS', None) == 'True'
if on_rtd:
# readthedocs.org uses their theme by default, so no need to specify it
# build rest-api, since RTD doesn't run make
# build both metrics and rest-api, since RTD doesn't run make
from subprocess import check_call as sh
sh(['make', 'rest-api'], cwd=docs)
sh(['make', 'metrics', 'rest-api'], cwd=docs)
# -- Spell checking -------------------------------------------------------

View File

@@ -83,7 +83,6 @@ these will be moved at a future review of the roadmap.
- (prometheus?) API for resource monitoring
- tracking activity on single-user servers instead of the proxy
- notes and activity tracking per API token
- UI for managing named servers
### Later

View File

@@ -4,23 +4,23 @@ The default Authenticator uses [PAM][] to authenticate system users with
their username and password. With the default Authenticator, any user
with an account and password on the system will be allowed to login.
## Create a whitelist of users
## Create a set of allowed users
You can restrict which users are allowed to login with a whitelist,
`Authenticator.whitelist`:
You can restrict which users are allowed to login with a set,
`Authenticator.allowed_users`:
```python
c.Authenticator.whitelist = {'mal', 'zoe', 'inara', 'kaylee'}
c.Authenticator.allowed_users = {'mal', 'zoe', 'inara', 'kaylee'}
```
Users in the whitelist are added to the Hub database when the Hub is
Users in the `allowed_users` set are added to the Hub database when the Hub is
started.
## Configure admins (`admin_users`)
Admin users of JupyterHub, `admin_users`, can add and remove users from
the user `whitelist`. `admin_users` can take actions on other users'
the user `allowed_users` set. `admin_users` can take actions on other users'
behalf, such as stopping and restarting their servers.
A set of initial admin users, `admin_users` can configured be as follows:
@@ -28,7 +28,7 @@ A set of initial admin users, `admin_users` can configured be as follows:
```python
c.Authenticator.admin_users = {'mal', 'zoe'}
```
Users in the admin list are automatically added to the user `whitelist`,
Users in the admin set are automatically added to the user `allowed_users` set,
if they are not already present.
Each authenticator may have different ways of determining whether a user is an
@@ -53,12 +53,12 @@ sure your users know if admin_access is enabled.**
Users can be added to and removed from the Hub via either the admin
panel or the REST API. When a user is **added**, the user will be
automatically added to the whitelist and database. Restarting the Hub
will not require manually updating the whitelist in your config file,
automatically added to the allowed users set and database. Restarting the Hub
will not require manually updating the allowed users set in your config file,
as the users will be loaded from the database.
After starting the Hub once, it is not sufficient to **remove** a user
from the whitelist in your config file. You must also remove the user
from the allowed users set in your config file. You must also remove the user
from the Hub's database, either by deleting the user from JupyterHub's
admin page, or you can clear the `jupyterhub.sqlite` database and start
fresh.

View File

@@ -7,7 +7,7 @@ It can be used in a class of students, a corporate data science group or scienti
research group. It is a multi-user **Hub** that spawns, manages, and proxies multiple
instances of the single-user `Jupyter notebook`_ server.
To make life easier, JupyterHub have distributions. Be sure to
To make life easier, JupyterHub has distributions. Be sure to
take a look at them before continuing with the configuration of the broad
original system of `JupyterHub`_. Today, you can find two main cases:

View File

@@ -26,6 +26,10 @@ Before installing JupyterHub, you will need:
The `nodejs-legacy` package installs the `node` executable and is currently
required for npm to work on Debian/Ubuntu.
- A [pluggable authentication module (PAM)](https://en.wikipedia.org/wiki/Pluggable_authentication_module)
to use the [default Authenticator](./getting-started/authenticators-users-basics.md).
PAM is often available by default on most distributions, if this is not the case it can be installed by
using the operating system's package manager.
- TLS certificate and key for HTTPS communication
- Domain name

View File

@@ -52,7 +52,7 @@ c.GitHubOAuthenticator.oauth_callback_url = os.environ['OAUTH_CALLBACK_URL']
c.LocalAuthenticator.create_system_users = True
# specify users and admin
c.Authenticator.whitelist = {'rgbkrk', 'minrk', 'jhamrick'}
c.Authenticator.allowed_users = {'rgbkrk', 'minrk', 'jhamrick'}
c.Authenticator.admin_users = {'jhamrick', 'rgbkrk'}
# uses the default spawner

View File

@@ -83,8 +83,11 @@ server {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# websocket headers
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_buffering off;
}
# Managing requests to verify letsencrypt host
@@ -139,6 +142,20 @@ Now restart `nginx`, restart the JupyterHub, and enjoy accessing
`https://HUB.DOMAIN.TLD` while serving other content securely on
`https://NO_HUB.DOMAIN.TLD`.
### SELinux permissions for nginx
On distributions with SELinux enabled (e.g. Fedora), one may encounter permission errors
when the nginx service is started.
We need to allow nginx to perform network relay and connect to the jupyterhub port. The
following commands do that:
```bash
semanage port -a -t http_port_t -p tcp 8000
setsebool -P httpd_can_network_relay 1
setsebool -P httpd_can_network_connect 1
```
Replace 8000 with the port the jupyterhub server is running from.
## Apache
@@ -199,8 +216,8 @@ In case of the need to run the jupyterhub under /jhub/ or other location please
httpd.conf amendments:
```bash
RewriteRule /jhub/(.*) ws://127.0.0.1:8000/jhub/$1 [P,L]
RewriteRule /jhub/(.*) http://127.0.0.1:8000/jhub/$1 [P,L]
RewriteRule /jhub/(.*) ws://127.0.0.1:8000/jhub/$1 [NE.P,L]
RewriteRule /jhub/(.*) http://127.0.0.1:8000/jhub/$1 [NE,P,L]
ProxyPass /jhub/ http://127.0.0.1:8000/jhub/
ProxyPassReverse /jhub/ http://127.0.0.1:8000/jhub/

View File

@@ -0,0 +1,30 @@
==============================
Configuration Reference
==============================
.. important::
Make sure the version of JupyterHub for this documentation matches your
installation version, as the output of this command may change between versions.
JupyterHub configuration
------------------------
As explained in the `Configuration Basics <../getting-started/config-basics.html#generate-a-default-config-file>`_
section, the ``jupyterhub_config.py`` can be automatically generated via
.. code-block:: bash
jupyterhub --generate-config
The following contains the output of that command for reference.
.. jupyterhub-generate-config::
JupyterHub help command output
------------------------------
This section contains the output of the command ``jupyterhub --help-all``.
.. jupyterhub-help-all::

View File

@@ -57,7 +57,7 @@ To do this we add to `/etc/sudoers` (use `visudo` for safe editing of sudoers):
For example:
```bash
# comma-separated whitelist of users that can spawn single-user servers
# comma-separated list of users that can spawn single-user servers
# this should include all of your Hub users
Runas_Alias JUPYTER_USERS = rhea, zoe, wash

View File

@@ -16,6 +16,7 @@ what happens under-the-hood when you deploy and configure your JupyterHub.
proxy
separate-proxy
rest
monitoring
database
templates
../events/index
@@ -24,3 +25,4 @@ what happens under-the-hood when you deploy and configure your JupyterHub.
config-ghoauth
config-proxy
config-sudo
config-reference

View File

@@ -0,0 +1,20 @@
Monitoring
==========
This section covers details on monitoring the state of your JupyterHub installation.
JupyterHub expose the ``/metrics`` endpoint that returns text describing its current
operational state formatted in a way `Prometheus <https://prometheus.io/docs/introduction/overview/>`_ understands.
Prometheus is a separate open source tool that can be configured to repeatedly poll
JupyterHub's ``/metrics`` endpoint to parse and save its current state.
By doing so, Prometheus can describe JupyterHub's evolving state over time.
This evolving state can then be accessed through Prometheus that expose its underlying
storage to those allowed to access it, and be presented with dashboards by a
tool like `Grafana <https://grafana.com/docs/grafana/latest/getting-started/what-is-grafana/>`_.
.. toctree::
:maxdepth: 2
metrics

View File

@@ -57,6 +57,9 @@ generating an API token is available from the JupyterHub user interface:
## Add API tokens to the config file
**This is deprecated. We are in no rush to remove this feature,
but please consider if service tokens are right for you.**
You may also add a dictionary of API tokens and usernames to the hub's
configuration file, `jupyterhub_config.py` (note that
the **key** is the 'secret-token' while the **value** is the 'username'):
@@ -67,6 +70,41 @@ c.JupyterHub.api_tokens = {
}
```
### Updating to admin services
The `api_tokens` configuration has been softly deprecated since the introduction of services.
We have no plans to remove it,
but users are encouraged to use service configuration instead.
If you have been using `api_tokens` to create an admin user
and a token for that user to perform some automations,
the services mechanism may be a better fit.
If you have the following configuration:
```python
c.JupyterHub.admin_users = {"service-admin",}
c.JupyterHub.api_tokens = {
"secret-token": "service-admin",
}
```
This can be updated to create an admin service, with the following configuration:
```python
c.JupyterHub.services = [
{
"name": "service-token",
"admin": True,
"api_token": "secret-token",
},
]
```
The token will have the same admin permissions,
but there will no longer be a user account created to house it.
The main noticeable difference is that there will be no notebook server associated with the account
and the service will not show up in the various user list pages and APIs.
## Make an API request
To authenticate your requests, pass the API token in the request's

View File

@@ -151,6 +151,8 @@ c.JupyterHub.services = [
{
'name': 'my-web-service',
'url': 'https://10.0.1.1:1984',
# any secret >8 characters, you'll use api_token to
# authenticate api requests to the hub from your service
'api_token': 'super-secret',
}
]
@@ -313,7 +315,7 @@ class MyHandler(HubAuthenticated, web.RequestHandler):
The HubAuth will automatically load the desired configuration from the Service
environment variables.
If you want to limit user access, you can whitelist users through either the
If you want to limit user access, you can specify allowed users through either the
`.hub_users` attribute or `.hub_groups`. These are sets that check against the
username and user group list, respectively. If a user matches neither the user
list nor the group list, they will not be allowed access. If both are left
@@ -331,7 +333,9 @@ and taking note of the following process:
1. retrieve the cookie `jupyterhub-services` from the request.
2. Make an API request `GET /hub/api/authorizations/cookie/jupyterhub-services/cookie-value`,
where cookie-value is the url-encoded value of the `jupyterhub-services` cookie.
This request must be authenticated with a Hub API token in the `Authorization` header.
This request must be authenticated with a Hub API token in the `Authorization` header,
for example using the `api_token` from your [external service's configuration](#externally-managed-services).
For example, with [requests][]:
```python

View File

@@ -27,8 +27,8 @@ Some examples include:
servers using batch systems
- [YarnSpawner](https://github.com/jupyterhub/yarnspawner) for spawning notebook
servers in YARN containers on a Hadoop cluster
- [RemoteSpawner](https://github.com/zonca/remotespawner) to spawn notebooks
and a remote server and tunnel the port via SSH
- [SSHSpawner](https://github.com/NERSC/sshspawner) to spawn notebooks
on a remote server using SSH
## Spawner control methods

View File

@@ -7,8 +7,8 @@ problem and how to resolve it.
[*Behavior*](#behavior)
- JupyterHub proxy fails to start
- sudospawner fails to run
- What is the default behavior when none of the lists (admin, whitelist,
group whitelist) are set?
- What is the default behavior when none of the lists (admin, allowed,
allowed groups) are set?
- JupyterHub Docker container not accessible at localhost
[*Errors*](#errors)
@@ -55,14 +55,14 @@ or add:
to the config file, `jupyterhub_config.py`.
### What is the default behavior when none of the lists (admin, whitelist, group whitelist) are set?
### What is the default behavior when none of the lists (admin, allowed, allowed groups) are set?
When nothing is given for these lists, there will be no admins, and all users
who can authenticate on the system (i.e. all the unix users on the server with
a password) will be allowed to start a server. The whitelist lets you limit
this to a particular set of users, and the admin_users lets you specify who
a password) will be allowed to start a server. The allowed username set lets you limit
this to a particular set of users, and admin_users lets you specify who
among them may use the admin interface (not necessary, unless you need to do
things like inspect other users' servers, or modify the userlist at runtime).
things like inspect other users' servers, or modify the user list at runtime).
### JupyterHub Docker container not accessible at localhost
@@ -152,7 +152,7 @@ You should see a similar 200 message, as above, in the Hub log when you first
visit your single-user notebook server. If you don't see this message in the log, it
may mean that your single-user notebook server isn't connecting to your Hub.
If you see 403 (forbidden) like this, it's a token problem:
If you see 403 (forbidden) like this, it's likely a token problem:
```
403 GET /hub/api/authorizations/cookie/jupyterhub-token-name/[secret] (@10.0.1.4) 4.14ms
@@ -196,6 +196,10 @@ After this, when you start your server via JupyterHub, it will build a
new container. If this was the underlying cause of the issue, you should see
your server again.
##### Proxy settings (403 GET)
When your whole JupyterHub sits behind a organization proxy (*not* a reverse proxy like NGINX as part of your setup and *not* the configurable-http-proxy) the environment variables `HTTP_PROXY`, `HTTPS_PROXY`, `http_proxy` and `https_proxy` might be set. This confuses the jupyterhub-singleuser servers: When connecting to the Hub for authorization they connect via the proxy instead of directly connecting to the Hub on localhost. The proxy might deny the request (403 GET). This results in the singleuser server thinking it has a wrong auth token. To circumvent this you should add `<hub_url>,<hub_ip>,localhost,127.0.0.1` to the environment variables `NO_PROXY` and `no_proxy`.
### Launching Jupyter Notebooks to run as an externally managed JupyterHub service with the `jupyterhub-singleuser` command returns a `JUPYTERHUB_API_TOKEN` error
[JupyterHub services](https://jupyterhub.readthedocs.io/en/stable/reference/services.html) allow processes to interact with JupyterHub's REST API. Example use-cases include:
@@ -328,8 +332,7 @@ notebook servers to default to JupyterLab:
### How do I set up JupyterHub for a workshop (when users are not known ahead of time)?
1. Set up JupyterHub using OAuthenticator for GitHub authentication
2. Configure whitelist to be an empty list in` jupyterhub_config.py`
3. Configure admin list to have workshop leaders be listed with administrator privileges.
2. Configure admin list to have workshop leaders be listed with administrator privileges.
Users will need a GitHub account to login and be authenticated by the Hub.