Bump to 2.2.1

changelog for 2.2.1

.pre-commit-config.yaml

@@ -6,11 +6,11 @@ repos:
         args:
           - --py36-plus
   - repo: https://github.com/asottile/reorder_python_imports
-    rev: v2.6.0
+    rev: v2.7.1
     hooks:
       - id: reorder-python-imports
   - repo: https://github.com/psf/black
-    rev: 21.12b0
+    rev: 22.1.0
     hooks:
       - id: black
   - repo: https://github.com/pre-commit/mirrors-prettier

docs/source/_static/rest-api.yml

@@ -6,7 +6,7 @@ info:
   description: The REST API for JupyterHub
   license:
     name: BSD-3-Clause
-  version: 2.1.1
+  version: 2.2.1
 servers:
   - url: /hub/api
 security:

docs/source/admin/log-messages.md

@@ -0,0 +1,37 @@
+# Common log messages emitted by JupyterHub
+
+When debugging errors and outages, looking at the logs emitted by
+JupyterHub is very helpful. This document tries to document some common
+log messages, and what they mean.
+
+## Failing suspected API request to not-running server
+
+### Example
+
+Your logs might be littered with lines that might look slightly scary
+
+```
+[W 2022-03-10 17:25:19.774 JupyterHub base:1349] Failing suspected API request to not-running server: /hub/user/<user-name>/api/metrics/v1
+```
+
+### Most likely cause
+
+This likely means is that the user's server has stopped running but they
+still have a browser tab open. For example, you might have 3 tabs open, and shut
+your server down via one. Or you closed your laptop, your server was
+culled for inactivity, and then you reopen your laptop again! The
+client side code (JupyterLab, Classic Notebook, etc) does not know
+yet that the server is dead, and continues to make some API requests.
+JupyterHub's architecture means that the proxy routes all requests that
+don't go to a running user server to the hub process itself. The hub
+process then explicitly returns a failure response, so the client knows
+that the server is not running anymore. This is used by JupyterLab to
+tell you your server is not running anymore, and offer you the option
+to let you restart it.
+
+Most commonly, you'll see this in reference to the `/api/metrics/v1`
+URL, used by [jupyter-resource-usage](https://github.com/jupyter-server/jupyter-resource-usage).
+
+### Actions you can take
+
+This log message is benign, and there is usually no action for you to take.

docs/source/changelog.md

@@ -6,6 +6,79 @@ command line for details.
 
 ## [Unreleased]
 
+## 2.2
+
+### 2.2.1 2021-03-11
+
+2.2.1 fixes a few small regressions in 2.2.0.
+
+([full changelog](https://github.com/jupyterhub/jupyterhub/compare/2.2.0...2.2.1))
+
+#### Bugs fixed
+
+- Fix clearing cookie with custom xsrf cookie options [#3823](https://github.com/jupyterhub/jupyterhub/pull/3823) ([@minrk](https://github.com/minrk), [@consideRatio](https://github.com/consideRatio))
+- Fix admin dashboard table sorting [#3822](https://github.com/jupyterhub/jupyterhub/pull/3822) ([@NarekA](https://github.com/NarekA), [@minrk](https://github.com/minrk), [@consideRatio](https://github.com/consideRatio))
+
+#### Maintenance and upkeep improvements
+
+- allow Spawner.server to be mocked without underlying orm_spawner [#3819](https://github.com/jupyterhub/jupyterhub/pull/3819) ([@minrk](https://github.com/minrk), [@yuvipanda](https://github.com/yuvipanda), [@consideRatio](https://github.com/consideRatio))
+
+#### Documentation
+
+- Add some docs on common log messages [#3820](https://github.com/jupyterhub/jupyterhub/pull/3820) ([@yuvipanda](https://github.com/yuvipanda), [@choldgraf](https://github.com/choldgraf), [@consideRatio](https://github.com/consideRatio))
+
+#### Contributors to this release
+
+([GitHub contributors page for this release](https://github.com/jupyterhub/jupyterhub/graphs/contributors?from=2022-03-07&to=2022-03-11&type=c))
+
+[@choldgraf](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Acholdgraf+updated%3A2022-03-07..2022-03-11&type=Issues) | [@consideRatio](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3AconsideRatio+updated%3A2022-03-07..2022-03-11&type=Issues) | [@minrk](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Aminrk+updated%3A2022-03-07..2022-03-11&type=Issues) | [@NarekA](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3ANarekA+updated%3A2022-03-07..2022-03-11&type=Issues) | [@yuvipanda](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Ayuvipanda+updated%3A2022-03-07..2022-03-11&type=Issues)
+
+# 2.2.0 2021-03-07
+
+JupyterHub 2.2.0 is a small release.
+The main new feature is the ability of Authenticators to [manage group membership](authenticator-groups),
+e.g. when the identity provider has its own concept of groups that should be preserved
+in JupyterHub.
+
+The links to access user servers from the admin page have been restored.
+
+([full changelog](https://github.com/jupyterhub/jupyterhub/compare/2.1.1...2.2.0))
+
+#### New features added
+
+- Enable `options_from_form(spawner, form_data)` signature from configuration file [#3791](https://github.com/jupyterhub/jupyterhub/pull/3791) ([@rcthomas](https://github.com/rcthomas), [@minrk](https://github.com/minrk))
+- Authenticator user group management [#3548](https://github.com/jupyterhub/jupyterhub/pull/3548) ([@thomafred](https://github.com/thomafred), [@minrk](https://github.com/minrk))
+
+#### Enhancements made
+
+- Add user token to JupyterLab PageConfig [#3809](https://github.com/jupyterhub/jupyterhub/pull/3809) ([@minrk](https://github.com/minrk), [@manics](https://github.com/manics), [@consideRatio](https://github.com/consideRatio))
+- show insecure-login-warning for all authenticators [#3793](https://github.com/jupyterhub/jupyterhub/pull/3793) ([@satra](https://github.com/satra), [@minrk](https://github.com/minrk))
+- short-circuit token permission check if token and owner share role [#3792](https://github.com/jupyterhub/jupyterhub/pull/3792) ([@minrk](https://github.com/minrk), [@consideRatio](https://github.com/consideRatio))
+- Named server support, access links in admin page [#3790](https://github.com/jupyterhub/jupyterhub/pull/3790) ([@NarekA](https://github.com/NarekA), [@minrk](https://github.com/minrk), [@ykazakov](https://github.com/ykazakov), [@manics](https://github.com/manics))
+
+#### Bugs fixed
+
+- Keep Spawner.server in sync with underlying orm_spawner.server [#3810](https://github.com/jupyterhub/jupyterhub/pull/3810) ([@minrk](https://github.com/minrk), [@manics](https://github.com/manics), [@GeorgianaElena](https://github.com/GeorgianaElena), [@consideRatio](https://github.com/consideRatio))
+- Replace failed spawners when starting new launch [#3802](https://github.com/jupyterhub/jupyterhub/pull/3802) ([@minrk](https://github.com/minrk), [@consideRatio](https://github.com/consideRatio))
+- Log proxy's public_url only when started by JupyterHub [#3781](https://github.com/jupyterhub/jupyterhub/pull/3781) ([@cqzlxl](https://github.com/cqzlxl), [@consideRatio](https://github.com/consideRatio), [@minrk](https://github.com/minrk))
+
+#### Documentation improvements
+
+- Apache2 Documentation: Updates Reverse Proxy Configuration (TLS/SSL, Protocols, Headers) [#3813](https://github.com/jupyterhub/jupyterhub/pull/3813) ([@rzo1](https://github.com/rzo1), [@minrk](https://github.com/minrk))
+- Update example to not reference an undefined scope [#3812](https://github.com/jupyterhub/jupyterhub/pull/3812) ([@ktaletsk](https://github.com/ktaletsk), [@minrk](https://github.com/minrk))
+- Apache: set X-Forwarded-Proto header [#3808](https://github.com/jupyterhub/jupyterhub/pull/3808) ([@manics](https://github.com/manics), [@consideRatio](https://github.com/consideRatio), [@rzo1](https://github.com/rzo1), [@tobi45](https://github.com/tobi45))
+- idle-culler example config missing closing bracket [#3803](https://github.com/jupyterhub/jupyterhub/pull/3803) ([@tmtabor](https://github.com/tmtabor), [@consideRatio](https://github.com/consideRatio))
+
+#### Behavior Changes
+
+- Stop opening PAM sessions by default [#3787](https://github.com/jupyterhub/jupyterhub/pull/3787) ([@minrk](https://github.com/minrk), [@consideRatio](https://github.com/consideRatio))
+
+#### Contributors to this release
+
+([GitHub contributors page for this release](https://github.com/jupyterhub/jupyterhub/graphs/contributors?from=2022-01-25&to=2022-03-07&type=c))
+
+[@blink1073](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Ablink1073+updated%3A2022-01-25..2022-03-07&type=Issues) | [@clkao](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Aclkao+updated%3A2022-01-25..2022-03-07&type=Issues) | [@consideRatio](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3AconsideRatio+updated%3A2022-01-25..2022-03-07&type=Issues) | [@cqzlxl](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Acqzlxl+updated%3A2022-01-25..2022-03-07&type=Issues) | [@dependabot](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Adependabot+updated%3A2022-01-25..2022-03-07&type=Issues) | [@dtaniwaki](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Adtaniwaki+updated%3A2022-01-25..2022-03-07&type=Issues) | [@fcollonval](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Afcollonval+updated%3A2022-01-25..2022-03-07&type=Issues) | [@GeorgianaElena](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3AGeorgianaElena+updated%3A2022-01-25..2022-03-07&type=Issues) | [@github-actions](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Agithub-actions+updated%3A2022-01-25..2022-03-07&type=Issues) | [@kshitija08](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Akshitija08+updated%3A2022-01-25..2022-03-07&type=Issues) | [@ktaletsk](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Aktaletsk+updated%3A2022-01-25..2022-03-07&type=Issues) | [@manics](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Amanics+updated%3A2022-01-25..2022-03-07&type=Issues) | [@minrk](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Aminrk+updated%3A2022-01-25..2022-03-07&type=Issues) | [@NarekA](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3ANarekA+updated%3A2022-01-25..2022-03-07&type=Issues) | [@pre-commit-ci](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Apre-commit-ci+updated%3A2022-01-25..2022-03-07&type=Issues) | [@rajat404](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Arajat404+updated%3A2022-01-25..2022-03-07&type=Issues) | [@rcthomas](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Arcthomas+updated%3A2022-01-25..2022-03-07&type=Issues) | [@ryogesh](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Aryogesh+updated%3A2022-01-25..2022-03-07&type=Issues) | [@rzo1](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Arzo1+updated%3A2022-01-25..2022-03-07&type=Issues) | [@satra](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Asatra+updated%3A2022-01-25..2022-03-07&type=Issues) | [@thomafred](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Athomafred+updated%3A2022-01-25..2022-03-07&type=Issues) | [@tmtabor](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Atmtabor+updated%3A2022-01-25..2022-03-07&type=Issues) | [@tobi45](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Atobi45+updated%3A2022-01-25..2022-03-07&type=Issues) | [@ykazakov](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Aykazakov+updated%3A2022-01-25..2022-03-07&type=Issues)
+
 ## 2.1
 
 ### 2.1.1 2021-01-25

docs/source/conf.py

@@ -21,6 +21,7 @@ extensions = [
     'myst_parser',
 ]
 
+myst_heading_anchors = 2
 myst_enable_extensions = [
     'colon_fence',
     'deflist',

docs/source/index-admin.rst

@@ -10,4 +10,5 @@ well as other information relevant to running your own JupyterHub over time.
 
    troubleshooting
    admin/upgrading
+   admin/log-messages
    changelog

docs/source/reference/authenticators.md

@@ -1,6 +1,6 @@
 # Authenticators
 
-The [Authenticator][] is the mechanism for authorizing users to use the
+The {class}`.Authenticator` is the mechanism for authorizing users to use the
 Hub and single user notebook servers.
 
 ## The default PAM Authenticator
@@ -137,8 +137,8 @@ via other mechanisms. One such example is using [GitHub OAuth][].
 
 Because the username is passed from the Authenticator to the Spawner,
 a custom Authenticator and Spawner are often used together.
-For example, the Authenticator methods, [pre_spawn_start(user, spawner)][]
-and [post_spawn_stop(user, spawner)][], are hooks that can be used to do
+For example, the Authenticator methods, {meth}`.Authenticator.pre_spawn_start`
+and {meth}`.Authenticator.post_spawn_stop`, are hooks that can be used to do
 auth-related startup (e.g. opening PAM sessions) and cleanup
 (e.g. closing PAM sessions).
 
@@ -223,7 +223,7 @@ If there are multiple keys present, the **first** key is always used to persist
 
 Typically, if `auth_state` is persisted it is desirable to affect the Spawner environment in some way.
 This may mean defining environment variables, placing certificate in the user's home directory, etc.
-The `Authenticator.pre_spawn_start` method can be used to pass information from authenticator state
+The {meth}`Authenticator.pre_spawn_start` method can be used to pass information from authenticator state
 to Spawner environment:
 
 ```python
@@ -247,10 +247,42 @@ class MyAuthenticator(Authenticator):
         spawner.environment['UPSTREAM_TOKEN'] = auth_state['upstream_token']
 ```
 
+(authenticator-groups)=
+
+## Authenticator-managed group membership
+
+:::{versionadded} 2.2
+:::
+
+Some identity providers may have their own concept of group membership that you would like to preserve in JupyterHub.
+This is now possible with `Authenticator.managed_groups`.
+
+You can set the config:
+
+```python
+c.Authenticator.manage_groups = True
+```
+
+to enable this behavior.
+The default is False for Authenticators that ship with JupyterHub,
+but may be True for custom Authenticators.
+Check your Authenticator's documentation for manage_groups support.
+
+If True, {meth}`.Authenticator.authenticate` and {meth}`.Authenticator.refresh_user` may include a field `groups`
+which is a list of group names the user should be a member of:
+
+- Membership will be added for any group in the list
+- Membership in any groups not in the list will be revoked
+- Any groups not already present in the database will be created
+- If `None` is returned, no changes are made to the user's group membership
+
+If authenticator-managed groups are enabled,
+all group-management via the API is disabled.
+
 ## pre_spawn_start and post_spawn_stop hooks
 
-Authenticators uses two hooks, [pre_spawn_start(user, spawner)][] and
-[post_spawn_stop(user, spawner)][] to add pass additional state information
+Authenticators uses two hooks, {meth}`.Authenticator.pre_spawn_start` and
+{meth}`.Authenticator.post_spawn_stop(user, spawner)` to add pass additional state information
 between the authenticator and a spawner. These hooks are typically used auth-related
 startup, i.e. opening a PAM session, and auth-related cleanup, i.e. closing a
 PAM session.
@@ -259,10 +291,7 @@ PAM session.
 
 Beginning with version 0.8, JupyterHub is an OAuth provider.
 
-[authenticator]: https://github.com/jupyterhub/jupyterhub/blob/HEAD/jupyterhub/auth.py
 [pam]: https://en.wikipedia.org/wiki/Pluggable_authentication_module
 [oauth]: https://en.wikipedia.org/wiki/OAuth
 [github oauth]: https://developer.github.com/v3/oauth/
 [oauthenticator]: https://github.com/jupyterhub/oauthenticator
-[pre_spawn_start(user, spawner)]: https://jupyterhub.readthedocs.io/en/latest/api/auth.html#jupyterhub.auth.Authenticator.pre_spawn_start
-[post_spawn_stop(user, spawner)]: https://jupyterhub.readthedocs.io/en/latest/api/auth.html#jupyterhub.auth.Authenticator.post_spawn_stop

docs/source/reference/config-proxy.md

@@ -165,7 +165,7 @@ As with nginx above, you can use [Apache](https://httpd.apache.org) as the rever
 First, we will need to enable the apache modules that we are going to need:
 
 ```bash
-a2enmod ssl rewrite proxy proxy_http proxy_wstunnel
+a2enmod ssl rewrite proxy headers proxy_http proxy_wstunnel
 ```
 
 Our Apache configuration is equivalent to the nginx configuration above:
@@ -188,13 +188,24 @@ Listen 443
 
   ServerName HUB.DOMAIN.TLD
 
+  # enable HTTP/2, if available
+  Protocols h2 http/1.1
+
+  # HTTP Strict Transport Security (mod_headers is required) (63072000 seconds)
+  Header always set Strict-Transport-Security "max-age=63072000"
+
   # configure SSL
   SSLEngine on
   SSLCertificateFile /etc/letsencrypt/live/HUB.DOMAIN.TLD/fullchain.pem
   SSLCertificateKeyFile /etc/letsencrypt/live/HUB.DOMAIN.TLD/privkey.pem
-  SSLProtocol All -SSLv2 -SSLv3
   SSLOpenSSLConfCmd DHParameters /etc/ssl/certs/dhparam.pem
-  SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH
+
+  # intermediate configuration from ssl-config.mozilla.org (2022-03-03)
+  # Please note, that this configuration might be out-dated - please update it accordingly using https://ssl-config.mozilla.org/
+  SSLProtocol             all -SSLv3 -TLSv1 -TLSv1.1
+  SSLCipherSuite          ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
+  SSLHonorCipherOrder     off
+  SSLSessionTickets       off
 
   # Use RewriteEngine to handle websocket connection upgrades
   RewriteEngine On
@@ -208,6 +219,7 @@ Listen 443
     # proxy to JupyterHub
     ProxyPass         http://127.0.0.1:8000/
     ProxyPassReverse  http://127.0.0.1:8000/
+    RequestHeader     set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
   </Location>
 </VirtualHost>
 ```

docs/source/reference/rest.md

@@ -113,7 +113,6 @@ c.JupyterHub.load_roles = [
         "scopes": [
             # specify the permissions the token should have
             "admin:users",
-            "admin:services",
         ],
         "services": [
             # assign the service the above permissions

docs/source/reference/services.md

@@ -83,6 +83,7 @@ c.JupyterHub.load_roles = [
             # 'admin:users' # needed if culling idle users as well
         ]
     }
+]
 
 c.JupyterHub.services = [
     {
@@ -208,23 +209,23 @@ can be used by services. You may go beyond this reference implementation and
 create custom hub-authenticating clients and services. We describe the process
 below.
 
-The reference, or base, implementation is the [`HubAuth`][hubauth] class,
+The reference, or base, implementation is the {class}`.HubAuth` class,
 which implements the API requests to the Hub that resolve a token to a User model.
 
 There are two levels of authentication with the Hub:
 
-- [`HubAuth`][hubauth] - the most basic authentication,
+- {class}`.HubAuth` - the most basic authentication,
   for services that should only accept API requests authorized with a token.
 
-- [`HubOAuth`][huboauth] - For services that should use oauth to authenticate with the Hub.
+- {class}`.HubOAuth` - For services that should use oauth to authenticate with the Hub.
   This should be used for any service that serves pages that should be visited with a browser.
 
 To use HubAuth, you must set the `.api_token`, either programmatically when constructing the class,
 or via the `JUPYTERHUB_API_TOKEN` environment variable.
 
 Most of the logic for authentication implementation is found in the
-[`HubAuth.user_for_token`][hubauth.user_for_token]
-methods, which makes a request of the Hub, and returns:
+{meth}`.HubAuth.user_for_token` methods,
+which makes a request of the Hub, and returns:
 
 - None, if no user could be identified, or
 - a dict of the following form:
@@ -245,6 +246,19 @@ action.
 HubAuth also caches the Hub's response for a number of seconds,
 configurable by the `cookie_cache_max_age` setting (default: five minutes).
 
+If your service would like to make further requests _on behalf of users_,
+it should use the token issued by this OAuth process.
+If you are using tornado,
+you can access the token authenticating the current request with {meth}`.HubAuth.get_token`.
+
+:::{versionchanged} 2.2
+
+{meth}`.HubAuth.get_token` adds support for retrieving
+tokens stored in tornado cookies after completion of OAuth.
+Previously, it only retrieved tokens from URL parameters or the Authorization header.
+Passing `get_token(handler, in_cookie=False)` preserves this behavior.
+:::
+
 ### Flask Example
 
 For example, you have a Flask service that returns information about a user.
@@ -370,11 +384,6 @@ section on securing the notebook viewer.
 
 [requests]: http://docs.python-requests.org/en/master/
 [services_auth]: ../api/services.auth.html
-[hubauth]: ../api/services.auth.html#jupyterhub.services.auth.HubAuth
-[huboauth]: ../api/services.auth.html#jupyterhub.services.auth.HubOAuth
-[hubauth.user_for_token]: ../api/services.auth.html#jupyterhub.services.auth.HubAuth.user_for_token
-[hubauthenticated]: ../api/services.auth.html#jupyterhub.services.auth.HubAuthenticated
-[huboauthenticated]: ../api/services.auth.html#jupyterhub.services.auth.HubOAuthenticated
 [nbviewer example]: https://github.com/jupyter/nbviewer#securing-the-notebook-viewer
 [fastapi example]: https://github.com/jupyterhub/jupyterhub/tree/HEAD/examples/service-fastapi
 [fastapi]: https://fastapi.tiangolo.com

docs/source/troubleshooting.md

@@ -275,7 +275,7 @@ where `ssl_cert` is example-chained.crt and ssl_key to your private key.
 
 Then restart JupyterHub.
 
-See also [JupyterHub SSL encryption](./getting-started/security-basics.html#ssl-encryption).
+See also {ref}`ssl-encryption`.
 
 ### Install JupyterHub without a network connection
 

examples/azuread-with-group-management/jupyterhub_config.py

@@ -0,0 +1,30 @@
+"""sample jupyterhub config file for testing
+
+configures jupyterhub with dummyauthenticator and simplespawner
+to enable testing without administrative privileges.
+"""
+
+c = get_config()  # noqa
+c.Application.log_level = 'DEBUG'
+
+from oauthenticator.azuread import AzureAdOAuthenticator
+import os
+
+c.JupyterHub.authenticator_class = AzureAdOAuthenticator
+
+c.AzureAdOAuthenticator.client_id = os.getenv("AAD_CLIENT_ID")
+c.AzureAdOAuthenticator.client_secret = os.getenv("AAD_CLIENT_SECRET")
+c.AzureAdOAuthenticator.oauth_callback_url = os.getenv("AAD_CALLBACK_URL")
+c.AzureAdOAuthenticator.tenant_id = os.getenv("AAD_TENANT_ID")
+c.AzureAdOAuthenticator.username_claim = "email"
+c.AzureAdOAuthenticator.authorize_url = os.getenv("AAD_AUTHORIZE_URL")
+c.AzureAdOAuthenticator.token_url = os.getenv("AAD_TOKEN_URL")
+c.Authenticator.manage_groups = True
+c.Authenticator.refresh_pre_spawn = True
+
+# Optionally set a global password that all users must use
+# c.DummyAuthenticator.password = "your_password"
+
+from jupyterhub.spawner import SimpleLocalProcessSpawner
+
+c.JupyterHub.spawner_class = SimpleLocalProcessSpawner

examples/azuread-with-group-management/requirements.txt

@@ -0,0 +1,2 @@
+oauthenticator
+pyjwt

jsx/src/components/ServerDashboard/ServerDashboard.jsx

@@ -10,6 +10,14 @@ import "./server-dashboard.css";
 import { timeSince } from "../../util/timeSince";
 import PaginationFooter from "../PaginationFooter/PaginationFooter";
 
+const AccessServerButton = ({ userName, serverName }) => (
+  <a href={`/user/${userName}/${serverName || ""}`}>
+    <button className="btn btn-primary btn-xs" style={{ marginRight: 20 }}>
+      Access Server
+    </button>
+  </a>
+);
+
 const ServerDashboard = (props) => {
   // sort methods
   var usernameDesc = (e) => e.sort((a, b) => (a.name > b.name ? 1 : -1)),
@@ -29,6 +37,7 @@ const ServerDashboard = (props) => {
 
   var [errorAlert, setErrorAlert] = useState(null);
   var [sortMethod, setSortMethod] = useState(null);
+  var [disabledButtons, setDisabledButtons] = useState({});
 
   var user_data = useSelector((state) => state.user_data),
     user_page = useSelector((state) => state.user_page),
@@ -72,6 +81,108 @@ const ServerDashboard = (props) => {
     user_data = sortMethod(user_data);
   }
 
+  const StopServerButton = ({ serverName, userName }) => {
+    var [isDisabled, setIsDisabled] = useState(false);
+    return (
+      <button
+        className="btn btn-danger btn-xs stop-button"
+        disabled={isDisabled}
+        onClick={() => {
+          setIsDisabled(true);
+          stopServer(userName, serverName)
+            .then((res) => {
+              if (res.status < 300) {
+                updateUsers(...slice)
+                  .then((data) => {
+                    dispatchPageUpdate(data, page);
+                  })
+                  .catch(() => {
+                    setIsDisabled(false);
+                    setErrorAlert(`Failed to update users list.`);
+                  });
+              } else {
+                setErrorAlert(`Failed to stop server.`);
+                setIsDisabled(false);
+              }
+              return res;
+            })
+            .catch(() => {
+              setErrorAlert(`Failed to stop server.`);
+              setIsDisabled(false);
+            });
+        }}
+      >
+        Stop Server
+      </button>
+    );
+  };
+
+  const StartServerButton = ({ serverName, userName }) => {
+    var [isDisabled, setIsDisabled] = useState(false);
+    return (
+      <button
+        className="btn btn-success btn-xs start-button"
+        disabled={isDisabled}
+        onClick={() => {
+          setIsDisabled(true);
+          startServer(userName, serverName)
+            .then((res) => {
+              if (res.status < 300) {
+                updateUsers(...slice)
+                  .then((data) => {
+                    dispatchPageUpdate(data, page);
+                  })
+                  .catch(() => {
+                    setErrorAlert(`Failed to update users list.`);
+                    setIsDisabled(false);
+                  });
+              } else {
+                setErrorAlert(`Failed to start server.`);
+                setIsDisabled(false);
+              }
+              return res;
+            })
+            .catch(() => {
+              setErrorAlert(`Failed to start server.`);
+              setIsDisabled(false);
+            });
+        }}
+      >
+        Start Server
+      </button>
+    );
+  };
+
+  const EditUserCell = ({ user }) => {
+    return (
+      <td>
+        <button
+          className="btn btn-primary btn-xs"
+          style={{ marginRight: 20 }}
+          onClick={() =>
+            history.push({
+              pathname: "/edit-user",
+              state: {
+                username: user.name,
+                has_admin: user.admin,
+              },
+            })
+          }
+        >
+          Edit User
+        </button>
+      </td>
+    );
+  };
+
+  let servers = user_data.flatMap((user) => {
+    let userServers = Object.values({
+      "": user.server || {},
+      ...(user.servers || {}),
+    });
+    return userServers.map((server) => [user, server]);
+  });
+
   return (
     <div className="container" data-testid="container">
       {errorAlert != null ? (
@@ -115,6 +226,14 @@ const ServerDashboard = (props) => {
                   testid="admin-sort"
                 />
               </th>
+              <th id="server-header">
+                Server{" "}
+                <SortHandler
+                  sorts={{ asc: usernameAsc, desc: usernameDesc }}
+                  callback={(method) => setSortMethod(() => method)}
+                  testid="server-sort"
+                />
+              </th>
               <th id="last-activity-header">
                 Last Activity{" "}
                 <SortHandler
@@ -227,88 +346,66 @@ const ServerDashboard = (props) => {
                 </Button>
               </td>
             </tr>
-            {user_data.map((e, i) => (
-              <tr key={i + "row"} className="user-row">
-                <td data-testid="user-row-name">{e.name}</td>
-                <td data-testid="user-row-admin">{e.admin ? "admin" : ""}</td>
-                <td data-testid="user-row-last-activity">
-                  {e.last_activity ? timeSince(e.last_activity) : "Never"}
-                </td>
-                <td data-testid="user-row-server-activity">
-                  {e.server != null ? (
-                    // Stop Single-user server
-                    <button
-                      className="btn btn-danger btn-xs stop-button"
-                      onClick={() =>
-                        stopServer(e.name)
-                          .then((res) => {
-                            if (res.status < 300) {
-                              updateUsers(...slice)
-                                .then((data) => {
-                                  dispatchPageUpdate(data, page);
-                                })
-                                .catch(() =>
-                                  setErrorAlert(`Failed to update users list.`)
-                                );
-                            } else {
-                              setErrorAlert(`Failed to stop server.`);
-                            }
-                            return res;
-                          })
-                          .catch(() => setErrorAlert(`Failed to stop server.`))
-                      }
-                    >
-                      Stop Server
-                    </button>
-                  ) : (
-                    // Start Single-user server
-                    <button
-                      className="btn btn-primary btn-xs start-button"
-                      onClick={() =>
-                        startServer(e.name)
-                          .then((res) => {
-                            if (res.status < 300) {
-                              updateUsers(...slice)
-                                .then((data) => {
-                                  dispatchPageUpdate(data, page);
-                                })
-                                .catch(() =>
-                                  setErrorAlert(`Failed to update users list.`)
-                                );
-                            } else {
-                              setErrorAlert(`Failed to start server.`);
-                            }
-                            return res;
-                          })
-                          .catch(() => {
-                            setErrorAlert(`Failed to start server.`);
-                          })
-                      }
-                    >
-                      Start Server
-                    </button>
-                  )}
-                </td>
-                <td>
-                  {/* Edit User */}
-                  <button
-                    className="btn btn-primary btn-xs"
-                    style={{ marginRight: 20 }}
-                    onClick={() =>
-                      history.push({
-                        pathname: "/edit-user",
-                        state: {
-                          username: e.name,
-                          has_admin: e.admin,
-                        },
-                      })
-                    }
-                  >
-                    edit user
-                  </button>
-                </td>
-              </tr>
-            ))}
+            {servers.map(([user, server], i) => {
+              server.name = server.name || "";
+              return (
+                <tr key={i + "row"} className="user-row">
+                  <td data-testid="user-row-name">{user.name}</td>
+                  <td data-testid="user-row-admin">
+                    {user.admin ? "admin" : ""}
+                  </td>
+
+                  <td data-testid="user-row-server">
+                    {server.name ? (
+                      <p class="text-secondary">{server.name}</p>
+                    ) : (
+                      <p style={{ color: "lightgrey" }}>[MAIN]</p>
+                    )}
+                  </td>
+                  <td data-testid="user-row-last-activity">
+                    {server.last_activity
+                      ? timeSince(server.last_activity)
+                      : "Never"}
+                  </td>
+                  <td data-testid="user-row-server-activity">
+                    {server.started ? (
+                      // Stop Single-user server
+                      <>
+                        <StopServerButton
+                          serverName={server.name}
+                          userName={user.name}
+                        />
+                        <AccessServerButton
+                          serverName={server.name}
+                          userName={user.name}
+                        />
+                      </>
+                    ) : (
+                      // Start Single-user server
+                      <>
+                        <StartServerButton
+                          serverName={server.name}
+                          userName={user.name}
+                        />
+                        <a
+                          href={`/spawn/${user.name}${
+                            server.name && "/" + server.name
+                          }`}
+                        >
+                          <button
+                            className="btn btn-secondary btn-xs"
+                            style={{ marginRight: 20 }}
+                          >
+                            Spawn Page
+                          </button>
+                        </a>
+                      </>
+                    )}
+                  </td>
+                  <EditUserCell user={user} />
+                </tr>
+              );
+            })}
           </tbody>
         </table>
         <PaginationFooter

jsx/src/util/withAPI.js

@@ -11,8 +11,10 @@ const withAPI = withProps(() => ({
       (data) => data.json()
     ),
   shutdownHub: () => jhapiRequest("/shutdown", "POST"),
-  startServer: (name) => jhapiRequest("/users/" + name + "/server", "POST"),
-  stopServer: (name) => jhapiRequest("/users/" + name + "/server", "DELETE"),
+  startServer: (name, serverName = "") =>
+    jhapiRequest("/users/" + name + "/servers/" + (serverName || ""), "POST"),
+  stopServer: (name, serverName = "") =>
+    jhapiRequest("/users/" + name + "/servers/" + (serverName || ""), "DELETE"),
   startAll: (names) =>
     names.map((e) => jhapiRequest("/users/" + e + "/server", "POST")),
   stopAll: (names) =>

jsx/yarn.lock

@@ -3664,9 +3664,9 @@ flatted@^3.1.0:
   integrity sha512-zAoAQiudy+r5SvnSw3KJy5os/oRJYHzrzja/tBDqrZtNhUw8bt6y8OBzMWcjWr+8liV8Eb6yOhw8WZ7VFZ5ZzA==
 
 follow-redirects@^1.0.0:
-  version "1.14.7"
-  resolved "https://registry.yarnpkg.com/follow-redirects/-/follow-redirects-1.14.7.tgz#2004c02eb9436eee9a21446a6477debf17e81685"
-  integrity sha512-+hbxoLbFMbRKDwohX8GkTataGqO6Jb7jGwpAlwgy2bIz25XtRm7KEzJM76R1WiNT5SwZkX4Y75SwBolkpmE7iQ==
+  version "1.14.8"
+  resolved "https://registry.yarnpkg.com/follow-redirects/-/follow-redirects-1.14.8.tgz#016996fb9a11a100566398b1c6839337d7bfa8fc"
+  integrity sha512-1x0S9UVJHsQprFcEC/qnNzBLcIxsjAV905f/UkQxbclCsoTWlacCNOpQa/anodLl2uaEKFhfWOvM2Qg77+15zA==
 
 for-in@^1.0.2:
   version "1.0.2"
@@ -7878,9 +7878,9 @@ urix@^0.1.0:
   integrity sha1-2pN/emLiH+wf0Y1Js1wpNQZ6bHI=
 
 url-parse@^1.4.3:
-  version "1.5.3"
-  resolved "https://registry.yarnpkg.com/url-parse/-/url-parse-1.5.3.tgz#71c1303d38fb6639ade183c2992c8cc0686df862"
-  integrity sha512-IIORyIQD9rvj0A4CLWsHkBBJuNqWpFQe224b6j9t/ABmquIS0qDU2pY6kl6AuOrL5OkCXHMCFNe1jBcuAggjvQ==
+  version "1.5.10"
+  resolved "https://registry.yarnpkg.com/url-parse/-/url-parse-1.5.10.tgz#9d3c2f736c1d75dd3bd2be507dcc111f1e2ea9c1"
+  integrity sha512-WypcfiRhfeUP9vvF0j6rw0J3hrWrw6iZv3+22h6iRMJ/8z1Tj6XfLP4DsUix5MhMPnXpiHDoKyoZ/bdCkwBCiQ==
   dependencies:
     querystringify "^2.1.1"
     requires-port "^1.0.0"

jupyterhub/_version.py

@@ -2,7 +2,7 @@
 # Copyright (c) Jupyter Development Team.
 # Distributed under the terms of the Modified BSD License.
 # version_info updated by running `tbump`
-version_info = (2, 1, 1, "", "")
+version_info = (2, 2, 1, "", "")
 
 # pep 440 version: no dot before beta/rc, but before .dev
 # 0.1.0rc1

jupyterhub/apihandlers/groups.py

@@ -33,6 +33,11 @@ class _GroupAPIHandler(APIHandler):
             raise web.HTTPError(404, "No such group: %s", group_name)
         return group
 
+    def check_authenticator_managed_groups(self):
+        """Raise error on group-management APIs if Authenticator is managing groups"""
+        if self.authenticator.manage_groups:
+            raise web.HTTPError(400, "Group management via API is disabled")
+
 
 class GroupListAPIHandler(_GroupAPIHandler):
     @needs_scope('list:groups')
@@ -68,6 +73,9 @@ class GroupListAPIHandler(_GroupAPIHandler):
     @needs_scope('admin:groups')
     async def post(self):
         """POST creates Multiple groups"""
+
+        self.check_authenticator_managed_groups()
+
         model = self.get_json_body()
         if not model or not isinstance(model, dict) or not model.get('groups'):
             raise web.HTTPError(400, "Must specify at least one group to create")
@@ -106,6 +114,7 @@ class GroupAPIHandler(_GroupAPIHandler):
     @needs_scope('admin:groups')
     async def post(self, group_name):
         """POST creates a group by name"""
+        self.check_authenticator_managed_groups()
         model = self.get_json_body()
         if model is None:
             model = {}
@@ -132,6 +141,7 @@ class GroupAPIHandler(_GroupAPIHandler):
     @needs_scope('delete:groups')
     def delete(self, group_name):
         """Delete a group by name"""
+        self.check_authenticator_managed_groups()
         group = self.find_group(group_name)
         self.log.info("Deleting group %s", group_name)
         self.db.delete(group)
@@ -145,6 +155,7 @@ class GroupUsersAPIHandler(_GroupAPIHandler):
     @needs_scope('groups')
     def post(self, group_name):
         """POST adds users to a group"""
+        self.check_authenticator_managed_groups()
         group = self.find_group(group_name)
         data = self.get_json_body()
         self._check_group_model(data)
@@ -163,6 +174,7 @@ class GroupUsersAPIHandler(_GroupAPIHandler):
     @needs_scope('groups')
     async def delete(self, group_name):
         """DELETE removes users from a group"""
+        self.check_authenticator_managed_groups()
         group = self.find_group(group_name)
         data = self.get_json_body()
         self._check_group_model(data)

jupyterhub/apihandlers/users.py

@@ -515,7 +515,7 @@ class UserServerAPIHandler(APIHandler):
                             user_name, self.named_server_limit_per_user
                         ),
                     )
-        spawner = user.spawners[server_name]
+        spawner = user.get_spawner(server_name, replace_failed=True)
         pending = spawner.pending
         if pending == 'spawn':
             self.set_header('Content-Type', 'text/plain')

jupyterhub/app.py

@@ -2001,6 +2001,9 @@ class JupyterHub(Application):
     async def init_groups(self):
         """Load predefined groups into the database"""
         db = self.db
+
+        if self.authenticator.manage_groups and self.load_groups:
+            raise ValueError("Group management has been offloaded to the authenticator")
         for name, usernames in self.load_groups.items():
             group = orm.Group.find(db, name)
             if group is None:
@@ -3147,7 +3150,12 @@ class JupyterHub(Application):
             self.last_activity_callback = pc
             pc.start()
 
-        self.log.info("JupyterHub is now running at %s", self.proxy.public_url)
+        if self.proxy.should_start:
+            self.log.info("JupyterHub is now running at %s", self.proxy.public_url)
+        else:
+            self.log.info(
+                "JupyterHub is now running, internal Hub API at %s", self.hub.url
+            )
         # Use atexit for Windows, it doesn't have signal handling support
         if _mswindows:
             atexit.register(self.atexit)

jupyterhub/auth.py

@@ -582,9 +582,13 @@ class Authenticator(LoggingConfigurable):
                 or None if Authentication failed.
 
                 The Authenticator may return a dict instead, which MUST have a
-                key `name` holding the username, and MAY have two optional keys
-                set: `auth_state`, a dictionary of of auth state that will be
-                persisted; and `admin`, the admin setting value for the user.
+                key `name` holding the username, and MAY have additional keys:
+
+                - `auth_state`, a dictionary of of auth state that will be
+                  persisted;
+                - `admin`, the admin setting value for the user
+                - `groups`, the list of group names the user should be a member of,
+                  if Authenticator.manage_groups is True.
         """
 
     def pre_spawn_start(self, user, spawner):
@@ -635,6 +639,19 @@ class Authenticator(LoggingConfigurable):
         """
         self.allowed_users.discard(user.name)
 
+    manage_groups = Bool(
+        False,
+        config=True,
+        help="""Let authenticator manage user groups
+
+        If True, Authenticator.authenticate and/or .refresh_user
+        may return a list of group names in the 'groups' field,
+        which will be assigned to the user.
+
+        All group-assignment APIs are disabled if this is True.
+        """,
+    )
+
     auto_login = Bool(
         False,
         config=True,
@@ -958,16 +975,24 @@ class PAMAuthenticator(LocalAuthenticator):
     ).tag(config=True)
 
     open_sessions = Bool(
-        True,
+        False,
         help="""
         Whether to open a new PAM session when spawners are started.
 
-        This may trigger things like mounting shared filsystems,
-        loading credentials, etc. depending on system configuration,
-        but it does not always work.
+        This may trigger things like mounting shared filesystems,
+        loading credentials, etc. depending on system configuration.
+
+        The lifecycle of PAM sessions is not correct,
+        so many PAM session configurations will not work.
 
         If any errors are encountered when opening/closing PAM sessions,
         this is automatically set to False.
+
+        .. versionchanged:: 2.2
+
+            Due to longstanding problems in the session lifecycle,
+            this is now disabled by default.
+            You may opt-in to opening sessions by setting this to True.
         """,
     ).tag(config=True)
 

jupyterhub/handlers/base.py

@@ -526,10 +526,16 @@ class BaseHandler(RequestHandler):
             path=url_path_join(self.base_url, 'services'),
             **kwargs,
         )
-        # clear tornado cookie
+        # clear_cookie only accepts a subset of set_cookie's kwargs
+        clear_xsrf_cookie_kwargs = {
+            key: value
+            for key, value in self.settings.get('xsrf_cookie_kwargs', {})
+            if key in {"path", "domain"}
+        }
+
         self.clear_cookie(
             '_xsrf',
-            **self.settings.get('xsrf_cookie_kwargs', {}),
+            **clear_xsrf_cookie_kwargs,
         )
         # Reset _jupyterhub_user
         self._jupyterhub_user = None
@@ -774,13 +780,22 @@ class BaseHandler(RequestHandler):
         # always ensure default roles ('user', 'admin' if admin) are assigned
         # after a successful login
         roles.assign_default_roles(self.db, entity=user)
+
+        # apply authenticator-managed groups
+        if self.authenticator.manage_groups:
+            group_names = authenticated.get("groups")
+            if group_names is not None:
+                user.sync_groups(group_names)
+
         # always set auth_state and commit,
         # because there could be key-rotation or clearing of previous values
         # going on.
         if not self.authenticator.enable_auth_state:
             # auth_state is not enabled. Force None.
             auth_state = None
+
         await user.save_auth_state(auth_state)
+
         return user
 
     async def login_user(self, data=None):
@@ -794,6 +809,7 @@ class BaseHandler(RequestHandler):
             self.set_login_cookie(user)
             self.statsd.incr('login.success')
             self.statsd.timing('login.authenticate.success', auth_timer.ms)
+
             self.log.info("User logged in: %s", user.name)
             user._auth_refreshed = time.monotonic()
             return user
@@ -1510,14 +1526,10 @@ class UserUrlHandler(BaseHandler):
 
         # if request is expecting JSON, assume it's an API request and fail with 503
         # because it won't like the redirect to the pending page
-        if (
-            get_accepted_mimetype(
-                self.request.headers.get('Accept', ''),
-                choices=['application/json', 'text/html'],
-            )
-            == 'application/json'
-            or 'api' in user_path.split('/')
-        ):
+        if get_accepted_mimetype(
+            self.request.headers.get('Accept', ''),
+            choices=['application/json', 'text/html'],
+        ) == 'application/json' or 'api' in user_path.split('/'):
             self._fail_api_request(user_name, server_name)
             return
 
@@ -1599,7 +1611,7 @@ class UserUrlHandler(BaseHandler):
         if redirects:
             self.log.warning("Redirect loop detected on %s", self.request.uri)
             # add capped exponential backoff where cap is 10s
-            await asyncio.sleep(min(1 * (2 ** redirects), 10))
+            await asyncio.sleep(min(1 * (2**redirects), 10))
             # rewrite target url with new `redirects` query value
             url_parts = urlparse(target)
             query_parts = parse_qs(url_parts.query)

jupyterhub/handlers/pages.py

@@ -151,7 +151,7 @@ class SpawnHandler(BaseHandler):
             self.redirect(url)
             return
 
-        spawner = user.spawners[server_name]
+        spawner = user.get_spawner(server_name, replace_failed=True)
 
         pending_url = self._get_pending_url(user, server_name)
 
@@ -237,7 +237,7 @@ class SpawnHandler(BaseHandler):
             if user is None:
                 raise web.HTTPError(404, "No such user: %s" % for_user)
 
-        spawner = user.spawners[server_name]
+        spawner = user.get_spawner(server_name, replace_failed=True)
 
         if spawner.ready:
             raise web.HTTPError(400, "%s is already running" % (spawner._log_name))
@@ -255,7 +255,7 @@ class SpawnHandler(BaseHandler):
             self.log.debug(
                 "Triggering spawn with supplied form options for %s", spawner._log_name
             )
-            options = await maybe_future(spawner.options_from_form(form_options))
+            options = await maybe_future(spawner.run_options_from_form(form_options))
             pending_url = self._get_pending_url(user, server_name)
             return await self._wrap_spawn_single_user(
                 user, server_name, spawner, pending_url, options
@@ -369,13 +369,9 @@ class SpawnPendingHandler(BaseHandler):
         auth_state = await user.get_auth_state()
 
         # First, check for previous failure.
-        if (
-            not spawner.active
-            and spawner._spawn_future
-            and spawner._spawn_future.done()
-            and spawner._spawn_future.exception()
-        ):
-            # Condition: spawner not active and _spawn_future exists and contains an Exception
+        if not spawner.active and spawner._failed:
+            # Condition: spawner not active and last spawn failed
+            # (failure is available as spawner._spawn_future.exception()).
             # Implicit spawn on /user/:name is not allowed if the user's last spawn failed.
             # We should point the user to Home if the most recent spawn failed.
             exc = spawner._spawn_future.exception()

jupyterhub/roles.py

@@ -403,6 +403,10 @@ def _token_allowed_role(db, token, role):
     if owner is None:
         raise ValueError(f"Owner not found for {token}")
 
+    if role in owner.roles:
+        # shortcut: token is assigned an exact role the owner has
+        return True
+
     expanded_scopes = _get_subscopes(role, owner=owner)
 
     implicit_permissions = {'inherit', 'read:inherit'}

jupyterhub/services/auth.py

@@ -501,11 +501,17 @@ class HubAuth(SingletonConfigurable):
     auth_header_name = 'Authorization'
     auth_header_pat = re.compile(r'(?:token|bearer)\s+(.+)', re.IGNORECASE)
 
-    def get_token(self, handler):
-        """Get the user token from a request
+    def get_token(self, handler, in_cookie=True):
+        """Get the token authenticating a request
+
+        .. versionchanged:: 2.2
+          in_cookie added.
+          Previously, only URL params and header were considered.
+          Pass `in_cookie=False` to preserve that behavior.
 
         - in URL parameters: ?token=<token>
         - in header: Authorization: token <token>
+        - in cookie (stored after oauth), if in_cookie is True
         """
 
         user_token = handler.get_argument('token', '')
@@ -516,8 +522,14 @@ class HubAuth(SingletonConfigurable):
             )
             if m:
                 user_token = m.group(1)
+        if not user_token and in_cookie:
+            user_token = self._get_token_cookie(handler)
         return user_token
 
+    def _get_token_cookie(self, handler):
+        """Base class doesn't store tokens in cookies"""
+        return None
+
     def _get_user_cookie(self, handler):
         """Get the user model from a cookie"""
         # overridden in HubOAuth to store the access token after oauth
@@ -553,8 +565,10 @@ class HubAuth(SingletonConfigurable):
         handler._cached_hub_user = user_model = None
         session_id = self.get_session_id(handler)
 
-        # check token first
-        token = self.get_token(handler)
+        # check token first, ignoring cookies
+        # because some checks are different when a request
+        # is token-authenticated (CORS-related)
+        token = self.get_token(handler, in_cookie=False)
         if token:
             user_model = self.user_for_token(token, session_id=session_id)
             if user_model:
@@ -614,11 +628,18 @@ class HubOAuth(HubAuth):
         """
         return self.cookie_name + '-oauth-state'
 
-    def _get_user_cookie(self, handler):
+    def _get_token_cookie(self, handler):
+        """Base class doesn't store tokens in cookies"""
         token = handler.get_secure_cookie(self.cookie_name)
+        if token:
+            # decode cookie bytes
+            token = token.decode('ascii', 'replace')
+        return token
+
+    def _get_user_cookie(self, handler):
+        token = self._get_token_cookie(handler)
         session_id = self.get_session_id(handler)
         if token:
-            token = token.decode('ascii', 'replace')
             user_model = self.user_for_token(token, session_id=session_id)
             if user_model is None:
                 app_log.warning("Token stored in cookie may have expired")

jupyterhub/singleuser/mixins.py

@@ -16,7 +16,6 @@ import random
 import secrets
 import sys
 import warnings
-from datetime import datetime
 from datetime import timezone
 from importlib import import_module
 from textwrap import dedent
@@ -493,7 +492,7 @@ class SingleUserNotebookAppMixin(Configurable):
                     i,
                     RETRIES,
                 )
-                await asyncio.sleep(min(2 ** i, 16))
+                await asyncio.sleep(min(2**i, 16))
             else:
                 break
         else:
@@ -680,6 +679,7 @@ class SingleUserNotebookAppMixin(Configurable):
         s['hub_prefix'] = self.hub_prefix
         s['hub_host'] = self.hub_host
         s['hub_auth'] = self.hub_auth
+        s['page_config_hook'] = self.page_config_hook
         csp_report_uri = s['csp_report_uri'] = self.hub_host + url_path_join(
             self.hub_prefix, 'security/csp-report'
         )
@@ -707,6 +707,18 @@ class SingleUserNotebookAppMixin(Configurable):
         self.patch_default_headers()
         self.patch_templates()
 
+    def page_config_hook(self, handler, page_config):
+        """JupyterLab page config hook
+
+        Adds JupyterHub info to page config.
+
+        Places the JupyterHub API token in PageConfig.token.
+
+        Only has effect on jupyterlab_server >=2.9
+        """
+        page_config["token"] = self.hub_auth.get_token(handler) or ""
+        return page_config
+
     def patch_default_headers(self):
         if hasattr(RequestHandler, '_orig_set_default_headers'):
             return

jupyterhub/spawner.py

@@ -11,6 +11,7 @@ import shutil
 import signal
 import sys
 import warnings
+from inspect import signature
 from subprocess import Popen
 from tempfile import mkdtemp
 from urllib.parse import urlparse
@@ -183,17 +184,38 @@ class Spawner(LoggingConfigurable):
     def last_activity(self):
         return self.orm_spawner.last_activity
 
+    # Spawner.server is a wrapper of the ORM orm_spawner.server
+    # make sure it's always in sync with the underlying state
+    # this is harder to do with traitlets,
+    # which do not run on every access, only on set and first-get
+    _server = None
+
     @property
     def server(self):
-        if hasattr(self, '_server'):
+        # always check that we're in sync with orm_spawner
+        if not self.orm_spawner:
+            # no ORM spawner, nothing to check
             return self._server
-        if self.orm_spawner and self.orm_spawner.server:
-            return Server(orm_server=self.orm_spawner.server)
+
+        orm_server = self.orm_spawner.server
+
+        if orm_server is not None and (
+            self._server is None or orm_server is not self._server.orm_server
+        ):
+            # self._server is not connected to orm_spawner
+            self._server = Server(orm_server=self.orm_spawner.server)
+        elif orm_server is None:
+            # no ORM server, clear it
+            self._server = None
+        return self._server
 
     @server.setter
     def server(self, server):
         self._server = server
-        if self.orm_spawner:
+        if self.orm_spawner is not None:
+            if server is not None and server.orm_server == self.orm_spawner.server:
+                # no change
+                return
             if self.orm_spawner.server is not None:
                 # delete the old value
                 db = inspect(self.orm_spawner.server).session
@@ -201,7 +223,13 @@ class Spawner(LoggingConfigurable):
             if server is None:
                 self.orm_spawner.server = None
             else:
+                if server.orm_server is None:
+                    self.log.warning(f"No ORM server for {self._log_name}")
                 self.orm_spawner.server = server.orm_server
+        elif server is not None:
+            self.log.warning(
+                "Setting Spawner.server for {self._log_name} with no underlying orm_spawner"
+            )
 
     @property
     def name(self):
@@ -424,6 +452,13 @@ class Spawner(LoggingConfigurable):
     def _default_options_from_form(self, form_data):
         return form_data
 
+    def run_options_from_form(self, form_data):
+        sig = signature(self.options_from_form)
+        if 'spawner' in sig.parameters:
+            return self.options_from_form(form_data, spawner=self)
+        else:
+            return self.options_from_form(form_data)
+
     def options_from_query(self, query_data):
         """Interpret query arguments passed to /spawn
 

jupyterhub/tests/test_api.py

@@ -1030,7 +1030,7 @@ async def test_never_spawn(app, no_patience, never_spawn):
     assert not app_user.spawner._spawn_pending
     status = await app_user.spawner.poll()
     assert status is not None
-    # failed spawn should decrements pending count
+    # failed spawn should decrement pending count
     assert app.users.count_active_users()['pending'] == 0
 
 
@@ -1039,9 +1039,16 @@ async def test_bad_spawn(app, bad_spawn):
     name = 'prim'
     user = add_user(db, app=app, name=name)
     r = await api_request(app, 'users', name, 'server', method='post')
+    # check that we don't re-use spawners that failed
+    user.spawners[''].reused = True
     assert r.status_code == 500
     assert app.users.count_active_users()['pending'] == 0
 
+    r = await api_request(app, 'users', name, 'server', method='post')
+    # check that we don't re-use spawners that failed
+    spawner = user.spawners['']
+    assert not getattr(spawner, 'reused', False)
+
 
 async def test_spawn_nosuch_user(app):
     r = await api_request(app, 'users', "nosuchuser", 'server', method='post')
@@ -1806,6 +1813,38 @@ async def test_group_add_delete_users(app):
     assert sorted(u.name for u in group.users) == sorted(names[2:])
 
 
+@mark.group
+async def test_auth_managed_groups(request, app, group, user):
+    group.users.append(user)
+    app.db.commit()
+    app.authenticator.manage_groups = True
+    request.addfinalizer(lambda: setattr(app.authenticator, "manage_groups", False))
+    # create groups
+    r = await api_request(app, 'groups', method='post')
+    assert r.status_code == 400
+    r = await api_request(app, 'groups/newgroup', method='post')
+    assert r.status_code == 400
+    # delete groups
+    r = await api_request(app, f'groups/{group.name}', method='delete')
+    assert r.status_code == 400
+    # add users to group
+    r = await api_request(
+        app,
+        f'groups/{group.name}/users',
+        method='post',
+        data=json.dumps({"users": [user.name]}),
+    )
+    assert r.status_code == 400
+    # remove users from group
+    r = await api_request(
+        app,
+        f'groups/{group.name}/users',
+        method='delete',
+        data=json.dumps({"users": [user.name]}),
+    )
+    assert r.status_code == 400
+
+
 # -----------------
 # Service API tests
 # -----------------

jupyterhub/tests/test_auth.py

@@ -7,6 +7,7 @@ from urllib.parse import urlparse
 
 import pytest
 from requests import HTTPError
+from traitlets import Any
 from traitlets.config import Config
 
 from .mocking import MockPAMAuthenticator
@@ -14,6 +15,7 @@ from .mocking import MockStructGroup
 from .mocking import MockStructPasswd
 from .utils import add_user
 from .utils import async_requests
+from .utils import get_page
 from .utils import public_url
 from jupyterhub import auth
 from jupyterhub import crypto
@@ -527,3 +529,71 @@ async def test_nullauthenticator(app):
         r = await async_requests.get(public_url(app))
     assert urlparse(r.url).path.endswith("/hub/login")
     assert r.status_code == 403
+
+
+class MockGroupsAuthenticator(auth.Authenticator):
+    authenticated_groups = Any()
+    refresh_groups = Any()
+
+    manage_groups = True
+
+    def authenticate(self, handler, data):
+        return {
+            "name": data["username"],
+            "groups": self.authenticated_groups,
+        }
+
+    async def refresh_user(self, user, handler):
+        return {
+            "name": user.name,
+            "groups": self.refresh_groups,
+        }
+
+
+@pytest.mark.parametrize(
+    "authenticated_groups, refresh_groups",
+    [
+        (None, None),
+        (["auth1"], None),
+        (None, ["auth1"]),
+        (["auth1"], ["auth1", "auth2"]),
+        (["auth1", "auth2"], ["auth1"]),
+        (["auth1", "auth2"], ["auth3"]),
+        (["auth1", "auth2"], ["auth3"]),
+    ],
+)
+async def test_auth_managed_groups(
+    app, user, group, authenticated_groups, refresh_groups
+):
+
+    authenticator = MockGroupsAuthenticator(
+        parent=app,
+        authenticated_groups=authenticated_groups,
+        refresh_groups=refresh_groups,
+    )
+
+    user.groups.append(group)
+    app.db.commit()
+    before_groups = [group.name]
+    if authenticated_groups is None:
+        expected_authenticated_groups = before_groups
+    else:
+        expected_authenticated_groups = authenticated_groups
+    if refresh_groups is None:
+        expected_refresh_groups = expected_authenticated_groups
+    else:
+        expected_refresh_groups = refresh_groups
+
+    with mock.patch.dict(app.tornado_settings, {"authenticator": authenticator}):
+        cookies = await app.login_user(user.name)
+        assert not app.db.dirty
+        groups = sorted(g.name for g in user.groups)
+        assert groups == expected_authenticated_groups
+
+        # force refresh_user on next request
+        user._auth_refreshed -= 10 + app.authenticator.auth_refresh_age
+        r = await get_page('home', app, cookies=cookies, allow_redirects=False)
+        assert r.status_code == 200
+        assert not app.db.dirty
+        groups = sorted(g.name for g in user.groups)
+        assert groups == expected_refresh_groups

jupyterhub/tests/test_pages.py

@@ -128,11 +128,20 @@ async def test_admin_sort(app, sort):
     assert r.status_code == 200
 
 
-async def test_spawn_redirect(app):
+@pytest.mark.parametrize("last_failed", [True, False])
+async def test_spawn_redirect(app, last_failed):
     name = 'wash'
     cookies = await app.login_user(name)
     u = app.users[orm.User.find(app.db, name)]
 
+    if last_failed:
+        # mock a failed spawn
+        last_spawner = u.spawners['']
+        last_spawner._spawn_future = asyncio.Future()
+        last_spawner._spawn_future.set_exception(RuntimeError("I failed!"))
+    else:
+        last_spawner = None
+
     status = await u.spawner.poll()
     assert status is not None
 
@@ -141,6 +150,10 @@ async def test_spawn_redirect(app):
     r.raise_for_status()
     print(urlparse(r.url))
     path = urlparse(r.url).path
+
+    # ensure we got a new spawner
+    assert u.spawners[''] is not last_spawner
+
     # make sure we visited hub/spawn-pending after spawn
     # if spawn was really quick, we might get redirected all the way to the running server,
     # so check history instead of r.url
@@ -258,6 +271,25 @@ async def test_spawn_page(app):
         assert FormSpawner.options_form in r.text
 
 
+async def test_spawn_page_after_failed(app, user):
+    cookies = await app.login_user(user.name)
+
+    # mock a failed spawn
+    last_spawner = user.spawners['']
+    last_spawner._spawn_future = asyncio.Future()
+    last_spawner._spawn_future.set_exception(RuntimeError("I failed!"))
+
+    with mock.patch.dict(app.users.settings, {'spawner_class': FormSpawner}):
+        r = await get_page('spawn', app, cookies=cookies)
+        spawner = user.spawners['']
+        # make sure we didn't reuse last spawner
+        assert isinstance(spawner, FormSpawner)
+        assert spawner is not last_spawner
+        assert r.url.endswith('/spawn')
+        spawner = user.spawners['']
+        assert FormSpawner.options_form in r.text
+
+
 async def test_spawn_page_falsy_callable(app):
     with mock.patch.dict(
         app.users.settings, {'spawner_class': FalsyCallableFormSpawner}

jupyterhub/tests/test_spawner.py

@@ -459,3 +459,80 @@ async def test_spawner_oauth_roles_bad(app, user):
     # raises ValueError if we try to assign a role that doesn't exist
     with pytest.raises(ValueError):
         await spawner.user.spawn()
+
+
+async def test_spawner_options_from_form(db):
+    def options_from_form(form_data):
+        return form_data
+
+    spawner = new_spawner(db, options_from_form=options_from_form)
+    form_data = {"key": ["value"]}
+    result = spawner.run_options_from_form(form_data)
+    for key, value in form_data.items():
+        assert key in result
+        assert result[key] == value
+
+
+async def test_spawner_options_from_form_with_spawner(db):
+    def options_from_form(form_data, spawner):
+        return form_data
+
+    spawner = new_spawner(db, options_from_form=options_from_form)
+    form_data = {"key": ["value"]}
+    result = spawner.run_options_from_form(form_data)
+    for key, value in form_data.items():
+        assert key in result
+        assert result[key] == value
+
+
+def test_spawner_server(db):
+    spawner = new_spawner(db)
+    spawner.orm_spawner = None
+    orm_spawner = orm.Spawner()
+    orm_server = orm.Server(base_url="/1/")
+    orm_spawner.server = orm_server
+    db.add(orm_spawner)
+    db.add(orm_server)
+    db.commit()
+    # initial: no orm_spawner
+    assert spawner.server is None
+    # assigning spawner.orm_spawner updates spawner.server
+    spawner.orm_spawner = orm_spawner
+    assert spawner.server is not None
+    assert spawner.server.orm_server is orm_server
+    # update orm_spawner.server without direct access on Spawner
+    orm_spawner.server = new_server = orm.Server(base_url="/2/")
+    db.commit()
+    assert spawner.server is not None
+    assert spawner.server.orm_server is not orm_server
+    assert spawner.server.orm_server is new_server
+    # clear orm_server via orm_spawner clears spawner.server
+    orm_spawner.server = None
+    db.commit()
+    assert spawner.server is None
+    # assigning spawner.server updates orm_spawner.server
+    orm_server = orm.Server(base_url="/3/")
+    db.add(orm_server)
+    db.commit()
+    spawner.server = server = Server(orm_server=orm_server)
+    db.commit()
+    assert spawner.server is server
+    assert spawner.orm_spawner.server is orm_server
+    # change orm spawner.server
+    orm_server = orm.Server(base_url="/4/")
+    db.add(orm_server)
+    db.commit()
+    spawner.server = server2 = Server(orm_server=orm_server)
+    assert spawner.server is server2
+    assert spawner.orm_spawner.server is orm_server
+    # clear server via spawner.server
+    spawner.server = None
+    db.commit()
+    assert spawner.orm_spawner.server is None
+
+    # test with no underlying orm.Spawner
+    # (only relevant for mocking, never true for actual Spawners)
+    spawner = Spawner()
+    spawner.server = Server.from_url("http://1.2.3.4")
+    assert spawner.server is not None
+    assert spawner.server.ip == "1.2.3.4"

jupyterhub/tests/test_user.py

@@ -1,5 +1,6 @@
 import pytest
 
+from .. import orm
 from ..user import UserDict
 from .utils import add_user
 
@@ -20,3 +21,35 @@ async def test_userdict_get(db, attr):
     assert userdict.get(key).id == u.id
     # `in` should find it now
     assert key in userdict
+
+
+@pytest.mark.parametrize(
+    "group_names",
+    [
+        ["isin1", "isin2"],
+        ["isin1"],
+        ["notin", "isin1"],
+        ["new-group", "isin1"],
+        [],
+    ],
+)
+def test_sync_groups(app, user, group_names):
+    expected = sorted(group_names)
+    db = app.db
+    db.add(orm.Group(name="notin"))
+    in_groups = [orm.Group(name="isin1"), orm.Group(name="isin2")]
+    for group in in_groups:
+        db.add(group)
+    db.commit()
+    user.groups = in_groups
+    db.commit()
+    user.sync_groups(group_names)
+    assert not app.db.dirty
+    after_groups = sorted(g.name for g in user.groups)
+    assert after_groups == expected
+    # double-check backref
+    for group in db.query(orm.Group):
+        if group.name in expected:
+            assert user.orm_user in group.users
+        else:
+            assert user.orm_user not in group.users

jupyterhub/user.py

@@ -253,6 +253,58 @@ class User:
     def spawner_class(self):
         return self.settings.get('spawner_class', LocalProcessSpawner)
 
+    def get_spawner(self, server_name="", replace_failed=False):
+        """Get a spawner by name
+
+        replace_failed governs whether a failed spawner should be replaced
+        or returned (default: returned).
+
+        .. versionadded:: 2.2
+        """
+        spawner = self.spawners[server_name]
+        if replace_failed and spawner._failed:
+            self.log.debug(f"Discarding failed spawner {spawner._log_name}")
+            # remove failed spawner, create a new one
+            self.spawners.pop(server_name)
+            spawner = self.spawners[server_name]
+        return spawner
+
+    def sync_groups(self, group_names):
+        """Synchronize groups with database"""
+
+        current_groups = {g.name for g in self.orm_user.groups}
+        new_groups = set(group_names)
+        if current_groups == new_groups:
+            # no change, nothing to do
+            return
+
+        # log group changes
+        new_groups = set(group_names).difference(current_groups)
+        removed_groups = current_groups.difference(group_names)
+        if new_groups:
+            self.log.info("Adding user {self.name} to group(s): {new_groups}")
+        if removed_groups:
+            self.log.info("Removing user {self.name} from group(s): {removed_groups}")
+
+        if group_names:
+            groups = (
+                self.db.query(orm.Group).filter(orm.Group.name.in_(group_names)).all()
+            )
+            existing_groups = {g.name for g in groups}
+            for group_name in group_names:
+                if group_name not in existing_groups:
+                    # create groups that don't exist yet
+                    self.log.info(
+                        f"Creating new group {group_name} for user {self.name}"
+                    )
+                    group = orm.Group(name=group_name)
+                    self.db.add(group)
+                    groups.append(group)
+            self.groups = groups
+        else:
+            self.groups = []
+        self.db.commit()
+
     async def save_auth_state(self, auth_state):
         """Encrypt and store auth_state"""
         if auth_state is None:
@@ -592,7 +644,7 @@ class User:
         api_token = self.new_api_token(note=note, roles=['server'])
         db.commit()
 
-        spawner = self.spawners[server_name]
+        spawner = self.get_spawner(server_name, replace_failed=True)
         spawner.server = server = Server(orm_server=orm_server)
         assert spawner.orm_spawner.server is orm_server
 

pyproject.toml

@@ -11,7 +11,7 @@ target_version = [
 github_url = "https://github.com/jupyterhub/jupyterhub"
 
 [tool.tbump.version]
-current = "2.1.1"
+current = "2.2.1"
 
 # Example of a semver regexp.
 # Make sure this matches current_version before

share/jupyterhub/templates/login.html

@@ -15,6 +15,11 @@
 {{ custom_html | safe }}
 {% elif login_service %}
 <div class="service-login">
+  <p id='insecure-login-warning' class='hidden'>
+  Warning: JupyterHub seems to be served over an unsecured HTTP connection.
+  We strongly recommend enabling HTTPS for JupyterHub.
+  </p>
+
   <a role="button" class='btn btn-jupyter btn-lg' href='{{authenticator_login_url}}'>
     Sign in with {{login_service}}
   </a>