0.8.0b2

build jupyterhub/singleuser on this repo

.dockerignore

@@ -4,3 +4,7 @@ jupyterhub_cookie_secret
 jupyterhub.sqlite
 jupyterhub_config.py
 node_modules
+docs
+.git
+dist
+build

.gitignore

@@ -3,7 +3,7 @@ node_modules
 *~
 .cache
 .DS_Store
-build
+/build
 dist
 docs/_build
 docs/source/_static/rest-api

.travis.yml

@@ -45,3 +45,5 @@ matrix:
       env: JUPYTERHUB_TEST_DB_URL=mysql+mysqlconnector://root@127.0.0.1/jupyterhub
     - python: 3.6
       env: JUPYTERHUB_TEST_DB_URL=postgresql://postgres@127.0.0.1/jupyterhub
+  allow_failures:
+    - python: nightly

MANIFEST.in

@@ -10,6 +10,7 @@ graft onbuild
 graft jupyterhub
 graft scripts
 graft share
+graft singleuser
 
 # Documentation
 graft docs

README.md

@@ -99,7 +99,7 @@ more configuration of the system.
 
 ## Configuration
 
-The [Getting Started](http://jupyterhub.readthedocs.io/en/latest/getting-started.html) section of the
+The [Getting Started](http://jupyterhub.readthedocs.io/en/latest/getting-started/index.html) section of the
 documentation explains the common steps in setting up JupyterHub.
 
 The [**JupyterHub tutorial**](https://github.com/jupyterhub/jupyterhub-tutorial)

docs/source/api/services.auth.rst

@@ -16,6 +16,12 @@ Module: :mod:`jupyterhub.services.auth`
 .. autoconfigurable:: HubAuth
     :members:
 
+:class:`HubOAuth`
+----------------
+
+.. autoconfigurable:: HubOAuth
+    :members:
+
 
 :class:`HubAuthenticated`
 -------------------------
@@ -23,3 +29,13 @@ Module: :mod:`jupyterhub.services.auth`
 .. autoclass:: HubAuthenticated
     :members:
 
+:class:`HubOAuthenticated`
+-------------------------
+
+.. autoclass:: HubOAuthenticated
+
+:class:`HubOAuthCallbackHandler`
+--------------------------------
+
+.. autoclass:: HubOAuthCallbackHandler
+

docs/source/changelog.md

@@ -7,12 +7,90 @@ command line for details.
 
 ## [Unreleased] 0.8
 
+JupyterHub 0.8 is a big release!
+
+Perhaps the biggest change is the use of OAuth to negotiate authentication
+between the Hub and single-user services.
+Due to this change, it is important that the single-user server
+and Hub are both running the same version of JupyterHub.
+If you are using containers (e.g. via DockerSpawner or KubeSpawner),
+this means upgrading jupyterhub in your user images at the same time as the Hub.
+In most cases, a
+
+    pip install jupyterhub==version
+
+in your Dockerfile is sufficient.
+
 #### Added
 
+- JupyterHub now defined a `.Proxy` API for custom
+  proxy implementations other than the default.
+  The defaults are unchanged,
+  but configuration of the proxy is now done on the `ConfigurableHTTPProxy` class instead of the top-level JupyterHub.
+  TODO: docs for writing a custom proxy.
+- Single-user servers and services
+  (anything that uses HubAuth)
+  can now accept token-authenticated requests via the Authentication header.
+- Authenticators can now store state in the Hub's database.
+  To do so, the `.authenticate` method should return a dict of the form
+
+  ```python
+  {
+      'username': 'name'
+      'state': {}
+  }
+  ```
+
+  This data will be encrypted and requires `JUPYTERHUB_CRYPT_KEY` environment variable to be set
+  and the `Authenticator.enable_auth_state` flag to be True.
+  If these are not set, auth_state returned by the Authenticator will not be stored.
+- There is preliminary support for multiple (named) servers per user in the REST API.
+  Named servers can be created via API requests, but there is currently no UI for managing them.
+- Add `LocalProcessSpawner.popen_kwargs` and `LocalProcessSpawner.shell_cmd`
+  for customizing how user server processes are launched.
+- Add `Authenticator.auto_login` flag for skipping the "Login with..." page explicitly.
+- Add `JupyterHub.hub_connect_ip` configuration
+  for the ip that should be used when connecting to the Hub.
+  This is promoting (and deprecating) `DockerSpawner.hub_ip_connect`
+  for use by all Spawners.
+- Add `Spawner.pre_spawn_hook(spawner)` hook for customizing
+  pre-spawn events.
+- Add `JupyterHub.active_server_limit` and `JupyterHub.concurrent_spawn_limit`
+  for limiting the total number of running user servers and the number of pending spawns, respectively.
+
+
 #### Changed
 
+- more arguments to spawners are now passed via environment variables (`.get_env()`)
+  rather than CLI arguments (`.get_args()`)
+- internally generated tokens no longer get extra hash rounds,
+  significantly speeding up authentication.
+  The hash rounds were deemed unnecessary because the tokens were already
+  generated with high entropy.
+- `JUPYTERHUB_API_TOKEN` env is available at all times,
+  rather than being removed during single-user start.
+  The token is now accessible to kernel processes,
+  enabling user kernels to make authenticated API requests to Hub-authenticated services.
+- Cookie secrets should be 32B hex instead of large base64 secrets.
+- pycurl is used by default, if available.
+
 #### Fixed
 
+So many things fixed!
+
+- Collisions are checked when users are renamed
+- Fix bug where OAuth authenticators could not logout users
+  due to being redirected right back through the login process.
+- If there are errors loading your config files,
+  JupyterHub will refuse to start with an informative error.
+  Previously, the bad config would be ignored and JupyterHub would launch with default configuration.
+- Raise 403 error on unauthorized user rather than redirect to login,
+  which could cause redirect loop.
+- Set `httponly` on cookies because it's prudent.
+- Improve support for MySQL as the database backend
+- Many race conditions and performance problems under heavy load have been fixed.
+- Fix alembic tagging of database schema versions.
+
 #### Removed
 
 - End support for Python 3.3 

docs/source/index.rst

@@ -67,6 +67,8 @@ Contents
 **Tutorials**
 
 * :doc:`tutorials/index`
+* :doc:`tutorials/upgrade-dot-eight`
+* `Zero to JupyterHub with Kubernetes <https://zero-to-jupyterhub.readthedocs.io/en/latest/>`_
 
 **Troubleshooting**
 

docs/source/tutorials/index.rst

@@ -4,4 +4,11 @@ Tutorials
 This section provides links to documentation that helps a user do a specific
 task.
 
-- `Zero to JupyterHub with Kubernetes <https://zero-to-jupyterhub.readthedocs.io/en/latest/>`_
+* :doc:`upgrade-dot-eight`
+* `Zero to JupyterHub with Kubernetes <https://zero-to-jupyterhub.readthedocs.io/en/latest/>`_
+
+.. toctree::
+   :maxdepth: 1
+   :hidden:
+
+   upgrade-dot-eight

docs/source/tutorials/upgrade-dot-eight.rst

@@ -0,0 +1,93 @@
+.. upgrade-dot-eight:
+
+Upgrading to JupyterHub version 0.8
+===================================
+
+This document will assist you in upgrading an existing JupyterHub deployment
+from version 0.7 to version 0.8.
+
+Upgrade checklist
+-----------------
+
+0. Review the release notes. Review any deprecated features and pay attention
+   to any backwards incompatible changes
+1. Backup JupyterHub database:
+    - ``jupyterhub.sqlite`` when using the default sqlite database
+    - Your JupyterHub database when using an RDBMS
+2. Backup the existing JupyterHub configuration file: ``jupyterhub_config.py``
+3. Shutdown the Hub
+4. Upgrade JupyterHub
+    - ``pip install -U jupyterhub`` when using ``pip``
+    - ``conda upgrade jupyterhub`` when using ``conda``
+5. Upgrade the database using run ```jupyterhub upgrade-db``
+6. Update the JupyterHub configuration file ``jupyterhub_config.py``
+
+Backup JupyterHub database
+--------------------------
+
+To prevent unintended loss of data or configuration information, you should
+back up the JupyterHub database (the default SQLite database or a RDBMS
+database using PostgreSQL, MySQL, or others supported by SQLAlchemy):
+
+- If using the default SQLite database, back up the ``jupyterhub.sqlite``
+  database.
+- If using an RDBMS database such as PostgreSQL, MySQL, or other supported by
+  SQLAlchemy, back up the JupyterHub database.
+
+.. note::
+
+    Losing the Hub database is often not a big deal. Information that resides only
+    in the Hub database includes:
+
+    - active login tokens (user cookies, service tokens)
+    - users added via GitHub UI, instead of config files
+    - info about running servers
+
+    If the following conditions are true, you should be fine clearing the Hub
+    database and starting over:
+
+    - users specified in config file
+    - user servers are stopped during upgrade
+    - don't mind causing users to login again after upgrade
+
+Backup JupyterHub configuration file
+------------------------------------
+
+Backup up your configuration file, ``jupyterhub_config.py``, to a secure
+location.
+
+Shutdown JupyterHub
+-------------------
+
+- Prior to shutting down JupyterHub, you should notify the Hub users of the
+  scheduled downtime.
+- Shutdown the JupyterHub service.
+
+Upgrade JupyterHub
+------------------
+
+Follow directions that correspond to your package manager, ``pip`` or ``conda``,
+for the new JupyterHub release:
+
+- ``pip install -U jupyterhub`` for ``pip``
+- ``conda upgrade jupyterhub`` for ``conda``
+
+Upgrade the proxy, authenticator, or spawner if needed.
+
+Upgrade JupyterHub database
+---------------------------
+
+To run the upgrade process for JupyterHub databases, enter::
+
+    jupyterhub upgrade-db
+
+Update the JupyterHub configuration file
+----------------------------------------
+
+Create a new JupyterHub configuration file or edit a copy of the existing
+file ``jupyterhub_config.py``.
+
+Start JupyterHub
+----------------
+
+Start JupyterHub with the same command that you used before the upgrade.

examples/service-whoami-flask/README.md

@@ -8,7 +8,7 @@ Uses `jupyterhub.services.HubAuth` to authenticate requests with the Hub in a [f
 
         jupyterhub --ip=127.0.0.1
 
-2. Visit http://127.0.0.1:8000/services/whoami
+2. Visit http://127.0.0.1:8000/services/whoami or http://127.0.0.1:8000/services/whoami-oauth
 
 After logging in with your local-system credentials, you should see a JSON dump of your user info:
 

examples/service-whoami-flask/jupyterhub_config.py

@@ -9,5 +9,13 @@ c.JupyterHub.services = [
         'environment': {
             'FLASK_APP': 'whoami-flask.py',
         }
-    }
+    },
+    {
+        'name': 'whoami-oauth',
+        'url': 'http://127.0.0.1:10201',
+        'command': ['flask', 'run', '--port=10201'],
+        'environment': {
+            'FLASK_APP': 'whoami-oauth.py',
+        }
+    },
 ]

examples/service-whoami-flask/whoami-flask.py

@@ -17,7 +17,7 @@ prefix = os.environ.get('JUPYTERHUB_SERVICE_PREFIX', '/')
 
 auth = HubAuth(
     api_token=os.environ['JUPYTERHUB_API_TOKEN'],
-    cookie_cache_max_age=60,
+    cache_max_age=60,
 )
 
 app = Flask(__name__)

examples/service-whoami-flask/whoami-oauth.py

@@ -0,0 +1,70 @@
+#!/usr/bin/env python3
+"""
+whoami service authentication with the Hub
+"""
+
+from functools import wraps
+import json
+import os
+
+from flask import Flask, redirect, request, Response, make_response
+
+from jupyterhub.services.auth import HubOAuth
+
+
+prefix = os.environ.get('JUPYTERHUB_SERVICE_PREFIX', '/')
+
+auth = HubOAuth(
+    api_token=os.environ['JUPYTERHUB_API_TOKEN'],
+    cache_max_age=60,
+)
+
+app = Flask(__name__)
+
+
+def authenticated(f):
+    """Decorator for authenticating with the Hub via OAuth"""
+    @wraps(f)
+    def decorated(*args, **kwargs):
+        token = request.cookies.get(auth.cookie_name)
+        if token:
+            user = auth.user_for_token(token)
+        else:
+            user = None
+        if user:
+            return f(user, *args, **kwargs)
+        else:
+            # redirect to login url on failed auth
+            state = auth.generate_state(next_url=request.path)
+            response = make_response(redirect(auth.login_url + '&state=%s' % state))
+            response.set_cookie(auth.state_cookie_name, state)
+            return response
+    return decorated
+
+
+@app.route(prefix)
+@authenticated
+def whoami(user):
+    return Response(
+        json.dumps(user, indent=1, sort_keys=True),
+        mimetype='application/json',
+        )
+
+@app.route(prefix + 'oauth_callback')
+def oauth_callback():
+    code = request.args.get('code', None)
+    if code is None:
+        return 403
+
+    # validate state field
+    arg_state = request.args.get('state', None)
+    cookie_state = request.cookies.get(auth.state_cookie_name)
+    if arg_state != cookie_state:
+        # state doesn't match
+        return 403
+
+    token = auth.token_for_code(code)
+    next_url = auth.get_next_url(cookie_state) or prefix
+    response = make_response(redirect(next_url))
+    response.set_cookie(auth.cookie_name, token)
+    return response

examples/service-whoami/README.md

@@ -2,13 +2,15 @@
 
 Uses `jupyterhub.services.HubAuthenticated` to authenticate requests with the Hub.
 
+There is an implementation each of cookie-based `HubAuthenticated` and OAuth-based `HubOAuthenticated`.
+
 ## Run
 
 1. Launch JupyterHub and the `whoami service` with
 
         jupyterhub --ip=127.0.0.1
 
-2. Visit http://127.0.0.1:8000/services/whoami
+2. Visit http://127.0.0.1:8000/services/whoami or http://127.0.0.1:8000/services/whoami-oauth
 
 After logging in with your local-system credentials, you should see a JSON dump of your user info:
 

examples/service-whoami/jupyterhub_config.py

@@ -6,5 +6,10 @@ c.JupyterHub.services = [
         'name': 'whoami',
         'url': 'http://127.0.0.1:10101',
         'command': [sys.executable, './whoami.py'],
-    }
+    },
+    {
+        'name': 'whoami-oauth',
+        'url': 'http://127.0.0.1:10102',
+        'command': [sys.executable, './whoami-oauth.py'],
+    },
 ]

examples/service-whoami/whoami-oauth.py

@@ -13,10 +13,10 @@ from tornado.ioloop import IOLoop
 from tornado.httpserver import HTTPServer
 from tornado.web import RequestHandler, Application, authenticated
 
-from jupyterhub.services.auth import HubAuthenticated
+from jupyterhub.services.auth import HubOAuthenticated, HubOAuthCallbackHandler
+from jupyterhub.utils import url_path_join
 
-
-class WhoAmIHandler(HubAuthenticated, RequestHandler):
+class WhoAmIHandler(HubOAuthenticated, RequestHandler):
     hub_users = {getuser()} # the users allowed to access this service
 
     @authenticated
@@ -27,9 +27,10 @@ class WhoAmIHandler(HubAuthenticated, RequestHandler):
 
 def main():
     app = Application([
-        (os.environ['JUPYTERHUB_SERVICE_PREFIX'] + '/?', WhoAmIHandler),
+        (os.environ['JUPYTERHUB_SERVICE_PREFIX'], WhoAmIHandler),
+        (url_path_join(os.environ['JUPYTERHUB_SERVICE_PREFIX'], 'oauth_callback'), HubOAuthCallbackHandler),
         (r'.*', WhoAmIHandler),
-    ])
+    ], cookie_secret=os.urandom(32))
     
     http_server = HTTPServer(app)
     url = urlparse(os.environ['JUPYTERHUB_SERVICE_URL'])

examples/service-whoami/whoami.py

@@ -27,7 +27,7 @@ def main():
     app = Application([
         (os.environ['JUPYTERHUB_SERVICE_PREFIX'] + '/?', WhoAmIHandler),
         (r'.*', WhoAmIHandler),
-    ], login_url='/hub/login')
+    ])
     
     http_server = HTTPServer(app)
     url = urlparse(os.environ['JUPYTERHUB_SERVICE_URL'])

jupyterhub/_version.py

@@ -7,7 +7,7 @@ version_info = (
     0,
     8,
     0,
-    'b1',
+    'b2',
 )
 
 __version__ = '.'.join(map(str, version_info))

jupyterhub/app.py

@@ -62,7 +62,7 @@ from .utils import (
 from .auth import Authenticator, PAMAuthenticator
 from .crypto import CryptKeeper
 from .spawner import Spawner, LocalProcessSpawner
-from .objects import Hub
+from .objects import Hub, Server
 
 # For faking stats
 from .emptyclass import EmptyClass
@@ -1180,7 +1180,7 @@ class JupyterHub(Application):
             if not service.url:
                 continue
             try:
-                yield service.orm.server.wait_up(timeout=1)
+                yield Server.from_orm(service.orm.server).wait_up(timeout=1)
             except TimeoutError:
                 self.log.warning("Cannot connect to %s service %s at %s", service.kind, name, service.url)
             else:
@@ -1557,7 +1557,7 @@ class JupyterHub(Application):
                 tries = 10 if service.managed else 1
                 for i in range(tries):
                     try:
-                        yield service.orm.server.wait_up(http=True, timeout=1)
+                        yield Server.from_orm(service.orm.server).wait_up(http=True, timeout=1)
                     except TimeoutError:
                         if service.managed:
                             status = yield service.spawner.poll()

jupyterhub/handlers/pages.py

@@ -146,14 +146,19 @@ class AdminHandler(BaseHandler):
         available = {'name', 'admin', 'running', 'last_activity'}
         default_sort = ['admin', 'name']
         mapping = {
-            'running': '_server_id'
+            'running': orm.Spawner.server_id,
         }
+        for name in available:
+            if name not in mapping:
+                mapping[name] = getattr(orm.User, name)
+
         default_order = {
             'name': 'asc',
             'last_activity': 'desc',
             'admin': 'desc',
             'running': 'desc',
         }
+
         sorts = self.get_arguments('sort') or default_sort
         orders = self.get_arguments('order')
 
@@ -176,11 +181,11 @@ class AdminHandler(BaseHandler):
 
         # this could be one incomprehensible nested list comprehension
         # get User columns
-        cols = [ getattr(orm.User, mapping.get(c, c)) for c in sorts ]
+        cols = [ mapping[c] for c in sorts ]
         # get User.col.desc() order objects
         ordered = [ getattr(c, o)() for c, o in zip(cols, orders) ]
 
-        users = self.db.query(orm.User).order_by(*ordered)
+        users = self.db.query(orm.User).join(orm.Spawner).order_by(*ordered)
         users = [ self._user_from_orm(u) for u in users ]
         running = [ u for u in users if u.running ]
 

jupyterhub/services/auth.py

@@ -9,11 +9,15 @@ model describing the authenticated user.
 authenticate with the Hub.
 
 """
+
+import base64
+import json
 import os
 import re
 import socket
 import time
 from urllib.parse import quote, urlencode
+import uuid
 import warnings
 
 import requests
@@ -397,6 +401,14 @@ class HubOAuth(HubAuth):
         """
         return self.oauth_client_id
 
+    @property
+    def state_cookie_name(self):
+        """The cookie name for storing OAuth state
+
+        This cookie is only live for the duration of the OAuth handshake.
+        """
+        return self.cookie_name + '-oauth-state'
+
     def _get_user_cookie(self, handler):
         token = handler.get_secure_cookie(self.cookie_name)
         if token:
@@ -476,6 +488,84 @@ class HubOAuth(HubAuth):
 
         return token_reply['access_token']
 
+    def _encode_state(self, state):
+        """Encode a state dict as url-safe base64"""
+        # trim trailing `=` because 
+        json_state = json.dumps(state)
+        return base64.urlsafe_b64encode(
+            json_state.encode('utf8')
+        ).decode('ascii').rstrip('=')
+
+    def _decode_state(self, b64_state):
+        """Decode a base64 state
+
+        Always returns a dict.
+        The dict will be empty if the state is invalid.
+        """
+        if isinstance(b64_state, str):
+            b64_state = b64_state.encode('ascii')
+        if len(b64_state) != 4:
+            # restore padding
+            b64_state = b64_state + (b'=' * (4 - len(b64_state) % 4))
+        try:
+            json_state = base64.urlsafe_b64decode(b64_state).decode('utf8')
+        except ValueError:
+            app_log.error("Failed to b64-decode state: %r", b64_state)
+            return {}
+        try:
+            return json.loads(json_state)
+        except ValueError:
+            app_log.error("Failed to json-decode state: %r", json_state)
+            return {}
+
+    def set_state_cookie(self, handler, next_url=None):
+        """Generate an OAuth state and store it in a cookie
+
+        Parameters
+        ----------
+        handler (RequestHandler): A tornado RequestHandler
+        next_url (str): The page to redirect to on successful login
+
+        Returns
+        -------
+        state (str): The OAuth state that has been stored in the cookie (url safe, base64-encoded)
+        """
+        b64_state = self.generate_state(next_url)
+        kwargs = {
+            'path': self.base_url,
+            'httponly': True,
+            'expires_days': 1,
+        }
+        if handler.request.protocol == 'https':
+            kwargs['secure'] = True
+        handler.set_secure_cookie(
+            self.state_cookie_name,
+            b64_state,
+            **kwargs
+        )
+        return b64_state
+
+    def generate_state(self, next_url=None):
+        """Generate a state string, given a next_url redirect target
+
+        Parameters
+        ----------
+        next_url (str): The URL of the page to redirect to on successful login.
+
+        Returns
+        -------
+        state (str): The base64-encoded state string.
+        """
+        return self._encode_state({
+            'uuid': uuid.uuid4().hex,
+            'next_url': next_url
+        })
+
+    def get_next_url(self, b64_state=''):
+        """Get the next_url for redirection, given an encoded OAuth state"""
+        state = self._decode_state(b64_state)
+        return state.get('next_url') or self.base_url
+
     def set_cookie(self, handler, access_token):
         """Set a cookie recording OAuth result"""
         kwargs = {
@@ -565,8 +655,14 @@ class HubAuthenticated(object):
 
     def get_login_url(self):
         """Return the Hub's login URL"""
-        app_log.debug("Redirecting to login url: %s" % self.hub_auth.login_url)
-        return self.hub_auth.login_url
+        login_url = self.hub_auth.login_url
+        app_log.debug("Redirecting to login url: %s", login_url)
+        if isinstance(self.hub_auth, HubOAuthenticated):
+            # add state argument to OAuth url
+            state = self.hub_auth.set_state_cookie(self, next_url=self.request.uri)
+            return url_concat(login_url, {'state': state})
+        else:
+            return login_url
 
     def check_hub_user(self, model):
         """Check whether Hub-authenticated user or service should be allowed.
@@ -657,6 +753,21 @@ class HubOAuthCallbackHandler(HubOAuthenticated, RequestHandler):
         code = self.get_argument("code", False)
         if not code:
             raise HTTPError(400, "oauth callback made without a token")
+
+        # validate OAuth state
+        arg_state = self.get_argument("state", None)
+        cookie_state = self.get_secure_cookie(self.hub_auth.state_cookie_name)
+        next_url = None
+        if arg_state or cookie_state:
+            # clear cookie state now that we've consumed it
+            self.clear_cookie(self.hub_auth.state_cookie_name)
+            if isinstance(cookie_state, bytes):
+                cookie_state = cookie_state.decode('ascii', 'replace')
+            # check that state matches
+            if arg_state != cookie_state:
+                app_log.debug("oauth state %r != %r", arg_state, cookie_state)
+                raise HTTPError(403, "oauth state does not match")
+            next_url = self.hub_auth.get_next_url(cookie_state)
         # TODO: make async (in a Thread?)
         token = self.hub_auth.token_for_code(code)
         user_model = self.hub_auth.user_for_token(token)
@@ -664,7 +775,6 @@ class HubOAuthCallbackHandler(HubOAuthenticated, RequestHandler):
             raise HTTPError(500, "oauth callback failed to identify a user")
         app_log.info("Logged-in user %s", user_model)
         self.hub_auth.set_cookie(self, token)
-        next_url = self.get_argument('next', '') or self.hub_auth.base_url
-        self.redirect(next_url)
+        self.redirect(next_url or self.hub_auth.base_url)
 
 

jupyterhub/singleuser.py

@@ -22,6 +22,7 @@ except ImportError:
 
 from traitlets import (
     Bool,
+    Bytes,
     Unicode,
     CUnicode,
     default,
@@ -115,20 +116,6 @@ class OAuthCallbackHandler(HubOAuthCallbackHandler, IPythonHandler):
     @property
     def hub_auth(self):
         return self.settings['hub_auth']
-    
-    def get(self):
-        code = self.get_argument("code", False)
-        if not code:
-            raise HTTPError(400, "oauth callback made without a token")
-        # TODO: make async (in a Thread?)
-        token = self.hub_auth.token_for_code(code)
-        user_model = self.hub_auth.user_for_token(token)
-        if user_model is None:
-            raise HTTPError(500, "oauth callback failed to identify a user")
-        self.log.info("Logged-in user %s", user_model)
-        self.hub_auth.set_cookie(self, token)
-        next_url = self.get_argument('next', '') or self.base_url
-        self.redirect(next_url)
 
 
 # register new hub related command-line aliases
@@ -192,6 +179,15 @@ class SingleUserNotebookApp(NotebookApp):
     subcommands = {}
     version = __version__
     classes = NotebookApp.classes + [HubOAuth]
+    
+    # don't store cookie secrets
+    cookie_secret_file = ''
+    # always generate a new cookie secret on launch
+    # ensures that each spawn clears any cookies from previous session,
+    # triggering OAuth again
+    cookie_secret = Bytes()
+    def _cookie_secret_default(self):
+        return os.urandom(32)
 
     user = CUnicode().tag(config=True)
     group = CUnicode().tag(config=True)

jupyterhub/tests/mocking.py

@@ -7,8 +7,6 @@ import threading
 from unittest import mock
 from urllib.parse import urlparse
 
-import requests
-
 from tornado import gen
 from tornado.concurrent import Future
 from tornado.ioloop import IOLoop
@@ -58,6 +56,13 @@ class MockSpawner(LocalProcessSpawner):
     def _cmd_default(self):
         return [sys.executable, '-m', 'jupyterhub.tests.mocksu']
 
+    use_this_api_token = None
+    def start(self):
+        if self.use_this_api_token:
+            self.api_token = self.use_this_api_token
+        elif self.will_resume:
+            self.use_this_api_token = self.api_token
+        return super().start()
 
 class SlowSpawner(MockSpawner):
     """A spawner that takes a few seconds to start"""

jupyterhub/tests/test_pages.py

@@ -85,11 +85,25 @@ def test_admin_not_admin(app):
 @pytest.mark.gen_test
 def test_admin(app):
     cookies = yield app.login_user('admin')
-    r = yield get_page('admin', app, cookies=cookies)
+    r = yield get_page('admin', app, cookies=cookies, allow_redirects=False)
     r.raise_for_status()
     assert r.url.endswith('/admin')
 
 
+@pytest.mark.parametrize('sort', [
+    'running',
+    'last_activity',
+    'admin',
+    'name',
+])
+@pytest.mark.gen_test
+def test_admin_sort(app, sort):
+    cookies = yield app.login_user('admin')
+    r = yield get_page('admin?sort=%s' % sort, app, cookies=cookies)
+    r.raise_for_status()
+    assert r.status_code == 200
+
+
 @pytest.mark.gen_test
 def test_spawn_redirect(app):
     name = 'wash'

jupyterhub/tests/test_spawner.py

@@ -15,11 +15,13 @@ from unittest import mock
 import pytest
 from tornado import gen
 
-from ..user import User
 from ..objects import Hub, Server
+from .. import orm
 from .. import spawner as spawnermod
 from ..spawner import LocalProcessSpawner, Spawner
-from .. import orm
+from ..user import User
+from ..utils import new_token
+from .test_api import add_user
 from .utils import async_requests
 
 _echo_sleep = """
@@ -270,3 +272,77 @@ def test_inherit_ok():
 
         def poll():
             pass
+
+
+@pytest.mark.gen_test
+def test_spawner_reuse_api_token(db, app):
+    # setup: user with no tokens, whose spawner has set the .will_resume flag
+    user = add_user(app.db, app, name='snoopy')
+    spawner = user.spawner
+    assert user.api_tokens == []
+    # will_resume triggers reuse of tokens
+    spawner.will_resume = True
+    # first start: gets a new API token
+    yield user.spawn()
+    api_token = spawner.api_token
+    found = orm.APIToken.find(app.db, api_token)
+    assert found
+    assert found.user.name == user.name
+    assert user.api_tokens == [found]
+    yield user.stop()
+    # second start: should reuse the token
+    yield user.spawn()
+    # verify re-use of API token
+    assert spawner.api_token == api_token
+    # verify that a new token was not created
+    assert user.api_tokens == [found]
+
+
+@pytest.mark.gen_test
+def test_spawner_insert_api_token(db, app):
+    """Token provided by spawner is not in the db
+    
+    Insert token into db as a user-provided token.
+    """
+    # setup: new user, double check that they don't have any tokens registered
+    user = add_user(app.db, app, name='tonkee')
+    spawner = user.spawner
+    assert user.api_tokens == []
+
+    # setup: spawner's going to use a token that's not in the db
+    api_token = new_token()
+    assert not orm.APIToken.find(app.db, api_token)
+    user.spawner.use_this_api_token = api_token
+    # The spawner's provided API token would already be in the db
+    # unless there is a bug somewhere else (in the Spawner),
+    # but handle it anyway.
+    yield user.spawn()
+    assert spawner.api_token == api_token
+    found = orm.APIToken.find(app.db, api_token)
+    assert found
+    assert found.user.name == user.name
+    assert user.api_tokens == [found]
+    yield user.stop()
+
+
+@pytest.mark.gen_test
+def test_spawner_bad_api_token(db, app):
+    """Tokens are revoked when a Spawner gets another user's token"""
+    # we need two users for this one
+    user = add_user(app.db, app, name='antimone')
+    spawner = user.spawner
+    other_user = add_user(app.db, app, name='alabaster')
+    assert user.api_tokens == []
+    assert other_user.api_tokens == []
+
+    # create a token owned by alabaster that antimone's going to try to use
+    other_token = other_user.new_api_token()
+    spawner.use_this_api_token = other_token
+    assert len(other_user.api_tokens) == 1
+
+    # starting a user's server with another user's token
+    # should revoke it
+    with pytest.raises(ValueError):
+        yield user.spawn()
+    assert orm.APIToken.find(app.db, other_token) is None
+    assert other_user.api_tokens == []

jupyterhub/user.py

@@ -385,12 +385,31 @@ class User(HasTraits):
                 # prior to 0.7, spawners had to store this info in user.server themselves.
                 # Handle < 0.7 behavior with a warning, assuming info was stored in db by the Spawner.
                 self.log.warning("DEPRECATION: Spawner.start should return (ip, port) in JupyterHub >= 0.7")
-            if spawner.api_token != api_token:
+            if spawner.api_token and spawner.api_token != api_token:
                 # Spawner re-used an API token, discard the unused api_token
                 orm_token = orm.APIToken.find(self.db, api_token)
                 if orm_token is not None:
                     self.db.delete(orm_token)
                     self.db.commit()
+                # check if the re-used API token is valid
+                found = orm.APIToken.find(self.db, spawner.api_token)
+                if found:
+                    if found.user is not self.orm_user:
+                        self.log.error("%s's server is using %s's token! Revoking this token.",
+                            self.name, (found.user or found.service).name)
+                        self.db.delete(found)
+                        self.db.commit()
+                        raise ValueError("Invalid token for %s!" % self.name)
+                else:
+                    # Spawner.api_token has changed, but isn't in the db.
+                    # What happened? Maybe something unclean in a resumed container.
+                    self.log.warning("%s's server specified its own API token that's not in the database",
+                        self.name
+                    )
+                    # use generated=False because we don't trust this token
+                    # to have been generated properly
+                    self.new_api_token(spawner.api_token, generated=False)
+
         except Exception as e:
             if isinstance(e, gen.TimeoutError):
                 self.log.warning("{user}'s server failed to start in {s} seconds, giving up".format(

singleuser/Dockerfile

@@ -0,0 +1,12 @@
+# Build as jupyterhub/singleuser
+# Run with the DockerSpawner in JupyterHub
+
+ARG BASE_IMAGE=jupyter/base-notebook
+FROM $BASE_IMAGE
+MAINTAINER Project Jupyter <jupyter@googlegroups.com>
+
+ADD install_jupyterhub /tmp/install_jupyterhub
+ARG JUPYTERHUB_VERSION=master
+# install pinned jupyterhub and ensure notebook is installed
+RUN python3 /tmp/install_jupyterhub && \
+    python3 -m pip install notebook

singleuser/README.md

@@ -0,0 +1,36 @@
+# jupyterhub/singleuser
+
+Built from the `jupyter/base-notebook` base image.
+
+This image contains a single user notebook server for use with
+[JupyterHub](https://github.com/jupyterhub/jupyterhub). In particular, it is meant
+to be used with the
+[DockerSpawner](https://github.com/jupyterhub/dockerspawner/blob/master/dockerspawner/dockerspawner.py)
+class to launch user notebook servers within docker containers.
+
+The only thing this image accomplishes is pinning the jupyterhub version on top of base-notebook.
+In most cases, one of the Jupyter [docker-stacks](https://github.com/jupyter/docker-stacks) is a better choice.
+You will just have to make sure that you have the right version of JupyterHub installed in your image,
+which can usually be accomplished with one line:
+
+```Dockerfile
+FROM jupyter/base-notebook:5ded1de07260
+RUN pip3 install jupyterhub==0.7.2
+```
+
+The dockerfile that builds this image exposes `BASE_IMAGE` and `JUPYTERHUB_VERSION` as build args, so you can do:
+
+    docker build -t singleuser \
+      --build-arg BASE_IMAGE=jupyter/scipy-notebook \
+      --build-arg JUPYTERHUB_VERSION=0.8.0 \
+      .
+
+in this directory to get a new image `singleuser` that is based on `jupyter/scipy-notebook` with JupyterHub 0.8, for example.
+
+This particular image runs as the `jovyan` user, with home directory at `/home/jovyan`.
+
+## Note on persistence
+
+This home directory, `/home/jovyan`, is *not* persistent by default,
+so some configuration is required unless the directory is to be used
+with temporary or demonstration JupyterHub deployments.

singleuser/hooks/build

@@ -0,0 +1,11 @@
+#!/bin/bash
+set -ex
+
+stable=0.7
+
+for V in master 0.7; do
+    docker build --build-arg JUPYTERHUB_VERSION=$V -t $DOCKER_REPO:$V .
+done
+
+echo "tagging $IMAGE_NAME"
+docker tag $DOCKER_REPO:$stable $IMAGE_NAME

singleuser/hooks/post_push

@@ -0,0 +1,23 @@
+#!/bin/bash
+
+for V in master 0.7; do
+  docker push $DOCKER_REPO:$V
+done
+
+function get_hub_version() {
+  rm -f hub_version
+  V=$1
+  docker run --rm -v $PWD:/version -u $(id -u) -i $DOCKER_REPO:$V sh -c 'jupyterhub --version > /version/hub_version'
+  hub_xyz=$(cat hub_version)
+  split=( ${hub_xyz//./ } )
+  hub_xy="${split[0]}.${split[1]}"
+}
+# tag e.g. 0.7.2 with 0.7
+get_hub_version 0.7
+docker tag $DOCKER_REPO:0.7 $DOCKER_REPO:$hub_xyz
+docker push $DOCKER_REPO:$hub_xyz
+
+# tag e.g. 0.8 with master
+get_hub_version master
+docker tag $DOCKER_REPO:master $DOCKER_REPO:$hub_xy
+docker push $DOCKER_REPO:$hub_xy

singleuser/install_jupyterhub

@@ -0,0 +1,21 @@
+#!/usr/bin/env python
+import os
+from subprocess import check_call
+import sys
+
+V = os.environ['JUPYTERHUB_VERSION']
+
+pip_install = [
+    sys.executable, '-m', 'pip', 'install', '--no-cache', '--upgrade',
+    '--upgrade-strategy', 'only-if-needed',
+]
+if V == 'master':
+    req = 'https://github.com/jupyterhub/jupyterhub/archive/master.tar.gz'
+else:
+    version_info = [ int(part) for part in V.split('.') ]
+    version_info[-1] += 1
+    upper_bound = '.'.join(map(str, version_info))
+    vs = '>=%s,<%s' % (V, upper_bound)
+    req = 'jupyterhub%s' % vs
+
+check_call(pip_install + [req])