release 0.9.4

- delete oauth clients for servers when they shutdown
- avoid deleting oauth clients for servers still running across an 0.8 -> 0.9 upgrade, when the oauth client ids changed from `user-NAME` to `jupyterhub-user-NAME`

.travis.yml

@@ -62,5 +62,7 @@ matrix:
     - python: 3.6
       env:
         - JUPYTERHUB_TEST_DB_URL=postgresql://postgres@127.0.0.1/jupyterhub
+    - python: 3.7
+      dist: xenial
   allow_failures:
     - python: nightly

README.md

@@ -11,8 +11,8 @@
 
 
 [![PyPI](https://img.shields.io/pypi/v/jupyterhub.svg)](https://pypi.python.org/pypi/jupyterhub)
-[![Documentation Status](https://readthedocs.org/projects/jupyterhub/badge/?version=latest)](http://jupyterhub.readthedocs.org/en/latest/?badge=latest)
-[![Documentation Status](http://readthedocs.org/projects/jupyterhub/badge/?version=0.7.2)](http://jupyterhub.readthedocs.io/en/0.7.2/?badge=0.7.2)
+[![Documentation Status](https://readthedocs.org/projects/jupyterhub/badge/?version=latest)](https://jupyterhub.readthedocs.org/en/latest/?badge=latest)
+[![Documentation Status](http://readthedocs.org/projects/jupyterhub/badge/?version=0.7.2)](https://jupyterhub.readthedocs.io/en/0.7.2/?badge=0.7.2)
 [![Build Status](https://travis-ci.org/jupyterhub/jupyterhub.svg?branch=master)](https://travis-ci.org/jupyterhub/jupyterhub)
 [![Circle CI](https://circleci.com/gh/jupyterhub/jupyterhub.svg?style=shield&circle-token=b5b65862eb2617b9a8d39e79340b0a6b816da8cc)](https://circleci.com/gh/jupyterhub/jupyterhub)
 [![codecov.io](https://codecov.io/github/jupyterhub/jupyterhub/coverage.svg?branch=master)](https://codecov.io/github/jupyterhub/jupyterhub?branch=master)
@@ -124,7 +124,7 @@ more configuration of the system.
 
 ## Configuration
 
-The [Getting Started](http://jupyterhub.readthedocs.io/en/latest/getting-started/index.html) section of the
+The [Getting Started](https://jupyterhub.readthedocs.io/en/latest/getting-started/index.html) section of the
 documentation explains the common steps in setting up JupyterHub.
 
 The [**JupyterHub tutorial**](https://github.com/jupyterhub/jupyterhub-tutorial)
@@ -233,7 +233,7 @@ our JupyterHub [Gitter](https://gitter.im/jupyterhub/jupyterhub) channel.
 
 - [Reporting Issues](https://github.com/jupyterhub/jupyterhub/issues)
 - [JupyterHub tutorial](https://github.com/jupyterhub/jupyterhub-tutorial)
-- [Documentation for JupyterHub](http://jupyterhub.readthedocs.io/en/latest/) | [PDF (latest)](https://media.readthedocs.org/pdf/jupyterhub/latest/jupyterhub.pdf) | [PDF (stable)](https://media.readthedocs.org/pdf/jupyterhub/stable/jupyterhub.pdf)
+- [Documentation for JupyterHub](https://jupyterhub.readthedocs.io/en/latest/) | [PDF (latest)](https://media.readthedocs.org/pdf/jupyterhub/latest/jupyterhub.pdf) | [PDF (stable)](https://media.readthedocs.org/pdf/jupyterhub/stable/jupyterhub.pdf)
 - [Documentation for JupyterHub's REST API](http://petstore.swagger.io/?url=https://raw.githubusercontent.com/jupyter/jupyterhub/master/docs/rest-api.yml#/default)
 - [Documentation for Project Jupyter](http://jupyter.readthedocs.io/en/latest/index.html) | [PDF](https://media.readthedocs.org/pdf/jupyter/latest/jupyter.pdf)
 - [Project Jupyter website](https://jupyter.org)

dev-requirements.txt

@@ -1,5 +1,6 @@
 -r requirements.txt
 mock
+beautifulsoup4
 codecov
 cryptography
 pytest-cov
@@ -8,3 +9,6 @@ pytest>=3.3
 notebook
 requests-mock
 virtualenv
+# temporary pin of attrs for jsonschema 0.3.0a1
+# seems to be a pip bug
+attrs>=17.4.0

docs/environment.yml

@@ -1,3 +1,5 @@
+# ReadTheDocs uses the `environment.yaml` so make sure to update that as well
+# if you change the dependencies of JupyterHub in the various `requirements.txt`
 name: jhub_docs
 channels:
   - conda-forge
@@ -17,3 +19,4 @@ dependencies:
   - recommonmark==0.4.0
   - async_generator
   - prometheus_client
+  - attrs>=17.4.0

docs/requirements.txt

@@ -1,3 +1,5 @@
+# ReadTheDocs uses the `environment.yaml` so make sure to update that as well
+# if you change this file
 -r ../requirements.txt
 sphinx>=1.7
 recommonmark==0.4.0

docs/rest-api.yml

@@ -3,7 +3,7 @@ swagger: '2.0'
 info:
   title: JupyterHub
   description: The REST API for JupyterHub
-  version: 0.9.0dev
+  version: 0.9.4
   license:
     name: BSD-3-Clause
 schemes:

docs/source/changelog.md

@@ -9,6 +9,53 @@ command line for details.
 
 ## 0.9
 
+### [0.9.4] 2018-09-24
+
+JupyterHub 0.9.4 is a small bugfix release.
+
+- Fixes an  issue that required all running user servers to be restarted
+  when performing an upgrade from 0.8 to 0.9.
+- Fixes content-type for API endpoints back to `application/json`.
+  It was `text/html` in 0.9.0-0.9.3.
+
+### [0.9.3] 2018-09-12
+
+JupyterHub 0.9.3 contains small bugfixes and improvements
+
+- Fix token page and model handling of `expires_at`.
+  This field was missing from the REST API model for tokens
+  and could cause the token page to not render
+- Add keep-alive to progress event stream to avoid proxies dropping
+  the connection due to inactivity
+- Documentation and example improvements
+- Disable quit button when using notebook 5.6
+- Prototype new feature (may change prior to 1.0):
+  pass requesting Handler to Spawners during start,
+  accessible as `self.handler`
+
+### [0.9.2] 2018-08-10
+
+JupyterHub 0.9.2 contains small bugfixes and improvements.
+
+- Documentation and example improvements
+- Add `Spawner.consecutive_failure_limit` config for aborting the Hub if too many spawns fail in a row.
+- Fix for handling SIGTERM when run with asyncio (tornado 5)
+- Windows compatibility fixes
+
+
+### [0.9.1] 2018-07-04
+
+JupyterHub 0.9.1 contains a number of small bugfixes on top of 0.9.
+
+- Use a PID file for the proxy to decrease the likelihood that a leftover proxy process will prevent JupyterHub from restarting
+- `c.LocalProcessSpawner.shell_cmd` is now configurable
+- API requests to stopped servers (requests to the hub for `/user/:name/api/...`) fail with 404 rather than triggering a restart of the server
+- Compatibility fix for notebook 5.6.0 which will introduce further
+  security checks for local connections
+- Managed services always use localhost to talk to the Hub if the Hub listening on all interfaces
+- When using a URL prefix, the Hub route will be `JupyterHub.base_url` instead of unconditionally `/`
+- additional fixes and improvements
+
 ### [0.9.0] 2018-06-15
 
 JupyterHub 0.9 is a major upgrade of JupyterHub.
@@ -95,7 +142,7 @@ and tornado < 5.0.
 - Added "Start All" button to admin page for launching all user servers at once.
 - Services have an `info` field which is a dictionary.
   This is accessible via the REST API.
-- `JupyterHub.extra_handlers` allows defining additonal tornado RequestHandlers attached to the Hub.
+- `JupyterHub.extra_handlers` allows defining additional tornado RequestHandlers attached to the Hub.
 - API tokens may now expire.
   Expiry is available in the REST model as `expires_at`,
   and settable when creating API tokens by specifying `expires_in`.
@@ -379,7 +426,11 @@ Fix removal of `/login` page in 0.4.0, breaking some OAuth providers.
 First preview release
 
 
-[Unreleased]: https://github.com/jupyterhub/jupyterhub/compare/0.9.0...HEAD
+[Unreleased]: https://github.com/jupyterhub/jupyterhub/compare/0.9.4...HEAD
+[0.9.4]: https://github.com/jupyterhub/jupyterhub/compare/0.9.3...0.9.4
+[0.9.3]: https://github.com/jupyterhub/jupyterhub/compare/0.9.2...0.9.3
+[0.9.2]: https://github.com/jupyterhub/jupyterhub/compare/0.9.1...0.9.2
+[0.9.1]: https://github.com/jupyterhub/jupyterhub/compare/0.9.0...0.9.1
 [0.9.0]: https://github.com/jupyterhub/jupyterhub/compare/0.8.1...0.9.0
 [0.8.1]: https://github.com/jupyterhub/jupyterhub/compare/0.8.0...0.8.1
 [0.8.0]: https://github.com/jupyterhub/jupyterhub/compare/0.7.2...0.8.0

docs/source/getting-started/security-basics.rst

@@ -45,7 +45,7 @@ is important that these files be put in a secure location on your server, where
 they are not readable by regular users.
 
 If you are using a **chain certificate**, see also chained certificate for SSL
-in the JupyterHub `troubleshooting FAQ <troubleshooting>`_.
+in the JupyterHub `Troubleshooting FAQ <../troubleshooting.html>`_.
 
 Using letsencrypt
 ~~~~~~~~~~~~~~~~~

docs/source/index.rst

@@ -58,6 +58,8 @@ Contents
 * :doc:`reference/services`
 * :doc:`reference/rest`
 * :doc:`reference/upgrading`
+* :doc:`reference/templates`
+* :doc:`reference/config-user-env`
 * :doc:`reference/config-examples`
 * :doc:`reference/config-ghoauth`
 * :doc:`reference/config-proxy`

docs/source/reference/authenticators.md

@@ -226,5 +226,5 @@ Beginning with version 0.8, JupyterHub is an OAuth provider.
 [OAuth]: https://en.wikipedia.org/wiki/OAuth
 [GitHub OAuth]: https://developer.github.com/v3/oauth/
 [OAuthenticator]: https://github.com/jupyterhub/oauthenticator
-[pre_spawn_start(user, spawner)]: http://jupyterhub.readthedocs.io/en/latest/api/auth.html#jupyterhub.auth.Authenticator.pre_spawn_start
-[post_spawn_stop(user, spawner)]: http://jupyterhub.readthedocs.io/en/latest/api/auth.html#jupyterhub.auth.Authenticator.post_spawn_stop
+[pre_spawn_start(user, spawner)]: https://jupyterhub.readthedocs.io/en/latest/api/auth.html#jupyterhub.auth.Authenticator.pre_spawn_start
+[post_spawn_stop(user, spawner)]: https://jupyterhub.readthedocs.io/en/latest/api/auth.html#jupyterhub.auth.Authenticator.post_spawn_stop

docs/source/reference/config-sudo.md

@@ -70,7 +70,7 @@ Cmnd_Alias JUPYTER_CMD = /usr/local/bin/sudospawner
 rhea ALL=(JUPYTER_USERS) NOPASSWD:JUPYTER_CMD
 ```
 
-It might be useful to modifiy `secure_path` to add commands in path.
+It might be useful to modify `secure_path` to add commands in path.
 
 As an alternative to adding every user to the `/etc/sudoers` file, you can
 use a group in the last line above, instead of `JUPYTER_USERS`:

docs/source/reference/config-user-env.md

@@ -125,7 +125,7 @@ sure are available, I can install their specs system-wide (in /usr/local) with:
 There are two broad categories of user environments that depend on what
 Spawner you choose:
 
-- Multi-user hosts (shared sytem)
+- Multi-user hosts (shared system)
 - Container-based
 
 How you configure user environments for each category can differ a bit

docs/source/reference/proxy.md

@@ -2,7 +2,7 @@
 
 JupyterHub 0.8 introduced the ability to write a custom implementation of the
 proxy. This enables deployments with different needs than the default proxy,
-configurable-http-proxy (CHP). CHP is a single-process nodejs proxy that they
+configurable-http-proxy (CHP). CHP is a single-process nodejs proxy that the
 Hub manages by default as a subprocess (it can be run externally, as well, and
 typically is in production deployments).
 

docs/source/reference/services.md

@@ -15,7 +15,7 @@ This section provides the following information about Services:
 ## Definition of a Service
 
 When working with JupyterHub, a **Service** is defined as a process that interacts
-with the Hub's REST API. A Service may perform a specific or
+with the Hub's REST API. A Service may perform a specific
 action or task. For example, the following tasks can each be a unique Service:
 
 - shutting down individuals' single user notebook servers that have been idle
@@ -205,9 +205,9 @@ To use HubAuth, you must set the `.api_token`, either programmatically when cons
 or via the `JUPYTERHUB_API_TOKEN` environment variable.
 
 Most of the logic for authentication implementation is found in the 
-[`HubAuth.user_for_cookie`](services.auth.html#jupyterhub.services.auth.HubAuth.user_for_cookie)
+[`HubAuth.user_for_cookie`][HubAuth.user_for_cookie] 
 and in the 
-[`HubAuth.user_for_token`](services.auth.html#jupyterhub.services.auth.HubAuth.user_for_token)
+[`HubAuth.user_for_token`][HubAuth.user_for_token]
 methods, which makes a request of the Hub, and returns:
 
 - None, if no user could be identified, or
@@ -359,14 +359,16 @@ and taking note of the following process:
    ```
 
 An example of using an Externally-Managed Service and authentication is
-in [nbviewer README]_ section on securing the notebook viewer,
+in [nbviewer README][nbviewer example] section on securing the notebook viewer,
 and an example of its configuration is found [here](https://github.com/jupyter/nbviewer/blob/master/nbviewer/providers/base.py#L94).
-nbviewer can also be run as a Hub-Managed Service as described [nbviewer README]_
+nbviewer can also be run as a Hub-Managed Service as described [nbviewer README][nbviewer example]
 section on securing the notebook viewer.
 
 
 [requests]: http://docs.python-requests.org/en/master/
 [services_auth]: ../api/services.auth.html
 [HubAuth]: ../api/services.auth.html#jupyterhub.services.auth.HubAuth
+[HubAuth.user_for_cookie]: ../api/services.auth.html#jupyterhub.services.auth.HubAuth.user_for_cookie
+[HubAuth.user_for_token]: ../api/services.auth.html#jupyterhub.services.auth.HubAuth.user_for_token
 [HubAuthenticated]: ../api/services.auth.html#jupyterhub.services.auth.HubAuthenticated
 [nbviewer example]: https://github.com/jupyter/nbviewer#securing-the-notebook-viewer

docs/source/reference/spawners.md

@@ -46,7 +46,16 @@ Most `Spawner.start` functions will look similar to this example:
 def start(self):
     self.ip = '127.0.0.1'
     self.port = random_port()
-    yield self._actually_start_server_somehow()
+    # get environment variables,
+    # several of which are required for configuring the single-user server
+    env = self.get_env()
+    cmd = []
+    # get jupyterhub command to run,
+    # typically ['jupyterhub-singleuser']
+    cmd.extend(self.cmd)
+    cmd.extend(self.get_args())
+
+    yield self._actually_start_server_somehow(cmd, env)
     return (self.ip, self.port)
 ```
 
@@ -187,7 +196,7 @@ allocate. Attempting to use more memory than this limit will cause errors. The
 single-user notebook server can discover its own memory limit by looking at
 the environment variable `MEM_LIMIT`, which is specified in absolute bytes.
 
-`c.Spawner.mem_guarantee`: Sometimes, a **guarantee** of a *minumum amount of
+`c.Spawner.mem_guarantee`: Sometimes, a **guarantee** of a *minimum amount of
 memory* is desirable. In this case, you can set `c.Spawner.mem_guarantee` to
 to provide a guarantee that at minimum this much memory will always be
 available for the single-user notebook server to use. The environment variable

docs/source/reference/templates.md

@@ -75,7 +75,7 @@ the top of all pages.  The more specific variables
 `announcement_login`, `announcement_spawn`, `announcement_home`, and
 `announcement_logout` are more specific and only show on their
 respective pages (overriding the global `announcement` variable).
-Note that changing these varables require a restart, unlike direct
+Note that changing these variables require a restart, unlike direct
 template extension.
 
 You can get the same effect by extending templates, which allows you

docs/source/spelling_wordlist.txt

@@ -166,7 +166,7 @@ startup
 statsd
 stdin
 stdout
-stoppped
+stopped
 subclasses
 subcommand
 subdomain

examples/bootstrap-script/jupyterhub_config.py

@@ -1,9 +1,14 @@
-# Example for a Spawner.pre_spawn_hook
-# create a directory for the user before the spawner starts
-
+"""
+Example for a Spawner.pre_spawn_hook
+create a directory for the user before the spawner starts
+"""
+# pylint: disable=import-error
 import os
 import shutil
+from jupyter_client.localinterfaces import public_ips
+
 def create_dir_hook(spawner):
+    """ Create directory """
     username = spawner.user.name # get the username
     volume_path = os.path.join('/volumes/jupyterhub', username)
     if not os.path.exists(volume_path):
@@ -12,23 +17,24 @@ def create_dir_hook(spawner):
         # ...
 
 def clean_dir_hook(spawner):
+    """ Delete directory """
     username = spawner.user.name # get the username
     temp_path = os.path.join('/volumes/jupyterhub', username, 'temp')
     if os.path.exists(temp_path) and os.path.isdir(temp_path):
         shutil.rmtree(temp_path)
 
 # attach the hook functions to the spawner
+# pylint: disable=undefined-variable
 c.Spawner.pre_spawn_hook = create_dir_hook
 c.Spawner.post_stop_hook = clean_dir_hook
 
 # Use the DockerSpawner to serve your users' notebooks
 c.JupyterHub.spawner_class = 'dockerspawner.DockerSpawner'
-from jupyter_client.localinterfaces import public_ips
 c.JupyterHub.hub_ip = public_ips()[0]
 c.DockerSpawner.hub_ip_connect = public_ips()[0]
 c.DockerSpawner.container_ip = "0.0.0.0"
 
 # You can now mount the volume to the docker container as we've
 # made sure the directory exists
+# pylint: disable=bad-whitespace
 c.DockerSpawner.volumes = { '/volumes/jupyterhub/{username}/': '/home/jovyan/work' }
-

examples/cull-idle/cull_idle_servers.py

@@ -186,10 +186,16 @@ def cull_idle(url, api_token, inactive_limit, cull_users=False, max_age=0, concu
                 log_name, format_td(age), format_td(inactive))
             return False
 
+        if server_name:
+            # culling a named server
+            delete_url = url + "/users/%s/servers/%s" % (
+                quote(user['name']), quote(server['name'])
+            )
+        else:
+            delete_url = url + '/users/%s/server' % quote(user['name'])
+
         req = HTTPRequest(
-            url=url + '/users/%s/server' % quote(user['name']),
-            method='DELETE',
-            headers=auth_header,
+            url=delete_url, method='DELETE', headers=auth_header,
         )
         resp = yield fetch(req)
         if resp.code == 202:

examples/service-announcement/README.md

@@ -0,0 +1,60 @@
+
+# Simple Announcement Service Example
+
+This is a simple service that allows administrators to manage announcements
+that appear when JupyterHub renders pages.
+
+To run the service as a hub-managed service simply include in your JupyterHub
+configuration file something like:
+
+    c.JupyterHub.services = [
+            {
+                'name': 'announcement',
+                'url': 'http://127.0.0.1:8888',
+                'command': ["python", "-m", "announcement"],
+            }
+    ]
+
+This starts the announcements service up at `/services/announcement` when
+JupyterHub launches.  By default the announcement text is empty.
+
+The `announcement` module has a configurable port (default 8888) and an API
+prefix setting.  By default the API prefix is `JUPYTERHUB_SERVICE_PREFIX` if
+that environment variable is set or `/` if it is not.
+
+## Managing the Announcement
+
+Admin users can set the announcement text with an API token:
+
+    $ curl -X POST -H "Authorization: token <token>"                        \
+        -d "{'announcement':'JupyterHub will be upgraded on August 14!'}"   \
+        https://.../services/announcement
+
+Anyone can read the announcement:
+
+    $ curl https://.../services/announcement | python -m json.tool
+    {
+        announcement: "JupyterHub will be upgraded on August 14!",
+        timestamp: "...",
+        user: "..."
+    }
+
+The time the announcement was posted is recorded in the `timestamp` field and
+the user who posted the announcement is recorded in the `user` field.
+
+To clear the announcement text, just DELETE.  Only admin users can do this.
+
+    $ curl -X POST -H "Authorization: token <token>"                        \
+        https://.../services/announcement
+
+## Seeing the Announcement in JupyterHub
+
+To be able to render the announcement, include the provide `page.html` template
+that extends the base `page.html` template.  Set `c.JupyterHub.template_paths`
+in JupyterHub's configuration to include the path to the extending template.
+The template changes the `announcement` element and does a JQuery `$.get()` call
+to retrieve the announcement text.
+
+JupyterHub's configurable announcement template variables can be set for various
+pages like login, logout, spawn, and home.  Including the template provided in
+this example overrides all of those.

examples/service-announcement/announcement.py

@@ -0,0 +1,73 @@
+
+import argparse
+import datetime
+import json
+import os
+
+from jupyterhub.services.auth import HubAuthenticated
+from tornado import escape, gen, ioloop, web
+
+
+class AnnouncementRequestHandler(HubAuthenticated, web.RequestHandler):
+    """Dynamically manage page announcements"""
+
+    hub_users = []
+    allow_admin = True
+
+    def initialize(self, storage):
+        """Create storage for announcement text"""
+        self.storage = storage
+
+    @web.authenticated
+    def post(self):
+        """Update announcement"""
+        doc = escape.json_decode(self.request.body)
+        self.storage["announcement"] = doc["announcement"]
+        self.storage["timestamp"] = datetime.datetime.now().isoformat()
+        self.storage["user"] = user["name"]
+        self.write_to_json(self.storage)
+
+    def get(self):
+        """Retrieve announcement"""
+        self.write_to_json(self.storage)
+
+    @web.authenticated
+    def delete(self):
+        """Clear announcement"""
+        self.storage["announcement"] = ""
+        self.write_to_json(self.storage)
+
+    def write_to_json(self, doc):
+        """Write dictionary document as JSON"""
+        self.set_header("Content-Type", "application/json; charset=UTF-8")
+        self.write(escape.utf8(json.dumps(doc)))
+
+
+def main():
+    args = parse_arguments()
+    application = create_application(**vars(args))
+    application.listen(args.port)
+    ioloop.IOLoop.current().start()
+
+
+def parse_arguments():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--api-prefix", "-a",
+            default=os.environ.get("JUPYTERHUB_SERVICE_PREFIX", "/"),
+            help="application API prefix")
+    parser.add_argument("--port", "-p",
+            default=8888,
+            help="port for API to listen on",
+            type=int)
+    return parser.parse_args()
+
+
+def create_application(api_prefix="/", 
+        handler=AnnouncementRequestHandler,
+        **kwargs):
+    storage = dict(announcement="", timestamp="", user="")
+    return web.Application([(api_prefix, handler, dict(storage=storage))])
+
+
+if __name__ == "__main__":
+    main()

examples/service-announcement/jupyterhub_config.py

@@ -0,0 +1,15 @@
+
+# To run the announcement service managed by the hub, add this.
+
+c.JupyterHub.services = [
+        {
+            'name': 'announcement',
+            'url': 'http://127.0.0.1:8888',
+            'command': ["python", "-m", "announcement"],
+        }
+]
+
+# The announcements need to get on the templates somehow, see page.html
+# for an example of how to do this.
+
+c.JupyterHub.template_paths = ["templates"]

examples/service-announcement/templates/page.html

@@ -0,0 +1,14 @@
+{% extends "templates/page.html" %}
+{% block announcement %}
+<div class="container text-center announcement">
+</div>
+{% endblock %}
+
+{% block script %}
+{{ super() }}
+<script>
+$.get("/services/announcement/", function(data) {
+    $(".announcement").html(data["announcement"]);
+});
+</script>
+{% endblock %}

hooks/post_push

@@ -11,12 +11,16 @@ function get_hub_version() {
   hub_xyz=$(cat hub_version)
   split=( ${hub_xyz//./ } )
   hub_xy="${split[0]}.${split[1]}"
+  # add .dev on hub_xy so it's 1.0.dev
+  if [[ ! -z "${split[3]}" ]]; then
+    hub_xy="${hub_xy}.${split[3]}"
+  fi
 }
 
 
 get_hub_version
 
-# when building master, push 0.9.0 as well
+# when building master, push 0.9.0.dev as well
 docker tag $DOCKER_REPO:$DOCKER_TAG $DOCKER_REPO:$hub_xyz
 docker push $DOCKER_REPO:$hub_xyz
 docker tag $ONBUILD:$DOCKER_TAG $ONBUILD:$hub_xyz

jupyterhub/_version.py

@@ -6,9 +6,9 @@
 version_info = (
     0,
     9,
-    0,
-    "",  # release (b1, rc1)
-    # "dev",  # dev
+    4,
+    "",  # release (b1, rc1, or "" for final or dev)
+    # "dev",  # dev or nothing
 )
 
 # pep 440 version: no dot before beta/rc, but before .dev

jupyterhub/apihandlers/base.py

@@ -2,6 +2,7 @@
 # Copyright (c) Jupyter Development Team.
 # Distributed under the terms of the Modified BSD License.
 
+from datetime import datetime
 import json
 
 from http.client import responses
@@ -13,12 +14,25 @@ from .. import orm
 from ..handlers import BaseHandler
 from ..utils import isoformat, url_path_join
 
+
 class APIHandler(BaseHandler):
+    """Base class for API endpoints
+
+    Differences from page handlers:
+
+    - JSON responses and errors
+    - strict referer checking for Cookie-authenticated requests
+    - strict content-security-policy
+    - methods for REST API models
+    """
 
     @property
     def content_security_policy(self):
         return '; '.join([super().content_security_policy, "default-src 'none'"])
 
+    def get_content_type(self):
+        return 'application/json'
+
     def check_referer(self):
         """Check Origin for cross-site API requests.
 
@@ -156,6 +170,7 @@ class APIHandler(BaseHandler):
             'kind': kind,
             'created': isoformat(token.created),
             'last_activity': isoformat(token.last_activity),
+            'expires_at': isoformat(expires_at),
         }
         model.update(extra)
         return model
@@ -253,3 +268,13 @@ class APIHandler(BaseHandler):
 
     def options(self, *args, **kwargs):
         self.finish()
+
+
+class API404(APIHandler):
+    """404 for API requests
+
+    Ensures JSON 404 errors for malformed URLs
+    """
+    async def prepare(self):
+        await super().prepare()
+        raise web.HTTPError(404)

jupyterhub/apihandlers/users.py

@@ -428,6 +428,9 @@ class UserAdminAccessAPIHandler(APIHandler):
 
 class SpawnProgressAPIHandler(APIHandler):
     """EventStream handler for pending spawns"""
+
+    keepalive_interval = 8
+
     def get_content_type(self):
         return 'text/event-stream'
 
@@ -440,6 +443,23 @@ class SpawnProgressAPIHandler(APIHandler):
             # raise Finish to halt the handler
             raise web.Finish()
 
+    _finished = False
+    def on_finish(self):
+        self._finished = True
+
+    async def keepalive(self):
+        """Write empty lines periodically
+
+        to avoid being closed by intermediate proxies
+        when there's a large gap between events.
+        """
+        while not self._finished:
+            try:
+                self.write("\n\n")
+            except (StreamClosedError, RuntimeError):
+                return
+            await asyncio.sleep(self.keepalive_interval)
+
     @admin_or_self
     async def get(self, username, server_name=''):
         self.set_header('Cache-Control', 'no-cache')
@@ -453,6 +473,9 @@ class SpawnProgressAPIHandler(APIHandler):
             # user has no such server
             raise web.HTTPError(404)
         spawner = user.spawners[server_name]
+
+        # start sending keepalive to avoid proxies closing the connection
+        asyncio.ensure_future(self.keepalive())
         # cases:
         # - spawner already started and ready
         # - spawner not running at all

jupyterhub/app.py

@@ -973,6 +973,8 @@ class JupyterHub(Application):
         h.extend(self.extra_handlers)
 
         h.append((r'/logo', LogoHandler, {'path': self.logo_file}))
+        h.append((r'/api/(.*)', apihandlers.base.API404))
+
         self.handlers = self.add_url_prefix(self.hub_prefix, h)
         # some extra handlers, outside hub_prefix
         self.handlers.extend([
@@ -1108,7 +1110,18 @@ class JupyterHub(Application):
         else:
             hub_args['ip'] = self.hub_ip
             hub_args['port'] = self.hub_port
-        self.hub = Hub(**hub_args)
+
+        # routespec for the Hub is the *app* base url
+        # not the hub URL, so it receives requests for non-running servers
+        # use `/` with host-based routing so the Hub
+        # gets requests for all hosts
+        host = ''
+        if self.subdomain_host:
+            routespec = '/'
+        else:
+            routespec = self.base_url
+
+        self.hub = Hub(routespec=routespec, **hub_args)
 
         if self.hub_connect_ip:
             self.hub.connect_ip = self.hub_connect_ip
@@ -1495,6 +1508,10 @@ class JupyterHub(Application):
         for user in self.users.values():
             for spawner in user.spawners.values():
                 oauth_client_ids.add(spawner.oauth_client_id)
+                # avoid deleting clients created by 0.8
+                # 0.9 uses `jupyterhub-user-...` for the client id, while
+                # 0.8 uses just `user-...`
+                oauth_client_ids.add(spawner.oauth_client_id.split('-', 1)[1])
 
         client_store = self.oauth_provider.client_authenticator.client_store
         for i, oauth_client in enumerate(self.db.query(orm.OAuthClient)):
@@ -1910,8 +1927,7 @@ class JupyterHub(Application):
 
     def sigterm(self, signum, frame):
         self.log.critical("Received SIGTERM, shutting down")
-        self.io_loop.stop()
-        self.atexit()
+        raise SystemExit(128 + signum)
 
     _atexit_ran = False
 
@@ -1921,6 +1937,7 @@ class JupyterHub(Application):
             return
         self._atexit_ran = True
         # run the cleanup step (in a new loop, because the interrupted one is unclean)
+        asyncio.set_event_loop(asyncio.new_event_loop())
         IOLoop.clear_current()
         loop = IOLoop()
         loop.make_current()

jupyterhub/handlers/base.py

@@ -6,6 +6,7 @@
 import copy
 from datetime import datetime, timedelta
 from http.client import responses
+import json
 import math
 import random
 import re
@@ -601,7 +602,7 @@ class BaseHandler(RequestHandler):
 
         self.log.debug("Initiating spawn for %s", user_server_name)
 
-        spawn_future = user.spawn(server_name, options)
+        spawn_future = user.spawn(server_name, options, handler=self)
 
         self.log.debug("%i%s concurrent spawns",
             spawn_pending_count,
@@ -656,6 +657,7 @@ class BaseHandler(RequestHandler):
         # hook up spawner._spawn_future so that other requests can await
         # this result
         finish_spawn_future = spawner._spawn_future = maybe_future(finish_user_spawn())
+
         def _clear_spawn_future(f):
             # clear spawner._spawn_future when it's done
             # keep an exception around, though, to prevent repeated implicit spawns
@@ -664,10 +666,44 @@ class BaseHandler(RequestHandler):
                 spawner._spawn_future = None
             # Now we're all done. clear _spawn_pending flag
             spawner._spawn_pending = False
+
         finish_spawn_future.add_done_callback(_clear_spawn_future)
 
+        # when spawn finishes (success or failure)
+        # update failure count and abort if consecutive failure limit
+        # is reached
+        def _track_failure_count(f):
+            if f.exception() is None:
+                # spawn succeeded, reset failure count
+                self.settings['failure_count'] = 0
+                return
+            # spawn failed, increment count and abort if limit reached
+            self.settings.setdefault('failure_count', 0)
+            self.settings['failure_count'] += 1
+            failure_count = self.settings['failure_count']
+            failure_limit = spawner.consecutive_failure_limit
+            if failure_limit and 1 < failure_count < failure_limit:
+                self.log.warning(
+                    "%i consecutive spawns failed.  "
+                    "Hub will exit if failure count reaches %i before succeeding",
+                    failure_count, failure_limit,
+                )
+            if failure_limit and failure_count >= failure_limit:
+                self.log.critical(
+                    "Aborting due to %i consecutive spawn failures", failure_count
+                )
+                # abort in 2 seconds to allow pending handlers to resolve
+                # mostly propagating errors for the current failures
+                def abort():
+                    raise SystemExit(1)
+                IOLoop.current().call_later(2, abort)
+
+        finish_spawn_future.add_done_callback(_track_failure_count)
+
         try:
-            await gen.with_timeout(timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future)
+            await gen.with_timeout(
+                timedelta(seconds=self.slow_spawn_timeout), finish_spawn_future
+            )
         except gen.TimeoutError:
             # waiting_for_response indicates server process has started,
             # but is yet to become responsive.
@@ -865,6 +901,11 @@ class PrefixRedirectHandler(BaseHandler):
     """
     def get(self):
         uri = self.request.uri
+        # Since self.base_url will end with trailing slash.
+        # Ensure uri will end with trailing slash when matching
+        # with self.base_url.
+        if not uri.endswith('/'):
+            uri += '/'
         if uri.startswith(self.base_url):
             path = self.request.uri[len(self.base_url):]
         else:
@@ -884,6 +925,13 @@ class UserSpawnHandler(BaseHandler):
     which will in turn send her to /user/alice/notebooks/mynotebook.ipynb.
     """
 
+    def _fail_api_request(self, user):
+        """Fail an API request to a not-running server"""
+        self.set_status(404)
+        self.set_header("Content-Type", "application/json")
+        self.write(json.dumps({"message": "%s is not running" % user.name}))
+        self.finish()
+
     async def get(self, name, user_path):
         if not user_path:
             user_path = '/'
@@ -910,6 +958,11 @@ class UserSpawnHandler(BaseHandler):
             # otherwise redirect users to their own server
             should_spawn = (current_user and current_user.name == name)
 
+        if "api" in user_path.split("/") and not user.active:
+            # API request for not-running server (e.g. notebook UI left open)
+            # Avoid triggering a spawn.
+            self._fail_api_request(user)
+            return
 
         if should_spawn:
             # if spawning fails for any reason, point users to /hub/home to retry

jupyterhub/handlers/pages.py

@@ -111,7 +111,11 @@ class SpawnHandler(BaseHandler):
             if user.spawner._spawn_future and user.spawner._spawn_future.done():
                 user.spawner._spawn_future = None
             # not running, no form. Trigger spawn by redirecting to /user/:name
-            self.redirect(user.url)
+            url = user.url
+            if self.request.query:
+                # add query params
+                url += '?' + self.request.query
+            self.redirect(url)
 
     @web.authenticated
     async def post(self, for_user=None):
@@ -243,9 +247,11 @@ class TokenPageHandler(BaseHandler):
             api_tokens.append(token)
 
         # group oauth client tokens by client id
+        # AccessTokens have expires_at as an integer timestamp
+        now_timestamp = now.timestamp()
         oauth_tokens = defaultdict(list)
         for token in user.oauth_tokens:
-            if token.expires_at and token.expires_at < now:
+            if token.expires_at and token.expires_at < now_timestamp:
                 self.log.warning("Deleting expired token")
                 self.db.delete(token)
                 self.db.commit()

jupyterhub/objects.py

@@ -63,6 +63,9 @@ class Server(HasTraits):
     @validate('connect_url')
     def _connect_url_add_prefix(self, proposal):
         """Ensure connect_url includes base_url"""
+        if not proposal.value:
+            # Don't add the prefix if the setting is being cleared
+            return proposal.value
         urlinfo = urlparse(proposal.value)
         if not urlinfo.path.startswith(self.base_url):
             urlinfo = urlinfo._replace(path=self.base_url)
@@ -185,6 +188,7 @@ class Hub(Server):
         )
         return self
     public_host = Unicode()
+    routespec = Unicode()
 
     @property
     def api_url(self):

jupyterhub/orm.py

@@ -746,7 +746,7 @@ def new_session_factory(url="sqlite:///:memory:",
     Base.metadata.create_all(engine)
 
     # We set expire_on_commit=False, since we don't actually need
-    # SQLAlchemy to expire objects after commiting - we don't expect
+    # SQLAlchemy to expire objects after committing - we don't expect
     # concurrent runs of the hub talking to the same db. Turning
     # this off gives us a major performance boost
     session_factory = sessionmaker(bind=engine,

jupyterhub/proxy.py

@@ -22,8 +22,10 @@ import asyncio
 from functools import wraps
 import json
 import os
+import signal
 from subprocess import Popen
-from urllib.parse import quote
+import time
+from urllib.parse import quote, urlparse
 
 from tornado import gen
 from tornado.httpclient import AsyncHTTPClient, HTTPRequest, HTTPError
@@ -301,13 +303,13 @@ class Proxy(LoggingConfigurable):
         user_routes = {path for path, r in routes.items() if 'user' in r['data']}
         futures = []
 
-        good_routes = {'/'}
+        good_routes = {self.app.hub.routespec}
 
-        hub = self.app.hub
-        if '/' not in routes:
+        hub = self.hub
+        if self.app.hub.routespec not in routes:
             futures.append(self.add_hub_route(hub))
         else:
-            route = routes['/']
+            route = routes[self.app.hub.routespec]
             if route['target'] != hub.host:
                 self.log.warning("Updating default route %s → %s", route['target'], hub.host)
                 futures.append(self.add_hub_route(hub))
@@ -365,8 +367,8 @@ class Proxy(LoggingConfigurable):
 
     def add_hub_route(self, hub):
         """Add the default route for the Hub"""
-        self.log.info("Adding default route for Hub: / => %s", hub.host)
-        return self.add_route('/', self.hub.host, {'hub': True})
+        self.log.info("Adding default route for Hub: %s => %s", hub.routespec, hub.host)
+        return self.add_route(hub.routespec, self.hub.host, {'hub': True})
 
     async def restore_routes(self):
         self.log.info("Setting up routes on new proxy")
@@ -437,8 +439,22 @@ class ConfigurableHTTPProxy(Proxy):
                       help="""The command to start the proxy"""
                       )
 
+    pid_file = Unicode(
+        "jupyterhub-proxy.pid",
+        config=True,
+        help="File in which to write the PID of the proxy process.",
+    )
+
     _check_running_callback = Any(help="PeriodicCallback to check if the proxy is running")
 
+    def _check_pid(self, pid):
+        if os.name == 'nt':
+            import psutil
+            if not psutil.pid_exists(pid):
+                raise ProcessLookupError
+        else:
+            os.kill(pid, 0)
+
     def __init__(self, **kwargs):
         super().__init__(**kwargs)
         # check for required token if proxy is external
@@ -448,9 +464,77 @@ class ConfigurableHTTPProxy(Proxy):
                 " if Proxy.should_start is False" % self.__class__.__name__
             )
 
+    def _check_previous_process(self):
+        """Check if there's a process leftover and shut it down if so"""
+        if not self.pid_file or not os.path.exists(self.pid_file):
+            return
+        pid_file = os.path.abspath(self.pid_file)
+        self.log.warning("Found proxy pid file: %s", pid_file)
+        try:
+            with open(pid_file, "r") as f:
+                pid = int(f.read().strip())
+        except ValueError:
+            self.log.warning("%s did not appear to contain a pid", pid_file)
+            self._remove_pid_file()
+            return
 
+        try:
+            self._check_pid(pid)
+        except ProcessLookupError:
+            self.log.warning("Proxy no longer running at pid=%s", pid)
+            self._remove_pid_file()
+            return
+
+        # if we got here, CHP is still running
+        self.log.warning("Proxy still running at pid=%s", pid)
+        if os.name != 'nt':
+            sig_list = [signal.SIGTERM] * 2 + [signal.SIGKILL]
+        for i in range(3):
+            try:
+                if os.name == 'nt':
+                    self._terminate_win(pid)
+                else:
+                    os.kill(pid,sig_list[i])
+            except ProcessLookupError:
+                break
+            time.sleep(1)
+            try:
+                self._check_pid(pid)
+            except ProcessLookupError:
+                break
+
+        try:
+            self._check_pid(pid)
+        except ProcessLookupError:
+            self.log.warning("Stopped proxy at pid=%s", pid)
+            self._remove_pid_file()
+            return
+        else:
+            raise RuntimeError("Failed to stop proxy at pid=%s", pid)
+
+    def _write_pid_file(self):
+        """write pid for proxy to a file"""
+        self.log.debug("Writing proxy pid file: %s", self.pid_file)
+        with open(self.pid_file, "w") as f:
+            f.write(str(self.proxy_process.pid))
+
+    def _remove_pid_file(self):
+        """Cleanup pid file for proxy after stopping"""
+        if not self.pid_file:
+            return
+        self.log.debug("Removing proxy pid file %s", self.pid_file)
+        try:
+            os.remove(self.pid_file)
+        except FileNotFoundError:
+            self.log.debug("PID file %s already removed", self.pid_file)
+            pass
 
     async def start(self):
+        """Start the proxy process"""
+        # check if there is a previous instance still around
+        self._check_previous_process()
+
+        # build the command to launch
         public_server = Server.from_url(self.public_url)
         api_server = Server.from_url(self.api_url)
         env = os.environ.copy()
@@ -496,6 +580,8 @@ class ConfigurableHTTPProxy(Proxy):
             )
             raise
 
+        self._write_pid_file()
+
         def _check_process():
             status = self.proxy_process.poll()
             if status is not None:
@@ -519,28 +605,34 @@ class ConfigurableHTTPProxy(Proxy):
         self._check_running_callback = pc
         pc.start()
 
-    def _kill_proc_tree(self, pid):
+    def _terminate_win(self, pid):
+        # On Windows we spawned a shell on Popen, so we need to
+        # terminate all child processes as well
         import psutil
+
         parent = psutil.Process(pid)
         children = parent.children(recursive=True)
         for child in children:
             child.kill()
         psutil.wait_procs(children, timeout=5)
 
+    def _terminate(self):
+        """Terminate our process"""
+        if os.name == 'nt':
+            self._terminate_win(self.proxy_process.pid)
+        else:
+            self.proxy_process.terminate()
+
     def stop(self):
         self.log.info("Cleaning up proxy[%i]...", self.proxy_process.pid)
         if self._check_running_callback is not None:
             self._check_running_callback.stop()
         if self.proxy_process.poll() is None:
             try:
-                if os.name == 'nt':
-                     # On Windows we spawned a shell on Popen, so we need to
-                    # terminate all child processes as well
-                    self._kill_proc_tree(self.proxy_process.pid)
-                else:
-                    self.proxy_process.terminate()
+                self._terminate()
             except Exception as e:
                 self.log.error("Failed to terminate proxy process: %s", e)
+        self._remove_pid_file()
 
     async def check_running(self):
         """Check if the proxy is still running"""
@@ -549,6 +641,7 @@ class ConfigurableHTTPProxy(Proxy):
         self.log.error("Proxy stopped with exit code %r",
                        'unknown' if self.proxy_process is None else self.proxy_process.poll()
                        )
+        self._remove_pid_file()
         await self.start()
         await self.restore_routes()
 

jupyterhub/services/service.py

@@ -39,8 +39,10 @@ A hub-managed service with no URL::
 
 """
 
+import copy
 import pipes
 import shutil
+import os
 from subprocess import Popen
 
 from traitlets import (
@@ -105,6 +107,8 @@ class _ServiceSpawner(LocalProcessSpawner):
     def start(self):
         """Start the process"""
         env = self.get_env()
+        if os.name == 'nt':
+            env['SYSTEMROOT'] = os.environ['SYSTEMROOT']
         cmd = self.cmd
 
         self.log.info("Spawning %s", ' '.join(pipes.quote(s) for s in cmd))
@@ -304,6 +308,15 @@ class Service(LoggingConfigurable):
             env['JUPYTERHUB_SERVICE_URL'] = self.url
             env['JUPYTERHUB_SERVICE_PREFIX'] = self.server.base_url
 
+        hub = self.hub
+        if self.hub.ip in ('0.0.0.0', ''):
+            # if the Hub is listening on all interfaces,
+            # tell services to connect via localhost
+            # since they are always local subprocesses
+            hub = copy.deepcopy(self.hub)
+            hub.connect_url = ''
+            hub.connect_ip = '127.0.0.1'
+
         self.spawner = _ServiceSpawner(
             cmd=self.command,
             environment=env,

jupyterhub/singleuser.py

@@ -214,6 +214,9 @@ class SingleUserNotebookApp(NotebookApp):
     version = __version__
     classes = NotebookApp.classes + [HubOAuth]
 
+    # disable single-user app's localhost checking
+    allow_remote_access = True
+
     # don't store cookie secrets
     cookie_secret_file = ''
     # always generate a new cookie secret on launch
@@ -295,6 +298,7 @@ class SingleUserNotebookApp(NotebookApp):
     # disble some single-user configurables
     token = ''
     open_browser = False
+    quit_button = False
     trust_xheaders = True
     login_handler_class = JupyterHubLoginHandler
     logout_handler_class = JupyterHubLogoutHandler

jupyterhub/spawner.py

@@ -161,6 +161,7 @@ class Spawner(LoggingConfigurable):
     admin_access = Bool(False)
     api_token = Unicode()
     oauth_client_id = Unicode()
+    handler = Any()
 
     will_resume = Bool(False,
         help="""Whether the Spawner will resume on next start
@@ -195,6 +196,19 @@ class Spawner(LoggingConfigurable):
         """
     ).tag(config=True)
 
+    consecutive_failure_limit = Integer(
+        0,
+        help="""
+        Maximum number of consecutive failures to allow before
+        shutting down JupyterHub.
+
+        This helps JupyterHub recover from a certain class of problem preventing launch
+        in contexts where the Hub is automatically restarted (e.g. systemd, docker, kubernetes).
+
+        A limit of 0 means no limit and consecutive failures will not be tracked.
+        """,
+    ).tag(config=True)
+
     start_timeout = Integer(60,
         help="""
         Timeout (in seconds) before giving up on starting of single-user server.
@@ -1002,7 +1016,7 @@ class LocalProcessSpawner(Spawner):
             which could change what the jupyterhub-singleuser launch command does.
             Only use this for trusted users.
         """
-    )
+    ).tag(config=True)
 
     proc = Instance(Popen,
         allow_none=True,

jupyterhub/tests/mocking.py

@@ -86,6 +86,8 @@ class MockSpawner(LocalProcessSpawner):
         pass
 
     def user_env(self, env):
+        if self.handler:
+            env['HANDLER_ARGS'] = self.handler.request.query
         return env
 
     @default('cmd')

jupyterhub/tests/test_api.py

@@ -100,6 +100,8 @@ def api_request(app, *api_path, **kwargs):
     assert "frame-ancestors 'self'" in resp.headers['Content-Security-Policy']
     assert ujoin(app.hub.base_url, "security/csp-report") in resp.headers['Content-Security-Policy']
     assert 'http' not in resp.headers['Content-Security-Policy']
+    if not kwargs.get('stream', False) and resp.content:
+        assert resp.headers.get('content-type') == 'application/json'
     return resp
 
 
@@ -604,6 +606,32 @@ def test_spawn(app):
     assert app.users.count_active_users()['pending'] == 0
 
 
+@mark.gen_test
+def test_spawn_handler(app):
+    """Test that the requesting Handler is passed to Spawner.handler"""
+    db = app.db
+    name = 'salmon'
+    user = add_user(db, app=app, name=name)
+    app_user = app.users[name]
+
+    # spawn via API with ?foo=bar
+    r = yield api_request(app, 'users', name, 'server', method='post', params={'foo': 'bar'})
+    r.raise_for_status()
+
+    # verify that request params got passed down
+    # implemented in MockSpawner
+    url = public_url(app, user)
+    r = yield async_requests.get(ujoin(url, 'env'))
+    env = r.json()
+    assert 'HANDLER_ARGS' in env
+    assert env['HANDLER_ARGS'] == 'foo=bar'
+    # make user spawner.handler doesn't persist after spawn finishes
+    assert app_user.spawner.handler is None
+
+    r = yield api_request(app, 'users', name, 'server', method='delete')
+    r.raise_for_status()
+
+
 @mark.slow
 @mark.gen_test
 def test_slow_spawn(app, no_patience, slow_spawn):
@@ -720,6 +748,8 @@ def test_progress(request, app, no_patience, slow_spawn):
     r = yield api_request(app, 'users', name, 'server/progress', stream=True)
     r.raise_for_status()
     request.addfinalizer(r.close)
+    assert r.headers['content-type'] == 'text/event-stream'
+
     ex = async_requests.executor
     line_iter = iter(r.iter_lines(decode_unicode=True))
     evt = yield ex.submit(next_event, line_iter)
@@ -781,6 +811,7 @@ def test_progress_ready(request, app):
     r = yield api_request(app, 'users', name, 'server/progress', stream=True)
     r.raise_for_status()
     request.addfinalizer(r.close)
+    assert r.headers['content-type'] == 'text/event-stream'
     ex = async_requests.executor
     line_iter = iter(r.iter_lines(decode_unicode=True))
     evt = yield ex.submit(next_event, line_iter)
@@ -800,6 +831,7 @@ def test_progress_bad(request, app, no_patience, bad_spawn):
     r = yield api_request(app, 'users', name, 'server/progress', stream=True)
     r.raise_for_status()
     request.addfinalizer(r.close)
+    assert r.headers['content-type'] == 'text/event-stream'
     ex = async_requests.executor
     line_iter = iter(r.iter_lines(decode_unicode=True))
     evt = yield ex.submit(next_event, line_iter)
@@ -821,6 +853,7 @@ def test_progress_bad_slow(request, app, no_patience, slow_bad_spawn):
     r = yield api_request(app, 'users', name, 'server/progress', stream=True)
     r.raise_for_status()
     request.addfinalizer(r.close)
+    assert r.headers['content-type'] == 'text/event-stream'
     ex = async_requests.executor
     line_iter = iter(r.iter_lines(decode_unicode=True))
     evt = yield ex.submit(next_event, line_iter)
@@ -1055,7 +1088,7 @@ def test_get_proxy(app):
     r = yield api_request(app, 'proxy')
     r.raise_for_status()
     reply = r.json()
-    assert list(reply.keys()) == ['/']
+    assert list(reply.keys()) == [app.hub.routespec]
 
 
 @mark.gen_test
@@ -1188,14 +1221,19 @@ def test_token_as_user_deprecated(app, as_user, for_user, status):
 
 
 @mark.gen_test
-@mark.parametrize("headers, status, note", [
-    ({}, 200, 'test note'),
-    ({}, 200, ''),
-    ({'Authorization': 'token bad'}, 403, ''),
+@mark.parametrize("headers, status, note, expires_in", [
+    ({}, 200, 'test note', None),
+    ({}, 200, '', 100),
+    ({'Authorization': 'token bad'}, 403, '', None),
 ])
-def test_get_new_token(app, headers, status, note):
+def test_get_new_token(app, headers, status, note, expires_in):
+    options = {}
     if note:
-        body = json.dumps({'note': note})
+        options['note'] = note
+    if expires_in:
+        options['expires_in'] = expires_in
+    if options:
+        body = json.dumps(options)
     else:
         body = ''
     # request a new token
@@ -1213,6 +1251,10 @@ def test_get_new_token(app, headers, status, note):
     assert reply['user'] == 'admin'
     assert reply['created']
     assert 'last_activity' in reply
+    if expires_in:
+        assert isinstance(reply['expires_at'], str)
+    else:
+        assert reply['expires_at'] is None
     if note:
         assert reply['note'] == note
     else:

jupyterhub/tests/test_pages.py

@@ -1,7 +1,9 @@
 """Tests for HTML pages"""
 
+import sys
 from urllib.parse import urlencode, urlparse
 
+from bs4 import BeautifulSoup
 from tornado import gen
 from tornado.httputil import url_concat
 
@@ -168,6 +170,31 @@ def test_spawn_redirect(app):
     assert path == ujoin(app.base_url, '/user/%s/' % name)
 
 
+@pytest.mark.gen_test
+def test_spawn_handler_access(app):
+    name = 'winston'
+    cookies = yield app.login_user(name)
+    u = app.users[orm.User.find(app.db, name)]
+
+    status = yield u.spawner.poll()
+    assert status is not None
+
+    # spawn server via browser link with ?arg=value
+    r = yield get_page('spawn', app, cookies=cookies, params={'arg': 'value'})
+    r.raise_for_status()
+
+    # verify that request params got passed down
+    # implemented in MockSpawner
+    r = yield async_requests.get(ujoin(public_url(app, u), 'env'))
+    env = r.json()
+    assert 'HANDLER_ARGS' in env
+    assert env['HANDLER_ARGS'] == 'arg=value'
+
+    # stop server
+    r = yield api_request(app, 'users', name, 'server', method='delete')
+    r.raise_for_status()
+
+
 @pytest.mark.gen_test
 def test_spawn_admin_access(app, admin_access):
     """GET /user/:name as admin with admin-access spawns user's server"""
@@ -216,11 +243,12 @@ def test_spawn_form(app):
         orm_u = orm.User.find(app.db, 'jones')
         u = app.users[orm_u]
         yield u.stop()
-
-        r = yield async_requests.post(ujoin(base_url, 'spawn?next=/user/jones/tree'), cookies=cookies, data={
-            'bounds': ['-1', '1'],
-            'energy': '511keV',
-        })
+        next_url = ujoin(app.base_url, 'user/jones/tree')
+        r = yield async_requests.post(
+            url_concat(ujoin(base_url, 'spawn'), {'next': next_url}),
+            cookies=cookies,
+            data={'bounds': ['-1', '1'], 'energy': '511keV'},
+        )
         r.raise_for_status()
         assert r.history
         assert u.spawner.user_options == {
@@ -236,13 +264,13 @@ def test_spawn_form_admin_access(app, admin_access):
         base_url = ujoin(public_host(app), app.hub.base_url)
         cookies = yield app.login_user('admin')
         u = add_user(app.db, app=app, name='martha')
+        next_url = ujoin(app.base_url, 'user', u.name, 'tree')
 
         r = yield async_requests.post(
-            ujoin(base_url, 'spawn/{0}?next=/user/{0}/tree'.format(u.name)),
-            cookies=cookies, data={
-            'bounds': ['-3', '3'],
-            'energy': '938MeV',
-        })
+            url_concat(ujoin(base_url, 'spawn', u.name), {'next': next_url}),
+            cookies=cookies,
+            data={'bounds': ['-3', '3'], 'energy': '938MeV'},
+        )
         r.raise_for_status()
         assert r.history
         assert r.url.startswith(public_url(app, u))
@@ -570,3 +598,57 @@ def test_announcements(app, announcements):
             app.authenticator.auto_login = auto_login
         r.raise_for_status()
         assert_announcement("logout", r.text)
+
+
+@pytest.mark.gen_test
+def test_token_page(app):
+    name = "cake"
+    cookies = yield app.login_user(name)
+    r = yield get_page("token", app, cookies=cookies)
+    r.raise_for_status()
+    assert urlparse(r.url).path.endswith('/hub/token')
+    def extract_body(r):
+        soup = BeautifulSoup(r.text, "html5lib")
+        import re
+        # trim empty lines
+        return re.sub(r"(\n\s*)+", "\n", soup.body.find(class_="container").text)
+    body = extract_body(r)
+    assert "Request new API token" in body, body
+    # no tokens yet, no lists
+    assert "API Tokens" not in body, body
+    assert "Authorized Applications" not in body, body
+
+    # request an API token
+    user = app.users[name]
+    token = user.new_api_token(expires_in=60, note="my-test-token")
+    app.db.commit()
+
+    r = yield get_page("token", app, cookies=cookies)
+    r.raise_for_status()
+    body = extract_body(r)
+    assert "API Tokens" in body, body
+    assert "my-test-token" in body, body
+    # no oauth tokens yet, shouldn't have that section
+    assert "Authorized Applications" not in body, body
+
+    # spawn the user to trigger oauth, etc.
+    # request an oauth token
+    user.spawner.cmd = [sys.executable, '-m', 'jupyterhub.singleuser']
+    r = yield get_page("spawn", app, cookies=cookies)
+    r.raise_for_status()
+
+    r = yield get_page("token", app, cookies=cookies)
+    r.raise_for_status()
+    body = extract_body(r)
+    assert "API Tokens" in body, body
+    assert "Server at %s" % user.base_url in body, body
+    assert "Authorized Applications" in body, body
+
+
+@pytest.mark.gen_test
+def test_server_not_running_api_request(app):
+    cookies = yield app.login_user("bees")
+    r = yield get_page("user/bees/api/status", app, hub=False, cookies=cookies)
+    assert r.status_code == 404
+    assert r.headers["content-type"] == "application/json"
+    assert r.json() == {"message": "bees is not running"}

jupyterhub/tests/test_proxy.py

@@ -79,7 +79,7 @@ def test_external_proxy(request):
 
     # test if api service has a root route '/'
     routes = yield app.proxy.get_all_routes()
-    assert list(routes.keys()) == ['/']
+    assert list(routes.keys()) == [app.hub.routespec]
     
     # add user to the db and start a single user server
     name = 'river'
@@ -95,7 +95,7 @@ def test_external_proxy(request):
     if app.subdomain_host:
         host = '%s.%s' % (name, urlparse(app.subdomain_host).hostname)
     user_spec = host + user_path
-    assert sorted(routes.keys()) == ['/', user_spec]
+    assert sorted(routes.keys()) == [app.hub.routespec, user_spec]
 
     # teardown the proxy and start a new one in the same place
     proxy.terminate()
@@ -113,7 +113,7 @@ def test_external_proxy(request):
 
     # check that the routes are correct
     routes = yield app.proxy.get_all_routes()
-    assert sorted(routes.keys()) == ['/', user_spec]
+    assert sorted(routes.keys()) == [app.hub.routespec, user_spec]
 
     # teardown the proxy, and start a new one with different auth and port
     proxy.terminate()
@@ -146,7 +146,7 @@ def test_external_proxy(request):
 
     # check that the routes are correct
     routes = yield app.proxy.get_all_routes()
-    assert sorted(routes.keys()) == ['/', user_spec]
+    assert sorted(routes.keys()) == [app.hub.routespec, user_spec]
 
 
 @pytest.mark.gen_test

jupyterhub/user.py

@@ -331,7 +331,7 @@ class User:
             url_parts.extend(['server/progress'])
         return url_path_join(*url_parts)
 
-    async def spawn(self, server_name='', options=None):
+    async def spawn(self, server_name='', options=None, handler=None):
         """Start the user's spawner
 
         depending from the value of JupyterHub.allow_named_servers
@@ -361,6 +361,9 @@ class User:
         spawner.server = server = Server(orm_server=orm_server)
         assert spawner.orm_spawner.server is orm_server
 
+        # pass requesting handler to the spawner
+        # e.g. for processing GET params
+        spawner.handler = handler
         # Passing user_options to the spawner
         spawner.user_options = options or {}
         # we are starting a new server, make sure it doesn't restore state
@@ -484,6 +487,9 @@ class User:
             # raise original exception
             spawner._start_pending = False
             raise e
+        finally:
+            # clear reference to handler after start finishes
+            spawner.handler = None
         spawner.start_polling()
 
         # store state
@@ -552,11 +558,25 @@ class User:
             # remove server entry from db
             spawner.server = None
             if not spawner.will_resume:
-                # find and remove the API token if the spawner isn't
+                # find and remove the API token and oauth client if the spawner isn't
                 # going to re-use it next time
                 orm_token = orm.APIToken.find(self.db, api_token)
                 if orm_token:
                     self.db.delete(orm_token)
+                # remove oauth client as well
+                # handle upgrades from 0.8, where client id will be `user-USERNAME`,
+                # not just `jupyterhub-user-USERNAME`
+                client_ids = (
+                    spawner.oauth_client_id,
+                    spawner.oauth_client_id.split('-', 1)[1],
+                )
+                for oauth_client in (
+                    self.db
+                        .query(orm.OAuthClient)
+                        .filter(orm.OAuthClient.identifier.in_(client_ids))
+                ):
+                    self.log.debug("Deleting oauth client %s", oauth_client.identifier)
+                    self.db.delete(oauth_client)
             self.db.commit()
         finally:
             spawner.orm_spawner.started = None

readthedocs.yml

@@ -4,3 +4,8 @@ conda:
     file: docs/environment.yml
 python:
   version: 3
+formats:
+  - htmlzip
+  - epub
+  # pdf disabled due to bug in sphinx 1.8 + recommonmark
+  # - pdf

share/jupyterhub/templates/admin.html

@@ -53,20 +53,20 @@
         {%- endif -%}
       </td>
       <td class="server-col col-sm-2 text-center">
-        <a role="button" class="stop-server btn btn-xs btn-danger {% if not u.running %}hidden{% endif %}">stop server</span>
-        <a role="button" class="start-server btn btn-xs btn-primary {% if u.running %}hidden{% endif %}">start server</span>
+        <a role="button" class="stop-server btn btn-xs btn-danger {% if not u.running %}hidden{% endif %}">stop server</a>
+        <a role="button" class="start-server btn btn-xs btn-primary {% if u.running %}hidden{% endif %}">start server</a>
       </td>
       <td class="server-col col-sm-1 text-center">
         {% if admin_access %}
-        <a role="button" class="access-server btn btn-xs btn-primary {% if not u.running %}hidden{% endif %}">access server</span>
+        <a role="button" class="access-server btn btn-xs btn-primary {% if not u.running %}hidden{% endif %}">access server</a>
         {% endif %}
       </td>
       <td class="edit-col col-sm-1 text-center">
-        <a role="button" class="edit-user btn btn-xs btn-primary">edit</span>
+        <a role="button" class="edit-user btn btn-xs btn-primary">edit</a>
       </td>
       <td class="edit-col col-sm-1 text-center">
         {% if u.name != user.name %}
-        <a role="button" class="delete-user btn btn-xs btn-danger">delete</span>
+        <a role="button" class="delete-user btn btn-xs btn-danger">delete</a>
         {% endif %}
       </td>
       {% endblock user_row %}

share/jupyterhub/templates/spawn_pending.html

@@ -16,6 +16,7 @@
       </div>
       <p id="progress-message"></p>
     </div>
+  </div>
   <div class="row">
     <div class="col-md-8 col-md-offset-2">
       <details id="progress-details">
@@ -24,6 +25,7 @@
       </details>
     </div>
   </div>
+</div>
 
 {% endblock %}
 

share/jupyterhub/templates/stop_pending.html

@@ -23,7 +23,7 @@
 require(["jquery"], function ($) {
   $("#refresh").click(function () {
     window.location.reload();
-  })
+  });
   setTimeout(function () {
     window.location.reload();
   }, 5000);

share/jupyterhub/templates/token.html

@@ -24,7 +24,6 @@
   </div>
 
   <div class="row">
-    <p>
       <div id="token-area" class="col-md-6 col-md-offset-3" style="display: none;">
         <div class="panel panel-default">
           <div class="panel-heading">
@@ -41,7 +40,6 @@
           </div>
         </div>
       </div>
-    </p>
   </div>
 
   {% if api_tokens %}
@@ -62,7 +60,7 @@
       </thead>
       <tbody>
         {% for token in api_tokens %}
-        <tr class="token-row" data-token-id="{{token.api_id}}"">
+        <tr class="token-row" data-token-id="{{token.api_id}}">
           {% block token_row scoped %}
           <td class="note-col col-sm-5">{{token.note}}</td>
           <td class="time-col col-sm-3">
@@ -111,7 +109,7 @@
       <tbody>
         {% for client in oauth_clients %}
         <tr class="token-row"
-          data-token-id="{{ client['token_id'] }}"">
+          data-token-id="{{ client['token_id'] }}">
           {% block client_row scoped %}
           <td class="note-col col-sm-5">{{ client['description'] }}</td>
           <td class="time-col col-sm-3">
@@ -129,8 +127,7 @@
             {%- endif -%}
           </td>
           <td class="col-sm-1 text-center">
-            <button class="revoke-token-btn btn btn-xs btn-danger">revoke</a>
-          </button>
+            <button class="revoke-token-btn btn btn-xs btn-danger">revoke</button>
           {% endblock client_row %}
         </tr>
         {% endfor %}
@@ -139,11 +136,11 @@
   </div>
   {% endif %}
 </div>
-{% endblock %}
+{% endblock main %}
 
 {% block script %}
 {{ super() }}
 <script type="text/javascript">
 require(["token"]);
 </script>
-{% endblock %}
+{% endblock script %}

singleuser/hooks/build

@@ -1,7 +1,7 @@
 #!/bin/bash
 set -ex
 
-stable=0.8
+stable=0.9
 
 for V in master $stable; do
     docker build --build-arg JUPYTERHUB_VERSION=$V -t $DOCKER_REPO:$V .

singleuser/hooks/post_push

@@ -1,6 +1,7 @@
 #!/bin/bash
+set -ex
 
-stable=0.8
+stable=0.9
 for V in master $stable; do
   docker push $DOCKER_REPO:$V
 done
@@ -12,6 +13,10 @@ function get_hub_version() {
   hub_xyz=$(cat hub_version)
   split=( ${hub_xyz//./ } )
   hub_xy="${split[0]}.${split[1]}"
+  # add .dev on hub_xy so it's 1.0.dev
+  if [[ ! -z "${split[3]}" ]]; then
+    hub_xy="${hub_xy}.${split[3]}"
+  fi
 }
 # tag e.g. 0.8.1 with 0.8
 get_hub_version $stable
@@ -22,3 +27,5 @@ docker push $DOCKER_REPO:$hub_xyz
 get_hub_version master
 docker tag $DOCKER_REPO:master $DOCKER_REPO:$hub_xy
 docker push $DOCKER_REPO:$hub_xy
+docker tag $DOCKER_REPO:master $DOCKER_REPO:$hub_xyz
+docker push $DOCKER_REPO:$hub_xyz