0.9.1

add python 3.7 to travis

.travis.yml

@@ -62,5 +62,7 @@ matrix:
     - python: 3.6
       env:
         - JUPYTERHUB_TEST_DB_URL=postgresql://postgres@127.0.0.1/jupyterhub
+    - python: 3.7
+      dist: xenial
   allow_failures:
     - python: nightly

docs/source/changelog.md

@@ -9,6 +9,19 @@ command line for details.
 
 ## 0.9
 
+### [0.9.1] 2018-07-04
+
+JupyterHub 0.9.1 contains a number of small bugfixes on top of 0.9.
+
+- Use a PID file for the proxy to decrease the likelihood that a leftover proxy process will prevent JupyterHub from restarting
+- `c.LocalProcessSpawner.shell_cmd` is now configurable
+- API requests to stopped servers (requests to the hub for `/user/:name/api/...`) fail with 404 rather than triggering a restart of the server
+- Compatibility fix for notebook 5.6.0 which will introduce further
+  security checks for local connections
+- Managed services always use localhost to talk to the Hub if the Hub listening on all interfaces
+- When using a URL prefix, the Hub route will be `JupyterHub.base_url` instead of unconditionally `/`
+- additional fixes and improvements
+
 ### [0.9.0] 2018-06-15
 
 JupyterHub 0.9 is a major upgrade of JupyterHub.
@@ -379,7 +392,8 @@ Fix removal of `/login` page in 0.4.0, breaking some OAuth providers.
 First preview release
 
 
-[Unreleased]: https://github.com/jupyterhub/jupyterhub/compare/0.9.0...HEAD
+[Unreleased]: https://github.com/jupyterhub/jupyterhub/compare/0.9.1...HEAD
+[0.9.1]: https://github.com/jupyterhub/jupyterhub/compare/0.9.0...0.9.1
 [0.9.0]: https://github.com/jupyterhub/jupyterhub/compare/0.8.1...0.9.0
 [0.8.1]: https://github.com/jupyterhub/jupyterhub/compare/0.8.0...0.8.1
 [0.8.0]: https://github.com/jupyterhub/jupyterhub/compare/0.7.2...0.8.0

docs/source/index.rst

@@ -58,6 +58,8 @@ Contents
 * :doc:`reference/services`
 * :doc:`reference/rest`
 * :doc:`reference/upgrading`
+* :doc:`reference/templates`
+* :doc:`reference/config-user-env`
 * :doc:`reference/config-examples`
 * :doc:`reference/config-ghoauth`
 * :doc:`reference/config-proxy`

docs/source/reference/proxy.md

@@ -2,7 +2,7 @@
 
 JupyterHub 0.8 introduced the ability to write a custom implementation of the
 proxy. This enables deployments with different needs than the default proxy,
-configurable-http-proxy (CHP). CHP is a single-process nodejs proxy that they
+configurable-http-proxy (CHP). CHP is a single-process nodejs proxy that the
 Hub manages by default as a subprocess (it can be run externally, as well, and
 typically is in production deployments).
 

docs/source/reference/services.md

@@ -15,7 +15,7 @@ This section provides the following information about Services:
 ## Definition of a Service
 
 When working with JupyterHub, a **Service** is defined as a process that interacts
-with the Hub's REST API. A Service may perform a specific or
+with the Hub's REST API. A Service may perform a specific
 action or task. For example, the following tasks can each be a unique Service:
 
 - shutting down individuals' single user notebook servers that have been idle

docs/source/reference/spawners.md

@@ -46,7 +46,16 @@ Most `Spawner.start` functions will look similar to this example:
 def start(self):
     self.ip = '127.0.0.1'
     self.port = random_port()
-    yield self._actually_start_server_somehow()
+    # get environment variables,
+    # several of which are required for configuring the single-user server
+    env = self.get_env()
+    cmd = []
+    # get jupyterhub command to run,
+    # typically ['jupyterhub-singleuser']
+    cmd.extend(self.cmd)
+    cmd.extend(self.get_args())
+
+    yield self._actually_start_server_somehow(cmd, env)
     return (self.ip, self.port)
 ```
 

jupyterhub/_version.py

@@ -6,9 +6,9 @@
 version_info = (
     0,
     9,
-    0,
-    "",  # release (b1, rc1)
-    # "dev",  # dev
+    1,
+    "",  # release (b1, rc1, or "" for final)
+    # "dev",  # dev or nothing
 )
 
 # pep 440 version: no dot before beta/rc, but before .dev

jupyterhub/app.py

@@ -1108,7 +1108,18 @@ class JupyterHub(Application):
         else:
             hub_args['ip'] = self.hub_ip
             hub_args['port'] = self.hub_port
-        self.hub = Hub(**hub_args)
+
+        # routespec for the Hub is the *app* base url
+        # not the hub URL, so it receives requests for non-running servers
+        # use `/` with host-based routing so the Hub
+        # gets requests for all hosts
+        host = ''
+        if self.subdomain_host:
+            routespec = '/'
+        else:
+            routespec = self.base_url
+
+        self.hub = Hub(routespec=routespec, **hub_args)
 
         if self.hub_connect_ip:
             self.hub.connect_ip = self.hub_connect_ip

jupyterhub/handlers/base.py

@@ -6,6 +6,7 @@
 import copy
 from datetime import datetime, timedelta
 from http.client import responses
+import json
 import math
 import random
 import re
@@ -884,6 +885,13 @@ class UserSpawnHandler(BaseHandler):
     which will in turn send her to /user/alice/notebooks/mynotebook.ipynb.
     """
 
+    def _fail_api_request(self, user):
+        """Fail an API request to a not-running server"""
+        self.set_status(404)
+        self.set_header("Content-Type", "application/json")
+        self.write(json.dumps({"message": "%s is not running" % user.name}))
+        self.finish()
+
     async def get(self, name, user_path):
         if not user_path:
             user_path = '/'
@@ -910,6 +918,11 @@ class UserSpawnHandler(BaseHandler):
             # otherwise redirect users to their own server
             should_spawn = (current_user and current_user.name == name)
 
+        if "api" in user_path.split("/") and not user.active:
+            # API request for not-running server (e.g. notebook UI left open)
+            # Avoid triggering a spawn.
+            self._fail_api_request(user)
+            return
 
         if should_spawn:
             # if spawning fails for any reason, point users to /hub/home to retry

jupyterhub/objects.py

@@ -63,6 +63,9 @@ class Server(HasTraits):
     @validate('connect_url')
     def _connect_url_add_prefix(self, proposal):
         """Ensure connect_url includes base_url"""
+        if not proposal.value:
+            # Don't add the prefix if the setting is being cleared
+            return proposal.value
         urlinfo = urlparse(proposal.value)
         if not urlinfo.path.startswith(self.base_url):
             urlinfo = urlinfo._replace(path=self.base_url)
@@ -185,6 +188,7 @@ class Hub(Server):
         )
         return self
     public_host = Unicode()
+    routespec = Unicode()
 
     @property
     def api_url(self):

jupyterhub/proxy.py

@@ -22,8 +22,10 @@ import asyncio
 from functools import wraps
 import json
 import os
+import signal
 from subprocess import Popen
-from urllib.parse import quote
+import time
+from urllib.parse import quote, urlparse
 
 from tornado import gen
 from tornado.httpclient import AsyncHTTPClient, HTTPRequest, HTTPError
@@ -301,13 +303,13 @@ class Proxy(LoggingConfigurable):
         user_routes = {path for path, r in routes.items() if 'user' in r['data']}
         futures = []
 
-        good_routes = {'/'}
+        good_routes = {self.app.hub.routespec}
 
-        hub = self.app.hub
-        if '/' not in routes:
+        hub = self.hub
+        if self.app.hub.routespec not in routes:
             futures.append(self.add_hub_route(hub))
         else:
-            route = routes['/']
+            route = routes[self.app.hub.routespec]
             if route['target'] != hub.host:
                 self.log.warning("Updating default route %s → %s", route['target'], hub.host)
                 futures.append(self.add_hub_route(hub))
@@ -365,8 +367,8 @@ class Proxy(LoggingConfigurable):
 
     def add_hub_route(self, hub):
         """Add the default route for the Hub"""
-        self.log.info("Adding default route for Hub: / => %s", hub.host)
-        return self.add_route('/', self.hub.host, {'hub': True})
+        self.log.info("Adding default route for Hub: %s => %s", hub.routespec, hub.host)
+        return self.add_route(hub.routespec, self.hub.host, {'hub': True})
 
     async def restore_routes(self):
         self.log.info("Setting up routes on new proxy")
@@ -437,6 +439,12 @@ class ConfigurableHTTPProxy(Proxy):
                       help="""The command to start the proxy"""
                       )
 
+    pid_file = Unicode(
+        "jupyterhub-proxy.pid",
+        config=True,
+        help="File in which to write the PID of the proxy process.",
+    )
+
     _check_running_callback = Any(help="PeriodicCallback to check if the proxy is running")
 
     def __init__(self, **kwargs):
@@ -448,9 +456,72 @@ class ConfigurableHTTPProxy(Proxy):
                 " if Proxy.should_start is False" % self.__class__.__name__
             )
 
+    def _check_previous_process(self):
+        """Check if there's a process leftover and shut it down if so"""
+        if not self.pid_file or not os.path.exists(self.pid_file):
+            return
+        pid_file = os.path.abspath(self.pid_file)
+        self.log.warning("Found proxy pid file: %s", pid_file)
+        try:
+            with open(pid_file, "r") as f:
+                pid = int(f.read().strip())
+        except ValueError:
+            self.log.warning("%s did not appear to contain a pid", pid_file)
+            self._remove_pid_file()
+            return
 
+        try:
+            os.kill(pid, 0)
+        except ProcessLookupError:
+            self.log.warning("Proxy no longer running at pid=%s", pid)
+            self._remove_pid_file()
+            return
+
+        # if we got here, CHP is still running
+        self.log.warning("Proxy still running at pid=%s", pid)
+        for i, sig in enumerate([signal.SIGTERM] * 2 + [signal.SIGKILL]):
+            try:
+                os.kill(pid, signal.SIGTERM)
+            except ProcessLookupError:
+                break
+            time.sleep(1)
+            try:
+                os.kill(pid, 0)
+            except ProcessLookupError:
+                break
+
+        try:
+            os.kill(pid, 0)
+        except ProcessLookupError:
+            self.log.warning("Stopped proxy at pid=%s", pid)
+            self._remove_pid_file()
+            return
+        else:
+            raise RuntimeError("Failed to stop proxy at pid=%s", pid)
+
+    def _write_pid_file(self):
+        """write pid for proxy to a file"""
+        self.log.debug("Writing proxy pid file: %s", self.pid_file)
+        with open(self.pid_file, "w") as f:
+            f.write(str(self.proxy_process.pid))
+
+    def _remove_pid_file(self):
+        """Cleanup pid file for proxy after stopping"""
+        if not self.pid_file:
+            return
+        self.log.debug("Removing proxy pid file %s", self.pid_file)
+        try:
+            os.remove(self.pid_file)
+        except FileNotFoundError:
+            self.log.debug("PID file %s already removed", self.pid_file)
+            pass
 
     async def start(self):
+        """Start the proxy process"""
+        # check if there is a previous instance still around
+        self._check_previous_process()
+
+        # build the command to launch
         public_server = Server.from_url(self.public_url)
         api_server = Server.from_url(self.api_url)
         env = os.environ.copy()
@@ -496,6 +567,8 @@ class ConfigurableHTTPProxy(Proxy):
             )
             raise
 
+        self._write_pid_file()
+
         def _check_process():
             status = self.proxy_process.poll()
             if status is not None:
@@ -519,13 +592,20 @@ class ConfigurableHTTPProxy(Proxy):
         self._check_running_callback = pc
         pc.start()
 
-    def _kill_proc_tree(self, pid):
-        import psutil
-        parent = psutil.Process(pid)
-        children = parent.children(recursive=True)
-        for child in children:
-            child.kill()
-        psutil.wait_procs(children, timeout=5)
+    def _terminate(self):
+        """Terminate our process"""
+        if os.name == 'nt':
+            # On Windows we spawned a shell on Popen, so we need to
+            # terminate all child processes as well
+            import psutil
+
+            parent = psutil.Process(self.proxy_process.pid)
+            children = parent.children(recursive=True)
+            for child in children:
+                child.kill()
+            psutil.wait_procs(children, timeout=5)
+        else:
+            self.proxy_process.terminate()
 
     def stop(self):
         self.log.info("Cleaning up proxy[%i]...", self.proxy_process.pid)
@@ -533,14 +613,10 @@ class ConfigurableHTTPProxy(Proxy):
             self._check_running_callback.stop()
         if self.proxy_process.poll() is None:
             try:
-                if os.name == 'nt':
-                     # On Windows we spawned a shell on Popen, so we need to
-                    # terminate all child processes as well
-                    self._kill_proc_tree(self.proxy_process.pid)
-                else:
-                    self.proxy_process.terminate()
+                self._terminate()
             except Exception as e:
                 self.log.error("Failed to terminate proxy process: %s", e)
+        self._remove_pid_file()
 
     async def check_running(self):
         """Check if the proxy is still running"""
@@ -549,6 +625,7 @@ class ConfigurableHTTPProxy(Proxy):
         self.log.error("Proxy stopped with exit code %r",
                        'unknown' if self.proxy_process is None else self.proxy_process.poll()
                        )
+        self._remove_pid_file()
         await self.start()
         await self.restore_routes()
 

jupyterhub/services/service.py

@@ -39,6 +39,7 @@ A hub-managed service with no URL::
 
 """
 
+import copy
 import pipes
 import shutil
 from subprocess import Popen
@@ -304,6 +305,15 @@ class Service(LoggingConfigurable):
             env['JUPYTERHUB_SERVICE_URL'] = self.url
             env['JUPYTERHUB_SERVICE_PREFIX'] = self.server.base_url
 
+        hub = self.hub
+        if self.hub.ip in ('0.0.0.0', ''):
+            # if the Hub is listening on all interfaces,
+            # tell services to connect via localhost
+            # since they are always local subprocesses
+            hub = copy.deepcopy(self.hub)
+            hub.connect_url = ''
+            hub.connect_ip = '127.0.0.1'
+
         self.spawner = _ServiceSpawner(
             cmd=self.command,
             environment=env,

jupyterhub/singleuser.py

@@ -214,6 +214,9 @@ class SingleUserNotebookApp(NotebookApp):
     version = __version__
     classes = NotebookApp.classes + [HubOAuth]
 
+    # disable single-user app's localhost checking
+    allow_remote_access = True
+
     # don't store cookie secrets
     cookie_secret_file = ''
     # always generate a new cookie secret on launch

jupyterhub/spawner.py

@@ -1002,7 +1002,7 @@ class LocalProcessSpawner(Spawner):
             which could change what the jupyterhub-singleuser launch command does.
             Only use this for trusted users.
         """
-    )
+    ).tag(config=True)
 
     proc = Instance(Popen,
         allow_none=True,

jupyterhub/tests/test_api.py

@@ -1055,7 +1055,7 @@ def test_get_proxy(app):
     r = yield api_request(app, 'proxy')
     r.raise_for_status()
     reply = r.json()
-    assert list(reply.keys()) == ['/']
+    assert list(reply.keys()) == [app.hub.routespec]
 
 
 @mark.gen_test

jupyterhub/tests/test_pages.py

@@ -216,11 +216,12 @@ def test_spawn_form(app):
         orm_u = orm.User.find(app.db, 'jones')
         u = app.users[orm_u]
         yield u.stop()
-
-        r = yield async_requests.post(ujoin(base_url, 'spawn?next=/user/jones/tree'), cookies=cookies, data={
-            'bounds': ['-1', '1'],
-            'energy': '511keV',
-        })
+        next_url = ujoin(app.base_url, 'user/jones/tree')
+        r = yield async_requests.post(
+            url_concat(ujoin(base_url, 'spawn'), {'next': next_url}),
+            cookies=cookies,
+            data={'bounds': ['-1', '1'], 'energy': '511keV'},
+        )
         r.raise_for_status()
         assert r.history
         assert u.spawner.user_options == {
@@ -236,13 +237,13 @@ def test_spawn_form_admin_access(app, admin_access):
         base_url = ujoin(public_host(app), app.hub.base_url)
         cookies = yield app.login_user('admin')
         u = add_user(app.db, app=app, name='martha')
+        next_url = ujoin(app.base_url, 'user', u.name, 'tree')
 
         r = yield async_requests.post(
-            ujoin(base_url, 'spawn/{0}?next=/user/{0}/tree'.format(u.name)),
-            cookies=cookies, data={
-            'bounds': ['-3', '3'],
-            'energy': '938MeV',
-        })
+            url_concat(ujoin(base_url, 'spawn', u.name), {'next': next_url}),
+            cookies=cookies,
+            data={'bounds': ['-3', '3'], 'energy': '938MeV'},
+        )
         r.raise_for_status()
         assert r.history
         assert r.url.startswith(public_url(app, u))
@@ -570,3 +571,12 @@ def test_announcements(app, announcements):
             app.authenticator.auto_login = auto_login
         r.raise_for_status()
         assert_announcement("logout", r.text)
+
+
+@pytest.mark.gen_test
+def test_server_not_running_api_request(app):
+    cookies = yield app.login_user("bees")
+    r = yield get_page("user/bees/api/status", app, hub=False, cookies=cookies)
+    assert r.status_code == 404
+    assert r.headers["content-type"] == "application/json"
+    assert r.json() == {"message": "bees is not running"}

jupyterhub/tests/test_proxy.py

@@ -79,7 +79,7 @@ def test_external_proxy(request):
 
     # test if api service has a root route '/'
     routes = yield app.proxy.get_all_routes()
-    assert list(routes.keys()) == ['/']
+    assert list(routes.keys()) == [app.hub.routespec]
     
     # add user to the db and start a single user server
     name = 'river'
@@ -95,7 +95,7 @@ def test_external_proxy(request):
     if app.subdomain_host:
         host = '%s.%s' % (name, urlparse(app.subdomain_host).hostname)
     user_spec = host + user_path
-    assert sorted(routes.keys()) == ['/', user_spec]
+    assert sorted(routes.keys()) == [app.hub.routespec, user_spec]
 
     # teardown the proxy and start a new one in the same place
     proxy.terminate()
@@ -113,7 +113,7 @@ def test_external_proxy(request):
 
     # check that the routes are correct
     routes = yield app.proxy.get_all_routes()
-    assert sorted(routes.keys()) == ['/', user_spec]
+    assert sorted(routes.keys()) == [app.hub.routespec, user_spec]
 
     # teardown the proxy, and start a new one with different auth and port
     proxy.terminate()
@@ -146,7 +146,7 @@ def test_external_proxy(request):
 
     # check that the routes are correct
     routes = yield app.proxy.get_all_routes()
-    assert sorted(routes.keys()) == ['/', user_spec]
+    assert sorted(routes.keys()) == [app.hub.routespec, user_spec]
 
 
 @pytest.mark.gen_test

share/jupyterhub/templates/admin.html

@@ -53,20 +53,20 @@
         {%- endif -%}
       </td>
       <td class="server-col col-sm-2 text-center">
-        <a role="button" class="stop-server btn btn-xs btn-danger {% if not u.running %}hidden{% endif %}">stop server</span>
-        <a role="button" class="start-server btn btn-xs btn-primary {% if u.running %}hidden{% endif %}">start server</span>
+        <a role="button" class="stop-server btn btn-xs btn-danger {% if not u.running %}hidden{% endif %}">stop server</a>
+        <a role="button" class="start-server btn btn-xs btn-primary {% if u.running %}hidden{% endif %}">start server</a>
       </td>
       <td class="server-col col-sm-1 text-center">
         {% if admin_access %}
-        <a role="button" class="access-server btn btn-xs btn-primary {% if not u.running %}hidden{% endif %}">access server</span>
+        <a role="button" class="access-server btn btn-xs btn-primary {% if not u.running %}hidden{% endif %}">access server</a>
         {% endif %}
       </td>
       <td class="edit-col col-sm-1 text-center">
-        <a role="button" class="edit-user btn btn-xs btn-primary">edit</span>
+        <a role="button" class="edit-user btn btn-xs btn-primary">edit</a>
       </td>
       <td class="edit-col col-sm-1 text-center">
         {% if u.name != user.name %}
-        <a role="button" class="delete-user btn btn-xs btn-danger">delete</span>
+        <a role="button" class="delete-user btn btn-xs btn-danger">delete</a>
         {% endif %}
       </td>
       {% endblock user_row %}

share/jupyterhub/templates/spawn_pending.html

@@ -15,6 +15,7 @@
         </div>
       </div>
       <p id="progress-message"></p>
+    </div>
   </div>
   <div class="row">
     <div class="col-md-8 col-md-offset-2">
@@ -23,6 +24,7 @@
         <div id="progress-log"></div>
       </details>
     </div>
+  </div>
 </div>
 
 {% endblock %}

share/jupyterhub/templates/stop_pending.html

@@ -23,7 +23,7 @@
 require(["jquery"], function ($) {
   $("#refresh").click(function () {
     window.location.reload();
-  })
+  });
   setTimeout(function () {
     window.location.reload();
   }, 5000);

share/jupyterhub/templates/token.html

@@ -24,7 +24,6 @@
   </div>
 
   <div class="row">
-    <p>
       <div id="token-area" class="col-md-6 col-md-offset-3" style="display: none;">
         <div class="panel panel-default">
           <div class="panel-heading">
@@ -41,7 +40,6 @@
           </div>
         </div>
       </div>
-    </p>
   </div>
 
   {% if api_tokens %}
@@ -62,7 +60,7 @@
       </thead>
       <tbody>
         {% for token in api_tokens %}
-        <tr class="token-row" data-token-id="{{token.api_id}}"">
+        <tr class="token-row" data-token-id="{{token.api_id}}">
           {% block token_row scoped %}
           <td class="note-col col-sm-5">{{token.note}}</td>
           <td class="time-col col-sm-3">
@@ -111,7 +109,7 @@
       <tbody>
         {% for client in oauth_clients %}
         <tr class="token-row"
-          data-token-id="{{ client['token_id'] }}"">
+          data-token-id="{{ client['token_id'] }}">
           {% block client_row scoped %}
           <td class="note-col col-sm-5">{{ client['description'] }}</td>
           <td class="time-col col-sm-3">
@@ -129,8 +127,7 @@
             {%- endif -%}
           </td>
           <td class="col-sm-1 text-center">
-            <button class="revoke-token-btn btn btn-xs btn-danger">revoke</a>
-          </button>
+            <button class="revoke-token-btn btn btn-xs btn-danger">revoke</button>
           {% endblock client_row %}
         </tr>
         {% endfor %}
@@ -139,11 +136,11 @@
   </div>
   {% endif %}
 </div>
-{% endblock %}
+{% endblock main %}
 
 {% block script %}
 {{ super() }}
 <script type="text/javascript">
 require(["token"]);
 </script>
-{% endblock %}
+{% endblock script %}