release 0.7.1

Added Toree troubleshooting to docs

.travis.yml

@@ -2,6 +2,7 @@
 language: python
 sudo: false
 python:
+    - 3.6-dev
     - 3.5
     - 3.4
     - 3.3

README.md

@@ -131,11 +131,11 @@ Some examples, meant as illustration and testing of this concept:
 ----
 
 ## Docker
-A ready to go [docker image for JupyterHub](https://hub.docker.com/r/jupyterhub/jupyterhub/) gives a straightforward deployment of JupyterHub.
+A starter [docker image for JupyterHub](https://hub.docker.com/r/jupyterhub/jupyterhub/) gives a baseline deployment of JupyterHub.
 
-*Note: This `jupyterhub/jupyterhub` docker image is only an image for running the Hub service itself.
-It does not require the other Jupyter components, such as Notebook installation, which are needed by the single-user servers.
-To run the single-user servers, which may be on the same system as the Hub or not, Jupyter Notebook version 4 or greater must be installed.*
+**Important:** This `jupyterhub/jupyterhub` image contains only the Hub itself, with no configuration. In general, one needs
+to make a derivative image, with at least a `jupyterhub_config.py` setting up an Authenticator and/or a Spawner. To run the
+single-user servers, which may be on the same system as the Hub or not, Jupyter Notebook version 4 or greater must be installed.
 
 #### Starting JupyterHub with docker
 The JupyterHub docker image can be started with the following command:

docs/package.json

@@ -8,7 +8,7 @@
   "author": "",
   "license": "BSD-3-Clause",
   "devDependencies": {
-    "bootprint": "^0.8.5",
+    "bootprint": "^0.10.0",
     "bootprint-openapi": "^0.17.0"
   }
 }

docs/source/changelog.md

@@ -9,6 +9,20 @@ command line for details.
 
 ## 0.7
 
+### [0.7.1] - 2016-01-02
+
+#### Added
+
+- `Spawner.will_resume` for signalling that a single-user server is paused instead of stopped.
+  This is needed for cases like `DockerSpawner.remove_containers = False`,
+  where the first API token is re-used for subsequent spawns.
+- Warning on startup about single-character usernames,
+   caused by common `set('string')` typo in config.
+
+#### Fixed
+
+- Removed spurious warning about empty `next_url`, which is AOK.
+
 ### [0.7.0] - 2016-12-2
 
 #### Added
@@ -118,8 +132,9 @@ Fix removal of `/login` page in 0.4.0, breaking some OAuth providers.
 First preview release
 
 
-[Unreleased]: https://github.com/jupyterhub/jupyterhub/compare/0.7.0...HEAD
-[Unreleased]: https://github.com/jupyterhub/jupyterhub/compare/0.6.1...0.7.0
+[Unreleased]: https://github.com/jupyterhub/jupyterhub/compare/0.7.1...HEAD
+[0.7.1]: https://github.com/jupyterhub/jupyterhub/compare/0.7.0...0.7.1
+[0.7.0]: https://github.com/jupyterhub/jupyterhub/compare/0.6.1...0.7.0
 [0.6.1]: https://github.com/jupyterhub/jupyterhub/compare/0.6.0...0.6.1
 [0.6.0]: https://github.com/jupyterhub/jupyterhub/compare/0.5.0...0.6.0
 [0.5]: https://github.com/jupyterhub/jupyterhub/compare/0.4.1...0.5.0

docs/source/rest.md

@@ -67,4 +67,4 @@ Note: The Swagger specification is being renamed the [OpenAPI Initiative][].
 
 [on swagger's petstore]: http://petstore.swagger.io/?url=https://raw.githubusercontent.com/jupyterhub/jupyterhub/master/docs/rest-api.yml#!/default
 [OpenAPI Initiative]: https://www.openapis.org/
-[JupyterHub REST API]: ./api/index.html
+[JupyterHub REST API]: ./_static/rest-api/index.html

docs/source/services.md

@@ -54,7 +54,7 @@ If a service is also to be managed by the Hub, it has a few extra options:
         externally.
       - If a command is specified for launching the Service, the Service will
         be started and managed by the Hub.
-- `env: dict` - environment variables to add to the current env
+- `environment: dict` - additional environment variables for the Service.
 - `user: str` - the name of a system user to manage the Service. If
   unspecified, run as the same user as the Hub.
 
@@ -99,7 +99,7 @@ c.JupyterHub.services = [
 A Hub-Managed Service may also be configured with additional optional
 parameters, which describe the environment needed to start the Service process:
 
-- `env: dict` - additional environment variables for the Service.
+- `environment: dict` - additional environment variables for the Service.
 - `user: str` - name of the user to run the server if different from the Hub.
    Requires Hub to be root.
 - `cwd: path` directory in which to run the Service, if different from the

docs/source/troubleshooting.md

@@ -249,3 +249,26 @@ jupyter kernelspec list
 ```bash
 jupyterhub --debug
 ```
+
+## Toree integration with HDFS rack awareness script
+
+The Apache Toree kernel will an issue, when running with JupyterHub, if the standard HDFS 
+rack awareness script is used. This will materialize in the logs as a repeated WARN:
+
+```bash
+16/11/29 16:24:20 WARN ScriptBasedMapping: Exception running /etc/hadoop/conf/topology_script.py some.ip.address
+ExitCodeException exitCode=1:   File "/etc/hadoop/conf/topology_script.py", line 63
+    print rack
+             ^
+SyntaxError: Missing parentheses in call to 'print'
+
+    at `org.apache.hadoop.util.Shell.runCommand(Shell.java:576)`
+```
+
+In order to resolve this issue, there are two potential options.
+
+1. Update HDFS core-site.xml, so the parameter "net.topology.script.file.name" points to a custom 
+script (e.g. /etc/hadoop/conf/custom_topology_script.py). Copy the original script and change the first line point
+to a python two installation (e.g. /usr/bin/python).
+2. In spark-env.sh add a Python 2 installation to your path (e.g. export PATH=/opt/anaconda2/bin:$PATH).
+

jupyterhub/app.py

@@ -146,18 +146,18 @@ class NewToken(Application):
 
 class UpgradeDB(Application):
     """Upgrade the JupyterHub database schema."""
-    
+
     name = 'jupyterhub-upgrade-db'
     version = jupyterhub.__version__
     description = """Upgrade the JupyterHub database to the current schema.
-    
+
     Usage:
 
         jupyterhub upgrade-db
     """
     aliases = common_aliases
     classes = []
-    
+
     def _backup_db_file(self, db_file):
         """Backup a database file"""
         if not os.path.exists(db_file):
@@ -171,7 +171,7 @@ class UpgradeDB(Application):
             backup_db_file = '{}.{}.{}'.format(db_file, timestamp, i)
         if os.path.exists(backup_db_file):
             self.exit("backup db file already exists: %s" % backup_db_file)
-        
+
         self.log.info("Backing up %s => %s", db_file, backup_db_file)
         shutil.copy(db_file, backup_db_file)
 
@@ -222,12 +222,12 @@ class JupyterHub(Application):
         Authenticator,
         PAMAuthenticator,
     ])
-    
+
     load_groups = Dict(List(Unicode()),
         help="""Dict of 'group': ['usernames'] to load at startup.
-        
+
         This strictly *adds* groups and users to groups.
-        
+
         Loading one set of groups, then starting JupyterHub again with a different
         set will not remove users or groups from previous launches.
         That must be done through the API.
@@ -414,7 +414,7 @@ class JupyterHub(Application):
 
     api_tokens = Dict(Unicode(),
         help="""PENDING DEPRECATION: consider using service_tokens
-        
+
         Dict of token:username to be loaded into the database.
 
         Allows ahead-of-time generation of API tokens for use by externally managed services,
@@ -437,14 +437,14 @@ class JupyterHub(Application):
         Allows ahead-of-time generation of API tokens for use by externally managed services.
         """
     ).tag(config=True)
-    
+
     services = List(Dict(),
         help="""List of service specification dictionaries.
-        
+
         A service
-        
+
         For instance::
-        
+
             services = [
                 {
                     'name': 'cull_idle',
@@ -454,7 +454,7 @@ class JupyterHub(Application):
                     'name': 'formgrader',
                     'url': 'http://127.0.0.1:1234',
                     'token': 'super-secret',
-                    'env': 
+                    'environment': 
                 }
             ]
         """
@@ -608,7 +608,7 @@ class JupyterHub(Application):
         Instance(logging.Handler),
         help="Extra log handlers to set on JupyterHub logger",
     ).tag(config=True)
-    
+
     statsd = Any(allow_none=False, help="The statsd client, if any. A mock will be used if we aren't using statsd")
     @default('statsd')
     def _statsd(self):
@@ -919,7 +919,7 @@ class JupyterHub(Application):
         # The whitelist set and the users in the db are now the same.
         # From this point on, any user changes should be done simultaneously
         # to the whitelist set and user db, unless the whitelist is empty (all users allowed).
-    
+
     @gen.coroutine
     def init_groups(self):
         """Load predefined groups into the database"""
@@ -941,7 +941,7 @@ class JupyterHub(Application):
                     db.add(user)
                 group.users.append(user)
         db.commit()
-    
+
     @gen.coroutine
     def _add_tokens(self, token_dict, kind):
         """Add tokens for users or services to the database"""
@@ -982,13 +982,13 @@ class JupyterHub(Application):
             else:
                 self.log.debug("Not duplicating token %s", orm_token)
         db.commit()
-    
+
     @gen.coroutine
     def init_api_tokens(self):
         """Load predefined API tokens (for services) into database"""
         yield self._add_tokens(self.service_tokens, kind='service')
         yield self._add_tokens(self.api_tokens, kind='user')
-    
+
     def init_services(self):
         self._service_map.clear()
         if self.domain:
@@ -1458,7 +1458,7 @@ class JupyterHub(Application):
         except Exception as e:
             self.log.critical("Failed to start proxy", exc_info=True)
             self.exit(1)
-        
+
         for service_name, service in self._service_map.items():
             if not service.managed:
                 continue

jupyterhub/auth.py

@@ -56,6 +56,17 @@ class Authenticator(LoggingConfigurable):
         """
     ).tag(config=True)
 
+    @observe('whitelist')
+    def _check_whitelist(self, change):
+        short_names = [name for name in change['new'] if len(name) <= 1]
+        if short_names:
+            sorted_names = sorted(short_names)
+            single = ''.join(sorted_names)
+            string_set_typo = "set('%s')" % single
+            self.log.warning("whitelist contains single-character names: %s; did you mean set([%r]) instead of %s?",
+                sorted_names[:8], single, string_set_typo,
+            )
+
     custom_html = Unicode(
         help="""
         HTML form to be overridden by authenticators if they want a custom authentication form.

jupyterhub/handlers/pages.py

@@ -28,7 +28,7 @@ class RootHandler(BaseHandler):
     """
     def get(self):
         next_url = self.get_argument('next', '')
-        if not next_url.startswith('/'):
+        if next_url and not next_url.startswith('/'):
             self.log.warning("Disallowing redirect outside JupyterHub: %r", next_url)
             next_url = ''
         if next_url and next_url.startswith(url_path_join(self.base_url, 'user/')):

jupyterhub/services/service.py

@@ -70,12 +70,12 @@ class _MockUser(HasTraits):
 
 class _ServiceSpawner(LocalProcessSpawner):
     """Subclass of LocalProcessSpawner
-    
+
     Removes notebook-specific-ness from LocalProcessSpawner.
     """
     cwd = Unicode()
     cmd = Command(minlen=0)
-    
+
     def make_preexec_fn(self, name):
         if not name or name == getuser():
             # no setuid if no name
@@ -116,25 +116,25 @@ class Service(LoggingConfigurable):
     - url: str (None)
         The URL where the service is/should be.
         If specified, the service will be added to the proxy at /services/:name
-    
+
     If a service is to be managed by the Hub, it has a few extra options:
-    
+
     - command: (str/Popen list)
         Command for JupyterHub to spawn the service.
         Only use this if the service should be a subprocess.
         If command is not specified, it is assumed to be managed
         by a
-    - env: dict
-        environment variables to add to the current env
+    - environment: dict
+        Additional environment variables for the service.
     - user: str
         The name of a system user to become.
         If unspecified, run as the same user as the Hub.
     """
-    
+
     # inputs:
     name = Unicode(
         help="""The name of the service.
-        
+
         If the service has an http endpoint, it
         """
     ).tag(input=True)
@@ -143,14 +143,14 @@ class Service(LoggingConfigurable):
     ).tag(input=True)
     url = Unicode(
         help="""URL of the service.
-        
+
         Only specify if the service runs an HTTP(s) endpoint that.
         If managed, will be passed as JUPYTERHUB_SERVICE_URL env.
         """
     ).tag(input=True)
     api_token = Unicode(
         help="""The API token to use for the service.
-        
+
         If unspecified, an API token will be generated for managed services.
         """
     ).tag(input=True)

jupyterhub/spawner.py

@@ -52,6 +52,17 @@ class Spawner(LoggingConfigurable):
     authenticator = Any()
     api_token = Unicode()
 
+    will_resume = Bool(False,
+        help="""Whether the Spawner will resume on next start
+
+
+        Default is False where each launch of the Spawner will be a new instance.
+        If True, an existing Spawner will resume instead of starting anew
+        (e.g. resuming a Docker container),
+        and API tokens in use when the Spawner stops will not be deleted.
+        """
+    )
+
     ip = Unicode('127.0.0.1',
         help="""
         The IP address (or hostname) the single-user server should listen on.

jupyterhub/user.py

@@ -234,6 +234,12 @@ class User(HasTraits):
                 # prior to 0.7, spawners had to store this info in user.server themselves.
                 # Handle < 0.7 behavior with a warning, assuming info was stored in db by the Spawner.
                 self.log.warning("DEPRECATION: Spawner.start should return (ip, port) in JupyterHub >= 0.7")
+            if spawner.api_token != api_token:
+                # Spawner re-used an API token, discard the unused api_token
+                orm_token = orm.APIToken.find(self.db, api_token)
+                if orm_token is not None:
+                    self.db.delete(orm_token)
+                    self.db.commit()
         except Exception as e:
             if isinstance(e, gen.TimeoutError):
                 self.log.warning("{user}'s server failed to start in {s} seconds, giving up".format(
@@ -313,10 +319,13 @@ class User(HasTraits):
             if self.server:
                 # cleanup server entry from db
                 self.db.delete(self.server)
-            orm_token = orm.APIToken.find(self.db, api_token)
-            if orm_token:
-                self.db.delete(orm_token)
             self.server = None
+            if not spawner.will_resume:
+                # find and remove the API token if the spawner isn't
+                # going to re-use it next time
+                orm_token = orm.APIToken.find(self.db, api_token)
+                if orm_token:
+                    self.db.delete(orm_token)
             self.db.commit()
         finally:
             self.stop_pending = False

jupyterhub/version.py

@@ -6,7 +6,8 @@
 version_info = (
     0,
     7,
-    0,
+    1,
+    # 'dev',
 )
 
 __version__ = '.'.join(map(str, version_info))