hazza/jupyterhub
1
0
Fork 0
mirror of https://github.com/jupyterhub/jupyterhub.git synced 2025-10-14 21:43:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #2055 from LLNL/end-to-end-ssl

Browse Source 
End-to-end SSL
This commit is contained in:
Min RK
2018-10-17 13:27:03 +02:00
committed by GitHub
parent 2d6e7186aa 7a055e65db
commit 2d94b2999f
25 changed files with 915 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docs/source/reference/proxy.md
View File

@@ -59,6 +59,18 @@ These methods **may** be coroutines.
`c.Proxy.should_start` is a configurable flag that determines whether the
Hub should call these methods when the Hub itself starts and stops.
## Encryption
When using `internal_ssl` to encrypt traffic behind the proxy, at minimum,
your `Proxy` will need client ssl certificates which the `Hub` must be made
aware of. These can be generated with the command `jupyterhub --generate-certs`
which will write them to the `internal_certs_location` in folders named
`proxy_api` and `proxy_client`. Alternatively, these can be provided to the
hub via the `jupyterhub_config.py` file by providing a `dict` of named paths
to the `external_authorities` option. The hub will include all certificates
provided in that `dict` in the trust bundle utilized by all internal
components.
### Purely external proxies
Probably most custom proxies will be externally managed,

27
docs/source/reference/spawners.md
View File

@@ -260,3 +260,30 @@ in the single-user notebook server when a guarantee is being provided.
**The spawner's underlying system or cluster is responsible for enforcing these
limits and providing these guarantees.** If these values are set to `None`, no
limits or guarantees are provided, and no environment values are set.
### Encryption
Communication between the `Proxy`, `Hub`, and `Notebook` can be secured by
turning on `internal_ssl` in `jupyterhub_config.py`. For a custom spawner to
utilize these certs, there are two methods of interest on the base `Spawner`
class: `.create_certs` and `.move_certs`.
The first method, `.create_certs` will sign a key-cert pair using an internally
trusted authority for notebooks. During this process, `.create_certs` can
apply `ip` and `dns` name information to the cert via an `alt_names` `kwarg`.
This is used for certificate authentication (verification). Without proper
verification, the `Notebook` will be unable to communicate with the `Hub` and
vice versa when `internal_ssl` is enabled. For example, given a deployment
using the `DockerSpawner` which will start containers with `ips` from the
`docker` subnet pool, the `DockerSpawner` would need to instead choose a
container `ip` prior to starting and pass that to `.create_certs` (TODO: edit).
In general though, this method will not need to be changed and the default
`ip`/`dns` (localhost) info will suffice.
When `.create_certs` is run, it will `.create_certs` in a default, central
location specified by `c.JupyterHub.internal_certs_location`. For `Spawners`
that need access to these certs elsewhere (i.e. on another host altogether),
the `.move_certs` method can be overridden to move the certs appropriately.
Again, using `DockerSpawner` as an example, this would entail moving certs
to a directory that will get mounted into the container this spawner starts.

17
docs/source/reference/websecurity.md
View File

@@ -99,6 +99,23 @@ single-user server, and not the environment(s) in which the user's kernel(s)
may run. Installing additional packages in the kernel environment does not
pose additional risk to the web application's security.
### Encrypt internal connections with SSL/TLS
By default, all communication on the server, between the proxy, hub, and single
-user notebooks is performed unencrypted. Setting the `internal_ssl` flag in
`jupyterhub_config.py` secures the aforementioned routes. Turning this
feature on does require that the enabled `Spawner` can use the certificates
generated by the `Hub` (the default `LocalProcessSpawner` can, for instance).
It is also important to note that this encryption **does not** (yet) cover the
`zmq tcp` sockets between the Notebook client and kernel. While users cannot
submit arbitrary commands to another user's kernel, they can bind to these
sockets and listen. When serving untrusted users, this eavesdropping can be
mitigated by setting `KernelManager.transport` to `ipc`. This applies standard
Unix permissions to the communication sockets thereby restricting
communication to the socket owner. The `internal_ssl` option will eventually
extend to securing the `tcp` sockets as well.
## Security audits
We recommend that you do periodic reviews of your deployment's security. It's

248
jupyterhub/app.py
View File

@@ -16,11 +16,11 @@ from operator import itemgetter
import os
import re
import signal
import socket
import sys
from textwrap import dedent
from urllib.parse import unquote, urlparse, urlunparse
if sys.version_info[:2] < (3, 3):
raise ValueError("Python < 3.3 not supported: %s" % sys.version)
@@ -63,6 +63,7 @@ from .utils import (
maybe_future,
url_path_join,
print_stacks, print_ps_info,
make_ssl_context,
)
# classes for config
from .auth import Authenticator, PAMAuthenticator
@@ -102,6 +103,8 @@ flags = {
"set log level to logging.DEBUG (maximize logging output)"),
'generate-config': ({'JupyterHub': {'generate_config': True}},
"generate default config file"),
'generate-certs': ({'JupyterHub': {'generate_certs': True}},
"generate certificates used for internal ssl"),
'no-db': ({'JupyterHub': {'db_url': 'sqlite:///:memory:'}},
"disable persisting state database to disk"
),
@@ -258,6 +261,9 @@ class JupyterHub(Application):
generate_config = Bool(False,
help="Generate default config file",
).tag(config=True)
generate_certs = Bool(False,
help="Generate certs used for internal ssl",
).tag(config=True)
answer_yes = Bool(False,
help="Answer yes to any questions (e.g. confirm overwrite)"
).tag(config=True)
@@ -318,6 +324,101 @@ class JupyterHub(Application):
When setting this, you should also set ssl_key
"""
).tag(config=True)
internal_ssl = Bool(False,
help="""Enable SSL for all internal communication
This enables end-to-end encryption between all JupyterHub components.
JupyterHub will automatically create the necessary certificate
authority and sign notebook certificates as they're created.
"""
).tag(config=True)
internal_certs_location = Unicode('internal-ssl',
help="""The location to store certificates automatically created by
JupyterHub.
Use with internal_ssl
"""
).tag(config=True)
recreate_internal_certs = Bool(False,
help="""Recreate all certificates used within JupyterHub on restart.
Note: enabling this feature requires restarting all notebook servers.
Use with internal_ssl
"""
).tag(config=True)
external_ssl_authorities = Dict(
help="""Dict authority:dict(files). Specify the key, cert, and/or
ca file for an authority. This is useful for externally managed
proxies that wish to use internal_ssl.
The files dict has this format (you must specify at least a cert)::
{
'key': '/path/to/key.key',
'cert': '/path/to/cert.crt',
'ca': '/path/to/ca.crt'
}
The authorities you can override: 'hub-ca', 'notebooks-ca',
'proxy-api-ca', 'proxy-client-ca', and 'services-ca'.
Use with internal_ssl
"""
).tag(config=True)
internal_ssl_authorities = Dict(
default_value={
'hub-ca': None,
'notebooks-ca': None,
'proxy-api-ca': None,
'proxy-client-ca': None,
'services-ca': None,
},
help="""Dict authority:dict(files). When creating the various
CAs needed for internal_ssl, these are the names that will be used
for each authority.
Use with internal_ssl
"""
)
internal_ssl_components_trust = Dict(
help="""Dict component:list(components). This dict specifies the
relationships of components secured by internal_ssl.
"""
)
internal_trust_bundles = Dict(
help="""Dict component:path. These are the paths to the trust bundles
that each component should have. They will be set during
`init_internal_ssl`.
Use with internal_ssl
"""
)
internal_ssl_key = Unicode(
help="""The key to be used for internal ssl"""
)
internal_ssl_cert = Unicode(
help="""The cert to be used for internal ssl"""
)
internal_ssl_ca = Unicode(
help="""The certificate authority to be used for internal ssl"""
)
internal_proxy_certs = Dict(
help=""" Dict component:dict(cert files). This dict contains the certs
generated for both the proxy API and proxy client.
"""
)
trusted_alt_names = List(Unicode(),
help="""Names to include in the subject alternative name.
These names will be used for server name verification. This is useful
if JupyterHub is being run behind a reverse proxy or services using ssl
are on different hosts.
Use with internal_ssl
"""
).tag(config=True)
ip = Unicode('',
help="""The public facing ip of the whole JupyterHub application
(specifically referred to as the proxy).
@@ -1093,6 +1194,105 @@ class JupyterHub(Application):
# store the loaded trait value
self.cookie_secret = secret
def init_internal_ssl(self):
"""Create the certs needed to turn on internal SSL."""
if self.internal_ssl:
from certipy import Certipy, CertNotFoundError
certipy = Certipy(store_dir=self.internal_certs_location,
remove_existing=self.recreate_internal_certs)
# Here we define how trust should be laid out per each component
self.internal_ssl_components_trust = {
'hub-ca': list(self.internal_ssl_authorities.keys()),
'proxy-api-ca': ['hub-ca', 'services-ca', 'notebooks-ca'],
'proxy-client-ca': ['hub-ca', 'notebooks-ca'],
'notebooks-ca': ['hub-ca', 'proxy-client-ca'],
'services-ca': ['hub-ca', 'proxy-api-ca'],
}
hub_name = 'hub-ca'
# If any external CAs were specified in external_ssl_authorities
# add records of them to Certipy's store.
self.internal_ssl_authorities.update(self.external_ssl_authorities)
for authority, files in self.internal_ssl_authorities.items():
if files:
self.log.info("Adding CA for %s", authority)
certipy.store.add_record(
authority, is_ca=True, files=files)
self.internal_trust_bundles = certipy.trust_from_graph(
self.internal_ssl_components_trust)
default_alt_names = ["IP:127.0.0.1", "DNS:localhost"]
if self.subdomain_host:
default_alt_names.append("DNS:%s" % urlparse(self.subdomain_host).hostname)
# The signed certs used by hub-internal components
try:
internal_key_pair = certipy.store.get_record("hub-internal")
except CertNotFoundError:
alt_names = list(default_alt_names)
# In the event the hub needs to be accessed externally, add
# the fqdn and (optionally) rev_proxy to the set of alt_names.
alt_names += (["DNS:" + socket.getfqdn()]
+ self.trusted_alt_names)
self.log.info(
"Adding CA for %s: %s",
"hub-internal",
";".join(alt_names),
)
internal_key_pair = certipy.create_signed_pair(
"hub-internal",
hub_name,
alt_names=alt_names,
)
else:
self.log.info("Using existing hub-internal CA")
# Create the proxy certs
proxy_api = 'proxy-api'
proxy_client = 'proxy-client'
for component in [proxy_api, proxy_client]:
ca_name = component + '-ca'
alt_names = default_alt_names + self.trusted_alt_names
try:
record = certipy.store.get_record(component)
except CertNotFoundError:
self.log.info(
"Generating signed pair for %s: %s",
component,
';'.join(alt_names),
)
record = certipy.create_signed_pair(
component,
ca_name,
alt_names=alt_names,
)
else:
self.log.info("Using existing %s CA", component)
self.internal_proxy_certs[component] = {
"keyfile": record['files']['key'],
"certfile": record['files']['cert'],
"cafile": record['files']['cert'],
}
self.internal_ssl_key = internal_key_pair['files']['key']
self.internal_ssl_cert = internal_key_pair['files']['cert']
self.internal_ssl_ca = self.internal_trust_bundles[hub_name]
# Configure the AsyncHTTPClient. This will affect anything using
# AsyncHTTPClient.
ssl_context = make_ssl_context(
self.internal_ssl_key,
self.internal_ssl_cert,
cafile=self.internal_ssl_ca,
)
AsyncHTTPClient.configure(
None, defaults={"ssl_options" : ssl_context}
)
def init_db(self):
"""Create the database connection"""
@@ -1127,6 +1327,9 @@ class JupyterHub(Application):
hub_args = dict(
base_url=self.hub_prefix,
public_host=self.subdomain_host,
certfile=self.internal_ssl_cert,
keyfile=self.internal_ssl_key,
cafile=self.internal_ssl_ca,
)
if self.hub_bind_url:
# ensure hub_prefix is set on bind_url
@@ -1168,6 +1371,8 @@ class JupyterHub(Application):
._replace(path=self.hub_prefix)
)
self.hub.connect_url = self.hub_connect_url
if self.internal_ssl:
self.hub.proto = 'https'
async def init_users(self):
"""Load users into and from the database"""
@@ -1375,6 +1580,7 @@ class JupyterHub(Application):
orm_service.admin = spec.get('admin', False)
self.db.commit()
service = Service(parent=self,
app=self,
base_url=self.base_url,
db=self.db, orm=orm_service,
domain=domain, host=host,
@@ -1631,6 +1837,14 @@ class JupyterHub(Application):
concurrent_spawn_limit=self.concurrent_spawn_limit,
spawn_throttle_retry_range=self.spawn_throttle_retry_range,
active_server_limit=self.active_server_limit,
internal_ssl=self.internal_ssl,
internal_certs_location=self.internal_certs_location,
internal_authorities=self.internal_ssl_authorities,
internal_trust_bundles=self.internal_trust_bundles,
internal_ssl_key=self.internal_ssl_key,
internal_ssl_cert=self.internal_ssl_cert,
internal_ssl_ca=self.internal_ssl_ca,
trusted_alt_names=self.trusted_alt_names,
)
# allow configured settings to have priority
settings.update(self.tornado_settings)
@@ -1661,7 +1875,7 @@ class JupyterHub(Application):
@catch_config_error
async def initialize(self, *args, **kwargs):
super().initialize(*args, **kwargs)
if self.generate_config or self.subapp:
if self.generate_config or self.generate_certs or self.subapp:
return
self.load_config_file(self.config_file)
self.init_logging()
@@ -1704,6 +1918,7 @@ class JupyterHub(Application):
self.init_pycurl()
self.init_secrets()
self.init_internal_ssl()
self.init_db()
self.init_hub()
self.init_proxy()
@@ -1862,8 +2077,28 @@ class JupyterHub(Application):
loop.stop()
return
if self.generate_certs:
self.load_config_file(self.config_file)
if not self.internal_ssl:
self.log.warn("You'll need to enable `internal_ssl` "
"in the `jupyterhub_config` file to use "
"these certs.")
self.internal_ssl = True
self.init_internal_ssl()
self.log.info("Certificates written to directory `{}`".format(
self.internal_certs_location))
loop.stop()
return
ssl_context = make_ssl_context(
self.internal_ssl_key,
self.internal_ssl_cert,
cafile=self.internal_ssl_ca,
check_hostname=False
)
# start the webserver
self.http_server = tornado.httpserver.HTTPServer(self.tornado_application, xheaders=True)
self.http_server = tornado.httpserver.HTTPServer(self.tornado_application, ssl_options=ssl_context, xheaders=True)
bind_url = urlparse(self.hub.bind_url)
try:
if bind_url.scheme.startswith('unix+'):
@@ -1913,7 +2148,12 @@ class JupyterHub(Application):
tries = 10 if service.managed else 1
for i in range(tries):
try:
await Server.from_orm(service.orm.server).wait_up(http=True, timeout=1)
ssl_context = make_ssl_context(
self.internal_ssl_key,
self.internal_ssl_cert,
cafile=self.internal_ssl_ca
)
await Server.from_orm(service.orm.server).wait_up(http=True, timeout=1, ssl_context=ssl_context)
except TimeoutError:
if service.managed:
status = await service.spawner.poll()

14
jupyterhub/objects.py
View File

@@ -15,7 +15,7 @@ from .traitlets import URLPrefix
from . import orm
from .utils import (
url_path_join, can_connect, wait_for_server,
wait_for_http_server, random_port,
wait_for_http_server, random_port, make_ssl_context,
)
class Server(HasTraits):
@@ -35,6 +35,9 @@ class Server(HasTraits):
cookie_name = Unicode('')
connect_url = Unicode('')
bind_url = Unicode('')
certfile = Unicode()
keyfile = Unicode()
cafile = Unicode()
@default('bind_url')
def bind_url_default(self):
@@ -157,10 +160,15 @@ class Server(HasTraits):
uri=self.base_url,
)
def wait_up(self, timeout=10, http=False):
def wait_up(self, timeout=10, http=False, ssl_context=None):
"""Wait for this server to come up"""
if http:
return wait_for_http_server(self.url, timeout=timeout)
ssl_context = ssl_context or make_ssl_context(
self.keyfile, self.certfile, cafile=self.cafile)
return wait_for_http_server(
self.url, timeout=timeout, ssl_context=ssl_context)
else:
return wait_for_server(self._connect_ip, self._connect_port, timeout=timeout)

42
jupyterhub/proxy.py
View File

@@ -39,10 +39,11 @@ from traitlets import (
from jupyterhub.traitlets import Command
from traitlets.config import LoggingConfigurable
from .metrics import CHECK_ROUTES_DURATION_SECONDS
from .objects import Server
from . import utils
from .utils import url_path_join
from .metrics import CHECK_ROUTES_DURATION_SECONDS
from .utils import url_path_join, make_ssl_context
def _one_at_a_time(method):
@@ -436,9 +437,22 @@ class ConfigurableHTTPProxy(Proxy):
token = utils.new_token()
return token
api_url = Unicode('http://127.0.0.1:8001', config=True,
api_url = Unicode(config=True,
help="""The ip (or hostname) of the proxy's API endpoint"""
)
@default('api_url')
def _api_url_default(self):
url = '127.0.0.1:8001'
proto = 'http'
if self.app.internal_ssl:
proto = 'https'
return "{proto}://{url}".format(
proto=proto,
url=url,
)
command = Command('configurable-http-proxy', config=True,
help="""The command to start the proxy"""
)
@@ -558,6 +572,28 @@ class ConfigurableHTTPProxy(Proxy):
cmd.extend(['--ssl-key', self.ssl_key])
if self.ssl_cert:
cmd.extend(['--ssl-cert', self.ssl_cert])
if self.app.internal_ssl:
proxy_api = 'proxy-api'
proxy_client = 'proxy-client'
api_key = self.app.internal_proxy_certs[proxy_api]['keyfile']
api_cert = self.app.internal_proxy_certs[proxy_api]['certfile']
api_ca = self.app.internal_trust_bundles[proxy_api + '-ca']
client_key = self.app.internal_proxy_certs[proxy_client]['keyfile']
client_cert = self.app.internal_proxy_certs[proxy_client]['certfile']
client_ca = self.app.internal_trust_bundles[proxy_client + '-ca']
cmd.extend(['--api-ssl-key', api_key])
cmd.extend(['--api-ssl-cert', api_cert])
cmd.extend(['--api-ssl-ca', api_ca])
cmd.extend(['--api-ssl-request-cert'])
cmd.extend(['--api-ssl-reject-unauthorized'])
cmd.extend(['--client-ssl-key', client_key])
cmd.extend(['--client-ssl-cert', client_cert])
cmd.extend(['--client-ssl-ca', client_ca])
cmd.extend(['--client-ssl-request-cert'])
cmd.extend(['--client-ssl-reject-unauthorized'])
if self.app.statsd_host:
cmd.extend([
'--statsd-host', self.app.statsd_host,

25
jupyterhub/services/auth.py
View File

@@ -196,6 +196,27 @@ class HubAuth(SingletonConfigurable):
def _default_login_url(self):
return self.hub_host + url_path_join(self.hub_prefix, 'login')
keyfile = Unicode('',
help="""The ssl key to use for requests
Use with certfile
"""
).tag(config=True)
certfile = Unicode('',
help="""The ssl cert to use for requests
Use with keyfile
"""
).tag(config=True)
client_ca = Unicode('',
help="""The ssl certificate authority to use to verify requests
Use with keyfile and certfile
"""
).tag(config=True)
cookie_name = Unicode('jupyterhub-services',
help="""The name of the cookie I should be looking for"""
).tag(config=True)
@@ -277,6 +298,10 @@ class HubAuth(SingletonConfigurable):
allow_404 = kwargs.pop('allow_404', False)
headers = kwargs.setdefault('headers', {})
headers.setdefault('Authorization', 'token %s' % self.api_token)
if "cert" not in kwargs and self.certfile and self.keyfile:
kwargs["cert"] = (self.certfile, self.keyfile)
if self.client_ca:
kwargs["verify"] = self.client_ca
try:
r = requests.request(method, url, **kwargs)
except requests.ConnectionError as e:

4
jupyterhub/services/service.py
View File

@@ -224,6 +224,7 @@ class Service(LoggingConfigurable):
domain = Unicode()
host = Unicode()
hub = Any()
app = Any()
proc = Any()
# handles on globals:
@@ -331,6 +332,9 @@ class Service(LoggingConfigurable):
server=self.orm.server,
host=self.host,
),
internal_ssl=self.app.internal_ssl,
internal_certs_location=self.app.internal_certs_location,
internal_trust_bundles=self.app.internal_trust_bundles,
)
self.spawner.start()
self.proc = self.spawner.proc

24
jupyterhub/singleuser.py
View File

@@ -43,7 +43,7 @@ from notebook.base.handlers import IPythonHandler
from ._version import __version__, _check_version
from .log import log_request
from .services.auth import HubOAuth, HubOAuthenticated, HubOAuthCallbackHandler
from .utils import url_path_join
from .utils import url_path_join, make_ssl_context
# Authenticate requests with the Hub
@@ -245,6 +245,18 @@ class SingleUserNotebookApp(NotebookApp):
hub_prefix = Unicode('/hub/').tag(config=True)
@default('keyfile')
def _keyfile_default(self):
return os.environ.get('JUPYTERHUB_SSL_KEYFILE') or ''
@default('certfile')
def _certfile_default(self):
return os.environ.get('JUPYTERHUB_SSL_CERTFILE') or ''
@default('client_ca')
def _client_ca_default(self):
return os.environ.get('JUPYTERHUB_SSL_CLIENT_CA') or ''
@default('hub_prefix')
def _hub_prefix_default(self):
base_url = os.environ.get('JUPYTERHUB_BASE_URL') or '/'
@@ -379,6 +391,13 @@ class SingleUserNotebookApp(NotebookApp):
- exit if I can't connect at all
- check version and warn on sufficient mismatch
"""
ssl_context = make_ssl_context(
self.keyfile,
self.certfile,
cafile=self.client_ca,
)
AsyncHTTPClient.configure(None, defaults={"ssl_options" : ssl_context})
client = AsyncHTTPClient()
RETRIES = 5
for i in range(1, RETRIES+1):
@@ -424,6 +443,9 @@ class SingleUserNotebookApp(NotebookApp):
api_url=self.hub_api_url,
hub_prefix=self.hub_prefix,
base_url=self.base_url,
keyfile=self.keyfile,
certfile=self.certfile,
client_ca=self.client_ca,
)
# smoke check
if not self.hub_auth.oauth_client_id:

166
jupyterhub/spawner.py
View File

@@ -14,6 +14,7 @@ import shutil
import signal
import sys
import warnings
import pwd
from subprocess import Popen
from tempfile import mkdtemp
@@ -27,7 +28,7 @@ from tornado.ioloop import PeriodicCallback
from traitlets.config import LoggingConfigurable
from traitlets import (
Any, Bool, Dict, Instance, Integer, Float, List, Unicode, Union,
observe, validate,
default, observe, validate,
)
from .objects import Server
@@ -162,6 +163,12 @@ class Spawner(LoggingConfigurable):
if self.orm_spawner:
return self.orm_spawner.name
return ''
hub = Any()
authenticator = Any()
internal_ssl = Bool(False)
internal_trust_bundles = Dict()
internal_certs_location = Unicode('')
cert_paths = Dict()
admin_access = Bool(False)
api_token = Unicode()
oauth_client_id = Unicode()
@@ -644,6 +651,11 @@ class Spawner(LoggingConfigurable):
if self.cpu_guarantee:
env['CPU_GUARANTEE'] = str(self.cpu_guarantee)
if self.cert_paths:
env['JUPYTERHUB_SSL_KEYFILE'] = self.cert_paths['keyfile']
env['JUPYTERHUB_SSL_CERTFILE'] = self.cert_paths['certfile']
env['JUPYTERHUB_SSL_CLIENT_CA'] = self.cert_paths['cafile']
return env
def template_namespace(self):
@@ -684,6 +696,115 @@ class Spawner(LoggingConfigurable):
"""
return s.format(**self.template_namespace())
trusted_alt_names = List(Unicode())
ssl_alt_names = List(
Unicode(),
config=True,
help="""List of SSL alt names
May be set in config if all spawners should have the same value(s),
or set at runtime by Spawner that know their names.
"""
)
@default('ssl_alt_names')
def _default_ssl_alt_names(self):
# by default, use trusted_alt_names
# inherited from global app
return list(self.trusted_alt_names)
ssl_alt_names_include_local = Bool(
True,
config=True,
help="""Whether to include DNS:localhost, IP:127.0.0.1 in alt names""",
)
async def create_certs(self):
"""Create and set ownership for the certs to be used for internal ssl
Keyword Arguments:
alt_names (list): a list of alternative names to identify the
server by, see:
https://en.wikipedia.org/wiki/Subject_Alternative_Name
override: override the default_names with the provided alt_names
Returns:
dict: Path to cert files and CA
This method creates certs for use with the singleuser notebook. It
enables SSL and ensures that the notebook can perform bi-directional
SSL auth with the hub (verification based on CA).
If the singleuser host has a name or ip other than localhost,
an appropriate alternative name(s) must be passed for ssl verification
by the hub to work. For example, for Jupyter hosts with an IP of
10.10.10.10 or DNS name of jupyter.example.com, this would be:
alt_names=["IP:10.10.10.10"]
alt_names=["DNS:jupyter.example.com"]
respectively. The list can contain both the IP and DNS names to refer
to the host by either IP or DNS name (note the `default_names` below).
"""
from certipy import Certipy
default_names = ["DNS:localhost", "IP:127.0.0.1"]
alt_names = []
alt_names.extend(self.ssl_alt_names)
if self.ssl_alt_names_include_local:
alt_names = default_names + alt_names
self.log.info("Creating certs for %s: %s",
self._log_name,
';'.join(alt_names),
)
common_name = self.user.name or 'service'
certipy = Certipy(store_dir=self.internal_certs_location)
notebook_component = 'notebooks-ca'
notebook_key_pair = certipy.create_signed_pair(
'user-' + common_name,
notebook_component,
alt_names=alt_names,
overwrite=True
)
paths = {
"keyfile": notebook_key_pair['files']['key'],
"certfile": notebook_key_pair['files']['cert'],
"cafile": self.internal_trust_bundles[notebook_component],
}
return paths
async def move_certs(self, paths):
"""Takes certificate paths and makes them available to the notebook server
Arguments:
paths (dict): a list of paths for key, cert, and CA.
These paths will be resolvable and readable by the Hub process,
but not necessarily by the notebook server.
Returns:
dict: a list (potentially altered) of paths for key, cert,
and CA.
These paths should be resolvable and readable
by the notebook server to be launched.
`.move_certs` is called after certs for the singleuser notebook have
been created by create_certs.
By default, certs are created in a standard, central location defined
by `internal_certs_location`. For a local, single-host deployment of
JupyterHub, this should suffice. If, however, singleuser notebooks
are spawned on other hosts, `.move_certs` should be overridden to move
these files appropriately. This could mean using `scp` to copy them
to another host, moving them to a volume mounted in a docker container,
or exporting them as a secret in kubernetes.
"""
return paths
def get_args(self):
"""Return the arguments to be passed after self.cmd
@@ -927,7 +1048,6 @@ def set_user_setuid(username, chdir=True):
home directory.
"""
import grp
import pwd
user = pwd.getpwnam(username)
uid = user.pw_uid
gid = user.pw_gid
@@ -1088,6 +1208,48 @@ class LocalProcessSpawner(Spawner):
env = self.user_env(env)
return env
async def move_certs(self, paths):
"""Takes cert paths, moves and sets ownership for them
Arguments:
paths (dict): a list of paths for key, cert, and CA
Returns:
dict: a list (potentially altered) of paths for key, cert,
and CA
Stage certificates into a private home directory
and make them readable by the user.
"""
key = paths['keyfile']
cert = paths['certfile']
ca = paths['cafile']
user = pwd.getpwnam(self.user.name)
uid = user.pw_uid
gid = user.pw_gid
home = user.pw_dir
# Create dir for user's certs wherever we're starting
out_dir = "{home}/.jupyterhub/jupyterhub-certs".format(home=home)
shutil.rmtree(out_dir, ignore_errors=True)
os.makedirs(out_dir, 0o700, exist_ok=True)
# Move certs to users dir
shutil.move(paths['keyfile'], out_dir)
shutil.move(paths['certfile'], out_dir)
shutil.copy(paths['cafile'], out_dir)
key = os.path.join(out_dir, os.path.basename(paths['keyfile']))
cert = os.path.join(out_dir, os.path.basename(paths['certfile']))
ca = os.path.join(out_dir, os.path.basename(paths['cafile']))
# Set cert ownership to user
for f in [out_dir, key, cert, ca]:
shutil.chown(f, user=uid, group=gid)
return {"keyfile": key, "certfile": cert, "cafile": ca}
async def start(self):
"""Start the single-user server."""
self.port = random_port()

52
jupyterhub/tests/conftest.py
View File

@@ -27,14 +27,18 @@ Fixtures to add functionality or spawning behavior
# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.
import asyncio
from getpass import getuser
import logging
import os
from pytest import fixture, raises
import sys
from subprocess import TimeoutExpired
from unittest import mock
from pytest import fixture, raises
from tornado import ioloop, gen
from tornado.httpclient import HTTPError
from unittest import mock
from tornado.platform.asyncio import AsyncIOMainLoop
from .. import orm
from .. import crypto
@@ -42,6 +46,7 @@ from ..utils import random_port
from . import mocking
from .mocking import MockHub
from .utils import ssl_setup
from .test_services import mockservice_cmd
import jupyterhub.services.service
@@ -51,24 +56,41 @@ _db = None
@fixture(scope='module')
def app(request, io_loop):
def ssl_tmpdir(tmpdir_factory):
return tmpdir_factory.mktemp('ssl')
@fixture(scope='module')
def app(request, io_loop, ssl_tmpdir):
"""Mock a jupyterhub app for testing"""
mocked_app = MockHub.instance(log_level=logging.DEBUG)
mocked_app = None
ssl_enabled = getattr(request.module, "ssl_enabled", False)
kwargs = dict(log_level=logging.DEBUG)
if ssl_enabled:
kwargs.update(
dict(
internal_ssl=True,
internal_certs_location=str(ssl_tmpdir),
)
)
@gen.coroutine
def make_app():
yield mocked_app.initialize([])
yield mocked_app.start()
mocked_app = MockHub.instance(**kwargs)
io_loop.run_sync(make_app)
async def make_app():
await mocked_app.initialize([])
await mocked_app.start()
def fin():
# disconnect logging during cleanup because pytest closes captured FDs prematurely
mocked_app.log.handlers = []
MockHub.clear_instance()
mocked_app.stop()
try:
mocked_app.stop()
except Exception as e:
print("Error stopping Hub: %s" % e, file=sys.stderr)
request.addfinalizer(fin)
io_loop.run_sync(make_app)
return mocked_app
@@ -106,8 +128,13 @@ def db():
@fixture(scope='module')
def io_loop(request):
"""Same as pytest-tornado.io_loop, but re-scoped to module-level"""
ioloop.IOLoop.configure(AsyncIOMainLoop)
aio_loop = asyncio.new_event_loop()
asyncio.set_event_loop(aio_loop)
io_loop = ioloop.IOLoop()
io_loop.make_current()
assert asyncio.get_event_loop() is aio_loop
assert io_loop.asyncio_loop is aio_loop
def _close():
io_loop.clear_current()
@@ -162,7 +189,10 @@ def _mockservice(request, app, url=False):
'admin': True,
}
if url:
spec['url'] = 'http://127.0.0.1:%i' % random_port()
if app.internal_ssl:
spec['url'] = 'https://127.0.0.1:%i' % random_port()
else:
spec['url'] = 'http://127.0.0.1:%i' % random_port()
io_loop = app.io_loop

35
jupyterhub/tests/mocking.py
View File

@@ -40,7 +40,7 @@ from tornado import gen
from tornado.concurrent import Future
from tornado.ioloop import IOLoop
from traitlets import default
from traitlets import Bool, default
from ..app import JupyterHub
from ..auth import PAMAuthenticator
@@ -49,7 +49,7 @@ from ..objects import Server
from ..spawner import LocalProcessSpawner
from ..singleuser import SingleUserNotebookApp
from ..utils import random_port, url_path_join
from .utils import async_requests
from .utils import async_requests, ssl_setup
from pamela import PAMError
@@ -95,6 +95,10 @@ class MockSpawner(LocalProcessSpawner):
def _cmd_default(self):
return [sys.executable, '-m', 'jupyterhub.tests.mocksu']
def move_certs(self, paths):
"""Return the paths unmodified"""
return paths
use_this_api_token = None
def start(self):
if self.use_this_api_token:
@@ -217,6 +221,14 @@ class MockHub(JupyterHub):
db_file = None
last_activity_interval = 2
log_datefmt = '%M:%S'
external_certs = None
log_level = 10
def __init__(self, *args, **kwargs):
if 'internal_certs_location' in kwargs:
cert_location = kwargs['internal_certs_location']
kwargs['external_certs'] = ssl_setup(cert_location, 'hub-ca')
super().__init__(*args, **kwargs)
@default('subdomain_host')
def _subdomain_host_default(self):
@@ -228,7 +240,7 @@ class MockHub(JupyterHub):
port = urlparse(self.subdomain_host).port
else:
port = random_port()
return 'http://127.0.0.1:%i/@/space%%20word/' % port
return 'http://127.0.0.1:%i/@/space%%20word/' % (port,)
@default('ip')
def _ip_default(self):
@@ -270,6 +282,18 @@ class MockHub(JupyterHub):
self.db.expire(service)
return super().init_services()
test_clean_db = Bool(True)
def init_db(self):
"""Ensure we start with a clean user list"""
super().init_db()
if self.test_clean_db:
for user in self.db.query(orm.User):
self.db.delete(user)
for group in self.db.query(orm.Group):
self.db.delete(group)
self.db.commit()
@gen.coroutine
def initialize(self, argv=None):
self.pid_file = NamedTemporaryFile(delete=False).name
@@ -310,12 +334,16 @@ class MockHub(JupyterHub):
def login_user(self, name):
"""Login a user by name, returning her cookies."""
base_url = public_url(self)
external_ca = None
if self.internal_ssl:
external_ca = self.external_certs['files']['ca']
r = yield async_requests.post(base_url + 'hub/login',
data={
'username': name,
'password': name,
},
allow_redirects=False,
verify=external_ca,
)
r.raise_for_status()
assert r.cookies
@@ -373,6 +401,7 @@ class StubSingleUserSpawner(MockSpawner):
evt = threading.Event()
print(args, env)
def _run():
asyncio.set_event_loop(asyncio.new_event_loop())
io_loop = IOLoop()
io_loop.make_current()
io_loop.add_callback(lambda : evt.set())

15
jupyterhub/tests/mockservice.py
View File

@@ -23,6 +23,7 @@ import requests
from tornado import web, httpserver, ioloop
from jupyterhub.services.auth import HubAuthenticated, HubOAuthenticated, HubOAuthCallbackHandler
from jupyterhub.utils import make_ssl_context
class EchoHandler(web.RequestHandler):
@@ -85,7 +86,19 @@ def main():
(r'.*', EchoHandler),
], cookie_secret=os.urandom(32))
server = httpserver.HTTPServer(app)
ssl_context = None
key = os.environ.get('JUPYTERHUB_SSL_KEYFILE') or ''
cert = os.environ.get('JUPYTERHUB_SSL_CERTFILE') or ''
ca = os.environ.get('JUPYTERHUB_SSL_CLIENT_CA') or ''
if key and cert and ca:
ssl_context = make_ssl_context(
key,
cert,
cafile = ca,
check_hostname = False)
server = httpserver.HTTPServer(app, ssl_options=ssl_context)
server.listen(url.port, url.hostname)
try:
ioloop.IOLoop.instance().start()

17
jupyterhub/tests/mocksu.py
View File

@@ -14,9 +14,11 @@ Handlers and their purpose include:
import argparse
import json
import sys
import os
from tornado import web, httpserver, ioloop
from .mockservice import EnvHandler
from ..utils import make_ssl_context
class EchoHandler(web.RequestHandler):
def get(self):
@@ -34,7 +36,20 @@ def main(args):
(r'.*', EchoHandler),
])
server = httpserver.HTTPServer(app)
ssl_context = None
key = os.environ.get('JUPYTERHUB_SSL_KEYFILE') or ''
cert = os.environ.get('JUPYTERHUB_SSL_CERTFILE') or ''
ca = os.environ.get('JUPYTERHUB_SSL_CLIENT_CA') or ''
if key and cert and ca:
ssl_context = make_ssl_context(
key,
cert,
cafile = ca,
check_hostname = False
)
server = httpserver.HTTPServer(app, ssl_options=ssl_context)
server.listen(args.port)
try:
ioloop.IOLoop.instance().start()

29
jupyterhub/tests/test_api.py
View File

@@ -91,11 +91,17 @@ def api_request(app, *api_path, **kwargs):
headers = kwargs.setdefault('headers', {})
if 'Authorization' not in headers and not kwargs.pop('noauth', False):
headers.update(auth_header(app.db, 'admin'))
# make a copy to avoid modifying arg in-place
kwargs['headers'] = h = {}
h.update(headers)
h.update(auth_header(app.db, 'admin'))
url = ujoin(base_url, 'api', *api_path)
method = kwargs.pop('method', 'get')
f = getattr(async_requests, method)
if app.internal_ssl:
kwargs['cert'] = (app.internal_ssl_cert, app.internal_ssl_key)
kwargs["verify"] = app.internal_ssl_ca
resp = yield f(url, **kwargs)
assert "frame-ancestors 'self'" in resp.headers['Content-Security-Policy']
assert ujoin(app.hub.base_url, "security/csp-report") in resp.headers['Content-Security-Policy']
@@ -575,15 +581,19 @@ def test_spawn(app):
assert spawner.server.base_url == ujoin(app.base_url, 'user/%s' % name) + '/'
url = public_url(app, user)
r = yield async_requests.get(url)
kwargs = {}
if app.internal_ssl:
kwargs['cert'] = (app.internal_ssl_cert, app.internal_ssl_key)
kwargs["verify"] = app.internal_ssl_ca
r = yield async_requests.get(url, **kwargs)
assert r.status_code == 200
assert r.text == spawner.server.base_url
r = yield async_requests.get(ujoin(url, 'args'))
r = yield async_requests.get(ujoin(url, 'args'), **kwargs)
assert r.status_code == 200
argv = r.json()
assert '--port' in ' '.join(argv)
r = yield async_requests.get(ujoin(url, 'env'))
r = yield async_requests.get(ujoin(url, 'env'), **kwargs)
env = r.json()
for expected in ['JUPYTERHUB_USER', 'JUPYTERHUB_BASE_URL', 'JUPYTERHUB_API_TOKEN']:
assert expected in env
@@ -620,8 +630,11 @@ def test_spawn_handler(app):
# verify that request params got passed down
# implemented in MockSpawner
kwargs = {}
if app.external_certs:
kwargs['verify'] = app.external_certs['files']['ca']
url = public_url(app, user)
r = yield async_requests.get(ujoin(url, 'env'))
r = yield async_requests.get(ujoin(url, 'env'), **kwargs)
env = r.json()
assert 'HANDLER_ARGS' in env
assert env['HANDLER_ARGS'] == 'foo=bar'
@@ -1614,7 +1627,11 @@ def test_get_service(app, mockservice_url):
def test_root_api(app):
base_url = app.hub.url
url = ujoin(base_url, 'api')
r = yield async_requests.get(url)
kwargs = {}
if app.internal_ssl:
kwargs['cert'] = (app.internal_ssl_cert, app.internal_ssl_key)
kwargs["verify"] = app.internal_ssl_ca
r = yield async_requests.get(url, **kwargs)
r.raise_for_status()
expected = {
'version': jupyterhub.__version__

58
jupyterhub/tests/test_app.py
View File

@@ -64,7 +64,7 @@ def test_generate_config():
@pytest.mark.gen_test
def test_init_tokens():
def test_init_tokens(request):
with TemporaryDirectory() as td:
db_file = os.path.join(td, 'jupyterhub.sqlite')
tokens = {
@@ -72,7 +72,11 @@ def test_init_tokens():
'also-super-secret': 'gordon',
'boagasdfasdf': 'chell',
}
app = MockHub(db_url=db_file, api_tokens=tokens)
kwargs = {'db_url': db_file, 'api_tokens': tokens}
ssl_enabled = getattr(request.module, "ssl_enabled", False)
if ssl_enabled:
kwargs['internal_certs_location'] = td
app = MockHub(**kwargs)
yield app.initialize([])
db = app.db
for token, username in tokens.items():
@@ -82,7 +86,7 @@ def test_init_tokens():
assert user.name == username
# simulate second startup, reloading same tokens:
app = MockHub(db_url=db_file, api_tokens=tokens)
app = MockHub(**kwargs)
yield app.initialize([])
db = app.db
for token, username in tokens.items():
@@ -93,27 +97,35 @@ def test_init_tokens():
# don't allow failed token insertion to create users:
tokens['short'] = 'gman'
app = MockHub(db_url=db_file, api_tokens=tokens)
app = MockHub(**kwargs)
with pytest.raises(ValueError):
yield app.initialize([])
assert orm.User.find(app.db, 'gman') is None
def test_write_cookie_secret(tmpdir):
def test_write_cookie_secret(tmpdir, request):
secret_path = str(tmpdir.join('cookie_secret'))
hub = MockHub(cookie_secret_file=secret_path)
kwargs = {'cookie_secret_file': secret_path}
ssl_enabled = getattr(request.module, "ssl_enabled", False)
if ssl_enabled:
kwargs['internal_certs_location'] = str(tmpdir)
hub = MockHub(**kwargs)
hub.init_secrets()
assert os.path.exists(secret_path)
assert os.stat(secret_path).st_mode & 0o600
assert not os.stat(secret_path).st_mode & 0o177
def test_cookie_secret_permissions(tmpdir):
def test_cookie_secret_permissions(tmpdir, request):
secret_file = tmpdir.join('cookie_secret')
secret_path = str(secret_file)
secret = os.urandom(COOKIE_SECRET_BYTES)
secret_file.write(binascii.b2a_hex(secret))
hub = MockHub(cookie_secret_file=secret_path)
kwargs = {'cookie_secret_file': secret_path}
ssl_enabled = getattr(request.module, "ssl_enabled", False)
if ssl_enabled:
kwargs['internal_certs_location'] = str(tmpdir)
hub = MockHub(**kwargs)
# raise with public secret file
os.chmod(secret_path, 0o664)
@@ -126,18 +138,26 @@ def test_cookie_secret_permissions(tmpdir):
assert hub.cookie_secret == secret
def test_cookie_secret_content(tmpdir):
def test_cookie_secret_content(tmpdir, request):
secret_file = tmpdir.join('cookie_secret')
secret_file.write('not base 64: uñiço∂e')
secret_path = str(secret_file)
os.chmod(secret_path, 0o660)
hub = MockHub(cookie_secret_file=secret_path)
kwargs = {'cookie_secret_file': secret_path}
ssl_enabled = getattr(request.module, "ssl_enabled", False)
if ssl_enabled:
kwargs['internal_certs_location'] = str(tmpdir)
hub = MockHub(**kwargs)
with pytest.raises(SystemExit):
hub.init_secrets()
def test_cookie_secret_env(tmpdir):
hub = MockHub(cookie_secret_file=str(tmpdir.join('cookie_secret')))
def test_cookie_secret_env(tmpdir, request):
kwargs = {'cookie_secret_file': str(tmpdir.join('cookie_secret'))}
ssl_enabled = getattr(request.module, "ssl_enabled", False)
if ssl_enabled:
kwargs['internal_certs_location'] = str(tmpdir)
hub = MockHub(**kwargs)
with patch.dict(os.environ, {'JPY_COOKIE_SECRET': 'not hex'}):
with pytest.raises(ValueError):
@@ -150,12 +170,16 @@ def test_cookie_secret_env(tmpdir):
@pytest.mark.gen_test
def test_load_groups():
def test_load_groups(tmpdir, request):
to_load = {
'blue': ['cyclops', 'rogue', 'wolverine'],
'gold': ['storm', 'jean-grey', 'colossus'],
}
hub = MockHub(load_groups=to_load)
kwargs = {'load_groups': to_load}
ssl_enabled = getattr(request.module, "ssl_enabled", False)
if ssl_enabled:
kwargs['internal_certs_location'] = str(tmpdir)
hub = MockHub(**kwargs)
hub.init_db()
yield hub.init_users()
yield hub.init_groups()
@@ -178,7 +202,11 @@ def test_resume_spawners(tmpdir, request):
request.addfinalizer(p.stop)
@gen.coroutine
def new_hub():
app = MockHub()
kwargs = {}
ssl_enabled = getattr(request.module, "ssl_enabled", False)
if ssl_enabled:
kwargs['internal_certs_location'] = str(tmpdir)
app = MockHub(test_clean_db=False, **kwargs)
app.config.ConfigurableHTTPProxy.should_start = False
app.config.ConfigurableHTTPProxy.auth_token = 'unused'
yield app.initialize([])

7
jupyterhub/tests/test_internal_ssl_api.py Normal file
View File

@@ -0,0 +1,7 @@
"""Tests for the SSL enabled REST API."""
# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.
from jupyterhub.tests.test_api import *
ssl_enabled = True

10
jupyterhub/tests/test_internal_ssl_app.py Normal file
View File

@@ -0,0 +1,10 @@
"""Test the JupyterHub entry point with internal ssl"""
# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.
import sys
import jupyterhub.tests.mocking
from jupyterhub.tests.test_app import *
ssl_enabled = True

77
jupyterhub/tests/test_internal_ssl_connections.py Normal file
View File

@@ -0,0 +1,77 @@
"""Tests for jupyterhub internal_ssl connections"""
import time
from subprocess import check_output
import sys
from urllib.parse import urlparse
import pytest
import jupyterhub
from tornado import gen
from unittest import mock
from requests.exceptions import SSLError
from .utils import async_requests
from .test_api import add_user
ssl_enabled = True
@gen.coroutine
def wait_for_spawner(spawner, timeout=10):
"""Wait for an http server to show up
polling at shorter intervals for early termination
"""
deadline = time.monotonic() + timeout
def wait():
return spawner.server.wait_up(timeout=1, http=True)
while time.monotonic() < deadline:
status = yield spawner.poll()
assert status is None
try:
yield wait()
except TimeoutError:
continue
else:
break
yield wait()
@pytest.mark.gen_test
def test_connection_hub_wrong_certs(app):
"""Connecting to the internal hub url fails without correct certs"""
with pytest.raises(SSLError):
kwargs = {'verify': False}
r = yield async_requests.get(app.hub.url, **kwargs)
r.raise_for_status()
@pytest.mark.gen_test
def test_connection_proxy_api_wrong_certs(app):
"""Connecting to the proxy api fails without correct certs"""
with pytest.raises(SSLError):
kwargs = {'verify': False}
r = yield async_requests.get(app.proxy.api_url, **kwargs)
r.raise_for_status()
@pytest.mark.gen_test
def test_connection_notebook_wrong_certs(app):
"""Connecting to a notebook fails without correct certs"""
with mock.patch.dict(
app.config.LocalProcessSpawner,
{'cmd': [sys.executable, '-m', 'jupyterhub.tests.mocksu']}
):
user = add_user(app.db, app, name='foo')
yield user.spawn()
yield wait_for_spawner(user.spawner)
spawner = user.spawner
status = yield spawner.poll()
assert status is None
with pytest.raises(SSLError):
kwargs = {'verify': False}
r = yield async_requests.get(spawner.server.url, **kwargs)
r.raise_for_status()

7
jupyterhub/tests/test_internal_ssl_spawner.py Normal file
View File

@@ -0,0 +1,7 @@
"""Tests for process spawning with internal_ssl"""
# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.
from jupyterhub.tests.test_spawner import *
ssl_enabled = True

8
jupyterhub/tests/test_spawner.py
View File

@@ -253,6 +253,7 @@ def test_shell_cmd(db, tmpdir, request):
r.raise_for_status()
env = r.json()
assert env['TESTVAR'] == 'foo'
yield s.stop()
def test_inherit_overwrite():
@@ -401,8 +402,13 @@ def test_spawner_routing(app, name):
yield user.spawn()
yield wait_for_spawner(user.spawner)
yield app.proxy.add_user(user)
kwargs = {'allow_redirects': False}
if app.internal_ssl:
kwargs['cert'] = (app.internal_ssl_cert, app.internal_ssl_key)
kwargs["verify"] = app.internal_ssl_ca
url = url_path_join(public_url(app, user), "test/url")
r = yield async_requests.get(url, allow_redirects=False)
r = yield async_requests.get(url, **kwargs)
r.raise_for_status()
assert r.url == url
assert r.text == urlparse(url).path
yield user.stop()

16
jupyterhub/tests/utils.py
View File

@@ -1,6 +1,8 @@
from concurrent.futures import ThreadPoolExecutor
import requests
from certipy import Certipy
class _AsyncRequests:
"""Wrapper around requests to return a Future from request methods
@@ -13,10 +15,24 @@ class _AsyncRequests:
requests_method = getattr(requests, name)
return lambda *args, **kwargs: self.executor.submit(requests_method, *args, **kwargs)
# async_requests.get = requests.get returning a Future, etc.
async_requests = _AsyncRequests()
class AsyncSession(requests.Session):
"""requests.Session object that runs in the background thread"""
def request(self, *args, **kwargs):
return async_requests.executor.submit(super().request, *args, **kwargs)
def ssl_setup(cert_dir, authority_name):
# Set up the external certs with the same authority as the internal
# one so that certificate trust works regardless of chosen endpoint.
certipy = Certipy(store_dir=cert_dir)
alt_names = ['DNS:localhost', 'IP:127.0.0.1']
internal_authority = certipy.create_ca(authority_name, overwrite=True)
external_certs = certipy.create_signed_pair('external', authority_name,
overwrite=True,
alt_names=alt_names)
return external_certs

37
jupyterhub/user.py
View File

@@ -10,7 +10,7 @@ from sqlalchemy import inspect
from tornado import gen
from tornado.log import app_log
from .utils import maybe_future, url_path_join
from .utils import maybe_future, url_path_join, make_ssl_context
from . import orm
from ._version import _check_version, __version__
@@ -229,6 +229,12 @@ class User:
client_id = 'jupyterhub-user-%s' % quote(self.name)
if server_name:
client_id = '%s-%s' % (client_id, quote(server_name))
trusted_alt_names = []
trusted_alt_names.extend(self.settings.get('trusted_alt_names', []))
if self.settings.get('subdomain_host'):
trusted_alt_names.append('DNS:' + self.domain)
spawn_kwargs = dict(
user=self,
orm_spawner=orm_spawner,
@@ -239,7 +245,19 @@ class User:
db=self.db,
oauth_client_id=client_id,
cookie_options = self.settings.get('cookie_options', {}),
trusted_alt_names=trusted_alt_names,
)
if self.settings.get('internal_ssl'):
ssl_kwargs = dict(
internal_ssl=self.settings.get('internal_ssl'),
internal_trust_bundles=self.settings.get(
'internal_trust_bundles'),
internal_certs_location=self.settings.get(
'internal_certs_location'),
)
spawn_kwargs.update(ssl_kwargs)
# update with kwargs. Mainly for testing.
spawn_kwargs.update(kwargs)
spawner = spawner_class(**spawn_kwargs)
@@ -431,6 +449,11 @@ class User:
try:
# run optional preparation work to bootstrap the notebook
await maybe_future(spawner.run_pre_spawn_hook())
if self.settings.get('internal_ssl'):
self.log.debug("Creating internal SSL certs for %s", spawner._log_name)
hub_paths = await maybe_future(spawner.create_certs())
spawner.cert_paths = await maybe_future(spawner.move_certs(hub_paths))
self.log.debug("Calling Spawner.start for %s", spawner._log_name)
f = maybe_future(spawner.start())
# commit any changes in spawner.start (always commit db changes before yield)
db.commit()
@@ -442,7 +465,8 @@ class User:
pass
else:
# >= 0.7 returns (ip, port)
url = 'http://%s:%i' % url
proto = 'https' if self.settings['internal_ssl'] else 'http'
url = '%s://%s:%i' % ((proto,) + url)
urlinfo = urlparse(url)
server.proto = urlinfo.scheme
server.ip = urlinfo.hostname
@@ -526,8 +550,15 @@ class User:
spawner.orm_spawner.state = spawner.get_state()
db.commit()
spawner._waiting_for_response = True
key = self.settings.get('internal_ssl_key')
cert = self.settings.get('internal_ssl_cert')
ca = self.settings.get('internal_ssl_ca')
ssl_context = make_ssl_context(key, cert, cafile=ca)
try:
resp = await server.wait_up(http=True, timeout=spawner.http_timeout)
resp = await server.wait_up(
http=True,
timeout=spawner.http_timeout,
ssl_context=ssl_context)
except Exception as e:
if isinstance(e, TimeoutError):
self.log.warning(

22
jupyterhub/utils.py
View File

@@ -16,6 +16,7 @@ import os
import socket
import sys
import threading
import ssl
import uuid
import warnings
@@ -70,6 +71,21 @@ def can_connect(ip, port):
else:
return True
def make_ssl_context(
keyfile, certfile, cafile=None,
verify=True, check_hostname=True):
"""Setup context for starting an https server or making requests over ssl.
"""
if not keyfile or not certfile:
return None
purpose = ssl.Purpose.SERVER_AUTH if verify else ssl.Purpose.CLIENT_AUTH
ssl_context = ssl.create_default_context(purpose, cafile=cafile)
ssl_context.load_cert_chain(certfile, keyfile)
ssl_context.check_hostname = check_hostname
return ssl_context
async def exponential_backoff(
pass_func,
fail_message,
@@ -166,12 +182,16 @@ async def wait_for_server(ip, port, timeout=10):
)
async def wait_for_http_server(url, timeout=10):
async def wait_for_http_server(url, timeout=10, ssl_context=None):
"""Wait for an HTTP Server to respond at url.
Any non-5XX response code will do, even 404.
"""
loop = ioloop.IOLoop.current()
tic = loop.time()
client = AsyncHTTPClient()
if ssl_context:
client.ssl_options = ssl_context
async def is_reachable():
try:
r = await client.fetch(url, follow_redirects=False)

1
requirements.txt
View File

@@ -10,3 +10,4 @@ python-dateutil
SQLAlchemy>=1.1
requests
prometheus_client>=0.0.21
certipy>=0.1.2