Compare commits

..

190 Commits
0.4.1 ... 0.6.0

Author SHA1 Message Date
Min RK
3046971064 release 0.6 2016-04-25 14:10:29 +02:00
Min RK
30498f97c4 Merge pull request #543 from robnagler:master
Allow jupyterhub-singleuser to run on python 2 install

closes #543
2016-04-25 11:35:40 +02:00
robnagler
d9d68efa55 run with default python, which might be python 2 2016-04-25 11:31:17 +02:00
Min RK
4125dc7ad0 Merge pull request #542 from willingc/doc-addition
Add troubleshooting documentation for 500 issue
2016-04-22 15:23:28 +02:00
Carol Willing
13600894fb Changed link re: Min's tip 2016-04-22 06:11:53 -07:00
Carol Willing
1b796cd871 Add links 2016-04-22 05:37:30 -07:00
Carol Willing
e7889dc12e Add 500 error to troubleshooting docs 2016-04-22 05:36:15 -07:00
Carol Willing
244a3b1000 Merge pull request #541 from minrk/cookie-referer
check referer only if there is a valid user cookie
2016-04-22 05:03:05 -07:00
Carol Willing
05dfda469f Merge pull request #540 from minrk/0.6
Changelog for 0.6
2016-04-22 04:21:44 -07:00
Min RK
6b19ee792d check referer only if there is a valid user cookie
avoids misleading "Blocking Cross Origin..." message
when there's no logged-in user for API requests.
2016-04-22 13:16:13 +02:00
Min RK
ace38d744a Changelog for 0.6 2016-04-22 12:50:49 +02:00
Min RK
56a5ed8c87 Merge pull request #539 from minrk/unused-email
add ignored -e arg to docker login
2016-04-22 10:50:15 +02:00
Min RK
60e8a76476 add ignored -e arg to docker login
doesn't appear to be needed on more recent docker
2016-04-22 10:24:40 +02:00
Min RK
552800ceb7 add sec doc
reviewed on security list
2016-04-22 10:20:18 +02:00
Carol Willing
7dd1900f5f Merge pull request #521 from minrk/docker-onbuild
Move docker onbuild step to jupyterhub-onbuild
2016-04-21 18:26:43 -07:00
Min RK
35c261d0ed better Dockerfile comments from Carol 2016-04-21 21:32:36 +02:00
Min RK
fa34ce64b7 include dockerfiles in manifest 2016-04-21 13:50:35 +02:00
Min RK
f0504420a9 move docker onbuild to directory 2016-04-21 13:50:27 +02:00
Min RK
8666f3a46c push onbuild image to docker hub with circle-ci 2016-04-21 13:38:15 +02:00
Carol Willing
60d6019cf7 Merge pull request #534 from jupyterhub/willingc-patch-1
Fix post move links to jupyterhub org in README
2016-04-20 20:05:39 -07:00
Carol Willing
173daeeb09 Fix post move links to jupyterhub org in README 2016-04-20 20:00:18 -07:00
Carol Willing
cf988dca4d Merge pull request #531 from minrk/extra-log-file-doc
[DOC] Note that extra_log_file only affects Hub's logs
2016-04-20 06:00:01 -07:00
Min RK
ffc2faabf7 [DOC] Note that extra_log_file only affects Hub's logs
not single-user server logs, or anything else.
2016-04-20 14:45:22 +02:00
Min RK
9fed0334c8 jupyterhub path in dockerfiles 2016-04-20 14:43:25 +02:00
Min RK
8b61eb7347 install from miniconda 4.0.5
- checksum miniconda installer
- move jupyterhub src to /src/jupyterhub
2016-04-19 13:21:25 +02:00
Min RK
9cdda101c7 Move onbuild step to jupyterhub-onbuild
Removes onbuild from from jupyter/jupyterhub image,
though it remains incomplete and will not run without a config file.
2016-04-19 13:21:24 +02:00
Carol Willing
f3bbca80ea Merge pull request #528 from minrk/test-cookie-secret
exercise cookie secret loading in tests
2016-04-19 04:01:36 -07:00
Carol Willing
ce30f28449 Merge pull request #527 from minrk/polish-520
polish cookie-secret PR
2016-04-19 03:56:41 -07:00
Min RK
6cb58c17e7 exercise cookie secret loading in tests 2016-04-19 11:15:48 +02:00
Min RK
183e244490 polish cookie-secret PR
- fix a couple of typos
- use ValueError instead of assert to ensure error is raised even when Python optimizes-out asserts
2016-04-19 10:15:10 +02:00
Min RK
d5cd5115a5 Merge pull request #520 from robnagler/master
cookie_secret file must be base64
2016-04-19 10:10:58 +02:00
robnagler
bbd3b22490 incorrect log call in previous checkin 2016-04-18 16:25:03 +00:00
robnagler
e02daf01ad Fix jupyter/jupyterhub#520: exit if any errors parsing file; Also, fix abstraction use of getenv/os.environ (use one or the other, not both) 2016-04-18 15:35:31 +00:00
robnagler
af1e253f8a Fix jupyter/jupyterhub#522 2016-04-18 15:16:01 +00:00
Min RK
491da69994 typo 2016-04-18 12:51:35 +02:00
Min RK
0737600d3c Merge pull request #515 from proversity-org/master
Post handler for  requesting authorization tokens; authenticated via form.
2016-04-18 11:23:24 +02:00
dominic
c7f542e79e Add tests for form based token generation 2016-04-18 10:27:38 +02:00
robnagler
21213c97c6 cookie_secret file is decoded by binascii.a2b_base64 so need to document it must be Base64. Added better doc for other values, and included description of "cookie_secret" parameter as well 2016-04-17 23:35:06 +00:00
Carol Willing
b36cd92ae6 Merge pull request #517 from minrk/load-tokens
allow pre-loading API tokens from config
2016-04-15 06:49:40 -07:00
Min RK
094ac451c7 Don't allow bad tokens to create tokens in the db 2016-04-15 12:42:52 +02:00
Min RK
fa4b666693 allow pre-loading API tokens from config
This is the first small part of easing the pain of services,
which is generating the API tokens,
and used to require initializing the JupyterHub database.
2016-04-14 16:45:40 +02:00
Carol Willing
ce9dc2093c Merge pull request #514 from minrk/docker-readme
revisions to docker notes
2016-04-14 07:05:02 -07:00
dominic
9fd97a8d63 Keep line spacing consistent. Don't do anything if authenticator not defined. 2016-04-14 15:39:26 +02:00
Min RK
2261a0e21d revisions to docker notes
- link to Docker docs on volumes
- name container `jupyterhub`
- wording
2016-04-14 15:35:58 +02:00
dominic
a7a1c32a03 Add post handler for form based auth 2016-04-14 09:32:42 +02:00
Matthias Bussonnier
dfd01bbf5f Merge pull request #503 from minrk/disable-npm-progress
disable npm progress when installing
2016-04-08 17:36:02 -07:00
Min RK
b11a5be781 disable npm progress when installing
apparently faster, but should also fix unicode errors
2016-04-08 16:35:23 -07:00
Kyle Kelley
8b6950055b Merge pull request #501 from minrk/set-login-at-root-redirect
set login cookie when redirecting
2016-04-06 12:35:50 -05:00
Min RK
e8a298be00 set login cookie when redirecting
should avoid one possible redirect-loop case when the single-user cookie is invalid, but the Hub cookie is valid.
2016-04-06 10:18:23 -07:00
Min RK
69f24acac2 Merge pull request #499 from yuvipanda/statsd
Emit metrics via statsd
2016-04-05 09:23:20 -07:00
YuviPanda
9ffebd0c5e Send metrics about various redirects from User spawning 2016-04-01 14:05:02 -07:00
YuviPanda
2dd3d3c448 Send timing info about spawner success / failure 2016-04-01 10:20:37 -07:00
YuviPanda
4644e7019e Send metrics about running and active users
Uses the standard user last-updated activity callback
2016-04-01 10:20:37 -07:00
YuviPanda
5a15d7a219 Actually start the timer 2016-04-01 10:20:37 -07:00
YuviPanda
788129da12 Send metrics for login and logout actions 2016-04-01 10:20:37 -07:00
YuviPanda
cac5175c9b Send CSP metrics to statsd 2016-04-01 10:20:37 -07:00
YuviPanda
80556360ac Add statsd to the base request handler 2016-04-01 10:20:37 -07:00
YuviPanda
3dca0df55f Add statsd to the base JupyterHub app
Not actually emitting any metrics yet
2016-04-01 10:20:37 -07:00
Min RK
62a5e9dbce Merge pull request #497 from yuvipanda/env-callable
Allow environment config values to be callable
2016-04-01 09:57:52 -07:00
YuviPanda
45fcdc75c0 Add docs about callables in environment configurable 2016-03-31 23:44:08 -07:00
Min RK
f1bdf6247a Merge pull request #500 from yuvipanda/fix-url-encoding-4
Use User.url instead of constructing it manually
2016-03-31 22:15:58 -07:00
YuviPanda
80932a51f4 Use User.url instead of constructing it manually
This fixes issues with URL encoding when redirecting users to
their own notebook instances
2016-03-31 17:28:33 -07:00
Min RK
c8774c44d4 Merge pull request #498 from yuvipanda/statsd-configurable
Mark statsd_prefix as configurable as well
2016-03-31 15:08:10 -07:00
YuviPanda
bf2629450c Mark statsd_prefix as configurable as well 2016-03-31 13:46:37 -07:00
YuviPanda
705ff78715 Allow environment config values to be callable
This allows deployments to configure environment variables
that need to be different for each user / container (such as
credentials for various services, etc).
2016-03-31 11:52:53 -07:00
Min RK
a13119a79f Merge pull request #496 from yuvipanda/statsd
Allow specifying statsd host/port/prefix info
2016-03-31 11:18:09 -07:00
YuviPanda
6932719e4e Convert port into string (so that .join works) 2016-03-31 10:32:49 -07:00
YuviPanda
68a750fc7a Use 'Integer' rather than 'Int' for config traitlet 2016-03-30 19:04:57 -07:00
YuviPanda
c6d05d0840 Allow specifying statsd host/port/prefix info
Currently only passes it through to CHP. This is needed
for the cases when JupyterHub spawns and maintains CHP.
2016-03-30 18:59:32 -07:00
Carol Willing
2bbfd75f4d Merge pull request #495 from Carreau/add-import
Import warnings, used on line 215, not imported.
2016-03-29 15:52:39 -07:00
Matthias Bussonnier
26f0e8ea5c Import warnings, used on line 215, not imported. 2016-03-29 15:36:22 -07:00
Carol Willing
552e5caa11 Merge pull request #494 from jupyter/Codecov-badge
Add codecov Badge.
2016-03-29 15:23:54 -07:00
Matthias Bussonnier
7753187e51 Add codecov Badge. 2016-03-29 15:04:17 -07:00
Carol Willing
bddadc7522 Merge pull request #493 from minrk/traitlets-4-1-again
use traitlets 4.1 APIs
2016-03-29 14:51:51 -07:00
Min RK
195eea55f3 log.warning 2016-03-29 09:22:32 -07:00
Min RK
7a2794af7c use traitlets-4.1 observe/default decorators 2016-03-27 10:41:36 -07:00
Min RK
fa48620076 use traitlets-4.1 .tag(config=True) API 2016-03-27 10:29:36 -07:00
Min RK
e4cfe01c4a require traitlets 4.1 2016-03-27 10:21:41 -07:00
Carol Willing
b35e506220 Merge pull request #479 from minrk/config-env
Make Spawner.env configurable
2016-03-24 07:59:44 -07:00
Carol Willing
dd3ed1bf75 Merge pull request #490 from minrk/disable-pam-session
Allow disabling PAM sessions
2016-03-24 07:57:00 -07:00
Min RK
40368b8f55 Allow disabling PAM sessions
it's often buggy and rarely necessary,
so allow it to be disabled when it's causing problems.

It's still on by default for backward-compatibility,
though maybe it shouldn't be.
2016-03-23 23:24:54 +01:00
Min RK
d0f1520642 Add Spawner.environment configurable
instead of making existing Spawner.env configurable

Spawner.env is deprecated
2016-03-22 13:48:26 +01:00
Carol Willing
28c8265c3d Merge pull request #487 from minrk/fix-failed-login-for-none
Fix 'failed login for None' message
2016-03-21 04:51:20 -07:00
Min RK
1d1a8ba78b Fix 'failed login for None' message
on failed login, get username from form data, not the guaranteed-None return value of authenticate
2016-03-21 12:01:31 +01:00
Min RK
a1c764593c travis_retry tests
to hide intermittent failures and enable laziness
2016-03-15 10:37:03 +01:00
Min RK
06902afa2d Merge pull request #481 from willingc/issue-417
Add additional documentation on --no-SSL option
2016-03-15 10:12:18 +01:00
Min RK
6d46f10cfa Merge pull request #480 from willingc/issue-458
Update the configuration section of docs
2016-03-15 10:11:54 +01:00
Carol Willing
b71f34eb3c Fix transposed version number 2016-03-14 16:57:12 -07:00
Carol Willing
11df935f34 Fix awkward wording 2016-03-14 16:54:04 -07:00
Carol Willing
19b6468889 Add no-SSL option to docs 2016-03-14 16:48:49 -07:00
Carol Willing
d2dddd6c82 Update the configuration section of docs, add example 2016-03-14 16:21:24 -07:00
Min RK
5d140fb889 Merge pull request #478 from willingc/readme-docker
Update README re: docker image contents
2016-03-11 22:24:27 +01:00
Matthias Bussonnier
2bf8683905 Merge pull request #477 from willingc/doc-sphinx
Use latest version of Sphinx to fix RTD "Edit on GitHub"
2016-03-11 10:46:52 -08:00
Carol Willing
2dba7f4f61 Update README re: docker image contents 2016-03-11 10:05:13 -08:00
Carol Willing
2820ba319f Update sphinx version for md on rtd 2016-03-11 07:55:49 -08:00
Min RK
be7a627c11 Make Spawner.env configurable
moves `_env_default` logic to `get_env`,
so that `Spawner.env` can be safely configurable
2016-03-11 12:34:49 +01:00
Matthias Bussonnier
2cb1618937 Merge pull request #467 from minrk/add-user-more-often
Call `add_user` more often
2016-03-10 14:45:56 -08:00
Min RK
c9e0c5fe04 Merge pull request #474 from minrk/user.url
allow user.url to be accessed without the server running
2016-03-10 10:28:11 +01:00
Min RK
922956def2 allow user.url to be accessed without the server running
Reduces the number of different ways we need to build the same URLs.
2016-03-09 09:30:50 +01:00
Min RK
c6c699ea89 Merge pull request #472 from yuvipanda/fix-user-encoding
Use encoded URL when redirecting user notebooks
2016-03-09 09:20:43 +01:00
YuviPanda
e0219d0363 Use encoded URL when redirecting user notebooks
Otherwise it breaks for usernames that have url unsafe
characters.
2016-03-08 18:41:35 -08:00
Matthias Bussonnier
f7dab558e4 Merge pull request #468 from minrk/clean-the-pool
set default pool_recycle if using mysql
2016-03-08 11:28:59 -08:00
Min RK
74e558dad2 set default pool_recycle if using mysql 2016-03-08 10:58:18 +01:00
Min RK
96269fac0f Call add_user more often
- Ensures add_user is called as part of startup *for all users*.
  This was previously only true for users not already in the db.
- Normalize usernames in whitelist and admin sets
- Call add_user on new users logged in when there is no whitelist.
2016-03-08 10:49:02 +01:00
Min RK
a0501c6ee4 set patch version to 0 on release 2016-03-08 09:55:44 +01:00
Min RK
ea2ed75ab2 back to dev 2016-03-08 09:00:41 +01:00
Min RK
fc6435825c release 0.5.0 2016-03-08 08:57:33 +01:00
Min RK
b3ab48eb68 Merge pull request #463 from minrk/moar-coverage
Increase some test coverage
2016-03-07 17:13:20 +01:00
Carol Willing
a212151c09 Merge pull request #461 from minrk/0.5
0.5 changelog
2016-03-07 08:07:19 -08:00
Min RK
67ccfc7eb7 increase some test coverage 2016-03-07 16:13:57 +01:00
Min RK
9af103c673 fixes for handling failed chdir in spawners 2016-03-07 15:12:30 +01:00
Min RK
82643adfb6 stop_pending also counts as not running 2016-03-07 14:27:40 +01:00
Min RK
74df94d15a 0.5 changelog 2016-03-07 13:54:40 +01:00
Min RK
da1b9bdd80 Merge pull request #460 from yuvipanda/mysql-fix
Add lengths to all Unicode() columns
2016-03-07 10:36:17 +01:00
Min RK
18675ef6df Merge pull request #453 from minrk/timeout-in-is-up
use the same connection check everywhere
2016-03-07 10:35:12 +01:00
YuviPanda
bf9dea5522 Add lengths to all Unicode() ones
- Otherwise does not work with MySQL
- Change JSONDict to be TEXT (Unbounded) rather than VARCHAR.
  This makes most sense, since you can't index these anyway.
- The 'ip' field in Server is set to 255, since that is the
  max allowed length of DNS entries.
- Most of the rest of the Unicodes have approximately high
  values that most people should not mostly run into
  (famous last words).
2016-03-06 18:26:25 -08:00
Min RK
62e30c1d79 Merge pull request #457 from shreddd/default_url
Enable default_url to pass in to notebook server
2016-03-06 10:33:52 +01:00
shreddd
1316196542 Update spawner.py
type
2016-03-05 12:24:39 -08:00
Shreyas Cholia
1a377bd03a comment on default_url being used with notebook_dir 2016-03-05 12:16:10 -08:00
Shreyas Cholia
66a99ce881 Add support for default_url 2016-03-05 12:05:58 -08:00
shreddd
481debcb80 Merge pull request #1 from jupyter/master
sync master
2016-03-05 12:04:09 -08:00
Carol Willing
03c25b5cac Merge pull request #452 from minrk/redundant-use-subdomain
remove redundant use_subdomains
2016-03-05 11:52:43 -08:00
Carol Willing
26c060d2c5 Merge pull request #456 from willingc/readme-clarify
Add minor clarification to README
2016-03-05 10:57:13 -08:00
Carol Willing
7ff42f9b55 Add @betatim's suggested wording 2016-03-05 10:43:45 -08:00
Carol Willing
a35d8a6262 Add minor clarification 2016-03-05 10:14:44 -08:00
Carol Willing
8f39e1f8f9 Merge pull request #455 from betatim/readme-fix
README uses two different names for docker container
2016-03-05 10:08:34 -08:00
Tim Head
ff19b799c4 container -> cont for consistency 2016-03-05 09:19:15 +01:00
Kyle Kelley
e547949aee Merge pull request #433 from minrk/disable-user-config
allow disabling user configuration of single-user servers
2016-03-04 09:57:45 -06:00
Min RK
31be00b49f failure to connect may be a timeout 2016-03-04 16:28:57 +01:00
Min RK
4533d96002 use the same connection check everywhere
avoids inconsistencies in error handling
2016-03-04 16:28:57 +01:00
Min RK
7f89f1a2a0 expose disable_user_config as Spawner.disable_user_config 2016-03-04 14:41:40 +01:00
Min RK
aed29e1db8 Simplify filter to exclude config in the home directory 2016-03-04 11:43:45 +01:00
Min RK
49bee25820 allow disabling user configuration of single-user servers 2016-03-04 11:43:45 +01:00
Min RK
838c8eb057 Merge pull request #448 from daradib/redirect
Redirect requests to logged in user
2016-03-04 11:15:56 +01:00
Min RK
be5860822d remove redundant use_subdomains
non-empty subdomain_host is enough
2016-03-04 11:11:41 +01:00
Dara Adib
5a10d304c9 Redirect user to login page when not logged in 2016-03-02 16:55:33 -08:00
Dara Adib
fdd3746f54 Add test for user redirect 2016-03-02 16:18:02 -08:00
Dara Adib
4d55a48a79 Redirect requests to logged in user
If a user, alice, requests /user/bob/notebooks/mynotebook.ipynb,
redirect her to /user/alice/notebooks/mynotebook.ipynb.
Currently, such requests get stuck in a redirect loop because
the request will be redirected to login page with a next parameter
that when followed is again redirected.

When notebook_dir is consistent across users, this will allow
users to share notebook URLs. Fixes #424.
2016-03-02 16:15:50 -08:00
Min RK
b2ece48239 reverse arguments in check_routes 2016-03-01 19:42:55 +01:00
Kyle Kelley
6375ba30b7 Merge pull request #445 from minrk/check-routes-pending
Don't add users with spawn_pending to the proxy
2016-03-01 09:19:42 -06:00
Min RK
f565f8ac53 Don't add users with spawn_pending to the proxy
check_routes checks for missing routes for running users.
This is meant for when the proxy has been relaunched outside the Hub.

If spawners are slow to start, it's possible for check_routes to fire in the middle of spawning,
triggering addition of the user's server (which has no defined location yet) to the proxy before it's up.
If the spawning fails, the route will remain indefinitely (because it never should have been added in the first place), and the user will see 503 until their server is launched manually again.

Checking `spawn_pending` in user.running prevents this.
2016-03-01 15:18:51 +01:00
Kyle Kelley
5ec05822f1 Merge pull request #436 from minrk/subdomains
allow running single-user servers on subdomains
2016-02-28 09:49:45 -06:00
Min RK
335b47d7c1 include protocol in subdomain_host
makes everything easier, and tests are passing with and without subdomains (yay!)
2016-02-28 11:12:41 +01:00
Min RK
f922561003 Tests are passing with subdomains 2016-02-26 17:32:55 +01:00
Min RK
79df83f0d3 Allow getting users by name 2016-02-26 17:32:55 +01:00
Min RK
29416463ff proxy needs user dict, which has proxy path
this won't be needed if/when I make a schema change, where domain is included in the Server table.
2016-02-26 17:32:55 +01:00
Min RK
dd2e1ef758 turn off subdomains by default 2016-02-26 17:32:55 +01:00
Min RK
a9b8542ec7 pass hub's host to single-user servers via hub_host 2016-02-26 17:32:54 +01:00
Min RK
a4ae2ec2d8 consolidate cookie setting in _set_user_cookie 2016-02-26 17:32:54 +01:00
Min RK
b54bfad8c2 [WIP]: allow running single-user servers on subdomains
relies on CHP's host-based routing (a feature I didn't add!)

requires wildcard DNS and wildcard SSL for a proper setup

still lots to workout and cleanup in terms of cookies and where to use host, domain, path, but it works locally.
2016-02-26 17:32:54 +01:00
Min RK
724bf7c4ce Merge pull request #441 from jupyter/revert-440-master
Revert "Do not consider `@` character url-safe"
2016-02-26 09:06:46 +01:00
Kyle Kelley
fccc954fb4 Merge pull request #442 from minrk/never-poll-before-start-is-done
avoid calling Spawner.poll during Spawner.start
2016-02-25 08:20:35 -06:00
Kyle Kelley
74385a6906 Merge pull request #443 from minrk/catch-options-from-form
catch exceptions in options_from_form
2016-02-25 08:19:16 -06:00
Min RK
dd66fe63c0 catch exceptions in options_from_form
Allows form validation to be implemented in options_from_form, as well as start.
2016-02-25 12:02:23 +01:00
Min RK
e74934cb17 avoid calling Spawner.poll during Spawner.start
moves `spawn_pending` flag to only around start, not the HTTP wait.

Some Spawners may not know how to poll until start has finished (DockerSpawner).
Let's not require that they do.
2016-02-25 10:13:51 +01:00
Min RK
450281a90a Revert "Do not consider @ character url-safe" 2016-02-25 09:04:25 +01:00
Kyle Kelley
6e7fc0574e Merge pull request #440 from ResearchComputing/master
Do not consider `@` character url-safe
2016-02-24 23:58:45 -06:00
Jonathon Anderson
fc49aac02b Do not consider `@' character url-safe
Usernames that have an `@'-separated domain component
break JupyterHub when the server expects to see query
strings that contain an `@', when browsers and other
clients send `%40'.
2016-02-24 16:48:23 -07:00
Kyle Kelley
097d883905 Merge pull request #435 from minrk/debug-no-server
add debug logging for adding users with no running server
2016-02-20 06:04:12 -08:00
Min RK
cb55118f70 add debug logging for adding users with no running server
in check_routes, it has been reported that users without a running server are attempted to be added.

So something is wrong, either in sqlalchemy or my understanding of what it does (likely the latter),
because a filter for users with a non-None server is returning at least one result whose server is None.
2016-02-20 14:22:50 +01:00
Carol Willing
2a3c87945e Merge pull request #434 from rgbkrk/ssl
Don't let the default include `--no-ssl`.
2016-02-18 16:48:06 -08:00
Kyle Kelley
2b2aacedc6 Don't let the default include --no-ssl. 2016-02-18 16:27:53 -08:00
Kyle Kelley
8ebec52827 Merge pull request #431 from ObiWahn/master
Update README.md
2016-02-18 16:25:56 -08:00
Jan Christoph Uhde
1642cc30c8 fix: run vs exec and split sentence 2016-02-19 00:13:02 +01:00
Kyle Kelley
1645d8f0c0 Merge pull request #432 from minrk/no-port-retries
disable port_retries in single-user server
2016-02-18 06:40:51 -08:00
Min RK
8d390819a1 disable port_retries in single-user server
since Spawners won't notice that the server has started somewhere other than where it was asked to
2016-02-18 09:03:45 +01:00
Jan Christoph Uhde
c7dd18bb03 Update README.md 2016-02-16 22:58:27 +01:00
Min RK
84b7de4d21 set x bit on jupyterhub-singleuser 2016-02-15 21:50:55 +01:00
Carol Willing
161df53143 Merge pull request #426 from takluyver/docs-intro
Add overview to landing page
2016-02-13 11:12:35 -08:00
Thomas Kluyver
1cfd6cf12e Fix grammaros 2016-02-13 18:18:23 +00:00
Thomas Kluyver
d40dcc35fb Reword intro 2016-02-13 16:44:41 +00:00
Thomas Kluyver
a570e95602 Add my overview to intro
Closes gh-425
2016-02-13 15:29:08 +00:00
Thomas Kluyver
e4e43521ee Close code block 2016-02-13 15:28:37 +00:00
Min RK
1b2c21a99c Merge pull request #423 from minrk/custom-logo
allow overriding logo
2016-02-11 15:03:02 +01:00
Min RK
e28eda6386 exercise some static file handlers in tests 2016-02-09 15:38:44 +01:00
Min RK
39c171cce7 allow overriding logo
by specifying JupyterHub.logo_file

also ensures single-user server always has the same logo image as the Hub
2016-02-09 15:38:34 +01:00
Min RK
c81cefd768 Merge pull request #372 from minrk/require-notebook-4
drop support for single-user server from IPython 3.x
2016-02-09 14:42:12 +01:00
Min RK
325f137265 Merge pull request #421 from Fokko/add-docker-label
Added label to dockerfile for referencing
2016-02-09 14:41:48 +01:00
Fokko Driesprong
1ae795df18 Changed domain of the label to .org 2016-02-09 14:16:50 +01:00
Fokko Driesprong
2aacd5e28b Added label to dockerfile for referencing 2016-02-08 17:16:20 +01:00
Kyle Kelley
6e1425e2c0 Merge pull request #417 from minrk/require-confirm-insecure
require confirmation for JupyterHub to run without SSL
2016-02-05 19:27:37 -06:00
Carol Willing
010db6ce72 Merge pull request #416 from willingc/doc-warn
Add more prominent message for https
2016-02-04 14:21:52 -08:00
Min RK
ce8d782220 no-ssl in changelog 2016-02-04 23:00:54 +01:00
Min RK
90c2b23fc0 require confirmation for JupyterHub to run without SSL
ensures folks deploying JupyterHub on HTTP have been told what's up.
2016-02-04 23:00:54 +01:00
Carol Willing
32685aeac1 Add more prominent message for https 2016-02-04 13:42:13 -08:00
Min RK
01c5608104 update version requirements in README 2016-02-04 22:41:18 +01:00
Min RK
a35f6298f0 drop support for single-user server from IPython 3.x 2016-02-04 22:40:44 +01:00
Min RK
8955d6aed4 Merge pull request #411 from minrk/one-two-seven
use 127.0.0.1 instead of localhost
2016-02-04 20:37:09 +01:00
Min RK
cafbf8b990 back to dev 2016-02-03 21:05:48 +01:00
Min RK
f626d2f6e5 use 127.0.0.1 instead of localhost
localhost can cause some issues on badly behaved or misconfigured systems,
and 127 seems simpler.
2016-02-03 10:30:09 +01:00
46 changed files with 1856 additions and 722 deletions

View File

@@ -12,6 +12,10 @@ before_install:
install:
- pip install -f travis-wheels/wheelhouse -r dev-requirements.txt .
script:
- py.test --cov jupyterhub jupyterhub/tests -v
- travis_retry py.test --cov jupyterhub jupyterhub/tests -v
after_success:
- codecov
matrix:
include:
- python: 3.5
env: JUPYTERHUB_TEST_SUBDOMAIN_HOST=http://127.0.0.1.xip.io:8000

View File

@@ -1,12 +1,27 @@
# A base docker image that includes juptyerhub and IPython master
# An incomplete base Docker image for running JupyterHub
#
# Build your own derivative images starting with
# Add your configuration to create a complete derivative Docker image.
#
# FROM jupyter/jupyterhub:latest
# Include your configuration settings by starting with one of two options:
#
# Option 1:
#
# FROM jupyterhub/jupyterhub:latest
#
# And put your configuration file jupyterhub_config.py in /srv/jupyterhub/jupyterhub_config.py.
#
# Option 2:
#
# Or you can create your jupyterhub config and database on the host machine, and mount it with:
#
# docker run -v $PWD:/srv/jupyterhub -t jupyterhub/jupyterhub
#
# NOTE
# If you base on jupyterhub/jupyterhub-onbuild
# your jupyterhub_config.py will be added automatically
# from your docker directory.
FROM debian:jessie
MAINTAINER Jupyter Project <jupyter@googlegroups.com>
# install nodejs, utf8 locale
@@ -22,7 +37,8 @@ RUN apt-get -y update && \
ENV LANG C.UTF-8
# install Python with conda
RUN wget -q https://repo.continuum.io/miniconda/Miniconda3-3.9.1-Linux-x86_64.sh -O /tmp/miniconda.sh && \
RUN wget -q https://repo.continuum.io/miniconda/Miniconda3-4.0.5-Linux-x86_64.sh -O /tmp/miniconda.sh && \
echo 'a7bcd0425d8b6688753946b59681572f63c2241aed77bf0ec6de4c5edc5ceeac */tmp/miniconda.sh' | shasum -a 256 -c - && \
bash /tmp/miniconda.sh -f -b -p /opt/conda && \
/opt/conda/bin/conda install --yes python=3.5 sqlalchemy tornado jinja2 traitlets requests pip && \
/opt/conda/bin/pip install --upgrade pip && \
@@ -32,19 +48,16 @@ ENV PATH=/opt/conda/bin:$PATH
# install js dependencies
RUN npm install -g configurable-http-proxy && rm -rf ~/.npm
WORKDIR /srv/
ADD . /srv/jupyterhub
WORKDIR /srv/jupyterhub/
ADD . /src/jupyterhub
WORKDIR /src/jupyterhub
RUN python setup.py js && pip install . && \
rm -rf node_modules ~/.cache ~/.npm
rm -rf $PWD ~/.cache ~/.npm
RUN mkdir -p /srv/jupyterhub/
WORKDIR /srv/jupyterhub/
# Derivative containers should add jupyterhub config,
# which will be used when starting the application.
EXPOSE 8000
ONBUILD ADD jupyterhub_config.py /srv/jupyterhub/jupyterhub_config.py
CMD ["jupyterhub", "-f", "/srv/jupyterhub/jupyterhub_config.py"]
LABEL org.jupyter.service="jupyterhub"
CMD ["jupyterhub"]

View File

@@ -4,7 +4,9 @@ include setupegg.py
include bower.json
include package.json
include *requirements.txt
include Dockerfile
graft onbuild
graft jupyterhub
graft scripts
graft share

View File

@@ -3,9 +3,11 @@
Questions, comments? Visit our Google Group:
[![Google Group](https://img.shields.io/badge/-Google%20Group-lightgrey.svg)](https://groups.google.com/forum/#!forum/jupyter)
[![Build Status](https://travis-ci.org/jupyter/jupyterhub.svg?branch=master)](https://travis-ci.org/jupyter/jupyterhub)
[![Circle CI](https://circleci.com/gh/jupyter/jupyterhub.svg?style=shield&circle-token=b5b65862eb2617b9a8d39e79340b0a6b816da8cc)](https://circleci.com/gh/jupyter/jupyterhub)
[![Build Status](https://travis-ci.org/jupyterhub/jupyterhub.svg?branch=master)](https://travis-ci.org/jupyterhub/jupyterhub)
[![Circle CI](https://circleci.com/gh/jupyterhub/jupyterhub.svg?style=shield&circle-token=b5b65862eb2617b9a8d39e79340b0a6b816da8cc)](https://circleci.com/gh/jupyterhub/jupyterhub)
[![Documentation Status](https://readthedocs.org/projects/jupyterhub/badge/?version=latest)](http://jupyterhub.readthedocs.org/en/latest/?badge=latest)
[![codecov.io](https://codecov.io/github/jupyter/jupyterhub/coverage.svg?branch=master)](https://codecov.io/github/jupyter/jupyterhub?branch=master)
JupyterHub, a multi-user server, manages and proxies multiple instances of the single-user <del>IPython</del> Jupyter notebook server.
@@ -25,7 +27,7 @@ Basic principles:
## Dependencies
JupyterHub requires [IPython](https://ipython.org/install.html) >= 3.0 (current master) and [Python](https://www.python.org/downloads/) >= 3.3.
JupyterHub itself requires [Python](https://www.python.org/downloads/) 3.3. To run the single-user servers (which may be on the same system as the Hub or not), [Jupyter Notebook](https://jupyter.readthedocs.org/en/latest/install.html) ≥ 4 is required.
Install [nodejs/npm](https://www.npmjs.com/), which is available from your
package manager. For example, install on Linux (Debian/Ubuntu) using:
@@ -50,14 +52,15 @@ Notes on the `pip` command used in the installation directions below:
## Installation
JupyterHub can be installed with pip:
JupyterHub can be installed with pip, and the proxy with npm:
npm install -g configurable-http-proxy
pip3 install jupyterhub
If you plan to run notebook servers locally, you may also need to install the
Jupyter ~~IPython~~ notebook:
pip3 install notebook
pip3 install --upgrade notebook
### Development install
@@ -116,6 +119,27 @@ Some examples, meant as illustration and testing of this concept:
- Using GitHub OAuth instead of PAM with [OAuthenticator](https://github.com/jupyter/oauthenticator)
- Spawning single-user servers with Docker, using the [DockerSpawner](https://github.com/jupyter/dockerspawner)
### Docker
There is a ready to go [docker image for JupyterHub](https://hub.docker.com/r/jupyter/jupyterhub/).
[Note: This `jupyter/jupyterhub` docker image is only an image for running the Hub service itself.
It does not require the other Jupyter components, which are needed by the single-user servers.
To run the single-user servers, which may be on the same system as the Hub or not, installation of Jupyter Notebook ≥ 4 is required.]
The JupyterHub docker image can be started with the following command:
docker run -d --name jupyterhub jupyter/jupyterhub jupyterhub
This command will create a container named `jupyterhub` that you can stop and resume with `docker stop/start`.
It will be listening on all interfaces at port 8000, so this is perfect to test JupyterHub on your desktop or laptop.
If you want to run docker on a computer that has a public IP then you should (as in MUST) secure it with ssl by
adding ssl options to your docker configuration or using a ssl enabled proxy.
[Mounting volumes](https://docs.docker.com/engine/userguide/containers/dockervolumes/) will
allow you to store data outside the docker image (host system) so it will be persistent, even when you start
a new image. The command `docker exec -it jupyterhub bash` will spawn a root shell in your docker
container. You can use it to create system users in the container. These accounts will be used for authentication
in jupyterhub's default configuration. In order to run without SSL (for testing purposes only), you'll need to set `--no-ssl` explicitly.
# Getting help
We encourage you to ask questions on the mailing list:

View File

@@ -8,4 +8,12 @@ dependencies:
test:
override:
- docker build -t jupyter/jupyterhub .
- docker build -t jupyterhub/jupyterhub .
- docker build -t jupyterhub/jupyterhub-onbuild:${CIRCLE_TAG:-latest} onbuild
deployment:
hub:
branch: master
commands:
- docker login -u $DOCKER_USER -p $DOCKER_PASS -e unused@example.com
- docker push jupyterhub/jupyterhub-onbuild:${CIRCLE_TAG:-latest}

View File

@@ -1,3 +1,3 @@
-r ../requirements.txt
sphinx
sphinx>=1.3.6
recommonmark==0.4.0

View File

@@ -77,6 +77,7 @@ For simple mappings, a configurable dict `Authenticator.username_map` is used to
c.Authenticator.username_map = {
'service-name': 'localname'
}
```
### Validating usernames

View File

@@ -2,6 +2,32 @@
See `git log` for a more detailed summary.
## 0.6
- JupyterHub has moved to a new `jupyterhub` namespace on GitHub and Docker. What was `juptyer/jupyterhub` is now `jupyterhub/jupyterhub`, etc.
- `jupyterhub/jupyterhub` image on DockerHub no longer loads the jupyterhub_config.py in an ONBUILD step. A new `jupyterhub/jupyterhub-onbuild` image does this
- Add statsd support, via `c.JupyterHub.statsd_{host,port,prefix}`
- Update to traitlets 4.1 `@default`, `@observe` APIs for traits
- Allow disabling PAM sessions via `c.PAMAuthenticator.open_sessions = False`. This may be needed on SELinux-enabled systems, where our PAM session logic often does not work properly
- Add `Spawner.environment` configurable, for defining extra environment variables to load for single-user servers
- JupyterHub API tokens can be pregenerated and loaded via `JupyterHub.api_tokens`, a dict of `token: username`.
- JupyterHub API tokens can be requested via the REST API, with a POST request to `/api/authorizations/token`.
This can only be used if the Authenticator has a username and password.
- Various fixes for user URLs and redirects
## 0.5
- Single-user server must be run with Jupyter Notebook ≥ 4.0
- Require `--no-ssl` confirmation to allow the Hub to be run without SSL (e.g. behind SSL termination in nginx)
- Add lengths to text fields for MySQL support
- Add `Spawner.disable_user_config` for preventing user-owned configuration from modifying single-user servers.
- Fixes for MySQL support.
- Add ability to run each user's server on its own subdomain. Requires wildcard DNS and wildcard SSL to be feasible. Enable subdomains by setting `JupyterHub.subdomain_host = 'https://jupyterhub.domain.tld[:port]'`.
- Use `127.0.0.1` for local communication instead of `localhost`, avoiding issues with DNS on some systems.
- Fix race that could add users to proxy prematurely if spawning is slow.
## 0.4
### 0.4.1

View File

@@ -22,6 +22,11 @@ There are three main categories of processes run by the `jupyterhub` command lin
## JupyterHub's default behavior
**IMPORTANT:** In its default configuration, JupyterHub requires SSL encryption (HTTPS) to run.
**You should not run JupyterHub without SSL encryption on a public network.**
See [Security documentation](#Security) for how to configure JupyterHub to use SSL, and in
certain cases, e.g. behind SSL termination in nginx, allowing the hub to run with no SSL
by requiring `--no-ssl` (as of [version 0.5](./changelog.html)).
To start JupyterHub in its default configuration, type the following at the command line:
@@ -44,10 +49,6 @@ or any other public IP or domain pointing to your system.
In their default configuration, the other services, the **Hub** and **Single-User Servers**,
all communicate with each other on localhost only.
**NOTE:** In its default configuration, JupyterHub runs without SSL encryption (HTTPS).
You should not run JupyterHub without SSL encryption on a public network.
See [Security documentation](#Security) for how to configure JupyterHub to use SSL.
By default, starting JupyterHub will write two files to disk in the current working directory:
- `jupyterhub.sqlite` is the sqlite database containing all of the state of the **Hub**.
@@ -65,23 +66,13 @@ The location of these files can be specified via configuration, discussed below.
JupyterHub is configured in two ways:
1. Command-line arguments
2. Configuration files
1. Configuration file
2. Command-line arguments
Type the following for brief information about the command line arguments:
jupyterhub -h
or:
jupyterhub --help-all
for the full command line help.
By default, JupyterHub will look for a configuration file (can be missing)
### Configuration file
By default, JupyterHub will look for a configuration file (which may not be created yet)
named `jupyterhub_config.py` in the current working directory.
You can create an empty configuration file with
You can create an empty configuration file with:
jupyterhub --generate-config
@@ -93,6 +84,23 @@ values. You can load a specific config file with:
See also: [general docs](http://ipython.org/ipython-doc/dev/development/config.html)
on the config system Jupyter uses.
### Command-line arguments
Type the following for brief information about the command-line arguments:
jupyterhub -h
or:
jupyterhub --help-all
for the full command line help.
All configurable options are technically configurable on the command-line,
even if some are really inconvenient to type. Just replace the desired option,
c.Class.trait, with --Class.trait. For example, to configure
c.Spawner.notebook_dir = '~/assignments' from the command-line:
jupyterhub --Spawner.notebook_dir='~/assignments'
## Networking
@@ -148,6 +156,9 @@ c.JupyterHub.hub_port = 54321
## Security
**IMPORTANT:** In its default configuration, JupyterHub requires SSL encryption (HTTPS) to run.
**You should not run JupyterHub without SSL encryption on a public network.**
Security is the most important aspect of configuring Jupyter. There are three main aspects of the
security configuration:
@@ -179,6 +190,10 @@ Some cert files also contain the key, in which case only the cert is needed. It
these files be put in a secure location on your server, where they are not readable by regular
users.
Note: In certain cases, e.g. behind SSL termination in nginx, allowing no SSL
running on the hub may be desired. To run the Hub without SSL, you must opt
in by configuring and confirming the `--no-ssl` option, added as of [version 0.5](./changelog.html).
## Cookie secret
The cookie secret is an encryption key, used to encrypt the browser cookies used for
@@ -190,26 +205,36 @@ as follows:
c.JupyterHub.cookie_secret_file = '/srv/jupyterhub/cookie_secret'
```
The content of this file should be a long random string. An example would be to generate this
file as:
The content of this file should be a long random string encoded in MIME Base64. An example would be to generate this file as:
```bash
openssl rand -hex 1024 > /srv/jupyterhub/cookie_secret
openssl rand -base64 2048 > /srv/jupyterhub/cookie_secret
```
In most deployments of JupyterHub, you should point this to a secure location on the file
system, such as `/srv/jupyterhub/cookie_secret`. If the cookie secret file doesn't exist when
the Hub starts, a new cookie secret is generated and stored in the file. The recommended
permissions for the cookie secret file should be 600 (owner-only rw).
the Hub starts, a new cookie secret is generated and stored in the file. The
file must not be readable by group or other or the server won't start.
The recommended permissions for the cookie secret file are 600 (owner-only rw).
If you would like to avoid the need for files, the value can be loaded in the Hub process from
the `JPY_COOKIE_SECRET` environment variable:
the `JPY_COOKIE_SECRET` environment variable, which is a hex-encoded string. You
can set it this way:
```bash
export JPY_COOKIE_SECRET=`openssl rand -hex 1024`
```
For security reasons, this environment variable should only be visible to the Hub.
If you set it dynamically as above, all users will be logged out each time the
Hub starts.
You can also set the secret in the configuration file itself as a binary string:
```python
c.JupyterHub.cookie_secret = bytes.fromhex('VERY LONG SECRET HEX STRING')
```
## Proxy authentication token

View File

@@ -1,22 +1,34 @@
.. JupyterHub documentation master file, created by
sphinx-quickstart on Mon Jan 4 16:31:09 2016.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
JupyterHub
==========
.. note:: This is the official documentation for JupyterHub. This project is
under active development.
JupyterHub is a server that gives multiple users access to Jupyter notebooks,
running an independent Jupyter notebook server for each user.
JupyterHub is a multi-user server that manages and proxies multiple instances
of the single-user Jupyter notebook server.
To use JupyterHub, you need a Unix server (typically Linux) running
somewhere that is accessible to your team on the network. The JupyterHub server
can be on an internal network at your organisation, or it can run on the public
internet (in which case, take care with `security <getting-started.html#security>`__).
Users access JupyterHub in a web browser, by going to the IP address or
domain name of the server.
Three actors:
Different :doc:`authenticators <authenticators>` control access
to JupyterHub. The default one (pam) uses the user accounts on the server where
JupyterHub is running. If you use this, you will need to create a user account
on the system for each user on your team. Using other authenticators, you can
allow users to sign in with e.g. a Github account, or with any single-sign-on
system your organisation has.
* multi-user Hub (tornado process)
* `configurable http proxy <https://github.com/jupyter/configurable-http-proxy>`_ (node-http-proxy)
* multiple single-user IPython notebook servers (Python/IPython/tornado)
Next, :doc:`spawners <spawners>` control how JupyterHub starts
the individual notebook server for each user. The default spawner will
start a notebook server on the same machine running under their system username.
The other main option is to start each server in a separate container, often
using Docker.
JupyterHub runs as three separate parts:
* The multi-user Hub (Python & Tornado)
* A `configurable http proxy <https://github.com/jupyter/configurable-http-proxy>`_ (NodeJS)
* Multiple single-user Jupyter notebook servers (Python & Tornado)
Basic principles:
@@ -34,6 +46,7 @@ Contents:
getting-started
howitworks
websecurity
.. toctree::
:maxdepth: 2

View File

@@ -2,10 +2,98 @@
This document is under active development.
## Networking
When troubleshooting, you may see unexpected behaviors or receive an error
message. These two lists provide links to identifying the cause of the
problem and how to resolve it.
If JupyterHub proxy fails to start:
## Behavior problems
- [JupyterHub proxy fails to start](#jupyterhub-proxy-fails-to-start)
## Errors
- [500 error after spawning a single-user server](#500-error-after-spawning-my-single-user-server)
----
## JupyterHub proxy fails to start
If you have tried to start the JupyterHub proxy and it fails to start:
- check if the JupyterHub IP configuration setting is
``c.JupyterHub.ip = '*'``; if it is, try ``c.JupyterHub.ip = ''``
- Try starting with ``jupyterhub --ip=0.0.0.0``
- Try starting with ``jupyterhub --ip=0.0.0.0``
----
## 500 error after spawning my single-user server
You receive a 500 error when accessing the URL `/user/you/...`. This is often
seen when your single-user server cannot check your cookies with the Hub.
There are two likely reasons for this:
1. The single-user server cannot connect to the Hub's API (networking
configuration problems)
2. The single-user server cannot *authenticate* its requests (invalid token)
### Symptoms:
The main symptom is a failure to load *any* page served by the single-user
server, met with a 500 error. This is typically the first page at `/user/you`
after logging in or clicking "Start my server". When a single-user server
receives a request, it makes an API request to the Hub to check if the cookie
corresponds to the right user. This request is logged.
If everything is working, it will look like this:
```
200 GET /hub/api/authorizations/cookie/jupyter-hub-token-name/[secret] (@10.0.1.4) 6.10ms
```
You should see a similar 200 message, as above, in the Hub log when you first
visit your single-user server. If you don't see this message in the log, it
may mean that your single-user server isn't connecting to your Hub.
If you see 403 (forbidden) like this, it's a token problem:
```
403 GET /hub/api/authorizations/cookie/jupyter-hub-token-name/[secret] (@10.0.1.4) 4.14ms
```
Check the logs of the single-user server, which may have more detailed
information on the cause.
### Causes and resolutions:
#### No authorization request
If you make an API request and it is not received by the server, you likely
have a network configuration issue. Often, this happens when the Hub is only
listening on 127.0.0.1 (default) and the single-user servers are not on the
same 'machine' (can be physically remote, or in a docker container or VM). The
fix for this case is to make sure that `c.JupyterHub.hub_ip` is an address
that all single-user servers can connect to, e.g.:
```python
c.JupyterHub.hub_ip = '10.0.0.1'
```
#### 403 GET /hub/api/authorizations/cookie
If you receive a 403 error, the API token for the single-user server is likely
invalid. Commonly, the 403 error is caused by resetting the JupyterHub
database (either removing jupyterhub.sqlite or some other action) while
leaving single-user servers running. This happens most frequently when using
DockerSpawner, because Docker's default behavior is to stop/start containers
which resets the JupyterHub database, rather than destroying and recreating
the container every time. This means that the same API token is used by the
server for its whole life, until the container is rebuilt.
The fix for this Docker case is to remove any Docker containers seeing this
issue (typicaly all containers created before a certain point in time):
docker rm -f jupyter-name
After this, when you start your server via JupyterHub, it will build a
new container. If this was the underlying cause of the issue, you should see
your server again.

View File

@@ -0,0 +1,63 @@
# Web Security in JupyterHub
JupyterHub is designed to be a simple multi-user server for modestly sized groups of semi-trusted users.
While the design reflects serving semi-trusted users,
JupyterHub is not necessarily unsuitable for serving untrusted users.
Using JupyterHub with untrusted users does mean more work and much care is required to secure a Hub against untrusted users,
with extra caution on protecting users from each other as the Hub is serving untrusted users.
One aspect of JupyterHub's design simplicity for semi-trusted users is that the Hub and single-user servers are placed in a single domain, behind a [proxy][configurable-http-proxy].
As a result, if the Hub is serving untrusted users,
many of the web's cross-site protections are not applied between single-user servers and the Hub,
or between single-user servers and each other,
since browsers see the whole thing (proxy, Hub, and single user servers) as a single website.
To protect users from each other, a user must never be able to write arbitrary HTML and serve it to another user on the Hub's domain.
JupyterHub's authentication setup prevents this because only the owner of a given single-user server is allowed to view user-authored pages served by their server.
To protect all users from each other, JupyterHub administrators must ensure that:
* A user does not have permission to modify their single-user server:
- A user may not install new packages in the Python environment that runs their server.
- If the PATH is used to resolve the single-user executable (instead of an absolute path), a user may not create new files in any PATH directory that precedes the directory containing jupyterhub-singleuser.
- A user may not modify environment variables (e.g. PATH, PYTHONPATH) for their single-user server.
* A user may not modify the configuration of the notebook server (the ~/.jupyter or JUPYTER_CONFIG_DIR directory).
If any additional services are run on the same domain as the Hub, the services must never display user-authored HTML that is neither sanitized nor sandboxed (e.g. IFramed) to any user that lacks authentication as the author of a file.
## Mitigations
There are two main configuration options provided by JupyterHub to mitigate these issues:
### Subdomains
JupyterHub 0.5 adds the ability to run single-user servers on their own subdomains,
which means the cross-origin protections between servers has the desired effect,
and user servers and the Hub are protected from each other.
A user's server will be at `username.jupyter.mydomain.com`, etc.
This requires all user subdomains to point to the same address,
which is most easily accomplished with wildcard DNS.
Since this spreads the service across multiple domains, you will need wildcard SSL, as well.
Unfortunately, for many institutional domains, wildcard DNS and SSL are not available,
but if you do plan to serve untrusted users, enabling subdomains is highly encouraged,
as it resolves all of the cross-site issues.
### Disabling user config
If subdomains are not available or not desirable,
0.5 also adds an option `Spawner.disable_use_config`,
which you can set to prevent the user-owned configuration files from being loaded.
This leaves only package installation and PATHs as things the admin must enforce.
For most Spawners, PATH is not something users an influence,
but care should be taken to ensure that the Spawn does *not* evaluate shell configuration files prior to launching the server.
Package isolation is most easily handled by running the single-user server in a virtualenv with disabled system-site-packages.
## Extra notes
It is important to note that the control over the environment only affects the single-user server,
and not the environment(s) in which the user's kernel(s) may run.
Installing additional packages in the kernel environment does not pose additional risk to the web application's security.
[configurable-http-proxy]: https://github.com/jupyterhub/configurable-http-proxy

View File

@@ -1,4 +1,4 @@
FROM jupyter/jupyterhub
FROM jupyter/jupyterhub-onbuild
MAINTAINER Jupyter Project <jupyter@googlegroups.com>

View File

@@ -1,5 +1,5 @@
"""
Example JuptyerHub config allowing users to specify environment variables and notebook-server args
Example JupyterHub config allowing users to specify environment variables and notebook-server args
"""
import shlex

View File

@@ -6,7 +6,7 @@
import json
from urllib.parse import quote
from tornado import web
from tornado import web, gen
from .. import orm
from ..utils import token_authenticated
from .base import APIHandler
@@ -20,13 +20,25 @@ class TokenAPIHandler(APIHandler):
raise web.HTTPError(404)
self.write(json.dumps(self.user_model(self.users[orm_token.user])))
@gen.coroutine
def post(self):
if self.authenticator is not None:
data = self.get_json_body()
username = yield self.authenticator.authenticate(self, data)
if username is None:
raise web.HTTPError(403)
user = self.find_user(username)
api_token = user.new_api_token()
self.write(json.dumps({"Authentication":api_token}))
else:
raise web.HTTPError(404)
class CookieAPIHandler(APIHandler):
@token_authenticated
def get(self, cookie_name, cookie_value=None):
cookie_name = quote(cookie_name, safe='')
if cookie_value is None:
self.log.warn("Cookie values in request body is deprecated, use `/cookie_name/cookie_value`")
self.log.warning("Cookie values in request body is deprecated, use `/cookie_name/cookie_value`")
cookie_value = self.request.body
else:
cookie_value = cookie_value.encode('utf8')
@@ -39,4 +51,5 @@ class CookieAPIHandler(APIHandler):
default_handlers = [
(r"/api/authorizations/cookie/([^/]+)(?:/([^/]+))?", CookieAPIHandler),
(r"/api/authorizations/token/([^/]+)", TokenAPIHandler),
(r"/api/authorizations/token", TokenAPIHandler),
]

View File

@@ -26,25 +26,29 @@ class APIHandler(BaseHandler):
# If no header is provided, assume it comes from a script/curl.
# We are only concerned with cross-site browser stuff here.
if not host:
self.log.warn("Blocking API request with no host")
self.log.warning("Blocking API request with no host")
return False
if not referer:
self.log.warn("Blocking API request with no referer")
self.log.warning("Blocking API request with no referer")
return False
host_path = url_path_join(host, self.hub.server.base_url)
referer_path = referer.split('://', 1)[-1]
if not (referer_path + '/').startswith(host_path):
self.log.warn("Blocking Cross Origin API request. Referer: %s, Host: %s",
self.log.warning("Blocking Cross Origin API request. Referer: %s, Host: %s",
referer, host_path)
return False
return True
def get_current_user_cookie(self):
"""Override get_user_cookie to check Referer header"""
if not self.check_referer():
cookie_user = super().get_current_user_cookie()
# check referer only if there is a cookie user,
# avoiding misleading "Blocking Cross Origin" messages
# when there's no cookie set anyway.
if cookie_user and not self.check_referer():
return None
return super().get_current_user_cookie()
return cookie_user
def get_json_body(self):
"""Return the body of the request as JSON data."""
@@ -86,7 +90,7 @@ class APIHandler(BaseHandler):
model = {
'name': user.name,
'admin': user.admin,
'server': user.server.base_url if user.running else None,
'server': user.url if user.running else None,
'pending': None,
'last_activity': user.last_activity.isoformat(),
}

View File

@@ -28,7 +28,7 @@ class ProxyAPIHandler(APIHandler):
@gen.coroutine
def post(self):
"""POST checks the proxy to ensure"""
yield self.proxy.check_routes()
yield self.proxy.check_routes(self.users)
@admin_only
@@ -59,7 +59,7 @@ class ProxyAPIHandler(APIHandler):
self.proxy.auth_token = model['auth_token']
self.db.commit()
self.log.info("Updated proxy at %s", server.bind_url)
yield self.proxy.check_routes()
yield self.proxy.check_routes(self.users)

View File

@@ -41,7 +41,7 @@ class UserListAPIHandler(APIHandler):
continue
user = self.find_user(name)
if user is not None:
self.log.warn("User %s already exists" % name)
self.log.warning("User %s already exists" % name)
else:
to_create.append(name)
@@ -195,7 +195,7 @@ class UserAdminAccessAPIHandler(APIHandler):
@admin_only
def post(self, name):
current = self.get_current_user()
self.log.warn("Admin user %s has requested access to %s's server",
self.log.warning("Admin user %s has requested access to %s's server",
current.name, name,
)
if not self.settings.get('admin_access', False):

File diff suppressed because it is too large Load Diff

View File

@@ -15,7 +15,7 @@ from tornado import gen
import pamela
from traitlets.config import LoggingConfigurable
from traitlets import Bool, Set, Unicode, Dict, Any
from traitlets import Bool, Set, Unicode, Dict, Any, default, observe
from .handlers.login import LoginHandler
from .utils import url_path_join
@@ -29,19 +29,19 @@ class Authenticator(LoggingConfigurable):
"""
db = Any()
admin_users = Set(config=True,
admin_users = Set(
help="""set of usernames of admin users
If unspecified, only the user that launches the server will be admin.
"""
)
whitelist = Set(config=True,
).tag(config=True)
whitelist = Set(
help="""Username whitelist.
Use this to restrict which users can login.
If empty, allow any user to attempt login.
"""
)
).tag(config=True)
custom_html = Unicode('',
help="""HTML login form for custom handlers.
Override in form-based custom authenticators
@@ -55,16 +55,17 @@ class Authenticator(LoggingConfigurable):
"""
)
username_pattern = Unicode(config=True,
username_pattern = Unicode(
help="""Regular expression pattern for validating usernames.
If not defined: allow any username.
"""
)
def _username_pattern_changed(self, name, old, new):
if not new:
).tag(config=True)
@observe('username_pattern')
def _username_pattern_changed(self, change):
if not change['new']:
self.username_regex = None
self.username_regex = re.compile(new)
self.username_regex = re.compile(change['new'])
username_regex = Any()
@@ -77,14 +78,14 @@ class Authenticator(LoggingConfigurable):
return True
return bool(self.username_regex.match(username))
username_map = Dict(config=True,
username_map = Dict(
help="""Dictionary mapping authenticator usernames to JupyterHub users.
Can be used to map OAuth service names to local users, for instance.
Used in normalize_username.
"""
)
).tag(config=True)
def normalize_username(self, username):
"""Normalize a username.
@@ -246,12 +247,12 @@ class LocalAuthenticator(Authenticator):
Checks for local users, and can attempt to create them if they exist.
"""
create_system_users = Bool(False, config=True,
create_system_users = Bool(False,
help="""If a user is added that doesn't exist on the system,
should I try to create the system user?
"""
)
add_user_cmd = Command(config=True,
).tag(config=True)
add_user_cmd = Command(
help="""The command to use for creating users as a list of strings.
For each element in the list, the string USERNAME will be replaced with
@@ -271,7 +272,9 @@ class LocalAuthenticator(Authenticator):
when the user 'river' is created.
"""
)
).tag(config=True)
@default('add_user_cmd')
def _add_user_cmd_default(self):
if sys.platform == 'darwin':
raise ValueError("I don't know how to create users on OS X")
@@ -283,13 +286,12 @@ class LocalAuthenticator(Authenticator):
return ['adduser', '-q', '--gecos', '""', '--disabled-password']
group_whitelist = Set(
config=True,
help="Automatically whitelist anyone in this group.",
)
def _group_whitelist_changed(self, name, old, new):
).tag(config=True)
@observe('group_whitelist')
def _group_whitelist_changed(self, change):
if self.whitelist:
self.log.warn(
self.log.warning(
"Ignoring username whitelist because group whitelist supplied!"
)
@@ -351,12 +353,24 @@ class LocalAuthenticator(Authenticator):
class PAMAuthenticator(LocalAuthenticator):
"""Authenticate local Linux/UNIX users with PAM"""
encoding = Unicode('utf8', config=True,
encoding = Unicode('utf8',
help="""The encoding to use for PAM"""
)
service = Unicode('login', config=True,
).tag(config=True)
service = Unicode('login',
help="""The PAM service to use for authentication."""
)
).tag(config=True)
open_sessions = Bool(True,
help="""Whether to open PAM sessions when spawners are started.
This may trigger things like mounting shared filsystems,
loading credentials, etc. depending on system configuration,
but it does not always work.
It can be disabled with::
c.PAMAuthenticator.open_sessions = False
"""
).tag(config=True)
@gen.coroutine
def authenticate(self, handler, data):
@@ -369,23 +383,31 @@ class PAMAuthenticator(LocalAuthenticator):
pamela.authenticate(username, data['password'], service=self.service)
except pamela.PAMError as e:
if handler is not None:
self.log.warn("PAM Authentication failed (@%s): %s", handler.request.remote_ip, e)
self.log.warning("PAM Authentication failed (%s@%s): %s", username, handler.request.remote_ip, e)
else:
self.log.warn("PAM Authentication failed: %s", e)
self.log.warning("PAM Authentication failed: %s", e)
else:
return username
def pre_spawn_start(self, user, spawner):
"""Open PAM session for user"""
if not self.open_sessions:
return
try:
pamela.open_session(user.name, service=self.service)
except pamela.PAMError as e:
self.log.warn("Failed to open PAM session for %s: %s", user.name, e)
self.log.warning("Failed to open PAM session for %s: %s", user.name, e)
self.log.warning("Disabling PAM sessions from now on.")
self.open_sessions = False
def post_spawn_stop(self, user, spawner):
"""Close PAM session for user"""
if not self.open_sessions:
return
try:
pamela.close_session(user.name, service=self.service)
except pamela.PAMError as e:
self.log.warn("Failed to close PAM session for %s: %s", user.name, e)
self.log.warning("Failed to close PAM session for %s: %s", user.name, e)
self.log.warning("Disabling PAM sessions from now on.")
self.open_sessions = False

13
jupyterhub/emptyclass.py Normal file
View File

@@ -0,0 +1,13 @@
"""
Simple empty class that returns itself for all functions called on it.
This allows us to call any method of any name on this, and it'll return another
instance of itself that'll allow any method to be called on it.
Primarily used to mock out the statsd client when statsd is not being used
"""
class EmptyClass:
def empty_function(self, *args, **kwargs):
return self
def __getattr__(self, attr):
return self.empty_function

View File

@@ -46,27 +46,39 @@ class BaseHandler(RequestHandler):
@property
def base_url(self):
return self.settings.get('base_url', '/')
@property
def version_hash(self):
return self.settings.get('version_hash', '')
@property
def subdomain_host(self):
return self.settings.get('subdomain_host', '')
@property
def domain(self):
return self.settings['domain']
@property
def db(self):
return self.settings['db']
@property
def users(self):
return self.settings.setdefault('users', {})
@property
def hub(self):
return self.settings['hub']
@property
def proxy(self):
return self.settings['proxy']
@property
def statsd(self):
return self.settings['statsd']
@property
def authenticator(self):
return self.settings.get('authenticator', None)
@@ -75,28 +87,28 @@ class BaseHandler(RequestHandler):
"""Roll back any uncommitted transactions from the handler."""
self.db.rollback()
super().finish(*args, **kwargs)
#---------------------------------------------------------------
# Security policies
#---------------------------------------------------------------
@property
def csp_report_uri(self):
return self.settings.get('csp_report_uri',
url_path_join(self.hub.server.base_url, 'security/csp-report')
)
@property
def content_security_policy(self):
"""The default Content-Security-Policy header
Can be overridden by defining Content-Security-Policy in settings['headers']
"""
return '; '.join([
"frame-ancestors 'self'",
"report-uri " + self.csp_report_uri,
])
def set_default_headers(self):
"""
Set any headers passed as tornado_settings['headers'].
@@ -105,7 +117,7 @@ class BaseHandler(RequestHandler):
"""
headers = self.settings.get('headers', {})
headers.setdefault("Content-Security-Policy", self.content_security_policy)
for header_name, header_content in headers.items():
self.set_header(header_name, header_content)
@@ -116,7 +128,7 @@ class BaseHandler(RequestHandler):
@property
def admin_users(self):
return self.settings.setdefault('admin_users', set())
@property
def cookie_max_age_days(self):
return self.settings.get('cookie_max_age_days', None)
@@ -133,7 +145,7 @@ class BaseHandler(RequestHandler):
return None
else:
return orm_token.user
def _user_for_cookie(self, cookie_name, cookie_value=None):
"""Get the User for a given cookie, if there is one"""
cookie_id = self.get_secure_cookie(
@@ -143,41 +155,41 @@ class BaseHandler(RequestHandler):
)
def clear():
self.clear_cookie(cookie_name, path=self.hub.server.base_url)
if cookie_id is None:
if self.get_cookie(cookie_name):
self.log.warn("Invalid or expired cookie token")
self.log.warning("Invalid or expired cookie token")
clear()
return
cookie_id = cookie_id.decode('utf8', 'replace')
u = self.db.query(orm.User).filter(orm.User.cookie_id==cookie_id).first()
user = self._user_from_orm(u)
if user is None:
self.log.warn("Invalid cookie token")
self.log.warning("Invalid cookie token")
# have cookie, but it's not valid. Clear it and start over.
clear()
return user
def _user_from_orm(self, orm_user):
"""return User wrapper from orm.User object"""
if orm_user is None:
return
return self.users[orm_user]
def get_current_user_cookie(self):
"""get_current_user from a cookie token"""
return self._user_for_cookie(self.hub.server.cookie_name)
def get_current_user(self):
"""get current username"""
user = self.get_current_user_token()
if user is not None:
return user
return self.get_current_user_cookie()
def find_user(self, name):
"""Get a user by name
return None if no such user
"""
orm_user = orm.User.find(db=self.db, name=name)
@@ -192,57 +204,60 @@ class BaseHandler(RequestHandler):
self.db.add(u)
self.db.commit()
user = self._user_from_orm(u)
self.authenticator.add_user(user)
return user
def clear_login_cookie(self, name=None):
if name is None:
user = self.get_current_user()
else:
user = self.find_user(name)
kwargs = {}
if self.subdomain_host:
kwargs['domain'] = self.domain
if user and user.server:
self.clear_cookie(user.server.cookie_name, path=user.server.base_url)
self.clear_cookie(self.hub.server.cookie_name, path=self.hub.server.base_url)
self.clear_cookie(user.server.cookie_name, path=user.server.base_url, **kwargs)
self.clear_cookie(self.hub.server.cookie_name, path=self.hub.server.base_url, **kwargs)
def _set_user_cookie(self, user, server):
# tornado <4.2 have a bug that consider secure==True as soon as
# 'secure' kwarg is passed to set_secure_cookie
if self.request.protocol == 'https':
kwargs = {'secure': True}
else:
kwargs = {}
if self.subdomain_host:
kwargs['domain'] = self.domain
self.log.debug("Setting cookie for %s: %s, %s", user.name, server.cookie_name, kwargs)
self.set_secure_cookie(
server.cookie_name,
user.cookie_id,
path=server.base_url,
**kwargs
)
def set_server_cookie(self, user):
"""set the login cookie for the single-user server"""
# tornado <4.2 have a bug that consider secure==True as soon as
# 'secure' kwarg is passed to set_secure_cookie
if self.request.protocol == 'https':
kwargs = {'secure':True}
else:
kwargs = {}
self.set_secure_cookie(
user.server.cookie_name,
user.cookie_id,
path=user.server.base_url,
**kwargs
)
self._set_user_cookie(user, user.server)
def set_hub_cookie(self, user):
"""set the login cookie for the Hub"""
# tornado <4.2 have a bug that consider secure==True as soon as
# 'secure' kwarg is passed to set_secure_cookie
if self.request.protocol == 'https':
kwargs = {'secure':True}
else:
kwargs = {}
self.set_secure_cookie(
self.hub.server.cookie_name,
user.cookie_id,
path=self.hub.server.base_url,
**kwargs
)
self._set_user_cookie(user, self.hub.server)
def set_login_cookie(self, user):
"""Set login cookies for the Hub and single-user server."""
if self.subdomain_host and not self.request.host.startswith(self.domain):
self.log.warning(
"Possibly setting cookie on wrong domain: %s != %s",
self.request.host, self.domain)
# create and set a new cookie token for the single-user server
if user.server:
self.set_server_cookie(user)
# create and set a new cookie token for the hub
if not self.get_current_user_cookie():
self.set_hub_cookie(user)
@gen.coroutine
def authenticate(self, data):
auth = self.authenticator
@@ -268,7 +283,7 @@ class BaseHandler(RequestHandler):
@property
def spawner_class(self):
return self.settings.get('spawner_class', LocalProcessSpawner)
@gen.coroutine
def spawn_single_user(self, user, options=None):
if user.spawn_pending:
@@ -280,7 +295,7 @@ class BaseHandler(RequestHandler):
@gen.coroutine
def finish_user_spawn(f=None):
"""Finish the user spawn by registering listeners and notifying the proxy.
If the spawner is slow to start, this is passed as an async callback,
otherwise it is called immediately.
"""
@@ -289,38 +304,48 @@ class BaseHandler(RequestHandler):
return
toc = IOLoop.current().time()
self.log.info("User %s server took %.3f seconds to start", user.name, toc-tic)
self.statsd.timing('spawner.success', (toc - tic) * 1000)
yield self.proxy.add_user(user)
user.spawner.add_poll_callback(self.user_stopped, user)
try:
yield gen.with_timeout(timedelta(seconds=self.slow_spawn_timeout), f)
except gen.TimeoutError:
if user.spawn_pending:
# still in Spawner.start, which is taking a long time
# we shouldn't poll while spawn_pending is True
self.log.warning("User %s server is slow to start", user.name)
# schedule finish for when the user finishes spawning
IOLoop.current().add_future(f, finish_user_spawn)
else:
# start has finished, but the server hasn't come up
# check if the server died while we were waiting
status = yield user.spawner.poll()
if status is None:
# hit timeout, but spawn is still pending
self.log.warn("User %s server is slow to start", user.name)
# hit timeout, but server's running. Hope that it'll show up soon enough,
# though it's possible that it started at the wrong URL
self.log.warning("User %s server is slow to become responsive", user.name)
# schedule finish for when the user finishes spawning
IOLoop.current().add_future(f, finish_user_spawn)
else:
toc = IOLoop.current().time()
self.statsd.timing('spawner.failure', (toc - tic) * 1000)
raise web.HTTPError(500, "Spawner failed to start [status=%s]" % status)
else:
raise
else:
yield finish_user_spawn()
@gen.coroutine
def user_stopped(self, user):
"""Callback that fires when the spawner has stopped"""
status = yield user.spawner.poll()
if status is None:
status = 'unknown'
self.log.warn("User %s server stopped, with exit code: %s",
self.log.warning("User %s server stopped, with exit code: %s",
user.name, status,
)
yield self.proxy.delete_user(user)
yield user.stop()
@gen.coroutine
def stop_single_user(self, user):
if user.stop_pending:
@@ -331,7 +356,7 @@ class BaseHandler(RequestHandler):
@gen.coroutine
def finish_stop(f=None):
"""Finish the stop action by noticing that the user is stopped.
If the spawner is slow to stop, this is passed as an async callback,
otherwise it is called immediately.
"""
@@ -340,13 +365,13 @@ class BaseHandler(RequestHandler):
return
toc = IOLoop.current().time()
self.log.info("User %s server took %.3f seconds to stop", user.name, toc-tic)
try:
yield gen.with_timeout(timedelta(seconds=self.slow_stop_timeout), f)
except gen.TimeoutError:
if user.stop_pending:
# hit timeout, but stop is still pending
self.log.warn("User %s server is slow to stop", user.name)
self.log.warning("User %s server is slow to stop", user.name)
# schedule finish for when the server finishes stopping
IOLoop.current().add_future(f, finish_stop)
else:
@@ -426,7 +451,7 @@ class Template404(BaseHandler):
class PrefixRedirectHandler(BaseHandler):
"""Redirect anything outside a prefix inside.
Redirects /foo to /prefix/foo, etc.
"""
def get(self):
@@ -437,22 +462,27 @@ class PrefixRedirectHandler(BaseHandler):
class UserSpawnHandler(BaseHandler):
"""Requests to /user/name handled by the Hub
should result in spawning the single-user server and
being redirected to the original.
"""Redirect requests to /user/name/* handled by the Hub.
If logged in, spawn a single-user server and redirect request.
If a user, alice, requests /user/bob/notebooks/mynotebook.ipynb,
redirect her to /user/alice/notebooks/mynotebook.ipynb, which should
in turn call this function.
"""
@gen.coroutine
def get(self, name):
def get(self, name, user_path):
current_user = self.get_current_user()
if current_user and current_user.name == name:
# logged in, spawn the server
# logged in as correct user, spawn the server
if current_user.spawner:
if current_user.spawn_pending:
# spawn has started, but not finished
self.statsd.incr('redirects.user_spawn_pending', 1)
html = self.render_template("spawn_pending.html", user=current_user)
self.finish(html)
return
# spawn has supposedly finished, check on the status
status = yield current_user.spawner.poll()
if status is not None:
@@ -465,25 +495,38 @@ class UserSpawnHandler(BaseHandler):
self.set_login_cookie(current_user)
without_prefix = self.request.uri[len(self.hub.server.base_url):]
target = url_path_join(self.base_url, without_prefix)
if self.subdomain_host:
target = current_user.host + target
self.redirect(target)
self.statsd.incr('redirects.user_after_login')
elif current_user:
# logged in as a different user, redirect
self.statsd.incr('redirects.user_to_user', 1)
target = url_path_join(current_user.url, user_path or '')
self.redirect(target)
else:
# not logged in to the right user,
# clear any cookies and reload (will redirect to login)
# not logged in, clear any cookies and reload
self.statsd.incr('redirects.user_to_login', 1)
self.clear_login_cookie()
self.redirect(url_concat(
self.settings['login_url'],
{'next': self.request.uri,
}))
{'next': self.request.uri},
))
class CSPReportHandler(BaseHandler):
'''Accepts a content security policy violation report'''
@web.authenticated
def post(self):
'''Log a content security policy violation report'''
self.log.warn("Content security violation: %s",
self.request.body.decode('utf8', 'replace'))
self.log.warning(
"Content security violation: %s",
self.request.body.decode('utf8', 'replace')
)
# Report it to statsd as well
self.statsd.incr('csp_report')
default_handlers = [
(r'/user/([^/]+)/?.*', UserSpawnHandler),
(r'/user/([^/]+)(/.*)?', UserSpawnHandler),
(r'/security/csp-report', CSPReportHandler),
]

View File

@@ -20,6 +20,7 @@ class LogoutHandler(BaseHandler):
self.clear_login_cookie(name)
user.other_user_cookies = set([])
self.redirect(self.hub.server.base_url, permanent=False)
self.statsd.incr('logout')
class LoginHandler(BaseHandler):
@@ -35,6 +36,7 @@ class LoginHandler(BaseHandler):
)
def get(self):
self.statsd.incr('login.request')
next_url = self.get_argument('next', '')
if not next_url.startswith('/'):
# disallow non-absolute next URLs (e.g. full URLs)
@@ -43,7 +45,7 @@ class LoginHandler(BaseHandler):
if user:
if not next_url:
if user.running:
next_url = user.server.base_url
next_url = user.url
else:
next_url = self.hub.server.base_url
# set new login cookie
@@ -61,8 +63,13 @@ class LoginHandler(BaseHandler):
for arg in self.request.arguments:
data[arg] = self.get_argument(arg)
auth_timer = self.statsd.timer('login.authenticate').start()
username = yield self.authenticate(data)
auth_timer.stop(send=False)
if username:
self.statsd.incr('login.success')
self.statsd.timing('login.authenticate.success', auth_timer.ms)
user = self.user_from_username(username)
already_running = False
if user.spawner:
@@ -78,7 +85,9 @@ class LoginHandler(BaseHandler):
self.redirect(next_url)
self.log.info("User logged in: %s", username)
else:
self.log.debug("Failed login for %s", username)
self.statsd.incr('login.failure')
self.statsd.timing('login.authenticate.failure', auth_timer.ms)
self.log.debug("Failed login for %s", data.get('username', 'unknown user'))
html = self._render(
login_error='Invalid username or password',
username=username,

View File

@@ -8,25 +8,26 @@ from tornado import web, gen
from .. import orm
from ..utils import admin_only, url_path_join
from .base import BaseHandler
from .login import LoginHandler
from urllib.parse import quote
class RootHandler(BaseHandler):
"""Render the Hub root page.
If logged in, redirects to:
- single-user server if running
- hub home, otherwise
Otherwise, renders login page.
"""
def get(self):
user = self.get_current_user()
if user:
if user.running:
url = user.server.base_url
url = user.url
self.log.debug("User is running: %s", url)
self.set_login_cookie(user) # set cookie
else:
url = url_path_join(self.hub.server.base_url, 'home')
self.log.debug("User is not running: %s", url)
@@ -49,9 +50,9 @@ class HomeHandler(BaseHandler):
class SpawnHandler(BaseHandler):
"""Handle spawning of single-user servers via form.
GET renders the form, POST handles form submission.
Only enabled when Spawner.options_form is defined.
"""
def _render_form(self, message=''):
@@ -67,7 +68,7 @@ class SpawnHandler(BaseHandler):
"""GET renders form for spawning with user-specified options"""
user = self.get_current_user()
if user.running:
url = user.server.base_url
url = user.url
self.log.debug("User is running: %s", url)
self.redirect(url)
return
@@ -75,16 +76,15 @@ class SpawnHandler(BaseHandler):
self.finish(self._render_form())
else:
# not running, no form. Trigger spawn.
url = url_path_join(self.base_url, 'user', user.name)
self.redirect(url)
self.redirect(user.url)
@web.authenticated
@gen.coroutine
def post(self):
"""POST spawns with user-specified options"""
user = self.get_current_user()
if user.running:
url = user.server.base_url
url = user.url
self.log.warning("User is already running: %s", url)
self.redirect(url)
return
@@ -93,15 +93,15 @@ class SpawnHandler(BaseHandler):
form_options[key] = [ bs.decode('utf8') for bs in byte_list ]
for key, byte_list in self.request.files.items():
form_options["%s_file"%key] = byte_list
options = user.spawner.options_from_form(form_options)
try:
options = user.spawner.options_from_form(form_options)
yield self.spawn_single_user(user, options=options)
except Exception as e:
self.log.error("Failed to spawn single-user server with form", exc_info=True)
self.finish(self._render_form(str(e)))
return
self.set_login_cookie(user)
url = user.server.base_url
url = user.url
self.redirect(url)
class AdminHandler(BaseHandler):
@@ -122,14 +122,14 @@ class AdminHandler(BaseHandler):
}
sorts = self.get_arguments('sort') or default_sort
orders = self.get_arguments('order')
for bad in set(sorts).difference(available):
self.log.warn("ignoring invalid sort: %r", bad)
self.log.warning("ignoring invalid sort: %r", bad)
sorts.remove(bad)
for bad in set(orders).difference({'asc', 'desc'}):
self.log.warn("ignoring invalid order: %r", bad)
self.log.warning("ignoring invalid order: %r", bad)
orders.remove(bad)
# add default sort as secondary
for s in default_sort:
if s not in sorts:
@@ -139,17 +139,17 @@ class AdminHandler(BaseHandler):
orders.append(default_order[col])
else:
orders = orders[:len(sorts)]
# this could be one incomprehensible nested list comprehension
# get User columns
cols = [ getattr(orm.User, mapping.get(c, c)) for c in sorts ]
# get User.col.desc() order objects
ordered = [ getattr(c, o)() for c, o in zip(cols, orders) ]
users = self.db.query(orm.User).order_by(*ordered)
users = [ self._user_from_orm(u) for u in users ]
running = [ u for u in users if u.running ]
html = self.render_template('admin.html',
user=self.get_current_user(),
admin_access=self.settings.get('admin_access', False),

View File

@@ -1,6 +1,7 @@
# Copyright (c) Jupyter Development Team.
# Distributed under the terms of the Modified BSD License.
import os
from tornado.web import StaticFileHandler
class CacheControlStaticFilesHandler(StaticFileHandler):
@@ -14,4 +15,14 @@ class CacheControlStaticFilesHandler(StaticFileHandler):
def set_extra_headers(self, path):
if "v" not in self.request.arguments:
self.add_header("Cache-Control", "no-cache")
class LogoHandler(StaticFileHandler):
"""A singular handler for serving the logo."""
def get(self):
return super().get('')
@classmethod
def get_absolute_path(cls, root, path):
"""We only serve one file, ignore relative path"""
return os.path.abspath(root)

View File

@@ -4,15 +4,13 @@
# Distributed under the terms of the Modified BSD License.
from datetime import datetime
import errno
import json
import socket
from tornado import gen
from tornado.log import app_log
from tornado.httpclient import HTTPRequest, AsyncHTTPClient
from sqlalchemy.types import TypeDecorator, VARCHAR
from sqlalchemy.types import TypeDecorator, TEXT
from sqlalchemy import (
inspect,
Column, Integer, ForeignKey, Unicode, Boolean,
@@ -26,7 +24,7 @@ from sqlalchemy import create_engine
from .utils import (
random_port, url_path_join, wait_for_server, wait_for_http_server,
new_token, hash_token, compare_token, localhost,
new_token, hash_token, compare_token, can_connect,
)
@@ -39,7 +37,7 @@ class JSONDict(TypeDecorator):
"""
impl = VARCHAR
impl = TEXT
def process_bind_param(self, value, dialect):
if value is not None:
@@ -59,26 +57,26 @@ Base.log = app_log
class Server(Base):
"""The basic state of a server
connection and cookie info
"""
__tablename__ = 'servers'
id = Column(Integer, primary_key=True)
proto = Column(Unicode, default='http')
ip = Column(Unicode, default='')
proto = Column(Unicode(15), default='http')
ip = Column(Unicode(255), default='') # could also be a DNS name
port = Column(Integer, default=random_port)
base_url = Column(Unicode, default='/')
cookie_name = Column(Unicode, default='cookie')
base_url = Column(Unicode(255), default='/')
cookie_name = Column(Unicode(255), default='cookie')
def __repr__(self):
return "<Server(%s:%s)>" % (self.ip, self.port)
@property
def host(self):
ip = self.ip
if ip in {'', '0.0.0.0'}:
# when listening on all interfaces, connect to localhost
ip = localhost()
ip = '127.0.0.1'
return "{proto}://{ip}:{port}".format(
proto=self.proto,
ip=ip,
@@ -91,52 +89,34 @@ class Server(Base):
host=self.host,
uri=self.base_url,
)
@property
def bind_url(self):
"""representation of URL used for binding
Never used in APIs, only logging,
since it can be non-connectable value, such as '', meaning all interfaces.
"""
if self.ip in {'', '0.0.0.0'}:
return self.url.replace('localhost', self.ip or '*', 1)
return self.url.replace('127.0.0.1', self.ip or '*', 1)
return self.url
@gen.coroutine
def wait_up(self, timeout=10, http=False):
"""Wait for this server to come up"""
if http:
yield wait_for_http_server(self.url, timeout=timeout)
else:
yield wait_for_server(self.ip or localhost(), self.port, timeout=timeout)
yield wait_for_server(self.ip or '127.0.0.1', self.port, timeout=timeout)
def is_up(self):
"""Is the server accepting connections?"""
try:
socket.create_connection((self.ip or localhost(), self.port))
except socket.error as e:
if e.errno == errno.ENETUNREACH:
try:
socket.create_connection((self.ip or '127.0.0.1', self.port))
except socket.error as e:
if e.errno == errno.ECONNREFUSED:
return False
else:
raise
else:
return True
elif e.errno == errno.ECONNREFUSED:
return False
else:
raise
else:
return True
return can_connect(self.ip or '127.0.0.1', self.port)
class Proxy(Base):
"""A configurable-http-proxy instance.
A proxy consists of the API server info and the public-facing server info,
plus an auth token for configuring the proxy table.
"""
@@ -147,7 +127,7 @@ class Proxy(Base):
public_server = relationship(Server, primaryjoin=_public_server_id == Server.id)
_api_server_id = Column(Integer, ForeignKey('servers.id'))
api_server = relationship(Server, primaryjoin=_api_server_id == Server.id)
def __repr__(self):
if self.public_server:
return "<%s %s:%s>" % (
@@ -155,7 +135,7 @@ class Proxy(Base):
)
else:
return "<%s [unconfigured]>" % self.__class__.__name__
def api_request(self, path, method='GET', body=None, client=None):
"""Make an authenticated API request of the proxy"""
client = client or AsyncHTTPClient()
@@ -176,10 +156,14 @@ class Proxy(Base):
def add_user(self, user, client=None):
"""Add a user's server to the proxy table."""
self.log.info("Adding user %s to proxy %s => %s",
user.name, user.server.base_url, user.server.host,
user.name, user.proxy_path, user.server.host,
)
yield self.api_request(user.server.base_url,
if user.spawn_pending:
raise RuntimeError(
"User %s's spawn is pending, shouldn't be added to the proxy yet!", user.name)
yield self.api_request(user.proxy_path,
method='POST',
body=dict(
target=user.server.host,
@@ -187,30 +171,15 @@ class Proxy(Base):
),
client=client,
)
@gen.coroutine
def delete_user(self, user, client=None):
"""Remove a user's server to the proxy table."""
self.log.info("Removing user %s from proxy", user.name)
yield self.api_request(user.server.base_url,
yield self.api_request(user.proxy_path,
method='DELETE',
client=client,
)
@gen.coroutine
def add_all_users(self):
"""Update the proxy table from the database.
Used when loading up a new proxy.
"""
db = inspect(self).session
futures = []
for user in db.query(User):
if (user.server):
futures.append(self.add_user(user))
# wait after submitting them all
for f in futures:
yield f
@gen.coroutine
def get_routes(self, client=None):
@@ -219,17 +188,42 @@ class Proxy(Base):
return json.loads(resp.body.decode('utf8', 'replace'))
@gen.coroutine
def check_routes(self, routes=None):
"""Check that all users are properly"""
def add_all_users(self, user_dict):
"""Update the proxy table from the database.
Used when loading up a new proxy.
"""
db = inspect(self).session
futures = []
for orm_user in db.query(User):
user = user_dict[orm_user]
if user.running:
futures.append(self.add_user(user))
# wait after submitting them all
for f in futures:
yield f
@gen.coroutine
def check_routes(self, user_dict, routes=None):
"""Check that all users are properly routed on the proxy"""
if not routes:
routes = yield self.get_routes()
have_routes = { r['user'] for r in routes.values() if 'user' in r }
futures = []
db = inspect(self).session
for user in db.query(User).filter(User.server != None):
for orm_user in db.query(User).filter(User.server != None):
user = user_dict[orm_user]
if not user.running:
# Don't add users to the proxy that haven't finished starting
continue
if user.server is None:
# This should never be True, but seems to be on rare occasion.
# catch filter bug, either in sqlalchemy or my understanding of its behavior
self.log.error("User %s has no server, but wasn't filtered out.", user)
continue
if user.name not in have_routes:
self.log.warn("Adding missing route for %s", user.name)
self.log.warning("Adding missing route for %s (%s)", user.name, user.server)
futures.append(self.add_user(user))
for f in futures:
yield f
@@ -238,9 +232,9 @@ class Proxy(Base):
class Hub(Base):
"""Bring it all together at the hub.
The Hub is a server, plus its API path suffix
the api_url is the full URL plus the api_path suffix on the end
of the server base_url.
"""
@@ -248,12 +242,13 @@ class Hub(Base):
id = Column(Integer, primary_key=True)
_server_id = Column(Integer, ForeignKey('servers.id'))
server = relationship(Server, primaryjoin=_server_id == Server.id)
host = ''
@property
def api_url(self):
"""return the full API url (with proto://host...)"""
return url_path_join(self.server.url, 'api')
def __repr__(self):
if self.server:
return "<%s %s:%s>" % (
@@ -265,31 +260,31 @@ class Hub(Base):
class User(Base):
"""The User table
Each user has a single server,
and multiple tokens used for authorization.
API tokens grant access to the Hub's REST API.
These are used by single-user servers to authenticate requests,
and external services to manipulate the Hub.
Cookies are set with a single ID.
Resetting the Cookie ID invalidates all cookies, forcing user to login again.
A `state` column contains a JSON dict,
used for restoring state of a Spawner.
"""
__tablename__ = 'users'
id = Column(Integer, primary_key=True, autoincrement=True)
name = Column(Unicode)
name = Column(Unicode(1023))
# should we allow multiple servers per user?
_server_id = Column(Integer, ForeignKey('servers.id'))
server = relationship(Server, primaryjoin=_server_id == Server.id)
admin = Column(Boolean, default=False)
last_activity = Column(DateTime, default=datetime.utcnow)
api_tokens = relationship("APIToken", backref="user")
cookie_id = Column(Unicode, default=new_token)
cookie_id = Column(Unicode(1023), default=new_token)
state = Column(JSONDict)
other_user_cookies = set([])
@@ -307,12 +302,22 @@ class User(Base):
cls=self.__class__.__name__,
name=self.name,
)
def new_api_token(self):
"""Create a new API token"""
def new_api_token(self, token=None):
"""Create a new API token
If `token` is given, load that token.
"""
assert self.id is not None
db = inspect(self).session
token = new_token()
if token is None:
token = new_token()
else:
if len(token) < 8:
raise ValueError("Tokens must be at least 8 characters, got %r" % token)
found = APIToken.find(db, token)
if found:
raise ValueError("Collision on token: %s..." % token[:4])
orm_token = APIToken(user_id=self.id)
orm_token.token = token
db.add(orm_token)
@@ -330,29 +335,29 @@ class User(Base):
class APIToken(Base):
"""An API token"""
__tablename__ = 'api_tokens'
@declared_attr
def user_id(cls):
return Column(Integer, ForeignKey('users.id'))
id = Column(Integer, primary_key=True)
hashed = Column(Unicode)
prefix = Column(Unicode)
hashed = Column(Unicode(1023))
prefix = Column(Unicode(1023))
prefix_length = 4
algorithm = "sha512"
rounds = 16384
salt_bytes = 8
@property
def token(self):
raise AttributeError("token is write-only")
@token.setter
def token(self, token):
"""Store the hashed value and prefix for a token"""
self.prefix = token[:self.prefix_length]
self.hashed = hash_token(token, rounds=self.rounds, salt=self.salt_bytes, algorithm=self.algorithm)
def __repr__(self):
return "<{cls}('{pre}...', user='{u}')>".format(
cls=self.__class__.__name__,
@@ -373,7 +378,7 @@ class APIToken(Base):
for orm_token in prefix_match:
if orm_token.match(token):
return orm_token
def match(self, token):
"""Is this my token?"""
return compare_token(self.hashed, token)
@@ -383,6 +388,8 @@ def new_session_factory(url="sqlite:///:memory:", reset=False, **kwargs):
"""Create a new session at url"""
if url.startswith('sqlite'):
kwargs.setdefault('connect_args', {'check_same_thread': False})
elif url.startswith('mysql'):
kwargs.setdefault('pool_recycle', 60)
if url.endswith(':memory:'):
# If we're using an in-memory database, ensure that only one connection

View File

@@ -10,8 +10,9 @@ import pwd
import signal
import sys
import grp
import warnings
from subprocess import Popen
from tempfile import TemporaryDirectory
from tempfile import mkdtemp
from tornado import gen
from tornado.ioloop import IOLoop, PeriodicCallback
@@ -22,7 +23,7 @@ from traitlets import (
)
from .traitlets import Command
from .utils import random_port, localhost
from .utils import random_port
class Spawner(LoggingConfigurable):
"""Base class for spawning single-user notebook servers.
@@ -41,39 +42,38 @@ class Spawner(LoggingConfigurable):
hub = Any()
authenticator = Any()
api_token = Unicode()
ip = Unicode(localhost(), config=True,
ip = Unicode('127.0.0.1',
help="The IP address (or hostname) the single-user server should listen on"
)
start_timeout = Integer(60, config=True,
).tag(config=True)
start_timeout = Integer(60,
help="""Timeout (in seconds) before giving up on the spawner.
This is the timeout for start to return, not the timeout for the server to respond.
Callers of spawner.start will assume that startup has failed if it takes longer than this.
start should return when the server process is started and its location is known.
"""
)
).tag(config=True)
http_timeout = Integer(
30, config=True,
http_timeout = Integer(30,
help="""Timeout (in seconds) before giving up on a spawned HTTP server
Once a server has successfully been spawned, this is the amount of time
we wait before assuming that the server is unable to accept
connections.
"""
)
).tag(config=True)
poll_interval = Integer(30, config=True,
poll_interval = Integer(30,
help="""Interval (in seconds) on which to poll the spawner."""
)
).tag(config=True)
_callbacks = List()
_poll_callback = Any()
debug = Bool(False, config=True,
debug = Bool(False,
help="Enable debug-logging of the single-user server"
)
).tag(config=True)
options_form = Unicode("", config=True, help="""
options_form = Unicode("", help="""
An HTML form for options a user can specify on launching their server.
The surrounding `<form>` element and the submit button are already provided.
@@ -87,7 +87,7 @@ class Spawner(LoggingConfigurable):
<option value="A">The letter A</option>
<option value="B">The letter B</option>
</select>
""")
""").tag(config=True)
def options_from_form(self, form_data):
"""Interpret HTTP form data
@@ -113,32 +113,58 @@ class Spawner(LoggingConfigurable):
'VIRTUAL_ENV',
'LANG',
'LC_ALL',
], config=True,
],
help="Whitelist of environment variables for the subprocess to inherit"
)
env = Dict()
def _env_default(self):
env = {}
for key in self.env_keep:
if key in os.environ:
env[key] = os.environ[key]
env['JPY_API_TOKEN'] = self.api_token
return env
).tag(config=True)
env = Dict(help="""Deprecated: use Spawner.get_env or Spawner.environment
cmd = Command(['jupyterhub-singleuser'], config=True,
- extend Spawner.get_env for adding required env in Spawner subclasses
- Spawner.environment for config-specified env
""")
environment = Dict(
help="""Environment variables to load for the Spawner.
Value could be a string or a callable. If it is a callable, it will
be called with one parameter, which will be the instance of the spawner
in use. It should quickly (without doing much blocking operations) return
a string that will be used as the value for the environment variable.
"""
).tag(config=True)
cmd = Command(['jupyterhub-singleuser'],
help="""The command used for starting notebooks."""
)
args = List(Unicode, config=True,
).tag(config=True)
args = List(Unicode(),
help="""Extra arguments to be passed to the single-user server"""
)
).tag(config=True)
notebook_dir = Unicode('', config=True,
notebook_dir = Unicode('',
help="""The notebook directory for the single-user server
`~` will be expanded to the user's home directory
`%U` will be expanded to the user's username
"""
)
).tag(config=True)
default_url = Unicode('',
help="""The default URL for the single-user server.
Can be used in conjunction with --notebook-dir=/ to enable
full filesystem traversal, while preserving user's homedir as
landing page for notebook
`%U` will be expanded to the user's username
"""
).tag(config=True)
disable_user_config = Bool(False,
help="""Disable per-user configuration of single-user servers.
This prevents any config in users' $HOME directories
from having an effect on their server.
"""
).tag(config=True)
def __init__(self, **kwargs):
super(Spawner, self).__init__(**kwargs)
@@ -185,12 +211,34 @@ class Spawner(LoggingConfigurable):
self.api_token = ''
def get_env(self):
"""Return the environment we should use
Default returns a copy of self.env.
"""Return the environment dict to use for the Spawner.
This applies things like `env_keep`, anything defined in `Spawner.environment`,
and adds the API token to the env.
Use this to access the env in Spawner.start to allow extension in subclasses.
"""
return self.env.copy()
env = {}
if self.env:
warnings.warn("Spawner.env is deprecated, found %s" % self.env, DeprecationWarning)
env.update(self.env)
for key in self.env_keep:
if key in os.environ:
env[key] = os.environ[key]
# config overrides. If the value is a callable, it will be called with
# one parameter - the current spawner instance - and the return value
# will be assigned to the environment variable. This will be called at
# spawn time.
for key, value in self.environment.items():
if callable(value):
env[key] = value(self)
else:
env[key] = value
env['JPY_API_TOKEN'] = self.api_token
return env
def get_args(self):
"""Return the arguments to be passed after self.cmd"""
@@ -199,6 +247,7 @@ class Spawner(LoggingConfigurable):
'--port=%i' % self.user.server.port,
'--cookie-name=%s' % self.user.server.cookie_name,
'--base-url=%s' % self.user.server.base_url,
'--hub-host=%s' % self.hub.host,
'--hub-prefix=%s' % self.hub.server.base_url,
'--hub-api-url=%s' % self.hub.api_url,
]
@@ -207,8 +256,14 @@ class Spawner(LoggingConfigurable):
if self.notebook_dir:
self.notebook_dir = self.notebook_dir.replace("%U",self.user.name)
args.append('--notebook-dir=%s' % self.notebook_dir)
if self.default_url:
self.default_url = self.default_url.replace("%U",self.user.name)
args.append('--NotebookApp.default_url=%s' % self.default_url)
if self.debug:
args.append('--debug')
if self.disable_user_config:
args.append('--disable-user-config')
args.extend(self.args)
return args
@@ -302,12 +357,13 @@ def _try_setcwd(path):
try:
os.chdir(path)
except OSError as e:
exc = e # break exception instance out of except scope
print("Couldn't set CWD to %s (%s)" % (path, e), file=sys.stderr)
path, _ = os.path.split(path)
else:
return
print("Couldn't set CWD at all (%s), using temp dir" % e, file=sys.stderr)
td = TemporaryDirectory().name
print("Couldn't set CWD at all (%s), using temp dir" % exc, file=sys.stderr)
td = mkdtemp()
os.chdir(td)
@@ -342,15 +398,15 @@ class LocalProcessSpawner(Spawner):
This is the default spawner for JupyterHub.
"""
INTERRUPT_TIMEOUT = Integer(10, config=True,
INTERRUPT_TIMEOUT = Integer(10,
help="Seconds to wait for process to halt after SIGINT before proceeding to SIGTERM"
)
TERM_TIMEOUT = Integer(5, config=True,
).tag(config=True)
TERM_TIMEOUT = Integer(5,
help="Seconds to wait for process to halt after SIGTERM before proceeding to SIGKILL"
)
KILL_TIMEOUT = Integer(5, config=True,
).tag(config=True)
KILL_TIMEOUT = Integer(5,
help="Seconds to wait for process to halt after SIGKILL before giving up"
)
).tag(config=True)
proc = Instance(Popen, allow_none=True)
pid = Integer(0)
@@ -486,5 +542,5 @@ class LocalProcessSpawner(Spawner):
status = yield self.poll()
if status is None:
# it all failed, zombie process
self.log.warn("Process %i never died", self.pid)
self.log.warning("Process %i never died", self.pid)

View File

@@ -1,7 +1,7 @@
"""mock utilities for testing"""
import os
import sys
from datetime import timedelta
from tempfile import NamedTemporaryFile
import threading
@@ -13,11 +13,13 @@ from tornado import gen
from tornado.concurrent import Future
from tornado.ioloop import IOLoop
from ..spawner import LocalProcessSpawner
from traitlets import default
from ..app import JupyterHub
from ..auth import PAMAuthenticator
from .. import orm
from ..utils import localhost
from ..spawner import LocalProcessSpawner
from ..utils import url_path_join
from pamela import PAMError
@@ -44,7 +46,7 @@ class MockSpawner(LocalProcessSpawner):
def user_env(self, env):
return env
@default('cmd')
def _cmd_default(self):
return [sys.executable, '-m', 'jupyterhub.tests.mocksu']
@@ -66,6 +68,7 @@ class SlowSpawner(MockSpawner):
class NeverSpawner(MockSpawner):
"""A spawner that will never start"""
@default('start_timeout')
def _start_timeout_default(self):
return 1
@@ -90,6 +93,7 @@ class FormSpawner(MockSpawner):
class MockPAMAuthenticator(PAMAuthenticator):
@default('admin_users')
def _admin_users_default(self):
return {'admin'}
@@ -109,13 +113,23 @@ class MockHub(JupyterHub):
"""Hub with various mock bits"""
db_file = None
confirm_no_ssl = True
last_activity_interval = 2
@default('subdomain_host')
def _subdomain_host_default(self):
return os.environ.get('JUPYTERHUB_TEST_SUBDOMAIN_HOST', '')
@default('ip')
def _ip_default(self):
return localhost()
return '127.0.0.1'
@default('authenticator_class')
def _authenticator_class_default(self):
return MockPAMAuthenticator
@default('spawner_class')
def _spawner_class_default(self):
return MockSpawner
@@ -124,7 +138,8 @@ class MockHub(JupyterHub):
def start(self, argv=None):
self.db_file = NamedTemporaryFile()
self.db_url = 'sqlite:///' + self.db_file.name
self.pid_file = NamedTemporaryFile(delete=False).name
self.db_url = self.db_file.name
evt = threading.Event()
@@ -161,13 +176,33 @@ class MockHub(JupyterHub):
self.db_file.close()
def login_user(self, name):
r = requests.post(self.proxy.public_server.url + 'hub/login',
base_url = public_url(self)
r = requests.post(base_url + 'hub/login',
data={
'username': name,
'password': name,
},
allow_redirects=False,
)
r.raise_for_status()
assert r.cookies
return r.cookies
def public_host(app):
if app.subdomain_host:
return app.subdomain_host
else:
return app.proxy.public_server.host
def public_url(app):
return public_host(app) + app.proxy.public_server.base_url
def user_url(user, app):
if app.subdomain_host:
host = user.host
else:
host = public_host(app)
return host + user.server.base_url

View File

@@ -2,9 +2,8 @@
import json
import time
from datetime import timedelta
from queue import Queue
from urllib.parse import urlparse
from urllib.parse import urlparse, quote
import requests
@@ -14,6 +13,7 @@ from .. import orm
from ..user import User
from ..utils import url_path_join as ujoin
from . import mocking
from .mocking import public_url, user_url
def check_db_locks(func):
@@ -41,7 +41,7 @@ def check_db_locks(func):
def find_user(db, name):
return db.query(orm.User).filter(orm.User.name==name).first()
def add_user(db, app=None, **kwargs):
orm_user = orm.User(**kwargs)
db.add(orm_user)
@@ -81,17 +81,17 @@ def test_auth_api(app):
db = app.db
r = api_request(app, 'authorizations', 'gobbledygook')
assert r.status_code == 404
# make a new cookie token
user = db.query(orm.User).first()
api_token = user.new_api_token()
# check success:
r = api_request(app, 'authorizations/token', api_token)
assert r.status_code == 200
reply = r.json()
assert reply['name'] == user.name
# check fail
r = api_request(app, 'authorizations/token', api_token,
headers={'Authorization': 'no sir'},
@@ -105,7 +105,7 @@ def test_auth_api(app):
def test_referer_check(app, io_loop):
url = app.hub.server.url
url = ujoin(public_url(app), app.hub.server.base_url)
host = urlparse(url).netloc
user = find_user(app.db, 'admin')
if user is None:
@@ -115,7 +115,7 @@ def test_referer_check(app, io_loop):
# stop the admin's server so we don't mess up future tests
io_loop.run_sync(lambda : app.proxy.delete_user(app_user))
io_loop.run_sync(app_user.stop)
r = api_request(app, 'users',
headers={
'Authorization': '',
@@ -152,7 +152,7 @@ def test_get_users(app):
db = app.db
r = api_request(app, 'users')
assert r.status_code == 200
users = sorted(r.json(), key=lambda d: d['name'])
for u in users:
u.pop('last_activity')
@@ -230,21 +230,21 @@ def test_add_multi_user(app):
reply = r.json()
r_names = [ user['name'] for user in reply ]
assert names == r_names
for name in names:
user = find_user(db, name)
assert user is not None
assert user.name == name
assert not user.admin
# try to create the same users again
r = api_request(app, 'users', method='post',
data=json.dumps({'usernames': names}),
)
assert r.status_code == 400
names = ['a', 'b', 'ab']
# try to create the same users again
r = api_request(app, 'users', method='post',
data=json.dumps({'usernames': names}),
@@ -265,7 +265,7 @@ def test_add_multi_user_admin(app):
reply = r.json()
r_names = [ user['name'] for user in reply ]
assert names == r_names
for name in names:
user = find_user(db, name)
assert user is not None
@@ -298,7 +298,7 @@ def test_delete_user(app):
mal = add_user(db, name='mal')
r = api_request(app, 'users', 'mal', method='delete')
assert r.status_code == 204
def test_make_admin(app):
db = app.db
@@ -321,7 +321,7 @@ def test_make_admin(app):
def get_app_user(app, name):
"""Get the User object from the main thread
Needed for access to the Spawner.
No ORM methods should be called on the result.
"""
@@ -350,21 +350,25 @@ def test_spawn(app, io_loop):
assert not app_user.spawn_pending
status = io_loop.run_sync(app_user.spawner.poll)
assert status is None
assert user.server.base_url == '/user/%s' % name
r = requests.get(ujoin(app.proxy.public_server.url, user.server.base_url))
url = user_url(user, app)
print(url)
r = requests.get(url)
assert r.status_code == 200
assert r.text == user.server.base_url
r = requests.get(ujoin(app.proxy.public_server.url, user.server.base_url, 'args'))
r = requests.get(ujoin(url, 'args'))
assert r.status_code == 200
argv = r.json()
for expected in ['--user=%s' % name, '--base-url=%s' % user.server.base_url]:
assert expected in argv
if app.subdomain_host:
assert '--hub-host=%s' % app.subdomain_host in argv
r = api_request(app, 'users', name, 'server', method='delete')
assert r.status_code == 204
assert 'pid' not in user.state
status = io_loop.run_sync(app_user.spawner.poll)
assert status == 0
@@ -379,18 +383,19 @@ def test_slow_spawn(app, io_loop):
name = 'zoe'
user = add_user(db, app=app, name=name)
r = api_request(app, 'users', name, 'server', method='post')
app.tornado_settings['spawner_class'] = mocking.MockSpawner
r.raise_for_status()
assert r.status_code == 202
app_user = get_app_user(app, name)
assert app_user.spawner is not None
assert app_user.spawn_pending
assert not app_user.stop_pending
@gen.coroutine
def wait_spawn():
while app_user.spawn_pending:
yield gen.sleep(0.1)
io_loop.run_sync(wait_spawn)
assert not app_user.spawn_pending
status = io_loop.run_sync(app_user.spawner.poll)
@@ -412,13 +417,13 @@ def test_slow_spawn(app, io_loop):
assert r.status_code == 202
assert app_user.spawner is not None
assert app_user.stop_pending
io_loop.run_sync(wait_stop)
assert not app_user.stop_pending
assert app_user.spawner is not None
r = api_request(app, 'users', name, 'server', method='delete')
assert r.status_code == 400
def test_never_spawn(app, io_loop):
app.tornado_settings['spawner_class'] = mocking.NeverSpawner
@@ -428,15 +433,16 @@ def test_never_spawn(app, io_loop):
name = 'badger'
user = add_user(db, app=app, name=name)
r = api_request(app, 'users', name, 'server', method='post')
app.tornado_settings['spawner_class'] = mocking.MockSpawner
app_user = get_app_user(app, name)
assert app_user.spawner is not None
assert app_user.spawn_pending
@gen.coroutine
def wait_pending():
while app_user.spawn_pending:
yield gen.sleep(0.1)
io_loop.run_sync(wait_pending)
assert not app_user.spawn_pending
status = io_loop.run_sync(app_user.spawner.poll)
@@ -450,6 +456,76 @@ def test_get_proxy(app, io_loop):
assert list(reply.keys()) == ['/']
def test_cookie(app):
db = app.db
name = 'patience'
user = add_user(db, app=app, name=name)
r = api_request(app, 'users', name, 'server', method='post')
assert r.status_code == 201
assert 'pid' in user.state
app_user = get_app_user(app, name)
cookies = app.login_user(name)
# cookie jar gives '"cookie-value"', we want 'cookie-value'
cookie = cookies[user.server.cookie_name][1:-1]
r = api_request(app, 'authorizations/cookie', user.server.cookie_name, "nothintoseehere")
assert r.status_code == 404
r = api_request(app, 'authorizations/cookie', user.server.cookie_name, quote(cookie, safe=''))
r.raise_for_status()
reply = r.json()
assert reply['name'] == name
# deprecated cookie in body:
r = api_request(app, 'authorizations/cookie', user.server.cookie_name, data=cookie)
r.raise_for_status()
reply = r.json()
assert reply['name'] == name
def test_token(app):
name = 'book'
user = add_user(app.db, app=app, name=name)
token = user.new_api_token()
r = api_request(app, 'authorizations/token', token)
r.raise_for_status()
user_model = r.json()
assert user_model['name'] == name
r = api_request(app, 'authorizations/token', 'notauthorized')
assert r.status_code == 404
def test_get_token(app):
name = 'user'
user = add_user(app.db, app=app, name=name)
r = api_request(app, 'authorizations/token', method='post', data=json.dumps({
'username': name,
'password': name,
}))
assert r.status_code == 200
data = r.content.decode("utf-8")
token = json.loads(data)
assert not token['Authentication'] is None
def test_bad_get_token(app):
name = 'user'
password = 'fake'
user = add_user(app.db, app=app, name=name)
r = api_request(app, 'authorizations/token', method='post', data=json.dumps({
'username': name,
'password': password,
}))
assert r.status_code == 403
def test_options(app):
r = api_request(app, 'users', method='options')
r.raise_for_status()
assert 'Access-Control-Allow-Headers' in r.headers
def test_bad_json_body(app):
r = api_request(app, 'users', method='post', data='notjson')
assert r.status_code == 400
def test_shutdown(app):
r = api_request(app, 'shutdown', method='post', data=json.dumps({
'servers': True,

View File

@@ -1,10 +1,17 @@
"""Test the JupyterHub entry point"""
import binascii
import os
import re
import sys
from subprocess import check_output
from subprocess import check_output, Popen, PIPE
from tempfile import NamedTemporaryFile, TemporaryDirectory
from unittest.mock import patch
import pytest
from .mocking import MockHub
from .. import orm
def test_help_all():
out = check_output([sys.executable, '-m', 'jupyterhub', '--help-all']).decode('utf8', 'replace')
@@ -23,10 +30,23 @@ def test_token_app():
def test_generate_config():
with NamedTemporaryFile(prefix='jupyterhub_config', suffix='.py') as tf:
cfg_file = tf.name
out = check_output([sys.executable, '-m', 'jupyterhub',
'--generate-config', '-f', cfg_file]
).decode('utf8', 'replace')
with open(cfg_file, 'w') as f:
f.write("c.A = 5")
p = Popen([sys.executable, '-m', 'jupyterhub',
'--generate-config', '-f', cfg_file],
stdout=PIPE, stdin=PIPE)
out, _ = p.communicate(b'n')
out = out.decode('utf8', 'replace')
assert os.path.exists(cfg_file)
with open(cfg_file) as f:
cfg_text = f.read()
assert cfg_text == 'c.A = 5'
p = Popen([sys.executable, '-m', 'jupyterhub',
'--generate-config', '-f', cfg_file],
stdout=PIPE, stdin=PIPE)
out, _ = p.communicate(b'x\ny')
out = out.decode('utf8', 'replace')
assert os.path.exists(cfg_file)
with open(cfg_file) as f:
cfg_text = f.read()
@@ -34,3 +54,89 @@ def test_generate_config():
assert cfg_file in out
assert 'Spawner.cmd' in cfg_text
assert 'Authenticator.whitelist' in cfg_text
def test_init_tokens():
with TemporaryDirectory() as td:
db_file = os.path.join(td, 'jupyterhub.sqlite')
tokens = {
'super-secret-token': 'alyx',
'also-super-secret': 'gordon',
'boagasdfasdf': 'chell',
}
app = MockHub(db_file=db_file, api_tokens=tokens)
app.initialize([])
db = app.db
for token, username in tokens.items():
api_token = orm.APIToken.find(db, token)
assert api_token is not None
user = api_token.user
assert user.name == username
# simulate second startup, reloading same tokens:
app = MockHub(db_file=db_file, api_tokens=tokens)
app.initialize([])
db = app.db
for token, username in tokens.items():
api_token = orm.APIToken.find(db, token)
assert api_token is not None
user = api_token.user
assert user.name == username
# don't allow failed token insertion to create users:
tokens['short'] = 'gman'
app = MockHub(db_file=db_file, api_tokens=tokens)
# with pytest.raises(ValueError):
app.initialize([])
assert orm.User.find(app.db, 'gman') is None
def test_write_cookie_secret(tmpdir):
secret_path = str(tmpdir.join('cookie_secret'))
hub = MockHub(cookie_secret_file=secret_path)
hub.init_secrets()
assert os.path.exists(secret_path)
assert os.stat(secret_path).st_mode & 0o600
assert not os.stat(secret_path).st_mode & 0o177
def test_cookie_secret_permissions(tmpdir):
secret_file = tmpdir.join('cookie_secret')
secret_path = str(secret_file)
secret = os.urandom(1024)
secret_file.write(binascii.b2a_base64(secret))
hub = MockHub(cookie_secret_file=secret_path)
# raise with public secret file
os.chmod(secret_path, 0o664)
with pytest.raises(SystemExit):
hub.init_secrets()
# ok with same file, proper permissions
os.chmod(secret_path, 0o660)
hub.init_secrets()
assert hub.cookie_secret == secret
def test_cookie_secret_content(tmpdir):
secret_file = tmpdir.join('cookie_secret')
secret_file.write('not base 64: uñiço∂e')
secret_path = str(secret_file)
os.chmod(secret_path, 0o660)
hub = MockHub(cookie_secret_file=secret_path)
with pytest.raises(SystemExit):
hub.init_secrets()
def test_cookie_secret_env(tmpdir):
hub = MockHub(cookie_secret_file=str(tmpdir.join('cookie_secret')))
with patch.dict(os.environ, {'JPY_COOKIE_SECRET': 'not hex'}):
with pytest.raises(ValueError):
hub.init_secrets()
with patch.dict(os.environ, {'JPY_COOKIE_SECRET': 'abc123'}):
hub.init_secrets()
assert hub.cookie_secret == binascii.a2b_hex('abc123')
assert not os.path.exists(hub.cookie_secret_file)

View File

@@ -20,7 +20,7 @@ def test_server(db):
assert server.proto == 'http'
assert isinstance(server.port, int)
assert isinstance(server.cookie_name, str)
assert server.host == 'http://localhost:%i' % server.port
assert server.host == 'http://127.0.0.1:%i' % server.port
assert server.url == server.host + '/'
assert server.bind_url == 'http://*:%i/' % server.port
server.ip = '127.0.0.1'
@@ -93,6 +93,16 @@ def test_tokens(db):
found = orm.APIToken.find(db, 'something else')
assert found is None
secret = 'super-secret-preload-token'
token = user.new_api_token(secret)
assert token == secret
assert len(user.api_tokens) == 3
# raise ValueError on collision
with pytest.raises(ValueError):
user.new_api_token(token)
assert len(user.api_tokens) == 3
def test_spawn_fails(db, io_loop):
orm_user = orm.User(name='aeofel')

View File

@@ -1,6 +1,6 @@
"""Tests for HTML pages"""
from urllib.parse import urlparse
from urllib.parse import urlencode, urlparse
import requests
@@ -8,12 +8,11 @@ from ..utils import url_path_join as ujoin
from .. import orm
import mock
from .mocking import FormSpawner
from .mocking import FormSpawner, public_url, public_host, user_url
from .test_api import api_request
def get_page(path, app, **kw):
base_url = ujoin(app.proxy.public_server.host, app.hub.server.base_url)
base_url = ujoin(public_url(app), app.hub.server.base_url)
print(base_url)
return requests.get(ujoin(base_url, path), **kw)
@@ -22,15 +21,17 @@ def test_root_no_auth(app, io_loop):
routes = io_loop.run_sync(app.proxy.get_routes)
print(routes)
print(app.hub.server)
r = requests.get(app.proxy.public_server.host)
url = public_url(app)
print(url)
r = requests.get(url)
r.raise_for_status()
assert r.url == ujoin(app.proxy.public_server.host, app.hub.server.base_url, 'login')
assert r.url == ujoin(url, app.hub.server.base_url, 'login')
def test_root_auth(app):
cookies = app.login_user('river')
r = requests.get(app.proxy.public_server.host, cookies=cookies)
r = requests.get(public_url(app), cookies=cookies)
r.raise_for_status()
assert r.url == ujoin(app.proxy.public_server.host, '/user/river')
assert r.url == user_url(app.users['river'], app)
def test_home_no_auth(app):
r = get_page('home', app, allow_redirects=False)
@@ -62,6 +63,7 @@ def test_admin(app):
r.raise_for_status()
assert r.url.endswith('/admin')
def test_spawn_redirect(app, io_loop):
name = 'wash'
cookies = app.login_user(name)
@@ -100,7 +102,7 @@ def test_spawn_page(app):
def test_spawn_form(app, io_loop):
with mock.patch.dict(app.users.settings, {'spawner_class': FormSpawner}):
base_url = ujoin(app.proxy.public_server.host, app.hub.server.base_url)
base_url = ujoin(public_url(app), app.hub.server.base_url)
cookies = app.login_user('jones')
orm_u = orm.User.find(app.db, 'jones')
u = app.users[orm_u]
@@ -121,7 +123,7 @@ def test_spawn_form(app, io_loop):
def test_spawn_form_with_file(app, io_loop):
with mock.patch.dict(app.users.settings, {'spawner_class': FormSpawner}):
base_url = ujoin(app.proxy.public_server.host, app.hub.server.base_url)
base_url = ujoin(public_url(app), app.hub.server.base_url)
cookies = app.login_user('jones')
orm_u = orm.User.find(app.db, 'jones')
u = app.users[orm_u]
@@ -147,3 +149,98 @@ def test_spawn_form_with_file(app, io_loop):
'content_type': 'application/unknown'},
}
def test_user_redirect(app):
name = 'wash'
cookies = app.login_user(name)
r = get_page('/user/baduser', app, cookies=cookies)
r.raise_for_status()
print(urlparse(r.url))
path = urlparse(r.url).path
assert path == '/user/%s' % name
r = get_page('/user/baduser/test.ipynb', app, cookies=cookies)
r.raise_for_status()
print(urlparse(r.url))
path = urlparse(r.url).path
assert path == '/user/%s/test.ipynb' % name
r = get_page('/user/baduser/test.ipynb', app)
r.raise_for_status()
print(urlparse(r.url))
path = urlparse(r.url).path
assert path == '/hub/login'
query = urlparse(r.url).query
assert query == urlencode({'next': '/hub/user/baduser/test.ipynb'})
def test_login_fail(app):
name = 'wash'
base_url = public_url(app)
r = requests.post(base_url + 'hub/login',
data={
'username': name,
'password': 'wrong',
},
allow_redirects=False,
)
assert not r.cookies
def test_login_redirect(app, io_loop):
cookies = app.login_user('river')
user = app.users['river']
# no next_url, server running
io_loop.run_sync(user.spawn)
r = get_page('login', app, cookies=cookies, allow_redirects=False)
r.raise_for_status()
assert r.status_code == 302
assert '/user/river' in r.headers['Location']
# no next_url, server not running
io_loop.run_sync(user.stop)
r = get_page('login', app, cookies=cookies, allow_redirects=False)
r.raise_for_status()
assert r.status_code == 302
assert '/hub/' in r.headers['Location']
# next URL given, use it
r = get_page('login?next=/hub/admin', app, cookies=cookies, allow_redirects=False)
r.raise_for_status()
assert r.status_code == 302
assert r.headers['Location'].endswith('/hub/admin')
def test_logout(app):
name = 'wash'
cookies = app.login_user(name)
r = requests.get(public_host(app) + app.tornado_settings['logout_url'], cookies=cookies)
r.raise_for_status()
login_url = public_host(app) + app.tornado_settings['login_url']
assert r.url == login_url
assert r.cookies == {}
def test_login_no_whitelist_adds_user(app):
auth = app.authenticator
mock_add_user = mock.Mock()
with mock.patch.object(auth, 'add_user', mock_add_user):
cookies = app.login_user('jubal')
user = app.users['jubal']
assert mock_add_user.mock_calls == [mock.call(user)]
def test_static_files(app):
base_url = ujoin(public_url(app), app.hub.server.base_url)
print(base_url)
r = requests.get(ujoin(base_url, 'logo'))
r.raise_for_status()
assert r.headers['content-type'] == 'image/png'
r = requests.get(ujoin(base_url, 'static', 'images', 'jupyter.png'))
r.raise_for_status()
assert r.headers['content-type'] == 'image/png'
r = requests.get(ujoin(base_url, 'static', 'css', 'style.min.css'))
r.raise_for_status()
assert r.headers['content-type'] == 'text/css'

View File

@@ -4,6 +4,7 @@ import json
import os
from queue import Queue
from subprocess import Popen
from urllib.parse import urlparse
from .. import orm
from .mocking import MockHub
@@ -34,6 +35,8 @@ def test_external_proxy(request, io_loop):
'--api-port', str(proxy_port),
'--default-target', 'http://%s:%i' % (app.hub_ip, app.hub_port),
]
if app.subdomain_host:
cmd.append('--host-routing')
proxy = Popen(cmd, env=env)
def _cleanup_proxy():
if proxy.poll() is None:
@@ -60,7 +63,11 @@ def test_external_proxy(request, io_loop):
r.raise_for_status()
routes = io_loop.run_sync(app.proxy.get_routes)
assert sorted(routes.keys()) == ['/', '/user/river']
user_path = '/user/river'
if app.subdomain_host:
domain = urlparse(app.subdomain_host).hostname
user_path = '/%s.%s' % (name, domain) + user_path
assert sorted(routes.keys()) == ['/', user_path]
# teardown the proxy and start a new one in the same place
proxy.terminate()
@@ -76,7 +83,7 @@ def test_external_proxy(request, io_loop):
# check that the routes are correct
routes = io_loop.run_sync(app.proxy.get_routes)
assert sorted(routes.keys()) == ['/', '/user/river']
assert sorted(routes.keys()) == ['/', user_path]
# teardown the proxy again, and start a new one with different auth and port
proxy.terminate()
@@ -90,13 +97,16 @@ def test_external_proxy(request, io_loop):
'--api-port', str(proxy_port),
'--default-target', 'http://%s:%i' % (app.hub_ip, app.hub_port),
]
if app.subdomain_host:
cmd.append('--host-routing')
proxy = Popen(cmd, env=env)
wait_for_proxy()
# tell the hub where the new proxy is
r = api_request(app, 'proxy', method='patch', data=json.dumps({
'port': proxy_port,
'protocol': 'http',
'ip': app.ip,
'auth_token': new_auth_token,
}))
r.raise_for_status()
@@ -113,7 +123,8 @@ def test_external_proxy(request, io_loop):
# check that the routes are correct
routes = io_loop.run_sync(app.proxy.get_routes)
assert sorted(routes.keys()) == ['/', '/user/river']
assert sorted(routes.keys()) == ['/', user_path]
def test_check_routes(app, io_loop):
proxy = app.proxy
@@ -123,13 +134,24 @@ def test_check_routes(app, io_loop):
r.raise_for_status()
zoe = orm.User.find(app.db, 'zoe')
assert zoe is not None
zoe = app.users[zoe]
before = sorted(io_loop.run_sync(app.proxy.get_routes))
assert '/user/zoe' in before
io_loop.run_sync(app.proxy.check_routes)
assert zoe.proxy_path in before
io_loop.run_sync(lambda : app.proxy.check_routes(app.users))
io_loop.run_sync(lambda : proxy.delete_user(zoe))
during = sorted(io_loop.run_sync(app.proxy.get_routes))
assert '/user/zoe' not in during
io_loop.run_sync(app.proxy.check_routes)
assert zoe.proxy_path not in during
io_loop.run_sync(lambda : app.proxy.check_routes(app.users))
after = sorted(io_loop.run_sync(app.proxy.get_routes))
assert '/user/zoe' in after
assert zoe.proxy_path in after
assert before == after
def test_patch_proxy_bad_req(app):
r = api_request(app, 'proxy', method='patch')
assert r.status_code == 400
r = api_request(app, 'proxy', method='patch', data='notjson')
assert r.status_code == 400
r = api_request(app, 'proxy', method='patch', data=json.dumps([]))
assert r.status_code == 400

View File

@@ -4,9 +4,14 @@
# Distributed under the terms of the Modified BSD License.
import logging
import os
import signal
import sys
import tempfile
import time
from unittest import mock
from tornado import gen
from .. import spawner as spawnermod
from ..spawner import LocalProcessSpawner
@@ -39,13 +44,14 @@ def new_spawner(db, **kwargs):
kwargs.setdefault('INTERRUPT_TIMEOUT', 1)
kwargs.setdefault('TERM_TIMEOUT', 1)
kwargs.setdefault('KILL_TIMEOUT', 1)
kwargs.setdefault('poll_interval', 1)
return LocalProcessSpawner(db=db, **kwargs)
def test_spawner(db, io_loop):
spawner = new_spawner(db)
io_loop.run_sync(spawner.start)
assert spawner.user.server.ip == 'localhost'
assert spawner.user.server.ip == '127.0.0.1'
# wait for the process to get to the while True: loop
time.sleep(1)
@@ -59,7 +65,7 @@ def test_spawner(db, io_loop):
def test_single_user_spawner(db, io_loop):
spawner = new_spawner(db, cmd=['jupyterhub-singleuser'])
io_loop.run_sync(spawner.start)
assert spawner.user.server.ip == 'localhost'
assert spawner.user.server.ip == '127.0.0.1'
# wait for http server to come up,
# checking for early termination every 1s
def wait():
@@ -110,3 +116,53 @@ def test_stop_spawner_stop_now(db, io_loop):
status = io_loop.run_sync(spawner.poll)
assert status == -signal.SIGTERM
def test_spawner_poll(db, io_loop):
first_spawner = new_spawner(db)
user = first_spawner.user
io_loop.run_sync(first_spawner.start)
proc = first_spawner.proc
status = io_loop.run_sync(first_spawner.poll)
assert status is None
user.state = first_spawner.get_state()
assert 'pid' in user.state
# create a new Spawner, loading from state of previous
spawner = new_spawner(db, user=first_spawner.user)
spawner.start_polling()
# wait for the process to get to the while True: loop
io_loop.run_sync(lambda : gen.sleep(1))
status = io_loop.run_sync(spawner.poll)
assert status is None
# kill the process
proc.terminate()
for i in range(10):
if proc.poll() is None:
time.sleep(1)
else:
break
assert proc.poll() is not None
io_loop.run_sync(lambda : gen.sleep(2))
status = io_loop.run_sync(spawner.poll)
assert status is not None
def test_setcwd():
cwd = os.getcwd()
with tempfile.TemporaryDirectory() as td:
td = os.path.realpath(os.path.abspath(td))
spawnermod._try_setcwd(td)
assert os.path.samefile(os.getcwd(), td)
os.chdir(cwd)
chdir = os.chdir
temp_root = os.path.realpath(os.path.abspath(tempfile.gettempdir()))
def raiser(path):
path = os.path.realpath(os.path.abspath(path))
if not path.startswith(temp_root):
raise OSError(path)
chdir(path)
with mock.patch('os.chdir', raiser):
spawnermod._try_setcwd(cwd)
assert os.getcwd().startswith(temp_root)
os.chdir(cwd)

View File

@@ -21,7 +21,7 @@ class Command(List):
kwargs.setdefault('minlen', 1)
if isinstance(default_value, str):
default_value = [default_value]
super().__init__(Unicode, default_value, **kwargs)
super().__init__(Unicode(), default_value, **kwargs)
def validate(self, obj, value):
if isinstance(value, str):

View File

@@ -2,7 +2,7 @@
# Distributed under the terms of the Modified BSD License.
from datetime import datetime, timedelta
from urllib.parse import quote
from urllib.parse import quote, urlparse
from tornado import gen
from tornado.log import app_log
@@ -12,7 +12,7 @@ from sqlalchemy import inspect
from .utils import url_path_join
from . import orm
from traitlets import HasTraits, Any, Dict
from traitlets import HasTraits, Any, Dict, observe, default
from .spawner import LocalProcessSpawner
@@ -38,6 +38,12 @@ class UserDict(dict):
def __getitem__(self, key):
if isinstance(key, User):
key = key.id
elif isinstance(key, str):
orm_user = self.db.query(orm.User).filter(orm.User.name==key).first()
if orm_user is None:
raise KeyError("No such user: %s" % key)
else:
key = orm_user
if isinstance(key, orm.User):
# users[orm_user] returns User(orm_user)
orm_user = key
@@ -69,22 +75,24 @@ class UserDict(dict):
class User(HasTraits):
@default('log')
def _log_default(self):
return app_log
settings = Dict()
db = Any(allow_none=True)
@default('db')
def _db_default(self):
if self.orm_user:
return inspect(self.orm_user).session
def _db_changed(self, name, old, new):
@observe('db')
def _db_changed(self, change):
"""Changing db session reacquires ORM User object"""
# db session changed, re-get orm User
if self.orm_user:
id = self.orm_user.id
self.orm_user = new.query(orm.User).filter(orm.User.id==id).first()
self.orm_user = change['new'].query(orm.User).filter(orm.User.id==id).first()
self.spawner.db = self.db
orm_user = None
@@ -139,6 +147,8 @@ class User(HasTraits):
@property
def running(self):
"""property for whether a user has a running server"""
if self.spawn_pending or self.stop_pending:
return False # server is not running if spawn or stop is still pending
if self.server is None:
return False
return True
@@ -148,6 +158,41 @@ class User(HasTraits):
"""My name, escaped for use in URLs, cookies, etc."""
return quote(self.name, safe='@')
@property
def proxy_path(self):
if self.settings.get('subdomain_host'):
return url_path_join('/' + self.domain, self.server.base_url)
else:
return self.server.base_url
@property
def domain(self):
"""Get the domain for my server."""
# FIXME: escaped_name probably isn't escaped enough in general for a domain fragment
return self.escaped_name + '.' + self.settings['domain']
@property
def host(self):
"""Get the *host* for my server (domain[:port])"""
# FIXME: escaped_name probably isn't escaped enough in general for a domain fragment
parsed = urlparse(self.settings['subdomain_host'])
h = '%s://%s.%s' % (parsed.scheme, self.escaped_name, parsed.netloc)
return h
@property
def url(self):
"""My URL
Full name.domain/path if using subdomains, otherwise just my /base/url
"""
if self.settings.get('subdomain_host'):
return '{host}{path}'.format(
host=self.host,
path=self.base_url,
)
else:
return self.base_url
@gen.coroutine
def spawn(self, options=None):
"""Start the user's spawner"""
@@ -183,7 +228,7 @@ class User(HasTraits):
yield gen.with_timeout(timedelta(seconds=spawner.start_timeout), f)
except Exception as e:
if isinstance(e, gen.TimeoutError):
self.log.warn("{user}'s server failed to start in {s} seconds, giving up".format(
self.log.warning("{user}'s server failed to start in {s} seconds, giving up".format(
user=self.name, s=spawner.start_timeout,
))
e.reason = 'timeout'
@@ -206,11 +251,12 @@ class User(HasTraits):
self.state = spawner.get_state()
self.last_activity = datetime.utcnow()
db.commit()
self.spawn_pending = False
try:
yield self.server.wait_up(http=True, timeout=spawner.http_timeout)
except Exception as e:
if isinstance(e, TimeoutError):
self.log.warn(
self.log.warning(
"{user}'s server never showed up at {url} "
"after {http_timeout} seconds. Giving up".format(
user=self.name,
@@ -232,7 +278,6 @@ class User(HasTraits):
), exc_info=True)
# raise original TimeoutError
raise e
self.spawn_pending = False
return self
@gen.coroutine

View File

@@ -30,22 +30,32 @@ def random_port():
ISO8601_ms = '%Y-%m-%dT%H:%M:%S.%fZ'
ISO8601_s = '%Y-%m-%dT%H:%M:%SZ'
def can_connect(ip, port):
"""Check if we can connect to an ip:port
return True if we can connect, False otherwise.
"""
try:
socket.create_connection((ip, port))
except socket.error as e:
if e.errno not in {errno.ECONNREFUSED, errno.ETIMEDOUT}:
app_log.error("Unexpected error connecting to %s:%i %s",
ip, port, e
)
return False
else:
return True
@gen.coroutine
def wait_for_server(ip, port, timeout=10):
"""wait for any server to show up at ip:port"""
loop = ioloop.IOLoop.current()
tic = loop.time()
while loop.time() - tic < timeout:
try:
socket.create_connection((ip, port))
except socket.error as e:
if e.errno != errno.ECONNREFUSED:
app_log.error("Unexpected error waiting for %s:%i %s",
ip, port, e
)
yield gen.sleep(0.1)
else:
if can_connect(ip, port):
return
else:
yield gen.sleep(0.1)
raise TimeoutError("Server at {ip}:{port} didn't respond in {timeout} seconds".format(
**locals()
))
@@ -68,14 +78,14 @@ def wait_for_http_server(url, timeout=10):
if e.code != 599:
# we expect 599 for no connection,
# but 502 or other proxy error is conceivable
app_log.warn("Server at %s responded with error: %s", url, e.code)
app_log.warning("Server at %s responded with error: %s", url, e.code)
yield gen.sleep(0.1)
else:
app_log.debug("Server at %s responded with %s", url, e.code)
return
except (OSError, socket.error) as e:
if e.errno not in {errno.ECONNABORTED, errno.ECONNREFUSED, errno.ECONNRESET}:
app_log.warn("Failed to connect to %s (%s)", url, e)
app_log.warning("Failed to connect to %s (%s)", url, e)
yield gen.sleep(0.1)
else:
return
@@ -195,35 +205,3 @@ def url_path_join(*pieces):
return result
def localhost():
"""Return localhost or 127.0.0.1"""
if hasattr(localhost, '_localhost'):
return localhost._localhost
binder = connector = None
try:
binder = socket.socket()
binder.bind(('localhost', 0))
binder.listen(1)
port = binder.getsockname()[1]
def accept():
try:
conn, addr = binder.accept()
except ConnectionAbortedError:
pass
else:
conn.close()
t = Thread(target=accept)
t.start()
connector = socket.create_connection(('localhost', port), timeout=10)
t.join(timeout=10)
except (socket.error, socket.gaierror) as e:
warnings.warn("localhost doesn't appear to work, using 127.0.0.1\n%s" % e, RuntimeWarning)
localhost._localhost = '127.0.0.1'
else:
localhost._localhost = 'localhost'
finally:
if binder:
binder.close()
if connector:
connector.close()
return localhost._localhost

View File

@@ -5,8 +5,9 @@
version_info = (
0,
4,
1,
6,
0,
# 'dev',
)
__version__ = '.'.join(map(str, version_info))

11
onbuild/Dockerfile Normal file
View File

@@ -0,0 +1,11 @@
# JupyterHub Dockerfile that loads your jupyterhub_config.py
#
# Adds ONBUILD step to jupyter/jupyterhub to load your juptyerhub_config.py into the image
#
# Derivative images must have jupyterhub_config.py next to the Dockerfile.
FROM jupyterhub/jupyterhub
ONBUILD ADD jupyterhub_config.py /srv/jupyterhub/jupyterhub_config.py
CMD ["jupyterhub", "-f", "/srv/jupyterhub/jupyterhub_config.py"]

10
onbuild/README.md Normal file
View File

@@ -0,0 +1,10 @@
# JupyterHub onbuild image
If you base a Dockerfile on this image:
FROM juptyerhub/jupyterhub-onbuild:0.6
...
then your `jupyterhub_config.py` adjacent to your Dockerfile will be loaded into the image and used by JupyterHub.
This is how the `jupyter/jupyterhub` docker image behaved prior to 0.6.

View File

@@ -1,6 +1,7 @@
traitlets>=4
traitlets>=4.1
tornado>=4.1
jinja2
pamela
statsd
sqlalchemy>=1.0
requests

View File

@@ -1,4 +1,4 @@
#!/usr/bin/env python3
#!/usr/bin/env python
"""Extend regular notebook server to be aware of multiuser things."""
# Copyright (c) Jupyter Development Team.
@@ -17,34 +17,27 @@ from jinja2 import ChoiceLoader, FunctionLoader
from tornado import ioloop
from tornado.web import HTTPError
try:
import notebook
except ImportError:
raise ImportError("JupyterHub single-user server requires notebook >= 4.0")
from IPython.utils.traitlets import (
from traitlets import (
Bool,
Integer,
Unicode,
CUnicode,
)
try:
import notebook
# 4.x
except ImportError:
from IPython.html.notebookapp import NotebookApp, aliases as notebook_aliases
from IPython.html.auth.login import LoginHandler
from IPython.html.auth.logout import LogoutHandler
from notebook.notebookapp import (
NotebookApp,
aliases as notebook_aliases,
flags as notebook_flags,
)
from notebook.auth.login import LoginHandler
from notebook.auth.logout import LogoutHandler
from IPython.html.utils import url_path_join
from distutils.version import LooseVersion as V
import IPython
if V(IPython.__version__) < V('3.0'):
raise ImportError("JupyterHub Requires IPython >= 3.0, found %s" % IPython.__version__)
else:
from notebook.notebookapp import NotebookApp, aliases as notebook_aliases
from notebook.auth.login import LoginHandler
from notebook.auth.logout import LogoutHandler
from notebook.utils import url_path_join
from notebook.utils import url_path_join
# Define two methods to attach to AuthenticatedHandler,
@@ -54,7 +47,7 @@ class JupyterHubLoginHandler(LoginHandler):
@staticmethod
def login_available(settings):
return True
@staticmethod
def verify_token(self, cookie_name, encrypted_cookie):
"""method for token verification"""
@@ -62,7 +55,7 @@ class JupyterHubLoginHandler(LoginHandler):
if encrypted_cookie in cookie_cache:
# we've seen this token before, don't ask upstream again
return cookie_cache[encrypted_cookie]
hub_api_url = self.settings['hub_api_url']
hub_api_key = self.settings['hub_api_key']
r = requests.get(url_path_join(
@@ -85,7 +78,7 @@ class JupyterHubLoginHandler(LoginHandler):
data = r.json()
cookie_cache[encrypted_cookie] = data
return data
@staticmethod
def get_user(self):
"""alternative get_current_user to query the central server"""
@@ -94,7 +87,7 @@ class JupyterHubLoginHandler(LoginHandler):
# since this may be called again when trying to render the error page
if hasattr(self, '_cached_user'):
return self._cached_user
self._cached_user = None
my_user = self.settings['user']
encrypted_cookie = self.get_cookie(self.cookie_name)
@@ -116,7 +109,9 @@ class JupyterHubLoginHandler(LoginHandler):
class JupyterHubLogoutHandler(LogoutHandler):
def get(self):
self.redirect(url_path_join(self.settings['hub_prefix'], 'logout'))
self.redirect(
self.settings['hub_host'] +
url_path_join(self.settings['hub_prefix'], 'logout'))
# register new hub related command-line aliases
@@ -125,9 +120,18 @@ aliases.update({
'user' : 'SingleUserNotebookApp.user',
'cookie-name': 'SingleUserNotebookApp.cookie_name',
'hub-prefix': 'SingleUserNotebookApp.hub_prefix',
'hub-host': 'SingleUserNotebookApp.hub_host',
'hub-api-url': 'SingleUserNotebookApp.hub_api_url',
'base-url': 'SingleUserNotebookApp.base_url',
})
flags = dict(notebook_flags)
flags.update({
'disable-user-config': ({
'SingleUserNotebookApp': {
'disable_user_config': True
}
}, "Disable user-controlled configuration of the notebook server.")
})
page_template = """
{% extends "templates/page.html" %}
@@ -141,8 +145,21 @@ page_template = """
>
Control Panel</a>
{% endblock %}
{% block logo %}
<img src='{{logo_url}}' alt='Jupyter Notebook'/>
{% endblock logo %}
"""
def _exclude_home(path_list):
"""Filter out any entries in a path list that are in my home directory.
Used to disable per-user configuration.
"""
home = os.path.expanduser('~')
for p in path_list:
if not p.startswith(home):
yield p
class SingleUserNotebookApp(NotebookApp):
"""A Subclass of the regular NotebookApp that is aware of the parent multiuser context."""
user = CUnicode(config=True)
@@ -150,12 +167,23 @@ class SingleUserNotebookApp(NotebookApp):
self.log.name = new
cookie_name = Unicode(config=True)
hub_prefix = Unicode(config=True)
hub_host = Unicode(config=True)
hub_api_url = Unicode(config=True)
aliases = aliases
flags = flags
open_browser = False
trust_xheaders = True
login_handler_class = JupyterHubLoginHandler
logout_handler_class = JupyterHubLogoutHandler
port_retries = 0 # disable port-retries, since the Spawner will tell us what port to use
disable_user_config = Bool(False, config=True,
help="""Disable user configuration of single-user server.
Prevents user-writable files that normally configure the single-user server
from being loaded, ensuring admins have full control of configuration.
"""
)
cookie_cache_lifetime = Integer(
config=True,
@@ -182,7 +210,37 @@ class SingleUserNotebookApp(NotebookApp):
def _clear_cookie_cache(self):
self.log.debug("Clearing cookie cache")
self.tornado_settings['cookie_cache'].clear()
def migrate_config(self):
if self.disable_user_config:
# disable config-migration when user config is disabled
return
else:
super(SingleUserNotebookApp, self).migrate_config()
@property
def config_file_paths(self):
path = super(SingleUserNotebookApp, self).config_file_paths
if self.disable_user_config:
# filter out user-writable config dirs if user config is disabled
path = list(_exclude_home(path))
return path
@property
def nbextensions_path(self):
path = super(SingleUserNotebookApp, self).nbextensions_path
if self.disable_user_config:
path = list(_exclude_home(path))
return path
def _static_custom_path_default(self):
path = super(SingleUserNotebookApp, self)._static_custom_path_default()
if self.disable_user_config:
path = list(_exclude_home(path))
return path
def start(self):
# Start a PeriodicCallback to clear cached cookies. This forces us to
# revalidate our user with the Hub at least every
@@ -193,7 +251,7 @@ class SingleUserNotebookApp(NotebookApp):
self.cookie_cache_lifetime * 1e3,
).start()
super(SingleUserNotebookApp, self).start()
def init_webapp(self):
# load the hub related settings into the tornado settings dict
env = os.environ
@@ -202,26 +260,28 @@ class SingleUserNotebookApp(NotebookApp):
s['user'] = self.user
s['hub_api_key'] = env.pop('JPY_API_TOKEN')
s['hub_prefix'] = self.hub_prefix
s['hub_host'] = self.hub_host
s['cookie_name'] = self.cookie_name
s['login_url'] = self.hub_prefix
s['login_url'] = self.hub_host + self.hub_prefix
s['hub_api_url'] = self.hub_api_url
s['csp_report_uri'] = url_path_join(self.hub_prefix, 'security/csp-report')
s['csp_report_uri'] = self.hub_host + url_path_join(self.hub_prefix, 'security/csp-report')
super(SingleUserNotebookApp, self).init_webapp()
self.patch_templates()
def patch_templates(self):
"""Patch page templates to add Hub-related buttons"""
self.jinja_template_vars['logo_url'] = self.hub_host + url_path_join(self.hub_prefix, 'logo')
env = self.web_app.settings['jinja2_env']
env.globals['hub_control_panel_url'] = \
url_path_join(self.hub_prefix, 'home')
self.hub_host + url_path_join(self.hub_prefix, 'home')
# patch jinja env loading to modify page template
def get_page(name):
if name == 'page.html':
return page_template
orig_loader = env.loader
env.loader = ChoiceLoader([
FunctionLoader(get_page),

View File

@@ -166,7 +166,7 @@ class Bower(BaseCommand):
if self.should_run_npm():
print("installing build dependencies with npm")
check_call(['npm', 'install'], cwd=here)
check_call(['npm', 'install', '--progress=false'], cwd=here)
os.utime(self.node_modules)
env = os.environ.copy()

View File

@@ -10,7 +10,7 @@
{% endif %}
<a id="start" class="btn btn-lg btn-success"
{% if user.running %}
href="{{base_url}}user/{{user.name}}/"
href="{{ user.url }}"
{% else %}
href="{{base_url}}spawn"
{% endif %}

View File

@@ -82,7 +82,7 @@
<div id="header" class="navbar navbar-static-top">
<div class="container">
<span id="jupyterhub-logo" class="pull-left"><a href="{{base_url}}"><img src='{{static_url("images/jupyter.png") }}' alt='JupyterHub' class='jpy-logo' title='Home'/></a></span>
<span id="jupyterhub-logo" class="pull-left"><a href="{{base_url}}"><img src='{{base_url}}logo' alt='JupyterHub' class='jpy-logo' title='Home'/></a></span>
{% block login_widget %}

View File

@@ -133,6 +133,7 @@ def untag(vs, push=False):
v2 = parse_vs(vs)
v2.append('dev')
v2[1] += 1
v2[2] = 0
vs2 = unparse_vs(v2)
patch_version(vs2, repo_root)
with cd(repo_root):