fee4ee23c0
**Environment**
* image: k8s-hub (`jupyterhub/k8s-hub:0.11.1`);
* `authenticator_class: dummy`;
* db: cocroachdb (`sqlalchemy-cocroachdb`).
**Description:**
`save_bearer_token` method (`provider.py`) passes a float value to the `expires_at` field (int).
A user can create a notebook, it gets successfully scheduled, and then, once the pod is up and ready, the user is unable to enter the notebook, because jupyterhub cannot save a token. In logs, we can see the following:
```
[I 2021-05-29 14:45:04.302 JupyterHub log:181] 302 GET /hub/api/oauth2/authorize?client_id=jupyterhub-user-user2&redirect_uri=%2Fuser%2Fuser2%2Foauth_callback&response_type=code&state=[secret] -> /user/user2/oauth_callback?code=[secret]&state=[secret] (user2 40.113.125.116) 73.98ms
[E 2021-05-29 14:45:04.424 JupyterHub web:1789] Uncaught exception POST /hub/api/oauth2/token (10.42.80.10)
HTTPServerRequest(protocol='http', host='hub:8081', method='POST', uri='/hub/api/oauth2/token', version='HTTP/1.1', remote_ip='10.42.80.10')
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/tornado/web.py", line 1702, in _execute
result = method(*self.path_args, **self.path_kwargs)
File "/usr/local/lib/python3.8/dist-packages/jupyterhub/apihandlers/auth.py", line 324, in post
headers, body, status = self.oauth_provider.create_token_response(
File "/usr/local/lib/python3.8/dist-packages/oauthlib/oauth2/rfc6749/endpoints/base.py", line 116, in wrapper
return f(endpoint, uri, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/oauthlib/oauth2/rfc6749/endpoints/token.py", line 118, in create_token_response
return grant_type_handler.create_token_response(
File "/usr/local/lib/python3.8/dist-packages/oauthlib/oauth2/rfc6749/grant_types/authorization_code.py", line 313, in create_token_response
self.request_validator.save_token(token, request)
File "/usr/local/lib/python3.8/dist-packages/jupyterhub/oauth/provider.py", line 281, in save_token
return self.save_bearer_token(token, request, *args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/jupyterhub/oauth/provider.py", line 354, in save_bearer_token
self.db.commit()
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1042, in commit
self.transaction.commit()
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 504, in commit
self._prepare_impl()
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 483, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 2536, in flush
self._flush(objects)
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 2678, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__
compat.raise_(
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 2638, in _flush
flush_context.execute()
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 586, in execute
persistence.save_obj(
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 239, in save_obj
_emit_insert_statements(
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1135, in _emit_insert_statements
result = cached_connections[connection].execute(
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1316, in _execute_context
self._handle_dbapi_exception(
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
util.raise_(
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 593, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.DatatypeMismatch) value type decimal doesn't match type int of column "expires_at"
HINT: you will need to rewrite or cast the expression
[SQL: INSERT INTO oauth_access_tokens (client_id, grant_type, expires_at, refresh_token, refresh_expires_at, user_id, session_id, hashed, prefix, created, last_activity) VALUES (%(client_id)s, %(grant_type)s, %(expires_at)s, %(refresh_token)s, %(refresh_expires_at)s, %(user_id)s, %(session_id)s, %(hashed)s, %(prefix)s, %(created)s, %(last_activity)s) RETURNING oauth_access_tokens.id]
[parameters: {'client_id': 'jupyterhub-user-user2', 'grant_type': 'authorization_code', 'expires_at': 1622303104.418992, 'refresh_token': 'FVJ8S4is0367LlEMnxIiEIoTOeoxhf', 'refresh_expires_at': None, 'user_id': 662636890939424770, 'session_id': '4e041a2bfcb34a34a00033a281bc1236', 'hashed': 'sha512:1:3b18deae37fbf50a:03df035736960af14e19196e1d13fd74f55c21f17405119f80e75817ff37c7567fab089a3d40b97a57f94b54065ee56f7260895352516b9facb989d656f05be8', 'prefix': 't11z', 'created': datetime.datetime(2021, 5, 29, 14, 45, 4, 421305), 'last_activity': None}]
(Background on this error at: http://sqlalche.me/e/13/f405)
[W 2021-05-29 14:45:04.430 JupyterHub base:110] Rolling back session due to database error (psycopg2.errors.DatatypeMismatch) value type decimal doesn't match type int of column "expires_at"
HINT: you will need to rewrite or cast the expression
[SQL: INSERT INTO oauth_access_tokens (client_id, grant_type, expires_at, refresh_token, refresh_expires_at, user_id, session_id, hashed, prefix, created, last_activity) VALUES (%(client_id)s, %(grant_type)s, %(expires_at)s, %(refresh_token)s, %(refresh_expires_at)s, %(user_id)s, %(session_id)s, %(hashed)s, %(prefix)s, %(created)s, %(last_activity)s) RETURNING oauth_access_tokens.id]
[parameters: {'client_id': 'jupyterhub-user-user2', 'grant_type': 'authorization_code', 'expires_at': 1622303104.418992, 'refresh_token': 'FVJ8S4is0367LlEMnxIiEIoTOeoxhf', 'refresh_expires_at': None, 'user_id': 662636890939424770, 'session_id': '4e041a2bfcb34a34a00033a281bc1236', 'hashed': 'sha512:1:3b18deae37fbf50a:03df035736960af14e19196e1d13fd74f55c21f17405119f80e75817ff37c7567fab089a3d40b97a57f94b54065ee56f7260895352516b9facb989d656f05be8', 'prefix': 't11z', 'created': datetime.datetime(2021, 5, 29, 14, 45, 4, 421305), 'last_activity': None}]
(Background on this error at: http://sqlalche.me/e/13/f405)
[E 2021-05-29 14:45:04.443 JupyterHub log:173] {
"Host": "hub:8081",
"User-Agent": "python-requests/2.25.1",
"Accept-Encoding": "gzip, deflate",
"Accept": "*/*",
"Connection": "keep-alive",
"Content-Type": "application/x-www-form-urlencoded",
"Authorization": "token [secret]",
"Content-Length": "190"
}
[E 2021-05-29 14:45:04.444 JupyterHub log:181] 500 POST /hub/api/oauth2/token (user2 10.42.80.10) 63.28ms
```
Everything went well, when I changed:
`expires_at=orm.OAuthAccessToken.now() + token['expires_in'],`
to:
`expires_at=int(orm.OAuthAccessToken.now() + token['expires_in']),`
That's what this PR is about.
As a sidenote, `black` formatter adjusted the `orm_client = orm.OAuthClient(identifier=client_id,)` line, but I guess it should be fine. Please, feel free to revert this change if needed.
(Upd): added the missing `int` conversion.
Signed-off-by: Erik Sundell <erik.i.sundell@gmail.com>
Technical Overview | Installation | Configuration | Docker | Contributing | License | Help and Resources
JupyterHub
With JupyterHub you can create a multi-user Hub which spawns, manages, and proxies multiple instances of the single-user Jupyter notebook server.
Project Jupyter created JupyterHub to support many users. The Hub can offer notebook servers to a class of students, a corporate data science workgroup, a scientific research project, or a high performance computing group.
Technical overview
Three main actors make up JupyterHub:
- multi-user Hub (tornado process)
- configurable http proxy (node-http-proxy)
- multiple single-user Jupyter notebook servers (Python/Jupyter/tornado)
Basic principles for operation are:
- Hub launches a proxy.
- Proxy forwards all requests to Hub by default.
- Hub handles login, and spawns single-user servers on demand.
- Hub configures proxy to forward url prefixes to the single-user notebook servers.
JupyterHub also provides a REST API for administration of the Hub and its users.
Installation
Check prerequisites
-
A Linux/Unix based system
-
Python 3.5 or greater
-
-
If you are using
conda, the nodejs and npm dependencies will be installed for you by conda. -
If you are using
pip, install a recent version of nodejs/npm. For example, install it on Linux (Debian/Ubuntu) using:sudo apt-get install npm nodejs-legacyThe
nodejs-legacypackage installs thenodeexecutable and is currently required for npm to work on Debian/Ubuntu.
-
-
If using the default PAM Authenticator, a pluggable authentication module (PAM).
-
TLS certificate and key for HTTPS communication
-
Domain name
Install packages
Using conda
To install JupyterHub along with its dependencies including nodejs/npm:
conda install -c conda-forge jupyterhub
If you plan to run notebook servers locally, install the Jupyter notebook or JupyterLab:
conda install notebook
conda install jupyterlab
Using pip
JupyterHub can be installed with pip, and the proxy with npm:
npm install -g configurable-http-proxy
python3 -m pip install jupyterhub
If you plan to run notebook servers locally, you will need to install the Jupyter notebook package:
python3 -m pip install --upgrade notebook
Run the Hub server
To start the Hub server, run the command:
jupyterhub
Visit https://localhost:8000 in your browser, and sign in with your unix
PAM credentials.
Note: To allow multiple users to sign into the server, you will need to
run the jupyterhub command as a privileged user, such as root.
The wiki
describes how to run the server as a less privileged user, which requires
more configuration of the system.
Configuration
The Getting Started section of the documentation explains the common steps in setting up JupyterHub.
The JupyterHub tutorial provides an in-depth video and sample configurations of JupyterHub.
Create a configuration file
To generate a default config file with settings and descriptions:
jupyterhub --generate-config
Start the Hub
To start the Hub on a specific url and port 10.0.1.2:443 with https:
jupyterhub --ip 10.0.1.2 --port 443 --ssl-key my_ssl.key --ssl-cert my_ssl.cert
Authenticators
| Authenticator | Description |
|---|---|
| PAMAuthenticator | Default, built-in authenticator |
| OAuthenticator | OAuth + JupyterHub Authenticator = OAuthenticator |
| ldapauthenticator | Simple LDAP Authenticator Plugin for JupyterHub |
| kerberosauthenticator | Kerberos Authenticator Plugin for JupyterHub |
Spawners
| Spawner | Description |
|---|---|
| LocalProcessSpawner | Default, built-in spawner starts single-user servers as local processes |
| dockerspawner | Spawn single-user servers in Docker containers |
| kubespawner | Kubernetes spawner for JupyterHub |
| sudospawner | Spawn single-user servers without being root |
| systemdspawner | Spawn single-user notebook servers using systemd |
| batchspawner | Designed for clusters using batch scheduling software |
| yarnspawner | Spawn single-user notebook servers distributed on a Hadoop cluster |
| wrapspawner | WrapSpawner and ProfilesSpawner enabling runtime configuration of spawners |
Docker
A starter docker image for JupyterHub gives a baseline deployment of JupyterHub using Docker.
Important: This jupyterhub/jupyterhub image contains only the Hub itself,
with no configuration. In general, one needs to make a derivative image, with
at least a jupyterhub_config.py setting up an Authenticator and/or a Spawner.
To run the single-user servers, which may be on the same system as the Hub or
not, Jupyter Notebook version 4 or greater must be installed.
The JupyterHub docker image can be started with the following command:
docker run -p 8000:8000 -d --name jupyterhub jupyterhub/jupyterhub jupyterhub
This command will create a container named jupyterhub that you can
stop and resume with docker stop/start.
The Hub service will be listening on all interfaces at port 8000, which makes this a good choice for testing JupyterHub on your desktop or laptop.
If you want to run docker on a computer that has a public IP then you should (as in MUST) secure it with ssl by adding ssl options to your docker configuration or by using a ssl enabled proxy.
Mounting volumes will allow you to store data outside the docker image (host system) so it will be persistent, even when you start a new image.
The command docker exec -it jupyterhub bash will spawn a root shell in your docker
container. You can use the root shell to create system users in the container.
These accounts will be used for authentication in JupyterHub's default configuration.
Contributing
If you would like to contribute to the project, please read our
contributor documentation
and the CONTRIBUTING.md. The CONTRIBUTING.md file
explains how to set up a development installation, how to run the test suite,
and how to contribute to documentation.
For a high-level view of the vision and next directions of the project, see the JupyterHub community roadmap.
A note about platform support
JupyterHub is supported on Linux/Unix based systems.
JupyterHub officially does not support Windows. You may be able to use JupyterHub on Windows if you use a Spawner and Authenticator that work on Windows, but the JupyterHub defaults will not. Bugs reported on Windows will not be accepted, and the test suite will not run on Windows. Small patches that fix minor Windows compatibility issues (such as basic installation) may be accepted, however. For Windows-based systems, we would recommend running JupyterHub in a docker container or Linux VM.
Additional Reference: Tornado's documentation on Windows platform support
License
We use a shared copyright model that enables all contributors to maintain the copyright on their contributions.
All code is licensed under the terms of the revised BSD license.
Help and resources
We encourage you to ask questions on the Jupyter mailing list. To participate in development discussions or get help, talk with us on our JupyterHub Gitter channel.
- Reporting Issues
- JupyterHub tutorial
- Documentation for JupyterHub | PDF (latest) | PDF (stable)
- Documentation for JupyterHub's REST API
- Documentation for Project Jupyter | PDF
- Project Jupyter website
- Project Jupyter community
JupyterHub follows the Jupyter Community Guides.
Technical Overview | Installation | Configuration | Docker | Contributing | License | Help and Resources