sync with main

.github/workflows/release.yml

@@ -149,7 +149,7 @@ jobs:
           branchRegex: ^\w[\w-.]*$
 
       - name: Build and push jupyterhub
-        uses: docker/build-push-action@37abcedcc1da61a57767b7588cb9d03eb57e28b3
+        uses: docker/build-push-action@3b5e8027fcad23fda98b2e3ac259d8d67585f671
         with:
           context: .
           platforms: linux/amd64,linux/arm64
@@ -170,7 +170,7 @@ jobs:
           branchRegex: ^\w[\w-.]*$
 
       - name: Build and push jupyterhub-onbuild
-        uses: docker/build-push-action@37abcedcc1da61a57767b7588cb9d03eb57e28b3
+        uses: docker/build-push-action@3b5e8027fcad23fda98b2e3ac259d8d67585f671
         with:
           build-args: |
             BASE_IMAGE=${{ fromJson(steps.jupyterhubtags.outputs.tags)[0] }}
@@ -191,7 +191,7 @@ jobs:
           branchRegex: ^\w[\w-.]*$
 
       - name: Build and push jupyterhub-demo
-        uses: docker/build-push-action@37abcedcc1da61a57767b7588cb9d03eb57e28b3
+        uses: docker/build-push-action@3b5e8027fcad23fda98b2e3ac259d8d67585f671
         with:
           build-args: |
             BASE_IMAGE=${{ fromJson(steps.onbuildtags.outputs.tags)[0] }}
@@ -215,7 +215,7 @@ jobs:
           branchRegex: ^\w[\w-.]*$
 
       - name: Build and push jupyterhub/singleuser
-        uses: docker/build-push-action@37abcedcc1da61a57767b7588cb9d03eb57e28b3
+        uses: docker/build-push-action@3b5e8027fcad23fda98b2e3ac259d8d67585f671
         with:
           build-args: |
             JUPYTERHUB_VERSION=${{ github.ref_type == 'tag' && github.ref_name || format('git:{0}', github.sha) }}

.pre-commit-config.yaml

@@ -24,7 +24,7 @@ repos:
 
   # Autoformat: Python code
   - repo: https://github.com/PyCQA/autoflake
-    rev: v2.0.0
+    rev: v2.0.1
     hooks:
       - id: autoflake
         # args ref: https://github.com/PyCQA/autoflake#advanced-usage
@@ -39,7 +39,7 @@ repos:
 
   # Autoformat: Python code
   - repo: https://github.com/psf/black
-    rev: 22.12.0
+    rev: 23.1.0
     hooks:
       - id: black
 

docs/source/changelog.md

@@ -1,3 +1,5 @@
+(changelog)=
+
 # Changelog
 
 For detailed changes from the prior release, click on the version number, and
@@ -205,7 +207,7 @@ More info in {ref}`available-scopes-target`.
 
 - The admin UI can now show more detailed info about users and their servers in a drop-down details table:
 
-  ![Details view in admin UI](./images/dropdown-details-3.0.png)
+  ![Details view in admin UI](/images/dropdown-details-3.0.png)
 
 - Several bugfixes and improvements in the new admin UI.
 - Direct access to the Hub's database is deprecated.
@@ -1353,7 +1355,7 @@ whether it was through discussion, testing, documentation, or development.
 - There is now full UI support for managing named servers.
   With named servers, each jupyterhub user may have access to more than one named server. For example, a professor may access a server named `research` and another named `teaching`.
 
-  ![named servers on the home page](./images/named-servers-home.png)
+  ![named servers on the home page](/images/named-servers-home.png)
 
 - Authenticators can now expire and refresh authentication data by implementing
   `Authenticator.refresh_user(user)`.

docs/source/contributing/security.md

@@ -1,7 +1,7 @@
 # Reporting security issues in Jupyter or JupyterHub
 
 If you find a security vulnerability in Jupyter or JupyterHub,
-whether it is a failure of the security model described in {doc}`../reference/websecurity`
+whether it is a failure of the security model described in [Security Overview](web-security)
 or a failure in implementation,
 please report it to <mailto:security@ipython.org>.
 

docs/source/contributing/setup.md

@@ -103,7 +103,7 @@ a more detailed discussion.
    The default database engine is `sqlite` so if you are just trying
    to get up and running quickly for local development that should be
    available via [Python](https://docs.python.org/3.5/library/sqlite3.html).
-   See {doc}`/reference/database` for details on other supported databases.
+   See [The Hub's Database](hub-database) for details on other supported databases.
 
 6. You are now ready to start JupyterHub!
 

docs/source/explanation/admin/capacity-planning.md

@@ -40,7 +40,7 @@ The rest is going to be up to your users.
 Per-user overhead from JupyterHub is typically negligible
 up to at least a few hundred concurrent active users.
 
-```{figure} ../images/mybinder-hub-components-cpu-memory.png
+```{figure} /images/mybinder-hub-components-cpu-memory.png
 JupyterHub component resource usage for mybinder.org.
 ```
 
@@ -200,7 +200,7 @@ The limit here is actually Kubernetes' pods per node, not memory _or_ CPU.
 This is likely a extreme case, as many Binder users come from clicking links on webpages
 without any actual intention of running code.
 
-```{figure} ../images/mybinder-load5.png
+```{figure} /images/mybinder-load5.png
 mybinder.org node CPU usage is low with 50-150 users sharing just 8 cores
 ```
 
@@ -277,7 +277,7 @@ showing >90% of users using less than 10% CPU and 200MB,
 but a few outliers near the limit of 1 CPU and 2GB of RAM.
 This is the kind of information you can use to tune your requests and limits.
 
-![Snapshot from JupyterHub's Grafana dashboards on mybinder.org](../images/mybinder-user-resources.png)
+![Snapshot from JupyterHub's Grafana dashboards on mybinder.org](/images/mybinder-user-resources.png)
 
 [prometheus]: https://prometheus.io
 [grafana]: https://grafana.com

docs/source/explanation/admin/database.md

@@ -1,3 +1,5 @@
+(hub-database)=
+
 # The Hub's Database
 
 JupyterHub uses a database to store information about users, services, and other data needed for operating the Hub.
@@ -80,7 +82,7 @@ Additionally, there is usually _very_ little load on the database itself.
 By far the most taxing activity on the database is the 'list all users' endpoint, primarily used by the [idle-culling service](https://github.com/jupyterhub/jupyterhub-idle-culler).
 Database-based optimizations have been added to make even these operations feasible for large numbers of users:
 
-1. State filtering on [GET /users](./rest-api.md) with `?state=active`,
+1. State filtering on [GET /users](jupyterhub-rest-API) with `?state=active`,
    which limits the number of results in the query to only the relevant subset (added in JupyterHub 1.3), rather than all users.
 2. [Pagination](api-pagination) of all list endpoints, allowing the request of a large number of resources to be more fairly balanced with other Hub activities across multiple requests (added in 2.0).
 

docs/source/explanation/admin/oauth.md

@@ -255,7 +255,7 @@ To authenticate this request, the single token stored in the encrypted cookie is
 
 If the user model matches who should be allowed (e.g. Danez),
 then the request is allowed.
-See {doc}`../rbac/scopes` for how JupyterHub uses scopes to determine authorized access to servers and services.
+See [Scopes in JupyterHub](jupyterhub-scopes) for how JupyterHub uses scopes to determine authorized access to servers and services.
 
 _the end_
 

docs/source/explanation/admin/websecurity.md

@@ -1,3 +1,5 @@
+(web-security)=
+
 # Security Overview
 
 The **Security Overview** section helps you learn about:

docs/source/explanation/index.md

@@ -1,8 +1,30 @@
 # Explanation
 
-The explanation guides are written to provide big-picture explanations of how JupyterHub works. They are meant to build your understanding of particular topics.
+_Explanation_ documentation provide big-picture descriptions of how JupyterHub works. This section is meant to build your understanding of particular topics.
+
+## Administration
+
+This section provides information relevant to running your own JupyterHub over time.
+
+```{toctree}
+:maxdepth: 1
+
+admin/capacity-planning
+admin/database
+admin/websecurity
+admin/oauth
+```
+
+## JupyterHub RBAC
+
+This section covers how Role Based Access Control (RBAC) is implemented in JupyterHub to control access to Jupyterhub's API resources.
+
+<!---
+The JupyterHub RBAC source files are contained in the source/rbac folder
+--->
 
 ```{toctree}
 :maxdepth: 2
 
+../rbac/index
 ```

docs/source/getting-started/index.md

@@ -1,17 +0,0 @@
-# Get Started
-
-This section covers how to configure and customize JupyterHub for your
-needs. It contains information about authentication, networking, security, and
-other topics that are relevant to individuals or organizations deploying their
-own JupyterHub.
-
-```{toctree}
-:maxdepth: 2
-
-config-basics
-networking-basics
-security-basics
-authenticators-users-basics
-spawners-basics
-services-basics
-```

docs/source/index-admin.md

@@ -1,11 +0,0 @@
-# Administrator's Guide
-
-This guide covers best-practices, tips, common questions and operations, as
-well as other information relevant to running your own JupyterHub over time.
-
-```{toctree}
-:maxdepth: 2
-
-admin/capacity-planning
-changelog
-```

docs/source/index.md

@@ -55,8 +55,8 @@ Documentation sections (reorganization in-progress)
 
 tutorial/index.md
 howto/index.md
-explanation/index.md
 reference/index.md
+explanation/index.md
 faq/index.md
 ```
 
@@ -81,14 +81,6 @@ Today, you can find two main use cases:
 _It is important to evaluate these distributions before you can continue with the
 configuration of JupyterHub_.
 
-### Getting Started
-
-```{toctree}
-:maxdepth: 2
-
-getting-started/index
-```
-
 ### Technical Reference
 
 ```{toctree}
@@ -97,14 +89,6 @@ getting-started/index
 reference/index
 ```
 
-### Administrators guide
-
-```{toctree}
-:maxdepth: 2
-
-index-admin
-```
-
 ### API Reference
 
 ```{toctree}
@@ -113,14 +97,6 @@ index-admin
 api/index
 ```
 
-### RBAC Reference
-
-```{toctree}
-:maxdepth: 2
-
-rbac/index
-```
-
 ### Contributing
 
 We welcome you to contribute to JupyterHub in ways that are most exciting

docs/source/rbac/index.md

@@ -1,4 +1,9 @@
-(RBAC)=
+<!---
+RBAC docs are part of the Explanation section of the JupyterHub documentation.
+As a result, this index file is referenced in the toctree within the explanation/index.md file.
+--->
+
+(rbac)=
 
 # JupyterHub RBAC
 

docs/source/rbac/scopes.md

@@ -1,8 +1,10 @@
+(jupyterhub-scopes)=
+
 # Scopes in JupyterHub
 
 A scope has a syntax-based design that reveals which resources it provides access to. Resources are objects with a type, associated data, relationships to other resources, and a set of methods that operate on them (see [RESTful API](https://restful-api-design.readthedocs.io/en/latest/resources.html) documentation for more information).
 
-`<resource>` in the RBAC scope design refers to the resource name in the [JupyterHub's API](../reference/rest-api.md) endpoints in most cases. For instance, `<resource>` equal to `users` corresponds to JupyterHub's API endpoints beginning with _/users_.
+`<resource>` in the RBAC scope design refers to the resource name in the [JupyterHub's API](jupyterhub-rest-API) endpoints in most cases. For instance, `<resource>` equal to `users` corresponds to JupyterHub's API endpoints beginning with _/users_.
 
 (scope-conventions-target)=
 
@@ -298,6 +300,6 @@ Custom scope _filters_ are NOT supported.
 
 ### Scopes and APIs
 
-The scopes are also listed in the [](../reference/rest-api.md) documentation. Each API endpoint has a list of scopes which can be used to access the API; if no scopes are listed, the API is not authenticated and can be accessed without any permissions (i.e., no scopes).
+The scopes are also listed in the [](jupyterhub-rest-API) documentation. Each API endpoint has a list of scopes which can be used to access the API; if no scopes are listed, the API is not authenticated and can be accessed without any permissions (i.e., no scopes).
 
 Listed scopes by each API endpoint reflect the "lowest" permissions required to gain any access to the corresponding API. For example, posting user's activity (_POST /users/:name/activity_) needs `users:activity` scope. If scope `users` is passed during the request, the access will be granted as the required scope is a subscope of the `users` scope. If, on the other hand, `read:users:activity` scope is passed, the access will be denied.

docs/source/rbac/tech-implementation.md

@@ -65,7 +65,7 @@ If the token's scopes are a subset of the token owner's scopes, the token is iss
 
 {ref}`Figure 1 <token-request-chart>` below illustrates the steps involved. The orange rectangles highlight where in the process the roles and scopes are resolved.
 
-```{figure} ../images/rbac-token-request-chart.png
+```{figure} /images/rbac-token-request-chart.png
 :align: center
 :name: token-request-chart
 
@@ -91,7 +91,7 @@ The passed scopes are compared to the scopes required to access the API as follo
 
 {ref}`Figure 2 <api-request-chart>` illustrates this process highlighting the steps where the role and scope resolutions as well as filtering occur in orange.
 
-```{figure} ../images/rbac-api-request-chart.png
+```{figure} /images/rbac-api-request-chart.png
 :align: center
 :name: api-request-chart
 

docs/source/rbac/upgrade.md

@@ -45,7 +45,7 @@ OAuth token is issued by the Hub to a single-user server when the user logs in.
 
 API token is issued by the Hub to a single-user server when launched and is used to communicate with the Hub's APIs such as posting activity or completing the OAuth flow. This token has no expiry by default.
 
-API tokens can also be issued to users via API ([_/hub/token_](../reference/urls.md) or [_POST /users/:username/tokens_](../reference/rest-api.md)) and services via `jupyterhub_config.py` to perform API requests.
+API tokens can also be issued to users via API ([_/hub/token_](jupyterhub-url) or [_POST /users/:username/tokens_](jupyterhub-rest-API)) and services via `jupyterhub_config.py` to perform API requests.
 
 ### With RBAC
 

docs/source/rbac/use-cases.md

@@ -3,13 +3,13 @@
 To determine which scopes a role should have, one can follow these steps:
 
 1. Determine what actions the role holder should have/have not access to
-2. Match the actions against the [JupyterHub's APIs](../reference/rest-api.md)
+2. Match the actions against the [JupyterHub's APIs](jupyterhub-rest-API)
 3. Check which scopes are required to access the APIs
 4. Combine scopes and subscopes if applicable
 5. Customize the scopes with filters if needed
 6. Define the role with required scopes and assign to users/services/groups/tokens
 
-Below, different use cases are presented on how to use the [RBAC framework](./index.md)
+Below, different use cases are presented on how to use the [RBAC framework](rbac)
 
 ## Service to cull idle servers
 

docs/source/reference/index.md

@@ -1,3 +1,5 @@
+(reference-index)=
+
 # Technical Reference
 
 This section covers more of the details of the JupyterHub architecture, as well as
@@ -8,15 +10,11 @@ what happens under-the-hood when you deploy and configure your JupyterHub.
 
 technical-overview
 urls
-websecurity
 authenticators
 spawners
 services
 rest-api
-server-api
 monitoring
-database
 ../events/index
 config-reference
-oauth
 ```

docs/source/reference/technical-overview.md

@@ -110,7 +110,7 @@ working directory:
   This file needs to persist so that a **Hub** server restart will avoid
   invalidating cookies. Conversely, deleting this file and restarting the server
   effectively invalidates all login cookies. The cookie secret file is discussed
-  in the [Cookie Secret section of the Security Settings document](../getting-started/security-basics.md).
+  in the [Cookie Secret section of the Security Settings document](security-basics).
 
 The location of these files can be specified via configuration settings. It is
 recommended that these files be stored in standard UNIX filesystem locations,

docs/source/reference/urls.md

@@ -1,3 +1,5 @@
+(jupyterhub-url)=
+
 # JupyterHub URL scheme
 
 This document describes how JupyterHub routes requests.

docs/source/tutorial/api/server-api.md

@@ -252,7 +252,7 @@ data: {"progress": 100, "ready": true, "message": "Server ready at /user/test-us
 
 Here is a Python example for consuming an event stream:
 
-```{literalinclude} ../../../examples/server-api/start-stop-server.py
+```{literalinclude} ../../../../examples/server-api/start-stop-server.py
 :language: python
 :pyobject: event_stream
 ```
@@ -285,7 +285,7 @@ The only way to wait for a server to stop is to poll it and wait for the server
 
 This Python code snippet can be used to stop a server and the wait for the process to complete:
 
-```{literalinclude} ../../../examples/server-api/start-stop-server.py
+```{literalinclude} ../../../../examples/server-api/start-stop-server.py
 :language: python
 :pyobject: stop_server
 ```
@@ -325,7 +325,7 @@ In summary, the processes involved in managing servers on behalf of users are:
 The example below demonstrates starting and stopping servers via the JupyterHub API,
 including waiting for them to start via the progress API and waiting for them to stop by polling the user model.
 
-```{literalinclude} ../../../examples/server-api/start-stop-server.py
+```{literalinclude} ../../../../examples/server-api/start-stop-server.py
 :language: python
 :start-at: def event_stream
 :end-before: def main

docs/source/tutorial/getting-started/config-basics.md

@@ -1,7 +1,7 @@
 # Configuration Basics
 
 This section contains basic information about configuring settings for a JupyterHub
-deployment. The [Technical Reference](../reference/index)
+deployment. The [Technical Reference](reference-index)
 documentation provides additional details.
 
 This section will help you learn how to:

docs/source/tutorial/getting-started/services-basics.md

@@ -14,7 +14,7 @@ document will:
 
 - explain some basic information about API tokens
 - clarify that API tokens can be used to authenticate to
-  single-user servers as of [version 0.8.0](../changelog)
+  single-user servers as of [version 0.8.0](changelog)
 - show how the [jupyterhub_idle_culler][] script can be:
   - used in a Hub-managed service
   - run as a standalone script
@@ -29,19 +29,19 @@ Hub via the REST API.
 To run such an external service, an API token must be created and
 provided to the service.
 
-As of [version 0.6.0](../changelog), the preferred way of doing
+As of [version 0.6.0](changelog), the preferred way of doing
 this is to first generate an API token:
 
 ```bash
 openssl rand -hex 32
 ```
 
-In [version 0.8.0](../changelog), a TOKEN request page for
+In [version 0.8.0](changelog), a TOKEN request page for
 generating an API token is available from the JupyterHub user interface:
 
-![Request API TOKEN page](../images/token-request.png)
+![Request API TOKEN page](/images/token-request.png)
 
-![API TOKEN success page](../images/token-request-success.png)
+![API TOKEN success page](/images/token-request-success.png)
 
 ### Step 2: Pass environment variable with token to the Hub
 

docs/source/tutorial/index.md

@@ -1,10 +1,10 @@
 # Tutorials
 
-This section of the documentation provides step-by-step tutorials to help you achieve a specific goal. The tutorials should be a good place to start learning about JupyterHub and how it works.
+_Tutorials_ provide step-by-step lessons to help you achieve a specific goal. They should be a good place to start learning about JupyterHub and how it works.
 
 ## Installation
 
-These sections cover how to get up-and-running with JupyterHub. They cover
+This section covers how to get up-and-running with JupyterHub. It covers
 some basics of the tools needed to deploy JupyterHub as well as how to get it
 running on your own infrastructure.
 
@@ -15,3 +15,31 @@ installation/quickstart
 installation/installation-basics
 installation/quickstart-docker
 ```
+
+## Getting Started
+
+This section covers how to configure and customize JupyterHub for your
+needs. It contains information about authentication, networking, security, and
+other topics that are relevant to individuals or organizations deploying their
+own JupyterHub.
+
+```{toctree}
+:maxdepth: 1
+
+getting-started/config-basics
+getting-started/networking-basics
+getting-started/security-basics
+getting-started/authenticators-users-basics
+getting-started/services-basics
+getting-started/spawners-basics
+```
+
+## Working with the JupyterHub API
+
+JupyterHub's functionalities can be accessed using its API. In this section, we cover how to use the JupyterHub API to achieve specific goals, for example, starting servers.
+
+```{toctree}
+:maxdepth: 1
+
+api/server-api
+```

examples/service-fastapi/app/security.py

@@ -32,6 +32,7 @@ if os.environ.get("JUPYTERHUB_OAUTH_SCOPES"):
 else:
     access_scopes = ["access:services"]
 
+
 ### For consideration: optimize performance with a cache instead of
 ### always hitting the Hub api?
 async def get_current_user(

jsx/src/util/jhapiUtil.js

@@ -8,7 +8,7 @@ export const jhapiRequest = (endpoint, method, data) => {
   if (xsrfToken) {
     // add xsrf token to url parameter
     var sep = endpoint.indexOf("?") === -1 ? "?" : "&";
-    suffix = sep + "_xsrf=" + xsrf_token;
+    suffix = sep + "_xsrf=" + xsrfToken;
   }
   return fetch(api_url + endpoint + suffix, {
     method: method,

jupyterhub/_memoize.py

@@ -83,6 +83,7 @@ def lru_cache_key(key_func, maxsize=1024):
 
     def cache_func(func):
         cache = LRUCache(maxsize=maxsize)
+
         # the actual decorated function:
         @wraps(func)
         def cached(*args, **kwargs):

jupyterhub/app.py

@@ -2929,7 +2929,6 @@ class JupyterHub(Application):
         init_spawners_future.add_done_callback(log_init_time)
 
         try:
-
             # don't allow a zero timeout because we still need to be sure
             # that the Spawner objects are defined and pending
             await gen.with_timeout(

jupyterhub/auth.py

@@ -357,6 +357,7 @@ class Authenticator(LoggingConfigurable):
                         ),
                         DeprecationWarning,
                     )
+
                     # use old name instead of new
                     # if old name is overridden in subclass
                     def _new_calls_old(old_name, *args, **kwargs):

jupyterhub/handlers/base.py

@@ -664,7 +664,12 @@ class BaseHandler(RequestHandler):
             next_url = "//" + next_url.lstrip("/")
         parsed_next_url = urlparse(next_url)
 
-        if (next_url + '/').startswith((f'{proto}://{host}/', f'//{host}/',)) or (
+        if (next_url + '/').startswith(
+            (
+                f'{proto}://{host}/',
+                f'//{host}/',
+            )
+        ) or (
             self.subdomain_host
             and parsed_next_url.netloc
             and ("." + parsed_next_url.netloc).endswith(

jupyterhub/orm.py

@@ -8,9 +8,7 @@ from datetime import datetime, timedelta
 
 import alembic.command
 import alembic.config
-import sqlalchemy
 from alembic.script import ScriptDirectory
-from packaging.version import parse as parse_version
 from sqlalchemy import (
     Boolean,
     Column,
@@ -31,18 +29,12 @@ from sqlalchemy import (
 from sqlalchemy.orm import (
     Session,
     backref,
+    declarative_base,
     interfaces,
     object_session,
     relationship,
     sessionmaker,
 )
-
-try:
-    from sqlalchemy.orm import declarative_base
-except ImportError:
-    # sqlalchemy < 1.4
-    from sqlalchemy.ext.declarative import declarative_base
-
 from sqlalchemy.pool import StaticPool
 from sqlalchemy.types import LargeBinary, Text, TypeDecorator
 from tornado.log import app_log
@@ -912,27 +904,19 @@ def register_ping_connection(engine):
 
     @event.listens_for(engine, "engine_connect")
     def ping_connection(connection, branch=None):
-        if branch:
+        # TODO: remove unused branch arg when we require sqlalchemy 2.0
-            # "branch" refers to a sub-connection of a connection,
-            # we don't want to bother pinging on these.
-            return
 
         # turn off "close with result".  This flag is only used with
         # "connectionless" execution, otherwise will be False in any case
         save_should_close_with_result = connection.should_close_with_result
         connection.should_close_with_result = False
 
-        if parse_version(sqlalchemy.__version__) < parse_version("1.4"):
-            one = [1]
-        else:
-            one = 1
-
         try:
             # run a SELECT 1. use a core select() so that
             # the SELECT of a scalar value without a table is
             # appropriately formatted for the backend
             with connection.begin() as transaction:
-                connection.scalar(select(one))
+                connection.scalar(select(1))
         except exc.DBAPIError as err:
             # catch SQLAlchemy's DBAPIError, which is a wrapper
             # for the DBAPI's exception.  It includes a .connection_invalidated
@@ -948,7 +932,7 @@ def register_ping_connection(engine):
                 # here also causes the whole connection pool to be invalidated
                 # so that all stale connections are discarded.
                 with connection.begin() as transaction:
-                    connection.scalar(select(one))
+                    connection.scalar(select(1))
             else:
                 raise
         finally:
@@ -972,11 +956,8 @@ def check_db_revision(engine):
 
     from .dbutil import _temp_alembic_ini
 
-    if hasattr(engine.url, "render_as_string"):
+    # alembic needs the password if it's in the URL
-        # sqlalchemy >= 1.4
+    engine_url = engine.url.render_as_string(hide_password=False)
-        engine_url = engine.url.render_as_string(hide_password=False)
-    else:
-        engine_url = str(engine.url)
 
     with _temp_alembic_ini(engine_url) as ini:
         cfg = alembic.config.Config(ini)
@@ -1067,6 +1048,8 @@ def new_session_factory(
     elif url.startswith('mysql'):
         kwargs.setdefault('pool_recycle', 60)
 
+    kwargs.setdefault("future", True)
+
     if url.endswith(':memory:'):
         # If we're using an in-memory database, ensure that only one connection
         # is ever created.

jupyterhub/scopes.py

@@ -685,7 +685,7 @@ def _check_scope_access(api_handler, req_scope, **kwargs):
             req_scope,
         )
         return True
-    for (filter_, filter_value) in kwargs.items():
+    for filter_, filter_value in kwargs.items():
         if filter_ in sub_scope and filter_value in sub_scope[filter_]:
             app_log.debug("Argument-based access to %s via %s", api_name, req_scope)
             return True

jupyterhub/services/auth.py

@@ -1149,6 +1149,7 @@ class HubAuthenticated:
         except UserNotAllowed as e:
             # cache None, in case get_user is called again while processing the error
             self._hub_auth_user_cache = None
+
             # Override redirect so if/when tornado @web.authenticated
             # tries to redirect to login URL, 403 will be raised instead.
             # This is not the best, but avoids problems that can be caused

jupyterhub/singleuser/mixins.py

@@ -943,6 +943,7 @@ def make_singleuser_app(App):
     merged_flags = {}
     merged_flags.update(empty_parent_app.flags or {})
     merged_flags.update(flags)
+
     # create mixed-in App class, bringing it all together
     class SingleUserNotebookApp(SingleUserNotebookAppMixin, App):
         aliases = merged_aliases

jupyterhub/tests/conftest.py

@@ -291,7 +291,6 @@ async def _mockservice(request, app, external=False, url=False):
             spec['url'] = 'http://127.0.0.1:%i' % random_port()
 
     if external:
-
         spec['oauth_redirect_uri'] = 'http://127.0.0.1:%i' % random_port()
 
     event_loop = asyncio.get_running_loop()

jupyterhub/tests/selenium/test_browser.py

@@ -168,7 +168,6 @@ async def test_open_url_login(
     form_action,
     user,
 ):
-
     await open_url(app, browser, path=url)
     url_new = url_path_join(public_host(app), app.hub.base_url, url_concat(url, params))
     await in_thread(browser.get, url_new)
@@ -375,9 +374,10 @@ async def test_spawn_pending_server_ready(app, browser, user):
     await webdriver_wait(browser, EC.staleness_of(button_start))
     # checking that server is running and two butons present on the home page
     home_page = url_path_join(public_host(app), ujoin(app.base_url, "hub/home"))
-    await in_thread(browser.get, home_page)
     while not user.spawner.ready:
         await asyncio.sleep(0.01)
+    await in_thread(browser.get, home_page)
+    await wait_for_ready(browser)
     assert is_displayed(browser, (By.ID, "stop"))
     assert is_displayed(browser, (By.ID, "start"))
 
@@ -990,3 +990,344 @@ async def test_oauth_page(
 
     # compare the scopes on the service page with the expected scope list
     assert sorted(authorized_scopes) == sorted(expected_scopes)
+
+
+# ADMIN UI
+
+
+async def open_admin_page(app, browser, user):
+    """Login as `user` and open the admin page"""
+
+    admin_page = url_escape(app.base_url) + "hub/admin"
+    await open_url(app, browser, path="/login?next=" + admin_page)
+    await login(browser, user.name, pass_w=str(user.name))
+    # waiting for loading of admin page elements
+    await webdriver_wait(
+        browser,
+        lambda browser: is_displayed(
+            browser, (By.XPATH, '//div[@class="resets"]/div[@data-testid="container"]')
+        ),
+    )
+
+
+def create_list_of_users(create_user_with_scopes, n):
+    return [create_user_with_scopes(["users"]) for i in range(1, n)]
+
+
+async def test_open_admin_page(app, browser, admin_user):
+
+    await open_admin_page(app, browser, admin_user)
+    assert '/hub/admin' in browser.current_url
+
+
+def get_users_buttons(browser, class_name):
+    """returns the list of buttons in the user row(s) that match this class name"""
+
+    all_btns = browser.find_elements(
+        By.XPATH,
+        f'//*[@data-testid="user-row-server-activity"]//button[contains(@class,"{class_name}")]',
+    )
+    return all_btns
+
+
+async def click_and_wait_paging_btn(browser, buttons_number):
+    """interecrion with paging buttons, where number 1 = previous and number 2 = next"""
+    # number 1 - previous button, number 2 - next button
+    await click(
+        browser,
+        (
+            By.XPATH,
+            f'//*[@class="pagination-footer"]//button[contains(@class, "btn-light")][{buttons_number}]',
+        ),
+    )
+
+
+async def test_start_stop_all_servers_on_admin_page(app, browser, admin_user):
+
+    await open_admin_page(app, browser, admin_user)
+    # get total count of users from db
+    users_count_db = app.db.query(orm.User).count()
+
+    start_all_btn = browser.find_element(
+        By.XPATH, '//button[@type="button" and @data-testid="start-all"]'
+    )
+    stop_all_btn = browser.find_element(
+        By.XPATH, '//button[@type="button" and @data-testid="stop-all"]'
+    )
+
+    # verify Start All and Stop All buttons are displayed
+    assert start_all_btn.is_displayed() and stop_all_btn.is_displayed()
+
+    async def click_all_btns(browser, btn_type, btn_await):
+        await click(
+            browser,
+            (By.XPATH, f'//button[@type="button" and @data-testid="{btn_type}"]'),
+        )
+        await webdriver_wait(
+            browser,
+            EC.visibility_of_all_elements_located(
+                (
+                    By.XPATH,
+                    '//*[@data-testid="user-row-server-activity"]//button[contains(@class, "%s")]'
+                    % str(btn_await),
+                )
+            ),
+        )
+
+    users = browser.find_elements(By.XPATH, '//td[@data-testid="user-row-name"]')
+    # verify that all servers are not started
+    # users´numbers are the same as numbers of the start button and the Spawn page button
+    # no Stop server buttons are displayed
+    # no access buttons are displayed
+
+    class_names = ["stop-button", "primary", "start-button", "secondary"]
+    btns = {
+        class_name: get_users_buttons(browser, class_name) for class_name in class_names
+    }
+    print(btns)
+    assert (
+        len(btns["start-button"])
+        == len(btns["secondary"])
+        == len(users)
+        == users_count_db
+    )
+    assert not btns["stop-button"] and not btns["primary"]
+
+    # start all servers via the Start All
+    await click_all_btns(browser, "start-all", "stop-button")
+    # Start All and Stop All are still displayed
+    assert start_all_btn.is_displayed() and stop_all_btn.is_displayed()
+
+    # users´numbers are the same as numbers of the stop button and the Access button
+    # no Start server buttons are displayed
+    # no Spawn page buttons are displayed
+    btns = {
+        class_name: get_users_buttons(browser, class_name) for class_name in class_names
+    }
+    assert (
+        len(btns["stop-button"]) == len(btns["primary"]) == len(users) == users_count_db
+    )
+    assert not btns["start-button"] and not btns["secondary"]
+
+    # stop all servers via the Stop All
+    await click_all_btns(browser, "stop-all", "start-button")
+
+    # verify that all servers are stopped
+    # users´numbers are the same as numbers of the start button and the Spawn page button
+    # no Stop server buttons are displayed
+    # no access buttons are displayed
+    assert start_all_btn.is_displayed() and stop_all_btn.is_displayed()
+    btns = {
+        class_name: get_users_buttons(browser, class_name) for class_name in class_names
+    }
+    assert (
+        len(btns["start-button"])
+        == len(btns["secondary"])
+        == len(users)
+        == users_count_db
+    )
+    assert not btns["stop-button"] and not btns["primary"]
+
+
+@pytest.mark.parametrize("added_count_users", [10, 47, 48, 49, 110])
+async def test_paging_on_admin_page(
+    app, browser, admin_user, added_count_users, create_user_with_scopes
+):
+
+    create_list_of_users(create_user_with_scopes, added_count_users)
+    await open_admin_page(app, browser, admin_user)
+    users = browser.find_elements(By.XPATH, '//td[@data-testid="user-row-name"]')
+
+    # get total count of users from db
+    users_count_db = app.db.query(orm.User).count()
+    # get total count of users from UI page
+    users_list = [user.text for user in users]
+    displaying = browser.find_element(
+        By.XPATH, '//*[@class="pagination-footer"]//*[contains(text(),"Displaying")]'
+    )
+    btn_previous = browser.find_element(
+        By.XPATH, '//*[@class="pagination-footer"]//span[contains(text(),"Previous")]'
+    )
+    btn_next = browser.find_element(
+        By.XPATH, '//*[@class="pagination-footer"]//span[contains(text(),"Next")]'
+    )
+    assert f"0-{min(users_count_db,50)}" in displaying.text
+    if users_count_db > 50:
+        assert btn_next.get_dom_attribute("class") == "active-pagination"
+        # click on Next button
+        await click_and_wait_paging_btn(browser, buttons_number=2)
+        if users_count_db <= 100:
+            assert f"50-{users_count_db}" in displaying.text
+        else:
+            assert "50-100" in displaying.text
+            assert btn_next.get_dom_attribute("class") == "active-pagination"
+        assert btn_previous.get_dom_attribute("class") == "active-pagination"
+        # click on Previous button
+        await click_and_wait_paging_btn(browser, buttons_number=1)
+    else:
+        assert btn_previous.get_dom_attribute("class") == "inactive-pagination"
+        assert btn_next.get_dom_attribute("class") == "inactive-pagination"
+
+
+@pytest.mark.parametrize(
+    "added_count_users, search_value",
+    [
+        # the value of search is absent =>the expected result null records are found
+        (10, "not exists"),
+        # a search value is a middle part of users name (number,symbol,letter)
+        (25, "r_5"),
+        # a search value equals to number
+        (50, "1"),
+        # searching result shows on more than one page
+        (60, "user"),
+    ],
+)
+async def test_search_on_admin_page(
+    app,
+    browser,
+    admin_user,
+    create_user_with_scopes,
+    added_count_users,
+    search_value,
+):
+
+    create_list_of_users(create_user_with_scopes, added_count_users)
+    await open_admin_page(app, browser, admin_user)
+    element_search = browser.find_element(By.XPATH, '//input[@name="user_search"]')
+    element_search.send_keys(search_value)
+    await asyncio.sleep(1)
+    # get the result of the search from db
+    users_count_db_filtered = (
+        app.db.query(orm.User).filter(orm.User.name.like(f'%{search_value}%')).count()
+    )
+    filtered_list_on_page = browser.find_elements(By.XPATH, '//*[@class="user-row"]')
+    # check that count of users matches with number of users on the footer
+    displaying = browser.find_element(
+        By.XPATH, '//*[@class="pagination-footer"]//*[contains(text(),"Displaying")]'
+    )
+    # check that users names contain the search value in the filtered list
+    for element in filtered_list_on_page:
+        name = element.find_element(
+            By.XPATH,
+            '//*[@data-testid="user-row-name"]//span[contains(@data-testid, "user-name-div")]',
+        )
+        assert search_value in name.text
+    if users_count_db_filtered <= 50:
+        assert "0-" + str(users_count_db_filtered) in displaying.text
+        assert len(filtered_list_on_page) == users_count_db_filtered
+    else:
+        assert "0-50" in displaying.text
+        assert len(filtered_list_on_page) == 50
+        # click on Next button to verify that the rest part of filtered list is displayed on the next page
+        await click_and_wait_paging_btn(browser, buttons_number=2)
+        filtered_list_on_next_page = browser.find_elements(
+            By.XPATH, '//*[@class="user-row"]'
+        )
+        assert users_count_db_filtered - 50 == len(filtered_list_on_next_page)
+        for element in filtered_list_on_next_page:
+            name = element.find_element(
+                By.XPATH,
+                '//*[@data-testid="user-row-name"]//span[contains(@data-testid, "user-name-div")]',
+            )
+        assert search_value in name.text
+
+
+@pytest.mark.parametrize("added_count_users,index_user_1, index_user_2", [(5, 1, 0)])
+async def test_start_stop_server_on_admin_page(
+    app,
+    browser,
+    admin_user,
+    create_user_with_scopes,
+    added_count_users,
+    index_user_1,
+    index_user_2,
+):
+    async def start_user(browser, expected_user):
+        start_button_xpath = f'//a[contains(@href, "spawn/{expected_user[0]}")]/preceding-sibling::button[contains(@class, "start-button")]'
+        await click(browser, (By.XPATH, start_button_xpath))
+        start_btn = browser.find_element(By.XPATH, start_button_xpath)
+        await wait_for_ready(browser)
+        await webdriver_wait(browser, EC.staleness_of(start_btn))
+
+    async def spawn_user(browser, app, expected_user):
+        spawn_button_xpath = f'//a[contains(@href, "spawn/{expected_user[1]}")]/button[contains(@class, "secondary")]'
+        await click(browser, (By.XPATH, spawn_button_xpath))
+        while (
+            not app.users[1].spawner.ready
+            and f"/hub/spawn-pending/{expected_user[1]}" in browser.current_url
+        ):
+            await webdriver_wait(browser, EC.url_contains(f"/user/{expected_user[1]}/"))
+
+    async def access_srv_user(browser, expected_user):
+        access_buttons_xpath = '//*[@data-testid="user-row-server-activity"]//button[contains(@class, "primary")]'
+        for i, ex_user in enumerate(expected_user):
+            access_btn_xpath = f'//a[contains(@href, "user/{expected_user[i]}")]/button[contains(@class, "primary")]'
+            await click(browser, (By.XPATH, access_btn_xpath))
+            if not f"/user/{expected_user[i]}/" in browser.current_url:
+                await webdriver_wait(
+                    browser, EC.url_contains(f"/user/{expected_user[i]}/")
+                )
+            browser.back()
+
+    async def stop_srv_users(browser, expected_user):
+        for i, ex_user in enumerate(expected_user):
+            stop_btn_xpath = f'//a[contains(@href, "user/{expected_user[i]}")]/preceding-sibling::button[contains(@class, "stop-button")]'
+            stop_btn = browser.find_element(By.XPATH, stop_btn_xpath)
+            await click(browser, (By.XPATH, stop_btn_xpath))
+            await webdriver_wait(browser, EC.staleness_of(stop_btn))
+
+    create_list_of_users(create_user_with_scopes, added_count_users)
+    await open_admin_page(app, browser, admin_user)
+    users = browser.find_elements(By.XPATH, '//td[@data-testid="user-row-name"]')
+    users_list = [user.text for user in users]
+
+    expected_user = [users_list[index_user_1], users_list[index_user_2]]
+    spawn_page_btns = browser.find_elements(
+        By.XPATH,
+        '//*[@data-testid="user-row-server-activity"]//a[contains(@href, "spawn/")]',
+    )
+
+    for i, user in enumerate(users):
+        spawn_page_btn = spawn_page_btns[i]
+        user_from_table = user.text
+        link = spawn_page_btn.get_attribute('href')
+    assert f"/spawn/{user_from_table}" in link
+
+    # click on Start button
+    await start_user(browser, expected_user)
+    class_names = ["stop-button", "primary", "start-button", "secondary"]
+    btns = {
+        class_name: get_users_buttons(browser, class_name) for class_name in class_names
+    }
+    assert len(btns["stop-button"]) == 1
+
+    # click on Spawn page button
+    await spawn_user(browser, app, expected_user)
+    assert f"/user/{expected_user[1]}/" in browser.current_url
+
+    # open  the Admin page
+    await open_url(app, browser, "/admin")
+    # wait for javascript to finish loading
+    await wait_for_ready(browser)
+    assert "/hub/admin" in browser.current_url
+    btns = {
+        class_name: get_users_buttons(browser, class_name) for class_name in class_names
+    }
+    assert len(btns["stop-button"]) == len(btns["primary"]) == 2
+
+    # click on the Access button
+    await access_srv_user(browser, expected_user)
+
+    assert "/hub/admin" in browser.current_url
+    btns = {
+        class_name: get_users_buttons(browser, class_name) for class_name in class_names
+    }
+    assert len(btns["stop-button"]) == 2
+
+    # click on Stop button for both users
+    await stop_srv_users(browser, expected_user)
+    btns = {
+        class_name: get_users_buttons(browser, class_name) for class_name in class_names
+    }
+    assert len(btns["stop-button"]) == 0
+    assert len(btns["primary"]) == 0

jupyterhub/tests/test_app.py

@@ -233,7 +233,6 @@ def test_cookie_secret_string_():
 
 
 async def test_load_groups(tmpdir, request):
-
     to_load = {
         'blue': {
             'users': ['cyclops', 'rogue', 'wolverine'],

jupyterhub/tests/test_auth.py

@@ -559,7 +559,6 @@ class MockGroupsAuthenticator(auth.Authenticator):
 async def test_auth_managed_groups(
     app, user, group, authenticated_groups, refresh_groups
 ):
-
     authenticator = MockGroupsAuthenticator(
         parent=app,
         authenticated_groups=authenticated_groups,

jupyterhub/tests/test_db.py

@@ -5,6 +5,7 @@ from glob import glob
 from subprocess import check_call
 
 import pytest
+from packaging.version import parse as V
 from pytest import raises
 from traitlets.config import Config
 
@@ -25,8 +26,14 @@ def generate_old_db(env_dir, hub_version, db_url):
     env_pip = os.path.join(env_dir, 'bin', 'pip')
     env_py = os.path.join(env_dir, 'bin', 'python')
     check_call([sys.executable, '-m', 'virtualenv', env_dir])
+    pkgs = ['jupyterhub==' + hub_version]
+
     # older jupyterhub needs older sqlachemy version
-    pkgs = ['jupyterhub==' + hub_version, 'sqlalchemy<1.4']
+    if V(hub_version) < V("2"):
+        pkgs.append('sqlalchemy<1.4')
+    elif V(hub_version) < V("3.1.1"):
+        pkgs.append('sqlalchemy<2')
+
     if 'mysql' in db_url:
         pkgs.append('mysql-connector-python')
     elif 'postgres' in db_url:

jupyterhub/tests/test_memoize.py

@@ -25,7 +25,6 @@ def test_lru_cache():
 
 
 def test_lru_cache_key():
-
     call_count = 0
 
     @lru_cache_key(frozenset)
@@ -47,7 +46,6 @@ def test_lru_cache_key():
 
 
 def test_do_not_cache():
-
     call_count = 0
 
     @lru_cache_key(lambda arg: arg)

jupyterhub/tests/test_roles.py

@@ -1243,7 +1243,6 @@ async def test_admin_role_respects_config():
     ],
 )
 async def test_admin_role_membership(in_db, role_users, admin_users, expected_members):
-
     load_roles = []
     if role_users is not None:
         load_roles.append({"name": "admin", "users": role_users})

jupyterhub/tests/test_scopes.py

@@ -971,7 +971,6 @@ def test_intersect_groups(request, db, left, right, expected, groups):
 async def test_list_users_filter(
     app, group, create_service_with_scopes, scopes, expected
 ):
-
     # create users:
     for i in (1, 2):
         user = add_user(app.db, app, name=f'in-{i}')
@@ -1028,7 +1027,6 @@ async def test_list_users_filter(
 async def test_list_groups_filter(
     request, app, create_service_with_scopes, scopes, expected
 ):
-
     # create groups:
     groups = []
     for i in (1, 2, 3):

jupyterhub/tests/utils.py

@@ -14,20 +14,6 @@ from jupyterhub.objects import Server
 from jupyterhub.roles import assign_default_roles, update_roles
 from jupyterhub.utils import url_path_join as ujoin
 
-try:
-    from sqlalchemy.exc import RemovedIn20Warning
-except ImportError:
-
-    class RemovedIn20Warning(DeprecationWarning):
-        """
-        I only exist so I can be used in warnings filters in pytest.ini
-
-        I will never be displayed.
-
-        sqlalchemy 1.4 introduces RemovedIn20Warning,
-        but we still test against older sqlalchemy.
-        """
-
 
 class _AsyncRequests:
     """Wrapper around requests to return a Future from request methods

jupyterhub/utils.py

@@ -710,7 +710,7 @@ def get_accepted_mimetype(accept_header, choices=None):
     Return `None` if choices is given and no match is found,
     or nothing is specified.
     """
-    for (mime, params, q) in _parse_accept_header(accept_header):
+    for mime, params, q in _parse_accept_header(accept_header):
         if choices:
             if mime in choices:
                 return mime

pytest.ini

@@ -20,5 +20,5 @@ markers =
     selenium: web tests that run with selenium
 
 filterwarnings =
-    error:.*:jupyterhub.tests.utils.RemovedIn20Warning
+    ignore:.*The new signature is "def engine_connect\(conn\)"*:sqlalchemy.exc.SADeprecationWarning
-    ignore:.*event listener has changed as of version 2.0.*:sqlalchemy.exc.SADeprecationWarning
+    ignore:.*The new signature is "def engine_connect\(conn\)"*:sqlalchemy.exc.SAWarning

requirements.txt

@@ -11,6 +11,6 @@ prometheus_client>=0.4.0
 psutil>=5.6.5; sys_platform == 'win32'
 python-dateutil
 requests
-SQLAlchemy>=1.1
+SQLAlchemy>=1.4
 tornado>=5.1
 traitlets>=4.3.2

setup.py

@@ -38,7 +38,7 @@ def get_data_files():
     """Get data files in share/jupyter"""
 
     data_files = []
-    for (d, dirs, filenames) in os.walk(share_jupyterhub):
+    for d, dirs, filenames in os.walk(share_jupyterhub):
         rel_d = os.path.relpath(d, here)
         data_files.append((rel_d, [os.path.join(rel_d, f) for f in filenames]))
     return data_files
@@ -238,7 +238,7 @@ class CSS(BaseCommand):
         earliest_target = sorted(mtime(t) for t in targets)[0]
 
         # check if any .less files are newer than the generated targets
-        for (dirpath, dirnames, filenames) in os.walk(static):
+        for dirpath, dirnames, filenames in os.walk(static):
             for f in filenames:
                 if f.endswith('.less'):
                     path = pjoin(static, dirpath, f)