Bump to 4.0.2

Backport PR #4534 on branch 4.x (Changelog for 4.0.2)

docs/source/_static/rest-api.yml

@@ -6,7 +6,7 @@ info:
   description: The REST API for JupyterHub
   license:
     name: BSD-3-Clause
-  version: 4.1.0.dev
+  version: 4.0.2
 servers:
   - url: /hub/api
 security:

docs/source/howto/configuration/config-user-env.md

@@ -45,7 +45,7 @@ additional packages.
 
 ## Configuring Jupyter and IPython
 
-[Jupyter](https://jupyter-notebook.readthedocs.io/en/stable/config_overview.html)
+[Jupyter](https://jupyter-notebook.readthedocs.io/en/stable/configuring/config_overview.html)
 and [IPython](https://ipython.readthedocs.io/en/stable/development/config.html)
 have their own configuration systems.
 
@@ -212,13 +212,31 @@ By default, the single-user server launches JupyterLab,
 which is based on [Jupyter Server][].
 
 This is the default server when running JupyterHub ≥ 2.0.
-To switch to using the legacy Jupyter Notebook server, you can set the `JUPYTERHUB_SINGLEUSER_APP` environment variable
+To switch to using the legacy Jupyter Notebook server (notebook < 7.0), you can set the `JUPYTERHUB_SINGLEUSER_APP` environment variable
 (in the single-user environment) to:
 
 ```bash
 export JUPYTERHUB_SINGLEUSER_APP='notebook.notebookapp.NotebookApp'
 ```
 
+:::{note}
+
+```
+JUPYTERHUB_SINGLEUSER_APP='notebook.notebookapp.NotebookApp'
+```
+
+is only valid for notebook < 7. notebook v7 is based on jupyter-server,
+and the default jupyter-server application must be used.
+Selecting the new notebook UI is no longer a matter of selecting the server app to launch,
+but only the default URL for users to visit.
+To use notebook v7 with JupyterHub, leave the default singleuser app config alone (or specify `JUPYTERHUB_SINGLEUSER_APP=jupyter-server`) and set the default _URL_ for user servers:
+
+```python
+c.Spawner.default_url = '/tree/'
+```
+
+:::
+
 [jupyter server]: https://jupyter-server.readthedocs.io
 [jupyter notebook]: https://jupyter-notebook.readthedocs.io
 

docs/source/reference/changelog.md

@@ -10,6 +10,34 @@ command line for details.
 
 ## 4.0
 
+### 4.0.2 - 2023-08-10
+
+([full changelog](https://github.com/jupyterhub/jupyterhub/compare/4.0.1...4.0.2))
+
+#### Enhancements made
+
+- avoid counting failed requests to not-running servers as 'activity' [#4491](https://github.com/jupyterhub/jupyterhub/pull/4491) ([@minrk](https://github.com/minrk), [@consideRatio](https://github.com/consideRatio))
+- improve permission-denied errors for various cases [#4489](https://github.com/jupyterhub/jupyterhub/pull/4489) ([@minrk](https://github.com/minrk), [@consideRatio](https://github.com/consideRatio))
+
+#### Bugs fixed
+
+- set root_dir when using singleuser extension [#4503](https://github.com/jupyterhub/jupyterhub/pull/4503) ([@minrk](https://github.com/minrk), [@consideRatio](https://github.com/consideRatio), [@manics](https://github.com/manics))
+- Allow setting custom log_function in tornado_settings in SingleUserServer [#4475](https://github.com/jupyterhub/jupyterhub/pull/4475) ([@grios-stratio](https://github.com/grios-stratio), [@minrk](https://github.com/minrk))
+
+#### Documentation improvements
+
+- doc: update notebook config URL [#4523](https://github.com/jupyterhub/jupyterhub/pull/4523) ([@minrk](https://github.com/minrk), [@manics](https://github.com/manics))
+- document how to use notebook v7 with jupyterhub [#4522](https://github.com/jupyterhub/jupyterhub/pull/4522) ([@minrk](https://github.com/minrk), [@manics](https://github.com/manics))
+
+#### Contributors to this release
+
+The following people contributed discussions, new ideas, code and documentation contributions, and review.
+See [our definition of contributors](https://github-activity.readthedocs.io/en/latest/#how-does-this-tool-define-contributions-in-the-reports).
+
+([GitHub contributors page for this release](https://github.com/jupyterhub/jupyterhub/graphs/contributors?from=2023-06-08&to=2023-08-10&type=c))
+
+@agelosnm ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Aagelosnm+updated%3A2023-06-08..2023-08-10&type=Issues)) | @consideRatio ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3AconsideRatio+updated%3A2023-06-08..2023-08-10&type=Issues)) | @diocas ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Adiocas+updated%3A2023-06-08..2023-08-10&type=Issues)) | @grios-stratio ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Agrios-stratio+updated%3A2023-06-08..2023-08-10&type=Issues)) | @jhgoebbert ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Ajhgoebbert+updated%3A2023-06-08..2023-08-10&type=Issues)) | @jtpio ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Ajtpio+updated%3A2023-06-08..2023-08-10&type=Issues)) | @kosmonavtus ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Akosmonavtus+updated%3A2023-06-08..2023-08-10&type=Issues)) | @kreuzert ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Akreuzert+updated%3A2023-06-08..2023-08-10&type=Issues)) | @manics ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Amanics+updated%3A2023-06-08..2023-08-10&type=Issues)) | @martinRenou ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3AmartinRenou+updated%3A2023-06-08..2023-08-10&type=Issues)) | @minrk ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Aminrk+updated%3A2023-06-08..2023-08-10&type=Issues)) | @opoplawski ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Aopoplawski+updated%3A2023-06-08..2023-08-10&type=Issues)) | @Ph0tonic ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3APh0tonic+updated%3A2023-06-08..2023-08-10&type=Issues)) | @sgaist ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Asgaist+updated%3A2023-06-08..2023-08-10&type=Issues)) | @trungleduc ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Atrungleduc+updated%3A2023-06-08..2023-08-10&type=Issues)) | @yuvipanda ([activity](https://github.com/search?q=repo%3Ajupyterhub%2Fjupyterhub+involves%3Ayuvipanda+updated%3A2023-06-08..2023-08-10&type=Issues))
+
 ### 4.0.1 - 2023-06-08
 
 ([full changelog](https://github.com/jupyterhub/jupyterhub/compare/4.0.0...4.0.1))

jupyterhub/_version.py

@@ -2,7 +2,7 @@
 # Copyright (c) Jupyter Development Team.
 # Distributed under the terms of the Modified BSD License.
 # version_info updated by running `tbump`
-version_info = (4, 1, 0, "", "dev")
+version_info = (4, 0, 2, "", "")
 
 # pep 440 version: no dot before beta/rc, but before .dev
 # 0.1.0rc1

jupyterhub/apihandlers/base.py

@@ -89,6 +89,11 @@ class APIHandler(BaseHandler):
         if not hasattr(self, '_jupyterhub_user'):
             # called too early to check if we're token-authenticated
             return
+        if self._jupyterhub_user is None and 'Origin' not in self.request.headers:
+            # don't raise xsrf if auth failed
+            # don't apply this shortcut to actual cross-site requests, which have an 'Origin' header,
+            # which would reveal if there are credentials present
+            return
         if getattr(self, '_token_authenticated', False):
             # if token-authenticated, ignore XSRF
             return

jupyterhub/handlers/base.py

@@ -236,11 +236,13 @@ class BaseHandler(RequestHandler):
     def check_xsrf_cookie(self):
         try:
             return super().check_xsrf_cookie()
-        except Exception as e:
-            # ensure _juptyerhub_user is defined on rejected requests
+        except web.HTTPError as e:
+            # ensure _jupyterhub_user is defined on rejected requests
             if not hasattr(self, "_jupyterhub_user"):
                 self._jupyterhub_user = None
             self._resolve_roles_and_scopes()
+            # rewrite message because we use this on methods other than POST
+            e.log_message = e.log_message.replace("POST", self.request.method)
             raise
 
     @property
@@ -1431,6 +1433,12 @@ class UserUrlHandler(BaseHandler):
     # accept token auth for API requests that are probably to non-running servers
     _accept_token_auth = True
 
+    # don't consider these redirects 'activity'
+    # if the redirect is followed and the subsequent action taken,
+    # _that_ is activity
+    def _record_activity(self, obj, timestamp=None):
+        return False
+
     def _fail_api_request(self, user_name='', server_name=''):
         """Fail an API request to a not-running server"""
         self.log.warning(

jupyterhub/scopes.py

@@ -845,6 +845,15 @@ def needs_scope(*scopes):
     def scope_decorator(func):
         @functools.wraps(func)
         def _auth_func(self, *args, **kwargs):
+            if not self.current_user:
+                # not authenticated at all, fail with more generic message
+                # this is the most likely permission error - missing or mis-specified credentials,
+                # don't indicate that they have insufficient permissions.
+                raise web.HTTPError(
+                    403,
+                    "Missing or invalid credentials.",
+                )
+
             sig = inspect.signature(func)
             bound_sig = sig.bind(self, *args, **kwargs)
             bound_sig.apply_defaults()
@@ -853,6 +862,11 @@ def needs_scope(*scopes):
                 self.expanded_scopes = {}
                 self.parsed_scopes = {}
 
+            try:
+                end_point = self.request.path
+            except AttributeError:
+                end_point = self.__name__
+
             s_kwargs = {}
             for resource in {'user', 'server', 'group', 'service'}:
                 resource_name = resource + '_name'
@@ -860,14 +874,10 @@ def needs_scope(*scopes):
                     resource_value = bound_sig.arguments[resource_name]
                     s_kwargs[resource] = resource_value
             for scope in scopes:
-                app_log.debug("Checking access via scope %s", scope)
+                app_log.debug("Checking access to %s via scope %s", end_point, scope)
                 has_access = _check_scope_access(self, scope, **s_kwargs)
                 if has_access:
                     return func(self, *args, **kwargs)
-            try:
-                end_point = self.request.path
-            except AttributeError:
-                end_point = self.__name__
             app_log.warning(
                 "Not authorizing access to {}. Requires any of [{}], not derived from scopes [{}]".format(
                     end_point, ", ".join(scopes), ", ".join(self.expanded_scopes)

jupyterhub/singleuser/app.py

@@ -6,7 +6,7 @@
 .. versionchanged:: 2.0
 
     Default app changed to launch `jupyter labhub`.
-    Use JUPYTERHUB_SINGLEUSER_APP=notebook.notebookapp.NotebookApp for the legacy 'classic' notebook server.
+    Use JUPYTERHUB_SINGLEUSER_APP='notebook' for the legacy 'classic' notebook server (requires notebook<7).
 """
 import os
 
@@ -27,7 +27,25 @@ JUPYTERHUB_SINGLEUSER_APP = _app_shortcuts.get(
     JUPYTERHUB_SINGLEUSER_APP.replace("_", "-"), JUPYTERHUB_SINGLEUSER_APP
 )
 
+
 if JUPYTERHUB_SINGLEUSER_APP:
+    if JUPYTERHUB_SINGLEUSER_APP in {"notebook", _app_shortcuts["notebook"]}:
+        # better error for notebook v7, which uses jupyter-server
+        # when the legacy notebook server is requested
+        try:
+            from notebook import __version__
+        except ImportError:
+            # will raise later
+            pass
+        else:
+            # check if this failed because of notebook v7
+            _notebook_major_version = int(__version__.split(".", 1)[0])
+            if _notebook_major_version >= 7:
+                raise ImportError(
+                    f"JUPYTERHUB_SINGLEUSER_APP={JUPYTERHUB_SINGLEUSER_APP} is not valid with notebook>=7 (have notebook=={__version__}).\n"
+                    f"Leave $JUPYTERHUB_SINGLEUSER_APP unspecified (or use the default JUPYTERHUB_SINGLEUSER_APP=jupyter-server), "
+                    'and set `c.Spawner.default_url = "/tree"` to make notebook v7 the default UI.'
+                )
     App = import_item(JUPYTERHUB_SINGLEUSER_APP)
 else:
     App = None

jupyterhub/singleuser/extension.py

@@ -483,6 +483,11 @@ class JupyterHubSingleUser(ExtensionApp):
         cfg.answer_yes = True
         self.config.FileContentsManager.delete_to_trash = False
 
+        # load Spawner.notebook_dir configuration, if given
+        root_dir = os.getenv("JUPYTERHUB_ROOT_DIR", None)
+        if root_dir:
+            cfg.root_dir = os.path.expanduser(root_dir)
+
         # load http server config from environment
         url = urlparse(os.environ['JUPYTERHUB_SERVICE_URL'])
         if url.port:
@@ -617,7 +622,9 @@ class JupyterHubSingleUser(ExtensionApp):
         app.web_app.settings[
             "page_config_hook"
         ] = app.identity_provider.page_config_hook
-        app.web_app.settings["log_function"] = log_request
+        # if the user has configured a log function in the tornado settings, do not override it
+        if not 'log_function' in app.config.ServerApp.get('tornado_settings', {}):
+            app.web_app.settings["log_function"] = log_request
         # add jupyterhub version header
         headers = app.web_app.settings.setdefault("headers", {})
         headers["X-JupyterHub-Version"] = __version__

jupyterhub/singleuser/mixins.py

@@ -669,7 +669,8 @@ class SingleUserNotebookAppMixin(Configurable):
         # load the hub-related settings into the tornado settings dict
         self.init_hub_auth()
         s = self.tornado_settings
-        s['log_function'] = log_request
+        # if the user has configured a log function in the tornado settings, do not override it
+        s.setdefault('log_function', log_request)
         s['user'] = self.user
         s['group'] = self.group
         s['hub_prefix'] = self.hub_prefix

jupyterhub/tests/extension/__init__.py

@@ -30,6 +30,7 @@ class JupyterHubTestHandler(JupyterHandler):
         info = {
             "current_user": self.current_user,
             "config": self.app.config,
+            "root_dir": self.contents_manager.root_dir,
             "disable_user_config": getattr(self.app, "disable_user_config", None),
             "settings": self.settings,
             "config_file_paths": self.app.config_file_paths,

jupyterhub/tests/test_api.py

@@ -122,6 +122,41 @@ async def test_xsrf_check(app, username, method, path, xsrf_in_url):
         assert r.status_code == 403
 
 
+@mark.parametrize(
+    "auth, expected_message",
+    [
+        ("", "Missing or invalid credentials"),
+        ("cookie_no_xsrf", "'_xsrf' argument missing from GET"),
+        ("cookie_xsrf_mismatch", "XSRF cookie does not match GET argument"),
+        ("token_no_scope", "requires any of [list:users]"),
+        ("cookie_no_scope", "requires any of [list:users]"),
+    ],
+)
+async def test_permission_error_messages(app, user, auth, expected_message):
+    # 1. no credentials, should be 403 and not mention xsrf
+
+    url = public_url(app, path="hub/api/users")
+
+    kwargs = {}
+    kwargs["headers"] = headers = {}
+    kwargs["params"] = params = {}
+    if auth == "token_no_scope":
+        token = user.new_api_token()
+        headers["Authorization"] = f"Bearer {token}"
+    elif "cookie" in auth:
+        cookies = kwargs["cookies"] = await app.login_user(user.name)
+        if auth == "cookie_no_scope":
+            params["_xsrf"] = cookies["_xsrf"]
+        if auth == "cookie_xsrf_mismatch":
+            params["_xsrf"] = "somethingelse"
+
+    r = await async_requests.get(url, **kwargs)
+    assert r.status_code == 403
+    response = r.json()
+    message = response["message"]
+    assert expected_message in message
+
+
 # --------------
 # User API tests
 # --------------

jupyterhub/tests/test_singleuser.py

@@ -2,6 +2,7 @@
 import os
 import sys
 from contextlib import nullcontext
+from pprint import pprint
 from subprocess import CalledProcessError, check_output
 from unittest import mock
 from urllib.parse import urlencode, urlparse
@@ -171,9 +172,7 @@ async def test_disable_user_config(request, app, tmpdir, full_spawn):
     )
     r.raise_for_status()
     info = r.json()
-    import pprint
-
-    pprint.pprint(info)
+    pprint(info)
     assert info['disable_user_config']
     server_config = info['config']
     settings = info['settings']
@@ -198,6 +197,79 @@ async def test_disable_user_config(request, app, tmpdir, full_spawn):
                 assert_not_in_home(path, key)
 
 
+@pytest.mark.parametrize("extension", [True, False])
+@pytest.mark.parametrize("notebook_dir", ["", "~", "~/sub", "ABS"])
+async def test_notebook_dir(
+    request, app, tmpdir, user, full_spawn, extension, notebook_dir
+):
+    if extension:
+        try:
+            import jupyter_server  # noqa
+        except ImportError:
+            pytest.skip("needs jupyter-server 2")
+        else:
+            if jupyter_server.version_info < (2,):
+                pytest.skip("needs jupyter-server 2")
+
+    token = user.new_api_token(scopes=["access:servers!user"])
+    headers = {"Authorization": f"Bearer {token}"}
+
+    spawner = user.spawner
+    if extension:
+        user.spawner.environment["JUPYTERHUB_SINGLEUSER_EXTENSION"] = "1"
+    else:
+        user.spawner.environment["JUPYTERHUB_SINGLEUSER_EXTENSION"] = "0"
+
+    home_dir = tmpdir.join("home").mkdir()
+    sub_dir = home_dir.join("sub").mkdir()
+    with sub_dir.join("subfile.txt").open("w") as f:
+        f.write("txt\n")
+    abs_dir = tmpdir.join("abs").mkdir()
+    with abs_dir.join("absfile.txt").open("w") as f:
+        f.write("absfile\n")
+
+    if notebook_dir:
+        expected_root_dir = notebook_dir.replace("ABS", str(abs_dir)).replace(
+            "~", str(home_dir)
+        )
+    else:
+        expected_root_dir = str(home_dir)
+
+    spawner.notebook_dir = notebook_dir.replace("ABS", str(abs_dir))
+
+    # home_dir is defined on SimpleSpawner
+    user.spawner.home_dir = home = str(home_dir)
+    spawner.environment["HOME"] = home
+    await user.spawn()
+    await app.proxy.add_user(user)
+    url = public_url(app, user)
+    r = await async_requests.get(
+        url_path_join(public_url(app, user), 'jupyterhub-test-info'), headers=headers
+    )
+    r.raise_for_status()
+    info = r.json()
+    pprint(info)
+
+    assert info["root_dir"] == expected_root_dir
+    # secondary check: make sure it has the intended effect on root_dir
+    r = await async_requests.get(
+        url_path_join(public_url(app, user), 'api/contents/'), headers=headers
+    )
+    r.raise_for_status()
+    root_contents = sorted(item['name'] for item in r.json()['content'])
+
+    # check contents
+    if not notebook_dir or notebook_dir == "~":
+        # use any to avoid counting possible automatically created files in $HOME
+        assert 'sub' in root_contents
+    elif notebook_dir == "ABS":
+        assert 'absfile.txt' in root_contents
+    elif notebook_dir == "~/sub":
+        assert 'subfile.txt' in root_contents
+    else:
+        raise ValueError(f"No contents check for {notebook_dir}")
+
+
 def test_help_output():
     out = check_output(
         [sys.executable, '-m', 'jupyterhub.singleuser', '--help-all']

pyproject.toml

@@ -43,7 +43,7 @@ target_version = [
 github_url = "https://github.com/jupyterhub/jupyterhub"
 
 [tool.tbump.version]
-current = "4.1.0.dev"
+current = "4.0.2"
 
 # Example of a semver regexp.
 # Make sure this matches current_version before