diff --git a/docs/index.rst b/docs/index.rst index f9a912f2..91271b16 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -13,6 +13,7 @@ Table of Contents using/common using/specifics using/recipes + using/troubleshooting .. toctree:: :maxdepth: 2 diff --git a/docs/using/troubleshooting.md b/docs/using/troubleshooting.md index f4042df1..dc7d9a3d 100644 --- a/docs/using/troubleshooting.md +++ b/docs/using/troubleshooting.md @@ -4,66 +4,179 @@ When troubleshooting, you may see unexpected behaviors or receive an error messa ## Permission Issues -### Permission denied when mounting local directories +### Permission denied when mounting volumes If you are running a Docker container while mounting a local volume or host directory using the `-v` flag like so: ```bash - docker run -it --rm \ - -v :/home/jovyan/work jupyter/datascience-notebook +docker run -it --rm \ + -p 8888:8888 \ + -v : \ + jupyter/minimal-notebook:latest ``` -you might face permissions issues when trying to access the mounted directory: +you might face permissions issues when trying to access the mounted volume: ```bash -$ ls -l foo.txt --rw-r--r-- 1 root root 0 Jan 131 10:36 foo.txt -$ echo hi > foo.txt --bash: foo.txt: Permission denied +# assuming we mounted the volume in /home/jovyan/stagingarea +$ ls -ld ~/stagingarea/ +drwxr-xr-x 2 root root 4096 Feb 1 12:55 stagingarea/ + +$ touch stagingarea/kale.txt +touch: cannot touch 'stagingarea/kale.txt': Permission denied ``` **Some things to try:** -1. Matching the container's UID/GID with the host's +1. **Change ownership of the volume mount** - Docker handles mounting host directories differently to mounting volumes, even though the syntax is basically the same (i.e. `-v`). + You can change the ownership of the volume mount using the `chown` command. For example, to change the ownership of the volume mount to the jovyan user: + + ```bash + docker run -d \ + -v : \ + -p 8888:8888 \ + --user root \ + -e CHOWN_EXTRA="" \ + -e CHOWN_EXTRA_OPTS="-R" \ + jupyter/minimal-notebook + ``` + + where: + + - `CHOWN_EXTRA=`: will change the owner and group of the specified container directory (non recursive by default). You need to provide full paths starting with `/`. + - `CHOWN_EXTRA_OPTS=R`: will recursively the owner and group of of the directory specified in `CHOWN_EXTRA`. + - `--user root`: must run the container with the root user to perform the change of ownership. + + so now accessing the mount should work as expected: + + ```bash + # assuming we mounted the volume in /home/jovyan/stagingarea + $ ls -ld ~/stagingarea + drwxr-xr-x 2 jovyan users 4096 Feb 1 12:55 stagingarea/ + + $ touch stagingarea/kale.txt + $ ls -la ~/stagingarea/ + -rw-r--r-- 1 jovyan users 0 Feb 1 14:41 kale.txt + ``` + + **Note**: If you are mounting your volume inside the `/home/` directory you can use the `-e CHOWN_HOME=yes` and `CHOWN_HOME_OPTS="-R"` flags instead of the + `-e CHOWN_EXTRA` and `-e CHOWN_EXTRA_OPTS` in the example above. + +2. **Matching the container's UID/GID with the host's** + + Docker handles mounting host directories differently to mounting volumes, even though the syntax is essentially the same (i.e. `-v`). When you initialize a Docker container using the flag `-v` the host directories are bind mounted directly into the container. - Therefore, the permissions and ownership are **exactly the same** as the ones in your local host (including user ids) which may result in permissions errors like the one displayed above. + Therefore, the permissions and ownership are copied over and will be **exactly the same** as the ones in your local host + (including user ids) which may result in permissions errors like the one displayed above. - Suppose your local user has a `UID` of `1000`. To fix the UID discrepancies between your local directories and the container's + Suppose your local user has a `UID` and `GID` of `1234`. To fix the UID discrepancies between your local directories and the container's directoriess, you need to set `NB_UID` and `NB_GID` to match the that of the local user: ```bash - docker run -it --rm \ - -e NB_UID=1000 \ - -e NB_GID=1000 \ - --user root \ - -v :/home/jovyan/work jupyter/datascience-notebook + $ docker run -it --rm \ + --user root \ + -p 8888:8888 \ + -e NB_UID=1234 \ + -e NB_GID=1234 \ + -v $(PWD)/test:/home/jovyan/work \ + jupyter/minimal-notebook:latest + + # you should see an output similar to this + Update jovyan's UID:GID to 1234:1234 + Running as jovyan: bash ``` - Note that you need to use the flag `--user root` to ensure that the `UID` is updated at runtime. + Some important things to notice: - The caveat with this approach is that since these changes are applied at runtime, you will need to to re-run the same - command with the appropriate flags and environment variables if you need to recreate the container. + - You must use `--user root` to ensure that the `UID` is updated at runtime + - The caveat with this approach is that since these changes are applied at runtime, you will need to to re-run the same command + with the appropriate flags and environment variables if you need to recreate the container -**Other tips:** + **Note** that this approach only updates the UID and GID of the existing jovyan user: + + ```bash + $ id + uid=1234(jovyan) gid=1234(jovyan) groups=1234(jovyan),100(users) + ``` + +If on top of changing the UID and GID you also need to create a new user (and thus ensure this user has a home directory) you can use the following command: + +```bash + docker run -it --rm \ + --user root \ + -e NB_UID=1234 \ + -e NB_GID=1234 \ + -e NB_USER=trallard \ + -e CHOWN_HOME=yes \ + -e CHOWN_HOME_OPTS="-R" \ + -w "/home/${NB_USER}" \ + -v $(PWD)/test:/home/trallard/work \ + jupyter/minimal-notebook start.sh + + # expected output + Updated the jovyan user: + - username: jovyan -> trallard + - home dir: /home/jovyan -> /home/trallard + Update trallard's UID:GID to 1234:1234 + Attempting to copy /home/jovyan to /home/trallard... + Success! + Ensuring /home/trallard is owned by 1234:1234 + Running as trallard: bash +``` + +**Additional tips and troubleshooting commands:** - Use absolute paths when using the `-v` flag: ```bash - -v $(PWD)/:/home/jovyan/work + -v $(PWD)/:/home/jovyan/work ``` in this example, we use the syntax `$(PWD)` which will be replaced with the full path to the current directory. The destination path should also be an absolute path starting with a `/` such as `home/jovyan/work`. -- You might want to consider using the Docker native `--user ` and `--group-add `: +- You might want to consider using the Docker native `--user ` and `--group-add users` flags instead of `-e NB_GID` and `-e NB_UID`: ```bash - # note this will use the same UID from the user calling the command + # note this will use the same UID from + # the user calling the command thus matching the local host + docker run -it --rm \ - --user "$(id -u)" --group-add users \ # $(id -u) prints the UID of the user that called docker run - -v :/home/jovyan/work jupyter/datascience-notebook + --user "$(id -u)" --group-add users \ # $(id -u) prints the UID of the user that called docker run + -v :/home/jovyan/work jupyter/datascience-notebook ``` this command will not only launch the container with a specific user UID, but also add the that user to the `users` group so that it can modify the files in the default `/home` and `/opt/conda` directories. + Further avoiding issues when trying to `conda install` additional packages. + +- Use `docker inspect ` and look for the [`Mounts` section](https://docs.docker.com/storage/volumes/#start-a-container-with-a-volume) to verify that the volume was created and mounted accordingly: + + ```json + # for example for a my-vol volume created with docker volume create + + "Mounts": [ + { + "Type": "volume", + "Name": "my-vol", + "Source": "/var/lib/docker/volumes/stagingarea/_data", + "Destination": "/home/jovyan/stagingarea", + "Driver": "local", + "Mode": "z", + "RW": true, + "Propagation": "" + } + ], + ``` + + diff --git a/test/foo b/test/foo new file mode 100644 index 00000000..257cc564 --- /dev/null +++ b/test/foo @@ -0,0 +1 @@ +foo