Merge pull request #2189 from tdonohue/docker_cleanup

Minor docker / compose updates based on recent changes to backend

Dockerfile

@@ -2,20 +2,27 @@
 # See https://github.com/DSpace/dspace-angular/tree/main/docker for usage details
 
 FROM node:18-alpine
-WORKDIR /app
-ADD . /app/
-EXPOSE 4000
 
 # Ensure Python and other build tools are available
 # These are needed to install some node modules, especially on linux/arm64
 RUN apk add --update python3 make g++ && rm -rf /var/cache/apk/*
 
+WORKDIR /app
+ADD . /app/
+EXPOSE 4000
+
 # We run yarn install with an increased network timeout (5min) to avoid "ESOCKETTIMEDOUT" errors from hub.docker.com
 # See, for example https://github.com/yarnpkg/yarn/issues/5540
 RUN yarn install --network-timeout 300000
 
+# When running in dev mode, 4GB of memory is required to build & launch the app.
+# This default setting can be overridden as needed in your shell, via an env file or in docker-compose.
+# See Docker environment var precedence: https://docs.docker.com/compose/environment-variables/envvars-precedence/
+ENV NODE_OPTIONS="--max_old_space_size=4096"
+
 # On startup, run in DEVELOPMENT mode (this defaults to live reloading enabled, etc).
 # Listen / accept connections from all IP addresses.
 # NOTE: At this time it is only possible to run Docker container in Production mode
-# if you have a public IP. See https://github.com/DSpace/dspace-angular/issues/1485
+# if you have a public URL. See https://github.com/DSpace/dspace-angular/issues/1485
+ENV NODE_ENV development
 CMD yarn serve --host 0.0.0.0

docker/README.md

@@ -6,7 +6,20 @@
 If you wish to run DSpace on Docker in production, we recommend building your own Docker images. You are welcome to borrow ideas/concepts from the below images in doing so. But, the below images should not be used "as is" in any production scenario.
 ***
 
-## 'Dockerfile' in root directory
+## Overview
+The scripts in this directory can be used to start the DSpace User Interface (frontend) in Docker.
+Optionally, the backend (REST API) might also be started in Docker.
+
+For additional options/settings in starting the backend (REST API) in Docker, see the Docker Compose
+documentation for the backend: https://github.com/DSpace/DSpace/blob/main/dspace/src/main/docker-compose/README.md
+
+## Root directory
+
+The root directory of this project contains all the Dockerfiles which may be referenced by
+the Docker compose scripts in this 'docker' folder.
+
+### Dockerfile
+
 This Dockerfile is used to build a *development* DSpace 7 Angular UI image, published as 'dspace/dspace-angular'
 
 ```
@@ -20,6 +33,8 @@ Admins to our DockerHub repo can manually publish with the following command.
 docker push dspace/dspace-angular:dspace-7_x
 ```
 
+### Dockerfile.dist
+
 The `Dockerfile.dist` is used to generate a *production* build and runtime environment.
 
 ```bash
@@ -29,7 +44,7 @@ docker build -f Dockerfile.dist -t dspace/dspace-angular:dspace-7_x-dist .
 
 A default/demo version of this image is built *automatically*.
 
-## docker directory
+## 'docker' directory
 - docker-compose.yml
   - Starts DSpace Angular with Docker Compose from the current branch.  This file assumes that a DSpace 7 REST instance will also be started in Docker.
 - docker-compose-rest.yml
@@ -54,25 +69,41 @@ docker-compose -f docker/docker-compose.yml build
 
 ## To start DSpace (REST and Angular) from your branch
 
+This command provides a quick way to start both the frontend & backend from this single codebase
 ```
 docker-compose -p d7 -f docker/docker-compose.yml -f docker/docker-compose-rest.yml up -d
 ```
 
+Keep in mind, you may also start the backend by cloning the 'DSpace/DSpace' GitHub repository separately. See the next section.
+
+
 ## Run DSpace REST and DSpace Angular from local branches.
+
+This section assumes that you have clones *both* the 'DSpace/DSpace' and 'DSpace/dspace-angular' GitHub
+repositories. When both are available locally, you can spin up both in Docker and have them work together.
+
 _The system will be started in 2 steps. Each step shares the same docker network._
 
-From DSpace/DSpace (build as needed)
+From 'DSpace/DSpace' clone (build first as needed):
 ```
 docker-compose -p d7 up -d
 ```
 
-From DSpace/DSpace-angular
+NOTE: More detailed instructions on starting the backend via Docker can be found in the [Docker Compose instructions for the Backend](https://github.com/DSpace/DSpace/blob/main/dspace/src/main/docker-compose/README.md).
+
+From 'DSpace/dspace-angular' clone (build first as needed)
 ```
 docker-compose -p d7 -f docker/docker-compose.yml up -d
 ```
 
+At this point, you should be able to access the UI from http://localhost:4000,
+and the backend at http://localhost:8080/server/
+
 ## Run DSpace Angular dist build with DSpace Demo site backend
 
+This allows you to run the Angular UI in *production* mode, pointing it at the demo backend
+(https://api7.dspace.org/server/).
+
 ```
 docker-compose -f docker/docker-compose-dist.yml pull
 docker-compose -f docker/docker-compose-dist.yml build
@@ -104,9 +135,10 @@ Load assetstore content and trigger a re-index of the repository
 docker-compose -p d7 -f docker/cli.yml -f docker/cli.assetstore.yml run --rm dspace-cli
 ```
 
-## End to end testing of the rest api (runs in travis).
+## End to end testing of the REST API (runs in GitHub Actions CI).
-_In this instance, only the REST api runs in Docker using the Entities dataset. Travis will perform CI testing of Angular using Node to drive the tests._
+_In this instance, only the REST api runs in Docker using the Entities dataset. GitHub Actions will perform CI testing of Angular using Node to drive the tests.  See `.github/workflows/build.yml` for more details._
 
+This command is only really useful for testing our Continuous Integration process.
 ```
-docker-compose -p d7ci -f docker/docker-compose-travis.yml up -d
+docker-compose -p d7ci -f docker/docker-compose-ci.yml up -d
 ```

docker/docker-compose-dist.yml

@@ -19,6 +19,10 @@ services:
       DSPACE_UI_HOST: dspace-angular
       DSPACE_UI_PORT: '4000'
       DSPACE_UI_NAMESPACE: /
+      # NOTE: When running the UI in production mode (which the -dist image does),
+      # these DSPACE_REST_* variables MUST point at a public, HTTPS URL.
+      # This is because Server Side Rendering (SSR) currently requires a public URL,
+      # see this bug: https://github.com/DSpace/dspace-angular/issues/1485
       DSPACE_REST_SSL: 'true'
       DSPACE_REST_HOST: api7.dspace.org
       DSPACE_REST_PORT: 443

docker/docker-compose-rest.yml

@@ -39,7 +39,7 @@ services:
       # proxies.trusted.ipranges: This setting is required for a REST API running in Docker to trust requests 
       # from the host machine. This IP range MUST correspond to the 'dspacenet' subnet defined above.
       proxies__P__trusted__P__ipranges: '172.23.0'
-    image: dspace/dspace:dspace-7_x-test
+    image: "${DOCKER_OWNER:-dspace}/dspace:${DSPACE_VER:-dspace-7_x-test}"
     depends_on:
     - dspacedb
     networks:
@@ -82,8 +82,7 @@ services:
   # DSpace Solr container  
   dspacesolr:
     container_name: dspacesolr
-    # Uses official Solr image at https://hub.docker.com/_/solr/
+    image: "${DOCKER_OWNER:-dspace}/dspace-solr:${DSPACE_VER:-dspace-7_x}"
-    image: solr:8.11-slim
     # Needs main 'dspace' container to start first to guarantee access to solr_configs
     depends_on:
     - dspace
@@ -96,28 +95,26 @@ services:
     tty: true
     working_dir: /var/solr/data
     volumes:
-    # Mount our "solr_configs" volume available under the Solr's configsets folder (in a 'dspace' subfolder)
-    # This copies the Solr configs from main 'dspace' container into 'dspacesolr' via that volume
-    - solr_configs:/opt/solr/server/solr/configsets/dspace
     # Keep Solr data directory between reboots
     - solr_data:/var/solr/data
     # Initialize all DSpace Solr cores using the mounted local configsets (see above), then start Solr
     # * First, run precreate-core to create the core (if it doesn't yet exist). If exists already, this is a no-op
-    # * Second, copy updated configs from mounted configsets to this core. If it already existed, this updates core
+    # * Second, copy configsets to this core:
-    #   to the latest configs. If it's a newly created core, this is a no-op.
+    #   Updates to Solr configs require the container to be rebuilt/restarted:
+    #   `docker-compose -p d7 -f docker/docker-compose.yml -f docker/docker-compose-rest.yml up -d --build dspacesolr`
     entrypoint:
     - /bin/bash
     - '-c'
     - |
       init-var-solr
       precreate-core authority /opt/solr/server/solr/configsets/dspace/authority
-      cp -r -u /opt/solr/server/solr/configsets/dspace/authority/* authority
+      cp -r /opt/solr/server/solr/configsets/dspace/authority/* authority
       precreate-core oai /opt/solr/server/solr/configsets/dspace/oai
-      cp -r -u /opt/solr/server/solr/configsets/dspace/oai/* oai
+      cp -r /opt/solr/server/solr/configsets/dspace/oai/* oai
       precreate-core search /opt/solr/server/solr/configsets/dspace/search
-      cp -r -u /opt/solr/server/solr/configsets/dspace/search/* search
+      cp -r /opt/solr/server/solr/configsets/dspace/search/* search
       precreate-core statistics /opt/solr/server/solr/configsets/dspace/statistics
-      cp -r -u /opt/solr/server/solr/configsets/dspace/statistics/* statistics
+      cp -r /opt/solr/server/solr/configsets/dspace/statistics/* statistics
       exec solr -f
 volumes:
   assetstore: