Refactor docker deployment and user setup (#863)
* Fix #862, address #826 and maybe some other tickets: reimplemented the docker-compose user setup to enhance product setup experience and cover much more cases.
* make unique DB container name and use it in communication from Photoview
* Removed unnecessary healthcheck for photoview from docker-compose.example.yml, as it is defined in the Dockerfile; optimized Dockerfile combining all RUN directives of PROD stage into 1, which will produce single layer and save some space; added Dockerfile-dev, docker-compose-dev.yml, and new "dev" and "dev-down" directives into Makefile, which allows to setup development env in Docker. Instructions of how to use it are in comments at the beginning of Dockerfile-dev and docker-compose-dev.yml files
* Set RWX permissions to the application's working folder for any user, so that the image could be later run with non-root permissions and the app still be able to do needed operations in the FS
* Enhanced the "Getting started" section in the readme; added the `help` target and enhanced comments in the Makefile; commented out the `docker system prune -f` with the comment about the command and why it is there; added optional and commented by default `7zz` commands to the `backup` section of the Makefile
* Use `slim` base image for final photoview image
* Implement SQLite support according to the PR #851
* Removed deprecated `version` line from compose files; optimized dockerfile to build with less layers and run as non-root; mapped only Photoview related services to Watchtower by default instead of updating all running images on a host; added template for Postgres to the .env; reverted compose executable definition, so the new compose is called when present; added a tip about `lnav` to help
* fix a typo in the username; add support of PostgreSQL; split and optimize backup target in Makefile
* Fixed some typos and styling in Readme, excluded dev-environment setup from the PR; added a list of tips on how to secure Photoview in the Advanced setup section of Readme
* Implemented many security improvements, suggested by @Omar007, switched to the dedicated Darktable's repo to install the latest released version, as asked in #935; switched Watchtower to labels instead of profiles
* forgot the compose file
* move face models back to /app folder; comment out and document unnecessary vars in compose; fix a typo in a few vars
* Exclude Makefile in the root folder from git; documented multiple mounts case better; fixed incorrect SQLite DB path
* Fixed several bugs after complete testing cycle with all 3 DBs
* removed hardcoded port in Dockerfile
* Pin the major version for the `photoview` image for stability
* Revert back to the port 80 inside the container on product owner's request
* Provide a minimal compose file and update the readme accordingly
* Handle incorrect media file and folder permissions; set correct permissions for storage folder; fix healthcheck command for postgres
---------
Co-authored-by: Konstantin Koval <kkb@ukr.net>
2024-05-15 10:58:02 +02:00
|
|
|
MAKEFLAGS += --always-make
|
|
|
|
|
|
|
|
-include .env
|
|
|
|
export
|
|
|
|
|
|
|
|
## Ensure compatibility with "docker-compose" (old) and "docker compose" (new).
|
|
|
|
HAS_DOCKER_COMPOSE_NO_DASH := $(shell docker compose version)
|
|
|
|
ifdef HAS_DOCKER_COMPOSE_NO_DASH
|
|
|
|
DOCKER_COMPOSE=docker compose
|
|
|
|
else
|
|
|
|
DOCKER_COMPOSE=docker-compose
|
|
|
|
endif
|
|
|
|
## If you want to use only the new compose command, comment previous section
|
|
|
|
## and uncomment next line:
|
|
|
|
# DOCKER_COMPOSE=docker compose
|
|
|
|
|
|
|
|
help:
|
|
|
|
@echo 'PhotoView Docker Compose management scenarios simplification'
|
|
|
|
@echo 'USAGE:'
|
|
|
|
@echo 'make <target>'
|
|
|
|
@echo ''
|
|
|
|
@echo 'Targets:'
|
|
|
|
@echo ' help Prints this usage info.'
|
|
|
|
@echo ' all Pulls fresh Docker images from the Registry and (re)starts the service.'
|
|
|
|
@echo ' Useful for the 1st start or update scenarios.'
|
|
|
|
@echo ' update The same as `all`, created for convenience.'
|
|
|
|
@echo ' start Creates folders for service data in the ${HOST_PHOTOVIEW_LOCATION} if not exist,'
|
|
|
|
@echo ' and starts the service. Optionally runs a Docker system cleanup, if uncommented.'
|
|
|
|
@echo ' stop Just stops the service, keeping all containers and volumes in Docker.'
|
|
|
|
@echo ' restart Simply stops and starts the service.'
|
|
|
|
@echo ' backup Verifies service database and creates new service backup'
|
|
|
|
@echo ' in the ${HOST_PHOTOVIEW_BACKUP}/<date of execution> using .tar.xz by default.'
|
|
|
|
@echo ' If you want to use 7zz instead (which is faster), read the comment on top of the target script.'
|
|
|
|
@echo ' pull Pulls fresh Docker images from the Registry.'
|
|
|
|
@echo ' readable Makes sure that the ${HOST_PHOTOVIEW_MEDIA_ROOT} and all its files and subdirectories'
|
|
|
|
@echo ' are searchable and readable by other users.'
|
|
|
|
@echo ' terminal Starts a Bash shell session inside the `photoview` container for troubleshooting.'
|
|
|
|
@echo ' logs Shows the last 100 lines (if the command not modified) from the log,'
|
|
|
|
@echo ' stays listening for new lines, and shows them interactively. Ctrl + C to exit.'
|
|
|
|
@echo ' It can be used as a source for a log viewer, like "make logs | lnav", so that you can'
|
|
|
|
@echo ' interactively filter and search for needed info.'
|
|
|
|
@echo ' down The same as `stop`, but also removes containers and volumes from Docker. Your data is safe.'
|
|
|
|
@echo ' remove Removes the service from Docker, including all items.'
|
|
|
|
@echo ' uninstall Stops and removes the service from Docker, including all items.'
|
|
|
|
@echo ''
|
|
|
|
all: pull restart
|
|
|
|
uninstall: down remove
|
|
|
|
restart: stop start
|
|
|
|
update: pull restart
|
|
|
|
pull:
|
|
|
|
$(DOCKER_COMPOSE) pull --ignore-pull-failures
|
|
|
|
start:
|
|
|
|
@## The next line is for MariaDB. If you use SQLite or PostgreSQL, comment it out and uncomment the corresponding line instead
|
|
|
|
mkdir -p ${HOST_PHOTOVIEW_LOCATION}/database/mariadb
|
|
|
|
@## The next line is for SQLite
|
|
|
|
@# mkdir -p ${HOST_PHOTOVIEW_LOCATION}/database
|
|
|
|
@## The next line is for PostgreSQL
|
|
|
|
@# mkdir -p ${HOST_PHOTOVIEW_LOCATION}/database/postgres
|
2024-06-23 14:28:02 +02:00
|
|
|
@# If you don't want to give 777 permissions to this folder, create a group with GID=999 (or user with UID=999 and group
|
|
|
|
@# with GID=999 and that user) and make the folder being owned by it, so you can give the 570 or 750 permissions correspondingly.
|
Refactor docker deployment and user setup (#863)
* Fix #862, address #826 and maybe some other tickets: reimplemented the docker-compose user setup to enhance product setup experience and cover much more cases.
* make unique DB container name and use it in communication from Photoview
* Removed unnecessary healthcheck for photoview from docker-compose.example.yml, as it is defined in the Dockerfile; optimized Dockerfile combining all RUN directives of PROD stage into 1, which will produce single layer and save some space; added Dockerfile-dev, docker-compose-dev.yml, and new "dev" and "dev-down" directives into Makefile, which allows to setup development env in Docker. Instructions of how to use it are in comments at the beginning of Dockerfile-dev and docker-compose-dev.yml files
* Set RWX permissions to the application's working folder for any user, so that the image could be later run with non-root permissions and the app still be able to do needed operations in the FS
* Enhanced the "Getting started" section in the readme; added the `help` target and enhanced comments in the Makefile; commented out the `docker system prune -f` with the comment about the command and why it is there; added optional and commented by default `7zz` commands to the `backup` section of the Makefile
* Use `slim` base image for final photoview image
* Implement SQLite support according to the PR #851
* Removed deprecated `version` line from compose files; optimized dockerfile to build with less layers and run as non-root; mapped only Photoview related services to Watchtower by default instead of updating all running images on a host; added template for Postgres to the .env; reverted compose executable definition, so the new compose is called when present; added a tip about `lnav` to help
* fix a typo in the username; add support of PostgreSQL; split and optimize backup target in Makefile
* Fixed some typos and styling in Readme, excluded dev-environment setup from the PR; added a list of tips on how to secure Photoview in the Advanced setup section of Readme
* Implemented many security improvements, suggested by @Omar007, switched to the dedicated Darktable's repo to install the latest released version, as asked in #935; switched Watchtower to labels instead of profiles
* forgot the compose file
* move face models back to /app folder; comment out and document unnecessary vars in compose; fix a typo in a few vars
* Exclude Makefile in the root folder from git; documented multiple mounts case better; fixed incorrect SQLite DB path
* Fixed several bugs after complete testing cycle with all 3 DBs
* removed hardcoded port in Dockerfile
* Pin the major version for the `photoview` image for stability
* Revert back to the port 80 inside the container on product owner's request
* Provide a minimal compose file and update the readme accordingly
* Handle incorrect media file and folder permissions; set correct permissions for storage folder; fix healthcheck command for postgres
---------
Co-authored-by: Konstantin Koval <kkb@ukr.net>
2024-05-15 10:58:02 +02:00
|
|
|
mkdir -p ${HOST_PHOTOVIEW_LOCATION}/storage
|
|
|
|
chmod 777 ${HOST_PHOTOVIEW_LOCATION}/storage
|
|
|
|
$(DOCKER_COMPOSE) up -d --remove-orphans
|
|
|
|
@## Uncomment the next line if you want to run an automatic cleanup of Docker leftovers
|
|
|
|
@## Make sure to read the Docker documentation to understand how it works
|
|
|
|
@## Please note that this command is applied to the Docker host affecting all hosted services, not only the PhotoView
|
|
|
|
@# docker system prune -f
|
|
|
|
stop:
|
|
|
|
$(DOCKER_COMPOSE) stop
|
|
|
|
down:
|
|
|
|
$(DOCKER_COMPOSE) down -v
|
|
|
|
remove:
|
|
|
|
$(DOCKER_COMPOSE) rm -s -v
|
|
|
|
terminal:
|
|
|
|
$(DOCKER_COMPOSE) exec photoview bash
|
|
|
|
logs:
|
|
|
|
$(DOCKER_COMPOSE) logs --tail=100 -f
|
|
|
|
readable:
|
|
|
|
find ${HOST_PHOTOVIEW_MEDIA_ROOT} -type d -exec chmod o+rx {} \;
|
|
|
|
find ${HOST_PHOTOVIEW_MEDIA_ROOT} -type f -exec chmod o+r {} \;
|
|
|
|
## By default the `backup` uses MariaDB. If you use SQLite or PostgreSQL, replace the `backup-mariadb` with corresponding target
|
|
|
|
## keeping the order of the calls the same
|
|
|
|
## -----------------------
|
|
|
|
## To see the content of the *.tar.xz use the command `tar -tvJf archive_name.tar.xz`
|
|
|
|
## To unpack the *.tar.xz into current folder use the command `tar -xJf archive_name.tar.xz`
|
|
|
|
## -----------------------
|
|
|
|
## The backup script creates .tar.xz archives. This type of archives provides great compression rate, but utilizes a lot of
|
|
|
|
## resources and time. It was selected, because it is pre-installed on most distros.
|
|
|
|
## However, you could replace it with the 7zz, which uses much less resources with comparable compression rate.
|
|
|
|
## Make sure to install the 7zz first and then comment out the 2 lines with tar command (1st one in the corresponding DB backup
|
|
|
|
## target, 2nd one in the `backup-post` target), uncomment corresponding lines with 7zz command
|
|
|
|
backup: backup-pre backup-mariadb backup-post
|
|
|
|
backup-pre:
|
|
|
|
mkdir -p ${HOST_PHOTOVIEW_BACKUP}
|
|
|
|
mkdir ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`
|
|
|
|
backup-mariadb:
|
|
|
|
$(DOCKER_COMPOSE) exec mariadb mysqlcheck -u root --password=${MARIADB_ROOT_PASSWORD} --check --check-upgrade --flush --process-views=YES --auto-repair --all-databases
|
|
|
|
$(DOCKER_COMPOSE) exec mariadb mysqlcheck -u root --password=${MARIADB_ROOT_PASSWORD} --optimize --flush --auto-repair --all-databases
|
|
|
|
$(DOCKER_COMPOSE) exec mariadb mariadb-dump -u root --password=${MARIADB_ROOT_PASSWORD} -e -x --all-databases -- > ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/mariaDB_mysql_dump.sql
|
|
|
|
tar -cJf ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/mariaDB_mysql_dump.tar.xz ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/mariaDB_mysql_dump.sql --remove-files
|
|
|
|
@# 7zz a -mx=9 ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/mariaDB_mysql_dump.7z ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/mariaDB_mysql_dump.sql && rm ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/mariaDB_mysql_dump.sql
|
|
|
|
backup-sqlite:
|
|
|
|
$(DOCKER_COMPOSE) exec photoview sh -c 'echo "PRAGMA integrity_check;" | sqlite3 ${PHOTOVIEW_SQLITE_PATH}'
|
|
|
|
$(DOCKER_COMPOSE) exec photoview sh -c 'echo "VACUUM;" | sqlite3 ${PHOTOVIEW_SQLITE_PATH}'
|
|
|
|
$(DOCKER_COMPOSE) exec photoview sh -c 'echo ".backup \"${PHOTOVIEW_SQLITE_PATH}_backup.db\"" | sqlite3 ${PHOTOVIEW_SQLITE_PATH}'
|
|
|
|
tar -cJf ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/sqlite_backup.tar.xz ${HOST_PHOTOVIEW_LOCATION}/database/photoview.db_backup.db && rm ${HOST_PHOTOVIEW_LOCATION}/database/photoview.db_backup.db
|
|
|
|
@# 7zz a -mx=9 ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/sqlite_backup.7z ${HOST_PHOTOVIEW_LOCATION}/database/photoview.db_backup.db && rm ${HOST_PHOTOVIEW_LOCATION}/database/photoview.db_backup.db
|
|
|
|
backup-postgres:
|
|
|
|
$(DOCKER_COMPOSE) exec postgres sh -c 'PGPASSWORD=${PGSQL_PASSWORD} psql -U ${PGSQL_USER} -d ${PGSQL_DATABASE} -c "VACUUM (VERBOSE, ANALYZE)"'
|
|
|
|
$(DOCKER_COMPOSE) exec postgres sh -c 'PGPASSWORD=${PGSQL_PASSWORD} pg_dump -U ${PGSQL_USER} -c --if-exists -C -v ${PGSQL_DATABASE} --' > ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/postgres_pg_dump.sql
|
|
|
|
tar -cJf ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/postgres_pg_dump.tar.xz ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/postgres_pg_dump.sql && rm ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/postgres_pg_dump.sql
|
|
|
|
@# 7zz a -mx=9 ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/postgres_pg_dump.7z ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/postgres_pg_dump.sql && rm ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/postgres_pg_dump.sql
|
|
|
|
backup-post:
|
|
|
|
cp ${HOST_PHOTOVIEW_LOCATION}/Makefile ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/
|
|
|
|
cp ${HOST_PHOTOVIEW_LOCATION}/docker-compose.yml ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/
|
|
|
|
cp ${HOST_PHOTOVIEW_LOCATION}/.env ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/
|
|
|
|
tar -cJf ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/storage.tar.xz ${HOST_PHOTOVIEW_LOCATION}/storage
|
|
|
|
@# 7zz a -mx=9 ${HOST_PHOTOVIEW_BACKUP}/`date +%Y-%m-%d`/storage.7z ${HOST_PHOTOVIEW_LOCATION}/storage
|