Docker Consolidation Project, Part Six

Docker Consolidation Project, Part Six
Photo by Harshil Gudka / Unsplash

Build a Dedicated Docker Test Server

So I've gotten a number of my containers successfully migrated over to DOCKER-01 so the last thing I wanted to work on for this was to make a test machine that I can use to play with new containers.

Steps performed on HV-03 via Windows Admin Center

  • I navigate to Virtual Machine
  • I click on Add and then New
    • Name: TEST-01
    • Generation: 2 (Recommended)
    • Virtual CPUs: 4
    • Memory (GB): 16
    • Network: Default external switch
    • New Disk:
      • Size: 100GB
    • Select Install an operating system later
    • I click on Create
  • I select TEST-01
  • I click on Settings
  • I go to Disks
    • I select Use an existing virtual hard disk or ISO image file
      • Path: c:\Images\Debian\debian-12.1.0-amd64-netinst.iso
    • I click on OK
    • I click on Save disk settings
  • I move to Boot Order
    • DVD
    • Hard Disk
    • Network adapter
    • I click on Save boot order
  • I navigate to Security
    • Enable Secure Boot: Yes
    • Template: Microsoft UEFI Certificate Authority
    • I click on Save security setting
  • I click on Close

Install TEST-01 Operating System

Steps performed on TEST-01 via console

  • I click on Install
  • Language: English
  • Location: United States
  • Keyboard: American English
  • Hostname: TEST-01
  • Domain name: domain.local
  • I set root's password
  • Full Name: david
  • Username: david
  • I set david's password
  • Timezone: Eastern
  • Partition: Guided - use entire disk
  • Scan extra installation media: no
  • Package Manager: United States
  • Archive mirror: deb.debian.org
  • HTTP proxy information: (blank)
  • Participate in the package usage survey: yes
  • Software to install:
    • SSH Server
    • standard system utilities
  • I click Continue
  • I unmount my Debian ISO and reboot the server

Configure Operating System

Configure Sudo

Steps performed on TEST-01

  • apt install sudo
  • /sbin/adduser david sudo
    • Adding user 'david' to group 'sudo' ...
    • Done.

Configure Network

Steps performed on TEST-01

  • nano /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug eth0
iface eth0 inet static
address 10.10.10.XXX
gateway 10.10.10.254
netmask 255.255.255.0
  • I save the file interfaces
  • systemctl restart networking

Steps performed on DC-02 via Windows Admin Center

  • I navigate to DNS
  • I select domain.local
  • I click on Create a new DNS record
    • DNS record type: HOST (A)
    • Record Name: TEST1
    • FQDN: TEST1.domain.local
    • IP Address: 10.10.10.XXX
    • Time to live: 3600
    • I click on Create

Configure SSH Key

Steps performed on desktop via Windows Terminal

  • ssh-keygen -t ed25519
  • Move-Item -Path c:\Users\david\filename* -Destination c:\Users\david\.ssh -Force
  • I open Windows Terminal's Settings
  • I open the JSON file and add the following:
            {
                "colorScheme": "Ubuntu-ColorScheme",
                "commandline": "ssh -i \"~/.ssh/test-01\" [email protected]",
                "experimental.retroTerminalEffect": false,
                "font": 
                {
                    "face": "Cascadia Code"
                },
                "guid": "{0caa0dad-35be-5f56-a8ff-XXXXXXXXXXXX}",
                "hidden": false,
                "name": "TEST-01",
                "tabTitle": "TEST-01"
            },
  • I make sure the GUID is unique and save JSON

Configure SSH

Steps performed on TEST-01

  • mkdir /home/david/.ssh
  • nano /home/david/.ssh/authorized_keys
  • I paste in the public key and save authorized_keys
  • chmod 600 /home/david/.ssh/authorized_keys
  • sudo nano /etc/ssh/sshd_config
    • PermitRootLogin no
    • PubkeyAuthentication yes
    • PubkeyAcceptedKeyTypes ssh-ed25519
    • PasswordAuthentication no
    • AuthorizedKeysFile /home/david/.ssh/authorized_keys
  • I save the file sshd_config
  • sudo sshd -t
  • sudo systemctl restart ssh

Configure Firewall

Steps performed on TEST-01

  • sudo apt install ufw
  • sudo ufw allow 22
  • sudo ufw enable

Configure Fail2Ban

Steps performed on TEST-01

  • sudo apt install fail2ban
  • sudo nano /etc/fail2ban/jail.local
[DEFAULT]
bantime = 100m
findtime = 5m
maxretry = 5
ignoreip = 10.10.10.XXX

[sshd]
enabled = true
port = ssh
logpath = %(sshd_log)s
backend = %(sshd_backend)s
  • I save the file jail.local
  • sudo systemctl enable fail2ban
  • sudo systemctl start fail2ban

Install Docker and Docker Compose

Steps performed on TEST-01

  • sudo apt install docker
  • sudo apt install docker-compose

Well that takes care of my test server, which I wanted to have running before I migrated Photoprism and Paperless-NGX. I've been using both quite a lot and so I have a lot of data in each and I would rather not have to start over with them. So my plan is to test migrating them to TEST-01 and when successful I can just repeat the steps on DOCKER-01. Seems reasonable to me.

Migrate Paperless-NGX

It looks like the recommended way to do this is with the exporter and importer. I'll start by setting up a container for Paperless-NGX on TEST-01.

Steps performed on TEST-01

  • sudo mkdir /apps/Paperless-NGX
  • cd /apps/Paperless-NGX
  • sudo nano docker-compose.env
PAPERLESS_URL=http://test1.domain.local:8000
PAPERLESS_TIME_ZONE=America/Detroit
PAPERLESS_OCR_LANGUAGE=eng
PAPERLESS_SECRET_KEY=
  • I saved docker-compose.env
  • sudo nano docker-compose.yml
# docker-compose file for running paperless from the Docker Hub.
# This file contains everything paperless needs to run.
# Paperless supports amd64, arm and arm64 hardware.
#
# All compose files of paperless configure paperless in the following way:
#
# - Paperless is (re)started on system boot, if it was running before shutdown.
# - Docker volumes for storing data are managed by Docker.
# - Folders for importing and exporting files are created in the same directory
#   as this file and mounted to the correct folders inside the container.
# - Paperless listens on port 8000.
#
# In addition to that, this docker-compose file adds the following optional
# configurations:
#
# - Instead of SQLite (default), MariaDB is used as the database server.
# - Apache Tika and Gotenberg servers are started with paperless and paperless
#   is configured to use these services. These provide support for consuming
#   Office documents (Word, Excel, Power Point and their LibreOffice counter-
#   parts.
#
# To install and update paperless with this file, do the following:
#
# - Copy this file as 'docker-compose.yml' and the files 'docker-compose.env'
#   and '.env' into a folder.
# - Run 'docker-compose pull'.
# - Run 'docker-compose run --rm webserver createsuperuser' to create a user.
# - Run 'docker-compose up -d'.
#
# For more extensive installation and update instructions, refer to the
# documentation.

version: "3.4"
services:
  broker:
    image: docker.io/library/redis:7
    restart: unless-stopped
    volumes:
#      - redisdata:/data
      - /var/lib/docker/volumes/paperless_redisdata:/data
  db:
    image: docker.io/library/mariadb:10
    restart: unless-stopped
    volumes:
#      - dbdata:/var/lib/mysql
      - /var/lib/docker/volumes/paperless_dbdata:/var/lib/mysql
    environment:
      MARIADB_HOST: paperless
      MARIADB_DATABASE: paperless
      MARIADB_USER: paperless
      MARIADB_PASSWORD: paperless
      MARIADB_ROOT_PASSWORD: paperless
    ports:
      - "3306:3306"

  webserver:
    image: ghcr.io/paperless-ngx/paperless-ngx:latest
    restart: unless-stopped
    depends_on:
      - db
      - broker
      - gotenberg
      - tika
    ports:
      - 8000:8000
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000"]
      interval: 30s
      timeout: 10s
      retries: 5
    volumes:
#      - data:/usr/src/paperless/data
      - /apps/Paperless-NGX/data:/usr/src/paperless/data
#      - media:/usr/src/paperless/media
      - /mnt/Paperless-NGX:/usr/src/paperless/media
      - ./export:/usr/src/paperless/export
      - /apps/Paperless-NGX/consume:/usr/src/paperless/consume
    env_file: docker-compose.env
    environment:
      PAPERLESS_REDIS: redis://broker:6379
      PAPERLESS_DBENGINE: mariadb
      PAPERLESS_DBHOST: db
      PAPERLESS_DBUSER: paperless # only needed if non-default username
      PAPERLESS_DBPASS: paperless # only needed if non-default password
      PAPERLESS_DBPORT: 3306
      PAPERLESS_TIKA_ENABLED: 1
      PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000
      PAPERLESS_TIKA_ENDPOINT: http://tika:9998

  gotenberg:
# Updated the image from gotenberg:7.4 to gotenberg:7.8
    image: docker.io/gotenberg/gotenberg:7.8
    restart: unless-stopped
# These lines were part of the original compose file but appear to have been depreciated for newer version.
#    environment:
#      CHROMIUM_DISABLE_ROUTES: 1
# These lines were added on the update from gotenberg:7.4 to gotenberg:7.8
    # The gotenberg chromium route is used to convert .eml files. We do not
    # want to allow external content like tracking pixels or even javascript.
    command:
      - "gotenberg"
      - "--chromium-disable-javascript=true"
      - "--chromium-allow-list=file:///tmp/.*"

  tika:
    image: ghcr.io/paperless-ngx/tika:latest
    restart: unless-stopped

#volumes:
#  data:
#  media:
#  dbdata:
#  redisdata:
  • I saved the docker-compose.yml file

I work towards setting up a test mount on my fileserver for this test.

..... and this is a real bummer because at this point I lost the rest of my notes for migrating Paperless-NGX. Obsidian crashed, and either it lost the rest of my notes or iCloud did. Either way, it was successful though so I was able to remove my test environment from TEST-01 and decommissioned PAPERLESS-01.