Portainer Stack Persistent Storage

nmincone · June 22, 2024, 8:25pm

Infos:

Used Zammad version: latest
Used Zammad installation type: Portainer stack, Debian
Operating system: Bookworm
Browser + version: All

Expected behavior:

Converted docker pull from git repo into a Portainer stack

Actual behavior:

Init fails with URI error.

Steps to reproduce the behavior:

Deploying stack from within Portainer
I was able to get Zammad up and running using docker pull and experiemented with it for a few days. Since then I have been working on a customer Portainer stack to allow easier control of env variables and persistent data locations from within Portainer.
I am having difficulty getting my stack to work properly and was hoping for some insight, perhaps after looking at this for so long I am missing something.

Here is the stack…

---
version: '3.8'

x-shared:
  zammad-service: &zammad-service
    environment: &zammad-environment
      MEMCACHE_SERVERS: ${MEMCACHE_SERVERS}
      POSTGRESQL_DB: ${POSTGRES_DB}
      POSTGRESQL_HOST: ${POSTGRES_HOST}
      POSTGRESQL_USER: ${POSTGRES_USER}
      POSTGRESQL_PASS: ${POSTGRES_PASS}
      POSTGRESQL_PORT: ${POSTGRES_PORT}
      POSTGRESQL_OPTIONS: ${POSTGRESQL_OPTIONS}
      POSTGRESQL_DB_CREATE:
      REDIS_URL: ${REDIS_URL}
      # Backup settings
      BACKUP_DIR: ${BACKUP_DIR}
      BACKUP_TIME: ${BACKUP_TIME}
      HOLD_DAYS: ${HOLD_DAYS}
      TZ: ${TZ}
      # Allow passing in these variables via .env:
      AUTOWIZARD_JSON:
      AUTOWIZARD_RELATIVE_PATH:
      ELASTICSEARCH_ENABLED:
      ELASTICSEARCH_HOST:
      ELASTICSEARCH_PORT:
      ELASTICSEARCH_SCHEMA:
      ELASTICSEARCH_NAMESPACE:
      ELASTICSEARCH_REINDEX:
      ELASTICSEARCH_SSL_VERIFY:
      NGINX_PORT:
      NGINX_SERVER_NAME:
      NGINX_SERVER_SCHEME:
      RAILS_TRUSTED_PROXIES:
      ZAMMAD_WEB_CONCURRENCY:
      ZAMMAD_SESSION_JOBS:
      ZAMMAD_PROCESS_SCHEDULED:
      ZAMMAD_PROCESS_DELAYED_JOBS_WORKERS:
    image: ${IMAGE_REPO}:${VERSION}
    restart: ${RESTART}
    volumes:
      - /home/docker/zammad/storage:/opt/zammad/storage
    depends_on:
      - zammad-memcached
      - zammad-postgresql
      - zammad-redis

services:

  zammad-backup:
    <<: *zammad-service
    command: ["zammad-backup"]
    volumes:
      - /home/docker/zammad/zammad-backup:/var/tmp/zammad
      - /home/docker/zammad/zammad-storage:/opt/zammad/storage:ro
    user: 0:0

  zammad-elasticsearch:
    image: bitnami/elasticsearch:${ELASTICSEARCH_VERSION:-8.12.2}
    restart: ${RESTART}
    volumes:
      - /home/docker/zammad/elasticsearch-data:/bitnami/elasticsearch/data

  zammad-init:
    <<: *zammad-service
    command: ["zammad-init"]
    depends_on:
      - zammad-postgresql
    restart: on-failure
    user: 0:0

  zammad-memcached:
    command: memcached -m 256M
    image: memcached:${MEMCACHE_VERSION:-1.6.25-alpine}
    restart: ${RESTART}

  zammad-nginx:
    <<: *zammad-service
    command: ["zammad-nginx"]
    expose:
      - "${NGINX_PORT}"
    ports:
      - "${NGINX_EXPOSE_PORT}:${NGINX_PORT}"
    depends_on:
      - zammad-railsserver

  zammad-postgresql:
    environment:
      POSTGRES_DB: ${POSTGRES_DB}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASS}
    image: postgres:${POSTGRES_VERSION}
    restart: ${RESTART}
    volumes:
      - /home/docker/zammad/postgresql-data:/var/lib/postgresql/data

  zammad-railsserver:
    <<: *zammad-service
    command: ["zammad-railsserver"]

  zammad-redis:
    image: redis:${REDIS_VERSION}
    restart: ${RESTART}
    volumes:
      - /home/docker/zammad/redis-data:/data

  zammad-scheduler:
    <<: *zammad-service
    command: ["zammad-scheduler"]

  zammad-websocket:
    <<: *zammad-service
    command: ["zammad-websocket"]

volumes:
  elasticsearch-data:
    driver: local
  postgresql-data:
    driver: local
  redis-data:
    driver: local
  zammad-backup:
    driver: local
  zammad-storage:
    driver: local

Here are the env variables…

IMAGE_REPO=ghcr.io/zammad/zammad
VERSION=latest
ELASTICSEARCH_VERSION=8.12.2
MEMCACHE_SERVERS=zammad-memcached:11211
MEMCACHE_VERSION=1.6.25-alpine
NGINX_PORT=8080
NGINX_EXPOSE_PORT=8083
POSTGRES_DB=zammad_production
POSTGRES_PASS=password
POSTGRES_USER=zammad
POSTGRES_HOST=zammad-postgresql
POSTGRES_PORT=5432
POSTGRES_VERSION=15.6-alpine
POSTGRESQL_OPTIONS=?pool=50
REDIS_URL=redis://zammad-redis:6379
REDIS_VERSION=7.2.4-alpine
BACKUP_DIR=/home/nmincone/docker/zammad/backup
BACKUP_TIME=-3:00
HOLD_DAYS=-10
TZ=America/New_York
RESTART=unless-stopped

Any help would be appreciated! I would like to keep persistent data for easier migration/backups on another server.

YetAnotherGerrit · June 24, 2024, 6:18am

Have you tried the original docker-compose.yml? We just had a hackathon where one team updated everything to be deployable via Portainer. It should work out of the box.

nmincone · June 24, 2024, 2:42pm

Hi, yes I have. The stack and env provided above is a direct result of this. There are a number of env variables in the stack that are not in the git files, where it lists
‘Allow passing in these variables via .env:’
Can I assume they go to defaults in the container or do all need to be assigned? Perhaps I missed something??
As stated the docker pull option seemed to work, the Portainer Stack template I used came from here zammad-docker-compose

byPARSE · July 3, 2024, 10:25pm

Hi,

we have all the needed variables now in defaults, yes. You can directly
import the stack from git with only these two settings in portainer:

Repository URL: “GitHub - zammad/zammad-docker-compose: Zammad Docker images for docker-compose”
Repository reference: “refs/heads/master”

Then click “Deploy stack”. After some time (let the init-container do his job first) Zammad can be accessed via Port 8080. That’s it!

Feel free to manipulate some of the default envs via manually inserted variables in portainer.

Here you find the official documentation:
https://docs.zammad.org/en/latest/install/docker-compose.html#deployment-with-portainer

sunny geetings
byPARSE

nmincone · July 6, 2024, 8:47pm

I’ve tried installing via Portainer 10+ times using the directions…
Every container states; waiting for init container to finish install or update…
ElasticSearch logs show; ERROR: Elasticsearch exited unexpectedly, with exit code 1
Postresql logs show; LOG: incomplete startup packet
NGINX logs show; waiting for init container to finish install or update…
Zammad Backup logs show; waiting for init container to finish install or update…
Websocket logs show; waiting for init container to finish install or update…
INit logs show; initialising / updating database…
rake aborted!
URI::InvalidURIError: bad URI(is not URI?): postgres://zammad:E^$6bx@zammad-postgresql:5432/zammad_production?pool=50 (URI::InvalidURIError)

What am I doing incorrectly???

---
version: '3.8'

x-shared:
  zammad-service: &zammad-service
    environment: &zammad-environment
      MEMCACHE_SERVERS: ${MEMCACHE_SERVERS:-zammad-memcached:11211}
      POSTGRESQL_DB: ${POSTGRES_DB:-zammad_production}
      POSTGRESQL_HOST: ${POSTGRES_HOST:-zammad-postgresql}
      POSTGRESQL_USER: ${POSTGRES_USER:-zammad}
      POSTGRESQL_PASS: ${POSTGRES_PASS:-zammad}
      POSTGRESQL_PORT: ${POSTGRES_PORT:-5432}
      POSTGRESQL_OPTIONS: ${POSTGRESQL_OPTIONS:-?pool=50}
      POSTGRESQL_DB_CREATE:
      REDIS_URL: ${REDIS_URL:-redis://zammad-redis:6379}
      # Backup settings
      BACKUP_DIR: "${BACKUP_DIR:-/var/tmp/zammad}"
      BACKUP_TIME: "${BACKUP_TIME:-03:00}"
      HOLD_DAYS: "${HOLD_DAYS:-10}"
      TZ: "${TZ:-Europe/Berlin}"
      # Allow passing in these variables via .env:
      AUTOWIZARD_JSON:
      AUTOWIZARD_RELATIVE_PATH:
      ELASTICSEARCH_ENABLED:
      ELASTICSEARCH_HOST:
      ELASTICSEARCH_PORT:
      ELASTICSEARCH_SCHEMA:
      ELASTICSEARCH_NAMESPACE:
      ELASTICSEARCH_REINDEX:
      ELASTICSEARCH_SSL_VERIFY:
      NGINX_PORT:
      NGINX_SERVER_NAME:
      NGINX_SERVER_SCHEME:
      RAILS_TRUSTED_PROXIES:
      ZAMMAD_WEB_CONCURRENCY:
      ZAMMAD_SESSION_JOBS:
      ZAMMAD_PROCESS_SCHEDULED:
      ZAMMAD_PROCESS_DELAYED_JOBS_WORKERS:
    image: ${IMAGE_REPO:-ghcr.io/zammad/zammad}:${VERSION:-6.3.1-38}
    restart: ${RESTART:-always}
    volumes:
      - /home/docker/zammad/storage:/opt/zammad/storage
    depends_on:
      - zammad-memcached
      - zammad-postgresql
      - zammad-redis

services:

  zammad-backup:
    <<: *zammad-service
    command: ["zammad-backup"]
    volumes:
      - /home/docker/zammad/backup:/var/tmp/zammad
      - /home/docker/zammad/storage:/opt/zammad/storage:ro
    user: 0:0

  zammad-elasticsearch:
    image: bitnami/elasticsearch:${ELASTICSEARCH_VERSION:-8.12.2}
    restart: ${RESTART:-always}
    volumes:
      - /home/docker/zammad/elasticsearch-data:/bitnami/elasticsearch/data

  zammad-init:
    <<: *zammad-service
    command: ["zammad-init"]
    depends_on:
      - zammad-postgresql
    restart: on-failure
    user: 0:0

  zammad-memcached:
    command: memcached -m 256M
    image: memcached:${MEMCACHE_VERSION:-1.6.25-alpine}
    restart: ${RESTART:-always}

  zammad-nginx:
    <<: *zammad-service
    command: ["zammad-nginx"]
    expose:
      - "${NGINX_PORT:-8080}"
    ports:
      - "${NGINX_EXPOSE_PORT:-8080}:${NGINX_PORT:-8080}"
    depends_on:
      - zammad-railsserver

  zammad-postgresql:
    environment:
      POSTGRES_DB: ${POSTGRES_DB:-zammad_production}
      POSTGRES_USER: ${POSTGRES_USER:-zammad}
      POSTGRES_PASSWORD: ${POSTGRES_PASS:-zammad}
    image: postgres:${POSTGRES_VERSION:-15.6-alpine}
    restart: ${RESTART:-always}
    volumes:
      - /home/docker/zammad/postgresql-data:/var/lib/postgresql/data

  zammad-railsserver:
    <<: *zammad-service
    command: ["zammad-railsserver"]

  zammad-redis:
    image: redis:${REDIS_VERSION:-7.2.4-alpine}
    restart: ${RESTART:-always}
    volumes:
      - /home/docker/zammad/redis-data:/data

  zammad-scheduler:
    <<: *zammad-service
    command: ["zammad-scheduler"]

  zammad-websocket:
    <<: *zammad-service
    command: ["zammad-websocket"]

volumes:
  elasticsearch-data:
    driver: local
  postgresql-data:
    driver: local
  redis-data:
    driver: local
  zammad-backup:
    driver: local
  zammad-storage:
    driver: local

env

ELASTICSEARCH_VERSION=8.12.2
IMAGE_REPO=ghcr.io/zammad/zammad
MEMCACHE_SERVERS=zammad-memcached:11211
MEMCACHE_VERSION=1.6.25-alpine
NGINX_PORT=8080
NGINX_EXPOSE_PORT=8008
POSTGRES_DB=zammad_production
POSTGRES_PASS=secret_password
POSTGRES_USER=zammad
POSTGRES_HOST=zammad-postgresql
POSTGRES_PORT=5432
POSTGRES_VERSION=15.6-alpine
POSTGRESQL_OPTIONS=?pool=50
REDIS_URL=redis://zammad-redis:6379
REDIS_VERSION=7.2.4-alpine
RESTART=unless-stopped
VERSION=latest

nmincone · July 7, 2024, 9:40pm

Ok, so the installation problem seemed to be it just did not like my database password… that was causing the URI error.
I can’t get elastic search to work. I’ll look into this next.

nmincone · July 8, 2024, 2:31am

So, I had a rouge Java instance preventing ElasticSearch from running I was able to remove with

ps aux | grep 'java'
kill -9 <PID>

But within Zammad when I click reports I get this error.

Unable to process GET request to elasticsearch URL 'http://zammad-elasticsearch:9200/zammad_production_ticket/_search'. Check the response and payload for detailed information:

Response:
#<UserAgent::Result:0x00007fd615a98fd8 @success=false, @body=nil, @data=nil, @code=0, @content_type=nil, @error="#<RuntimeError: Unable to process http call '#<Net::HTTPServiceUnavailable 503 Service Unavailable readbody=true>'>", @header=nil>

Payload:
{"size":0,"query":{"bool":{"must":[{"range":{"created_at":{"from":"2024-01-01T05:00:00Z","to":"2025-01-01T04:59:59Z"}}},{"bool":{"must":[{"bool":{"must_not":[{"term":{"state.name.keyword":"merged"}}]}}]}}]}},"aggs":{"time_buckets":{"date_histogram":{"field":"created_at","calendar_interval":"month","time_zone":"America/New_York"}}},"sort":[{"updated_at":{"order":"desc"}},"_score"]}

Payload size: 0M

Any idea what may cause this from within the app?

nmincone · July 8, 2024, 6:54pm

I figured it out… I was able to start fress with the docker compose, changing only the NPM port, database password, and persistent data location and I’m up and running!!
Looking forward to test driving Zammad!! Thanks you, and thank you to all the devs working on this project!

system · July 15, 2024, 6:54pm

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.