Scheduler fails with "ActiveRecord::ConnectionTimeoutError: could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds); all pooled connections were in use>"

Technical assistance
3

Thanks, I’m not exactly familiar with how all max thread limits affect each other.
So I reverted all environment settings back to default, increased the postgresql limit, and the issue still shows. What could be the next debugging step, or what should I try now?

PS.
From what I’ve read (link below), this issue has likely not to do with postgresql but with rails configuration. That’s why I tried upping/changing the rails thread limit environment variables.

https :// judoscale .com/blog/active-record-connection-pool

root@svrdocker01:/home/username/docker_zammad# docker-compose up -d
Starting docker_zammad_zammad-elasticsearch_1 ... done
Starting docker_zammad_zammad-postgresql_1    ... done
Starting docker_zammad_zammad-redis_1         ... done
Starting docker_zammad_zammad-memcached_1     ... done
Starting docker_zammad_zammad-init_1          ... done
Starting docker_zammad_zammad-scheduler_1     ... done
Starting docker_zammad_zammad-railsserver_1   ... done
Starting docker_zammad_zammad-websocket_1     ... done
Starting docker_zammad_zammad-nginx_1         ... done

root@svrdocker01:/home/username/docker_zammad# docker logs -f docker_zammad_zammad-scheduler_1
[..snip..]
E, [2024-05-15T10:24:08.457665#1-150100] ERROR -- : execute Channel.fetch (try_count 0) exited with error #<ActiveRecord::ConnectionTimeoutError: could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds); all pooled connections were in use> in: 5.000267993 seconds.

username@svrdocker01:~/docker_zammad$ docker exec -it  docker_zammad_zammad-scheduler_1 /bin/bash
zammad@0be657ce42d3:~$ cat config/database/database.yml
default: &default
  pool: 50
  timeout: 15000
  encoding: utf8
  adapter: postgresql

production:
  <<: *default
  pool: 50
  timeout: 15000
  database: zammad_production

username@svrdocker01:~/docker_zammad$ docker exec -it  docker_zammad_zammad-postgresql_1 /bin/bash
15d11848aeb6:/# cat ./var/lib/postgresql/data/postgresql.conf | grep max
max_connections = 5000                  # (change requires restart)

This is my .env (possibly relevant bits):

~/docker_zammad$ cat .env
ELASTICSEARCH_VERSION=8.12.2
IMAGE_REPO=ghcr.io/zammad/zammad
MEMCACHE_SERVERS=zammad-memcached:11211
MEMCACHE_VERSION=1.6.25-alpine
POSTGRES_VERSION=15.6-alpine
REDIS_VERSION=7.2.4-alpine
RESTART=always
VERSION=6.3.0       # including minor updates

And docker-compose.yml (should be default):

x-shared:
  zammad-service: &zammad-service
    environment: &zammad-environment
      MEMCACHE_SERVERS: ${MEMCACHE_SERVERS}
      POSTGRESQL_DB: ${POSTGRES_DB}
      POSTGRESQL_HOST: ${POSTGRES_HOST}
      POSTGRESQL_USER: ${POSTGRES_USER}
      POSTGRESQL_PASS: ${POSTGRES_PASS}
      POSTGRESQL_PORT: ${POSTGRES_PORT}
      REDIS_URL: ${REDIS_URL}
      # Allow passing in these variables via .env:
      AUTOWIZARD_JSON:
      AUTOWIZARD_RELATIVE_PATH:
      ELASTICSEARCH_ENABLED:
      ELASTICSEARCH_HOST:
      ELASTICSEARCH_PORT:
      ELASTICSEARCH_SCHEMA:
      ELASTICSEARCH_NAMESPACE:
      ELASTICSEARCH_REINDEX:
      ELASTICSEARCH_SSL_VERIFY:
      NGINX_PORT:
      NGINX_SERVER_NAME:
      NGINX_SERVER_SCHEME:
      POSTGRESQL_DB_CREATE:
      POSTGRESQL_OPTIONS:
      RAILS_TRUSTED_PROXIES:
      ZAMMAD_WEB_CONCURRENCY:
      ZAMMAD_SESSION_JOBS:
      ZAMMAD_PROCESS_SCHEDULED:
      ZAMMAD_PROCESS_DELAYED_JOBS_WORKERS: