Zammad Slow performance in new host

Technical assistance
1

Hello Team,

* Zammad Version: 6.3.0
* Installation Type: Source
* Operating System: Ubuntu 22.04
* Browser + Version: Microsoft Edge Version 128.0.2739.42

###Expected Behavior:

Optimal performance.

###Actual Behavior:

Slow performance during busy periods.

###Steps to Reproduce:

We have been working with Zammad for about 5 months, hosted on Azure Cloud. The Azure environment has the following configuration:

Zammad Host:

* Ubuntu 22.04
* 8 CPUs
* 32GB RAM
* env:
	* ZAMMAD_PROCESS_SCHEDULED_JOBS_WORKERS=1
	* MEMCACHE_SERVERS=localhost:11211
	* ZAMMAD_SESSION_JOBS_CONCURRENT=8
	* ZAMMAD_PROCESS_DELAYED_JOBS_WORKERS=8
	* Database: pool: 50

Elastic Host:

* Ubuntu 22.04
* 2 CPUs
* 16GB RAM

Azure Database PostgreSQL:

* 2 CPUs
* 16GB RAM
* 5000 IOPs Storage Performance
* CONFIG:
	* max_connections:2000

We plan to migrate this operation to a dedicated host running Proxmox (qemu/kvm).

The physical host has the following configuration:

* 48 x Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz (2 Sockets)
* 256GB RAM
* 2x2TB NVME ZFS RAID 0

The guest virtual machines are configured as follows:

Zammad Host:

* Ubuntu 22.04
* 16 CPUs
* 32GB RAM
* env(We tried a some diferent configurations):
	* MEMCACHE_SERVERS="localhost:11211"
	* WEB_CONCURRENCY=8
	* ZAMMAD_PROCESS_DELAYED_JOBS_WORKERS=16
	* ZAMMAD_PROCESS_SCHEDULED_JOBS_WORKERS=1
	* ZAMMAD_SESSION_JOBS_CONCURRENT=60

Elastic Host:

* Ubuntu 22.04
* 8 CPUs
* 32GB RAM

PostgreSQL Host:

* Ubuntu 22.04
* 4 CPUs
* 16GB RAM
* CONFIG:
	* max_connections = 200000

When running Zammad on Azure, everything works well consistently. However, after migrating to the dedicated host, we’ve experienced a noticeable drop in performance for entire system (GUI, API, Scheduled Jobs) during busy hours.

We migrated using the procedure outlined in Migrate Zammad to New Host — Zammad System Documentation documentation

Actually we have 15 agents online during busy hours, 15 tickets opened by minute through API and most of than closed automatically.
The database has amount of 200000 tickets.

Some information:

rails r "p Sessions.list.uniq.count"  
I, [2024-08-29T18:08:18.872978 #466541]  INFO -- : ActionCable is using the redis instance at redis://localhost:6379.
I, [2024-08-29T18:08:18.880975#466541-6060]  INFO -- : Using memcached as Rails cache store.
I, [2024-08-29T18:08:18.881068#466541-6060]  INFO -- : Using the File back end for Zammad's web socket session store.
11

rails r "p Overview.count" 
I, [2024-08-29T18:13:58.536268 #468828]  INFO -- : ActionCable is using the redis instance at redis://localhost:6379.
I, [2024-08-29T18:13:58.544970#468828-6060]  INFO -- : Using memcached as Rails cache store.
I, [2024-08-29T18:13:58.545071#468828-6060]  INFO -- : Using the File back end for Zammad's web socket session store.
51

rails r "p Scheduler.where(active: false).pluck(:name)" 
I, [2024-08-29T18:14:50.039008 #469200]  INFO -- : ActionCable is using the redis instance at redis://localhost:6379.
I, [2024-08-29T18:14:50.047149#469200-6060]  INFO -- : Using memcached as Rails cache store.
I, [2024-08-29T18:14:50.047244#469200-6060]  INFO -- : Using the File back end for Zammad's web socket session store.
[]

Can anyone help me with this issue?

Thanks in advanced.

2

Your tunings look somewhat high for the number of cores. Also it seems that you haven’t set the most important tuning parameter: WEB_CONCURRENCY At least on azure site it seems. A web concurrency of 8 should be plenty for the number of calls and agents you’re calling.

4

Thank you very much for your help, @MrGeneration. Indeed, I did not have the WEB_CONCURRENCY parameter enabled on Azure, and it was working as expected there. We noticed the slowdown after migrating to the dedicated server. On the dedicated server, I enabled the WEB_CONCURRENCY=8 parameter, but it did not have any effect.

My knowledge of Ruby on Rails is still limited, and even after reading the documentation, I couldn’t understand how to adjust the parameters in the best way.

Do you have any tips on what I can do to identify the cause of the slowdown?

5

Which config file contains the WEB_CONCURRENCY variable? Thank you.

6

No file, you can set it via the rails console

See the performance Tipps here: Configuration via Environment Variables — Zammad System Documentation documentation