[SOLVED] LDAP Performance Issues

Technical assistance
1

Info:

  • Zammad 2.4.x (updated 05/06/2018)
  • RPM Installation Source via Zammad Repository
  • Server OS = RHEL 7 (VMware VM)
  • Chrome 66

LDAP syncs every hour as per the default. The problem is that when this is happening its hanging the web application. There are around 3000 users in our AD although all but 9 of them are skipped according to the filters I have in place. There doesn’t appear to be a simple way to change the base DN to focus on a specific OU so I cannot reduce the number of parsed records. The server is taking around 8 minutes to perform the sync. If any agent makes changes to a ticket while the sync is in progress the interface hangs until the sync process crashes out with a record locking error.

I have made performance related changes as per this thread: https://community.zammad.org/t/high-latency-timeouts-tickets-twice/476/3

The server CPU utilization is at 100% while the sync is running even after the above changes. The server is running 4 CPU’s, 16GB memory, 500GB non-SSD storage. Obviously this level of performance means the helpdesk is almost unusable for 8 minutes of every hour. Am I missing something with the settings? Is it possible to have it only sync once overnight instead of every hour? I’ve tried adjusting the Import_Job code block in the Schedulers.rb file to 1.Day, but this has had no effect.

Steps to reproduce the behavior:

  • Attempt to use the helpdesk while LDAP sync is running.
2

I can’t fix your problem, but I can give you a data point.

On a weekend when no one is using it, I can do a LDAP sync in maybe 20-30 seconds, during a working day it syncs about 4 - 6 accounts a minute, and takes 5+ minutes. It’s still perfectly usable while it’s syncing though.

I had a different problem with LDAP sync in where if there was a communication problem with the ldap server while it was syncing an account, zammad would disable that account. I’m not sure what the problem is yet but I was seeing an account getting disabled 2 or 3 times a day, so I turn the ldap sync off.

You could turn the ldap integration off (it doesn’t delete any accounts) and then run a cron job that turns it on after hours just as a temporary work around. I’m not sure of the details to turn the ldap integration on and off via the api or rails console, but no doubt it’s possible to do.

This is just me guessing here, I’m guessing ldap sync makes a lot of elasticsearch queries, how much memory have you given to elasticsearch?

Maybe in different terminals run top and tail elasticsearch logs, scheduler logs, railserver logs and see what’s it’s doing while you run an ldap sync. Pay attention to swap, if you start swapping your performance is going to tank. reconfigure so swap is rarely used. Swapping out is OK, swapping in, though, is a killer of performance.

I have a bit more memory and more cores than you, these are my settings, I don’t know if they are good or bad.

elasticsearch
“ES_JAVA_OPTS=-Xms16g -Xmx16g”

You’ve got 16G, so you wouldn’t go over 8, and I think 8 assumes nothing else is using memory on the server, so the best figure might be 4 or even lower, maybe 2? I have no idea.

MEMCACHED_SIZE=16G

railsserver
- WEB_CONCURRENCY=8
- MAX_THREADS=32
- MIN_THREADS=8

The above is a lot of processes and threads which uses a lot of memory for rails and a lot of connections to postgres which also uses a lot of memory if all the connections decide to query at the same time.

3

Hi Chesty

Thanks for the input. Just an update on this, after tweaking the settings as per your post and the article I linked to it appears to have sorted the performance problems.

Thanks again.

Sully

4

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.