Zammad system not processing emails

Technical assistance
1

Infos:

  • Used Zammad version: 3.0.0
  • Used Zammad installation source: rpm
  • Operating system: centos 7
  • Browser + version: Firefox 66.0.5

Expected behavior:

  • Zammad process emails.

  • get unreceived emails after restarting the email process.

Actual behavior:

  • First we did not receive any email from zammad since yesterday. After doing a zammad restart the new emails start coming up but not the old ones.

  • We would like to investigate the main issue and fix it so it not reproduce again.

Please could someone help us on that?

Kindly found out the production.log file shared link:

Thank’s in advance.

2

Well sorry, but a greppet “ERROR”-Logfile won’t help us anywhere.
But aside from that, you got a ton of deadlocks and unexpected database server closed connection errors there you might want to fix.

Please ensure your hardware does meet the minimum requirenments of Zammad.
At least 4GB of RAM, or better like so:
https://docs.zammad.org/en/latest/prerequisites-hardware.html

Check your message log, your error log for database server to see why it dies.
Also ensure your storage is all good and not too slow.

Last but not least, could you please answer the following questions?

  • Is this a Zammad installation with any manual changes?
  • What was the last change before this issue arised?

Please provide the output of the following commands (zammad run rails c):

Overview.count
User.count
User.joins(roles: :permissions).where(roles: { active: true }, permissions: { name: 'ticket.agent', active: true }).where.not(id: 1).count
Ticket.count
Organization.count
3

Hello @MrGeneration

Thank’s for your reply.

You’re right i can see deadlocks timeout from the database logs:

Process 23239 waits for ShareLock on transaction 28677595; blocked by process 9304.
	Process 9304 waits for ShareLock on transaction 28677594; blocked by process 23239.
	Process 23239: SELECT  "taskbars".* FROM "taskbars" WHERE "taskbars"."id" = $1 LIMIT $2 FOR UPDATE
	Process 9304: SELECT  "taskbars".* FROM "taskbars" WHERE "taskbars"."id" = $1 LIMIT $2 FOR UPDATE

I think raising the deadlock timeout could fix it, otherwise an upgrade of the zammad version is required regarding this post: [solved] Failed to run scheduled job 'Generate user based stats.'. Cause: Failed to run Stats.generate after 10 tries #<RuntimeError: No settings with area 'Dashboard::Stats' defined>

Please what do you suggest?

To answer your questions:

• Is this a Zammad installation with any manual changes?
Yes, the change i did is i have increased the amount of the application server by executing
this command:
zammad config:set WEB_CONCURRENCY=2

• What was the last change before this issue arised?
The last change is that the database was upgraded to postgresql11-server-11.5.(Installed in a separate machine).

Please found out the output of the following commands (zammad run rails c):

Overview.count:
=> 19

User.count:
=> 5442

User.joins(roles: :permissions).where(roles: { active: true }, permissions: { name: 'ticket.agent', active: true }).where.not(id: 1).count:
=> 60

Ticket.count:
=> 33908

Organization.count:
=> 5

Extra:
Hardware information: 2x4 CPU Cores and 16GB RAM.

Thank’s for your support.

4

Please any update here?

5

@MrGeneration

We planned to migrate zammad to an aws instance, for that please we would like to have an answer or suggestions regarding the above issue to improve the new zammad installation so we don’t meet the same problem.

Thank you.

6

How many concurrent users do you have? We had the problem that to many tickets in our overview affected the performance. The scheduler Stats.generate couldn’t work fine. The impact was that other processes stopped working, like the process “E-Mail to ticket” & “Ticket to E-Mail”. We have 15 groups and every group has an overview called “Closed Tickets”. There are about 1000-3000 closed tickets. I changed the configuration. All agents just can see the last 5 days of their closed tickets. Now the performance is fine and there isn’t a missing mail anymore.

7

Please note that I cannot provide any help for AWS instances, as I don’t have any experience with AWS.

Anyway, your specs shouldn’t be of any problem.
However, if you have deadlocks and the database lives on another host, it’s proberbly not the Zammad system, but your database system having some kind of issue.

Possibly too high I/O causing the database to be too slow or so.
Especially with slow I/O when updating the same ticket in a very short time period, this can get troublesome.

Issue is that even changing WEB_CONCURRENCY and other tweaks in Zammad most proberbly won’t help you here. Sorry.

8

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.