[solved] Failed to run scheduled job 'Generate user based stats.'. Cause: Failed to run Stats.generate after 10 tries #<RuntimeError: No settings with area 'Dashboard::Stats' defined>

mpmichael · February 22, 2019, 8:45am

Hello,

System: Ubuntu 18.04
Using: Apt sources

since the Migration to Zammad 2.9 we are getting this error (seen in the GUI/System/Monitoring)

* Failed to run scheduled job 'Generate user based stats.'. Cause: Failed to run Stats.generate after 10 tries #<RuntimeError: No settings with area 'Dashboard::Stats' defined>

The production Log says:

E, [2019-02-22T09:34:36.278306 #7886-70256318823760] ERROR -- : execute Stats.generate (try_count 0) exited with error #<RuntimeError: No settings with area 'Dashboard::Stats' defined>
E, [2019-02-22T09:34:37.303971 #7886-70256318823760] ERROR -- : execute Stats.generate (try_count 1) exited with error #<RuntimeError: No settings with area 'Dashboard::Stats' defined>
E, [2019-02-22T09:34:39.332674 #7886-70256318823760] ERROR -- : execute Stats.generate (try_count 2) exited with error #<RuntimeError: No settings with area 'Dashboard::Stats' defined>
E, [2019-02-22T09:34:42.357197 #7886-70256318823760] ERROR -- : execute Stats.generate (try_count 3) exited with error #<RuntimeError: No settings with area 'Dashboard::Stats' defined>
E, [2019-02-22T09:34:46.380354 #7886-70256318823760] ERROR -- : execute Stats.generate (try_count 4) exited with error #<RuntimeError: No settings with area 'Dashboard::Stats' defined>
E, [2019-02-22T09:34:51.406240 #7886-70256318823760] ERROR -- : execute Stats.generate (try_count 5) exited with error #<RuntimeError: No settings with area 'Dashboard::Stats' defined>
E, [2019-02-22T09:34:57.436242 #7886-70256318823760] ERROR -- : execute Stats.generate (try_count 6) exited with error #<RuntimeError: No settings with area 'Dashboard::Stats' defined>
E, [2019-02-22T09:35:04.459956 #7886-70256318823760] ERROR -- : execute Stats.generate (try_count 7) exited with error #<RuntimeError: No settings with area 'Dashboard::Stats' defined>
E, [2019-02-22T09:35:12.480528 #7886-70256318823760] ERROR -- : execute Stats.generate (try_count 8) exited with error #<RuntimeError: No settings with area 'Dashboard::Stats' defined>
E, [2019-02-22T09:35:21.503353 #7886-70256318823760] ERROR -- : execute Stats.generate (try_count 9) exited with error #<RuntimeError: No settings with area 'Dashboard::Stats' defined>

With the click on the Button “Restart failed Jobs” it works for some Minutes - then the Error returns.
System and Zammad was already restarted.
Any Ideas how to solve this?
Best regards
Martin

MrGeneration · February 22, 2019, 10:38am

Your elasticsearch is up and running?
Also, did you adjust any Zammad code?

mpmichael · February 22, 2019, 11:40am

Hello,

thank’s for taking care.

Elastic Seach ist up and running:

● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2019-02-22 10:35:15 CET; 2h 3min ago
     Docs: http://www.elastic.co
  Process: 1086 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS)
 Main PID: 1182 (java)
    Tasks: 47 (limit: 4583)
   CGroup: /system.slice/elasticsearch.service
           └─1182 /usr/bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+AlwaysPreTouch -server -Xss1m -Djava.

Feb 22 10:35:15 support systemd[1]: Starting Elasticsearch...
Feb 22 10:35:15 support systemd[1]: Started Elasticsearch.

The Zammad code was not modified…

Best Regards
Martin

mpmichael · February 22, 2019, 11:52am

Hello,

I was diggin a bit deeper and found some errors ind the /var/log/postgresql/postgresql-10-main.log

2019-02-22 09:59:55.363 CET [13604] zammad@zammad ERROR:  deadlock detected
2019-02-22 09:59:55.363 CET [13604] zammad@zammad DETAIL:  Process 13604 waits for ShareLock on transaction 1031948; blocked by process 8218.
        Process 8218 waits for ShareLock on transaction 1031947; blocked by process 13604.
        Process 13604: SELECT  "taskbars".* FROM "taskbars" WHERE "taskbars"."id" = $1 LIMIT $2 FOR UPDATE
        Process 8218: SELECT  "taskbars".* FROM "taskbars" WHERE "taskbars"."id" = $1 LIMIT $2 FOR UPDATE
2019-02-22 09:59:55.363 CET [13604] zammad@zammad HINT:  See server log for query details.
2019-02-22 09:59:55.363 CET [13604] zammad@zammad CONTEXT:  while locking tuple (5,3) in relation "taskbars"
2019-02-22 09:59:55.363 CET [13604] zammad@zammad STATEMENT:  SELECT  "taskbars".* FROM "taskbars" WHERE "taskbars"."id" = $1 LIMIT $2 FOR UPDATE
2019-02-22 10:34:39.899 CET [1293] LOG:  received fast shutdown request
2019-02-22 10:34:39.903 CET [1293] LOG:  aborting any active transactions
2019-02-22 10:34:39.904 CET [21622] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.904 CET [14751] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.904 CET [20355] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.904 CET [14749] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.908 CET [14866] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.911 CET [13604] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.917 CET [8046] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.918 CET [12357] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.917 CET [10882] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.918 CET [8662] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.918 CET [8016] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.918 CET [12646] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.918 CET [10869] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.918 CET [12648] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.918 CET [8771] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.918 CET [10872] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.918 CET [11135] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.921 CET [7985] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.917 CET [8003] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.921 CET [13601] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.920 CET [12353] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.921 CET [13605] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.918 CET [10871] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.922 CET [12356] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.925 CET [12355] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.925 CET [8218] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.925 CET [8015] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.925 CET [12354] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.928 CET [8776] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:39.929 CET [7991] zammad@zammad FATAL:  terminating connection due to administrator command
2019-02-22 10:34:40.098 CET [1293] LOG:  worker process: logical replication launcher (PID 1306) exited with exit code 1
2019-02-22 10:34:40.160 CET [1301] LOG:  shutting down
2019-02-22 10:34:40.229 CET [1293] LOG:  database system is shut down
2019-02-22 10:35:15.822 CET [1297] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2019-02-22 10:35:15.825 CET [1297] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2019-02-22 10:35:15.881 CET [1305] LOG:  database system was shut down at 2019-02-22 10:34:40 CET
2019-02-22 10:35:15.895 CET [1297] LOG:  database system is ready to accept connections
2019-02-22 10:35:16.434 CET [1326] [unknown]@[unknown] LOG:  incomplete startup packet

Could the problem caused by a Problem in the Database ?
Best regards
Martin

MrGeneration · February 22, 2019, 3:35pm

That’s possible yes.
You might consiver setting max_connections to a higher value than 50 (e.g. 500) and afterwards restart your services:

systemctl restart postgresql
systemctl restart zammad

afterwards you should be fine again

mpmichael · February 22, 2019, 4:40pm

Hello,

thank you for your hint - but this didn’t solve the Problem.

I’ve to investigate further…
Best regards

Martin

mpmichael · February 23, 2019, 7:45am

Hello,

I’ve seen, that apt-get upgrade offered me a update for zammad. I did the update and the error was gone?!
Bevore I had Version 2.9.x after the upgrade the Version shown was 2.9.x again.
A bit strange - looks like a “quickfix” - but anyhow - I’m lucky now.
Best regards
Martin

MrGeneration · February 25, 2019, 11:01am

Maybe the installation scripts weren’t run completely, this might explain it.
Any way, glad it’s fixed for now!

mpmichael · March 8, 2019, 2:23pm

Hello,

yes - it is working fine for about 10 Days. So we can close this issue.

Best regards
Martin

system · July 6, 2019, 2:23pm

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.