Monitoring has fixed interval of 8 Minutes?

sstidl1 · November 19, 2019, 1:51pm

Infos:

Used Zammad version: latest-docker as of 18.11.2019
Used Zammad installation source: (source, package, …): docker
Operating system: debian10
Browser + version: irrelevant

See https://github.com/zammad/zammad/issues/2812

Hi!

I am not sure, if this is intentional:
in monitoring-controller it seems to have a fixed 8 minutes check.

zammad/app/controllers/monitoring_controller.rb Line 81

ttps://github.com/zammad/zammad/blob/1c63e966758f42ec101ffd8adb99606f1958fde4/app/controllers/monitoring_controller.rb#L81

we are getting scheduler may not run (last execution of Stats.generate 38 minutes over) - please contact your system administrator in the latest zammad-docker-compose version

there is an there is an interval of 11 minutes here:

zammad/db/seeds/schedulers.rb

Line 142 in fae1949

so it is very easy to have the monitoring sensor https://YOURSITEHERE/api/v1/monitoring/health_check?token=XXX in the red.

In our case it must be something in addition, as the stats get over an hour old.

please comment.

regards
stefan

sstidl1 · November 19, 2019, 1:54pm

I think this is a bug, but Mr Generation closed it and sent me here

MrGeneration · November 21, 2019, 6:04pm

Hey Stefan,

thanks again for the time you took.
I’ve read your post over and over and think to understand where you’re aiming to. Hopefully I’ll get it right (in this case it’s not a bug ;( )

Zammad regulary checks if key jobs have been run and, at some point, throws a monitoring alert (like yours) where in this case it thinks that your scheduler might not be running.

In fact, it’s possible that your scheduler is perfectly fine, but something else is blocking Zammad or the runtime of this tasks (for whatever reason).

Now, to understand how these 38 minutes come up (and why they don’t match with the 11 minute period), we’ll need to take a closer look on how a scheduler job attribute looks like in Zammad:

Scheduler.find_by(name:'Generate user based stats.')
 => #<Scheduler id: 17, name: "Generate user based stats.", method: "Stats.generate", period: 660, running: 0, last_run: "2019-11-21 17:54:13", prio: 2, pid: "", note: nil, error_message: "", status: "ok", active: true, updated_by_id: 1, created_by_id: 1, created_at: "2019-11-14 14:51:39", updated_at: "2019-11-21 17:54:13">

Our monitoring checks for this key: last_run. It contains the last timestamp when this task has run (no matter if successful or not).

Actuually the following code part is responsible for the scheduler based checks:

github.com

zammad/zammad/blob/fae194918eb8a07036060a1b41f1b33c45e03cad/app/controllers/monitoring_controller.rb#L78-L88


      
          # scheduler running check
          Scheduler.where('active = ? AND period > 300', true).where.not(last_run: nil).order(last_run: :asc, period: :asc).each do |scheduler|
            diff = Time.zone.now - (scheduler.last_run + scheduler.period.seconds)
            next if diff < 8.minutes
          
            issues.push "scheduler may not run (last execution of #{scheduler.method} #{helpers.time_ago_in_words(Time.zone.now - diff.seconds)} over) - please contact your system administrator"
            break
          end
          if Scheduler.where(active: true, last_run: nil).count == Scheduler.where(active: true).count
            issues.push 'scheduler not running'
          end

Zammad will alert you if the job is 8 minutes overdue. The above monitoring message is the result of that. So while Stats.generate should run every 11 minutes, it hasn’t done so for at least 38 minutes.

The above issue can have several reasons but most likely is a performance issue.
Please ensure that your machine (or better ressources) do meet the following requirenments:
https://docs.zammad.org/en/latest/prerequisites-hardware.html

Note that you need at least free 4 GB of ram for your docker-compose to run the Zammad construct.

I hope that helps.

Bests
Marcel

system · March 20, 2020, 6:04pm

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.