OTRS Import timeout

Infos:

  • Used Zammad version: 3.0.0
  • Used Zammad installation source: package
  • Operating system: Debian 8
  • Browser + version: Chome 75.0.3770.80 (Official Build) (32-bit)

Expected behavior:

  • Continue importing.

Actual behavior:

  • Import stops after 4 hours. Appears to be the maximum time to a background process/worker.

Steps to reproduce the behavior:

  • Import a long otrs database (we have ≈84.000 tickets) using the gui interface.

There is a way to increase this max import time to 1 day or something upper?

There’s, as far as I’m aware, no time limit in that action.
Please check (and provide) your logfiles, the error should be found inside there. Possibly the combination of production.log of Zammad and OTRS-Logs are needed at this part.

Are both machines inside a data centre or at least on the same site? If one of the machines is e.g. on an office DSL connections, this might also be the root of the problem (e.g. slow connection speed or even disconnects)

Yes, both machines are in the same place.

The exact error message that we get is: Delayed::Worker.max_run_time is only 14400 seconds (4 hours). I have also a screenshot of the error that appears in GUI:

This screenshot is from a second run to confirm the problem an that is the reason because of there is less than 84.000 tickets.

About the log files, i can’t fetch the otrs logs from this import, but I have the logs from production.log, posting only 1000 last lines:
https://www.codepile.net/pile/9LaxNewV

I mean that the problem is near line 643:

@thorsteneckel
Does this info help you to test this or do you need more input?

Great catch @bmsoft-de and welcome to the Community. For such long running imports I’d recommend to use the Zammad rails console for starting the import. To avoid the timeout make sure to run the following command before starting the import Delayed::Worker.max_run_time = 7.days. Please note that this change is only valid for the lifetime of the Zammad rails console.

@MrGeneration can you please add a hint to the console import documentation?

Yes, we checked it, and we think that this limit is not used when running from console (i mean because there is not Delayed::Worker and this is only used for GUI background tasks). So changing this variable doesn’t matters because in console we don’t need it and doesn’t affects to GUI import, there is a way to modify the GUI import timeout?

We also tried to import by console without success at the moment (now we get a new error caused by a fk -maybe then if we don’t found the cause we create a new thread-), and this is the reason of try to use the GUI importer. I haven’t deep in code, but searching in the logs appears to work with diff import, and avoid/skip this error.

I also know that maybe it’s possible import a diff using the rails console as described in docs:
https://docs.zammad.org/en/latest/migration-otrs.html#importing-a-diff

But this also doesn’t work for us, all the time give us the same result: tickets import is done without continue.

You can put that line at the end of the file config/initializers/delayed_jobs_settings_reset.rb and restart all services. Please make sure that you perform a fresh import (by deleting all present data in Zammad again). The diff import only works correctly after a regular import was finished successfully.

Hint has been added and will be available within the next 15 minutes.

1 Like

Hi @MrGeneration, I think that the hint added is confusing, because when running import in console the Delayed::Worker is not used (this is only for GUI background processes). So, running this command do nothing in rails console because his lifetime and scope is only the rails console.

Maybe you can update to:

Sorry for taking so long and asking back.
I’m not really getting smart from the above, can you please check if the below is correct?

If so, I’ll gladly update the docs:

[...]
Extending import time for big installations (optional)
------------------------------------------------------

Optional, if you're having a bigger installation or running in timeouts like:
``Delayed::Worker.max_run_time is only 14400 seconds (4 hours)`` you need to do the following:

For importing via console
^^^^^^^^^^^^^^^^^^^^^^^^^

* open the file ``config/initializers/delayed_jobs_settings_reset.rb`` and add the following at the end of it:
  :: 
    
    Delayed::Worker.max_run_time = 7.days

* Restart the Zammad-Service (``systemctl restart zammad``)

For importing via browser (not recommended on big installations)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Run below in a Zammad console and ensure to not close it during import:
  :: 
    
    Delayed::Worker.max_run_time = 7.days


.. Note:: The above setting is only valid for the lifetime of the Zammad rails console.
  If you close the console, the change is reset to the default value.
 
Enter the following commands in the rails console
-------------------------------------------------
[...]

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.