Migration from Kayako - running into issues with ticket import - looking for general advice

Infos:

  • Used Zammad version: v6.3.1
  • Used Zammad installation type: Package Install
  • Operating system: Ubuntu 24.04 LTS
  • Browser + version: Multiple, Chrome, Firefox, Edge

Looking for more general guidence here rather than help with a specific item here, but here goes;

Expected behavior: Currently are attempting to perform a import from Kayako and we are seeing the following error in the window that displays the status updates;

‘’‘{undefined local variable or method `e’ for Sequencer::Unit::Import::Kayako::Request:Class}‘’’

In looking into the system it appears to have properly imported the Groups, Organizations, and Users, but get’s stuck on the tickets showing a progress of 3168/3700 tickets - and no matter how long we let it sit no further progress is shown.

Running a command line to check the status of the update as per the documentation shows the following;

‘’‘{
root@support:/var/log/zammad# zammad run rails r “pp ImportJob.find_by(name: ‘Import::Kayako’)”
#<ImportJob:0x00007fee2071efb0
id: 1,
name: “Import::Kayako”,
dry_run: false,
payload: {},
result:
{“Groups”=>{“skipped”=>0, “created”=>9, “updated”=>0, “unchanged”=>0, “failed”=>0, “deactivated”=>0, “sum”=>9, “total”=>9},
“Organizations”=>{“skipped”=>0, “created”=>69143, “updated”=>0, “unchanged”=>0, “failed”=>0, “deactivated”=>0, “sum”=>69143, “total”=>69143},
“Users”=>{“skipped”=>0, “created”=>247573, “updated”=>0, “unchanged”=>0, “failed”=>0, “deactivated”=>0, “sum”=>247573, “total”=>247573},
“Tickets”=>{“skipped”=>7, “created”=>3607, “updated”=>4, “unchanged”=>0, “failed”=>0, “deactivated”=>0, “sum”=>3618, “total”=>3700},
“error”=>"undefined local variable or method `e’ for Sequencer::Unit::Import::Kayako::Request:Class"},
started_at: Wed, 21 Aug 2024 22:28:48.223000000 UTC +00:00,
finished_at: Thu, 22 Aug 2024 09:06:28.987000000 UTC +00:00,
created_at: Wed, 21 Aug 2024 22:28:48.006000000 UTC +00:00,
updated_at: Thu, 22 Aug 2024 09:06:28.988000000 UTC +00:00>
}‘’’

In looking into the production log I can find the specific error listed as being a in the production log and that points to a specific ticket but there’s nothing special about it - so I have removed the ticket, rolled back the server and started the process again, however I wonder is there a better way to handle this type of situation and avoid having to restart the entire migration process over? I’ve got 500k+ tickets that we are looking at having to move and if it’s going to be restarting the process this frequently this is going to be a long haul to get things moved over.

Any better suggestions for managing that migration process?

Hi @azilliax_far. Please provide a snippet out of the production.log file when the error happened. The trace might be interesting + helpful to understand what happens (and maybe we’re able to provide a fix afterward).

I wound up rolling back and starting the import process again after removing that specific problem ticket. The second run of the import is already past the point where the last one failed out. If it locks up again I’ll grab the log and share what I can here.

1 Like

Ok so the issue did reoccur, here’s a copy of the section of the prodcution log where I can find that specific error;

{
E, [2024-08-22T22:00:01.224941#15808-149140] ERROR -- : ImportJob 'Import::Kayako' failed: undefined local variable or method `e' for Sequencer::Unit::Import::Kayako::Request:Class
E, [2024-08-22T22:00:01.225141#15808-149140] ERROR -- : undefined local variable or method `e' for Sequencer::Unit::Import::Kayako::Request:Class (NameError)
lib/sequencer/unit/import/kayako/requester.rb:19:in `rescue in block in request'
lib/sequencer/unit/import/kayako/requester.rb:8:in `block in request'
lib/sequencer/unit/import/kayako/requester.rb:7:in `times'
lib/sequencer/unit/import/kayako/requester.rb:7:in `request'
lib/sequencer/unit/import/kayako/request.rb:18:in `response'
lib/sequencer/unit/common/provider/attribute.rb:25:in `value'
lib/sequencer/unit/common/provider/attribute.rb:30:in `ignore?'
lib/sequencer/unit/common/provider/attribute.rb:6:in `process'
lib/sequencer/unit/base.rb:247:in `process'
lib/sequencer.rb:76:in `block (4 levels) in process'
lib/mixin/start_finish_logger.rb:9:in `log_start_finish'
lib/sequencer.rb:75:in `block (3 levels) in process'
lib/sequencer/state.rb:151:in `process'
lib/sequencer.rb:73:in `block (2 levels) in process'
lib/sequencer/units.rb:27:in `block in each'
lib/sequencer/units.rb:26:in `each'
lib/sequencer/units.rb:26:in `each'
lib/sequencer.rb:71:in `each_with_index'
lib/sequencer.rb:71:in `block in process'
lib/mixin/start_finish_logger.rb:9:in `log_start_finish'
lib/sequencer.rb:69:in `process'
lib/sequencer.rb:24:in `process'
lib/sequencer/unit/import/kayako/sub_sequence/generic.rb:14:in `block in process'
lib/sequencer/unit/import/kayako/sub_sequence/generic.rb:12:in `loop'
lib/sequencer/unit/import/kayako/sub_sequence/generic.rb:12:in `each_with_index'
lib/sequencer/unit/import/kayako/sub_sequence/generic.rb:12:in `process'
lib/sequencer/unit/import/common/model/mixin/skip/action.rb:42:in `process'
lib/sequencer/unit/base.rb:247:in `process'
lib/sequencer.rb:76:in `block (4 levels) in process'
lib/mixin/start_finish_logger.rb:9:in `log_start_finish'
lib/sequencer.rb:75:in `block (3 levels) in process'
lib/sequencer/state.rb:151:in `process'
lib/sequencer.rb:73:in `block (2 levels) in process'
lib/sequencer/units.rb:27:in `block in each'
lib/sequencer/units.rb:26:in `each'
lib/sequencer/units.rb:26:in `each'
lib/sequencer.rb:71:in `each_with_index'
lib/sequencer.rb:71:in `block in process'
lib/mixin/start_finish_logger.rb:9:in `log_start_finish'
lib/sequencer.rb:69:in `process'
lib/sequencer.rb:24:in `process'
lib/sequencer/unit/import/kayako/perform.rb:13:in `block in process'
lib/sequencer/unit/import/kayako/perform.rb:12:in `each'
lib/sequencer/unit/import/kayako/perform.rb:12:in `process'
lib/sequencer/unit/import/common/model/mixin/skip/action.rb:42:in `process'
lib/sequencer/unit/base.rb:247:in `process'
lib/sequencer.rb:76:in `block (4 levels) in process'
lib/mixin/start_finish_logger.rb:9:in `log_start_finish'
lib/sequencer.rb:75:in `block (3 levels) in process'
lib/sequencer/state.rb:151:in `process'
lib/sequencer.rb:73:in `block (2 levels) in process'
lib/sequencer/units.rb:27:in `block in each'
lib/sequencer/units.rb:26:in `each'
lib/sequencer/units.rb:26:in `each'
lib/sequencer.rb:71:in `each_with_index'
lib/sequencer.rb:71:in `block in process'
lib/mixin/start_finish_logger.rb:9:in `log_start_finish'
lib/sequencer.rb:69:in `process'
lib/sequencer.rb:24:in `process'
lib/sequencer/unit/import/kayako/sub_sequence/generic.rb:14:in `block in process'
lib/sequencer/unit/import/kayako/sub_sequence/generic.rb:12:in `loop'
lib/sequencer/unit/import/kayako/sub_sequence/generic.rb:12:in `each_with_index'
lib/sequencer/unit/import/kayako/sub_sequence/generic.rb:12:in `process'
lib/sequencer/unit/base.rb:247:in `process'
lib/sequencer.rb:76:in `block (4 levels) in process'
lib/mixin/start_finish_logger.rb:9:in `log_start_finish'
lib/sequencer.rb:75:in `block (3 levels) in process'
lib/sequencer/state.rb:151:in `process'
lib/sequencer.rb:73:in `block (2 levels) in process'
lib/sequencer/units.rb:27:in `block in each'
lib/sequencer/units.rb:26:in `each'
lib/sequencer/units.rb:26:in `each'
lib/sequencer.rb:71:in `each_with_index'
lib/sequencer.rb:71:in `block in process'
lib/mixin/start_finish_logger.rb:9:in `log_start_finish'
lib/sequencer.rb:69:in `process'
lib/sequencer.rb:24:in `process'
lib/import/mixin/sequence.rb:17:in `process'
lib/import/kayako.rb:8:in `start'
app/models/import_job.rb:26:in `start'
app/jobs/async_import_job.rb:11:in `perform'
lib/background_services/service/process_delayed_jobs.rb:25:in `block (2 levels) in launch'
lib/background_services/service/process_delayed_jobs.rb:23:in `block in launch'
lib/background_services/service/process_delayed_jobs.rb:20:in `loop'
lib/background_services/service/process_delayed_jobs.rb:20:in `launch'
lib/background_services/service.rb:27:in `block in run'
lib/application_handle_info.rb:19:in `use'
lib/background_services/service.rb:33:in `block in run_in_service_context'
lib/background_services/service.rb:32:in `run_in_service_context'
lib/background_services/service.rb:26:in `run'
lib/background_services.rb:67:in `block in start_as_thread'
}

I’m not sure what to look at here - this time around it’s not seeming to reference a specific ticket this time around.

Hi @azilliax_far. There was an issue in the used code when it comes down to exception handling. We improved it. In case you want to try it out, you have to update to the latest version and start the import again. Maybe it’s giving more information afterward what the real issue is about.

1 Like

Ok, I’ve run a update from the source and I’ve started the migration process again, will post back here with the results once It’s had a chance to process.

1 Like

Well so far I’m further along in the import process than last time around so hopefully that update will do the trick and see it through to the end. @fliebe92 thanks for helping out.

Ok, because I don’t want to be that guy who posts and never replies about the eventual outcome we were able to figure out the specific tickets that were causing problems with the import and given that it was only a couple of basically spam tickets we simply removed those from Kayako and then the import was able to complete.

We just had to roll back in the logs, identify the last ticket that was mentioned in the logs, and that pointed us to where the problems where coming from.

1 Like

Thank you for keeping us updated! :slight_smile: