Unable to rebuild ElasticSearch index after migrate Zammad to new VM (missing attachment in storage)

Infos:

  • Used Zammad version: 5.1.1
  • Used Zammad installation type: Deb package, MySql, File storage
  • Operating system: Ubuntu 20.04

Expected behavior:

Rebuild index to use search in Zammad

Actual behavior:

Error while indexing, file not found (attachment missing in /opt/zammad/storage)

Steps to reproduce the behavior:

root@mgc-srv-1:~# zammad run rake searchindex:rebuild --trace
** Invoke searchindex:rebuild (first_time)
** Invoke environment (first_time)
** Execute environment
nil versions are discouraged and will be deprecated in Rubygems 4
** Invoke searchindex:configured (first_time)
** Invoke environment 
** Execute searchindex:configured
** Invoke searchindex:version_supported (first_time)
** Invoke environment 
** Execute searchindex:version_supported
** Execute searchindex:rebuild
** Execute searchindex:drop
drop indexes...done
** Execute searchindex:drop_pipeline
delete pipeline (pipeline)... done
** Execute searchindex:create
create indexes...done
** Execute searchindex:create_pipeline
create pipeline (pipeline)... done
** Execute searchindex:reload
reload data...
 reload Ticket::State
  - started at 2022-05-19 11:13:14 UTC
	7/7
  - took 0 seconds
 reload Ticket::Priority
  - started at 2022-05-19 11:13:15 UTC
	3/3
  - took 0 seconds
 reload Ticket
  - started at 2022-05-19 11:13:15 UTC
rake aborted!
Unable to send Ticket.find(9747).search_index_update_backend backend: #<Errno::ENOENT: No such file or directory @ rb_sysopen - /opt/zammad/storage/fs/8735/192b/47d5f/10480/8c4db0a/adcaacf/a01bc1bf96ceffb46d676f0d3427c10d>
/opt/zammad/app/models/concerns/has_search_index_backend.rb:231:in `rescue in block in search_index_reload'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:225:in `block in search_index_reload'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/config/initializers/active_record_as_batches.rb:27:in `as_batches'
/opt/zammad/config/initializers/active_record_as_batches.rb:44:in `as_batches'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:223:in `search_index_reload'
/opt/zammad/lib/tasks/search_index_es.rake:50:in `block (3 levels) in <main>'
/opt/zammad/lib/tasks/search_index_es.rake:46:in `each'
/opt/zammad/lib/tasks/search_index_es.rake:46:in `block (2 levels) in <main>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/opt/zammad/lib/tasks/search_index_es.rake:65:in `block (2 levels) in <main>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:80:in `run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/opt/zammad/vendor/bundle/ruby/2.7.0/bin/rake:23:in `load'
/opt/zammad/vendor/bundle/ruby/2.7.0/bin/rake:23:in `<top (required)>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:58:in `load'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:58:in `kernel_load'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:23:in `run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli.rb:479:in `exec'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli.rb:31:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli.rb:25:in `start'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/exe/bundle:49:in `block in <top (required)>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/exe/bundle:37:in `<top (required)>'
/opt/zammad/bin/bundle:121:in `load'
/opt/zammad/bin/bundle:121:in `<main>'

Caused by:
Errno::ENOENT: No such file or directory @ rb_sysopen - /opt/zammad/storage/fs/8735/192b/47d5f/10480/8c4db0a/adcaacf/a01bc1bf96ceffb46d676f0d3427c10d
/opt/zammad/app/models/store/provider/file.rb:30:in `binread'
/opt/zammad/app/models/store/provider/file.rb:30:in `get'
/opt/zammad/app/models/store/file.rb:62:in `content'
/opt/zammad/app/models/store.rb:8:in `content'
/opt/zammad/app/models/ticket/search_index.rb:86:in `search_index_attribute_lookup_file_oversized?'
/opt/zammad/app/models/ticket/search_index.rb:52:in `block (2 levels) in search_index_attribute_lookup'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/app/models/ticket/search_index.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:70:in `block (2 levels) in find_each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:70:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:70:in `block in find_each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:136:in `block in find_in_batches'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:238:in `block in in_batches'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:222:in `loop'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:222:in `in_batches'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:135:in `find_in_batches'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:69:in `find_each'
/opt/zammad/app/models/ticket/search_index.rb:22:in `each'
/opt/zammad/app/models/ticket/search_index.rb:22:in `search_index_attribute_lookup'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:178:in `search_index_update_backend'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:226:in `block in search_index_reload'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/config/initializers/active_record_as_batches.rb:27:in `as_batches'
/opt/zammad/config/initializers/active_record_as_batches.rb:44:in `as_batches'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:223:in `search_index_reload'
/opt/zammad/lib/tasks/search_index_es.rake:50:in `block (3 levels) in <main>'
/opt/zammad/lib/tasks/search_index_es.rake:46:in `each'
/opt/zammad/lib/tasks/search_index_es.rake:46:in `block (2 levels) in <main>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/opt/zammad/lib/tasks/search_index_es.rake:65:in `block (2 levels) in <main>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:80:in `run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/opt/zammad/vendor/bundle/ruby/2.7.0/bin/rake:23:in `load'
/opt/zammad/vendor/bundle/ruby/2.7.0/bin/rake:23:in `<top (required)>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:58:in `load'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:58:in `kernel_load'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:23:in `run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli.rb:479:in `exec'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli.rb:31:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli.rb:25:in `start'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/exe/bundle:49:in `block in <top (required)>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/exe/bundle:37:in `<top (required)>'
/opt/zammad/bin/bundle:121:in `load'
/opt/zammad/bin/bundle:121:in `<main>'
Tasks: TOP => searchindex:rebuild

Can we ignore missing files to continue indexing ?

Did you use backup script and restore in new VM?

No, because old installation was not working anymore after 5.1.1 upgrade.
So I copied my database with mysqldump, made a new installation and run the elasticsearch reindex.

My old configuration was storage in database. So normally nothing was missing.

I think after the VM crashed I lost some attachments.

Can we delete the ticket which has missing file in storage ?
If yes, how?

Removing storage attachments is nothing trivial at all and requires a lot of knowledge of Zammad.
This is the reason that nobody will be able to help you.

I won’t as this is a possible security issue for our copy cats out there. I’m sorry, it’s just too time consuming.

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.