Unable to rebuild ElasticSearch index after migrate Zammad to new VM (missing attachment in storage)

Technical assistance
1

Infos:

  • Used Zammad version: 5.1.1
  • Used Zammad installation type: Deb package, MySql, File storage
  • Operating system: Ubuntu 20.04

Expected behavior:

Rebuild index to use search in Zammad

Actual behavior:

Error while indexing, file not found (attachment missing in /opt/zammad/storage)

Steps to reproduce the behavior:

root@mgc-srv-1:~# zammad run rake searchindex:rebuild --trace
** Invoke searchindex:rebuild (first_time)
** Invoke environment (first_time)
** Execute environment
nil versions are discouraged and will be deprecated in Rubygems 4
** Invoke searchindex:configured (first_time)
** Invoke environment 
** Execute searchindex:configured
** Invoke searchindex:version_supported (first_time)
** Invoke environment 
** Execute searchindex:version_supported
** Execute searchindex:rebuild
** Execute searchindex:drop
drop indexes...done
** Execute searchindex:drop_pipeline
delete pipeline (pipeline)... done
** Execute searchindex:create
create indexes...done
** Execute searchindex:create_pipeline
create pipeline (pipeline)... done
** Execute searchindex:reload
reload data...
 reload Ticket::State
  - started at 2022-05-19 11:13:14 UTC
	7/7
  - took 0 seconds
 reload Ticket::Priority
  - started at 2022-05-19 11:13:15 UTC
	3/3
  - took 0 seconds
 reload Ticket
  - started at 2022-05-19 11:13:15 UTC
rake aborted!
Unable to send Ticket.find(9747).search_index_update_backend backend: #<Errno::ENOENT: No such file or directory @ rb_sysopen - /opt/zammad/storage/fs/8735/192b/47d5f/10480/8c4db0a/adcaacf/a01bc1bf96ceffb46d676f0d3427c10d>
/opt/zammad/app/models/concerns/has_search_index_backend.rb:231:in `rescue in block in search_index_reload'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:225:in `block in search_index_reload'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/config/initializers/active_record_as_batches.rb:27:in `as_batches'
/opt/zammad/config/initializers/active_record_as_batches.rb:44:in `as_batches'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:223:in `search_index_reload'
/opt/zammad/lib/tasks/search_index_es.rake:50:in `block (3 levels) in <main>'
/opt/zammad/lib/tasks/search_index_es.rake:46:in `each'
/opt/zammad/lib/tasks/search_index_es.rake:46:in `block (2 levels) in <main>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/opt/zammad/lib/tasks/search_index_es.rake:65:in `block (2 levels) in <main>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:80:in `run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/opt/zammad/vendor/bundle/ruby/2.7.0/bin/rake:23:in `load'
/opt/zammad/vendor/bundle/ruby/2.7.0/bin/rake:23:in `<top (required)>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:58:in `load'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:58:in `kernel_load'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:23:in `run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli.rb:479:in `exec'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli.rb:31:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli.rb:25:in `start'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/exe/bundle:49:in `block in <top (required)>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/exe/bundle:37:in `<top (required)>'
/opt/zammad/bin/bundle:121:in `load'
/opt/zammad/bin/bundle:121:in `<main>'

Caused by:
Errno::ENOENT: No such file or directory @ rb_sysopen - /opt/zammad/storage/fs/8735/192b/47d5f/10480/8c4db0a/adcaacf/a01bc1bf96ceffb46d676f0d3427c10d
/opt/zammad/app/models/store/provider/file.rb:30:in `binread'
/opt/zammad/app/models/store/provider/file.rb:30:in `get'
/opt/zammad/app/models/store/file.rb:62:in `content'
/opt/zammad/app/models/store.rb:8:in `content'
/opt/zammad/app/models/ticket/search_index.rb:86:in `search_index_attribute_lookup_file_oversized?'
/opt/zammad/app/models/ticket/search_index.rb:52:in `block (2 levels) in search_index_attribute_lookup'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/app/models/ticket/search_index.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:70:in `block (2 levels) in find_each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:70:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:70:in `block in find_each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:136:in `block in find_in_batches'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:238:in `block in in_batches'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:222:in `loop'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:222:in `in_batches'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:135:in `find_in_batches'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/batches.rb:69:in `find_each'
/opt/zammad/app/models/ticket/search_index.rb:22:in `each'
/opt/zammad/app/models/ticket/search_index.rb:22:in `search_index_attribute_lookup'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:178:in `search_index_update_backend'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:226:in `block in search_index_reload'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/activerecord-6.0.4.8/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/config/initializers/active_record_as_batches.rb:27:in `as_batches'
/opt/zammad/config/initializers/active_record_as_batches.rb:44:in `as_batches'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:223:in `search_index_reload'
/opt/zammad/lib/tasks/search_index_es.rake:50:in `block (3 levels) in <main>'
/opt/zammad/lib/tasks/search_index_es.rake:46:in `each'
/opt/zammad/lib/tasks/search_index_es.rake:46:in `block (2 levels) in <main>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/opt/zammad/lib/tasks/search_index_es.rake:65:in `block (2 levels) in <main>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:80:in `run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/rake-13.0.6/exe/rake:27:in `<top (required)>'
/opt/zammad/vendor/bundle/ruby/2.7.0/bin/rake:23:in `load'
/opt/zammad/vendor/bundle/ruby/2.7.0/bin/rake:23:in `<top (required)>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:58:in `load'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:58:in `kernel_load'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli/exec.rb:23:in `run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli.rb:479:in `exec'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli.rb:31:in `dispatch'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/cli.rb:25:in `start'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/exe/bundle:49:in `block in <top (required)>'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/lib/bundler/friendly_errors.rb:103:in `with_friendly_errors'
/opt/zammad/vendor/bundle/ruby/2.7.0/gems/bundler-2.2.33/exe/bundle:37:in `<top (required)>'
/opt/zammad/bin/bundle:121:in `load'
/opt/zammad/bin/bundle:121:in `<main>'
Tasks: TOP => searchindex:rebuild

Can we ignore missing files to continue indexing ?

2

Did you use backup script and restore in new VM?

3

No, because old installation was not working anymore after 5.1.1 upgrade.
So I copied my database with mysqldump, made a new installation and run the elasticsearch reindex.

My old configuration was storage in database. So normally nothing was missing.

4

I think after the VM crashed I lost some attachments.

Can we delete the ticket which has missing file in storage ?
If yes, how?

5

Removing storage attachments is nothing trivial at all and requires a lot of knowledge of Zammad.
This is the reason that nobody will be able to help you.

I won’t as this is a possible security issue for our copy cats out there. I’m sorry, it’s just too time consuming.

6

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.