SOLVED - Index creation for elasticsearch: Continue for missing files

Technical assistance
1

Infos:

  • Used Zammad version: 5.2.3
  • Used Zammad installation type: source
  • Operating system: Ubuntu-20.04
  • Browser + version: N/A
  • Elastic search: 7.8.1

Expected behavior:

  • Report errors during index creation, continue anyways

Actual behavior:

  • An error leads to termination of the index creation
zammad@dp-zammad-2004:~$ rake zammad:searchindex:rebuild
Dropping indexes... done.
Deleting pipeline... done.
...
  - Ticket...
rake aborted!
Unable to send Ticket.find(6395).search_index_update_backend backend: #<Errno::ENOENT: No such file or directory @ rb_sysopen - /opt/zammad/storage/fs/1e5a/fe05/e6da6/49767/767ba49/c32be68/e2d22b79481fcfdfea61d531692276c9>
/opt/zammad/app/models/concerns/has_search_index_backend.rb:208:in `rescue in block in search_index_reload'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:202:in `block in search_index_reload'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/config/initializers/active_record_as_batches.rb:27:in `as_batches'
/opt/zammad/config/initializers/active_record_as_batches.rb:44:in `as_batches'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:200:in `search_index_reload'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:54:in `block (5 levels) in <main>'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:53:in `block (4 levels) in <main>'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:51:in `each'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:51:in `block (3 levels) in <main>'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:71:in `block (3 levels) in <main>'

Caused by:
Errno::ENOENT: No such file or directory @ rb_sysopen - /opt/zammad/storage/fs/1e5a/fe05/e6da6/49767/767ba49/c32be68/e2d22b79481fcfdfea61d531692276c9
/opt/zammad/app/models/store/provider/file.rb:30:in `binread'
/opt/zammad/app/models/store/provider/file.rb:30:in `get'
/opt/zammad/app/models/store/file.rb:62:in `content'
/opt/zammad/app/models/store.rb:8:in `content'
/opt/zammad/app/models/ticket/search_index.rb:86:in `search_index_attribute_lookup_file_oversized?'
/opt/zammad/app/models/ticket/search_index.rb:52:in `block (2 levels) in search_index_attribute_lookup'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/app/models/ticket/search_index.rb:48:in `block in search_index_attribute_lookup'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/batches.rb:71:in `block (2 levels) in find_each'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/batches.rb:71:in `each'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/batches.rb:71:in `block in find_each'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/batches.rb:138:in `block in find_in_batches'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/batches.rb:245:in `block in in_batches'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/batches.rb:229:in `loop'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/batches.rb:229:in `in_batches'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/batches.rb:137:in `find_in_batches'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/batches.rb:70:in `find_each'
/opt/zammad/app/models/ticket/search_index.rb:22:in `each'
/opt/zammad/app/models/ticket/search_index.rb:22:in `search_index_attribute_lookup'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:151:in `search_index_update_backend'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:203:in `block in search_index_reload'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/.rvm/gems/ruby-3.0.4/gems/activerecord-6.1.7/lib/active_record/relation/delegation.rb:88:in `each'
/opt/zammad/config/initializers/active_record_as_batches.rb:27:in `as_batches'
/opt/zammad/config/initializers/active_record_as_batches.rb:44:in `as_batches'
/opt/zammad/app/models/concerns/has_search_index_backend.rb:200:in `search_index_reload'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:54:in `block (5 levels) in <main>'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:53:in `block (4 levels) in <main>'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:51:in `each'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:51:in `block (3 levels) in <main>'
/opt/zammad/lib/tasks/zammad/search_index_es.rake:71:in `block (3 levels) in <main>'
Tasks: TOP => zammad:searchindex:rebuild
(See full trace by running task with --trace)

Steps to reproduce the behavior:

I guess the error is caused by me not completely restoring my backup. I restored the
database only but not any “folder”. So maybe the attachments have been stored within folders
on my old machine and these are not missing.

  • Is my assumption correct?
  • Is there a way to cleanup the database? Meaning delete all references/links to non-existing external files?
  • Is there a way to complete the index creation ignoring the errors?

Thanks for your help! I browsed the documentation but somehow I didn’t find anything related to these topics…

Best regards, Uli

2

Answering to myself: I restored the folder /opt/zammad/storage/fs. Since the disk crashed,
there are some files missing in there. I ended up with a couple of records within the table “store_files” pointing to non-existing files. I deleted these records and the related records within “stores” and now the index creation seems to work again.

3

Dear future anon finding this thread.
Please note that the stated “solution” in this thread is a very dangerous operation and not suggested.

This can lead to seriously degraded service, data loss or even a nuked installation.

4

Thx for this clarification. What’s the non-dangerous way to handle this situation? Is there any?

5

There’s non.
Missing attachments means corruption on your storage.

Apart from trying to find out what tickets are affected and removing them, there’s basically nothing you can do. Like with everything that’s gone I’m afraid.

6

OK, so this topic can be closed. Thx for your help and advice.

7

This topic was automatically closed 120 days after the last reply. New replies are no longer allowed.