Infos:
- Used Zammad version: 6.5.0
- Used Zammad installation type: package
- Operating system: Ubuntu 22.04.5 LTS
The elasticsearch service is regularly crashing and needs to be restarted. It seems to be going down and not coming back up during certain updates. I believe this error caused one of the crashes [2024-11-26T01:30:00,102][INFO ][o.e.x.m.MlDailyMaintenanceService] [zammad] Successfully completed [ML] maintenance task: triggerDeleteExpiredDataTask
[2024-11-26T02:41:37,727][WARN ][o.e.c.c.ClusterBootstrapService] [zammad] this node is locked into cluster UUID [QU1EBDDPSoSZsM4mchwG7A] but [cluster.initial_master_nodes] is set to [zammad]; remove this setting to avoid possible data loss caused by subsequent cluster bootstrap attempts; for further information see Important Elasticsearch configuration | Elasticsearch Guide [8.16] | Elastic
[2024-11-26T04:15:57,873][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [zammad] fatal error in thread [elasticsearch[zammad][masterService#updateTask][T#1892]], exiting
java.util.ServiceConfigurationError: Error loading SPI class list from URL: jar:file:///usr/share/elasticsearch/modules/x-pack-security/x-pack-security-8.16.0.jar!/META-INF/services/org.elasticsearch.index.SlowLogFieldProvider
at org.elasticsearch.plugins.spi.SPIClassIterator.loadNextProfile(SPIClassIterator.java:136) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.plugins.spi.SPIClassIterator.hasNext(SPIClassIterator.java:148) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.plugins.PluginsService.createExtensions(PluginsService.java:380) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.plugins.PluginsService.loadServiceProviders(PluginsService.java:335) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.indices.IndicesService.loadSlowLogFieldProvider(IndicesService.java:1432) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.indices.IndicesService.createIndexMapperServiceForValidation(IndicesService.java:829) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.cluster.metadata.MetadataMappingService$PutMappingExecutor.execute(MetadataMappingService.java:111) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.cluster.service.MasterService.innerExecuteTasks(MasterService.java:1075) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:1038) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.cluster.service.MasterService.executeAndPublishBatch(MasterService.java:245) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.lambda$run$2(MasterService.java:1691) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.action.ActionListener.run(ActionListener.java:452) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.run(MasterService.java:1688) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.cluster.service.MasterService$5.lambda$doRun$0(MasterService.java:1283) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.action.ActionListener.run(ActionListener.java:452) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.cluster.service.MasterService$5.doRun(MasterService.java:1262) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:1023) ~[elasticsearch-8.16.0.jar:?]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27) ~[elasticsearch-8.16.0.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
at java.lang.Thread.run(Thread.java:1575) ~[?:?]
Caused by: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/modules/x-pack-security/x-pack-security-8.16.0.jar
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) ~[?:?]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:56) ~[?:?]
at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:172) ~[?:?]
at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99) ~[?:?]
at java.nio.file.Files.readAttributes(Files.java:1865) ~[?:?]
at java.util.zip.ZipFile$Source.get(ZipFile.java:1512) ~[?:?]
at java.util.zip.ZipFile$CleanableResource.(ZipFile.java:744) ~[?:?]
at java.util.zip.ZipFile.(ZipFile.java:254) ~[?:?]
at java.util.zip.ZipFile.(ZipFile.java:181) ~[?:?]
at java.util.jar.JarFile.(JarFile.java:345) ~[?:?]
at sun.net.www.protocol.jar.URLJarFile.(URLJarFile.java:73) ~[?:?]
at sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:59) ~[?:?]
at sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:168) ~[?:?]
at sun.net.www.protocol.jar.JarFileFactory.getOrCreate(JarFileFactory.java:91) ~[?:?]
at sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:117) ~[?:?]
at sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:160) ~[?:?]
at org.elasticsearch.plugins.spi.SPIClassIterator.loadNextProfile(SPIClassIterator.java:112) ~[elasticsearch-8.16.0.jar:?]
… 20 more