[solved]Postgresql Service starts before the NFS Shares can mount


#1
  • Used Zammad version: 2.9.x
  • Used Zammad installation source: Zammad install via DEB
  • Operating system: Linux Debian 9.x

Expected behavior:

  • After a server restart, Zammad should function properly.

Actual behavior:

Note: I moved the Postgresql database to an NFS drive to run the database directly on a new one if the server fails.
Everything works perfectly until the moment I restart the VM.
Zammad then has problems loading certain things.
Restarting the postgresql service solves the problem. Do you have an idea to solve the problem without restarting the service every time ? Of course there are only 3 words each time after restarting the VM.
You could solve this with the fstab and a script to restart the postgresql service but are there any other solutions ?

Apr 15 13:52:30 srvvzammad001 zammad-websocket-1.service[1268]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/applicat
Apr 15 13:52:30 srvvzammad001 zammad-websocket-1.service[1268]:   /opt/zammad/config/environment.rb:5:in `<top (required)>'
Apr 15 13:52:30 srvvzammad001 zammad-websocket-1.service[1268]:   script/websocket-server.rb:14:in `require'
Apr 15 13:52:30 srvvzammad001 zammad-websocket-1.service[1268]:   script/websocket-server.rb:14:in `<top (required)>'
Apr 15 13:52:30 srvvzammad001 systemd[1]: zammad-worker-1.service: Service hold-off time over, scheduling restart.
Apr 15 13:52:30 srvvzammad001 systemd[1]: Stopped zammad-worker-1.service.
Apr 15 13:52:30 srvvzammad001 systemd[1]: Started zammad-worker-1.service.
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]: => Booting Puma
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]: => Rails 5.1.6.2 application starting in production
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]: => Run `rails server -h` for more startup options
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]: Exiting
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]: bundler: failed to load command: script/rails (script/rails)
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]: PG::ConnectionBad: could not connect to server: No such file or directory
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]: #011Is the server running locally and accepting
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]: #011connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/pg-0.21.0/lib/pg.rb:56:in `initialize'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/pg-0.21.0/lib/pg.rb:56:in `new'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/pg-0.21.0/lib/pg.rb:56:in `connect'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/codapter.rb:697:in `connect'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/codapter.rb:221:in `initialize'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/codapter.rb:38:in `new'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/codapter.rb:38:in `postgresql_connection'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/conection_pool.rb:759:in `new_connection'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/conection_pool.rb:803:in `checkout_new_connection'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/conection_pool.rb:782:in `try_to_checkout_new_connection'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/conection_pool.rb:743:in `acquire_connection'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/conection_pool.rb:500:in `checkout'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/conection_pool.rb:374:in `connection'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/conection_pool.rb:931:in `retrieve_connection'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/coetrieve_connection'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.6.2/lib/active_record/connection'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/config/initializers/db_preferences_postgresql.rb:14:in `<top (required)
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6.2/lib/active_support/
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6.2/lib/active_support/in load'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6.2/lib/active_support/ependency'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6.2/lib/active_support/
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/engine.rb:655:alizer'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6.2/lib/active_support/ument'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/engine.rb:654:
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/engine.rb:612::Engine>'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/engine.rb:611:
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/engine.rb:611:
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/initializable.
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/initializable.
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/initializable.izers'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:228:in `block in tsort_each'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:350:in `block (2 levels) in ent'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:422:in `block (2 levels) in ent_from'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:431:in `each_strongly_connect
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:421:in `block in each_strongl
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/initializable.
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/initializable.
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:415:in `call'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:415:in `each_strongly_connect
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:349:in `block in each_strongl
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:347:in `each'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:347:in `call'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:347:in `each_strongly_connect
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:226:in `tsort_each'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/ruby-2.4.4/lib/ruby/2.4.0/tsort.rb:205:in `tsort_each'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/initializable.
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/application.rb
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/config/environment.rb:5:in `<top (required)>'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6.2/lib/active_support/e'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6.2/lib/active_support/in require'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6.2/lib/active_support/ependency'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.6.2/lib/active_support/e'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   config.ru:3:in `block in <main>'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.6/lib/rack/builder.rb:55:in `ins
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.6/lib/rack/builder.rb:55:in `ini
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   config.ru:in `new'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   config.ru:in `<main>'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.6/lib/rack/builder.rb:49:in `eva
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.6/lib/rack/builder.rb:49:in `new
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.6/lib/rack/builder.rb:40:in `par
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.6/lib/rack/server.rb:319:in `bui'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.6/lib/rack/server.rb:219:in `app
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/commands/serve'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.6/lib/rack/server.rb:354:in `wra
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/rack-2.0.6/lib/rack/server.rb:283:in `sta
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/commands/servert'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/commands/serveock in perform'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/commands/servep'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/commands/serverform'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor/command.rb:27:in `ru
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor/invocation.rb:126:in
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/thor-0.20.3/lib/thor.rb:387:in `dispatch'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/command/base.r
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/command.rb:44:
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   /opt/zammad/vendor/bundle/ruby/2.4.0/gems/railties-5.1.6.2/lib/rails/commands.rb:16
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   script/rails:6:in `require'
Apr 15 13:52:31 srvvzammad001 zammad-web-1.service[1242]:   script/rails:6:in `<top (required)>'
Apr 15 13:52:31 srvvzammad001 systemd[1]: zammad-websocket-1.service: Main process exited, code=exited, status=1/FAILURE
Apr 15 13:52:31 srvvzammad001 systemd[1]: zammad-websocket-1.service: Unit entered failed state.
Apr 15 13:52:31 srvvzammad001 systemd[1]: zammad-websocket-1.service: Failed with result 'exit-code'.
Apr 15 13:52:31 srvvzammad001 systemd[1]: zammad-web-1.service: Main process exited, code=exited, status=1/FAILURE
Apr 15 13:52:31 srvvzammad001 systemd[1]: zammad-web-1.service: Unit entered failed state.
Apr 15 13:52:31 srvvzammad001 systemd[1]: zammad-web-1.service: Failed with result 'exit-code'.
Apr 15 13:52:31 srvvzammad001 systemd[1]: zammad-websocket-1.service: Service hold-off time over, scheduling restart.
Apr 15 13:52:31 srvvzammad001 systemd[1]: zammad-web-1.service: Service hold-off time over, scheduling restart.
Apr 15 13:52:31 srvvzammad001 systemd[1]: Stopped zammad-web-1.service.
Apr 15 13:52:31 srvvzammad001 systemd[1]: Started zammad-web-1.service.
Apr 15 13:52:31 srvvzammad001 systemd[1]: Stopped zammad-websocket-1.service.
Apr 15 13:52:31 srvvzammad001 systemd[1]: Started zammad-websocket-1.service.

Steps to reproduce the behavior:

  • Install Zamamd on a new Debian VM
  • Move the postgresql dircetory to a NFS Share
  • Edit the /etc/postgresql/9.6/main/postgresql.conf to youre new directory
  • Edit the /etc/fstab for atuo mounting the new NFS Share
  • Reboot the VM

#2

I think it’s because the postgresql service loads before loading the mount point. Does anyone have an idea how to start the service after loading the mount point?


#3

Your postgresql-Server is not running. Ensure it’s enabled and running, then try to start Zammad again.


#4

I was able to solve my problem :slight_smile:

The problem is simply that systemd starts all services by default paralell unless there is something else written in the individual services.This is the reason why Postgres loads before the autofs and wants to access a share that does not yet exist. I use autofs to mount the shares instead of fstab.
And in the 2 Postgres services under [Unit] I entered the following:
After=autofs.service (in the file: /lib/systemd/system/postgresql.service)
After=autofs.service (In the file: /lib/systemd/system/postgresql@.service)

I hope I can help someone with this.