Apache Error AH01114 HTTP: failed to make connection to backend

Hi,
right after the weekend, my Zammad-server serves me an error on monday morning.
On the website, I got the error “503 Service Unavailable”. In the apache2 log I see this error:

[Mon Oct 23 09:31:00.409985 2023] [proxy:error] [pid 3892:tid 140117930927872] (111)Connection refused: AH00957: http: attempt to connect to 127.0.0.1:3000 (127.0.0.1:3000) failed
[Mon Oct 23 09:31:00.410019 2023] [proxy_http:error] [pid 3892:tid 140117930927872] [client 10.215.224.52:63624] AH01114: HTTP: failed to make connection to backend: 127.0.0.1

But first my technical situation:
-Zammad Installation from source
-Version? Can’t find a version-number on cli. Can’t access GUI to look for it. Last update was in beginning of 2023.
-Server: Debian GNU/Linux 11 (bullseye)
-Webserver: Apache2
-Database: PostgreSQL
-Browser: MS-Edge Version 118.0.2088.57 and Firefox 118.0.1
-I don’t have a Firewall running on this machine
-Server is on-premise

I don ‘t know what happened on week and, but after it, Zammad doesn’t work. So, I have to troubleshot this situation. I not sure how.

Up to Now I know:

-Apache error.log shows the message above.
-Zammad.conf looks like this:

ServerTokens Prod

<VirtualHost *:80>
    ServerName karlx021.intern.ias-gruppe.de
    Redirect permanent / https://fqdn-of-server
</VirtualHost>

<VirtualHost *:443>
    SSLEngine on
    SSLProtocol all -SSLv2 -SSLv3
    SSLCipherSuite EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH

    SSLCertificateFile /etc/certificate-ias/fqdn-of-server.de.crt
    SSLCertificateKeyFile /etc/certificate-ias/ fqdn-of-server.de.key.pem


    #SSLCertificateFile /etc/self-signed-certivicate/certificate.pem
    #SSLCertificateKeyFile /etc/self-signed-certivicate/key.pem
    #SSLCertificateChainFile /etc/ssl/lets-encrypt-x3-cross-signed.pem
    SSLOpenSSLConfCmd DHParameters /etc/self-signed-certivicate/dhparam.pem

    # replace 'localhost' with your fqdn if you want to use zammad from remote
    ServerName localhost

    ## don't loose time with IP address lookups
    HostnameLookups Off

    ## needed for named virtual hosts
    UseCanonicalName Off

    ## configures the footer on server-generated documents
    ServerSignature Off

    ProxyRequests Off
    ProxyPreserveHost On

    <Proxy 127.0.0.1:3000>
      Require local
    </Proxy>

    ProxyPass /assets !
    ProxyPass /favicon.ico !
    ProxyPass /apple-touch-icon.png !
    ProxyPass /robots.txt !
    ProxyPass /ws ws://127.0.0.1:6042/
    ProxyPass / http://127.0.0.1:3000/

    # change this line in an SSO setup
    RequestHeader unset X-Forwarded-User
    RequestHeader set X_FORWARDED_PROTO 'https'
    # Use settings below if proxying does not work and you receive HTTP-Errror 404
    # if you use the settings below, make sure to comment out the above two options
    # This may not apply to all systems, applies to openSuse
    #ProxyPass /ws ws://127.0.0.1:6042/ "retry=1 acque=3000 timeout=600 keepalive=On"
    #ProxyPass / http://127.0.0.1:3000/ "retry=1 acque=3000 timeout=600 keepalive=On"

    DocumentRoot "/opt/zammad/public"

    <Directory />
        Options FollowSymLinks
        AllowOverride None
    </Directory>

    <Directory "/opt/zammad/public">
        Options FollowSymLinks
              Require all granted
    </Directory>
</VirtualHost>

Netstat will show me:

# netstat -tlpen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       User       Inode      PID/Program name
tcp        0      0 0.0.0.0:5666            0.0.0.0:*               LISTEN      120        1799       639/nrpe
tcp        0      0 127.0.0.1:6379          0.0.0.0:*               LISTEN      121        815        642/redis-server 12
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      0          743        666/sshd: /usr/sbin
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN      0          12073      637/cupsd
tcp        0      0 127.0.0.1:5432          0.0.0.0:*               LISTEN      119        14522      793/postgres
tcp        0      0 127.0.0.1:6042          0.0.0.0:*               LISTEN      118        198694     3372/script/websock
tcp6       0      0 :::443                  :::*                    LISTEN      0          199884     3891/apache2
tcp6       0      0 :::5666                 :::*                    LISTEN      120        1800       639/nrpe
tcp6       0      0 ::1:6379                :::*                    LISTEN      121        816        642/redis-server 12
tcp6       0      0 :::80                   :::*                    LISTEN      0          199880     3891/apache2
tcp6       0      0 127.0.0.1:9200          :::*                    LISTEN      117        16104      765/java
tcp6       0      0 ::1:9200                :::*                    LISTEN      117        17548      765/java
tcp6       0      0 127.0.0.1:9300          :::*                    LISTEN      117        17536      765/java
tcp6       0      0 ::1:9300                :::*                    LISTEN      117        17529      765/java
tcp6       0      0 :::22                   :::*                    LISTEN      0          754        666/sshd: /usr/sbin
tcp6       0      0 ::1:631                 :::*                    LISTEN      0          12072      637/cupsd
tcp6       0      0 ::1:5432                :::*                    LISTEN      119        14521      793/postgres

Postgres log shows me:


2023-10-23 09:32:20.692 CEST [4022] zammad@zammad ERROR:  could not serialize access due to read/write dependencies among transactions
2023-10-23 09:32:20.692 CEST [4022] zammad@zammad DETAIL:  Reason code: Canceled on identification as a pivot, during commit attempt.
2023-10-23 09:32:20.692 CEST [4022] zammad@zammad HINT:  The transaction might succeed if retried.
2023-10-23 09:32:20.692 CEST [4022] zammad@zammad STATEMENT:  COMMIT
2023-10-23 09:32:20.692 CEST [4022] zammad@zammad WARNING:  there is no transaction in progress
2023-10-23 09:32:20.745 CEST [4023] zammad@zammad ERROR:  could not serialize access due to read/write dependencies among transactions
2023-10-23 09:32:20.745 CEST [4023] zammad@zammad DETAIL:  Reason code: Canceled on identification as a pivot, during write.
2023-10-23 09:32:20.745 CEST [4023] zammad@zammad HINT:  The transaction might succeed if retried.
2023-10-23 09:32:20.745 CEST [4023] zammad@zammad STATEMENT:  INSERT INTO "active_job_locks" ("lock_key", "active_job_id", "created_at", "updated_at") VALUES ($1, $2, $3, $4) RETURNING "id"
2023-10-23 09:32:20.746 CEST [4023] zammad@zammad ERROR:  current transaction is aborted, commands ignored until end of transaction block
2023-10-23 09:32:20.746 CEST [4023] zammad@zammad STATEMENT:  DEALLOCATE a1
2023-10-23 09:32:20.747 CEST [4023] zammad@zammad ERROR:  current transaction is aborted, commands ignored until end of transaction block
2023-10-23 09:32:20.747 CEST [4023] zammad@zammad STATEMENT:  DEALLOCATE a2
2023-10-23 09:32:20.798 CEST [4022] zammad@zammad ERROR:  could not serialize access due to read/write dependencies among transactions
2023-10-23 09:32:20.798 CEST [4022] zammad@zammad DETAIL:  Reason code: Canceled on identification as a pivot, during write.
2023-10-23 09:32:20.798 CEST [4022] zammad@zammad HINT:  The transaction might succeed if retried.
2023-10-23 09:32:20.798 CEST [4022] zammad@zammad STATEMENT:  UPDATE "active_job_locks" SET "updated_at" = $1 WHERE "active_job_locks"."id" = $2
2023-10-23 09:32:20.798 CEST [4022] zammad@zammad ERROR:  current transaction is aborted, commands ignored until end of transaction block
2023-10-23 09:32:20.798 CEST [4022] zammad@zammad STATEMENT:  DEALLOCATE a1
2023-10-23 09:32:20.799 CEST [4022] zammad@zammad ERROR:  current transaction is aborted, commands ignored until end of transaction block
2023-10-23 09:32:20.799 CEST [4022] zammad@zammad STATEMENT:  DEALLOCATE a2
2023-10-23 09:32:20.799 CEST [4022] zammad@zammad ERROR:  current transaction is aborted, commands ignored until end of transaction block
2023-10-23 09:32:20.799 CEST [4022] zammad@zammad STATEMENT:  DEALLOCATE a3
2023-10-23 09:32:20.800 CEST [4022] zammad@zammad ERROR:  current transaction is aborted, commands ignored until end of transaction block
2023-10-23 09:32:20.800 CEST [4022] zammad@zammad STATEMENT:  DEALLOCATE a4

Question:
in Zammad.conf I have this entry:

ProxyPass / http://127.0.0.1:3000/

But in output of “netstat -tlpen” there is no open Port 3000. Which service should run on port 3000?

Zammad spawns several processes depending on your performance tuning. For communication with users via Web, it requires the railsserver and websocket services.
They’re running on Port 3000 and 6042 by default.

Your processes either died for whatever reason or where stopped (for whatever reason). Start them.

Thanks for your fast answer.
whats the name of this two daemens?
They should have been startet, when i did the last reboot this morning.

https://docs.zammad.org/en/latest/install/source.html#manage-services-of-zammad

thanks for this link.

this services are running:

root@zammad:~# service zammad-web status
● zammad-web.service
     Loaded: loaded (/etc/systemd/system/zammad-web.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2023-10-23 12:56:50 CEST; 20min ago
   Main PID: 531 (sleep)
      Tasks: 1 (limit: 19138)
     Memory: 148.0K
        CPU: 821us
     CGroup: /system.slice/zammad-web.service
             └─531 /bin/sleep infinity

Oct 23 12:56:50 zammad systemd[1]: Started zammad-web.service.
root@zammad:~# service zammad-worker status
● zammad-worker.service
     Loaded: loaded (/etc/systemd/system/zammad-worker.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2023-10-23 12:56:50 CEST; 21min ago
   Main PID: 535 (sleep)
      Tasks: 1 (limit: 19138)
     Memory: 144.0K
        CPU: 914us
     CGroup: /system.slice/zammad-worker.service
             └─535 /bin/sleep infinity

Oct 23 12:56:50 zammad systemd[1]: Started zammad-worker.service.
root@zammad:~# service zammad-websocket status
● zammad-websocket.service
     Loaded: loaded (/etc/systemd/system/zammad-websocket.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2023-10-23 12:56:50 CEST; 21min ago
   Main PID: 533 (sleep)
      Tasks: 1 (limit: 19138)
     Memory: 144.0K
        CPU: 1ms
     CGroup: /system.slice/zammad-websocket.service
             └─533 /bin/sleep infinity

Oct 23 12:56:50 zammad systemd[1]: Started zammad-websocket.service.
root@zammad:~# service zammad status
● zammad.service
     Loaded: loaded (/etc/systemd/system/zammad.service; enabled; vendor preset: enabled)
     Active: active (running) since Mon 2023-10-23 12:56:50 CEST; 21min ago
   Main PID: 530 (sleep)
      Tasks: 1 (limit: 19138)
     Memory: 188.0K
        CPU: 630us
     CGroup: /system.slice/zammad.service
             └─530 /bin/sleep infinity

Oct 23 12:56:50 zammad systemd[1]: Started zammad.service.
root@zammad:~#

This topic was automatically closed 360 days after the last reply. New replies are no longer allowed.