Tables `http_logs` and friends are eating all my storage space!

Feature requests
1

Using zammad 6.3.1, on a test environment, so with a very low traffic. After one week of no usage, I found my local disk full.
I did some inspection, and I found that the http_logs and friends tables are eating all of my space.

zammad=# select pg_size_pretty( pg_relation_size( oid ) ), relname
from pg_class where relname not like 'pg%'
order by pg_relation_size( oid ) desc;


pg_size_pretty |                             relname                             
----------------+-----------------------------------------------------------------
 9643 MB        | http_logs
 403 MB         | http_logs_pkey
 403 MB         | index_http_logs_on_created_at
 136 MB         | index_http_logs_on_created_by_id
 136 MB         | index_http_logs_on_facility
 26 MB          | translations
 3704 kB        | translations_pkey
 3000 kB        | index_sessions_on_session_id
 1800 kB        | index_translations_on_source
 1168 kB        | index_translations_on_locale
 1032 kB        | index_sessions_on_updated_at
 728 kB         | sessions_pkey
 328 kB         | index_sessions_on_persistent
 160 kB         | import_jobs
 112 kB         | settings
 64 kB          | sql_features
 56 kB          | delayed_jobs
...

In the production.rb file I’ve config.log_level = :info.
After digging into some record from http_logs, I found that is related to LDAP:

zammad=# select direction, facility, method from http_logs order by created_at desc limit 10;
 direction | facility | method 
-----------+----------+--------
 out       | ldap     | tcp
 out       | ldap     | tcp
 out       | ldap     | tcp
 out       | ldap     | tcp
 out       | ldap     | tcp
 out       | ldap     | tcp
 out       | ldap     | tcp
 out       | ldap     | tcp
 out       | ldap     | tcp
 out       | ldap     | tcp

and they contain messages like:

irection | facility | method |                         request                         
-----------+----------+--------+---------------------------------------------------------
 out       | ldap     | tcp    | ---                                                    +
           |          |        | content:                                               +
           |          |        |   firstname: !ruby/string:Net::BER::BerIdentifiedString+
           |          |        |     str: XXXXXXX                                       +
           |          |        |     ber_identifier: 4                                  +
           |          |        |   lastname: !ruby/string:Net::BER::BerIdentifiedString +
           |          |        |     str: XXXXX                                       +
           |          |        |     ber_identifier: 4                                  +
           |          |        |   email: XXXX@yyyy.it                    +
           |          |        |   login: !ruby/string:Net::BER::BerIdentifiedString    +
           |          |        |     str: XXXX@yyyy.it                    +
           |          |        |     ber_identifier: 4                                  +
           |          |        |   phone:                                               +
           |          |        |

Why is that happening?
May I truncate safely the table, because so far the system is not responding at all.

1 Like
2

I’ve tried to truncate the table, and it seems to have produced no disaster. I’m monitoring the situation, and after one hour the table has already reached 60MB in size.

1 Like
3

We have a very large Active Directory. My table http_logs is 16 GB big.
The data is for me trash. I don’t know for what it is and I can’t disable it. I expanded my disk for that last week :joy:

See this thread: Reduce http_logs in database - #3 by ozturkinc

1 Like
4

Thanks for the hint @Dennis1993 , unluckily, the following does not seem to work:

$ sudo -u postgres psql -c "select pg_size_pretty( pg_relation_size( 'http_logs' ) );"  zammad
 pg_size_pretty 
----------------
 86 MB
(1 riga)

$ sudo -u zammad zammad run rails r 'HttpLog.cleanup(1.day)'

$ sudo -u postgres psql -c "select pg_size_pretty( pg_relation_size( 'http_logs' ) );"  zammad
 pg_size_pretty 
----------------
 86 MB

$ sudo -u postgres psql -c "vacuum full http_logs;"  zammad

$ sudo -u postgres psql -c "select pg_size_pretty( pg_relation_size( 'http_logs' ) );"  zammad
 pg_size_pretty 
----------------
 29 MB

and the table is not shrinking as I would expect, since all the logs are of today. However, this is at least something I can schedule to ensure the table will stay under control.