Excessive disk usage on pfSense
Was looking at my configuration the other day and noticed that was running out of space on pfSense. These were the files that were the culprit:
For some unknown reason, I am running out of space. I was able to determine where it is, but not how to control it:
[2.6.0-RELEASE][root@pfSense-backup3.home.arpa]/usr/local/datastore/sqlite: ls -l
total 3134179
-rw-r--r-- 1 root wheel 45056 May 4 12:50 alert_all.sqlite
-rw-r--r-- 1 root wheel 4803751936 May 22 08:41 conn_all.sqlite
-rw-r--r-- 1 root wheel 1972715520 May 22 08:41 dns_all.sqlite
-rw-r--r-- 1 root wheel 214843392 May 22 08:40 http_all.sqlite
-rw-r--r-- 1 root wheel 8192 May 4 12:50 sip_all.sqlite
-rw-r--r-- 1 root wheel 907161600 May 22 08:40 tls_all.sqlite
Was able to find out that these files belong to ZenArmor. Went into my configuration and was only saving 2 days of history. From my perspective 2 days of logs should not over 4.5GB of space.
-
Official comment
Hi,
It could be an issue with indey files. Please try reset reporting (Settings - Data Management) and check again. If it doesn't solve the problem, can you open a ticket via Send Feedback on Zenconsole?
Comment actions -
I reset the files, which did erase them, and they recreated. Planning on rebuilding this system today because I generally would like more files.
System is configured with ZFS with LZ4 compression, and was expecting a higher level of compression.than the now current 1.50x.
Not sure what compression was when the logs were 4.6GB, but looks better now, and I know the ZFS is compressing them, and with this script, can look at them in the future if they get out of hand.:
[2.6.0-RELEASE][root@pfSense-backup3.home.arpa]/: find . -type f -iname "*all.sqlite" -exec sh -c "echo -n ' de-compressed: '; du -Ah '{}'; echo -n 'compressed (used): '; du -h '{}'; echo ' '" \;
de-compressed: 44K ./usr/local/datastore/sqlite/alert_all.sqlite
compressed (used): 4.5K ./usr/local/datastore/sqlite/alert_all.sqlite
de-compressed: 8.0K ./usr/local/datastore/sqlite/sip_all.sqlite
compressed (used): 4.5K ./usr/local/datastore/sqlite/sip_all.sqlite
de-compressed: 73M ./usr/local/datastore/sqlite/tls_all.sqlite
compressed (used): 30M ./usr/local/datastore/sqlite/tls_all.sqlite
de-compressed: 443M ./usr/local/datastore/sqlite/conn_all.sqlite
compressed (used): 187M ./usr/local/datastore/sqlite/conn_all.sqlite
de-compressed: 184M ./usr/local/datastore/sqlite/dns_all.sqlite
compressed (used): 78M ./usr/local/datastore/sqlite/dns_all.sqlite
de-compressed: 58M ./usr/local/datastore/sqlite/http_all.sqlite
compressed (used): 12M ./usr/local/datastore/sqlite/http_all.sqlite -
Looks like it is under control, While the files are large, they are manageable:
[2.6.0-RELEASE][root@pfSense-backup3.home.arpa]/usr/local/datastore/sqlite: ls -l
total 2629139
-rw-r--r-- 1 root wheel 45056 May 22 11:28 alert_all.sqlite
-rw-r--r-- 1 root wheel 4064116736 Jun 1 10:04 conn_all.sqlite
-rw-r--r-- 1 root wheel 1679486976 Jun 1 10:03 dns_all.sqlite
-rw-r--r-- 1 root wheel 189747200 Jun 1 10:02 http_all.sqlite
-rw-r--r-- 1 root wheel 8192 May 22 11:28 sip_all.sqlite
-rw-r--r-- 1 root wheel 644341760 Jun 1 10:04 tls_all.sqliteand with ZFS as the filesystem, the compression is helping:
[2.6.0-RELEASE][root@pfSense-backup3.home.arpa]/usr/local/datastore/sqlite: find . -type f -iname "*all.sqlite" -exec sh -c "echo -n ' de-compressed: '; du -Ah '{}'; echo -n 'compressed (used): '; du -h '{}'; echo ' '" \;
de-compressed: 44K ./alert_all.sqlite
compressed (used): 4.5K ./alert_all.sqlite
de-compressed: 8.0K ./sip_all.sqlite
compressed (used): 4.5K ./sip_all.sqlite
de-compressed: 615M ./tls_all.sqlite
compressed (used): 244M ./tls_all.sqlite
de-compressed: 3.8G ./conn_all.sqlite
compressed (used): 1.6G ./conn_all.sqlite
de-compressed: 1.6G ./dns_all.sqlite
compressed (used): 669M ./dns_all.sqlite
de-compressed: 181M ./http_all.sqlite
compressed (used): 45M ./http_all.sqlite
Please sign in to leave a comment.
Comments
4 comments