Memory Leak with ZenArmor with OPNsense
# Zenarmor Memory Leak Report
**Subject:** Zenarmor memory leak --- mbuf jumbo page allocations
consuming all RAM + swap on OPNsense
------------------------------------------------------------------------
## Environment
- **Platform:** OPNsense (VM with 16 GB RAM, 8 GB swap)
- **Zenarmor mode:** \[please fill in: Protect or Passive Inspect\]
- **Interfaces inspected:** \[please fill in, e.g., WAN/LAN\]
## Problem Description
- Upon boot, memory usage is reasonable (\~40--45% of 16 GB).\
- After enabling Zenarmor, memory usage steadily climbs over time (45%
→ 57% → 68% → 75% → 97%).\
- No userland processes (eastpect, ipdrstreamer) show increasing RSS
usage --- their memory footprint remains stable (eastpect \~297M,
ipdrstreamer \~59M).\
- Instead, **kernel memory consumption rises due to mbuf jumbo
allocations**, visible in `netstat -m` and `vmstat -z`.\
- Eventually, the system consumes nearly all 16 GB RAM and pushes \>4
GB into swap. At that point, performance degrades severely and the
firewall becomes unstable.\
- Stopping Zenarmor (`service eastpect stop`) does not free the kernel
memory --- mbuf jumbo allocations remain pinned until reboot.
## Evidence Collected
### `top` (before leak progression)
- 45% RAM in use, swap = 0\
- eastpect: VSZ 10G, RES \~297M (stable)\
- ipdrstreamer: VSZ 1300M, RES \~59M each (stable)
### `top` (at 97% RAM, 4.3 GB swap in use)
Mem: 8131M Active, 3745M Laundry, 3278M Wired, 371M Free
Swap: 8192M total, 4277M used
- eastpect RES dropped to \~156M (likely swapped), but kernel memory
was exhausted
### `netstat -m`
Early:
5101/6329/11430 mbufs in use
4368/4776/9144/501877 4k jumbo clusters in use
Later:
8472/9416/501877 4k jumbo clusters in use
960k total jumbo page allocations
### `vmstat -z | egrep "mbuf|packet"` (growth over time)
- mbuf_jumbo_page allocations climbing steadily:
- 847,417 → 858,250 → 960,671
- Current in use steadily rising with no release
## Conclusion
This appears to be a **kernel memory leak in the mbuf jumbo page pool
triggered by Zenarmor's netmap packet engine (`eastpect`)**.\
Userland process memory remains stable, but kernel allocations grow
until the system runs out of memory and starts swapping.
## Temporary Mitigation
- Disabling Zenarmor (`service eastpect stop`) halts new allocations,
but does not release memory already consumed.\
- Full memory recovery requires a reboot.\
- To avoid full crashes, I am considering limiting mbuf pools via
`sysctl`, but this would lead to packet drops.
## Request
Can you please confirm if this is a known issue with the current
Zenarmor release?\
- Is there a patch or a tuning recommendation to prevent unbounded jumbo
mbuf allocations?\
- If not, I am happy to provide full logs (`top`, `netstat -m`,
`vmstat -z`) over time to help diagnose further.
------------------------------------------------------------------------
**Thank you for your assistance. I can reproduce this consistently and
will support with more debugging info if needed.**
-
Official comment
Hi Graham,
There is no known issue. It could be a netmap issue as well. We need to investigate your logs are configuration. Please share them via "Have Feedback" option in the bottom left corner of UI.
Comment actions
Please sign in to leave a comment.
Comments
1 comment