I have a docker container running on AWS EC2 Ubuntu machine.
It runs on Ubuntu 20.04 with 4GB memory. But even though the utilization is 34% of memory, I get a warning with below details.
Total virtual memory: 34.31% - 1.24 GB of 3.61 GB, Committed: 114% - 4.13 GB of 3.61 GB virtual memory (warn/crit at 100.00%/150.00% used)**WARN**
Can someone help to address this? I see a couple of posts in this forum, with similar symptoms. But not mentioned about the fix they applied.
Thanks
You can only address this issue by understanding the application that is running on this host.
Committed memory is memory allocated by processes but currently not used. The Linux kernel overprovisions memory in this case. Hence the warning at 100% used RAM. If suddenly all allocated memory was used by the processes you would get an out of memory error after filling swap space.
If this is quite normal behaviour by the application and it checks RAM limits by itself you could increase the thresholds for this service check.
TOP - Status
top - 15:40:16 up 1 day, 10:34, 1 user, load average: 0.45, 0.58, 0.63
Tasks: 192 total, 1 running, 188 sleeping, 0 stopped, 3 zombie
%Cpu(s): 7.4 us, 1.9 sy, 0.0 ni, 90.3 id, 0.4 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 42.2/3691.7 [|||||||||||||||||||||||||||||||||||||||||| ]
MiB Swap: 0.0/0.0 [ ]
ps aux | head -1; ps aux | sort -rnk 4 | head -5
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
ubuntu 2403354 0.0 3.9 180008 148512 ? S 12:34 0:08 /usr/sbin/apache2 -f /omd/sites/controller/etc/apache/apache.conf
ubuntu 2363449 0.1 3.8 178408 146844 ? S 12:04 0:14 /usr/sbin/apache2 -f /omd/sites/controller/etc/apache/apache.conf
ubuntu 2607033 0.2 3.7 171920 140364 ? S 15:16 0:04 /usr/sbin/apache2 -f /omd/sites/controller/etc/apache/apache.conf
root 167 0.0 3.7 277024 142084 ? S<s Nov08 0:15 /lib/systemd/systemd-journald
ubuntu 2607034 0.2 3.3 158172 126460 ? S 15:16 0:04 /usr/sbin/apache2 -f /omd/sites/controller/etc/apache/apache.conf
Looks like the apache in CMK container is causing the utilization. Can someone help to optimize it.
IMHO column 4 is the wrong column. Sort by column 5 again.
This is how it looks like:
ps aux | head -1; ps aux | sort -rnk 5 | head -4
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1046 0.4 1.4 1680420 55152 ? Ssl Nov08 21:12 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
root 588 0.0 0.6 1493044 26376 ? Ssl Nov08 2:25 /usr/bin/containerd
root 969 0.0 0.4 1395272 15892 ? Sl Nov08 0:09 /snap/amazon-ssm-agent/4046/ssm-agent-worker
root 573 0.0 0.2 1382880 10156 ? Ssl Nov08 0:14 /snap/amazon-ssm-agent/4046/amazon-ssm-agent
All these processes allocate much more memory than they actually use.
If this is normal behaviour you may need to adjust the thresholds for committed memory.
Hi,
for me this is totally normal behavior on Linux, especially for servers with that less total memory and running some containers. There are quite a lot and good web pages around which explain exactly overcommiting means and why this happens.
there is a really good blog entry describing exactly whats happening but unfortunately it’s only in German (Linux und overcommit_memory (lug-erding.de)). I haven’t found a good practical blog after a quick search, but linux overcommit_memory would do the job.
Long story short, as Robert mentioned, higher the thresholds for this host (i would disable it completely in this case) or spend a bit more memory on this host. Additionally you can tune your system a bit to not overcommit memory but this could lead into problems with your container not starting anymore (really limited memory on this host).
I’m working to fine-tune the server. I’ll let you know the final outcome.
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed. Contact an admin if you think this should be re-opened.