[Check_mk (english)] tmpfs will not release (no free space)

k11 · December 24, 2014, 9:08am

Hi everyone,

just got a strange problem: tmpfs will not release, then stuck whole OMD (web page monitoring services turns grey, can’t add new host). if i force omd service restart, apache will fail.

only OS reboot can fix it. ( for 1 day), the tmpfs folder will eat up 100% in less than 1 day)

searched seems no one have same problem, is there anything i can do to fix this? or i need more ram?

thanks a lot.

my config:

OMD 1.2 x64

CentOS 6.6 X64, up to date

RAM: 2GB

hq site: 41 hosts (which will ran into problem, it’s tmpfs will keep 100% after OS reboot for less than 1 day), monitoring windows(snmp)/network devices(snmp)/linux(agent)

service site: 61 hosts

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root

                  211G 5.8G 194G 3% /

tmpfs 939M 0 939M 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home

                  9.9G 151M 9.2G 2% /home

tmpfs 939M 243M 696M 26% /opt/omd/sites/hq/tmp

tmpfs 939M 3.0M 936M 1% /opt/omd/sites/service/tmp

Regards,

KT

···

MarsellusWallace · December 24, 2014, 9:25am

Hi KT,

You need more RAM! If you have 2GB and run 2 sites each running a tmpfs with 900MB you have how much memory left? Right… 200MB.

Also running two sites with that env is not the best choice.

Kind regards, Happy Holidays,
Marcel

···

k19840@126.com k19840@126.com schrieb am Mi., 24. Dez. 2014 10:09:

Hi everyone,

just got a strange problem: tmpfs will not release, then stuck whole OMD (web page monitoring services turns grey, can’t add new host). if i force omd service restart, apache will fail.

only OS reboot can fix it. ( for 1 day), the tmpfs folder will eat up 100% in less than 1 day)

searched seems no one have same problem, is there anything i can do to fix this? or i need more ram?

thanks a lot.

my config:

OMD 1.2 x64

CentOS 6.6 X64, up to date

RAM: 2GB

hq site: 41 hosts (which will ran into problem, it’s tmpfs will keep 100% after OS reboot for less than 1 day), monitoring windows(snmp)/network devices(snmp)/linux(agent)

service site: 61 hosts

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root
                  211G 5.8G 194G 3% /
tmpfs 939M 0 939M 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home
tmpfs 939M 243M 696M 26% /opt/omd/sites/hq/tmp

tmpfs 939M 3.0M 936M 1% /opt/omd/sites/service/tmp

Regards,

KT

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

andreas-doehler · December 24, 2014, 10:15am

Hi KT,

beside what Marcel already mentioned, what is inside your tmpfs?

243MByte usage looks very high for such a small site.

Is it possible that you have other processes / scripts who store data there?

Best regards,

Andreas

···

k19840@126.com k19840@126.com schrieb am Wed Dec 24 2014 at 10:09:12:

Hi everyone,

just got a strange problem: tmpfs will not release, then stuck whole OMD (web page monitoring services turns grey, can’t add new host). if i force omd service restart, apache will fail.

only OS reboot can fix it. ( for 1 day), the tmpfs folder will eat up 100% in less than 1 day)

searched seems no one have same problem, is there anything i can do to fix this? or i need more ram?

thanks a lot.

my config:

OMD 1.2 x64

CentOS 6.6 X64, up to date

RAM: 2GB

hq site: 41 hosts (which will ran into problem, it’s tmpfs will keep 100% after OS reboot for less than 1 day), monitoring windows(snmp)/network devices(snmp)/linux(agent)

service site: 61 hosts

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root
                  211G 5.8G 194G 3% /
tmpfs 939M 0 939M 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home
tmpfs 939M 243M 696M 26% /opt/omd/sites/hq/tmp

tmpfs 939M 3.0M 936M 1% /opt/omd/sites/service/tmp

Regards,

KT

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

k11 · December 24, 2014, 11:19am

Hi Andreas,

just standard installation, and the VM(CENT OS 6.6 mini installation) only have OMD installed. no other script/app will touch that folder or store anything else over there.

but really strange is hq site have less host got that problem, service site have more host but have no problem.

* just add more ram. ( to 2.5G). and saw the tmpfs size still counting in every minute.

[root@monitor ~]# top

top - 19:07:31 up 1 min, 1 user, load average: 0.10, 0.06, 0.02

Tasks: 203 total, 1 running, 202 sleeping, 0 stopped, 0 zombie

Cpu(s): 2.3%us, 1.7%sy, 0.0%ni, 95.9%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st

Mem: 2413464k total, 469188k used, 1944276k free, 16040k buffers

Swap: 4063224k total, 0k used, 4063224k free, 185840k cached

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root

                  211G 5.9G 194G 3% /

tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home

                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 7.1M 1.2G 1% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

* 1 minute later

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root

                  211G 5.9G 194G 3% /

tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home

                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 8.7M 1.2G 1% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

[root@monitor ~]#

thanks everyone & regards

have a nice chrismas.

KT

···

From: Andreas Döhler

Date: 2014-12-24 18:15

To: k19840@126.com; checkmk-en

Subject: Re: [Check_mk (english)] tmpfs will not release (no free space)

Hi KT,

beside what Marcel already mentioned, what is inside your tmpfs?

243MByte usage looks very high for such a small site.

Is it possible that you have other processes / scripts who store data there?

Best regards,

Andreas

k19840@126.com k19840@126.com schrieb am Wed Dec 24 2014 at 10:09:12:

Hi everyone,

just got a strange problem: tmpfs will not release, then stuck whole OMD (web page monitoring services turns grey, can’t add new host). if i force omd service restart, apache will fail.

only OS reboot can fix it. ( for 1 day), the tmpfs folder will eat up 100% in less than 1 day)

searched seems no one have same problem, is there anything i can do to fix this? or i need more ram?

thanks a lot.

my config:

OMD 1.2 x64

CentOS 6.6 X64, up to date

RAM: 2GB

hq site: 41 hosts (which will ran into problem, it’s tmpfs will keep 100% after OS reboot for less than 1 day), monitoring windows(snmp)/network devices(snmp)/linux(agent)

service site: 61 hosts

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root
                  211G 5.8G 194G 3% /
tmpfs 939M 0 939M 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home
tmpfs 939M 243M 696M 26% /opt/omd/sites/hq/tmp

tmpfs 939M 3.0M 936M 1% /opt/omd/sites/service/tmp

Regards,

KT

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

andreas-doehler · December 24, 2014, 11:29am

That’s no answer to the question what’s inside. Please take a look inside the tmpfs and locate the files who consume your space. Then it is way easier to give some advice.

br
Andreas

···

k19840@126.com k19840@126.com schrieb am Mi., 24. Dez. 2014 12:19:

Hi Andreas,

just standard installation, and the VM(CENT OS 6.6 mini installation) only have OMD installed. no other script/app will touch that folder or store anything else over there.

but really strange is hq site have less host got that problem, service site have more host but have no problem.

* just add more ram. ( to 2.5G). and saw the tmpfs size still counting in every minute.

[root@monitor ~]# top

top - 19:07:31 up 1 min, 1 user, load average: 0.10, 0.06, 0.02

Tasks: 203 total, 1 running, 202 sleeping, 0 stopped, 0 zombie

Cpu(s): 2.3%us, 1.7%sy, 0.0%ni, 95.9%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st

Mem: 2413464k total, 469188k used, 1944276k free, 16040k buffers

Swap: 4063224k total, 0k used, 4063224k free, 185840k cached

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root

                  211G 5.9G 194G 3% /

tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 7.1M 1.2G 1% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

* 1 minute later

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root

                  211G 5.9G 194G 3% /

tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 8.7M 1.2G 1% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

[root@monitor ~]#

thanks everyone & regards

have a nice chrismas.

KT

From: Andreas Döhler

Date: 2014-12-24 18:15

To: k19840@126.com; checkmk-en

Subject: Re: [Check_mk (english)] tmpfs will not release (no free space)

Hi KT,

beside what Marcel already mentioned, what is inside your tmpfs?

243MByte usage looks very high for such a small site.

Is it possible that you have other processes / scripts who store data there?

Best regards,

Andreas

k19840@126.com k19840@126.com schrieb am Wed Dec 24 2014 at 10:09:12:

Hi everyone,

just got a strange problem: tmpfs will not release, then stuck whole OMD (web page monitoring services turns grey, can’t add new host). if i force omd service restart, apache will fail.

only OS reboot can fix it. ( for 1 day), the tmpfs folder will eat up 100% in less than 1 day)

searched seems no one have same problem, is there anything i can do to fix this? or i need more ram?

thanks a lot.

my config:

OMD 1.2 x64

CentOS 6.6 X64, up to date

RAM: 2GB

hq site: 41 hosts (which will ran into problem, it’s tmpfs will keep 100% after OS reboot for less than 1 day), monitoring windows(snmp)/network devices(snmp)/linux(agent)

service site: 61 hosts

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root
                  211G 5.8G 194G 3% /
tmpfs 939M 0 939M 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home
tmpfs 939M 243M 696M 26% /opt/omd/sites/hq/tmp

tmpfs 939M 3.0M 936M 1% /opt/omd/sites/service/tmp

Regards,

KT

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

k11 · December 24, 2014, 11:43am

hi Andreas,

thanks. list as below

[root@monitor tmp]# ls
apache check_mk dokuwiki icinga lock nagios nagvis php pnp4nagios rrdcached rrdcached.pid run shinken thruk
[root@monitor tmp]# du -h
0 ./nagvis/share
0 ./nagvis/tmpl/cache
0 ./nagvis/tmpl/compile
0 ./nagvis/tmpl
0 ./nagvis
0 ./apache/fcgid_sock
4.0K ./apache/run
8.0K ./apache
0 ./thruk/reports
0 ./thruk
0 ./rrdcached
0 ./php/wsdl-cache
27M ./php/session
0 ./php/upload

27M ./php

24K ./nagios/checkresults

0 ./nagios/tmp

584K ./nagios

0 ./shinken/run

0 ./shinken/tmp

0 ./shinken

768K ./check_mk/cache

152K ./check_mk/counters

920K ./check_mk

4.0K ./pnp4nagios/run

0 ./pnp4nagios/lock

4.0K ./pnp4nagios/stats

8.0K ./pnp4nagios

0 ./run/mysql

0 ./run

4.0K ./lock

0 ./dokuwiki

28M .

[root@monitor session]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root

                  211G 5.9G 194G 3% /

tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home

                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 29M 1.2G 3% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

in /php/session are full of files like these:

sess_58go55cg5u9tev1gnbcret8s81 sess_aima3kkt4iadlsv4af0p385en6 sess_g6hduao808p0kq039k87e8trf3 sess_l8ubmjsecrf4qgtgi54dchgvs5 sess_qg65de17ua09rmmk9ojb9pkv02

sess_58n4m97e5jo2j5thu94fojvnu1 sess_ain65to2b9353o5jd71li1ljf1 sess_g6q8rthsfomm83g34kjtv8p0m5 sess_l8vvgaf2etk2apfcespa3fl0k7 sess_qg7fc4k8i05lr4nq4sevve6ia1

PHP Version

[root@monitor session]# php -v

PHP 5.3.3 (cli) (built: Aug 6 2014 05:54:27)

BR,
KT

···

From: Andreas Döhler

Date: 2014-12-24 19:29

To: k19840@126.com; checkmk-en

Subject: Re: Re: [Check_mk (english)] tmpfs will not release (no free space)

That’s no answer to the question what’s inside. Please take a look inside the tmpfs and locate the files who consume your space. Then it is way easier to give some advice.

br
Andreas

k19840@126.com k19840@126.com schrieb am Mi., 24. Dez. 2014 12:19:

Hi Andreas,

just standard installation, and the VM(CENT OS 6.6 mini installation) only have OMD installed. no other script/app will touch that folder or store anything else over there.

but really strange is hq site have less host got that problem, service site have more host but have no problem.

* just add more ram. ( to 2.5G). and saw the tmpfs size still counting in every minute.

[root@monitor ~]# top

top - 19:07:31 up 1 min, 1 user, load average: 0.10, 0.06, 0.02

Tasks: 203 total, 1 running, 202 sleeping, 0 stopped, 0 zombie

Cpu(s): 2.3%us, 1.7%sy, 0.0%ni, 95.9%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st

Mem: 2413464k total, 469188k used, 1944276k free, 16040k buffers

Swap: 4063224k total, 0k used, 4063224k free, 185840k cached

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root

                  211G 5.9G 194G 3% /

tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 7.1M 1.2G 1% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

* 1 minute later

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root

                  211G 5.9G 194G 3% /

tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 8.7M 1.2G 1% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

[root@monitor ~]#

thanks everyone & regards

have a nice chrismas.

KT

From: Andreas Döhler

Date: 2014-12-24 18:15

To: k19840@126.com; checkmk-en

Subject: Re: [Check_mk (english)] tmpfs will not release (no free space)

Hi KT,

beside what Marcel already mentioned, what is inside your tmpfs?

243MByte usage looks very high for such a small site.

Is it possible that you have other processes / scripts who store data there?

Best regards,

Andreas

k19840@126.com k19840@126.com schrieb am Wed Dec 24 2014 at 10:09:12:

Hi everyone,

just got a strange problem: tmpfs will not release, then stuck whole OMD (web page monitoring services turns grey, can’t add new host). if i force omd service restart, apache will fail.

only OS reboot can fix it. ( for 1 day), the tmpfs folder will eat up 100% in less than 1 day)

searched seems no one have same problem, is there anything i can do to fix this? or i need more ram?

thanks a lot.

my config:

OMD 1.2 x64

CentOS 6.6 X64, up to date

RAM: 2GB

hq site: 41 hosts (which will ran into problem, it’s tmpfs will keep 100% after OS reboot for less than 1 day), monitoring windows(snmp)/network devices(snmp)/linux(agent)

service site: 61 hosts

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root
                  211G 5.8G 194G 3% /
tmpfs 939M 0 939M 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home
tmpfs 939M 243M 696M 26% /opt/omd/sites/hq/tmp

tmpfs 939M 3.0M 936M 1% /opt/omd/sites/service/tmp

Regards,

KT

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

andreas-doehler · December 24, 2014, 12:13pm

That means there is a problem with your Apache.
There are not many things inside OMD who create php session files. I think one is PNP4Nagios.

But it looks more like a problem in your setup. If you look at your Apache log files there should be also some errors.

br

Andreas

···

k19840@126.com k19840@126.com schrieb am Wed Dec 24 2014 at 12:43:55:

hi Andreas,

thanks. list as below

[root@monitor tmp]# ls
apache check_mk dokuwiki icinga lock nagios nagvis php pnp4nagios rrdcached rrdcached.pid run shinken thruk
[root@monitor tmp]# du -h
0 ./nagvis/share
0 ./nagvis/tmpl/cache
0 ./nagvis/tmpl/compile
0 ./nagvis/tmpl
0 ./nagvis
0 ./apache/fcgid_sock
4.0K ./apache/run
8.0K ./apache
0 ./thruk/reports
0 ./thruk
0 ./rrdcached
0 ./php/wsdl-cache
27M ./php/session
0 ./php/upload

27M ./php

24K ./nagios/checkresults

0 ./nagios/tmp

584K ./nagios

0 ./shinken/run

0 ./shinken/tmp

0 ./shinken

768K ./check_mk/cache

152K ./check_mk/counters

920K ./check_mk

4.0K ./pnp4nagios/run

0 ./pnp4nagios/lock

4.0K ./pnp4nagios/stats

8.0K ./pnp4nagios

0 ./run/mysql

0 ./run

4.0K ./lock

0 ./dokuwiki

28M .

[root@monitor session]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root
                  211G 5.9G 194G 3% /
tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 29M 1.2G 3% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

in /php/session are full of files like these:

sess_58go55cg5u9tev1gnbcret8s81 sess_aima3kkt4iadlsv4af0p385en6 sess_g6hduao808p0kq039k87e8trf3 sess_l8ubmjsecrf4qgtgi54dchgvs5 sess_qg65de17ua09rmmk9ojb9pkv02

sess_58n4m97e5jo2j5thu94fojvnu1 sess_ain65to2b9353o5jd71li1ljf1 sess_g6q8rthsfomm83g34kjtv8p0m5 sess_l8vvgaf2etk2apfcespa3fl0k7 sess_qg7fc4k8i05lr4nq4sevve6ia1

PHP Version

[root@monitor session]# php -v

PHP 5.3.3 (cli) (built: Aug 6 2014 05:54:27)

Copyright (c) 1997-2010 The PHP Group

Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies

BR,
KT

From: Andreas Döhler

Date: 2014-12-24 19:29

To: k19840@126.com; checkmk-en

Subject: Re: Re: [Check_mk (english)] tmpfs will not release (no free space)

That’s no answer to the question what’s inside. Please take a look inside the tmpfs and locate the files who consume your space. Then it is way easier to give some advice.

br
Andreas

k19840@126.com k19840@126.com schrieb am Mi., 24. Dez. 2014 12:19:

Hi Andreas,

just standard installation, and the VM(CENT OS 6.6 mini installation) only have OMD installed. no other script/app will touch that folder or store anything else over there.

but really strange is hq site have less host got that problem, service site have more host but have no problem.

* just add more ram. ( to 2.5G). and saw the tmpfs size still counting in every minute.

[root@monitor ~]# top

top - 19:07:31 up 1 min, 1 user, load average: 0.10, 0.06, 0.02

Tasks: 203 total, 1 running, 202 sleeping, 0 stopped, 0 zombie

Cpu(s): 2.3%us, 1.7%sy, 0.0%ni, 95.9%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st

Mem: 2413464k total, 469188k used, 1944276k free, 16040k buffers

Swap: 4063224k total, 0k used, 4063224k free, 185840k cached

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root

                  211G 5.9G 194G 3% /

tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 7.1M 1.2G 1% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

* 1 minute later

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root

                  211G 5.9G 194G 3% /

tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 8.7M 1.2G 1% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

[root@monitor ~]#

thanks everyone & regards

have a nice chrismas.

KT

From: Andreas Döhler

Date: 2014-12-24 18:15

To: k19840@126.com; checkmk-en

Subject: Re: [Check_mk (english)] tmpfs will not release (no free space)

Hi KT,

beside what Marcel already mentioned, what is inside your tmpfs?

243MByte usage looks very high for such a small site.

Is it possible that you have other processes / scripts who store data there?

Best regards,

Andreas

k19840@126.com k19840@126.com schrieb am Wed Dec 24 2014 at 10:09:12:

Hi everyone,

just got a strange problem: tmpfs will not release, then stuck whole OMD (web page monitoring services turns grey, can’t add new host). if i force omd service restart, apache will fail.

only OS reboot can fix it. ( for 1 day), the tmpfs folder will eat up 100% in less than 1 day)

searched seems no one have same problem, is there anything i can do to fix this? or i need more ram?

thanks a lot.

my config:

OMD 1.2 x64

CentOS 6.6 X64, up to date

RAM: 2GB

hq site: 41 hosts (which will ran into problem, it’s tmpfs will keep 100% after OS reboot for less than 1 day), monitoring windows(snmp)/network devices(snmp)/linux(agent)

service site: 61 hosts

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root
                  211G 5.8G 194G 3% /
tmpfs 939M 0 939M 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home
tmpfs 939M 243M 696M 26% /opt/omd/sites/hq/tmp

tmpfs 939M 3.0M 936M 1% /opt/omd/sites/service/tmp

Regards,

KT

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

k11 · December 25, 2014, 6:45am

hi Andreas,

problem fixed, caused by external tools. (it will keep read monitor records via http api, and it was configed to get hunderds non-exist host info)

thanks.

···

BR
KT

From: Andreas Döhler

Date: 2014-12-24 20:13

To: k19840@126.com; checkmk-en

Subject: Re: Re: [Check_mk (english)] tmpfs will not release (no free space)

That means there is a problem with your Apache.
There are not many things inside OMD who create php session files. I think one is PNP4Nagios.
But it looks more like a problem in your setup. If you look at your Apache log files there should be also some errors.

br

Andreas

k19840@126.com k19840@126.com schrieb am Wed Dec 24 2014 at 12:43:55:

hi Andreas,

thanks. list as below

[root@monitor tmp]# ls
apache check_mk dokuwiki icinga lock nagios nagvis php pnp4nagios rrdcached rrdcached.pid run shinken thruk
[root@monitor tmp]# du -h
0 ./nagvis/share
0 ./nagvis/tmpl/cache
0 ./nagvis/tmpl/compile
0 ./nagvis/tmpl
0 ./nagvis
0 ./apache/fcgid_sock
4.0K ./apache/run
8.0K ./apache
0 ./thruk/reports
0 ./thruk
0 ./rrdcached
0 ./php/wsdl-cache
27M ./php/session
0 ./php/upload

27M ./php

24K ./nagios/checkresults

0 ./nagios/tmp

584K ./nagios

0 ./shinken/run

0 ./shinken/tmp

0 ./shinken

768K ./check_mk/cache

152K ./check_mk/counters

920K ./check_mk

4.0K ./pnp4nagios/run

0 ./pnp4nagios/lock

4.0K ./pnp4nagios/stats

8.0K ./pnp4nagios

0 ./run/mysql

0 ./run

4.0K ./lock

0 ./dokuwiki

28M .

[root@monitor session]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root
                  211G 5.9G 194G 3% /
tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 29M 1.2G 3% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

in /php/session are full of files like these:

sess_58go55cg5u9tev1gnbcret8s81 sess_aima3kkt4iadlsv4af0p385en6 sess_g6hduao808p0kq039k87e8trf3 sess_l8ubmjsecrf4qgtgi54dchgvs5 sess_qg65de17ua09rmmk9ojb9pkv02

sess_58n4m97e5jo2j5thu94fojvnu1 sess_ain65to2b9353o5jd71li1ljf1 sess_g6q8rthsfomm83g34kjtv8p0m5 sess_l8vvgaf2etk2apfcespa3fl0k7 sess_qg7fc4k8i05lr4nq4sevve6ia1

PHP Version

[root@monitor session]# php -v

PHP 5.3.3 (cli) (built: Aug 6 2014 05:54:27)

Copyright (c) 1997-2010 The PHP Group

Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies

BR,
KT

From: Andreas Döhler

Date: 2014-12-24 19:29

To: k19840@126.com; checkmk-en

Subject: Re: Re: [Check_mk (english)] tmpfs will not release (no free space)

That’s no answer to the question what’s inside. Please take a look inside the tmpfs and locate the files who consume your space. Then it is way easier to give some advice.

br
Andreas

k19840@126.com k19840@126.com schrieb am Mi., 24. Dez. 2014 12:19:

Hi Andreas,

just standard installation, and the VM(CENT OS 6.6 mini installation) only have OMD installed. no other script/app will touch that folder or store anything else over there.

but really strange is hq site have less host got that problem, service site have more host but have no problem.

* just add more ram. ( to 2.5G). and saw the tmpfs size still counting in every minute.

[root@monitor ~]# top

top - 19:07:31 up 1 min, 1 user, load average: 0.10, 0.06, 0.02

Tasks: 203 total, 1 running, 202 sleeping, 0 stopped, 0 zombie

Cpu(s): 2.3%us, 1.7%sy, 0.0%ni, 95.9%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st

Mem: 2413464k total, 469188k used, 1944276k free, 16040k buffers

Swap: 4063224k total, 0k used, 4063224k free, 185840k cached

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root

                  211G 5.9G 194G 3% /

tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 7.1M 1.2G 1% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

* 1 minute later

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root

                  211G 5.9G 194G 3% /

tmpfs 1.2G 0 1.2G 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home

tmpfs 1.2G 8.7M 1.2G 1% /opt/omd/sites/hq/tmp

tmpfs 1.2G 1.1M 1.2G 1% /opt/omd/sites/service/tmp

[root@monitor ~]#

thanks everyone & regards

have a nice chrismas.

KT

From: Andreas Döhler

Date: 2014-12-24 18:15

To: k19840@126.com; checkmk-en

Subject: Re: [Check_mk (english)] tmpfs will not release (no free space)

Hi KT,

beside what Marcel already mentioned, what is inside your tmpfs?

243MByte usage looks very high for such a small site.

Is it possible that you have other processes / scripts who store data there?

Best regards,

Andreas

k19840@126.com k19840@126.com schrieb am Wed Dec 24 2014 at 10:09:12:

Hi everyone,

just got a strange problem: tmpfs will not release, then stuck whole OMD (web page monitoring services turns grey, can’t add new host). if i force omd service restart, apache will fail.

only OS reboot can fix it. ( for 1 day), the tmpfs folder will eat up 100% in less than 1 day)

searched seems no one have same problem, is there anything i can do to fix this? or i need more ram?

thanks a lot.

my config:

OMD 1.2 x64

CentOS 6.6 X64, up to date

RAM: 2GB

hq site: 41 hosts (which will ran into problem, it’s tmpfs will keep 100% after OS reboot for less than 1 day), monitoring windows(snmp)/network devices(snmp)/linux(agent)

service site: 61 hosts

[root@monitor ~]# df -h

Filesystem Size Used Avail Use% Mounted on

/dev/mapper/vg_monitor-lv_root
                  211G 5.8G 194G 3% /
tmpfs 939M 0 939M 0% /dev/shm

/dev/sda1 485M 55M 405M 12% /boot

/dev/mapper/vg_monitor-lv_home
                  9.9G 151M 9.2G 2% /home
tmpfs 939M 243M 696M 26% /opt/omd/sites/hq/tmp

tmpfs 939M 3.0M 936M 1% /opt/omd/sites/service/tmp

Regards,

KT

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en