[Check_mk (english)] Perfdata cleanup

Archive checkmk-en
1

I’m interested.

···

-------- Original message --------
From: Marcel Schulte schulte.marcel@gmail.com
Date: 08/14/2015 16:38 (GMT-05:00)
To: paulmonitoring paulmonitoring@gmail.com, Andreas Döhler andreas.doehler@gmail.com, Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de, omd-users@lists.mathias-kettner.de
Subject: Re: [Check_mk (english)] Perfdata cleanup

Hi,

I’m currently working on a weekly cronjob doing the cleanup for me.

What’s already working:

  • Find data of deleted hosts and remove the whole host folder from perfect.
  • Find all remaining files with modification time older than a configurable amount of days (defaults to 90) and remove them.

The latter will be rewritten to only remove the file if it’s known not to be needed anymore. I think this will be usable mid of next week, there’s more important work to do first :wink:

Whole thing is written in python and works inside OMD sites without modifications. For non-OMD installations a single variable must be changed.

…I could post the mentioned working part at Monday if anyone is interested.

Best regards,
Marcel

paulmonitoring paulmonitoring@gmail.com schrieb am Fr., 14. Aug. 2015 22:24:

I echo what Andreas says. Be cautious. In the past I may or may not have deleted a lot of data by accident from the perfdata dir. :wink:

Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone

-------- Original message --------
From: Andreas Döhler andreas.doehler@gmail.com
Date: 08/14/2015 15:32 (GMT-05:00)
To: Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de, omd-users@lists.mathias-kettner.de
Subject: Re: [Check_mk (english)] Perfdata cleanup

Yes and no.
Perfdata directory will only get bigger with new hosts in the moment you create one host the total space is allocated. What I do on my systems is from time to time a find with older then 30-60 days on this directory. With this I remove obsolete perfdata from deleted machines on non existent checks.

But pay attention what you delete there :slight_smile:

Best regards
Andreas

Gerardo Ferreyra raptorg83@gmail.com schrieb am Fr., 14. Aug. 2015 21:19:

Hi guys.
A quick one. I have hundreds of servers being monitored by check_mk.

I’m seeing that perfdata directory is getting very big.

/opt/omd/sites/prod/var/pnp4nagios/perfdata

I need performance data, of course.

My question.

I have no problem (for now) to increase the disk, but… is there a internal functionality on check_mk (wato) I need to activate, that rotates this data? Set to… retain X months of data? Something like that?

I see “Diskspace cleanup” on global settings. Is that?

Thank you very much!

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.

October 18th-20th, 2015

http://mathias-kettner.com/conference

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.

October 18th-20th, 2015

http://mathias-kettner.com/conference

2

What I have used and modify is this.

https://github.com/lingej/pnp4nagios/blob/master/scripts/check_pnp_rrds.pl.in

Might be worth something to some.

Not only can you add this as a check (classic Nagios/icinga), but there is a command line options –deletexml –deleterrd that can be used to help in cleanup. The script itself, when using those you have to hit Y to delete for each, but easily modified if you want to add it to a cron job to delete.

Just another suggestion for aiding in cleanup.

···

From: checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de] On Behalf Of paulmonitoring
Sent: Friday, August 14, 2015 5:45 PM
To: Marcel Schulte schulte.marcel@gmail.com; Andreas Döhler andreas.doehler@gmail.com; Gerardo Ferreyra raptorg83@gmail.com; checkmk-en@lists.mathias-kettner.de checkmk-en@lists.mathias-kettner.de; omd-users@lists.mathias-kettner.de
Subject: Re: [Check_mk (english)] Perfdata cleanup

I’m interested.

-------- Original message --------
From: Marcel Schulte schulte.marcel@gmail.com
Date: 08/14/2015 16:38 (GMT-05:00)
To: paulmonitoring paulmonitoring@gmail.com, Andreas Döhler andreas.doehler@gmail.com, Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de, omd-users@lists.mathias-kettner.de
Subject: Re: [Check_mk (english)] Perfdata cleanup

Hi,

I’m currently working on a weekly cronjob doing the cleanup for me.

What’s already working:

  • Find data of deleted hosts and remove the whole host folder from perfect.
  • Find all remaining files with modification time older than a configurable amount of days (defaults to 90) and remove them.

The latter will be rewritten to only remove the file if it’s known not to be needed anymore. I think this will be usable mid of next week, there’s more important work to do first :wink:

Whole thing is written in python and works inside OMD sites without modifications. For non-OMD installations a single variable must be changed.

…I could post the mentioned working part at Monday if anyone is interested.

Best regards,
Marcel

paulmonitoring paulmonitoring@gmail.com schrieb am Fr., 14. Aug. 2015 22:24:

I echo what Andreas says. Be cautious. In the past I may or may not have deleted a lot of data by accident from the perfdata dir. :wink:

Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone

-------- Original message --------
From: Andreas Döhler andreas.doehler@gmail.com
Date: 08/14/2015 15:32 (GMT-05:00)
To: Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de, omd-users@lists.mathias-kettner.de
Subject: Re: [Check_mk (english)] Perfdata cleanup

Yes and no.
Perfdata directory will only get bigger with new hosts in the moment you create one host the total space is allocated. What I do on my systems is from time to time a find with older then 30-60 days on this directory. With this I remove obsolete perfdata from deleted machines on non existent checks.

But pay attention what you delete there :slight_smile:

Best regards
Andreas

Gerardo Ferreyra raptorg83@gmail.com schrieb am Fr., 14. Aug. 2015 21:19:

Hi guys.

A quick one. I have hundreds of servers being monitored by check_mk.

I’m seeing that perfdata directory is getting very big.

/opt/omd/sites/prod/var/pnp4nagios/perfdata

I need performance data, of course.

My question.

I have no problem (for now) to increase the disk, but… is there a internal functionality on check_mk (wato) I need to activate, that rotates this data? Set to… retain X months of data? Something like that?

I see “Diskspace cleanup” on global settings. Is that?

Thank you very much!

checkmk-en mailing list
checkmk-en@lists.mathias-kettner.de
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.
October 18th-20th, 2015
http://mathias-kettner.com/conference

checkmk-en mailing list
checkmk-en@lists.mathias-kettner.de
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!
Book your place now and be part of it.
October 18th-20th, 2015
http://mathias-kettner.com/conference

3

Hi guys (and girls, if any),

I finished my cronjob script to cleanup perfdata files.

The script is intended to be run as OMD siteuser - but will complain about which variables to tweak if it’s not :wink:

If you’re not using CMC (“Check_MK Micro Core”) you must set line 48 to False

Here’s the usage:

···

$cleanup_perfdata.py -h

usage: cleanup_perfdata.py [-h] [-x XDAYS] [-n] [-N] [-v] [-V]

cleanup_perfdata: remove perfdata files not needed anymore

optional arguments:

-h, --help show this help message and exit

-x XDAYS, --xdays XDAYS

set fileage in days [default: 90]

-n, --dryrun do not delete anything, just report [default: False]

-N ‘dryrun’ but non-verbose, just stats [default: False]

-v, --verbose set verbosity level [default: False]

-V, --version show program’s version number and exit

Dryrun (“-n|–dryrun”) implies “-v|–verbose”.

Try different XDAYS with dryrun to get a clue what’s going on :wink:

Comments are very much appreciated.

https://github.com/dasmarci/check_mk/blob/master/scripts/cleanup_perfdata.py

Regards,

Marcel

Jeffrey J. Gerschkow jgerschk@comcast.net schrieb am Sa., 15. Aug. 2015 um 14:37 Uhr:

What I have used and modify is this.

https://github.com/lingej/pnp4nagios/blob/master/scripts/check_pnp_rrds.pl.in

Might be worth something to some.

Not only can you add this as a check (classic Nagios/icinga), but there is a command line options –deletexml –deleterrd that can be used to help in cleanup. The script itself, when using those you have to hit Y to delete for each, but easily modified if you want to add it to a cron job to delete.

Just another suggestion for aiding in cleanup.

From: checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de] On Behalf Of paulmonitoring
Sent: Friday, August 14, 2015 5:45 PM
To: Marcel Schulte schulte.marcel@gmail.com; Andreas Döhler andreas.doehler@gmail.com; Gerardo Ferreyra raptorg83@gmail.com; checkmk-en@lists.mathias-kettner.de checkmk-en@lists.mathias-kettner.de; omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

I’m interested.

-------- Original message --------
From: Marcel Schulte schulte.marcel@gmail.com
Date: 08/14/2015 16:38 (GMT-05:00)
To: paulmonitoring paulmonitoring@gmail.com, Andreas Döhler andreas.doehler@gmail.com, Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de, omd-users@lists.mathias-kettner.de
Subject: Re: [Check_mk (english)] Perfdata cleanup

Hi,

I’m currently working on a weekly cronjob doing the cleanup for me.

What’s already working:

  • Find data of deleted hosts and remove the whole host folder from perfect.
  • Find all remaining files with modification time older than a configurable amount of days (defaults to 90) and remove them.

The latter will be rewritten to only remove the file if it’s known not to be needed anymore. I think this will be usable mid of next week, there’s more important work to do first :wink:

Whole thing is written in python and works inside OMD sites without modifications. For non-OMD installations a single variable must be changed.

…I could post the mentioned working part at Monday if anyone is interested.

Best regards,
Marcel

paulmonitoring paulmonitoring@gmail.com schrieb am Fr., 14. Aug. 2015 22:24:

I echo what Andreas says. Be cautious. In the past I may or may not have deleted a lot of data by accident from the perfdata dir. :wink:

Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone

-------- Original message --------
From: Andreas Döhler andreas.doehler@gmail.com
Date: 08/14/2015 15:32 (GMT-05:00)
To: Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de, omd-users@lists.mathias-kettner.de
Subject: Re: [Check_mk (english)] Perfdata cleanup

Yes and no.
Perfdata directory will only get bigger with new hosts in the moment you create one host the total space is allocated. What I do on my systems is from time to time a find with older then 30-60 days on this directory. With this I remove obsolete perfdata from deleted machines on non existent checks.

But pay attention what you delete there :slight_smile:

Best regards
Andreas

Gerardo Ferreyra raptorg83@gmail.com schrieb am Fr., 14. Aug. 2015 21:19:

Hi guys.

A quick one. I have hundreds of servers being monitored by check_mk.

I’m seeing that perfdata directory is getting very big.

/opt/omd/sites/prod/var/pnp4nagios/perfdata

I need performance data, of course.

My question.

I have no problem (for now) to increase the disk, but… is there a internal functionality on check_mk (wato) I need to activate, that rotates this data? Set to… retain X months of data? Something like that?

I see “Diskspace cleanup” on global settings. Is that?

Thank you very much!

checkmk-en mailing list
checkmk-en@lists.mathias-kettner.de
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.
October 18th-20th, 2015
http://mathias-kettner.com/conference

checkmk-en mailing list
checkmk-en@lists.mathias-kettner.de
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!
Book your place now and be part of it.
October 18th-20th, 2015
http://mathias-kettner.com/conference

4

I had to add

old_files = {}

at around line 197 at the start of process_host, otherwise deleting a host which no longer exists creates an exception.

Cheers,

Phil

···

Hi guys (and girls, if any),

I finished my cronjob script to cleanup perfdata files.

The script is intended to be run as OMD siteuser - but will complain about which variables to tweak if it’s not :wink:

If you’re not using CMC (“Check_MK Micro Core”) you must set line 48 to False

Here’s the usage:

$cleanup_perfdata.py -h

usage: cleanup_perfdata.py [-h] [-x XDAYS] [-n] [-N] [-v] [-V]

cleanup_perfdata: remove perfdata files not needed anymore

optional arguments:

-h, --help show this help message and exit

-x XDAYS, --xdays XDAYS

set fileage in days [default: 90]

-n, --dryrun do not delete anything, just report [default: False]

-N ‘dryrun’ but non-verbose, just stats [default: False]

-v, --verbose set verbosity level [default: False]

-V, --version show program’s version number and exit

Dryrun (“-n|–dryrun”) implies “-v|–verbose”.

Try different XDAYS with dryrun to get a clue what’s going on :wink:

Comments are very much appreciated.

https://github.com/dasmarci/check_mk/blob/master/scripts/cleanup_perfdata.py

Regards,

Marcel

Jeffrey J. Gerschkow jgerschk@comcast.net schrieb am Sa., 15. Aug. 2015 um 14:37 Uhr:

What I have used and modify is this.

https://github.com/lingej/pnp4nagios/blob/master/scripts/check_pnp_rrds.pl.in

Might be worth something to some.

Not only can you add this as a check (classic Nagios/icinga), but there is a command
line options –deletexml –deleterrd that can be used to help in cleanup. The script itself, when using those you have to hit Y to delete for each, but easily modified if you want to add it to a cron job to delete.

Just another suggestion for aiding in cleanup.

From:
checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
On Behalf Of paulmonitoring
Sent: Friday, August 14, 2015 5:45 PM
To: Marcel Schulte schulte.marcel@gmail.com; Andreas Döhler andreas.doehler@gmail.com; Gerardo Ferreyra <raptorg83@gmail.com >;
checkmk-en@lists.mathias-kettner.de checkmk-en@lists.mathias-kettner.de;
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

I’m interested.

-------- Original message --------
From: Marcel Schulte schulte.marcel@gmail.com

Date: 08/14/2015 16:38 (GMT-05:00)
To: paulmonitoring paulmonitoring@gmail.com, Andreas Döhler andreas.doehler@gmail.com, Gerardo Ferreyra <raptorg83@gmail.com >,
“checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Hi,

I’m currently working on a weekly cronjob doing the cleanup for me.

What’s already working:

  • Find data of deleted hosts and remove the whole host folder from perfect.
  • Find all remaining files with modification time older than a configurable amount of days (defaults to 90) and remove them.

The latter will be rewritten to only remove the file if it’s known not to be needed anymore. I think this will be usable mid of next week, there’s more important work to do first :wink:

Whole thing is written in python and works inside OMD sites without modifications. For non-OMD installations a single variable must be changed.

…I could post the mentioned working part at Monday if anyone is interested.

Best regards,
Marcel

paulmonitoring paulmonitoring@gmail.com schrieb am Fr., 14. Aug. 2015 22:24:

I echo what Andreas says. Be cautious. In the past I may or may not have deleted a lot of data by accident from the perfdata dir. :wink:

Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone

-------- Original message --------
From: Andreas Döhler andreas.doehler@gmail.com

Date: 08/14/2015 15:32 (GMT-05:00)
To: Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Yes and no.
Perfdata directory will only get bigger with new hosts in the moment you create one host the total space is allocated. What I do on my systems is from time to time a find with older then 30-60 days on this directory. With this I remove obsolete perfdata from
deleted machines on non existent checks.

But pay attention what you delete there :slight_smile:

Best regards
Andreas

Gerardo Ferreyra raptorg83@gmail.com schrieb am Fr., 14. Aug. 2015 21:19:

Hi guys.

A quick one. I have hundreds of servers being monitored by check_mk.

I’m seeing that perfdata directory is getting very big.

/opt/omd/sites/prod/var/pnp4nagios/perfdata

I need performance data, of course.

My question.

I have no problem (for now) to increase the disk, but… is there a internal functionality on check_mk (wato) I need to activate, that rotates this data? Set
to… retain X months of data? Something like that?

I see “Diskspace cleanup” on global settings. Is that?

Thank you very much!

checkmk-en mailing list
checkmk-en@lists.mathias-kettner.de
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.
October 18th-20th, 2015
http://mathias-kettner.com/conference

checkmk-en mailing list
checkmk-en@lists.mathias-kettner.de
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!
Book your place now and be part of it.
October 18th-20th, 2015
http://mathias-kettner.com/conference

5

Alternatively, indent

if old_files.get(‘xml’, None):

all_files = process_xml_files(host, all_files, old_files)

if old_files.get(‘rrd’, None):

process_rrd_files(host, all_files, old_files)

an extra level.

Phil

···

–

Phil Randal

Infrastructure Engineer

Hoople Ltd | Thorn Office Centre | Hereford | HR2 6JT

Tel : 01432 260415 |Email:
phil.randal@hoopleltd.co.uk

General email:
enquiries@hoopleltd.co.uk

Website: www.hoopleltd.co.uk

From: Randal, Phil
Sent: 17 August 2015 16:20
To: ‘Marcel Schulte’; checkmk-en@lists.mathias-kettner.de; omd-users@lists.mathias-kettner.de
Subject: RE: [Check_mk (english)] Perfdata cleanup

I had to add

old_files = {}

at around line 197 at the start of process_host, otherwise deleting a host which no longer exists creates an exception.

Cheers,

Phil

–

Phil Randal

Infrastructure Engineer

Hoople Ltd | Thorn Office Centre | Hereford | HR2 6JT

Tel : 01432 260415 |Email:
phil.randal@hoopleltd.co.uk

General email:
enquiries@hoopleltd.co.uk

Website: www.hoopleltd.co.uk

From:
checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
On Behalf Of Marcel Schulte
Sent: 17 August 2015 15:26
To: checkmk-en@lists.mathias-kettner.de;
omd-users@lists.mathias-kettner.de
Subject: Re: [Check_mk (english)] Perfdata cleanup

Hi guys (and girls, if any),

I finished my cronjob script to cleanup perfdata files.

The script is intended to be run as OMD siteuser - but will complain about which variables to tweak if it’s not :wink:

If you’re not using CMC (“Check_MK Micro Core”) you must set line 48 to False

Here’s the usage:

$cleanup_perfdata.py -h

usage: cleanup_perfdata.py [-h] [-x XDAYS] [-n] [-N] [-v] [-V]

cleanup_perfdata: remove perfdata files not needed anymore

optional arguments:

-h, --help show this help message and exit

-x XDAYS, --xdays XDAYS

set fileage in days [default: 90]

-n, --dryrun do not delete anything, just report [default: False]

-N ‘dryrun’ but non-verbose, just stats [default: False]

-v, --verbose set verbosity level [default: False]

-V, --version show program’s version number and exit

Dryrun (“-n|–dryrun”) implies “-v|–verbose”.

Try different XDAYS with dryrun to get a clue what’s going on :wink:

Comments are very much appreciated.

https://github.com/dasmarci/check_mk/blob/master/scripts/cleanup_perfdata.py

Regards,

Marcel

Jeffrey J. Gerschkow jgerschk@comcast.net schrieb am Sa., 15. Aug. 2015 um 14:37 Uhr:

What I have used and modify is this.

https://github.com/lingej/pnp4nagios/blob/master/scripts/check_pnp_rrds.pl.in

Might be worth something to some.

Not only can you add this as a check (classic Nagios/icinga), but there is a command
line options –deletexml –deleterrd that can be used to help in cleanup. The script itself, when using those you have to hit Y to delete for each, but easily modified if you want to add it to a cron job to delete.

Just another suggestion for aiding in cleanup.

From:
checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
On Behalf Of paulmonitoring
Sent: Friday, August 14, 2015 5:45 PM
To: Marcel Schulte schulte.marcel@gmail.com; Andreas Döhler andreas.doehler@gmail.com; Gerardo Ferreyra <raptorg83@gmail.com >;
checkmk-en@lists.mathias-kettner.de checkmk-en@lists.mathias-kettner.de;
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

I’m interested.

-------- Original message --------
From: Marcel Schulte schulte.marcel@gmail.com

Date: 08/14/2015 16:38 (GMT-05:00)
To: paulmonitoring paulmonitoring@gmail.com, Andreas Döhler andreas.doehler@gmail.com, Gerardo Ferreyra <raptorg83@gmail.com >,
“checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Hi,

I’m currently working on a weekly cronjob doing the cleanup for me.

What’s already working:

  • Find data of deleted hosts and remove the whole host folder from perfect.
  • Find all remaining files with modification time older than a configurable amount of days (defaults to 90) and remove them.

The latter will be rewritten to only remove the file if it’s known not to be needed anymore. I think this will be usable mid of next week, there’s more important work to do first :wink:

Whole thing is written in python and works inside OMD sites without modifications. For non-OMD installations a single variable must be changed.

…I could post the mentioned working part at Monday if anyone is interested.

Best regards,
Marcel

paulmonitoring paulmonitoring@gmail.com schrieb am Fr., 14. Aug. 2015 22:24:

I echo what Andreas says. Be cautious. In the past I may or may not have deleted a lot of data by accident from the perfdata dir. :wink:

Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone

-------- Original message --------
From: Andreas Döhler andreas.doehler@gmail.com

Date: 08/14/2015 15:32 (GMT-05:00)
To: Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Yes and no.
Perfdata directory will only get bigger with new hosts in the moment you create one host the total space is allocated. What I do on my systems is from time to time a find with older then 30-60 days on this directory. With this I remove obsolete perfdata from
deleted machines on non existent checks.

But pay attention what you delete there :slight_smile:

Best regards
Andreas

Gerardo Ferreyra raptorg83@gmail.com schrieb am Fr., 14. Aug. 2015 21:19:

Hi guys.

A quick one. I have hundreds of servers being monitored by check_mk.

I’m seeing that perfdata directory is getting very big.

/opt/omd/sites/prod/var/pnp4nagios/perfdata

I need performance data, of course.

My question.

I have no problem (for now) to increase the disk, but… is there a internal functionality on check_mk (wato) I need to activate, that rotates this data? Set
to… retain X months of data? Something like that?

I see “Diskspace cleanup” on global settings. Is that?

Thank you very much!

checkmk-en mailing list
checkmk-en@lists.mathias-kettner.de
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.
October 18th-20th, 2015
http://mathias-kettner.com/conference

checkmk-en mailing list
checkmk-en@lists.mathias-kettner.de
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!
Book your place now and be part of it.
October 18th-20th, 2015
http://mathias-kettner.com/conference

6

Hi Marcel,

i like your script, but you can achieve very similar results with a bash one liner:

for x in $(find $OMD_ROOT/var/pnp4nagios/perfdata –mtime +30); do rm -rf „$x“; done # this can be dangerous!

Ok, it can’t actively look for vanished hosts, but they will disappear after 30 days of inactivity of their folder.

Not perfect, but it was done quick and dirty.

Mit freundlichen Grüßen

i.A. Jakub Tusz

image001.jpg

···

Telefon:
Fax:
E-Mail:

+49(721)6107 5905
+49(721)6107 5909
jakub.tusz@vbk.karlsruhe.de

Kfm. Geschäftsführer: Dr. Alexander Pischon, Techn. Geschäftsführer: Ascan Egerer

Vorsitzender des Aufsichtsrates: Oberbürgermeister Dr. Frank Mentrup

Amtsgericht Mannheim HRB 107847

Von: checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
Im Auftrag von Marcel Schulte
Gesendet: Montag, 17. August 2015 16:26
An: checkmk-en@lists.mathias-kettner.de; omd-users@lists.mathias-kettner.de
Betreff: Re: [Check_mk (english)] Perfdata cleanup

Hi guys (and girls, if any),

I finished my cronjob script to cleanup perfdata files.

The script is intended to be run as OMD siteuser - but will complain about which variables to tweak if it’s not :wink:

If you’re not using CMC (“Check_MK Micro Core”) you must set line 48 to False

Here’s the usage:

$cleanup_perfdata.py -h

usage: cleanup_perfdata.py [-h] [-x XDAYS] [-n] [-N] [-v] [-V]

cleanup_perfdata: remove perfdata files not needed anymore

optional arguments:

-h, --help show this help message and exit

-x XDAYS, --xdays XDAYS

set fileage in days [default: 90]

-n, --dryrun do not delete anything, just report [default: False]

-N ‘dryrun’ but non-verbose, just stats [default: False]

-v, --verbose set verbosity level [default: False]

-V, --version show program’s version number and exit

Dryrun (“-n|–dryrun”) implies “-v|–verbose”.

Try different XDAYS with dryrun to get a clue what’s going on :wink:

Comments are very much appreciated.

https://github.com/dasmarci/check_mk/blob/master/scripts/cleanup_perfdata.py

Regards,

Marcel

Jeffrey J. Gerschkow jgerschk@comcast.net schrieb am Sa., 15. Aug. 2015 um 14:37 Uhr:

What I have used and modify is this.

https://github.com/lingej/pnp4nagios/blob/master/scripts/check_pnp_rrds.pl.in

Might be worth something to some.

Not only can you add this as a check (classic Nagios/icinga), but there is a command line
options –deletexml –deleterrd that can be used to help in cleanup. The script itself, when using those you have to hit Y to delete for each, but easily modified if you want to add it to a cron job to delete.

Just another suggestion for aiding in cleanup.

From:
checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
On Behalf Of paulmonitoring
Sent: Friday, August 14, 2015 5:45 PM
To: Marcel Schulte schulte.marcel@gmail.com; Andreas Döhler andreas.doehler@gmail.com; Gerardo Ferreyra <raptorg83@gmail.com >;
checkmk-en@lists.mathias-kettner.de checkmk-en@lists.mathias-kettner.de;
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

I’m interested.

-------- Original message --------
From: Marcel Schulte schulte.marcel@gmail.com

Date: 08/14/2015 16:38 (GMT-05:00)
To: paulmonitoring paulmonitoring@gmail.com, Andreas Döhler andreas.doehler@gmail.com, Gerardo Ferreyra <raptorg83@gmail.com >,
“checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Hi,

I’m currently working on a weekly cronjob doing the cleanup for me.

What’s already working:

  • Find data of deleted hosts and remove the whole host folder from perfect.
  • Find all remaining files with modification time older than a configurable amount of days (defaults to 90) and remove them.

The latter will be rewritten to only remove the file if it’s known not to be needed anymore. I think this will be usable mid of next week, there’s more important work to do first :wink:

Whole thing is written in python and works inside OMD sites without modifications. For non-OMD installations a single variable must be changed.

…I could post the mentioned working part at Monday if anyone is interested.

Best regards,
Marcel

paulmonitoring paulmonitoring@gmail.com schrieb am Fr., 14. Aug. 2015 22:24:

I echo what Andreas says. Be cautious. In the past I may or may not have deleted a lot of data by accident from the perfdata dir. :wink:

Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone

-------- Original message --------
From: Andreas Döhler andreas.doehler@gmail.com

Date: 08/14/2015 15:32 (GMT-05:00)
To: Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Yes and no.
Perfdata directory will only get bigger with new hosts in the moment you create one host the total space is allocated. What I do on my systems is from time to time a find with older then 30-60 days on this directory. With this I remove obsolete perfdata from
deleted machines on non existent checks.

But pay attention what you delete there :slight_smile:

Best regards
Andreas

Gerardo Ferreyra raptorg83@gmail.com schrieb am Fr., 14. Aug. 2015 21:19:

Hi guys.

A quick one. I have hundreds of servers being monitored by check_mk.

I’m seeing that perfdata directory is getting very big.

/opt/omd/sites/prod/var/pnp4nagios/perfdata

I need performance data, of course.

My question.

I have no problem (for now) to increase the disk, but… is there a internal functionality on check_mk (wato) I need to activate, that rotates this data? Set
to… retain X months of data? Something like that?

I see “Diskspace cleanup” on global settings. Is that?

Thank you very much!

checkmk-en mailing list
checkmk-en@lists.mathias-kettner.de
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.
October 18th-20th, 2015
http://mathias-kettner.com/conference

checkmk-en mailing list
checkmk-en@lists.mathias-kettner.de
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!
Book your place now and be part of it.
October 18th-20th, 2015
http://mathias-kettner.com/conference

7

Hi Jakub,

I know, had such jobs in former - until I realized those quick-and-dirty jobs removed too many files still needed.

Think of hosts in downtime or where the Check_MK service fails - and no more data is written. Host and services are still there but the data does not get updated so they where removed…

Another failing thing was a configuration of mine where an switch interface was configured to ignore ifOperStatus because it’s down most time. The if check does not provide perfdata if an interface is down so the service’s perfdata was only updated in rare cases where the interface was up - and my oneliner removed XML and RRD files :frowning:

That’s why I tried to find out how to prevent losing needed data.

Best regards,

Marcel

image001.jpg

···

Tusz, Jakub Jakub.Tusz@kvv.karlsruhe.de schrieb am Di., 18. Aug. 2015 um 12:55 Uhr:

Hi Marcel,

i like your script, but you can achieve very similar results with a bash one liner:

for x in $(find $OMD_ROOT/var/pnp4nagios/perfdata –mtime +30); do rm -rf „$x“; done # this can be dangerous!

Ok, it can’t actively look for vanished hosts, but they will disappear after 30 days of inactivity of their folder.

Not perfect, but it was done quick and dirty.

Mit freundlichen Grüßen

i.A. Jakub Tusz

Jakub Tusz

Abt.: V5-IT - Verwaltung | Informationstechnologie

VBK - Verkehrsbetriebe Karlsruhe GmbH

Tullastraße 71, D-76131 Karlsruhe

Postfach 1140, D-76001 Karlsruhe

Telefon:

Fax:

E-Mail:

+49(721)6107 5905

+49(721)6107 5909

jakub.tusz@vbk.karlsruhe.de

Kfm. Geschäftsführer: Dr. Alexander Pischon, Techn. Geschäftsführer: Ascan Egerer

Vorsitzender des Aufsichtsrates: Oberbürgermeister Dr. Frank Mentrup

Amtsgericht Mannheim HRB 107847

Von: checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
Im Auftrag von Marcel Schulte
Gesendet: Montag, 17. August 2015 16:26
An: checkmk-en@lists.mathias-kettner.de; omd-users@lists.mathias-kettner.de
Betreff: Re: [Check_mk (english)] Perfdata cleanup

Hi guys (and girls, if any),

I finished my cronjob script to cleanup perfdata files.

The script is intended to be run as OMD siteuser - but will complain about which variables to tweak if it’s not :wink:

If you’re not using CMC (“Check_MK Micro Core”) you must set line 48 to False

Here’s the usage:

$cleanup_perfdata.py -h

usage: cleanup_perfdata.py [-h] [-x XDAYS] [-n] [-N] [-v] [-V]

cleanup_perfdata: remove perfdata files not needed anymore

optional arguments:

-h, --help show this help message and exit

-x XDAYS, --xdays XDAYS

set fileage in days [default: 90]

-n, --dryrun do not delete anything, just report [default: False]

-N ‘dryrun’ but non-verbose, just stats [default: False]

-v, --verbose set verbosity level [default: False]

-V, --version show program’s version number and exit

Dryrun (“-n|–dryrun”) implies “-v|–verbose”.

Try different XDAYS with dryrun to get a clue what’s going on :wink:

Comments are very much appreciated.

https://github.com/dasmarci/check_mk/blob/master/scripts/cleanup_perfdata.py

Regards,

Marcel

Jeffrey J. Gerschkow jgerschk@comcast.net schrieb am Sa., 15. Aug. 2015 um 14:37 Uhr:

What I have used and modify is this.

https://github.com/lingej/pnp4nagios/blob/master/scripts/check_pnp_rrds.pl.in

Might be worth something to some.

Not only can you add this as a check (classic Nagios/icinga), but there is a command line
options –deletexml –deleterrd that can be used to help in cleanup. The script itself, when using those you have to hit Y to delete for each, but easily modified if you want to add it to a cron job to delete.

Just another suggestion for aiding in cleanup.

From:
checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
On Behalf Of paulmonitoring
Sent: Friday, August 14, 2015 5:45 PM
To: Marcel Schulte schulte.marcel@gmail.com; Andreas Döhler andreas.doehler@gmail.com; Gerardo Ferreyra <raptorg83@gmail.com >;
checkmk-en@lists.mathias-kettner.de checkmk-en@lists.mathias-kettner.de;
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

I’m interested.

-------- Original message --------

From: Marcel Schulte schulte.marcel@gmail.com

Date: 08/14/2015 16:38 (GMT-05:00)

To: paulmonitoring paulmonitoring@gmail.com, Andreas Döhler andreas.doehler@gmail.com, Gerardo Ferreyra <raptorg83@gmail.com >,
“checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Hi,

I’m currently working on a weekly cronjob doing the cleanup for me.

What’s already working:

  • Find data of deleted hosts and remove the whole host folder from perfect.

  • Find all remaining files with modification time older than a configurable amount of days (defaults to 90) and remove them.

The latter will be rewritten to only remove the file if it’s known not to be needed anymore. I think this will be usable mid of next week, there’s more important work to do first :wink:

Whole thing is written in python and works inside OMD sites without modifications. For non-OMD installations a single variable must be changed.

…I could post the mentioned working part at Monday if anyone is interested.

Best regards,

Marcel

paulmonitoring paulmonitoring@gmail.com schrieb am Fr., 14. Aug. 2015 22:24:

I echo what Andreas says. Be cautious. In the past I may or may not have deleted a lot of data by accident from the perfdata dir. :wink:

Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone

-------- Original message --------

From: Andreas Döhler andreas.doehler@gmail.com

Date: 08/14/2015 15:32 (GMT-05:00)

To: Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Yes and no.

Perfdata directory will only get bigger with new hosts in the moment you create one host the total space is allocated. What I do on my systems is from time to time a find with older then 30-60 days on this directory. With this I remove obsolete perfdata from
deleted machines on non existent checks.

But pay attention what you delete there :slight_smile:

Best regards

Andreas

Gerardo Ferreyra raptorg83@gmail.com schrieb am Fr., 14. Aug. 2015 21:19:

Hi guys.

A quick one. I have hundreds of servers being monitored by check_mk.

I’m seeing that perfdata directory is getting very big.

/opt/omd/sites/prod/var/pnp4nagios/perfdata

I need performance data, of course.

My question.

I have no problem (for now) to increase the disk, but… is there a internal functionality on check_mk (wato) I need to activate, that rotates this data? Set
to… retain X months of data? Something like that?

I see “Diskspace cleanup” on global settings. Is that?

Thank you very much!

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.

October 18th-20th, 2015

http://mathias-kettner.com/conference

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.

October 18th-20th, 2015

http://mathias-kettner.com/conference

Diese E-Mail kann vertrauliche und/oder rechtlich geschützte Informationen enthalten. Wenn Sie nicht der richtige Adressat sind, so bitten wir Sie, sofort den Absender zu informieren und diese E-Mail vollständig zu löschen.
Das unerlaubte Kopieren, Weiterleiten, Verbreiten oder Verwenden dieser E-Mail und deren Inhalte ist nicht gestattet.

8

Are you guys saying that the perf data will never get cleared by OMD or check_mk on its own if the host got removed?

image001.jpg

···

On Tue, Aug 18, 2015 at 3:55 AM, Tusz, Jakub Jakub.Tusz@kvv.karlsruhe.de wrote:

Hi Marcel,

i like your script, but you can achieve very similar results with a bash one liner:

for x in $(find $OMD_ROOT/var/pnp4nagios/perfdata –mtime +30); do rm -rf „$x“; done # this can be dangerous!

Ok, it can’t actively look for vanished hosts, but they will disappear after 30 days of inactivity of their folder.

Not perfect, but it was done quick and dirty.

Mit freundlichen Grüßen

i.A. Jakub Tusz

Jakub Tusz

Abt.: V5-IT - Verwaltung | Informationstechnologie

VBK - Verkehrsbetriebe Karlsruhe GmbH

Tullastraße 71, D-76131 Karlsruhe

Postfach 1140, D-76001 Karlsruhe

Telefon:

Fax:

E-Mail:

+49(721)6107 5905

+49(721)6107 5909

jakub.tusz@vbk.karlsruhe.de

Kfm. Geschäftsführer: Dr. Alexander Pischon, Techn. Geschäftsführer: Ascan Egerer

Vorsitzender des Aufsichtsrates: Oberbürgermeister Dr. Frank Mentrup

Amtsgericht Mannheim HRB 107847

Von: checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
Im Auftrag von Marcel Schulte
Gesendet: Montag, 17. August 2015 16:26
An: checkmk-en@lists.mathias-kettner.de; omd-users@lists.mathias-kettner.de
Betreff: Re: [Check_mk (english)] Perfdata cleanup

Hi guys (and girls, if any),

I finished my cronjob script to cleanup perfdata files.

The script is intended to be run as OMD siteuser - but will complain about which variables to tweak if it’s not :wink:

If you’re not using CMC (“Check_MK Micro Core”) you must set line 48 to False

Here’s the usage:

$cleanup_perfdata.py -h

usage: cleanup_perfdata.py [-h] [-x XDAYS] [-n] [-N] [-v] [-V]

cleanup_perfdata: remove perfdata files not needed anymore

optional arguments:

-h, --help show this help message and exit

-x XDAYS, --xdays XDAYS

set fileage in days [default: 90]

-n, --dryrun do not delete anything, just report [default: False]

-N ‘dryrun’ but non-verbose, just stats [default: False]

-v, --verbose set verbosity level [default: False]

-V, --version show program’s version number and exit

Dryrun (“-n|–dryrun”) implies “-v|–verbose”.

Try different XDAYS with dryrun to get a clue what’s going on :wink:

Comments are very much appreciated.

https://github.com/dasmarci/check_mk/blob/master/scripts/cleanup_perfdata.py

Regards,

Marcel

Jeffrey J. Gerschkow jgerschk@comcast.net schrieb am Sa., 15. Aug. 2015 um 14:37 Uhr:

What I have used and modify is this.

https://github.com/lingej/pnp4nagios/blob/master/scripts/check_pnp_rrds.pl.in

Might be worth something to some.

Not only can you add this as a check (classic Nagios/icinga), but there is a command line
options –deletexml –deleterrd that can be used to help in cleanup. The script itself, when using those you have to hit Y to delete for each, but easily modified if you want to add it to a cron job to delete.

Just another suggestion for aiding in cleanup.

From:
checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
On Behalf Of paulmonitoring
Sent: Friday, August 14, 2015 5:45 PM
To: Marcel Schulte schulte.marcel@gmail.com; Andreas Döhler andreas.doehler@gmail.com; Gerardo Ferreyra <raptorg83@gmail.com >;
checkmk-en@lists.mathias-kettner.de checkmk-en@lists.mathias-kettner.de;
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

I’m interested.

-------- Original message --------

From: Marcel Schulte schulte.marcel@gmail.com

Date: 08/14/2015 16:38 (GMT-05:00)

To: paulmonitoring paulmonitoring@gmail.com, Andreas Döhler andreas.doehler@gmail.com, Gerardo Ferreyra <raptorg83@gmail.com >,
“checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Hi,

I’m currently working on a weekly cronjob doing the cleanup for me.

What’s already working:

  • Find data of deleted hosts and remove the whole host folder from perfect.

  • Find all remaining files with modification time older than a configurable amount of days (defaults to 90) and remove them.

The latter will be rewritten to only remove the file if it’s known not to be needed anymore. I think this will be usable mid of next week, there’s more important work to do first :wink:

Whole thing is written in python and works inside OMD sites without modifications. For non-OMD installations a single variable must be changed.

…I could post the mentioned working part at Monday if anyone is interested.

Best regards,

Marcel

paulmonitoring paulmonitoring@gmail.com schrieb am Fr., 14. Aug. 2015 22:24:

I echo what Andreas says. Be cautious. In the past I may or may not have deleted a lot of data by accident from the perfdata dir. :wink:

Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone

-------- Original message --------

From: Andreas Döhler andreas.doehler@gmail.com

Date: 08/14/2015 15:32 (GMT-05:00)

To: Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Yes and no.

Perfdata directory will only get bigger with new hosts in the moment you create one host the total space is allocated. What I do on my systems is from time to time a find with older then 30-60 days on this directory. With this I remove obsolete perfdata from
deleted machines on non existent checks.

But pay attention what you delete there :slight_smile:

Best regards

Andreas

Gerardo Ferreyra raptorg83@gmail.com schrieb am Fr., 14. Aug. 2015 21:19:

Hi guys.

A quick one. I have hundreds of servers being monitored by check_mk.

I’m seeing that perfdata directory is getting very big.

/opt/omd/sites/prod/var/pnp4nagios/perfdata

I need performance data, of course.

My question.

I have no problem (for now) to increase the disk, but… is there a internal functionality on check_mk (wato) I need to activate, that rotates this data? Set
to… retain X months of data? Something like that?

I see “Diskspace cleanup” on global settings. Is that?

Thank you very much!

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.

October 18th-20th, 2015

http://mathias-kettner.com/conference

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.

October 18th-20th, 2015

http://mathias-kettner.com/conference

Diese E-Mail kann vertrauliche und/oder rechtlich geschützte Informationen enthalten. Wenn Sie nicht der richtige Adressat sind, so bitten wir Sie, sofort den Absender zu informieren und diese E-Mail vollständig zu löschen.
Das unerlaubte Kopieren, Weiterleiten, Verbreiten oder Verwenden dieser E-Mail und deren Inhalte ist nicht gestattet.

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.

October 18th-20th, 2015

http://mathias-kettner.com/conference

9

Hi Henry,

absolutely, you’ll have to do that on your own. Okay, maybe the host’s whole perfdata folder could get removed when host is deleted in WATO or via webapi…

But what about old/left RRD and/or XML files? When should they get deleted?

Services get created when inventoried and they create RRD/XML, too, if not available. Service deletion is done by reinventory. Reinventory means “delete all (inventoried) services, then do an inventory” - should all perfdata get deleted because their services get removed?

It’s a difficult (and maybe dangerous) decision - so so users should do that themselves.

Regards,

Marcel

image001.jpg

···

On Tue, Aug 18, 2015 at 3:55 AM, Tusz, Jakub Jakub.Tusz@kvv.karlsruhe.de wrote:

Hi Marcel,

i like your script, but you can achieve very similar results with a bash one liner:

for x in $(find $OMD_ROOT/var/pnp4nagios/perfdata –mtime +30); do rm -rf „$x“; done # this can be dangerous!

Ok, it can’t actively look for vanished hosts, but they will disappear after 30 days of inactivity of their folder.

Not perfect, but it was done quick and dirty.

Mit freundlichen Grüßen

i.A. Jakub Tusz

Jakub Tusz

Abt.: V5-IT - Verwaltung | Informationstechnologie

VBK - Verkehrsbetriebe Karlsruhe GmbH

Tullastraße 71, D-76131 Karlsruhe

Postfach 1140, D-76001 Karlsruhe

Telefon:

Fax:

E-Mail:

+49(721)6107 5905

+49(721)6107 5909

jakub.tusz@vbk.karlsruhe.de

Kfm. Geschäftsführer: Dr. Alexander Pischon, Techn. Geschäftsführer: Ascan Egerer

Vorsitzender des Aufsichtsrates: Oberbürgermeister Dr. Frank Mentrup

Amtsgericht Mannheim HRB 107847

Von: checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
Im Auftrag von Marcel Schulte
Gesendet: Montag, 17. August 2015 16:26
An: checkmk-en@lists.mathias-kettner.de; omd-users@lists.mathias-kettner.de
Betreff: Re: [Check_mk (english)] Perfdata cleanup

Hi guys (and girls, if any),

I finished my cronjob script to cleanup perfdata files.

The script is intended to be run as OMD siteuser - but will complain about which variables to tweak if it’s not :wink:

If you’re not using CMC (“Check_MK Micro Core”) you must set line 48 to False

Here’s the usage:

$cleanup_perfdata.py -h

usage: cleanup_perfdata.py [-h] [-x XDAYS] [-n] [-N] [-v] [-V]

cleanup_perfdata: remove perfdata files not needed anymore

optional arguments:

-h, --help show this help message and exit

-x XDAYS, --xdays XDAYS

set fileage in days [default: 90]

-n, --dryrun do not delete anything, just report [default: False]

-N ‘dryrun’ but non-verbose, just stats [default: False]

-v, --verbose set verbosity level [default: False]

-V, --version show program’s version number and exit

Dryrun (“-n|–dryrun”) implies “-v|–verbose”.

Try different XDAYS with dryrun to get a clue what’s going on :wink:

Comments are very much appreciated.

https://github.com/dasmarci/check_mk/blob/master/scripts/cleanup_perfdata.py

Regards,

Marcel

Jeffrey J. Gerschkow jgerschk@comcast.net schrieb am Sa., 15. Aug. 2015 um 14:37 Uhr:

What I have used and modify is this.

https://github.com/lingej/pnp4nagios/blob/master/scripts/check_pnp_rrds.pl.in

Might be worth something to some.

Not only can you add this as a check (classic Nagios/icinga), but there is a command line
options –deletexml –deleterrd that can be used to help in cleanup. The script itself, when using those you have to hit Y to delete for each, but easily modified if you want to add it to a cron job to delete.

Just another suggestion for aiding in cleanup.

From:
checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
On Behalf Of paulmonitoring
Sent: Friday, August 14, 2015 5:45 PM
To: Marcel Schulte schulte.marcel@gmail.com; Andreas Döhler andreas.doehler@gmail.com; Gerardo Ferreyra <raptorg83@gmail.com >;
checkmk-en@lists.mathias-kettner.de checkmk-en@lists.mathias-kettner.de;
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

I’m interested.

-------- Original message --------

From: Marcel Schulte schulte.marcel@gmail.com

Date: 08/14/2015 16:38 (GMT-05:00)

To: paulmonitoring paulmonitoring@gmail.com, Andreas Döhler andreas.doehler@gmail.com, Gerardo Ferreyra <raptorg83@gmail.com >,
“checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Hi,

I’m currently working on a weekly cronjob doing the cleanup for me.

What’s already working:

  • Find data of deleted hosts and remove the whole host folder from perfect.

  • Find all remaining files with modification time older than a configurable amount of days (defaults to 90) and remove them.

The latter will be rewritten to only remove the file if it’s known not to be needed anymore. I think this will be usable mid of next week, there’s more important work to do first :wink:

Whole thing is written in python and works inside OMD sites without modifications. For non-OMD installations a single variable must be changed.

…I could post the mentioned working part at Monday if anyone is interested.

Best regards,

Marcel

paulmonitoring paulmonitoring@gmail.com schrieb am Fr., 14. Aug. 2015 22:24:

I echo what Andreas says. Be cautious. In the past I may or may not have deleted a lot of data by accident from the perfdata dir. :wink:

Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone

-------- Original message --------

From: Andreas Döhler andreas.doehler@gmail.com

Date: 08/14/2015 15:32 (GMT-05:00)

To: Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Yes and no.

Perfdata directory will only get bigger with new hosts in the moment you create one host the total space is allocated. What I do on my systems is from time to time a find with older then 30-60 days on this directory. With this I remove obsolete perfdata from
deleted machines on non existent checks.

But pay attention what you delete there :slight_smile:

Best regards

Andreas

Gerardo Ferreyra raptorg83@gmail.com schrieb am Fr., 14. Aug. 2015 21:19:

Hi guys.

A quick one. I have hundreds of servers being monitored by check_mk.

I’m seeing that perfdata directory is getting very big.

/opt/omd/sites/prod/var/pnp4nagios/perfdata

I need performance data, of course.

My question.

I have no problem (for now) to increase the disk, but… is there a internal functionality on check_mk (wato) I need to activate, that rotates this data? Set
to… retain X months of data? Something like that?

I see “Diskspace cleanup” on global settings. Is that?

Thank you very much!

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.

October 18th-20th, 2015

http://mathias-kettner.com/conference

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.

October 18th-20th, 2015

http://mathias-kettner.com/conference

Diese E-Mail kann vertrauliche und/oder rechtlich geschützte Informationen enthalten. Wenn Sie nicht der richtige Adressat sind, so bitten wir Sie, sofort den Absender zu informieren und diese E-Mail vollständig zu löschen.
Das unerlaubte Kopieren, Weiterleiten, Verbreiten oder Verwenden dieser E-Mail und deren Inhalte ist nicht gestattet.

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.

October 18th-20th, 2015

http://mathias-kettner.com/conference

10

Like Marcel said, you have to do it at your own.

And nope, Check_MK/OMD does not remove the xml/rrd files when you delete a host (found some test hosts from a year ago yesterday).

Mit freundlichen Grüßen

i.A. Jakub Tusz

image001.jpg

···

Telefon:
Fax:
E-Mail:

+49(721)6107 5905
+49(721)6107 5909
jakub.tusz@vbk.karlsruhe.de

Kfm. Geschäftsführer: Dr. Alexander Pischon, Techn. Geschäftsführer: Ascan Egerer

Vorsitzender des Aufsichtsrates: Oberbürgermeister Dr. Frank Mentrup

Amtsgericht Mannheim HRB 107847

Von: Marcel Schulte [mailto:schulte.marcel@gmail.com]
Gesendet: Dienstag, 18. August 2015 21:48
An: Henry Huang; Tusz, Jakub
Cc: checkmk-en@lists.mathias-kettner.de
Betreff: Re: [Check_mk (english)] Perfdata cleanup

Hi Henry,

absolutely, you’ll have to do that on your own. Okay, maybe the host’s whole perfdata folder could get removed when host is deleted in WATO or via webapi…

But what about old/left RRD and/or XML files? When should they get deleted?

Services get created when inventoried and they create RRD/XML, too, if not available. Service deletion is done by reinventory. Reinventory means “delete all (inventoried) services, then do an inventory” - should all perfdata get deleted
because their services get removed?

It’s a difficult (and maybe dangerous) decision - so so users should do that themselves.

Regards,

Marcel

Henry Huang red.rain.seven@gmail.com schrieb am Di., 18. Aug. 2015 um 18:36 Uhr:

Are you guys saying that the perf data will never get cleared by OMD or check_mk on its own if the host got removed?

On Tue, Aug 18, 2015 at 3:55 AM, Tusz, Jakub Jakub.Tusz@kvv.karlsruhe.de wrote:

Hi Marcel,

i like your script, but you can achieve very similar results with a bash one liner:

for x in $(find $OMD_ROOT/var/pnp4nagios/perfdata –mtime +30); do rm -rf „$x“; done # this can be dangerous!

Ok, it can’t actively look for vanished hosts, but they will disappear after 30 days of inactivity of their folder.

Not perfect, but it was done quick and dirty.

Mit freundlichen Grüßen

i.A. Jakub Tusz

Jakub Tusz

Abt.: V5-IT - Verwaltung | Informationstechnologie

VBK - Verkehrsbetriebe Karlsruhe GmbH

Tullastraße 71, D-76131 Karlsruhe

Postfach 1140, D-76001 Karlsruhe

Telefon:

Fax:

E-Mail:

+49(721)6107 5905

+49(721)6107 5909

jakub.tusz@vbk.karlsruhe.de

Kfm. Geschäftsführer: Dr. Alexander Pischon, Techn. Geschäftsführer: Ascan Egerer

Vorsitzender des Aufsichtsrates: Oberbürgermeister Dr. Frank Mentrup

Amtsgericht Mannheim HRB 107847

Von:
checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
Im Auftrag von Marcel Schulte
Gesendet: Montag, 17. August 2015 16:26
An: checkmk-en@lists.mathias-kettner.de;
omd-users@lists.mathias-kettner.de
Betreff: Re: [Check_mk (english)] Perfdata cleanup

Hi guys (and girls, if any),

I finished my cronjob script to cleanup perfdata files.

The script is intended to be run as OMD siteuser - but will complain about which variables to tweak if it’s not :wink:

If you’re not using CMC (“Check_MK Micro Core”) you must set line 48 to False

Here’s the usage:

$cleanup_perfdata.py -h

usage: cleanup_perfdata.py [-h] [-x XDAYS] [-n] [-N] [-v] [-V]

cleanup_perfdata: remove perfdata files not needed anymore

optional arguments:

-h, --help show this help message and exit

-x XDAYS, --xdays XDAYS

set fileage in days [default: 90]

-n, --dryrun do not delete anything, just report [default: False]

-N ‘dryrun’ but non-verbose, just stats [default: False]

-v, --verbose set verbosity level [default: False]

-V, --version show program’s version number and exit

Dryrun (“-n|–dryrun”) implies “-v|–verbose”.

Try different XDAYS with dryrun to get a clue what’s going on :wink:

Comments are very much appreciated.

https://github.com/dasmarci/check_mk/blob/master/scripts/cleanup_perfdata.py

Regards,

Marcel

Jeffrey J. Gerschkow jgerschk@comcast.net schrieb am Sa., 15. Aug. 2015 um 14:37 Uhr:

What I have used and modify is this.

https://github.com/lingej/pnp4nagios/blob/master/scripts/check_pnp_rrds.pl.in

Might be worth something to some.

Not only can you add this as a check (classic Nagios/icinga), but there is a command line
options –deletexml –deleterrd that can be used to help in cleanup. The script itself, when using those you have to hit Y to delete for each, but easily modified if you want to add it to a cron job to delete.

Just another suggestion for aiding in cleanup.

From:
checkmk-en-bounces@lists.mathias-kettner.de [mailto:checkmk-en-bounces@lists.mathias-kettner.de]
On Behalf Of paulmonitoring
Sent: Friday, August 14, 2015 5:45 PM
To: Marcel Schulte schulte.marcel@gmail.com; Andreas Döhler andreas.doehler@gmail.com; Gerardo Ferreyra <raptorg83@gmail.com >;
checkmk-en@lists.mathias-kettner.de checkmk-en@lists.mathias-kettner.de;
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

I’m interested.

-------- Original message --------

From: Marcel Schulte schulte.marcel@gmail.com

Date: 08/14/2015 16:38 (GMT-05:00)

To: paulmonitoring paulmonitoring@gmail.com, Andreas Döhler andreas.doehler@gmail.com, Gerardo Ferreyra <raptorg83@gmail.com >,
“checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Hi,

I’m currently working on a weekly cronjob doing the cleanup for me.

What’s already working:

  • Find data of deleted hosts and remove the whole host folder from perfect.

  • Find all remaining files with modification time older than a configurable amount of days (defaults to 90) and remove them.

The latter will be rewritten to only remove the file if it’s known not to be needed anymore. I think this will be usable mid of next week, there’s more important work to do first :wink:

Whole thing is written in python and works inside OMD sites without modifications. For non-OMD installations a single variable must be changed.

…I could post the mentioned working part at Monday if anyone is interested.

Best regards,

Marcel

paulmonitoring paulmonitoring@gmail.com schrieb am Fr., 14. Aug. 2015 22:24:

I echo what Andreas says. Be cautious. In the past I may or may not have deleted a lot of data by accident from the perfdata dir. :wink:

Sent via the Samsung GALAXY S® 5, an AT&T 4G LTE smartphone

-------- Original message --------

From: Andreas Döhler andreas.doehler@gmail.com

Date: 08/14/2015 15:32 (GMT-05:00)

To: Gerardo Ferreyra raptorg83@gmail.com, “checkmk-en@lists.mathias-kettner.de” checkmk-en@lists.mathias-kettner.de,
omd-users@lists.mathias-kettner.de

Subject: Re: [Check_mk (english)] Perfdata cleanup

Yes and no.

Perfdata directory will only get bigger with new hosts in the moment you create one host the total space is allocated. What I do on my systems is from time to time a find with older then 30-60 days on this directory. With this I remove obsolete perfdata from
deleted machines on non existent checks.

But pay attention what you delete there :slight_smile:

Best regards

Andreas

Gerardo Ferreyra raptorg83@gmail.com schrieb am Fr., 14. Aug. 2015 21:19:

Hi guys.

A quick one. I have hundreds of servers being monitored by check_mk.

I’m seeing that perfdata directory is getting very big.

/opt/omd/sites/prod/var/pnp4nagios/perfdata

I need performance data, of course.

My question.

I have no problem (for now) to increase the disk, but… is there a internal functionality on check_mk (wato) I need to activate, that rotates this data? Set
to… retain X months of data? Something like that?

I see “Diskspace cleanup” on global settings. Is that?

Thank you very much!

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.

October 18th-20th, 2015

http://mathias-kettner.com/conference

checkmk-en mailing list

checkmk-en@lists.mathias-kettner.de

http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!

Book your place now and be part of it.

October 18th-20th, 2015

http://mathias-kettner.com/conference

Diese E-Mail kann vertrauliche und/oder rechtlich geschützte Informationen enthalten. Wenn Sie nicht der richtige Adressat sind, so bitten wir Sie, sofort den Absender
zu informieren und diese E-Mail vollständig zu löschen. Das unerlaubte Kopieren, Weiterleiten, Verbreiten oder Verwenden dieser E-Mail und deren Inhalte ist nicht gestattet.

checkmk-en mailing list
checkmk-en@lists.mathias-kettner.de
http://lists.mathias-kettner.de/mailman/listinfo/checkmk-en

We’ll meet in Munich for the 2nd Check_MK Conference!
Book your place now and be part of it.
October 18th-20th, 2015
http://mathias-kettner.com/conference