"live_example.py" for Using LQL in Python get stucked

[jorgejuan]

CMK version: Checkmk Enterprise Edition 2.1.0p6
OS version: Red Hat Enterprise Linux 8.6 (Ootpa)

**Error message: The “live_example.py” for Using LQL in Python that Check_mk uploaded in the check_mk docs get stucked in the ‘while’ loop and never ends.

This is where the code get stucked:

# receive the reply as a JSON string
chunks = []
while len(chunks) == 0 or chunks[-1] != "":
    chunks.append(sock.recv(4096))
sock.close()
reply = "".join(chunks)

Please, Any help about what is going wrong? Could someone share a running example?

BR.

[thl-cmk]

I guess you mean this sample script in the official Checkmk User Guide. I played a bit with it myself and I think there are some issues:

the sock.sendall expects byte encoded data not a string

sock.sendall("GET status\nOutputFormat: json\n")

so this should be changed to

sock.sendall("GET status\nOutputFormat: json\n".encode())

than you ned the decode the recived data from bytes into str. Change

chunks.append(sock.recv(4096))

to

chunks.append(sock.recv(4096).decode())

the last issue I run into is the default blocking state of the socket (see [ python: loop with socket.recv()](python: loop with socket.recv() - Stack Overflow)

After I changed the socket type from blocking to non blocking (sock.setblocking(False)) and added a sleep timer (sleep(5)). I was finaly able to get the script working.

sock = socket.socket(family, socket.SOCK_STREAM)
sock.connect(address)
sock.setblocking(False)
sock.sendall(str.encode("GET status\nOutputFormat: json\n"))
sock.shutdown(socket.SHUT_WR)
sleep(5)

The issue with this might be the sleep timer, as it depends on how long your querry takes to complete.

@mschlenker Can you take a look at the sample LQL script in the documentation? It may need some polishing

[mschlenker (Mattias Schlenker)]

Thanks. Bookmarked the thread for my collection for late Q3 when “11. Automation and development” is on the agenda. (Still might have a look earlier)

[jorgejuan]

Hi,

Thank you so much for your attention.

I added your suggestions but still failing. Sorry, I have poor knowledge about sockets.

Here is the error:

socket.error: [Errno 11] Resource temporarily unavailable

…and here is the code:

#!/usr/bin/env python
# Sample program for accessing Livestatus from Python

import json, os, socket
import time

# for local site only: file path to socket
address = "%s/tmp/run/live" % os.getenv("OMD_ROOT")
# for local/remote sites: TCP address/port for Livestatus socket
# address = ("localhost", 6557)

# connect to Livestatus
family = socket.AF_INET if type(address) == tuple else socket.AF_UNIX
sock = socket.socket(family, socket.SOCK_STREAM)
sock.connect(address)
sock.setblocking(False)
time.sleep(5)

# send our request. sock.sendall expects byte encoded data
sock.sendall("GET status\nOutputFormat: json\n".encode())
sock.shutdown(socket.SHUT_WR)

# you ned the decode the recived data from bytes into str, and convert as a JSON
chunks = []
while len(chunks) == 0 or chunks[-1] != "":
    chunks.append(sock.recv(4096).decode())
sock.close()
reply = "".join(chunks)

# print the parsed reply
print(json.loads(reply))

Forums say that is because of the “non-blocking” state.

Anyway, This is so frustrating and complicated. I tried differenta approach using “lq” and os.popen() and it worked

   command = "lq \"GET comments\nColumns: service_comments_with_info\nFilter: host_name = %s\nFilter: service_description = %s\" | awk 'NR==1'" % \
                (hostname, servicedesc)
               
            output = os.popen(command).read()  # nosec

BR.

[thl-cmk]

you need to move the time.sleep(5) behind the sock.shutdown(socket.SHUT_WR). And I asume you try this directly as siteuser on the CMK server.

Here my variant of the script:

#!/usr/bin/env python
# Sample program for accessing Livestatus from Python

import json
import os
import socket
from time import sleep
from pprint import pp


def get_lq_data(querry: str):
    address = "%s/tmp/run/live" % os.getenv("OMD_ROOT")
    family = socket.AF_INET if type(address) == tuple else socket.AF_UNIX
    sock = socket.socket(family, socket.SOCK_STREAM)
    sock.connect(address)
    sock.setblocking(False)
    sock.sendall(querry.encode())
    sock.shutdown(socket.SHUT_WR)
    sleep(1)
    chunks = []
    while len(chunks) == 0 or chunks[-1] != "":
        chunks.append(sock.recv(4096).decode())
    sock.close()
    print(f'# of chunks: {len(chunks)}')
    if len(chunks):
        reply = json.loads("".join(chunks))
        return reply
    else:
        return {}


pp(get_lq_data("GET hosts\nColumns: name address\nOutputFormat: json\nFilter: host_name ~ ro01.*\nColumnHeaders: on\n"))

returns:

[['name', 'address'], ['ro01-auth', '192.168.10.124'], ['ro01wlc', '192.168.10.142'], ['ro01', '192.168.10.124'], ['ro01-noauth', '192.168.10.124']]

[jorgejuan]

Hi,

It worked!!

Thank you so much for the help.

BR.

[sebkir]

This slipped through the cracks. Sorry about that.

The example is now fixed in the user guide as well. Please do check out the fixed example in the user guide, because it works without having to use sleep(n).

Retrieving status data via Livestatus

Thanks to @thl-cmk and @chauhan_sudhir.

[thl-cmk]

@sebkir

This slipped through the cracks. Sorry about that.

Better late than never THX for updating.

Your example works for me. I have remove the sock.setblocking(False) and sleep(1) from my scrpit. Works now without them to. No idea why it was needed before.