Custom graphs crash after upgrade from 2.2p34 to 2.3p17

Troubleshooting
1

CMK version: 2.3p17
OS version: RHEL 8.10

After upgrade from 2.2p34 to 2.3p17 i have encountered a GUI crash because of a problem with the custom graphs:

Exception	TypeError (('rrd', 'JAVA-1', 'GarbageColl, 'BootstrapHeap_GB', 'max'))
Traceback	  File "/omd/sites/MON/lib/python3/cmk/gui/wsgi/applications/checkmk.py", line 188, in _process_request
    resp = page_handler()
  File "/omd/sites/MON/lib/python3/cmk/gui/wsgi/applications/utils.py", line 108, in _call_auth
    func()
  File "/omd/sites/MON/lib/python3/cmk/gui/pages.py", line 187, in wrapper
    return hc().handle_page()
  File "/omd/sites/MON/lib/python3/cmk/gui/pages.py", line 51, in handle_page
    self.page()
  File "/omd/sites/MON/lib/python3/cmk/gui/pages.py", line 146, in <lambda>
    "page": lambda self: self._wrapped_callable[0](),
  File "/omd/sites/MON/lib/python3/cmk/gui/main.py", line 33, in page_index
    SidebarRenderer().show(title, content)
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/__init__.py", line 378, in show
    self._show_sidebar()
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/__init__.py", line 412, in _show_sidebar
    self._show_sidebar_head()
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/__init__.py", line 618, in _show_sidebar_head
    MainMenuRenderer().show()
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/main_menu.py", line 40, in show
    self._show_main_menu_content()
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/main_menu.py", line 44, in _show_main_menu_content
    for menu_item in self._get_main_menu_items():
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/main_menu.py", line 84, in _get_main_menu_items
    if not menu.topics():
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/_snapin/_views.py", line 163, in <lambda>
    topics=lambda: view_menu_topics(include_reports=True),
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/_snapin/_views.py", line 61, in view_menu_topics
    return make_topic_menu(view_menu_items(include_reports))
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/_snapin/_views.py", line 73, in view_menu_items
    for page in page_type.load().pages():
  File "/omd/sites/MON/lib/python3/cmk/gui/graphing/cee/_customgraphs.py", line 434, in load
    for k, v in super().load().instances_dict().items():
  File "/omd/sites/MON/lib/python3/cmk/gui/pagetypes.py", line 955, in load
    instances.add_instance((user_id, name), cls.deserialize(raw_page_dict))
  File "/omd/sites/MON/lib/python3/cmk/gui/graphing/cee/_customgraphs.py", line 403, in deserialize
    return cls(_deserialize_custom_graph_page_spec(page_dict))
  File "/omd/sites/MON/lib/python3/cmk/gui/graphing/cee/_customgraphs.py", line 339, in _deserialize_custom_graph_page_spec
    metrics=[_deserialize_custom_graph_metric_spec(m) for m in metrics],
  File "/omd/sites/MON/lib/python3/cmk/gui/graphing/cee/_customgraphs.py", line 267, in _deserialize_custom_graph_metric_spec
    expression=_deserialize_expression(metric["expression"]),
  File "/omd/sites/MON/lib/python3/cmk/gui/graphing/cee/_customgraphs.py", line 256, in _deserialize_expression
    raise TypeError(expression)
Local Variables	{'expression': ('rrd',
                'JAVA-1',
                'GarbageColl',
                'BootstrapHeap_GB',
                'max')}
Crash Type	gui
Time	2024-10-11 09:14:38
Operating System	Red Hat Enterprise Linux release 8.10 (Ootpa)
Checkmk Version	2.3.0p17
Edition	cee
Core	cmc
Python Version	3.12.3 (main, Sep 11 2024, 10:14:33) [GCC 13.2.0]
Python Module Paths	/omd/sites/MON/local/lib/python3
/omd/sites/MON/lib/python3/cloud
/omd/sites/MON/lib/python312.zip
/omd/sites/MON/lib/python3.12
/omd/sites/MON/lib/python3.12/lib-dynload
/omd/sites/MON/lib/python3.12/site-packages
/omd/sites/MON/lib/python3
Details
Page	index.py
Request Method	GET
HTTP Parameters	POST / GET Variables

Referer	 
Username	cmkadmin
User Agent	Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36 Edg/129.0.0.0
Mobile GUI	 
SSL	 
Language	en

Has anybody had a similar problem?

2

Hi, we have a longer chat about the topic here:

Updating to v2.3 - pagetypes failed - Troubleshooting - Checkmk Community

Best regards
Chris

1 Like
3

Hi,

We’ve found the following working solution: removing all occurrences of user_custom_graph.mk files from ~/var/check_mk/web folder.

It’s important, that the GUI crashes even if other users’ web folder contains this file!

So far it seems this issue only occurs if a custom graph was created on a much older version of CMK - if a custom graph is created under v2.2 then it doesn’t break the GUI after upgrading to v2.3, but we’ve had these files possibly since v1.5.

2 Likes
4

Hi, well I have also found that. But this is not a solution for me since we have really a lot of custom graphs.

5

Hi,

FYI: werk 17390 has addressed this bug.

I can confirm that the problem is not present anymore on cmk v2.3 p20.

1 Like
6

Unfortunately I still get the GUI crash with the following error:
TypeError ((‘rrd’, ‘SRV15’, ‘CPU utilization’, ‘user’, ‘max’))

Traceback

  File "/omd/sites/MON/lib/python3/cmk/gui/wsgi/applications/checkmk.py", line 188, in _process_request
    resp = page_handler()
  File "/omd/sites/MON/lib/python3/cmk/gui/wsgi/applications/utils.py", line 113, in _call_auth
    func()
  File "/omd/sites/MON/lib/python3/cmk/gui/pages.py", line 187, in wrapper
    return hc().handle_page()
  File "/omd/sites/MON/lib/python3/cmk/gui/pages.py", line 51, in handle_page
    self.page()
  File "/omd/sites/MON/lib/python3/cmk/gui/pages.py", line 146, in <lambda>
    "page": lambda self: self._wrapped_callable[0](),
  File "/omd/sites/MON/lib/python3/cmk/gui/main.py", line 33, in page_index
    SidebarRenderer().show(title, content)
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/__init__.py", line 383, in show
    self._show_sidebar()
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/__init__.py", line 417, in _show_sidebar
    self._show_sidebar_head()
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/__init__.py", line 627, in _show_sidebar_head
    MainMenuRenderer().show()
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/main_menu.py", line 41, in show
    self._show_main_menu_content()
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/main_menu.py", line 45, in _show_main_menu_content
    for menu_item in self._get_main_menu_items():
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/main_menu.py", line 85, in _get_main_menu_items
    if not menu.topics():
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/_snapin/_views.py", line 163, in <lambda>
    topics=lambda: view_menu_topics(include_reports=True),
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/_snapin/_views.py", line 61, in view_menu_topics
    return make_topic_menu(view_menu_items(include_reports))
  File "/omd/sites/MON/lib/python3/cmk/gui/sidebar/_snapin/_views.py", line 73, in view_menu_items
    for page in page_type.load().pages():
  File "/omd/sites/MON/lib/python3/cmk/gui/graphing/cee/_customgraphs.py", line 448, in load
    for k, v in super().load().instances_dict().items():
  File "/omd/sites/MON/lib/python3/cmk/gui/pagetypes.py", line 962, in load
    instances.add_instance((user_id, name), cls.deserialize(raw_page_dict))
  File "/omd/sites/MON/lib/python3/cmk/gui/graphing/cee/_customgraphs.py", line 417, in deserialize
    return cls(_deserialize_custom_graph_page_spec(page_dict))
  File "/omd/sites/MON/lib/python3/cmk/gui/graphing/cee/_customgraphs.py", line 353, in _deserialize_custom_graph_page_spec
    metrics=[_deserialize_custom_graph_metric_spec(m) for m in metrics],
  File "/omd/sites/MON/lib/python3/cmk/gui/graphing/cee/_customgraphs.py", line 279, in _deserialize_custom_graph_metric_spec
    expression=_deserialize_expression(metric["expression"]),
  File "/omd/sites/MON/lib/python3/cmk/gui/graphing/cee/_customgraphs.py", line 268, in _deserialize_expression
    raise TypeError(expression)

Local Variables	

{'expression': ('rrd', 'SRV15', 'CPU utilization', 'user', 'max')}
7

The support found in the user_custom_graphs file of a particular user the name of the device with space at the last character (difficult to understand how it occured however the first graphs were created already in version 1.6). When we removed that, the GUI survived after the upgrade.