RobotMK: Application Launch with ImageHorizon

rolandg · January 30, 2023, 2:05pm

Hi,
I have a very specific question regarding RobotMK.
Maybe @simonm or someone else can help me here.

I have a virtual machine running with Debian and Gnome to make all my robotmk tests.
Gnome is actually not needed during normal operations since all tests run headless, but it’s very convinient to have a desktop when writing new tests to actually see what happens.

Today I wrote a new test for checking if a RDP login works.
So I start up remmina, press tab, down and enter to launch the first (and only) item in the list.
Then I wait for the Google Chrome Icon to show up, which sits on the desktop.
As soon as it’s there within 50 seconds, the app will be closed and a success is reported.

This works very good when I start it in gnome, but for robotmk it must run headless.
So, this is my crontab:
*/10 * * * * /usr/bin/xvfb-run /usr/bin/robotmk-runner.py

It uses xvfb-run to open a virtual x11 session to make that happen. Unfortunately, this does not work for a robot file like this:

*** Settings ***
Library        ImageHorizonLibrary
Test Setup     Open Remmina
Test Teardown  Close Remmina

*** Keywords ***
Open Remmina
    # Launch Application  /usr/bin/remmina  alias=remmina
    Press Combination  key.win  key.a
    Sleep   1s

    Press Combination  key.r
    Press Combination  key.e
    Press Combination  key.m
    Press Combination  key.enter
    Sleep   1s

Close Remmina
    # Terminate Application  remmina
    Press Combination  key.alt  key.f4
    Sleep   2s
    Press Combination  key.alt  key.f4

*** Test Cases ***
RDP-Test DC1 with Horizon and Remmina

    Set Reference Folder    /usr/lib/check_mk_agent/refimages
    Set Screenshot Folder   /usr/lib/check_mk_agent/screenshots
    Set Confidence          0.9

    Press Combination  key.tab
    Press Combination  key.down
    Press Combination  key.enter

    Wait For         chrome  timeout=50

Since I do not see what’s happening inside the virtual x session, I don’t know what’s going on. The failure screenshot is completely black.
I also tried the Launch/Terminate Application keywords. They launch remmina, but there is a plugin called glibsecret which keeps encrypted passwords I guess, and this plugin fails to start, when I use these keywords.

Any thoughts on this one?
Is there a better way to implement an End-to-end-RDP-Test with a debian VM?

Thanks for reading,
Roland.

simonm · January 30, 2023, 2:24pm

Hi Roland,

a black screenshot is an indication that there is no “real” desktop visible for the library at all.
(I would have been surprised if it had worked out of the box.)

I found an issue where someone hacked some lines into the imagehorizonlbrary to get XVFB working:

github.com/eficode/robotframework-imagehorizonlibrary

Is it possible to add headless run ?

opened 11:39AM - 22 Jun 17 UTC

closed 12:55PM - 05 Jul 17 UTC

david-guenault

Well i know it is technically possible with some workaround. I just hacked the l…ibrary to add test running in xvfb. This is a quick and (realy) dirty implementation. _**modify the __init__.py at the root library folder**_ ` ... # required modules import os import Xlib.display from pyvirtualdisplay.smartdisplay import SmartDisplay # new method to add running test in xvfb def start_xvfb(self, width=1920, height=1080, depth=24): '''Allow to execute tests in a virtual display (Xvfb)''' LOGGER.info("STARTING VIRTUAL DISPLAY : %sx%sx%s " % (width, height, depth)) display = SmartDisplay(visible=0, size=(width, height), color_depth=depth) display.start() SmartDisplay() # this is the tricky part ! ag._pyautogui_x11._display = Xlib.display.Display(os.environ["DISPLAY"]) LOGGER.info("DISPLAY IS : %s " % os.environ["DISPLAY"]) .... ` All of this is just a copy/paste implementation from this post : http://fredtantini.free.fr/blog/index.php?article58/automatiser-des-actions-avec-selenium-pyautogui-et-xvfb Of course this is just a proof of concept that just work (except you have to take care of windows decoration) and just for Gnu/linux distro.

See also this:

(I had to use google translate )

Since I am planning a fork of the imagehorizonlibrary anyway (to finally include edge detection as a feature), XVFB support would be another interesting feature.

Try to see if this hack would work for you in general. We can then talk about whether this could be included in the new autogui-library.

simonm · January 30, 2023, 2:54pm

A hint to your code example (may be interesting for others…)

You can omit the keyword set_reference_folder by specifying the folder already during import:

Library ImageHorizonLibrary reference_folder=${CURDIR}${/}images

CURDIR is a robot internal variable that points to the current directory where the robot file is located. (I would prefer to have the images as well be within the robot folder under version control)
The slash in brackets - yes I know, looks stupid But this makes your test OS independent.

Apart from that: well done; Starting and closing the application under test is best done in the setup/teardown phase of the suite or test.

rolandg · January 31, 2023, 1:56pm

Hi,
thanks for your research, but I didn’t get this hack to run.
I didn’t find out which file I should edit, but I guess it wouldn’t be update persistent anyway. So, if another colleague runs an update on the machine, the checks won’t work anymore.

However, there is also a web application, so I used that instead. However the users normally use a RDP-Tool to log in, so a real e2e-test should try this method - so I will be very happy when XVFB support will join the toolbox.

I have another question about the robotmk-runner.py
The checks for this customer run every 10 minutes. Sometimes a single suite fails and sometimes this is a false positve. My colleagues from the support team would like to be able to rerun a single suite and update the state file to check this specific service again.
Is this possible?

Cheers,
Roland.

simonm · January 31, 2023, 2:19pm

Of course, this would not be update safe. Just for testing.
If you want, you can open the first feature request on https://github.com/MarketSquare/robotframework-autogui/issues

Yes, it is. Just go into the bakery rule where you can define the suites to run.
There is a section “re-execution of failed tests”. This is especially for flaky tests which sometimes “somehow” fail. After the first suite execution, Robotmk executes the failed tests as often as specified and in the end merges tha “last and best” result of each test into a new result file.
See also here:

Regards,
Simon