Find Image function on Linux works, doesn't, works, dies

Post here if you need help using Actiona
Post Reply
Beddhist
Posts: 9
Joined: 18 Apr 2021, 11:26

Find Image function on Linux works, doesn't, works, dies

Post by Beddhist »

Hi All,

I have an old Athlon PC here that used to run Windows (XP?) fine. I upgraded RAM to 8 GB (new) and installed Linux Mint. It had a lot of problems with the inbuilt Geforce 6150SE, so I bit the bullet and bought a GT 710 for it. It's now a fresh install of Mint 20.3 with the latest Nvidia graphics driver installed.

Its sole purpose in life now is to run a WebGL-based online game in Firefox, for which I have written a couple of Actiona scripts to save me from endless clicking. Everything works fine and stable, except the Find Image function. I have written a procedure that finds the image, puts the position into a variable, then calls another procedure, which clicks the target. When it works this works really well, as it does on another PC running Windows 10.

Seemingly at random the search fails, then works again and eventually Actiona dies, with a truncated core being dumped by systemd. Everything else continues working. Occasionally, instead of just Actiona dying the machine either suddenly reboots, or it freezes. The latter sometimes puts the NIC into some kind of spamming mode, which takes down the entire LAN. Pressing reset sometimes results in a corrupted POST display with the PC freezing at that point, requiring a power off/on.

Suspecting a problem with the graphics card or driver, I have tried 3 older versions - no change. I also tried the open source Xorg-nouveau driver that is installed by Mint as default. With this the image search worked every time, but after an hour or two the PC crashed/froze/rebooted. Given the fact that using the nouveau driver caused a change my hunch is that there is indeed a driver problem, or a hardware problem, both of which I cannot begin to debug, since I don't have any other hardware to try. I'm also taking this to the Mint forum. Perhaps there is a setting in the bios that may help, but I'm not hopeful.

Thanks for your time,
Peter.
eureka
Posts: 204
Joined: 08 Mar 2016, 22:18

Re: Find Image function on Linux works, doesn't, works, dies

Post by eureka »

For such an issue my troubleshooting approach would be to decouple Actiona from some tasks which might alternatively be run through another route (at least to eliminate Actiona during troubleshooting).

That is, use some other means to find Image through running a Command in Actiona (cutting out Find Image object).

Or write a secondary Actiona script which is only a small testrig to Find image and wrap a lot of diagnostics in the script to see if/where it breaks from continuous play (Loop).

Basically, reshuffle the cards.
Beddhist
Posts: 9
Joined: 18 Apr 2021, 11:26

Re: Find Image function on Linux works, doesn't, works, dies

Post by Beddhist »

I already know that it is the find image function that dies. It is indicated in the status box while the core dump occurs. The only other thing I can say is that it only happens when it actually finds the target image. If the target doesn't exist it will not fail. If I disable the find function the script will run literally all day.

Debugging the function itself is way beyond my level of (non-)expertise. ;-)

PS: I have two procedures that search for different images and both fail.
eureka
Posts: 204
Joined: 08 Mar 2016, 22:18

Re: Find Image function on Linux works, doesn't, works, dies

Post by eureka »

Perhaps if you just post a test script with only a single Action - Find Image - with parameters
which you have set - it can be inspected here. By posting the single Action (FindImage.ascr) the internal parameters can be inspected, whereas screenshot does not give that detail. My debugging thoughts are to place FindImage.ascr into a wrapper which dumps logs using built in Exceptions tab settings.
Beddhist
Posts: 9
Joined: 18 Apr 2021, 11:26

Re: Find Image function on Linux works, doesn't, works, dies

Post by Beddhist »

OK, here we go:

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>
<scriptfile>
    <settings program="actiona" version="3.10.1" scriptVersion="1.1.0" os="GNU/Linux"/>
    <actions>
        <action name="ActionBeginProcedure" version="1.0.0"/>
        <action name="ActionNoop" version="1.0.0"/>
        <action name="ActionClick" version="1.0.0"/>
        <action name="ActionVariable" version="1.0.0"/>
        <action name="ActionEndProcedure" version="1.0.0"/>
        <action name="ActionFindImage" version="1.1.0"/>
    </actions>
    <parameters/>
    <resources/>
    <script pauseBefore="0" pauseAfter="0">
        <action name="ActionVariable">
            <exception id="0" action="0" line=""/>
            <exception id="1" action="0" line=""/>
            <exception id="2" action="1" line=""/>
            <exception id="32" action="0" line=""/>
            <parameter name="colorValue">
                <subParameter name="value" code="0"></subParameter>
            </parameter>
            <parameter name="positionValue">
                <subParameter name="unit" code="0">0</subParameter>
                <subParameter name="value" code="0">642:669</subParameter>
            </parameter>
            <parameter name="type">
                <subParameter name="value" code="0">position</subParameter>
            </parameter>
            <parameter name="value">
                <subParameter name="value" code="0"></subParameter>
            </parameter>
            <parameter name="variable">
                <subParameter name="value" code="0">CollectChestButtonCoord</subParameter>
            </parameter>
        </action>
        <action name="ActionBeginProcedure" label="FindChest">
            <exception id="0" action="0" line=""/>
            <exception id="1" action="0" line=""/>
            <exception id="2" action="1" line=""/>
            <parameter name="name">
                <subParameter name="value" code="0">FindChest</subParameter>
            </parameter>
        </action>
        <action name="ActionFindImage" timeout="500">
            <exception id="0" action="0" line=""/>
            <exception id="1" action="0" line=""/>
            <exception id="2" action="1" line=""/>
            <exception id="32" action="0" line=""/>
            <parameter name="confidence">
                <subParameter name="value" code="0"></subParameter>
            </parameter>
            <parameter name="confidenceMinimum">
                <subParameter name="value" code="0">70</subParameter>
            </parameter>
            <parameter name="downPyramidCount">
                <subParameter name="value" code="0">1</subParameter>
            </parameter>
            <parameter name="ifFound">
                <subParameter name="action" code="0">call_procedure</subParameter>
                <subParameter name="line" code="0">OpenChest</subParameter>
            </parameter>
            <parameter name="ifNotFound">
                <subParameter name="action" code="0">wait</subParameter>
                <subParameter name="line" code="0"></subParameter>
            </parameter>
            <parameter name="imageToFind">
                <subParameter name="value" code="0">/home/peter/ts-scripts/chest.png</subParameter>
            </parameter>
            <parameter name="imageToSearchIn">
                <subParameter name="value" code="0"></subParameter>
            </parameter>
            <parameter name="maximumMatches">
                <subParameter name="value" code="0">1</subParameter>
            </parameter>
            <parameter name="method">
                <subParameter name="value" code="0">correlationcoefficient</subParameter>
            </parameter>
            <parameter name="position">
                <subParameter name="value" code="0">ChestCoord</subParameter>
            </parameter>
            <parameter name="searchDelay">
                <subParameter name="value" code="0">100</subParameter>
            </parameter>
            <parameter name="searchExpansion">
                <subParameter name="value" code="0">15</subParameter>
            </parameter>
            <parameter name="source">
                <subParameter name="value" code="0">window</subParameter>
            </parameter>
            <parameter name="windowName">
                <subParameter name="value" code="0">TrainStation - Pixel Federation Games — Mozilla Firefox</subParameter>
            </parameter>
            <parameter name="windowRelativePosition">
                <subParameter name="value" code="0">false</subParameter>
            </parameter>
        </action>
        <action name="ActionEndProcedure">
            <exception id="0" action="0" line=""/>
            <exception id="1" action="0" line=""/>
            <exception id="2" action="1" line=""/>
        </action>
        <action name="ActionNoop" enabled="false">
            <exception id="0" action="0" line=""/>
            <exception id="1" action="0" line=""/>
            <exception id="2" action="1" line=""/>
        </action>
        <action name="ActionBeginProcedure" label="OpenChest">
            <exception id="0" action="0" line=""/>
            <exception id="1" action="0" line=""/>
            <exception id="2" action="1" line=""/>
            <parameter name="name">
                <subParameter name="value" code="0">OpenChest</subParameter>
            </parameter>
        </action>
        <action name="ActionClick">
            <exception id="0" action="0" line=""/>
            <exception id="1" action="0" line=""/>
            <exception id="2" action="1" line=""/>
            <exception id="32" action="0" line=""/>
            <exception id="33" action="0" line=""/>
            <parameter name="action">
                <subParameter name="value" code="0">pressRelease</subParameter>
            </parameter>
            <parameter name="amount">
                <subParameter name="value" code="0">1</subParameter>
            </parameter>
            <parameter name="button">
                <subParameter name="value" code="0">left</subParameter>
            </parameter>
            <parameter name="position">
                <subParameter name="unit" code="0">0</subParameter>
                <subParameter name="value" code="1">ChestCoord</subParameter>
            </parameter>
            <parameter name="positionOffset">
                <subParameter name="unit" code="0">0</subParameter>
                <subParameter name="value" code="0"></subParameter>
            </parameter>
            <parameter name="restoreCursorPosition">
                <subParameter name="value" code="0">false</subParameter>
            </parameter>
        </action>
        <action name="ActionClick">
            <exception id="0" action="0" line=""/>
            <exception id="1" action="0" line=""/>
            <exception id="2" action="1" line=""/>
            <exception id="32" action="0" line=""/>
            <exception id="33" action="0" line=""/>
            <parameter name="action">
                <subParameter name="value" code="0">pressRelease</subParameter>
            </parameter>
            <parameter name="amount">
                <subParameter name="value" code="0">1</subParameter>
            </parameter>
            <parameter name="button">
                <subParameter name="value" code="0">left</subParameter>
            </parameter>
            <parameter name="position">
                <subParameter name="unit" code="0">0</subParameter>
                <subParameter name="value" code="1">CollectChestButtonCoord</subParameter>
            </parameter>
            <parameter name="positionOffset">
                <subParameter name="unit" code="0">0</subParameter>
                <subParameter name="value" code="0"></subParameter>
            </parameter>
            <parameter name="restoreCursorPosition">
                <subParameter name="value" code="0">false</subParameter>
            </parameter>
        </action>
        <action name="ActionEndProcedure">
            <exception id="0" action="0" line=""/>
            <exception id="1" action="0" line=""/>
            <exception id="2" action="1" line=""/>
        </action>
    </script>
</scriptfile>
This should be fully functional, except for the missing chest.png file.

The script has just run for a few hours with the proc enabled. It came to a halt when FF had gobbled up 4GB RAM and "Web content" another 11GB. The PC only has 8GB. There seems to be a memory leak either in FF or in the game code. Happens on Windows, too.
eureka
Posts: 204
Joined: 08 Mar 2016, 22:18

Re: Find Image function on Linux works, doesn't, works, dies

Post by eureka »

I inspected the script as an XML file using XMLCopyEditor and it seems valid .. if you temporarily snip the top declaration just to validate XML .. It is needed to run in Actiona.

<?xml version="1.0" encoding="UTF-8"?>

My only suggestion is that you leverage the Exceptions tab and GotoLine for each of the 4 exceptions (or any of the exceptions). Then you can log all exceptions in Console or into a log file.
Beddhist
Posts: 9
Joined: 18 Apr 2021, 11:26

Re: Find Image function on Linux works, doesn't, works, dies

Post by Beddhist »

Thanks Eureka, I'll try that at the next opportunity.

The more I play with it and think about it the more I believe this may have something to do with the memory leak. On the
Windows PC I've had Actiona disappear on me a few times - always when the machine got too busy swapping itself to death.
Post Reply