How to direct key events to a specific window?

Post here if you need help using Actiona
Post Reply
ptsl
Posts: 5
Joined: 26 Sep 2019, 10:54

How to direct key events to a specific window?

Post by ptsl » 26 Sep 2019, 10:58

If I start my application without using Actiona, the new application gets keyboard/mouse focus. However, if I run devilspie2 which matches the application window a runs a script using actexec, the application loses focus and the keys "typed" by Actiona go to the console instead.

Is it possible to direct key events generated by Actiona to a specific window?

eureka
Posts: 84
Joined: 08 Mar 2016, 22:18

Re: How to direct key events to a specific window?

Post by eureka » 27 Sep 2019, 08:35

It is not clear to me where devilspie2 fits into your workflow. It offers some similar features to Actiona. Why is it needed?
Can you not use the Window action in Actiona to focus on a window?

ptsl
Posts: 5
Joined: 26 Sep 2019, 10:54

Re: How to direct key events to a specific window?

Post by ptsl » 27 Sep 2019, 11:59

As a developer, I start the same application over and over again. And every time I need to open the same file. This is the series of steps I wanted to automate:
* Whenever a new window titled "My Application" is open, click on menu -> File -> Open -> select a file.

This is how I'm doing it:
* I'm using devilspie2 to match the "My Application" window title and launch actexec if a match is found.
* actexec runs a script which sends Alt+F (File) O (Open) and a sequence of Down keys to select the correct file.

I know this workflow is awkward (maybe it's a remnant from the times I used xdotool, a very unpractical automation tool).

Is there a way to replace devilspie2? (Is Actiona able to match window titles?)
Is there a way to manipulate menus and dialogs directly without having to send the keyboard shortcuts?
Unfortunately, I had a (quick) look at Actiona actions and didn't a find a suitable-looking action.

eureka
Posts: 84
Joined: 08 Mar 2016, 22:18

Re: How to direct key events to a specific window?

Post by eureka » 28 Sep 2019, 13:43

I too have used devilspie2 and xdotool. In fact you can run a command in Actiona to run an xdotool command if you wish. This sometimes reduces the number of key commands.

Q. Is there a way to replace devilspie2? (Is Actiona able to match window titles?)

Yes. Actiona can match window titles.
Add a Window object to your action list in gui.
Input parameters include "window title".
In fact click on the dropdown arrow to the right of the window title field and you see a list of all open window titles.
You can also use wild card * in title string.

Another relevant object is Window Condition.


Q. Is there a way to manipulate menus and dialogs directly without having to send the keyboard shortcuts?

This depends on the target menu or dialog. You can choose for targetting either
- keyboard sequences using Key object
- absolute mouse coordinates using Move Cursor object
- images (captured in screenshot) using Find Image object
- or even command line using Command object (if the target can be controlled via CLI). For example some apps can open a file via command line if you place the filepath as an argument to the command. This works in Atom editor.

You can get mouse coordinates of target by using Move Cursor object, click on the + sign and move the cursor to the point in your target menu or dialogue.

It is safer to first maximise your target windows to ensure consistency of targetting by coordinates. And this approach may not work if you send your script to other users perhaps with different environments.

[Later thought]
I will add a note here on workflow. Your workflow assumes that you use Actiona as your controller of your workflow. What I have realised from my own experiments in development is that Actiona scripts can be launched from applications by adding Actiona as a script or plugin. That is, turning the workflow around on its head so that Actiona scripts can be run as plugins or helper scripts. I use Atom editor extensively for development and I have added a custom button in Atom which launches an Actiona script. This means that I can pass arguments such as filepaths. To conclude think of Actiona as part of a toolchain. Similarly I use Krusader as my dual pane File Manager, which links to Atom and I can launch Actiona as a helper script. Many applications can run in CLI mode. Yet another approach is to add a custom URL Protocol Handler which launches Actiona through hyperclicks in your browser.

ptsl
Posts: 5
Joined: 26 Sep 2019, 10:54

Re: How to direct key events to a specific window?

Post by ptsl » 28 Sep 2019, 15:53

eureka - Your answer is thorough and exhausting and I really appreciate it.

I am glad that Actiona can be used as the workflow "master", this is much easier to do than my old devilspie2+xdotool / Actiona combination.

But I still didn't figure out how to use it. I found the Window action and configure it to match for "My Application" window title, wait 1 s and minimize the window. This is just my test. But it seems that the script gets executed only once - when I press the Execute button.

Of course I can do a neverending loop using Goto - but than the GUI seems to be executing very frantically - with the progress bar moving from 0 to 100 % and starting over again and again. And I don't know how to react to different windows with different actions. (Maybe using multiple if's and having them tested in a loop again?)

This is a contrast with devilspie2, which sits quietly in the background and (I believe) it works in a trigger-action mode. This is what puzzles me in Actiona - I can't see a separation between triggers and actions.

Did I get anything wrong?

eureka
Posts: 84
Joined: 08 Mar 2016, 22:18

Re: How to direct key events to a specific window?

Post by eureka » 28 Sep 2019, 17:00

But it seems that the script gets executed only once - when I press the Execute button.
If the test script only minimises a window then the basic script is finished. Job done. No need to keep active or looping looking for an event. i rarely use loops since each execution of a script performs part of a workflow. In fact I have on occasions used a "master" Actiona script which runs multiple *.ascr scripts in hierarchical sequence using actexec myscript.ascr in a list of command objects. But Actiona cannot run dual sessions and so I use a hack to clone actexec binary as actexec > actexec2. Not documented.

It would help if you describe in text a typical workflow you wish to automate. Reading again the title of your post it seems that you want to use key events to drive your workflow. Remember that the purpose of Actiona is to emulate key strokes not to react to them. Could you use devilspie2 as front end watcher to trigger actexec to execute some actions on a key event?

If you add Internal > Console object after each object you can dump variables to console which shows in right

Consider writing some frequently used scripts as procedures. Begin, end, call procedure.

ptsl
Posts: 5
Joined: 26 Sep 2019, 10:54

Re: How to direct key events to a specific window?

Post by ptsl » 28 Sep 2019, 17:18

This is how my workflow used with devilspie2 + xdotool. As I said before, it was organized in a trigger-action fashion.

The application I'm working on is basically a form editor. Open a form, fill in some fields, generate the document for the customer.

1.) Trigger: the application has been launched (the main window has appeared on the screen). Action: click Menu -> File -> Open.
2.) Trigger: the Open Form window has appeared. Action: select the customer.
3.) Trigger: the form window has been opened. Action: populate some fields.

Now it occurres to me that maybe with Actiona I should not have been waiting for the application to be launched, but rather to launch the application *from within* Actiona. Then to have Actiona click on the menu and insert some pauses to wait for the Open Dialog to appear. Seems that I was too confined in the mindset set by devilspie2's mode of operation.

Again, thank you for your time and for replying to my questions!

eureka
Posts: 84
Joined: 08 Mar 2016, 22:18

Re: How to direct key events to a specific window?

Post by eureka » 28 Sep 2019, 17:41

Actiona will serve that requirement. To add to your form entry automation experiments you might try sikuli.

http://slides.sikuli.org/examples/driver/mailchimp/

ptsl
Posts: 5
Joined: 26 Sep 2019, 10:54

Re: How to direct key events to a specific window?

Post by ptsl » 30 Sep 2019, 09:56

Thank you for the suggestion! Definitely will have a look!

Post Reply