Basically, it works on the fact that you take screenshots and make a script to automate your UI.
Check out the video to really get a grasp of what it can do:
http://groups.csail.mit.edu/uid/sikuli/
It's really limitless. Today I used it to open up a bunch of putty tabs and sign me into our development server. It's so easy to use even a cave man can do it that even non-programmers could get the hang of it.
Maybe I can get it to automate some things in nethack for me, bwaahahaha.
Anyhow, I'm going to demonstrate the script I used to actually post this blog message:
Yes seriously. It's really that easy.
Happy hacking!
