 Gesture Recognition in FlashPosted on March 7, 2005 by Grant SkinnerThis is an older experiment, but I think it’s still fairly interesting, and I wanted to post it to the blog so that it has a home on my site. This system demonstrates converting user gestures into machine readable information – in this case, it simulates Palm graffiti recognition. Click and drag to write graffiti characters on the input pad, and the system will convert them into characters on screen. Supports almost the full range of graffiti inputs including letters, numbers, uppercase, backspace and punctuation.
Here’s a graffiti guide to help you out – the dot shows the starting point of the stroke. It also supports caps shift (upwards stroke), caps lock (two upwards strokes), and punctuation shift (single tap).
This algorithm was also put to use by Random Media on the Shinobi website. For a full explanation of the algorithm, you can click here. The cool thing is that this is all based on a very generic pattern recognition engine that I wrote, which can be used to identify patterns in any kind of linear 2 axis data.
Follow @gskinner on Twitter for more news and views on interactive media.
|
|
|
16 Comments
As always, really nice, mate.
but... Somehow all the things I did turned out wrong at the text field.
probably you made a chart on how to draw those letters to come up right, like the palm has.
Posted by: Matti on Mar 8, 2005 2:27am
Added a graffiti guide. I haven't had any problem drawing any of the characters - it's at least as accurate as my old Palm Pro (though that's not saying much). Then again, it is trained to my graffiti input.
Posted by: Grant Skinner on Mar 8, 2005 12:07pm URL: http://gskinner.com/
Very nice. I got all the letters fine. I'll be taking a look at this soon.
Cheers.
+LA
Posted by: LordAlex on Mar 9, 2005 7:40am URL: http://www.lordalex.org
Really nice!
The things that can be done with flash never stop amazing me.
Posted by: kembond on Mar 10, 2005 10:07am URL: http://www.kembond.com
A more reliable algorithm may go like this:
Record the position of the mouse at different intervals. Create two sub-graffities, x against t and y against t. For example, A maps to rising line in x versus t and a up and down line in y against t.
Normalize those and do a fit to find the good letter. Because it uses speed instead of position, and because x and y are functions of t instead of things doing loops and such, it should be faster and more reliable.
Posted by: Patrick Mineault on Mar 10, 2005 8:59pm URL: http://www.5etdemi.com/blog
Someone actually released a system based on discrete gestures similar to what you described. It was less processor intensive, but far less accurate - this is because it relies on the accuracy of independent strokes, rather than a comparison across an entire character.
Posted by: Grant Skinner on Mar 13, 2005 10:31pm URL: http://gskinner.com/
Hello, very nice !
The only problem is the url
http://www.gskinner.com/playpen/graffiti.html
isn't working yet... Any idea ?
Ciao, r0main
Posted by: r0main on Mar 29, 2005 2:54am
i have a palm pilot and this is crap!!!!!!!
Posted by: slitix on May 17, 2005 6:14pm
I also would like to see the http://www.gskinner.com/playpen/graffiti.html page. Thanks!
Posted by: deecay on May 26, 2005 2:34am
Really sweet!! Excellent work indeed!!
Posted by: Harsha Hegde on Jun 1, 2005 2:36pm
very nice
good idea
Posted by: david on Sep 29, 2006 6:28pm URL: http://www.supermali.com
Why can't I seem to get numbers to work?
Also I would like to see the link fixed to the http://www.gskinner.com/playpen/graffiti.html page
Cheers!
Nice work by the way
Posted by: Ryan Preece on Nov 2, 2006 4:56am URL: http://www.visual-voice.co.uk
gesture recognition with webcam.
First engine for recognition with webcam
small video presentation for my program
http://www.robotdimitron.com/oblici
Autor: Dimitron
Posted by: dimitron on Jul 23, 2007 7:22am URL: http://www.robotdimitron.com/oblici
First engine for gesture recognition using webcam with hand control( no color detection )
http://interaktivnamatematika.com/oblici/
I am from Macedonia where is born Alexander Macednien, its small country with bigs ideas.
Thank you Grant Skinner for pioner idea and realisation of this program
Posted by: dimitron on Mar 16, 2008 2:09pm URL: http://interaktivnamatematika.com/oblici
Hi Grant,
I'm really interested in looking at the source for this but it seems to have disappeared. Can you let me know whether or not you plan to post it again?
Thanks as always,
Dave
Posted by: Dave on Jul 13, 2009 7:59am
Nice Flash Example. Do u have more like that ?
Posted by: Creationpool on Jun 11, 2010 11:02am URL: http://www.creationpool.de