Monthly Archives: July 2013
Since the Gesture based Technology PLC was created in November last year, our focus has expanded to many devices and this has led to the setup of a central portal page that links to all the hardware that we are currently using with our students.
So if you are interested in using iPads, Eyegaze, Kinect, LeapMotion or any gesture based tech visit the GestureSEN wiki for information about how to use these devices to promote engagement, movement and creativity with students with severe learning difficulties.
Tuesday 23rd July was the day that my LeapMotion was delivered. I was very excited about this piece of technology and the impact it could have on access and engagement for the students that I work with.
For those who don’t know the LeapMotion is a small, USB sized device that works like a mini-kinect to track movements of the fingers. It’s ‘working area’ is an invisible cube about a metre across above the device and it will pick up fine hand and finger movements- these can then be converted instantly to effects on the screen- so causing visual and audio effects. Unlike the Kinect – which does whole body tracking. the Leap will only track hand and finger movement which potentially limits the audience that we would use the device with.
First impressions of the Leap are that it is very accurate in detecting hand and finger movement and I quickly went to Airspace (the Leap Motion app store) to see what has been developed for the device. When playing around with the apps, I found those that require fine motor skills were quite difficult to use for example Blocks – a jenga -like game – which requires you to ‘pinch’ your fingers to remove the blocks. However where I feel the Leap will be most useful for our students is in its ability to track hand movements and the apps Midnight, Flocking, Airharp and Vitrun – beautifully demonstrated this. I found myself engaged with these apps in particular Midnight and spent time trying to manipulate the environment to see what I could create.
After the initial trial at home, I was eager to try the Leap out with our students. Considering it was the last day of term, I still managed to find time for one of our students to have a go with the LeapMotion. The student has muscular dystrophy and has limited movement in his arms. I was intrigued to see how he got on with using the Leap, as I found even when I used it, the positioning of the device was crucial. He started with Midnight and really enjoyed making changes to the environment in front of him. He soon got to grips with changing fingers to make things happen and mentioned that he like the different colours that he was making on screen.
As any typical teenage boy, he is quite keen with playing computer games, so he tried out Cut the Rope. This game requires quite fine motor skills – using your finger to swipe the ropes to cut them. This is where I noticed that positioning of the device for those with difficulty with their mobility is crucial . He was quite determined to continue the game but he did get frustrated on occasions when his finger swipes were not fast enough to cut the rope. The last app that he tried was Vitrun and he had greater success with this app as it requires you to hold you hand out flat above the device to move the ball forward and backwards. Again, he enjoyed this app and was engaged for long periods of time which was promising to see.
Apps that I recommend to try:
Midnight – $2.39 A beautiful particle system that is controlled using your hand. Also plays music to add to the experience. Watch the video below:
Flocking – Free – Fish will follow you fingers on screen. Really nice and simple sensory application.
Airharp – $1.19 – exactly what it says, play a harp using you hands – nice interaction for simple cause and effect.
Vitrun Air – $3.59 – Simple game that involves moving a ball thorugh a range of obstacles with very easy controls.
I am impressed with the Leap and can see potential for it in an SEN setting as an enabling device that allows those with limited movement to be able to interact with a screen. Factors that will be important in the success of using this device are similar to all devices such as iPads, switches, Eygaze – positioning and app selection. Hopefully now that the Leap has been released, I am hoping that more developers will create content for it and I look forward to seeing what they develop for it.
Anthony Rhys (@trinityfieldsit) has setup a LeapSEN wiki which over the next couple of months we hope to fill with information about how to get the best out of the Leap for SEN students as part of the Gesture Based PLC group. For more information about this and other gesture based tech visit –http://kinectsen.wikispaces.com/