Blog Archives

KinectSEN – exploring gesture based technology to engage all learners

Last Thursday, I had the pleasure to be invited to be part of the professional learning community (PLC) looking into gesture based technology and the impact it can have on special needs learners. Gesture Based Technology (GBT) includes technology that involves a natural user interface for its input. This can be the Kinect, iPads, Eyegaze, mobile floor projectors to name a few of the technologies and is mainly used at present in gaming consoles at home. The power of natural user interfaces is that is allows students with SEN to be included in sessions and enable them to explore movement, creativity and engagement. From the evidence that I have seen so far, it gives students an opportunity to be actively involved in effecting their environment and allows them to do things that they simply could not do before. What is great to see is the instant effect that students have using this technology and this achieved by them moving in whatever way they can.

One of the mobile interactive floor projectors

The PLC is co-ordinated by the fantastic Anthony Rhys (@trinityfieldsit) and from looking around Trinity Fields School it was amazing to see the progress the school had made in enabling students to interact in their lessons using GBT. The day focused on how this technology can be used to enable students who are often withdrawn and shy to engage in various activities. It was great to see the evidence of other students using this technology to encourage creativity, movement, engagement and exploration.

kinect flow.JPG

Over the next few months, I will be hoping to blog the progress that has been made in incorporating GBT into our schools and present evidence for this. I came away from the day even more enthused to make this happen and it was great to meet other practitioners just as interested in the technology to help engage thise student with severe learning difficulties. We are currently looking into adding this to our sensory room so that we have an interactive floor and wall display at the fraction of the cost that some SEN companies would charge.

If you are interested in GBT and the use of the Kinect and would like to find out more about how different schools are incorporating this technology please visit the Kinect Wiki site at


Using innovative technologies in the classroom – The Echoes Project

This post is the third in a series of posts on the Festival of Social Sciences – Using Innovative Technologies in the Classroom event that took place in Birmingham on Thursday 8th November. You can view the first post here and the second one here.

The Echoes project

Echoes is project led by Dr Kaska Porayska-Pomsta from the Univertisty of London with the focus to support children on the autistic spectrum with their social communication skills in particular joint attention. Many autistic children fail to pick up on the social cues given by others which can often lead to misunderstandings for many children in the spectrum.

She realised that technology could have a role to play in supporting children in developing these essential social skills and so developed the Echoes project. The idea is to provide children with an environment for them to practise these skills on something that isn't human and isn't going to take offence.

Echoes uses touch screen technology alongside cameras to pick up information from the user so that the virtual autonomous agent Andy can react appropriately to their input. Andy's traits are preselected by the team which determines how he reacts to the information he receives. In addition the cameras allow him to understand where the child might be looking. His personality has been built using artificial intelligence which enables him to react differently to different children. Depending on the child's development, Andy will react differently and adapt his actions accordingly. For example, if a child touches an object frequently, Andy might think the child likes that object and then would concentrate on using that object in different activities.

By observing children playing with Andy, it allows teachers to gain a better understanding of a child's strengths and weaknesses and explore questioning in relation to learning and development. I really enjoyed having the chance to experiment with the technology and Andy would instantly show me what the activity was and also signed in makaton to promote further interaction.

What really excites me with this project is the plans for the future. Kaska explained they are exploring different routes to open up the accessibility of the project. One use could be through using the Microsoft Kinect and using the tracking capabilities of this technology alongside Eyegaze cameras. This would be something that in my particular setting would be of great benefit as it opens up the project to wide range of students.

After having the chance to play with the project I left feeling that this technology could benefit many students in enabling to practice social skills, interaction and co-operation in a safe environment to prepare them for transferring these skills into real world contexts.

Related Links

The Echoes Project

The SHAPE project, technologies to enhance learning for young people on the Autistic Spectrum


Using Innovative Technologies in the Classroom – COSPATIAL Project

This post is the second in a series of posts on the Festival of Social Sciences- Using Innovative Technologies in the Classroom event that took place in Birmingham on Thursday 8th November. You can view the first post here.


The main emphasis of the COSPATIAL project is finding ways to adapt existing classroom technologies to engage children with autism in learning social skills. The project is being led by Dr Sarah Parsons of the University of Southampton and Dr Sue Cobb of the University of Nottingham.  Thanks go to Dr Sarah Parsons for talking us through the project and although we did not get to see the project in action it was clear to see how the technology would be used with Autistic Spectrum Condition (ASC) students.

The COSPATIAL project focuses on two types of technologies:

  • Collaborative Virtual Environments
  • Shared Active Surfaces

In this post I am going to concentrate on the Collaborative Virtual Environments. These allow multiple users to interact and communicate within a shared virtual space. They do this by accessing the space on individual laptops that are in the same room. There are 3 different programs that the students can use each working on different elements of social interaction from communication to working together to complete a task.

Talk2U – students communicate with the support of the teacher to help explain social conversation. The students press the the buttons on screen to communicate with each other and this can be on a variety of different topics.

Talk2U – Students can practice social conversation with the support of the teacher

Block Challenge – the students work together to build a tower of blocks. The students have to communicate with each other to complete the task. Again this task looks to promote skills that students on the Autistic Spectrum tend to need support with.

Block Challenge – where students work together to build a tower of blocks

Face2Face – Students communicate in a shared environment using video avatars. The space is flexible and can be adapted for different users and activities.

I felt that this project is really interesting as it allowed students with ASC to work in an environment that enables them to communicate and collaborate through a safe and comfortable medium. The idea of the project is to give the students a space to practice these skills before moving them onto to real-life interactions which can be extremely daunting for students with ASC.

What I also like about this was that they are using existing technology and the application can run on a variety of machines. Unlike some high-tech solutions on the market this application would benefit many school running on a limited budget.

Links –

The COSPATIAL Project –

The SHAPE Project, technologies to enhance learning for young people in the Autism Spectrum –

Next Post – Echoes Project

Guided Access – iOS 6

One of the great new features of iOS 6 is the guided access feature. This will prove extremely useful when working with students who have the tendency to keep pressing the home button to close the app.

Also another great feature of this, is that you can select parts of the screen to be inactive. This is particularly useful when using free apps that have adverts as I found that sometimes students will accidentally press the adverts, which then leads them away from the app.

To set up, guided access, you need to turn it on first. Go to Settings, General and click Accessibility. In there you will find Guided Access. Select this and switch it to on, plus set a pass code to enable you to turn it off.

Once you have set this up, load the app you want to use and triple press the home button. Press guided access which will lead you to the setup screen. Here you have the option to select parts of the screen to make inactive, and turn off touch and motion altogether. To select parts of the screen, just draw a circle around the bits you want to make inactive. Then all you have to do is press done and the app is ready to go. To exit, guided access, just triple press the home button and enter the passcode to exit.

A really excellent feature when using iPads with SEN.


PuppetPals HD – Aesop’s Fables

I was first shown this app at a training day and really liked the simplistic interface for recording your own shows. I was looking forward to using this app with my students to allow them to create their own movies. I chose the theme of Aesop’s Fables and the lesson was planned so that everything that they needed to do they would do on the iPad. The reason I chose a theme rather than letting the students come up with their own creations was that some of my ASC students struggle with creativity and I wanted a level playing field for all students to participate in the activity.

They first had to choose an Aesop Fables that they were going to recreate. These were available on iTunes U for free which allowed them to listen the story and note down the key points. Next they created a mind map using the app ‘Popplet’ and noted down what characters, settings and scenes they needed for their movie. Once this had been done the students got searching on the internet for the various images they would need.

20120227-115139.jpg   20120227-115145.jpg

Now onto Puppetpals. When you first start a new show you need to select the actors. Puppetpals comes preloaded with various actors and backgrounds but would recommend purchasing the ‘Directors Pass’ which allows you to download more themes. The students found importing the images really easy as they had saved them to the camera roll. Puppetpals allows you to import pictures from the camera library or take a photo using the iPad’s camera. If you want more high res images, some students used the digital camera then transferred them using the camera connection kit (more on that in a later post). Once they had selected the images, Puppetpals allows you to draw round a particular part of the picture to cut out the background allowing you to create your actors. So once all the actors had been imported and selected the next step was for them to choose their backgrounds. Again the students imported pictures from the camera role and resized the images to make their backgrounds. Once the backgrounds had been selected it was onto filming.

20120227-115200.jpg    20120227-115205.jpg

20120227-115211.jpg    20120227-115215.jpg

With Puppetpals, when you press the record button everything you do on screen is recorded as well as any sounds made. The students soon learned that they had to be quiet during the recording process as any background noise would be included. What really struck me about Puppetpals, was how quickly students with Severe to Moderate Learning Difficulties pick up the controls as if it was second nature. Some students struggled with the pinch-to-zoom motion, to increase/decrease the size of the characters, due to their physical limitations, but overall they were able to quickly film their scenes using the friendly and intuitive interface. What was also nice to see was the students discussing amongst themselves the different roles they would undertake and making decisions collaboratively as a group. Once each scene was made they saved them and then exported them into iMovie to make the final edits. The students enjoyed making these videos and were excited to share them with each other.

To conclude, I found using Puppetpals in the lesson as an excellent and simple tool to help students create their own movies. The interface is very simple to use and all students were able to participate in the activity. Below are their finished movies, enjoy!!

%d bloggers like this: