After several months of trial & error we have finally completed a working first prototype of the TUNE glove, recently re-named the “Reading Glove”.  The Reading Glove combines an Innovations ID-12 RFID reader, an XBee series 2 module (on a Lilypad XBee Radio), and Arduino Lilypad, and a simple homemade power supply to transmit RFID tag information wirelessly to a Laptop running Max MSP.  Read on for specific details on how it works, what it does, and how to make one of your own.

The complete circuit

Overview:

Watch our Video of the Reading Glove in Action
The Reading Glove is an RFID based interface for interaction with tangible objects.  It allows interactors to manipulate and handle tagged objects in order to access digital information that has been “embedded” in them.  Each object in this interaction is marked with a unique RFID tag.  These unique identifiers allow the object to be associated with specific digital information, in the form of audio, projected visualizations, and text.  The glove is comprised of an Arduino Lillypad Microcontroller, an Xbee Series two wireless radio, and an Innovations ID-12 RFID reader, embedded in the palm of a soft fabric glove.  The Reading Glove transmits the Tag information to a computer running Max MSP, which uses the tag information to trigger digital events.

The Reading Glove is used to reveal fragments of a story associated with a collection of 10 objects.  The objects represent tangible gateways to narrative meaning.  They are a form of “boundary object”, situated in between the world of the reader and the world of the fiction.  Readers are invited to play with the objects as they explore the interactive narrative, which is presented in the form of audio narration. As the reader interacts with the objects she is provided with fragments of a linear story, which may be pieced together like a puzzle to reveal the matrix of events, characters, themes, and associations surrounding the objects.

Some objects have only one story fragment, or “lexia” associated with them, while others are associated with multiple audio files.  These are presented randomly to the interactor: repeated interactions have the potential to reveal new pieces of the narrative.  The Narrative is written to take advantage of the interactive and non-linear nature of the system.  The story is written to include a web of “cognitive hyperlinks” which help guide the reader through the story.   For example, when she selects the camera, she learns about a roll of film which was hidden inside a coffee grinder.  When she selects the coffee grinder she learns about a rose which was used as a signal.  In this way, the story provides clues about how to piece itself together.

Objects with multiple lexia also provide the reader with clues about their relative position within the chronology of the story.  When a reader selects the Top Hat she might hear about the character selecting the Hat as part of a disguise, or she might hear about the character pulling the hat over his face to hide from his enemies.  Each occurrence of the object provides the reader with clues to the chronology of the story at an intra-object level, which in turn provide the reader with clues at an inter-object level.

This prototype allows us to explore questions about the relationship between tangible interaction and narrative meaning.  A short user study indicates the potential advantages of this glove-based interaction over a handheld device or stationary RFID scanner for interactive storytelling. Throughout the design of this system I have been working on the assumption that the use of the glove for object selection had narratively desirable advantages over a handheld device or stationary RFID scanner.
In particular I hypothesized that the glove based interaction would force the interactor to “grapple” with the objects explicitly, ideally foregrounding the objects in the reader’s mind.  If successful, this would reinforce the role of the objects as “loci of meaning”.  They would be more semantically rich than if used in a non-glove-based setting.  To interrogate this assumption, I ran two informal parallel pilot user studies.

In one group, users were asked to wear the glove while exploring the prototype.  In the second group, the glove was left sitting on the table, and users were instructed to scan objects over it without picking it up.  Both groups were allowed to interact for as long as they liked, and both groups were asked the same set of questions after they had completed their interaction.

Even with a small pool of participants, several trends became apparent, if only anecdotally so:  Interactors wearing the glove appeared more likely to play with the objects, whereas interactors scanning objects over the stationary glove were more likely to scan an object and then set it down.  I asked participants to recount the narrative events to the best of their ability after each session, and also asked targeted questions about object/story associations.  Participants in the first group appeared more able to recount the details of the story, and also demonstrated a stronger understanding of the role of the objects in the story.

While the informality of this study does not allow for generalizable claims about the value of the glove based interaction, it does indicate the promise of this approach, and has provided important information for the design of future studies using this prototype.
Building the Reading Glove

Here, we provide a brief overview of what went into the creation of the Reading Glove.
Parts Needed:

Tools Needed:

  • Soldering Iron and solder
  • Multimeter
  • Wire cutters/strippers
  • Alligator Clips

The Hardware:

Most of our effort was spent troubleshooting the hardware.  Arduino Lilypads are surprisingly delicate, and alligator clips slip around on their pads easily, which greatly increases the likelihood of a chip-frying short.  After blowing up our first Lilypad we invested in some cheap sheets of hobby-foam from a local dollar store, which we cut into Lilypad sized discs.  These reduced slippage, and prevented further shorting.

Perhaps the biggest challenge we faced was providing sufficient power to each of the components.  Our first design attempted to tie together a triad of Lilypad AAA power supplies, but we were unable to make this work.  Instead we opted to use a simple 5v regulator, with common + and – rails for all of the components to clip onto.  Here is a diagram for the final working circuit:

Glove Diagram v.3

At the top of this post is a picture of everything wired together and working.
The Software:

The brains of this project is the Lilypad Arduino, which will be making sense of the information coming from the RFID reader before routing it to the XBee radio for transmission. See Arduino Code below for complete program.

For information on configuring the XBee radios see this earlier post.
On the server side, we used Max MSP to process the serial information and map it onto the collection of sounds that we recorded.  Here is a copy of the Max Patch.

Arduino Code
Here is the code (you should auto-format this in Arduino after copying and pasting it.)

// RFID reader ID-12 for Arduino
// Based on code by BARRAGAN
// and code from HC Gilje – http://hcgilje.wordpress.com/resources/rfid_id12_tagreader/
// Modified for Arudino by djmatic
// Modified for ID-12 and checksum by Martijn The – http://www.martijnthe.nl/
// Modified for use with the XBee wireless radion by Karen and Josh Tanenbaum
// Use the diagram here to wire the ID-12 and the XBee: http://www.flickr.com/photos/jtanenbaum/4156417140/
// Remark: disconnect the rx serial wire to the ID-12 when uploading the sketch
#include   //Found here: http://arduiniana.org/libraries/NewSoftSerial/

#define rxPin 2
#define txPin 3
#define ledPin 13

byte pinState = 0;
NewSoftSerial xbee(rxPin,txPin);

void setup() {
pinMode(rxPin, INPUT);
pinMode(txPin, OUTPUT);
pinMode(ledPin, OUTPUT);
Serial.begin(9600);                                 // connect to the serial port
xbee.begin(9600);
}

void loop () {
byte i = 0;
byte val = 0;
byte code[6];
byte checksum = 0;
byte bytesread = 0;
byte tempbyte = 0;

if(Serial.available() > 0) {
if((val = Serial.read()) == 2) {                  // check for header
bytesread = 0;
while (bytesread < 12) {                        // read 10 digit code + 2 digit checksum if( Serial.available() > 0) {
val = Serial.read();
if((val == 0x0D)||(val == 0x0A)||(val == 0x03)||(val == 0x02)) { // if header or stop bytes before the 10 digit reading
break;                                    // stop reading
}

// Do Ascii/Hex conversion:
if ((val >= ‘0’) && (val <= '9')) { val = val - '0'; } else if ((val >= ‘A’) && (val <= 'F')) { val = 10 + val - 'A'; }

// Every two hex-digits, add byte to code:
if (bytesread & 1 == 1) {
// make some space for this hex-digit by
// shifting the previous hex-digit with 4 bits to the left:
code[bytesread >> 1] = (val | (tempbyte << 4));

if (bytesread >> 1 != 5) {                // If we’re at the checksum byte,
checksum ^= code[bytesread >> 1];       // Calculate the checksum… (XOR)
};
}
else {
tempbyte = val;                           // Store the first hex digit first…
};

bytesread++;                                // ready to read next digit
}
}

// Output to Serial:

if (bytesread == 12) {                          // if 12 digit read is complete
for (i=0; i<5; i++) { if (code[i] < 16) Serial.print("0"); xbee.print(code[i], HEX); xbee.print(" "); } toggle(ledPin); xbee.println(); xbee.println(); }

bytesread = 0;
}
}
}

void toggle(int pinNum) {
// set the LED pin using the pinState variable:
digitalWrite(pinNum, pinState);
// if pinState = 0, set it to 1, and vice versa:
pinState = !pinState;
}  

So, there you have it!  Software and Hardware documentation of the working Reading Glove!

Here are links to related earlier posts about the development of the system, and to our photo-documentation of the prototyping process:

RFID Reader Success

Lilypad + XBee Wireless Communication Achieved

ID 12 RFID Reader + Arduino LilyPad

The Pictures