Quick Response

We were looking for a way to create interactions in the natural spaces of the building, on the walls, in the windows, and to avoid a “kiosk” interface as much as possible. By choosing the QR code, we used the visitor’s smartphone as the primary interface for the installation.

Most people now recognize the QR form but are still unsure of its purpose, It’s generally seen as a hyperlink to product or service information. Our goal was to give it a new context, reframing the QR as a digital actuator rather than a passive code.

We knew the number of QR-capable smartphones was limited, but accepted the constraint. To simplify development, we supported Android and iOS only. (It was still surprising to see how many designers and artists continue to use Blackberry devices!)

Although real-time interaction is often easier through native applications, we chose to build web apps for compatibility across platforms. Visitors without smartphones remained engaged by watching others interact and by joining conversations that followed. The most interesting part was seeing the discovery process unfold.

We needed a way for users to communicate with our interactive pieces, and for those pieces to communicate with each other. We decided to use a shared language that all applications could understand, and OSC (Open Sound Control) was a natural fit. This offered a friendly learning curve and broad support across environments such as Processing, ActionScript, Arduino, and openFrameworks.

To manage the communication traffic, we used Node.js, a server-side JavaScript runtime, together with the modules Socket.io and Node-OSC. This combination handled all messaging, allowing us to build applications that received real-time input from smartphone clients and could sync and broadcast across devices. The server shared OSC data with multiple interactive applications as needed.

The key advantage of this setup was that all client interfaces remained web-based, enabling interaction from across the room or across the network. Because the entire communication layer was written in JavaScript, it was both powerful and flexible.

System diagram for QR-driven interaction at CCS, 2011

Late in development we discovered high-level libraries like Express and Now.js, which likely would have saved us time, but the process itself was invaluable. It remains unclear whether Node’s asynchronous nature improved performance or throughput, but it is something worth exploring further.

 

These technologies allowed teams to create virtually-controlled physical displays. Mobile clients talking to servers via the internet, servers talking to computers on a local network running Flash and Processing, Computers talking (sans wires) to Arduino microcontrollers. All in all a pretty powerful and flexible system, I hope we continue to build on it!