Gesture Controller with Microsoft Kinect for EIZO OR Monitor

A showcase for MEDICA

EIZO GmbH, specifically its Operating Rooms department, was considering demonstrating contactless gesture control for its CuratOR Surgical Panel (a customisable wall console for clinical use) as a showcase at MEDICA 2015 in Düsseldorf.

Hands on or rather "HandsUp" for CONZE

The goal was to develop a fault-tolerant gesture and voice control concept that would allow the user to switch between different signal layouts from a variety of image sources. These signal layouts were meant to be processed and merged by the EIZO Large Monitor Manager (LMM). The controller software should be activated with a simple “HandsUp” gesture or voice command. A swiping gesture or voice command was supposed to linearly switch between the layouts.

Analytical approach with practical result

Everything started by evaluating a wide range of sensor devices: Microsoft Kinect, Leap Motion, Thalmic Labs Myo and many other human-computer interaction technologies were checked for their suitability in a medical working environment. To bring EIZO’s vision to life, the software architects at CONZE combined the proven C# development environment with an intuitive user interface based on the WPF-GUI framework. Before the coding started, parameters like positioning, sequence and movement needed to be pre-defined precisely for all swiping gestures. Microsoft Kinect’s Skeletal Tracking feature served as the basis for the motion processing. With this technology, the joints of up to four users could be located and tracked simultaneously. The Voice control of the controller software was activated with the command “Hey CuratOR”. Inputs and Layouts were switched with the command “back” or “next” for the respective direction. The communication between the CONZE Gesture Controller Software and the EIZO LMM was realized with a configured HTTP interface to display gestures and image sources separately.