Reading #3: Manual Deskterity: An Exploration of Simultaneous Pen + Touch Direct Input

Comments

Steven Hennessy

Shena Hoffman

Reference

Manual Deskterity: An Exploration of Simultaneous Pen + Touch Direct Input

Ken Hinckley, Kohi Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, Bill Buxton

CHI 2010 - Atlanta

Summary

Microsoft Research has created a prototype digital drafting table, using the Microsoft Surface and newly developed Deskterity software. The system is unique in that it exist to study the use of both direct touch and pen input. The paper identifies that many mainstream devices are now relying on touch as a mechanism of input. Recently, digitizers which can detect both a pen input and finger touch input have been developed. This opens the door for a world of devices which can allow users the experience of using both their hands and a pen as input. Microsoft develops a general rule of using touch for selection, translation, scaling,etc. and the pen for writing. Additionally, a third type of action is generated from combining both pen and touch input simultaneously. Several combinations of input are explored, as well as the results of the input. Much emphasis is placed on developing a set of compound gestures. Studies were conducted with several designers and results were generally favorable. Future work includes expanding from a demo into a set of techniques usable across many applications.

Discussion

This project is an exciting glimpse at the future world of human computer interaction techniques. The authors from Microsoft do an excellent job at assessing the current landscape of our devices, as well as predicting the future path of these devices. Additionally, the related works section of this paper is a great joy to see -- the authors really appear to have performed a thorough review of recent research efforts. I found most of the reactions to compound inputs to be predictable and desirable. The idea of using the photo or object as a straight edge on another object is great! A few of the more “novel” actions felt like they were trying to hard for the “wow factor”, while ignoring what feels natural. Once the context menu came into play, the natural feel was lost. The stapling action was also a bit off feeling in my opinion. But, aside from a few nitpicks, this project is a significant contribution and will be of importance to our everyday lives soon.

Take a few minutes to watch the demo video: