Adapting user interfaces for gestural interaction with the flexible action and articulated skeleton toolkit

Evan Suma, David Krum, Belinda Lange, Sebastian Koenig, Albert Rizzo, Mark Bolas

    Research output: Contribution to journalArticlepeer-review

    55 Citations (Scopus)

    Abstract

    We present the Flexible Action and Articulated Skeleton Toolkit (FAAST), a middleware software framework for integrating full-body interaction with virtual environments, video games, and other user interfaces. This toolkit provides a complete end-to-end solution that includes a graphical user interface for custom gesture creation, sensor configuration, skeletal tracking, action recognition, and a variety of output mechanisms to control third party applications, allowing virtually any PC application to be repurposed for gestural control even if it does not explicit support input from motion sensors. To facilitate intuitive and transparent gesture design, we define a syntax for representing human gestures using rule sets that correspond to the basic spatial and temporal components of an action. These individual rules form primitives that, although conceptually simple on their own, can be combined both simultaneously and in sequence to form sophisticated gestural interactions. In addition to presenting the system architecture and our approach for representing and designing gestural interactions, we also describe two case studies that evaluated the use of FAAST for controlling first-person video games and improving the accessibility of computing interfaces for individuals with motor impairments. Thus, this work represents an important step toward making gestural interaction more accessible for practitioners, researchers, and hobbyists alike.

    Original languageEnglish
    Pages (from-to)193-201
    Number of pages9
    JournalCOMPUTERS & GRAPHICS-UK
    Volume37
    Issue number3
    DOIs
    Publication statusPublished - 2013

    Keywords

    • Gesture
    • Middleware
    • Natural interaction
    • User interfaces
    • Video games

    Fingerprint

    Dive into the research topics of 'Adapting user interfaces for gestural interaction with the flexible action and articulated skeleton toolkit'. Together they form a unique fingerprint.

    Cite this