This paper is the second of three research papers published as part of my PhD thesis.
One challenge I ran into when implementing the demos for my first paper was that I found it difficult to track that state of any user interface element that had more than two states. The aim of this paper was to modify the initial architecture to make it easy for developers to track the probabilistic state of user interface elements.
The key insight in this paper was to use a Monte Carlo approach to maintain a probabilistically accurate description of the user interface that can be used to make informed choices about actions. Samples are used to approximate the distribution of possible inputs, possible interactor states that result from inputs, and possible actions (callbacks and feedback) interactors may execute.
For example, a touch event is often represented as a single point, but in fact the finger is touching an entire area. My system approximates this distribution by generating a collection of individual touch events (Figure 2).
Because each sample is certain, the developer can specify most of the behavior of interactors in a familiar, non-probabilistic fashion. This approach retains all the advantages of maintaining information about uncertainty while minimizing the need for the developer to work in probabilistic terms. This paper presents a working implementation of our framework and illustrates the power of these techniques within a paint program that includes three different kinds of uncertain input.