PhD Thesis: A Monte Carlo Approach to Probabilistic User Interfaces

Schwarz, J., 2015. Monte Carlo Methods for Managing Uncertain User Interfaces. Carnegie Mellon University.

Current user interface toolkits provide effective techniques for acting on user input. However, many input handling systems make the assumption that all input events are certain, and are not built to handle ambiguity such as multiple possible inputs from a recognizer. This is unfortunately at odds with recent interaction trends towards voice, gesture and touch, all of which come with a great deal of uncertainty.

This dissertation presents a new user interface architecture that treats user input as an uncertain process, approximates the probability distribution over possible interfaces using Monte Carlo sampling, and enables interface developers to easily build probabilistic user interfaces without needing to think probabilistically.

This architecture is embodied in the JULIA toolkit: a JavaScript User interface Library for tracking Interface Alternatives. To demonstrate the versatility and power of this architecture, the dissertation presents a collection of applications and interaction techniques built using the JULIA toolkit. This architecture provides the foundation for a new era of nondeterministic user interfaces that leverage probabilistic models to better infer user intent.

Understanding the Architecture

I've been told that Figure 1 provides the best overview of the approach in the toolkit. Next, I'd suggest watching my talk. Finally, my probabilistic feedback paper provides a concise architecture overview that I'd recommend reading.

If you have questions about the architecture, please don't hesitate to contact me.

Figure 1. Overview of how the JULIA toolkit dispatches an input event, manages state, and renders feedback to the user when a user does something.