Julia Schwarz

I build software that leverages probabilistic modeling, signal processing, and machine learning to improve user interfaces. I primarily work on developing new interaction techniques for touchscreens and other natural user interfaces.

I received a BS from University of Washington, and a PhD from Carnegie Mellon University. I'm currently heading up the R&D team at Qeexo, a company I co-founded in 2012.

You can find me on GitHub, Twitter, StackOverflow, and LinkedIn.

Publications my output as a PhD student

PhD Thesis: A Monte Carlo Approach to Probabilistic User Interfaces

Julia Schwarz
PhD Thesis, Carnegie Mellon University
Describes a probabilistic user interface architecture that uses monte carlo methods to track alternate input interpretations

An Architecture for Generating Interactive Feedback in Probabilistic User Interfaces

Julia Schwarz, Jennifer Mankoff, Scott E. Hudson
In Proceedings of CHI '15, Honorable Mention for Best Paper
A user interface architecture that not only tracks probabilistic interface state, but communicates this uncertainty back to the user in a meaningful way. This is accomplished by tracking possible interfaces, reducing the number of possible interfaces to a few exemplars, and fusing exemplars to communicate uncertainty and allow for disambiguation.


Xiang 'Anthony' Chen, Julia Schwarz, Chris Harrison, Jennifer Mankoff, Scott E. Hudson
In Proceedings of UIST '14
A collection of interaction techniques that interweave touch events with in-air gestures.

Probabilistic Palm Rejection Using Spatiotemporal Touch Features and Iterative Classification

Julia Schwarz, Robert Xiao, Jennifer Mankoff, Scott E. Hudson and Chris Harrison
In Proceedings of CHI ‘14
An algorithm for discriminating unintentional palm inputs from intentional pen inputs. This algorithm improves over existing palm rejection methods, reducing accidental palm inputs to 0.016 per pen stroke, while correctly passing 98% of stylus inputs.

Replacing 'Wave to Engage' on Xbox One by Combining Body Pose, Gaze and Motion to Determine Intention to Interact

Julia Schwarz, Charles Marais, Tommer Leyvand, Scott E. Hudson, Jennifer Mankoff
In Proceedings of CHI ‘14
A new algorithm for determining when to pay attention to a user’s actions, and when to ignore them. This work helped to replace the ‘wave to engage’ gesture on Xbox One.

TouchTools: Leveraging Familiarity and Skill with Physical Tools to Augment Touch Interaction

Chris Harrison, Robert Xiao, Julia Schwarz and Scott E. Hudson
In Proceedings of CHI ‘14
TouchTools draws upon user familiarity with tools from the real world and bring them to interactive surfaces. To invoke a tool, users replicate a tool’s real-world grasp and press it to the screen as though it were physically present.

Phone as a Pixel: Enabling Ad-Hoc, Large Scale Displays using Mobile Devices

Julia Schwarz, David Klionsky, Chris Harrison, Paul Dietz, and Andy Wilson
In Proceedings of CHI ‘14
A system for building crowd sized displays using personal mobile phones. A camera watches the crowd and looks for flashing patterns identifying each individual phone. Once phones are identified, they can display a unique color or animation based on their location.

TapSense: Enhancing Finger Interaction on Touch Surfaces

Chris Harrison, Julia Schwarz, and Scott E. Hudson
In Proceedings of UIST '11
A touchscreen that detects not only touch location, but touch type. This is accomplished by performing acoustic analysis when a finger strikes the screen. The technology is now being commercialized at Qeexo.

Monte Carlo Methods for Managing Interactive State, Action and Feedback Under Uncertainty

Julia Schwarz, Jennifer Mankoff, Scott E. Hudson, Andy Wilson
In Proceedings of UIST '11
A probabilistic input architecture that tracks the likelihood of interface alternatives by deterministically simulating hundreds of input alternatives and aggregating results. This allows develpers to build uncertain interfaces without having to think probabilistically.

Augmenting Web Pages and Search Results to Support Credibility Assessment

Julia Schwarz and Meredith Ringel-Morris
In Proceedings of CHI '11
How do you decide what to believe online? It turns out that showing aggregate statistics such as number of viewers (particularly expert viewers e.g. doctors, lawyers, etc.) significantly helps people asses the credibility of web page content. This is especially helpful when viewing search results.

A Robust and Flexible Framework for Handling Inputs with Uncertainty

Julia Schwarz, Scott E. Hudson, and Jennifer Mankoff
In Proceedings of UIST '10
Presents a general probabilistic input dispatch architecture where interface elements are queried regarding their suitability to handle an event.

Cord Input: An Intuitive, High-Accuracy, Multi-Degree-of-Freedom Input Method for Mobile Devices

Julia Schwarz, Chris Harrison, Scott E. Hudson, and Jennifer Mankoff
In Proceedings of CHI '10
I built a cord which detected twist, pull, and touch, with the intention of turning a simple headphone cord into an input device.

Reflections of Everyday Activities in Spending Data

Julia Schwarz, Jennifer Mankoff, Scott Matthews
In Proceedings of CHI '09
Your purchases reveal your whereabouts, your habits, preferences, and even your carbon footprint! This project aims to identify what credit companies already infer about your activies.

Notable Side Projects output from evenings and weekends

Headshot: A Windows Phone Application that Uses Computer Vision to Help You Take Selfies

Julia Schwarz
Windows Phone
The high quality camera on your phone is rear-facing, which means that you can't see a preview of yourself when trying to take a selfie. This makes it hard to get your face in the camera's frame. Headshot helps you take selfies by detecting your face in the camera preview and telling you how to adjust your camera. This application won a company-wide development contest at Microsoft and has been featured in Windows Phone App Store. It has over 225,000 downloads as of June 2014.

A Face Detection Library for Windows Phone 7

Julia Schwarz
Windows Phone
A C# implementation of Viola-Jones, able to load OpenCV XML files. First face detection library available for Windows Phone. I wrote this library while working on Headshot.

Window Manipulation Using the Microsoft Touch Mouse

Julia Schwarz and David Klionsky
An application enabling multitouch gestures on a Microsoft Touch mouse to move and resize windows. This application was built for the 2011 UIST Student Innovation Contest where it won first place in the 'Most Useful' category.

Inhabited Web

Julia Schwarz, Chris Harrison and Bryan Pendleton
A small visualization next to a browser's scroll bar. Triangles are used to represent users and their current scroll position, allowing people to see where others are congregating. This application won the Yahoo HackU contest at Carnegie Mellon in 2009.

For more prior work, please see the archives, which contain older sites and projects (before 2010).

My other skills include juggling, skiing , and raft-building.