Programming the smart home: ‘If this, then that’

The idea of a smart home sounds promising enough. Who doesn’t want a house full of automated gadgets — from light switches to appliances to heating systems — that know exactly when to turn on, turn off, heat up or power down?

But in order for all those devices to do what they’re supposed to do, they’ll need to be programed — a task the average homeowner might not have the interest or the tech-savvy to perform. And nobody wants to call tech support just to turn on a light.

A group of computer science researchers from Brown and Carnegie Mellon universities may have found a workable programming solution. Through a series of surveys and experiments, the researchers show that a style of programming they term “trigger-action programming” provides a powerful and intuitive means of talking to smart home gadgets.

The research was presented today at the Conference on Human Factors in Computing Systems (CHI2014) in Toronto. The work was co-authored by Blase Ur, a graduate student at Carnegie Mellon, Brown undergraduates Melwyn Pak Yong Ho and Elyse McManus, and Michael Littman, professor of computer science at Brown.

The trigger-action paradigm is already gaining steam on the web. It’s used perhaps most prominently on the website IFTTT.com (If This, Then That), which helps people automate tasks across various Internet services. Users create “recipes” using simple if-then statements — for example: “If somebody tags me in a Facebook photo, then upload it to Instagram.” The website interfaces with both service providers, and the action happens automatically each time it’s triggered.

“We live in a world now that’s populated by machines that are supposed to make our lives easier, but we can’t talk to them. Everybody out there should be able to tell their machines what to do.” Credit: Mike Cohea/Brown UniversityIFTTT.com started out as a tool to link websites, but it has recently added the capability to command a few internet-connected devices, like Belkin’s WeMo power outlet and the Philips Hue lightbulb. That got Littman and his team thinking perhaps the trigger-action model employed by IFTTT might be a good fit for home automation.

“As a programming model, it’s simple and there are real people using it to control their devices,” Littman said. “But the question we asked in this paper was: Does it work for the [home automation] tasks people want to do, or is it perhaps too simple?”

To find out, the researchers started by asking workers on Mechanical Turk, Amazon’s crowdsourcing marketplace, what they might want a hypothetical smart home to do. Then the team evaluated answers from 318 respondents to see if those activities would require some kind of programming, and if so, whether the program could be expressed as triggers and actions.

The survey responses varied from the mundane (“start the coffee pot from the bedroom”) to the outlandish (“I would want my home to be able to shoot lasers at intruders.”). A majority of activities people wanted — “Notify me when my pet gets out of the back yard” or “Start brewing coffee 15 minutes before my alarm” — would require some kind of programming to get the devices on the same page as the user. Most of the programming tasks fit nicely into the trigger-action format, the survey found. Seventy-eight percent of responses could be expressed as a single trigger and a single action. Another 22 percent involved some combination of multiple triggers or multiple actions.

The next step for the researchers was to see how well people could actually design recipes to accomplish tasks. To do that, they used two interfaces designed by McManus, one of the undergraduate researchers, and enlisted Mechanical Turkers to make recipes with the interfaces.

“We based both [interfaces] on ‘If This Then That,’” McManus said. “But then we made one of them slightly more complex, so you could add multiple triggers and multiple actions.”

The study showed that participants were able to use both interfaces — the simpler one and the one with multiple triggers and actions — fairly well. Participants who didn’t have any programming experience performed just as well on the tasks as those who did.

Taken together, the researchers say, the results suggest that trigger-action programming is flexible enough to do what people want a smart home to do, and simple enough that non-programmers can use it. Melwyn Pak, one of the Brown undergraduates on the project, finds that encouraging.

“People are more than ready to have some form of finer control of their devices,” he said of the results. “You just need to give them a tool that allows them to operate those devices in an intuitive way.”

Littman, who has been studying end-user programming of electronic devices for several years, agrees.

“We live in a world now that’s populated by machines that are supposed to make our lives easier, but we can’t talk to them,” he said. “Everybody out there should be able to tell their machines what to do. This paper is our attempt to start thinking about how to bridge that gap.”