Google Watching You Watch TV?

Sebadoh posted on Friday about Google getting into the satellite radio business. Well, it looks like Google is also making forays into television. Not producing, mind you, but monitoring what you watch and producing related internet content.

In a research paper presented last week at interactive television conference Euro ITV in Athens, Greece, Google researchers Michele Covell and Shumeet Baluja propose using ambient-audio identification technology to capture TV sound with a laptop PC to identify the show that is the source of the sound and to use that information to immediately return personalized Internet content to the PC.

"We showed how to sample the ambient sound emitted from a TV and automatically determine what is being watched from a small signature of the sound—all with complete privacy and minuscule effort," Covell and Baluja write on the Google Research Blog. "The system could keep up with users while they channel surf, presenting them with a real-time forum about a live political debate one minute and an ad-hoc chat room for a sporting event in the next."

… According to Boralv, the system wouldn’t be that intrusive. She writes, "If you were watching the news and wanted to delve deeper, this type of system could allow you to do that easily by automatically collecting related material and Web links for you. The beauty of the system that Michele and Shumeet describe is that it wouldn’t be a distraction. If you don’t want it you can ignore it and the PC browser will quietly update pages without bothering anyone—no input required and no audible output to form a distraction."

Those appalled by the prospect of Google tapping your television take heart: The proposal suggests user privacy would be respected. "[O]ur approach will not ‘overhear’ conversations," the paper says. "Furthermore, no one receiving (or intercepting) these statistics is able to eavesdrop, on such conversations, since the original audio does not leave the viewer’s computer." Perhaps there’s a lesson here for the National Security Agency.

Creepy? Or just the next step in increased synergy between new media and old? I can’t decide, but the Interactive TV Today blog describes in a bit more detail how Google might "overhear" what you’re watching.

[The system] would use a PC microphone to sample the ambient sound emitted by a television set and automatically determine what is being watched on that set from a small signature of that sound. The system would then use the data thus gathered to automatically present the viewer–in real time on his or her Web browser–with contextually relevant information, and with ad-hoc services that would enable social interaction around programming: thus, if the viewer were watching a sporting event, the system might present him or her with an ad-hoc fan forum; and if the viewer were to switch to a movie, it might present him or her with maps of the locales featured in the movie or with a bio of the actor currently appearing on the TV screen.

The system is composed of three distinct components: "a client-side interface, an audio-database server (with mass-media audio statistics), and a social-application Web server," the three scientists write in the paper’s introduction. "The client-side interface samples and irreversibly compresses the viewer’s ambient audio to summary statistics. [Note: the authors note that this irreversible mapping would serve to protect the viewer’s privacy.] These statistics are streamed from the viewer’s personal computer to the audio-database server for identification of the background audio (e.g., ‘Seinfeld’ episode 6101, minute 3:03). The audio database transmits this information to the social-application server, which provides personalized and interactive content back to the viewer."

One more question. How does this relate to Google partnering with Viacom to test sharing ad revenues from web videos? Is there a possible tie-in here? Maybe Google monitors what you’re watching and turns up related web videos? Clips from other shows you might be interested in, from Google’s ad partners?