Google Knows What We Do Online and May Soon Know Everything Else About Us, Too

Google just wrapped up its yearly developers conference in Mountain View, and the company’s recent investments in artificial intelligence and machine learning were on full display. The biggest news of the conference revolved around the demonstration of Duplex, the AI behind the voice of Google Assistant.

Google played two conversations at the conference. In the first conversation, the Google Assistant booked a haircut appointment for its “client.” There were audible gasps from attendees of the conference — it was nearly impossible to tell that the Google Assistant was a robot and not a human. After showcasing its newest developments in AI, Google CEO Sundar Pichai said it will be a while before this product is available to the general public. The company will need to evaluate how it signifies to the humans on the other side of the conversation that they are indeed talking to a robot. Ethical implications and concerns abound.

The business opportunity of this AI is clear, though. Improvements in the artificial intelligence behind Google Assistant would make robots even more capable of completing tasks, which could be appealing to customers and boost sales. This would be a boon for Google, which is trying to position itself to be a major player in the next iteration of the online world: the Internet of Things.

Google has succeeded in maintaining a dominant position as the internet has moved from computers to mobile devices. Now we are shifting into what people are calling the Internet of Things — IoT — if you want to use the jargon.

The Internet of Things is a fancy way of talking about how the internet is embedded in more and more consumer products — in stuff like televisions, lights, TVs and cars. These “smart” devices can be communicated with and manipulated through the web by users. They are also outfitted with sensors and, in some cases, cameras that can capture and transmit all sorts of data about their users. Us.

This makes the IoT very enticing for Google. All of these smart objects are untapped fountains of personal information, and personal information is the financial lifeblood of Google. It drives the company’s online advertising business model.

Who knows what kind of intimate information the objects around us will be able to gather in the years to come: What we eat, when we sleep, what we do at home. The potential is massive. The IoT is not just a new frontier of the internet — it’s a new frontier of data collection.

We already use Google to send email, search the web, navigate, watch videos, compose documents, store files. The list goes on and on.  From all these free web services, Google has already scraped arguably the largest and most valuable treasure trove of online user data owned by a company. Facebook is its only major competitor.

Now Google is trying to get into our homes and cars, to collect a whole other set of personal data to combine with the online info it already has its hands on. It’s trying to build an autonomous car. It acquired and then subsumed Nest, which has been expanded from a smart thermostat” to a smart home product. It is also competing in the smart speaker market. This is all on the hardware side. But Google is also making a strong push into the IoT on the software side.

San Jose State University engineering professor Ahmed Banafa attended Google’s annual developers conference. Banafa said just from looking at what Google is demo-ing, you can tell the company is trying to cement itself as a big player in the Internet of Things.

This, Banafa said, is part of why Google is investing so heavily in artificial intelligence, particularly the AI behind speech recognition and replication. The company with the best speech AI will have an edge in the market for smart speakers, which is currently a key product in the IoT.

Right now, smart speakers are the main portals to the growing world of connected things around us. The voice-activated speaker acts as a hub — a device we talk to that can talk to our other devices. Control that device, and you’ve got the robot at the center of the robot action.

Apple, Amazon and Google are in a three-way race over speech AI. Apple has Siri, Amazon has Alexa, and Google has Google Assistant. Amazon still dominates the smart speaker market, controlling some 70 percent. But Google is gaining. It has made large advances in the quality of the artificial intelligence behind Google Assistant.

Google has another play to insert itself into the IoT software. At the developers conference it rolled out a new operating system for smart objects to run on. The OS is called Android Things, and if it succeeds in catching on, it means Google would have a back door into the IoT. Google would not have to build the smart fridges and toasters, but could still have a hand in them because they would be running on Google software.

Abhishek Nagaraj is a professor at UC Berkeley’s Haas School of Business, and he thinks this is a bigger play for Google. “What is going to be more important for them is not so much to control the hardware,” he said, “but to control the operating system or the software that runs on these things.”

If Google succeeds in getting manufacturers to use its software, the company has a chance of collecting data from our smart fridges, lights, TVs and everything else that ends up getting connected to the web. That data can then be combined with all that Google already knows about what we do online. Together, it could make an unrivaled picture of who we are, what we do and how we behave.

Jennifer King is the director of consumer privacy at the Center for Internet and Society at Stanford Law School. “You’re starting to look at a world where their reach basically gets into almost every corner of your daily existence,” King said.

She said Facebook is Google’s only major competitor in this race to gather personal information and monetize it through advertising. Inserting itself into the objects around us could be a way for Google to get a leg up on its rival. It would open up an entirely new vein of personal data. This would be a boon for a company that makes most of its money from advertisers who would love access to our inner workings.

Recently, Facebook has borne the brunt of concern over the privacy, handling and misuse of personal information. With the Cambridge Analytica scandal, the public got a taste of the potential power of data. It could potentially be used to manipulate us. Subtly alter our desires. Influence our election choices. And for years we have been willing to give away our data and the data of all of our “friends” just to take an online quiz that promised to tell us what Harry Potter character we most resemble.

Google has succeeded in keeping its user-data-collection business model out of the privacy-misuse spotlight this go-round. That feat could become trickier as the company moves deeper into our homes and intimate spaces.

“The more ability they get to draw inferences about every aspect of your life, that’s where things I think get to become more uncomfortable,” King said.

What if this data falls into the hands of someone you don’t want to have it? How could it be used to influence your behavior, to shape who you are? What does it mean to be a human in a world where more and more of life is measured and quantified and recorded? Where you can’t tell a human from a robot?