Share

Tweet

Share

Share

Share

Email

Comments

On Tuesday, Facebook CEO Mark Zuckerberg’s face accompanied by Morgan Freeman’s voice, took over the Internet by storm. The home-video Zuckerberg released via his personal Facebook profile and featuring his home AI assistant, reminds one once again how the young tech tycoon is living at least a few years out ahead of anyone reading this post.

My personal challenge for 2016 was to build a simple AI to run my home — like Jarvis in Iron Man.

Zuckerberg wrote in a note in reference to his regular practice of setting an annual personal goal for himself.

It was in August that the founding executive shared that he hoped to soon demonstrate his home’s artificial intelligence system – named Jarvis, which he could talk to on phone and computer, that could control his home, including lights, temperature, appliances, music and security, that learned his tastes and patterns, that could learn new words and concepts, and that could even entertain his toddler Max. An excited Zuckerberg had then promised a demo in September, revealing that the AI responded to his voice alone and ignored even his wife.

The AI, Jarvis’s, demo finally making it public on December 20th, could be seen using several artificial intelligence techniques, including natural language processing, speech recognition, face recognition, and reinforcement learning, written in Python, PHP and Objective C.

The demo has taken its sweet while in making it to the masses, but the heavily produced video (Come to think of it … Jarvis is voiced by none other than Morgan Freeman!) has been worth its wait. The personal project is an insightful means to have an understanding of all the internal technology Facebook engineers get to use, as well as a thorough overview of home automation.

The platform looks like a powerfully capable voice-powered command center for the home. However, with the help of “all these Facebook engineers who are doing all this awesome AI work in speech recognition, face recognition,” his challenge this year hasn’t necessarily been a personal Waterloo for Zuckerberg. He confessed on Monday:

In some ways, this challenge was easier than I expected. In fact, my running challenge (I also set out to run 365 miles in 2016) took more total time.

The only challenge Zuckerberg noted much more complicated than he had expected was simply connecting and communicating with all of the different systems in his home.

In addition to featuring a sublime shirt canon and the sassy Nickleback burn we all loved, the demo released at Zuckerberg’s profile has triggered an avalanche of public feedback and suggestions which could be employed in making the AI more efficient.
The two most noticeable suggestions have been the following:

For assistants like Jarvis to be able to control everything in homes for more people, there need to be more devices connected and the industry needs to develop common APIs and standards for the devices to talk to each other.

In the longer term, Zuck plans to explore teaching Jarvis how to learn new skills itself rather than being taught how to perform specific tasks. Regarding the employment of Jarvis in Messenger, he opined that the future AI products cannot be solely focused on voice and will need a private messaging interface as well. He shared:

I have always been optimistic about AI bots, but my experience with Jarvis has made me even more optimistic that we’ll all communicate with bots like Jarvis in the future.

We can see that this type of visual AI system has the scope to be resourceful in solving the context problem so that the AI can correctly respond to context-free requests without being provided a location. Like most aspects of this AI, vision is most useful when it informs a broader model of the world, connected with other abilities. The more context the system has, the smarter is gets overall.

In general, Zuckerberg’s AI has been demonstrated to have used open-ended requests more frequently than more specific asks. No commercial products available in the market at the moment can do this, and therein lies a big opportunity for a commercially viable product.

Considering the ways to make this AI available to the world, Zuckerberg shared that he had thought of open sourcing his code, but the current AI tightly tied to his own home, appliances & network configuration, restricted his options. The Facebook CEO expressed that if he ever built a layer that abstracted more home automation functionality, he might release that. More significantly, he hoped to employ it as a solid foundation to build a new product.