Jarvis Demo

Jarvis System

This project was to integrate Arduino and laptop in order to build a Smart Assistant at home. I call it Jarvis. There’s an Arduino-like open source firmware to help us prototype Iot products : Nodemcu. As an Iot device, it is less than $2 WI-FI MCU ESP8266 integrated and easy to prototyping development kit. And I need a laptop as a central management system. It uses several techniques including speech recognition, web crawler, socket, sqlite, Messenger API written in Python and Nodejs.

1.Light Control

Nodemcu *1
Relay *1
LCD *1
Plug-in *1

2.Door Control

Motor 755 *1
Relay *1
Power supply *1
Nodemcu *1

3.User Interfaces

There're three ways to communicate with Jarvis Server : Voice, Chat server, and Facebook messenger. I applied Google Speech API to realize Speech-To-Text. I also wrote a chat web in Nodejs, so it is feasible to control my room as long as I can connect to my wifi network. However, Facebook Messenger is much powerful. I used "fbchat" library in Python to login to Facebook and keep listening to my command, and then used "forever" Nodejs package to keep the program alive even when the program stops. In this way we are allowed to communicate with Jarvis in public network.

4. Add new command

Every time I create a new command, I need to write a new behavior code first in “behavior” directory, and then execute “AddEntryToDataBase.py” to add the new behavior code and corresponding “command” to the database.

5.Play Music

There are different ways to play music like spotify or itunes. I prefered listening to music on Youtube, so I implemented the control of music using os module in python to open a youtube URL in firefox browser and if I want to turn off the music, just kill the browser.

6.Ask for weather & news

I applied crontab to grab weather and news information every three hours. First write an web crawler program and set up cron job using crontab. Crontab is a time-based job scheduler in Unix-like computer operating systems. In the program, I need to store data as JSON file, and everytime I call Jarvis for weather and news, Jarvis reads the corresponding data and shows the information.

7.Ask for Dynamic Public IP

In my laptop, I've activated ssh and sftp server for server management. However, because of dynamic public IP address, I am not able to connect to my server when I go outside. So, when I built a messenger bot in my laptop, I came up with an idea that Jarvis maybe could send me the dynamic IP. In my home network, I used Network Address Translation (NAT) to assign the public address to my laptop inside the private network.

Code Here