Under the Hood: An Overview of Tesla’s Neural Network Solution for Self-Driving Cars

A screenshot showing Elon Musk during Tesla's "Autonomy Day"
Elon Musk speaking during Tesla's Autonomy Day

On April 22, 2019 Elon Musk graced a small stage to outline, with a few key members of his Tesla team, where the current autonomous vehicle development process is, and where it is going at Tesla.  Over the course of a couple hours, the team discussed how they have accomplished building a highly complicated autonomous vehicle system called Tesla Autopilot. Elon Musk has promised the arrival of fully self-driving Teslas in the coming year.  In order to explain how this will be accomplished, the team walked through their neural network infrastructure. The room filled with investors and journalists often struggled to grasp the content and likely, many people on the livestream did so as well.  This blog simply will seek to outline how you can get a better understanding of what’s going on under the hood at Tesla.

Image of multiple cars on the road, with illustrations of how Tesla cars "see" other objects on the road

Elon Musk, when asked about Lidar technology for autonomous cars, laughed and indicated that he believes that it is “doomed”.  Lidar is essentially a laser that can measure distances in the outside world. Musk and his team outline how they have fully bought into a complex neural network approach to solving the self-driving question.  The presentation ranged from examining the hardware to the software behind the Tesla computer vision approach. Key concepts like “hidden layers”, “Relu”, and the all-important labeled data dilemma were peppered throughout the presentation.  I will attempt to provide a resource to understand some of these concepts.

Understanding neural networks can be tricky, but there are countless high-quality courses online which can aid in becoming familiar with them.  One of these courses, “Neural Networks and Convolutional Neural Networks Essential Training” taught by Jonathon Fernandes on Lynda.com, gives a broad overview of some of the technology being used for cutting edge technological development at companies like Tesla.  It walks you through a progression of steps to understand the key building blocks that build deep neural networks.

Screenshot of Lynda.com course entitled "Neural Networks and Convolutional Neural Networks Essential Training"

Fernandes explains the differences in activation functions like Relu, sigmoid, and TanH which directly help you understand some questions asked at the Tesla presentation.  He also talks through key concepts like backpropagation and gradient descent which are fundamental aspects of any neural network including Tesla’s.

The course the goes on to demonstrate how, using Keras, you too can build a neural network for image recognition.  Through doing this, it becomes evident how important labeled training data is, which serves as a key aspect in Tesla’s argument about why they will win the race to fully autonomous driving. In finishing this online tutorial, you will not only understand neural and convolutional neural networks a little better, but you will also understand just how incredible the work is that Tesla has been able to accomplish. If you are anything like me, you will likely be left wondering how the Tesla team has managed to build a system that may propel society into an age post human driving.

Image of a fly with computer code beneath, demonstrating the process of neural network analysis