TinyML by Daniel Situnayake -- Read -- Imperial Library of Trantor

Index

Preface

Conventions Used in This Book Using Code Examples O’Reilly Online Learning How to Contact Us Acknowledgments

1. Introduction

Embedded Devices Changing Landscape

2. Getting Started

Who Is This Book Aimed At? What Hardware Do You Need? What Software Do You Need? What Do We Hope You’ll Learn?

3. Getting Up to Speed on Machine Learning

What Machine Learning Actually Is The Deep Learning Workflow

Decide on a Goal Collect a Dataset Design a Model Architecture Train the Model Convert the Model Run Inference Evaluate and Troubleshoot

Wrapping Up

4. The “Hello World” of TinyML: Building and Training a Model

What We’re Building Our Machine Learning Toolchain

Python and Jupyter Notebooks Google Colaboratory TensorFlow and Keras

Building Our Model

Importing Dependencies Generating Data Splitting the Data Defining a Basic Model

Training Our Model

Training Metrics Graphing the History Improving Our Model Testing

Converting the Model for TensorFlow Lite

Converting to a C File

Wrapping Up

5. The “Hello World” of TinyML: Building an Application

Walking Through the Tests

Including Dependencies Setting Up the Test Getting Ready to Log Data Mapping Our Model Creating an AllOpsResolver Defining a Tensor Arena Creating an Interpreter Inspecting the Input Tensor Running Inference on an Input Reading the Output Running the Tests

Project File Structure Walking Through the Source

Starting with main_functions.cc Handling Output with output_handler.cc Wrapping Up main_functions.cc Understanding main.cc Running Our Application

Wrapping Up

6. The “Hello World” of TinyML: Deploying to Microcontrollers

What Exactly Is a Microcontroller? Arduino

Handling Output on Arduino Running the Example Making Your Own Changes

SparkFun Edge

Handling Output on SparkFun Edge Running the Example Testing the Program Viewing Debug Data Making Your Own Changes

ST Microelectronics STM32F746G Discovery Kit

Handling Output on STM32F746G Running the Example Making Your Own Changes

Wrapping Up

7. Wake-Word Detection: Building an Application

What We’re Building Application Architecture

Introducing Our Model All the Moving Parts

Walking Through the Tests

The Basic Flow The Audio Provider The Feature Provider The Command Recognizer The Command Responder

Listening for Wake Words

Running Our Application

Deploying to Microcontrollers

Arduino SparkFun Edge ST Microelectronics STM32F746G Discovery Kit

Wrapping Up

8. Wake-Word Detection: Training a Model

Training Our New Model

Training in Colab

Using the Model in Our Project

Replacing the Model Updating the Labels Updating command_responder.cc Other Ways to Run the Scripts

How the Model Works

Visualizing the Inputs How Does Feature Generation Work? Understanding the Model Architecture Understanding the Model Output

Training with Your Own Data

The Speech Commands Dataset Training on Your Own Dataset How to Record Your Own Audio Data Augmentation Model Architectures

Wrapping Up

9. Person Detection: Building an Application

What We’re Building Application Architecture

Introducing Our Model All the Moving Parts

Walking Through the Tests

The Basic Flow The Image Provider The Detection Responder

Detecting People Deploying to Microcontrollers

Arduino SparkFun Edge

Wrapping Up

10. Person Detection: Training a Model

Picking a Machine Setting Up a Google Cloud Platform Instance Training Framework Choice Building the Dataset Training the Model TensorBoard Evaluating the Model Exporting the Model to TensorFlow Lite

Exporting to a GraphDef Protobuf File Freezing the Weights Quantizing and Converting to TensorFlow Lite Converting to a C Source File

Training for Other Categories Understanding the Architecture Wrapping Up

11. Magic Wand: Building an Application

What We’re Building Application Architecture

Introducing Our Model All the Moving Parts

Walking Through the Tests

The Basic Flow The Accelerometer Handler The Gesture Predictor The Output Handler

Detecting Gestures Deploying to Microcontrollers

Arduino SparkFun Edge

Wrapping Up

12. Magic Wand: Training a Model

Training a Model

Training in Colab Other Ways to Run the Scripts

How the Model Works

Visualizing the Input Understanding the Model Architecture

Training with Your Own Data

Capturing Data Modifying the Training Scripts Training Using the New Model

Wrapping Up

Learning Machine Learning What’s Next

13. TensorFlow Lite for Microcontrollers

What Is TensorFlow Lite for Microcontrollers?

TensorFlow TensorFlow Lite TensorFlow Lite for Microcontrollers Requirements Why Is the Model Interpreted? Project Generation

Build Systems

Specializing Code Makefiles Writing Tests

Supporting a New Hardware Platform

Printing to a Log Implementing DebugLog() Running All the Targets Integrating with the Makefile Build

Supporting a New IDE or Build System Integrating Code Changes Between Projects and Repositories Contributing Back to Open Source Supporting New Hardware Accelerators Understanding the File Format

FlatBuffers

Porting TensorFlow Lite Mobile Ops to Micro

Separate the Reference Code Create a Micro Copy of the Operator Port the Test to the Micro Framework Build a Bazel Test Add Your Op to AllOpsResolver Build a Makefile Test

Wrapping Up

14. Designing Your Own TinyML Applications

The Design Process Do You Need a Microcontroller, or Would a Larger Device Work? Understanding What’s Possible Follow in Someone Else’s Footsteps Find Some Similar Models to Train Look at the Data Wizard of Oz-ing Get It Working on the Desktop First

15. Optimizing Latency

First Make Sure It Matters Hardware Changes Model Improvements

Estimating Model Latency How to Speed Up Your Model

Quantization Product Design Code Optimizations

Performance Profiling

Optimizing Operations

Look for Implementations That Are Already Optimized Write Your Own Optimized Implementation Taking Advantage of Hardware Features Accelerators and Coprocessors

Contributing Back to Open Source Wrapping Up

16. Optimizing Energy Usage

Developing Intuition

Typical Component Power Usage Hardware Choice

Measuring Real Power Usage Estimating Power Usage for a Model Improving Power Usage

Duty Cycling Cascading Design

Wrapping Up

17. Optimizing Model and Binary Size

Understanding Your System’s Limits Estimating Memory Usage

Flash Usage RAM Usage

Ballpark Figures for Model Accuracy and Size on Different Problems

Speech Wake-Word Model Accelerometer Predictive Maintenance Model Person Presence Detection

Model Choice Reducing the Size of Your Executable

Measuring Code Size How Much Space Is Tensorflow Lite for Microcontrollers Taking? OpResolver Understanding the Size of Individual Functions Framework Constants

Truly Tiny Models Wrapping Up

18. Debugging

Accuracy Loss Between Training and Deployment

Preprocessing Differences Debugging Preprocessing On-Device Evaluation

Numerical Differences

Are the Differences a Problem? Establish a Metric Compare Against a Baseline Swap Out Implementations

Mysterious Crashes and Hangs

Desktop Debugging Log Tracing Shotgun Debugging Memory Corruption

Wrapping Up

19. Porting Models from TensorFlow to TensorFlow Lite

Understand What Ops Are Needed Look at Existing Op Coverage in Tensorflow Lite Move Preprocessing and Postprocessing into Application Code Implement Required Ops if Necessary Optimize Ops Wrapping Up

20. Privacy, Security, and Deployment

Privacy

The Privacy Design Document Using a PDD

Security

Protecting Models

Deployment

Moving from a Development Board to a Product

Wrapping Up

21. Learning More

The TinyML Foundation SIG Micro The TensorFlow Website Other Frameworks Twitter Friends of TinyML Wrapping Up

A. Using and Generating an Arduino Library Zip B. Capturing Audio on Arduino Index

← Prev
Back
Next →

← Prev
Back
Next →