Human Activity Recognition with LSTM on M5Stack

An LSTM trained on the UCI HAR dataset and deployed on an M5Stack Gray for real-time on-device inference

This project implements Human Activity Recognition (HAR) with a Long Short-Term Memory (LSTM) network and deploys it on an M5Stack Gray, an ESP32-based IoT device, for real-time inference from its built-in IMU sensor. The model is trained on the UCI Human Activity Recognition dataset to recognize six activities: walking, walking upstairs, walking downstairs, sitting, standing and laying.

From PyTorch to the device.

  1. Train the LSTM in PyTorch and keep the best checkpoint.
  2. Convert the model to TensorFlow Lite.
  3. Convert the .tflite file into a C header (xxd -i) and mark the arrays const, so the model fits in the device flash.
  4. Load it with TensorFlowLite for ESP32 in an Arduino sketch and classify live IMU readings on the M5Stack.
Left: confusion matrix on the test set (normalized to % of total test data). Right: real-time prediction running on the M5Stack.
Accuracy over 650 training epochs (learning rate 0.0015).

Links: Code