Use TensorFlow.js in a React Native app

In this tutorial you'll install and run a React Native example app that uses a TensorFlow pose detection model (MoveNet.SinglePose.Lightning) to do real-time pose detection. Built on the TensorFlow.js platform adapter for React Native, the app supports both portrait and landscape modes with the front and back cameras.

Prerequisites

To complete this tutorial, you need the following installed in your development environment:

The TensorFlow.js React Native platform adapter depends on expo-gl and expo-gl-cpp, so you must use a version of React Native that's supported by Expo.

Install and run the example app

  1. Clone or download the tfjs-examples repository.

  2. Change into the react-native/pose-detection directory:

    cd tfjs-examples/react-native/pose-detection

  3. Install dependencies:

    yarn

  4. Run the example app locally:

    yarn start

When you run the app, the terminal displays a QR code.

Starting Metro Bundler
▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄
█ ▄▄▄▄▄ █▄▀ ▀   ███ ▄▄▄▄▄ █
█ █   █ █   █▀ █ ▀█ █   █ █
█ █▄▄▄█ █▄█▀ ▄▀█▄▀█ █▄▄▄█ █
█▄▄▄▄▄▄▄█▄█ █ █▄▀ █▄▄▄▄▄▄▄█
█▄  ▄▀█▄█ █ ▄ ▀▀▄▄▄██▄ ▄▀▄█
██▄▀▀█▀▄█▀  ▄▄▀ █▀█ ▀██▀███
█▄▄▀ ▀▀▄▄▄ ▄▀ ▄█ ▄█ ▄ █ █▀█
█▀▄▄ ▄▄▄▀ ▄  █▄██ ▀▀█▀▀█ ▀█
███▄▄██▄█▀▄██▄  ▄ ▄▄▄ ▀▄█▀█
█ ▄▄▄▄▄ ██  ▀██▀█ █▄█ █▄▄ █
█ █   █ █▀█ ███▀▀▄▄   █▀ ▀█
█ █▄▄▄█ █ ▀▀▀▀▀▄▀▄▀▄█▄▄ ▄██
█▄▄▄▄▄▄▄█▄██▄▄██▄██████▄▄▄█

› Metro waiting on exp://192.168.0.6:19000

Scan the QR code with a phone or other test device that has Expo Go installed. The example app should open in Expo Go.

The screenshots below show the app detecting and rendering keypoints of the user's body.

Portrait Landscape

To understand how the TensorFlow.js libraries are used in the example, look at App.tsx. The comments in that file explain how to configure tensor size, load the model, run pose detection, and more.

To learn more about pose detection using TensorFlow.js, see this blog post.

More about the Reactive Native platform adapter

The TensorFlow.js platform adapter for React Native provides GPU-accelerated execution of TensorFlow.js and supports all major modes of TensorFlow.js usage:

  • Support for both model inference and training
  • GPU support with WebGL via expo-gl
  • Support for loading pretrained models from the web
  • IOHandlers to support loading models from async storage and models that are compiled into the app bundle

For full installation instructions, see the React Native platform adapter README.

Troubleshoot TensorFlow.js for React Native

If your app crashes on startup, you might need to alter your setup. To learn more about setting up the platform adapter for React Native, see the README.