TensorFlow Model Analysis

TensorFlow Model Analysis (TFMA) is a library for evaluating TensorFlow models. It allows users to evaluate their models on large amounts of data in a distributed manner, using the same metrics defined in their trainer. These metrics can be computed over different slices of data and visualized in Jupyter notebooks.

TFMA Slicing Metrics Browser

Installation

The recommended way to install TFMA is using the PyPI package:

pip install tensorflow-model-analysis

Currently, TFMA requires that TensorFlow is installed but does not have an explicit dependency on the TensorFlow PyPI package. See the TensorFlow install guides for instructions.

To enable TFMA visualization in Jupyter Notebook:

  jupyter nbextension enable --py widgetsnbextension
  jupyter nbextension enable --py tensorflow_model_analysis

Notable Dependencies

TensorFlow is required.

Apache Beam is required; it's the way that efficient distributed computation is supported. By default, Apache Beam runs in local mode but can also run in distributed mode using Google Cloud Dataflow and other Apache Beam runners.

Apache Arrow is also required. TFMA uses Arrow to represent data internally in order to make use of vectorized numpy functions.

Getting Started

For instructions on using TFMA, see the get started guide.

Compatible Versions

The following table is the TFMA package versions that are compatible with each other. This is determined by our testing framework, but other untested combinations may also work.

tensorflow-model-analysis apache-beam[gcp] pyarrow tensorflow tensorflow-metadata tfx-bsl
GitHub master 2.24.0 0.17.0 nightly (1.x/2.x) 0.24.0 0.24.1
0.24.3 2.24.0 0.17.0 1.15 / 2.3 0.24.0 0.24.1
0.24.2 2.23.0 0.17.0 1.15 / 2.3 0.24.0 0.24.0
0.24.1 2.23.0 0.17.0 1.15 / 2.3 0.24.0 0.24.0
0.24.0 2.23.0 0.17.0 1.15 / 2.3 0.24.0 0.24.0
0.23.0 2.23.0 0.17.0 1.15 / 2.3 0.23.0 0.23.0
0.22.2 2.20.0 0.16.0 1.15 / 2.2 0.22.2 0.22.0
0.22.1 2.20.0 0.16.0 1.15 / 2.2 0.22.2 0.22.0
0.22.0 2.20.0 0.16.0 1.15 / 2.2 0.22.0 0.22.0
0.21.6 2.19.0 0.15.0 1.15 / 2.1 0.21.0 0.21.3
0.21.5 2.19.0 0.15.0 1.15 / 2.1 0.21.0 0.21.3
0.21.4 2.19.0 0.15.0 1.15 / 2.1 0.21.0 0.21.3
0.21.3 2.17.0 0.15.0 1.15 / 2.1 0.21.0 0.21.0
0.21.2 2.17.0 0.15.0 1.15 / 2.1 0.21.0 0.21.0
0.21.1 2.17.0 0.15.0 1.15 / 2.1 0.21.0 0.21.0
0.21.0 2.17.0 0.15.0 1.15 / 2.1 0.21.0 0.21.0
0.15.4 2.16.0 0.15.0 1.15 / 2.0 n/a 0.15.1
0.15.3 2.16.0 0.15.0 1.15 / 2.0 n/a 0.15.1
0.15.2 2.16.0 0.15.0 1.15 / 2.0 n/a 0.15.1
0.15.1 2.16.0 0.15.0 1.15 / 2.0 n/a 0.15.0
0.15.0 2.16.0 0.15.0 1.15 n/a n/a
0.14.0 2.14.0 n/a 1.14 n/a n/a
0.13.1 2.11.0 n/a 1.13 n/a n/a
0.13.0 2.11.0 n/a 1.13 n/a n/a
0.12.1 2.10.0 n/a 1.12 n/a n/a
0.12.0 2.10.0 n/a 1.12 n/a n/a
0.11.0 2.8.0 n/a 1.11 n/a n/a
0.9.2 2.6.0 n/a 1.9 n/a n/a
0.9.1 2.6.0 n/a 1.10 n/a n/a
0.9.0 2.5.0 n/a 1.9 n/a n/a
0.6.0 2.4.0 n/a 1.6 n/a n/a

Questions

Please direct any questions about working with TFMA to Stack Overflow using the tensorflow-model-analysis tag.