Tune in to the first Women in ML Symposium this Tuesday, October 19 at 9am PST Register now


Runs TensorFlow model analysis.

Used in the notebooks

Used in the tutorials

It runs a Beam pipeline to compute the slicing metrics exported in TensorFlow Eval SavedModel and returns the results.

This is a simplified API for users who want to quickly get something running locally. Users who wish to create their own Beam pipelines can use the Evaluate PTransform instead.

eval_shared_model Optional shared model (single-model evaluation) or list of shared models (multi-model evaluation). Only required if needed by default extractors, evaluators, or writers.
eval_config Eval config.
data_location The location of the data files.
file_format The file format of the data, can be either 'text' or 'tfrecords' for now. By default, 'tfrecords' will be used.
output_path The directory to output metrics and results to. If None, we use a temporary directory.
extractors Optional list of Extractors to apply to Extracts. Typically these will be added by calling the default_extractors function. If no extractors are provided, default_extractors (non-materialized) will be used.
evaluators Optional list of Evaluators for evaluating Extracts. Typically these will be added by calling the default_evaluators function. If no evaluators are provided, default_evaluators will be used.
writers Optional list of Writers for writing Evaluation output. Typically these will be added by calling the default_writers function. If no writers are provided, default_writers will be used.
pipeline_options Optional arguments to run the Pipeline, for instance whether to run directly.
slice_spec Deprecated (use EvalConfig).
write_config Deprecated (use EvalConfig).
compute_confidence_intervals Deprecated (use EvalConfig).
min_slice_size Deprecated (use EvalConfig).
random_seed_for_testing Provide for deterministic tests only.
schema Optional tf.Metadata schema of the input data.

An EvalResult that can be used with the TFMA visualization functions.

ValueError If the file_format is unknown to us.