tfx.v1.components.ExampleValidator

A TFX component to validate input examples.

tfx.v1.components.ExampleValidator(
    statistics: types.BaseChannel,
    schema: types.BaseChannel,
    exclude_splits: Optional[List[str]] = None,
    custom_validation_config: Optional[custom_validation_config_pb2.CustomValidationConfig] = None
)

Used in the notebooks

Used in the tutorials

The ExampleValidator component uses Tensorflow Data Validation to validate the statistics of some splits on input examples against a schema.

The ExampleValidator component identifies anomalies in training and serving data. The component can be configured to detect different classes of anomalies in the data. It can:

  • perform validity checks by comparing data statistics against a schema that codifies expectations of the user.
  • run custom validations based on an optional SQL-based config.

Schema Based Example Validation The ExampleValidator component identifies any anomalies in the example data by comparing data statistics computed by the StatisticsGen component against a schema. The schema codifies properties which the input data is expected to satisfy, and is provided and maintained by the user.

Example

# Performs anomaly detection based on statistics and data schema.
validate_stats = ExampleValidator(
    statistics=statistics_gen.outputs['statistics'],
    schema=infer_schema.outputs['schema'])

Component outputs contains:

See the ExampleValidator guide for more details.

statistics A BaseChannel of type standard_artifacts.ExampleStatistics.
schema A BaseChannel of type standard_artifacts.Schema. required
exclude_splits Names of splits that the example validator should not validate. Default behavior (when exclude_splits is set to None) is excluding no splits.
custom_validation_config Optional configuration for specifying SQL-based custom validations.

outputs Component's output channel dict.