tfx.components.StatisticsGen

Official TFX StatisticsGen component.

Inherits From: BaseBeamComponent, BaseComponent, BaseNode

View aliases

Main aliases

tfx.components.statistics_gen.component.StatisticsGen

tfx.components.StatisticsGen(
    examples: tfx.types.Channel = None,
    schema: Optional[tfx.types.Channel] = None,
    stats_options: Optional[tfdv.StatsOptions] = None,
    exclude_splits: Optional[List[Text]] = None
)

Used in the notebooks

Used in the tutorials
TFX Estimator Component Tutorial Graph-based Neural Structured Learning in TFX Fairness Indicators Lineage Case Study MLMD Model Card Toolkit Demo

The StatisticsGen component generates features statistics and random samples over training data, which can be used for visualization and validation. StatisticsGen uses Apache Beam and approximate algorithms to scale to large datasets.

Please see https://www.tensorflow.org/tfx/data_validation for more details.

Example

  # Computes statistics over data for visualization and example validation.
  statistics_gen = StatisticsGen(examples=example_gen.outputs['examples'])

Component outputs contains:

• statistics: Channel of type standard_artifacts.ExampleStatistics for statistics of each split provided in the input examples.

Args
examples	A Channel of ExamplesPath type, likely generated by the ExampleGen component. This needs to contain two splits labeled train and eval. required
schema	A Schema channel to use for automatically configuring the value of stats options passed to TFDV.
stats_options	The StatsOptions instance to configure optional TFDV behavior. When stats_options.schema is set, it will be used instead of the schema channel input. Due to the requirement that stats_options be serialized, the slicer functions and custom stats generators are dropped and are therefore not usable.
exclude_splits	Names of splits where statistics and sample should not be generated. Default behavior (when exclude_splits is set to None) is excluding no splits.

Attributes
outputs	Component's output channel dict.

Methods

with_beam_pipeline_args

View source

with_beam_pipeline_args(
    beam_pipeline_args: Iterable[str]
) -> "BaseBeamComponent"

Add per component Beam pipeline args.

Args
beam_pipeline_args	List of Beam pipeline args to be added to the Beam executor spec.

Returns
the same component itself.