Module: tfx.components.example_gen.utils

Utilities for ExampleGen components.

Functions

calculate_splits_fingerprint_span_and_version(...): Calculates the fingerprint of files in a URI matching split patterns.

date_to_span_number(...): Calculated span from date as number of days since unix epoch.

dict_to_example(...): Converts dict to tf example.

generate_output_split_names(...): Return output split name based on input and output config.

make_default_input_config(...): Returns default input config.

make_default_output_config(...): Returns default output config based on input config.

span_number_to_date(...): Given a span number, convert it to the corresponding calendar date.

DATE_SPECS

DAY_SPEC '{DD}'
EXAMPLES_KEY 'examples'
FINGERPRINT_PROPERTY_NAME 'input_fingerprint'
INPUT_BASE_KEY 'input_base'
INPUT_CONFIG_KEY 'input_config'
MONTH_SPEC '{MM}'
OUTPUT_CONFIG_KEY 'output_config'
OUTPUT_DATA_FORMAT_KEY 'output_data_format'
PAYLOAD_FORMAT_PROPERTY_NAME 'payload_format'
RANGE_CONFIG_KEY 'range_config'
SPAN_FULL_REGEX '{SPAN}|{SPAN:(?P<width>.*?)}'
SPAN_PROPERTY_NAME 'span'
SPAN_SPEC '{SPAN}'
SPAN_SPEC_WIDTH_REGEX '{SPAN:(?P<width>.*?)}'
UNIX_EPOCH_DATE

VERSION_FULL_REGEX '{VERSION}|{VERSION:(?P<width>.*?)}'
VERSION_PROPERTY_NAME 'version'
VERSION_SPEC '{VERSION}'
VERSION_SPEC_WIDTH_REGEX '{VERSION:(?P<width>.*?)}'
YEAR_SPEC '{YYYY}'