tfx.orchestration.portable.cache_utils.get_cache_context

Gets cache context for a potential node execution.

tfx.orchestration.portable.cache_utils.get_cache_context(
    metadata_handler: tfx.orchestration.metadata.Metadata,
    pipeline_node: pipeline_pb2.PipelineNode,
    pipeline_info: pipeline_pb2.PipelineInfo,
    input_artifacts: Optional[Mapping[Text, Sequence[types.Artifact]]] = None,
    output_artifacts: Optional[Mapping[Text, Sequence[types.Artifact]]] = None,
    parameters: Optional[Mapping[Text, Any]] = None
) -> metadata_store_pb2.Context

The cache key is generated by applying SHA-256 hashing function on:

  • Serialized pipeline info.
  • Serialized ExecutorSpec of the node.
  • Serialized input artifacts if any.
  • Serialized output artifacts if any. The uri was removed during the process.
  • Serialized parameters if any.
  • Serialized module file content if module file is present in parameters.

metadata_handler A handler to access MLMD store.
pipeline_node A pipeline_pb2.PipelineNode instance to represent the node.
pipeline_info Information of the pipeline.
input_artifacts Input artifacts of the potential execution. The order of the artifacts under a key matters when calculating the cache key.
output_artifacts Output artifacts skeleton of the potential execution. The order of the artifadts under a key matters when calculating the cache key.
parameters Parameters of the potential execution.

A metadata_store_pb2.Context for the cache key.