model_card_toolkit.ModelCardToolkit

ModelCardToolkit provides utilities to generate a ModelCard.

model_card_toolkit.ModelCardToolkit(
    output_dir: Optional[str] = None,
    mlmd_source: Optional[model_card_toolkit.MlmdSource] = None,
    source: Optional[model_card_toolkit.Source] = None
)

ModelCardToolkit is a tool for ML practitioners to create Model Cards, documentation for model information such as owners, use cases, training and evaluation data, performance, etc. A Model Card document can be displayed in output formats including HTML, Markdown, etc.

The ModelCardToolkit includes an API designed for a human-in-the-loop process to elaborate the ModelCard. If model training is integrated with ML Metadata (e.g., TFX pipelines), the ModelCardToolkit can further populate ModelCard fields by extract metadata and lineage from the model's MLMD instance.

The ModelCardToolkit organizes the ModelCard assets (e.g., structured data, plots, and UI templates) in a user-specified directory, and updates them incrementally via its API.

Standard workflow:

import model_card_toolkit

# Initialize the Model Card Toolkit with a path to store generate assets
model_card_dir_path = ...
mct = model_card_toolkit.ModelCardToolkit(model_card_dir_path)

# Initialize the ModelCard, which can be freely populated
model_card = mct.scaffold_assets()
model_card.model_details.name = 'My Model'

# Write the model card data to a proto file
mct.update_model_card(model_card)

# Return the model card document as an HTML page
html = mct.export_format()

output_dir The path where MCT assets (such as data files and model cards) are written to. If not provided, a temp directory is used.
mlmd_source The ML Metadata Store to retrieve metadata and lineage information about the model. If given, a set of model card properties can be auto-populated from the store.
source A collection of sources to extract data for a model card. This can be used instead of mlmd_source, or alongside it. Useful when using tools like TensorFlow Model Analysis and Data Validation without writing to a MLMD store.

ValueError If a model cannot be found at mlmd_source.model_uri.

Methods

export_format

View source

export_format(
    model_card: Optional[Union[model_card_toolkit.ModelCard, model_card_pb2.ModelCard]] = None,
    template_path: Optional[str] = None,
    output_file: Optional[str] = None
) -> str

Generates a model card document based on the MCT assets.

The model card document is both returned by this function, as well as saved to output_file.

Args
model_card The ModelCard object, generated from scaffold_assets(). If not provided, it will be read from the ModelCard proto file in the assets directory.
template_path The file path of the Jinja template. If not provided, the default template will be used.
output_file The file name of the generated model card. If not provided, the default 'model_card.html' will be used. If the file already exists, then it will be overwritten.

Returns
The model card file content.

Raises
ValueError If export_format is called before scaffold_assets has generated model card assets.

scaffold_assets

View source

scaffold_assets(
    json: Optional[Union[Dict[str, Any], str]] = None
) -> model_card_toolkit.ModelCard

Generates the Model Card Tookit assets.

Assets include the ModelCard proto file, Model Card document, and jinja template. These are written to the output_dir declared at initialization.

An assets directory is created if one does not already exist.

If the MCT is initialized with a mlmd_source, it further auto-populates ModelCard properties and generates plots for model performance and data distributions. The ModelCard is saved as an Artifact to the mlmd_source.

Args
json An optional JSON object which can be used to populate fields in the model card. This can be provided as either a dictionary or a string. If provided, any fields used here will overwrite fields populated by mlmd_source.

Returns
A ModelCard representing the given model.

Raises
FileNotFoundError on failure to copy the template files.

update_model_card

View source

update_model_card(
    model_card: Union[model_card_toolkit.ModelCard, model_card_pb2.ModelCard]
) -> None

Updates the Proto file in the MCT assets directory.

Args
model_card The updated model card to write back.

Raises
Error when the given model_card is invalid w.r.t. the schema.