|View source on GitHub|
Utility class for saving/loading experiment metrics via a CSV file.
tff.simulation.CSVMetricsManager( csv_filepath: str )
||A string specifying the file to write and read metrics from.|
If the file at
The last round number passed to
If not metrics have been written, this will be
clear_metrics( round_num: int ) -> None
Clear out metrics at and after a given starting
By using this method, this class can be used upon restart of an experiment
round_num to ensure that no duplicate rows of data exist in
the CSV file. This method will atomically update the stored CSV file.
Note that if
clear_metrics(round_num=0) is called, all metrics are cleared
in a more performant manner. Rather than removing all rows of the CSV with
round numbers greater than or equal to
round_num, we simply remove all
rows of the CSV when
||A nonnegative integer indicating the starting round number for clearing metrics from the manager's associated CSV.|
get_metrics() -> Tuple[Sequence[str], List[Dict[str, Any]]]
Retrieve the stored experiment metrics data for all rounds.
A sequence representing all possible keys for the metrics, and a list
containing experiment metrics data for all rounds. Each entry in the list
is a dictionary corresponding to a given round. The data has been
flattened, with the column names equal to the path in the original nested
metric structure. There is a fieldname
save_metrics( metrics: Mapping[str, Any], round_num: int ) -> None
Updates the stored metrics data with metrics for a specific round.
round_num must be nonnegative, and larger than the latest
round number for which metrics exist in the stored metrics data. For
example, calling this method with
round_num = 3 then
round_num = 7
is acceptable, but calling the method with
round_num = 6 then
round_num = 6 (or anything less than 6) is not supported.
This method will atomically update the stored CSV file. Also, if stored
metrics already exist and
metrics contains a new, previously unseen metric
name, a new column in the dataframe will be added for that metric, and all
previous rows will fill in with NaN values for the metric.
The metrics written are the leaf node tensors of the metrics_to_append
structure. Purely scalar tensors will be written as scalars in the CSV,
while tensors with non-zero rank will be written as a list of lists. For
example, the tensor
tf.ones([2, 2]) will be written to the CSV as
'[[1.0, 1.0], [1.0, 1.0]'.
A nested structure of metrics collected during
Communication round at which
||If the provided round number is negative.|
||If the provided round number is less than or equal to the latest round number in the stored metrics data.|