tfds.core.SplitInfo

Wraps proto.SplitInfo with an additional property.

file_instructions Returns the list of dict(filename, take, skip).

This allows for creating your own tf.data.Dataset using the low-level TFDS values.

file_instructions = info.splits['train[75%:]'].file_instructions
instruction_ds = tf.data.Dataset.from_generator(
lambda: file_instructions,
output_types={
'filename': tf.string,
'take': tf.int64,
'skip': tf.int64,
},
)
ds = instruction_ds.interleave(
lambda f: tf.data.TFRecordDataset(
f['filename']).skip(f['skip']).take(f['take'])
)

When skip=0 and take=-1, the full shard will be read, so the ds.skip and ds.take could be skipped.

filenames Returns the list of filenames.
num_examples

num_shards

Methods

get_proto

View source

__eq__

View source