|View source on GitHub|
Abstract interface for data backends.
Used in the notebooks
|Used in the tutorials|
A data backend is a component that can resolve symbolic references to data
as URIs, and locally materialize the associated payloads. Data backends are
used in tandem with the
data_executor that queries them as it encounters
data building block.
materialize( data, type_spec )
data with the given
The form of the materialized payload must be such that it can be understood
by the downstream components of the executor stack. For example, if the data
backend is plugged into a stack based on an eager TensorFlow executor, the
accepted forms of payload would include Numpy-like objects, tensors, as well
as instances of eager
tf.data.Dataset, and structures thereof. It is the
responsibility of the code that constructs the executor stack with the given
data backend to ensure that the types of payload materialized are compatible
with what the downstream components of the executor stack can accept.
A symbolic reference to the data to be materialized locally. Must be
an instance of
An instance of
|The materialized payload.|