DataLayouts

public final class DataLayouts

Exposes DataLayout instances of data formats frequently used in linear algebra computation.

Example of usage:

// Storing boolean values in a ByteDataBuffer
 BooleanDataBuffer boolBuffer = DataLayouts.BOOL.applyTo(byteDataBuffer);

 // Allocating a new buffer of 256 half floats
 FloatDataBuffer halfBuffer = DataLayouts.FLOAT16.applyTo(DataBuffers.ofShorts(256 * DataLayouts.FLOAT16.scale());

Fields

public static final FloatDataLayout<ShortDataBuffer>	BFLOAT16	Data layout for converting 16-bit bfloats to/from short values.
public static final BooleanDataLayout<ByteDataBuffer>	BOOL	Data layout for converting booleans to/from byte values.
public static final FloatDataLayout<ShortDataBuffer>	FLOAT16	Data layout for converting 16-bit half floats to/from short values.

Public Constructors

DataLayouts()

Public Methods

static DataLayout<DataBuffer<byte[]>, String>

ofStrings(Charset charset)

Creates a data layout for converting strings to/from byte sequences.

Inherited Methods

From class java.lang.Object

boolean	equals(Object arg0)
final Class<?>	getClass()
int	hashCode()
final void	notify()
final void	notifyAll()
String	toString()
final void	wait(long arg0, int arg1)
final void	wait(long arg0)
final void	wait()

Fields

public static final FloatDataLayout<ShortDataBuffer> BFLOAT16

Data layout for converting 16-bit bfloats to/from short values.

This format used to be specific to TensorFlow but has now been adopted more broadly in the machine learning field. It is optimized for fast conversion with single-precision 32-bit floating points by simply shifting their value and truncating the mantissa to only 7 bits.

Therefore, this is a lost of precision in the fraction part compared to the IEEE-754 half-precision floating point specification (see FLOAT16 but it has a larger range of possible values in the whole part as it preserves the 8-bit exponent and uses the same bias, (i.e. an absolute range above 0 of approximately [10^-40, 3.39 × 10³⁸]

Some CPUs support the bfloat16 format natively for better performances.

public static final BooleanDataLayout<ByteDataBuffer> BOOL

Data layout for converting booleans to/from byte values.

Since there is no Java NIO boolean buffer, this layout is particularly useful for mapping booleans values to standard byte buffers. The conversion between a boolean and a byte requires explicit type casting.

public static final FloatDataLayout<ShortDataBuffer> FLOAT16

Data layout for converting 16-bit half floats to/from short values.

Half floats are stored in memory accordingly to the IEEE-754 half-precision floating point specification, and are converted to/from 32-bit floats in the user space.

There is a potential loss of precision when converting a single float (32-bit) to a half float (16-bit). Absolute range of values above 0 for a half float is approximately [5.96 × 10^-8, 6.55 × 10⁴] and their decimal part is rounded up to a 10 bits mantissa.

In general, half float computation perform better on GPUs since, in general, CPUs do not support this format natively.

Public Constructors

public DataLayouts ()

Public Methods

public static DataLayout<DataBuffer<byte[]>, String> ofStrings (Charset charset)

Creates a data layout for converting strings to/from byte sequences.

This layout requires a charset in parameter to specify how the strings must be encoded/decoded as byte sequences. So a new layout instance is always returned.

Parameters

charset	charset to use

Returns

a new string layout