tfm.nlp.networks.PackedSequenceEmbedding

An embedding network supporting packed sequences and position ids.

tfm.nlp.networks.PackedSequenceEmbedding(
    vocab_size,
    type_vocab_size,
    embedding_width,
    hidden_size,
    max_seq_length,
    initializer,
    dropout_rate,
    use_position_id=False,
    pack_multiple_sequences=False,
    **kwargs
)

This network implements an embedding layer similar to the one described in "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" (https://arxiv.org/abs/1810.04805). On top of it, it supports to (1) pack multiple sequences into one sequence and (2) allow additional "position_ids" as input.

Args
vocab_size	The size of the token vocabulary.
type_vocab_size	The size of the type vocabulary.
embedding_width	Width of token embeddings.
hidden_size	The output size for this encoder.
max_seq_length	The maximum sequence length for this encoder.
initializer	The initializer for the embedding portion of this encoder.
dropout_rate	The dropout rate to apply before the encoding layers.
pack_multiple_sequences	If True, we can feed multiple sequences into one sequence for training and inference (they don't impact each other).
use_position_id	Whether to expect position_ids as an input to the network. If False, the position_ids will be inferred: (1) when pack_multiple_sequences is False, we assume the position ids are 0, 1, 2, ..., seq_length - 1; (2) when pack_multiple_sequences is True, there may be multiple sub sequences, and for each sub sequence, its position ids start from 0, 1, 2, ...

Methods

call

call(
    inputs, training=None, mask=None
)

Calls the model on new inputs and returns the outputs as tensors.

In this case call() just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs).

Note: This method should not be called directly. It is only meant to be overridden when subclassing tf.keras.Model. To call a model on an input, always use the __call__() method, i.e. model(inputs), which relies on the underlying call() method.

Args
inputs	Input tensor, or dict/list/tuple of input tensors.
training	Boolean or boolean scalar tensor, indicating whether to run the Network in training mode or inference mode.
mask	A mask or list of masks. A mask can be either a boolean tensor or None (no mask). For more details, check the guide here.

Returns
A tensor if there is a single output, or a list of tensors if there are more than one outputs.

get_embedding_table

View source

get_embedding_table()