tfm.nlp.models.BertSpanLabeler

Span labeler model based on a BERT-style transformer-based encoder.

tfm.nlp.models.BertSpanLabeler(
    network,
    initializer='glorot_uniform',
    output='logits',
    **kwargs
)

This is an implementation of the network structure surrounding a transformer encoder as described in "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding" (https://arxiv.org/abs/1810.04805).

The BertSpanLabeler allows a user to pass in a transformer encoder, and instantiates a span labeling network based on a single dense layer.

Note: that the model is constructed by Keras Functional API.

Args
network	A transformer network. This network should output a sequence output and a classification output. Furthermore, it should expose its embedding table via a get_embedding_table method.
initializer	The initializer (if any) to use in the span labeling network. Defaults to a Glorot uniform initializer.
output	The output style for this network. Can be either logit' or predictions.

Attributes
checkpoint_items

Methods

call

call(
    inputs, training=None, mask=None
)

Calls the model on new inputs and returns the outputs as tensors.

In this case call() just reapplies all ops in the graph to the new inputs (e.g. build a new computational graph from the provided inputs).

Note: This method should not be called directly. It is only meant to be overridden when subclassing tf.keras.Model. To call a model on an input, always use the __call__() method, i.e. model(inputs), which relies on the underlying call() method.

Args
inputs	Input tensor, or dict/list/tuple of input tensors.
training	Boolean or boolean scalar tensor, indicating whether to run the Network in training mode or inference mode.
mask	A mask or list of masks. A mask can be either a boolean tensor or None (no mask). For more details, check the guide here.

Returns
A tensor if there is a single output, or a list of tensors if there are more than one outputs.