Class SentenceVectorsDL

java.lang.Object
opennlp.dl.AbstractDL
opennlp.dl.vectors.SentenceVectorsDL
All Implemented Interfaces:
Closeable, AutoCloseable, opennlp.tools.embeddings.TextEmbedder

@ThreadSafe public class SentenceVectorsDL extends AbstractDL implements opennlp.tools.embeddings.TextEmbedder
Facilitates the generation of sentence vectors using a sentence-transformers model converted to ONNX.

The model inputs follow the standard single-segment BERT encoding: attention_mask is 1 for every real token and token_type_ids is 0 throughout.

The sentence vector is read from a sentence_embedding output of shape [batch, hidden] if the model has one. Otherwise the token vectors of the first [batch, tokens, hidden] output are pooled with the configured Pooling. By default the token vectors are averaged and the result is scaled to unit length, as sentence-transformers does for the MiniLM family. Text longer than the maximum length is truncated, keeping the final [SEP] token.

Release note (OpenNLP 3.0.0): prior releases sent an all-zero attention_mask and all-one token_type_ids, so the encoder attended to nothing and the output vectors were incorrect, and they returned the raw [CLS] vector instead of the pooled sentence vector. Additionally, tokenization now performs BERT basic tokenization (lower casing and accent stripping by default, see WordpieceEncoder) before wordpiece. Output vectors change with the corrected encoding, tokenization and pooling; any embeddings persisted from the previous behavior are not comparable with the corrected output and must be re-embedded.

This class is thread-safe and may be shared across threads: the inference methods hold no per-call instance state and the underlying OrtSession supports concurrent execution. This thread-safety guarantee applies until close() is called; callers must not race close() with inference methods.

embedAll(List) batches its texts according to the configured PaddingStrategy. By default it does not pad on the CPU and runs one inference per distinct tokenized length. On a GPU it pads to the longest text of a batch, which runs texts of mixed lengths together, except for a model with a sentence_embedding output, which is not padded by default because such a model may ignore the attention mask. No inference exceeds 16384 token positions, except that a single text whose encoding is wider runs as an inference of its own.

  • Field Details

    • DEFAULT_MAX_LENGTH

      public static final int DEFAULT_MAX_LENGTH
      The maximum number of tokens per input if none is given, the BERT position limit.
      See Also:
    • DEFAULT_CPU_PADDING

      public static final PaddingStrategy DEFAULT_CPU_PADDING
      The PaddingStrategy on the CPU, or on a GPU for a pooled output, if none is given.
    • DEFAULT_GPU_PADDING

      public static final PaddingStrategy DEFAULT_GPU_PADDING
      The PaddingStrategy on a GPU if none is given and the model has no pooled output, so padding is masked out of the pooling, and the vocabulary has a padding token. A GPU runs one wide inference faster than many narrow ones.
  • Constructor Details

    • SentenceVectorsDL

      public SentenceVectorsDL(File model, File vocabulary) throws ai.onnxruntime.OrtException, IOException
      Instantiates a sentence vector generator for an uncased model. Input text is lower cased and accent stripped during tokenization, as required by uncased models such as the sentence-transformers MiniLM family.
      Parameters:
      model - The file name of a sentence vectors ONNX model.
      vocabulary - The file name of the vocabulary file for the model.
      Throws:
      ai.onnxruntime.OrtException - Thrown if the model cannot be loaded.
      IOException - Thrown if errors occurred loading the model or vocabulary.
      opennlp.tools.util.InvalidFormatException - Thrown if a JSON vocabulary is malformed, has an unsupported layout, or sets a lower casing that disagrees with the lower casing the component is configured with.
    • SentenceVectorsDL

      public SentenceVectorsDL(File model, File vocabulary, boolean lowerCase) throws ai.onnxruntime.OrtException, IOException
      Instantiates a sentence vector generator that averages the token vectors, scales the result to unit length and truncates input to 512 tokens.
      Parameters:
      model - The file name of a sentence vectors ONNX model.
      vocabulary - The file name of the vocabulary file for the model.
      lowerCase - true for uncased models (lower casing and accent stripping during tokenization), false for cased models.
      Throws:
      ai.onnxruntime.OrtException - Thrown if the model cannot be loaded.
      IOException - Thrown if errors occurred loading the model or vocabulary.
      opennlp.tools.util.InvalidFormatException - Thrown if a JSON vocabulary is malformed, has an unsupported layout, or sets a lower casing that disagrees with the lower casing the component is configured with.
    • SentenceVectorsDL

      public SentenceVectorsDL(File model, File vocabulary, boolean lowerCase, Pooling pooling, boolean normalize, int maxLength) throws ai.onnxruntime.OrtException, IOException
      Instantiates a sentence vector generator using ONNX models.
      Parameters:
      model - The file name of a sentence vectors ONNX model.
      vocabulary - The file name of the vocabulary file for the model.
      lowerCase - true for uncased models (lower casing and accent stripping during tokenization), false for cased models.
      pooling - How token vectors are pooled. Not used for a model with a sentence_embedding output. Must not be null.
      normalize - true to scale every vector to unit length.
      maxLength - The maximum number of tokens per input, [CLS] and [SEP] included; longer input is truncated. Must be at least 2.
      Throws:
      IllegalArgumentException - Thrown if pooling is null, if maxLength is less than 2, or if the model has no output of shape [batch, hidden] or [batch, tokens, hidden].
      ai.onnxruntime.OrtException - Thrown if the model cannot be loaded.
      IOException - Thrown if errors occurred loading the model or vocabulary.
      opennlp.tools.util.InvalidFormatException - Thrown if a JSON vocabulary is malformed, has an unsupported layout, or sets a lower casing that disagrees with the lower casing the component is configured with.
    • SentenceVectorsDL

      public SentenceVectorsDL(File model, File vocabulary, boolean lowerCase, SentenceVectorsOptions vectorOptions, InferenceOptions inferenceOptions) throws ai.onnxruntime.OrtException, IOException
      Instantiates a sentence vector generator with vector settings and ONNX session settings. With PaddingStrategy.MAX_LENGTH, a probe for an unknown output dimension also runs at the configured maximum length and can fail if it exceeds the model's position limit.
      Parameters:
      model - The file name of a sentence vectors ONNX model.
      vocabulary - The file name of the vocabulary file for the model.
      lowerCase - true for uncased models, false for cased models. A lower case setting in inferenceOptions takes precedence.
      vectorOptions - The pooling, normalization, truncation and padding settings. Must not be null. PaddingStrategy.AUTO selects the device and model default.
      inferenceOptions - The ONNX session settings. Must not be null.
      Throws:
      IllegalArgumentException - Thrown if vectorOptions or inferenceOptions is null, if the inference options contain invalid split settings or a negative GPU device id, if the model has no output of shape [batch, hidden] or [batch, tokens, hidden], or if padding is requested without a vocabulary padding token.
      ai.onnxruntime.OrtException - Thrown if the model cannot be loaded, the output dimension probe fails, or the requested GPU is unavailable. ONNX Runtime does not fall back to the CPU.
      IOException - Thrown if the model or vocabulary cannot be read.
      opennlp.tools.util.InvalidFormatException - Thrown if a JSON vocabulary is malformed, has an unsupported layout, or disagrees with the configured lower casing.
      Since:
      3.0.0
  • Method Details

    • getVectors

      public float[] getVectors(String sentence) throws ai.onnxruntime.OrtException
      Generates vectors given a sentence.
      Parameters:
      sentence - The input sentence.
      Returns:
      The sentence vector.
      Throws:
      IllegalArgumentException - Thrown if sentence is null.
      ai.onnxruntime.OrtException - Thrown if an error occurs during inference.
    • embed

      public float[] embed(CharSequence text)

      Empty or unrecognized input is still run through the model as the wrapped [CLS] [SEP] sequence rather than returning a zero vector.

      Specified by:
      embed in interface opennlp.tools.embeddings.TextEmbedder
    • embedAll

      public float[][] embedAll(List<? extends CharSequence> texts)

      The inputs are tokenized up front and batched according to the configured PaddingStrategy, so that no inference exceeds 16384 token positions; a single text whose encoding is wider runs alone. Without padding, every row is computed from the tensors its single-input call would have used. A padded position is 0 in the attention mask, so vector i matches what embed(CharSequence) returns for input i up to floating-point rounding, provided the model honors the attention mask. A sentence_embedding output that ignores the mask changes with padding.

      Specified by:
      embedAll in interface opennlp.tools.embeddings.TextEmbedder
    • dimension

      public int dimension()

      Read from the model's declared output shape, or from one inference at construction if the model declares the hidden dimension dynamically.

      Specified by:
      dimension in interface opennlp.tools.embeddings.TextEmbedder
    • close

      public void close()
      Closes the ONNX session. An OrtException from the session is rethrown as the cause of an UncheckedIOException.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Specified by:
      close in interface opennlp.tools.embeddings.TextEmbedder
      Overrides:
      close in class AbstractDL