Class NormalizationProfile

java.lang.Object
opennlp.tools.util.normalizer.NormalizationProfile

public final class NormalizationProfile extends Object
Per-language normalization settings, mirroring how OpenNLP already selects a Snowball stemmer by language. A profile pairs a language with its Snowball SnowballStemmer.ALGORITHM and the diacritic fold appropriate for that language (if any).

The accentFold normalizer is the language's diacritic transform for a matching form, or null when folding is not appropriate. In the profiles of NormalizationProfiles it is the generic AccentFoldCharSequenceNormalizer for English, Catalan, French, Italian, Portuguese and Spanish (where accented letters are matching variants of their base letter), the German-specific GermanUmlautCharSequenceNormalizer (a-umlaut to ae, eszett to ss, ...) for German, and null for every other language, including those where diacritics mark distinct letters (such as the Nordic languages) and the non-Latin scripts. This is a search-recall choice, not a statement of linguistic correctness; callers can build a TermAnalyzer directly to override it.

A profile is immutable and thread-safe. Two profiles are equal when their language, stemmer algorithm and diacritic fold are equal.

  • Constructor Details

    • NormalizationProfile

      public NormalizationProfile(String language, SnowballStemmer.ALGORITHM stemmerAlgorithm, opennlp.tools.util.normalizer.CharSequenceNormalizer accentFold)
      Creates a profile.
      Parameters:
      language - The language, as an ISO 639-3 code (for example "eng"). Must not be null or blank.
      stemmerAlgorithm - The Snowball algorithm for the language. Must not be null.
      accentFold - The diacritic fold for the language, or null for none.
      Throws:
      IllegalArgumentException - if language or stemmerAlgorithm is null, or if language is blank.
  • Method Details

    • language

      public String language()
      Returns the language, as an ISO 639-3 code.
      Returns:
      the language, as an ISO 639-3 code
    • stemmerAlgorithm

      public SnowballStemmer.ALGORITHM stemmerAlgorithm()
      Returns the Snowball algorithm for the language.
      Returns:
      the Snowball algorithm for the language
    • accentFold

      public opennlp.tools.util.normalizer.CharSequenceNormalizer accentFold()
      Returns the diacritic fold for the language, or null for none.
      Returns:
      the diacritic fold for the language, or null for none
    • stemmerFactory

      public opennlp.tools.stemmer.StemmerFactory stemmerFactory()
      Returns a thread-safe factory for this language's Snowball stemmer. The factory may be shared; each minted stemmer is confined to the calling thread.
      Returns:
      a thread-safe factory for this language's Snowball stemmer
    • newStemmer

      public opennlp.tools.stemmer.Stemmer newStemmer()
      Returns a new Stemmer for this language. The returned SnowballStemmer is thread-safe and may be shared across threads.
      Returns:
      a new Stemmer for this language
    • matchingAnalyzer

      public TermAnalyzer matchingAnalyzer()
      Returns the matching analyzer for this language: NFC, case folding, the language's diacritic fold when it has one, then stemming. The analyzer is built on the first call, and every call on this profile returns the same instance. It is thread-safe, and repeated words resolve from a bounded per-thread stem cache instead of being re-stemmed; the cache pays off on threads that live across many calls, such as a fixed platform-thread pool. Call TermAnalyzer.clearThreadLocalState() when a pooled thread no longer uses the analyzer.
      Returns:
      the analyzer.
    • equals

      public boolean equals(Object obj)
      Overrides:
      equals in class Object
    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object
    • toString

      public String toString()
      Overrides:
      toString in class Object