Class NormalizationProfile
SnowballStemmer.ALGORITHM and the
diacritic fold appropriate for that language (if any).
The accentFold normalizer is the language's diacritic transform for a matching form, or
null when folding is not appropriate. In the profiles of NormalizationProfiles it
is the generic AccentFoldCharSequenceNormalizer for English, Catalan, French, Italian,
Portuguese and Spanish (where accented letters are matching variants of their base letter), the
German-specific GermanUmlautCharSequenceNormalizer (a-umlaut to ae, eszett to
ss, ...) for German, and null for every other language, including those where
diacritics mark distinct letters (such as the Nordic languages) and the non-Latin scripts. This
is a search-recall choice, not a statement of linguistic correctness; callers can build a
TermAnalyzer directly to override it.
A profile is immutable and thread-safe. Two profiles are equal when their language, stemmer algorithm and diacritic fold are equal.
-
Constructor Summary
ConstructorsConstructorDescriptionNormalizationProfile(String language, SnowballStemmer.ALGORITHM stemmerAlgorithm, opennlp.tools.util.normalizer.CharSequenceNormalizer accentFold) Creates a profile. -
Method Summary
Modifier and TypeMethodDescriptionopennlp.tools.util.normalizer.CharSequenceNormalizerReturns the diacritic fold for the language, ornullfor none.booleaninthashCode()language()Returns the language, as an ISO 639-3 code.Returns the matching analyzer for this language: NFC, case folding, the language's diacritic fold when it has one, then stemming.opennlp.tools.stemmer.StemmerReturns a newStemmerfor this language.Returns the Snowball algorithm for the language.opennlp.tools.stemmer.StemmerFactoryReturns a thread-safe factory for this language's Snowball stemmer.toString()
-
Constructor Details
-
NormalizationProfile
public NormalizationProfile(String language, SnowballStemmer.ALGORITHM stemmerAlgorithm, opennlp.tools.util.normalizer.CharSequenceNormalizer accentFold) Creates a profile.- Parameters:
language- The language, as an ISO 639-3 code (for example"eng"). Must not benullor blank.stemmerAlgorithm- The Snowball algorithm for the language. Must not benull.accentFold- The diacritic fold for the language, ornullfor none.- Throws:
IllegalArgumentException- iflanguageorstemmerAlgorithmisnull, or iflanguageis blank.
-
-
Method Details
-
language
Returns the language, as an ISO 639-3 code.- Returns:
- the language, as an ISO 639-3 code
-
stemmerAlgorithm
Returns the Snowball algorithm for the language.- Returns:
- the Snowball algorithm for the language
-
accentFold
public opennlp.tools.util.normalizer.CharSequenceNormalizer accentFold()Returns the diacritic fold for the language, ornullfor none.- Returns:
- the diacritic fold for the language, or
nullfor none
-
stemmerFactory
public opennlp.tools.stemmer.StemmerFactory stemmerFactory()Returns a thread-safe factory for this language's Snowball stemmer. The factory may be shared; each minted stemmer is confined to the calling thread.- Returns:
- a thread-safe factory for this language's Snowball stemmer
-
newStemmer
public opennlp.tools.stemmer.Stemmer newStemmer()Returns a newStemmerfor this language. The returnedSnowballStemmeris thread-safe and may be shared across threads.- Returns:
- a new
Stemmerfor this language
-
matchingAnalyzer
Returns the matching analyzer for this language: NFC, case folding, the language's diacritic fold when it has one, then stemming. The analyzer is built on the first call, and every call on this profile returns the same instance. It is thread-safe, and repeated words resolve from a bounded per-thread stem cache instead of being re-stemmed; the cache pays off on threads that live across many calls, such as a fixed platform-thread pool. CallTermAnalyzer.clearThreadLocalState()when a pooled thread no longer uses the analyzer.- Returns:
- the analyzer.
-
equals
-
hashCode
-
toString
-