Class DependencyEvaluator

java.lang.Object
opennlp.tools.util.eval.Evaluator<opennlp.tools.depparse.DependencySample>
opennlp.tools.depparse.DependencyEvaluator

public class DependencyEvaluator extends Evaluator<opennlp.tools.depparse.DependencySample>
Measures the quality of a DependencyParser against gold samples with the two standard scores: the unlabeled attachment score (UAS, the fraction of tokens with the correct head) and the labeled attachment score (LAS, the fraction of tokens with the correct head and relation label).

Both scores are also kept over the tokens that are not punctuation, because treebank evaluations customarily leave punctuation out of the attachment scores. A token counts as punctuation when its gold part-of-speech tag satisfies the predicate given at construction; by default that is the universal tag UNIVERSAL_PUNCTUATION_TAG.

Since:
3.0.0
  • Field Details

    • UNIVERSAL_PUNCTUATION_TAG

      public static final String UNIVERSAL_PUNCTUATION_TAG
      The universal part-of-speech tag of punctuation, which the default constructor excludes from the punctuation-free scores.
      See Also:
  • Constructor Details

    • DependencyEvaluator

      public DependencyEvaluator(opennlp.tools.depparse.DependencyParser parser, opennlp.tools.depparse.DependencyEvaluationMonitor... listeners)
      Initializes a DependencyEvaluator that treats tokens tagged UNIVERSAL_PUNCTUATION_TAG as punctuation.
      Parameters:
      parser - The parser to evaluate. Must not be null.
      listeners - The listeners told about each evaluated sample; null entries are ignored.
      Throws:
      IllegalArgumentException - Thrown if parser is null.
    • DependencyEvaluator

      public DependencyEvaluator(opennlp.tools.depparse.DependencyParser parser, Predicate<String> punctuationTag, opennlp.tools.depparse.DependencyEvaluationMonitor... listeners)
      Initializes a DependencyEvaluator with a custom notion of punctuation.
      Parameters:
      parser - The parser to evaluate. Must not be null.
      punctuationTag - Decides from a gold part-of-speech tag whether the token is punctuation and therefore left out of the punctuation-free scores. Must not be null.
      listeners - The listeners told about each evaluated sample; null entries are ignored.
      Throws:
      IllegalArgumentException - Thrown if parser or punctuationTag is null.
  • Method Details

    • getUas

      public double getUas()
      Returns:
      The unlabeled attachment score over all evaluated tokens.
    • getLas

      public double getLas()
      Returns:
      The labeled attachment score over all evaluated tokens.
    • getWordCount

      public long getWordCount()
      Returns:
      The number of tokens scored so far.
    • getUasExcludingPunctuation

      public double getUasExcludingPunctuation()
      Returns:
      The unlabeled attachment score over the evaluated tokens that are not punctuation.
    • getLasExcludingPunctuation

      public double getLasExcludingPunctuation()
      Returns:
      The labeled attachment score over the evaluated tokens that are not punctuation.
    • getWordCountExcludingPunctuation

      public long getWordCountExcludingPunctuation()
      Returns:
      The number of tokens scored so far that are not punctuation.