Class DependencyCrossValidator
java.lang.Object
opennlp.tools.depparse.DependencyCrossValidator
Cross validator for
DependencyParserME.
The samples are split into k folds. For each fold a parser is trained with
DependencyParserME.train(String, ObjectStream, TrainingParameters) on the other k - 1 folds and scored on the
held-out fold with a DependencyEvaluator. The unlabeled and labeled attachment
scores are accumulated over all evaluated tokens, so every token of the input counts
once, in the fold it was held out from. The listeners given at construction are told
about each held-out sample.
- Since:
- 3.0.0
- See Also:
-
Constructor Summary
ConstructorsConstructorDescriptionDependencyCrossValidator(String languageCode, opennlp.tools.util.TrainingParameters params, Predicate<String> punctuationTag, opennlp.tools.depparse.DependencyEvaluationMonitor... listeners) Initializes aDependencyCrossValidatorwith a custom notion of punctuation.DependencyCrossValidator(String languageCode, opennlp.tools.util.TrainingParameters params, opennlp.tools.depparse.DependencyEvaluationMonitor... listeners) Initializes aDependencyCrossValidatorthat treats tokens taggedDependencyEvaluator.UNIVERSAL_PUNCTUATION_TAGas punctuation. -
Method Summary
-
Constructor Details
-
DependencyCrossValidator
public DependencyCrossValidator(String languageCode, opennlp.tools.util.TrainingParameters params, opennlp.tools.depparse.DependencyEvaluationMonitor... listeners) Initializes aDependencyCrossValidatorthat treats tokens taggedDependencyEvaluator.UNIVERSAL_PUNCTUATION_TAGas punctuation.- Parameters:
languageCode- The ISO language code of the samples. Must not benull.params- TheTrainingParametersfor the parser of each fold. Must not benull.listeners- Thelistenerstold about each held-out sample;nullentries are ignored.- Throws:
IllegalArgumentException- Thrown iflanguageCodeorparamsisnull.
-
DependencyCrossValidator
public DependencyCrossValidator(String languageCode, opennlp.tools.util.TrainingParameters params, Predicate<String> punctuationTag, opennlp.tools.depparse.DependencyEvaluationMonitor... listeners) Initializes aDependencyCrossValidatorwith a custom notion of punctuation.- Parameters:
languageCode- The ISO language code of the samples. Must not benull.params- TheTrainingParametersfor the parser of each fold. Must not benull.punctuationTag- Decides from a gold part-of-speech tag whether the token is punctuation and therefore left out of the punctuation-free scores. Must not benull.listeners- Thelistenerstold about each held-out sample;nullentries are ignored.- Throws:
IllegalArgumentException- Thrown iflanguageCode,paramsorpunctuationTagisnull.
-
-
Method Details
-
evaluate
public void evaluate(opennlp.tools.util.ObjectStream<opennlp.tools.depparse.DependencySample> samples, int folds) throws IOException Runs the cross validation and adds the scores of every fold to the totals.- Parameters:
samples- The samples to train and test with. Must not benulland must supportObjectStream.reset(), because every fold reads the stream from the start. The stream is not closed.folds- The number of folds. Must be at least2.- Throws:
IOException- Thrown if reading the samples fails.IllegalArgumentException- Thrown ifsamplesisnull,foldsis below2, or the training parameters do not select an event model trainer.
-
getUas
public double getUas()- Returns:
- The unlabeled attachment score over all tokens evaluated so far.
-
getLas
public double getLas()- Returns:
- The labeled attachment score over all tokens evaluated so far.
-
getWordCount
public long getWordCount()- Returns:
- The number of tokens evaluated so far; over one complete run this is the number of tokens in the samples, because every token is held out once.
-
getUasExcludingPunctuation
public double getUasExcludingPunctuation()- Returns:
- The unlabeled attachment score over the evaluated tokens that are not punctuation.
-
getLasExcludingPunctuation
public double getLasExcludingPunctuation()- Returns:
- The labeled attachment score over the evaluated tokens that are not punctuation.
-
getWordCountExcludingPunctuation
public long getWordCountExcludingPunctuation()- Returns:
- The number of evaluated tokens that are not punctuation.
-