Class Confusables

java.lang.Object
opennlp.tools.util.normalizer.Confusables

public final class Confusables extends Object
Computes the Unicode confusable skeleton of text, following the skeleton algorithm defined in UTS #39 (Unicode Security Mechanisms). Two strings are confusable, for example Latin "paypal" and a version using Cyrillic lookalikes, exactly when their skeletons are equal.

The mapping is loaded on first use from the confusables.txt resource of the Unicode security data. If that resource is missing or cannot be read, every call that needs it throws IllegalStateException or UncheckedIOException. The skeleton of a string is NFD(map(NFD(s))): decompose, replace each code point with its prototype, and decompose again. This changes length and offsets, so it is a matching-only comparison form, not an offset-preserving transform.

This implements only the UTS #39 skeleton transform and the confusable-detection test built on skeleton equality. The other mechanisms defined in the report, such as identifier restriction levels, mixed-script and whole-script confusable detection, and the bidirectional skeleton, are out of scope.

  • Method Details

    • skeleton

      public static String skeleton(CharSequence text)
      Returns the confusable skeleton of text: NFD(map(NFD(text))) where map replaces each code point with its UTS #39 prototype. The skeleton is for comparison only; it is not human-readable text and does not preserve offsets.
      Parameters:
      text - The text to reduce.
      Returns:
      The skeleton.
      Throws:
      IllegalArgumentException - Thrown if text is null.
      IllegalStateException - Thrown if the bundled confusables.txt resource is missing.
      UncheckedIOException - Thrown if the bundled confusables.txt resource cannot be read.
    • confusable

      public static boolean confusable(CharSequence left, CharSequence right)
      Returns whether left and right are confusable. They are confusable when their skeletons are equal.
      Parameters:
      left - The first string.
      right - The second string.
      Returns:
      whether left and right are confusable
      Throws:
      IllegalArgumentException - Thrown if left or right is null.
      IllegalStateException - Thrown if the bundled confusables.txt resource is missing.
      UncheckedIOException - Thrown if the bundled confusables.txt resource cannot be read.