Value Lookup

Overview

The 'Vocabulary' property defines the set of allowed values for the lookup operation. Only values present in the vocabulary are considered valid and included in the output, unless the 'Allow Miss' option is enabled in 'Lookup Options'.

The vocabulary can be configured using local entries (one per line), by referencing one or more external Lexicons, or by including values from the parent field's value list. This provides flexibility for both simple and complex validation scenarios.

How It Works

• Each value is checked against the vocabulary after exclusions are applied.
• If the value is not found and 'Fuzzy Lookup' is enabled, the closest match above the minimum similarity threshold is used.
• If 'Translate' is enabled in 'Lookup Options', key-value pairs in the vocabulary are used to map input values to replacements.
• If 'Allow Miss' is enabled, values not found in the vocabulary are still included in the output.

Usage Scenarios

• Enforce a controlled list of valid codes, names, or terms for a Data Field.
• Normalize extracted values by mapping synonyms or abbreviations to a standard form.
• Correct minor spelling or OCR errors using fuzzy matching.

Best Practices

• Use local entries for small, field-specific lists.
• Reference external Lexicons for large or shared vocabularies.
• Regularly review and update the vocabulary to ensure accuracy and completeness.
• Avoid circular references when including other Lexicons.

Overview

The 'Exclusions' property defines a set of values that are always discarded, regardless of whether they appear in the vocabulary. This provides a way to filter out unwanted, invalid, or blacklisted values from the output.

How It Works

• Each value is first checked against the exclusions list.
• If the value appears in the exclusions, it is discarded and not processed further.
• Exclusions take precedence over the vocabulary and all other options.

Usage Scenarios

• Discard placeholder values such as "N/A", "Unknown", or "None".
• Filter out blacklisted or prohibited terms.
• Prevent specific values from being included in the output, even if they are otherwise valid.

Best Practices

• Use local entries for a short list of exclusions.
• Reference external Lexicons for shared or frequently updated exclusion lists.
• Keep the exclusions list focused on truly unwanted values to avoid discarding valid data.

Can be one of the following types:

Overview

The Fuzzy Lookup Options class defines how fuzzy matching is performed when a value is not found by exact match in the vocabulary during a Value Lookup operation. Fuzzy lookup is a secondary search that can automatically resolve minor differences caused by OCR inaccuracy, spelling mistakes, or inflected word forms, improving data quality and extraction robustness.

Fuzzy matching uses edit distance algorithms (such as Levenshtein distance) to compare the input value to entries in the vocabulary, scoring each candidate by similarity. The closest match above the configured similarity threshold is selected, optionally using custom weightings to account for common OCR or typographical errors.

How Fuzzy Lookup Works

1. If a value is not found in the vocabulary by exact match, fuzzy lookup is attempted (if enabled).
2. Each candidate in the vocabulary is compared to the input value using edit distance.
3. The best match above the 'MinimumSimilarity' threshold is selected as the replacement.
4. Custom weightings (via the 'Weightings' property) can adjust the cost of specific character substitutions, insertions, or deletions.
5. Optionally, fuzzy matching can be limited to the top N entries or to an alternate vocabulary for performance.

Usage Scenarios

• Correcting common OCR errors (e.g., "1NVOICE" → "INVOICE", "O" ↔ "0", "B" ↔ "8").
• Handling misspellings or inflected forms in user-entered or scanned data.
• Improving extraction accuracy in noisy or low-quality documents.

Best Practices

• Test fuzzy lookup settings with representative data to ensure desired correction and avoid over-matching.
• Regularly review and update weightings and alternate vocabularies as data quality or requirements change.
• Use in combination with exclusions and translation options in Value Lookup for comprehensive value normalization.

A combination of the following flags:

Overview

The LookupOption enum defines flags that modify how Value Lookup processes and outputs values. These options allow you to control whether values must match the vocabulary, how punctuation and case are handled, and whether translation or normalization is performed.

Usage

Combine options as needed to achieve strict validation, flexible normalization, or custom output formatting. These flags are typically set via the 'Lookup Options' property on a Value Lookup object.

Available Options

• None: No special options are enabled; only exact matches in the vocabulary are accepted.
• AllowMiss: Values not found in the vocabulary are still included in the output, allowing for fuzzy repair or translation without strict filtering.
• CleanKey: All punctuation and control characters are removed from values before lookup, enabling matches that ignore formatting differences.
• UseListCase: Output values reflect the case of the vocabulary entry instead of the document value, ensuring standardized casing.
• Translate: Output values are replaced with mapped values from the vocabulary when configured as a 'Lookup' lexicon (key-value pairs).

Best Practices

• Use 'None' for strict validation.
• Enable 'AllowMiss' for normalization or correction scenarios.
• Use 'CleanKey' to handle values with inconsistent punctuation.
• Enable 'Translate' to map synonyms, abbreviations, or codes to standardized forms.
• Combine options as needed for your use case.