TfModes

Specifies how the term frequency (TF) of features should be calculated for classification models.

Remarks

The 'TfModes' enumeration controls the method used to compute term frequency (TF) for features during document classification. The choice of mode affects how feature occurrences are weighted, which in turn influences similarity calculations between documents and trained Document Types.

Overview

• Term Frequency (TF): Measures how often a feature (such as a word or token) appears in a document, relative to the document's size or other features.
• The selected mode determines how TF is normalized or scaled, impacting sensitivity to document length, feature repetition, and feature prominence.

Available Modes

• Normal:
  Normalizes feature counts by the total number of features in the document, making TF independent of document size.
• Logarithmic:
  Applies a logarithmic scale to feature counts, reducing the impact of very frequent features. Provided for backward compatibility.
• Augmented:
  Normalizes feature counts by the most frequent feature in the document, allowing document size to play a greater role in classification. Includes a dampering factor controlled by the 'Frequency Scaling' property.

Practical Guidance

• Use 'Normal' for most scenarios where document length should not affect classification.
• Use 'Logarithmic' for legacy models or when you want to reduce the influence of highly repetitive features.
• Use 'Augmented' when you want longer documents or repeated features to have more influence, or when tuning with the 'Frequency Scaling' property.

Can be one of the following values:

Name	Value		Description
Normal	0	►	Normalizes term frequency to the size of the document. Each feature's count is divided by the total number of features in the document. This mode provides matching that is independent of document size, so short and long documents are compared on equal footing. Note: In some cases, this can produce false positives, as a very short document may match a long document with high similarity if the same features are present.
Logarithmic	1	►	Scales term frequency logarithmically. Applies a logarithmic function to feature counts, reducing the impact of very frequent features. Useful for legacy models or when you want to minimize the influence of repeated features. Provided primarily for backward compatibility.
Augmented	2	►	Normalizes term frequency to the most-frequent feature in the document, with optional dampering. Each feature's count is divided by the count of the most frequent feature, then scaled by the 'Frequency Scaling' property. This mode allows document size and feature repetition to play a greater role in classification. The dampering factor ('Frequency Scaling') controls how much frequency matters: higher values increase the effect, lower values reduce it. Useful when you want to tune the sensitivity to repeated features or document length.

Used By

Lexical