AI Collection Reader

Choose the model that best fits your extraction needs and available resources. The selected model determines the language understanding, accuracy, and cost of the extraction operation. Common choices include gpt-3.5-turbo for general-purpose extraction and gpt-4o for more complex or nuanced documents.

Guidance

• Use newer or more capable models for documents with complex structure or ambiguous content.
• Consider cost and performance trade-offs when selecting a model.
• The list of available models is determined by your LLM Connector configuration.

Can be one of the following types:

The Chat Parameters class defines a set of options for tuning the behavior of LLM completion requests in Grooper. These parameters allow users to control the randomness, diversity, and topicality of generated responses, enabling tailored outputs for a wide range of scenarios.

Typical usage involves adjusting one or more parameters to guide the model’s output toward the desired style or content. For example, increasing randomness for creative tasks, or reducing repetition for summarization.

Configuration Guidance

• Adjust either 'Temperature' or 'Top P' to control randomness and diversity, but not both simultaneously.
• Use 'Presence Penalty' and 'Frequency Penalty' together to reduce repetition and encourage broader topic coverage.

Usage in Grooper

Chat Parameters are used throughout Grooper to influence LLM completions for document processing, data extraction, and other AI-driven tasks.

Impact

Proper configuration of ChatParameters can significantly affect the quality, creativity, and reliability of LLM outputs. Experimentation is recommended to find the best settings for your specific use case.

Use this property to limit extraction to a subset of fields, tables, or sections. This reduces prompt complexity and focuses the LLM on only the data you need.

• If left empty, all visible, non-computed descendant Data Elements are included.
• Add specific Data Fields, Data Tables, or Data Sections to target extraction and improve performance.
• Useful for incremental extraction or when only certain data is needed from a document.

Example:
To extract only the "Invoice Date" and "Total Amount" fields, add those fields to this list.

Provide additional guidance or requirements for the LLM to follow during extraction. Instructions are appended to the prompt and can clarify expectations, formatting, or special rules.

• Use clear, concise language to direct the LLM.
• Examples: "Extract only dates in MM/DD/YYYY format." or "Ignore handwritten notes."
• Leave blank for standard extraction behavior.

Guidance on Prompt Engineering

Well-crafted instructions can significantly improve extraction accuracy, especially for documents with ambiguous or inconsistent layouts. Use this property to specify formatting requirements, business rules, or to clarify how the LLM should interpret certain document features.

• Be explicit about required formats (e.g., "Return all phone numbers in the format (###) ###-####").
• Instruct the LLM to ignore irrelevant content or to prioritize certain fields.
• For complex documents, break down instructions into bullet points or numbered steps.

Troubleshooting

If extraction results are incomplete or inconsistent, try refining your instructions. Test with different phrasings or add clarifying examples to guide the LLM toward the desired output.

Chunking enables AI Collection Reader to process documents that are too large to fit into the context window of a large language model (LLM). By splitting a document into smaller chunks, each chunk can be processed independently, ensuring that even extremely large or multi-page documents can be extracted without exceeding LLM context limitations.

• Set to a value greater than 1 to enable chunked processing.
• Leave blank or set to 0 to process the entire section as a single chunk.
• Adjust based on document size, LLM context window, and desired throughput.

Why Chunking Is Needed

Most LLMs have a maximum context window (token limit) that restricts how much text can be processed in a single request. For very large documents, attempting to extract all data at once may exceed this limit, resulting in errors or incomplete extraction. Chunking divides the document into manageable segments, each of which fits within the LLM's context window.

Chunking Strategies

• Use chunking for statements, logs, reports, or any document where the total content exceeds the LLM's context capacity.
• Choose a chunk size that ensures each chunk (including quoted content and prompt) remains well below the model's token limit.
• Smaller chunk sizes may improve reliability but increase the number of LLM calls.
• Larger chunk sizes reduce the number of calls but may risk exceeding token limits or introducing more noise.
• Experiment with different chunk sizes to find the best balance for your documents and LLM model.

When chunked processing is enabled, this property controls the degree of parallelism. Increasing this value can speed up extraction for large documents, but may increase resource usage.

• Applies only to collection sections with chunking enabled.
• Typical values range from 1 (sequential) to 32 (high parallelism).
• Adjust based on available system resources and desired performance.

Parallel Processing Considerations

Parallel processing allows multiple chunks to be extracted simultaneously, reducing total processing time for large documents. However, higher degrees of parallelism will consume more CPU and memory resources.

• Set a lower value if running on a shared or resource-constrained system.
• Increase the value to maximize throughput on dedicated or high-performance servers.
• Monitor system performance and adjust as needed to avoid overloading the environment.

Best Practices

• Start with the default value and adjust incrementally.
• For cloud or virtualized environments, coordinate with your IT team to determine safe parallelism levels.
• If you encounter throttling or resource contention, reduce the value.

Can be one of the following types:

The quoting method determines how the relevant portion of the document is selected and formatted for inclusion in the LLM prompt. This can impact extraction accuracy and efficiency.

• Choose a quoting method that best matches the structure and content of your documents.
• If left blank, the entire input document is included by default.
• Advanced quoting methods can help focus the LLM on specific regions, fields, or layouts.

Quoting Strategies

The choice of quoting method affects both the context provided to the LLM and the cost of the operation (since larger prompts consume more tokens). Use targeted quoting to reduce noise and focus the LLM on the most relevant content.

• Use region-based quoting for documents with well-defined sections or headers.
• Use layout or table quoting for structured data, such as invoices or forms.
• For free-form documents, consider quoting the entire page or document.

Custom Quoting

Custom quoting methods can be created to handle unique document layouts or to preprocess content (e.g., removing watermarks, normalizing whitespace). This is especially useful for documents with non-standard formatting.

When enabled, the LLM is instructed to return data in a strict JSON format matching the extraction schema. This improves reliability and parsing accuracy, especially for complex or nested data.

• Supported by OpenAI models such as GPT-4o, GPT-4o-mini, GPT-4 Turbo, and recent GPT-3.5 Turbo models.
• When disabled, a best-effort parsing approach is used, which may be less reliable.
• Enable this option whenever possible for the most robust extraction results.

Controls how many times the AI Section Reader will attempt to re-issue a request to the LLM service if an error occurs during extraction. Retries are triggered by issues such as network timeouts, incomplete or malformed responses, or other transient failures that may prevent a successful extraction on the first attempt.

Configuration Guidance

• Increase this value if you experience intermittent connectivity issues or occasional LLM service instability.
• Decrease the value to reduce wait times in environments where failures are rare or rapid feedback is preferred.
• The allowed range is 1 to 8. Setting a value outside this range is not permitted.
• Each retry uses the same prompt and parameters as the original request.
• When 'Use Structured Output' is disabled, increasing 'Max Tries' can help ensure that documents are processed successfully, as the LLM may occasionally return unparseable or incomplete responses. Allowing more retries gives the system additional opportunities to obtain a valid result.

Impact

A higher value for 'Max Tries' can improve reliability in unstable environments, ensuring that temporary issues do not cause extraction to fail. However, increasing the number of retries may also result in longer processing times if the LLM service is unavailable or consistently returns errors.

If all attempts fail, the extraction operation will be unsuccessful, and an error will be logged in the diagnostics.

Example

• If 'Max Tries' is set to 3, the AI Section Reader will attempt up to three requests before reporting a failure.