AI Collection Reader

The Data Generator object configures how Grooper uses large language models (LLMs) to generate structured data from unstructured document content. It is embedded as a property within various Grooper features that leverage generative AI, such as advanced data extraction, document classification, and content transformation.

Configuration Guidance

• Choose a model that matches your accuracy, speed, and cost requirements.
• Use structured output mode whenever possible for the most reliable results.
• Adjust chat parameters to fine-tune the LLM's behavior for your specific use case.
• Lowering the 'Temperature' can improve consistency and reduce variability in responses, especially for structured data extraction.
• Increase 'Max Tries' if you encounter intermittent failures or unparseable responses.

How It Works

When Grooper performs an extraction using a Data Generator:

1. The extraction schema (as a JSON object) and system/user messages are sent to the LLM.
2. The LLM is prompted to return data in the required JSON format.
3. If 'Use Structured Output' is enabled and supported by the model, the LLM is given explicit instructions to return strictly valid JSON matching the schema.
4. Grooper parses the LLM's response and maps it to the extraction schema.
5. If the response is invalid, incomplete, or a transient error occurs, Grooper retries the request up to 'Max Tries'.

This approach enables robust, schema-driven extraction from a wide variety of document types, even when traditional rule-based methods are insufficient.

Diagnostics and Logging

When extraction is performed, Data Generator logs diagnostic artifacts to aid in troubleshooting and auditing:

• Chat Log.jsonl: A line-delimited JSON log of all messages exchanged with the LLM during extraction.
• Response Data.json: The raw JSON data returned by the LLM for each extraction attempt.
• LLM Completion Operation: Timing and performance metrics for each LLM request, recorded in the diagnostic log.

These artifacts are available in the extraction diagnostics and can be used to review LLM interactions, analyze failures, and optimize configuration.

Use this property to limit extraction to a subset of fields, tables, or sections. This reduces prompt complexity and focuses the LLM on only the data you need.

• If left empty, all visible, non-computed descendant Data Elements are included.
• Add specific Data Fields, Data Tables, or Data Sections to target extraction and improve performance.
• Useful for incremental extraction or when only certain data is needed from a document.

Example:
To extract only the "Invoice Date" and "Total Amount" fields, add those fields to this list.

Provide additional guidance or requirements for the LLM to follow during extraction. Instructions are appended to the prompt and can clarify expectations, formatting, or special rules.

• Use clear, concise language to direct the LLM.
• Examples: "Extract only dates in MM/DD/YYYY format." or "Ignore handwritten notes."
• Leave blank for standard extraction behavior.

Guidance on Prompt Engineering

Well-crafted instructions can significantly improve extraction accuracy, especially for documents with ambiguous or inconsistent layouts. Use this property to specify formatting requirements, business rules, or to clarify how the LLM should interpret certain document features.

• Be explicit about required formats (e.g., "Return all phone numbers in the format (###) ###-####").
• Instruct the LLM to ignore irrelevant content or to prioritize certain fields.
• For complex documents, break down instructions into bullet points or numbered steps.

Troubleshooting

If extraction results are incomplete or inconsistent, try refining your instructions. Test with different phrasings or add clarifying examples to guide the LLM toward the desired output.

Chunking enables AI Collection Reader to process documents that are too large to fit into the context window of a large language model (LLM). By splitting a document into smaller chunks, each chunk can be processed independently, ensuring that even extremely large or multi-page documents can be extracted without exceeding LLM context limitations.

• Set to a value greater than 1 to enable chunked processing.
• Leave blank or set to 0 to process the entire section as a single chunk.
• Adjust based on document size, LLM context window, and desired throughput.

Why Chunking Is Needed

Most LLMs have a maximum context window (token limit) that restricts how much text can be processed in a single request. For very large documents, attempting to extract all data at once may exceed this limit, resulting in errors or incomplete extraction. Chunking divides the document into manageable segments, each of which fits within the LLM's context window.

Chunking Strategies

• Use chunking for statements, logs, reports, or any document where the total content exceeds the LLM's context capacity.
• Choose a chunk size that ensures each chunk (including quoted content and prompt) remains well below the model's token limit.
• Smaller chunk sizes may improve reliability but increase the number of LLM calls.
• Larger chunk sizes reduce the number of calls but may risk exceeding token limits or introducing more noise.
• Experiment with different chunk sizes to find the best balance for your documents and LLM model.

When chunked processing is enabled, this property controls the degree of parallelism. Increasing this value can speed up extraction for large documents, but may increase resource usage.

• Applies only to collection sections with chunking enabled.
• Typical values range from 1 (sequential) to 32 (high parallelism).
• Adjust based on available system resources and desired performance.

Parallel Processing Considerations

Parallel processing allows multiple chunks to be extracted simultaneously, reducing total processing time for large documents. However, higher degrees of parallelism will consume more CPU and memory resources.

• Set a lower value if running on a shared or resource-constrained system.
• Increase the value to maximize throughput on dedicated or high-performance servers.
• Monitor system performance and adjust as needed to avoid overloading the environment.

Best Practices

• Start with the default value and adjust incrementally.
• For cloud or virtualized environments, coordinate with your IT team to determine safe parallelism levels.
• If you encounter throttling or resource contention, reduce the value.

Can be one of the following types:

The quoting method determines how the relevant portion of the document is selected and formatted for inclusion in the LLM prompt. This can impact extraction accuracy and efficiency.

• Choose a quoting method that best matches the structure and content of your documents.
• If left blank, the entire input document is included by default.
• Advanced quoting methods can help focus the LLM on specific regions, fields, or layouts.

Quoting Strategies

The choice of quoting method affects both the context provided to the LLM and the cost of the operation (since larger prompts consume more tokens). Use targeted quoting to reduce noise and focus the LLM on the most relevant content.

• Use region-based quoting for documents with well-defined sections or headers.
• Use layout or table quoting for structured data, such as invoices or forms.
• For free-form documents, consider quoting the entire page or document.

Custom Quoting

Custom quoting methods can be created to handle unique document layouts or to preprocess content (e.g., removing watermarks, normalizing whitespace). This is especially useful for documents with non-standard formatting.