Retrieval Plan

Defines a plan for retrieving external information before an AI Assistant generates a response to a user inquiry.

Remarks

A Retrieval Plan orchestrates how an AI Assistant gathers supporting information from external resources—such as Search Indexes, Database Tables, Web Services, or Bing Search—to enhance the accuracy, context, and reliability of its responses.

Retrieval-Augmented Generation (RAG) is a technique that empowers the assistant to supplement its generative capabilities with up-to-date, domain-specific, or authoritative data. The Retrieval Plan is the mechanism by which the assistant determines which retrieval tools to invoke, in what order, and with what criteria, before composing a final answer for the user.

How Retrieval Plans Work

When a user submits a question, the AI Assistant evaluates the conversation context and available Resource References (Search Index, Database Table, Web Service, Bing Search) to determine if additional information is needed. If so, it generates a Retrieval Plan that specifies:

• Which retrieval tools to use: Each tool corresponds to a specific resource type (e.g., semantic search, SQL query, web API call).
• What information to retrieve: The plan describes the data or documents needed to answer the user's question.
• How to retrieve it: Criteria such as search queries, filters, or parameters are included for each tool.
• Chained retrievals: If multiple steps are required (e.g., first retrieve a list, then fetch details), the plan can specify additional instructions for subsequent retrievals.

The plan is executed before the assistant generates its response, ensuring that answers are informed by the most relevant and current data available.

Retrieval Plans are generated automatically by the AI Assistant during chat sessions. End users do not configure plans directly; instead, they configure Resource References and retrieval options on the assistant. The plan is created in response to each user inquiry, based on the conversation context and available resources.

The plan is visible in diagnostic logs and may be surfaced in the user interface for transparency or troubleshooting.

Supported Resource Types

The AI Assistant can be configured with a variety of resource types, each enabling different retrieval capabilities. These resources are added to the assistant's 'Resources' property and are referenced in Retrieval Plans as needed:

• Search Index Reference: Enables semantic (vector) and keyword-based search over indexed document collections. Use for question answering, document lookup, and knowledge discovery.
• Table Reference: Exposes a database table for direct querying. Use for structured data retrieval, reporting, and analytics.
• Web Service Reference: Connects to external APIs described by RAML definitions. Use for dynamic data retrieval, integration, or automation via web services.
• Bing Search Reference: Provides real-time web search using Bing. Use for fact-checking, accessing current events, or augmenting knowledge with internet sources.

Each resource type is configured independently and can be combined to create a comprehensive retrieval strategy tailored to your business needs.

Multi-Hop Retrieval

Some user questions require synthesizing information from multiple sources or performing a sequence of dependent retrievals. This is known as multi-hop retrieval. In Grooper, multi-hop retrieval allows the AI Assistant to chain together several retrieval operations, where the output of one step informs the next.

The maximum number of retrieval steps (or "hops") is controlled by the 'Max Retrieval Depth' property on the AI Assistant. For example, setting this value to 1 restricts the assistant to a single retrieval operation per inquiry, while higher values (up to 8) enable more sophisticated, multi-step plans.

How Multi-Hop Retrieval Works:

• The Retrieval Plan may include a nextStep property with instructions for additional retrievals.
• After each retrieval, the plan is re-evaluated to determine if further steps are needed.
• This process continues until the maximum depth is reached or no further retrievals are required.

Example Multi-Hop Scenario:

1. User asks: "List all customers who purchased product X last quarter and show their most recent order status."
2. The first retrieval step queries the Database Table for customers who purchased product X.
3. The next step retrieves the latest order status for each customer found in the previous step.
4. The assistant combines the results and generates a comprehensive response.

Multi-hop retrieval is essential for answering complex, multi-part questions and for scenarios where information must be correlated across different resources.

Diagnostic Artifacts

When a Retrieval Plan is generated, the following diagnostic artifacts may be logged:

• Retrieval Plan Schema: The JSON schema describing the structure of the plan.
• Retrieval Plan Conversation: The full conversation between the assistant and the planning model, including all instructions and responses.

These artifacts are useful for auditing, troubleshooting, and refining retrieval strategies.

Example Workflow

1. User asks: "What is the latest status of order 12345?"
2. The AI Assistant generates a Retrieval Plan that includes:
   • A SQL query tool targeting the 'Orders' Database Table to retrieve the status.
   • A brief description: "Query the Orders table for order 12345 and return the current status."
3. The plan is executed, results are retrieved, and the assistant composes a response using the fresh data.

Configuring Custom Retrieval Instructions

The behavior of Retrieval Plans can be tailored by providing custom retrieval instructions on the AI Assistant. These instructions guide how the assistant selects, prioritizes, and interacts with knowledge resources when formulating a plan.

Custom instructions are set using the 'Retrieval Instructions' property on the AI Assistant. This property allows you to:

• Specify preferred search strategies (e.g., "Always check search indexes before querying database tables").
• Set thresholds and limits, such as the maximum number of results or retrieval depth.
• Indicate when to prompt the user for clarification.
• Define rules for combining or ranking results from multiple resources.

Example Custom Retrieval Instructions:

For policy-related questions, search the 'Policies' search index first. If no relevant results are found, check the 'Procedures' index.
Limit database queries to 10 results unless the user specifies a different amount.
If a user's question could apply to multiple resources, ask which resource they want to use before proceeding.

Best Practices:

• Keep instructions concise and focused on actionable guidance.
• Update instructions as your data sources or business needs evolve.
• Test the assistant's behavior with a variety of queries to ensure the retrieval plan aligns with your intent.

By configuring custom retrieval instructions, you can ensure that the AI Assistant generates Retrieval Plans that align with your business rules, compliance requirements, and user expectations, resulting in more accurate and context-aware responses.