HTTP Import

• Each HTTPResource defines a starting point for import, such as a website root or a specific page.
• Configure each resource to control which pages are included, which links are followed, and how deep the crawl goes.
• See the HTTP Resource and Hyperlink Selector documentation for details on advanced crawling and filtering.

• Use this to throttle requests and avoid overloading web servers.
• Example: Set to 1000 to wait 1 second between each page fetch.

When 'Sparse Import' is enabled, Grooper creates a Content Link for each imported document instead of copying the file content into the repository. This reduces storage requirements and speeds up imports, especially for large volumes. Content can be loaded on demand later in the process, and all Grooper activities work with sparse documents as with normal documents.

Use 'Skip Count' to restart an import that was previously stopped, or to perform incremental imports by skipping items already processed.

Use 'Max Items' to limit the number of documents or folders imported in a single operation. This can be combined with 'Skip Count' for incremental or staged imports.

The 'Batch Creation' property defines rules for grouping imported items into batches, including batch size, structure, and organization. Adjust these settings to match your workflow and processing requirements.

The 'Batch Name Options' property allows you to configure naming conventions for new batches, including dynamic values based on process, content type, or other metadata.

Assigns a Content Type to each imported document or folder. This controls classification, data extraction, and downstream processing. Set this property to ensure imported content is correctly identified and processed in Grooper.