HTTP Resource

Represents a web site or web page to be imported usines HTTP Import.

Remarks

An HTTPResource defines a starting point for importing web content into Grooper using the HTTP Import provider.

How it works

• URL: Set this to the root of the website or a specific page you want to import.
• RelativePageUrls: (Optional) Specify a list of relative URLs to include, if you want to restrict import to certain pages under the root URL.
• LinkSelectors: Add one or more HyperlinkSelector objects to control which links are followed from each page, enabling recursive crawling, filtering, and advanced navigation.
• Enabled: Use this to temporarily include or exclude this resource from import runs.
• EnforceScope: When enabled, only URLs that are within the root URL's scope will be followed, preventing the crawler from leaving the intended site or section.
• Description: Add a description to help identify the purpose or scope of this resource.

Example

To import all product pages from a website:

• Set URL to https://example.com
• Add a LinkSelector with a selector like a.product-link and enable recursion to follow all product links.

Notes

• The combination of RelativePageUrls and LinkSelectors allows you to precisely control which pages are imported and how deep the crawl goes.
• Use LinkSelectors to include/exclude links, apply URL patterns, and manage recursion for complex site structures.
• Each HTTPResource operates independently, so you can import from multiple sites or sections in a single import operation.

Properties

Name	Type		Description
URL	String	►	The URL of a web site or a web page. Requiredtrue Set this to the root URL of the website you want to import, or to a specific page. All crawling and link following will be relative to this URL unless overridden by other settings. Example: https://example.com
Relative Page Urls	String	►	Optional list of relative URLs specifying pages in the website to be included. EditorText Property Editor ConverterPg Text Lines Converter Enter one relative URL per line (e.g., /products/page1, /about/contact). If set, only these specific pages will be imported, starting from the root URL. Useful for restricting import to a subset of pages within a site. If left blank, the import will start from the root URL and follow links as defined by LinkSelectors.
Link Selectors	Hyperlink Selector[]	►	The set of links to follow on each HTML page. EditorObject Collection Editor ConverterCollection Converter Add one or more HyperlinkSelector objects to control which links are followed from each imported page. Use selectors to target specific links (e.g., a.product-link), exclude certain links, or enable recursive crawling. Advanced options allow filtering by URL patterns, limiting crawl depth, and combining multiple selectors for complex navigation. If no selectors are defined, only the starting pages are imported.
Enabled	Boolean	►	If set to false, this resource will be skipped during import. DefaultTrue Use this to temporarily disable a resource without deleting its configuration. When unchecked, no pages from this resource will be imported.
Enforce Scope	Boolean	►	When true, URLs not rooted at URL are ignored. DefaultTrue When enabled, only links and pages that begin with the root URL will be followed or imported. Prevents the crawler from leaving the intended site or section. Disable this if you want to allow cross-site or external link crawling (not recommended for most use cases).
Description	String	►	A description for this HTTP Resource. Requiredtrue Use this field to describe the purpose, scope, or any notes about this resource. Helpful for documentation and distinguishing between multiple resources in a project.

See Also

Hyperlink Selector

Used By

HTTP Import