Hyperlink Selector

• Enter a CSS selector to target the links you want to follow (e.g., a, a.product-link, div.listing a).
• This property is required.
• Only links matching this selector will be considered for crawling.

• Use to exclude links that should not be followed, such as navigation or footer links.
• Example: th a will exclude links inside table headers.
• If left blank, no exclusions are applied.

• Enable to crawl through multiple levels of pages, following links found on each subsequent page.
• Disable to only follow links from the starting page.
• Useful for crawling entire sections or hierarchies of a site.

• Add additional HyperlinkSelector objects to define different crawling rules for child pages.
• Use this to apply different selectors or patterns at different levels of the crawl.
• If left empty, the same selector is used for all levels (if Recursive is enabled).

Can be one of the following types:

• Only URLs matching this pattern will be followed.
• Use to restrict crawling to certain URL formats (e.g., /products/.*).
• Leave blank to include all URLs.

Can be one of the following types:

• URLs matching this pattern will be skipped, even if they match the main selector.
• Use to filter out unwanted pages (e.g., /login, /cart).
• Leave blank to not exclude any URLs by pattern.

Can be one of the following types:

• If a URL matches this pattern, no further links will be followed from that page.
• Use to mark certain pages as endpoints in the crawl (e.g., product detail pages).
• Only relevant if Recursive or child LinkSelectors are used.