Crawl Pilot Web Scraper

Turn any webpage into structured data with a visual point-and-click interface. No coding required.

Crawl Pilot is a browser-based data extraction tool with image downloading and AI text summarization — all running locally in your browser with no data collection.

FEATURES
────────────────────────────────────────────

VISUAL LIST EXTRACTOR

Click on any repeating element on a page — a product card, a headline, a listing — and Crawl Pilot identifies all similar items automatically. No CSS selectors or code needed.

How it works:
1. Open the side panel
2. Click "Extract a List"
3. Click any item on the page
4. The extension detects the pattern and highlights matching items
5. Choose which fields to extract (text, links, images, attributes)
6. Export to Excel, CSV, or JSON

Key capabilities:
• Pattern detection based on DOM structure and element hierarchy
• Automatic column separation for distinct data fields
• Nested data support for complex page layouts
• Live preview table while configuring columns
• Multi-level extraction — visit detail pages to grab additional fields

PAGINATION SUPPORT
────────────────────────────────────────────

Handles multi-page content automatically:

• Next Button Mode — Clicks through "Next", "Load More", or numbered pages while accumulating data. Configurable delay between pages.
• Infinite Scroll Mode — Scrolls down to trigger lazy-loaded content. Configurable scroll speed and maximum depth.
• Indexed Scroll — For single-page apps with virtual viewports. Tracks seen items to prevent duplicates.

PAGE EXTRACTOR
────────────────────────────────────────────

Process multiple URLs in sequence:
• Paste a list of URLs or import from CSV
• Define an extraction template for which fields to collect
• Results stream in real-time with progress tracking
• Auto-resumes if interrupted — progress saved locally
• Templates can be saved and reused

IMAGE DOWNLOADER
────────────────────────────────────────────

Finds and downloads images from any webpage, including those that load lazily or are rendered dynamically.

Discovery:
• Scrolls the page to trigger lazy-loaded images
• Detects CSS background images and canvas-rendered images
• Parses lazy-load attributes (data-src, srcset, etc.)
• Works on single-page applications with dynamic content

Live Watch Mode:
• Continuously monitors the page for newly loaded images
• Images appear in the gallery as they're discovered

Filtering & Organization:
• Filter by format (JPG, PNG, SVG, GIF, WebP, AVIF)
• Filter by dimensions (min/max width and height)
• Group images by similar size
• Search by URL or alt text

Download Options:
• Download individual images or batch download as ZIP
• Download all images in a size group at once

AI TEXT SUMMARIZER
────────────────────────────────────────────

Summarize and analyze page content using AI:
• Get concise summaries of articles, papers, or documentation
• Ask questions about the page content
• Automatically identifies key takeaways

BROWSER UTILITIES
────────────────────────────────────────────

Right-Click Unlocker:
• Re-enables context menu on pages that disable it
• Restores copy, paste, and text selection
• One toggle — works immediately, no reload needed

USE CASES
────────────────────────────────────────────

Research & Analysis:
• Collect structured data from directories, catalogs, and public listings
• Gather pricing information across multiple pages for comparison
• Extract article metadata for literature reviews
• Build datasets from tables and repeated page elements

Business & Productivity:
• Turn product listings into organized spreadsheets
• Compile contact information from business directories
• Monitor content changes across multiple pages
• Extract job postings, event listings, or property details in bulk

Design & Content:
• Download image collections filtered by size and format
• Audit image assets across a website
• Summarize long articles or documentation quickly
• Collect reference material from multiple sources

WHY CRAWL PILOT
────────────────────────────────────────────

• No coding — visual point-and-click interface for all features
• No cloud dependency — everything runs in your browser tab
• No account or signup — install and start using immediately
• Works on modern websites — handles dynamic content, lazy loading, and infinite scroll
• Multiple export formats — Excel, CSV, JSON, and ZIP for images
• Saves progress — extraction state persists if interrupted

PRIVACY & DATA
────────────────────────────────────────────

• All processing happens locally in your browser
• No data is sent to external servers
• No account required
• No tracking or analytics