Wild Spider

web pages are crawled by being loaded into browser using multiple tabs parallelly

Wild Spider: Chrome Extension for Web Crawling

Wild Spider is a Chrome extension that allows web pages to be crawled by being loaded into the browser using multiple tabs parallelly. It uses the current URL as the starting point and fetches all the links on the page to open them in multiple tabs. Please note that the more tabs you use, the more computer resources will be used.

Download

Extension stats

By: Xuan Wu

Users: 41

Rating: 1.00

(1)

Version: 0.0.3 (Last updated: 2019-03-08)

Creation date: 2019-03-08

Risk impact: High risk impact

Risk likelihood:

Low risk likelihood

Manifest version: 2

Permissions:

tabs
activeTab
webNavigation

Size: 123.90K

Ranking

# 91,980 ▲ 2,354

Other rankings

#7418 in Tools ▲ 51

Other platforms

Wild Tamer (v2.43)

4.18

(203,928) 5,998,219+

Not available on Firefox

Not available on Edge

Want to check extension ranking and stats more quickly for other Chrome extensions? Install Chrome-Stats extension to view Chrome-Stats data as you browse the Chrome Web Store.

Extension summary

Analyze keywords

WATCH OUT: more tabs you use, more computer resources (CPU, memory) will be used, and each page costs a bit disk to save the content (in IndexedDb, accessible from extensions -> Inspect views: background page).

The "spider" works in this way:

The current url is used as the starting point, and it's loaded again in a new tab.
After this page is loaded, fetch all the links on the page.
Get all the links on the page, including relative urls.
Open the extracted link parallelly in all the tabs used (by default 3, set in eventPage).
repeat 2-4

All source code at: https://github.com/nobodxbodon/ChromeCrawlerWildSpider