EvilZone

Hacking and Security => Hacking and Security => : Moiz June 15, 2013, 05:02:12 PM

: How To Crawl A webSIte Using Web Robot
: Moiz June 15, 2013, 05:02:12 PM

How Can I Crawl Any Site With Help Of Robot or A Spider Or Crawler For

Example I want to crawl Evil Zone To Get All Links Listed in it So Suggest me A Good Spider And If there A tutorial Please Post It

: Re: How To Crawl A webSIte Using Web Robot
: vezzy June 15, 2013, 05:33:50 PM

Use wget or write your own dedicated web scraper with some sort of HTML parsing library.

: Re: How To Crawl A webSIte Using Web Robot
: Kulverstukas June 15, 2013, 07:25:14 PM

Capitalizing Every Word In A Sentence Is Very Cool.

But what you really want is a web cloner, I can suggest HTTrack (http://www.httrack.com/). Used it few times, worked fantastic.

: Re: How To Crawl A webSIte Using Web Robot
: kenjoe41 June 15, 2013, 11:53:52 PM

Bash or python would do the job just fine. But that's just me.