EvilZone

Programming and Scripting => Scripting Languages => : d4rkcat December 14, 2014, 06:44:56 PM

: [Python] Multithreaded 4chan image downloader
: d4rkcat December 14, 2014, 06:44:56 PM
I wrote this after seeing this thread  (https://evilzone.org/scripting-languages/4chan-image-download-in-ruby/)by Fur on a Ruby 4chan downloader.
I decided it was a nice idea and so gave it an upgrade to a real scripting language, python.
It is blazingly fast and if you have a great connection, you can increase the number of threads with the -n switch to make it even faster.

Usage:
:
./threaddumper.py -b b -t 584978710
Will download every image in the 584978710 thread on /b/

:
./threaddumper.py -b b -t 584978710 -m
Will download every image in the 584978710 thread on /b/ and then monitor the thread for new images and download them when available.

:
./chandumper.py -b b -q
Will download every image in every thread on /b/ and be quiet with the output.

threaddumper.py:

[gist]d4rkcat/a00241bc00bbbd4ba2ff[/gist]

chandumper.py:

[gist]d4rkcat/22b1343d97da7100b2ad[/gist]
: Re: [Python] Multithreaded 4chan image downloader
: Kulverstukas December 14, 2014, 07:28:49 PM
Very good, but is there any practical use of it?
: Re: [Python] Multithreaded 4chan image downloader
: d4rkcat December 15, 2014, 03:18:43 AM
Very good, but is there any practical use of it?

LOL, no ofc not.
I guess someone could learn how download shit 30 times faster than normal.
It entertained me for an hour or so.
: Re: [Python] Multithreaded 4chan image downloader
: Psycho_Coder December 15, 2014, 06:41:24 AM
Good work. YOu could gain significant faster parsing and with lxml instead of beautiful soup. Also try to avoid urllib2 and move to python requests. urllib3 would be fine as well.
: Re: [Python] Multithreaded 4chan image downloader
: d4rkcat December 16, 2014, 05:07:59 AM
Good work. YOu could gain significant faster parsing and with lxml instead of beautiful soup. Also try to avoid urllib2 and move to python requests. urllib3 would be fine as well.

Great ideas, done and done.
Thank you for your advice.  ;)
+1