EvilZone
Programming and Scripting => Scripting Languages => : Matriplex December 23, 2013, 10:14:57 PM
-
I got a little bored this afternoon and decided to learn a bit of regex. Then I decided to see if I could fetch the most recent posts section from good ol' ez using said method. I guess it worked.
It prints out the topic, who posted, and in which section. Then below, it prints the url. If you are running *nix you should be able to right click and go to url which would obviously open it in your browser.
Example:
Re: How to find the owner name and details of a mobile number? by boriswc (General discussion)
http://evilzone.org/general-discussion/how-to-find-the-owner-name-and-details-of-a-mobile-number/msg73088/
Re: Best free proxy? by boriswc (Hacking and Security)
http://evilzone.org/hacking-and-security/best-free-proxy/msg73087/
Feel free to remove the colors if you they hurt your eyes. Do so by just removing the red.format() and blue.format()'s
import urllib
import urllib2
import re
red = "\033[01;31m{0}\033[00m"
blue = "\033[1;36m{0}\033[00m"
url = ('http://www.evilzone.org/')
sock = urllib.urlopen(url)
ch = sock.read()
sock.close()
x = ch.find('<dl id="ic_recentposts" class="middletext">')
patingr = re.compile('<strong><a.+?>.+?</a></strong> by <a.+?>.+?</a> \(<a.+?>.+?</a>\)')
list = patingr.findall(ch, x)
print
for s in list:
print blue.format(re.sub('<[^>]*>', '', s))
url = re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', s)
for l in url:
if not "profile" in l and ";PHPSESSID" in l:
print red.format(''.join((l.split(';', 1)[0]).split('?topicseen', 1)))
print
Also, I am relatively new to regex so keep that in mind. I know using .+? is probably not the most efficient way, but it gets the job done. However I would very much like to know if you guys would do it a different way.
Enjoy
-
I have done something similar a while ago because I wanted to feed conky with latest topics etc.
Since a new version of the board will be coming out and an API will probably be implemented I didnt bother do anything else on the code.
But guess the mechanism is pretty much the same.
-
Since a new version of the board will be coming out and an API will probably be implemented I didnt bother do anything else on the code.
Hm, I forgot about the new version. I guess I'll just update the code when it does come out.
And an API? Why didn't I hear about this?? My day has been made.
-
my best guess would be that this is python? doesnt actually state that anywhere though?
Its pretty obvious isn't it.
-
This is weird. I don't see where it logs in?
Also wouldn't it be easier to parse RSS feeds? :)
-
This is weird. I don't see where it logs in?
Also wouldn't it be easier to parse RSS feeds? :)
Recent posts and unread posts are 2 different things, recent does not require login.
RSS feed is indeed much easier than HTML parsing, man I hate that.
-
Well, im not a python coder. Its just obviously not PHP or Perl. doesnt leave many options as far as popular scripting languages go... but there are potentially other niche languages i wouldnt know as well... so...
Well its one of the few languages that uses forced indention instead of brackets and stuff.
-
This is weird. I don't see where it logs in?
Also wouldn't it be easier to parse RSS feeds? :)
I wanted to see if I could parse the HTML using regex, just for a learning experience :)
Suppose it would be easier to parse RSS feeds, but as I said, learning experience.
Well, im not a python coder. Its just obviously not PHP or Perl. doesnt leave many options as far as popular scripting languages go... but there are potentially other niche languages i wouldnt know as well... so...
No worries, not everybody knows a language at first glance. Next time, look at some methods or functions that are called that look like they are language specific, e.g; import urllib2, and research that one line on Google. You should be able to find out pretty fast :)
-
I wanted to see if I could parse the HTML using regex, just for a learning experience :)
Read this if you haven't already: http://stackoverflow.com/a/1732454/1552152
Also try this next time: http://www.crummy.com/software/BeautifulSoup/
For extracting tags and very simple HTML parsing Regex is fine, but if you want to do complex stuff, consider a proper HTML parsing lib.