EvilZone
Hacking and Security => Tutorials => : daxda November 15, 2013, 03:11:29 PM
-
The long version
I decided to add support for Bing in my search engine scraper which is written in Python (view the source here (https://evilzone.org/scripting-languages/%28source%29-gscrape-py/))
Once implemented and tested, I found that you get a maximum of 10 results per query which is way too less for my taste.
So I tried searching what kind of parameters Bing offers, unlike Google there wasn't one for the amount of results.
Changing the amount of results in the settings (http://www.bing.com/account/general) of Bing changed the request accordingly, turns out Bing uses a Cookie to decide how many results will get displayed.
The full request will look like this:
GET /search?q=query&go=&qs=ds&form=QBLH&filt=all HTTP/1.1
Host: www.bing.com
User-Agent: fuck you :)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: MUID=2AD1C087FaF66E6810F6CS11EED76E63; MUIDB=2AD1CF7EFF66E6810F6C5E1EED76E63; OrigMUID=2AD1C08sEFF66E6De0F6R501EED76E63%2ca7d25648b73544c8b8982d16c6452d1e; SRCHD=MS=30896x3&D=30894h2&AF=QBLH; SRCHUID=V=2&GUID=702445F423FDA4B0B82096DEA33499C79; SRCHUSR=AUTOREDIR=0&GEOVAR=&DOB=20131115; _UR=OMW=0&OMD=13028909743; SRCHHPGUSR=NEWWND=0&NRSLT=50&SRCHLANG=&AS=0&ADLT=OFF; _FS=NU=1; _SS=SID=2DB1B23g905CF4D4D82A02y1C3AAC297C7662&bIm=570781&nhIm=08-; _HOP=; SCRHDN=ASD=0&DURL=#; WLS=TS=63520119834
DNT: 1
Connection: keep-alive
After a long read on the Cookie I came by the parameter called "NRSLT=50", which rang my bells. (Number of ReSuLTs)
I've tested out what arguments in the Cookie you have to pass to still get a valid query, this was the result:
Cookie: SRCHHPGUSR=&NEWWND=0&NRSLT=50&SRCHLANG=
The short version
Add Cookie to request ^