Author Topic: [AU3][FUNC]"DeepWeb" Crawler  (Read 903 times)

0 Members and 1 Guest are viewing this topic.

Offline B1N4RY2.0

  • /dev/null
  • *
  • Posts: 11
  • Cookies: -3
  • These are not the 1s and 0s you are looking for...
    • View Profile
[AU3][FUNC]"DeepWeb" Crawler
« on: July 31, 2015, 02:38:31 pm »
This is basically a port from a python example found here on EZ, the original author: proxx
 https://evilzone.org/scripting-languages/%27hidden%27-website-finder/msg69531/#msg69531

It's not fast by any means, and could use a far better method of storing/checking IPs already tested ( which would definitely improve performance ). I may end up recoding this example to use raw sockets and check the DNS returns myself rather than relying on Autoit's version as well. If I do, I'll more than likely update this source code :)

Usage:
Code: [Select]
Do
   
   _dwCrawler() ; Returns True if we found port 80 open and returned no DNS
   
Until 0 = 1

Source:
Code: [Select]
#include <Inet.au3>
OnAutoItExitRegister("_onExit")

TCPStartup()

Func _dwCrawler()
   Local $ipAddr = _genIP()
   Local $strCheck = StringInStr(FileRead("Tested_IPs.txt"), $IP)
   
   If $strCheck = 0 Then
      _testIP($ipAddr)
      FileWrite("Tested_IPs.txt", $ipAddr & @CRLF)
   EndIf
   
EndFunc ;==> _dwCrawler()

Func _genIP()
   Local $blockA = Random(1, 254, 1)
   Local $blockB = Random(0, 255, 1)
   Local $blockC = Random(0, 254, 1)
   Local $blockD = Random(0, 255, 1)

   While $blockA == 192 or $blockA == 168 or $blockA == 10
      $blockA = Random(0, 255, 1)
   WEnd
   
   Local $ipAddr = $blockA & "." & $blockB & "." & $blockC & "." & $blockD
   
   Return $ipAddr
EndFunc ;==> _genIP()

Func _testIP($ipAddr)
   Local $sPortCheck = TCPConnect($ipAddr, 80)
   
   if $sPortCheck = - 1 Then
      ConsoleWrite("Port 80 seems to be closed on: " & $ipAddr & @CRLF)
     
      Return False
   EndIf

   Local $sResult = _TCPIpToName($ipAddr)
   
   If @error Then
      ConsoleWrite(@CRLF & "--> There seems to be no DNS associated with the webserver @ " & $ipAddr & @CRLF & @CRLF)
      FileWrite("Hidden_IPs.txt", $ipAddr & @CRLF)
     
      Return True
   Else
      ConsoleWrite(@CRLF & "--> We found a registered webserver: " & $ipAddr & " " & $sResult & @CRLF & @CRLF)
     
      Return False
   EndIf
   
EndFunc ;==> _testIP()

Func _onExit()
   
   TCPShutdown()
   
EndFunc ;==> _onExit()

-edit-
Definitely going to have to change the way DNS is acquired, picked up some "false positives" due to cloud flare protection ( not able to directly address the site with IP, need a "properly configured HEADER" AutoIt doesn't seem to use a DNS service to check the TcpIpToName, but rather checks the site itself and returns the long domain name. But here is some samples it found after a couple hours of running.

Quote from: DeepWeb Crawler
63.250.21.251
104.27.199.78
104.20.17.249
149.152.180.101
104.27.139.146
104.202.81.90
158.123.106.200
68.65.120.254
104.25.47.83
173.83.152.63
104.149.251.6
104.233.170.238
116.166.20.31
198.41.135.173
50.6.54.120
155.73.209.198
116.168.127.69
173.246.182.205
23.253.167.169
66.248.200.237
137.175.44.138
108.187.251.15
104.25.28.92
116.153.16.183
206.82.195.141
206.251.180.24
208.74.202.232
76.56.175.218
147.155.248.244
23.225.16.178
200.35.143.197
174.35.7.183
104.130.128.193
8.24.254.193
99.192.205.143
50.6.2.93
50.118.77.238
198.154.202.171
155.73.230.143
209.18.121.13
163.238.116.145
159.174.113.132
66.160.197.119
107.148.73.23
104.202.81.136
164.159.245.34
205.237.69.79a
107.178.181.189
12.169.71.91
64.250.117.55
72.52.203.95
42.167.175.188
107.170.71.116
172.252.112.174
63.228.175.32
104.28.159.178
191.238.44.36
155.73.147.175
155.40.5.3
173.247.233.201
198.170.66.124
198.105.187.126
66.160.132.223
74.205.95.240
207.150.210.102
128.165.104.167
65.36.129.250
45.43.30.54
69.20.14.217
« Last Edit: August 01, 2015, 04:59:06 am by B1N4RY2.0 »