Today fyodor/nmap gave a talk at defcon ("Nmap: Scanning the Internet"). It was one of the better if not the best presentation at defcon for me. Fyodor presents his research with a lot of charm, fun and motivation.
Nmap can now be used to scan the entire Internet.
Before joining THC I was doing research for Team-Teso. In 2000 one of our problems at Teso was that many script kiddies entered the arena
and started setting up DDoS hosts and owning like mad. Hacking became mainstream.
At Teso we did not like script kiddies and we abhorred those doing DDoS. A small group of Teso and some friends reverse engineered the backdoors and started scanning for them. Our objective was to discourage script kiddies and stop DDoS attacks (by removing the DDoS agents).
Techniques
We developed a new scanner (called 'bscan', not published but a handful of people had it) that was capable of scanning
the internet.
The main features of bscan were:
- Raw SYN scanner. Full TCP/IP stack in userland.
- Using ghost IP and ghost MAC (untraceable)
- Modular. We developed loadable modules for telnet handshake, bind, http (HEAD / HTTP/1.0), ...
- Sending out 50.000 or more syn packets per second.
- Running on linux, sunos/solaris and bsd.
In short the scanner was capable of scanning the entire Internet (0.0.0.0 - 239.255.255.255). The scanner retrieved all Web Server versions
or telnet banners within hours.
Fyodor's nmap was developed for a different reason. The features of nmap are far superior to bscan. Bscan was a tool and nmap is a professional application.
Results
All this is history now and I think that 7 years after the development the time has come to share some of the stuff that we learned
while scanning the Internet:
| 1. | The Internet is full of hosts that do not comply with the RFC. |
| 2. | There are hosts on the Internet that keep sending ACK packets for hours even if you send back FIN, RST or ICMP error messages. They just wont stop sending! |
| 3. | Sometimes you send a SYN to one host and you get the SYN/ACK back from a different host (asymmetric NAT). |
| 4. | There are entire class A networks with no hosts in them at all (The Black Holes of the Internet). |
| 5. | Never scan sequential. If a remote class B or class C is hit with 50k SYNCs per second the serving router of the target network will start sending out ARP requests to resolve the MAC of all these hosts. ARP requests are broadcast messages. This will overload some hosts on the target 'local' network which will crash or not respond for several seconds while processing the ARP requests. You will miss those hosts. Scan 'spread spectrum' and increment the IP by 256 or a similar value. |
| 6. | The first syn packet is often lost. When scanning 10-20 class A networks in 'spread spectrum mode (-X option in bscan) then the router of a large network (e.g. class B) still has to resolve several hundred ARP entries per second. Some routers can not handle this and will start dropping SYN packets if the MAC is not known and can not be resolved because the router is already busy resolving other MAC addresses. |
| 7. | Coordinate with your people that you are the only one scanning the Internet. Same reason as above: If two people scan at the same time the target hosts have to process to many ARP requests and both of you will miss hosts. |
| 8. | Never wait longer than 3 seconds for a host to complete. If it takes longer than 3 seconds for a host to reply you are not interested in owning that host anyway. |
| 9. | Be kind to other administrators. We set up a charity ("The Institute for Internet Statistics") to have a reasonable explanation for any IT administrator who complained about our scanning activities. |
The scanner was usually started on 5-10 Internet hosts in parallel. A big thanks at this point to the IT Administrators of the
various universities in Germany who let us use their hosts for scanning (legally!).
A typical TCP port scan of the Internet took between 8-16 hours.
Stories
There was a nice side effect of cleaning the internet from script kiddies and their backdoors: Teso had a full list of all
server versions of all hosts on the Internet. No longer had team teso to scan for vulnerable hosts. We just looked them up in our
log files.
One day one of the German hackers who helped Teso came home drunk and decided to start another scan for a script kiddie
backdoor that was running on TCP port 33645. He initiated a scan and set source port to 443 and destination port
to 33645. The morning after (and being sober again) he saw that various security mailing lists discussed a new
0-day vulnerability against HTTPS (port 443). Apparently someone was scanning with massive speed the HTTPS ports on
the Internet. He looked again of what scan he started the night before: He mistakenly swapped source and destination port while drunk and scanned for port 443 instead for port 33465.
These mails can still be found on the archives of various mailing lists around xmas 2002.
Lesson learned: Do not drink & hack.
We were not the only ones who scanned the Internet. We heart of an Israeli research group who did it in 1998.
In 2002/2003 Dan Kaminsky published another tool called scanrand. His tool is public. Try it.
Final Notes
These days bscan is old and not up to date anymore.
Whatever you do make sure it's legal and does not cause trouble to other people.
regards,
someone
