Yeah so these are ideas that date back to the '90s, so it is genuinely weird to see Hoglund talking about them in 2009. But for whatever it's worth: if ever a tool needed its balls blown off, it's nmap. I have nothing but respect for Fyodor, but that space badly needs real competition.
There is, for what it's worth, very little reason why the best scanner need be written in C. Scanners aren't even I/O bound in their most common use case; they're timer-bound.
If you feel the need to justify nmap being writting in C, which you shouldn't, just remember: The people who wrote nmap know C, and managed to write fucking nmap in it.
If you want a better one written in something else, then get cracking...
He's just saying they used what they knew best. For example, there's no reason Git has to be written in C[1]. Mercurial, which is Python, is doing just fine. C is what Linus and the kernel hackers knew best.
[1] Yeah, I know it was mostly hacked together using Bash and Perl, but that was a long time ago. Now it's mostly C. I'm suspecting they rewrote the performance-critical stuff in C, and left everything else alone. Not sure, though; I don't follow Git development.
Not sure why I'm bothering to respond since you're being rather trolly.. but neither. I rarely program in C, and I rarely have a use for nmap.
Rather I just intensely dislike when people pick language choice to focus on. You pick your language based on what your team knows, and if you can ship with it. Picking it for any other reason just makes you come off in a PHB type of way. If you have real criticisms and suggestions to make, then do so and make the world a better place in the process.
I am saying something very simple that seems to have caused your head to explode. It is: you could write n+1map in Python using Twisted, or Ruby using EventMachine, or Erlang, and probably end up faster (not because of the language, but because you'd design for speed and the language doesn't matter) than nmap is in C.
This isn't a controversial statement. Nobody thinks nmap is the fastest possible scanner. To argue that n+1map should be written in C, you have to make a case for what aspect of a port scanner benefits from C. Which is why I pointed out: nmap doesn't even look I/O bound; it looks timer-bound. And in most cases, neither I/O-bound nor timer-bound programs benefit greatly from locality, cheaper copies, or (within reason) optimized memory allocation and layout.
I believe the problem is that you seem to be saying c should only be used when performance matters. They think the language choice is irrelevant and shouldn't have even been mentioned. Or so I believe.
For my part, I am an avid C programmer; it's my first language, and virtually the only one I used through the '90s. And I will say: you should avoid writing in C unless you need to. The cases I can come up with off the top of my head where it's necessary:
* When performance matters and can be gained through writing C code --- for instance, if you're compute-bound, or if you need fast access to data structures.
* When building incremental improvements to large C codebases --- ie, writing a loadable kernel module.
* When you're deploying in small-footprint environments.
What are the other cases where C makes sense?
I brought up the C thing not because I want to take potshots at what is probably my favorite language, but because a majority of the developers on HN don't write C, and you'd hate for them not to take a crack at competing with nmap because they had a faulty belief that they'd need to use C.
C is a bad choice for a new port scanner started in 2011. I've already written comments here justifying that statement. If you really think I'm wrong, it might be fun to hash out why. (I've got a fairly decent background in nmap's problem domain, for what it's worth.)
I don't have a problem with C, although if I had to do it I would use C++ for object-oriented support and more or less guaranteed portability. Possibly also to compile in a lua interpreter. That said, I'm not a domain expert and was responding to the question of whether nmap should have been coded in C rather than what the best language would be for something started NOW.
I don't know what the best alternatives would be and while I doubt I can contribute much to the discussion, I've read your other posts on security with interest and would be interested to hear which language you'd pick and why.
Personally, I think Lisp is a fantastic language to write a scanner in; you can express async state machines in it naturally, and most Lisps have very good foreign interfaces, which would allow you to just call into libpcap. Lisp is also good for DSLs --- look how Seibel did the little expression language for binary formats in his Lisp book; that's exactly the problem statement that drove me to make my own dumb language in '97 to express packet formats.
Virtually any language will work for this (which is why you should avoid C). About the only thing I'd steer you clear of is a JVM language. The JVM is ordinarily a win, but here, where you want the path-of-least-resistance to getting pcap into your program, it's a bit of a pain. So scratch Scala from the list. Other than that, have fun.
Nmap isn't fundamentally flawed. It's ambitious and sprawling in ways that detract from its basic purpose. It wants to be a general-purpose network surveying tool, but is constrained to the UX of a port scanner.
Someone else should write the best possible port scanner and free nmap up to be the giant network profiling system it clearly wants to be.
"It wants to be a general-purpose network surveying tool, but is constrained to the UX of a port scanner."
What exactly does that mean? Nmap certainly isn't constrained to the UX of a port scanner. In fact, their expansion wasn't even in the form of bundled build, you can get all the tools separated, like: nmap, nping, ncat, ncrack...
So actually, it really isn't detracting from its basic purpose, you can still run simple network scans. But, if you got the knowledge, skills and the other tools installed, you can then also do much than advanced network scans, like vulnerability discovery and exploitation, passwords bruteforcing, etc...
Your comment isn't making much sense to me, please help me understand.
It has N different kinds of "stealth" scans that all produce the same kernel of valuable information, it has an elaborate (and, against most corporate networks from external vantage points, not particularly accurate or helpful) OS detection scheme, it runs scripts, it produces stdout and file output in a variety of complicated formats (it really just wants to be a service hooked up to a database), it runs scripts, IT RUNS SCRIPTS, come on. This isn't a serious argument. You just like nmap. That's fine; I think nmap is impressive too.
Oh, and it takes _for_ _ever_ to run. I'll pastie the 50 line EventMachine scripts I race it with when people on my team get stuck waiting for 12-hour nmap runs.
Sure. You know how simple this task is when you have an event loop, right? Here, I'll write a port scanner right now:
class HostProbe
module Probe
attr_accessor :bp
def connection_completed
@win = true
end
def unbind
self.bp.closed(self, @win)
end
end
def closed(obj, won)
@inflight_now -= 1
port = @inflight_q.delete(obj)
if won
puts "%TARGET-LISTENING: #{ @target }:#{ port }"
end
fill()
end
def sweep
Generator.new do |g|
(1..65535).each {|port| g.yield port}
end
end
def fill
@inflight_now.upto(@inflight_max) {
@inflight_now += 1
if(port = @sweep.next)
EventMachine::connect(@target, port, Probe) do |c|
c.bp = self
@inflight_q[c] = port
end
end
}
end
def initialize(host, opts={})
@target = host
@sweep = sweep
@inflight_q = {}
@inflight_now = 0
@inflight_max = opts[:inflight_max] || 10
fill()
end
end
EventMachine::run {
HostProbe.new(ARGV[0])
}
I think that may be it? I'm not even going to bother seeing if it evaluates. I write that stupid script once a month or so. There is probably a set of nmap options that crushes it for speed, but with the default options and a firewalled target network (ie: most professional nmap targets), I lap nmap with it.
Bit twiddling is much easier in other languages than in C. And virtually every mainstream language has a pcap binding for the packet reading and injection (although a good straight-socket scanner could still give nmap a run for its money).
There is, for what it's worth, very little reason why the best scanner need be written in C. Scanners aren't even I/O bound in their most common use case; they're timer-bound.