It's a combination of a scientific and engineering limitation. The Curiosity rover's power source generates an average of about 1/6th of a horsepower of electrical power. In practice this is used to trickle charge the batteries of the rover over night and provide slightly higher total power during the day for driving, even so we're talking about a fraction of a horse power to drive a nearly one tonne vehicle the size of an SUV. The Opportunity rover is smaller and lighter (180 kg) but also solar powered and so has much less total energy to work with per day.
If the goal were merely speed we could build faster rovers but to do so would mean removing a lot of scientific instruments so we'd be able to get to different locations but then we'd only be able to do a tiny amount of science compared to what we can do with Curiosity or Opportunity.
How much of this is also a reflection of autonomous navigation, risk avoidance, and proceeding at a deliberate pace? While we've got vehicles on Earth which can navigate established highways at, well, highway speed (Google cars), the established road network on Mars is substantially less extensive, and the downside risks (in terms of mission outcome) of even a small mishap on Mars would be very high.
So: for both engineering reasons (power supply & motors vs. scientific payload) and risk mitigation, you end up with systems which are mobile, but only at a very slow crawl.
Incidentally: Mars also limits the options for higher-mobility systems. The atmosphere is too thin for conventional winged aircraft to be terribly effective (they'd have huge wingspan requirements relative to payload, and very large turning radius and stall speeds). Balloons would be impractical. The gravitational force is high enough that a multi-hop lander relying on rocket thrust would be expensive.
Some sort of robotic tumbleweed / beachball might work, though unless these had some means of getting themselves unstuck, would likely just scatter over an area of terrain (erm, marscape?), before settling against an outcropping or at the bottom of a ravine, crater, canyon, etc.
Another reason why the rovers are slower than cars: cars have air-breathing engines. On Mars, your propulsion system needs to be totally self-contained, or else solar powered.
Why can't they just use nuclear power? Didn't the viking probes from the 70's use that? I might be wrong, but I thought that curiosity actually did use nuclear.
Curiosity does use a nuclear power source (an RTG) but it only provides a little over 100 Watts of power. As I said, the primary use of the RTG is not to drive the rover but to power the batteries over night to drive the rover during the day. Even so, this still amounts to less than one horsepower available to drive a nearly one tonne vehicle.
Like the Viking probes, Curiosity uses the heat from natural decay, not fission. It's a much more foolproof system than a reactor, but it only generates 110W, meaning it needs batteries to handle bursts of demand over 110W.
If the goal were merely speed we could build faster rovers but to do so would mean removing a lot of scientific instruments so we'd be able to get to different locations but then we'd only be able to do a tiny amount of science compared to what we can do with Curiosity or Opportunity.