Or at least have the self driving cars driving only along specific routes well marked, well maintained autonomous driving routes similar to light rail/bus lanes so pedestrians know to be extra careful.
And for long haul trucking applications, freeway lanes at specific times of late night/early morning could be dedicated for autonomous driving.