Here's what I have always said. Its never gonna happen. The reason: Lawyers. Eventually there will be a tragic death and it will be blamed on the driverless car. The lawyers will sue anybody and everybody involved into oblivion. It will kill the industry.
You could argue all day about the overall statistics, about how human drivers kill others all the time, and how safe the driverless car is overall, etc. It won't matter. When the right case comes around and a young child is killed, or a whole family, and there is any inkling that the driverless car was at fault, they will lose.
Heck, there are plenty that have been killed already, mostly in those Teslas. Those Teslas do doubt will drive themselves, but you aren't "supposed" to do that. But of course, people do it all the time. I always make a game when I am on the freeways, whenever I go by a Tesla, I look over and see if the driver has their hands on the wheel. I would say, somewhere around half the time, they don't. This is easiest in the Bay Area, where there are tons of them driving around.
Tesla so far has avoided the lawsuits, based on the fact that they have specifically said to the owners "you are supposed to keep your hands on the wheel and pay attention". Then Musk winks and says hey we have a new and improved self driving mode. Whether this is fair or not, you decide. The Joshua Brown crash was a total fail of the system and he was wiped out into oblivion. But, no lawsuit, as it was deemed his fault for not paying attention and letting the car drive itself.
There are just too many possible shortcomings that will require ever-increasing levels of AI and computing power to resolve, some of which have been mentioned. The fog, the rain, the dust, the unplanned construction detours, and so on. And the hacking? If it can done, some griefer somewhere will do it.