Depends when you start as well of course. It's leaving from as soon as it hits your bloodstream, not from when you finish drinking.
I don't believe that everyone gets rid at 1 unit per hour either, although that's probably a fair average. A 7 stone petite lady will probably get rid at a slower rate than a 16 stone professional footballer.
For those who are whiter than white, bear in mind that the vast majority of deaths on UK roads are caused by sober people. And that even when a driver involved in an accident has been drinking, that doesn't necessarily mean that he (or she) caused the accident. It does, however, suit the authorities to publish statistics which suggest this is the case, same as they do with speeding.
Just to be clear, I don't condone drink driving or driving with excessive speed for the conditions under any circumstances.