It's really no problem as long as I'm under 45 MPH... as soon as I get over that speed, the wind noise (even with the windows up and my windrestrictor) makes it impossible for people to hear me.
For the most part I try and refrain from using the phone too much while I'm driving anyway.