I know what you are talking about. I noticed this especially since I drive using cockpit view and there's no wind in that view. A few times I've cycled out of cockpit view by mistake and I hear the loud wind noise in all views. Sometimes is stays on when I go back into cockpit view.
The opposite almost occurs in another Codemasters game, GRID Autosport, where if you lose your side window or windscreen, you can hear loud wind rushing past, but as soon as you use the "look behind" button, the wind disappears! Don't think they've quite got this wind thing nailed down!