You don't need everyone to make that decision to make a meaningful difference in congestion. If everyone drove at 4 am instead, the highway would be empty at 8 am!
Keep in mind, that person has other choices: live closer to work, work closer to home, take transit, carpool, etc.
My workplace has flex hours. Some people work 7-3 to avoid traffic. Other people work 9:30/10 - 6 + use the gym after work. You don't need people who start shiftwork at 9 am to drive at 4 am and sit in their car at their workplace parking lot for 4 hours before their shift starts to make a difference in congestion.