I live in a hurricane area and have gone through a number of major ones.
To answer your question there are many who simply cannot get out of the way of a hurricane. If they have a car, they dont have the money to put gas in it. There are others who are to sick or immobile to travel. Coastline areas are often evacuated with people going to shelters in an effort to be safe. Of course, if you recall what happened at the Superdome in New Orleans after Katrina, it makes sense why some might rethink that option.
Overall, I am with you. A cat 1 hurricane, I laugh at. However, if a cat 4 is barreling down on me, I pick my ass up and go. Sadly, that isnt the case for all people.