Are you sure? Gas consumption is notoriously inelastic. West coast gasoline is already a dollar or more than it costs on the east coast. Do poor people drive less in California than in Florida?
Gas consumption is inelastic in the short term, but everything is elastic in the long term.
If you want proof of this, just look at what happens to sales of large vs small cars when the price of gas changes.
I think everyone drives less in California than in Florida. (Google says ~14,500 miles annually per licensed driver in Florida, versus ~12,500 miles in California.) Gas prices are a factor in this.