That's a lot, but if we only used oil in the US for the military, we'd still be using 95% less oil, and we'd also still be producing enough oil to have an economy of scale.
In any case, realistically speaking, we aren't coming for oil tomorrow. Instead, we need to come for coal immediately. If it takes us decades each time we cut our oil consumption in half, that's one thing... but the coal, that needs to stop sooner.