There are two fundamental problems.
First is the scrollbar thing I mentioned. It means that you can’t use it for layout purposes; you thought you could sit four blocks side-by-side with `width: 25vw`? (And that was pretty much the whole reason why people wanted viewport units in the first place—this being before flexbox provided alternatives that are generally acceptable, though not without flaws, or grid provided often better alternatives.) Sorry, that’ll only work on a platform with overlay scrollbars, not where scrollbars actually take up space. There’s fun history around Firefox’s implementation where they made it possible to get the “right” behaviour, but no one else implemented that, so it was eventually removed from the spec.
Secondly, it was also predicated on the idea that the physical viewport size won’t be changing all the time; but on mobile platforms where the address bar can get out of the way as you scroll that’s simply not true, so you’re stuck with browsers having to decide between two interpretations: vh picking either the smallest or largest value and thus not actually reflecting the viewport height half the time, or vh reflecting the viewport height and causing the page layout to jump around in a jarring fashion when the address bar expands or collapses, when vh is used as part of the layout.
The first can be fixed in two ways (which can be combined): firstly, by defining new units that exclude document element scrollbars (e.g. 100vw2 ≅ calc(100vw - env(scrollbar-layout-width-auto, 0px)), to define it in terms of what follows); secondly, by exposing the layout width of scrollbars to CSS, e.g. as env(scrollbar-layout-width-auto) for the width with `scrollbar-width: auto` and env(scrollbar-layout-width-thin) for the width with `scrollbar-width: thin`. These values would be something like 17px and 8px on Firefox on Windows, and 0px and 0px on platforms with overlay scrollbars.
The second, you probably need to expose new constants for the possible extreme values of viewport height. e.g. I could imagine env(viewport-min-height, 100vh) and env(viewport-max-height, 100vh) working, which would then allow developers to select one or the other, as suited their purpose. Or define new units opinionated on which value should be used, and then deprecate vw and vh since they’re inconsistently implemented.
(In using env(), I must caution that it’s currently rather broken for some sorts of situations: https://github.com/w3c/csswg-drafts/issues/3285.)