I think having a unified workflow is a good thing, and that that can exist along side important special cases.

A tool like inkscape gets a ton of "casual" once-in-a-while use and some amount of frequent power-user use. It's tricky to facilitate both kinds of use, but I think that should be a goal.

What you wrote is not ridiculous like the following statement, but consider "Yeah, you have to draw by using your text editor on an SVG file. There are not two workflows, one for drawings and one for text, which is a good thing."

If I were approaching the problem fresh, though, I would look into the presentation of the modal functions as the root of the issue. The source of a lot of complexity in image editing software is confusion over layers, masks, groups, and selections - they have some conceptual overlap and can be applied modally(e.g. shape as rendering element OR mask OR selection). If the UI buries the difference it just adds to the confusion.