Basically I'd say there are 3 parts in learning improvisation: the theoretical part (which is already a complex one because there are several possible theoretical approaches to jazz improvisation), the imitation part (learn and play back existing phrases, or better complete solos from the masters), the by-ear / singing part (the toughest one that most actually don't reach, at least reliably).
Don't forget that there is definitely a large social part in jazz improvisation; I'd rate my improvisation ability as uninteresting most of the time; I can get "in the zone" accidentally by myself, but more often it's a band thing: playing with the right people often enough and long enough to get together in "the zone". Yes, that's exactly the same zone as the programming one; you're lost in the music, feeling what's coming next and what notes should be played by whom (there only comes your technical ability in the picture) without thinking about it.
In my personal case, a long practice of classical piano hampered my early capability at improvisation for a long time and I needed the crutch of theory, and to intellectualize the process. Some blessed people "ear" the right notes without needing any justification "why" they are the right ones.
When you've got a long practice of your instrument, the difficulty is to free yourself from the reassuring but useless knowledge and habits you have that bring you to play this scale or this phrase because it's "in your fingers". The best way to reach that point is to have hundreds of ready phrases in all tones "in your fingers", and then try to forget them and listen to the music. Hopefully, you'll feel what goes where, like an unrolling, animated puzzle, or a Tetris game.
Personally I've found Kent Hewitt's advice to be very useful, I think it may help a lot of people. What's great in his playing is that he keeps it very simple, but always richly melodic. https://www.youtube.com/channel/UCdmjw5sm9Kn83TB_rA_QBCw
Another piece of advice I can give you is to learn to recognize chords and all the different ways they're built (by stacking thirds or fourths, etc) and how they come in succession (the usual II V I VI and friends) to get a better feel of what comes here or there.
