Manipulating actual flight controls is probably hard.
Manipulating sensors is probably comparably easy, and, if you can adjust the AOA sensors to make the plane think its in level flight while spoofing altitude data, you can probably manipulate the pilots (auto or not) into a CFIT.
It's happened on other flights where pilots became confused by conflicting information (AF447 and many others) so giving them consistently wrong data is going to cause a consistently bad output.
Anyway though I deeply agree that once you have physical access all bets are off. Sure, you can put a bug in which causes a complicated aircraft failure but you could also just put a time-delayed container of thermite or tannerite in the right place.
I'd be much more worried about a software update causing a major issue, either intentionally or even more likely and more worrying, accidentally ALA 737Max.