The passage of the Human Rights Act (which brings Convention rights into domestic law) was not a requirement of EU membership and we could have repealed it while still an EU member (this would be separate from leaving the ECHR itself). Moreover, reducing the impact of the convention and repealing/"reforming" the HRA was most notably proposed in the past by Theresa May as Home Secretary, who campaigned for Remain.

I don't think the ECHR/HRA is particularly likely to go away here any time soon. However the government may attempt to reform the way judicial reviews can be brought to limit their impact or remove the ability to bring "last minute" challenges which could have been brought in a more timely way (theoretically this is already supposed to be the case but in practice the politically controversial cases aren't often dismissed for lack of timeliness).