It would be nice if this were true, but I did an experiment where I created a new account on a VM with a new IP not previously connected to me and it almost immediately started serving me right wing slop regardless of who I followed. It seems obvious to me from this anecdote that it’s not as simple as following or unfollowing.