Maybe you are, maybe you aren't. As the website's operator, I don't want to manually have to go back-and-forth with everyone who forgets their password to try and verify their identity through their past activity etc.

Making it optional for the people who won't forget is fine, there's no way to know who that will be though. And if your password manager fail you'll be here on HN pointing out how I made a website with not a single fallback mechanism to get back in your account. Why would anyone do that?