You just don't sign a contract where you give a buyers agent commission, although again this limits the pool. Frankly, buyers agents are largely a scam in my view. With MLS, Zillow, Redfin, etc, their 'experience' seems mostly worthless. It certainly is not worth 3% of the cost of a house.
If anything, sellers agents are more useful since they actually offer something valuable to the real estate experience (e.g., listing housing, photos, providing buyers access, etc).