On desktop and mobile devices:
While playing a video, select the Settings button.
Locate the Ambient Mode setting in the list of preferences.
Toggle it to off to disable Ambient Mode for all videos on YouTube (in that browser).
It's in the same popup used for video quality and playback speed.