Depends on the cartridge. Any of the ones with a battery save (like Zelda or Final Fantasy) need to be opened up, have the welded in battery clipped off the board, have a CR2032 socket soldered on instead, and then put a new battery in and reassemble.

It's a LOT of pain to get a NES cart with a dead battery working again. That's probably why they opted not to put a slot in this thing -- many people's dusty old carts are dead, and not knowing any better, they'd blame the new console.