1) Lose the passphrase. The device does quiz the user on words of the passphrase upon setup, to be sure the user at least wrote it down. Hopefully this reinforces its importance. Ledger provides a card to write the words, with prominent instructions to save it somewhere safe.

2) When sending, don't verify on the device's display that you're sending the money where you think. But the display is right there on the device, and displays the destination address while waiting for you to push a button right next to it.

For smart contracts it helps to get a larger device that shows all the contract parameters, but for simple sends that's all there is to it.