You don't always need an operator license but, when you don't, you typically do need properly certified equipment. (This is also why you don't need a license to use Wi-Fi.) The rules vary by service and by frequency band.
I'm no expert on boat RADARs, but I'd bet that, 1) they operate in bands allocated for that sort of use and 2) that they have been either type accepted or certified by the FCC to comply with whatever rules govern their band of operation.