Your assumption is correct. Men's and women's toilet facilities are counted separately for each sex by OSHA. The exception is single-occupancy rooms, which can count for either sex.
(It's not clear the exact algorithm one should use in counting a mixture of multi- and single-person restrooms, but I would presume it's something like, count all the men that can use your male restrooms, count all the women that can use your female restrooms, and take everyone left over and see if they fit is your unisex restrooms.)
