Technically fudge is just a crystalline sugar candy with a certain water percentage. Milk is optional (and frequently omitted). Reese's peanut butter cups are fudge, for example.
This explains better what I mean. https://en.wikipedia.org/wiki/Fudge#Texture - milk is added only as a stabilizer, and many candies that resemble fudge in texture incorporate milk or other stabilizers to prevent too rapid a crystallization. Fondant is just fudge without stabilizers.
