The gravity field of a spherical object only depends on its mass and not its density/radius. So for the Roche limit only the density/radius of the object at risk of breaking apart (called minor object on wikipedia) (earth) should matter, not the one of the major object (black hole).
Wikipedia gives the formula d=R_m*(2m_M/m_m)^{1/3} which matches that intuition.