That wouldn't do it by itself, actually.
Let us examine the Village of Virtuous Women, where there are 100 men and 100 women. 99 wives are faithful to their husbands. All the men are unfaithful with the 100th wife (except, of course, her husband -- who I really feel for).
Average number of partners per man: (2 + 2 + 2 + ... + 1) / 100 = 1.99
Average number of partners per woman: (1 + 1 + 1... + 100) / 100 = 1.99
You can also justify this with "Sum over a gender of number of sexual partners equals the sum over the gender of sexual partnerships, and if we only consider heterosexual partnerships, then this must be symmetric for both genders. Thus, since number of people in both genders is approximately equal, average number of sexual partners must be approximately equal."
So what actually causes the disparity? It has to be either a) untruthfulness or b) unsampled outliers. For example, if in the Village of Faithful Women the unfaithful wife got missed in your phone survey, you'd blow the results completely now wouldn't you.
But in the instant case, it is highly likely that the answer is in fact untruthfulness (both overreporting and underreporting).