That's a common sentiment among people who aren't terribly familiar with the scientific background for dark matter. I would urge you to go take a look at the wikipedia entry on dark matter and read through it, it's a good overview of a lot of the lines of evidence. The fact is that scientists have always been reluctant to believe in the existence of some exotic, new form of matter that hasn't been seen or directly detected, but in trying to disprove its existence or to prove that it's really just something else they've consistently, time after time, been stuck with that as the only option that explains the evidence. And there is quite a lot of evidence, from cosmological simulations to gravitational lensing studies to galactic rotational curves and so much more.

If dark matter is what we think it probably is then it would actually make for a fairly elegant explanation. The overwhelmingly likely story is that dark matter is a kind of "supersymmetric" particle which requires very high energy conditions to create, conditions which do not generally exist much anymore, and which interacts very weakly or not at all with electromagnetic and strong nuclear forces. Highly energetic conditions were quite common in the early period of the Universe just after the big bang though, and if it were at all possible for such weakly interacting particles to exist then they would have been created in abundance, sapping a lot of the total mass/energy away from the Universe as they "fell out" of the energetic soup of the Universe. Eventually the Universe would have cooled enough to cause creation of such particles to halt, after which it would have gone on to create the more ordinary matter we know today (photons, electrons, protons, etc.)

Fortunately we're likely to get a lot more data on the subject in the next few decades. As I mentioned our capabilities of directly testing the existence of dark matter particles are increasing to the point where within one to two decades we will likely have direct confirmation of their existence.

I put the chances of dark matter being something other than a weakly interacting massive particle to be fairly low, though not impossible of course.

As for dark energy, the evidence for the accelerating expansion of the Universe is much newer and the hypotheses attempting to explain it much more circumspect, I wouldn't be surprised if it turned out to be something other than the crude guesses we've made at present.