For instance, in species with sexual differentiation, two males or two females cannot mate, but are considered the same species.
For a more precise definition, you would have to, say, look at the probability that a pair of organisms of the appropriate sex from two different populations would be able to mate, and produce viable offspring (that is, offspring that could themselves mate and produce more offspring).
Asking about a single organism that happens to be infertile, or a single pair that happens not to be able to mate or produce viable offspring, is not particularly interesting, but at a population level you could use a statistical test to get a probability distribution, rather than a yes or no answer.
See https://en.wikipedia.org/wiki/Species#Difficulty_defining_or... for more information.
Note also that it can be the case that two different species can mate, and produce offspring, but the offspring itself is not viable (cannot mate and produce offspring). This happens in the case of mating two closely related species, such as when mating a horse and donkey to get a mule (https://en.wikipedia.org/wiki/Mule), or a lion and a tiger to get a liger (https://en.wikipedia.org/wiki/Liger).
Plants can clone itselves so to be infertile is not the same as to be extinct for them. Infertile also means normally something like "99,9999% unable to produce seeds" but with enough time and a high number of tries, planets will align in the polinisation of two "unfertile" stems of the same plant. At the end seeds will be produced and the new species will exit the 'trap' perfectly (or all DNA of the father will be discarded reverting to the original mother species). Is just a question of time.
The concept of species exists purely as a human, scientific abstraction: evolution/biology doesn't give us absolute categories that you can unambiguously put any organism into.
That's a somewhat rare version of a general pattern where that abstraction is leaky. Much more common is a temporal version of that. A might beget A1 and B, with A1 begetting A2 and B begetting C, continuing all the way down. And though A can interbreed with A1 and B, and B with C, etc, Z can't interbreed with A25. That's speciation, which happens all the time. The only thing that makes it a clear species is defining the concept species while abandoning time invariance.
In all seriousness, species definition is a case of trying to define a black/white classification in a gradient. It might sort of work in higher animals, such as mammals. But it certainly breaks down in bacteria, which is clearly seen since it's possible to compare genomes on population level.
See https://en.wikipedia.org/wiki/Subspecies
"A taxonomist decides whether to recognize a subspecies or not. A common way to decide is that organisms belonging to different subspecies of the same species are capable of interbreeding and producing fertile offspring, but they do not interbreed in nature due to geographic isolation or other factors."
"The domestic dog lost its recognition as a separate species in 1993, and is now considered a subspecies of gray wolf."
The biological species concept is probably one of the most popular and very similar to what you are suggesting. In this concept, a species is an interbreeding population that is reproductively isolated from other populations (either due to genetics or geography).