Another way of looking at it: If we have a function f(x), we can have an algorithm that computes f(x). In fact, there are an infinite number of algorithms that compute f(x). But there are also an infinite number of algorithms that compute f'(x), where f'(x) is an approximation of f(x). And such algorithms are called heuristics for f(x).
