Interesting fact about foldl'. Regardless, in practice it is strict and tail recursive. As I mentioned earlier, this does not mean the same thing as constant space unless the reduction function returns a fixed size result.
Yes, you can guarantee that a linked list in Java is finite because Java does not support codata.
Haskell's tail call recursion is also often optimized to be allocation-free, unless, again, it is generating some data structure.