I agree. I think the French are doing the right thing here. I only mentioned it because it's one of the many things governments do which encourage and normalise more coming.
British politician could try to strike a deal with the French to stop them helping individual who try to enter the UK illegally, but obviously it would only make sense for them to do that if the British were also trying to stop people entering the country illegally. Like you say, we can't expect the French to defend UK borders.
In regards to leaving the EU this wouldn't change the situation meaningfully, it would just allow for more cooperation in how illegal migrants are distributed across Europe. Like the UK, EU countries are also legally not allowed to deport Afghans. It's not as if the EU doesn't face similar problems.