I built a small project that cleans CSV files for ML and data analysis.
Using pandas in the backend, cleaning a 65 MB file took over 200 seconds and frequently timed out. Just parsing the upload took over 120 seconds.
After migrating to Polars and adding in-memory DataFrame caching between the upload and cleaning endpoints, the same file now cleans in under 4 seconds. Smaller files feel nearly instant.
Has anyone else made this migration in a production app? Curious about edge cases you hit, especially with type inference, null handling, and lazy vs eager evaluation.
