Being a maintainer of the fastest XML library for Rust, I strongly disagree that XML is inherently fast to parse, and I question any such claim which comes with no evidence. Especially when it has remained unchanged on their page since (at least) 2008 [0]. Have you actually tested that claim or are you taking it at face value?
IME the XML spec is so complex that you either end up with a slow but compliant parser or a fast one that doesn't implement the spec completely.
JSON, unlike XML, is minimal enough that writing an entire compliant parser with SIMD intrinsics [1] is actually practically feasible. That library claims 3 GBps parsing speed, which could theoretically process your 120kb of data in 1/25000th of a second instead of 2/1000ths of a second.
I would wager that JSON is faster to parse, on balance.
[0] https://web.archive.org/web/20080209172554/https://rapidxml....
[1] https://github.com/simdjson/simdjson