This is part of him pleading guilty to these charges, one of which is over "more than one classified memorandum from a government intelligence agency". It's not him personally being shifty about it--that's what the charge is.
But of course then the question is how he was able to access them given the impression I get is his office was rather low security, not a SCIF.
He refers to the "T-SCIF" many times in this statement, 'T' meaning 'Temporary'. There's even a photo in the chat logs published by Wired.