The text of the 4th amendment says: "The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated..."

Which of those are things you transmit between people? Note the use of the word "paper" (connotative of something you keep in a desk, like a diary) versus the word "letter" (connotative of something you send to someone else).

The addresses of Postal mail have never been considered "your information" protected by the 4th amendment. The contents of Postal mail are considered protected, but given that the protection does not extend to UPS/FedEx/etc, it seems to be based on the unique status of the Postal service as an organ of the government.