Close your office door?
Noise and visual distractions are largely created by design decisions...that loft with brick walls and hardwood floors looks really cool, but carpet, acoustical tile, and gypsum absorb sound energy. Open space plans in live spaces make the issues worse.
With noise, the second strategy is sound masking. For example, having a small fan generating ambient sounds will reduce the distractions caused by sounds to which we are attuned such as people's voices. Having your earphones on low serves a similar purpose.
Opening a window helps on two levels. It reduces the hard surface reflective area and introduces masking noise. But like all anti-distraction measures, it requires the occupant to have meaningful control over their environment.
I once had an assigned office and where the desk and return placed my back to the door (and hallway beyond) and staring at a blank wall behind my monitor. I got crap from my boss for rearranging the furniture so that I could face the door because it required walking around the end of the desk to sit at it...that's what working for architects is like.
Good luck.