I generally use rxvt because xterm is too slow. It doesn't have menus or URL highlighting and scrollbars are nonfunctional in the presence of screen, tmux, Emacs or generally any interesting terminal app.

On Windows I use mintty and I turn off scrollbars there. I simply don't use the mouse to interact with the terminal other than to select text, and that's with selection buffer to copy.

Speed is highly relevant to me. Most modern terminal emulators are very slow, most noticeable when you get a lot of output in a panel in something like tmux.

(Simplistic benchmarks that test full screen scrolling usually hand the crown to terminals that don't bother to refresh the screen with everything output, but that's not the only bit of a terminal emulator that can be slow.)