The UI needs to be more complicated than that. I've forgotten to note the finish time, and have needed to update things after the fact. Suppose, for example, that I've forgotten to exclude lunch from the timing system. With a pen&paper system I can mark "(lunch, 45 minutes)" after the fact. There are plenty of computer-based systems which can do the same, but their UIs are more complicated than the one you just outlined.

Also, with pen&paper billing is usually quantized to 10 or 15 minute intervals, which makes it easier to do the arithmetic.