Yes, it can get tricky if you have to scroll a bunch, e.g. moving a file in a big directory into a subfolder, trying to hit that one pixel where it will scroll up, or using two other fingers to attempt to scroll, while holding the drag finger down...(CLI pros, you win this one).
I would like a desktop pick and place that works like drag and drop, you click and then it sticks to the cursor, but you are free to do whatever gestures until you click again.