I've tested this out and it's definitely noticeably slower. It
actually happens infrequently enough that I may actually be willing to
trade the speed for the correctness, but I'll use it for a while and
see how it goes.

Realistically, it looks like it may be the difference between being
able to scroll one line at a time reasonably and not, which is going
to hit spacemacs users and people that are used to scrolling in that
way hard. I know scrolling in that way is not the recommended
approach, but it's nicer experience than having the screen and the
point jump half way across the screen every time you need to see one
more line.

Aaron