The above is a screenshot of Emacs with the patch applied opened in GIMP
and zoomed in very far.  I've placed the crosshair on the problematic
pixel column.

If you know why that happens, this could constitute a fix with that
tweak applied.

Just using :align-to header-line-indent-width appears to have even worse
results, because ISTM that it gets rounded after scaling is applied, so
it is of the wrong width by different amounts for each text-scale.  Even
without any text-scale applied, it has the same extra-pixel problem.

I think the above is close to correct, though, it perhaps implies that
header-line-indent-mode should get some love in order to be
pixel-perfect in pixel specifications too.

>> Interestingly, changing text-scale-remap-header-line appears to have no
>> effect, but that variable ought to be accounted for somehow also, I
>> think?  Though, if the line-numbers and header line always use the same
>> face, then the above should be correct, I think?
>
> If you take a look at ruler-mode.el, you will see that it overrides
> header-line-format by its own code.

Yes, but the name and logic in text-scale seemed to imply to me that
text-scale would take the header-line-format set up by ruler-mode and
"alter" it somehow.
-- 
Arsen Arsenović