One can still measure the actual character width using
window-text-pixel-size and take it into account.
 That's correct. width is pixel width returned by (posn-at-point) and w is (font-get (font-at (point)) :size). 

This assumes that, e.g., #x2001 takes the same horizontal space as the
SPC character (or the same as a column)?
The code is only assuming what the specs prescribe. It makes no guarantee for non-standard fonts.

The complete code in action, if anybody is interested, is here https://gitlab.com/atamariya/emacs/-/blob/dev/lisp/cedet/company.el#L2562 .