The issues with `scroll-down-command' (and `next-line', below) remain.
I find the following change fixes the problem for `scroll-down-command',
though I'm not sure whether it's the right thing to do.

--- a/src/window.c
+++ b/src/window.c
@@ -5148,7 +5148,7 @@ window_scroll_pixel_based (Lisp_Object window, int n, bool whole, bool noerror)
         in the scroll margin at the bottom.  */
       move_it_to (&it, PT, -1,
                  (it.last_visible_y - WINDOW_HEADER_LINE_HEIGHT (w)
-                  - this_scroll_margin - 1),
+                  - this_scroll_margin - 1 - frame_line_height),
                  -1,
                  MOVE_TO_POS | MOVE_TO_Y);

>>>
>>> Also, did you test these changes with scroll-conservatively set to
>>> 101?  If not, please do, as that setting activates some code parts
>>> that no other option does.
>>
>> I hadn't; trying it out now, it seems to cause `next-line' to also have
>> the bad behaviour of `scroll-down-command' where point is one line too
>> far down.

For `next-line', the initial difference seems to be in `line-move'.
When `scroll-conservatively' is non-zero (I found no change at 101, in
particular), it calls `vertical-motion` (via `line-move-visual'),
otherwise `line-move-partial'.

    (defun line-move (arg &optional noerror _to-end try-vscroll)
    ...
        (unless (and auto-window-vscroll try-vscroll
                     ;; Only vscroll for single line moves
                     (= (abs arg) 1)
                     ;; Under scroll-conservatively, the display engine
                     ;; does this better.
                     (zerop scroll-conservatively)
                     ...
                     (line-move-partial arg noerror))
          ...
              (prog1 (line-move-visual arg noerror)

The problem involves partial lines.  In a window where
(window-screen-lines) returns 7.222, doing M-: (vertical-motion '(0.0
. 1)) does not scroll the window, which lets point end up 1 line away
from the center.  Doing M-: (vertical-motion '(0.0 . -1)) does scroll
the window, keeping the point in the center (as expected).

Adjusting the window so that (window-screen-lines) returns 7.0, there is
no discrepancy between up and down motion.

I guess there is some incorrect boundary condition in try_scrolling,
though I haven't worked out where.