After some debugging work, I think I find the cause.

See the discussion: https://lists.gnu.org/archive/html/emacs-devel/2019-10/msg01096.html