I just realized that the initial term-height didn't use floor to round down the value of window-text-height. This was masked by a hook I have in my configuration. I attached another patch that fixes the issue.