I didn't think X worked this way, though, so perhaps it is some libinput setting. Or are you using XWayland, which may behave differently than standard X?