> It is not clean to make this work in font-lock mode only.

Agreed.

  > It is worth a bit more complexity to make the code work in
  > all cases.  And it is also more modular not to depend on
  > what font-lock does for this mode.

I moved the stuff away from `after-change-functions' _and_ font-lock
mode.  Could people please try whether it works as intended.