Below is a patch that seems to fix this. I haven’t tested the actual patch but I’ve added its changed to my init file (adding the new defvar, and redefining erc-display-message and erc-lurker-update-status) for the last couple of days, and it seems to work as intended. Thanks to Noam for helping me with this over on #emacs.