I wrote a simple patch for merely checking whether the buffer about to be killed happens to be the remember-notes buffer.