bug#50877: 28.0.50; Gnus: nnimap backend is extremely slow to initialise new groups

[Eric Abrahamsen <eric@ericabrahamsen.net>]

Morgan Willcock <mwillcock@precedence.co.uk> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> You could try reverting 20f7fa691b7c2859b96550d9ccb326bf394e160d and see
>> if that fixes it. That change went in in April, though, so unless you
>> haven't updated for a while (or you've been seeing this problem for a
>> long time) it might not be likely.
>
> Reverting that commit fixes it for me.

I'm cc'ing Lars and Stefan Kangas, who are the most "involved" here. The
benchmarks are gruesome:

(defun old-gnus-set-difference (list1 list2)
  "Return a list of elements of LIST1 that do not appear in LIST2."
  (let ((hash2 (make-hash-table :test 'eq))
        (result nil))
    (dolist (elt list2) (puthash elt t hash2))
    (dolist (elt list1)
      (unless (gethash elt hash2)
        (setq result (cons elt result))))
    (nreverse result)))

(let* ((common1 (number-sequence 0 200))
       (common2 (number-sequence 501 700))
       (l1 (append common1
		   (number-sequence 200 500 2)
		   common2))
       (l2 (append common1
		   (number-sequence 201 500 2)
		   common2))
       (results (list
		 (benchmark-run 1000 (old-gnus-set-difference l1 l2))
		 (benchmark-run 1000 (seq-difference l1 l2 #'eq)))))
  results)->
  ((1.842791826 7 1.4057783080000021)
   (14.284496493999999 4 0.7823414249999985))

This is a particularly bad example -- when there was no common2
`seq-difference' performed better, but I think it still around 6 seconds
to the old function's 2.

Probably still worth a dedicated function for Gnus? (Or a rewrite of
`seq-difference'...)

Eric