bug#51982: Erroneous handling of local variables in byte-compiled nested lambdas

["Mattias Engdegård" <mattiase@acm.org>]

21 nov. 2021 kl. 08.59 skrev Michael Heerdegen <michael_heerdegen@web.de>:

> If arbitrary fixes would be allowed at any point of the development cycle
> the result would be worse than what we have now.  It's also a quite
> common policy.

Yes, but it does cause some inconvenience in a project with Emacs's geological release pace. It is difficult to defend an inability to rapidly release an updated version when a serious bug has been found.

Now, regarding the actual bug. Consider the function

1 (defun f (x)
2   (lambda ()
3     (let ((g (lambda () x)))
4       (let ((x 'a))
5         (list x (funcall g))))))

First of all, the variable x is free in the function starting in line 2, so that function is converted to a closure capturing that variable explicitly.

Next, the function bound to g will be lambda-lifted; ie, converted to (lambda (x) x) which means that the call to g in line 5 must be amended to include the value of x. However, we can't just change (funcall g) to (funcall g x) because x is shadowed by the binding in 4, so a new variable is introduced for this purpose.

The result after cconv is essentially (without the fix):

1 (defun f (x)
2   (internal-make-closure nil (x) nil
3     (let ((g (lambda (x) x)))
4       (let ((x 'a)
5             (closed-x x))
6         (list x (funcall g closed-x))))))

But x is not the right expression for closed-x, because it is a captured variable. The patch fixes this:

1 (defun f (x)
2   (internal-make-closure nil (x) nil
3     (let ((g (lambda (x) x)))
4       (let ((x 'a)
5             (closed-x (internal-get-closed-var 0)))
6         (list x (funcall g closed-x))))))

As I mentioned previously, it would be probably be better to elide closed-x entirely and produce

1 (defun f (x)
2   (internal-make-closure nil (x) nil
3     (let ((g (lambda (x) x)))
4       (let ((x 'a))
5         (list x (funcall g (internal-get-closed-var 0)))))))

In other words, the bug occurs when a variable is captured, lambda-lifted, and shadowed.
I need to take another good look at the code to make sure the change is correct (more eyes on it would be appreciated).