Re: cl-block less lexical than expected

[Robin Tarsiger <rtt@dasyatidae.com>]

I wrote:
 > I have to think that since catch and throw both take evaluated
 > expressions for their tags, a more suitable implementation might
 > do a static transformation to choose an out-of-the-way lexical
 > _variable_ name which would then be let-bound to a gensym during
 > the body. cl-return-from could then try to evaluate the variable
 > to get the tag;

Stefan Monnier wrote:
> Sounds good.  As you noted, this was not an option when these macros
> were written because of the need for statically scoped variables.

So I've been experimenting a little with modifying cl-macs.el to make
this happen. I think I have it about right, but here's what I've found
that's tricky:

   - cl--block-wrapper currently assumes it's receiving exactly a form
     produced by cl--block and invasively unwraps it, so they have to
     be kept in sync. This isn't too bad when it's just a catch form,
     but the new form is much more highly nested.

     I imagine having cl--block-wrapper do most of the expansion itself
     and be implemented as macro+cmacro rather than passthrough+cmacro
     would be cleaner here?

   - The new form is more highly nested not just because of the let to
     keep the tag in a variable, but because of an unwind-protect used
     to clear out the variable when the body is exited. This is so that
     lambdas that cl-return-from the enclosing block function properly
     when called while the block is executing, but can _not_ erroneously
     return from a separate activation of the same block if called after
     the original activation of the block has exited. Instead, they will
     wind up throwing to a dummy tag and signaling a recognizable error
     thereby.

     Naturally, the existing code misbehaves in this scenario as well.

   - I currently have cl-block warn if lexical-binding is nil. I did a
     quick pass over the Lisp files included with Emacs to make sure
     all files with cl-block-alikes used lexical-binding, but I don't
     know what impact this would have on external packages.

   - It would be nice to have cl-return-from warn if it doesn't see the
     name in the list of extant blocks that the cl--block-wrapper cmacro
     already uses to optimize away the catch, but that's not reliable
     enough, right? It doesn't play nicely with macrostep, for instance.
     Is there a better way to do this?

   - Documenting the new behavior is a bit weird due to the potential
     interaction with lexical-binding. I have a few paragraphs in the
     docstrings Texinfo manual indicating that cl-return-from may
     behave erratically if there are any non-lexical-binding cl-blocks
     in effect.

     The existing documentation is not quite correct anyway.

More general questions for the list:

   - Is this a worthwhile thing to fix in Emacs? I would say yes, else
     I wouldn't be trying it, but I don't know if other people agree.

     The complexity increase in documentation from interactions with
     non-lexical-binding code is awkward, especially. How much do we
     need to worry about that? Is the general trend for lexical-binding
     to be strongly recommended enough that not giving details of
     what happens if it's off (other than that the correct use cases
     should still work) is okay?

   - If so, do I sound reasonably on the right track above? Would it be
     helpful to present a draft patch?

   - The reason I have not already presented a draft patch is because in
     futures where the code is integrated into Emacs, I would have to
     perform a copyright assignment. I do not object to this, but I also
     do not use my government-recognized name as my social name in this
     context. I am okay with the FSF keeping a record of my government
     name for legal purposes, but I would rather it not be included in
     publications anyplace too obvious. Is this a request that I can
     expect to be honored? What would be the impact of this on the
     assignment procedure? I am a US citizen.

-RTT