bug#63648: 29.0.90; project.el: with switch-use-entire-map, switch-project errors on non-project commands

[Dmitry Gutov <dmitry@gutov.dev>]

On 31/08/2023 19:36, Juri Linkov wrote:
>>>> - Unfortunately, using default-directory instead of the specialized
>>>>     variable which we added lately (project-current-directory-override)
>>>>     brings back the bug it was added for: https://debbugs.gnu.org/58784. The
>>>>     switch to a different design didn't fix the problem of the temporary
>>>>     binding for d-d in the buffer which is current when the command is
>>>>     executed. So adding the next-default-directory variable might not be the
>>>>     best idea after all. But the advice thingy can set a binding for any
>>>>     variable, including the *-override one.
>>> I didn't test yet the cases from bug#58784.  So this might require more
>>> changes.
>>
>> We tried to make the default-directory binding work with a couple of
>> different changes - it didn't work out.
> 
> The problem is that let-binding overrides the buffer-local value:

Yep.

> ```
> (let ((default-directory "/tmp/"))
>    (list default-directory
>          (buffer-local-value 'default-directory (current-buffer))))
> => ("/tmp/" "/tmp/")
> ```
> 
> Here is the shortest test case: 'C-x p p C-b' shows buffers
> from two projects when using let-binding for default-directory,
> because 'project-buffers' relies on
> 
>    (buffer-local-value 'default-directory buf)
> 
> This could be fixed by adding special-handling of the default-directory
> for the current buffer in 'project-buffers'.

What kind of special handling? The "real" buffer-local value is hidden 
until the "let" exists, the global value is nil, and if the buffer is 
not a file-visiting one, there is no other file name to test against.

Finally, whatever special handling we invent, would have to be mirrored 
by all subsequent new commands (built-in and third-party) which look up 
the value of default-directory. Especially project-related ones. How to 
popularize that knowledge, would be the next question for whatever 
solution we invent.

>> A general question regarding this approach, for me, is: is a "prefix
>> command" a real command?
> 
> It's a real command that prepares some transient values for the next command.
> Most existing prefix commands have no problems because they modify global
> values.  But 'default-directory' is the first buffer-local variable
> used for the next command, therefore it requires special-handling.
> Too bad that 'default-directory' is not a function.

I suppose the new code could check against some property or dynamic var 
to distinguish prefix commands from "terminal" ones.

>> And would they "swallow" these prepared variable bindings too early?
> 
> The scope of the modified variable depends on concrete needs.
> For example, set-transient-map restores its old value in pre-command-hook,
> but display-buffer-override-next-command does this in post-command-hook.

That can also work - though the odds of getting into an unrecoverable 
state (such as one I described) would be higher.