bug#72849: [PATCH] Keep project's exec-path during with-temp-buffer call

[kobarity <kobarity@gmail.com>]

Evgenii Klimov wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Date: Wed, 28 Aug 2024 00:13:25 +0100
> >> From:  Evgenii Klimov via "Bug reports for GNU Emacs,
> >>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> >> 
> >> `with-temp-buffer' doesn't respect buffer-local environment variables,
> >> `exec-path' in this case.  Which results in executables not being found,
> >> or the wrong versions of executables being picked up.  E.g. if
> >> environment variable is modified via .dir-local file or direnv/envrc
> >> package.
> >
> > Hmm, this doesn't look clean to me: exec-path is just one variable,
> > what makes it special here?
> >
> > Moreover, it sounds like python-shell-with-environment, which
> > python-shell-prompt-detect calls, already attempts to have
> > buffer-local value of exec-path to be available to Python, so why
> > isn't that working for you?  And if it isn't work, I think we should
> > amend python-shell-with-environment to do this, so we don't need to do
> > it "by hand".
> 
> Indeed, my initial approach is too manual.
> 
> Here the problem that I have: I don't use Python's "venv" module to
> create virtual environment for the project.  Instead, I use GNU Guix's
> "guix shell" command [1] which provides augmented PATH and PYTHONPATH,
> etc. to link project's dependencies.  Then, envrc.el package picks up
> these environment variables and makes them buffer-local project-wise
> (`exec-path' and `process-environment').
> 
> You are correct that `python-shell-with-environment' provides
> buffer-local variables, but `with-temp-buffer' treats `exec-path' and
> `process-environment' variables very specially.
> 
> I didn't find this behavior in documentation, but look at this example:
> 
>   (setq-default exec-path (list "global" "list"))
>   (setq-local exec-path (cons "local"
>                               (default-value 'exec-path)))
>   (setq-default myvar (list "global" "list"))
>   (setq-local myvar (cons "local" (default-value 'myvar)))
> 
>   (let ((exec-path exec-path) 		; takes buffer-local
>         (myvar myvar)) 			; takes buffer-local
>     (with-temp-buffer
>       (insert (car exec-path) 		; uses global
>               "\n"
>               (car myvar)) 		; uses `let'-binded
>       (buffer-string)))
>   ;; => "global
>   ;;     local"
> 
>   (require 'cl-lib)
>   (let ((myvar myvar))
>     ;; temporarily binds buffer-local value to global symbol
>     (cl-letf (((default-value 'exec-path) exec-path))
>       (with-temp-buffer
>         ;; global variable is used, but it's value is temporarily equal
>         ;; to buffer-local value
>         (insert (car exec-path)
>                 "\n"
>                 (car myvar))
>         (buffer-string))))
>   ;; => "local
>   ;;     local"
> 
> It's a simplified and expanded version of
> `python-shell-with-environment' and `python-shell-prompt-detect'.  As
> you can see, `exec-path' is treated differently inside of
> `with-temp-buffer' and `cl-letf' is needed to force `with-temp-buffer'
> to use buffer-local value of `exec-path'.
> 
> In the new patch attached I show how this can be overcome.  Don't know
> if you'll consider my use case too narrow and specific, but I'll be glad
> to hear your thoughts on this.
> 
> [1] https://guix.gnu.org/en/manual/en/guix.html#Invoking-guix-environment

Thank you for your explanation and the new patch.  It helped me to
understand the problem correctly.  I also confirmed that the new patch
resolves the issue.  I don't think it's a special use case, as I
sometimes set `exec-path' locally using .dir-locals.el.  I agree with
the new patch.