bug#72849: [PATCH] Keep project's exec-path during with-temp-buffer call

[Eli Zaretskii <eliz@gnu.org>]

> Date: Fri, 30 Aug 2024 23:43:57 +0900
> From: kobarity <kobarity@gmail.com>
> Cc: 72849@debbugs.gnu.org
> 
> Evgenii Klimov wrote:
> > Eli Zaretskii <eliz@gnu.org> writes:
> > 
> > >> Date: Wed, 28 Aug 2024 00:13:25 +0100
> > >> From:  Evgenii Klimov via "Bug reports for GNU Emacs,
> > >>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> > >> 
> > >> `with-temp-buffer' doesn't respect buffer-local environment variables,
> > >> `exec-path' in this case.  Which results in executables not being found,
> > >> or the wrong versions of executables being picked up.  E.g. if
> > >> environment variable is modified via .dir-local file or direnv/envrc
> > >> package.
> > >
> > > Hmm, this doesn't look clean to me: exec-path is just one variable,
> > > what makes it special here?
> > >
> > > Moreover, it sounds like python-shell-with-environment, which
> > > python-shell-prompt-detect calls, already attempts to have
> > > buffer-local value of exec-path to be available to Python, so why
> > > isn't that working for you?  And if it isn't work, I think we should
> > > amend python-shell-with-environment to do this, so we don't need to do
> > > it "by hand".
> > 
> > Indeed, my initial approach is too manual.
> > 
> > Here the problem that I have: I don't use Python's "venv" module to
> > create virtual environment for the project.  Instead, I use GNU Guix's
> > "guix shell" command [1] which provides augmented PATH and PYTHONPATH,
> > etc. to link project's dependencies.  Then, envrc.el package picks up
> > these environment variables and makes them buffer-local project-wise
> > (`exec-path' and `process-environment').
> > 
> > You are correct that `python-shell-with-environment' provides
> > buffer-local variables, but `with-temp-buffer' treats `exec-path' and
> > `process-environment' variables very specially.
> > 
> > I didn't find this behavior in documentation, but look at this example:
> > 
> >   (setq-default exec-path (list "global" "list"))
> >   (setq-local exec-path (cons "local"
> >                               (default-value 'exec-path)))
> >   (setq-default myvar (list "global" "list"))
> >   (setq-local myvar (cons "local" (default-value 'myvar)))
> > 
> >   (let ((exec-path exec-path) 		; takes buffer-local
> >         (myvar myvar)) 			; takes buffer-local
> >     (with-temp-buffer
> >       (insert (car exec-path) 		; uses global
> >               "\n"
> >               (car myvar)) 		; uses `let'-binded
> >       (buffer-string)))
> >   ;; => "global
> >   ;;     local"
> > 
> >   (require 'cl-lib)
> >   (let ((myvar myvar))
> >     ;; temporarily binds buffer-local value to global symbol
> >     (cl-letf (((default-value 'exec-path) exec-path))
> >       (with-temp-buffer
> >         ;; global variable is used, but it's value is temporarily equal
> >         ;; to buffer-local value
> >         (insert (car exec-path)
> >                 "\n"
> >                 (car myvar))
> >         (buffer-string))))
> >   ;; => "local
> >   ;;     local"
> > 
> > It's a simplified and expanded version of
> > `python-shell-with-environment' and `python-shell-prompt-detect'.  As
> > you can see, `exec-path' is treated differently inside of
> > `with-temp-buffer' and `cl-letf' is needed to force `with-temp-buffer'
> > to use buffer-local value of `exec-path'.
> > 
> > In the new patch attached I show how this can be overcome.  Don't know
> > if you'll consider my use case too narrow and specific, but I'll be glad
> > to hear your thoughts on this.
> > 
> > [1] https://guix.gnu.org/en/manual/en/guix.html#Invoking-guix-environment
> 
> Thank you for your explanation and the new patch.  It helped me to
> understand the problem correctly.  I also confirmed that the new patch
> resolves the issue.  I don't think it's a special use case, as I
> sometimes set `exec-path' locally using .dir-locals.el.  I agree with
> the new patch.

Thanks to both of you.  I've now installed this on the emacs-30
release branch, and I'm therefore closing this bug.