bug#54405: 27.1; How to reset connection-local variable

bug#54405: 27.1; How to reset connection-local variable

[David Landell]

[-- Attachment #1: Type: text/plain, Size: 958 bytes --]

Hi,
Is it possible to reset a connection local variable to the default 
in a buffer when default-directory changes from remote to local?

I have attached an example ert test to demonstrate the issue. I would
kind of expect the supplied code to work but I may be doing something
wrong and there might be some other way of doing this. Couldn't find
anything in the manual though.

To run the code: emacs -Q  -l ./connection-local.el 

The problem is a real world problem from my package rg.el
(https://github.com/dajva/rg.el). It is similar to the grep.el package
with the added possibility to rerun a searches with modified parameters
from the results buffer. So a typical use case would be to do a search
on a remote host and then change the default-directory in the results
buffer to do the same search on the local computer. The connection local
is used to store the path to the ripgrep executable that is driving the
search.

Best regards,
David Landell


[-- Attachment #2: Code to reproduce --]
[-- Type: application/emacs-lisp, Size: 814 bytes --]

[-- Attachment #3: Type: text/plain, Size: 4541 bytes --]



In GNU Emacs 27.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20, cairo version 1.16.0)
 of 2020-09-19 built on lgw01-amd64-021
Windowing system distributor 'The X.Org Foundation', version 11.0.12013000
System Description: Linux Mint 20.2

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Ran 1 tests, 0 results were as expected, 1 unexpected
Making completion list...

Configured using:
 'configure --build=x86_64-linux-gnu --prefix=/usr
 '--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
 '--infodir=${prefix}/share/info' --sysconfdir=/etc --localstatedir=/var
 --disable-silent-rules '--libdir=${prefix}/lib/x86_64-linux-gnu'
 '--libexecdir=${prefix}/lib/x86_64-linux-gnu' --disable-maintainer-mode
 --disable-dependency-tracking --prefix=/usr --sharedstatedir=/var/lib
 --libexecdir=/usr/lib --localstatedir=/var/lib
 --infodir=/usr/share/info --mandir=/usr/share/man
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/27.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/27.1/site-lisp:/usr/share/emacs/site-lisp
 --program-suffix=27 --with-modules --with-file-notification=inotify
 --with-mailutils --with-harfbuzz --with-json --with-x=yes
 --with-x-toolkit=gtk3 --with-lcms2 --with-cairo --with-xpm=yes
 --with-gif=yes --with-gnutls=yes --with-jpeg=yes --with-png=yes
 --with-tiff=yes --with-xwidgets 'CFLAGS=-g -O2
 -fdebug-prefix-map=/build/emacs27-bifpWT/emacs27-27.1~1.git86d8d76aa3=. -fstack-protector-strong
 -Wformat -Werror=format-security -no-pie' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro
 -no-pie''

Configured features:
XPM JPEG TIFF GIF PNG RSVG CAIRO SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF
ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS XWIDGETS
LIBSYSTEMD JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LC_MONETARY: sv_SE.UTF-8
  value of $LC_NUMERIC: sv_SE.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs text-property-search mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils tramp-cache tramp-sh
tramp tramp-loaddefs trampver tramp-integration tramp-compat shell
pcomplete comint ansi-color ring parse-time iso8601 time-date ls-lisp
format-spec auth-source cl-seq eieio eieio-core eieio-loaddefs
password-cache json subr-x map files-x thingatpt cl-extra seq byte-opt
bytecomp byte-compile cconv cl-macs gv ert pp ewoc debug backtrace
help-mode easymenu find-func cl-loaddefs cl-lib tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting xwidget-internal cairo
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 76214 9618)
 (symbols 48 9041 1)
 (strings 32 26513 2007)
 (string-bytes 1 933622)
 (vectors 16 15642)
 (vector-slots 8 191670 10146)
 (floats 8 60 122)
 (intervals 56 347 0)
 (buffers 1000 15))

bug#54405: 27.1; How to reset connection-local variable

[Michael Albinus]

David Landell <david.landell@sunnyhill.email> writes:

> Hi,

Hi David,

> Is it possible to reset a connection local variable to the default
> in a buffer when default-directory changes from remote to local?

There does not exist a dedicated function for resetting. What I
recommend is to use the macro `with-connection-local-variables', which
runs BODY with applied connection-local variables, and a cleanup to the
previous state afterwards.

> Best regards,
> David Landell

Best regards, Michael.

bug#54405: 27.1; How to reset connection-local variable

[David Landell]

Thanks Michael for the answer,

Michael Albinus <michael.albinus@gmx.de> writes:

> There does not exist a dedicated function for resetting. What I
> recommend is to use the macro `with-connection-local-variables', which
> runs BODY with applied connection-local variables, and a cleanup to the
> previous state afterwards.
>

This solves my small demo test case but unfortunately it doesn't solve
the main use case I slightly described. In the latter case something
else is calling hack-connection-local-variables-apply outside of the
resetting macro. I think it's possibly from tramp since there is a
process running in the buffer on the remote host.

If I understand the criteria matching correctly, it matches only the
fields available so if I use only the :host key, an unrelated criteria
will match as long as it is using the same :host key, right?

So, seems I should use my own :application key but then I can't use
the resetting macro since that is not applying locals with a unique :application
key. There is something there with 'tramp' being applied by default that
I don't understand though.

Anyway, this is possible to work around now that I think I understand
it.

/David

>
> Best regards, Michael.

bug#54405: 27.1; How to reset connection-local variable

[Michael Albinus]

David Landell <david.landell@sunnyhill.email> writes:

> Thanks Michael for the answer,

Hi David,

> This solves my small demo test case but unfortunately it doesn't solve
> the main use case I slightly described. In the latter case something
> else is calling hack-connection-local-variables-apply outside of the
> resetting macro. I think it's possibly from tramp since there is a
> process running in the buffer on the remote host.

Yes, Tramp does this in its connection buffers.

> If I understand the criteria matching correctly, it matches only the
> fields available so if I use only the :host key, an unrelated criteria
> will match as long as it is using the same :host key, right?

Yes.

> So, seems I should use my own :application key but then I can't use
> the resetting macro since that is not applying locals with a unique :application
> key. There is something there with 'tramp' being applied by default that
> I don't understand though.

`with-connection-local-variables' calls
`connection-local-criteria-for-default-directory' w/o an APPLICATION
argument, that's why `tramp' is used by default. Perhaps we shall extend
`with-connection-local-variables' to accept another :application
key. But for the time being, you might copy
`with-connection-local-variables' into your own package under another
name, and modify it accordingly.

> Anyway, this is possible to work around now that I think I understand
> it.
>
> /David

Best regards, Michael.

bug#54405: 27.1; How to reset connection-local variable

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi David,

>> So, seems I should use my own :application key but then I can't use
>> the resetting macro since that is not applying locals with a unique :application
>> key. There is something there with 'tramp' being applied by default that
>> I don't understand though.
>
> `with-connection-local-variables' calls
> `connection-local-criteria-for-default-directory' w/o an APPLICATION
> argument, that's why `tramp' is used by default. Perhaps we shall extend
> `with-connection-local-variables' to accept another :application
> key. But for the time being, you might copy
> `with-connection-local-variables' into your own package under another
> name, and modify it accordingly.

I've pushed a change to the Emacs master git branch, which allows to
change the application in `with-connection-local-variables'. The Elisp
manual has been extended with

--8<---------------cut here---------------start------------->8---
 -- Variable: connection-local-default-application
     The default application, a symbol, to be applied in
     ‘with-connection-local-variables’.  It defaults to ‘tramp’, but in
     case you want to overwrite Tramp’s settings temporarily, you could
     let-bind it like

          (connection-local-set-profile-variables
            'my-remote-perl
            '((perl-command-name . "/usr/local/bin/perl5")
              (perl-command-switch . "-e %s")))

          (connection-local-set-profiles
            '(:application 'my-app :protocol "ssh" :machine "remotehost")
            'my-remote-perl)

          (let ((default-directory "/ssh:remotehost:/working/dir/")
                (connection-local-default-application 'my-app))
            (with-connection-local-variables
              do something useful))

     This variable must not be changed globally.
--8<---------------cut here---------------end--------------->8---

Does this suffice?

>> /David

Best regards, Michael.

bug#54405: 27.1; How to reset connection-local variable

[David Landell]

Excellent,

> Michael Albinus <michael.albinus@gmx.de> writes:
>
>
> I've pushed a change to the Emacs master git branch, which allows to
> change the application in `with-connection-local-variables'. The Elisp
> manual has been extended with
>
> Does this suffice?
>

This seems to work fine as far as I can tell and as far as my testing
goes with this feature isolated. There is however a small complication
which I don't know if it is to be considered a bug or not.

Still this fails with my package. I discovered that this is because
`hack-dir-local-variables-non-file-buffer' is called at some point for a
totally unrelated variable to be applied. The actual trigger point is
`hack-local-variables-apply'.

So further debugging indicates that the connection-local variable that
is applied (and reset) in `with-connection-local-variables' is also
added to `file-local-variables-alist' which makes it a buffer local
during the call to `hack-local-variables-apply'.

Question is if the file-local should be reset at the same time as
`kill-local-variable' is called in `with-connection-local-variables'?

Anyway, on my end this can be resolved, I have a workaround in place. So
this is mostly FYI.

Cheers,
David

>
> Best regards, Michael.

bug#54405: 27.1; How to reset connection-local variable

[Michael Albinus]

Version: 29.1

David Landell <david.landell@sunnyhill.email> writes:

Hi David,

> This seems to work fine as far as I can tell and as far as my testing
> goes with this feature isolated. There is however a small complication
> which I don't know if it is to be considered a bug or not.
>
> Still this fails with my package. I discovered that this is because
> `hack-dir-local-variables-non-file-buffer' is called at some point for a
> totally unrelated variable to be applied. The actual trigger point is
> `hack-local-variables-apply'.
>
> So further debugging indicates that the connection-local variable that
> is applied (and reset) in `with-connection-local-variables' is also
> added to `file-local-variables-alist' which makes it a buffer local
> during the call to `hack-local-variables-apply'.
>
> Question is if the file-local should be reset at the same time as
> `kill-local-variable' is called in `with-connection-local-variables'?

File-local, directory-local and connection-local variables are set
independently. All of them are implemented as buffer-local variables,
that means, if a given variable is both file-local and connection-local
(or directory-local), there might be conflicts.

Until now, there is no mechanism to avoid this (AFAIK).

> Anyway, on my end this can be resolved, I have a workaround in place. So
> this is mostly FYI.

OK, closing the bug.

> Cheers,
> David

Best regards, Michael.

bug#54405: 27.1; How to reset connection-local variable

[Drew Adams]

> File-local, directory-local and connection-local variables are set
> independently. All of them are implemented as buffer-local variables,
> that means, if a given variable is both file-local and connection-local
> (or directory-local), there might be conflicts.
> 
> Until now, there is no mechanism to avoid this (AFAIK).

Good to know.  Thx.

I haven't checked the doc for each of those kinds
of var.  But do we mention what you say in the
doc?

E.g., do we say that a var that's local in more
than one of these ways might be problematic ("there
might be conflicts")?  Should the doc say clearly
what happens (e.g., what determines the order or
priority)?

bug#54405: 27.1; How to reset connection-local variable

[Michael Albinus]

Drew Adams <drew.adams@oracle.com> writes:

Hi Drew,

> E.g., do we say that a var that's local in more
> than one of these ways might be problematic ("there
> might be conflicts")?  Should the doc say clearly
> what happens (e.g., what determines the order or
> priority)?

It is documented that in case a variable has both file-local and
directory-local settings, the file-local value will be used. See
(info "(emacs)Directory Variables") (at the bottom).

Otherwise, it is kind of undetermined, I believe. Every package (author)
is free to call `hack-local-variables', `hack-dir-local-variables' or
`hack-connection-local-variables' in any order.

Best regards, Michael.