server-local variables

server-local variables

[Michael Albinus]

Hi,

a long standing wishlist for Tramp are server-local (sometimes called
connection-local) variables. It was expressed for the first time in
<https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3736#35>, but there are
more discussions about in bug reports and discussions on emacs-devel and
tramp-devel.

I'm playing with a small patch for files.el, which provides a mechanism
similar to directory-local variables. Most important are the following
functions:

--8<---------------cut here---------------start------------->8---
server-locals-set-class is a compiled Lisp function in
‘../../../src/emacs/lisp/files.el’.

(server-locals-set-class REGEXP CLASS)

Add CLASS for remote servers.
REGEXP is a regular expression identifying remote servers, a string.
CLASS is the name of a project class, a symbol.

When a connection to a remote server is opened, the
server-specific variables from CLASS are applied to the
corresponding process buffer.  The variables for a class are
defined using ‘server-locals-set-class-variables’.
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
server-locals-set-class-variables is a compiled Lisp function in
‘../../../src/emacs/lisp/files.el’.

(server-locals-set-class-variables CLASS VARIABLES)

Map the symbol CLASS to a list of variable settings.
VARIABLES is a list that declares server-local variables for the
class.  An element in VARIABLES is an alist whose elements are of
the form (VARIABLE . VALUE).

When a connection to a remote server is opened, the server’s
classes are found.  A server may be assigned a class using
‘server-locals-set-directory-class’.  Then variables are set in
the server’s process buffer according to the VARIABLES list of
the class.  The list is processed in order.
--8<---------------cut here---------------end--------------->8---

--8<---------------cut here---------------start------------->8---
hack-server-local-variables-apply is a compiled Lisp function in
‘../../../src/emacs/lisp/files.el’.

(hack-server-local-variables-apply)

Apply server-local variables for the server identified by ‘default-directory’.
Other local variables, like file-local and dir-local variables,
will not be touched.
--8<---------------cut here---------------end--------------->8---

With these functions, one could write the following code snippet,
pushing a server-local variable `some-useful-setting' for all remote
connections using the "sudo" method:

--8<---------------cut here---------------start------------->8---
(server-locals-set-class-variables
 'remote '((some-useful-setting . some-useful-value)))

(server-locals-set-class "^/sudo:" 'remote)

;; This must be applied in a buffer with remote `default-directory'.
(hack-server-local-variables-apply)
--8<---------------cut here---------------end--------------->8---

Comments?

Best regards, Michael.

Re: server-local variables

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Sun, 30 Oct 2016 19:39:38 +0100
> 
> Comments?

Do these functions really have to live in files.el?  They are used by
Tramp, don't they?

Re: server-local variables

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

> Do these functions really have to live in files.el?  They are used by
> Tramp, don't they?

Sure, Tramp (and ange-ftp, perhaps also url-handlers) will use them
internally. But they shall also be good for other packages. Bug#24394
discusses `shell-file-name' and `shell-command-switch'. Bug#3736
discusses server-local settings for `null-device'. And in bug#18940 I've
proposed server-local settings of `process-environment' (well, Stefan did
object, but this was not discussed until its end, I believe).

If you fear that it would inflate files.el too much, we could locate
these functions in files-x.el.

Best regards, Michael.

Re: server-local variables

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: emacs-devel@gnu.org
> Date: Sun, 30 Oct 2016 21:28:48 +0100
> 
> > Do these functions really have to live in files.el?  They are used by
> > Tramp, don't they?
> 
> Sure, Tramp (and ange-ftp, perhaps also url-handlers) will use them
> internally. But they shall also be good for other packages. Bug#24394
> discusses `shell-file-name' and `shell-command-switch'. Bug#3736
> discusses server-local settings for `null-device'. And in bug#18940 I've
> proposed server-local settings of `process-environment' (well, Stefan did
> object, but this was not discussed until its end, I believe).
> 
> If you fear that it would inflate files.el too much, we could locate
> these functions in files-x.el.

My reasoning was that if this doesn't have to be preloaded, it should
live in a package that's most likely to be loaded when they are needed
anyway.  I think Tramp is the likely user; I find it much harder to
imagine the others using it, but whenever I use a remote file name,
Tramp is auto-loaded, so Tramp sounds like a good place.

I don't object to files-x.el, if you think that is better.

Re: server-local variables

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

> My reasoning was that if this doesn't have to be preloaded, it should
> live in a package that's most likely to be loaded when they are needed
> anyway.  I think Tramp is the likely user; I find it much harder to
> imagine the others using it, but whenever I use a remote file name,
> Tramp is auto-loaded, so Tramp sounds like a good place.
>
> I don't object to files-x.el, if you think that is better.

I do not want to restrict server-local variables to Tramp, even if there
isn't such other use case yet.

I'll push my patch to files-x.el in a couple of days, waiting for other
possible comments first.

Best regards, Michael.

PS: Starting tomorrow, I'll be offline for the rest of this week.

Re: server-local variables

[Phil Sainty]

Personally, 'server' in an Emacs context makes me think firstly
of server.el, not a remote host.

I think 'host-local' would be a better name than 'server-local' ?

Re: server-local variables

[Ted Zlatanov]

On Sun, 30 Oct 2016 19:39:38 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> With these functions, one could write the following code snippet,
MA> pushing a server-local variable `some-useful-setting' for all remote
MA> connections using the "sudo" method:

MA> (server-locals-set-class-variables
MA>  'remote '((some-useful-setting . some-useful-value)))

MA> (server-locals-set-class "^/sudo:" 'remote)

MA> ;; This must be applied in a buffer with remote `default-directory'.
MA> (hack-server-local-variables-apply)

That's very interesting. Some questions:

* how are multiple classes merged?
* can classes be assigned by other means besides regexp?
* why not an obarray?
* a macro `with-process-classes' would be nice

It seems like these are process-local variables, in the sense that you
may want them per user@host:port combination (or other process
parameters), not just hostname. Thus it may be better to make this a
process-level facility.

I think the GnuTLS code could use this facility as well. Right now
gnutls.el has ad-hoc facilities for this purpose, if I understand you
correctly.

Ted

Re: server-local variables

[Óscar Fuentes]

Phil Sainty <psainty@orcon.net.nz> writes:

> Personally, 'server' in an Emacs context makes me think firstly
> of server.el, not a remote host.

+1

Re: server-local variables

[Stefan Monnier]

> * why not an obarray?

FWIW, this question *very* rarely makes sense.  Usually the better
question in this regard is:

  why an obarray?

Other than trying to implement Emacs modules, I can't think of any
situation where obarrays are a better option than, say, hash tables.


        Stefan

Re: server-local variables

[Stefan Monnier]

> I do not want to restrict server-local variables to Tramp, even if there
> isn't such other use case yet.

I have the suspicion that if another user than Tramp ever comes along,
it could have a really hard time using your API, because your API is
designed very clearly from Tramp's point of view.

I can agree that it might be desirable that this feature should be able
to set non-Tramp variables on a per-host basis (e.g. shell-file-name,
null-device, ...), but other than that I think it makes sense to add it
as a Tramp-only API at least at first.


        Stefan

Re: server-local variables

[Michael Albinus]

Phil Sainty <psainty@orcon.net.nz> writes:

> Personally, 'server' in an Emacs context makes me think firstly
> of server.el, not a remote host.
>
> I think 'host-local' would be a better name than 'server-local' ?

Well, it is not only the host which counts, also user name or method
could be used to distinguish the different variable sets for different
purposes. So one could use `connection-local' instead of `server-local',
if people prefer.

Best regards, Michael.

Re: server-local variables

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> That's very interesting. Some questions:
>
> * how are multiple classes merged?

Several calls of `server-locals-set-class-variables', like this:

--8<---------------cut here---------------start------------->8---
(server-locals-set-class-variables
 'remote-bash
 '((shell-file-name . "/bin/bash")
   ;; Command line switch.
   (shell-command-switch . "-c")
   ;; Command line switch.
   (shell-interactive-switch . "-i")
   ;; Login shell switch.
   (shell-login-switch . "--login")))

(server-locals-set-class-variables
 'remote-nullfile
 '((nullfile . "/dev/null")))

(server-locals-set-class
 "my-user@my-remote-host" 'remote-bash)
(server-locals-set-class
 "my-user@my-remote-host" 'remote-nullfile)
--8<---------------cut here---------------end--------------->8---

And for another user, you could apply the same class, or another one:

--8<---------------cut here---------------start------------->8---
(server-locals-set-class-variables
 'remote-ksh
 '((shell-file-name . "/bin/ksh")
   ...))

(server-locals-set-class
 "another-user@my-remote-host" 'remote-ksh)
(server-locals-set-class
 "another-user@my-remote-host" 'remote-nullfile)
--8<---------------cut here---------------end--------------->8---

> * can classes be assigned by other means besides regexp?

No, not foreseen yet.

> * why not an obarray?

See Stefan

> * a macro `with-process-classes' would be nice

How shall it look like?

> It seems like these are process-local variables, in the sense that you
> may want them per user@host:port combination (or other process
> parameters), not just hostname. Thus it may be better to make this a
> process-level facility.

Process buffers are the first, obvious use case for this. But I don't
want to restrict it to them. People could use it also for setting other
local variables in any remote buffer, maybe even for specific modes.

> Ted

Best regards, Michael.

Re: server-local variables

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I do not want to restrict server-local variables to Tramp, even if there
>> isn't such other use case yet.
>
> I have the suspicion that if another user than Tramp ever comes along,
> it could have a really hard time using your API, because your API is
> designed very clearly from Tramp's point of view.

It fits Tramp's need, yes. But the design is a simple clone of
`dir-locals-set-class-variables', `dir-locals-set-directory-class' and
`hack-dir-local-variables-non-file-buffer'.

> I can agree that it might be desirable that this feature should be able
> to set non-Tramp variables on a per-host basis (e.g. shell-file-name,
> null-device, ...), but other than that I think it makes sense to add it
> as a Tramp-only API at least at first.

It would be easier for me, because I would have more control over it :-)

But I still believe there will be non-Tramp use cases. As already said,
ange-ftp comes to mind.

>         Stefan

Best regards, Michael.

Re: server-local variables

[Ted Zlatanov]

On Mon, 31 Oct 2016 17:26:07 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Ted Zlatanov <tzz@lifelogs.com> writes:
>> That's very interesting. Some questions:
>> 
>> * how are multiple classes merged?

MA> (server-locals-set-class
MA>  "my-user@my-remote-host" 'remote-bash)
MA> (server-locals-set-class
MA>  "my-user@my-remote-host" 'remote-nullfile)

That seems awkward compared to (server-locals-set-classes CRITERIA CLASSES...)
(in other words, I expect mixing classes to be the common case, so it
should be easy).

Otherwise the API looks reasonable to me :)

>> * can classes be assigned by other means besides regexp?

MA> No, not foreseen yet.

I'd make it a regexp or a callable thing, that will cover 99% of the
cases I can imagine.

>> * why not an obarray?

MA> See Stefan

OK, it seemed like you were basically designing a scoped symbol table
per connection, so an obarray seemed sensible. I haven't worked with
them so it was just a question for my education. But if they are
actively discouraged for most usage, the ELisp manual section on
Creating Symbols should mention that. I was not aware.

>> * a macro `with-process-classes' would be nice

MA> How shall it look like?

I'd use it in a process to widen or narrow permissions and environment:

...connect to a host, then...

(with-process-classes temporary-sudo-override temporary-shell-override ansible-host-environment
  ...do something)

It would take care of setting the connection-local variables inside the
scope and unwinding them afterwards.

MA> Process buffers are the first, obvious use case for this. But I don't
MA> want to restrict it to them. People could use it also for setting other
MA> local variables in any remote buffer, maybe even for specific modes.
...
MA> But I still believe there will be non-Tramp use cases. As already said,
MA> ange-ftp comes to mind.

OK, please take a look at the GnuTLS glue code in gnutls.el and you'll
see what I need from this facility. I'm happy to start using it there as
soon as the API is solidified. I have a TODO item (discussed with Lars a
few months ago) to implement something like this facility so it's great
to see it's not an isolated need :)

Ted

Re: server-local variables

[Phil Sainty]

On 2016-11-01 04:42, Michael Albinus wrote:
> Well, it is not only the host which counts, also user name or method
> could be used to distinguish the different variable sets for different
> purposes. So one could use `connection-local' instead of 
> `server-local',
> if people prefer.

Yes, that makes good sense.

Re: server-local variables

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> MA> (server-locals-set-class
> MA>  "my-user@my-remote-host" 'remote-bash)
> MA> (server-locals-set-class
> MA>  "my-user@my-remote-host" 'remote-nullfile)
>
> That seems awkward compared to (server-locals-set-classes CRITERIA CLASSES...)
> (in other words, I expect mixing classes to be the common case, so it
> should be easy).

Makes sense. Noted.

>>> * can classes be assigned by other means besides regexp?
>
> MA> No, not foreseen yet.
>
> I'd make it a regexp or a callable thing, that will cover 99% of the
> cases I can imagine.

Possible. Noted.

>>> * a macro `with-process-classes' would be nice
>
> MA> How shall it look like?
>
> I'd use it in a process to widen or narrow permissions and environment:
>
> ...connect to a host, then...
>
> (with-process-classes temporary-sudo-override temporary-shell-override
> ansible-host-environment
>   ...do something)
>
> It would take care of setting the connection-local variables inside the
> scope and unwinding them afterwards.

I see. The problem is unwinding the local variables; they could exist
already as local variables when entering the macro. But in general it
shall be possible, when implementing proper book-keeping.

Btw, I wouldn't call it with-*process*-classes. server-local (or
connection-local) variables could be applied to any buffer with a remote
default-directory. 

> MA> Process buffers are the first, obvious use case for this. But I don't
> MA> want to restrict it to them. People could use it also for setting other
> MA> local variables in any remote buffer, maybe even for specific modes.
> ...
> MA> But I still believe there will be non-Tramp use cases. As already said,
> MA> ange-ftp comes to mind.
>
> OK, please take a look at the GnuTLS glue code in gnutls.el and you'll
> see what I need from this facility. I'm happy to start using it there as
> soon as the API is solidified. I have a TODO item (discussed with Lars a
> few months ago) to implement something like this facility so it's great
> to see it's not an isolated need :)

I've looked shortly on this. Seems like you need a mechanism for
`gnutls-boot-parameters'.

> Ted

Best regards, Michael.

RE: server-local variables

[Drew Adams]

> > * why not an obarray?
> 
> FWIW, this question *very* rarely makes sense.  Usually the better
> question in this regard is: why an obarray?

You give no reasons...

> Other than trying to implement Emacs modules, I can't think of any
> situation where obarrays are a better option than, say, hash tables.

An obarray is a kind of hash table.  But it also provides a separate
namespace for symbols.  Using an obarray can make sense when you want
a separate symbol namespace.  Is that only "*very* rarely"?

I use an obarray in `synonyms.el', for example, for the names in a
dictionary (thesaurus).  (https://www.emacswiki.org/emacs/Synonyms)

Why do you think it "*very* rarely makes sense" to use an obarray?

I don't disagree that "Why an obarray?" is a useful question.
I do think that giving advice/guidelines without reasons is not
so helpful.  Especially coming from someone who is (rightfully)
respected as an expert and authority, wouldn't it be more helpful
to explain your admonition a bit?

Re: server-local variables

[Stefan Monnier]

> I use an obarray in `synonyms.el', for example, for the names in a
> dictionary (thesaurus).  (https://www.emacswiki.org/emacs/Synonyms)

That's exactly an example where you use obarrays to do something similar
to modules.

> I do think that giving advice/guidelines without reasons is not
> so helpful.

I did give a reason.


        Stefan

RE: server-local variables

[Drew Adams]

> > I use an obarray in `synonyms.el', for example, for the
> > names in a dictionary (thesaurus).
> > (https://www.emacswiki.org/emacs/Synonyms)
> 
> That's exactly an example where you use obarrays to do
> something similar to modules.

Oh, that's what you meant by "modules"?  Had you mentioned
"namespaces" (or even just separate sets of "symbols", or
even "something similar to Common Lisp packages"), that
might have been clearer.

(Especially in light of the recent discussions of adding
a different sort of "modules" to Emacs.)

See Ted's reply, where it seems he thought of an obarray
for exactly the reasons you and I apparently agree that
obarrays _can_ be appropriate: to wall off sets of symbols:

> OK, it seemed like you were basically designing a scoped
> symbol table per connection, so an obarray seemed sensible.
> I haven't worked with them so it was just a question for
> my education.  But if they are actively discouraged for
> most usage, the ELisp manual section on Creating Symbols
> should mention that.  I was not aware.

He seems to have taken your admonition somewhat similarly
to how I did.

Had you just said that you don't think separate symbol
namespaces are needed here, I don't think there would have
been any confusion.  It was not clear, to me at least, why
you said that obarray use cases are *very* rare. 

And if it _is_ needed here to have separate sets of symbols
(some of which could have the same name), then "Why not an
obarray?" isn't an unreasonable question.

Perhaps your reply was just your way of asking "Do we need
separate sets of symbols here?", or your way of saying that
we do not?

Re: server-local variables

[Ted Zlatanov]

On Mon, 31 Oct 2016 21:47:32 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Ted Zlatanov <tzz@lifelogs.com> writes:

>> (with-process-classes temporary-sudo-override temporary-shell-override
>> ansible-host-environment
>> ...do something)
>> 
>> It would take care of setting the connection-local variables inside the
>> scope and unwinding them afterwards.

MA> I see. The problem is unwinding the local variables; they could exist
MA> already as local variables when entering the macro. But in general it
MA> shall be possible, when implementing proper book-keeping.

That would be great, thank you for considering it.

MA> Btw, I wouldn't call it with-*process*-classes. server-local (or
MA> connection-local) variables could be applied to any buffer with a remote
MA> default-directory.

Any name is fine with me. It's really a pattern match against some
buffer characteristics and has a lot in common with file-and-dir-local
variables (which pattern match against the file location).

>> OK, please take a look at the GnuTLS glue code in gnutls.el and you'll
>> see what I need from this facility. I'm happy to start using it there as
>> soon as the API is solidified. I have a TODO item (discussed with Lars a
>> few months ago) to implement something like this facility so it's great
>> to see it's not an isolated need :)

MA> I've looked shortly on this. Seems like you need a mechanism for
MA> `gnutls-boot-parameters'.

Yes, exactly. It needs to be flexible and augmentable to allow layering
of GnuTLS options; currently it's pretty much the opposite.

The GnuTLS priority strings, which are on the C side, have interesting
parallels with your concept of classes that represent sets of parameters.

Thanks
Ted

Re: server-local variables

[Michael Albinus]

Hi,

finally, I've committed a patch to master implementing connection-local
variables.  For details, see

(info "(elisp)Connection Local Variables")

Best regards, Michael.

GnuTLS per-connection variables (was: server-local variables)

[Ted Zlatanov]

On Mon, 14 Nov 2016 15:02:50 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> finally, I've committed a patch to master implementing connection-local
MA> variables.  For details, see

MA> (info "(elisp)Connection Local Variables")

Looks great, thank you.

I looked at migrating gnutls.el to use this new facility. I have some
questions (and maybe Lars and others can add more).

* how will users see and configure the settings per class?

* how will users select the class per server? The NSM may be useful
  here, since it already does some per-connection checks and storage.

* how to migrate users from the current defcustoms in
  (customize-group 'gnutls) to this new system? Do we support both for
  some time?

Thanks
Ted

Re: GnuTLS per-connection variables

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

Hi Ted,

> I looked at migrating gnutls.el to use this new facility. I have some
> questions (and maybe Lars and others can add more).
>
> * how will users see and configure the settings per class?

Connection local variables are specified so far for on Lisp level. No
(end-) user is provided yet.

Let's take the example from the manual:

--8<---------------cut here---------------start------------->8---
(connection-local-set-class-variables
  'remote-bash
  '((shell-file-name . "/bin/bash")
    (shell-command-switch . "-c")
    (shell-interactive-switch . "-i")
    (shell-login-switch . "-l")))

(connection-local-set-class-variables
  'remote-null-device
  '((null-device "/dev/null")))

(connection-local-set-classes
  "^/ssh:" 'remote-bash 'remote-null-device)
--8<---------------cut here---------------end--------------->8---

Then there are two undocumented (in the manual) functions which tel you
the settings:

(connection-local-get-class-variables 'remote-bash)

=> ((shell-file-name . "/bin/bash")
    (shell-command-switch . "-c")
    (shell-interactive-switch . "-i")
    (shell-login-switch . "-l"))

(connection-local-get-classes "^/ssh:")
=> (remote-bash remote-null-device)

> * how will users select the class per server? The NSM may be useful
>   here, since it already does some per-connection checks and storage.

Again, as of today on Lisp level, per `connection-local-set-classes'.

> * how to migrate users from the current defcustoms in
>   (customize-group 'gnutls) to this new system? Do we support both for
>   some time?

That I don't know. Maybe one adds a new defcustom group per server
on-the-fly, and let the variables declare inside every such group. A
post customization function shall move those variables into
connection-local variables then.

And maybe one could write code to support such approach for a general
use case, not only for gnutls.

> Thanks
> Ted

Best regards, Michael.

Re: GnuTLS per-connection variables

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

Hi Ted,

> I looked at migrating gnutls.el to use this new facility. I have some
> questions (and maybe Lars and others can add more).
>
> * how will users see and configure the settings per class?
>
> * how will users select the class per server? The NSM may be useful
>   here, since it already does some per-connection checks and storage.

Over the weekend, I've played a little bit with this. As a result there
are two new functions (see attached patch), which could give some support.

The following scenario shall give you an idea. I assume, you have some
connection-local variables you want to offer. Declare them as alist,
where the `cdr' of every element might be the default value to use:

(defvar gnutls-boot-parameters-default
  '((gnutls-priority . "priority-string")
    (gnutls-hostname . "localhost")
    (gnutls-loglevel . 123)
    (gnutls-min-prime-bits . 1024)
    (gnutls-trustfiles . "/etc/trustfiles")
    (gnutls-crlfiles . "/etc/crlfiles")
    (gnutls-keylist)
    (gnutls-verify-flags a b c)
    (gnutls-verify-error . t)
    (gnutls-callbacks))
  "Connection-local variables and their initial values.")

Please don't discuss the values with me, I've used them ramdomly :-)

After that, you call

(connection-local-custom-create-custom
 'gnutls-boot-parameters 'gnutls-boot-parameters-default)

This creates a new custom option `gnutls-boot-parameters', which you
could edit now in the appearing customization buffer. You should add
regular expressions, which identify the remote servers you want to
provide with connection-local variables. Let's assume you add the two
regular expressions "^/ssh:" "and remote-host". If you set these values
in the custom buffer, the custom option `gnutls-boot-parameters' is
created with the following values:

gnutls-boot-parameters
=> ("^/ssh:" "remote-host")

Furthermore, two additional custom options are created, which represent
intermediate classes
`connection-local--class-^/ssh:-gnutls-boot-parameters-default' and
`connection-local--class-remote-host-gnutls-boot-parameters-default'. The
names sound strange, but this could be changed. You don't need them
explicitely.

Now follow the link at the bottom of the customization buffer to the
custom group connection-local. Beside the new custom option
`gnutls-boot-parameters', you see also the two new options representing
the classes. Customize them. For example, in the custom option
`connection-local--class-^/ssh:-gnutls-boot-parameters-default' you
might select the keys `gnutls-priority' and `gnutls-hostname', and in
the custom option
`connection-local--class-remote-host-gnutls-boot-parameters-default' you
might select the keys `gnutls-min-prime-bits' and `gnutls-crlfiles'. In
all cases, you could also change the values of the selected keys.

Apply the changes, and now you have as desired

connection-local-criteria-alist
=> (("remote-host" connection-local--class-remote-host-gnutls-boot-parameters-default)
    ("^/ssh:" connection-local--class-^/ssh:-gnutls-boot-parameters-default))

connection-local-class-alist
=> ((connection-local--class-remote-host-gnutls-boot-parameters-default
    (gnutls-min-prime-bits . 2048)
    (gnutls-trustfiles . "/etc/trustfiles")
    (gnutls-crlfiles . "/opt/crlfiles"))
   (connection-local--class-^/ssh:-gnutls-boot-parameters-default
    (gnutls-priority . "priority-string")
    (gnutls-hostname . "localhost")))

All of this is far from being perfect, but you see the idea.

> * how to migrate users from the current defcustoms in
>   (customize-group 'gnutls) to this new system? Do we support both for
>   some time?

For this I still have no idea.

> Thanks
> Ted

Best regards, Michael.

--8<---------------cut here---------------start------------->8---
*** /home/albinus/src/emacs/lisp/files-x.el.~1228055b320dbed92ab400c4a95813a2b8023909~	2016-11-28 19:26:52.537538722 +0100
--- /home/albinus/src/emacs/lisp/files-x.el	2016-11-28 18:02:11.790554444 +0100
***************
*** 559,564 ****
--- 559,570 ----
  (setq ignored-local-variables
        (cons 'connection-local-variables-alist ignored-local-variables))
  
+ (defgroup connection-local nil
+   "Declare connection-local files."
+   :group 'files
+ ;  :link '(custom-manual "(tramp)Top")
+   :version "26.1")
+ 
  (defvar connection-local-class-alist '()
    "Alist mapping connection-local variable classes (symbols) to variable lists.
  Each element in this list has the form (CLASS VARIABLES).
***************
*** 636,641 ****
--- 642,734 ----
  class.  The list is processed in order."
    (setf (alist-get class connection-local-class-alist) variables))
  
+ (defun connection-local-custom-set-variables (criteria template)
+   "Add connection-local variables for remote servers.
+ CRITERIA is either a regular expression identifying a remote
+ server, or a function with one argument IDENTIFICATION, which
+ returns non-nil when a remote server shall apply variables.  If
+ CRITERIA is nil, it always applies.
+ 
+ TEMPLATE, a symbol, contains a list of connection-local variables
+ and their initial values.  Every element of this list is a cons
+ cell of type (VARIABLE . VALUE).
+ 
+ A new customizable variable is created on-the-fly, which is used
+ as connection-local class in order to create a link between
+ CRITERIA and the variables.  A customization buffer opens in
+ order to provide an interface for setting the connection-local
+ variables derived from TEMPLATE."
+   (unless (symbolp template) (error "Invalid template %s" template))
+   (let ((class
+          (intern (format "connection-local--class-%s-%s" criteria template)))
+         (type
+          `(alist
+            :key-type variable :value-type sexp
+            :options
+            ,(mapcar
+              (lambda (x)
+                "Compute alist options, depending on TEMPLATE."
+                (unless (consp x) (error "Invalid argument %s in template" x))
+                (list
+                 (car x)
+                 ;; We support only some few types.  All other types
+                 ;; are handled as sexp.
+                 (if (null (cdr x))
+                     'sexp
+                   (list
+                    (cond
+                     ((numberp (cdr x)) 'number)
+                     ((stringp (cdr x)) 'string)
+                     ((functionp (cdr x)) 'function)
+ ;;                    ((booleanp (cdr x)) 'boolean)
+                     ((symbolp (cdr x)) 'symbol)
+ ;;                    ((consp (cdr x)) 'cons)
+ ;;                    ((vectorp (cdr x)) 'vector)
+ ;;                    ((listp (cdr x)) 'list)
+                     (t 'sexp))
+                    :value (cdr x)))))
+              (symbol-value template))))
+         (setfunction
+          `(lambda (symbol value)
+             (set symbol value)
+             (connection-local-set-class-variables symbol value)
+             (connection-local-set-classes ,criteria symbol))))
+     (eval
+      `(progn
+         (defcustom ,class nil
+           ,(format
+             "Connection-local class for criteria %S and template %S."
+             criteria template)
+           :group 'connection-local
+           :type type
+           :set setfunction)))))
+ 
+ (defun connection-local-custom-create-custom (option template)
+   "Create custom variable OPTION, and initialize it with TEMPLATE.
+ TEMPLATE, a symbol, contains a list of connection-local variables
+ and their initial values.  Every element of this list is a cons
+ cell of type (VARIABLE . VALUE).
+ OPTION can be used then to customize the connection-local
+ variables according to different remote connections."
+   (unless (symbolp option) (error "Invalid custom variable %s" option))
+   (unless (symbolp template) (error "Invalid template %s" template))
+   (let ((type '(repeat regexp))
+         (setfunction
+          `(lambda (symbol value)
+             (set symbol value)
+             (dolist (val value)
+               (connection-local-custom-set-variables val ',template)))))
+     (eval
+      `(progn
+         (defcustom ,option nil
+           ,(format
+             "Connection-local class for custom option %S and template %S."
+             option template)
+           :group 'connection-local
+           :type type
+           :set setfunction)
+         (customize-variable ',option)))))
+ 
  (defun hack-connection-local-variables ()
    "Read per-connection local variables for the current buffer.
  Store the connection-local variables in `connection-local-variables-alist'.
--8<---------------cut here---------------end--------------->8---

Re: GnuTLS per-connection variables

[Ted Zlatanov]

On Thu, 24 Nov 2016 13:50:30 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

>> * how to migrate users from the current defcustoms in
>> (customize-group 'gnutls) to this new system? Do we support both for
>> some time?

MA> That I don't know. Maybe one adds a new defcustom group per server
MA> on-the-fly, and let the variables declare inside every such group. A
MA> post customization function shall move those variables into
MA> connection-local variables then.

MA> And maybe one could write code to support such approach for a general
MA> use case, not only for gnutls.

Thanks for the great explanation. I'll see how all this integrates with
gnutls.el. The dynamic customization suggestion sounds very similar to
the Gnus group/topic customization logic, which generates customization
logic as well. I also think there's an interesting connection with
dir-local variables, which also have the concepts of context sensitivity
and hierarchy.

Ted

Re: GnuTLS per-connection variables

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Ted,

> Over the weekend, I've played a little bit with this. As a result there
> are two new functions (see attached patch), which could give some support.

I'm undecided how to continue. Would this approach help you? Shall I
continue to work this direction?

Best regards, Michael.

Re: GnuTLS per-connection variables

[Ted Zlatanov]

On Mon, 05 Dec 2016 16:09:31 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Michael Albinus <michael.albinus@gmx.de> writes:

>> Over the weekend, I've played a little bit with this. As a result there
>> are two new functions (see attached patch), which could give some support.

MA> I'm undecided how to continue. Would this approach help you? Shall I
MA> continue to work this direction?

I started playing with it. I think it's a good direction. I also think
the general idea of a "customizable hierarchy of contexts to set
variable values" keeps getting reinvented in many parts of Emacs, so I
was looking at the Gnus topic/group customization code as well.

I think the right approach is to focus on the data side and leave
user-side Customize support for later. The Customize UI is just not
right to support all the use cases in a way that won't hopelessly
confuse the user. So we may have to implement special UI to make the
hierarchy visible and understandable, but considering the Emacs user
base I think that will not be a strong requirement.

From my side, in gnutls.el, I will start using the code and probably
extend it here and there. So I think your patch is a good start for us.

Thanks
Ted

Re: GnuTLS per-connection variables

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> From my side, in gnutls.el, I will start using the code and probably
> extend it here and there. So I think your patch is a good start for us.

Since the patch plays mainly with defcustom's I still don't know whether
it shall be committed to master, or whether you only want to take the
ideas out.

> Thanks
> Ted

Best regards, Michael.