Two problems with directory-local variables

Two problems with directory-local variables

[Marcin Borkowski]

Hi all,

I am writing a blog post about directory local variables
(http://mbork.pl/2018-09-17_Emacs_directory_variables), and I discovered
two quirks with them.

1. The docs don't seem to mention that they are processed _before_
file-local variables.  (In particular, this also means before
find-file-hook, whose docstring could also mention that.)

2. More seriously, apparently the "eval" keyword doesn't work with
directory variables the way it does with file variables.  The manual
says:
,----
| You can specify the variables ‘mode’, ‘eval’, and ‘unibyte’ in your
| ‘.dir-locals.el’, and they have the same meanings as they would have in
| file local variables.
`----
but I found this is not the case.  This works (as a file variable):

--8<---------------cut here---------------start------------->8---
;; Local Variables:
;; eval: (message "hello")
;; End:
--8<---------------cut here---------------end--------------->8---

but this entry in .dir-locals.el does not work:

--8<---------------cut here---------------start------------->8---
((nil . ((eval '(message "hello")))))
--8<---------------cut here---------------end--------------->8---

yielding

,----
| File local-variables error: (invalid-function '(message hello))
`----

(I have no idea why the quotes are missing here, btw.)

When I type this in .dir-locals.el:

--8<---------------cut here---------------start------------->8---
((nil . ((eval (lambda () (message "hello"))))))
--8<---------------cut here---------------end--------------->8---

it works as expected.

Shouldn't the manual be updated?  (If so, I can make a patch.)

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: Two problems with directory-local variables

[Phil Sainty]

On 2018-09-17 20:08, Marcin Borkowski wrote:
> but this entry in .dir-locals.el does not work:
> ((nil . ((eval '(message "hello")))))

That is because the syntax is incorrect.  You want:

((nil . ((eval . (message "hello")))))

or equivalently, without dotted pair syntax in the eval:

((nil . ((eval message "hello"))))

Note that your version was equivalent to evaluating:

('(message "hello"))


> When I type this in .dir-locals.el:
> ((nil . ((eval (lambda () (message "hello"))))))
> it works as expected.

Indeed, as that's evaluating this form:

((lambda () (message "hello")))

Although I wouldn't recommend that over the former
syntax.


-Phil

Re: Two problems with directory-local variables

[Marcin Borkowski]

On 2018-09-17, at 11:28, Phil Sainty <psainty@orcon.net.nz> wrote:

> On 2018-09-17 20:08, Marcin Borkowski wrote:
>> but this entry in .dir-locals.el does not work:
>> ((nil . ((eval '(message "hello")))))
>
> That is because the syntax is incorrect.  You want:
>
> ((nil . ((eval . (message "hello")))))
>
> or equivalently, without dotted pair syntax in the eval:
>
> ((nil . ((eval message "hello"))))

Thanks!

Actually, this is what `add-dir-local-variable' puts in .dir-locals.el,
which looks a bit weird.  It seems that I forgot about the dotted pairs,
and "corrected" it to the wrong form.  Thanks a lot!  I'll update my
blog post soon.

> Note that your version was equivalent to evaluating:
>
> ('(message "hello"))

That explains (at least partially) the weird error message I got
("(invalid-function (message hello))").

>> When I type this in .dir-locals.el:
>> ((nil . ((eval (lambda () (message "hello"))))))
>> it works as expected.
>
> Indeed, as that's evaluating this form:
>
> ((lambda () (message "hello")))
>
> Although I wouldn't recommend that over the former
> syntax.

Do I get it correctly that this a Scheme-ish syntax with the lambda
expression in place of the function name?

Thanks a lot,

-- 
Marcin Borkowski
http://mbork.pl

Re: Two problems with directory-local variables

[Juri Linkov]

>>> but this entry in .dir-locals.el does not work:
>>> ((nil . ((eval '(message "hello")))))
>>
>> That is because the syntax is incorrect.  You want:
>>
>> ((nil . ((eval . (message "hello")))))
>>
>> or equivalently, without dotted pair syntax in the eval:
>>
>> ((nil . ((eval message "hello"))))
>
> Thanks!
>
> Actually, this is what `add-dir-local-variable' puts in .dir-locals.el,
> which looks a bit weird.

If you think there is a bug in add-dir-local-variable, please show a recipe
how did you get an incorrect setting.  When I tried, it correctly added

  ((nil
    (eval message "hello")))

Re: Two problems with directory-local variables

[Phil Sainty]

On 2018-09-18 11:15, Juri Linkov wrote:
> When I tried [add-dir-local-variable] it correctly added
>   ((nil
>     (eval message "hello")))

On this note, while I don't use the commands myself, I've often wished
that add-dir-local-variable would generate the dotted-pair syntax,
because I've often seen confusion about the syntax of dir-locals arising
from dotted vs non-dotted possibilities, and I feel strongly that the
dotted syntax is the most readable for these files.

i.e. To my mind, (eval message "hello") is liable to confuse people,
whereas (eval . (message "hello")) is rather clear.

Of course we wouldn't want this to end up in the file either:
(eval . (message . ("hello" . nil)))

I guess we'd need some code to only 'dot' the high level elements,
but not the values.

I don't suppose anything like that already exists?


-Phil

Re: Two problems with directory-local variables

[Juri Linkov]

>> When I tried [add-dir-local-variable] it correctly added
>>   ((nil
>>     (eval message "hello")))
>
> On this note, while I don't use the commands myself, I've often wished
> that add-dir-local-variable would generate the dotted-pair syntax,
> because I've often seen confusion about the syntax of dir-locals arising
> from dotted vs non-dotted possibilities, and I feel strongly that the
> dotted syntax is the most readable for these files.
>
> i.e. To my mind, (eval message "hello") is liable to confuse people,
> whereas (eval . (message "hello")) is rather clear.
>
> Of course we wouldn't want this to end up in the file either:
> (eval . (message . ("hello" . nil)))
>
> I guess we'd need some code to only 'dot' the high level elements,
> but not the values.
>
> I don't suppose anything like that already exists?

I don't know if something exists that could allow printing dotted syntax for

  (pp-to-string '(eval message "hello"))

In its implementation I see the references to print-escape-newlines
and print-quoted, but there is no print-dotted.

Re: Two problems with directory-local variables

[Stefan Monnier]

>> On this note, while I don't use the commands myself, I've often wished
>> that add-dir-local-variable would generate the dotted-pair syntax,
>> because I've often seen confusion about the syntax of dir-locals arising
>> from dotted vs non-dotted possibilities, and I feel strongly that the
>> dotted syntax is the most readable for these files.

Agreed.  We should probably have a `print-alist` kind of function for
that purpose and then try and make use of it from add-dir-local-variable.

> I don't know if something exists that could allow printing dotted syntax for
>
>   (pp-to-string '(eval message "hello"))
>
> In its implementation I see the references to print-escape-newlines
> and print-quoted, but there is no print-dotted.

I don't think we could add a print-dotted because as you say we don't
want (eval . (message . ("hello" . nil))) and only the human coder can
know which cons cell should be printed dotted and which shouldn't.


        Stefan

Re: Two problems with directory-local variables

[Juri Linkov]

>>> On this note, while I don't use the commands myself, I've often wished
>>> that add-dir-local-variable would generate the dotted-pair syntax,
>>> because I've often seen confusion about the syntax of dir-locals arising
>>> from dotted vs non-dotted possibilities, and I feel strongly that the
>>> dotted syntax is the most readable for these files.
>
> Agreed.  We should probably have a `print-alist` kind of function for
> that purpose and then try and make use of it from add-dir-local-variable.
>
>> I don't know if something exists that could allow printing dotted syntax for
>>
>>   (pp-to-string '(eval message "hello"))
>>
>> In its implementation I see the references to print-escape-newlines
>> and print-quoted, but there is no print-dotted.
>
> I don't think we could add a print-dotted because as you say we don't
> want (eval . (message . ("hello" . nil))) and only the human coder can
> know which cons cell should be printed dotted and which shouldn't.

Examples in (info "(emacs) Directory Variables") recommend dotted syntax
for alists on the first level:

     ((nil . ((indent-tabs-mode . t)
              (fill-column . 80)))
      (c-mode . ((c-file-style . "BSD")
                 (subdirs . nil))))

as well as for the second level:

      (("src/imported"
        . ((nil . ((change-log-default-name
                    . "ChangeLog.local"))))))

Taking into account the original problematic case with `eval',
dotted alist should be also on the third level:

      (("src/imported"
        . ((nil . ((eval
                    . (message "hello")))))))

Given that, I don't see what would be a good choice for generalization:

1. Since requirements for this feature are too vague, it doesn't seem
   to fit well into low-level printing functions.

2. Reformatting the printed output in pp-buffer to add dots where necessary
   also doesn't look like an elegant solution.

3. Implementing print-alist poses a question how to define at which levels
   to print alists with dotted syntax.

4. Since only add-dir-local-variable knows the semantics of used data
   structures, the simplest way would be to implement printing of each level
   explicitly in add-dir-local-variable.

RE: Two problems with directory-local variables

[Drew Adams]

> Given that, I don't see what would be a good choice for generalization:
> 
> 1. Since requirements for this feature are too vague, it doesn't seem
>    to fit well into low-level printing functions.
> 
> 2. Reformatting the printed output in pp-buffer to add dots where necessary
>    also doesn't look like an elegant solution.
> 
> 3. Implementing print-alist poses a question how to define at which levels
>    to print alists with dotted syntax.
> 
> 4. Since only add-dir-local-variable knows the semantics of used data
>    structures, the simplest way would be to implement printing of each level
>    explicitly in add-dir-local-variable.

I don't think there's a good design for this. Whatever some designer's intention
beforehand might be, the result will be misleading in at least some contexts.

And the problem is that when it's misleading (showing some dots but not
others it can be even more confusing for users than consistently doing either
what we do now - no dots for a list cdr, at all levels - or adding dots at all levels.

I think (so far, but I might change my mind if I see a good argument) that this
is really something that users of Lisp just have to get used to. It can be a
gotcha - like getting used to quoting (when to quote and when not to). Lisp
has a few such gotchas.

We could I suppose have `print-dot-levels', like we have `print-level', to give
code control over the level (top-down). That might be a little better than
`print-dotted' (Boolean). But is it really worth it (needed)? I tend not to
think it's worth it to have just a top-level `print-alist', in any case (and the
alist might not be at top level).

As Stefan said, "only the human coder can know which cons cell should be
printed dotted and which shouldn't." I'd change "cell" to "cells", but
otherwise I pretty much agree.

Re: Two problems with directory-local variables

[Juri Linkov]

> I don't think there's a good design for this. Whatever some designer's intention
> beforehand might be, the result will be misleading in at least some contexts.
>
> And the problem is that when it's misleading (showing some dots but not
> others it can be even more confusing for users than consistently doing either
> what we do now - no dots for a list cdr, at all levels - or adding dots at all levels.
>
> I think (so far, but I might change my mind if I see a good argument) that this
> is really something that users of Lisp just have to get used to. It can be a
> gotcha - like getting used to quoting (when to quote and when not to). Lisp
> has a few such gotchas.
>
> We could I suppose have `print-dot-levels', like we have `print-level', to give
> code control over the level (top-down). That might be a little better than
> `print-dotted' (Boolean). But is it really worth it (needed)? I tend not to
> think it's worth it to have just a top-level `print-alist', in any case (and the
> alist might not be at top level).
>
> As Stefan said, "only the human coder can know which cons cell should be
> printed dotted and which shouldn't." I'd change "cell" to "cells", but
> otherwise I pretty much agree.

The root cause of the problem is requiring the alist value to be stored
in the CDR that prints with confusing dotted syntax especially bad for
hand-editing, instead of storing the value in the CAR of the CDR printed
with simpler list syntax.