Error: Setting the shell in sh-mode via a local variable does not work

Error: Setting the shell in sh-mode via a local variable does not work

[Johannes Kastl]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dear all,

first of all, sorry if I failed in finding the answer to my question
on the web.

I have a file called 'whatever.spec.in', that basically is a rpm
spec file. (Technically, the spec file is generated from this file).
So, I'd like to have the nice rpm sh-mode, that my emacs uses for
files called 'whatever.spec'.

I googled a lot about major-modes, interpreter-mode-alist and all
kinds of stuff, even derived my own major-mode, but all fail, except
this:

(add-hook 'sh-mode-hook
          (lambda ()
            (if (string-match "\\.spec\\.in$" buffer-file-name)
                                (sh-set-shell "rpm"))))

This works. But:

The file contains some local variables at the end, that should
achieve the same:

# Local variables:
# mode: shell-script
# sh-shell: rpm
# end:

But regardless of what I put in the 'mode' and 'sh-shell' lines, I
always end up with shell-script[bash] when opening the file as
whatever.spec.in. If I rename it to whatever.spec, I get
shell-script[rpm].

Is this intended behaviour? Shouldn't the 'sh-shell:' line work? I
get no warning about malformed lines, so I think it is syntactically
correct.

Putting these local variables into an empty file call whatever.txt
gets me shell-script[bash], so apparently the 'mode:' line is used.

Thanks in advance, and please excuse the long descriptions...

Johannes

P.S.: System is OSX 10.10, emacs is GNU Emacs 24.5.1 from macports
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with SeaMonkey - http://www.enigmail.net/

iEYEARECAAYFAlaBZNIACgkQzi3gQ/xETbJLQgCfeBj+bVh4IzYCIb+CEMokqnko
MJYAn1dUEBM7w3VWCQa+/01gtKLcGulB
=r6ms
-----END PGP SIGNATURE-----

Re: Error: Setting the shell in sh-mode via a local variable does not work

[Emanuel Berg]

Johannes Kastl <mail@ojkastl.de> writes:

> I have a file called 'whatever.spec.in', that
> basically is a rpm spec file. (Technically, the spec
> file is generated from this file). So, I'd like to
> have the nice rpm sh-mode, that my emacs uses for
> files called 'whatever.spec'.

I know two ways of doing that.

The first way is the best. I include my code as well.
It solves your problem at line 2:

(let ((modes '(
     ("\\.spec.in\\'"                                           . (lambda () (interactive) (progn (sh-mode) (sh-set-shell "rpm"))))
     ("\\.bal\\'"                                               . balance-mode)
     ("\\.gpi\\'"                                               . gnuplot-mode)
     ("\\.\\(glsl\\|oil\\|ssc\\|dfy\\)\\'"                      . c-mode)
     ("\\.lu\\'"                                                . lua-mode)
     ("\\.\\(service\\|list\\|theme\\|mailrc\\|sed\\|inc\\)\\'" . conf-mode)
     ("\\.pic\\'"                                               . nroff-mode)
     ("\\.xr\\'"                                                . conf-xdefaults-mode)
     ("inputrc\\'"                                              . conf-colon-mode)
     ("KILL\\'"                                                 . emacs-lisp-mode)
     ("README\\'"                                               . text-mode)
     ("\\.service\\'"                                           . emacs)
     )))
  (setq auto-mode-alist (append modes auto-mode-alist)) )

Then the second way. Here you have to add

    # spec.in

to every file. That isn't a lot of work, but I think
it is unaestethic. Files shouldn't be ABOUT
technology, they should be about what you can do, and
are doing WITH technology. Anyway:

    (setq magic-mode-alist '(("# spec.in" . (lambda () (interactive) (progn (sh-mode) (sh-set-shell "rpm"))))))

;; another example:
;; (setq magic-mode-alist '(("/\\* cpp \\*/" . c++-mode)))

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Error: Setting the shell in sh-mode via a local variable does not work

[Johannes Kastl]

[-- Attachment #1: Type: text/plain, Size: 696 bytes --]

Dear Emanuel,

thanks for your answer.

Am 29.12.15 schrieb Emanuel Berg:
> Johannes Kastl <mail@ojkastl.de> writes:
> 
>> I have a file called 'whatever.spec.in', that
>> basically is a rpm spec file. (Technically, the spec
>> file is generated from this file). So, I'd like to
>> have the nice rpm sh-mode, that my emacs uses for
>> files called 'whatever.spec'.
> 
> I know two ways of doing that.

Sorry if my question was rather long, but you are answering the part
that I already had a solution for. Adding an entry to .emacs to add a
hook so sh-mode works.

But my main question was, why the local variable "sh-shell: rpm" set
in a file does not work...

Johannes


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 244 bytes --]

Re: Error: Setting the shell in sh-mode via a local variable does not work

[Emanuel Berg]

Johannes Kastl <mail@johannes-kastl.de> writes:

> But my main question was, why the local variable
> "sh-shell: rpm" set in a file does not work...

What variable is that?

How is it set in a file?

In what way does it not work?

Or better yet: What do you want to do/happen, which
you cannot do/doesn't happen?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Error: Setting the shell in sh-mode via a local variable does not work

[Robert Thorpe]

Emanuel Berg <embe8573@student.uu.se> writes:

> Johannes Kastl <mail@johannes-kastl.de> writes:
>
>> But my main question was, why the local variable
>> "sh-shell: rpm" set in a file does not work...
>
> What variable is that?
>
> How is it set in a file?
>
> In what way does it not work?
>
> Or better yet: What do you want to do/happen, which
> you cannot do/doesn't happen?

All of that was described in the original message that Johannes sent.

This bug report describes the same problem, but it says that it was
fixed in Emacs 24.5.
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17333

BR,
Robert Thorpe

Re: Error: Setting the shell in sh-mode via a local variable does not work

[Emanuel Berg]

Robert Thorpe <rt@robertthorpeconsulting.com> writes:

>>> But my main question was, why the local variable
>>> "sh-shell: rpm" set in a file does not work...
>>
>> What variable is that?
>> How is it set in a file?
>> In what way does it not work?
>> Or better yet: What do you want to do/happen, which
>> you cannot do/doesn't happen?
>
> All of that was described in the original message
> that Johannes sent.

Okay...?

Getting the buffer in the desired mode works for me
with the code I provided, on:

    GNU Emacs 24.5.1 (x86_64-pc-linux-gnu, GTK+
    Version 3.18.6) of 2015-12-14 on binet, modified
    by Debian

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Error: Setting the shell in sh-mode via a local variable does not work

[Johannes Kastl]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dear Robert,

Am 30.12.15 schrieb Robert Thorpe:
> 
> All of that was described in the original message that Johannes 
> sent.
> 
> This bug report describes the same problem, but it says that it
> was fixed in Emacs 24.5. 
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=17333

Thanks for the link. But I only read that it is fixed in version 25.1,
so I am not getting the fix in my 24.5.1.

> bug Marked as fixed in versions 25.1. Request was from Glenn 
> Morris <rgm <at> gnu.org> to control <at> debbugs.gnu.org. (Sat,
> 04 Oct 2014 16:36:04 GMT) Full text and rfc822 format available.

> bug No longer marked as fixed in versions 24.5. Request was from 
> Glenn Morris <rgm <at> gnu.org> to control <at> debbugs.gnu.org. 
> (Sat, 04 Oct 2014 16:36:04 GMT) Full text and rfc822 format 
> available.

OK, until I get 25.x on my mac I can live with my .emacs workaround.

Johannes
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
Comment: Using GnuPG with SeaMonkey - http://www.enigmail.net/

iEYEARECAAYFAlaEF4gACgkQzi3gQ/xETbJzowCeMd63viBiaQOdru4j6P6lHQI2
tOoAoJHvB94yHX124AhZuKwBaRviSedL
=repM
-----END PGP SIGNATURE-----

Re: Error: Setting the shell in sh-mode via a local variable does not work

[Teemu Likonen]

[-- Attachment #1: Type: text/plain, Size: 1528 bytes --]

I'll add some simple Lisp programming notes that are unrelated to your
discussion. I hope you don't mind. :-)


Emanuel Berg [2015-12-29 02:32:38+01] wrote:

> (let ((modes '(

>      )))
>   (setq auto-mode-alist (append modes auto-mode-alist)) )

There you create a list MODES and then APPEND copies the whole list and
joins it to AUTO-MODE-ALIST. The original list is discarded. No problem,
it's just a configuration code. But in some other situation it might be
good idea to construct the list only once and use NCONC which only
traverses the lists through and modifies last conses to join it to the
next list:

    (let ((modes (list '(a . b)
                       ;; ...
                       )))
      (setq auto-mode-alist (nconc modes auto-mode-alist)))

Or:

    (setq auto-mode-alist
          (nconc (list '(a . b)
                       ;; ...
                       )
                 auto-mode-alist))

I used LIST function because, at least in Common Lisp, literal objects
(like lists created with QUOTE ' or literal strings) shouldn't be
modified. Consequences are undefined.

>     (setq magic-mode-alist '(("# spec.in" . (lambda () (interactive) (progn (sh-mode) (sh-set-shell "rpm"))))))

The PROGN is redundant because LAMBDA has an implicit PROGN:

    (lambda ()
      (interactive)
      (sh-mode)
      (sh-set-shell "rpm"))

-- 
/// Teemu Likonen   - .-..   <https://github.com/tlikonen> //
// PGP: 4E10 55DC 84E9 DFF6 13D7 8557 719D 69D3 2453 9450 ///

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

`append' vs. `nconc' (was: Re: Error: Setting the shell in sh-mode via a local variable does not work)

[Emanuel Berg]

Teemu Likonen <tlikonen@iki.fi> writes:

> There you create a list MODES and then APPEND copies
> the whole list and joins it to AUTO-MODE-ALIST.
> The original list is discarded. No problem, it's
> just a configuration code. But in some other
> situation it might be good idea to construct the
> list only once and use NCONC which only traverses
> the lists through and modifies last conses to join
> it to the next list:
>
>     (let ((modes (list '(a . b) ;; ... ))) (setq
> auto-mode-alist (nconc modes auto-mode-alist)))

So what you are saying, if `append' is replaced by
`nconc', the same thing happens, only one less list
has to be created?

I have several appends in my source, is there
a rule-of-thumb when to use `append' and when to use
`nconc'?

> The PROGN is redundant because LAMBDA has an implicit
> PROGN

OK, cool.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: `append' vs. `nconc'

[Teemu Likonen]

[-- Attachment #1: Type: text/plain, Size: 1046 bytes --]

Emanuel Berg [2015-12-30 16:41:51+01] wrote:

> So what you are saying, if `append' is replaced by `nconc', the same
> thing happens, only one less list has to be created?

Given the following:

    (append list1 list2 list3 last)

    (nconc list1 list2 list3 last)

APPEND copies LIST1, LIST2 and LIST3 but not LAST. All original lists
remain untouched but LAST becomes part of the new list returned by
APPEND.

NCONC doesn't copy lists but modifies LIST1's, LIST2's and LIST3's last
cons's CDR to point to the next list. NCONC returnes the joined list.
Only LAST remains untouched.

> I have several appends in my source, is there a rule-of-thumb when to
> use `append' and when to use `nconc'?

You can use NCONC if it's safe to modify the lists, that is, you know
where the lists come from and know how they are used. Don't modify a
list created with '(...). Use (list ...) instead.

-- 
/// Teemu Likonen   - .-..   <https://github.com/tlikonen> //
// PGP: 4E10 55DC 84E9 DFF6 13D7 8557 719D 69D3 2453 9450 ///

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: `append' vs. `nconc'

[Emanuel Berg]

Teemu Likonen <tlikonen@iki.fi> writes:

> Given the following:
>
>     (append list1 list2 list3 last)
>
>     (nconc list1 list2 list3 last)
>
> APPEND copies LIST1, LIST2 and LIST3 but not LAST.
> All original lists remain untouched but LAST becomes
> part of the new list returned by APPEND.

 I think I always used `append' as if it were

    (append list1 ... listn '())

probably the reason it never bit me was I never did
much with the lists returned.

> NCONC doesn't copy lists but modifies LIST1's,
> LIST2's and LIST3's last cons's CDR to point to the
> next list. NCONC returnes the joined list. Only LAST
> remains untouched.

OK, so for example this:

(setq completion-ignored-extensions
      (append completion-ignored-extensions '(".bcf" ".run.xml")) )

Is better as:

    (nconc completion-ignored-extensions (list ".bcf" ".run.xml"))

?

>> I have several appends in my source, is there
>> a rule-of-thumb when to use `append' and when to
>> use `nconc'?
>
> You can use NCONC if it's safe to modify the lists,
> that is, you know where the lists come from and know
> how they are used. Don't modify a list created with
> '(...). Use (list ...) instead.

OK.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: `append' vs. `nconc'

[Pascal J. Bourguignon]

Emanuel Berg <embe8573@student.uu.se> writes:

> OK, so for example this:
>
> (setq completion-ignored-extensions
>       (append completion-ignored-extensions '(".bcf" ".run.xml")) )
>
> Is better as:
>
>     (nconc completion-ignored-extensions (list ".bcf" ".run.xml"))
>
> ?

No.  You should never use nconc only for effect (just like delete, etc).

Always re-assign the modified list:

    (setf completion-ignored-extensions
          (nconc completion-ignored-extensions (list ".bcf" ".run.xml")))

otherwise it will break when the place is bound to nil.  (nil is a symbol
and cannot be mutated into a cons cell!).

-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: `append' vs. `nconc'

[Emanuel Berg]

"Pascal J. Bourguignon" <pjb@informatimago.com>
writes:

>> OK, so for example this: (setq
>> completion-ignored-extensions (append
>> completion-ignored-extensions '(".bcf" ".run.xml"))
>> ) Is better as: (nconc completion-ignored-extensions
>> (list ".bcf" ".run.xml")) ?
>
> No. You should never use nconc only for effect (just
> like delete, etc).

What do you mean by "effect"?

The reason I thought it fitting was the definitions of
`append' and `ncons'. Here I want to change
`completion-ignored-extensions'. `append' doesn't
change it, which is why the assignment with `setq' was
there. If it can't be removed (well, in the form of
`setf'), there is still the gain of not creating the
list unnecessarily, which in this case is a matter of
principle but in other cases (with long lists) is
a performance bottleneck?

> Always re-assign the modified list:
>
>     (setf completion-ignored-extensions (nconc
> completion-ignored-extensions (list ".bcf"
> ".run.xml")))
>
> otherwise it will break when the place is bound to
> nil. (nil is a symbol and cannot be mutated into
> a cons cell!).

?

What place is bound to nil, when?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: `append' vs. `nconc'

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Dec 31, 2015 at 05:00:07AM +0100, Emanuel Berg wrote:
> "Pascal J. Bourguignon" <pjb@informatimago.com>
> writes:
> 
> >> OK, so for example this: (setq
> >> completion-ignored-extensions (append
> >> completion-ignored-extensions '(".bcf" ".run.xml"))
> >> ) Is better as: (nconc completion-ignored-extensions
> >> (list ".bcf" ".run.xml")) ?
> >
> > No. You should never use nconc only for effect (just
> > like delete, etc).
> 
> What do you mean by "effect"?

Side effect, the big lurking Monster for functional programmers:

  (defvar foo 15)

  (defun monster ()
    (setq foo 42))

  (monster)
  ;; now foo is 42: Gaaah!
  (message "%S" foo)

Nconc has a side effect: after calling it, many of the lists are
changed.

Side effects make things more "interesting", because now things depend
on the order of evaluation of things: it does matter, e.g. wheter you
evaluate the arguments of a function (call) left-to-right or right-to-left
(or random!). Things like that. The equivalent in C might be

  int i = 42;
  printf("%d %d %d\n", i++, i++, i++)

or the classic:

  a[i] = i++;

(which leads to undefined behaviour, because compiler writers need some
wiggling room to make efficient compilers!).

Sometimes you call functions because you *want* the side effect (aka
"calling for effect"), as in

  (message "Happy New Year!")

So-called "pure functional languages" have to go towards some "interesting"
contortions for that.

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlaE8+0ACgkQBcgs9XrR2kYE0QCfcFXQTDijDWoh4GQKerTVEuuw
ragAn2SMKqwM3yfp5LQQt5MLQAEgdMjV
=iPp+
-----END PGP SIGNATURE-----

Re: `append' vs. `nconc'

[Emanuel Berg]

<tomas@tuxteam.de> writes:

> Side effect, the big lurking Monster for functional
> programmers:
>
>   (defvar foo 15)
>
>   (defun monster () (setq foo 42))
>
>   (monster) ;; now foo is 42: Gaaah! (message "%S"
> foo)
>
> Nconc has a side effect: after calling it, many of the
> lists are changed.

However, because of the potential situation with
a symbol being nil or the empty list, `nconc' can't
really be used only "for effect" - so it should return
the list and that in turn be assigned!

So a little bit of functional programming still has to
be employed! (Not that the FP people has the monopoly
on returning stuff tho they might think so themselves
in their opium dreams.)

On the other hand, `append' isn't side-effect free!

    (setq *digits* '(1 2 3 4 5))
    (setq *chars*  '(a b c d e))
    (setq *appended-list* (append *digits* *chars*))
    (setf (car (last *appended-list*)) 'x)
    (insert (format " ; %s" *appended-list*)) ; (1 2 3 4 5 a b c d x)
    (insert (format " ; %s" *chars*)) ; (a b c d x)

Perhaps (concatenate 'list ...) is the
functional function?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: `append' vs. `nconc'

[Pascal J. Bourguignon]

Emanuel Berg <embe8573@student.uu.se> writes:

> "Pascal J. Bourguignon" <pjb@informatimago.com>
> writes:
>
>>> OK, so for example this: (setq
>>> completion-ignored-extensions (append
>>> completion-ignored-extensions '(".bcf" ".run.xml"))
>>> ) Is better as: (nconc completion-ignored-extensions
>>> (list ".bcf" ".run.xml")) ?
>>
>> No. You should never use nconc only for effect (just
>> like delete, etc).
>
> What do you mean by "effect"?

Side effect, same thing.

(nconc nil '(1 2 3)) has no side effect!

> The reason I thought it fitting was the definitions of
> `append' and `ncons'. Here I want to change
> `completion-ignored-extensions'. `append' doesn't
> change it, which is why the assignment with `setq' was
> there. If it can't be removed (well, in the form of
> `setf'), there is still the gain of not creating the
> list unnecessarily, which in this case is a matter of
> principle but in other cases (with long lists) is
> a performance bottleneck?
>
>> Always re-assign the modified list:
>>
>>     (setf completion-ignored-extensions (nconc
>> completion-ignored-extensions (list ".bcf"
>> ".run.xml")))
>>
>> otherwise it will break when the place is bound to
>> nil. (nil is a symbol and cannot be mutated into
>> a cons cell!).
>
> ?
>
> What place is bound to nil, when?

Any place you want, when it is.

    (let ((mylist '()))
       (nconc mylist (list 1 2 3))
       mylist)
    --> nil

    (setf completion-ignored-extensions '())
    (nconc completion-ignored-extensions  (list ".o"))
    completion-ignored-extensions
    --> nil


-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: `append' vs. `nconc'

[Emanuel Berg]

Teemu Likonen <tlikonen@iki.fi> writes:

> Don't modify a list created with '(...). Use (list
> ...) instead.

What about the elements of the lists? If they are
themselves lists, can they be created with ' or is
this rule a "recursive descent" one?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: `append' vs. `nconc'

[Pascal J. Bourguignon]

Emanuel Berg <embe8573@student.uu.se> writes:

> Teemu Likonen <tlikonen@iki.fi> writes:
>
>> Don't modify a list created with '(...). Use (list
>> ...) instead.
>
> What about the elements of the lists? If they are
> themselves lists, can they be created with ' or is
> this rule a "recursive descent" one?

You must remember the literal/immutable status of each item at each
level.

Since you are incapable of remembering it, you should assume that the
whole input data is immutable and write purely functional code (ie. use
append, not nconc) in general.


-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: `append' vs. `nconc'

[Emanuel Berg]

"Pascal J. Bourguignon" <pjb@informatimago.com>
writes:

> You must remember the literal/immutable status of
> each item at each level.
>
> Since you are incapable of remembering it, you
> should assume that the whole input data is immutable
> and write purely functional code (ie. use append,
> not nconc) in general.

OK, so you use `nconc' when you yourself create the
lists by hand and thus know they are not empty, all
the while using `list' and not ', and when done you
assign the result to a variable associated with a list
INSTEAD of using `nconc' directly because that
variable can hold the empty list, i.e. nil, which
`nconc' contrary to `append' cannot handle. And you do
this to save time!

Now I got it right!

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: `append' vs. `nconc'

[tomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Dec 31, 2015 at 08:13:41AM +0100, Emanuel Berg wrote:
> "Pascal J. Bourguignon" <pjb@informatimago.com>
> writes:
> 
> > You must remember the literal/immutable status of
> > each item at each level.
> >
> > Since you are incapable of remembering it, you
> > should assume that the whole input data is immutable
> > and write purely functional code (ie. use append,
> > not nconc) in general.
> 
> OK, so you use `nconc' when you yourself create the
> lists by hand and thus know they are not empty, all
> the while using `list' and not ', and when done you
> assign the result to a variable associated with a list
> INSTEAD of using `nconc' directly because that
> variable can hold the empty list, i.e. nil, which
> `nconc' contrary to `append' cannot handle. And you do
> this to save time!

Or, to squint differently at it: you use nconc when "nobody
is looking", i.e. when you *know* that there are no references
to the data being mutated (unless you *want* that the world
changes for those other watchers too).

OTOH, nconc is "just an optimization", because GC isn't ideal.
So a good rule of thumb:

 - it's very obvious
   It's a local value you are keeping in your little scope.
   (One might argue that then, it's an idiomatic way to
   emphasize this fact to the (human) reader...)

 - you are really pressed for time 
   It's somewhere deep in many nested loops.

Otherwise... just don't use it.

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlaE9cUACgkQBcgs9XrR2kZmSQCfRKgjktl8mIPS6/Eh/XkkHNiE
0E8AoIFZKrKi9bB1q0Ujul/IdXad8WX6
=Q8vS
-----END PGP SIGNATURE-----

side effects, list sharing [was: `append' vs. `nconc']

[Drew Adams]

Thomas wrote:

> nconc is "just an optimization", because GC isn't ideal.
> So a good rule of thumb:
>   - it's very obvious ...
>   - you are really pressed for time ...
> Otherwise... just don't use it.

_Just don't use side effects_, at least those such as `nconc'
that share list structure, is the right starting place, and is
definitely the best rule of thumb.  You will _never_ go wrong
with it as your motto, in code you write.

If, however, you modify code written by others, who do not
play by this simple rule, then you had better learn what it
is all about.  (By "you" I mean each of us.)

Lisp is Lisp.  It is not Haskell or lambda calculus.

The two guidelines that Thomas listed are fine.  But the most
important use case for side effects using functions such as
`nconc' has to do primarily with neither space nor time
optimization: it is _sharing of list structure_ for combined
effect.

IOW, precisely the thing that can get you in trouble is the
main reason for intentionally using such functions: you
_want_ a change in one list to automatically change another
in the same way.  You do that by making them share some list
structure.

And yes, such things are fraught with gotchas, exactly like
the sharing of data using global variables.

When a bug rears its ugly head, it is just as likely that you
see only an ugly pimple on a foot or elbow or tail, and not
on the head at all, and it is likely that the noticeable
manifestation will occur only long after you made the change
to the list.  Bonjour les dégats !

Again, this is _exactly_ like the problems that arise from
using global variables.  They closely _couple_ together
different program pieces and their logic.  And this is used
in Lisp for exactly the same reason that shared data has
been used in other languages.

IOW, this is nothing new, even if its form might be new to 
someone who is unused to Lisp lists.  If you are coming to
Lisp from Haskell then you might have a lot to learn in
this regard.  If you are coming to it from Fortran then
the data-sharing part is not new (but the rest of Lisp is).

This is Real Lisp (TM), and it is not for the faint-hearted.
And yes, we can all be bitten occasionally by the gotchas,
and we _all are_.  And it can be difficult to track down the
culprit code.  Difficult or fun, depending on how much free
time you have and how much you enjoy playing with Lisp.

You can say that "nconc is 'just an optimization'", but that
is true only for some very general, or very complicated and
program-specific, meaning of "optimization".  And it is not
necessarily "because GC isn't ideal".

If you never step off into List-Modification Land then it
is not important how such so-called "destructive" functions
are characterized.

But if you do venture there, e.g., to modify some existing
code that shares list structure, then know how, where, and
why the structure is shared, to start with.  And even then,
keep your wits about you, and keep a look out for dragons
over your shoulder and under your feet.

Re: side effects, list sharing [was: `append' vs. `nconc']

[Emanuel Berg]

Drew Adams <drew.adams@oracle.com> writes:

> _Just don't use side effects_, at least those such
> as `nconc' that share list structure, is the right
> starting place, and is definitely the best rule of
> thumb. You will _never_ go wrong with it as your
> motto, in code you write.

As a rule of thumb it is too general to be useful, as
a motto I suppose it is OK.

> Lisp is Lisp. It is not Haskell or lambda calculus.

That's exactly right. There was a guy here, namely
Rusi, who spent much effort advocating FP. We'll see
if he jumps into the discussion.

> The two guidelines that Thomas listed are fine.
> But the most important use case for side effects using
> functions such as `nconc' has to do primarily with
> neither space nor time optimization: it is _sharing of
> list structure_ for combined effect.

Yeah, we understand that is what `nconc' do. I think
the reason Thomas said that is that was the direction
the discussion went because it all started with having
my configuration not having to copy the list as would
happen with `append'.

But other then time/space optimization, what would you
say is the reason not to play it safe/sane and just
return stuff, new stuff if necessary, but instead
modify them directly?

Besides a possible time/space gain, I can only see
really creative/insane programmers doing that!

> This is Real Lisp (TM), and it is not for the
> faint-hearted. And yes, we can all be bitten
> occasionally by the gotchas, and we _all are_.
> And it can be difficult to track down the culprit
> code. Difficult or fun, depending on how much free
> time you have and how much you enjoy playing
> with Lisp.

Indeed, only I think it is the other way around:
I would get an heart attack instantly if I were to
think, argue, and code like the FP fanatics.

Math is another thing tho because math only has to
make sense and be correct, it doesn't have to work.
Programming has to work and if it makes sense all the
better :)

-- 
underground experts united
http://user.it.uu.se/~embe8573

RE: side effects, list sharing [was: `append' vs. `nconc']

[Drew Adams]

> But other then time/space optimization, what would you
> say is the reason not to play it safe/sane and just
> return stuff, new stuff if necessary, but instead
> modify them directly?

You will know it if you see it or when you really need it!

If you don't see a need for sharing list structure then
playing it safe/sane is the right approach.

If you are modifying someone else's code, and you see
structure-modifying code, that is, code that indicates
that list-sharing is going on, then you need to wake up
and take that into consideration.

Otherwise, there is no need to worry about it - just
play it safe/sane and Bob's your uncle.

If you just want to learn about such use cases, look
through existing Lisp code for uses of `setcar' and
company.  I can almost guarantee that they are not
used gratuitously.

And unless you see them used in a clearly local and
safe/encapsulated fashion, look around for what's
really going on, to see why that code uses list-sharing.

Re: side effects, list sharing [was: `append' vs. `nconc']

[Emanuel Berg]

Drew Adams <drew.adams@oracle.com> writes:

>> But other then time/space optimization, what would
>> you say is the reason not to play it safe/sane and
>> just return stuff, new stuff if necessary, but
>> instead modify them directly?
>
> You will know it if you see it or when you really
> need it!

Morpheus:

    Unfortunately, no one can be *told* what the
    Matrix is. You have to see it for yourself.

> If you don't see a need for sharing list structure
> then playing it safe/sane is the right approach.

Say that you do a rope-and-wheel mechanism to have
a bathroom door close automatically. As a weight, you
use a bottle. How do you tie the bottle to the rope?
Do you just tie it as a kid would, with random loops
all over, and then pull tight with your muscles? Or do
you look up a professional knot in the BIG BOOK
OF KNOTS?

Just as using advanced Lisp to solve perhaps trivial,
and definitely non-critical problems of Emacs
configuration, it can be seen as hitting the
heavy bag.

If the real deal, whatever that is, ever appears,
you'll be more ready than otherwise. If it doesn't
appear, you at least have a good looking knot, some
cool Elisp, and a fit body!

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: `append' vs. `nconc'

[Emanuel Berg]

<tomas@tuxteam.de> writes:

> - it's very obvious It's a local value you are
> keeping in your little scope. (One might argue that
> then, it's an idiomatic way to emphasize this fact
> to the (human) reader...)

The thing here is, when it is very obvious and
everything is small and easy to handle, what is the
actual gain in time and space not using `append'?

Answer:

    -> 0

But OK, as a matter of principle it can be a cool
thing to know/do.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: `append' vs. `nconc'

[Barry Margolin]

In article <mailman.1440.1451588113.843.help-gnu-emacs@gnu.org>,
 Emanuel Berg <embe8573@student.uu.se> wrote:

> <tomas@tuxteam.de> writes:
> 
> > - it's very obvious It's a local value you are
> > keeping in your little scope. (One might argue that
> > then, it's an idiomatic way to emphasize this fact
> > to the (human) reader...)
> 
> The thing here is, when it is very obvious and
> everything is small and easy to handle, what is the
> actual gain in time and space not using `append'?
> 
> Answer:
> 
>     -> 0
> 
> But OK, as a matter of principle it can be a cool
> thing to know/do.

NCONC was created at least 40 years ago, when computers were slow and 
had very little memory (a modern cellphone is has the power of a roomful 
of 70's-era mainframes). And garbage collection was also very slow 
(generational GC hadn't been developed yet).

So optimizations like NCONC could be very useful back then. These days, 
the benefit is probably very little.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: `append' vs. `nconc'

[Emanuel Berg]

Barry Margolin <barmar@alum.mit.edu> writes:

>> The thing here is, when it is very obvious and
>> everything is small and easy to handle, what is the
>> actual gain in time and space not using `append'?
>> 
>> Answer:
>> 
>>     -> 0
>> 
>> But OK, as a matter of principle it can be a cool
>> thing to know/do.
>
> NCONC was created at least 40 years ago, when
> computers were slow and had very little memory (a
> modern cellphone is has the power of a roomful of
> 70's-era mainframes). And garbage collection was
> also very slow (generational GC hadn't been
> developed yet).
>
> So optimizations like NCONC could be very useful
> back then. These days, the benefit is probably
> very little.

Q.E.D. :)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: `append' vs. `nconc'

[Pascal J. Bourguignon]

Emanuel Berg <embe8573@student.uu.se> writes:

> "Pascal J. Bourguignon" <pjb@informatimago.com>
> writes:
>
>> You must remember the literal/immutable status of
>> each item at each level.
>>
>> Since you are incapable of remembering it, you
>> should assume that the whole input data is immutable
>> and write purely functional code (ie. use append,
>> not nconc) in general.
>
> OK, so you use `nconc' when you yourself create the
> lists by hand and thus know they are not empty, all
> the while using `list' and not ', and when done you
> assign the result to a variable associated with a list
> INSTEAD of using `nconc' directly because that
> variable can hold the empty list, i.e. nil, which
> `nconc' contrary to `append' cannot handle. And you do
> this to save time!

nconc can handle nil.  But in that case, it doesn't modify it! It just
return the nconc'ed list:

(nconc nil nil (list 1 2 3) nil (list 3 4 5) nil)
--> (1 2 3 3 4 5)

(let ((list nil))
   (list (nconc list nil (list 1 2 3) nil (list 3 4 5) nil)
         list))
--> ((1 2 3 3 4 5) ; returned
     nil) ; list is still bound to nil which cannot be mutated.

(let ((list nil))
   (list (setf list (nconc list nil (list 1 2 3) nil (list 3 4 5) nil))
         list))
--> ((1 2 3 3 4 5)  ; returned and set to list
     (1 2 3 3 4 5)) ; list has been set with setf, so it's ok.

The only case where you could forget the setf, is when you just return
the nconc'ed result:

    (defun f (list to-add)
      (setf list (nconc list to-add))
      list)

can be written just:

    (defun f (list to-add)
      (nconc list to-add))

But in such a case, notice that you will have to bind or use the result
of f:

   (let ((list (list 1 2 3)))
     (setf list (f list (list 4 5 6)))
     list)
   --> (1 2 3 4 5 6)

It's like with delete, but for the opposite reason: delete cannot
transform a cons cell into the symbol nil:

    (delete 1 (list 1)) --> nil

But in anycase, it is not specified how delete (or delq) modifies the
list structure, so even:

     (let ((list (list 1 2 3 4 5)))
        (delete 3 list)
        list)

is not guaranteed to return: (1 2 4 5)

Instead, you have to write:

     (let ((list (list 1 2 3 4 5)))
        (setf list (delete 3 list))
        list)
     --> (1 2 4 5)

to ensure that list is bound to the wanted result of delete.

It is general: when you have a function that mutate a structure and
return the new structure, if the exact way how it mutates it is not
specified or if it cannot always mutate it (eg for lists, when there's
an impossible transition nil <-> cons cell), you have use the result of
the function and cannot count on the mutation.

So, to use a non-mutating function:

    (setf list (remove element list))
    (setf list (append list tail))

and to use a mutating function:

    (setf list (delete element list))
    (setf list (nconc list tail))

In both cases you must rebind the result.


> Now I got it right!

almost. ;-)

-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: `append' vs. `nconc'

[Teemu Likonen]

[-- Attachment #1: Type: text/plain, Size: 578 bytes --]

Emanuel Berg [2015-12-31 04:50:19+01] wrote:

> Teemu Likonen <tlikonen@iki.fi> writes:
>> Don't modify a list created with '(...). Use (list ...) instead.
>
> What about the elements of the lists? If they are themselves lists,
> can they be created with ' or is this rule a "recursive descent" one?

NCONC doesn't touch the elements (i.e., CAR values of conses) so this is
fine:

    (setq foo (nconc (list '(a . 1) '(b . 2)) foo))

-- 
/// Teemu Likonen   - .-..   <https://github.com/tlikonen> //
// PGP: 4E10 55DC 84E9 DFF6 13D7 8557 719D 69D3 2453 9450 ///

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: `append' vs. `nconc'

[Emanuel Berg]

Teemu Likonen <tlikonen@iki.fi> writes:

> NCONC doesn't touch the elements (i.e., CAR values
> of conses) so this is fine:
>
>     (setq foo (nconc (list '(a . 1) '(b . 2)) foo))

OK!

Because, many cases where I replaced `apply' by
`nconc' were lists of lists.

What is the reason one has to use `list'? (I suppose
the ` isn't safe, either.)

And: In Pascal's examples he always used `setf'.
The help says it can be used for places as well as for
symbols. In the examples tho there has been only
symbols if I'm not mistaken.

(setq completion-ignored-extensions
      (nconc completion-ignored-extensions (list ".bcf" ".run.xml")) )

(setq ispell-skip-region-alist
      (nconc ispell-skip-region-alist (list '("`" . "\\(`\\|'\\)"))) )

(let*((write
       (list
         '("hp"       "http://user.it.uu.se/~embe8573")
         '("lenght"   "length")
         ;; ...
         '("thedate"  "" (lambda () (interactive) (show-time-and-date t))) ; INSERT
         ))
      (groups (list
                '("ctt" "comp.text.tex")
                '("geg" "gmane.emacs.gnus")
                ;; ...
                ))
      (both (nconc write groups)))
  (define-abbrev-table 'global-abbrev-table both) )

(let ((modes (list
     '("\\.bal\\'"  . balance-mode)
     '("\\.gpi\\'"  . gnuplot-mode)
     ;; ...
     )))
  (setq auto-mode-alist (nconc modes auto-mode-alist)) )

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: `append' vs. `nconc'

[Teemu Likonen]

[-- Attachment #1: Type: text/plain, Size: 1106 bytes --]

Emanuel Berg [2015-12-31 19:35:52+01] wrote:

> Teemu Likonen <tlikonen@iki.fi> writes:
>>     (setq foo (nconc (list '(a . 1) '(b . 2)) foo))

> What is the reason one has to use `list'? (I suppose the ` isn't safe,
> either.)

LIST function creates a fresh list when it is evaluated, usually at
runtime. The following applies to the Common Lisp language: Literal
objects created with ' ` (QUOTE ...) "string" etc. might be created at
compile time. Such objects might even be shared by all instances of such
literal object. Common Lisp compilers are allowed to do that. So, the
literal list '(a b c) in one place might actually become the very same
(as in EQ) object as literal '(a b c) in somewhere else. If you mutate
one object the results may show everywhere, which is not usually wanted.

Even though it may not apply to Emacs Lisp or all Common Lisp compilers
it's good habit to not mutate literally created objects. Be functional
with them! :-)

-- 
/// Teemu Likonen   - .-..   <https://github.com/tlikonen> //
// PGP: 4E10 55DC 84E9 DFF6 13D7 8557 719D 69D3 2453 9450 ///

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: `append' vs. `nconc'

[Emanuel Berg]

Teemu Likonen <tlikonen@iki.fi> writes:

> LIST function creates a fresh list when it is
> evaluated, usually at runtime. The following applies
> to the Common Lisp language: Literal objects created
> with ' ` (QUOTE ...) "string" etc. might be created
> at compile time. Such objects might even be shared
> by all instances of such literal object. Common Lisp
> compilers are allowed to do that. So, the literal
> list '(a b c) in one place might actually become the
> very same (as in EQ) object as literal '(a b c) in
> somewhere else. If you mutate one object the results
> may show everywhere, which is not usually wanted.
>
> Even though it may not apply to Emacs Lisp or all
> Common Lisp compilers it's good habit to not mutate
> literally created objects. Be functional with them!

OK, then what functions, other than `nconc', are
mutators (or, how do you now)?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: `append' vs. `nconc'

[Teemu Likonen]

[-- Attachment #1: Type: text/plain, Size: 350 bytes --]

Emanuel Berg [2016-01-01 14:23:26+01] wrote:

> OK, then what functions, other than `nconc', are mutators (or, how do
> you now)?

A manual or the specification says it. Such operation is called
"destructive".

-- 
/// Teemu Likonen   - .-..   <https://github.com/tlikonen> //
// PGP: 4E10 55DC 84E9 DFF6 13D7 8557 719D 69D3 2453 9450 ///

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: `append' vs. `nconc'

[Emanuel Berg]

Teemu Likonen <tlikonen@iki.fi> writes:

>> OK, then what functions, other than `nconc', are
>> mutators (or, how do you now)?
>
> A manual or the specification says it.
> Such operation is called "destructive".

A function is destructive if it changes its arguments.
I suppose it is even more destructive if it changes
things that aren't even passed to the function.

What I can tell this definition is straightforward so
it is rather a question of knowing what functions are
like that.

For example, the help for `nconc' says:

    (nconc &rest LISTS)

    Concatenate any number of lists by altering them.
    Only the last argument is not altered, and need not be
    a list.

And `append':

    (append &rest SEQUENCES)

    Concatenate all the arguments and make the result
    a list.
    The result is a list whose elements are the
    elements of all the arguments.
    Each argument may be a list, vector or string.
    The last argument is not copied, just used as the
    tail of the new list.

You need to be attentive but if you are, sure, it
is there.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: `append' vs. `nconc'

[Pascal J. Bourguignon]

Emanuel Berg <embe8573@student.uu.se> writes:

> What is the reason one has to use `list'? (I suppose
> the ` isn't safe, either.)

list returns a fresh list. ` doesn't. ` may return a structure that
shares literal substructures with others.

Typically, it will be a tail, but it can also be some cars:

    `((a . b) (c . ,x) ,@y d e f)

could translate to:

    (list* '(a . b) (cons 'c x) (append y '(d e f)))

or something similar.  Then each time you execute it, you get a list
whose tail is the same literal (d e f), and whose car is the same
literal cons (a . b).


> And: In Pascal's examples he always used `setf'.
> The help says it can be used for places as well as for
> symbols. In the examples tho there has been only
> symbols if I'm not mistaken.

It's simplier to forget about set and setq, and always use setf.
That frees two neurons to think about other things.


-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: `append' vs. `nconc'

[Emanuel Berg]

"Pascal J. Bourguignon" <pjb@informatimago.com>
writes:

> list returns a fresh list. ` doesn't. ` may return
> a structure that shares literal substructures
> with others.

I have lots of ' and ` and instead of examining all
those cases I think it is easier to look for the
mutators, and see if ' or ` are used in
their vicinity.

I found this page:

    http://www.emacswiki.org/emacs/DestructiveOperations

> It's simplier to forget about set and setq, and
> always use setf.

Oh, no! I have 207 `setq' in my initialization!

> That frees two neurons to think about other things.

I think we have spent too much time together lately:
now we are starting to *think* the same way!

-- 
underground experts united
http://user.it.uu.se/~embe8573

RE: `append' vs. `nconc'

[Drew Adams]

> I have lots of ' and ` and instead of examining all
> those cases I think it is easier to look for the
> mutators, and see if ' or ` are used in their vicinity.
>
> I found this page: http://www.emacswiki.org/emacs/DestructiveOperations

Yes, just keep an eye out for the mutators; that's all.
And it's not about ' or `.  Whenever "destructive" functions
are involved, you need to keep an eye out more generally -
but specifically wherever the structures get mutated or used
(referenced).

Re: `append' vs. `nconc'

[Pascal J. Bourguignon]

Emanuel Berg <embe8573@student.uu.se> writes:

> Teemu Likonen <tlikonen@iki.fi> writes:
>
>> There you create a list MODES and then APPEND copies
>> the whole list and joins it to AUTO-MODE-ALIST.
>> The original list is discarded. No problem, it's
>> just a configuration code. But in some other
>> situation it might be good idea to construct the
>> list only once and use NCONC which only traverses
>> the lists through and modifies last conses to join
>> it to the next list:
>>
>>     (let ((modes (list '(a . b) ;; ... ))) (setq
>> auto-mode-alist (nconc modes auto-mode-alist)))
>
> So what you are saying, if `append' is replaced by
> `nconc', the same thing happens, only one less list
> has to be created?
>
> I have several appends in my source, is there
> a rule-of-thumb when to use `append' and when to use
> `nconc'?

In doubt, use append.

If you are appending freshly consed lists, then it's safe to use nconc.

For example:

    (defun f (lists)
      (loop
        for list in lists
        append list))

    (defun f* (lists)
      (apply (function append) lists))

    (defun g (lists)
      (loop
         for list in lists
         nconc (mapcar (function 1+) list)))


In f, we are appending lists that are given to the function in the
parameter. Since we don't know whether those lists will be shared or
literal (we didn't specify anything about the lists), we should avoid
mutating them, therefore we use append in f.  A new copy of each of the
list will be used to build the result.

Notice that in f*, append does not copy the last list. We obtain a
result that shares structure with the arguments.  This may be a problem
or not, but in any case, that means that the result of f* is not a fresh
list, even if most of it is made of new cons cells, because of this
shared tail!

When you have a fixed set of lists to append, you can either use
concatenate, or add a '() as last argument to append:

        (concatenate 'list l1 l2 l3) --> a fresh list
        (append l1 l2 l3 '())        --> a fresh list
        (append l1 l2 l3)            --> a list sharing structure with l3.


In the case of g, since we create new lists (by way of mapcar), there
are no other reference to those lists, so we can use nconc, mutating
their last cdr to concatenate them without having to copy them.



Another way to say it: if you could use sort on your lists without
copy-list, then you can use nconc, otherwise you should use append:

(let ((l (list 1 2 3))
       s)
  (list (setf s (sort l '<))     ; ok
        (nconc s (list 4 5 6)))) ; ==> nconc ok

(let ((l '(1 2 3))
      s)
  (list (setf s (sort (copy-list l) '<))  ; copy-list needed
        (append l (list 4 5 6))           ; ==> use append
        (nconc s (list 4 5 6)))) ; s is a fresh copy ==> nconc ok


-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk