side effects, list sharing [was: `append' vs. `nconc']

[Drew Adams <drew.adams@oracle.com>]

Thomas wrote:

> nconc is "just an optimization", because GC isn't ideal.
> So a good rule of thumb:
>   - it's very obvious ...
>   - you are really pressed for time ...
> Otherwise... just don't use it.

_Just don't use side effects_, at least those such as `nconc'
that share list structure, is the right starting place, and is
definitely the best rule of thumb.  You will _never_ go wrong
with it as your motto, in code you write.

If, however, you modify code written by others, who do not
play by this simple rule, then you had better learn what it
is all about.  (By "you" I mean each of us.)

Lisp is Lisp.  It is not Haskell or lambda calculus.

The two guidelines that Thomas listed are fine.  But the most
important use case for side effects using functions such as
`nconc' has to do primarily with neither space nor time
optimization: it is _sharing of list structure_ for combined
effect.

IOW, precisely the thing that can get you in trouble is the
main reason for intentionally using such functions: you
_want_ a change in one list to automatically change another
in the same way.  You do that by making them share some list
structure.

And yes, such things are fraught with gotchas, exactly like
the sharing of data using global variables.

When a bug rears its ugly head, it is just as likely that you
see only an ugly pimple on a foot or elbow or tail, and not
on the head at all, and it is likely that the noticeable
manifestation will occur only long after you made the change
to the list.  Bonjour les dégats !

Again, this is _exactly_ like the problems that arise from
using global variables.  They closely _couple_ together
different program pieces and their logic.  And this is used
in Lisp for exactly the same reason that shared data has
been used in other languages.

IOW, this is nothing new, even if its form might be new to 
someone who is unused to Lisp lists.  If you are coming to
Lisp from Haskell then you might have a lot to learn in
this regard.  If you are coming to it from Fortran then
the data-sharing part is not new (but the rest of Lisp is).

This is Real Lisp (TM), and it is not for the faint-hearted.
And yes, we can all be bitten occasionally by the gotchas,
and we _all are_.  And it can be difficult to track down the
culprit code.  Difficult or fun, depending on how much free
time you have and how much you enjoy playing with Lisp.

You can say that "nconc is 'just an optimization'", but that
is true only for some very general, or very complicated and
program-specific, meaning of "optimization".  And it is not
necessarily "because GC isn't ideal".

If you never step off into List-Modification Land then it
is not important how such so-called "destructive" functions
are characterized.

But if you do venture there, e.g., to modify some existing
code that shares list structure, then know how, where, and
why the structure is shared, to start with.  And even then,
keep your wits about you, and keep a look out for dragons
over your shoulder and under your feet.