Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Štěpán Němec]

On Sat, 18 Apr 2020 16:01:14 -0400 (EDT)
Paul Eggert wrote:

> branch: emacs-27
> commit eebfb72c906755c0a80d92c11deee7ac9faf5f4b
> Author: Paul Eggert <eggert@cs.ucla.edu>
> Commit: Paul Eggert <eggert@cs.ucla.edu>
>
>     Document constant vs mutable objects better

I think this is great, except for a few places mentioned below.

> @@ -9547,7 +9557,7 @@ part of which is the address of the next pair.  The very last box
>  points to the symbol @code{nil}, which marks the end of the list.
>  
>  @need 1200
> -When a variable is set to a list with a function such as @code{setq},
> +When a variable is set to a list via @code{setq},

I understand you wanted to avoid implying that `setq' was a function,
but how about "setq or a similar function", "e.g. setq", or just leave
it at "set to a list"? Otherwise some people might wonder if `setq' is
somehow special (and the original formulation apparently tried to avoid
giving that impression).

>  it stores the address of the first box in the variable.  Thus,
>  evaluation of the expression
>  
> @@ -17092,7 +17102,7 @@ reminders.
>  
>  @cindex Mail aliases
>  @noindent
> -This @code{setq} command sets the value of the variable
> +This @code{setq} sets the value of the variable
>  @code{mail-aliases} to @code{t}.  Since @code{t} means true, the line
>  says, in effect, ``Yes, use mail aliases.''
>  
> @@ -17130,8 +17140,8 @@ The following turns off Indent Tabs mode:
>  @end smallexample
>  
>  Note that this line uses @code{setq-default} rather than the
> -@code{setq} command that we have seen before.  The @code{setq-default}
> -command sets values only in buffers that do not have their own local
> +@code{setq} that we have seen before.  The @code{setq-default}
                                          ^^^
With the following "command" removed, the article now seems
superfluous/awkward.

> +sets values only in buffers that do not have their own local
>  values for the variable.
>  

> -However, the other arguments (all but the last) must be lists.
> +However, the other arguments (all but the last) must be mutable lists.
                                                           ^^^^^^^
I don't think this is a good formulation, for reasons explained at the
bottom.

> @@ -1353,7 +1363,7 @@ Compare this with @code{memq}:
>       @result{} (1.2 1.3)
>  @end group
>  @group
> -(memq 1.2 '(1.1 1.2 1.3))  ; @r{@code{1.2} and @code{1.2} are not @code{eq}.}
> +(memq (list 2) '((1) (2)))  ; @r{@code{(list 2)} and @code{(2)} are not @code{eq}.}

I think this is missing the point, which is numeric comparison with `eq'
vs `eql' ("floating-point elements are compared by value"); no
mutability issues involved.

>       @result{} nil
>  @end group
>  @end example
> @@ -1373,11 +1383,11 @@ Compare this with @code{memq}:
>  
>  @example
>  @group
> -(member '(2) '((1) (2)))  ; @r{@code{(2)} and @code{(2)} are @code{equal}.}
> +(member (list 2) '((1) (2)))  ; @r{@code{(list 2)} and @code{(2)} are @code{equal}.}
>       @result{} ((2))
>  @end group
>  @group
> -(memq '(2) '((1) (2)))    ; @r{@code{(2)} and @code{(2)} are not @code{eq}.}
> +(memq (list 2) '((1) (2)))    ; @r{@code{(list 2)} and @code{(2)} are not @code{eq}.}

Not sure about this, either.

> @@ -1618,9 +1628,9 @@ keys may not be symbols:
>        '(("simple leaves" . oak)
>          ("compound leaves" . horsechestnut)))
>  
> -(assq "simple leaves" leaves)
> +(assq (copy-sequence "simple leaves") leaves)
>       @result{} nil
> -(assoc "simple leaves" leaves)
> +(assoc (copy-sequence "simple leaves") leaves)

This seems strange to me. Using `copy-sequence' just to get a key for
comparison?

> @@ -1237,7 +1237,8 @@ w
>  @end group
>  
>  @group
> -(setq x "asdfasfd")
> +;; @r{@code{copy-sequence} creates a mutable string.}

I think this comment is unnecessarily misleading, as it seems to imply
that literal strings are not mutable. IOW, "mutable" and "safely
mutable"/"mutating it is a good idea" should be distinguished.

> +(setq x (copy-sequence "asdfasfd"))
>       @result{} "asdfasfd"
>  (aset x 3 ?Z)
>       @result{} 90
> @@ -1246,6 +1247,10 @@ x
>  @end group
>  @end example
>  
> +The @var{array} should be mutable; that is, it should not be a constant,

Similarly here. There are languages, Lisps even, that distinguish
mutable and immutable strings/conses. In such languages, trying to
mutate an immutable data structure leads to an error being signalled.
Conflating "mutable" with "safely mutable" is IMO bound to induce
confusion.

> +such as the constants created via quoting or via self-evaluating forms.
> +@xref{Self-Evaluating Forms}.

  Thanks,

  Štěpán

RE: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Drew Adams]

> "mutable" and "safely mutable"/"mutating it is
> a good idea" should be distinguished.

Exactly.

> Similarly here. There are languages, Lisps even, that distinguish
> mutable and immutable strings/conses. In such languages, trying to
> mutate an immutable data structure leads to an error being signalled.
> Conflating "mutable" with "safely mutable" is IMO bound to induce
> confusion.

Ditto.

Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 3172 bytes --]

On 4/18/20 2:24 PM, Štěpán Němec wrote:

>> -When a variable is set to a list with a function such as @code{setq},
>> +When a variable is set to a list via @code{setq},
> 
> I understand you wanted to avoid implying that `setq' was a function,
> but how about "setq or a similar function",

Thanks, that sounds good. See attached patch. Unless otherwise specified, the 
patch simply adopts your other suggestions.
>> -However, the other arguments (all but the last) must be lists.
>> +However, the other arguments (all but the last) must be mutable lists.
>                                                             ^^^^^^^
> I don't think this is a good formulation, for reasons explained at the
> bottom.

See below.

>> -(memq 1.2 '(1.1 1.2 1.3))  ; @r{@code{1.2} and @code{1.2} are not @code{eq}.}
>> +(memq (list 2) '((1) (2)))  ; @r{@code{(list 2)} and @code{(2)} are not @code{eq}.}
> 
> I think this is missing the point, which is numeric comparison with `eq'
> vs `eql' ("floating-point elements are compared by value"); no
> mutability issues involved.

You're right that my change missed the point and needs to be fixed. However, 
mutability issues are relevant here: mutability came up partly because the Lisp 
compiler is free to share structure of the original program and this means those 
two instances of 1.2 might be eq. The attached patch reverts the abovementioned 
change but adds commentary about why that memq call might not return nil.

>> -(memq '(2) '((1) (2)))    ; @r{@code{(2)} and @code{(2)} are not @code{eq}.}
>> +(memq (list 2) '((1) (2)))    ; @r{@code{(list 2)} and @code{(2)} are not @code{eq}.}
> 
> Not sure about this, either.

It's the same point as the previous one.  The two instances of (2) in the old 
example might be eq so the old example might yield non-nil. But the (list 2) and 
the quoted (2) in the revised example cannot be eq, and this corrects the example.

>> > -(assq "simple leaves" leaves)
>> > +(assq (copy-sequence "simple leaves") leaves)
>> >       @result{} nil
>> > -(assoc "simple leaves" leaves)
>> > +(assoc (copy-sequence "simple leaves") leaves)
> 
> This seems strange to me. Using `copy-sequence' just to get a key for
> comparison?

The copy-sequence is needed to ensure that the assq/assoc string key argument is 
not eq to the string within the alist argument. The attached patch adds a 
comment "The @code{copy-sequence} means the keys are not @code{eq}."

> IOW, "mutable" and "safely
> mutable"/"mutating it is a good idea" should be distinguished.

The attached patch makes that distinction in a new section "Constants and 
mutability" that I hope clarifies some of the issues here. While writing this, I 
noticed that for Elisp we needn't separate the three notions 
(not-mutable-and-this-is-checked, not-safe-to-mutate, safely-mutable) 
everywhere; all we need to do is describe those notions in one spot, and 
everywhere else we can just use the term "mutable" for safely-mutable and the 
term "constant" for everything else.

Thanks for your detailed comments and suggestions. Further comments are welcome 
of course.

[-- Attachment #2: 0001-Improve-mutability-documentation.patch --]
[-- Type: text/x-patch, Size: 11505 bytes --]

From dca35b31d0a58efdcc698faf90493b96fa8e1406 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Sun, 19 Apr 2020 12:00:49 -0700
Subject: [PATCH] Improve mutability documentation
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

This change was inspired by comments from Štěpán Němec in:
https://lists.gnu.org/r/emacs-devel/2020-04/msg01063.html
* doc/lispref/objects.texi (Lisp Data Types): Mention mutability.
(Constants and mutability): New section.
* doc/lispintro/emacs-lisp-intro.texi (Lists diagrammed)
(Indent Tabs Mode): Improve wording.
* doc/lispref/eval.texi (Self-Evaluating Forms):
Say that they return constants.
* doc/lispref/lists.texi (Sets And Lists):
Fix memql mistake/confusion that I recently introduced.
---
 doc/lispintro/emacs-lisp-intro.texi |  4 +--
 doc/lispref/elisp.texi              |  1 +
 doc/lispref/eval.texi               |  7 +++++
 doc/lispref/lists.texi              | 16 +++++-----
 doc/lispref/objects.texi            | 45 +++++++++++++++++++++++++++++
 doc/lispref/sequences.texi          | 10 +++----
 doc/lispref/strings.texi            |  1 +
 7 files changed, 69 insertions(+), 15 deletions(-)

diff --git a/doc/lispintro/emacs-lisp-intro.texi b/doc/lispintro/emacs-lisp-intro.texi
index 630676d978..ea16d9ef15 100644
--- a/doc/lispintro/emacs-lisp-intro.texi
+++ b/doc/lispintro/emacs-lisp-intro.texi
@@ -9557,7 +9557,7 @@ part of which is the address of the next pair.  The very last box
 points to the symbol @code{nil}, which marks the end of the list.
 
 @need 1200
-When a variable is set to a list via @code{setq},
+When a variable is set to a list with an operation such as @code{setq},
 it stores the address of the first box in the variable.  Thus,
 evaluation of the expression
 
@@ -17140,7 +17140,7 @@ The following turns off Indent Tabs mode:
 @end smallexample
 
 Note that this line uses @code{setq-default} rather than the
-@code{setq} that we have seen before.  The @code{setq-default}
+@code{setq} that we have seen before; @code{setq-default}
 sets values only in buffers that do not have their own local
 values for the variable.
 
diff --git a/doc/lispref/elisp.texi b/doc/lispref/elisp.texi
index cfd96f7aa6..bba1b63115 100644
--- a/doc/lispref/elisp.texi
+++ b/doc/lispref/elisp.texi
@@ -297,6 +297,7 @@ Lisp Data Types
 * Circular Objects::        Read syntax for circular structure.
 * Type Predicates::         Tests related to types.
 * Equality Predicates::     Tests of equality between any two objects.
+* Constants and Mutability::  Whether an object's value can change.
 
 Programming Types
 
diff --git a/doc/lispref/eval.texi b/doc/lispref/eval.texi
index 46cfab164b..deb288943a 100644
--- a/doc/lispref/eval.texi
+++ b/doc/lispref/eval.texi
@@ -158,6 +158,12 @@ contents unchanged.
 @end group
 @end example
 
+  A self-evaluating form yields a constant, and you should not attempt
+to modify the constant's contents via @code{setcar}, @code{aset} or
+similar primitives.  The Lisp interpreter might unify the constants
+yielded by your program's self-evaluating forms, so that these
+constants might share structure.  @xref{Constants and Mutability}.
+
   It is common to write numbers, characters, strings, and even vectors
 in Lisp code, taking advantage of the fact that they self-evaluate.
 However, it is quite unusual to do this for types that lack a read
@@ -559,6 +565,7 @@ and vectors.)
 @defspec quote object
 This special form returns @var{object}, without evaluating it.
 The returned value is a constant, and should not be modified.
+@xref{Constants and Mutability}.
 @end defspec
 
 @cindex @samp{'} for quoting
diff --git a/doc/lispref/lists.texi b/doc/lispref/lists.texi
index c2771b0165..f1acc85616 100644
--- a/doc/lispref/lists.texi
+++ b/doc/lispref/lists.texi
@@ -866,16 +866,15 @@ foo                       ;; @r{@code{foo} was changed.}
 @node Modifying Lists
 @section Modifying Existing List Structure
 @cindex destructive list operations
-@cindex constant lists
 @cindex mutable lists
 
   You can modify the @sc{car} and @sc{cdr} contents of a cons cell with the
 primitives @code{setcar} and @code{setcdr}.  These are destructive
 operations because they change existing list structure.
-Destructive operations should be applied only to @dfn{mutable} lists,
+Destructive operations should be applied only to mutable lists,
 that is, lists constructed via @code{cons}, @code{list} or similar
 operations.  Lists created by quoting are constants and should not be
-changed by destructive operations.
+changed by destructive operations.  @xref{Constants and Mutability}.
 
 @cindex CL note---@code{rplaca} vs @code{setcar}
 @quotation
@@ -1169,9 +1168,9 @@ x
 @end group
 @end example
 
-However, the other arguments (all but the last) must be mutable lists.
+However, the other arguments (all but the last) should be mutable lists.
 
-A common pitfall is to use a quoted constant list as a non-last
+A common pitfall is to use a constant list as a non-last
 argument to @code{nconc}.  If you do this, the resulting behavior
 is undefined.  It is possible that your program will change
 each time you run it!  Here is what might happen (though this
@@ -1359,12 +1358,12 @@ Compare this with @code{memq}:
 
 @example
 @group
-(memql 1.2 '(1.1 1.2 1.3))  ; @r{@code{1.2} and @code{1.2} are @code{eql}.}
+(memql 1.2 '(1.1 1.2 1.3))  ; @r{@code{1.2} and @code{1.2} must be @code{eql}.}
      @result{} (1.2 1.3)
 @end group
 @group
-(memq (list 2) '((1) (2)))  ; @r{@code{(list 2)} and @code{(2)} are not @code{eq}.}
-     @result{} nil
+(memq 1.2 '(1.1 1.2 1.3))  ; @r{@code{1.2} and @code{1.2} need not be @code{eq}.}
+     @result{} nil         ; @r{... or it might be @code{(1.2 1.3)}.}
 @end group
 @end example
 @end defun
@@ -1628,6 +1627,7 @@ keys may not be symbols:
       '(("simple leaves" . oak)
         ("compound leaves" . horsechestnut)))
 
+;; @r{The @code{copy-sequence} means the keys are not @code{eq}.}
 (assq (copy-sequence "simple leaves") leaves)
      @result{} nil
 (assoc (copy-sequence "simple leaves") leaves)
diff --git a/doc/lispref/objects.texi b/doc/lispref/objects.texi
index 1c4e7e4d4e..98b001afd2 100644
--- a/doc/lispref/objects.texi
+++ b/doc/lispref/objects.texi
@@ -46,6 +46,10 @@ you store in it, type and all.  (Actually, a small number of Emacs
 Lisp variables can only take on values of a certain type.
 @xref{Variables with Restricted Values}.)
 
+  Some Lisp objects are @dfn{constant}: their values never change.
+Others are @dfn{mutable}: their values can be changed via destructive
+operations that involve side effects.
+
   This chapter describes the purpose, printed representation, and read
 syntax of each of the standard types in GNU Emacs Lisp.  Details on how
 to use these types can be found in later chapters.
@@ -59,6 +63,7 @@ to use these types can be found in later chapters.
 * Circular Objects::            Read syntax for circular structure.
 * Type Predicates::             Tests related to types.
 * Equality Predicates::         Tests of equality between any two objects.
+* Constants and Mutability::    Whether an object's value can change.
 @end menu
 
 @node Printed Representation
@@ -2373,3 +2378,43 @@ that for two strings to be equal, they have the same text properties.
 @end group
 @end example
 @end defun
+
+@node Constants and Mutability
+@section Constants and Mutability
+@cindex constants
+@cindex mutable objects
+
+  Some Lisp objects are constant: their values never change.
+For example, you can create a new integer by calculating one, but you
+cannot modify the value of an existing integer.
+
+  Other Lisp objects are mutable: their values can be changed
+via destructive operations involving side effects.  For example, an
+existing marker can be changed by moving the marker to point to
+somewhere else.
+
+  Although numbers are always constants and markers are always
+mutable, some types contain both constant and mutable members.  These
+types include conses, vectors, and strings.  For example, the string
+literal @code{"aaa"} yields a constant string, whereas the function
+call @code{(make-string 3 ?a)} yields a mutable string that can be
+changed via later calls to @code{aset}.
+
+  A program should not attempt to modify a constant because the
+resulting behavior is undefined: the Lisp interpreter might or might
+not detect the error, and if it does not detect the error the
+interpreter can behave unpredictably thereafter.  Another way to put
+this is that mutable objects are safe to change, whereas constants are
+not safely mutable: if you try to change a constant your program might
+behave as you expect but it might crash or worse.  This problem occurs
+with types that have both constant and mutable members, and that have
+mutators like @code{setcar} and @code{aset} that are valid on mutable
+objects but hazardous on constants.
+
+  When the same constant occurs multiple times in a program, the Lisp
+interpreter might save time or space by reusing existing constants or
+constant components.  For example, @code{(eq "abc" "abc")} returns
+@code{t} if the interpreter creates only one instance of the string
+constant @code{"abc"}, and returns @code{nil} if it creates two
+instances.  Lisp programs should be written so that they work
+regardless of whether this optimization is in use.
diff --git a/doc/lispref/sequences.texi b/doc/lispref/sequences.texi
index 7a3f26e584..62d60156fb 100644
--- a/doc/lispref/sequences.texi
+++ b/doc/lispref/sequences.texi
@@ -1249,7 +1249,7 @@ x
 
 The @var{array} should be mutable; that is, it should not be a constant,
 such as the constants created via quoting or via self-evaluating forms.
-@xref{Self-Evaluating Forms}.
+@xref{Constants and Mutability}.
 
 If @var{array} is a string and @var{object} is not a character, a
 @code{wrong-type-argument} error results.  The function converts a
@@ -1306,10 +1306,10 @@ same way in Lisp input.
 
   A vector, like a string or a number, is considered a constant for
 evaluation: the result of evaluating it is the same vector.  This does
-not evaluate or even examine the elements of the vector.  Vectors
-written with square brackets are constants and should not be modified
-via @code{aset} or other destructive operations.
-@xref{Self-Evaluating Forms}.
+not evaluate or even examine the elements of the vector.
+@xref{Self-Evaluating Forms}.  Vectors written with square brackets
+are constants and should not be modified via @code{aset} or other
+destructive operations.  @xref{Constants and Mutability}.
 
   Here are examples illustrating these principles:
 
diff --git a/doc/lispref/strings.texi b/doc/lispref/strings.texi
index 3acbf538dc..a4c9c2549c 100644
--- a/doc/lispref/strings.texi
+++ b/doc/lispref/strings.texi
@@ -384,6 +384,7 @@ usual value is @w{@code{"[ \f\t\n\r\v]+"}}.
   You can alter the contents of a mutable string via operations
 described in this section.  However, you should not try to use these
 operations to alter the contents of a constant string.
+@xref{Constants and Mutability}.
 
   The most basic way to alter the contents of an existing string is with
 @code{aset} (@pxref{Array Functions}).  @code{(aset @var{string}
-- 
2.17.1

Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Štěpán Němec]

On Sun, 19 Apr 2020 13:37:54 -0700
Paul Eggert wrote:

> Unless otherwise specified, the patch simply adopts your other
> suggestions.

Thank you, and sorry for aggravating your pain.

Regarding comparing two equal floats, strings or lists by `eq', you warn
about its undefinedness ("might not return nil") repeatedly, but can you
give an actual example where (eq 1.2 1.2) or (eq "string" "string")
returns non-nil in Elisp?

Otherwise, isn't it too hypothetical/theoretical to talk about in the
manual? I guess it is imaginable that a compiler might do that, but so
are other things that Elisp could do but doesn't.

>> IOW, "mutable" and "safely
>> mutable"/"mutating it is a good idea" should be distinguished.
>
> The attached patch makes that distinction in a new section "Constants
> and mutability" that I hope clarifies some of the issues here. While
> writing this, I noticed that for Elisp we needn't separate the three
> notions (not-mutable-and-this-is-checked, not-safe-to-mutate,
> safely-mutable) everywhere; all we need to do is describe those
> notions in one spot, and everywhere else we can just use the term
> "mutable" for safely-mutable and the term "constant" for everything
> else.

I think there is no disagreement about the issues involved, but about
the terms used to describe them. I still think that formulations like
"the other arguments (all but the last) should be mutable lists" are
unfortunate, because all Elisp lists are mutable. IOW, "mutable" and
"immutable" seem better suited for data structure classification than
Elisp best practices recommendations. All lists/strings/vectors are
mutable in Elisp. The manual had better describe under what
circumstances one should not mutate them, and I don't think calling the
same data structure once "mutable" and once not will help rather than
confuse or mystify.

> diff --git a/doc/lispref/eval.texi b/doc/lispref/eval.texi
> index 46cfab164b..deb288943a 100644
> --- a/doc/lispref/eval.texi
> +++ b/doc/lispref/eval.texi
> @@ -158,6 +158,12 @@ contents unchanged.
>  @end group
>  @end example
>  
> +  A self-evaluating form yields a constant, and you should not attempt
> +to modify the constant's contents via @code{setcar}, @code{aset} or
> +similar primitives.  The Lisp interpreter might unify the constants
           ^^^^^^^^^^
In the Elisp manual, "primitive" is defined as "A function which is
callable from Lisp but is actually written in C", but I think we want to
say that one shouldn't try to modify constants, period; no matter the
particular means used.

> +yielded by your program's self-evaluating forms, so that these
> +constants might share structure.  @xref{Constants and Mutability}.
> +
> @@ -1306,10 +1306,10 @@ same way in Lisp input.
>  
>    A vector, like a string or a number, is considered a constant for
>  evaluation: the result of evaluating it is the same vector.  This does
> -not evaluate or even examine the elements of the vector.  Vectors
> -written with square brackets are constants and should not be modified
> -via @code{aset} or other destructive operations.
> -@xref{Self-Evaluating Forms}.
> +not evaluate or even examine the elements of the vector.
> +@xref{Self-Evaluating Forms}.  Vectors written with square brackets
> +are constants and should not be modified via @code{aset} or other
> +destructive operations.  @xref{Constants and Mutability}.

This is a good formulation IMO: no "(im)mutable" needed. Such and such
are (considered) constant/literal, and should not be modified/modifying
them leads to undefined consequences.

  Thanks,

  Štěpán

Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Paul Eggert]

On 4/19/20 3:33 PM, Štěpán Němec wrote:

> Regarding comparing two equal floats, strings or lists by `eq', you warn
> about its undefinedness ("might not return nil") repeatedly, but can you
> give an actual example where (eq 1.2 1.2) or (eq "string" "string")
> returns non-nil in Elisp?

   (let ((str "abc"))
     (eq str "abc"))

This yields t when byte-compiled.

> I still think that formulations like
> "the other arguments (all but the last) should be mutable lists" are
> unfortunate, because all Elisp lists are mutable.

Pure lists are not mutable. (Admittedly these are getting rare...)

> All lists/strings/vectors are
> mutable in Elisp.

No, there are exceptions. For example the following makes my Emacs dump core, 
even though all it does is modify a string.

   (aset (symbol-name 'cons) 0 ?x)

It's trying to change the spelling of the name of the 'cons' function, but that 
spelling is in memory that the operating system protects on my platform.

> I don't think calling the
> same data structure once "mutable" and once not will help rather than
> confuse or mystify.

Yes, we should definitely be consistent in whatever terminology we use. And it's 
quite plausible that the terminology in the current (emacs-27) manual can be 
improved. However, I don't know what terminology people would prefer. ("Literal 
object" is one possibility, but it has its own problems.)


>> +  A self-evaluating form yields a constant, and you should not attempt
>> +to modify the constant's contents via @code{setcar}, @code{aset} or
>> +similar primitives.  The Lisp interpreter might unify the constants
>             ^^^^^^^^^^
> In the Elisp manual, "primitive" is defined as "A function which is
> callable from Lisp but is actually written in C", but I think we want to
> say that one shouldn't try to modify constants, period; no matter the
> particular means used.

Thanks, I changed "primitives" to "operations".

Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Michael Heerdegen]

Paul Eggert <eggert@cs.ucla.edu> writes:

> No, there are exceptions. For example the following makes my Emacs
> dump core, even though all it does is modify a string.
>
>   (aset (symbol-name 'cons) 0 ?x)
>
> It's trying to change the spelling of the name of the 'cons' function,
> but that spelling is in memory that the operating system protects on
> my platform.

What you didn't say (directly) is that this means your Emacs will crash
if you eval this.  It would have been...good.

Michael.

Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better, Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better, Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Štěpán Němec]

On Sun, 19 Apr 2020 16:48:46 -0700
Paul Eggert wrote:

> On 4/19/20 3:33 PM, Štěpán Němec wrote:
>
>> Regarding comparing two equal floats, strings or lists by `eq', you warn
>> about its undefinedness ("might not return nil") repeatedly, but can you
>> give an actual example where (eq 1.2 1.2) or (eq "string" "string")
>> returns non-nil in Elisp?
>
>   (let ((str "abc"))
>     (eq str "abc"))
>
> This yields t when byte-compiled.

OK, that's an interesting example, thanks.

I don't think it warrants changing documentation examples like

          (assoc "simple leaves" leaves)
               ⇒ ("simple leaves" . oak)

to

          (assoc (copy-sequence "simple leaves") leaves)
               ⇒ ("simple leaves" . oak)

though, because that's not the point that manual section is making:
what's important here (unlike the modification examples) is not to make
sure the string we're comparing is not the same object as one of the
compared list members. We want to use the appropriate predicate, so that
no matter the object identity, we get the correct result.

So, if you insist that saying that (assq "simple leaves" leaves) returns
nil is no good, how about, rather than giving examples that you would
never use in real code, we change the examples as follows:

          (assq "simple leaves" leaves)
               ⇒ unspecified ; might be nil or non-nil
          (assoc "simple leaves" leaves)
               ⇒ ("simple leaves" . oak)

Similarly for the others.

>> I still think that formulations like "the other arguments (all but
>> the last) should be mutable lists" are unfortunate, because all Elisp
>> lists are mutable.
>
> Pure lists are not mutable. (Admittedly these are getting rare...)
>
>> All lists/strings/vectors are mutable in Elisp.
>
> No, there are exceptions. For example the following makes my Emacs
> dump core, even though all it does is modify a string.
>
>   (aset (symbol-name 'cons) 0 ?x)
>
> It's trying to change the spelling of the name of the 'cons' function,
> but that spelling is in memory that the operating system protects on
> my platform.

This still seems more an issue of terminology: the examples you give IMO
don't illustrate that some strings or lists are mutable and some are
not; it illustrates that mutating some lists or strings has undefined
consequences.

The point I have been trying to make is that, especially now that
immutable (/persistent/functional) data structures are quite widespread,
using the mutable/immutable dichotomy in the way you do for Elisp is
confusing (and I suggest sticking to "undefined consequences" or "safely
mutable"; again: I would expect an attempt to modify an immutable data
structure to produce an error, not dump core or anything else, and I
think that's the general expectation with immutable data structures (and
I still think that saying "all Elisp lists/strings/vectors are mutable"
is valid in that sense).

On Mon, 20 Apr 2020 02:16:03 +0200
Michael Heerdegen wrote:

>> Regarding comparing two equal floats, strings or lists by `eq', you warn
>> about its undefinedness ("might not return nil") repeatedly, but can you
>> give an actual example where (eq 1.2 1.2) or (eq "string" "string")
>> returns non-nil in Elisp?
>
> (eq "" "") => t.  You don't even have to compile.

Yes, but that's a corner case, similar to there only being one empty
list (nil), and I'm not sure it is relevant to the manual sections in
discussion here. I'm not even sure it's relevant (for documentation or
coding style) at all, i.e., you probably wouldn't recommend using (eq
string "") instead of `string-empty-p' or `string='?

>> Otherwise, isn't it too hypothetical/theoretical to talk about in the
>> manual?
>
> It is important.  This once bit me, and it took a long time until I
> found out what was wrong.  I had to ask here (AFAIR that's the reason
> why an explanation was added to the manual).  And it was not code to
> provoke that kind of thing.

Here I assume you mean something else than the empty string case?

-- 
Štěpán

Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better, Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better, Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Michael Heerdegen]

Štěpán Němec <stepnem@gmail.com> writes:

> > (eq "" "") => t.  You don't even have to compile.
>
> Yes, but that's a corner case [...]

Sure.  OTOH it is a good demonstration of the fact that not every
"created" string is a new object.  That only one "" exists is one
optimization.  That the compiler sometimes maps (other) equal strings to
the same object is another one.  Anyway, I didn't intend to demonstrate
something special.  But that "" is unique was not as obvious as the
uniqueness of nil for me.

> > It is important.  This once bit me, and it took a long time until I
> > found out what was wrong.
> Here I assume you mean something else than the empty string case?

Yes, it was something different, I don't recall exactly, I guess I was
using strings as keys in some association structure or so.

Michael.

Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better, Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better, Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 2463 bytes --]

On 4/20/20 3:05 AM, Štěpán Němec wrote:

> how about, rather than giving examples that you would
> never use in real code, we change the examples as follows:
> 
>            (assq "simple leaves" leaves)
>                 ⇒ unspecified ; might be nil or non-nil
>            (assoc "simple leaves" leaves)
>                 ⇒ ("simple leaves" . oak)

Yes, that's fine. I installed the attached patch, which attempts to address 
this, along with other comments by you and Drew.

> Similarly for the others.

What similar examples are there? As far as I can tell, the other 
newly-introduced uses of copy-sequence are needed for mutability not uniqueness, 
so they aren't similar.

> the examples you give IMO
> don't illustrate that some strings or lists are mutable and some are
> not; it illustrates that mutating some lists or strings has undefined
> consequences.

I have a different impression, I think the string that (symbol-name 'lambda) 
returns really is constant. Currently Emacs doesn't behave well if you try to 
change that string (and Emacs should behave better): but it's a feature not a 
bug that you can't mess up the Lisp interpreter in such a fundamental way, and 
if we make improvements in this area they should not change the fact that these 
string values should stay constant.

> The point I have been trying to make is that, especially now that
> immutable (/persistent/functional) data structures are quite widespread,
> using the mutable/immutable dichotomy in the way you do for Elisp is
> confusing

But the Elisp documentation doesn't contrast "mutable" with "immutable"; it 
contrasts "mutable" with "constant". The documentation avoids the word 
"immutable" partly to try to avoid the confusion that you mention.

There are other languages where "constant" or "const" means "behavior is 
undefined if you try to change it", not "an exception is thrown if you try to 
change it"; C and C++ are notable examples. So it's not like the documentation 
is inventing terminology out of whole cloth here.

If one's expectation is that Emacs will throw an exception if one tries to 
modify a constant then that's unfortunate, because Emacs doesn't do that for all 
constants. For now, the manual documents what you can do safely in a Elisp code; 
if we improve Emacs so that it reliably signals an error when you try to modify 
any constant then we can change the manual to say so.

[-- Attachment #2: 0001-Improve-wording-about-constants.patch --]
[-- Type: text/x-patch, Size: 5478 bytes --]

From 400ff5cd195e81204edd9c69fa1b8bc3cb66b42d Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Wed, 22 Apr 2020 10:42:09 -0700
Subject: [PATCH] Improve wording about constants
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Thanks to Štěpán Němec and Drew Adams for reviews of recent changes.
* doc/lispref/eval.texi (Quoting): Give an example.
* doc/lispref/lists.texi (Association Lists): Simplify example code.
* doc/lispref/objects.texi (Lisp Data Types)
(Constants and Mutability): Clarify wording.
---
 doc/lispref/eval.texi    |  6 ++++++
 doc/lispref/lists.texi   |  7 +++----
 doc/lispref/objects.texi | 21 ++++++++++++++-------
 3 files changed, 23 insertions(+), 11 deletions(-)

diff --git a/doc/lispref/eval.texi b/doc/lispref/eval.texi
index 021604c514..baddce4d9c 100644
--- a/doc/lispref/eval.texi
+++ b/doc/lispref/eval.texi
@@ -606,6 +606,12 @@ Here are some examples of expressions that use @code{quote}:
 @end group
 @end example
 
+  Although the expressions @code{(list '+ 1 2)} and @code{'(+ 1 2)}
+both yield lists equal to @code{(+ 1 2)}, the former yields a
+freshly-minted mutable list whereas the latter yields a constant list
+built from conses that may be shared with other constants.
+@xref{Constants and Mutability}.
+
   Other quoting constructs include @code{function} (@pxref{Anonymous
 Functions}), which causes an anonymous lambda expression written in Lisp
 to be compiled, and @samp{`} (@pxref{Backquote}), which is used to quote
diff --git a/doc/lispref/lists.texi b/doc/lispref/lists.texi
index 1125af7bec..ea44e01f48 100644
--- a/doc/lispref/lists.texi
+++ b/doc/lispref/lists.texi
@@ -1625,10 +1625,9 @@ keys may not be symbols:
       '(("simple leaves" . oak)
         ("compound leaves" . horsechestnut)))
 
-;; @r{The @code{copy-sequence} means the keys are not @code{eq}.}
-(assq (copy-sequence "simple leaves") leaves)
-     @result{} nil
-(assoc (copy-sequence "simple leaves") leaves)
+(assq "simple leaves" leaves)
+     @result{} @r{Unspecified; might be @code{nil} or non-@code{nil}.}
+(assoc "simple leaves" leaves)
      @result{} ("simple leaves" . oak)
 @end smallexample
 @end defun
diff --git a/doc/lispref/objects.texi b/doc/lispref/objects.texi
index abd258eb53..1eda94ab63 100644
--- a/doc/lispref/objects.texi
+++ b/doc/lispref/objects.texi
@@ -46,7 +46,7 @@ you store in it, type and all.  (Actually, a small number of Emacs
 Lisp variables can only take on values of a certain type.
 @xref{Variables with Restricted Values}.)
 
-  Some Lisp objects are @dfn{constant}: their values never change.
+  Some Lisp objects are @dfn{constant}: their values should never change.
 Others are @dfn{mutable}: their values can be changed via destructive
 operations that involve side effects.
 
@@ -2384,22 +2384,28 @@ that for two strings to be equal, they have the same text properties.
 @cindex constants
 @cindex mutable objects
 
-  Some Lisp objects are constant: their values never change.
+  Some Lisp objects are constant: their values should never change
+during a single execution of Emacs running well-behaved Lisp code.
 For example, you can create a new integer by calculating one, but you
 cannot modify the value of an existing integer.
 
-  Other Lisp objects are mutable: their values can be changed
+  Other Lisp objects are mutable: it is safe to change their values
 via destructive operations involving side effects.  For example, an
 existing marker can be changed by moving the marker to point to
 somewhere else.
 
-  Although numbers are always constants and markers are always
+  Although all numbers are constants and all markers are
 mutable, some types contain both constant and mutable members.  These
 types include conses, vectors, strings, and symbols.  For example, the string
 literal @code{"aaa"} yields a constant string, whereas the function
 call @code{(make-string 3 ?a)} yields a mutable string that can be
 changed via later calls to @code{aset}.
 
+  A mutable object can become constant if it is passed to the
+@code{eval} function, because a program should not modify an object
+that is being evaluated.  The reverse does not occur: constant objects
+should stay constant.
+
   Trying to modify a constant variable signals an error
 (@pxref{Constant Variables}).
 A program should not attempt to modify other types of constants because the
@@ -2407,9 +2413,10 @@ resulting behavior is undefined: the Lisp interpreter might or might
 not detect the error, and if it does not detect the error the
 interpreter can behave unpredictably thereafter.  Another way to put
 this is that although mutable objects are safe to change and constant
-symbols reliably reject attempts to change them, other constants are
-not safely mutable: if you try to change one your program might
-behave as you expect but it might crash or worse.  This problem occurs
+variables reliably prevent attempts to change them, other constants
+are not safely mutable: if a misbehaving program tries to change such a
+constant then the constant's value might actually change, or the
+program might crash or worse.  This problem occurs
 with types that have both constant and mutable members, and that have
 mutators like @code{setcar} and @code{aset} that are valid on mutable
 objects but hazardous on constants.
-- 
2.17.1

Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better, Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better, Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Štěpán Němec]

On Wed, 22 Apr 2020 10:54:12 -0700
Paul Eggert wrote:

> On 4/20/20 3:05 AM, Štěpán Němec wrote:
>
>> how about, rather than giving examples that you would
>> never use in real code, we change the examples as follows:
>>            (assq "simple leaves" leaves)
>>                 ⇒ unspecified ; might be nil or non-nil
>>            (assoc "simple leaves" leaves)
>>                 ⇒ ("simple leaves" . oak)
>
> Yes, that's fine. I installed the attached patch, which attempts to
> address this, along with other comments by you and Drew.

Thanks.

>> Similarly for the others.
>
> What similar examples are there? As far as I can tell, the other
> newly-introduced uses of copy-sequence are needed for mutability not
> uniqueness, so they aren't similar.

I meant the list examples you changed: you would never actually use

 (memq (list 2) '((1) (2)))

instead of (memq '(2) '((1) (2))), same as you'd never use
`copy-sequence' to make sure you have a fresh comparison key.

Rather, the point of that section is to use `member' instead of `memq',
analogously to `assoc' vs `assq' above.

>> the examples you give IMO don't illustrate that some strings or lists
>> are mutable and some are not; it illustrates that mutating some lists
>> or strings has undefined consequences.
>
> I have a different impression, I think the string that (symbol-name
> 'lambda) returns really is constant. Currently Emacs doesn't behave
> well if you try to change that string (and Emacs should behave
> better): but it's a feature not a bug that you can't mess up the Lisp
> interpreter in such a fundamental way, and if we make improvements in
> this area they should not change the fact that these string values
> should stay constant.

I still think we are miscommunicating due to the different usage or
notion of "mutable". All I'm saying is that if you introduce wording
such as "the other arguments (all but the last) should be mutable
lists", that (to me, and I'm sure many other readers) implies that you
are talking about (types of) data structures, i.e. that there is such a
thing as mutable (and also immutable, because why would you use that
term otherwise?) lists/conses/strings as opposed to something else, as
there indeed is in some languages, including Scheme (e.g. SRFI 140) or
Racket.

I'm not saying that the difference between the values of (symbol-name
'lambda), "lambda" and (copy-sequence "lambda") or (list 1) vs '(1)
aren't real, important or worth clarifying, only that unqualified
"mutable" is an unfortunate term to use (and AFAICT you introduce this
usage to the documentation; it was pretty much nonexistent before your
recent changes).

-- 
Štěpán

Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 1770 bytes --]

On 5/2/20 5:50 AM, Štěpán Němec wrote:

> I meant the list examples you changed: you would never actually use
> 
>  (memq (list 2) '((1) (2)))
> 
> instead of (memq '(2) '((1) (2))), same as you'd never use
> `copy-sequence' to make sure you have a fresh comparison key.
> 
> Rather, the point of that section is to use `member' instead of `memq',
> analogously to `assoc' vs `assq' above.

I see your point. That's easy enough to fix; I installed the attached patch.

> if you introduce wording
> such as "the other arguments (all but the last) should be mutable
> lists", that (to me, and I'm sure many other readers) implies that you
> are talking about (types of) data structures, i.e. that there is such a
> thing as mutable (and also immutable, because why would you use that
> term otherwise?) lists/conses/strings as opposed to something else, as
> there indeed is in some languages, including Scheme (e.g. SRFI 140) or
> Racket.

That's OK. It's not a misimpression to think of things that way. True, Scheme
SRFI 140 has a lot more detail, with functions like istring? that Elisp lacks,
but the core idea is close enough and it would be nice if Elisp eventually grew
in the same direction as Scheme in this area (though obviously this would be a
much bigger project than merely fiddling with the documentation).

I just checked, and Scheme has gone in the Elisp direction recently in this
area: although R6RS took the strong position that safety is essential, R7RS says
it's not - e.g., R7RS like R5RS says that (car 0) can cause the interpreter to
crash (though of course interpreters are suggested to behave more nicely than
that). This change to the Scheme spec was for performance reasons that Elisp is
not immune to.

[-- Attachment #2: 0001-Make-memq-etc.-examples-more-like-they-were.patch --]
[-- Type: text/x-patch, Size: 2978 bytes --]

From 0a3731feef351f6af47bed1458aefb6cb481b5f9 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Sat, 2 May 2020 13:48:21 -0700
Subject: [PATCH] Make memq etc. examples more like they were
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Problem reported by Štěpán Němec in:
https://lists.gnu.org/r/emacs-devel/2020-05/msg00130.html
* doc/lispref/lists.texi (Sets And Lists, Association Lists):
Revert examples to be more like the way they were, using
self-evaluating expressions.  Be more consistent about listing
unspecified results.
---
 doc/lispref/lists.texi | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/doc/lispref/lists.texi b/doc/lispref/lists.texi
index ea44e01f48..fcaf4386b1 100644
--- a/doc/lispref/lists.texi
+++ b/doc/lispref/lists.texi
@@ -1242,8 +1242,8 @@ compare @var{object} against the elements of the list.  For example:
      @result{} (b c b a)
 @end group
 @group
-(memq '(2) '((1) (2)))    ; @r{@code{(2)} and @code{(2)} are not @code{eq}.}
-     @result{} nil
+(memq '(2) '((1) (2)))    ; @r{The two @code{(2)}s need not be @code{eq}.}
+     @result{} @r{Unspecified; might be @code{nil} or @code{((2))}.}
 @end group
 @end example
 @end defun
@@ -1356,12 +1356,12 @@ Compare this with @code{memq}:
 
 @example
 @group
-(memql 1.2 '(1.1 1.2 1.3))  ; @r{@code{1.2} and @code{1.2} must be @code{eql}.}
+(memql 1.2 '(1.1 1.2 1.3))  ; @r{@code{1.2} and @code{1.2} are @code{eql}.}
      @result{} (1.2 1.3)
 @end group
 @group
-(memq 1.2 '(1.1 1.2 1.3))  ; @r{@code{1.2} and @code{1.2} need not be @code{eq}.}
-     @result{} nil         ; @r{... or it might be @code{(1.2 1.3)}.}
+(memq 1.2 '(1.1 1.2 1.3))  ; @r{The two @code{1.2}s need not be @code{eq}.}
+     @result{} @r{Unspecified; might be @code{nil} or @code{(1.2 1.3)}.}
 @end group
 @end example
 @end defun
@@ -1380,12 +1380,12 @@ Compare this with @code{memq}:
 
 @example
 @group
-(member (list 2) '((1) (2)))  ; @r{@code{(list 2)} and @code{(2)} are @code{equal}.}
+(member '(2) '((1) (2)))  ; @r{@code{(2)} and @code{(2)} are @code{equal}.}
      @result{} ((2))
 @end group
 @group
-(memq (list 2) '((1) (2)))    ; @r{@code{(list 2)} and @code{(2)} are not @code{eq}.}
-     @result{} nil
+(memq '(2) '((1) (2)))    ; @r{The two @code{(2)}s need not be @code{eq}.}
+     @result{} @r{Unspecified; might be @code{nil} or @code{(2)}.}
 @end group
 @group
 ;; @r{Two strings with the same contents are @code{equal}.}
@@ -1626,7 +1626,7 @@ keys may not be symbols:
         ("compound leaves" . horsechestnut)))
 
 (assq "simple leaves" leaves)
-     @result{} @r{Unspecified; might be @code{nil} or non-@code{nil}.}
+     @result{} @r{Unspecified; might be @code{nil} or @code{("simple leaves" . oak)}.}
 (assoc "simple leaves" leaves)
      @result{} ("simple leaves" . oak)
 @end smallexample
-- 
2.17.1

Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Michael Heerdegen]

Štěpán Němec <stepnem@gmail.com> writes:

> Regarding comparing two equal floats, strings or lists by `eq', you warn
> about its undefinedness ("might not return nil") repeatedly, but can you
> give an actual example where (eq 1.2 1.2) or (eq "string" "string")
> returns non-nil in Elisp?

(eq "" "") => t.  You don't even have to compile.

> Otherwise, isn't it too hypothetical/theoretical to talk about in the
> manual?

It is important.  This once bit me, and it took a long time until I
found out what was wrong.  I had to ask here (AFAIR that's the reason
why an explanation was added to the manual).  And it was not code to
provoke that kind of thing.


Michael.

Re: emacs-27 eebfb72 1/2: Document constant vs mutable objects better

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Regarding comparing two equal floats, strings or lists by `eq', you warn
  > > about its undefinedness ("might not return nil") repeatedly, but can you
  > > give an actual example where (eq 1.2 1.2) or (eq "string" "string")
  > > returns non-nil in Elisp?

  > (eq "" "") => t.  You don't even have to compile.

I just evaluated (eq "string" "string") and got nil.

I think (eq "" "") is an exception.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)