bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Drew Adams]

Please consider adding a TEST comparer arg for plist keys.

In Elisp, a plist key need not be a symbol:

(plist-put (list "aaa" 1 "bbb" 2 "ccc" 3) "bbb" 42)

That "works" (and no error), but it doesn't do what's expected, since
the keys should be compared with `equal' or `string=', not `eq'.

And if it were not intended that that work then Elisp would raise a
wrong-type-argument error saying that the first arg isn't a plist.

And note that the Elisp manual says:

  A "property list" ("plist" for short) is a list of
  paired elements.  Each of the pairs associates a
  property name (usually a symbol) with a property or value.
                 ^^^^^^^

In GNU Emacs 26.3 (build 1, x86_64-w64-mingw32)
 of 2019-08-29
Repository revision: 96dd0196c28bc36779584e47fffcca433c9309cd
Windowing system distributor `Microsoft Corp.', version 10.0.19041
Configured using:
 `configure --without-dbus --host=x86_64-w64-mingw32
 --without-compress-install 'CFLAGS=-O2 -static -g3''

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Drew Adams]

That would also mean we wouldn't need `lax-plist-*' functions.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Jean Louis]

* Drew Adams <drew.adams@oracle.com> [2021-03-27 01:15]:
> Please consider adding a TEST comparer arg for plist keys.
> 
> In Elisp, a plist key need not be a symbol:
> 
> (plist-put (list "aaa" 1 "bbb" 2 "ccc" 3) "bbb" 42)
> 
> That "works" (and no error), but it doesn't do what's expected, since
> the keys should be compared with `equal' or `string=', not `eq'.
> 
> And if it were not intended that that work then Elisp would raise a
> wrong-type-argument error saying that the first arg isn't a plist.
> 
> And note that the Elisp manual says:
> 
>   A "property list" ("plist" for short) is a list of
>   paired elements.  Each of the pairs associates a
>   property name (usually a symbol) with a property or value.
>                  ^^^^^^^

That is right, I always had that problem with plist, and still
do. Amount of the coping code on my side would become redundant if
that would be corrected to have a test function.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Lars Ingebrigtsen]

Drew Adams <drew.adams@oracle.com> writes:

> Please consider adding a TEST comparer arg for plist keys.
>
> In Elisp, a plist key need not be a symbol:
>
> (plist-put (list "aaa" 1 "bbb" 2 "ccc" 3) "bbb" 42)
>
> That "works" (and no error), but it doesn't do what's expected, since
> the keys should be compared with `equal' or `string=', not `eq'.

plist-put doesn't ensure that the operation makes sense here, no, but we
can't really add that at this point, either.

I think adding a comparison function makes sense, but on the other
hand -- we seem to be moving towards using map.el more for these things
now, so I'm not sure there's much enthusiasm for that.  On the other
hand, the generic map functions have the problem that they...  can't
really be used like plist-put.  Sure

(map-put! (list 'a 1) 'b 2)

works fine, but you can't create a plist that way, which makes these
functions barely usable at all for handling plists/alists:

(map-put! nil 'b 2)

will signal an error.

So does anybody have an opinion here?  I think I'm in favour of adding a
comparison function for all three `plist-*' functions.

Drew Adams <drew.adams@oracle.com> writes:

> That would also mean we wouldn't need `lax-plist-*' functions.

Yes, those are horrible functions, and are barely used anywhere.  (And
`lax-plist-member' is missing.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47425: [External] : Re: bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Drew Adams]

> > Please consider adding a TEST comparer arg for plist keys.
> >
> > In Elisp, a plist key need not be a symbol:
> >
> > (plist-put (list "aaa" 1 "bbb" 2 "ccc" 3) "bbb" 42)
> >
> > That "works" (and no error), but it doesn't do what's expected, since
> > the keys should be compared with `equal' or `string=', not `eq'.
> 
> plist-put doesn't ensure that the operation makes sense here, no, but we
> can't really add that at this point, either.

Please elaborate.  I don't know what you're saying,
or why.

> I think adding a comparison function makes sense, but on the other
> hand -- we seem to be moving towards using map.el more for these things
> now, so I'm not sure there's much enthusiasm for that.  On the other
> hand, the generic map functions have the problem that they...  can't
> really be used like plist-put.

No.  Please do _not_ bring generic mapping into this.
This is a legitimate issue about plists and plist
functions.

> So does anybody have an opinion here?  I think I'm in favour of adding a
> comparison function for all three `plist-*' functions.

+1. 

> > That would also mean we wouldn't need `lax-plist-*' functions.
> 
> Yes, those are horrible functions, and are barely used anywhere.  (And
> `lax-plist-member' is missing.)

Agreed.  But their existence is an argument that the
intention was to provide for the use of `equal' as
an alternative test pred to `eq'.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Philipp]

> So does anybody have an opinion here?  I think I'm in favour of adding a
> comparison function for all three `plist-*' functions.

I'm not a fan of this since it would mean that these functions will no longer be side-effect-free.
Adding a function argument to a function means that the function can no longer really promise anything, so it leads to code that's harder to reason about.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Lars Ingebrigtsen]

Philipp <p.stephani2@gmail.com> writes:

> I'm not a fan of this since it would mean that these functions will no
> longer be side-effect-free.  Adding a function argument to a function
> means that the function can no longer really promise anything, so it
> leads to code that's harder to reason about.

`plist-put' already mutates state (of the plist), so it's not
side-effect-free...

The other ones are, though.  But...  I'm not sure that's really much of
a concern in practice.  (And Emacs should be able to reason about the
function, still, if the predicate passed in is side effect free, but I
guess we don't have the machinery for that in place?)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> I'm not a fan of this since it would mean that these functions will no
>> longer be side-effect-free.  Adding a function argument to a function
>> means that the function can no longer really promise anything, so it
>> leads to code that's harder to reason about.
>
> `plist-put' already mutates state (of the plist), so it's not
> side-effect-free...
>
> The other ones are, though.  But...  I'm not sure that's really much of
> a concern in practice.  (And Emacs should be able to reason about the
> function, still, if the predicate passed in is side effect free, but I
> guess we don't have the machinery for that in place?)

We don't.

Anyway, that was the only objection here, so I went ahead and made this
change in Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Mattias Engdegård]

> I'm not a fan of this since it would mean that these functions will no longer be side-effect-free. Adding a function argument to a function means that the function can no longer really promise anything, so it leads to code that's harder to reason about. 

No fan of it either. A plist is not intended as a general-purpose data structure; the `eq` key comparison is pretty much built in.

The cost of adding an equality predicate is not huge but also not really motivated.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Lars Ingebrigtsen]

Mattias Engdegård <mattiase@acm.org> writes:

> The cost of adding an equality predicate is not huge but also not
> really motivated.

The motivation is greater symmetry with assoc and friends.  That is,
people have been forced to use alists where they'd more naturally prefer
to use plists because of the eq restriction.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Lars Ingebrigtsen [2022-06-27 13:43:23] wrote:
> Mattias Engdegård <mattiase@acm.org> writes:
>> The cost of adding an equality predicate is not huge but also not
>> really motivated.
> The motivation is greater symmetry with assoc and friends.  That is,
> people have been forced to use alists where they'd more naturally prefer
> to use plists because of the eq restriction.

Then again, plists should never have existed, in my book.
They're just strictly worse than alists as datastructures.
Their only advantage is that sometimes when you write them by hand it
they're somewhat more compact (fewer dots and parentheses) but for those
cases `eq` is always good enough in my experience.

I'd be curious to hear of a use case where plists are better than alists
while at the same time requiring a non-eq comparison.


        Stefan

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Mattias Engdegård]

27 juni 2022 kl. 14.18 skrev Stefan Monnier <monnier@iro.umontreal.ca>:

> Then again, plists should never have existed, in my book.
> They're just strictly worse than alists as datastructures.

Indeed. For example, key-value pairs cannot be shared between different plists, and getting the nth key takes twice as long.
There should be no reason to promote or expand the use of plists.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Lars Ingebrigtsen]

Mattias Engdegård <mattiase@acm.org> writes:

> Indeed. For example, key-value pairs cannot be shared between
> different plists, and getting the nth key takes twice as long.
> There should be no reason to promote or expand the use of plists.

If you say "I want the nth", then, yes, it takes longer.  But nobody
does that with a plist or an alist -- they say "give me the value of
key", and that's an identical number of operations on a plist as on an
alist.

(Plist is car for eq, cdr + cdr for traversal, while alist is car + car
for eq, cdr for traversal.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Mattias Engdegård]

27 juni 2022 kl. 14.44 skrev Lars Ingebrigtsen <larsi@gnus.org>:

> If you say "I want the nth", then, yes, it takes longer.  But nobody
> does that with a plist or an alist

'Nobody' is too strong (done it a few times) but yes, it's not a common operation.

But the point is that alists has several more advantages: counting number of keys is twice as fast, iterating and/or transform the collection easily done with mapcar, mapcan, dolist, mapc etc. Plists have no real advantage from their structure.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Lars Ingebrigtsen]

Mattias Engdegård <mattiase@acm.org> writes:

> But the point is that alists has several more advantages: counting
> number of keys is twice as fast,

Nobody does that either.  😀

> iterating and/or transform the collection easily done with mapcar,
> mapcan, dolist, mapc etc. Plists have no real advantage from their
> structure.

Yes, it's true that we have a lot more infrastructure that's geared
towards simple lists, and you can use those more easily with alists.
But mapcar etc aren't idea for alists, either.  We do have `on' in
`cl-loop'.  Adding some syntax to pcase for plists might be nice.
(I.e., for pcase-dolist etc.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> `cl-loop'.  Adding some syntax to pcase for plists might be nice.

Not!


        Stefan

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> `cl-loop'.  Adding some syntax to pcase for plists might be nice.
>
> Not!

🥳

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Drew Adams]

> But the point is that alists has several more advantages: counting
> number of keys is twice as fast, iterating and/or transform the
> collection easily done with mapcar, mapcan, dolist, mapc etc. Plists
> have no real advantage from their structure.

Alists have a zillion advantages over plists.

That's not what this enhancement request is about.

A similar situation exists between vectors and
lists.  And even among different numeric types.
And even between Lisp symbols and strings.

It's the case that sometimes users use a type
that's less useful than another type in some
context.  That's irrelevant to whether plist
functions could benefit from a TEST arg.  IMO.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Drew Adams]

> > Then again, plists should never have existed, in my book.
> > They're just strictly worse than alists as datastructures.
> 
> Indeed. For example, key-value pairs cannot be shared between 
> different plists, and getting the nth key takes twice as long.
> There should be no reason to promote or expand the use of plists.

I happen to agree.  Adding a TEST arg for plist
functions isn't, IMO, promoting or expanding
their use.  It's just recognizing existing use.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I'd be curious to hear of a use case where plists are better than alists
> while at the same time requiring a non-eq comparison.

They're equivalent in features, yes, but people prefer plists for
simple structures because they look nicer.  I'd say they've proven their
popularity over half a century.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Drew Adams]

> > I'd be curious to hear of a use case where plists are better than
> > alists while at the same time requiring a non-eq comparison.
> 
> They're equivalent in features, yes, but people prefer plists for
> simple structures because they look nicer.  I'd say they've proven
> their popularity over half a century.

Yes, it's about recognizing that reality.

Unless you're going to somehow _prevent_ the
use of plists with keys that are other than,
e.g., symbols - in particular keys that are
strings, adding a TEST arg makes sense, IMO.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Drew Adams]

> Then again, plists should never have existed, in my book.
> They're just strictly worse than alists as datastructures.
> Their only advantage is that sometimes when you write them by hand it
> they're somewhat more compact (fewer dots and parentheses) but for
> those cases `eq` is always good enough in my experience.
>
> I'd be curious to hear of a use case where plists are better than
> alists while at the same time requiring a non-eq comparison.

Wow.  Stefan and I actually agree on something. ;-)

(Well, I wouldn't go as far as saying that they
never should have existed.  But as far as listie
things go, they're definitely nowhere near as
useful as alists.)

But whether plists should benefit from TEST
functions other than `eq' is unrelated to whether
alists might be generally - or even always "better"
than alists.

The fact is that some users use plists, even when
they could, and maybe even should, use alists.  And
this is likely to continue, if not increase, due to
experience with other languages and other key-value
representations (JSON etc.).

(Yes, IMO the manuals could tout the advantages of
alists more, or compare & contrast plists and alists
more.  But that's not a reason not to give plist
functions an optional TEST arg.)

Plists with, e.g., string keys aren't uncommon, and
that's likely to continue.  That's kinda the point,
here.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Plists with, e.g., string keys aren't uncommon, and

Really?  Can you give examples?


        Stefan

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Drew Adams]

> > Plists with, e.g., string keys aren't uncommon
>
> Really?  Can you give examples?

Maybe I could, but I won't try.  I take it that
you don't think so.  That's fine.

My expectation is that more users nowadays may
be inclined to use string keys than in the past,
because (as I said) of the influence of things
like JSON.  This enhancement request is to cater
to what I expect will not be all that uncommon.

And there may be more and more use of conversion
of alists (which do often have nonsymbol keys)
to plists.  For whatever reasons (good & bad).

Note too, BTW, that (not uncommon) != common.
___

CLTL2, "The Property List" says, in passing,
that `symbol-plist' keys are "typically" symbols.
It doesn't say that they _are_ symbols.

And it doesn't say anywhere that they need to
be distinguishable with just `eq'.

In any case, I'll grant you that using symbol
keys _is_ common.  (The same is true of alist
keys, BTW.)
___

BTW, CLTL2 also points to this as the essential
difference between a plist and an alist:

  a property list is an object with a unique
  identity; the operations for adding and
  removing property-list entries are destructive
  operations that alter the property list rather
  than making a new one. Association lists, on
  the other hand, are normally augmented
  non-destructively (without side effects) by
  adding new entries to the front (see acons and
  pairlis).

No mention of plist keys being symbols.

However, it's also true that CL defines `remprop'
as using `eq'.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Mattias Engdegård]

> And it doesn't say anywhere that they need to
> be distinguishable with just `eq'.

Operations on property lists in Common Lisp use `eq`. There is no option to use another equality predicate.

This change is just needless gold-plating. Let's revert.

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Lars Ingebrigtsen]

Mattias Engdegård <mattiase@acm.org> writes:

> This change is just needless gold-plating. Let's revert.

Nope.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Basil L. Contovounesios via Bug reports for GNU Emacs, the Swiss army knife of text editors]

(let ((plist (list :a t)))
  (setf (plist-get plist :a) nil)
  plist)
;; => (:a nil)

(let ((plist (list :a t)))
  (setf (plist-get plist :a #'eq) nil)
  plist)
;; error-> wrong-number-of-arguments

Do we want the expander in gv.el to understand the new argument?

There's also the question of map.el.  Custom predicates are deprecated
there, since they rely on knowing the underlying map type.  Do we want
plist operations to understand custom predicates regardless?

Thanks,

-- 
Basil

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Lars Ingebrigtsen]

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> (let ((plist (list :a t)))
>   (setf (plist-get plist :a) nil)
>   plist)
> ;; => (:a nil)
>
> (let ((plist (list :a t)))
>   (setf (plist-get plist :a #'eq) nil)
>   plist)
> ;; error-> wrong-number-of-arguments
>
> Do we want the expander in gv.el to understand the new argument?

Yup.

> There's also the question of map.el.  Custom predicates are deprecated
> there, since they rely on knowing the underlying map type.  Do we want
> plist operations to understand custom predicates regardless?

Using the map functions for plists doesn't really work, so I don't think
it matters much.  That is, if you end up with a nil MAP after doing some
deletes, you suddenly can't put anything into the map.

I think expanding map.el to try to handle plists was a mistake -- it's a
fun hack, but can't be used meaningfully in practice.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#47425: 26.3; `plist-get', `plist-put' and proposed TEST function

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

CL-style keyword arguments to modify list operations are clunky.
Unless we implement them in an extra-complicated way, they will slow
down the usual case, too.  I understand the motivation for proposing
this change, but let's solve it in a cleaner way that doesn't invite
copying the bad designs of CL.

Here are two ways that occur to me.

* The applications that want to compare property names with `equal'
could canonicalize those names.  For instance, intern the strings
you want to use as property names, perhaps in a special obarray.
Then you can use `plist-get' and `plist-put' on them and get the
results you want.

* Define functions `plist-get-equal' and `plist-put-equal'.
Instead of complicating the definitions of widely used functions `plist-get'
and `plist-put', it will define two new functions that most users won't
need to pay attention to.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#47425: 26.3; `plist-get', `plist-put' and proposed TEST function

[Drew Adams]

> CL-style keyword arguments to modify list operations are clunky.
> Unless we implement them in an extra-complicated way, they will slow
> down the usual case, too.  I understand the motivation for proposing
> this change, but let's solve it in a cleaner way that doesn't invite
> copying the bad designs of CL.

I don't think this bug thread has anything to
do with keyword arguments or even with keywords
(e.g. :foobar) in any other way.  Keywords are
anyway symbols, so they compare fine with `eq'.

The suggestion was to add a TEST optional arg,
not a keyword :test arg.  IOW, follow what we
usually do in Emacs Lisp.

The motivation here is to support plists with
keys, such as strings, that can't be usefully
compared with `eq'.

> Here are two ways that occur to me.
> 
> * The applications that want to compare property names with `equal'
> could canonicalize those names.  For instance, intern the strings
> you want to use as property names, perhaps in a special obarray.
> Then you can use `plist-get' and `plist-put' on them and get the
> results you want.

Someone writing code from scratch can always
use symbols as keys (plist properties).  That's
not the problem.

The idea is to support existing plists that
have string keys.  (And some users will end up
with such plists, one way or another.)

> * Define functions `plist-get-equal' and `plist-put-equal'.
> Instead of complicating the definitions of widely used functions
> `plist-get' and `plist-put', it will define two new functions
> that most users won't need to pay attention to.

That solution is OK also.  That won't help
someone who comes across plist-get and -put
and mistakenly tries to use them with string
keys, but that's OK.  (After all, we have memq
and member...)

bug#47425: 26.3; `plist-get', `plist-put' and proposed TEST function

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The suggestion was to add a TEST optional arg,
  > not a keyword :test arg.  IOW, follow what we
  > usually do in Emacs Lisp.

Sorry.  I thought I had seen a proposal for a keyword arg.

However, I still think that the other methods I proposed are cleaner
overall.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

bug#47425: 26.3; `plist-get', `plist-put' should accept a TEST function

[Drew Adams]

> > And it doesn't say anywhere that they need to
> > be distinguishable with just `eq'.
> 
> Operations on property lists in Common Lisp use `eq`.
> There is no option to use another equality predicate.

Yes, I mentioned that fact.

CL was of course defined 40 years ago.  And
its users, especially back then, didn't come
from backgrounds that commonly included using
key-value with string keys (such as JSON).

And CL didn't change pre-existing functions
(such as `remprop'), to give them :test args,
as much as it added such args to new functions.
CL sequence functions have :test, for example.
(And a plist is a sequence.).

Also, unlike Elisp, CL doesn't document any
use of plists other than `symbol-plist' - it
specifically didn't adopt the use of
"disembodied" plists, for example.
___

I don't think that providing a TEST arg will
needlessly encourage the use of non-symbol
keys.  I don't see how it will hurt, at all.

But if someone thinks it's very important to
discourage the use of non-symbol keys, then
that can be added to the doc string.  That's
not equivalent to not allowing a TEST arg.