unofficial mirror of bug-guile@gnu.org 
 help / color / mirror / Atom feed
* bug#41354: equal? has no sensible code path for symbols
@ 2020-05-17 10:49 David Kastrup
  2020-05-27 20:39 ` Ludovic Courtès
  0 siblings, 1 reply; 7+ messages in thread
From: David Kastrup @ 2020-05-17 10:49 UTC (permalink / raw)
  To: 41354


In Scheme, symbols can be compared using eq? for equality.  However,
since they have garbage-collected content attached, they do not meet the
predicate SCM_IMP in the short-circuit evaluation at the start of equal?
This means that unequal symbols compared using equal? fall through a
whole bunch of tests and end up in a general structural comparison
comparing their underlying string names.

This completely sabotages the semantics symbols are intended for.
Behavior for eqv? is similar but the fall-through at least is not as
expensive as it is for equal? .

-- 
David Kastrup





^ permalink raw reply	[flat|nested] 7+ messages in thread

* bug#41354: equal? has no sensible code path for symbols
  2020-05-17 10:49 bug#41354: equal? has no sensible code path for symbols David Kastrup
@ 2020-05-27 20:39 ` Ludovic Courtès
  2020-05-27 20:49   ` David Kastrup
  0 siblings, 1 reply; 7+ messages in thread
From: Ludovic Courtès @ 2020-05-27 20:39 UTC (permalink / raw)
  To: David Kastrup; +Cc: 41354

Hi David,

David Kastrup <dak@gnu.org> skribis:

> In Scheme, symbols can be compared using eq? for equality.  However,
> since they have garbage-collected content attached, they do not meet the
> predicate SCM_IMP in the short-circuit evaluation at the start of equal?
> This means that unequal symbols compared using equal? fall through a
> whole bunch of tests and end up in a general structural comparison
> comparing their underlying string names.

‘equal?’ starts by checking for eq-ness, which LGTM:

  SCM
  scm_equal_p (SCM x, SCM y)
  #define FUNC_NAME s_scm_i_equal_p
  {
    SCM_CHECK_STACK;
   tailrecurse:
    SCM_TICK;
    if (scm_is_eq (x, y))
      return SCM_BOOL_T;

Or were you referring to something else?

Thanks,
Ludo’.





^ permalink raw reply	[flat|nested] 7+ messages in thread

* bug#41354: equal? has no sensible code path for symbols
  2020-05-27 20:39 ` Ludovic Courtès
@ 2020-05-27 20:49   ` David Kastrup
  2020-05-28 16:06     ` Ludovic Courtès
  0 siblings, 1 reply; 7+ messages in thread
From: David Kastrup @ 2020-05-27 20:49 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 41354

Ludovic Courtès <ludo@gnu.org> writes:

> Hi David,
>
> David Kastrup <dak@gnu.org> skribis:
>
>> In Scheme, symbols can be compared using eq? for equality.  However,
>> since they have garbage-collected content attached, they do not meet the
>> predicate SCM_IMP in the short-circuit evaluation at the start of equal?
>> This means that unequal symbols compared using equal? fall through a
>> whole bunch of tests and end up in a general structural comparison
>> comparing their underlying string names.
>
> ‘equal?’ starts by checking for eq-ness, which LGTM:
>
>   SCM
>   scm_equal_p (SCM x, SCM y)
>   #define FUNC_NAME s_scm_i_equal_p
>   {
>     SCM_CHECK_STACK;
>    tailrecurse:
>     SCM_TICK;
>     if (scm_is_eq (x, y))
>       return SCM_BOOL_T;
>
> Or were you referring to something else?

I repeat: "This means that UNEQUAL symbols compared using equal? fall
through a whole bunch of tests and end up in a general structural
comparison comparing their underlying string names".

Lots of searches _end_ with an equal comparison (which is fast) but do a
lot of unequal comparisons before that (which is slow, even though
symbols that are not eq? will also not be equal?, so if you know you are
checking _symbols_, if they are not eq? you are done).

Symbols comparing as _unequal_ have no special path in equal?.

-- 
David Kastrup





^ permalink raw reply	[flat|nested] 7+ messages in thread

* bug#41354: equal? has no sensible code path for symbols
  2020-05-27 20:49   ` David Kastrup
@ 2020-05-28 16:06     ` Ludovic Courtès
  2020-05-28 16:50       ` David Kastrup
  0 siblings, 1 reply; 7+ messages in thread
From: Ludovic Courtès @ 2020-05-28 16:06 UTC (permalink / raw)
  To: David Kastrup; +Cc: 41354

[-- Attachment #1: Type: text/plain, Size: 2411 bytes --]

Hi,

David Kastrup <dak@gnu.org> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hi David,
>>
>> David Kastrup <dak@gnu.org> skribis:
>>
>>> In Scheme, symbols can be compared using eq? for equality.  However,
>>> since they have garbage-collected content attached, they do not meet the
>>> predicate SCM_IMP in the short-circuit evaluation at the start of equal?
>>> This means that unequal symbols compared using equal? fall through a
>>> whole bunch of tests and end up in a general structural comparison
>>> comparing their underlying string names.
>>
>> ‘equal?’ starts by checking for eq-ness, which LGTM:
>>
>>   SCM
>>   scm_equal_p (SCM x, SCM y)
>>   #define FUNC_NAME s_scm_i_equal_p
>>   {
>>     SCM_CHECK_STACK;
>>    tailrecurse:
>>     SCM_TICK;
>>     if (scm_is_eq (x, y))
>>       return SCM_BOOL_T;
>>
>> Or were you referring to something else?
>
> I repeat: "This means that UNEQUAL symbols compared using equal? fall
> through a whole bunch of tests and end up in a general structural
> comparison comparing their underlying string names".
>
> Lots of searches _end_ with an equal comparison (which is fast) but do a
> lot of unequal comparisons before that (which is slow, even though
> symbols that are not eq? will also not be equal?, so if you know you are
> checking _symbols_, if they are not eq? you are done).
>
> Symbols comparing as _unequal_ have no special path in equal?.

I was going to say that this is necessary for uninterned symbols, but it
turns out that uninterned symbols that look the same are not ‘equal?’:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (define a (make-symbol "x"))
scheme@(guile-user)> (define b (make-symbol "x"))
scheme@(guile-user)> (eq? a b)
$10 = #f
scheme@(guile-user)> (equal? a b)
$11 = #f
--8<---------------cut here---------------end--------------->8---

Thus we could go with the patch below, though I doubt it would make a
measurable difference (and it actually adds tests for other cases).

Thoughts?

Besides, in the common case where one is comparing against a symbol
literal, the question is moot:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,optimize (equal? 'x s)
$14 = (eq? 'x s)
--8<---------------cut here---------------end--------------->8---

Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 419 bytes --]

diff --git a/libguile/eq.c b/libguile/eq.c
index 627d6f09b..16c5bfb3f 100644
--- a/libguile/eq.c
+++ b/libguile/eq.c
@@ -303,6 +303,8 @@ scm_equal_p (SCM x, SCM y)
     return SCM_BOOL_F;
   if (SCM_IMP (y))
     return SCM_BOOL_F;
+  if (scm_is_symbol (x) || scm_is_symbol (y))
+    return SCM_BOOL_F;
   if (scm_is_pair (x) && scm_is_pair (y))
     {
       if (scm_is_false (scm_equal_p (SCM_CAR (x), SCM_CAR (y))))

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* bug#41354: equal? has no sensible code path for symbols
  2020-05-28 16:06     ` Ludovic Courtès
@ 2020-05-28 16:50       ` David Kastrup
  2020-05-29  8:05         ` Ludovic Courtès
  0 siblings, 1 reply; 7+ messages in thread
From: David Kastrup @ 2020-05-28 16:50 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 41354

Ludovic Courtès <ludo@gnu.org> writes:

> David Kastrup <dak@gnu.org> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> Hi David,
>>>
>>> David Kastrup <dak@gnu.org> skribis:
>>
>> Symbols comparing as _unequal_ have no special path in equal?.
>
> I was going to say that this is necessary for uninterned symbols, but it
> turns out that uninterned symbols that look the same are not ‘equal?’:
>
> scheme@(guile-user)> (define a (make-symbol "x"))
> scheme@(guile-user)> (define b (make-symbol "x"))
> scheme@(guile-user)> (eq? a b)
> $10 = #f
> scheme@(guile-user)> (equal? a b)
> $11 = #f

And it would be pretty horrible if they were, in my book.

> Thus we could go with the patch below, though I doubt it would make a
> measurable difference (and it actually adds tests for other cases).

It made a considerable measurable difference in LilyPond where it slowed
down the operation of assoc when used for symbol lookup (while assoc has
a short-circuit path to assq for SCM_IMP (key) that happens to have the
same problem of not being effective for symbols).  It took some
debugging to figure out why so much time was spent in equal? .

> Thoughts?
>
> Besides, in the common case where one is comparing against a symbol
> literal, the question is moot:
>
> scheme@(guile-user)> ,optimize (equal? 'x s)
> $14 = (eq? 'x s)

That is really quite irrelevant since the problem becomes visible when a
large number of comparisons in a row is done and if you were only
looking for a single constant key among a large set, you'd hardly have a
single constant key your code path would be looking for among that large
set.

> Ludo’.
>
> diff --git a/libguile/eq.c b/libguile/eq.c
> index 627d6f09b..16c5bfb3f 100644
> --- a/libguile/eq.c
> +++ b/libguile/eq.c
> @@ -303,6 +303,8 @@ scm_equal_p (SCM x, SCM y)
>      return SCM_BOOL_F;
>    if (SCM_IMP (y))
>      return SCM_BOOL_F;
> +  if (scm_is_symbol (x) || scm_is_symbol (y))
> +    return SCM_BOOL_F;
>    if (scm_is_pair (x) && scm_is_pair (y))
>      {
>        if (scm_is_false (scm_equal_p (SCM_CAR (x), SCM_CAR (y))))
>

Yes, that looks reasonable.  scm_is_symbol checks some tag subset that
the code for equal_p later looks at closer as well: if you worry about
the extra cost of the scm_is_symbol check, one could try folding the
symbol check into that later code passage, which would slow down the
symbol check and effect the more costly fallbacks less.  But since those
fallbacks _are_ more costly, I doubt it would be worth the trouble.

-- 
David Kastrup





^ permalink raw reply	[flat|nested] 7+ messages in thread

* bug#41354: equal? has no sensible code path for symbols
  2020-05-28 16:50       ` David Kastrup
@ 2020-05-29  8:05         ` Ludovic Courtès
  2021-01-19 21:53           ` David Kastrup
  0 siblings, 1 reply; 7+ messages in thread
From: Ludovic Courtès @ 2020-05-29  8:05 UTC (permalink / raw)
  To: David Kastrup; +Cc: 41354

Hi,

David Kastrup <dak@gnu.org> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> Thus we could go with the patch below, though I doubt it would make a
>> measurable difference (and it actually adds tests for other cases).
>
> It made a considerable measurable difference in LilyPond

You measured with and without the patch I sent?  Or something else?

>> diff --git a/libguile/eq.c b/libguile/eq.c
>> index 627d6f09b..16c5bfb3f 100644
>> --- a/libguile/eq.c
>> +++ b/libguile/eq.c
>> @@ -303,6 +303,8 @@ scm_equal_p (SCM x, SCM y)
>>      return SCM_BOOL_F;
>>    if (SCM_IMP (y))
>>      return SCM_BOOL_F;
>> +  if (scm_is_symbol (x) || scm_is_symbol (y))
>> +    return SCM_BOOL_F;
>>    if (scm_is_pair (x) && scm_is_pair (y))
>>      {
>>        if (scm_is_false (scm_equal_p (SCM_CAR (x), SCM_CAR (y))))
>>
>
> Yes, that looks reasonable.  scm_is_symbol checks some tag subset that
> the code for equal_p later looks at closer as well: if you worry about
> the extra cost of the scm_is_symbol check, one could try folding the
> symbol check into that later code passage, which would slow down the
> symbol check and effect the more costly fallbacks less.  But since those
> fallbacks _are_ more costly, I doubt it would be worth the trouble.

Looking at eq.c, I don’t see what “costly fallbacks” you’re referring
to.  For a symbol, AIUI, we end up here:

  switch (SCM_TYP7 (x))
    {
    default:
      /* Check equality between structs of equal type (see cell-type test above). */
      if (SCM_STRUCTP (x))
	{
	  if (SCM_INSTANCEP (x))
	    goto generic_equal;
	  else
	    return scm_i_struct_equalp (x, y);
	}
      break;   // <- here, meaning we return SCM_BOOL_F

All the checks leading to this line are type tag comparisons.

Am I overlooking something?

Thanks,
Ludo’.





^ permalink raw reply	[flat|nested] 7+ messages in thread

* bug#41354: equal? has no sensible code path for symbols
  2020-05-29  8:05         ` Ludovic Courtès
@ 2021-01-19 21:53           ` David Kastrup
  0 siblings, 0 replies; 7+ messages in thread
From: David Kastrup @ 2021-01-19 21:53 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 41354

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> David Kastrup <dak@gnu.org> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>
> [...]
>
>>> Thus we could go with the patch below, though I doubt it would make a
>>> measurable difference (and it actually adds tests for other cases).
>>
>> It made a considerable measurable difference in LilyPond
>
> You measured with and without the patch I sent?  Or something else?

It made a considerable measurable difference in LilyPond to use scm_eq
over scm_eqv when one variable was known to be a symbol and most
comparisons would have turned out false.

>
>>> diff --git a/libguile/eq.c b/libguile/eq.c
>>> index 627d6f09b..16c5bfb3f 100644
>>> --- a/libguile/eq.c
>>> +++ b/libguile/eq.c
>>> @@ -303,6 +303,8 @@ scm_equal_p (SCM x, SCM y)
>>>      return SCM_BOOL_F;
>>>    if (SCM_IMP (y))
>>>      return SCM_BOOL_F;
>>> +  if (scm_is_symbol (x) || scm_is_symbol (y))
>>> +    return SCM_BOOL_F;
>>>    if (scm_is_pair (x) && scm_is_pair (y))
>>>      {
>>>        if (scm_is_false (scm_equal_p (SCM_CAR (x), SCM_CAR (y))))
>>>
>>
>> Yes, that looks reasonable.  scm_is_symbol checks some tag subset that
>> the code for equal_p later looks at closer as well: if you worry about
>> the extra cost of the scm_is_symbol check, one could try folding the
>> symbol check into that later code passage, which would slow down the
>> symbol check and effect the more costly fallbacks less.  But since those
>> fallbacks _are_ more costly, I doubt it would be worth the trouble.
>
> Looking at eq.c, I don’t see what “costly fallbacks” you’re referring
> to.  For a symbol, AIUI, we end up here:
>
>   switch (SCM_TYP7 (x))
>     {
>     default:
>       /* Check equality between structs of equal type (see cell-type test above). */
>       if (SCM_STRUCTP (x))
> 	{
> 	  if (SCM_INSTANCEP (x))
> 	    goto generic_equal;
> 	  else
> 	    return scm_i_struct_equalp (x, y);
> 	}
>       break;   // <- here, meaning we return SCM_BOOL_F
>
> All the checks leading to this line are type tag comparisons.
>
> Am I overlooking something?

That "all the checks" amount to quite a bit when the whole point of a
symbol is being faster to compare than structured types?  The main
surprise for me was that a symbol is a non-immediate type even though on
second thought it is clear that the symbol name has to be stored
somewhere associated with the symbol value.  However, from the
performance semantics a symbol should not be markedly different from
immediate types to avoid violating reasonable user expectations about
the Scheme type system: their whole point is to be fast to compare, and
fast comparisons are particularly important where you go through a large
number of them (which usually implies that most comparisons will end up
false).  This is particularly important when both arguments are symbols.

The normal expectation of functions like assv would be that they would
be only marginally slower than assq when the search key is a symbol (and
particularly so if most key/value pairs have a symbol key).  Because the
outcome for eqv(symbol1, symbol2)->#f takes quite longer than the
outcome for eq(symbol1, symbol2)->#f, this expectation is not met.

-- 
David Kastrup





^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2021-01-19 21:53 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-17 10:49 bug#41354: equal? has no sensible code path for symbols David Kastrup
2020-05-27 20:39 ` Ludovic Courtès
2020-05-27 20:49   ` David Kastrup
2020-05-28 16:06     ` Ludovic Courtès
2020-05-28 16:50       ` David Kastrup
2020-05-29  8:05         ` Ludovic Courtès
2021-01-19 21:53           ` David Kastrup

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).