Generic functions: Specializers like `or' or `memq'?

Generic functions: Specializers like `or' or `memq'?

[Michael Heerdegen]

Hello,

I come back to this question again and again, so it's time to just ask:

Often I want to use a certain method implementation for several very
similar cases and want to avoid code duplication, but I don't know how
to express this using the existing method specializers.

For example, I want the `head' spec to match one of several symbols who
are synonymous.  Or instead of (eql SYM) I would rather want
(memq (SYMS...)).  Or I want to combine several SPECS using `or' because
the implementation is the same.

Am I'm missing something or - this is not possible currently, right?
Would it be possible to implement such stuff then?

TIA,
Michael.

Re: Generic functions: Specializers like `or' or `memq'?

[Philip Kaludercic]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Hello,
>
> I come back to this question again and again, so it's time to just ask:
>
> Often I want to use a certain method implementation for several very
> similar cases and want to avoid code duplication, but I don't know how
> to express this using the existing method specializers.
>
> For example, I want the `head' spec to match one of several symbols who
> are synonymous.  Or instead of (eql SYM) I would rather want
> (memq (SYMS...)).  Or I want to combine several SPECS using `or' because
> the implementation is the same.
>
> Am I'm missing something or - this is not possible currently, right?
> Would it be possible to implement such stuff then?

I haven't tried it myself, but it seems like you would have to use
cl-generic-generalizers (see cl-generic.el) to do so.

-- 
	Philip Kaludercic

Re: Generic functions: Specializers like `or' or `memq'?

[Michael Heerdegen]

Philip Kaludercic <philipk@posteo.net> writes:

> I haven't tried it myself, but it seems like you would have to use
> cl-generic-generalizers (see cl-generic.el) to do so.

Yes, it should be doable.  I find it a bit suspicious that this is not
already available.

Thanks,

Michael.

Re: Generic functions: Specializers like `or' or `memq'?

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> I haven't tried it myself, but it seems like you would have to use
>> cl-generic-generalizers (see cl-generic.el) to do so.
>
> Yes, it should be doable.  I find it a bit suspicious that this is not
> already available.

My understanding is that these more computationally-intensive
generalizers have the potential to seriously slow down method funcalls,
and so we're sort of gently discouraged from doing that. Obviously once
you've got this tool you're bound to try to use it everywhere, but I
think in general it's supposed to work better to move tightly-focused
generalization into a `cond' inside the method body, or what have you.

Is anyone else's brain incapable of maintaining the distinction between
"specializer" and "generalizer"? I cannot remember which is which, or
exactly how they differ, and its one of the major obstacles to me
actually reading and understanding cl-generic.el.

Eric

Re: Generic functions: Specializers like `or' or `memq'?

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Is anyone else's brain incapable of maintaining the distinction between
> "specializer" and "generalizer"? I cannot remember which is which, or
> exactly how they differ, and its one of the major obstacles to me
> actually reading and understanding cl-generic.el.

The manual doesn't even mention the term "generalizer".  I don't have a
clue.  Is the term common in Common Lisp?  The implementation in
cl-generic is hard to understand without some previous knowledge.

Michael.

Re: Generic functions: Specializers like `or' or `memq'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Michael Heerdegen wrote:

>> Is anyone else's brain incapable of maintaining the
>> distinction between "specializer" and "generalizer"?
>> I cannot remember which is which, or exactly how they
>> differ, and its one of the major obstacles to me actually
>> reading and understanding cl-generic.el.
>
> The manual doesn't even mention the term "generalizer".
> I don't have a clue. Is the term common in Common Lisp?
> The implementation in cl-generic is hard to understand
> without some previous knowledge.

Word ...

From the `cl-generic-generalizers' docstring:

  It's called a generalizer because it takes a specific object
  and returns a more general approximation, denoting a set of
  objects to which it belongs.

Think of the -er suffix, e.g. toaster. It is the thing that
does things. Like MacGyver. A real man of action, right?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Generic functions: Specializers like `or' or `memq'?

[Michael Heerdegen]

Emanuel Berg via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

> From the `cl-generic-generalizers' docstring:
>
>   It's called a generalizer because it takes a specific object
>   and returns a more general approximation, denoting a set of
>   objects to which it belongs.

And the relation and difference to specializers is?

Michael.

Re: Generic functions: Specializers like `or' or `memq'?

[Stefan Monnier via Users list for the GNU Emacs text editor]

>>> I haven't tried it myself, but it seems like you would have to use
>>> cl-generic-generalizers (see cl-generic.el) to do so.
>> Yes, it should be doable.  I find it a bit suspicious that this is not
>> already available.
> My understanding is that these more computationally-intensive
> generalizers have the potential to seriously slow down method funcalls,
> and so we're sort of gently discouraged from doing that.

Not really, actually, no.

The main problem is one of ordering between different methods when
several methods are applicable.
[ `cl-generic.el` solves this problem by forcing the generalizer to
  come with a priority (a number), which makes the ordering "trivial"
  for `cl-generic.el` but it can be hard to choose the right priority
  to use when defining a new specializer/generalizer.  ]

> Is anyone else's brain incapable of maintaining the distinction between
> "specializer" and "generalizer"?

A specializer is what you write in the `cl-defmethod`, it says this
method is specialized for arguments of a particular type.
It can be thought of as a type or a set of values.

A generalizer is a sort of function that takes a value and returns the
specializer to which it belongs (it "generalizes" from a value to a set
that includes this value).  Every kind of specializer comes with
a matching generalizer as part of its implementation.

This is not standard CLOS parlance, because CLOS does not include
a mechanism to define new kinds of specializers: you can dispatch on
(eql <VAL>) and you can dispatch on types and that's that (tho IIRC, you
can actually extend it via the MOP, potentially with the help of some
details of your Common-Lisp implementation).


        Stefan

Re: Generic functions: Specializers like `or' or `memq'?

[Stefan Monnier via Users list for the GNU Emacs text editor]

> This is not standard CLOS parlance, because CLOS does not include
> a mechanism to define new kinds of specializers: you can dispatch on
> (eql <VAL>) and you can dispatch on types and that's that (tho IIRC, you
> can actually extend it via the MOP, potentially with the help of some
> details of your Common-Lisp implementation).

BTW, I took the term "generalizer" from the following article (from
which I drew a fair bit of inspiration when designing `cl-generic.el`):

    Generalizers: New metaobjects for generalized dispatch
    http://research.gold.ac.uk/9924/1/els-specializers.pdf


-- Stefan

Re: Generic functions: Specializers like `or' or `memq'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Stefan Monnier via Users list for the GNU Emacs text editor wrote:

>> This is not standard CLOS parlance, because CLOS does not include
>> a mechanism to define new kinds of specializers: you can dispatch on
>> (eql <VAL>) and you can dispatch on types and that's that (tho IIRC, you
>> can actually extend it via the MOP, potentially with the help of some
>> details of your Common-Lisp implementation).
>
> BTW, I took the term "generalizer" from the following article (from
> which I drew a fair bit of inspiration when designing `cl-generic.el`):
>
>     Generalizers: New metaobjects for generalized dispatch
>     http://research.gold.ac.uk/9924/1/els-specializers.pdf

Ah you see... I said it smells like OO spirit. No one likes OO
and everyone likes "functional programming". Well, can you
program as well? It's ridiculous to the point that we should
boast we had it long before and it was functional even then.

Call me poison!

https://www.youtube.com/watch?v=qzBel5-CADE

-- 
underground experts united
https://dataswamp.org/~incal

Re: Generic functions: Specializers like `or' or `memq'?

[Eric Abrahamsen]

Stefan Monnier via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

>>>> I haven't tried it myself, but it seems like you would have to use
>>>> cl-generic-generalizers (see cl-generic.el) to do so.
>>> Yes, it should be doable.  I find it a bit suspicious that this is not
>>> already available.
>> My understanding is that these more computationally-intensive
>> generalizers have the potential to seriously slow down method funcalls,
>> and so we're sort of gently discouraged from doing that.
>
> Not really, actually, no.
>
> The main problem is one of ordering between different methods when
> several methods are applicable.
> [ `cl-generic.el` solves this problem by forcing the generalizer to
>   come with a priority (a number), which makes the ordering "trivial"
>   for `cl-generic.el` but it can be hard to choose the right priority
>   to use when defining a new specializer/generalizer.  ]

Thank you for correcting my FUD!

>> Is anyone else's brain incapable of maintaining the distinction between
>> "specializer" and "generalizer"?
>
> A specializer is what you write in the `cl-defmethod`, it says this
> method is specialized for arguments of a particular type.
> It can be thought of as a type or a set of values.
>
> A generalizer is a sort of function that takes a value and returns the
> specializer to which it belongs (it "generalizes" from a value to a set
> that includes this value).  Every kind of specializer comes with
> a matching generalizer as part of its implementation.

Okay, this is helpful, but my brain is still resisting. I also read the
docstring for `cl-generic-generalizers' from time to time, and I think
if I can really understand what it's saying, I'll have a handle on it.

My earliest confusion began when I decided that if I could find the part
of the code that runs the `eql' specializer (or do we call it the `eql'
generalizer?) and then worked back from there, I would be able to make
sense of everything. But I couldn't even find the bit of code that looks
like (eql <val> <tag>)! And still can't.

[...]

> BTW, I took the term "generalizer" from the following article (from
> which I drew a fair bit of inspiration when designing `cl-generic.el`):
>
>     Generalizers: New metaobjects for generalized dispatch
>     http://research.gold.ac.uk/9924/1/els-specializers.pdf

Thanks! I'm reading this now.

Eric

Re: Generic functions: Specializers like `or' or `memq'?

[Stefan Monnier via Users list for the GNU Emacs text editor]

> My earliest confusion began when I decided that if I could find the part
> of the code that runs the `eql' specializer (or do we call it the `eql'
> generalizer?) and then worked back from there, I would be able to make
> sense of everything. But I couldn't even find the bit of code that looks
> like (eql <val> <tag>)! And still can't.

Nah, that would be too easy.
Instead it happens implicitly via a lookup in a hash-table.

I suspect that the code for cl-defstruct specializer might be easier to
understand in this respect.


        Stefan

Re: Generic functions: Specializers like `or' or `memq'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Stefan Monnier via Users list for the GNU Emacs text editor wrote:

>> My earliest confusion began when I decided that if I could
>> find the part of the code that runs the `eql' specializer
>> (or do we call it the `eql' generalizer?) and then worked
>> back from there, I would be able to make sense of
>> everything. But I couldn't even find the bit of code that
>> looks like (eql <val> <tag>)! And still can't.
>
> Nah, that would be too easy. Instead it happens implicitly
> via a lookup in a hash-table.
>
> I suspect that the code for cl-defstruct specializer might
> be easier to understand in this respect.

What's the typical use case for this?

Except for "when it is needed" I mean ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: Generic functions: Specializers like `or' or `memq'?

[Michael Heerdegen]

Stefan Monnier via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

> The main problem is one of ordering between different methods when
> several methods are applicable.
> [ `cl-generic.el` solves this problem by forcing the generalizer to
>   come with a priority (a number), which makes the ordering "trivial"
>   for `cl-generic.el` but it can be hard to choose the right priority
>   to use when defining a new specializer/generalizer.  ]

I don't see this.  How would an `or' or `memql' specializer be different
from implementing the method several times for each alternative using
the same body?  There would be a problem only if the domains of the
specializers would not be disjoint.  But that can already happen now.  I
don't see why we would make the situation worse, unless the programmer
does really strange things like listing the same (sub)specializer in
mutliple `or' specializers.  This could be warned about.

I guess I have missed something.

Michael.

Re: Generic functions: Specializers like `or' or `memq'?

[Stefan Monnier via Users list for the GNU Emacs text editor]

Emanuel Berg via Users list for the GNU Emacs text editor [2021-11-05 22:36:25] wrote:
> Stefan Monnier via Users list for the GNU Emacs text editor wrote:
>>> My earliest confusion began when I decided that if I could
>>> find the part of the code that runs the `eql' specializer
>>> (or do we call it the `eql' generalizer?) and then worked
>>> back from there, I would be able to make sense of
>>> everything. But I couldn't even find the bit of code that
>>> looks like (eql <val> <tag>)! And still can't.
>>
>> Nah, that would be too easy. Instead it happens implicitly
>> via a lookup in a hash-table.
>>
>> I suspect that the code for cl-defstruct specializer might
>> be easier to understand in this respect.
>
> What's the typical use case for this?

Not sure what "this" refers to.  If you're talking about the use case
for adding new specializers/generalizers, it's quite rare.
It's mostly a detail of `cl-generic.el` used internally.


        Stefan

Re: Generic functions: Specializers like `or' or `memq'?

[Stefan Monnier via Users list for the GNU Emacs text editor]

>> [ `cl-generic.el` solves this problem by forcing the generalizer to
>>   come with a priority (a number), which makes the ordering "trivial"
>>   for `cl-generic.el` but it can be hard to choose the right priority
>>   to use when defining a new specializer/generalizer.  ]
>
> I don't see this.  How would an `or' or `memql' specializer be different
> from implementing the method several times for each alternative using
> the same body?

You need to specify the priority of things like

    (or (eql 4) integer)
and
    (or (eql 4) (eql 5))
and
    (or (eql 4) (eql "hello"))

The first wants to have lower priority than `integer`, the second should
have higher priority than `integer`, as for the third ... well ...

It's easier to move the code to a separate function and use individual
`cl-defmethod`s for each case, all of them implemented by calling that
separate function.


        Stefan

Re: Generic functions: Specializers like `or' or `memq'?

[Emanuel Berg via Users list for the GNU Emacs text editor]

Stefan Monnier via Users list for the GNU Emacs text editor wrote:

> Not sure what "this" refers to. If you're talking about the
> use case for adding new specializers/generalizers, it's
> quite rare. It's mostly a detail of `cl-generic.el`
> used internally.

Specifically I mean when do you use specializers/generalizers
in ... general.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Generic functions: Specializers like `or' or `memq'?

[Michael Heerdegen]

Stefan Monnier via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

>     (or (eql 4) integer)
> and
>     (or (eql 4) (eql 5))
> and
>     (or (eql 4) (eql "hello"))

That's not a good example for how specializers I wanted to be able to
write would look like, but I see the problem.

Michael.

Re: Generic functions: Specializers like `or' or `memq'?

[Stefan Monnier via Users list for the GNU Emacs text editor]

Michael Heerdegen [2021-11-07 02:48:04] wrote:
> Stefan Monnier via Users list for the GNU Emacs text editor
> <help-gnu-emacs@gnu.org> writes:
>
>>     (or (eql 4) integer)
>> and
>>     (or (eql 4) (eql 5))
>> and
>>     (or (eql 4) (eql "hello"))
>
> That's not a good example for how specializers I wanted to be able to
> write would look like, but I see the problem.

BTW, the above problem doesn't prevent defining an `or` specializer.
It just means that this specializer may not always behave quite like one
might expect :-(

I think a `memql` specializer would be easier and less likely to cause
problems: you could just give it a priority just below that of `eql`
(it may still get it wrong in that (memql true false maybe) would likely
get higher priority than a specializer like `boolean` which (if we were
to add such a type) would like only match `nil` and `t` and hence
should ideally have higher priorty instead).

Feel free to give us some concrete examples of the kinds of things you'd
like to do, if you want to see what it would take to support them.


        Stefan

Re: Generic functions: Specializers like `or' or `memq'?

[Michael Heerdegen]

Stefan Monnier via Users list for the GNU Emacs text editor
<help-gnu-emacs@gnu.org> writes:

> Feel free to give us some concrete examples of the kinds of things
> you'd like to do, if you want to see what it would take to support
> them.

Oh - nothing special.  It just seems that whenever I try to make use of
generic functions the rules for writing specializers seems unnecessarily
restrictive.

The last time it was implementing that transformation function regexp ->
PEG, where the "head" could be one of two or three synonyms.  There are
workarounds, as always, but too often I think "ok, so this again was not
such a good example for using a generic function...".

Michael.