unofficial mirror of guile-devel@gnu.org 
 help / color / mirror / Atom feed
* Re: [PATCH] add SRFI: srfi-121; generators
@ 2020-08-01  3:42 John Cowan
  2020-08-02 22:39 ` Mark H Weaver
  0 siblings, 1 reply; 16+ messages in thread
From: John Cowan @ 2020-08-01  3:42 UTC (permalink / raw)
  To: guile-devel

[-- Attachment #1: Type: text/plain, Size: 263 bytes --]

Mark Weaver wrote in July 2019:

Also, the provided implementations of 'generator-find', 'generator-any'
> and 'generator-every' are incorrect:


I appreciate your finding these bugs.  I wish, however, that you had also
sent them to <srfi-121@srfi.schemers.org>.

[-- Attachment #2: Type: text/html, Size: 578 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
  2020-08-01  3:42 [PATCH] add SRFI: srfi-121; generators John Cowan
@ 2020-08-02 22:39 ` Mark H Weaver
  0 siblings, 0 replies; 16+ messages in thread
From: Mark H Weaver @ 2020-08-02 22:39 UTC (permalink / raw)
  To: John Cowan; +Cc: guile-devel

Hi John,

John Cowan <cowan@ccil.org> wrote:
> Mark Weaver wrote in July 2019:
> 
>> Also, the provided implementations of 'generator-find', 'generator-any'
>> and 'generator-every' are incorrect:
> 
> 
> I appreciate your finding these bugs.  I wish, however, that you had also
> sent them to <srfi-121@srfi.schemers.org>.

Sorry about that.  As I recall, I didn't send those bug reports to the
SRFI-121 mailing list because I had assumed that the bugs were not in
the reference implementation.  I made that assumption because the buggy
procedures included comments that expressed uncertainty about what the
specification intended, e.g. "the spec would have me return #f, but I
think it must simply be wrong" and "a literal interpretation might say
it only terminates on #eof if (pred #eof) but I think this makes more
sense."

It didn't occur to me that the reference implementation of a finalized
SRFI would include comments like "the spec would have me return #f, but
I think it must simply be wrong".  However, I see now that I was
mistaken.  Indeed, those buggy implementations and confused comments are
in both the SRFI-121 and SRFI-158 reference implementations.

I *did* send bug reports to the SRFI-121 and SRFI-158 mailing lists
about inconsistencies in those specifications.  See:

  https://srfi-email.schemers.org/srfi-158/msg/11780787/
  https://srfi-email.schemers.org/srfi-158/msg/11781931/
  https://srfi-email.schemers.org/srfi-121/threads/2019/07/

I'm sorry to say it, but in my opinion SRFI-121 and SRFI-158 should be
deprecated and avoided.  The reference implementations do not match the
specifications, and the specifications themselves are self-contradictory
(see above).  Therefore, it's entirely possible that users of these
SRFIs may have contradictory expectations about how these procedures
behave.  Some may have read the spec one way, some may have read the
spec in a contradictory way, some may have learned from how the buggy
reference implementation behaves, and others may have learned from the
behavior of a different SRFI-121 implementation that doesn't have those
bugs.  At this point, I don't see how the problems can be fixed without
breaking some users' assumptions, and therefore breaking existing code.

I sincerely hope that these two SRFIs are an aberration, and not
representative of the quality of the R7RS-large effort.

      Mark



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
  2021-01-23  2:18     ` Arthur A. Gleckler
@ 2021-01-23  6:37       ` Mark H Weaver
  0 siblings, 0 replies; 16+ messages in thread
From: Mark H Weaver @ 2021-01-23  6:37 UTC (permalink / raw)
  To: Arthur A. Gleckler, Shiro Kawai; +Cc: srfi, John Cowan, guile-devel

Hi Arthur,

"Arthur A. Gleckler" <srfi@speechcode.com> writes:
> It's not a bad idea for the sample implementation to be as clear as
> possible at the expense of performance.

I agree that it's desirable for one of the sample implementations to be
as simple and clear as possible, for the purpose of clarifying the
specification.

> But it certainly wouldn't hurt to have a supplemental document making
> recommendations about possible performance improvements, or even a
> second implementation.

Sounds good.  For SRFIs such as 121 and 158, where efficiency is
important, I think that a second sample implementation tuned for
performance would be a useful addition.

     Regards,
       Mark



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
  2021-01-23  2:14   ` Shiro Kawai
@ 2021-01-23  2:18     ` Arthur A. Gleckler
  2021-01-23  6:37       ` Mark H Weaver
  0 siblings, 1 reply; 16+ messages in thread
From: Arthur A. Gleckler @ 2021-01-23  2:18 UTC (permalink / raw)
  To: Shiro Kawai; +Cc: Mark H Weaver, John Cowan, guile-devel, srfi

[-- Attachment #1: Type: text/plain, Size: 996 bytes --]

On Fri, Jan 22, 2021 at 6:15 PM Shiro Kawai <shiro.kawai@gmail.com> wrote:

> Hi Mark,
> I agree that the whole point of generators is performance.  When combined
> with lazy sequences (srfi-127), it covers many typical use cases of streams
> much more efficiently.
> I've written Gauche version with performance in mind, but it depends on a
> few non-standard features and can not be used as-is.  Besides, if we do go
> for performance, we need benchmark suites as well.  I'm interested in
> adopting Gauche version as the reference implementation but don't know when
> I have time.  If you have some code to improve the current reference
> implementation and can merge it to the repo, we can proceed gradually, I
> guess.
>

It's not a bad idea for the sample implementation to be as clear as
possible at the expense of performance.  But it certainly wouldn't hurt to
have a supplemental document making recommendations about possible
performance improvements, or even a second implementation.

[-- Attachment #2: Type: text/html, Size: 1324 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
  2021-01-23  0:58 ` Mark H Weaver
@ 2021-01-23  2:14   ` Shiro Kawai
  2021-01-23  2:18     ` Arthur A. Gleckler
  0 siblings, 1 reply; 16+ messages in thread
From: Shiro Kawai @ 2021-01-23  2:14 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: srfi, John Cowan, guile-devel

[-- Attachment #1: Type: text/plain, Size: 4781 bytes --]

Hi Mark,
I agree that the whole point of generators is performance.  When combined
with lazy sequences (srfi-127), it covers many typical use cases of streams
much more efficiently.
I've written Gauche version with performance in mind, but it depends on a
few non-standard features and can not be used as-is.  Besides, if we do go
for performance, we need benchmark suites as well.  I'm interested in
adopting Gauche version as the reference implementation but don't know when
I have time.  If you have some code to improve the current reference
implementation and can merge it to the repo, we can proceed gradually, I
guess.

--shiro


On Fri, Jan 22, 2021 at 2:59 PM Mark H Weaver <mhw@netris.org> wrote:

> Hi John,
>
> John Cowan <cowan@ccil.org> writes:
>
> > Mark: I'm interested to know if you have a sketch of ideas for a more
> > efficient implementation of SRFI 121/158.  You say it requires specific
> > knowledge of Guile internals, but are you willing to sketch how you might
> > do it?  Thanks.
>
> Here are a few suggestions off the top of my head, looking only at the
> latest SRFI-121 reference implementation:
>
> * In 'gcombine', 'generator-fold', 'generator-for-each', and possibly
>   also 'generator-unfold', it would be helpful to use 'case-lambda' to
>   provide specialized code for cases where the 'dotted' tail in the
>   argument list consists of 1 or 2 arguments.  When a procedure with a
>   dotted tail is called, it forces allocation of a new list, and last I
>   checked Guile does not include optimizations to avoid that allocation
>   where possible.  Worse, the general case requires calling 'map' and
>   allocating a new list every time a new item is requested.  It's a
>   great help to avoid these expenses in the most common cases.  For
>   example, see the definitions of 'map', 'for-each', and 'fold' in
>   Guile:
>
>
> https://git.savannah.gnu.org/cgit/guile.git/tree/module/ice-9/boot-9.scm?id=75b0db1a286f936a90683973efc2315a07c03b21#n214
>
> https://git.savannah.gnu.org/cgit/guile.git/tree/module/srfi/srfi-1.scm?id=75b0db1a286f936a90683973efc2315a07c03b21#n451
>
> * Avoid using 'define-values' in internal lexical contexts in Guile for
>   now.  Hopefully some day Guile's implementation of 'define-values'
>   will be more efficient, but for now it's implemented as a simple macro
>   that expands into code that mutates variables, which prevents several
>   other optimizations that could otherwise be done by Guile's compiler.
>   In particular, in 'gcombine', better use 'call-with-values' or
>   'receive'.
>
> * Several procedures are defined in terms of more general higher-order
>   procedures, or create intermediate lists/generators unnecessarily, for
>   the sake of making the code simpler.  In most contexts I would applaud
>   this practice, but there will be a significant price to pay in
>   efficiency.  For example, 'generator->reverse-list' with 1 argument is
>   implemented in terms of 'generator-fold', and with 2 arguments by
>   creating an intermediate generator using 'gtake'.
>
> * To make matters worse: 'gtake', as well as 'make-for-each-generator'
>   and 'make-unfold-generator', are defined in terms of coroutine
>   generators.  It's good to have coroutine generators available, but
>   they are quite expensive, and best avoided where efficiency is
>   desirable.
>
> * 'generator->vector' is defined using 'generator->list' and
>   'list->vector'.  That's bad enough, but digging deeper we see that
>   'generator->list' is implemented by reversing the result of
>   'generator->reverse-list', which as I've already pointed out is
>   defined using 'generator-fold', which currently involves calling 'map'
>   and 'append' and allocating temporary lists for each item generated.
>
> In general, it's helpful to go through the mental exercise of tracing an
> evaluation of each of these procedures, and more importantly to trace
> calls to the produced generators, to get some idea of the expense
> involved.
>
> This is just a sampling of the problems I noticed from a quick skim of
> the code, by no means comprehensive.
>
> I acknowledge that following the suggestions above will make the code
> much larger, uglier, and more difficult to understand.  I would not
> recommend such practices except in cases where efficiency is paramount.
>
> In this case, I think efficiency *is* paramount.  My rationale is that,
> like hash tables, generators inevitably force code that uses them to be
> written in an imperative style, and therefore they are best avoided in
> my opinion, *except* in cases where efficiency is paramount.
>
> To avoid being forced to write code in an imperative style, I suggest
> that in most cases streams are preferable to generators.
>
>       Regards,
>         Mark
>

[-- Attachment #2: Type: text/html, Size: 6123 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
  2021-01-21 18:39 John Cowan
@ 2021-01-23  0:58 ` Mark H Weaver
  2021-01-23  2:14   ` Shiro Kawai
  0 siblings, 1 reply; 16+ messages in thread
From: Mark H Weaver @ 2021-01-23  0:58 UTC (permalink / raw)
  To: John Cowan, guile-devel; +Cc: srfi

Hi John,

John Cowan <cowan@ccil.org> writes:

> Mark: I'm interested to know if you have a sketch of ideas for a more
> efficient implementation of SRFI 121/158.  You say it requires specific
> knowledge of Guile internals, but are you willing to sketch how you might
> do it?  Thanks.

Here are a few suggestions off the top of my head, looking only at the
latest SRFI-121 reference implementation:

* In 'gcombine', 'generator-fold', 'generator-for-each', and possibly
  also 'generator-unfold', it would be helpful to use 'case-lambda' to
  provide specialized code for cases where the 'dotted' tail in the
  argument list consists of 1 or 2 arguments.  When a procedure with a
  dotted tail is called, it forces allocation of a new list, and last I
  checked Guile does not include optimizations to avoid that allocation
  where possible.  Worse, the general case requires calling 'map' and
  allocating a new list every time a new item is requested.  It's a
  great help to avoid these expenses in the most common cases.  For
  example, see the definitions of 'map', 'for-each', and 'fold' in
  Guile:

  https://git.savannah.gnu.org/cgit/guile.git/tree/module/ice-9/boot-9.scm?id=75b0db1a286f936a90683973efc2315a07c03b21#n214
  https://git.savannah.gnu.org/cgit/guile.git/tree/module/srfi/srfi-1.scm?id=75b0db1a286f936a90683973efc2315a07c03b21#n451

* Avoid using 'define-values' in internal lexical contexts in Guile for
  now.  Hopefully some day Guile's implementation of 'define-values'
  will be more efficient, but for now it's implemented as a simple macro
  that expands into code that mutates variables, which prevents several
  other optimizations that could otherwise be done by Guile's compiler.
  In particular, in 'gcombine', better use 'call-with-values' or
  'receive'.

* Several procedures are defined in terms of more general higher-order
  procedures, or create intermediate lists/generators unnecessarily, for
  the sake of making the code simpler.  In most contexts I would applaud
  this practice, but there will be a significant price to pay in
  efficiency.  For example, 'generator->reverse-list' with 1 argument is
  implemented in terms of 'generator-fold', and with 2 arguments by
  creating an intermediate generator using 'gtake'.

* To make matters worse: 'gtake', as well as 'make-for-each-generator'
  and 'make-unfold-generator', are defined in terms of coroutine
  generators.  It's good to have coroutine generators available, but
  they are quite expensive, and best avoided where efficiency is
  desirable.

* 'generator->vector' is defined using 'generator->list' and
  'list->vector'.  That's bad enough, but digging deeper we see that
  'generator->list' is implemented by reversing the result of
  'generator->reverse-list', which as I've already pointed out is
  defined using 'generator-fold', which currently involves calling 'map'
  and 'append' and allocating temporary lists for each item generated.

In general, it's helpful to go through the mental exercise of tracing an
evaluation of each of these procedures, and more importantly to trace
calls to the produced generators, to get some idea of the expense
involved.

This is just a sampling of the problems I noticed from a quick skim of
the code, by no means comprehensive.

I acknowledge that following the suggestions above will make the code
much larger, uglier, and more difficult to understand.  I would not
recommend such practices except in cases where efficiency is paramount.

In this case, I think efficiency *is* paramount.  My rationale is that,
like hash tables, generators inevitably force code that uses them to be
written in an imperative style, and therefore they are best avoided in
my opinion, *except* in cases where efficiency is paramount.

To avoid being forced to write code in an imperative style, I suggest
that in most cases streams are preferable to generators.

      Regards,
        Mark



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
@ 2021-01-21 18:39 John Cowan
  2021-01-23  0:58 ` Mark H Weaver
  0 siblings, 1 reply; 16+ messages in thread
From: John Cowan @ 2021-01-21 18:39 UTC (permalink / raw)
  To: guile-devel; +Cc: srfi, Mark H Weaver

[-- Attachment #1: Type: text/plain, Size: 728 bytes --]

Back in July 2020, Mark Weaver wrote:

Also, the provided implementations of 'generator-find', 'generator-any'
> and 'generator-every' are incorrect:


Shiro Kawai has now fixed these bugs, and they have been merged into SRFI
121.  Note that even if it doesn't become part of Guile's
batteries-included, the code should still work on Guile as an R6RS
library.  However, the upward compatible SRFI 158 provides more generator
function as well as support for accumulators, the dual of generators.

Mark: I'm interested to know if you have a sketch of ideas for a more
efficient implementation of SRFI 121/158.  You say it requires specific
knowledge of Guile internals, but are you willing to sketch how you might
do it?  Thanks.

[-- Attachment #2: Type: text/html, Size: 1646 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
  2020-08-04 15:58     ` Marc Nieper-Wißkirchen
@ 2020-08-04 17:24       ` Dr. Arne Babenhauserheide
  0 siblings, 0 replies; 16+ messages in thread
From: Dr. Arne Babenhauserheide @ 2020-08-04 17:24 UTC (permalink / raw)
  To: Marc Nieper-Wißkirchen; +Cc: Mark H Weaver, John Cowan, guile-devel

[-- Attachment #1: Type: text/plain, Size: 1310 bytes --]


Marc Nieper-Wißkirchen <marc.nieper@gmail.com> writes:
>>> While I have been contributing to R7RS-large, I have to agree with you
>>> to some extent. Most of the SRFIs that have been voted into R7RS-large
>>> or are to be submitted for it don't have the quality of the R6RS or
>>> R7RS(-small) specifications
>
>> Inevitably so.  A large granite boulder cannot be cut like a diamond, but it has uses that a diamond does not (and vice versa, of course).
>
> There's already a large granite boulder called Python. When we want to
> advertise R7RS-large, which differences shall we point out (apart from
> the Lisp syntax)?

What I would like to see is a reduction step at the end. With Python we
have urllib, urllib2 and urllib3. Scheme offers the chance to clean up 

What Scheme brings to the table are multiple implementations for
different use-cases, along with a real standard.

Also first-class macros.

Though that doesn’t mean that letting quality lapse won’t hurt.

Do we actually have to take SRFI without changes into r7rs-large? Isn’t
an advantage of a standard that it can smooth out all those rough edges
you get when you bring together parts that were developed in isolation?

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1076 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
  2020-08-04 15:24   ` John Cowan
@ 2020-08-04 15:58     ` Marc Nieper-Wißkirchen
  2020-08-04 17:24       ` Dr. Arne Babenhauserheide
  0 siblings, 1 reply; 16+ messages in thread
From: Marc Nieper-Wißkirchen @ 2020-08-04 15:58 UTC (permalink / raw)
  To: John Cowan; +Cc: Mark H Weaver, guile-devel

Am Di., 4. Aug. 2020 um 17:24 Uhr schrieb John Cowan <cowan@ccil.org>:

>>  At the moment,
>> there is no general programmatic way to know whether a specific
>> implementation is up-to-date with respect to these post-finalization
>> notes or not.
>
>
> How could there be?  The implementations are written in a Turing-complete language.

If we had R6RS's library versions, we could just bump the version
number each time the API is changed due to a PFN. (Note that I am not
saying that R6RS's version numbers are the best tool; my point is more
that there are tools.)

>> A similar problem occurs with libraries that have already been voted
>> into R7RS-large. In the Red Edition, (scheme generator) stood for the
>> interface of SRFI 121; in the Tangerine Edition, it stands for SRFI
>> 158.

>
>
> I only did that because 158 is upward compatible with 121; indeed, bug-for-bug compatible, as it turned out.

SRFI 158 is not upward compatible in the full sense with SRFI 121
because it exports more identifiers. Of course, chances that some code
will break are low.

Likewise, SRFI 166, which will most likely replace SRFI 159, is not
fully upward compatible.

That said, these incompatible changes during the draft period of
R7RS-large are the lesser of two evils. As I said, if one needs a more
stable interface, just import (srfi 121) or (srfi 159) instead of
(scheme generator) or (scheme show).

>> While I have been contributing to R7RS-large, I have to agree with you
>> to some extent. Most of the SRFIs that have been voted into R7RS-large
>> or are to be submitted for it don't have the quality of the R6RS or
>> R7RS(-small) specifications

> Inevitably so.  A large granite boulder cannot be cut like a diamond, but it has uses that a diamond does not (and vice versa, of course).

There's already a large granite boulder called Python. When we want to
advertise R7RS-large, which differences shall we point out (apart from
the Lisp syntax)?

>> - Questions of tail context are often ignored.
>> - Higher-order procedures are often silent with respect to call/cc,
>> exceptions or side effects.
>
>
> Regrettably both true.  Scheme is so often used without effects that it's easy to forget they exist.

>> - Formal syntax is often missing.

> What further syntax do you want?  Procedures have only a single syntax, so all we need to know is what arguments exist and in what order.  Macros are few, but informal explanations (as in R[57]RS) seem sufficient to me.

For procedures, we don't need any formal syntax, of course. If it's
needed, it is for macros. If you take a look at many earlier SRFIs,
grammar production rules extending those of R5RS are given, which help
to understand the syntax.

On the other hand if you take a look at many other SRFIs, let's take
SRFI 204 for example, it's hard to know how to implement them just
from reading the spec and not the implementation.

>> - Formal semantics are almost always missing.

> That requires someone who can write their own Greek, which would not be me.

You don't need to write arcane formulas. Take SRFI 2, for example. The
semantics are pretty well laid out.

>> SRFI 121/158 is incidentally also an example for this. Take 'gappend'
>> from SRFI 158, for example. It doesn't specify when the various
>> generator arguments are being called.

> I don't follow you.  It is quite explicit that each is called until it is exhausted in left to right order.

For example, the current spec allows that "gappend" already calls the
various generators supplied as arguments (and stores their results)
before returning the resulting generator. It is specified, in which
order the values are delivered, but it isn't specified when the
supplied generators are asked for their values. There may be an
obvious order of procedure calls, but it is not in the spec.

Thus, if any of the supplied generators has side effects, we will have
an implementation-dependent result. Moreover, we don't know what
happens when the control flow steps out of or into one of the supplied
generator procedures through exceptions or call/cc.

"gappend" was just an example, though. There are many other examples
in many other SRFIs (my own probably as well) where we are not as
precise as R6RS or R7RS. This is somewhat a pity as R7RS-large should
be able to cover that part of R6RS that is not covered by R7RS-small
due to its, well, small size. So it would be great if the quality of
R7RS-large were at least as good as the quality of R6RS.



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
  2020-08-03 19:41 ` Marc Nieper-Wißkirchen
  2020-08-04 12:48   ` Marc Nieper-Wißkirchen
@ 2020-08-04 15:24   ` John Cowan
  2020-08-04 15:58     ` Marc Nieper-Wißkirchen
  1 sibling, 1 reply; 16+ messages in thread
From: John Cowan @ 2020-08-04 15:24 UTC (permalink / raw)
  To: Marc Nieper-Wißkirchen; +Cc: Mark H Weaver, guile-devel

[-- Attachment #1: Type: text/plain, Size: 4576 bytes --]

On Mon, Aug 3, 2020 at 3:41 PM Marc Nieper-Wißkirchen <marc.nieper@gmail.com>
wrote:

> It didn't occur to me that the reference implementation of a finalized
> > SRFI would include comments like "the spec would have me return #f, but
> > I think it must simply be wrong".


It certainly should not.  But it can happen when one person writes the spec
and another the implementation, and communication is less than perfect.
Process improvements since then should make this less likely, but
arse [sic] longa, vita brevis.


> > I'm sorry to say it, but in my opinion SRFI-121 and SRFI-158 should be
> > deprecated and avoided.  The reference implementations do not match the
> > specifications,


That can and shall be fixed.  The spec is stabilized, short of a
post-finalization note (which is never mandatory), but the implementation,
like any other program, must be presumed to have bugs, which is why
its behavior is *not* part of the spec (and why we don't use the term
"reference implementation" any more).


> > and the specifications themselves are self-contradictory (see above).


I'll look into that further.


> > At this point, I don't see how the problems can be fixed without
> > breaking some users' assumptions, and therefore breaking existing code.
>

*All* bug fixes, even those to bring an implementation into compliance with
its spec, risk breaking existing code.  Some people may have inserted
workarounds that are unavoidably not idempotent.

My favorite example of this is a stock-ticker service that transmitted
incorrect stock prices for stocks with 4-letter ticker symbols like AAPL.
The bug was easy to work around, and many people did.  When the bug was
eventually fixed, many clients were extremely annoyed, and insisted that
correct prices only be made available when using a versioning flag.  I
don't know the final outcome.  I hope the people responsible for the server
stood their ground.

 At the moment,
> there is no general programmatic way to know whether a specific
> implementation is up-to-date with respect to these post-finalization
> notes or not.
>

How could there be?  The implementations are written in a Turing-complete
language.


> A similar problem occurs with libraries that have already been voted
> into R7RS-large. In the Red Edition, (scheme generator) stood for the
> interface of SRFI 121; in the Tangerine Edition, it stands for SRFI
> 158.


I only did that because 158 is upward compatible with 121; indeed,
bug-for-bug compatible, as it turned out.


> While I have been contributing to R7RS-large, I have to agree with you
> to some extent. Most of the SRFIs that have been voted into R7RS-large
> or are to be submitted for it don't have the quality of the R6RS or
> R7RS(-small) specifications


Inevitably so.  A large granite boulder cannot be cut like a diamond, but
it has uses that a diamond does not (and vice versa, of course).


> - Questions of tail context are often ignored.
> - Higher-order procedures are often silent with respect to call/cc,
> exceptions or side effects.
>

Regrettably both true.  Scheme is so often used without effects that it's
easy to forget they exist.


> - Formal syntax is often missing.
>

What further syntax do you want?  Procedures have only a single syntax, so
all we need to know is what arguments exist and in what order.  Macros are
few, but informal explanations (as in R[57]RS) seem sufficient to me.

- Formal semantics are almost always missing.
>

That requires someone who can write their own Greek, which would not be me.


> SRFI 121/158 is incidentally also an example for this. Take 'gappend'
> from SRFI 158, for example. It doesn't specify when the various
> generator arguments are being called.
>

I don't follow you.  It is quite explicit that each is called until it is
exhausted in left to right order.


> PS: One of my own issues with SRFI 158 is that accumulators are not
> that well-designed from a logical point of view. I summarized it here:
> https://srfi-email.schemers.org/srfi-158/msg/6219505/. A follow-up
> thread begins here:
> https://srfi-email.schemers.org/srfi-158/msg/6557831/.
>

I stand behind my final reply in that thread as well as Shiro's.



John Cowan          http://vrici.lojban.org/~cowan        cowan@ccil.org
A mosquito cried out in his pain,
"A chemist has poisoned my brain!"
The cause of his sorrow / Was para-dichloro-
Diphenyltrichloroethane.                                (aka DDT)

[-- Attachment #2: Type: text/html, Size: 7292 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
  2020-08-03 19:41 ` Marc Nieper-Wißkirchen
@ 2020-08-04 12:48   ` Marc Nieper-Wißkirchen
  2020-08-04 15:24   ` John Cowan
  1 sibling, 0 replies; 16+ messages in thread
From: Marc Nieper-Wißkirchen @ 2020-08-04 12:48 UTC (permalink / raw)
  To: guile-devel; +Cc: Mark H Weaver, John Cowan

Am Mo., 3. Aug. 2020 um 21:41 Uhr schrieb Marc Nieper-Wißkirchen
<marc.nieper@gmail.com>:

> > I'm sorry to say it, but in my opinion SRFI-121 and SRFI-158 should be
> > deprecated and avoided.  The reference implementations do not match the
> > specifications, and the specifications themselves are self-contradictory
> > (see above).  Therefore, it's entirely possible that users of these
> > SRFIs may have contradictory expectations about how these procedures
> > behave.  Some may have read the spec one way, some may have read the
> > spec in a contradictory way, some may have learned from how the buggy
> > reference implementation behaves, and others may have learned from the
> > behavior of a different SRFI-121 implementation that doesn't have those
> > bugs.  At this point, I don't see how the problems can be fixed without
> > breaking some users' assumptions, and therefore breaking existing code.

> Of course, all this can be fixed, but by the sheer size of R7RS-large,
> I doubt that it ever will.

PPS That said, the more people are committing a bit of their spare
time to R7RS-large, the better the result will be. So if you are
inclined to improve SRFI 121/158 in the areas in which you have
perceived deficiencies, I am sure that John will put an improved
version on the next ballot.



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
       [not found] <mailman.61.1596470427.10776.guile-devel@gnu.org>
@ 2020-08-03 19:41 ` Marc Nieper-Wißkirchen
  2020-08-04 12:48   ` Marc Nieper-Wißkirchen
  2020-08-04 15:24   ` John Cowan
  0 siblings, 2 replies; 16+ messages in thread
From: Marc Nieper-Wißkirchen @ 2020-08-03 19:41 UTC (permalink / raw)
  To: guile-devel; +Cc: Mark H Weaver, John Cowan

Am Mo., 3. Aug. 2020 um 18:00 Uhr schrieb <guile-devel-request@gnu.org>:

> Message: 1
> Date: Sun, 02 Aug 2020 18:39:32 -0400
> From: Mark H Weaver <mhw@netris.org>
> To: John Cowan <cowan@ccil.org>
> Cc: guile-devel@gnu.org
> Subject: Re: [PATCH] add SRFI: srfi-121; generators
> Message-ID: <87v9i0zn7k.fsf@netris.org>
> Content-Type: text/plain

> It didn't occur to me that the reference implementation of a finalized
> SRFI would include comments like "the spec would have me return #f, but
> I think it must simply be wrong".  However, I see now that I was
> mistaken.  Indeed, those buggy implementations and confused comments are
> in both the SRFI-121 and SRFI-158 reference implementations.

To be fair, one should remark that the reference (or, rather, sample)
implementations do not belong to the SRFIs proper. That said, it is
surprising why the questions raised in the sample implementation
haven't been resolved during the draft period of the SRFI.

> I'm sorry to say it, but in my opinion SRFI-121 and SRFI-158 should be
> deprecated and avoided.  The reference implementations do not match the
> specifications, and the specifications themselves are self-contradictory
> (see above).  Therefore, it's entirely possible that users of these
> SRFIs may have contradictory expectations about how these procedures
> behave.  Some may have read the spec one way, some may have read the
> spec in a contradictory way, some may have learned from how the buggy
> reference implementation behaves, and others may have learned from the
> behavior of a different SRFI-121 implementation that doesn't have those
> bugs.  At this point, I don't see how the problems can be fixed without
> breaking some users' assumptions, and therefore breaking existing code.

There are a number of SRFIs with post-finalization notes, most of
which alter some behavior of their respective SRFIs. At the moment,
there is no general programmatic way to know whether a specific
implementation is up-to-date with respect to these post-finalization
notes or not.

A similar problem occurs with libraries that have already been voted
into R7RS-large. In the Red Edition, (scheme generator) stood for the
interface of SRFI 121; in the Tangerine Edition, it stands for SRFI
158. Of course, one can circumvent any problems by importing the
libraries through their SRFI names and not using the scheme namespace
before R7RS-large has been finalized.

> I sincerely hope that these two SRFIs are an aberration, and not
> representative of the quality of the R7RS-large effort.

While I have been contributing to R7RS-large, I have to agree with you
to some extent. Most of the SRFIs that have been voted into R7RS-large
or are to be submitted for it don't have the quality of the R6RS or
R7RS(-small) specifications. This is especially true in the following
areas:

- Questions of tail context are often ignored.
- Higher-order procedures are often silent with respect to call/cc,
exceptions or side effects.
- Formal syntax is often missing.
- Formal semantics are almost always missing.

In particular, the ignorance of call/cc in most SRFIs dealing with
higher-order procedures make this defining feature of Scheme hard to
use in a portable way.

SRFI 121/158 is incidentally also an example for this. Take 'gappend'
from SRFI 158, for example. It doesn't specify when the various
generator arguments are being called.

Of course, all this can be fixed, but by the sheer size of R7RS-large,
I doubt that it ever will.

Marc

PS: One of my own issues with SRFI 158 is that accumulators are not
that well-designed from a logical point of view. I summarized it here:
https://srfi-email.schemers.org/srfi-158/msg/6219505/. A follow-up
thread begins here:
https://srfi-email.schemers.org/srfi-158/msg/6557831/.



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
  2019-07-01  6:00   ` Mark H Weaver
@ 2019-07-01  6:21     ` Amar Singh
  0 siblings, 0 replies; 16+ messages in thread
From: Amar Singh @ 2019-07-01  6:21 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: guile-devel

On July 1, 2019 11:30:38 AM GMT+05:30, Mark H Weaver <mhw@netris.org> wrote:
>Hi again,
>
>I wrote earlier:
>> With this in mind, if SRFI-121 is to be added to Guile, it should be
>a
>> high performance implementation.  The implementation that you
>provided,
>> which I guess is primarily taken from the sample implementation, is
>far
>> too inefficient, at least on Guile.
>
>I should emphasize that there's no shame in not being able to meet my
>high expectations for efficiency in a SRFI-121 implementation.  The
>code
>that you provided would be quite reasonable in most contexts.
>
>In this particular case, because of the undesirability (IMO) of
>generators as an API due to their imperative nature, with their primary
>justification being the efficiency they can provide, I feel justified
>demanding high efficiency in this implementation.  Relatively few
>people
>would be able to meet those expectations.  It requires knowledge of
>Guile's implementation and compiler that relatively few people have.
>
>With that in mind, I hope that you will not be too discouraged by this,
>and that you will consider contributing to our community in the future.
>
>Also, I will try to find the time to send a followup message with more
>details on why the provided code would run inefficiently on Guile, and
>how to improve it.
>
>     Best regards,
>        Mark

Thanks, I will look at the issues you've pointed out.

The requirement to have a high performance implementation makes sense.
Amar Singh<nly@disroot.org>
----------------
Sent fromK-9 Mail. Please excuse my brevity.



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
  2019-07-01  5:06 ` Mark H Weaver
@ 2019-07-01  6:00   ` Mark H Weaver
  2019-07-01  6:21     ` Amar Singh
  0 siblings, 1 reply; 16+ messages in thread
From: Mark H Weaver @ 2019-07-01  6:00 UTC (permalink / raw)
  To: nly; +Cc: guile-devel

Hi again,

I wrote earlier:
> With this in mind, if SRFI-121 is to be added to Guile, it should be a
> high performance implementation.  The implementation that you provided,
> which I guess is primarily taken from the sample implementation, is far
> too inefficient, at least on Guile.

I should emphasize that there's no shame in not being able to meet my
high expectations for efficiency in a SRFI-121 implementation.  The code
that you provided would be quite reasonable in most contexts.

In this particular case, because of the undesirability (IMO) of
generators as an API due to their imperative nature, with their primary
justification being the efficiency they can provide, I feel justified
demanding high efficiency in this implementation.  Relatively few people
would be able to meet those expectations.  It requires knowledge of
Guile's implementation and compiler that relatively few people have.

With that in mind, I hope that you will not be too discouraged by this,
and that you will consider contributing to our community in the future.

Also, I will try to find the time to send a followup message with more
details on why the provided code would run inefficiently on Guile, and
how to improve it.

     Best regards,
        Mark



^ permalink raw reply	[flat|nested] 16+ messages in thread

* Re: [PATCH] add SRFI: srfi-121; generators
  2019-07-01  0:09 nly
@ 2019-07-01  5:06 ` Mark H Weaver
  2019-07-01  6:00   ` Mark H Weaver
  0 siblings, 1 reply; 16+ messages in thread
From: Mark H Weaver @ 2019-07-01  5:06 UTC (permalink / raw)
  To: nly; +Cc: guile-devel

Hi Amar,

> SRFI-121 Generators.
>
> All tests(49/49) are passing in my testing. I am not sure if the tests
> file is put in the correct place.
>
> From 0352f9be13aba1e8acc9a8f700f3673334d48d28 Mon Sep 17 00:00:00 2001
> From: Amar Singh <nly@disroot.org>
> Date: Mon, 1 Jul 2019 05:14:53 +0530
> Subject: [PATCH] add SRFI: srfi-121; generators

Thank you for this, but I'm sorry to say that I'm not inclined to add
this implementation of SRFI-121 to Guile.  The first sentence of
SRFI-121's "Rationale" section states:

  The main purpose of generators is high performance.

and I agree.  In fact, I would strongly discourage its use except in
cases where efficiency demands it.  Like hash tables, generators are
fundamentally imperative constructs, and they will tend to force code
built with them to be written in an imperative style.

With this in mind, if SRFI-121 is to be added to Guile, it should be a
high performance implementation.  The implementation that you provided,
which I guess is primarily taken from the sample implementation, is far
too inefficient, at least on Guile.

Also, the provided implementations of 'generator-find', 'generator-any'
and 'generator-every' are incorrect:

> +;; generator-find
> +(define (generator-find pred g)
> +  (let loop ((v (g)))
> +   ; A literal interpretation might say it only terminates on #eof if (pred #eof) but I think this makes more sense...
> +   (if (or (pred v) (eof-object? v))
> +     v
> +     (loop (g)))))

The SRFI says:

  Returns the first item from the generator gen that satisfies the
  predicate pred, or #f if no such item is found before gen is
  exhausted.

This specification is quite clear that #f should be returned if EOF is
encountered (i.e. "gen is exhausted").  Here, you are returning EOF in
that case, which is incorrect.

Also, I think that 'pred' should never be applied to EOF.

> +;; generator-any
> +(define (generator-any pred g)
> +  (let loop ((v (g)))
> +   (if (eof-object? v)
> +     #f
> +     (if (pred v)
> +       #t
> +       (loop (g))))))

This is incorrect.  If (pred v) returns a true value, the value returned
by (pred v) should be returned, not #t.

> +;; generator-every
> +(define (generator-every pred g)
> +  (let loop ((v (g)))
> +   (if (eof-object? v)
> +     #t
> +     (if (pred v)
> +       (loop (g))
> +       #f ; the spec would have me return #f, but I think it must simply be wrong...
> +       ))))

I can't make sense of the comment above.  Anyway, this is also
incorrect.  The specification is clear that if the generator is
exhausted after returning some items, (pred LAST-ITEM) should be
returned.  The only case where #t is returned is if the generator was
exhausted before calling 'generator-every'.

I might write my own implementation of SRFI-121 from scratch and add it
to Guile.

     Regards,
       Mark



^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH] add SRFI: srfi-121; generators
@ 2019-07-01  0:09 nly
  2019-07-01  5:06 ` Mark H Weaver
  0 siblings, 1 reply; 16+ messages in thread
From: nly @ 2019-07-01  0:09 UTC (permalink / raw)
  To: guile-devel


[-- Attachment #1.1: Type: text/plain, Size: 131 bytes --]

SRFI-121 Generators.

All tests(49/49) are passing in my testing. I am not sure if the tests
file is put in the correct place.

[-- Attachment #1.2: Type: text/html, Size: 398 bytes --]

[-- Attachment #2: 0001-add-SRFI-srfi-121-generators.patch --]
[-- Type: application/octet-stream, Size: 23279 bytes --]

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2021-01-23  6:37 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-01  3:42 [PATCH] add SRFI: srfi-121; generators John Cowan
2020-08-02 22:39 ` Mark H Weaver
  -- strict thread matches above, loose matches on Subject: below --
2021-01-21 18:39 John Cowan
2021-01-23  0:58 ` Mark H Weaver
2021-01-23  2:14   ` Shiro Kawai
2021-01-23  2:18     ` Arthur A. Gleckler
2021-01-23  6:37       ` Mark H Weaver
     [not found] <mailman.61.1596470427.10776.guile-devel@gnu.org>
2020-08-03 19:41 ` Marc Nieper-Wißkirchen
2020-08-04 12:48   ` Marc Nieper-Wißkirchen
2020-08-04 15:24   ` John Cowan
2020-08-04 15:58     ` Marc Nieper-Wißkirchen
2020-08-04 17:24       ` Dr. Arne Babenhauserheide
2019-07-01  0:09 nly
2019-07-01  5:06 ` Mark H Weaver
2019-07-01  6:00   ` Mark H Weaver
2019-07-01  6:21     ` Amar Singh

unofficial mirror of guile-devel@gnu.org 

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://yhetil.org/guile-devel/0 guile-devel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 guile-devel guile-devel/ https://yhetil.org/guile-devel \
		guile-devel@gnu.org
	public-inbox-index guile-devel

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.yhetil.org/yhetil.lisp.guile.devel
	nntp://news.gmane.io/gmane.lisp.guile.devel


AGPL code for this site: git clone http://ou63pmih66umazou.onion/public-inbox.git