[PATCH] Add nondestructive delq1, delv1, and delete1.

[PATCH] Add nondestructive delq1, delv1, and delete1.

[Richard Sent]

* libguile/list.c: (delq1): Create.
(delv1): Create.
(delete1): Create.
* doc/ref/api-data.texi (List Modification): Document it.
---

Hi Guile,

There was some discussion on IRC a while back about how it was odd
there were destructive delete1! and similar functions, but no
nondestructive variants. This patch aims to fix that.

 doc/ref/api-data.texi | 52 ++++++++++++++++++++++++-------------------
 libguile/list.c       | 36 ++++++++++++++++++++++++++++++
 libguile/list.h       |  3 +++
 3 files changed, 68 insertions(+), 23 deletions(-)

diff --git a/doc/ref/api-data.texi b/doc/ref/api-data.texi
index 3b9933d97..c94846842 100644
--- a/doc/ref/api-data.texi
+++ b/doc/ref/api-data.texi
@@ -6000,39 +6000,45 @@ Deleting}), and also an @code{lset-difference} which can delete
 multiple @var{item}s in one call (@ref{SRFI-1 Set Operations}).
 @end deffn
 
-@deffn {Scheme Procedure} delq! item lst
-@deffnx {Scheme Procedure} delv! item lst
-@deffnx {Scheme Procedure} delete! item lst
-@deffnx {C Function} scm_delq_x (item, lst)
-@deffnx {C Function} scm_delv_x (item, lst)
-@deffnx {C Function} scm_delete_x (item, lst)
-These procedures are destructive versions of @code{delq}, @code{delv}
-and @code{delete}: they modify the pointers in the existing @var{lst}
-rather than creating a new list.  Caveat evaluator: Like other
-destructive list functions, these functions cannot modify the binding of
-@var{lst}, and so cannot be used to delete the first element of
-@var{lst} destructively.
-@end deffn
-
-@deffn {Scheme Procedure} delq1! item lst
+@deffn {Scheme Procedure} delq1 item lst
 @deffnx {C Function} scm_delq1_x (item, lst)
-Like @code{delq!}, but only deletes the first occurrence of
+Like @code{delq}, but only deletes the first occurrence of
 @var{item} from @var{lst}.  Tests for equality using
-@code{eq?}.  See also @code{delv1!} and @code{delete1!}.
+@code{eq?}.  See also @code{delv1} and @code{delete1}.
 @end deffn
 
-@deffn {Scheme Procedure} delv1! item lst
+@deffn {Scheme Procedure} delv1 item lst
 @deffnx {C Function} scm_delv1_x (item, lst)
-Like @code{delv!}, but only deletes the first occurrence of
+Like @code{delv}, but only deletes the first occurrence of
 @var{item} from @var{lst}.  Tests for equality using
-@code{eqv?}.  See also @code{delq1!} and @code{delete1!}.
+@code{eqv?}.  See also @code{delq1} and @code{delete1}.
 @end deffn
 
-@deffn {Scheme Procedure} delete1! item lst
+@deffn {Scheme Procedure} delete1 item lst
 @deffnx {C Function} scm_delete1_x (item, lst)
-Like @code{delete!}, but only deletes the first occurrence of
+Like @code{delete}, but only deletes the first occurrence of
 @var{item} from @var{lst}.  Tests for equality using
-@code{equal?}.  See also @code{delq1!} and @code{delv1!}.
+@code{equal?}.  See also @code{delq1} and @code{delv1}.
+@end deffn
+
+@deffn {Scheme Procedure} delq! item lst
+@deffnx {Scheme Procedure} delv! item lst
+@deffnx {Scheme Procedure} delete! item lst
+@deffnx {Scheme Procedure} delq1! item lst
+@deffnx {Scheme Procedure} delv1! item lst
+@deffnx {Scheme Procedure} delete1! item lst
+@deffnx {C Function} scm_delq_x (item, lst)
+@deffnx {C Function} scm_delv_x (item, lst)
+@deffnx {C Function} scm_delete_x (item, lst)
+@deffnx {C Function} scm_delq1_x (item, lst)
+@deffnx {C Function} scm_delv1_x (item, lst)
+@deffnx {C Function} scm_delete1_x (item, lst)
+These procedures are destructive versions of @code{delq}, @code{delv},
+@code{delete}, @code{delq1}, @code{delv1}, and @code{delete1}: they
+modify the pointers in the existing @var{lst} rather than creating a new
+list. Caveat evaluator: Like other destructive list functions, these
+functions cannot modify the binding of @var{lst}, and so cannot be used
+to delete the first element of @var{lst} destructively.
 @end deffn
 
 @deffn {Scheme Procedure} filter pred lst
diff --git a/libguile/list.c b/libguile/list.c
index 8063a15d1..af8e6222e 100644
--- a/libguile/list.c
+++ b/libguile/list.c
@@ -949,6 +949,42 @@ SCM_DEFINE (scm_delete1_x, "delete1!", 2, 0, 0,
 }
 #undef FUNC_NAME
 
+SCM_DEFINE (scm_delq1, "delq1", 2, 0, 0,
+            (SCM item, SCM lst),
+	    "Like @code{delq}, but only deletes the first occurrence of\n"
+	    "@var{item} from @var{lst}.  Tests for equality using\n"
+	    "@code{eq?}.  See also @code{delv1} and @code{delete1}.")
+#define FUNC_NAME s_scm_delq1
+{
+  SCM copy = scm_list_copy (lst);
+  return scm_delq1_x (item, copy);
+}
+#undef FUNC_NAME
+
+SCM_DEFINE (scm_delv1, "delv1", 2, 0, 0,
+            (SCM item, SCM lst),
+	    "Like @code{delv}, but only deletes the first occurrence of\n"
+	    "@var{item} from @var{lst}.  Tests for equality using\n"
+	    "@code{eqv?}.  See also @code{delq1} and @code{delete1}.")
+#define FUNC_NAME s_scm_delv1
+{
+  SCM copy = scm_list_copy (lst);
+  return scm_delv1_x (item, copy);
+}
+#undef FUNC_NAME
+
+SCM_DEFINE (scm_delete1, "delete1", 2, 0, 0,
+            (SCM item, SCM lst),
+	    "Like @code{delete}, but only deletes the first occurrence of\n"
+	    "@var{item} from @var{lst}.  Tests for equality using\n"
+	    "@code{equal?}.  See also @code{delq1} and @code{delv1}.")
+#define FUNC_NAME s_scm_delete1
+{
+  SCM copy = scm_list_copy (lst);
+  return scm_delete1_x (item, copy);
+}
+#undef FUNC_NAME
+
 SCM_DEFINE (scm_filter, "filter", 2, 0, 0,
 	    (SCM pred, SCM list),
 	    "Return all the elements of 2nd arg @var{list} that satisfy predicate @var{pred}.\n"
diff --git a/libguile/list.h b/libguile/list.h
index 5ebcc8a82..bf9f6d90e 100644
--- a/libguile/list.h
+++ b/libguile/list.h
@@ -63,6 +63,9 @@ SCM_API SCM scm_delete (SCM item, SCM lst);
 SCM_API SCM scm_delq1_x (SCM item, SCM lst);
 SCM_API SCM scm_delv1_x (SCM item, SCM lst);
 SCM_API SCM scm_delete1_x (SCM item, SCM lst);
+SCM_API SCM scm_delq1 (SCM item, SCM lst);
+SCM_API SCM scm_delv1 (SCM item, SCM lst);
+SCM_API SCM scm_delete1 (SCM item, SCM lst);
 SCM_API SCM scm_filter (SCM pred, SCM list);
 SCM_API SCM scm_filter_x (SCM pred, SCM list);
 SCM_API SCM scm_copy_tree (SCM obj);
-- 
2.45.1

The Guile junk drawer and a C plea (was: [PATCH] Add nondestructive delq1, delv1, and delete1.)

[Thompson, David]

Hi Richard and all other Guilers, too,

What follows is not code review, but your patch felt like an
opportunity to provide some commentary about the trajectory of Guile
development that I've wanted to share for awhile.

First, I think Guile's default environment is a total mess.  It's the
very definition of a junk drawer.  There's over 1000 names in the
(guile) module!  Contrast this with R7RS-small's (scheme base) module
that only has 200ish.  Guile is an old project and I'm sure stuff just
accumulated over the years, but having so much in the default
environment makes it hard to know what a program actually uses because
many things that ought to be explicit imports are not. This makes it a
challenge to move Guile in a more "least authority" direction.  As a
rule, I think Guile should *not* add any additional names to the
default environment without an extremely good reason. Because (guile)
is imported implicitly, new names can cause clashes with existing code
that require #:replace to suppress the warning about shadowing core
bindings.  For example, the newish 'spawn' procedure collides with
'spawn' in (goblins core) in the Goblins project.  I think Guile needs
a (multi-year, multi-major version) plan to deprecate cruft and move
the good stuff into different modules.  Give a hoot, don't pollute
(the default environment)!

Second, please please please, no more C! Guile's substantial amount of
C code is a legacy of its origins decades ago, and we need to make it
clear to new users and contributors that new code should be written in
Scheme! The procedures in Richard's patch would be much more elegantly
written in Scheme and it would allow the compiler to gnaw on the code,
too.  Those experienced with Guile know that writing Scheme procedures
in C has all sorts of issues, like non-resumable continuations if a C
stack frame is captured, but we could probably do more to discourage
writing C in the docs and stuff.  It's also just no fun at all.  Who
actually wants to use that C API?  Furthermore, every procedure
implemented in C is a challenge for bringing Guile to new places like,
say, WebAssembly. Hoot is unable to import any module from Guile that
loads a C extension. Did you know that (srfi srfi-1) loads a C
extension?  Argh!  I'm hoping we on the Hoot team will fix that
particular issue soon.

Okay I should be going to bed instead of writing emails. That's all for now!

- Dave

Re: The Guile junk drawer and a C plea (was: [PATCH] Add nondestructive delq1, delv1, and delete1.)

[Mikael Djurfeldt]

[-- Attachment #1: Type: text/plain, Size: 3541 bytes --]

Your two points sound good.

However, I personally hope that Guile will continue to be friendly towards
those using it as an embedded language. There, a C API is important. There
*is* a downside in moving to Scheme from that perspective, but that is fine
as long as it is easy to call Scheme from C. The typical use case is when a
binding in the application which use Guile as embedded language needs to
manipulate Scheme data in its arguments in order to interpret them.

Regarding the junk, I very much agree. I also look forward to getting rid
of ice-9 :). As has been spoken about here previously, I suggest that we
design a new module hierarchy, introduce aliases for module bindings, and
still supply the old module hierarchy during a few years for backward
compatibility.

I don't see any harm in applying Richards patch, though, as things look
right now.

Best regards,
Mikael

Den lör 29 juni 2024 04:53Thompson, David <dthompson2@worcester.edu> skrev:

> Hi Richard and all other Guilers, too,
>
> What follows is not code review, but your patch felt like an
> opportunity to provide some commentary about the trajectory of Guile
> development that I've wanted to share for awhile.
>
> First, I think Guile's default environment is a total mess.  It's the
> very definition of a junk drawer.  There's over 1000 names in the
> (guile) module!  Contrast this with R7RS-small's (scheme base) module
> that only has 200ish.  Guile is an old project and I'm sure stuff just
> accumulated over the years, but having so much in the default
> environment makes it hard to know what a program actually uses because
> many things that ought to be explicit imports are not. This makes it a
> challenge to move Guile in a more "least authority" direction.  As a
> rule, I think Guile should *not* add any additional names to the
> default environment without an extremely good reason. Because (guile)
> is imported implicitly, new names can cause clashes with existing code
> that require #:replace to suppress the warning about shadowing core
> bindings.  For example, the newish 'spawn' procedure collides with
> 'spawn' in (goblins core) in the Goblins project.  I think Guile needs
> a (multi-year, multi-major version) plan to deprecate cruft and move
> the good stuff into different modules.  Give a hoot, don't pollute
> (the default environment)!
>
> Second, please please please, no more C! Guile's substantial amount of
> C code is a legacy of its origins decades ago, and we need to make it
> clear to new users and contributors that new code should be written in
> Scheme! The procedures in Richard's patch would be much more elegantly
> written in Scheme and it would allow the compiler to gnaw on the code,
> too.  Those experienced with Guile know that writing Scheme procedures
> in C has all sorts of issues, like non-resumable continuations if a C
> stack frame is captured, but we could probably do more to discourage
> writing C in the docs and stuff.  It's also just no fun at all.  Who
> actually wants to use that C API?  Furthermore, every procedure
> implemented in C is a challenge for bringing Guile to new places like,
> say, WebAssembly. Hoot is unable to import any module from Guile that
> loads a C extension. Did you know that (srfi srfi-1) loads a C
> extension?  Argh!  I'm hoping we on the Hoot team will fix that
> particular issue soon.
>
> Okay I should be going to bed instead of writing emails. That's all for
> now!
>
> - Dave
>
>

[-- Attachment #2: Type: text/html, Size: 4180 bytes --]

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 2253 bytes --]

Mikael Djurfeldt <mikael@djurfeldt.com> writes:

> design a new module hierarchy, introduce aliases for module bindings,
> and still supply the old module hierarchy during a few years for
> backward compatibility.

Please do not do this. It is a recipe for disaster.

Do you remember when Lilypond broke with Guile 2.0?
How long it took to get it working with modern Guile again?
This plan would cause the same — but much, much worse.

Lilypond is the one tool using Guile — the single tool — which actually
reigns supreme in its domain. Nothing else comes even close in quality.

The tools broken by breaking things “with sufficient warning” are
usually the most advanced ones. Specialized tools. The ones which rule
in their domain. That people depend on.

There’s something many useful things have in common: they work and need
little changes.

When the infrastructure these tools use intentionally breaks the tools
and requires constant upkeep just to keep working, this makes the
infrastructure volatile and unreliable.

Such large changes promise to make the system better, but they leave
it in a state of eternal semi-brokenness, because before the first
breakage is fully resolved, the next part gets rewritten and it causes
more breakage. And so the next breakage comes. Because doing such
“let’s just change it all” steps changes the culture.

Python tools had just kept working for years and years before Python 3.
After the release of Python 3, they broke every few years. The culture
had changed to one that accepts being volatile.

Making Guile volatile would make it a dumb idea to depend on it for
infrastructure. And I very much want to use Guile for my infrastructure.

I reduced my reliance on Python after having to spend time again and
again when my existing tools broke. Guile is becoming the replacement
for it. Please don’t plan to break it.

Please let us be a reliable foundation for infrastructure.

Best wishes,
Arne

PS: If this argument doesn’t suffice for you, before you discard it,
    please have a look at
    https://www.draketo.de/software/volatile-infrastructure
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: The Guile junk drawer and a C plea

[Damien Mattei]

[-- Attachment #1: Type: text/plain, Size: 3262 bytes --]

just my 2 cents...

about moving code from C to Scheme, it seems to be the Racket choice , they
had Racket written in C and now they have a new version based on Chez
Scheme that is mainly written in Scheme, they move a lot of code from C to
Scheme as far as i know...

About breaking backward compatibility , i understand it could be a
disaster... but if Python made this choice is to go forward in better
language, if you never move something you can not advance and create better
things. It is my critic of Scheme: have too little change in 40 years, even
if in Scheme+ i try to be 100% compatible with Scheme. By the way i do not
understand why a 100% written scheme program would no more work with a
different (internal) scheme implementation but i do not know Lilypond
design.

regards,
Damien

On Sat, Jun 29, 2024 at 10:13 AM Dr. Arne Babenhauserheide <arne_bab@web.de>
wrote:

> Mikael Djurfeldt <mikael@djurfeldt.com> writes:
>
> > design a new module hierarchy, introduce aliases for module bindings,
> > and still supply the old module hierarchy during a few years for
> > backward compatibility.
>
> Please do not do this. It is a recipe for disaster.
>
> Do you remember when Lilypond broke with Guile 2.0?
> How long it took to get it working with modern Guile again?
> This plan would cause the same — but much, much worse.
>
> Lilypond is the one tool using Guile — the single tool — which actually
> reigns supreme in its domain. Nothing else comes even close in quality.
>
> The tools broken by breaking things “with sufficient warning” are
> usually the most advanced ones. Specialized tools. The ones which rule
> in their domain. That people depend on.
>
> There’s something many useful things have in common: they work and need
> little changes.
>
> When the infrastructure these tools use intentionally breaks the tools
> and requires constant upkeep just to keep working, this makes the
> infrastructure volatile and unreliable.
>
> Such large changes promise to make the system better, but they leave
> it in a state of eternal semi-brokenness, because before the first
> breakage is fully resolved, the next part gets rewritten and it causes
> more breakage. And so the next breakage comes. Because doing such
> “let’s just change it all” steps changes the culture.
>
> Python tools had just kept working for years and years before Python 3.
> After the release of Python 3, they broke every few years. The culture
> had changed to one that accepts being volatile.
>
> Making Guile volatile would make it a dumb idea to depend on it for
> infrastructure. And I very much want to use Guile for my infrastructure.
>
> I reduced my reliance on Python after having to spend time again and
> again when my existing tools broke. Guile is becoming the replacement
> for it. Please don’t plan to break it.
>
> Please let us be a reliable foundation for infrastructure.
>
> Best wishes,
> Arne
>
> PS: If this argument doesn’t suffice for you, before you discard it,
>     please have a look at
>     https://www.draketo.de/software/volatile-infrastructure
> --
> Unpolitisch sein
> heißt politisch sein,
> ohne es zu merken.
> draketo.de
>

[-- Attachment #2: Type: text/html, Size: 4349 bytes --]

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 4473 bytes --]

Damien Mattei <damien.mattei@gmail.com> writes:

> About breaking backward compatibility , i understand it could be a
> disaster... but if Python made this choice

The experience of Python should not be encouragement. It was a disaster.

Many tools are *still* broken, especially specialist tools. More than
one and a half decades after Python 3 got introduced. The developers
continued to maintain Python 2 for 12 years — until 2020.

Python 3 still got takeup, because the AI craze hit and it was in a good
position to provide readable (yet brittle) APIs. But I personally lost
many weekends fixing existing tools. And I am sure that I’m not the only
one.

I am also certain that Python 3 caused lots of damage to Mercurial. Do
you remember when Mercurial showed that Python can compete in
performance with C while only using a minimal set of performance
critical tooling in C?

Mercurial competed on roughly equal terms with Git.

One distinguishing factor were thirdparty extensions which added
specialized capabilities. Many of these got broken with Python 3. I
personally gave up maintenance of two extensions because I wrote them
when I had free time for them, but as most other people, when they were
broken a when Mercurial finished the Python 3 transition, I didn’t have
that time anymore.

These extensions were useful, but many where not maintained. Before
Python 3 they did not *need* maintenance: they solved a problem and just
kept working. After Python 3 that was no longer the case.

Mercurial has become a niche tool now.

I do not want that fate for our tools.

> better things. It is my critic of Scheme: have too little change in 40
> years, even if in Scheme+ i try to be 100% compatible with Scheme.

Let’s look at what the one responsible for Python 3 said about changing
code:

>> No one wants their code to break, but they always want everyone elses
>> code to break by adding the keyword they want. — Guide van Rossum, 2018
https://youtu.be/Oiw23yfqQy8?t=163

He has a slide of what went wrong: https://youtu.be/Oiw23yfqQy8?t=769

- underrated Python’s popularity: people using every trick in the book
  of what worked.
- Underestimated the importance of 3rd party packages: dependencies on
  lots of tiny little modules that solve one little problem.
- clumsy migration: only *mostly working* auto-translation.
- no runtime compatibility: a single non-converted dependency
  blocked compatibility.
- supporting different versions in the same code is hard.

Despite those lessons learned, many tools are *still* broken,
especially specialist tools. Most of these are dying. More than one
and a half decades after Python 3 got introduced. The developers
continued to maintain Python 2 for 12 years — until 2020.

And even worse: some Python 3 point releases broke existing code again.
The big breakage had caused a cultural change. Breaking compatibility
just a bit was deemed OK now.

Python 3 still got takeup. I think it was, because the AI craze hit and
Python was in a good position to provide readable (yet brittle) APIs to
C++ code. But I personally lost many weekends fixing existing tools. And
while I can still use Python, I moved on. Which is part of the reason
why I’m here now. And I am sure that I’m not the only one.

If you have to invest lots of time anyway, you can just rewrite in
another language instead, which is much more enjoyable than spending
days over days searching for the source of some remaining
bytes-to-unicode breakages.

> By the way i do not understand why a 100% written scheme program would
> no more work with a different (internal) scheme implementation but i
> do not know Lilypond design.

Lilypond did a lot of optimization to be fast. It couldn’t afford small
slowdowns, because it is used by people to create large
production-quality scorebooks. So they did what worked to fulfill their
requirements.

Lilypond is C embedding Guile as scripting language. As the Guile
reference manual recommended using Guile back then. They had to use some
unclean options to reach the requirements of their users. 

And similar will be true for many other tools which manage to be best in
their domain. I hope that Guile will have many of them, and I want it to
be a safe choice to implement these.


Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

RE: The Guile junk drawer and a C plea (was: [PATCH] Addnondestructive delq1, delv1, and delete1.)

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1000 bytes --]

>However, I personally hope that Guile will continue to be friendly towards those using it as an embedded language. There, a C API is important. There *is* a downside in moving to Scheme from that perspective, but that is fine as long as it is easy to call Scheme from C. The typical use case is when a binding in the application which use Guile as embedded language needs to manipulate Scheme data in its arguments in order to interpret them.

Currently, there is some code to automatically generate Scheme bindings to C procedures (i.e. automatically generated scm_c_define_gsubr) (IIRC this is ‘snarfing’, but maybe that’s only a documentation thing).

It is possible to call the Scheme procedures from C (scm_call + module reflection API to get the procedure), but AFAIK there is no mechanism yet for _automatically_ generating the C wrappers to Scheme things yet. For the embedded language case, it would be useful to have such an automatic generator.

Best regards,
Maxime Devos.

[-- Attachment #2: Type: text/html, Size: 2243 bytes --]

RE: The Guile junk drawer and a C plea (was: [PATCH] Addnondestructive delq1, delv1, and delete1.)

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 4801 bytes --]


>Regarding the junk, I very much agree. I also look forward to getting rid of ice-9 :). As has been spoken about here previously, I suggest that we design a new module hierarchy, introduce aliases for module bindings, and still supply the old module hierarchy during a few years for backward compatibility.

On this, and on responses on potential harm introduces by incompatible changes:

It’s not like Python where semantics were changed in some changes, this is about simply renaming stuff. We can keep the (ice-9 ...) modules/bindings forever – if not in Guile, then in a separate Scheme library (so out-of-date Scheme libraries can still be used after installing this compatibility library with whatever is the local package manager).

The default environment is irrelevant to most Scheme libraries, since it is irrelevant to any library defining modules with ‘define-module’, ‘define-library’ or ‘library’ forms (which is pretty much the only reasonable way to define modules unless you are making your own system). This already reduces a lot of compatibility concerns. An important exception is anyone doing ‘eval’ or compiling expressions (think anyone doing the equivalent of “guile -l script.scm”.

Now let’s look at ‘eval’ and the like, in particular where it uses the default environment.

From the manual:

Scheme Procedure: eval exp module_or_state
C Function: scm_eval (exp, module_or_state)
Evaluate exp, a list representing a Scheme expression, in the top-level environment specified by module_or_state. While exp is evaluated (using primitive-eval), module_or_state is made the current module. The current module is reset to its previous value when eval returns. XXX - dynamic states. Example: (eval ’(+ 1 2) (interaction-environment))

Note that ‘eval’ does not even have a default environment, you need to specify an environment (in this case, a module) to evaluate the expression in! So, an option here is to let (guile) be the current messy (not recommended) environment, let (guile interaction-environment-2) (*) be a cleaned-up version, if at some point in time it is decided this cleaned-up version is not clean enough or that some bindings should be added, a new module (guile interaction-environment-3) could be defined, etc..

(Important point: never change the contents of one of these interaction environment modules, to ensure compatibility!)  (If at some point there are too many of these interaction environment, the old ones can be moved to separate libraries to be installed with the local package manager – this might even include (guile).)

This way, any user of ‘eval’ or the like can (and in case of ‘eval’, needs to) specify _which_ environment they want.

Next, we have the thunk ‘interaction-environment’:

>Scheme Procedure: interaction-environment
>C Function: scm_interaction_environment ()
>Return a specifier for the environment that contains implementation–defined bindings, typically a superset of those listed in the report. The intent is that this procedure will return the environment in which the implementation would evaluate expressions dynamically typed by the user.

For compatibility, this would need to be the messy (guile) module. It would also need to deprecated (but not actually removed) – best let users specify _which_ version they want, so compatibility is preserved.

Next there is ‘eval-string’ and ‘primitive-eval’, which evaluates in the current module (and hence doesn’t care about what is the interaction environment).

There is also “guile -l [...]” and the like, but I don’t think Lilypond and the like really care about what “guile -l [...]” uses as default environment, as long as there is a way of overriding the choice.

Also, on deprecation and removal: just because you deprecate something, doesn’t mean it has to be removed. And just because something deprecated will be removed or because something is volatile, doesn’t mean it has to be an incompatible change! 

We can do something similar to what Coq does – Coq sometimes does incompatible changes, but at the same time, when it does so, it also verifies that there are no impacted libraries, and if there are impacted libraries, the impacted libraries are first modified – by people making changes to Coq itself(*) – to update it according to the changes.  (It’s not done for _all_ libraries of course, but still a nice selection.)

(*) Actually I’m not sure if this about Coq itself or a selection of Coq libraries, but the point stands either way.

Also, in RnRS modules can have version modules. There isn’t currently support for defining multiple versions of modules in Guile, but there could be.

Best regards,
Maxime Devos.

[-- Attachment #2: Type: text/html, Size: 8490 bytes --]

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1824 bytes --]


Maxime Devos <maximedevos@telenet.be> writes:
>> I suggest that we design a new module hierarchy, introduce aliases for
>> module bindings, and still supply the old module hierarchy during a few
>> years for backward compatibility.
> Also, on deprecation and removal: just because you deprecate something, doesn’t mean it has to be removed. And just because something
> deprecated will be removed or because something is volatile, doesn’t mean it has to be an incompatible change! 

Change can be done without breaking things, but that’s not what was
proposed.

I’m not saying that there shouldn’t be change, just that there shouldn’t
be *breaking* change.

And that a new hierarchy we define is not guaranteed to actually *stay*
better. There would be errors, but different ones. We can only expect
that the new one will be significantly better in case where either the
computing environment changed substantially or where our knowledge and
skill of how to define such a structure improved substantially.

Also take into account whether we can adapt third party tutorials. These
are an often forgotten part of software and when they no longer work due
to chages, this seriously hampers adoption.

One such case was making modules declarative by default. This broke
tutorials which described how to create a live-REPL for interactive game
development. It cost me hours of time to find the cause. And then again
when I worked on a system with only Guile 2.0 where that module-option
#:declarative #f is illegal. The compatibility change for the REPL
completely broke the code on that other system. And the deployment
system had another Guile version again, causing more breakage.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: The Guile junk drawer and a C plea

[Lassi Kortela]

> And that a new hierarchy we define is not guaranteed to actually *stay*
> better. There would be errors, but different ones. We can only expect
> that the new one will be significantly better in case where either the
> computing environment changed substantially or where our knowledge and
> skill of how to define such a structure improved substantially.

Words of wisdom.

RE: The Guile junk drawer and a C plea

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2926 bytes --]

>Maxime Devos <maximedevos@telenet.be> writes:
>>> I suggest that we design a new module hierarchy, introduce aliases for
>>> module bindings, and still supply the old module hierarchy during a few
>>> years for backward compatibility.
>> Also, on deprecation and removal: just because you deprecate something, doesn’t mean it has to be removed. And just because something
>> deprecated will be removed or because something is volatile, doesn’t mean it has to be an incompatible change! 
>Change can be done without breaking things, but that’s not what was
proposed.

Unless you are referring to the ‘during a few years’ (and in particular, the implied ‘remove it afterwards’) or (*), I don’t follow – renaming things and keeping the old names available doesn’t break things.

>I’m not saying that there shouldn’t be change, just that there shouldn’t
be *breaking* change.

What breaking change are you referring to?

>And that a new hierarchy we define is not guaranteed to actually *stay*
better. There would be errors, but different ones. We can only expect
that the new one will be significantly better in case where either the
computing environment changed substantially or where our knowledge and
skill of how to define such a structure improved substantially.

>Also take into account whether we can adapt third party tutorials. These
are an often forgotten part of software and when they no longer work due
to chages, this seriously hampers adoption.

We don’t need to account for third party tutorials here, renaming things while keeping the old names available is a compatible change so third party tutorials remain valid (just a bit outdated in naming).

>One such case was making modules declarative by default. This broke
tutorials which described how to create a live-REPL for interactive game
development. It cost me hours of time to find the cause. And then again
when I worked on a system with only Guile 2.0 where that module-option
#:declarative #f is illegal. The compatibility change for the REPL
completely broke the code on that other system. And the deployment
system had another Guile version again, causing more breakage.

I am proposing a compatible change except for (*), not an incompatible change, but this is an example of an incompatible change instead of a compatible change, so this doesn’t seem an analogous situation unless this is about (*).

(*) different default environment for “guile -l” and the like.

And (*) can be resolved with some automatic warning messages – if things are in the (implicit) interaction environment and a binding could not be found, propose explicitly choosing (guile) as interaction environment).

p.s. forgot to mention that (define-module ...) also has default environment and hence should also be adjusted to explicitly choose _which_ environment.

Best regards,
Maxime Devos

[-- Attachment #2: Type: text/html, Size: 5117 bytes --]

GRFI [was: The Guile junk drawer and a C plea]

[Matt Wette]

How about a GRFI site to deal with proposals to Guile?

Re: GRFI [was: The Guile junk drawer and a C plea]

[Lassi Kortela]

> How about a GRFI site to deal with proposals to Guile?

I advise against starting a GRFI unless you can find plausible ways to 
solve the known problems with the SRFI process. In case you can solve 
these difficult problems which many smart people have failed to solve, 
perhaps the new process could be somehow merged with SRFI itself.

Empirically, it seems that most schemers will not watch many inboxes. 
Each separate design process will add its own inbox with subtly 
different rules and customs. That's non-trivial cognitive load.

Re: GRFI [was: The Guile junk drawer and a C plea]

[Jason Hemann]

[-- Attachment #1: Type: text/plain, Size: 1389 bytes --]

Was the GRFI proposal intended to work around the difficulties posed by a hard-fork of Guile (a la Python 2 vs. 3)? [as discussed in prior thread]

It seems to me that major breaking changes bring lots of downside, and I’m not sure the immediate benefits would outweigh them.

I think of the SRFI process as soft-standardization across different implementations, though. So, I’m not sure I follow. Absent some major language version split,  what the GRFI process would be _for_?

Best,

[PS: You—yes you—should consider writing up and submitting a paper to the upcoming Scheme Workshop https://icfp24.sigplan.org/home/scheme-2024 (July 18 deadline)]

Jason Hemann
Assistant Professor of Computer Science
Seton Hall University
On Jun 29, 2024 at 15:14 -0400, Lassi Kortela <lassi@lassi.io>, wrote:
> > How about a GRFI site to deal with proposals to Guile?
>
> I advise against starting a GRFI unless you can find plausible ways to
> solve the known problems with the SRFI process. In case you can solve
> these difficult problems which many smart people have failed to solve,
> perhaps the new process could be somehow merged with SRFI itself.
>
> Empirically, it seems that most schemers will not watch many inboxes.
> Each separate design process will add its own inbox with subtly
> different rules and customs. That's non-trivial cognitive load.
>

[-- Attachment #2: Type: text/html, Size: 2072 bytes --]

Re: GRFI [was: The Guile junk drawer and a C plea]

[Jean Abou Samra]

[-- Attachment #1: Type: text/plain, Size: 534 bytes --]

Le samedi 29 juin 2024 à 12:04 -0700, Matt Wette a écrit :
> How about a GRFI site to deal with proposals to Guile?

In the current state of Guile, even simple patches are routinely taking
months to get reviewed and merged by the maintainers, if they get replies
at all. Not assigning blame here, just stating the objective reality.
I think there would be little point in creating formal processes for
proposing large-scale changes if small-scale changes (by people other
than the maintainers) are already bottlenecked.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: The Guile junk drawer and a C plea

[Philip McGrath]

Hi,

On 6/28/24 22:52, Thompson, David wrote:
> First, I think Guile's default environment is a total mess.  It's
> the very definition of a junk drawer.  There's over 1000 names in
> the (guile) module!  Contrast this with R7RS-small's (scheme base)
> module that only has 200ish.  Guile is an old project and I'm sure
> stuff just accumulated over the years, but having so much in the
> default environment makes it hard to know what a program actually
> uses because many things that ought to be explicit imports are not.
> This makes it a challenge to move Guile in a more "least authority"
> direction.  As a rule, I think Guile should *not* add any additional
> names to the default environment without an extremely good reason.
> Because (guile) is imported implicitly, new names can cause clashes
> with existing code that require #:replace to suppress the warning
> about shadowing core bindings.  For example, the newish 'spawn'
> procedure collides with 'spawn' in (goblins core) in the Goblins
> project.  I think Guile needs a (multi-year, multi-major version)
> plan to deprecate cruft and move the good stuff into different
> modules.  Give a hoot, don't pollute (the default environment)!

On 6/29/24 06:41, Maxime Devos wrote:
> 
> The default environment is irrelevant to most Scheme libraries, since
> it is irrelevant to any library defining modules with
> ‘define-module’, ‘define-library’ or ‘library’ forms (which is pretty
> much the only reasonable way to define modules unless you are making
> your own system). This already reduces a lot of compatibility
> concerns. An important exception is anyone doing ‘eval’ or compiling
> expressions (think anyone doing the equivalent of “guile -l
> script.scm”.
> 

Unfortunately, the "default environment" in the sense of "core bindings"
definitely is relevant to libraries using `define-module`. For example,
in this module:

```
(define-module (foo)
   #:export (foo))
(define foo
   (cons 'a 1))
```

at least the bindings for `define`, `cons`, and `quote` are implicitly
imported from `(guile)`.

R6RS libraries don't have this problem. If instead you write:

```
#!r6rs
(library (foo)
   (export foo)
   (import (rnrs base))
   (define foo
     (cons 'a 1)))
```

the bindings for `define`, `cons`, and `quote` are explicitly imported
from `(rnrs base)`. If the `import` clause were empty, you would get
unbound identifier errors. (The report specifies the meaning of
`library` and its `export` and `import` sub-forms, but it also specifies
that they are not bound by any of the libraries specified by the report,
though `library` is often implemented as a binding in some sort of
implementation-specific start-up environment that is not in scope inside
a library.)

Similarly, in the Racket module:

```
(module foo racket/base
   (provide foo)
   (define foo
     (cons 'a 1)))
```

the module's language, `racket/base`, is explicitly the source of the
binding for `provide` as well as those for `define`, `cons`, and
`quote`. If you replaced `racket/base` with `typed/racket/base` or
`lazy`, you would get a valid module with different meanings for those
bindings.

Of course, you could avoid some indentation by writing the above as:

```
#lang racket/base
(provide foo)
(define foo
   (cons 'a 1))
```

Andy Wingo's "lessons learned from guile, the ancient & spry"
(<https://wingolog.org/archives/2020/02/07/lessons-learned-from-guile-the-ancient-spry>) 
concludes in part:

> But as far as next steps in language evolution, I think in the short
> term they are essentially to further enable change while further
> sedimenting good practices into Guile. On the change side, we need
> parallel installability for entire languages. Racket did a great job
> facilitating this with #lang and we should just adopt that.

I agree.

Obviously `#lang` does much more than this (we Racketeers tend to say 
`#lang` as shorthand for a bunch of complementary but mostly independent 
features), but I think a particularly important aspect is that each 
module should explicitly specify its "language"/"default 
environment"/"core bindings". If done well, this actually *avoids* the 
compatibility concerns some have raised: when you want to make a 
breaking change, you pick a new name, and things using the old name keep 
working, interoperably.

I wrote some thoughts about how Guile could gradually adopt `#lang` in a 
way that would provide for future compatibility in a sub-thread on this 
list last year: https://lists.gnu.org/r/guile-devel/2023-02/msg00075.html

My long-term pipe dream (though I have no particular plans to work on 
it), inspired by conversations with Christine, would be a grand, 
interoperable unification of Guile and Racket based on `#lang` and 
linklets: 
https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00010.html and, 
earlier, https://lists.gnu.org/archive/html/guix-devel/2021-08/msg00099.html

Philip

RE: The Guile junk drawer and a C plea

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 5673 bytes --]

>> The default environment is irrelevant to most Scheme libraries, since
> it is irrelevant to any library defining modules with
> ‘define-module’, ‘define-library’ or ‘library’ forms (which is pretty
> much the only reasonable way to define modules unless you are making
> your own system). This already reduces a lot of compatibility
> concerns. An important exception is anyone doing ‘eval’ or compiling
> expressions (think anyone doing the equivalent of “guile -l
> script.scm”.
> 

>Unfortunately, the "default environment" in the sense of "core bindings"
definitely is relevant to libraries using `define-module`. For example,
in this module:

```
(define-module (foo)
   #:export (foo))
(define foo
   (cons 'a 1))
```

at least the bindings for `define`, `cons`, and `quote` are implicitly
imported from `(guile)`.

Yes, that’s why #:pure? #true (IIRC the right keyword argument) should be used. (+ deprecation warning when #:pure? is absent?)

>R6RS libraries don't have this problem. If instead you write:

```
#!r6rs
(library (foo)
   (export foo)
   (import (rnrs base))
   (define foo
     (cons 'a 1)))
```

Here you are demonstrating how R6RS libraries have the _same_ problem. You should at least have included a version number next to (rnrs base). Who knows, maybe R8RS will rename ‘cons’ to ‘make-pair’?

As written, it is implicit which version of (rnrs base) is imported (*) – is it R6RS? Is it R7RS (no, because R7RS uses (scheme ...), but it _could_ have used (rnrs ...) instead)? A hypothetical future R8RS? (There might be (?) a rule that if no version is mentioned, the latest version available is used, but that’s its own source of incompatibilities ...)

(*) no #!r6rs does not count – that’s good for lexical syntax, but the version number of the _modules_ go into the (import ...). AFAICT, nowhere is the version number of the (rnrs ...) treated specially w.r.t. #!r6rs.

Compare this with how (implicitly) (guile) is imported – Guile doesn’t know _which_ version of the (guile) API should be used because it isn’t told (and currently, there is no option to tell Guile).

That a module is implicitly imported isn’t really a problem, what is the problem is that the _version_ (in terms of API, not implementation) is implicit.

> the bindings for `define`, `cons`, and `quote` are explicitly imported
from `(rnrs base)`. If the `import` clause were empty, you would get
unbound identifier errors. (The report specifies the meaning of
`library` and its `export` and `import` sub-forms, but it also specifies
that they are not bound by any of the libraries specified by the report,
though `library` is often implemented as a binding in some sort of
implementation-specific start-up environment that is not in scope inside
a library.)

>Similarly, in the Racket module:

```
(module foo racket/base
   (provide foo)
   (define foo
     (cons 'a 1)))
```

>the module's language, `racket/base`, is explicitly the source of the
binding for `provide` as well as those for `define`, `cons`, and
`quote`. If you replaced `racket/base` with `typed/racket/base` or
`lazy`, you would get a valid module with different meanings for those
bindings.

I could be wrong since I don’t the well how Racket is developed, but barring evidence to the contrary, I’d assume that the bindings in racket/base vary depending on the version of Racket – just like Guile’s default environment. Assuming this is true, then Racket has (in terms of implicitness) pretty much (not exactly, but _pretty much_) the same problem as Guile.

>Of course, you could avoid some indentation by writing the above as:

```
#lang racket/base
(provide foo)
(define foo
   (cons 'a 1))
```

Unless the API of racket/base never changes over time: same problem, no version number or equivalent is included.

>Andy Wingo's "lessons learned from guile, the ancient & spry"
(<https://wingolog.org/archives/2020/02/07/lessons-learned-from-guile-the-ancient-spry>) 
concludes in part:

> But as far as next steps in language evolution, I think in the short
> term they are essentially to further enable change while further
> sedimenting good practices into Guile. On the change side, we need
> parallel installability for entire languages. Racket did a great job
> facilitating this with #lang and we should just adopt that.
>
>I agree.
>
> Obviously `#lang` does much more than this (we Racketeers tend to say 
`#lang` as shorthand for a bunch of complementary but mostly independent 
features), but I think a particularly important aspect is that each 
module should explicitly specify its "language"/"default 
environment"/"core bindings". If done well, this actually *avoids* the 
compatibility concerns some have raised: when you want to make a 
breaking change, you pick a new name, and things using the old name keep 
working, interoperably.

The thing is, AFAICT using #lang does not imply “explicitly specifying the core bindings”, as mentioned previously – AFAIK Racket does not pick a new name every time the list of bindings is changed in some way.

There are benefits to have a standard way to indicate the language of a file (e.g. #lang) (and I guess if you want to you could define a bunch of fake languages that also include a couple of imported modules(and which these are depend on the precise fake language), though I don’t see why not just import those as modules), but that doesn’t mean we should repeat Racket’s mistakes with what else (and how) it also uses #lang for.

Best regards,
Maxime Devos

[-- Attachment #2: Type: text/html, Size: 9630 bytes --]

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]

Maxime Devos <maximedevos@telenet.be> writes:
> ```
> #!r6rs
> (library (foo)
>    (export foo)
>    (import (rnrs base))
>    (define foo
>      (cons 'a 1)))
> ```
> Here you are demonstrating how R6RS libraries have the _same_ problem. You should at least have included a version number next to (rnrs
> base). Who knows, maybe R8RS will rename ‘cons’ to ‘make-pair’?

Keep in mind that adding a version number does not resolve
backwards compatibility. It just moves it so source maintenance instead
of package maintenance — and then adds more complexity in keeping the
system compatible with different versions of itself.

You’ll then have to test every version of Scheme with every other
version, else you get breakage the moment a library updates to a newer
version of Scheme while another is still on the old version.

You can use methods to shift the load of backwards compatibility to
different groups, but you cannot remove it.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

RE: The Guile junk drawer and a C plea

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2896 bytes --]

>Maxime Devos <maximedevos@telenet.be> writes:
>> ```
>> #!r6rs
>> (library (foo)
>>    (export foo)
>>    (import (rnrs base))
>>    (define foo
>>      (cons 'a 1)))
>> ```
> Here you are demonstrating how R6RS libraries have the _same_ problem. You should at least have included a version number next to (rnrs
> base). Who knows, maybe R8RS will rename ‘cons’ to ‘make-pair’?

>Keep in mind that adding a version number does not resolve
backwards compatibility. It just moves it so source maintenance instead
of package maintenance — and then adds more complexity in keeping the
system compatible with different versions of itself.

>You’ll then have to test every version of Scheme with every other
version, else you get breakage the moment a library updates to a newer
version of Scheme while another is still on the old version.

No it doesn’t, because of _the version number_.  If you put (6) next to the module name, that’s the _exact_ version number, it’s not ‘6 or later’ or ‘maybe 6 whatever’, it’s ‘exactly 6’. You could load multiple versions of a module at the same time! (Not actually implemented currently, but it _could_ relatively easily be implemented.)

Also, what source maintenance? Software, that isn’t subject to changing external demands, doesn’t rot, and a _precise_ version number is included which implies the lack of changing external demand.

In terms of implementation, realistically there would be a single module version that contains the actual implementation, and various other version variants that only say ‘take these binding from that module (vN), with those renamings’ (and the occasional ‘this binding was removed / has different arguments / ..., let’s provide a small wrapper).

>You can use methods to shift the load of backwards compatibility to
different groups, but you cannot remove it.

I’m pretty sure you can remove backwards compatibility, just replace ‘cons’ by ‘make-pair’ in Guile code and see what happens.

I think it’s better though to preserve backwards compatibility, e.g. by implementing version number support for modules with a _well-defined_ API.

Note the emphasis on _well-defined_ -- if some API (in terms of semantics and not just bikeshedding over naming and argument order) is just what we have currently instead of something that is expected to remain the same (except perhaps for pure additions), and hence could easily change in the future, then version numbers are just going to lead to new problems. 

But in the case of _(guile)_, for most things in (guile) this isn’t really the case. Likewise, (rnrs ...) is pretty well-defined – there are some changes to RnRS stuff, but it’s mostly just additions, removals and reorderings – the semantics appear to have remained pretty much the same.

Best regards,
Maxime Devos

[-- Attachment #2: Type: text/html, Size: 5029 bytes --]

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 2957 bytes --]

Maxime Devos <maximedevos@telenet.be> writes:
> No it doesn’t, because of _the version number_. If you put (6) next to
> the module name, that’s the _exact_ version number, it’s not ‘6 or
> later’ or ‘maybe 6 whatever’, it’s ‘exactly 6’. You could load
> multiple versions of a module at the same time! (Not actually
> implemented currently, but it _could_ relatively easily be
> implemented.)

Yes, that’s what I mean.

And then I use a second library which gives me a comparator, so I pass
equal? from (rnrs base 7) to hash-table from (rnrs something 6).

Either this is allowed, then I just doubled my testing area, or this
is forbidden, then programs break on update of a library and need to
update all their libraries to the new dependency version.

> Also, what source maintenance? Software, that isn’t subject to changing external demands, doesn’t rot, and a _precise_ version number
> is included which implies the lack of changing external demand.

I am writing about not breaking on update.

> In terms of implementation, realistically there would be a single module version that contains the actual implementation, and various
> other version variants that only say ‘take these binding from that module (vN), with those renamings’ (and the occasional ‘this binding
> was removed / has different arguments / ..., let’s provide a small wrapper).

I’m seeing in Guix how much more complex this approach makes packaging
Haskell-programs. How Rust programs spawn 500 MiB of
source-with-dependency for small features.

>>You can use methods to shift the load of backwards compatibility to
>> different groups, but you cannot remove it.
>
> I’m pretty sure you can remove backwards compatibility

You cannot remove *the load of backwards compatibility*.

> But in the case of _(guile)_, for most things in (guile) this isn’t really the case. Likewise, (rnrs ...) is pretty well-defined – there are some
> changes to RnRS stuff, but it’s mostly just additions, removals and reorderings – the semantics appear to have remained pretty much
> the same.

Yes, and that is how it should be.

It’s when you don’t actually need version numbers. And if you added
them, you’d still have to ensure compatibility between different
versions, because libraries you use will have to be updated, even if
your own code just keeps working.

That’s why I say that the load of backwards compatibility cannot be
removed: it can only be shifted on other people.

But when the culture shifts so people say “hey, we have versioned APIs
now, let’s change everything around to fit this new style; it won’t
break existing clients (until we remove the old version)” (can you say
that you never saw people do that? I did see it), then it causes serious
problems.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

RE: The Guile junk drawer and a C plea

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 7228 bytes --]



Sent from Mail for Windows

From: Dr. Arne Babenhauserheide
Sent: Sunday, 30 June 2024 16:34
To: Maxime Devos
Cc: Philip McGrath; mikael@djurfeldt.com; Thompson, David; Richard Sent; guile-devel
Subject: Re: The Guile junk drawer and a C plea

>Maxime Devos <maximedevos@telenet.be> writes:
>> No it doesn’t, because of _the version number_. If you put (6) next to
>> the module name, that’s the _exact_ version number, it’s not ‘6 or
>> later’ or ‘maybe 6 whatever’, it’s ‘exactly 6’. You could load
>> multiple versions of a module at the same time! (Not actually
>> implemented currently, but it _could_ relatively easily be
>> implemented.)
>
>Yes, that’s what I mean.
>
>And then I use a second library which gives me a comparator, so I pass
>equal? from (rnrs base 7) to hash-table from (rnrs something 6).
>
>Either this is allowed, then I just doubled my testing area, or this
>is forbidden, then programs break on update of a library and need to
>update all their libraries to the new dependency version.

It does not double the testing area, because there is only a single combination here.
It also probably does not, because ‘equal?’ from (rnrs base (7)) would likewise be the same as (rnrs base (6)).

Also, this is allowed, because (IIRC) (rnrs something (6)) has an API where you specify _which_ hash and _which_ comparator – so it doesn’t care which comparator and hash (as long as they are compatible with each other and the comparator is, in fact, a comparator (reflexivity, transitivity, symmetry)). In particular, it breaks nothing.

This does not increase the testing area, because the testing area of (rnrs something (6)) is already infinite since it can’t care about which comparator in particular it is passed – 2*infinity = infinity for most notions of infinity. Since (rnrs something) can’t care about which comparator is used, it is redundant to test (rnrs something N) + [whatever equal?], it instead suffices to test (rnrs something N) and [whatever equal? + whatever hash] in isolation. 

There are situations where you can’t combine some things, this isn’t one of them.

I also never claimed that versioning is a panacea.

>> Also, what source maintenance? Software, that isn’t subject to changing external demands, doesn’t rot, and a _precise_ version number
>> is included which implies the lack of changing external demand.
>I am writing about not breaking on update.

What breaking of update? As mentioned previously, there is no breaking on update in the considered cases, and even if there were, there would still be breaking on update if instead of a version number the module was given a different name. At least the different version numbers give an indication of what might potentially the cause of the update.

>> In terms of implementation, realistically there would be a single module version that contains the actual implementation, and various
>> other version variants that only say ‘take these binding from that module (vN), with those renamings’ (and the occasional ‘this binding
>> was removed / has different arguments / ..., let’s provide a small wrapper).
>I’m seeing in Guix how much more complex this approach makes packaging
Haskell-programs. How Rust programs spawn 500 MiB of
source-with-dependency for small features.

This is not at all the same situation. Rust comes with entire _multiple implementation versions_ of dependencies, I’m just talking about _renaming_ and basic wrappers. It also is a problem of cargo-build-system, if you replace it by something saner (see mentions of antioxidant in guix-devel, which was mostly complete (almost everything builds!) when I last posted an update on it) plenty of cases of multiple versions disappear and sometimes dependencies disappear.

I’m not very familiar with Haskell packaging, but last time I did things with Haskell, I’m pretty sure I haven’t seen 500 MiB of ‘rename things’ packages.

Can you stop it with the strawmen?

>>>You can use methods to shift the load of backwards compatibility to
>>> different groups, but you cannot remove it.
>>
>> I’m pretty sure you can remove backwards compatibility
>
>You cannot remove *the load of backwards compatibility*.

>> But in the case of _(guile)_, for most things in (guile) this isn’t really the case. Likewise, (rnrs ...) is pretty well-defined – there are some
>> changes to RnRS stuff, but it’s mostly just additions, removals and reorderings – the semantics appear to have remained pretty much
>> the same.
>Yes, and that is how it should be.

So why do you keep bringing up the strawman of changing semantics then?

>It’s when you don’t actually need version numbers. And if you added
them, you’d still have to ensure compatibility between different
versions, because libraries you use will have to be updated, even if
your own code just keeps working.

Please remember my example of a hypothetical ‘cons’ (R6RS) -> make-pair (R8RS) transition that keeps the module name. In this case, you definitely something equivalent to a version number.

Also, no, you don’t need to update the library. That’s what the version number is for.

Also, you don’t need to ensure compatibility between different versions, because in my proposal (contrary to what appears to be your strawman), the two different versions would be the same implementation, just with different names of bindings (and maybe the occasional wrapper for reordering arguments or whatever). In this case, the two versions are trivially compatible with each other because they are the same thing.

>That’s why I say that the load of backwards compatibility cannot be
removed: it can only be shifted on other people.

This is false, you can also shift it on yourself (i.e. whoever does a bunch of renamings, also changes the version number and under the old version number adds some bindings referring to the new names in the new version number).

I guess this is technically a ‘load’, but the effort required is so minimal that this is hyperbole.

>But when the culture shifts so people say “hey, we have versioned APIs
now, let’s change everything around to fit this new style; it won’t
break existing clients (until we remove the old version)” (can you say
that you never saw people do that? I did see it), then it causes serious
problems.

If you are making up strawmen, that’s your problem, not mine.

“let’s change everything + remove old version” is an entirely different situation from  “do some reorganisation preserving semantics and only changing location/names + keep old version (referring to new version as a bunch of renamings)”.

Someone else said to remove the old version, not me.

Can you stop making strawmen and say what would be the problem with _renaming_/ _relocating_ when the old names are still preserved?

> (can you say that you never saw people do that? I did see it)

I can lie, yes. Also, don’t conflate people with me. If other people do that, you need to take it up with them, not me, because I’m not one of them

Regards,
Maxime Devos

[-- Attachment #2: Type: text/html, Size: 11926 bytes --]

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 3842 bytes --]

Maxime Devos <maximedevos@telenet.be> writes:
>> But when the culture shifts so people say “hey, we have versioned APIs
>> now, let’s change everything around to fit this new style; it won’t
>> break existing clients (until we remove the old version)” (can you say
>> that you never saw people do that? I did see it), then it causes serious
>> problems.

> Someone else said to remove the old version, not me.

It’s how this thread started, so that’s what I’m talking about.

I comment on the case without deleting them at the end.

> It also probably does not, because ‘equal?’ from (rnrs base (7)) would likewise be the same as (rnrs base (6)).
>
> Also, this is allowed, because (IIRC) (rnrs something (6)) has an API where you specify _which_ hash and _which_ comparator – so it doesn’t care
> which comparator and hash (as long as they are compatible with each other and the comparator is, in fact, a comparator (reflexivity,
> transitivity, symmetry)). In particular, it breaks nothing.

It breaks nothing right now, because people took care to keep the
semantics the same. They cared about backwards compatibility. But this
is what changes if we use versions as reason to ignore backwards
compatibility.

> This does not increase the testing area, because the testing area of
> (rnrs something (6)) is already infinite since it can’t care about
> which comparator in particular it is passed – 2*infinity = infinity
> for most notions of infinity.

This is sophistry: leaving out that there are non-infinite parts of this
infinity which are important to tested for most use-cases — and that
these non-infinite parts double.

> I also never claimed that versioning is a panacea.

Then I misunderstood you, because all I am doing is to warn, that
versioning APIs to make big changes has drawbacks and risks a cultural
shift towards breaking backwards compatibility.

I’ve seen this with Python 3. It shows again with Rust.

Scheme implementations have a pretty good history of backwards
compatibility and making that worse is what I worry about.

> Can you stop it with the strawmen?

It’s only a strawman if it isn’t real.

I am showing you examples where it leads when version numbers are
treated as a free pass to break backwards compatibility.

This is not a strawman, but the risk I see when a warning not to break
backwards compatibility is answered with “we need versions”.

>> That’s why I say that the load of backwards compatibility cannot be
>> removed: it can only be shifted on other people.
>
> This is false, you can also shift it on yourself (i.e. whoever does a bunch of renamings, also changes the version number and under the old
> version number adds some bindings referring to the new names in the new version number).
> 
> I guess this is technically a ‘load’, but the effort required is so minimal that this is hyperbole.

It sounds like this is the main point where our opinions differ. I do
not consider this effort as minimal. It may look small with the first
change, but it grows more than exponentially, because if you want to
give a guarantee, every version has to be tested against every
combination of already existing versions.

> what would be the problem with _renaming_/ _relocating_ when the old names are still preserved?

Off the cuff: You now have two sets of documentation to keep in sync
(including tutorials that no longer match the best-practices), you must
ensure that the implementations stay the same, you get code using mixed
and matched module hierarchies, dependent parts can get imported with
different names depending on which best-practices their authors
followed.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

#lang header

[Lassi Kortela]

Thank you for keeping an open line of communication between Racket and 
Scheme.

> Obviously `#lang` does much more than this (we Racketeers tend to say 
> `#lang` as shorthand for a bunch of complementary but mostly independent 
> features), but I think a particularly important aspect is that each 
> module should explicitly specify its "language"/"default 
> environment"/"core bindings". If done well, this actually *avoids* the 
> compatibility concerns some have raised: when you want to make a 
> breaking change, you pick a new name, and things using the old name keep 
> working, interoperably.
> 
> I wrote some thoughts about how Guile could gradually adopt `#lang` in a 
> way that would provide for future compatibility in a sub-thread on this 
> list last year: https://lists.gnu.org/r/guile-devel/2023-02/msg00075.html

The fundamental problem with #lang is that it comes from the language 
implementation (Racket), not from language specifications (e.g. R6RS or 
the Python Language Reference). You explain its other virtues well, but 
this problem is insurmountable.

#lang would only be a good idea if the established programming languages 
all agreed to it. When they don't, source files with #lang are limited 
to Racket. I agree that a source file should identify its own language, 
but the wider "ecosystem" will not get there in decades. If we tell 
languages what to do, we'll be stuck with unpopular languages.

Think about it: What would happen if one C compiler decided that all 
source files have to start with "#lang C" or "#lang C++"?

> My long-term pipe dream (though I have no particular plans to work on 
> it), inspired by conversations with Christine, would be a grand, 
> interoperable unification of Guile and Racket based on `#lang` and 
> linklets: 
> https://lists.gnu.org/archive/html/guix-devel/2021-10/msg00010.html and, 
> earlier, 
> https://lists.gnu.org/archive/html/guix-devel/2021-08/msg00099.html

I'm sympathetic to thinking in this general direction.

Re: The Guile junk drawer and a C plea

[Thompson, David]

Hi Mikael,

On Sat, Jun 29, 2024 at 3:38 AM Mikael Djurfeldt <mikael@djurfeldt.com> wrote:
>
> Your two points sound good.
>
> However, I personally hope that Guile will continue to be friendly towards those using it as an embedded language. There, a C API is important. There *is* a downside in moving to Scheme from that perspective, but that is fine as long as it is easy to call Scheme from C. The typical use case is when a binding in the application which use Guile as embedded language needs to manipulate Scheme data in its arguments in order to interpret them.

To clarify: I am not saying that Guile's C API should go away.  What I
am saying is that we should not introduce new standard library
procedures that are implemented in C.  Guile is already slowly
replacing C with Scheme (this is how we got suspendable ports, for
example) and I'd like to see that trend continue.

This work is being accelerated, in a way, it's just happening in the
Hoot project. Whenever we try to port a new program that calls a
(guile) procedure implemented in C, we write our own equivalent in
Scheme.  For example, I think we already have most, if not all, of
libguile/list.c covered.  We could upstream that code at some point.

> Regarding the junk, I very much agree. I also look forward to getting rid of ice-9 :). As has been spoken about here previously, I suggest that we design a new module hierarchy, introduce aliases for module bindings, and still supply the old module hierarchy during a few years for backward compatibility.

While I didn't explicitly bring up the ice-9 namespace, I am in favor
of transitioning away from it.  It confuses too many newcomers.  The
manual explains it, I think, but no one sees that and it's still a
frequently asked question.  I enjoyed reading Cat's Cradle in high
school English class as much as any US public school student, but the
fun reference isn't worth the confusion that it causes.

But to be very clear: Shrinking the exports in the (guile) module is
orthogonal to transitioning away from the ice-9 namespace.  I wouldn't
want the two conflated because the latter is much more controversial
than the former, I think.

> I don't see any harm in applying Richards patch, though, as things look right now.

I think the new procedures should simply be implemented in Scheme
rather than C. Those destructive procedures are already there, might
as well have nondestructive ones that are much more useful.  And
you'll save me porting 3 additional procedures for Hoot. ;)

- Dave

Re: The Guile junk drawer and a C plea (was: [PATCH] Add nondestructive delq1, delv1, and delete1.)

[Olivier Dion]

On Sat, 29 Jun 2024, Mikael Djurfeldt <mikael@djurfeldt.com> wrote:

[...]

> As has been spoken about here previously, I suggest that we
> design a new module hierarchy, introduce aliases for module bindings, and
> still supply the old module hierarchy during a few years for backward
> compatibility.

In the name of discoverability for new users, this must be done.
Aliasing (ice-9 *) to (guile *) for avoiding the "what's that none
standard ice-9 library?".  Also (srfi srfi-N) with something meaninful
for humans that don't want to lookup srfi.schemers.org or keep a
cheatsheet on their desk.  e.g., (srfi srfi-1) => (srfi list) or
something like that.

[...]
-- 
Olivier Dion
oldiob.ca

Library namespaces (guile ...) and (srfi ...)

[Lassi Kortela]

> Aliasing (ice-9 *) to (guile *) for avoiding the "what's that none
> standard ice-9 library?".

Sounds reasonable. "What is ice, and why version 9?"

> Also (srfi srfi-N) with something meaninful
> for humans that don't want to lookup srfi.schemers.org or keep a
> cheatsheet on their desk.  e.g., (srfi srfi-1) => (srfi list) or
> something like that.

This should be done Scheme-community-wide. There are existing 
discussions which have pretty much covered all the bases. But the 
resulting proposals have so far not been merged.

AFAIK the original proposal was SRFI 97. It is contentious because it 
prefixes numbers with colons. E.g. (srfi :1 lists).

R7RS standardizes the more natural (srfi 1) but does not mention 
anything about the mnemonic (srfi 1 lists) format.

So each gets one part right and the other part wrong.

There is also a debate about singular vs plural names. The R6RS camp 
tends to use plural, the R7RS camp singular. I looked into this once, 
and there are fewer singular names than plural names, so it might be 
easier to switch most stuff to plural (aliasing existing names).

Finally, SRFI 97 uses excessively verbose names in some cases. I 
suggested shorter names at 
https://htmlpreview.github.io/?https://github.com/pre-srfi/consistent-library-names/blob/master/survey.html 
but so far there has been no uptake.

There have been a few threads about the topic on the SRFI mailing lists. 
The latest one is 
https://srfi-email.schemers.org/srfi-discuss/msg/22363692/. The SRFI 97 
mailing list archive is at https://srfi-email.schemers.org/srfi-97/.

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1971 bytes --]

Olivier Dion <olivier.dion@polymtl.ca> writes:

> On Sat, 29 Jun 2024, Mikael Djurfeldt <mikael@djurfeldt.com> wrote:
>
> [...]
>
>> As has been spoken about here previously, I suggest that we
>> design a new module hierarchy, introduce aliases for module bindings, and
>> still supply the old module hierarchy during a few years for backward
>> compatibility.
>
> In the name of discoverability for new users, this must be done.
> Aliasing (ice-9 *) to (guile *) for avoiding the "what's that none
> standard ice-9 library?".

I remember wondering about that exactly once; never forgot the name
afterwards.

As long as ice-9 is kept, I would not mind an alias, but as long as
deprecation is on the table, I do object, because subsequent removal
would break a lot of working code and working tutorials.

Working code that needs no maintenance and would likely not receive any,
so it would just get broken and stay broken, driving people away from
Guile.

> Also (srfi srfi-N) with something meaninful
> for humans that don't want to lookup srfi.schemers.org or keep a
> cheatsheet on their desk.  e.g., (srfi srfi-1) => (srfi list) or
> something like that.

SRFI already suggests optional names, but you can use any name:

(import (srfi :1)) ;; works
(import (srfi :1 lists)) ;; works
(import (srfi :1 awesome-feature)) ;; also works, but maybe should warn

Though this is not supported by use-modules (yet?):

(use-modules (srfi srfi-1)) ;; works
(use-modules (srfi srfi-1 lists)) ;; error
(use-modules (srfi :1)) ;; error
(use-modules (srfi :1 lists)) ;; error

The correct step™ there would be to validate the name used in import
after the number against a list of accepted names.

If we were to use (srfi list), this would break the possibility to have
another srfi for lists (which is why SRFI does not do that).

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: The Guile junk drawer and a C plea

[MSavoritias]

Dr. Arne Babenhauserheide kirjoitti 17.7.2024 klo 13.45:

> Olivier Dion <olivier.dion@polymtl.ca> writes:
>
>> On Sat, 29 Jun 2024, Mikael Djurfeldt <mikael@djurfeldt.com> wrote:
>>
>> [...]
>>
>>> As has been spoken about here previously, I suggest that we
>>> design a new module hierarchy, introduce aliases for module bindings, and
>>> still supply the old module hierarchy during a few years for backward
>>> compatibility.
>> In the name of discoverability for new users, this must be done.
>> Aliasing (ice-9 *) to (guile *) for avoiding the "what's that none
>> standard ice-9 library?".
> I remember wondering about that exactly once; never forgot the name
> afterwards.
>
> As long as ice-9 is kept, I would not mind an alias, but as long as
> deprecation is on the table, I do object, because subsequent removal
> would break a lot of working code and working tutorials.
>
> Working code that needs no maintenance and would likely not receive any,
> so it would just get broken and stay broken, driving people away from
> Guile.

I mean deprecation doesn't mean it has to be removed immediately.

There will be some time before it is actually removed. We can even push 
it to Guile 4.

Given a long timeline i doubt older programs would be advised to be used 
or even work with recent guile. Given the rest of the cleaning 
discussion and any other breaking changes that is.

>> Also (srfi srfi-N) with something meaninful
>> for humans that don't want to lookup srfi.schemers.org or keep a
>> cheatsheet on their desk.  e.g., (srfi srfi-1) => (srfi list) or
>> something like that.
> SRFI already suggests optional names, but you can use any name:
>
> (import (srfi :1)) ;; works
> (import (srfi :1 lists)) ;; works
> (import (srfi :1 awesome-feature)) ;; also works, but maybe should warn
>
> Though this is not supported by use-modules (yet?):
>
> (use-modules (srfi srfi-1)) ;; works
> (use-modules (srfi srfi-1 lists)) ;; error
> (use-modules (srfi :1)) ;; error
> (use-modules (srfi :1 lists)) ;; error
>
> The correct step™ there would be to validate the name used in import
> after the number against a list of accepted names.
>
> If we were to use (srfi list), this would break the possibility to have
> another srfi for lists (which is why SRFI does not do that).
>
> Best wishes,
> Arne

Re: The Guile junk drawer and a C plea

[Lassi Kortela]

> I mean deprecation doesn't mean it has to be removed immediately.
> 
> There will be some time before it is actually removed. We can even push 
> it to Guile 4.
> 
> Given a long timeline i doubt older programs would be advised to be used 
> or even work with recent guile. Given the rest of the cleaning 
> discussion and any other breaking changes that is.

Arne's wisdom still holds. "Breaking" is bad, and "cleaning" does not 
guarantee keeping clean.

If the (guile ...) names are adopted, I suggest you keep the existing 
(ice-9 ...) libraries indefinitely but no longer advertise them in the 
manual.

BTW, 
https://www.gnu.org/software/guile/manual/html_node/Concept-Index.html 
only has one mention of ice-9.

Re: The Guile junk drawer and a C plea

[Olivier Dion]

On Wed, 17 Jul 2024, Lassi Kortela <lassi@lassi.io> wrote:
>> I mean deprecation doesn't mean it has to be removed immediately.
>> 
>> There will be some time before it is actually removed. We can even push 
>> it to Guile 4.
>> 
>> Given a long timeline i doubt older programs would be advised to be used 
>> or even work with recent guile. Given the rest of the cleaning 
>> discussion and any other breaking changes that is.
>
> Arne's wisdom still holds. "Breaking" is bad, and "cleaning" does not 
> guarantee keeping clean.
>
> If the (guile ...) names are adopted, I suggest you keep the existing 
> (ice-9 ...) libraries indefinitely but no longer advertise them in the 
> manual.

I concur.  We can keep ice-9 indefinitely for backward support but also
because it is a piece of Guile history.  Even when Guile 4 is out, I
think ice-9 should be kept to ease transition from 3 to 4.  At the same
time, major version bumping also mean API breakage .. so it might be a
good opportunity.

However, if a (guile ...) hierarchy is introduced, all mentions to
ice-9, except the historical note, ought to be removed from the manual.

[...]

-- 
Olivier Dion
oldiob.ca

Re: The Guile junk drawer and a C plea

[tomas]

[-- Attachment #1: Type: text/plain, Size: 952 bytes --]

On Wed, Jul 17, 2024 at 11:44:09AM -0400, Olivier Dion wrote:
> On Wed, 17 Jul 2024, Lassi Kortela <lassi@lassi.io> wrote:

[...]

> > If the (guile ...) names are adopted, I suggest you keep the existing 
> > (ice-9 ...) libraries indefinitely but no longer advertise them in the 
> > manual.
> 
> I concur.  We can keep ice-9 indefinitely for backward support but also
> because it is a piece of Guile history.  Even when Guile 4 is out, I
> think ice-9 should be kept to ease transition from 3 to 4.  At the same
> time, major version bumping also mean API breakage .. so it might be a
> good opportunity.

To me, this makes most sense.

> However, if a (guile ...) hierarchy is introduced, all mentions to
> ice-9, except the historical note, ought to be removed from the manual.

Well -- perhaps except an explicit mention as a "historical term" (with
index entry) to help people understand/fix old code...

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: The Guile junk drawer and a C plea

[Attila Lendvai]

i feel the urge to point out that nowadays (i.e. with tools like guix) there are new ways to "retain" backwards compatibility.

as long as one can trivially grab a guile-3-latest, or even depend on it indefinitely, then it's not such a big issue that guile-25 is not backwards compatible with guile-3 anymore.

IOW, if you don't want changes in your dependencies, then just don't update them.

nevertheless, a cost-benefit analysis still applies to refactorings, but i think the cost side is smaller than it is implied in this discussion.

(it's not an argument against retaining an ice-9 alias. and if it can be done without much of a maintenance burden, then possibly even indefinitely.)

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“In your journey to healing, you will learn to appreciate the many faceted qualities of others. Your early impressions will grow more accurate and you will use your trust more wisely. As your boundaries become more clearly defined, you will detect more quickly when others violate them. When the wounds are healed, the sharks will no longer circle.”
	— Richard Moskovitz, 'Lost in the Mirror'

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 931 bytes --]

Attila Lendvai <attila@lendvai.name> writes:

> IOW, if you don't want changes in your dependencies, then just don't update them.

This does not work.

You often have to update dependencies for security reasons. Got a new
gnutls or openssl or openssh with new cyphers you need to have a working
program — will Guile 3 get updated to support them or will you be forced
to migrate to Guile 4 to keep your tool working?

Do you want to create tools that people have to restrict to internal
networks a decade from now (anyone else thinking of companies still
bound to Windows XP?)?

Or do you want Guile tools to build a reliable path into the future
where we can consistently build upon existing work and actually stand on
the shoulders of giants?

If so, we must avoid repeatedly breaking the giants’ knees.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: The Guile junk drawer and a C plea

[Attila Lendvai]

>  > IOW, if you don't want changes in your dependencies, then just don't update them.
>  
>  This does not work.
>  
>  You often have to update dependencies for security reasons. Got a new
>  gnutls or openssl or openssh with new cyphers you need to have a working
>  program — will Guile 3 get updated to support them or will you be forced
>  to migrate to Guile 4 to keep your tool working?


fork off guile 3 into a branch, and backport those precious few security issues that you are suggesting will pop up.

and if backporting any of the fixes is too much burden, then add a warning and leave it unpatched.

it's not about destroying anything. it's about keeping engineering debt low, so that the invested human effort continues to give good yields.

or in short: it's possible to end up in an inadequate state by erring in both directions (i.e. too much reluctance for cleanup, and too much egerness for cleanup).

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 2477 bytes --]

Attila Lendvai <attila@lendvai.name> writes:

>>  > IOW, if you don't want changes in your dependencies, then just don't update them.
>>  
>>  This does not work.
>>  
>>  You often have to update dependencies for security reasons. Got a new
>>  gnutls or openssl or openssh with new cyphers you need to have a working
>>  program — will Guile 3 get updated to support them or will you be forced
>>  to migrate to Guile 4 to keep your tool working?
>
>
> fork off guile 3 into a branch, and backport those precious few
> security issues that you are suggesting will pop up.

Add modern formats you have to interact with. Websockets. HTTP3.

That’s why it does not work that way. What you’re suggesting is the path
to automatically turn working software into legacy software by creating
a constant upkeep cost to avoid becoming stale and unusable.

There are sometimes actually good reasons to break backwards
compatibility, but they are very, very few, and if you have an issue where you
think it is an actual reason to break backwards compatibility, it most
likely is not.

> it's about keeping engineering debt low, so that the invested human
> effort continues to give good yields.

You can take the easy route for that when creating a product people do
not build upon.


But Guile has the official mission to be a foundation for people to
build upon:

„Guile is the GNU Ubiquitous Intelligent Language for Extensions,
and the official extension language of the GNU project.“
— https://www.gnu.org/software/guile/

If you break backwards compatibility, you break extensions.


One of the promises of Guile is:

“Using any of the supported scripting languages, users can customize and
extend applications while they are running and see the changes take
place live!

Users can easily trade and share features by uploading and downloading
the scripts, instead of trading complex patches and recompiling their
applications.”


This only works if those scripts keep working.


That’s one reason why we have to think twice how to preserve backwards
compatibility.

You *can* usually keep engineering debt low without breaking existing
tools. Shim stuff. Build compatibility layers. And the few special
exceptions where that doesn’t work usually aren’t special enough to
break compatibility.


Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 841 bytes --]

MSavoritias <email@msavoritias.me> writes:
> There will be some time before it is actually removed.

So it will just take a bit longer until my setup gets broken.

> Given a long timeline i doubt older programs would be advised to be
> used or even work with recent guile. Given the rest of the cleaning
> discussion and any other breaking changes that is.

Do you have an idea how long lived many programs are?
That much of the high performance computing today is built on Fortran
code created decades ago?

One of the main reasons I have for using Scheme is that it is a clearly
specified language. That it stood the test of time. That it does not
just break my working tools.

Please stop recommending to destroy that.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: The Guile junk drawer and a C plea

[Greg Troxel]

I'll just add, as someone who works on packaging of guile in pkgsrc,
that we have had to have multiple versions for a long time.  There is
software useful enough for people to want, with various requirements.
Currently we have

  1.8
  2.0
  2.2
  3.0

If there were more concern about compatibility -- all 2.0 programs will
compile an work with 3.0 -- then we would not need to keep the old
versions.

  

RE: The Guile junk drawer and a C plea

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1157 bytes --]

>  1.8
>  2.0
>  2.2
>  3.0
>
>If there were more concern about compatibility -- all 2.0 programs will
>compile an work with 3.0 -- then we would not need to keep the old
>versions.

One of these changes is how #:autoload works. One of the options to preserve compatibility yet introduce the new behaviour, could have been to define ‘define-module2’ (to be used instead of the (deprecated) ‘define-module’) with the new semantics. Since their implementations would share almost all code, there wouldn’t be serious implementation costs(*). The only significant downside I see here is that ‘define-module2’ is a rather uncool name, but that’s a non-issue.

(*) I refuse to call it ‘maintenance burden’. Software isn’t hardware and doesn’t rot, in the absence of changes in external demands and absence of internal changes, the quality of software is independent of time. The ‘burden’ that would be referred to, is not some oil to applied or gears to regularly replace, but rather a cleaning up past mistakes (once per mistake) that probably could have been prevented in the first place.

Best regards,
Maxime Devos.

[-- Attachment #2: Type: text/html, Size: 2993 bytes --]

RE: The Guile junk drawer and a C plea

[Attila Lendvai]

> >If there were more concern about compatibility -- all 2.0 programs will
> 
> >compile an work with 3.0 -- then we would not need to keep the old
> 
> >versions.
> 
> One of these changes is how #:autoload works. One of the options to preserve compatibility yet introduce the new behaviour, could have been to define ‘define-module2’ (to be used instead of the (deprecated) ‘define-module’) with the new semantics. Since their implementations would share almost all code, there wouldn’t be serious implementation costs(*). The only significant downside I see here is that ‘define-module2’ is a rather uncool name, but that’s a non-issue.


i'd argue with the statement that an aesthetic glitch is a non-issue.

the short version: see the Broken Window phenomenon, and the Turing Tarpit.

the longer version:

you do not see how many potential contributors end up never touching the codebase because of stuff like `define-module2`. i know how i operate, and i've seen great coders exhibit similar behavior. i also participated in a few discussions about how and why we got our impressions and made our decisions.

there are various signs of disorder; a few major ones that quickly come to mind:

 - a messy filesystem structure

 - badly chosen names for abstractions

 - a lot of work for M-x whitespace-cleanup

 - inconsystent code formatting

 - lack of code comments next to kludges. kludges are ok, but
   uncommented kludges are a big red sign.

 - no attention for failing as early and as loudly as possible.

 - a lot of DWIM stuff, where e.g. some names are unnecessarily
   generated, which makes navigating (grep'ping) the codebase
   hopeless.  IOW, unnecessary hindrance for code discoverability.

these things give a quick impression about how much effort and annoyance it would be to delve into the codebase, and how quickly the invested effor would yield a bugfix, or a new feature, or answers to questions. and people decide based on this impression. and there's practically zero feedback about the number of people deciding *not* to contribute based on such impressions.

in my understanding of how these things work, this is very much *not* a non-issue.

and this is the reason i spoke up in this thread, not to argue for indiscriminate cleanups. retaining an ice-9 compatibility falls into the kludge category, i.e. it's ok if it's clearly marked as a kludge, and if it incurrs little extra maintenance costs.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“He alone is great and happy who fills his own station of independence, and has neither to command nor to obey.”
	— Johann Wolfgang von Goethe (1749–1832), 'With the Iron Hand' (1773), Act I

Re: [PATCH] The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 6923 bytes --]

Attila Lendvai <attila@lendvai.name> writes:

>> >If there were more concern about compatibility -- all 2.0 programs will
>> >compile an work with 3.0 -- then we would not need to keep the old
>> >versions.
>> 
>> One of these changes is how #:autoload works. One of the options to
>> preserve compatibility yet introduce the new behaviour, could have
>> been to define ‘define-module2’ (to be used instead of the
>> (deprecated) ‘define-module’) with the new semantics. Since their
>> implementations would share almost all code, there wouldn’t be
>> serious implementation costs(*). The only significant downside I see
>> here is that ‘define-module2’ is a rather uncool name, but that’s a
>> non-issue.
>
> i'd argue with the statement that an aesthetic glitch is a non-issue.
>
> the short version: see the Broken Window phenomenon, and the Turing Tarpit.

Do you know that the Broken Window theory has been debunked?
https://cssh.northeastern.edu/sccj/2019/05/21/researchers-debunk-broken-windows-theory-after-35-years/

To expand on that: none of the problems you name are connected to using
a not that beautiful name where it is *required* for backwards
compatibility.

Taking care of backwards compatibility is *to me* a sign that a project
takes the code and its users seriously, and all the signs you list are
not connected with accepting some annoyances where they are required to
keep tools working but rather with happily breaking tools that build
upon it. With choosing the easy path that’s right when seen *in isolation*
instead of the more complex one that’s right in the larger context of
people using the project.

If such breakage happens often, the project may have clean code, but its
beauty is shallow because it rejects the deeper understanding of how
others use the code and are affected by changes.

In contrast, thinking before adding an abstraction and tidying up
annoying parts without causing breakage yields a system that gives
confidence that the project will have my back for years to come.


Aside: the threads here made me realize something about Python and how
it’s used in AI stuff that I didn’t understand before. Before Python 3,
Mercurial was kind of the poster-child of Python best practices: Build
as much as possible directly in Python and only go down to C where it is
absolutely required to fulfill workflow requirements. Profile before
optimizing. Python 3 broke Mercurial horribly.

The new AI tools that form the main users of Python today are different.
They just use Python as a thin veneer for a huge C++ core. This
insulates them from breakages in Python, because the amount of complex
Python code they have is pretty small.¹

By breaking many of the tools that took a deep dive on Python to
actually follow the advertised best practices, these best practices
changed.

¹ I checked whether I’m wrong after writing this — because I might have
  been: I’m far from all-knowing, and I mostly see Python from the
  sidelines these days: I stopped being an enthusiastic Pythonista
  around 2013. But here’s a representative article about uses of Python:
  https://www.coursera.org/articles/what-is-python-used-for-a-beginners-guide-to-using-python
  - Data analysis/ML (thin veneer)
  - web development (framework²)
  - automation/scripting (thin veneer for IO and subprocesses)
  - testing/prototyping (thin veneer for compiler/bad idea³)
  - everyday tasks (thin veneer)

² with the frameworks you limit your own code and can hope that the
  framework will shield you from most language breakage. Even if it
  isn’t stable either, limiting your own code keeps maintenance cost
  low.

³ because the prototype may go into production and then it will be hit
  by the same problems that hit Mercurial.

> there are various signs of disorder; a few major ones that quickly come to mind:
>
>  - a messy filesystem structure
>  - badly chosen names for abstractions
>  - a lot of work for M-x whitespace-cleanup
>  - inconsystent code formatting
>  - lack of code comments next to kludges. kludges are ok, but
>    uncommented kludges are a big red sign.
>  - no attention for failing as early and as loudly as possible.
>  - a lot of DWIM stuff, where e.g. some names are unnecessarily
>    generated, which makes navigating (grep'ping) the codebase
>    hopeless.  IOW, unnecessary hindrance for code discoverability.

> and this is the reason i spoke up in this thread, not to argue for
> indiscriminate cleanups. retaining an ice-9 compatibility falls into
> the kludge category, i.e. it's ok if it's clearly marked as a kludge,
> and if it incurrs little extra maintenance costs.

With the pretty harsh words I wrote above, I think it’s important to say
this here again:

We agree on that. And I also think that marking kludges is important.

And that if we were to take that decision again, it would be good to
look for a better name than define-module2 (it’s too late for that, but
maybe keep this as a lesson for the future).


But I see an even worse problem which would be even easier to fix:

Lassi Kortela <lassi@lassi.io> writes:
> BTW,
> https://www.gnu.org/software/guile/manual/html_node/Concept-Index.html
> only has one mention of ice-9.

That would be a much easier explanation why ice-9 can cause problems to
newcomers. They are not told about it where they actually look.

It would be easy to state in more places "the standard library of guile
is called ice-9 (see [history])".

It would cause no breakage at all and fix the problem that newcomers
don’t know where to look.

I would suggest this as a start:


diff --git a/doc/ref/tour.texi b/doc/ref/tour.texi
index c0ecb1699..7142394a5 100644
--- a/doc/ref/tour.texi
+++ b/doc/ref/tour.texi
@@ -210,6 +210,15 @@ processing or command line parsing.  Additionally, there exist many
 Guile modules written by other Guile hackers, but which have to be
 installed manually.
 
+Most provided modules use one of three different prefixes:
+
+@itemize @bullet
+@item @code{ice-9} includes guile-specific modules: the standard library of Guile. @xref{Status, History of ice-9, History of ice-9}
+@item @code{scheme} includes modules from the RnRS standard: @url{https://standards.scheme.org/}.
+@item @code{srfi} includes Scheme Requests For Implementation; SRFI’s: @url{https://srfi.schemers.org/}.
+@end itemize
+
+
 Here is a sample interactive session that shows how to use the
 @code{(ice-9 popen)} module which provides the means for communicating
 with other processes over pipes together with the @code{(ice-9


Also we could do cleanups like merging the gc-benchmarks folder into the
benchmark-suite folder.


Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Name of the standard library

[Lassi Kortela]

> It would be easy to state in more places "the standard library of guile
> is called ice-9 (see [history])".

With no disrespect intended -- I understand it's a joke that was funny 
at one time -- "the standard library of Guile is called ice-9" sounds 
like "the unit of mass is called footballs". If so, why would a smart 
newbie learn more? One non-sequitur is likely to be followed by others.

The standard library of Guile should be called "guile", and this should 
be prominently advertised. That doesn't preclude an ice-9 alias.

Re: Name of the standard library

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1671 bytes --]

Lassi Kortela <lassi@lassi.io> writes:

>> It would be easy to state in more places "the standard library of guile
>> is called ice-9 (see [history])".
>
> With no disrespect intended -- I understand it's a joke that was funny
> at one time -- "the standard library of Guile is called ice-9" sounds
> like "the unit of mass is called footballs". If so, why would a smart
> newbie learn more?

I did.

Did you not?

(I know that this is a paradoxical question; I’m pointing to it, because
*we actually do not know*, so maybe we should refrain from discussing
the hypothetical smart newbie when all of us who are here would by
definition not match that description)

I actually liked that name — and still like it. Keep in mind that being
professional quality doesn’t require being teflon-proof naming. Python
includes `import this` and `import antigravity`. Because exposing ones
humanity isn’t a problem.


But on the topic of (guile ...) as name: I’m not sure whether (guile
...) is better. Because what then is (language ...)? What are (oop ...)
(sxml ...) and (web ...)?

Should all of these move into (guile ...)? Or should we provide the
modules without prefix? What should then actually move into (guile ...)?


I didn’t think of these before, because I didn’t start by looking at
existing code tree. Which was a mistake. These make it more dubious for
me whether a (guile ...) prefix is a good idea at all. Just aliasing
ice-9 would give the false impression that (web ...) and (language ...)
aren’t guile.


Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Name of the standard library

[Lassi Kortela]

>> If so, why would a smart newbie learn more?

> I did.
> 
> Did you not?

As a committed schemer, I forgive almost any obstacle.

> I actually liked that name — and still like it. Keep in mind that being
> professional quality doesn’t require being teflon-proof naming. Python
> includes `import this` and `import antigravity`. Because exposing ones
> humanity isn’t a problem.

Those are Easter eggs, not the default way to do everyday things.

> But on the topic of (guile ...) as name: I’m not sure whether (guile
> ...) is better. Because what then is (language ...)? What are (oop ...)
> (sxml ...) and (web ...)?
> 
> Should all of these move into (guile ...)?

IMHO they should move under (guile ...). Other Scheme implementations 
(e.g. Gauche) have the same problem: implementation-specific libraries 
are in the top-level namespace. This make it hard to figure out which 
libraries are portable, standardized, or third-party, and which ship 
with a particular Scheme implementation. I often write portable code, 
and in that context this is a clear issue.

Re: Name of the standard library

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1048 bytes --]

Lassi Kortela <lassi@lassi.io> writes:

>> But on the topic of (guile ...) as name: I’m not sure whether (guile
>> ...) is better. Because what then is (language ...)? What are (oop ...)
>> (sxml ...) and (web ...)?
>> Should all of these move into (guile ...)?
>
> IMHO they should move under (guile ...). Other Scheme implementations
> (e.g. Gauche) have the same problem: implementation-specific libraries
> are in the top-level namespace. This make it hard to figure out which
> libraries are portable, standardized, or third-party, and which ship
> with a particular Scheme implementation. I often write portable code,
> and in that context this is a clear issue.

Is anything except for (srfi ...) and (rnrs ...) expected to be
portable? I thought till now that if I want my code portable, an easy
way would be to restrict my imports to these.

What else is there that actually is portable, despite not being in
these?

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Portable code

[Lassi Kortela]

> Is anything except for (srfi ...) and (rnrs ...) expected to be
> portable? I thought till now that if I want my code portable, an easy
> way would be to restrict my imports to these.

The R6RS and R7RS library definition framework (which is broadly 
compatible across both standards) offers a realistic way to write 
portable libraries.

> What else is there that actually is portable, despite not being in
> these?

Most prominently, many of the packages at https://akkuscm.org/packages/ 
(R6RS and R7RS) and https://snow-fort.org/pkg (R7RS only).

The groundwork has been laid. It's a matter of will whether the Scheme 
community wants to push in the direction of more portability or less.

Re: Portable code

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1426 bytes --]

Lassi Kortela <lassi@lassi.io> writes:

>> Is anything except for (srfi ...) and (rnrs ...) expected to be
>> portable? I thought till now that if I want my code portable, an easy
>> way would be to restrict my imports to these.
>
> The R6RS and R7RS library definition framework (which is broadly
> compatible across both standards) offers a realistic way to write
> portable libraries.
>
>> What else is there that actually is portable, despite not being in
>> these?
>
> Most prominently, many of the packages at
> https://akkuscm.org/packages/ (R6RS and R7RS) and
> https://snow-fort.org/pkg (R7RS only).
>
> The groundwork has been laid. It's a matter of will whether the Scheme
> community wants to push in the direction of more portability or less.

Would it be possible to start into that by creating prefixes for the
different package repositories?

(akkuscm ...) and (snow-fort ...)

Maybe this could be automated for ones marked as portable.
(how do I find out whether they are portable? Do I have to try?)
(how do I find the license? In snow-fort I don’t see any license info,
 in akkuscm I see NOASSERTION for (slib minimize) but it should be BSD-3
 or public domain)

Though I see that for example (pfds ...) and (slib ...) are in both. And
akkuscm seems to import snow-fort.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Portable code

[Lassi Kortela]

> Would it be possible to start into that by creating prefixes for the
> different package repositories?
> 
> (akkuscm ...) and (snow-fort ...)

I advise against doing that, as the same package can be published to 
multiple repositories.

> (how do I find out whether they are portable? Do I have to try?)

Last I checked, you just have to try.

> (how do I find the license? In snow-fort I don’t see any license info,
>   in akkuscm I see NOASSERTION for (slib minimize) but it should be BSD-3
>   or public domain)

IIRC in Snow packages the license is declared manually in package.scm 
files, whereas Akku reads SPDX data from the source tree.

> Though I see that for example (pfds ...) and (slib ...) are in both. And
> akkuscm seems to import snow-fort.

Indeed, that is correct.

Re: Portable code

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1151 bytes --]

Lassi Kortela <lassi@lassi.io> writes:

>> Would it be possible to start into that by creating prefixes for the
>> different package repositories?
>> (akkuscm ...) and (snow-fort ...)
>
> I advise against doing that, as the same package can be published to
> multiple repositories.

Would that be a problem? All it needs is a reference that reliably works
in Guile. The akku/snow-fort equivalent for srfi.

Maybe even something like the shims in the r7rs benchmarks: they have
preludes for the different schemes with which they can run the tests.

Making a set of Guile preludes and for each package some info which
prelude is needed to run it could make the packages work out of the box
for many. And it could be generalized to many different Schemes.

>> (how do I find out whether they are portable? Do I have to try?)
>
> Last I checked, you just have to try.

Maybe we could start there for portability: create a list of packages
which work with Guile. Do you happen to have the start of such a list
from your testing?

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Portable code

[Lassi Kortela]

Arne wrote:

>>> Would it be possible to start into that by creating prefixes for the
>>> different package repositories?
>>> (akkuscm ...) and (snow-fort ...)

>> I advise against doing that, as the same package can be published to
>> multiple repositories.

> Would that be a problem? All it needs is a reference that reliably works
> in Guile. The akku/snow-fort equivalent for srfi.

I think it's not the best decomposition of the problem. At their core, 
Akku and Snow are package indexes. Pointers that say, "in order to 
obtain a copy of library X, use URL foo".

At the source code level, a Scheme library does not care where its 
dependencies are obtained from. A particular version of a particular 
package should perhaps be identified by a cryptographic hash of its 
source code. And any place you can obtain a blob with that hash is fair 
game. This is the Git model.

If we include the name of a package index in the library name, I think 
we're promoting package indexes too much. It's just not the right mental 
model, and that has secondary effects.

> Maybe even something like the shims in the r7rs benchmarks: they have
> preludes for the different schemes with which they can run the tests.
> 
> Making a set of Guile preludes and for each package some info which
> prelude is needed to run it could make the packages work out of the box
> for many. And it could be generalized to many different Schemes.

I recommend that each Scheme implementation natively support all the 
RnRS feature, and that the Scheme community come up with standard test 
suites for each RnRS edition. There is substantial work in this 
direction, IIRC championed by Will Clinger, but my memory is fuzzy.

>>> (how do I find out whether they are portable? Do I have to try?)

>> Last I checked, you just have to try.

> Maybe we could start there for portability: create a list of packages
> which work with Guile. Do you happen to have the start of such a list
> from your testing?

I haven't kept track of compatibility with any Scheme implementation.

The general principle is (or should be) that each package is a 
collection of libraries; each library imports other libraries; and any 
Scheme implementation that can support those libraries can support the 
library that depends on them. Where this is not the case, IMHO time is 
better spent fixing that than manually keeping compatibility lists.

RE: Name of the standard library

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1168 bytes --]

>Is anything except for (srfi ...) and (rnrs ...) expected to be
portable? I thought till now that if I want my code portable, an easy
way would be to restrict my imports to these.

>What else is there that actually is portable, despite not being in
these?

Maybe (scheme ...) is portable as well (idk)? (That’s for a RnRS, don’t recall the exact version).

That said, I think that in Scheme, standard is quite different from portable – if something standard is implemented, it will be (mostly) according to the standard, so in this way it is ‘portable’, but that’s a big ‘if’. For large Schemes (and small Schemes for which the RnRS or SRFI stuff is implemented in a separate library that can easily be installed(*)), I don’t expect much trouble, but for very minimalistic (or outdated Schemes, but that holds for other languages as well) you might possibly be in trouble.

(As mentioned earlier, I also want to mention that (srfi srfi-N) is not portable, it’s a Guile-ism.)

(*) and for small Schemes where the RnRS / SRFI stuff is implemented as a RnRS library importing only standard RnRS modules.

Best regards,
Maxime Devos.

[-- Attachment #2: Type: text/html, Size: 2614 bytes --]

Re: Name of the standard library

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1096 bytes --]

Maxime Devos <maximedevos@telenet.be> writes:

> That said, I think that in Scheme, standard is quite different from portable – if something standard is implemented, it will be (mostly) according
> to the standard, so in this way it is ‘portable’, but that’s a big ‘if’. For large Schemes (and small Schemes for which the RnRS or SRFI stuff is
> implemented in a separate library that can easily be installed(*)), I don’t expect much trouble, but for very minimalistic (or outdated Schemes,
> but that holds for other languages as well) you might possibly be in trouble.
>
> (As mentioned earlier, I also want to mention that (srfi srfi-N) is not portable, it’s a Guile-ism.)

Is (import (srfi :N)) portable in practice?

I switched from (use-modules ...) to (import ...) to get towards making
my code easier to re-use on different Schemes, but I didn’t actually try
using it on another Scheme implementation.

(I hope it’s OK that I pick your brain on this)

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

RE: Name of the standard library

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 775 bytes --]

>Is (import (srfi :N)) portable in practice?

I don’t know. I know that (srfi srfi-N) isn’t portable, but I don’t know what _is_ portable.  I don’t know if ‘(import ...)’ is standard either (sure it is as part of ‘define-library’, but I didn’t find it on its own in r7rs.pdf), 

That said, at least the name is standard, so it would be good to make it available under this name in Guile:

> https://srfi.schemers.org/srfi-97/srfi-97.html:
>A SRFI Library can be referenced by number, as in
>(srfi :1),

(srfi 1) is problematic, since ‘1’ is not an symbol (#{1}# is, but that’s not what has been choosen in SRFI 97).

> (I hope it’s OK that I pick your brain on this)

Sure, but on this subject it’s mostly empty.

Best regards,


[-- Attachment #2: Type: text/html, Size: 2282 bytes --]

Portable imports

[Lassi Kortela]

> I don’t know if ‘(import ...)’ is standard either (sure it is as part of 
> ‘define-library’, but I didn’t find it on its own in r7rs.pdf),

(import ...) is standard in both R6RS and R7RS, and supported by every 
serious implementation of those standards. Please spread it.

R7RS talks about "programs" and "libraries". These are technical terms 
with precise meanings.

A "program" corresponds to your typical Scheme script. IIRC it _has_ to 
start with (import ...).

A "library" is a (library ...) [in R6RS] or a (define-library ...) [in 
R7RS]. You can type (import ...) inside either.

>  > https://srfi.schemers.org/srfi-97/srfi-97.html:
> 
>  >A SRFI Library can be referenced by number, as in
> 
>  >(srfi :1),
> 
> (srfi 1) is problematic, since ‘1’ is not an symbol (#{1}# is, but 
> that’s not what has been choosen in SRFI 97).

In R7RS non-negative integers can be library name parts. Since these 
library names look natural, it would be good to backport this to R6RS 
implementations.

The colon causes endless grief when mapping library names to file names. 
For example, look at all the %3a in 
https://github.com/arcfide/chez-srfi. That's not even the worst of it.

RE: Portable imports

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1951 bytes --]

>In R7RS non-negative integers can be library name parts. Since these 
library names look natural, it would be good to backport this to R6RS 
implementations.

Then (library [...] (import (srfi 1)) [...]) would work, and since ‘library’ is (R6RS) standard and reasonably portable it would then appear that (srfi 1) is (R6RS) standard and portable, whereas it isn’t R6RS, and hence not a good idea to backport.

>The colon causes endless grief when mapping library names to file names. 
>For example, look at all the %3a in 
https://github.com/arcfide/chez-srfi. That's not even the worst of it.

I don’t think this is a problem for Guile? I don’t recall to what extent, but (srfi ...) modules are somewhat special-cased in Guile (or maybe it was integers in general for define-library) – maybe its implementation of define-library translates (srfi 1) to (srfi srfi-1) (I don’t recall the specifics). Hence, the file can simply be named “srfi/srfi-1.scm”.

For compatibility, both(**) (srfi srfi-N) (← non-standard Guile thingie) and (srfi :N) need to be supported anyway for ‘define-module’ (← Guile-specific interface), so which of them determines the file name is just a matter of convenience.

Also, AFAIK that %3a encoding isn’t necessary (and neither recognised(^)) in Guile – I don’t think Guile does any percent encoding(*). I think naming the file “srfi/:1.scm” would work fine, although it’s not something I’ve tried before. (There might be a problem with Makefile if ‘make’ doesn’t like the :, but I have some ideas for simple ways around that.)

(*) implication: you can’t have two different modules (foo/bar) and (foo bar) in Guile.
(^) (srfi %3a1) would mean the module has literally (srfi %3a1) as name.
(**) not entirely true, only supporting (srfi srfi-N) (in define-module) would be compatible, but that does not seem to be the future.

Best regards,
Maxime Devos

[-- Attachment #2: Type: text/html, Size: 3666 bytes --]

Encoding library names

[Lassi Kortela]

>  > In R7RS non-negative integers can be library name parts. Since these
> library names look natural, it would be good to backport this to R6RS
> implementations.
> 
> Then (library [...] (import (srfi 1)) [...]) would work, and since 
> ‘library’ is (R6RS) standard and reasonably portable it would then 
> appear that (srfi 1) is (R6RS) standard and portable, whereas it isn’t 
> R6RS, and hence not a good idea to backport.

For the time being, (library ...) is only available in R6RS 
implementations. But the next report (tentatively titled R7RS-large) is 
on track to be a merger of R6RS and R7RS, and hence will most likely 
support both (library ...) and (define-library ...) while merging their 
semantics in some way.

I would agree that interop between strict R6RS and other dialects of 
Scheme is important. To that end, the option to use numbers in R6RS 
library names using the : prefix is good to have. (R6RS does not have 
the vertical bar notation |123| to turn numbers into symbols, so strict 
R6RS code cannot even rely on that notation to encode numerical library 
name parts).

A further complication is that :123 is a keyword in some Scheme 
implementations. (This syntax comes from Common Lisp and Emacs Lisp, 
perhaps going as far back as Maclisp.) It might be best if any leading 
colon in a library name part is simply removed.

> > The colon causes endless grief when mapping library names to file names.
> > For example, look at all the %3a in
> > https://github.com/arcfide/chez-srfi. That's not even the worst of it.
> 
> I don’t think this is a problem for Guile? I don’t recall to what 
> extent, but (srfi ...) modules are somewhat special-cased in Guile (or 
> maybe it was integers in general for define-library) – maybe its 
> implementation of define-library translates (srfi 1) to (srfi srfi-1) (I 
> don’t recall the specifics). Hence, the file can simply be named 
> “srfi/srfi-1.scm”.
> 
> For compatibility, both(**) (srfi srfi-N) (ß non-standard Guile thingie) 
> and (srfi :N) need to be supported anyway for ‘define-module’ (ß 
> Guile-specific interface), so which of them determines the file name is 
> just a matter of convenience.
> 
> Also, AFAIK that %3a encoding isn’t necessary (and neither 
> recognised(^)) in Guile – I don’t think Guile does any percent 
> encoding(*). I think naming the file “srfi/:1.scm” would work fine, 
> although it’s not something I’ve tried before. (There might be a problem 
> with Makefile if ‘make’ doesn’t like the :, but I have some ideas for 
> simple ways around that.)

Guile should support non-negative integers in any library name, not only 
for the SRFI libraries. R7RS allows them anywhere.

I would recommend implementing the %-encoding of arbitrary UTF-8 bytes 
(several Scheme implementations do it when translating library names to 
file names) but to avoid using it in practice.

The : prefixes should be stripped before encoding a library name as a 
file name.

FAT, NTFS (and probably other file systems) do not allow colons in file 
names. I am not aware of any file system that forbids the percent sign.

> (*) implication: you can’t have two different modules (foo/bar) and (foo 
> bar) in Guile.

Percent-encoding the slash solves this easily.

> (^) (srfi %3a1) would mean the module has literally (srfi %3a1) as name.

AFAIK that is allowed by RnRS.

RE: Encoding library names

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 4621 bytes --]

>>  > In R7RS non-negative integers can be library name parts. Since these
>> library names look natural, it would be good to backport this to R6RS
>> implementations.
>> 
>> Then (library [...] (import (srfi 1)) [...]) would work, and since 
>> ‘library’ is (R6RS) standard and reasonably portable it would then 
>> appear that (srfi 1) is (R6RS) standard and portable, whereas it isn’t 
>> R6RS, and hence not a good idea to backport.

>For the time being, (library ...) is only available in R6RS 
implementations. But the next report (tentatively titled R7RS-large) is 
on track to be a merger of R6RS and R7RS, and hence will most likely 
support both (library ...) and (define-library ...) while merging their 
semantics in some way.

This is not _yet_ the case – AFAIK R7RS-large is still in progress. So, too soon to implement it for ‘library’ yet – ‘library’ is currently R6RS.

>I would agree that interop between strict R6RS and other dialects of 
Scheme is important.
> To that end, the option to use numbers in R6RS 
library names using the : prefix is good to have. (R6RS does not have 
the vertical bar notation |123| to turn numbers into symbols, so strict 
R6RS code cannot even rely on that notation to encode numerical library 
name parts).

In the case of SRFI, yes, since that’s what the relevant SRFI says the module names are, but you are formulating this much more generally.

>A further complication is that :123 is a keyword in some Scheme 
implementations. (This syntax comes from Common Lisp and Emacs Lisp, 
perhaps going as far back as Maclisp.) It might be best if any leading 
colon in a library name part is simply removed.

Guile isn’t one of those, so it’s not a problem. As I understand it, (a priori) (foo N), (foo :N) and (foo |N|) are three different module names, so this removal is simply incorrect (barring changes to RnRS). As such leading colons should be kept.

As part of a ‘module name -> file name’ mapping it seems a reasonable choice, but that’s a different matter.

(Not claiming that an implementation should in general support different (foo :N) (foo N) (foo |N|) modules, only that it should recognise them as different names.)

>> [...]
>Guile should support non-negative integers in any library name, not only 
for the SRFI libraries. R7RS allows them anywhere.

I never claimed it should be restricted to SRFI libraries. My comment was about the colon and the problems it would cause (more precisely, the lack of problems) (and about not doing it for R6RS library forms).

>I would recommend implementing the %-encoding of arbitrary UTF-8 bytes 
(several Scheme implementations do it when translating library names to 
file names) but to avoid using it in practice.

Err, no, I recommend _not_ doing that. Unicode brought us the option to be able to just type characters without special tricks, I don’t want Guile to regress to ASCII here.

Now, if Guile were to support both %-encoding (e.g. for : on Windows situations) but also supported (and preferred) just literally including the actual character in the filename (Unicode as intended, applied to file names), that would be fine, but since Guile’s current module implementation just directly maps module names to file names (+ search path), that’s currently not an option (start-up performance implications).  (I don’t think this is an unsurmountable problem, but it does require some reorganization in how Guile libraries are packaged and how Guile searches for .go/.scm.)

(Other option: %-encode only disallowed characters so there is a unique corresponding file name (modulo search paths), but which characters are disallowed depend on file system and OS, so that’s not practical.)

>The : prefixes should be stripped before encoding a library name as a 
file name.
>FAT, NTFS (and probably other file systems) do not allow colons in file 
names. I am not aware of any file system that forbids the percent sign.

NTFS supports colons just fine(*), it’s Windows that places restrictions on file names.
So, some change is indeed necessary (but not necessarily just always stripping the :, other options exist as well.)

(*) not completely sure – ‘:’ can refer to alternate data streams, but I don’t know whether that’s an NTFS or a Windows thing

>> (*) implication: you can’t have two different modules (foo/bar) and (foo 
>> bar) in Guile.
>
>Percent-encoding the slash solves this easily.

See above for how percent-encoding is a problem of its own.

Best regards,
Maxime Devos

[-- Attachment #2: Type: text/html, Size: 7563 bytes --]

Re: Portable imports

[tomas]

[-- Attachment #1: Type: text/plain, Size: 753 bytes --]

On Sat, Jul 20, 2024 at 09:18:53PM +0200, Maxime Devos wrote:

[...]

> Also, AFAIK that %3a encoding isn’t necessary (and neither recognised(^)) in Guile – I don’t think Guile does any percent encoding(*). I think naming the file “srfi/:1.scm” would work fine, although it’s not something I’ve tried before. (There might be a problem with Makefile if ‘make’ doesn’t like the :, but I have some ideas for simple ways around that.)

For civilised file systems, yes. Windows will balk at the colon.
(Windows balks at lots of funny things in file names: the colon,
I somewhat expect, from DOS inheritance, but AFAIR, "+" is also
a no-go).

I guess that's where this funky URL encoding was intended to fix.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

RE: Portable imports

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 336 bytes --]

>R7RS talks about "programs" and "libraries". These are technical terms 
with precise meanings.
>A "program" corresponds to your typical Scheme script. IIRC it _has_ to 
start with (import ...).

I’ve found it in r7rs.pdf now:

> 7.1.6. Programs and definitions
> <program> → <import declaration>+ <command or definition>+

[-- Attachment #2: Type: text/html, Size: 1611 bytes --]

Re: Name of the standard library

[Lassi Kortela]

>> That said, I think that in Scheme, standard is quite different from portable – if something standard is implemented, it will be (mostly) according
>> to the standard, so in this way it is ‘portable’, but that’s a big ‘if’. For large Schemes (and small Schemes for which the RnRS or SRFI stuff is
>> implemented in a separate library that can easily be installed(*)), I don’t expect much trouble, but for very minimalistic (or outdated Schemes,
>> but that holds for other languages as well) you might possibly be in trouble.

Indeed, Scheme doesn't have a clear separation between full-featured 
RnRS implementations and subset implementation. Several people have 
noted that this is confusing.

There are subsets of e.g. C, ML, Haskell, and Perl, but these are 
clearly indicated s such.

> Is (import (srfi :N)) portable in practice?

AFAIK:

(import (srfi :N)) is portable across all R6RS implementations.

(import (srfi N)) - without the colon - is portable across all R7RS 
implementations.

Mnemonic SRFI imports - e.g. (srfi 1 lists) - are currently not 
available in any R7RS implementation.

In R6RS land, at least some implementations support mnemonic SRFI 
imports using the form (srfi :1 lists). I'm not sure how many. There are 
discrepancies in the names (e.g. "lists") among some R6RS implementations.

It's really a mess that should be cleaned up. Any technical problems 
have been overcome - it's entirely a social issue. It should be pursued 
on the srfi-discuss mailing list.

> I switched from (use-modules ...) to (import ...) to get towards making
> my code easier to re-use on different Schemes, but I didn’t actually try
> using it on another Scheme implementation.

Thank you for making the effort.

The current crop of portable code has been written by very few schemers. 
With a few extra people we will rapidly get more impressive results. In 
my experience, the foundation provided by R6RS and R7RS is excellent and 
the few known practical problems can be solved.

https://github.com/arcfide/chez-srfi is a notable project in R6RS 
portability. Despite its name, it is not limited to Chez Scheme.

R7RS has an (include "...") mechanism which R6RS lacks. chez-srfi adds a 
similar mechanism on top of the R6RS library system. The chez-srfi 
mechanism is widely portable in practice.

Re: Name of the standard library

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 6455 bytes --]

Lassi Kortela <lassi@lassi.io> writes:

>>> That said, I think that in Scheme, standard is quite different from portable – if something standard is implemented, it will be (mostly) according
>>> to the standard, so in this way it is ‘portable’, but that’s a big ‘if’. For large Schemes (and small Schemes for which the RnRS or SRFI stuff is
>>> implemented in a separate library that can easily be installed(*)), I don’t expect much trouble, but for very minimalistic (or outdated Schemes,
>>> but that holds for other languages as well) you might possibly be in trouble.
>
> Indeed, Scheme doesn't have a clear separation between full-featured
> RnRS implementations and subset implementation. Several people have
> noted that this is confusing.

Maybe the r7rs-benchmark preludes and postludes could be a start for
documentation of that?
https://github.com/ecraven/r7rs-benchmarks/tree/master/src

Example: Guile3-prelude.scm:
https://github.com/ecraven/r7rs-benchmarks/blob/master/src/Guile3-prelude.scm

    (use-modules (ice-9 rdelim))
    (define-syntax import
      (syntax-rules ()
        ((import stuff ...)
         (begin) ;; do nothing
         )))
    (define flush-output-port force-output)
    (define current-second current-time)
    (define (jiffies-per-second) internal-time-units-per-second)
    (define current-jiffy get-internal-real-time)
    (define exact inexact->exact)
    (define inexact exact->inexact)
    (define (square x) (* x x))
    (define (write-string str out) (display str out)) ; sufficient for tail.scm
    (define (this-scheme-implementation-name) (string-append "guile3-" (version)))
    (read-enable 'r7rs-symbols)
    (print-enable 'r7rs-symbols)
    (use-modules (rnrs bytevectors)
    	     (rnrs base)
    	     (srfi srfi-9))


Could some of those be eligible to be merged into (guile-user) — the
interactive module of the REPL?

> There are subsets of e.g. C, ML, Haskell, and Perl, but these are
> clearly indicated s such.
>
>> Is (import (srfi :N)) portable in practice?
>
> AFAIK:
>
> (import (srfi :N)) is portable across all R6RS implementations.
>
> (import (srfi N)) - without the colon - is portable across all R7RS
> implementations.
>
> Mnemonic SRFI imports - e.g. (srfi 1 lists) - are currently not
> available in any R7RS implementation.
>
> In R6RS land, at least some implementations support mnemonic SRFI
> imports using the form (srfi :1 lists). I'm not sure how many. There
> are discrepancies in the names (e.g. "lists") among some R6RS
> implementations.
>
> It's really a mess that should be cleaned up. Any technical problems
> have been overcome - it's entirely a social issue. It should be
> pursued on the srfi-discuss mailing list.

Thank you for the overview!

Do you want to cross-post this part of your answer to the srfi-list?
(I think I should be on there — but I still have to finish srfi-234
before committing to additional things)

>> I switched from (use-modules ...) to (import ...) to get towards making
>> my code easier to re-use on different Schemes, but I didn’t actually try
>> using it on another Scheme implementation.
>
> Thank you for making the effort.
>
> The current crop of portable code has been written by very few
> schemers. With a few extra people we will rapidly get more impressive
> results. In my experience, the foundation provided by R6RS and R7RS is
> excellent and the few known practical problems can be solved.
>
> https://github.com/arcfide/chez-srfi is a notable project in R6RS
> portability. Despite its name, it is not limited to Chez Scheme.
>
> R7RS has an (include "...") mechanism which R6RS lacks. chez-srfi adds
> a similar mechanism on top of the R6RS library system. The chez-srfi
> mechanism is widely portable in practice.

Does that work the same as Guile’s (include "...")?

From the reference manual:

     -- Scheme Syntax: include file-name
         Open FILE-NAME, at expansion-time, and read the Scheme forms that
         it contains, splicing them into the location of the ‘include’,
         within a ‘begin’.
    
         If FILE-NAME is a relative path, it is searched for relative to the
         path that contains the file that the ‘include’ form appears in.
    
       If you are a C programmer, if ‘load’ in Scheme is like ‘dlopen’ in C,
    consider ‘include’ to be like the C preprocessor’s ‘#include’.  When you
    use ‘include’, it is as if the contents of the included file were typed
    in instead of the ‘include’ form.
    
       Because the code is included at compile-time, it is available to the
    macroexpander.  Syntax definitions in the included file are available to
    later code in the form in which the ‘include’ appears, without the need
    for ‘eval-when’.  (*Note Eval When::.)
    
       For the same reason, compiling a form that uses ‘include’ results in
    one compilation unit, composed of multiple files.  Loading the compiled
    file is one ‘stat’ operation for the compilation unit, instead of ‘2*N’
    in the case of ‘load’ (once for each loaded source file, and once each
    corresponding compiled file, in the best case).
    
       Unlike ‘load’, ‘include’ also works within nested lexical contexts.
    It so happens that the optimizer works best within a lexical context,
    because all of the uses of bindings in a lexical context are visible, so
    composing files by including them within a ‘(let () ...)’ can sometimes
    lead to important speed improvements.
    
       On the other hand, ‘include’ does have all the disadvantages of early
    binding: once the code with the ‘include’ is compiled, no change to the
    included file is reflected in the future behavior of the including form.
    
       Also, the particular form of ‘include’, which requires an absolute
    path, or a path relative to the current directory at compile-time, is
    not very amenable to compiling the source in one place, but then
    installing the source to another place.  For this reason, Guile provides
    another form, ‘include-from-path’, which looks for the source file to
    include within a load path.


Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Name of the standard library

[Lassi Kortela]

Arne wrote:

>> Indeed, Scheme doesn't have a clear separation between full-featured
>> RnRS implementations and subset implementation. Several people have
>> noted that this is confusing.

> Maybe the r7rs-benchmark preludes and postludes could be a start for
> documentation of that?
> https://github.com/ecraven/r7rs-benchmarks/tree/master/src
> 
> Example: Guile3-prelude.scm:
> https://github.com/ecraven/r7rs-benchmarks/blob/master/src/Guile3-prelude.scm
> 
>      (use-modules (ice-9 rdelim))
>      (define-syntax import
>        (syntax-rules ()
>          ((import stuff ...)
>           (begin) ;; do nothing
>           )))
>      (define flush-output-port force-output)
>      (define current-second current-time)
>      (define (jiffies-per-second) internal-time-units-per-second)
>      (define current-jiffy get-internal-real-time)
>      (define exact inexact->exact)
>      (define inexact exact->inexact)
>      (define (square x) (* x x))
>      (define (write-string str out) (display str out)) ; sufficient for tail.scm
>      (define (this-scheme-implementation-name) (string-append "guile3-" (version)))
>      (read-enable 'r7rs-symbols)
>      (print-enable 'r7rs-symbols)
>      (use-modules (rnrs bytevectors)
>      	     (rnrs base)
>      	     (srfi srfi-9))

Those might be a good start for filling in the gaps in RnRS support.

Some Scheme implemenetations have existing RnRS test suites. IIRC those 
are mostly cloned and hacked from Clinger's tests for Larceny. A test 
suite has the advantage that it is executable documentation of RnRS 
compatibility. Manual documentation is prone to mistakes.

> Could some of those be eligible to be merged into (guile-user) — the
> interactive module of the REPL?

I'm not qualified to answer that one.

>>> Is (import (srfi :N)) portable in practice?

>> AFAIK:
>>
>> (import (srfi :N)) is portable across all R6RS implementations.
>>
>> (import (srfi N)) - without the colon - is portable across all R7RS
>> implementations.
>>
>> [...]

> Thank you for the overview!
> 
> Do you want to cross-post this part of your answer to the srfi-list?

You're welcome. I think I have posted an equivalent overview to 
srfi-discuss in the past, but I can't find it.

>> R7RS has an (include "...") mechanism which R6RS lacks. chez-srfi adds
>> a similar mechanism on top of the R6RS library system. The chez-srfi
>> mechanism is widely portable in practice.

> Does that work the same as Guile’s (include "...")?
> 
>  From the reference manual:
> 
>       -- Scheme Syntax: include file-name
>           Open FILE-NAME, at expansion-time, and read the Scheme forms that
>           it contains, splicing them into the location of the ‘include’,
>           within a ‘begin’.
>      
>           If FILE-NAME is a relative path, it is searched for relative to the
>           path that contains the file that the ‘include’ form appears in.
> [...]

Yes, Guile's include sounds very similar to R7RS include. There may be 
obscure technical differences, but the intent is exactly the same.

RE: Name of the standard library

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1080 bytes --]

>But on the topic of (guile ...) as name: I’m not sure whether (guile
...) is better. Because what then is (language ...)? What are (oop ...)
(sxml ...) and (web ...)?

>Should all of these move into (guile ...)? Or should we provide the
modules without prefix? What should then actually move into (guile ...)?

>I didn’t think of these before, because I didn’t start by looking at
existing code tree. Which was a mistake. These make it more dubious for
me whether a (guile ...) prefix is a good idea at all. Just aliasing
ice-9 would give the false impression that (web ...) and (language ...)
aren’t guile.

Why not move (language xyz) into (guile language xyz) as well?

I’m not sure about all the top-level module thingies though, sometimes other implementations implement the same thing too. For example, Racket has an SXML implementation. If the API is compatible (and located under the same module name) (I don’t know if this is the case), keeping it under (sxml ...) would make sense. (If not, (guile sxml ...)?)

Best regards,
Maxime Devos

[-- Attachment #2: Type: text/html, Size: 4852 bytes --]

Re: Name of the standard library

[Lassi Kortela]

> Why not move (language xyz) into (guile language xyz) as well?

+1

> I’m not sure about all the top-level module thingies though, sometimes 
> other implementations implement the same thing too. For example, Racket 
> has an SXML implementation. If the API is compatible (and located under 
> the same module name) (I don’t know if this is the case), keeping it 
> under (sxml ...) would make sense. (If not, (guile sxml ...)?)

IMHO (sxml ...) would only be the best name for the library if the 
entire Scheme community agreed on what the API to SXML should be.

The tradition is that (scheme ...) and (rnrs ...) contain the libraries 
that are agreed upon in the Scheme reports, and (srfi ...) contains 
SRFIs. We don't (yet) have other standard namespaces, but perhaps we could.

The tradition in e.g. Python and C, where the standard library puts 
random libraries into the top-level namespace, starts from a culture 
that is dominated by one implementation. Scheme is the opposite.

Re: Name of the standard library

[MSavoritias]

Dr. Arne Babenhauserheide kirjoitti 20.7.2024 klo 17.52:
> Lassi Kortela <lassi@lassi.io> writes:
>
>>> It would be easy to state in more places "the standard library of guile
>>> is called ice-9 (see [history])".
>> With no disrespect intended -- I understand it's a joke that was funny
>> at one time -- "the standard library of Guile is called ice-9" sounds
>> like "the unit of mass is called footballs". If so, why would a smart
>> newbie learn more?
> I did.
>
> Did you not?
>
> (I know that this is a paradoxical question; I’m pointing to it, because
> *we actually do not know*, so maybe we should refrain from discussing
> the hypothetical smart newbie when all of us who are here would by
> definition not match that description)
>
> I actually liked that name — and still like it. Keep in mind that being
> professional quality doesn’t require being teflon-proof naming. Python
> includes `import this` and `import antigravity`. Because exposing ones
> humanity isn’t a problem.

Just a note here, I am that newbie :)

And guile is much easier than whatever inside joke "ice-9" is.

And no I don't need to "prove" myself by going through learning whatever 
ice-9 is.

We should make it easier to learn guile over time. I strongly disagree 
forcing newbies to go through the same things we did just to make a point.


Regards,

MSavoritias

>
> But on the topic of (guile ...) as name: I’m not sure whether (guile
> ...) is better. Because what then is (language ...)? What are (oop ...)
> (sxml ...) and (web ...)?
>
> Should all of these move into (guile ...)? Or should we provide the
> modules without prefix? What should then actually move into (guile ...)?
>
>
> I didn’t think of these before, because I didn’t start by looking at
> existing code tree. Which was a mistake. These make it more dubious for
> me whether a (guile ...) prefix is a good idea at all. Just aliasing
> ice-9 would give the false impression that (web ...) and (language ...)
> aren’t guile.
>
>
> Best wishes,
> Arne

Re: Name of the standard library

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 2820 bytes --]

MSavoritias <email@msavoritias.me> writes:

> Dr. Arne Babenhauserheide kirjoitti 20.7.2024 klo 17.52:
>> Lassi Kortela <lassi@lassi.io> writes:
>>>> It would be easy to state in more places "the standard library of guile
>>>> is called ice-9 (see [history])".
>>> With no disrespect intended -- I understand it's a joke that was funny
>>> at one time -- "the standard library of Guile is called ice-9" sounds
>>> like "the unit of mass is called footballs". If so, why would a smart
>>> newbie learn more?
>> (I know that this is a paradoxical question; I’m pointing to it, because
>> *we actually do not know*, so maybe we should refrain from discussing
>> the hypothetical smart newbie when all of us who are here would by
>> definition not match that description)
> Just a note here, I am that newbie :)

By definition of the smart newbie given in the thread, you can’t be
that, because you did keep learning :-)

(but that’s just a tongue in cheek answer …)

> We should make it easier to learn guile over time.

I agree. I just disagree that this is the best way for that.

> I strongly disagree forcing newbies to go through the same things we
> did just to make a point.

That’s not the goal.

The goal is to avoid breaking existing tools, tutorials, and skills.


Is it worse to learn about ice-9, srfi, web, and rnrs,
or to learn from one tutorial about ice-9 and web,
and from the other about guile and (guile web)?


I’ve written and published a book about Guile.¹ The printed copies and
downloaded PDFs will for ever and ever include ice-9, because I cannot
change it. I could not, if I wanted. So if we change from ice-9 to
guile, that should have higher benefit than the cost of inconsistency
between old and new documentation.

Changing best practices causes soft trauma² to those who learned the old
ones.


And we inherently have different main namespaces: rnrs and srfi are not
going away. They are deeply engrained in our infrastructure.

Additionally we’d have to ask whether these should actually be in
(guile ...). (ice-9 match) is actually from chibi-scheme, just with
three shims added so the upstream code runs unmodified. Calling that
(guile ...) would falsely imply that it is implementation specific.


The deeper I look into it, the less I think that a (guile ...) module
would be a good idea.

Instead I now think that we need to check our documentation why people
don’t find the different namespaces early and effortlessly enough that
they are obvious.


¹ https://www.draketo.de/py2guile
² https://drewdevault.com/2019/11/26/Avoid-traumatic-changes.html
  — another one who got hit by Python 3


Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Library names describe APIs

[Lassi Kortela]

Arne wrote:

> Additionally we’d have to ask whether these should actually be in
> (guile ...). (ice-9 match) is actually from chibi-scheme, just with
> three shims added so the upstream code runs unmodified. Calling that
> (guile ...) would falsely imply that it is implementation specific.
> 
> The deeper I look into it, the less I think that a (guile ...) module
> would be a good idea.

IMHO, a library name fundamentally specifies an API.

(import (scheme ...)) and (import (rnrs ...)) are stuff that is 
API-compatible with the Scheme reports. (import (srfi ...)) is similar.

(import (guile ...)) and (import (chibi ...)) would import stuff where 
the Guile and Chibi communities, respectively, are responsible for the 
API. Even when other Scheme implementations provide a library by that name.

I have more to say about this, but since this thread is massive as it 
is, I'll leave that for later.

Re: Name of the standard library

[Ricardo Wurmus]

MSavoritias <email@msavoritias.me> writes:

>> I actually liked that name — and still like it. Keep in mind that being
>> professional quality doesn’t require being teflon-proof naming. Python
>> includes `import this` and `import antigravity`. Because exposing ones
>> humanity isn’t a problem.
>
> Just a note here, I am that newbie :)
>
> And guile is much easier than whatever inside joke "ice-9" is.
>
> And no I don't need to "prove" myself by going through learning
> whatever ice-9 is.

Maybe I'm just pathologically uncurious, but I just "learned" it as an
opaque name and moved on.

Many years later I read "Cat's Cradle", which was very enjoyable, but
did not do anything to teach me about "ice-9".  (It pushed me into a
short Vonnegut reading frenzy; would recommend.)  There really is
nothing to *learn* here.

> We should make it easier to learn guile over time. I strongly disagree
> forcing newbies to go through the same things we did just to make a
> point.

There are countless obstacles when learning Guile (whether you're coming
from some other programming language or if Guile is your first).  The
name "ice-9" really isn't one of them.

-- 
Ricardo

Re: Name of the standard library

[Attila Lendvai]

> > With no disrespect intended -- I understand it's a joke that was funny
> > at one time -- "the standard library of Guile is called ice-9" sounds
> > like "the unit of mass is called footballs". If so, why would a smart
> > newbie learn more?
> 
> 
> I did.
> 
> Did you not?


yeah, because guix. so i wouldn't draw too many conclusions from this fact alone.

as a seasoned CL hacker, and a naive scheme newbie, i once even asked why guix is not made portable on top of other schemes. my motivation was to get better debugging facilities than what guile provides. it was so far from my expectations that i considered it easier to port guix to another scheme.

(there's no need to educate me, though. since then i learned the differences between the CL and scheme worlds, and understood the effort it would take.)

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“When law and morality contradict each other, the citizen has the cruel alternative of either losing his moral sense or losing his respect for the law.”
	— Frédéric Bastiat (1801–1850), 'The Law' (1850)

RE: Name of the standard library

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1287 bytes --]

>> It would be easy to state in more places "the standard library of guile
>> is called ice-9 (see [history])".

>With no disrespect intended -- I understand it's a joke that was funny 
at one time -- "the standard library of Guile is called ice-9" sounds 
like "the unit of mass is called footballs". If so, why would a smart 
newbie learn more? One non-sequitur is likely to be followed by others.

Why wouldn’t this smart newbie learn more? I mean, in an alternate universe, the CIPM and or their predecessors might have liked soccer (cf. Ice-9) very much and called the unit of mass the ‘football’ instead of the ‘(kilo)gram’. In that universe, “the unit of mass is called footballs” would be a perfectly reasonable sentence (not a non-sequitur) that doesn’t prevent (SI) measurement newbies from learning more about the other SI units – the units don’t need to share a name with SI.

In case of Guile, (guile ...) seems a bit better than (ice-9 ...) (in cases where compatibility with other Schemes is attempted), but the name (ice-9 ...) hardly seems an obstacle to me – you need to name it _something_, so mentioning in the documentation what this _something_ is seems pretty reasonable to me (not a non-sequitur).

Best regards,
Maxime Devos.

[-- Attachment #2: Type: text/html, Size: 2678 bytes --]

Re: Name of the standard library

[Lassi Kortela]

> Why wouldn’t this smart newbie learn more?

Many things are competing for his time.

> I mean, in an alternate 
> universe, the CIPM and or their predecessors might have liked soccer 
> (cf. Ice-9) very much and called the unit of mass the ‘football’ instead 
> of the ‘(kilo)gram’. In that universe, “the unit of mass is called 
> footballs” would be a perfectly reasonable sentence (not a non-sequitur) 
> that doesn’t prevent (SI) measurement newbies from learning more about 
> the other SI units – the units don’t need to share a name with SI.

While reality is indeed a fiction, it is a shared fiction. That's why 
the list type is "list", the unit of mass is "kilogram", etc.

Alternate universes are a distinct hobby. In programming, that hobby is 
represented by esoteric languages.

RE: Name of the standard library

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1357 bytes --]

>> Why wouldn’t this smart newbie learn more?
>Many things are competing for his time.

That’s always the case. What I meant was, what reason would this smart newbie have to not learn more that has to do with the statement “the standard library of Guile is name ice-9”?

> I mean, in an alternate 
> universe, the CIPM and or their predecessors might have liked soccer 
> (cf. Ice-9) very much and called the unit of mass the ‘football’ instead 
> of the ‘(kilo)gram’. In that universe, “the unit of mass is called 
> footballs” would be a perfectly reasonable sentence (not a non-sequitur) 
> that doesn’t prevent (SI) measurement newbies from learning more about 
> the other SI units – the units don’t need to share a name with SI.

>While reality is indeed a fiction, it is a shared fiction. That's why 
the list type is "list", the unit of mass is "kilogram", etc.

>Alternate universes are a distinct hobby. In programming, that hobby is 
represented by esoteric languages.

Yes, and in this alternate universe, the shared fiction would be naming the unit of mass the ‘football’. This alternate universe illustrates that a name can simply be name (even when it also is other things). That alternate universes are a hobby doesn’t prevent them from being illustrative.

Best regards,
Maxime Devos.

[-- Attachment #2: Type: text/html, Size: 2968 bytes --]

Re: [PATCH] The Guile junk drawer and a C plea

[Attila Lendvai]

> Do you know that the Broken Window theory has been debunked?
> https://cssh.northeastern.edu/sccj/2019/05/21/researchers-debunk-broken-windows-theory-after-35-years/


there's no need for scientific papers about something i can observe myself. both inside me, in my own reactions and judgments, and also in external reality, in the behavior of other people.

and re debunking: The Science™ has reached a point where anything and their opposite has a peer reviewed publication now, especially in social sciences where replicability is...

but anyway, i don't have a dog in this fight.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“He that would make his own liberty secure, must guard even his enemy from oppression; for if he violates this duty, he establishes a precedent that will reach to himself.”
	— Thomas Paine (1737–1809)

Re: [PATCH] The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 2359 bytes --]

Attila Lendvai <attila@lendvai.name> writes:

>> Do you know that the Broken Window theory has been debunked?
>> https://cssh.northeastern.edu/sccj/2019/05/21/researchers-debunk-broken-windows-theory-after-35-years/
>
> there's no need for scientific papers about something i can observe myself. both inside me, in my own reactions and judgments, and also in external reality, in the behavior of other people.

Then I don’t know whether you speak of the actual broken windows theory.

That theory says: if there are small problems like broken windows, crime
will go up, so being heavy handed even for the smallest kind of disorder
will reduce serious crimes.

That’s the broken windows theory. It was the basis for much of the
brutal law enforcement in the USA.

And it is wrong.

When people looked into the actual crime data, there was no evidence
that accepting small disturbances causes a higher rate serious
disturbances.

There is an actual effect of accepting violations: when a specific rule
is violated the first time and that gets tolerated *that specific rule*
stops to feel binding to other people.

> and re debunking: The Science™ has reached a point where anything and their opposite has a peer reviewed publication now, especially in social sciences where replicability is...

Despite the often poor replicability of social sciences, that’s not
true. There is observable reality, and that shows that the broken
windows theory is wrong.

> but anyway, i don't have a dog in this fight.

When it comes to general statements against science, I do.

I know the amount of work that goes into even a single publication, how
much more dilligence, effort, and skill that takes than writing a
newspaper article. How careful most (though sadly not all) people are
with their statements in publications. I only published two papers
myself and reviewed a few more, but that experience showed me the
difference between a scientific publication and just throwing some
statements into the web.

Also social sciences have ramped up their replication studies in recent
years. Because most scientists actually care about checking.

Social sciences are just under much worse financial pressure than STEM.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: [PATCH] The Guile junk drawer and a C plea

[Attila Lendvai]

> > > Do you know that the Broken Window theory has been debunked?
> > > https://cssh.northeastern.edu/sccj/2019/05/21/researchers-debunk-broken-windows-theory-after-35-years/
> > 
> > there's no need for scientific papers about something i can observe myself. both inside me, in my own reactions and judgments, and also in external reality, in the behavior of other people.
> 
> 
> Then I don’t know whether you speak of the actual broken windows theory.
> 
> That theory says: if there are small problems like broken windows, crime
> will go up, so being heavy handed even for the smallest kind of disorder
> will reduce serious crimes.


this is just a specific (mis)application of the general idea.

the general idea of the broken window phenomenon (at least as it's used in my circles) is that if there's a building that is seemingly not maintained (e.g. a broken window is left unrepaired for a critical length of time), then that building will deteriorate with an increasing speed compared to buildings that receive just a baseline level of maintenance. (e.g. the other windows will typically get broken by human action).

put differently, if it's not taken care of (i.e. if this is not someone's property) then for some people it's an invitation for a free-for-all.

and in a codebase: if i don't see the signs of a careful local gardener, then why would i put much effort into improving it as a visitor, or why would i even visit that specific garden among the many?


> > but anyway, i don't have a dog in this fight.
> 
> 
> When it comes to general statements against science, I do.


i didn't mean to say something against science (the methodology), but rather against The Science™ (i.e. contemporary academia and publishing gatekeepers, with its p hacking, citation circles, defunding as punishment, straight out censorship, etc... generally, giving up truth-seeking for delivering results based on political/financial agendas).


> I know the amount of work that goes into even a single publication, how
> much more dilligence, effort, and skill that takes than writing a
> newspaper article. How careful most (though sadly not all) people are
> with their statements in publications. I only published two papers
> myself and reviewed a few more, but that experience showed me the
> difference between a scientific publication and just throwing some
> statements into the web.


which is respectable and admirable behavior! and many of the scientists, individually, still pursue truth-seeking, regardless of where that may lead. but sadly this cannot be said about the institutions, i.e. about the scientific cooperation of the individual scientists.

the emergent behavior of the system has been successfully hijacked by politics. and this won't change until we implement censorship resistant publishing, and solve the decentralized financing of scientists. (pseudonymity may also be needed in certain fields, but lasting pseuconimity is a much harder nut to crack)

and let me finish with a hand-picked quote this time:

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“It is dangerous to be right in matters on which the established authorities are wrong.”
	— Voltaire (1694–1778), 'The Age of Louis XIV'

RE: The Guile junk drawer and a C plea

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 4544 bytes --]

> >If there were more concern about compatibility -- all 2.0 programs will
> 
> >compile an work with 3.0 -- then we would not need to keep the old
> 
> >versions.
> 
> One of these changes is how #:autoload works. One of the options to preserve compatibility yet introduce the new behaviour, could have been to define ‘define-module2’ (to be used instead of the (deprecated) ‘define-module’) with the new semantics. Since their implementations would share almost all code, there wouldn’t be serious implementation costs(*). The only significant downside I see here is that ‘define-module2’ is a rather uncool name, but that’s a non-issue.

>i'd argue with the statement that an aesthetic glitch is a non-issue.

>the short version: see the Broken Window phenomenon, and the Turing Tarpit.

'define-module2’ and ‘define-module’ aren’t Turing tarpits:

• Neither are Turing-complete (unless perhaps you do fancy stuff with merge-generics and custom (GOOPS) method metaclasses, but then it was your own choice to turn it into a Turing-tarpit).
• Both are (relatively speaking) easy to use. A slight difficulty for beginners is determining where in the file system to put the module (hence the relatively qualifier), but both define-module and define-module2 have the exact same problem, and AFAIK other programming languages aren’t any better at this (hence ‘relatively’).

‘define-module2’ is not a broken window, it is a fixed window. Neither is ‘define-module’, it would be slightly outdated in its precise semantics (and hence not recommended), but it does work decently.

The broken window here, is breaking the backwards incompatibility by _not_ doing ‘define-module2’ or the like. (Doesn’t need to be ‘define-module2’ per se, other methods for preserving compatibility exist as well.) ‘define-module2’ would be the replacement glass (for the broken window) here.

>the longer version:

>you do not see how many potential contributors end up never touching the codebase because of stuff like `define-module2`. i know how i operate, and i've seen great coders exhibit similar behavior. i also participated in a few discussions about how and why we got our impressions and made our decisions.

Neither do you (or me, for that matter) see how many potential contributors never touch Guile, or stop touching Guile, or stay touching Guile but are frustrated about Guile because of the _lack_ of stuff like ‘define-module2’. That ‘2’-suffix can be quite useful (for compatibility).

Also, almost by definition, those ‘great coders’ you are mentioning, aren’t actually great because of their lack of appreciation of backwards compatibility (*). Going by what I’m hearing about these coders, I think I prefer _not_ having them touch the codebase.

(*) In the past I’ve sometimes argued for _not_ preserving backwards compatibility, but my opinion changed over time and this is not the same situation.

>there are various signs of disorder; a few major ones that quickly come to mind:
>
> - a messy filesystem structure
>
> - badly chosen names for abstractions
>
> - a lot of work for M-x whitespace-cleanup
>
> - inconsystent code formatting
>
> - lack of code comments next to kludges. kludges are ok, but
>  uncommented kludges are a big red sign.
>
> - no attention for failing as early and as loudly as possible.
>
> - a lot of DWIM stuff, where e.g. some names are unnecessarily
>  generated, which makes navigating (grep'ping) the codebase
>   hopeless.  IOW, unnecessary hindrance for code discoverability.

None of these is the hypothetical ‘define-module2’. The only thing that comes close is ‘badly chosen name for abstractions’, but well, ‘define-module2’ would define a module, and it is the second (*) incompatible version of this method to defining modules. As such it seems a pretty decent choice of name (just a bit boring).

(*) or maybe third, I’m assuming things here about Guile <2.2 that I don’t actually know, the hypothetical ‘define-module2’ / ‘define-module’ is just an example.

> [...]
>
> and this is the reason i spoke up in this thread, not to argue for indiscriminate
> cleanups. retaining an ice-9 compatibility falls into the kludge category, i.e. it's ok if
> it's clearly marked as a kludge, and if it incurrs little extra maintenance costs.

Nowhere did I say anything about not properly deprecating ‘define-module’.

Best regards,
Maxime Devos

[-- Attachment #2: Type: text/html, Size: 12088 bytes --]

Re: The Guile junk drawer and a C plea

[Lassi Kortela]

> (import (srfi :1)) ;; works
> (import (srfi :1 lists)) ;; works

The colon in :1 is a hack due to a limitation in the R6RS spec.

I recommend that Guile support both 1 and :1 (as well as the symbol |1|) 
by internally converting to some normal form.

> (import (srfi :1 awesome-feature)) ;; also works, but maybe should warn

That should raise an error.

> (use-modules (srfi srfi-1)) ;; works
> (use-modules (srfi srfi-1 lists)) ;; error

The (srfi srfi-1) notation is idiosyncratic to Guile, and confuses 
schemers coming from any other implementation.

> The correct step™ there would be to validate the name used in import
> after the number against a list of accepted names.

+1

I recommend a general mechanism for library aliases.

> If we were to use (srfi list), this would break the possibility to have
> another srfi for lists (which is why SRFI does not do that).

Indeed. This is deliberate.

Re: The Guile junk drawer and a C plea (was: [PATCH] Add nondestructive delq1, delv1, and delete1.)

[Jean Abou Samra]

[-- Attachment #1: Type: text/plain, Size: 189 bytes --]

Le vendredi 28 juin 2024 à 22:52 -0400, Thompson, David a écrit :
> Who actually wants to use that C API?

lilypond $ git grep '\bscm_' '**/*.cc' '**/*.yy' '**/*.ll' | wc -l
3961


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: The Guile junk drawer and a C plea

[Richard Sent]

Hi David and all,

"Thompson, David" <dthompson2@worcester.edu> writes:

> As a rule, I think Guile should *not* add any additional names to the
> default environment without an extremely good reason. Because (guile)
> is imported implicitly, new names can cause clashes with existing code
> that require #:replace to suppress the warning about shadowing core
> bindings. For example, the newish 'spawn' procedure collides with
> 'spawn' in (goblins core) in the Goblins project. I think Guile needs
> a (multi-year, multi-major version) plan to deprecate cruft and move
> the good stuff into different modules. Give a hoot, don't pollute (the
> default environment)!
>
> Second, please please please, no more C! Guile's substantial amount of
> C code is a legacy of its origins decades ago, and we need to make it
> clear to new users and contributors that new code should be written in
> Scheme!

I see my little patch has ignited a comparatively oversized firestorm.
Oops. ;)

The current discussion is above my paygrade so I'll largely stay out of
it. (Although it's quite fascinating!)

In the specific context of this patch I feel like the risks of namespace
pollution and C maintenance burden are low since both are near identical
to what already exists. But of course, someone else may argue that this
is a death by a thousand cuts situation.

I will suggest that if a consensus is reached on what new code must
adhere to (e.g. no (guile) namespace pollution, no new procedures
written in C), it should be documented in the reference manual. There
are traces of this but nothing concrete [1]. What little documentation
exists is secreted away into the darkest corners of Texinfo. I believe a
top-level "Contributing" section in a similar vein to GNU Guix would be
a valuable addition.

It's quite possible as a first-time contributor I am missing something
that already exists. If so, oops again!

[1]: (guile) Status

-- 
Take it easy,
Richard Sent
Making my computer weirder one commit at a time.

Re: The Guile junk drawer and a C plea

[Thompson, David]

Hi Richard,

On Sat, Jun 29, 2024 at 3:02 PM Richard Sent
<richard@freakingpenguin.com> wrote:
>
> I see my little patch has ignited a comparatively oversized firestorm.
> Oops. ;)

I wasn't sure if my email would get 0 replies or a lot of replies...
but the latter happened. I just want to be clear that this is *not* a
judgement of your patch or saying that you made a mistake! A big
reason why I changed the subject line is because I wanted to separate
this discussion from the review of your patch. Thank you for sending a
patch to improve Guile!! And thank you for bringing some important
issues back to the front of my mind. :)

> The current discussion is above my paygrade so I'll largely stay out of
> it. (Although it's quite fascinating!)
>
> In the specific context of this patch I feel like the risks of namespace
> pollution and C maintenance burden are low since both are near identical
> to what already exists. But of course, someone else may argue that this
> is a death by a thousand cuts situation.

I agree that the risks are low in this case. It doesn't make sense to
have destructive list operations without nondestructive equivalents,
especially since mutation of pairs is strongly discouraged amongst
seasoned Schemers.  If it were my call (and it's not, to be clear),
I'd say that they should just be implemented in Scheme, at least.
There really is no good reason to implement Scheme procedures in C
anymore.

> I will suggest that if a consensus is reached on what new code must
> adhere to (e.g. no (guile) namespace pollution, no new procedures
> written in C), it should be documented in the reference manual. There
> are traces of this but nothing concrete [1]. What little documentation
> exists is secreted away into the darkest corners of Texinfo. I believe a
> top-level "Contributing" section in a similar vein to GNU Guix would be
> a valuable addition.

Right, there is no consensus about this right now. It's been talked
about vaguely on IRC and in person over the years (maybe the mailing
list, too, but I don't feel like checking). I'm just prodding the
issue again (uh oh, did I prod a hornet's nest?) to see if it creates
some positive momentum.

> It's quite possible as a first-time contributor I am missing something
> that already exists. If so, oops again!

Nope! Not missing a thing. There's literally no way for you to know
that there's a desire amongst some Guilers to discourage implementing
new standard library procedures in C and expanding the default
namespace.

I really hope I haven't discouraged you from future contributions by
using your patch as a case study. Thank you again for hacking on
Guile!

- Dave

Re: The Guile junk drawer and a C plea

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 2273 bytes --]

"Thompson, David" <dthompson2@worcester.edu> writes:
> <richard@freakingpenguin.com> wrote:
>> I will suggest that if a consensus is reached on what new code must
>> adhere to (e.g. no (guile) namespace pollution, no new procedures
>> written in C), it should be documented in the reference manual.
>
> Right, there is no consensus about this right now.

If there were consensus, there would be less talk about it :-)

>> In the specific context of this patch I feel like the risks of namespace
>> pollution and C maintenance burden are low since both are near identical
>> to what already exists.
>
> I agree that the risks are low in this case. It doesn't make sense to
> have destructive list operations without nondestructive equivalents,
> especially since mutation of pairs is strongly discouraged amongst
> seasoned Schemers.  If it were my call (and it's not, to be clear),
> I'd say that they should just be implemented in Scheme, at least.
> There really is no good reason to implement Scheme procedures in C
> anymore.

I think harmonizing the existing codebase and moving to Scheme should be
seen as orthogonal.

This does not increase the work when moving delete to Scheme by much
(asking in the conversion "was there a good reason not to have a
non-mutating version" might actually take up more time than just doing
the conversion alongside), so I would suggest to merge the patch.

There are nuances which could turn harmonizing into expanding the C in
the codebase (and thus creating a moving target for reducing the C
core), but I think this patch is purely harmonizing

>> It's quite possible as a first-time contributor I am missing something
>> that already exists. If so, oops again!
>
> Nope! Not missing a thing. There's literally no way for you to know
> that there's a desire amongst some Guilers to discourage implementing
> new standard library procedures in C and expanding the default
> namespace.

> I really hope I haven't discouraged you from future contributions by
> using your patch as a case study. Thank you again for hacking on
> Guile!

I want to second this: thank you for your patch!

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: The Guile junk drawer and a C plea

[Maxim Cournoyer]

Hi David,

"Thompson, David" <dthompson2@worcester.edu> writes:

> Hi Richard and all other Guilers, too,
>
> What follows is not code review, but your patch felt like an
> opportunity to provide some commentary about the trajectory of Guile
> development that I've wanted to share for awhile.
>
> First, I think Guile's default environment is a total mess.  It's the
> very definition of a junk drawer.  There's over 1000 names in the
> (guile) module!  Contrast this with R7RS-small's (scheme base) module
> that only has 200ish.  Guile is an old project and I'm sure stuff just
> accumulated over the years, but having so much in the default
> environment makes it hard to know what a program actually uses because
> many things that ought to be explicit imports are not. This makes it a
> challenge to move Guile in a more "least authority" direction.  As a
> rule, I think Guile should *not* add any additional names to the
> default environment without an extremely good reason. Because (guile)
> is imported implicitly, new names can cause clashes with existing code
> that require #:replace to suppress the warning about shadowing core
> bindings.  For example, the newish 'spawn' procedure collides with
> 'spawn' in (goblins core) in the Goblins project.  I think Guile needs
> a (multi-year, multi-major version) plan to deprecate cruft and move
> the good stuff into different modules.  Give a hoot, don't pollute
> (the default environment)!

You probably know this, but in case you or others don't, you can have a
clean, explicit environment by writing a R7RS-small compatible library
via 'define-library' in a .sld file.  I think that's a good standard and
portable way to resolve the messy default environment situation, which
we should embrace more.  On a related note, I have some patches
improving R7RS support in Guile on the tracker/mailing list, such as
#71304, which allowed me to import R7RS SRFI implementations as-is.

-- 
Thanks,
Maxim