automake test driver for Guile scheme patch review

automake test driver for Guile scheme patch review

[Karl Berry]

Hello Guile folk - back in May 2016, Mathieu Lirzin submitted a patch to
add a test driver for Guile Scheme to Automake.
https://lists.gnu.org/archive/html/automake-patches/2016-05/msg00000.html

It was never committed. It's been seven+ years, so before doing so, I
wanted to check if it was still useful, and valid. I know next to
nothing about Guile or Guix myself.

Please advise? If this is the wrong list, please forward? --thanks, karl.

(I'm not on this list, so please cc me with any replies.)

Re: automake test driver for Guile scheme patch review

[Freja Nordsiek]

Dear Karl Berry and Guilers,


I've been considering using Guile for a project as the next step after 
my initial proof of concept in shell script and very much would be using 
Automake's parallel-test harness, like this is for.

The tests I am writing now would not change (run program, check 
results), but having a test driver in Automake itself for Guile would 
make it easier to test the harder parts that my existing tests would not 
be able to do (interact with individual pieces of code to see if they do 
the right thing at a more granular level).

Otherwise, I would likely fall back to scheme library I wrote several 
years ago in 2017 to make an SRFI-64 inspired unit test framework that 
output in the TAP format (had Guile, R6RS, and R7RS compliant scheme and 
install support for Guile, Chicken, and Vicare) and could then be used 
in with Automake's TAP based test harness 
(https://github.com/frejanordsiek/unit-test-tap).

So, this patch, if revived, would be very nice for me, assuming I 
actually do make the Guile version of my project after finishing the 
proof of concept. I have to be honest here, it has been a long time 
since I worked with Guile (2017) and I am not sure how well it would fly 
with others interested in the project. So, I am only a maybe future user 
if this gets done. Maybe there are others interested in it as well.


Best regards,

Freja Nordsiek


On 19/11/2023 18:17, Karl Berry wrote:
> Hello Guile folk - back in May 2016, Mathieu Lirzin submitted a patch to
> add a test driver for Guile Scheme to Automake.
> https://lists.gnu.org/archive/html/automake-patches/2016-05/msg00000.html
>
> It was never committed. It's been seven+ years, so before doing so, I
> wanted to check if it was still useful, and valid. I know next to
> nothing about Guile or Guix myself.
>
> Please advise? If this is the wrong list, please forward? --thanks, karl.
>
> (I'm not on this list, so please cc me with any replies.)
>

Re: automake test driver for Guile scheme patch review

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 1706 bytes --]



Op 19-11-2023 om 18:17 schreef Karl Berry:
> Hello Guile folk - back in May 2016, Mathieu Lirzin submitted a patch to
> add a test driver for Guile Scheme to Automake.
> https://lists.gnu.org/archive/html/automake-patches/2016-05/msg00000.html
> 
> It was never committed. It's been seven+ years, so before doing so, I
> wanted to check if it was still useful, and valid. I know next to
> nothing about Guile or Guix myself.
> 
> Please advise? If this is the wrong list, please forward? --thanks, karl.
> 
> (I'm not on this list, so please cc me with any replies.)
> 
Useful, but I don't think it should be in Automake, because then the 
test driver is harder to update and it would add yet another source of 
bundling.  Instead, I think the test driver should be a separate 
package/program (with support in Automake for invoking the driver) or 
part of Guile (maybe a "guild test" subcommand or something like that).

I don't have any advise on how to get the automake maintainers to 
actually respond to the patch beyond ‘maybe try again?’.

Also, the test driver has been updated since then.  As far as I can 
tell, Guix is currently the upstream of the test driver (see: 
build-aux/test-driver.scm) -- while part of Guix, it should be 
straightforward enough to only install the test driver itself in 
distributions, without compiling or installing the rest of Guix.

(See, e.g. https://issues.guix.gnu.org/58365#9, where test-driver.scm 
(from Guix) is packaged as a Guix package, albeit not the best example 
because both are Guix.)

If you retry merging it into Automake, I recommend picking up the new 
version from Guix.

Best regards,
Maxime Devos

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: automake test driver for Guile scheme patch review

[Timothy Sample]

Hi,

Maxime Devos <maximedevos@telenet.be> writes:

> Op 19-11-2023 om 18:17 schreef Karl Berry:
>
>> Hello Guile folk - back in May 2016, Mathieu Lirzin submitted a patch
>> to add a test driver for Guile Scheme to Automake.
>> https://lists.gnu.org/archive/html/automake-patches/2016-05/msg00000.html
>>
>> It was never committed. It's been seven+ years, so before doing so, I
>> wanted to check if it was still useful, and valid.
>
> Useful, but I don't think it should be in Automake, because then the
> test driver is harder to update and it would add yet another source of
> bundling.

Well, it’s already a part of Automake as ‘contrib/test-driver.scm’:

https://git.savannah.gnu.org/cgit/automake.git/tree/contrib/test-driver.scm

> Also, the test driver has been updated since then.

That’s true.  Maxim Cournoyer added some useful features in 2021.
However, these features do not affect Automake.  They only make the
script more useful when running outside of Automake.  It would be nice
to have them in the Automake version, but I guess Maxim would have to
sign a copyright assignment for that.  (I’m not sure of Automake’s
policy there, so correct me if I’m wrong.)

I think that doing nothing other than closing that patch would be a
reasonable course of action.  :)


-- Tim

Re: automake test driver for Guile scheme patch review

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 2256 bytes --]



Op 20-11-2023 om 17:16 schreef Timothy Sample:
> Hi,
> 
> Maxime Devos <maximedevos@telenet.be> writes:
> 
>> Op 19-11-2023 om 18:17 schreef Karl Berry:
>>
>>> Hello Guile folk - back in May 2016, Mathieu Lirzin submitted a patch
>>> to add a test driver for Guile Scheme to Automake.
>>> https://lists.gnu.org/archive/html/automake-patches/2016-05/msg00000.html
>>>
>>> It was never committed. It's been seven+ years, so before doing so, I
>>> wanted to check if it was still useful, and valid.
>>
>> Useful, but I don't think it should be in Automake, because then the
>> test driver is harder to update and it would add yet another source of
>> bundling.
> 
> Well, it’s already a part of Automake as ‘contrib/test-driver.scm’:
> 
> https://git.savannah.gnu.org/cgit/automake.git/tree/contrib/test-driver.scm

Last time I tried, "autoreconf -i" doesn't pick it up, so apparently 
it's not part of Automake in any way that matters, it's only part of the 
Automake repository. (Also, it's undocumented.)

Besides, if it were added to Automake at one point, it can be removed 
from Automake (and added somwhere else) at another.

>> Also, the test driver has been updated since then.
> 
> That’s true.  Maxim Cournoyer added some useful features in 2021.
> However, these features do not affect Automake.  They only make the
> script more useful when running outside of Automake.  It would be nice
> to have them in the Automake version, but I guess Maxim would have to
> sign a copyright assignment for that.  (I’m not sure of Automake’s
> policy there, so correct me if I’m wrong.)

Looking at the commit messages, they are useful within Automake as well
do in fact, affect Automake.  For example, one of these changes is a new 
"--show-duration" option, which can be used within Automake by:

$ make check SCM_LOG_DRIVER_FLAGS="--brief=no --show-duration=yes"

(copied from doc/guix.texi).

> I think that doing nothing other than closing that patch would be a
> reasonable course of action.  :)

And how is "doing nothing" supposed to help with discoverability, ease 
of updating the test driver, clarity, unbundling and the goals of Freja 
Nordsiek?

Best regards,
Maxime Devos

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: automake test driver for Guile scheme patch review

[Karl Berry]

Thanks for the replies.

    MD> but I don't think it should be in Automake, because then the test
    driver is harder to update

Fine by me :).

    MD> I don't have any advise on how to get the automake maintainers

Sorry, I should have made it clear that I am, effectively, the automake
maintainer nowadays. Technically just a contributor, but no one else is
applying patches. (Jim Meyering is still making the releases, hopefully
with another one out before the end of the year.)

    TS> Well, it's already a part of Automake as
    contrib/test-driver.scm:

Sounds like I should remove it from there, since that version is stale,
as you both noted. It doesn't make sense to me to have a copy of a file
that is updated elsewhere. Better for it to be part of wherever it is
actually useful instead of putting me and laggard automake development
in the middle. Since it's been years since the initial patch+idea, and
no one has ever written about it, evidently the goal has been reached
outside of Automake.

    TS> I think that doing nothing other than closing that patch would be a
    reasonable course of action.  :)

It's nice to hear the same proposal twice :). Thus it shall be.

Freja, if you go forward with your project, evidently the conclusion is
that it's better to use the actively-maintained version in Guix.

--happy hacking, karl.

Re: automake test driver for Guile scheme patch review

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 768 bytes --]



Op 21-11-2023 om 18:33 schreef Maxime Devos:
>> I think that doing nothing other than closing that patch would be a
>> reasonable course of action.  :)
> 
> And how is "doing nothing" supposed to help with discoverability, ease 
> of updating the test driver, clarity, unbundling and the goals of Freja 
> Nordsiek?

By the extra context from Karl Berry (context: Karl Berry is effectively 
an automake maintainer): I now gather this is about what would be 
reasonable to do for the Automake maintainer, not about what would be 
reasonable for someone doing Guile things -- there are still other 
things beside closing the patch that need to happen, but those aren't 
things to do as part of Automake.

Best regards,
Maxime Devos

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: automake test driver for Guile scheme patch review

[Timothy Sample]

Hello,

[Dropping Karl since we are no longer discussing Automake maintenance.]

Maxime Devos <maximedevos@telenet.be> writes:

> [What about] discoverability, ease of updating the test driver,
> clarity, unbundling and the goals of Freja Nordsiek?

Maybe Guile should adopt the script as ‘guild test-am’ or somesuch.
Mathieu assigned copyright to the FSF for the Automake patch and I
believe Maxim would be willing to do the same for the more recent
patches.

If there’s interest and someone willing to commit the change, I
volunteer to integrate the script.  I can add a node in the manual after
“4.8 Distributing Guile Code” called “Testing Guile Code” that links to
Maxim’s forthcoming SRFI 64 documentation and explains how to use the
script with Automake.

Thoughts?


-- Tim

Re: automake test driver for Guile scheme patch review

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 1483 bytes --]



Op 22-11-2023 om 00:02 schreef Timothy Sample:
> Hello,
> 
> [Dropping Karl since we are no longer discussing Automake maintenance.]
> 
> Maxime Devos <maximedevos@telenet.be> writes:
> 
>> [What about] discoverability, ease of updating the test driver,
>> clarity, unbundling and the goals of Freja Nordsiek?
> 
> Maybe Guile should adopt the script as ‘guild test-am’ or somesuch.

While written for Automake, it's useful and usable outside it, so I 
would change 'am' to something else. Maybe: ‘guild run-test-suite’.

> Mathieu assigned copyright to the FSF for the Automake patch and I
> believe Maxim would be willing to do the same for the more recent
> patches.

Note: while accepted, that's not a requirement anymore.

> If there’s interest and someone willing to commit the change, I
> volunteer to integrate the script.  I can add a node in the manual after
> “4.8 Distributing Guile Code” called “Testing Guile Code” that links to
> Maxim’s forthcoming SRFI 64 documentation and explains how to use the
> script with Automake.
> 
> Thoughts?

I don't know if there is someone willing to commit the change (as in, I 
don't know, not I know there isn't someone willing), but I would say 
there is definitely interest.

(Doesn't matter for me whether it's in Guile or a thing packaged 
separately ... just not this script that gets copied around with 
somewhat unclear providence.)

Best regards,
Maxime Devos.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: automake test driver for Guile scheme patch review

[Freja Nordsiek]

I will check out the actively-maintained one from Guix, and likely use 
it. Almost certainly better than the mess I wrote.


Best regards,

Freja


On 21/11/2023 22:45, Karl Berry wrote:
> Thanks for the replies.
>
>      MD> but I don't think it should be in Automake, because then the test
>      driver is harder to update
>
> Fine by me :).
>
>      MD> I don't have any advise on how to get the automake maintainers
>
> Sorry, I should have made it clear that I am, effectively, the automake
> maintainer nowadays. Technically just a contributor, but no one else is
> applying patches. (Jim Meyering is still making the releases, hopefully
> with another one out before the end of the year.)
>
>      TS> Well, it's already a part of Automake as
>      contrib/test-driver.scm:
>
> Sounds like I should remove it from there, since that version is stale,
> as you both noted. It doesn't make sense to me to have a copy of a file
> that is updated elsewhere. Better for it to be part of wherever it is
> actually useful instead of putting me and laggard automake development
> in the middle. Since it's been years since the initial patch+idea, and
> no one has ever written about it, evidently the goal has been reached
> outside of Automake.
>
>      TS> I think that doing nothing other than closing that patch would be a
>      reasonable course of action.  :)
>
> It's nice to hear the same proposal twice :). Thus it shall be.
>
> Freja, if you go forward with your project, evidently the conclusion is
> that it's better to use the actively-maintained version in Guix.
>
> --happy hacking, karl.
>

Re: automake test driver for Guile scheme patch review

[Maxim Cournoyer]

Hi,

Timothy Sample <samplet@ngyro.com> writes:

> Hello,
>
> [Dropping Karl since we are no longer discussing Automake maintenance.]
>
> Maxime Devos <maximedevos@telenet.be> writes:
>
>> [What about] discoverability, ease of updating the test driver,
>> clarity, unbundling and the goals of Freja Nordsiek?
>
> Maybe Guile should adopt the script as ‘guild test-am’ or somesuch.
> Mathieu assigned copyright to the FSF for the Automake patch and I
> believe Maxim would be willing to do the same for the more recent
> patches.

I think having such a guild command would be nice for reuse.  I'm happy
to assign my copyright to the FSF, although that's not strictly needed
anymore as pointed to Maxime.

> If there’s interest and someone willing to commit the change, I
> volunteer to integrate the script.  I can add a node in the manual after
> “4.8 Distributing Guile Code” called “Testing Guile Code” that links to
> Maxim’s forthcoming SRFI 64 documentation and explains how to use the
> script with Automake.
>
> Thoughts?

Sounds good to me!  Thanks for the initiative.

-- 
Thanks,
Maxim