unofficial mirror of guile-user@gnu.org 
 help / color / mirror / Atom feed
* Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
@ 2021-03-13 22:42 Michael Schierl
  2021-03-14 13:57 ` [bootstrappable] " Jan Nieuwenhuizen
  2021-03-15 17:09 ` Ludovic Courtès
  0 siblings, 2 replies; 8+ messages in thread
From: Michael Schierl @ 2021-03-13 22:42 UTC (permalink / raw)
  To: guile-user, guix-devel, bootstrappable


Hello,


Jan Nieuwenhuizen wrote to guile-user@gnu.org[1] on 07 Jul 2017:
> Mark H Weaver writes:
>
>>> Does this mean Guile is not bootstrappable from source only?
>>
>> That's correct.  psyntax-pp.scm is not source code, and it is needed to
>> bootstrap Guile.
>
> I'm facing the same problem with Mes.  I have an implemenation of
> syntax-rules that is just about 200 lines of define-macro source code,
> but not syntax case.

>> Having said this, I agree that it would be better if psyntax.scm were
>> written in such a way that it could be bootstrapped without the use of
>> itself.  Maybe some day we'll rewrite it to make it so.
>
> That could be essential to our full source bootstrapping efforts so I'm
> very much interested!


For the record, I have written a psyntax implementation that can be used
by Guile (3.0.2) and does not require an expanded version of itself. It
is not ideal (not fully hygienic and does not support with-ellipsis),
but it works well enough to bootstrap a slightly patched version of
psyntax.scm, which then can be used to bootstrap "the real thing" and
then regenerate psyntax-pp.scm (resulting in a bit-for-bit identical
version if you run the bootstrap on 64-bit Linux).

The project is at
<https://github.com/schierlm/guile-psyntax-bootstrapping/>.

It may still contain some unnecessary code, and the patch for the
patched vesion is definitely not minimal, but for now I'm glad that it
works.

I may improve it later. Contributions are welcome as well.


Regards,


Michael


[1]: https://mail.gnu.org/archive/html/guile-user/2017-07/msg00011.html



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [bootstrappable] Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
  2021-03-13 22:42 Can Guile be bootstrapped from source without psyntax-pp.scm? Michael Schierl
@ 2021-03-14 13:57 ` Jan Nieuwenhuizen
  2021-03-14 14:18   ` Michael Schierl
  2021-03-15 17:09 ` Ludovic Courtès
  1 sibling, 1 reply; 8+ messages in thread
From: Jan Nieuwenhuizen @ 2021-03-14 13:57 UTC (permalink / raw)
  To: Michael Schierl; +Cc: guix-devel, guile-user, bootstrappable

Michael Schierl writes:

Hello,

> Jan Nieuwenhuizen wrote to guile-user@gnu.org[1] on 07 Jul 2017:
>> Mark H Weaver writes:
>>
>>>> Does this mean Guile is not bootstrappable from source only?
>>>
>>> That's correct.  psyntax-pp.scm is not source code, and it is needed to
>>> bootstrap Guile.
>>
>> I'm facing the same problem with Mes.  I have an implemenation of
>> syntax-rules that is just about 200 lines of define-macro source code,
>> but not syntax case.
>
>>> Having said this, I agree that it would be better if psyntax.scm were
>>> written in such a way that it could be bootstrapped without the use of
>>> itself.  Maybe some day we'll rewrite it to make it so.
>>
>> That could be essential to our full source bootstrapping efforts so I'm
>> very much interested!
>
> For the record, I have written a psyntax implementation that can be used
> by Guile (3.0.2) and does not require an expanded version of itself.

Oh, that's amazing!  I see that you are using make-syntax-transformer
(and others) which GNU Mes does not support yet; it only has
define-macro.  This may be a good reason/opportunity to work towards
better Guile support in Mes.

Greetings,
Janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [bootstrappable] Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
  2021-03-14 13:57 ` [bootstrappable] " Jan Nieuwenhuizen
@ 2021-03-14 14:18   ` Michael Schierl
  0 siblings, 0 replies; 8+ messages in thread
From: Michael Schierl @ 2021-03-14 14:18 UTC (permalink / raw)
  To: Jan Nieuwenhuizen; +Cc: guix-devel, guile-user, bootstrappable


Hello Jan,


Am 14.03.2021 um 14:57 schrieb Jan Nieuwenhuizen:
> Michael Schierl writes:
>
> Hello,
>
>> For the record, I have written a psyntax implementation that can be used
>> by Guile (3.0.2) and does not require an expanded version of itself.
>
> Oh, that's amazing!  I see that you are using make-syntax-transformer
> (and others) which GNU Mes does not support yet; it only has
> define-macro.  This may be a good reason/opportunity to work towards
> better Guile support in Mes.

In fact, I use make-syntax-transformer only because Guile does not have
native define-macro support (it uses psyntax to emulate it).

When there is native define-macro support, you can replace step1.scm by

(define s1*-define-macro define-macro)

(define-macro (s1*-expand-with-side-effects seff1 body seff2)
   (list '(lambda (a b . c) (apply values b))
         seff1
         (list 'call-with-values (list 'lambda '() body)
               '(lambda rest rest))
         seff2))

which will make the bootstrap even shorter.

Probably you should validate whether the evaluation order is right so
that s1*-expand-with-side-effects will really expand the side effect 1
before the body and side effect 2 after the body (there is an example in
psyntax-bootstrapping.scm line 76 which should display 1 2 3 and return 42).

You can skip step 2 as well, as you already have a quasiquote expander
that does not rely on psyntex.

The further steps up to step 7 will not use make-syntax-transformer, it
will come back in step 8 (patched psyntax.scm). But probably for step 8
you would have to patch your own psyntax.scm instead of patching guile's
version anyway.


Regards,


Michael



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [bootstrappable] Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
  2021-03-13 22:42 Can Guile be bootstrapped from source without psyntax-pp.scm? Michael Schierl
  2021-03-14 13:57 ` [bootstrappable] " Jan Nieuwenhuizen
@ 2021-03-15 17:09 ` Ludovic Courtès
  2021-03-15 19:50   ` Michael Schierl
  1 sibling, 1 reply; 8+ messages in thread
From: Ludovic Courtès @ 2021-03-15 17:09 UTC (permalink / raw)
  To: Michael Schierl; +Cc: guix-devel, guile-user, bootstrappable

Hi Michael,

Michael Schierl <schierlm@gmx.de> skribis:

> For the record, I have written a psyntax implementation that can be used
> by Guile (3.0.2) and does not require an expanded version of itself. It
> is not ideal (not fully hygienic and does not support with-ellipsis),
> but it works well enough to bootstrap a slightly patched version of
> psyntax.scm, which then can be used to bootstrap "the real thing" and
> then regenerate psyntax-pp.scm (resulting in a bit-for-bit identical
> version if you run the bootstrap on 64-bit Linux).
>
> The project is at
> <https://github.com/schierlm/guile-psyntax-bootstrapping/>.
>
> It may still contain some unnecessary code, and the patch for the
> patched vesion is definitely not minimal, but for now I'm glad that it
> works.

Woow, this is great news!  I think it would be great towards importing
it in Guile proper.

To do that, I think we should first get Andy’s opinion on the approach.
Then you could try to integrate the files into the Guile repo and adjust
the makefile machinery so that it uses this code to generate the initial
psyntax-pp.scm.

There may be additional things to look at, such as performance when
building from scratch.  We could still include the generated
psyntax-pp.scm in the tarball, like we do for some .go files, for those
who want a faster Guile build and are willing to make this tradeoff.

Thanks!

Ludo’.



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [bootstrappable] Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
  2021-03-15 17:09 ` Ludovic Courtès
@ 2021-03-15 19:50   ` Michael Schierl
  2021-03-16  8:59     ` Andy Wingo
  2021-03-17 17:25     ` Ludovic Courtès
  0 siblings, 2 replies; 8+ messages in thread
From: Michael Schierl @ 2021-03-15 19:50 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, guile-user, bootstrappable


Hello Ludo’,


Am 15.03.2021 um 18:09 schrieb Ludovic Courtès:
> Woow, this is great news!  I think it would be great towards importing
> it in Guile proper.
>
> To do that, I think we should first get Andy’s opinion on the approach.

I don't think upstream is very interested in having psyntax-pp.scm
bootstrappable. In Guile 3.0.3 they broke even the `make
ice-9/psyntax-pp.scm.gen` target, and did not repair it even in Guile
3.0.5, that's why I used 3.0.2 for the bootstrap. But I included a patch
to repair it in 3.0.5 in case you really want to bootstrap that version
(psyntax-pp.scm has not changed there). OTOH, from the git log it seems
like psyntax is currently being overhauled for the next release, so
probably my code would need some updates for the next version.

Also, in the last 15 years I avoided directly contributing to "GNU
projects" (with FSF as copyright holder in the license headers), reasons
below. But if anyone else takes my code and upstreams it, I won't object.

Regardless, even when not part of Guile, I believe this code is very
useful for both the live-bootstrap project and Guix to get their Guile
bootstrapped. And even if nobody ever updates it for 3.0.6+, you can
always bootstrap the later versions from an earlier Guile. And maybe a
variation of it lands in GNU Mes, too.


<rant>

And now for the reasons. It happened first to me 17 years ago, what
others would have called an honor, a private email from RMS himself if I
would consider upstreaming some of my code into GNU Emacs. I answered to
feel free to take it, since it is GPLv3+ (or was it GPLv2+ at that
point? not sure) anyway, and he replied that it is not that easy since
first they need a to have me sign "copyright assignment papers" and
asked for a postal address to send them. I was able to find an old
version of that assignment online and it included some clauses I was
unwilling to sign, so I asked if FSF could send me an electronic version
first before I give them my postal address so they can snail mail me the
dead-tree version, just to avoid work on their side assuming that I may
not be willing to sign that anyway. As FSF was unable/unwilling to do
so, it all stopped, until, years later somebody asked me to contribute
some of my code to ELPA. I guess I can spare you the details, they would
bore you.

I'm not at all against contracts, the http://developercertificate.org/
(which I agreed to before contributing a 2-line bugfix to the Linux
kernel) has recently got some traction, and also I've signed Google's
Contributor License Agreement. However, I would not sign Oracle's
Contributor License Agremment (the last version of it that I checked),
not because of the company but because of its contents.

</rant>


Regards,


Michael



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [bootstrappable] Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
  2021-03-15 19:50   ` Michael Schierl
@ 2021-03-16  8:59     ` Andy Wingo
  2021-05-13 20:22       ` Michael Schierl
  2021-03-17 17:25     ` Ludovic Courtès
  1 sibling, 1 reply; 8+ messages in thread
From: Andy Wingo @ 2021-03-16  8:59 UTC (permalink / raw)
  To: Michael Schierl
  Cc: guix-devel, Ludovic Courtès, guile-user, bootstrappable

On Mon 15 Mar 2021 20:50, Michael Schierl <schierlm@gmx.de> writes:

> Am 15.03.2021 um 18:09 schrieb Ludovic Courtès:
>> Woow, this is great news!  I think it would be great towards importing
>> it in Guile proper.
>>
>> To do that, I think we should first get Andy’s opinion on the approach.
>
> I don't think upstream is very interested in having psyntax-pp.scm
> bootstrappable.

Why do you say that? :)

> In Guile 3.0.3 they broke even the `make ice-9/psyntax-pp.scm.gen`
> target, and did not repair it even in Guile 3.0.5, that's why I used
> 3.0.2 for the bootstrap.

Strange -- I used it many times over the past couple months without
problems.  But I see now from your patches what the issue was -- and omg
how embarrassing, I must have a stale canonicalize.go file hanging
around in the tree.  Goes to show how important bootstrapping is!

Andy



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [bootstrappable] Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
  2021-03-15 19:50   ` Michael Schierl
  2021-03-16  8:59     ` Andy Wingo
@ 2021-03-17 17:25     ` Ludovic Courtès
  1 sibling, 0 replies; 8+ messages in thread
From: Ludovic Courtès @ 2021-03-17 17:25 UTC (permalink / raw)
  To: Michael Schierl; +Cc: guix-devel, guile-user, bootstrappable

Hi Michael,

Michael Schierl <schierlm@gmx.de> skribis:

> Am 15.03.2021 um 18:09 schrieb Ludovic Courtès:
>> Woow, this is great news!  I think it would be great towards importing
>> it in Guile proper.
>>
>> To do that, I think we should first get Andy’s opinion on the approach.
>
> I don't think upstream is very interested in having psyntax-pp.scm
> bootstrappable. In Guile 3.0.3 they broke even the `make
> ice-9/psyntax-pp.scm.gen` target, and did not repair it even in Guile
> 3.0.5, that's why I used 3.0.2 for the bootstrap. But I included a patch
> to repair it in 3.0.5 in case you really want to bootstrap that version
> (psyntax-pp.scm has not changed there). OTOH, from the git log it seems
> like psyntax is currently being overhauled for the next release, so
> probably my code would need some updates for the next version.

Andy made it clear that it was a bug.  There’s an interest in providing
a good bootstrapping story for Guile; that’s why there’s an interpreter
written in C, for example.  I’m sure we’d be happy to address psyntax
bootstrapping as well!

> Also, in the last 15 years I avoided directly contributing to "GNU
> projects" (with FSF as copyright holder in the license headers), reasons
> below. But if anyone else takes my code and upstreams it, I won't object.

Sure, we can discuss the details of how to integrate your code.  (Guile
incorporates code not initially written for Guile, such as sxml or
(ice-9 match), and the policy is to not require copyright assignment for
such code.)

> Regardless, even when not part of Guile, I believe this code is very
> useful for both the live-bootstrap project and Guix to get their Guile
> bootstrapped. And even if nobody ever updates it for 3.0.6+, you can
> always bootstrap the later versions from an earlier Guile. And maybe a
> variation of it lands in GNU Mes, too.

Yes, that would be great.  We’ll have to take a closer look, but I’m
under the impression that a fruitful approach for Guile would be to have
it maintained in-tree; that would immediately benefit all distros.

Thanks!

Ludo’.

PS: Not all GNU packages require copyright assignment.  Guix doesn’t,
    for instance, and we’d be happy to get your contributions.  :-)



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [bootstrappable] Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
  2021-03-16  8:59     ` Andy Wingo
@ 2021-05-13 20:22       ` Michael Schierl
  0 siblings, 0 replies; 8+ messages in thread
From: Michael Schierl @ 2021-05-13 20:22 UTC (permalink / raw)
  To: bootstrappable, Andy Wingo; +Cc: guix-devel, guile-user


Hello all,

Am 16.03.2021 um 09:59 schrieb Andy Wingo:
> On Mon 15 Mar 2021 20:50, Michael Schierl <schierlm@gmx.de> writes:

>> In Guile 3.0.3 they broke even the `make ice-9/psyntax-pp.scm.gen`
>> target, and did not repair it even in Guile 3.0.5, that's why I used
>> 3.0.2 for the bootstrap.
>
> Strange -- I used it many times over the past couple months without
> problems.  But I see now from your patches what the issue was -- and omg
> how embarrassing, I must have a stale canonicalize.go file hanging
> around in the tree.  Goes to show how important bootstrapping is!

Yes. Stale files required for building are an even bigger problem than
non-source artifacts in the source code required for building...

As the psyntax-pp.scm.gen target was fixed in 3.0.7, I updated my
bootstrap (the changes where fairly minor):

https://github.com/schierlm/guile-psyntax-bootstrapping/commit/e2f2b1d5271ff8fddb22706863bb0d719e844dd5

I also pushed two tags (guile-3.0.2 and guile 3.0.7) to the repo so that
users can have stable links for bootstrapping a particular Guile version
without having to fetch a particular commit id.


Regards,


Michael



^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2021-05-13 20:22 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-13 22:42 Can Guile be bootstrapped from source without psyntax-pp.scm? Michael Schierl
2021-03-14 13:57 ` [bootstrappable] " Jan Nieuwenhuizen
2021-03-14 14:18   ` Michael Schierl
2021-03-15 17:09 ` Ludovic Courtès
2021-03-15 19:50   ` Michael Schierl
2021-03-16  8:59     ` Andy Wingo
2021-05-13 20:22       ` Michael Schierl
2021-03-17 17:25     ` Ludovic Courtès

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).