unofficial mirror of guile-user@gnu.org 
 help / color / mirror / Atom feed
* Can Guile be bootstrapped from source without psyntax-pp.scm?
@ 2017-07-06 15:24 Alex Vong
  2017-07-07  1:42 ` Mark H Weaver
  0 siblings, 1 reply; 6+ messages in thread
From: Alex Vong @ 2017-07-06 15:24 UTC (permalink / raw)
  To: guile-user, bootstrappable

[-- Attachment #1: Type: text/plain, Size: 2928 bytes --]

Hello,


Recently, I've read the chapter on Macros in Guile's manual. The manual
says that Guile's expander originated from that of Chez Scheme's and
that version was portable to other schemes as well. So I search the
Internet and find it[0]. The website states that the expander needs an
expanded version of itself for bootstrapping, i.e. psyntax.ss needs
psyntax.pp for bootstrapping.

After playing for awhile, I realize the relationship of Guile's
psyntax.scm and psyntax-pp.scm is exactly analogus. One needs an
expanded version of psyntax.scm, i.e. psyntax-pp.scm for bootstrapping.

Does this mean Guile is not bootstrappable from source only? I've tried
to compile Guile 2.2.2 with psyntax-pp.scm removed and I get a
segfault[1]. It seems the interpreter relies on psyntax-pp.scm to
function properly. I don't know if I should bring this discussion to
guix-devel. Free to do so if you think it is on topic.


Cheers,
Alex


[0]: https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html
[1]:

  GEN      guile-procedures.texi
Uncaught exception:
Backtrace:
/bin/bash: line 1:  4113 Broken pipe             cat alist.doc array-handle.doc array-map.doc arrays.doc async.doc atomic.doc backtrace.doc boolean.doc bitvectors.doc bytevectors.doc chars.doc control.doc continuations.doc debug.doc deprecated.doc deprecation.doc dynl.doc dynwind.doc eq.doc error.doc eval.doc evalext.doc expand.doc extensions.doc fdes-finalizers.doc feature.doc filesys.doc fluids.doc foreign.doc fports.doc gc-malloc.doc gc.doc gettext.doc generalized-arrays.doc generalized-vectors.doc goops.doc gsubr.doc guardians.doc hash.doc hashtab.doc hooks.doc i18n.doc init.doc ioext.doc keywords.doc list.doc load.doc macros.doc mallocs.doc memoize.doc modules.doc numbers.doc objprop.doc options.doc pairs.doc ports.doc print.doc procprop.doc procs.doc promises.doc r6rs-ports.doc random.doc rdelim.doc read.doc rw.doc scmsigs.doc script.doc simpos.doc smob.doc sort.doc srcprop.doc srfi-1.doc srfi-4.doc srfi-13.doc srfi-14.doc srfi-60.doc stackchk.doc stacks.doc stime.doc strings.doc strorder.doc strports.doc struct.doc symbols.doc syntax.doc threads.doc throw.doc trees.doc unicode.doc uniform.doc values.doc variable.doc vectors.doc version.doc vports.doc weak-set.doc weak-table.doc weak-vector.doc dynl.doc posix.doc net_db.doc socket.doc regex-posix.doc
      4114 Segmentation fault      | GUILE_AUTO_COMPILE=0 ../meta/build-env guild snarf-check-and-output-texi > guile-procedures.texi
make[3]: *** [Makefile:3909: guile-procedures.texi] Error 1
make[3]: Leaving directory '/home/alexvong1995/scm/guile-2.2.2/libguile'
make[2]: *** [Makefile:2299: all] Error 2
make[2]: Leaving directory '/home/alexvong1995/scm/guile-2.2.2/libguile'
make[1]: *** [Makefile:1857: all-recursive] Error 1
make[1]: Leaving directory '/home/alexvong1995/scm/guile-2.2.2'
make: *** [Makefile:1743: all] Error 2

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
  2017-07-06 15:24 Can Guile be bootstrapped from source without psyntax-pp.scm? Alex Vong
@ 2017-07-07  1:42 ` Mark H Weaver
  2017-07-07 14:00   ` Alex Vong
  2017-07-07 14:55   ` Jan Nieuwenhuizen
  0 siblings, 2 replies; 6+ messages in thread
From: Mark H Weaver @ 2017-07-07  1:42 UTC (permalink / raw)
  To: Alex Vong; +Cc: guile-user, bootstrappable

Hi Alex,

Alex Vong <alexvong1995@gmail.com> writes:

> Recently, I've read the chapter on Macros in Guile's manual. The manual
> says that Guile's expander originated from that of Chez Scheme's and
> that version was portable to other schemes as well. So I search the
> Internet and find it[0]. The website states that the expander needs an
> expanded version of itself for bootstrapping, i.e. psyntax.ss needs
> psyntax.pp for bootstrapping.
>
> After playing for awhile, I realize the relationship of Guile's
> psyntax.scm and psyntax-pp.scm is exactly analogus. One needs an
> expanded version of psyntax.scm, i.e. psyntax-pp.scm for bootstrapping.
>
> Does this mean Guile is not bootstrappable from source only?

That's correct.  psyntax-pp.scm is not source code, and it is needed to
bootstrap Guile.  However, I made an effort some years ago to make
psyntax-pp.scm far smaller and more readable than the corresponding file
from upstream psyntax.  See:

  https://git.savannah.gnu.org/cgit/guile.git/commit/?h=stable-2.0&id=72ee0ef71b9a0514874976cdcf3ea9d5333db4b1
  https://git.savannah.gnu.org/cgit/guile.git/commit/?h=stable-2.0&id=1af6d2a717f499564fbbc297c79e00ac14b0dcf9

At this point, I believe it would be quite feasible for a single hacker
to audit our psyntax-pp.scm and compare it to psyntax.scm within a
reasonable time frame.  Furthermore, when we make local changes to
psyntax.scm, the corresponding changes to psyntax-pp.scm are localized
and quite easy to audit as well, so the full audit need not be repeated.

Having said this, I agree that it would be better if psyntax.scm were
written in such a way that it could be bootstrapped without the use of
itself.  Maybe some day we'll rewrite it to make it so.

      Mark



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
  2017-07-07  1:42 ` Mark H Weaver
@ 2017-07-07 14:00   ` Alex Vong
  2017-07-07 16:40     ` Mark H Weaver
  2017-07-07 14:55   ` Jan Nieuwenhuizen
  1 sibling, 1 reply; 6+ messages in thread
From: Alex Vong @ 2017-07-07 14:00 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: guile-user, bootstrappable

[-- Attachment #1: Type: text/plain, Size: 2542 bytes --]

Hi Mark,

Mark H Weaver <mhw@netris.org> writes:

> Hi Alex,
>
> Alex Vong <alexvong1995@gmail.com> writes:
>
>> Recently, I've read the chapter on Macros in Guile's manual. The manual
>> says that Guile's expander originated from that of Chez Scheme's and
>> that version was portable to other schemes as well. So I search the
>> Internet and find it[0]. The website states that the expander needs an
>> expanded version of itself for bootstrapping, i.e. psyntax.ss needs
>> psyntax.pp for bootstrapping.
>>
>> After playing for awhile, I realize the relationship of Guile's
>> psyntax.scm and psyntax-pp.scm is exactly analogus. One needs an
>> expanded version of psyntax.scm, i.e. psyntax-pp.scm for bootstrapping.
>>
>> Does this mean Guile is not bootstrappable from source only?
>
> That's correct.  psyntax-pp.scm is not source code, and it is needed to
> bootstrap Guile.  However, I made an effort some years ago to make
> psyntax-pp.scm far smaller and more readable than the corresponding file
> from upstream psyntax.  See:
>
>   https://git.savannah.gnu.org/cgit/guile.git/commit/?h=stable-2.0&id=72ee0ef71b9a0514874976cdcf3ea9d5333db4b1
>   https://git.savannah.gnu.org/cgit/guile.git/commit/?h=stable-2.0&id=1af6d2a717f499564fbbc297c79e00ac14b0dcf9
>
I see. So it used to be worse --- several millions LOC. Now it is only
~3000, which is shorter than Emacs's bytecomp.el

> At this point, I believe it would be quite feasible for a single hacker
> to audit our psyntax-pp.scm and compare it to psyntax.scm within a
> reasonable time frame.  Furthermore, when we make local changes to
> psyntax.scm, the corresponding changes to psyntax-pp.scm are localized
> and quite easy to audit as well, so the full audit need not be repeated.
>
It is really good that small change in source --> small change in
binary. A fancy way of saying it: the map between source and binary is
in some sense "continuious".

> Having said this, I agree that it would be better if psyntax.scm were
> written in such a way that it could be bootstrapped without the use of
> itself.  Maybe some day we'll rewrite it to make it so.
>
Agree.

For future interested readers:

I can imagine one way of doing it: First implement a low-level hygienic
macro expander without using macros and then write the syntax-case
expander using low-level macros.

To get started, you can watch the "Let's Build a Hygienic Macro Expander
— Strange Loop 2016, Matthew Flatt" video.

>       Mark

Cheers,
Alex

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
  2017-07-07  1:42 ` Mark H Weaver
  2017-07-07 14:00   ` Alex Vong
@ 2017-07-07 14:55   ` Jan Nieuwenhuizen
  1 sibling, 0 replies; 6+ messages in thread
From: Jan Nieuwenhuizen @ 2017-07-07 14:55 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: guile-user, bootstrappable

Mark H Weaver writes:

>> Does this mean Guile is not bootstrappable from source only?
>
> That's correct.  psyntax-pp.scm is not source code, and it is needed to
> bootstrap Guile.

I'm facing the same problem with Mes.  I have an implemenation of
syntax-rules that is just about 200 lines of define-macro source code,
but not syntax case.

The only useful thing I found was Andre van Tonder's syntax-case.  I
tried to geth it up and running, which just might be a way (it's not
clear to me from the documentation or code and it seems to have been a
lone effort that was abandoned -- could not find any references or
follow-ups) to have syntax-case without psyntax.  I gave up for now.

> Having said this, I agree that it would be better if psyntax.scm were
> written in such a way that it could be bootstrapped without the use of
> itself.  Maybe some day we'll rewrite it to make it so.

That could be essential to our full source bootstrapping efforts so I'm
very much interested!

Greetings,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
  2017-07-07 14:00   ` Alex Vong
@ 2017-07-07 16:40     ` Mark H Weaver
  0 siblings, 0 replies; 6+ messages in thread
From: Mark H Weaver @ 2017-07-07 16:40 UTC (permalink / raw)
  To: Alex Vong; +Cc: guile-user, bootstrappable

Alex Vong <alexvong1995@gmail.com> writes:

> Mark H Weaver <mhw@netris.org> writes:
>
>> At this point, I believe it would be quite feasible for a single hacker
>> to audit our psyntax-pp.scm and compare it to psyntax.scm within a
>> reasonable time frame.  Furthermore, when we make local changes to
>> psyntax.scm, the corresponding changes to psyntax-pp.scm are localized
>> and quite easy to audit as well, so the full audit need not be repeated.
>>
> It is really good that small change in source --> small change in
> binary. A fancy way of saying it: the map between source and binary is
> in some sense "continuious".

Sorry, I should have qualified my statement above with the word
"usually".  There are some kinds of local changes that result in
non-local changes to psyntax-pp.scm, e.g. if one of the macros are
changed.

A particularly pessimal case is when the 'syntax-object' structure is
changed, since its accessors are macros that are used throughout the
code.  This case actually happened recently in these commits:

  https://git.savannah.gnu.org/cgit/guile.git/commit/?h=stable-2.2&id=eb84c2f2da83cf04214bbacf4b33528ce09a5b1a
  https://git.savannah.gnu.org/cgit/guile.git/commit/?h=stable-2.2&id=a42bfae65f445178d3608433356ce132d1e7369e

As far as I'm aware, this is the first time such a large non-local
change has occurred since my work to minimize psyntax-pp.scm in 2012.

      Mark



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Can Guile be bootstrapped from source without psyntax-pp.scm?
@ 2021-03-13 22:42 Michael Schierl
  0 siblings, 0 replies; 6+ messages in thread
From: Michael Schierl @ 2021-03-13 22:42 UTC (permalink / raw)
  To: guile-user, guix-devel, bootstrappable


Hello,


Jan Nieuwenhuizen wrote to guile-user@gnu.org[1] on 07 Jul 2017:
> Mark H Weaver writes:
>
>>> Does this mean Guile is not bootstrappable from source only?
>>
>> That's correct.  psyntax-pp.scm is not source code, and it is needed to
>> bootstrap Guile.
>
> I'm facing the same problem with Mes.  I have an implemenation of
> syntax-rules that is just about 200 lines of define-macro source code,
> but not syntax case.

>> Having said this, I agree that it would be better if psyntax.scm were
>> written in such a way that it could be bootstrapped without the use of
>> itself.  Maybe some day we'll rewrite it to make it so.
>
> That could be essential to our full source bootstrapping efforts so I'm
> very much interested!


For the record, I have written a psyntax implementation that can be used
by Guile (3.0.2) and does not require an expanded version of itself. It
is not ideal (not fully hygienic and does not support with-ellipsis),
but it works well enough to bootstrap a slightly patched version of
psyntax.scm, which then can be used to bootstrap "the real thing" and
then regenerate psyntax-pp.scm (resulting in a bit-for-bit identical
version if you run the bootstrap on 64-bit Linux).

The project is at
<https://github.com/schierlm/guile-psyntax-bootstrapping/>.

It may still contain some unnecessary code, and the patch for the
patched vesion is definitely not minimal, but for now I'm glad that it
works.

I may improve it later. Contributions are welcome as well.


Regards,


Michael


[1]: https://mail.gnu.org/archive/html/guile-user/2017-07/msg00011.html



^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-03-13 22:42 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-07-06 15:24 Can Guile be bootstrapped from source without psyntax-pp.scm? Alex Vong
2017-07-07  1:42 ` Mark H Weaver
2017-07-07 14:00   ` Alex Vong
2017-07-07 16:40     ` Mark H Weaver
2017-07-07 14:55   ` Jan Nieuwenhuizen
  -- strict thread matches above, loose matches on Subject: below --
2021-03-13 22:42 Michael Schierl

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).