GNU Mes 0.19 released

GNU Mes 0.19 released

[Jan Nieuwenhuizen]

[-- Attachment #1: Type: text/plain, Size: 6640 bytes --]

We are pleased to announce the release of GNU Mes 0.19, representing
100 commits over 10 weeks.

Mes has now brought the Reduced Binary Seed bootstrap to Guix (bootstrap
a GNU/Linux system without binary GNU toolchain or equivalent) and work
is ongoing to audit and verify this bootstrap path in NixOS.

This release introduces strings as byte-array, hash-tables and native
structs.  While that does increase the footprint somewhat, it fixes
our performance issue; tinycc is now compiled in ~8min (WAS: ~1h30).

Next targets:

 - translate mes.c into unsnarfed mes.M2
 - use Gash to remove bash, coreutils&co, grep, sed, tar from the Guix
   bootstrap binaries
 - replace the NixOS bootstrap
 - use dietlibc, uClibc, ... for bootstrapping GNU (bash, binutils,
   gcc, tar) and remove Mes C lib+gnu?
 - bootstrap gcc-3.x or 4.x directly, drop initial gcc-2.95.3 target?
 - have M1+hex2 create gcc/tcc-usable object files?  archives?
 - Debian?
 - ARM, the Hurd?

Packages are available from Guix's core-updates branch.

* About

  GNU Mes[0] brings a Reduced Binary Seed bootstrap[1] to Guix[2] and
  potentially to any other interested GNU/Linux distribution, and aims
  to help create a full source bootstrap as part of the
  bootstrappable builds[3] effort.

  It consists of a mutual self-hosting Scheme interpreter written in
  ~5,000 LOC of simple C and a Nyacc-based C compiler written in Scheme.
  This mes.c is being simplified[4] to be transpiled by M2-Planet[5].

  The Scheme interpreter (mes.c) has a Garbage Collector, a library of
  loadable Scheme modules-- notably Dominique Boucher's LALR[6], Pre-R6RS
  [portable syntax-case[7] with R7RS ellipsis, Matt Wette's Nyacc[8] --and test
  suite just barely enough to support a simple REPL and simple
  C-compiler: MesCC.

  Mes+MesCC can compile an only lightly patched TinyCC[9] that is
  self-hosting.  Using this tcc and the Mes C library we now have a
  Reduced Binary Seed bootstrap for the gnutools triplet: glibc-2.2.5,
  binutils-2.20.1, gcc-2.95.3.  This is enough to bootstrap Guix for
  i686-linux and x86_64-linux.

  Mes is inspired by The Maxwell Equations of Software: LISP-1.5[10] -- John
  McCarthy page 13, GNU Guix's source/binary packaging transparency and
  Jeremiah Orians's stage0[11] ~500 byte self-hosting hex assembler.

* Download

  git clone git://git.savannah.gnu.org/mes.git

  Here are the compressed sources and a GPG detached signature[*]:
    https://ftp.gnu.org/gnu/mes/mes-0.19.tar.gz
    https://ftp.gnu.org/gnu/mes/mes-0.19.tar.gz.sig

  Use a mirror for higher download bandwidth:
    https://ftpmirror.gnu.org/mes/mes-0.19.tar.gz
    https://ftpmirror.gnu.org/mes/mes-0.19.tar.gz.sig

  Here are the MD5 and SHA1 checksums:

  99e134df87adc5fc5fd2c04941929c23  mes-0.19.tar.gz
  c9781b3b6a814acc985c2ac68caa111a56583bca  mes-0.19.tar.gz

  [*] Use a .sig file to verify that the corresponding file (without the
  .sig suffix) is intact.  First, be sure to download both the .sig file
  and the corresponding tarball.  Then, run a command like this:

    gpg --verify mes-0.19.tar.gz.sig

  If that command fails because you don't have the required public key,
  then run this command to import it:

    gpg --keyserver keys.gnupg.net --recv-keys 1A858392E331EAFDB8C27FFBF3C1A0D9C1D65273

  and rerun the 'gpg --verify' command.

Mes runs from the source tree and can also be built, packaged and
installed in Guix from a git checkout by running

    guix package -f .guix.scm

* Get informed, get involved

    See https://bootstrappable.org
    Join #bootstrappable on irc.freenode.net.

* Changes in 0.19 since 0.18
 ** Core
 *** The build system has been simplified.
 *** Mes now prints a backtrace upon error.
 *** Performance has been improved 2-8 times, making Mes 2-10 times slower than Guile.
 *** Mes now supports a module type and uses a `boot-module'.
 *** Mes now supports a hash_table type.
 *** Mes now supports a struct type.
 *** Mes now supports building a %bootstrap-mes seed from Guix.
 ** Language
 *** Records are now implemented using struct (WAS: vector).
 *** 44 new functions
 ceil, char-downcase, char-set-adjoin, char-set-complement,
 char-upcase, current-time, delete-file, dup, dup2, file-exists?,
 floor, frame-printer, get-internal-run-time, getcwd, gettimeofday,
 hash, hash-ref, hash-set!, hash-table-printer, hashq,
 hashq-get-handle, hashq-ref, hashq-set, inexact->exact,
 make-hash-table, make-stack, make-struct, module-define!,
 module-printer, module-ref, module-variable, read-line, round,
 stack-length, stack-ref, string-downcase, string-tokenize,
 string-upcase, struct-length, struct-ref, struct-set! struct-vtable,
 struct-vtable, with-error-to-file.
 ** MesCC
 *** Assembly defines have been cleaned-up: duplicates deleted, missing added, wrong fixed.
 *** MesCC now supports compiling GNU Bash and GNU Tar.
 **** 6 New functions
 getegid, geteuid, getppid, setgid, setuid, sigdelset, sigprocmask.
 **** 22 New macros
 EACCES, ENOSPC, ESPIPE, INT16_MAX, INT16_MIN, INT32_MAX, INT32_MIN,
 INT64_MAX, INT64_MIN, INT8_MAX, INT8_MIN, LLONG_MAX, LLONG_MIN,
 SIZE_MAX SYS_getegid, SYS_geteuid, SYS_setgid SYS_setuid, S_IRGRP,
 S_IROTH, S_IRWXG, S_IRWXO S_ISGID, S_ISUID, S_IWGRP, S_IWOTH, S_IXGRP,
 S_IXOTH, UINT16_MAX, UINT32_MAX, UINT64_MAX, UINT8_MAX,
 _POSIX_VERSION.
 ** Noteworthy bug fixes
 *** Mes now supports characters #\xNN.
 *** Mes now supports assq-ref and assoc-ref with alist == #f.
 *** Mes now supports \xNN in strings.  This allows using Nyacc-0.86.0.
 *** MesCC now supports the unary plus operator.
 *** MesCC now supports the `U' integer suffix.
 *** MesCC now comes with INTnn_MIN/MAX, UINTnn defines in stdint.h.
 *** MesCC now always exits non-zero when assembler or linker fail.

Greetings,
janneke

[0] https://www.gnu.org/software/mes
[1] http://joyofsource.com/reduced-binary-seed-bootstrap.html
[2] https://www.gnu.org/software/guix
[3] https://bootstrappable.org
[4] https://github.com/oriansj/mes-m2
[5] https://github.com/oriansj/m2-planet
[6] https://github.com/schemeway/lalr-scm
[7] https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html
[8] https://www.nongnu.org/nyacc
[9] https://gitlab.com/janneke/tinycc
[10] http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf
[11] https://github.com/oriansj/stage0

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: GNU Mes 0.19 released

[Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]


Jan Nieuwenhuizen <janneke@gnu.org> writes:

> Mes has now brought the Reduced Binary Seed bootstrap to Guix (bootstrap
> a GNU/Linux system without binary GNU toolchain or equivalent)

That’s awesome!

> This release introduces strings as byte-array, hash-tables and native
> structs.  While that does increase the footprint somewhat, it fixes
> our performance issue; tinycc is now compiled in ~8min (WAS: ~1h30).

Wow! I did not expect such a huge difference.

> Next targets:
>
>  - translate mes.c into unsnarfed mes.M2
>  - use Gash to remove bash, coreutils&co, grep, sed, tar from the Guix
>    bootstrap binaries

Is this about configure+make?

>  - bootstrap gcc-3.x or 4.x directly, drop initial gcc-2.95.3 target?

Do you have a document showing the bootstrap path — ideally in graph
form, i.e. with plantuml activity diagram?
http://plantuml.com/activity-diagram-legacy

>  - Debian?
- Gentoo?

>  - ARM, the Hurd?

The Hurd would be nice!

Thank you for your great work!

Best wishes,
Arne
--
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1076 bytes --]

Re: [bootstrappable] Re: GNU Mes 0.19 released

[Ricardo Wurmus]

Hi Arne,

> Do you have a document showing the bootstrap path — ideally in graph
> form, i.e. with plantuml activity diagram?
> http://plantuml.com/activity-diagram-legacy

You can see the current bootstrap path here:

    https://bootstrappable.org/projects/mes.html

This is without the path from virtually nothing to mes + mescc tools,
i.e. mes and mescc tools currently are bootstrap binaries here.

>>  - use Gash to remove bash, coreutils&co, grep, sed, tar from the Guix
>>    bootstrap binaries
>
> Is this about configure+make?

You can see in the graph that “bootstrap-binaries” is a collection of
statically linked binaries including tar, gzip, patch, coreutils, etc.
Most if not all of these could be removed with Gash.

--
Ricardo

Re: GNU Mes 0.19 released

[Jérémy Korwin-Zmijowski]

Very inspiring work !

Le 16 décembre 2018 21:21:21 GMT+01:00, Jan Nieuwenhuizen <janneke@gnu.org> a écrit :
>We are pleased to announce the release of GNU Mes 0.19, representing
>100 commits over 10 weeks.
>
>Mes has now brought the Reduced Binary Seed bootstrap to Guix
>(bootstrap
>a GNU/Linux system without binary GNU toolchain or equivalent) and work
>is ongoing to audit and verify this bootstrap path in NixOS.
>
>This release introduces strings as byte-array, hash-tables and native
>structs.  While that does increase the footprint somewhat, it fixes
>our performance issue; tinycc is now compiled in ~8min (WAS: ~1h30).
>
>Next targets:
>
> - translate mes.c into unsnarfed mes.M2
> - use Gash to remove bash, coreutils&co, grep, sed, tar from the Guix
>   bootstrap binaries
> - replace the NixOS bootstrap
> - use dietlibc, uClibc, ... for bootstrapping GNU (bash, binutils,
>   gcc, tar) and remove Mes C lib+gnu?
> - bootstrap gcc-3.x or 4.x directly, drop initial gcc-2.95.3 target?
> - have M1+hex2 create gcc/tcc-usable object files?  archives?
> - Debian?
> - ARM, the Hurd?
>
>Packages are available from Guix's core-updates branch.
>
>* About
>
>  GNU Mes[0] brings a Reduced Binary Seed bootstrap[1] to Guix[2] and
>  potentially to any other interested GNU/Linux distribution, and aims
>  to help create a full source bootstrap as part of the
>  bootstrappable builds[3] effort.
>
>  It consists of a mutual self-hosting Scheme interpreter written in
> ~5,000 LOC of simple C and a Nyacc-based C compiler written in Scheme.
>  This mes.c is being simplified[4] to be transpiled by M2-Planet[5].
>
>  The Scheme interpreter (mes.c) has a Garbage Collector, a library of
>loadable Scheme modules-- notably Dominique Boucher's LALR[6], Pre-R6RS
>[portable syntax-case[7] with R7RS ellipsis, Matt Wette's Nyacc[8]
>--and test
>  suite just barely enough to support a simple REPL and simple
>  C-compiler: MesCC.
>
>  Mes+MesCC can compile an only lightly patched TinyCC[9] that is
>  self-hosting.  Using this tcc and the Mes C library we now have a
>  Reduced Binary Seed bootstrap for the gnutools triplet: glibc-2.2.5,
>  binutils-2.20.1, gcc-2.95.3.  This is enough to bootstrap Guix for
>  i686-linux and x86_64-linux.
>
>Mes is inspired by The Maxwell Equations of Software: LISP-1.5[10] --
>John
>  McCarthy page 13, GNU Guix's source/binary packaging transparency and
>  Jeremiah Orians's stage0[11] ~500 byte self-hosting hex assembler.
>
>* Download
>
>  git clone git://git.savannah.gnu.org/mes.git
>
>  Here are the compressed sources and a GPG detached signature[*]:
>    https://ftp.gnu.org/gnu/mes/mes-0.19.tar.gz
>    https://ftp.gnu.org/gnu/mes/mes-0.19.tar.gz.sig
>
>  Use a mirror for higher download bandwidth:
>    https://ftpmirror.gnu.org/mes/mes-0.19.tar.gz
>    https://ftpmirror.gnu.org/mes/mes-0.19.tar.gz.sig
>
>  Here are the MD5 and SHA1 checksums:
>
>  99e134df87adc5fc5fd2c04941929c23  mes-0.19.tar.gz
>  c9781b3b6a814acc985c2ac68caa111a56583bca  mes-0.19.tar.gz
>
> [*] Use a .sig file to verify that the corresponding file (without the
> .sig suffix) is intact.  First, be sure to download both the .sig file
>  and the corresponding tarball.  Then, run a command like this:
>
>    gpg --verify mes-0.19.tar.gz.sig
>
>  If that command fails because you don't have the required public key,
>  then run this command to import it:
>
>gpg --keyserver keys.gnupg.net --recv-keys
>1A858392E331EAFDB8C27FFBF3C1A0D9C1D65273
>
>  and rerun the 'gpg --verify' command.
>
>Mes runs from the source tree and can also be built, packaged and
>installed in Guix from a git checkout by running
>
>    guix package -f .guix.scm
>
>* Get informed, get involved
>
>    See https://bootstrappable.org
>    Join #bootstrappable on irc.freenode.net.
>
>* Changes in 0.19 since 0.18
> ** Core
> *** The build system has been simplified.
> *** Mes now prints a backtrace upon error.
>*** Performance has been improved 2-8 times, making Mes 2-10 times
>slower than Guile.
> *** Mes now supports a module type and uses a `boot-module'.
> *** Mes now supports a hash_table type.
> *** Mes now supports a struct type.
> *** Mes now supports building a %bootstrap-mes seed from Guix.
> ** Language
> *** Records are now implemented using struct (WAS: vector).
> *** 44 new functions
> ceil, char-downcase, char-set-adjoin, char-set-complement,
> char-upcase, current-time, delete-file, dup, dup2, file-exists?,
> floor, frame-printer, get-internal-run-time, getcwd, gettimeofday,
> hash, hash-ref, hash-set!, hash-table-printer, hashq,
> hashq-get-handle, hashq-ref, hashq-set, inexact->exact,
> make-hash-table, make-stack, make-struct, module-define!,
> module-printer, module-ref, module-variable, read-line, round,
> stack-length, stack-ref, string-downcase, string-tokenize,
> string-upcase, struct-length, struct-ref, struct-set! struct-vtable,
> struct-vtable, with-error-to-file.
> ** MesCC
>*** Assembly defines have been cleaned-up: duplicates deleted, missing
>added, wrong fixed.
> *** MesCC now supports compiling GNU Bash and GNU Tar.
> **** 6 New functions
> getegid, geteuid, getppid, setgid, setuid, sigdelset, sigprocmask.
> **** 22 New macros
> EACCES, ENOSPC, ESPIPE, INT16_MAX, INT16_MIN, INT32_MAX, INT32_MIN,
> INT64_MAX, INT64_MIN, INT8_MAX, INT8_MIN, LLONG_MAX, LLONG_MIN,
> SIZE_MAX SYS_getegid, SYS_geteuid, SYS_setgid SYS_setuid, S_IRGRP,
> S_IROTH, S_IRWXG, S_IRWXO S_ISGID, S_ISUID, S_IWGRP, S_IWOTH, S_IXGRP,
> S_IXOTH, UINT16_MAX, UINT32_MAX, UINT64_MAX, UINT8_MAX,
> _POSIX_VERSION.
> ** Noteworthy bug fixes
> *** Mes now supports characters #\xNN.
> *** Mes now supports assq-ref and assoc-ref with alist == #f.
> *** Mes now supports \xNN in strings.  This allows using Nyacc-0.86.0.
> *** MesCC now supports the unary plus operator.
> *** MesCC now supports the `U' integer suffix.
> *** MesCC now comes with INTnn_MIN/MAX, UINTnn defines in stdint.h.
> *** MesCC now always exits non-zero when assembler or linker fail.
>
>Greetings,
>janneke
>
>[0] https://www.gnu.org/software/mes
>[1] http://joyofsource.com/reduced-binary-seed-bootstrap.html
>[2] https://www.gnu.org/software/guix
>[3] https://bootstrappable.org
>[4] https://github.com/oriansj/mes-m2
>[5] https://github.com/oriansj/m2-planet
>[6] https://github.com/schemeway/lalr-scm
>[7] https://www.cs.indiana.edu/chezscheme/syntax-case/old-psyntax.html
>[8] https://www.nongnu.org/nyacc
>[9] https://gitlab.com/janneke/tinycc
>[10]
>http://www.softwarepreservation.org/projects/LISP/book/LISP%201.5%20Programmers%20Manual.pdf
>[11] https://github.com/oriansj/stage0
>
>-- 
>Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
>Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

-- 
Jeko

Re: [bootstrappable] Re: GNU Mes 0.19 released

[Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1053 bytes --]


Ricardo Wurmus <rekado@elephly.net> writes:

> Hi Arne,
>
>> Do you have a document showing the bootstrap path — ideally in graph
>> form, i.e. with plantuml activity diagram?
>> http://plantuml.com/activity-diagram-legacy
>
> You can see the current bootstrap path here:
>
>     https://bootstrappable.org/projects/mes.html
>
> This is without the path from virtually nothing to mes + mescc tools,
> i.e. mes and mescc tools currently are bootstrap binaries here.

This is really cool! Thank you!

>>>  - use Gash to remove bash, coreutils&co, grep, sed, tar from the Guix
>>>    bootstrap binaries
>>
>> Is this about configure+make?
>
> You can see in the graph that “bootstrap-binaries” is a collection of
> statically linked binaries including tar, gzip, patch, coreutils, etc.
> Most if not all of these could be removed with Gash.

I do not see gmake in there. Is that covered by one of them, or can it
be built with them?

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1076 bytes --]

Re: [bootstrappable] Re: GNU Mes 0.19 released

[Jan Nieuwenhuizen]

Arne Babenhauserheide writes:

Hi Arne,

>> You can see the current bootstrap path here:
>>
>>     https://bootstrappable.org/projects/mes.html

> I do not see gmake in there. Is that covered by one of them, or can it
> be built with them?

`make-mesboot0' is the initial build of GNU make.  You can read the full
Guix bootstrap build recipe here:

    http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/commencement.scm?h=core-updates

Greetings,
janneke

-- 
Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org
Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com

Re: [bootstrappable] Re: GNU Mes 0.19 released

[Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 393 bytes --]


Jan Nieuwenhuizen <janneke@gnu.org> writes:
> `make-mesboot0' is the initial build of GNU make.  You can read the full
> Guix bootstrap build recipe here:
>
>     http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/commencement.scm?h=core-updates

Ah, nice!

Again: Thank you very much!

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1076 bytes --]