unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Creating a reliable bootstrap for building from source
  2017-05-14 17:29             ` pjotr.public12
@ 2017-05-14 18:30               ` Pjotr Prins
  2017-05-14 21:32                 ` Ludovic Courtès
  0 siblings, 1 reply; 12+ messages in thread
From: Pjotr Prins @ 2017-05-14 18:30 UTC (permalink / raw)
  To: Jan Nieuwenhuizen, guix-devel

Thinking about it, what I want to achieve is that we can take the
latest git tree and bootstrap by building guix and packages. This
should be easy, since I have guix running, but it is not. And the main
trouble is that the underlying build packages can differ over time. I
am looking at gcc versions and guile versions.  I.e., we are building
on shifting sands. How unguixy!

The combination of 'guix pull' held a promise, were it not that pull is
also iffy. Probably for pretty much the same reason.

The bootstrap+configure scripts try to work that, but actually
address a wider case. I.e. people who want to bootstrap in Debian etc.
I don't think we need al that. I write Makefile.guix for my projects
and they tend to be simple! Once you can assume Guix is there life
gets simple as a developer - except when you try to bootstrap :0

The instruction I would like to write for others is:

1. Install the latest bootstrap-guix-from-source package after a guix pull
2. git clone guix && cd guix
3. run make -f Makefile.guix  

(no configure is needed in guix!)

4. ./pre-inst guix etc. etc.

Now the bootstrap-guix-from-source package is easy to create and needs
one single test: build guix from the git tree. It should have a
dependency on the git tree itself, so it gets forced every time on the
build farm.

Once we have that we can even bootstrap using guix pack and such.

I realise potluck could be an alternative, but a working bootstrap
would make guix pull also reliable and we can test it effectively on
the build farm.

Pj. 

(unguixy, I like that. It applies to many deployment statements. The
world is an unguixy place)

^ permalink raw reply	[flat|nested] 12+ messages in thread

* RE: Creating a reliable bootstrap for building from source
@ 2017-05-14 19:54 Jeremiah
  2017-05-14 21:17 ` Pjotr Prins
  0 siblings, 1 reply; 12+ messages in thread
From: Jeremiah @ 2017-05-14 19:54 UTC (permalink / raw)
  To: guix-devel

> Thinking about it, what I want to achieve is that we can take the
> latest git tree and bootstrap by building guix and packages. This
> should be easy, since I have guix running, but it is not. And the main
> trouble is that the underlying build packages can differ over time. I
> am looking at gcc versions and guile versions.  I.e., we are building
> on shifting sands. How unguixy!

It is worse than that, Janneke and I are still trying to build out a
full source bootstrap.

Now mind you we have gotten quite a bit down that rabbit hole.
( I've build from a hex monitor to a Lexically scoped garbage
compacting/collecting lisp and Janneke built his rather impressive MES
which already supports large parts of the C language and enough to
bootstrap some rather important pieces)

But there still are many gaps left to close (how to bootstrap a 280 byte
hex monitor without a hex monitor or hex assembler, stage0-vm
downstrapping, MES tinycc bootstrapping, MES lisp bootstrapping, etc)

but ultimately shifting sands are the only grounds we can be certain
will be there.

So we better get comfortable minimizing our assumptions.

-Jeremiah

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Creating a reliable bootstrap for building from source
  2017-05-14 19:54 Creating a reliable bootstrap for building from source Jeremiah
@ 2017-05-14 21:17 ` Pjotr Prins
  0 siblings, 0 replies; 12+ messages in thread
From: Pjotr Prins @ 2017-05-14 21:17 UTC (permalink / raw)
  To: Jeremiah; +Cc: guix-devel

On Sun, May 14, 2017 at 03:54:27PM -0400, Jeremiah@pdp10.guru wrote:
> > Thinking about it, what I want to achieve is that we can take the
> > latest git tree and bootstrap by building guix and packages. This
> > should be easy, since I have guix running, but it is not. And the main
> > trouble is that the underlying build packages can differ over time. I
> > am looking at gcc versions and guile versions.  I.e., we are building
> > on shifting sands. How unguixy!
> 
> It is worse than that, Janneke and I are still trying to build out a
> full source bootstrap.
> 
> Now mind you we have gotten quite a bit down that rabbit hole.
> ( I've build from a hex monitor to a Lexically scoped garbage
> compacting/collecting lisp and Janneke built his rather impressive MES
> which already supports large parts of the C language and enough to
> bootstrap some rather important pieces)
> 
> But there still are many gaps left to close (how to bootstrap a 280 byte
> hex monitor without a hex monitor or hex assembler, stage0-vm
> downstrapping, MES tinycc bootstrapping, MES lisp bootstrapping, etc)
> 
> but ultimately shifting sands are the only grounds we can be certain
> will be there.
> 
> So we better get comfortable minimizing our assumptions.

What you are trying to do is even more heroic - bootstrapping all of
guix :)

What I want is achievable, simply have the build system as a tested
binary available. A tested builder package that is know to work
against the current tree. We pretty much have it, it is only not
formalized and tested on the build farm.

Pj.
-- 

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Creating a reliable bootstrap for building from source
  2017-05-14 18:30               ` Creating a reliable bootstrap for building from source Pjotr Prins
@ 2017-05-14 21:32                 ` Ludovic Courtès
  2017-05-15  1:20                   ` David Pirotte
                                     ` (2 more replies)
  0 siblings, 3 replies; 12+ messages in thread
From: Ludovic Courtès @ 2017-05-14 21:32 UTC (permalink / raw)
  To: Pjotr Prins; +Cc: guix-devel

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> The combination of 'guix pull' held a promise, were it not that pull is
> also iffy. Probably for pretty much the same reason.
>
> The bootstrap+configure scripts try to work that, but actually
> address a wider case. I.e. people who want to bootstrap in Debian etc.
> I don't think we need al that. I write Makefile.guix for my projects
> and they tend to be simple! Once you can assume Guix is there life
> gets simple as a developer - except when you try to bootstrap :0
>
> The instruction I would like to write for others is:
>
> 1. Install the latest bootstrap-guix-from-source package after a guix pull
> 2. git clone guix && cd guix
> 3. run make -f Makefile.guix  
>
> (no configure is needed in guix!)
>
> 4. ./pre-inst guix etc. etc.

I think there are two very different use cases.

As a user I want something like ‘apt-get update’, which is what ‘guix
pull’ tries to do.

For Guix developers, I think it’s reasonable to have a traditional GNU
build system.  After all, Guix is also a regular software package that
people can build from source with “./configure && make && make install”.

My 2¢,
Ludo’.

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Creating a reliable bootstrap for building from source
@ 2017-05-15  0:11 Jeremiah
  0 siblings, 0 replies; 12+ messages in thread
From: Jeremiah @ 2017-05-15  0:11 UTC (permalink / raw)
  To: pjotr.public12; +Cc: guix-devel

> What you are trying to do is even more heroic - bootstrapping all of
> guix :)

The real heros are those that check our work, report bugs and give us
insight into solving our most difficult problems.

And we LOVE patches and pull requests ;-D

> What I want is achievable, simply have the build system as a tested
> binary available. A tested builder package that is know to work
> against the current tree. We pretty much have it, it is only not
> formalized and tested on the build farm

If what you want is achievable and going to help, then please do it.
No need to ask, we always love people who take time to make things
better.

-Jeremiah

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Creating a reliable bootstrap for building from source
  2017-05-14 21:32                 ` Ludovic Courtès
@ 2017-05-15  1:20                   ` David Pirotte
  2017-05-15 13:27                     ` Ludovic Courtès
  2017-05-15  7:35                   ` Pjotr Prins
  2017-05-17  7:52                   ` Pjotr Prins
  2 siblings, 1 reply; 12+ messages in thread
From: David Pirotte @ 2017-05-15  1:20 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 1080 bytes --]

Heya,

> As a user I want something like ‘apt-get update’, which is what ‘guix
> pull’ tries to do.

For end-users, cool stuff would be:

	nguix	[ guix-ncurse

		u	[ update the aptitude cached list of all packages,
			[ installed or not... just package descriptions, nothing
			[ has been downloaded yet
		U	[ mark as 'wish to update'
		g	[ list the above marked for update packages, then fine tune
			[ using =, -, + ...
		g	ok, download what I really selected to be updated
			and update...

		note that the most important aptitude ncurse menu is 'Cancel pending
		actions' :)

	or using emacs, 'à la magit'

	M-x guix-status
		u
		U
		...

> For Guix developers, I think it’s reasonable to have a traditional GNU
> build system.  After all, Guix is also a regular software package that
> people can build from source with “./configure && make && make install”.

If I could just grab guile-gnutls and  “./configure && make && make install”
then I could compile, play, use, learn, contribute with/to guix...

Cheers,
David

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Creating a reliable bootstrap for building from source
  2017-05-14 21:32                 ` Ludovic Courtès
  2017-05-15  1:20                   ` David Pirotte
@ 2017-05-15  7:35                   ` Pjotr Prins
  2017-05-15 13:28                     ` Ludovic Courtès
  2017-05-17  7:52                   ` Pjotr Prins
  2 siblings, 1 reply; 12+ messages in thread
From: Pjotr Prins @ 2017-05-15  7:35 UTC (permalink / raw)
  To: Ludovic Court??s; +Cc: guix-devel

On Sun, May 14, 2017 at 11:32:06PM +0200, Ludovic Court??s wrote:
> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
> 
> > The combination of 'guix pull' held a promise, were it not that pull is
> > also iffy. Probably for pretty much the same reason.
> >
> > The bootstrap+configure scripts try to work that, but actually
> > address a wider case. I.e. people who want to bootstrap in Debian etc.
> > I don't think we need al that. I write Makefile.guix for my projects
> > and they tend to be simple! Once you can assume Guix is there life
> > gets simple as a developer - except when you try to bootstrap :0
> >
> > The instruction I would like to write for others is:
> >
> > 1. Install the latest bootstrap-guix-from-source package after a guix pull
> > 2. git clone guix && cd guix
> > 3. run make -f Makefile.guix  
> >
> > (no configure is needed in guix!)
> >
> > 4. ./pre-inst guix etc. etc.
> 
> I think there are two very different use cases.
> 
> As a user I want something like 'apt-get update', which is what 'guix
> pull' tries to do.

Sure. But from my previous E-mail you can see we are effectively using
pull to bootstrap the source tree build.

> For Guix developers, I think it's reasonable to have a traditional GNU
> build system.  After all, Guix is also a regular software package that
> people can build from source with './configure && make && make install'.

My point is that we can simplify. I like simple. Simple is good.

We can have both the configure and a simple Makefile.guix option. That
is what I do with my projects.

We do not need bootstrap, autoconf and configure on a running Guix
system. We do need it for other distributions.

Anyway, feel free to ignore this idea.

Pj.
-- 

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Creating a reliable bootstrap for building from source
  2017-05-15  1:20                   ` David Pirotte
@ 2017-05-15 13:27                     ` Ludovic Courtès
  0 siblings, 0 replies; 12+ messages in thread
From: Ludovic Courtès @ 2017-05-15 13:27 UTC (permalink / raw)
  To: David Pirotte; +Cc: guix-devel

Hi David,

David Pirotte <david@altosw.be> skribis:

>> As a user I want something like ‘apt-get update’, which is what ‘guix
>> pull’ tries to do.
>
> For end-users, cool stuff would be:
>
> 	nguix	[ guix-ncurse
>
> 		u	[ update the aptitude cached list of all packages,
> 			[ installed or not... just package descriptions, nothing
> 			[ has been downloaded yet
> 		U	[ mark as 'wish to update'
> 		g	[ list the above marked for update packages, then fine tune
> 			[ using =, -, + ...
> 		g	ok, download what I really selected to be updated
> 			and update...
>
> 		note that the most important aptitude ncurse menu is 'Cancel pending
> 		actions' :)
>
> 	or using emacs, 'à la magit'
>
> 	M-x guix-status
> 		u
> 		U
> 		...

How would that differ from what Emacs-Guix provides?

>> For Guix developers, I think it’s reasonable to have a traditional GNU
>> build system.  After all, Guix is also a regular software package that
>> people can build from source with “./configure && make && make install”.
>
> If I could just grab guile-gnutls and  “./configure && make && make install”
> then I could compile, play, use, learn, contribute with/to guix...

David, if your distro does not provide a guile-gnutls package, you can
always install Guix via the binary installation method:

  https://www.gnu.org/software/guix/manual/html_node/Binary-Installation.html

HTH!

Ludo’.

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Creating a reliable bootstrap for building from source
  2017-05-15  7:35                   ` Pjotr Prins
@ 2017-05-15 13:28                     ` Ludovic Courtès
  2017-05-17  7:47                       ` Pjotr Prins
  0 siblings, 1 reply; 12+ messages in thread
From: Ludovic Courtès @ 2017-05-15 13:28 UTC (permalink / raw)
  To: Pjotr Prins; +Cc: guix-devel

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Sun, May 14, 2017 at 11:32:06PM +0200, Ludovic Court??s wrote:
>> Pjotr Prins <pjotr.public12@thebird.nl> skribis:
>> 
>> > The combination of 'guix pull' held a promise, were it not that pull is
>> > also iffy. Probably for pretty much the same reason.
>> >
>> > The bootstrap+configure scripts try to work that, but actually
>> > address a wider case. I.e. people who want to bootstrap in Debian etc.
>> > I don't think we need al that. I write Makefile.guix for my projects
>> > and they tend to be simple! Once you can assume Guix is there life
>> > gets simple as a developer - except when you try to bootstrap :0
>> >
>> > The instruction I would like to write for others is:
>> >
>> > 1. Install the latest bootstrap-guix-from-source package after a guix pull
>> > 2. git clone guix && cd guix
>> > 3. run make -f Makefile.guix  
>> >
>> > (no configure is needed in guix!)
>> >
>> > 4. ./pre-inst guix etc. etc.
>> 
>> I think there are two very different use cases.
>> 
>> As a user I want something like 'apt-get update', which is what 'guix
>> pull' tries to do.
>
> Sure. But from my previous E-mail you can see we are effectively using
> pull to bootstrap the source tree build.

Sure.

>> For Guix developers, I think it's reasonable to have a traditional GNU
>> build system.  After all, Guix is also a regular software package that
>> people can build from source with './configure && make && make install'.
>
> My point is that we can simplify. I like simple. Simple is good.
>
> We can have both the configure and a simple Makefile.guix option. That
> is what I do with my projects.
>
> We do not need bootstrap, autoconf and configure on a running Guix
> system. We do need it for other distributions.

I agree.  build-aux/build-self.scm, which is what ‘guix pull’ runs to
build Guix, is close to what you’re suggesting, IIUC: a pure-Guile build
script.  WDYT?

Ludo’.

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Creating a reliable bootstrap for building from source
  2017-05-15 13:28                     ` Ludovic Courtès
@ 2017-05-17  7:47                       ` Pjotr Prins
  2017-05-19  8:58                         ` Ludovic Courtès
  0 siblings, 1 reply; 12+ messages in thread
From: Pjotr Prins @ 2017-05-17  7:47 UTC (permalink / raw)
  To: Ludovic Court??s; +Cc: guix-devel

On Mon, May 15, 2017 at 03:28:58PM +0200, Ludovic Court??s wrote:
> >> For Guix developers, I think it's reasonable to have a traditional GNU
> >> build system.  After all, Guix is also a regular software package that
> >> people can build from source with './configure && make && make install'.
> >
> > My point is that we can simplify. I like simple. Simple is good.
> >
> > We can have both the configure and a simple Makefile.guix option. That
> > is what I do with my projects.
> >
> > We do not need bootstrap, autoconf and configure on a running Guix
> > system. We do need it for other distributions.
> 
> I agree.  build-aux/build-self.scm, which is what ???guix pull??? runs to
> build Guix, is close to what you???re suggesting, IIUC: a pure-Guile build
> script.  WDYT?

It is interesting - especially the hoops jumping around guile
versioning ;) - but it still looks like it invokes the traditional
gnu-build-system. I think we can do without that too. 

Pj.
-- 

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Creating a reliable bootstrap for building from source
  2017-05-14 21:32                 ` Ludovic Courtès
  2017-05-15  1:20                   ` David Pirotte
  2017-05-15  7:35                   ` Pjotr Prins
@ 2017-05-17  7:52                   ` Pjotr Prins
  2 siblings, 0 replies; 12+ messages in thread
From: Pjotr Prins @ 2017-05-17  7:52 UTC (permalink / raw)
  To: Ludovic Court??s; +Cc: guix-devel

On Sun, May 14, 2017 at 11:32:06PM +0200, Ludovic Court??s wrote:
> For Guix developers, I think it???s reasonable to have a traditional GNU
> build system.  After all, Guix is also a regular software package that
> people can build from source with ???./configure && make && make install???.

This is the procedure I have now for creating a reliable bootstrap
from source:

The safest route is by using guix environment after starting
a clean shell

  screen -S guix-build # I tend to build in screen
  env -i /bin/bash --login --noprofile --norc
  ~/.guix-profile/bin/guix environment guix --ad-hoc help2man git strace \
    pkg-config less vim binutils coreutils grep --no-grafts
  bash # you may want this shell

In fact pick the most recent guix you have got, see 'ls
/gnu/store/*guix*/bin/guix' and run that command. Use the --no-grafts
switch if you have built packages that way before.

Note that you can start guix by installing the binary tar ball, or
copying it from another machine using the rather useful guix archive
or [[https://www.gnu.org/software/guix/news/creating-bundles-with-guix-pack.html][guix pack]] commands.

You may want to take a note of these running versions

  gcc --version
  guile --version

Next in the source tree

  rm -rf autom4te.cache/ # to be sure
  make clean
  ./bootstrap
  ./configure --localstatedir=/var
  make clean # to be really sure
  time make

It works reliably on all my setups :). Note that typically gcc is v4 or v5 and guile 2.0.x,
but never the same.

I am pleased with this until it stops working ... I think it should go in the manual and
we should have an automated test on this procedure.

Pj.
-- 

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: Creating a reliable bootstrap for building from source
  2017-05-17  7:47                       ` Pjotr Prins
@ 2017-05-19  8:58                         ` Ludovic Courtès
  0 siblings, 0 replies; 12+ messages in thread
From: Ludovic Courtès @ 2017-05-19  8:58 UTC (permalink / raw)
  To: Pjotr Prins; +Cc: guix-devel

Pjotr Prins <pjotr.public12@thebird.nl> skribis:

> On Mon, May 15, 2017 at 03:28:58PM +0200, Ludovic Court??s wrote:
>> >> For Guix developers, I think it's reasonable to have a traditional GNU
>> >> build system.  After all, Guix is also a regular software package that
>> >> people can build from source with './configure && make && make install'.
>> >
>> > My point is that we can simplify. I like simple. Simple is good.
>> >
>> > We can have both the configure and a simple Makefile.guix option. That
>> > is what I do with my projects.
>> >
>> > We do not need bootstrap, autoconf and configure on a running Guix
>> > system. We do need it for other distributions.
>> 
>> I agree.  build-aux/build-self.scm, which is what ???guix pull??? runs to
>> build Guix, is close to what you???re suggesting, IIUC: a pure-Guile build
>> script.  WDYT?
>
> It is interesting - especially the hoops jumping around guile
> versioning ;) - but it still looks like it invokes the traditional
> gnu-build-system.

No, (guix scripts pull) really just traverses the list of .scm files and
compiles them.

Ludo’.

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2017-05-19  8:58 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-05-14 19:54 Creating a reliable bootstrap for building from source Jeremiah
2017-05-14 21:17 ` Pjotr Prins
  -- strict thread matches above, loose matches on Subject: below --
2017-05-15  0:11 Jeremiah
2017-03-15 16:13 Ready for Guile 2.2! Ludovic Courtès
2017-04-22 22:34 ` ‘guix pull’ vs. transition to Guile 2.2 Ludovic Courtès
2017-05-09 21:22   ` Heads-up: " Ludovic Courtès
2017-05-14 13:50     ` Pjotr Prins
2017-05-14 15:35       ` Pjotr Prins
2017-05-14 16:13         ` Pjotr Prins
2017-05-14 16:28           ` Jan Nieuwenhuizen
2017-05-14 17:29             ` pjotr.public12
2017-05-14 18:30               ` Creating a reliable bootstrap for building from source Pjotr Prins
2017-05-14 21:32                 ` Ludovic Courtès
2017-05-15  1:20                   ` David Pirotte
2017-05-15 13:27                     ` Ludovic Courtès
2017-05-15  7:35                   ` Pjotr Prins
2017-05-15 13:28                     ` Ludovic Courtès
2017-05-17  7:47                       ` Pjotr Prins
2017-05-19  8:58                         ` Ludovic Courtès
2017-05-17  7:52                   ` Pjotr Prins

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).