Using bootstrap seeds write package definitions

Using bootstrap seeds write package definitions

[c4droid]

Hello guix!
I was using the Linux From Scratch for a while as my mainly linux distro, use soft link as package management. when I heard guix, I think I found solution for my LFS package management. I tried guix and guix system few month, Is time for switch my LFS package manager to guix.
When I watched the manual talk about bootstrapping guix, I don't know how to writing some package definitions with bootstrap-seeds, I was searching on the google and ask questions on irc, because building the LFS, want build the cross toolchain (stage 1) then build final system (stage 2), last is kernel and initramfs (stage 3). after that is build Beyond LFS (stage 4). I stuck with using bootstrap seeds writing stage 1 package definitions. So I ask for help on community, because the power of community is great. :)
p.s.: Sorry for my poor english
------
Best reguards
c4droid

Re: Using bootstrap seeds write package definitions

[Julien Lepiller]

Hi c4droid,

Le Sun, 16 May 2021 10:50:36 +0800,
"c4droid" <c4droid@foxmail.com> a écrit :

> Hello guix!
> I was using the Linux From Scratch for a while as my mainly linux
> distro, use soft link as package management. when I heard guix, I
> think I found solution for my LFS package management. I tried guix
> and guix system few month, Is time for switch my LFS package manager
> to guix.

I've been using LFS (and my own LFS-based "distro") for some years, so
I completely understand why you'd want to use guix. Actually I based my
"distro" on what I understood from Guix and the package manager from
Haiku. When I understood I implemented that badly, I switched to Guix
and Guix System entirely, no more LFS :D.

> When I watched the manual talk about bootstrapping guix, I don't know
> how to writing some package definitions with bootstrap-seeds, I was
> searching on the google and ask questions on irc, because building
> the LFS, want build the cross toolchain (stage 1) then build final
> system (stage 2), last is kernel and initramfs (stage 3). after that
> is build Beyond LFS (stage 4). I stuck with using bootstrap seeds
> writing stage 1 package definitions. So I ask for help on community,
> because the power of community is great. :)

So Guix comes with a set of packages, which are just recipes to build
them from source. There are two things you might want to do: you can
either use the existing packages or create your own recipes for the
whole system.

If you choose to use our packages, then there are again two options.
You can use the existing binary bootstrap (the only thing you wouldn't
build yourself), or build it yourself and replace the binaries used in
Guix by your own. Note that the bootstrap binaries are different from
the binaries built for the first chroot of LFS (the cross-compiler). We
do not use a binary GCC, but we instead start from mescc, a small C
compiler, and use it to build tcc, then gcc, with the goal of reducing
that down to stage0, which is only a few hundred bytes of binary, plus
sources of other packages.

# Using our recipes and bootstrap binaries

If you want to use our bootstrap binaries, there is nothing to do,
because Guix itself will be able to get the binary bootstrap and build
everything from it. Do not enable substitutes if you want to build
everything yourself.

# Providing your own bootstrap binaries

If you prefer to provide your own bootstrap, you'll have to build and
provide binaries for the bootstrap seeds we currently have in Guix,
that you see in the manual.

# Building your own packages

If you decide not to use our recipes, you must write your own. The set
of packages is rooted in the bootstrap seeds, as you noticed. However,
if you want to provide your own recipes, you are basically free to
provide any bootstrap and further packages you want. What you could do
is to build stage1 (cross toolchain), use that as your binary seed, and
build the other stages as guix packages. Note that for that, you'll
need to have guix already installed on the host system.

If you have never used Guix, or never written a package definition, you
might want to have a look at our short packaging tutorial at
https://guix.gnu.org/en/cookbook/en/html_node/Packaging-Tutorial.html#Packaging-Tutorial
(French and German are also available if you prefer one of these
languages).

There is some code in gnu/packages/bootstrap.scm that you might be
interested in. It has some interesting bits: first there are a few
binary seeds with hashes: they are statically linked binaries that are
needed to decompress other seeds. Then, we define a few special
procedures that replace the normal packaging procedures
(bootstrap-origin, package-from-tarball). You probably want to use them
or get inspiration from them to create your own bootstrap recipes
(packages that simply decompress a tarball containing a prebuilt
binary). The rest of the file lists these packages.

Then gnu/packages/commencement.scm is the very beginning of the package
graph. These packages use the binary seeds to build up to gcc, that we
later use in the gnu-build-system, etc. You will be interested in the
comments in this file too :)

# My own experience with my package manager (not guix)

If I recall correctly, what I did when I had my own "distro" (and
package manager), is that I first built the cross-toolchain and base
system from LFS (it was pre-10.0, and we built the cross-compiler and
use it firts outside the chroot to build the base build toolchain we
could then chroot to). I would first manually create a package from the
chroot, and install a first system with only that package, then chroot
to it. Then, from this system and the chroot package, I was able to
build the base system with my package manager. Once I had the base LFS
system (stage 2), that system had a dependency on the chroot package,
which was not acceptable for me, so I replaced all the recipes to
depend on the previously built version, turning stage2 into a set of
bootstrap binaries. I rebuilt stage2 from the new bootstrap, and
continued to write packages for other packages.

Once I had my packages, I also wrote some code to create and install
the initramfs and kernel (stage3).

At this point, I could boot the new system, and continue with stage 4 :)

Note that, in order to use Guix, you'll need Guix in the chroot
package and all its dependencies, which is bigger than what LFS makes
you build initially.

Hope this is useful! Before I can help you further, I think I need to
know which route you want to follow, as they are very different.

> p.s.: Sorry for my poor english

Hehe, don't worry. You can use your mother tongue if you prefer on this
mailing list, although I can only help in English or French.

> ------
> Best reguards c4droid

Re: Using bootstrap seeds write package definitions

[c4droid]

Julien Lepiller writes:

> Hi c4droid,
>
> Le Sun, 16 May 2021 10:50:36 +0800,
> "c4droid" <c4droid@foxmail.com> a écrit :
>
>> Hello guix!
>> I was using the Linux From Scratch for a while as my mainly linux
>> distro, use soft link as package management. when I heard guix, I
>> think I found solution for my LFS package management. I tried guix
>> and guix system few month, Is time for switch my LFS package manager
>> to guix.
>
> I've been using LFS (and my own LFS-based "distro") for some years, so
> I completely understand why you'd want to use guix. Actually I based my
> "distro" on what I understood from Guix and the package manager from
> Haiku. When I understood I implemented that badly, I switched to Guix
> and Guix System entirely, no more LFS :D.

I was use pkgtool from slackware and write some dirty shell script
emulate GNU Stow for my package management. It's so difficult for manage
soft link.

>
>> When I watched the manual talk about bootstrapping guix, I don't know
>> how to writing some package definitions with bootstrap-seeds, I was
>> searching on the google and ask questions on irc, because building
>> the LFS, want build the cross toolchain (stage 1) then build final
>> system (stage 2), last is kernel and initramfs (stage 3). after that
>> is build Beyond LFS (stage 4). I stuck with using bootstrap seeds
>> writing stage 1 package definitions. So I ask for help on community,
>> because the power of community is great. :)
>
> So Guix comes with a set of packages, which are just recipes to build
> them from source. There are two things you might want to do: you can
> either use the existing packages or create your own recipes for the
> whole system.

If I can, I want create my own recipes for the whole system with the
help of Guix power

>
> If you choose to use our packages, then there are again two options.
> You can use the existing binary bootstrap (the only thing you wouldn't
> build yourself), or build it yourself and replace the binaries used in
> Guix by your own. Note that the bootstrap binaries are different from
> the binaries built for the first chroot of LFS (the cross-compiler). We
> do not use a binary GCC, but we instead start from mescc, a small C
> compiler, and use it to build tcc, then gcc, with the goal of reducing
> that down to stage0, which is only a few hundred bytes of binary, plus
> sources of other packages.
>
> # Using our recipes and bootstrap binaries
>
> If you want to use our bootstrap binaries, there is nothing to do,
> because Guix itself will be able to get the binary bootstrap and build
> everything from it. Do not enable substitutes if you want to build
> everything yourself.
>
> # Providing your own bootstrap binaries
>
> If you prefer to provide your own bootstrap, you'll have to build and
> provide binaries for the bootstrap seeds we currently have in Guix,
> that you see in the manual.
>

I want to build my own bootstrap binaries, hope get more help with
create my own bootstraps

> # Building your own packages
>
> If you decide not to use our recipes, you must write your own. The set
> of packages is rooted in the bootstrap seeds, as you noticed. However,
> if you want to provide your own recipes, you are basically free to
> provide any bootstrap and further packages you want. What you could do
> is to build stage1 (cross toolchain), use that as your binary seed, and
> build the other stages as guix packages. Note that for that, you'll
> need to have guix already installed on the host system.
>
> If you have never used Guix, or never written a package definition, you
> might want to have a look at our short packaging tutorial at
> https://guix.gnu.org/en/cookbook/en/html_node/Packaging-Tutorial.html#Packaging-Tutorial
> (French and German are also available if you prefer one of these
> languages).
>
> There is some code in gnu/packages/bootstrap.scm that you might be
> interested in. It has some interesting bits: first there are a few
> binary seeds with hashes: they are statically linked binaries that are
> needed to decompress other seeds. Then, we define a few special
> procedures that replace the normal packaging procedures
> (bootstrap-origin, package-from-tarball). You probably want to use them
> or get inspiration from them to create your own bootstrap recipes
> (packages that simply decompress a tarball containing a prebuilt
> binary). The rest of the file lists these packages.
>
> Then gnu/packages/commencement.scm is the very beginning of the package
> graph. These packages use the binary seeds to build up to gcc, that we
> later use in the gnu-build-system, etc. You will be interested in the
> comments in this file too :)
>

If I can, I think use the mescc to build stage0 then build stage1, stage2,
stage3 is the best option.

I'll watch gnu/packages/commencement.scm comments later for get some
inspiration

I'm currently use a Guix System VM for my build machine, and I have a ci
for build continously

> # My own experience with my package manager (not guix)
>
> If I recall correctly, what I did when I had my own "distro" (and
> package manager), is that I first built the cross-toolchain and base
> system from LFS (it was pre-10.0, and we built the cross-compiler and
> use it firts outside the chroot to build the base build toolchain we
> could then chroot to). I would first manually create a package from the
> chroot, and install a first system with only that package, then chroot
> to it. Then, from this system and the chroot package, I was able to
> build the base system with my package manager. Once I had the base LFS
> system (stage 2), that system had a dependency on the chroot package,
> which was not acceptable for me, so I replaced all the recipes to
> depend on the previously built version, turning stage2 into a set of
> bootstrap binaries. I rebuilt stage2 from the new bootstrap, and
> continued to write packages for other packages.
>
> Once I had my packages, I also wrote some code to create and install
> the initramfs and kernel (stage3).
>
> At this point, I could boot the new system, and continue with stage 4 :)
>
> Note that, in order to use Guix, you'll need Guix in the chroot
> package and all its dependencies, which is bigger than what LFS makes
> you build initially.
>
> Hope this is useful! Before I can help you further, I think I need to
> know which route you want to follow, as they are very different.
>

When I see your experience for the package manager, I'm sure using mes
for build stage1 is right option 

>> p.s.: Sorry for my poor english
>
> Hehe, don't worry. You can use your mother tongue if you prefer on this
> mailing list, although I can only help in English or French.
>

Haha, my mother tongue is Mandarin Chinese, translate mandarin to
english and make sure the meanings right is difficult. Because english
is universal language in free software community, so I use english as
possible. :D

>> ------
>> Best reguards c4droid

Re: Using bootstrap seeds write package definitions

[c4droid]

Julien Lepiller writes:

> Hi c4droid,
>
> Le Sun, 16 May 2021 10:50:36 +0800,
> "c4droid" <c4droid@foxmail.com> a écrit :
>
>> Hello guix!
>> I was using the Linux From Scratch for a while as my mainly linux
>> distro, use soft link as package management. when I heard guix, I
>> think I found solution for my LFS package management. I tried guix
>> and guix system few month, Is time for switch my LFS package manager
>> to guix.
>
> I've been using LFS (and my own LFS-based "distro") for some years, so
> I completely understand why you'd want to use guix. Actually I based my
> "distro" on what I understood from Guix and the package manager from
> Haiku. When I understood I implemented that badly, I switched to Guix
> and Guix System entirely, no more LFS :D.
>
>> When I watched the manual talk about bootstrapping guix, I don't know
>> how to writing some package definitions with bootstrap-seeds, I was
>> searching on the google and ask questions on irc, because building
>> the LFS, want build the cross toolchain (stage 1) then build final
>> system (stage 2), last is kernel and initramfs (stage 3). after that
>> is build Beyond LFS (stage 4). I stuck with using bootstrap seeds
>> writing stage 1 package definitions. So I ask for help on community,
>> because the power of community is great. :)
>
> So Guix comes with a set of packages, which are just recipes to build
> them from source. There are two things you might want to do: you can
> either use the existing packages or create your own recipes for the
> whole system.
>
> If you choose to use our packages, then there are again two options.
> You can use the existing binary bootstrap (the only thing you wouldn't
> build yourself), or build it yourself and replace the binaries used in
> Guix by your own. Note that the bootstrap binaries are different from
> the binaries built for the first chroot of LFS (the cross-compiler). We
> do not use a binary GCC, but we instead start from mescc, a small C
> compiler, and use it to build tcc, then gcc, with the goal of reducing
> that down to stage0, which is only a few hundred bytes of binary, plus
> sources of other packages.
>
> # Using our recipes and bootstrap binaries
>
> If you want to use our bootstrap binaries, there is nothing to do,
> because Guix itself will be able to get the binary bootstrap and build
> everything from it. Do not enable substitutes if you want to build
> everything yourself.
>
> # Providing your own bootstrap binaries
>
> If you prefer to provide your own bootstrap, you'll have to build and
> provide binaries for the bootstrap seeds we currently have in Guix,
> that you see in the manual.
>
> # Building your own packages
>
> If you decide not to use our recipes, you must write your own. The set
> of packages is rooted in the bootstrap seeds, as you noticed. However,
> if you want to provide your own recipes, you are basically free to
> provide any bootstrap and further packages you want. What you could do
> is to build stage1 (cross toolchain), use that as your binary seed, and
> build the other stages as guix packages. Note that for that, you'll
> need to have guix already installed on the host system.
>
> If you have never used Guix, or never written a package definition, you
> might want to have a look at our short packaging tutorial at
> https://guix.gnu.org/en/cookbook/en/html_node/Packaging-Tutorial.html#Packaging-Tutorial
> (French and German are also available if you prefer one of these
> languages).
>
> There is some code in gnu/packages/bootstrap.scm that you might be
> interested in. It has some interesting bits: first there are a few
> binary seeds with hashes: they are statically linked binaries that are
> needed to decompress other seeds. Then, we define a few special
> procedures that replace the normal packaging procedures
> (bootstrap-origin, package-from-tarball). You probably want to use them
> or get inspiration from them to create your own bootstrap recipes
> (packages that simply decompress a tarball containing a prebuilt
> binary). The rest of the file lists these packages.
>
> Then gnu/packages/commencement.scm is the very beginning of the package
> graph. These packages use the binary seeds to build up to gcc, that we
> later use in the gnu-build-system, etc. You will be interested in the
> comments in this file too :)
>

I watched gnu/packages/bootstrap.scm and gnu/packages/commencement.scm,
I decide use mes tools and busybox utilities make bootstrap recipes and
my own bootstrap seeds, then follow the gnu/packages/commencement.scm
graph build cross toolchain and so on. But the way, the bootstrap and
commencement module give some inspiration for my idea.

> # My own experience with my package manager (not guix)
>
> If I recall correctly, what I did when I had my own "distro" (and
> package manager), is that I first built the cross-toolchain and base
> system from LFS (it was pre-10.0, and we built the cross-compiler and
> use it firts outside the chroot to build the base build toolchain we
> could then chroot to). I would first manually create a package from the
> chroot, and install a first system with only that package, then chroot
> to it. Then, from this system and the chroot package, I was able to
> build the base system with my package manager. Once I had the base LFS
> system (stage 2), that system had a dependency on the chroot package,
> which was not acceptable for me, so I replaced all the recipes to
> depend on the previously built version, turning stage2 into a set of
> bootstrap binaries. I rebuilt stage2 from the new bootstrap, and
> continued to write packages for other packages.
>
> Once I had my packages, I also wrote some code to create and install
> the initramfs and kernel (stage3).
>
> At this point, I could boot the new system, and continue with stage 4 :)
>
> Note that, in order to use Guix, you'll need Guix in the chroot
> package and all its dependencies, which is bigger than what LFS makes
> you build initially.
>
> Hope this is useful! Before I can help you further, I think I need to
> know which route you want to follow, as they are very different.
>
>> p.s.: Sorry for my poor english
>
> Hehe, don't worry. You can use your mother tongue if you prefer on this
> mailing list, although I can only help in English or French.
>
>> ------
>> Best reguards c4droid