Single-board-computer approach: don't make an installer, make the install?

Single-board-computer approach: don't make an installer, make the install?

[Christopher Lemmer Webber]

I looked recently at this tutorial:

  https://guix.gnu.org/blog/2017/porting-guixsd-to-armv7/

However, it strikes me: maybe there's another nice approach?  Would it
be nicer to just make an image that the user boots into directly on the
beagleboard, if it's going onto a microsd card anyway, rather than
building an installer image?

I haven't tried it yet but can't think of a good reason it wouldn't
work.  Has anyone tried that?

Side note, a RISC-V Beagleboard SoC is coming.  I suspect that might be
very good helping us start ramping up getting Guix very usable on
RISC-V:

  https://arstechnica.com/gadgets/2021/01/seeed-and-beagleboard-team-up-to-provide-a-new-risc-v-based-linux-pc/

Re: Single-board-computer approach: don't make an installer, make the install?

[Mathieu Othacehe]

Hello Christopher,

> I haven't tried it yet but can't think of a good reason it wouldn't
> work.  Has anyone tried that?

Sure, I would advise you to read this[1], which is a sort of follow-up
of this article. You can also listen that talk[2].

I have been successfully producing ready to use disk-images for Pine64,
Danny has done the same for Novena and there are a few people trying to
do so for their Pinebook-pro.

Thanks,

Mathieu

[1]: https://othacehe.org/the-guix-system-image-api.html
[2]: https://archive.fosdem.org/2020/schedule/event/ggaaattyp/

Re: Single-board-computer approach: don't make an installer, make the install?

[Christopher Lemmer Webber]

Mathieu Othacehe writes:

> Hello Christopher,
>
>> I haven't tried it yet but can't think of a good reason it wouldn't
>> work.  Has anyone tried that?
>
> Sure, I would advise you to read this[1], which is a sort of follow-up
> of this article. You can also listen that talk[2].

O-ho!  Very good!

Looking forward to watching the talk tonight. :)

> I have been successfully producing ready to use disk-images for Pine64,
> Danny has done the same for Novena and there are a few people trying to
> do so for their Pinebook-pro.

Extra, super good!

> Thanks,
>
> Mathieu
>
> [1]: https://othacehe.org/the-guix-system-image-api.html
> [2]: https://archive.fosdem.org/2020/schedule/event/ggaaattyp/

Re: Single-board-computer approach: don't make an installer, make the install?

[Joshua Branson via]

Mathieu Othacehe <othacehe@gnu.org> writes:

> Mathieu
>
> [1]: https://othacehe.org/the-guix-system-image-api.html

This is really well written.  May I edit this article and reproduce it
for the guix blog and perhaps the guix manual?  If ludo's ok with me
putting it on the blog, then I intend to end the blog post with "This
article first appeared in this personal blog (link here) and is
reproduced with permission."

> [2]: https://archive.fosdem.org/2020/schedule/event/ggaaattyp/
>

--
Joshua Branson
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help

enough other people get what they want." - Zig Ziglar

Re: Single-board-computer approach: don't make an installer, make the install?

[Christopher Lemmer Webber]

Joshua Branson writes:

> Mathieu Othacehe <othacehe@gnu.org> writes:
>
>> Mathieu
>>
>> [1]: https://othacehe.org/the-guix-system-image-api.html
>
> This is really well written.  May I edit this article and reproduce it
> for the guix blog and perhaps the guix manual?  If ludo's ok with me
> putting it on the blog, then I intend to end the blog post with "This
> article first appeared in this personal blog (link here) and is
> reproduced with permission."

It is good.

Maybe in the cookbook?

>> [2]: https://archive.fosdem.org/2020/schedule/event/ggaaattyp/
>>

Re: Single-board-computer approach: don't make an installer, make the install?

[Mathieu Othacehe]

Hello,

> This is really well written.  May I edit this article and reproduce it
> for the guix blog and perhaps the guix manual?  If ludo's ok with me
> putting it on the blog, then I intend to end the blog post with "This
> article first appeared in this personal blog (link here) and is
> reproduced with permission."

Thanks! I would happily review it :).

There's for sure some work left on that topic. I'd like to:

* Rename "disk-image" command to "image".
* Add "docker" as a new image type.
* Find a better way to enforce system/target constraints, as discussed
here[1].

Mathieu

[1]: https://issues.guix.gnu.org/45020

Re: Single-board-computer approach: don't make an installer, make the install?

[Joshua Branson via]

Mathieu Othacehe <othacehe@gnu.org> writes:

> Hello,
>
>> This is really well written.  May I edit this article and reproduce it
>> for the guix blog and perhaps the guix manual?  If ludo's ok with me
>> putting it on the blog, then I intend to end the blog post with "This
>> article first appeared in this personal blog (link here) and is
>> reproduced with permission."
>
> Thanks! I would happily review it :).
>
> There's for sure some work left on that topic. I'd like to:
>
> * Rename "disk-image" command to "image".
> * Add "docker" as a new image type.
> * Find a better way to enforce system/target constraints, as discussed
> here[1].

I'll go ahead and try to set up a patch for the cookbook.  

>
> Mathieu
>
> [1]: https://issues.guix.gnu.org/45020

--
Joshua Branson
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar

[PATCH] * doc/guix-cookbook.texi (Guix System Image API): new section

[Joshua Branson]

---
 doc/guix-cookbook.texi | 187 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 187 insertions(+)

diff --git a/doc/guix-cookbook.texi b/doc/guix-cookbook.texi
index 54ab99558e..2bc8fe8a99 100644
--- a/doc/guix-cookbook.texi
+++ b/doc/guix-cookbook.texi
@@ -1353,6 +1353,7 @@ reference.
 
 @menu
 * Customizing the Kernel::       Creating and using a custom Linux kernel on Guix System.
+* Guix System Image API::        Customizing the disk-image to target specific platforms.
 * Connecting to Wireguard VPN::  Connecting to a Wireguard VPN.
 * Customizing a Window Manager:: Handle customization of a Window manager on Guix System.
 * Running Guix on a Linode Server:: Running Guix on a Linode Server
@@ -1601,6 +1602,192 @@ likely that you'll need to modify the initrd on a machine using a custom
 kernel, since certain modules which are expected to be built may not be
 available for inclusion into the initrd.
 
+@node Guix System Image API
+@section Guix System Image API 
+
+Historically, Guix System is centered around an @code{operating-system}
+structure.  This structure contains various fields ranging from the
+bootloader and kernel declaration to the services to install.  This is
+useful to create an installer image, but the new Guix System image API
+makes it possible to create an image that the user boots into directly.
+For example, an image that targets the Beagleboard or PinePhone
+directly.
+
+Turning an @code{operating-system} structure into a disk-image requires
+additional information such as the image label, size and
+partitioning. To this end, we can use the new @code{image} record.
+
+@lisp
+(define-record-type* <image>
+  image make-image
+  image?
+  (name               image-name ;symbol
+                      (default #f))
+  (format             image-format) ;symbol
+  (target             image-target
+                      (default #f))
+  (size               image-size  ;size in bytes as integer
+                      (default 'guess))
+  (operating-system   image-operating-system  ;<operating-system>
+                      (default #f))
+  (partitions         image-partitions ;list of <partition>
+                      (default '()))
+  (compression?       image-compression? ;boolean
+                      (default #t))
+  (volatile-root?     image-volatile-root? ;boolean
+                      (default #t))
+  (substitutable?     image-substitutable? ;boolean
+                      (default #t)))
+@end lisp
+
+This record also contains the operating-system to instantiate. The
+@code{format} field defines the image type and can be @code{disk-image},
+@code{compressed-qcow2} or @code{iso9660}. In the future, it could be
+extended to @code{docker} or other image types.
+
+A new directory in the Guix sources is dedicated to images definition. For now
+there are two files:
+
+@itemize @bullet
+@item
+- @file{gnu/system/images/hurd.scm}
+
+@item
+- @file{gnu/system/images/pine64.scm}
+
+@end itemize
+
+Let's have a look to @file{pine64.scm}. It contains the
+@code{pine64-barebones-os} variable which is a minimal definition of an
+operating-system dedicated to the @b{Pine A64 LTS} board.
+
+@lisp
+(define pine64-barebones-os
+  (operating-system
+    (host-name "vignemale")
+    (timezone "Europe/Paris")
+    (locale "en_US.utf8")
+    (bootloader (bootloader-configuration
+                 (bootloader u-boot-pine64-lts-bootloader)
+                 (target "/dev/vda")))
+    (initrd-modules '())
+    (kernel linux-libre-arm64-generic)
+    (file-systems (cons (file-system
+                          (device (file-system-label "my-root"))
+                          (mount-point "/")
+                          (type "ext4"))
+                        %base-file-systems))
+    (services (cons (service agetty-service-type
+                             (agetty-configuration
+                              (extra-options '("-L")) ; no carrier detect
+                              (baud-rate "115200")
+                              (term "vt100")
+                              (tty "ttyS0")))
+                    %base-services))))
+@end lisp
+
+The @code{kernel} and @code{bootloader} fields are pointing to packages
+dedicated to this board.
+
+Right below, the @code{pine64-image-type} variable is also defined.
+
+@lisp
+(define pine64-image-type
+  (image-type
+   (name 'pine64-raw)
+   (constructor (cut image-with-os arm64-disk-image <>))))
+@end lisp
+
+It's using a record we haven't talked about yet, the @code{image-type} record,
+defined this way:
+
+@lisp
+(define-record-type* <image-type>
+  image-type make-image-type
+  image-type?
+  (name           image-type-name) ;symbol
+  (constructor    image-type-constructor)) ;<operating-system> -> <image>
+@end lisp
+
+The main purpose of this record is to associate a name to a procedure
+transforming an @code{operating-system} to an image. To understand why
+it is necessary, let's have a look to the command producing a disk-image
+from an @code{operating-system} configuration file:
+
+@example
+guix system disk-image my-os.scm
+@end example
+
+This command expects an @code{operating-system} configuration but how
+should we indicate that we want an image targeting a Pine64 board? We
+need to provide an extra information, the @code{image-type}, by passing
+the @code{--image-type} or @code{-t} flag, this way:
+
+@example
+guix system disk-image --image-type=pine64-raw my-os.scm
+@end example
+
+This @code{image-type} parameter points to the @code{pine64-image-type}
+defined above. Hence, the @code{operating-system} declared in
+@code{my-os.scm} will be applied the @code{(cut image-with-os
+arm64-disk-image <>)} procedure to turn it into an image.
+
+The resulting image looks like:
+
+@lisp
+(image
+   (format 'disk-image)
+   (target "aarch64-linux-gnu")
+   (operating-system my-os)
+   (partitions
+    (list (partition
+           (inherit root-partition)
+           (offset root-offset)))))
+@end lisp
+
+which is the aggregation of the @code{operating-system} defined in
+ @code{my-os.scm} to the @code{arm64-disk-image} record.
+
+But enough Scheme madness. What does this image API bring to the Guix user?
+
+One can run:
+
+@example
+mathieu@@cervin:~$ guix system --list-image-types
+The available image types are:
+
+   - pine64-raw
+   - hurd-raw
+   - hurd-qcow2
+   - iso9660
+   - uncompressed-iso9660
+   - raw
+   - qcow2
+@end example
+
+and by writing an @code{operating-system} file based on
+@code{pine64-barebones-os} or @code{hurd-barebones-os} run:
+
+@example
+guix system --image-type=pine64-raw my-pine-os.scm
+@end example
+
+or,
+
+@example
+guix system --image-type=hurd-raw my-hurd-os.scm
+@end example
+
+to get a disk-image that can directly be written to a support and booted from.
+
+Without changing anything to @code{my-hurd-os.scm}, calling:
+
+@example
+guix system --image-type=hurd-qcow2 my-hurd-os.scm
+@end example
+
+will instead produce a Hurd QEMU image.
+
 @node Connecting to Wireguard VPN
 @section Connecting to Wireguard VPN
 
-- 
2.30.0

Re: [PATCH] * doc/guix-cookbook.texi (Guix System Image API): new section

[Mathieu Othacehe]

Hello,

Thanks for this contribution! You should submit patches to
"guix-patches@gnu.org" instead of this mailing list.

There's also a small issue with the commit message which is placed as
title of this email.

> +* Guix System Image API::        Customizing the disk-image to target specific platforms.

Customizing disk images to target specific platforms, maybe?

> +@itemize @bullet
> +@item
> +- @file{gnu/system/images/hurd.scm}

You can remove the dash here.

> +@example
> +guix system disk-image my-os.scm
> +@end example

I have a pending patchset renaming "disk-image" command to "image". We
will need to update it accordingly.

Otherwise, it looks nice to me, but I may be biased here. It would be
nice if someone else could have a look.

Thanks,

Mathieu

Re: [PATCH] * doc/guix-cookbook.texi (Guix System Image API): new section

[Joshua Branson]

Mathieu Othacehe <othacehe@gnu.org> writes:

> Hello,
>
> Thanks for this contribution! You should submit patches to
> "guix-patches@gnu.org" instead of this mailing list.

I'll send one that way.

>
> There's also a small issue with the commit message which is placed as
> title of this email.
>
>> +* Guix System Image API::        Customizing the disk-image to target specific platforms.
>
> Customizing disk images to target specific platforms, maybe?
>
>> +@itemize @bullet
>> +@item
>> +- @file{gnu/system/images/hurd.scm}
>
> You can remove the dash here.
>
>> +@example
>> +guix system disk-image my-os.scm
>> +@end example
>
> I have a pending patchset renaming "disk-image" command to "image". We
> will need to update it accordingly.
>
> Otherwise, it looks nice to me, but I may be biased here. It would be
> nice if someone else could have a look.
>
> Thanks,
>
> Mathieu

I will update accordingly to your edits.

--
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar

[PATCH] * doc/guix-cookbook.texi (Guix System Image API): new section

[jbranso--- via]

---
 doc/guix-cookbook.texi | 187 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 187 insertions(+)

diff --git a/doc/guix-cookbook.texi b/doc/guix-cookbook.texi
index 54ab99558e..d9835c43f5 100644
--- a/doc/guix-cookbook.texi
+++ b/doc/guix-cookbook.texi
@@ -1353,6 +1353,7 @@ reference.
 
 @menu
 * Customizing the Kernel::       Creating and using a custom Linux kernel on Guix System.
+* Guix System Image API::        Customizing disk images to target specific platforms.
 * Connecting to Wireguard VPN::  Connecting to a Wireguard VPN.
 * Customizing a Window Manager:: Handle customization of a Window manager on Guix System.
 * Running Guix on a Linode Server:: Running Guix on a Linode Server
@@ -1601,6 +1602,192 @@ likely that you'll need to modify the initrd on a machine using a custom
 kernel, since certain modules which are expected to be built may not be
 available for inclusion into the initrd.
 
+@node Guix System Image API
+@section Guix System Image API 
+
+Historically, Guix System is centered around an @code{operating-system}
+structure.  This structure contains various fields ranging from the
+bootloader and kernel declaration to the services to install.  This is
+useful to create an installer image, but the new Guix System image API
+makes it possible to create an image that the user boots into directly.
+For example, an image that targets the Beagleboard or PinePhone
+directly.
+
+Turning an @code{operating-system} structure into a disk-image requires
+additional information such as the image label, size and
+partitioning. To this end, we can use the new @code{image} record.
+
+@lisp
+(define-record-type* <image>
+  image make-image
+  image?
+  (name               image-name ;symbol
+                      (default #f))
+  (format             image-format) ;symbol
+  (target             image-target
+                      (default #f))
+  (size               image-size  ;size in bytes as integer
+                      (default 'guess))
+  (operating-system   image-operating-system  ;<operating-system>
+                      (default #f))
+  (partitions         image-partitions ;list of <partition>
+                      (default '()))
+  (compression?       image-compression? ;boolean
+                      (default #t))
+  (volatile-root?     image-volatile-root? ;boolean
+                      (default #t))
+  (substitutable?     image-substitutable? ;boolean
+                      (default #t)))
+@end lisp
+
+This record also contains the operating-system to instantiate. The
+@code{format} field defines the image type and can be @code{disk-image},
+@code{compressed-qcow2} or @code{iso9660}. In the future, it could be
+extended to @code{docker} or other image types.
+
+A new directory in the Guix sources is dedicated to images definition. For now
+there are two files:
+
+@itemize @bullet
+@item
+@file{gnu/system/images/hurd.scm}
+
+@item
+@file{gnu/system/images/pine64.scm}
+
+@end itemize
+
+Let's have a look to @file{pine64.scm}. It contains the
+@code{pine64-barebones-os} variable which is a minimal definition of an
+operating-system dedicated to the @b{Pine A64 LTS} board.
+
+@lisp
+(define pine64-barebones-os
+  (operating-system
+    (host-name "vignemale")
+    (timezone "Europe/Paris")
+    (locale "en_US.utf8")
+    (bootloader (bootloader-configuration
+                 (bootloader u-boot-pine64-lts-bootloader)
+                 (target "/dev/vda")))
+    (initrd-modules '())
+    (kernel linux-libre-arm64-generic)
+    (file-systems (cons (file-system
+                          (device (file-system-label "my-root"))
+                          (mount-point "/")
+                          (type "ext4"))
+                        %base-file-systems))
+    (services (cons (service agetty-service-type
+                             (agetty-configuration
+                              (extra-options '("-L")) ; no carrier detect
+                              (baud-rate "115200")
+                              (term "vt100")
+                              (tty "ttyS0")))
+                    %base-services))))
+@end lisp
+
+The @code{kernel} and @code{bootloader} fields are pointing to packages
+dedicated to this board.
+
+Right below, the @code{pine64-image-type} variable is also defined.
+
+@lisp
+(define pine64-image-type
+  (image-type
+   (name 'pine64-raw)
+   (constructor (cut image-with-os arm64-disk-image <>))))
+@end lisp
+
+It's using a record we haven't talked about yet, the @code{image-type} record,
+defined this way:
+
+@lisp
+(define-record-type* <image-type>
+  image-type make-image-type
+  image-type?
+  (name           image-type-name) ;symbol
+  (constructor    image-type-constructor)) ;<operating-system> -> <image>
+@end lisp
+
+The main purpose of this record is to associate a name to a procedure
+transforming an @code{operating-system} to an image. To understand why
+it is necessary, let's have a look to the command producing a disk-image
+from an @code{operating-system} configuration file:
+
+@example
+guix system disk-image my-os.scm
+@end example
+
+This command expects an @code{operating-system} configuration but how
+should we indicate that we want an image targeting a Pine64 board? We
+need to provide an extra information, the @code{image-type}, by passing
+the @code{--image-type} or @code{-t} flag, this way:
+
+@example
+guix system disk-image --image-type=pine64-raw my-os.scm
+@end example
+
+This @code{image-type} parameter points to the @code{pine64-image-type}
+defined above. Hence, the @code{operating-system} declared in
+@code{my-os.scm} will be applied the @code{(cut image-with-os
+arm64-disk-image <>)} procedure to turn it into an image.
+
+The resulting image looks like:
+
+@lisp
+(image
+   (format 'disk-image)
+   (target "aarch64-linux-gnu")
+   (operating-system my-os)
+   (partitions
+    (list (partition
+           (inherit root-partition)
+           (offset root-offset)))))
+@end lisp
+
+which is the aggregation of the @code{operating-system} defined in
+ @code{my-os.scm} to the @code{arm64-disk-image} record.
+
+But enough Scheme madness. What does this image API bring to the Guix user?
+
+One can run:
+
+@example
+mathieu@@cervin:~$ guix system --list-image-types
+The available image types are:
+
+   - pine64-raw
+   - hurd-raw
+   - hurd-qcow2
+   - iso9660
+   - uncompressed-iso9660
+   - raw
+   - qcow2
+@end example
+
+and by writing an @code{operating-system} file based on
+@code{pine64-barebones-os} or @code{hurd-barebones-os} run:
+
+@example
+guix system --image-type=pine64-raw my-pine-os.scm
+@end example
+
+or,
+
+@example
+guix system --image-type=hurd-raw my-hurd-os.scm
+@end example
+
+to get a disk-image that can directly be written to a support and booted from.
+
+Without changing anything to @code{my-hurd-os.scm}, calling:
+
+@example
+guix system --image-type=hurd-qcow2 my-hurd-os.scm
+@end example
+
+will instead produce a Hurd QEMU image.
+
 @node Connecting to Wireguard VPN
 @section Connecting to Wireguard VPN
 
-- 
2.30.0

Re: Single-board-computer approach: don't make an installer, make the install?

[Joshua Branson]

Mathieu Othacehe <othacehe@gnu.org> writes:

> Hello,
>
>> This is really well written.  May I edit this article and reproduce it
>> for the guix blog and perhaps the guix manual?  If ludo's ok with me
>> putting it on the blog, then I intend to end the blog post with "This
>> article first appeared in this personal blog (link here) and is
>> reproduced with permission."
>
> Thanks! I would happily review it :).
>
> There's for sure some work left on that topic. I'd like to:
>

Now that we've got this bit included in the cookbook, would you be in
favor of also turning it into a guix blog post?  I'd be happy to submit
a patch.

>
> Mathieu
>
> [1]: https://issues.guix.gnu.org/45020
>

--
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar

Re: Single-board-computer approach: don't make an installer, make the install?

[Mathieu Othacehe]

Hello,

> Now that we've got this bit included in the cookbook, would you be in
> favor of also turning it into a guix blog post?  I'd be happy to submit
> a patch.

I think that the cookbook is a nice achievement for now. Maybe we could
postpone the guix blog article until the pending image command line
patches are pushed[1] and the relevant Guix documentation is written?

Thanks,

Mathieu

[1]:
https://issues.guix.gnu.org/45986
https://issues.guix.gnu.org/46017

Re: Single-board-computer approach: don't make an installer, make the install?

[Joshua Branson]

Mathieu Othacehe <othacehe@gnu.org> writes:

> I think that the cookbook is a nice achievement for now. Maybe we could
> postpone the guix blog article until the pending image command line
> patches are pushed[1] and the relevant Guix documentation is written?
>
> Thanks,
>
> Mathieu
>
> [1]:
> https://issues.guix.gnu.org/45986
> https://issues.guix.gnu.org/46017

I believe that these issues have been closed.  I suppose we could write
a blog post, if you believe that would be worth while.  


-- 
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar
  

Re: Single-board-computer approach: don't make an installer, make the install?

[Mathieu Othacehe]

Hey Joshua,

> I believe that these issues have been closed.  I suppose we could write
> a blog post, if you believe that would be worth while.  

Yes they were indeed. It would be nice but it's not on my list right
now. I you feel like writing it, I would happily review it :).

Thanks,

Mathieu

Re: Single-board-computer approach: don't make an installer, make the install?

[Joshua Branson]

Mathieu Othacehe <othacehe@gnu.org> writes:

> Hey Joshua,
>
>> I believe that these issues have been closed.  I suppose we could write
>> a blog post, if you believe that would be worth while.
>
> Yes they were indeed. It would be nice but it's not on my list right
> now. I you feel like writing it, I would happily review it :).

I'll add it to my list of things to do.  I still am hoping to get an
endlessh service merged too...I'm moving really slowly.  :)

>
> Thanks,
>
> Mathieu
>

--
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar