Gnome Boxes

Gnome Boxes

[Adriano Peluso]

Gnome Boxes is GUI app that manages virtual machines and OSs images to
run in them 

It's a sort of Qemu for the rest of us.

It offers a menu with images of OSs to download and run

There are several versions of NixOs, among many others

There's a recipe for Guix System by Julien Lepiller, it's here
https://gitlab.com/libosinfo/osinfo-db/-/blob/master/data/os/guix.gnu.org/guix-1.1.xml.in

In their irc channel, the Boxes people infornmed me that they filter
out images that have no download url, as it seems to be the case with
this recipe

In fact, as you can see, the download url is commented out (on line 15)

Why is this ?

Was there a specific problem with the download url ?

It's be nice if I could pach this and have Guix System in the Gnome
Boxes menu

Thanks

Re: Gnome Boxes

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 1176 bytes --]

Hi!

When submitting the MR to libosinfo, there were concerns that the .xz extension would cause problems with other software, so we left it out.

I think we should provide a .iso download directly, if we want gnome boxes to propose the guix system.

Le 9 février 2021 08:58:35 GMT-05:00, Adriano Peluso <randomlooser@riseup.net> a écrit :
>Gnome Boxes is GUI app that manages virtual machines and OSs images to
>run in them 
>
>It's a sort of Qemu for the rest of us.
>
>It offers a menu with images of OSs to download and run
>
>There are several versions of NixOs, among many others
>
>There's a recipe for Guix System by Julien Lepiller, it's here
>https://gitlab.com/libosinfo/osinfo-db/-/blob/master/data/os/guix.gnu.org/guix-1.1.xml.in
>
>In their irc channel, the Boxes people infornmed me that they filter
>out images that have no download url, as it seems to be the case with
>this recipe
>
>In fact, as you can see, the download url is commented out (on line 15)
>
>Why is this ?
>
>Was there a specific problem with the download url ?
>
>It's be nice if I could pach this and have Guix System in the Gnome
>Boxes menu
>
>Thanks

[-- Attachment #2: Type: text/html, Size: 1574 bytes --]

Re: Gnome Boxes

[Ludovic Courtès]

Hi,

Julien Lepiller <julien@lepiller.eu> skribis:

> When submitting the MR to libosinfo, there were concerns that the .xz extension would cause problems with other software, so we left it out.
>
> I think we should provide a .iso download directly, if we want gnome boxes to propose the guix system.

We could do that but… isn’t it a bit ridiculous?  I mean, compression is
a well-known technique to reduce bandwidth usage, right?  :-)

That said, we now produce compressed ISOs IIRC, so it might be that the
extra compression layer doesn’t buy us much.  Would be worth checking!

Thanks,
Ludo’.

Re: Gnome Boxes

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 886 bytes --]

Usually compression is provided by the webserver, but maybe ours is not configured to do that. I think we're the only distro to provide compressed isos.

Le 18 février 2021 12:18:44 GMT-05:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
>Hi,
>
>Julien Lepiller <julien@lepiller.eu> skribis:
>
>> When submitting the MR to libosinfo, there were concerns that the .xz
>extension would cause problems with other software, so we left it out.
>>
>> I think we should provide a .iso download directly, if we want gnome
>boxes to propose the guix system.
>
>We could do that but… isn’t it a bit ridiculous?  I mean, compression
>is
>a well-known technique to reduce bandwidth usage, right?  :-)
>
>That said, we now produce compressed ISOs IIRC, so it might be that the
>extra compression layer doesn’t buy us much.  Would be worth checking!
>
>Thanks,
>Ludo’.

[-- Attachment #2: Type: text/html, Size: 1278 bytes --]

Re: Gnome Boxes

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 569 bytes --]

Julien Lepiller 写道：
> Usually compression is provided by the webserver, but maybe ours 
> is not configured to do that.  I think we're the only distro to 
> provide compressed isos.

Really?  Most distribution ISOs use squashfs or similar with 
XZ/LZMA compression.  It doesn't make sense to compress that over 
the wire.

That said: XZ compression currently saves 27% (559M -> 405M). 
Transparently serving pre-compressed ISOs with nginx (gzip level 
9) would save about 25% (559M -> 415M), which is surprisingly 
similar.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

Re: Gnome Boxes

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 849 bytes --]

Sorry, a compressed .iso is probably common, a .iso.xz is very uncommon :). We even have had some reports of people trying to copy that directly to an installation media.

Le 18 février 2021 14:56:44 GMT-05:00, Tobias Geerinckx-Rice <me@tobias.gr> a écrit :
>Julien Lepiller 写道：
>> Usually compression is provided by the webserver, but maybe ours 
>> is not configured to do that.  I think we're the only distro to 
>> provide compressed isos.
>
>Really?  Most distribution ISOs use squashfs or similar with 
>XZ/LZMA compression.  It doesn't make sense to compress that over 
>the wire.
>
>That said: XZ compression currently saves 27% (559M -> 405M). 
>Transparently serving pre-compressed ISOs with nginx (gzip level 
>9) would save about 25% (559M -> 415M), which is surprisingly 
>similar.
>
>Kind regards,
>
>T G-R

[-- Attachment #2: Type: text/html, Size: 1245 bytes --]

Re: Gnome Boxes

[Evan Rowley]

[-- Attachment #1: Type: text/plain, Size: 1134 bytes --]

FreeBSD also provides .iso.xz. Some examples here:
http://ftp-archive.freebsd.org/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/13.0/

On Thu, Feb 18, 2021 at 4:23 PM Julien Lepiller <julien@lepiller.eu> wrote:

> Sorry, a compressed .iso is probably common, a .iso.xz is very uncommon
> :). We even have had some reports of people trying to copy that directly to
> an installation media.
>
> Le 18 février 2021 14:56:44 GMT-05:00, Tobias Geerinckx-Rice <me@tobias.gr>
> a écrit :
>>
>> Julien Lepiller 写道：
>>
>>> Usually compression is provided by the webserver, but maybe ours
>>> is not configured to do that.  I think we're the only distro to
>>> provide compressed isos.
>>>
>>
>> Really?  Most distribution ISOs use squashfs or similar with
>> XZ/LZMA compression.  It doesn't make sense to compress that over
>> the wire.
>>
>> That said: XZ compression currently saves 27% (559M -> 405M).
>> Transparently serving pre-compressed ISOs with nginx (gzip level
>> 9) would save about 25% (559M -> 415M), which is surprisingly
>> similar.
>>
>> Kind regards,
>>
>> T G-R
>>
>>

-- 
 - EJR

[-- Attachment #2: Type: text/html, Size: 1919 bytes --]

Installing via iso.xz, really best idea? -- was Re: Gnome Boxes

[Bengt Richter]

Hi,

On +2021-02-22 00:39:36 -0500, Evan Rowley wrote:
> FreeBSD also provides .iso.xz. Some examples here:
> http://ftp-archive.freebsd.org/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/13.0/
>

Looking there for an example:
--8<---------------cut here---------------start------------->8---
FreeBSD-13.0-BETA2-amd64-bootonly.iso       363927552  2021-Feb-12 07:47
FreeBSD-13.0-BETA2-amd64-bootonly.iso.xz     82351604  2021-Feb-12 07:47
--8<---------------cut here---------------end--------------->8---

That seems like a really worthwhile compression! (ratio 4.4:1)

However, I wonder if big .iso's are really the best way
to take advantage of the xz (or any) compression for installation purposes.

If the point of having the information in an .iso is to have it be bootable
with everything (drivers, UI, etc) needed to install to any raw disk anywhere,
ISTM that is wastefully general for my usual case:

I have several laptops all able to dd to a USB-attached SSD or thumbdrive.
I trust their impure apps to do the xfr and dd, if I can verify result hashes independently.
I don't need to boot anything but my laptop for these apps: they're on my laptops.
I don't intend to use my foreign laptop to synthesize content of the files to xfr by dd,
  so there is only dependency on the pure guix versions of fdisk, mkfs, etc used on
  the machine that executed
      guix system -make-custom-bootstrap-scripted-tarball my-neat-system.scm
  (in my dreams :) (maybe a modded version of guix pack could do it, but sizes?)
┌─────────────────────────────────────────────────────────────────────────────────┐
│ I don't want to modify the laptop I'm using (in fact, I want a guarantee        │
│ that it WON'T be modified as I use it to dd-write to the attached USB storage). │
└─────────────────────────────────────────────────────────────────────────────────┘
(boxed for emphasis :)

All I really need is a tarball with a bootstrap script that essentially
does a series of dd file transfers to selected block sequences on the target disk(s).

I am thinking this could be smaller and easier to use than using guix pack
to concoct the same functionality.

As the script starts, IWBN to be able to select target disk(s) interactively, probably using
lsblk to probe and display, but I want to avoid partitioning target disk(s) with foreign tools
(i.e. writing to the medium with other than dd).

I think read-only access to get existing (if any) partition locations and sizes and types is ok,
so the script can potentially choose between sparse files to dd into common sizes of disk and
partition spaces where e.g. just padding with zeroes is not workable. IWBN to be able to target
a toy 32-gb USB as well as the front end of a 500GB SSD with the same script/tarball.

Those files for dd can contain ready-made images of GPT pieces -- superblock, VFAT fs,
legacy-bootable grub blocks, /boot and / and maybe other specialized block-sequence images
(with mkfs and transfers into the populated fs images already done on a pure guix system
using loopback-mounted files for fs images), or whatever else may be needed.
(caveat hunch: subject to some init-hook binary patching on first boot (hopefully no grub-launched
or BIOS-validated thing like firmware update necessary if kernel init can do it ;/ ).

For the case where I want to modify my own laptop, e.g. to boot from the USB-attached
storage I just prepared, that depends on what its BIOS does for booting, but I guess
that is commonly just a matter of updating grub's .cfg.

For the case where I want to install to my own laptop's disk(s), I still don't like
booting a monster iso. If my laptop's existing OS can't go into a safe maintenace mode,
where the target disk is synced and unmounted, then I will have to reboot.

If that is from a bootable installer USB, I want a rescue-size bootable, not
the monster iso including everything (nor the job and wait of writing the iso image
to a USB just so I can boot it once).

ISTM a manifest for the installation job could be a file of symbolic by-hash links
or equivalent sha256sum listing, and a list of mirrors, specifying
directories where the by-hash links are findable, e.g. like
    https://example.com/pub/filedir/by-hash/
and links in that directory looking like e.g.,
    e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 -> ../zerolengthfile
(which are easy to generate selectively or wholesale for a filedir/ directory
just creating the ./by-hash/ subdirectory and making the links))

ISTM like a custom bootstrap.sh could safely wget each file,
piping directly to dd writing these chunks to raw disk slots,
afterwards checking the sha256 of every block sequence it wrote.
Done simply --  and irrespective of whether the by-hash links
and files were stored in a git repo or plain file directories.

I think "guix system" could automate the production of customized
my-neat-system-boostrap.sh scripts containing the right wget and dd commands
and sha256 verifications, while providing safe interactive choice
of target disk(s) at the start of running it.

Such a script, built to run on a specific but non-guile/guix-dependent
"foreign" platform like Linux x86_64 would be a nice way to share a
"my-neat-system" ... or a Guix release.

So it seems to me, anyway :)

(I know you can do all kinds of things with laptops already running guix.
A main point here is to be able to use a (borrowed even) foreign, non-guile/guix
laptop to install something onto raw USB-attached storage without transferring
anything from the state of the laptop doing the work (i.e., xfr only, no synthesis).

I.e., the system or data being transferred
could be for a totally different architecture or purpose.

For the paranoid, a first-boot (assuming new thing is bootable)
hook could re-check all the hashes on the new system.

(Please excuse all the handwaving :)

> On Thu, Feb 18, 2021 at 4:23 PM Julien Lepiller <julien@lepiller.eu> wrote:
> 
> > Sorry, a compressed .iso is probably common, a .iso.xz is very uncommon
> > :). We even have had some reports of people trying to copy that directly to
> > an installation media.
> >
> > Le 18 février 2021 14:56:44 GMT-05:00, Tobias Geerinckx-Rice <me@tobias.gr>
> > a écrit :
> >>
> >> Julien Lepiller 写道：
> >>
> >>> Usually compression is provided by the webserver, but maybe ours
> >>> is not configured to do that.  I think we're the only distro to
> >>> provide compressed isos.
> >>>
> >>
> >> Really?  Most distribution ISOs use squashfs or similar with
> >> XZ/LZMA compression.  It doesn't make sense to compress that over
> >> the wire.
> >>
> >> That said: XZ compression currently saves 27% (559M -> 405M).
> >> Transparently serving pre-compressed ISOs with nginx (gzip level
> >> 9) would save about 25% (559M -> 415M), which is surprisingly
> >> similar.
> >>
> >> Kind regards,
> >>
> >> T G-R
> >>
> >>
> 
> -- 
>  - EJR

Hm, I wonder how large a tarball "guix pack" as it works now  would create
to create something runnable with the my-neat-system-bootstrap.sh
functionality described above, vs depending on the foreign system's
bash, wget, dd, etc.


-- 
Regards,
Bengt Richter