Oniro or Guix on Zephyr kernel?

Oniro or Guix on Zephyr kernel?

[Peter Polidoro]

I just stumbled across Oniro[1], the Eclipse Foundation's new 
operating system.

It seems that its main goal is to be able to run a common 
operating system on multiple embedded kernels, either the Linux 
kernel for larger devices or the Zephyr kernel for smaller ones.

Since Guix System can run on both the Linux kernel and the Hurd 
kernel, could it, in theory, also run on the Zephyr kernel?

Could Guix System be an alternative to the Oniro project or is 
there something about Guix System which would make it unsuitable 
for embedded devices on an RTOS kernel?

Footnotes:
[1] 
https://blogs.eclipse.org/post/mike-milinkovich/introducing-oniro-vendor-neutral-open-source-os-next-gen-devices

Re: Oniro or Guix on Zephyr kernel?

[Csepp]

Peter Polidoro <peter@polidoro.io> writes:

> I just stumbled across Oniro[1], the Eclipse Foundation's new
> operating system.
>
> It seems that its main goal is to be able to run a common operating
> system on multiple embedded kernels, either the Linux kernel for
> larger devices or the Zephyr kernel for smaller ones.
>
> Since Guix System can run on both the Linux kernel and the Hurd
> kernel, could it, in theory, also run on the Zephyr kernel?
>
> Could Guix System be an alternative to the Oniro project or is there
> something about Guix System which would make it unsuitable for
> embedded devices on an RTOS kernel?
>
> Footnotes:
> [1]
> https://blogs.eclipse.org/post/mike-milinkovich/introducing-oniro-vendor-neutral-open-source-os-next-gen-devices

This doesn't quite answer your question but it's worth noting:
You can use Guix as a build system to compile to any kernel with a Linux
based cross compile toolchain.

I'm currently working on using it to compile MirageOS unikernels.

Reading about the kinds of devices Oniro and OpenHarmony supports, it
seems unlikely that you could run the Guix build system on the lower end
of the spectrum.  Guix already barely copes with lower end ARM and older
i386 devices.  It needs at least 1GB of RAM and plenty of storage,
preferably flash based.

If your device meets those requirements then there is not much reason to
run an embedded OS and if it doesn't then your only chance is cross
compiling to it.

There is actually quite a bit to be gained from using Guix to cross
compile to embedded devices or unikernels, but a lot more work is needed
IMHO.  Help with cross-toolchains is I think always welcome. UwU

Re: Oniro or Guix on Zephyr kernel?

[Joshua Branson]

Csepp <raingloom@riseup.net> writes:

> Peter Polidoro <peter@polidoro.io> writes:
>
>> I just stumbled across Oniro[1], the Eclipse Foundation's new
>> operating system.
>>
>> It seems that its main goal is to be able to run a common operating
>> system on multiple embedded kernels, either the Linux kernel for
>> larger devices or the Zephyr kernel for smaller ones.
>>
>> Since Guix System can run on both the Linux kernel and the Hurd
>> kernel, could it, in theory, also run on the Zephyr kernel?
>>
>> Could Guix System be an alternative to the Oniro project or is there
>> something about Guix System which would make it unsuitable for
>> embedded devices on an RTOS kernel?
>>
>> Footnotes:
>> [1]
>> https://blogs.eclipse.org/post/mike-milinkovich/introducing-oniro-vendor-neutral-open-source-os-next-gen-devices

Actually, my new friend Mitchell just created a blog post about using
GNU Guix for Zephyr kernels:

https://gnucode.me/building-toolchains-with-guix.html

So GNU Guix can definitely help with this endeavor.  :)

Joshua

Re: Oniro or Guix on Zephyr kernel?

[Peter Polidoro]

> Actually, my new friend Mitchell just created a blog post about 
> using
> GNU Guix for Zephyr kernels:
>
> https://gnucode.me/building-toolchains-with-guix.html

Great blog post, thank you! It makes me a little hesitant about 
Zephyr if they have just given up on other people building their 
SDK, but I am very glad all of you smart people are working on a 
Guix alternative.

I am still curious if other parts of Guix System could be useful 
in embedded environments if all of the packages are 
cross-compiled.

I am not sure of the detailed plans for Oniro, but I assume that 
it will provide some sort of abstraction layer and user space on 
top of either the Linux kernel or the embedded Zephyr kernel. 
Would it be possible to make some subset of Guix System into 
something equivalent or is an entirely new operating system really 
necessary for that purpose? Are most of the resource requirements 
for Guix System, 1 Gig of ram, etc, due to the package builder and 
Guix/Nix daemon? If it was possible to declare an instance of Guix 
System that did not include those and only used cross-compiled 
packages, could some portion of Guix System function in a similar 
way as Oniro, or is that a nonsense question?

Re: Oniro or Guix on Zephyr kernel?

[jbranso]

February 24, 2023 10:41 AM, "Peter Polidoro" <peter@polidoro.io> wrote:

>> Actually, my new friend Mitchell just created a blog post about > using
>> GNU Guix for Zephyr kernels:
>> 
>> https://gnucode.me/building-toolchains-with-guix.html
> 
> Great blog post, thank you! It makes me a little hesitant about Zephyr if they have just given up
> on other people building their SDK, but I am very glad all of you smart people are working on a
> Guix alternative.
> 
> I am still curious if other parts of Guix System could be useful in embedded environments if all of
> the packages are cross-compiled.
> 
> I am not sure of the detailed plans for Oniro, but I assume that it will provide some sort of
> abstraction layer and user space on top of either the Linux kernel or the embedded Zephyr kernel.
> Would it be possible to make some subset of Guix System into something equivalent or is an entirely
> new operating system really necessary for that purpose? Are most of the resource requirements for
> Guix System, 1 Gig of ram, etc, due to the package builder and Guix/Nix daemon? If it was possible
> to declare an instance of Guix System that did not include those and only used cross-compiled
> packages, could some portion of Guix System function in a similar way as Oniro, or is that a
> nonsense question?

I'm probably not the best person to ask about that.  I am adding in Mitchell to the discussion.

Mitchell, what do you think?  Could we use a subset of guix system in embedded environments?

Thanks,

Joshua

Re: Oniro or Guix on Zephyr kernel?

[Mitchell Schmeisser via Development of GNU Guix and the GNU System distribution.]

jbranso@dismail.de writes:

> February 24, 2023 10:41 AM, "Peter Polidoro" <peter@polidoro.io> wrote:
>
>>> Actually, my new friend Mitchell just created a blog post about > using
>>> GNU Guix for Zephyr kernels:
>>> 
>>> https://gnucode.me/building-toolchains-with-guix.html
>> 
>> Great blog post, thank you! It makes me a little hesitant about Zephyr if they have just given up
>> on other people building their SDK, but I am very glad all of you smart people are working on a
>> Guix alternative.
>> 
>> I am still curious if other parts of Guix System could be useful in embedded environments if all of
>> the packages are cross-compiled.
>> 
>> I am not sure of the detailed plans for Oniro, but I assume that it will provide some sort of
>> abstraction layer and user space on top of either the Linux kernel or the embedded Zephyr kernel.
>> Would it be possible to make some subset of Guix System into something equivalent or is an entirely
>> new operating system really necessary for that purpose? Are most of the resource requirements for
>> Guix System, 1 Gig of ram, etc, due to the package builder and Guix/Nix daemon? If it was possible
>> to declare an instance of Guix System that did not include those and only used cross-compiled
>> packages, could some portion of Guix System function in a similar way as Oniro, or is that a
>> nonsense question?
>
> I'm probably not the best person to ask about that.  I am adding in Mitchell to the discussion.
>
> Mitchell, what do you think?  Could we use a subset of guix system in embedded environments?
>
> Thanks,
>
> Joshua

>> It makes me a little hesitant about Zephyr if they have just
>> given up on other people building their SDK

In the embedded world this is not that uncommon. Building cross toolchains is
very difficult and many Zephyr users are developing on Windows where you
don't ever build anything yourself. 

A lot of embedded developers still pay top dollar for proprietary
toolchains.

>> I am not sure of the detailed plans for Oniro

I do not know what this is.


>> I am still curious if other parts of Guix System could be useful in embedded environments if all of
>> the packages are cross-compiled.

The short answer is Guix can run on embedded linux environments given
sufficient resources, but Guix will never run on the Zephyr kernel*.

Zephyr is targeted at extremely resource constrained devices (think
arduino or a TV remote). Many of these devices don't have an MMU and do not have such a
firm separation between kernel and user space.
It is an entirely different world from the one currently modeled by Guix.

My main reason for doing this was because Guix has already replaced
every other package manager on the system.
It felt wrong to use west's dependency management and the SDK installer
is a binary distribution. Guix already has packages for
`arm-none-eabi-toolchain` so mostly this is just copy and pasted and
adjusted to match the cmake expectations of the world.

I also thought relying on West to build production software was not the
best because it was too easy for me to accidentally modify the
environment. The only I could be sure about what I was deploying was to
do as Guix does (but worse) and clone the stuff down fresh from the
internet (all 3GB+ of it...) and "bootstrap" the environment.

Reproducibility issues pop up from time to time in the zephyr project and I
thought this work could give some sense of formality to their build
system.

>> Guix System, 1 Gig of ram, etc, due to the package builder and Guix/Nix daemon? If it was possible
>> to declare an instance of Guix System that did not include those and only used cross-compiled
>> packages, could some portion of Guix System function in a similar way as Oniro, or is that a
>> nonsense question?

What I understand your question to be is "Can we use Guix to describe an
embedded operating system which does not run Guix?" and I think the
answer is probably. I don't think the guix daemon is technically required
for the shepherd to boot (being the daemon is a shepherd process
itself).

I don't think it's a good idea because you need the daemon in order to
use `guix deploy`. Otherwise you have to make an installation image and
it can become "involved."


* Actually the zephyr kernel can drive large systems as well but if you
  want to run Guix on it you are better off using linux.
  Look at this video for some inspiration on how Guix can be used to
  manage these systems.
  https://archive.fosdem.org/2020/schedule/event/ggaaattyp/

Re: Oniro or Guix on Zephyr kernel?

[Tobias Geerinckx-Rice]

Hi,

On 2023-02-24 19:36, Mitchell Schmeisser wrote:
>                     I don't think the guix daemon is technically 
> required
> for the shepherd to boot (being the daemon is a shepherd process
> itself).

No, not at all.  I think this is a common (and not unreasonable!) 
assumption we Guixers forget exists.

The ‘Guix daemon’ is not required to run Guix software or Guix System, 
only for store operations (building, GC'ing, …).

The name ‘guix-build-daemon’ would have been closer to the truth.

Kind regards,

T G-R

Sent from a Web browser.  Excuse or enjoy my brevity.

Re: Oniro or Guix on Zephyr kernel?

[Felix Lechner via Development of GNU Guix and the GNU System distribution.]

Hi Mitchell,

On Fri, Feb 24, 2023 at 10:36 AM Mitchell Schmeisser via Development
of GNU Guix and the GNU System distribution. <guix-devel@gnu.org>
wrote:
>
> My main reason for doing this was because Guix has already replaced
> every other package manager on the system.

Kudos to this. You should get a prize!

Kind regards
Felix Lechner

Re: Oniro or Guix on Zephyr kernel?

[Peter Polidoro]

Thank you for all of your great work on packaging the Zephyr SDK for Guix! I had the same feelings about West and its approach to handling dependencies.

PlatformIO has a similar issue where parts of it are very useful, but it would be great to be able use Guix to manage firmware libraries instead of the platformio.ini files.

Are you also planning on making the firmware that you write to run on top of Zephyr into Guix packages as well? That could make it way easier to synchronize code on both a host machine and an embedded board that need to communicate with each other. 

I would love to read another blog post about using Guix to manage an entire embedded project including firmware if you are ever up for it!

Re: Oniro or Guix on Zephyr kernel?

[Mitchell Schmeisser via Development of GNU Guix and the GNU System distribution.]

> Are you also planning on making the firmware that you write to run on top of Zephyr into Guix packages as well? That could make it way easier to synchronize code on both a host machine and an embedded board that need to communicate with each other. 
> 
> I would love to read another blog post about using Guix to manage an entire embedded project including firmware if you are ever up for it!


It’s already written! It will be published shortly but if you want to look at my work https://github.com/guix-zephyr

-Mitchell

> On Feb 25, 2023, at 8:45 AM, Peter Polidoro <peter@polidoro.io> wrote:
> 
> Thank you for all of your great work on packaging the Zephyr SDK for Guix! I had the same feelings about West and its approach to handling dependencies.
> 
> PlatformIO has a similar issue where parts of it are very useful, but it would be great to be able use Guix to manage firmware libraries instead of the platformio.ini files.
> 
> Are you also planning on making the firmware that you write to run on top of Zephyr into Guix packages as well? That could make it way easier to synchronize code on both a host machine and an embedded board that need to communicate with each other. 
> 
> I would love to read another blog post about using Guix to manage an entire embedded project including firmware if you are ever up for it!
> 
> 

Re: Oniro or Guix on Zephyr kernel?

[Mitchell Schmeisser via Development of GNU Guix and the GNU System distribution.]

https://github.com/paperclip4465/guix-zephyr

> On Feb 25, 2023, at 9:15 AM, Mitchell Schmeisser <mitchellschmeisser@librem.one> wrote:
> 
> 
>> 
>> Are you also planning on making the firmware that you write to run on top of Zephyr into Guix packages as well? That could make it way easier to synchronize code on both a host machine and an embedded board that need to communicate with each other. 
>> 
>> I would love to read another blog post about using Guix to manage an entire embedded project including firmware if you are ever up for it!
> 
> 
> It’s already written! It will be published shortly but if you want to look at my work https://github.com/guix-zephyr
> 
> -Mitchell
> 
>> On Feb 25, 2023, at 8:45 AM, Peter Polidoro <peter@polidoro.io> wrote:
>> 
>> Thank you for all of your great work on packaging the Zephyr SDK for Guix! I had the same feelings about West and its approach to handling dependencies.
>> 
>> PlatformIO has a similar issue where parts of it are very useful, but it would be great to be able use Guix to manage firmware libraries instead of the platformio.ini files.
>> 
>> Are you also planning on making the firmware that you write to run on top of Zephyr into Guix packages as well? That could make it way easier to synchronize code on both a host machine and an embedded board that need to communicate with each other. 
>> 
>> I would love to read another blog post about using Guix to manage an entire embedded project including firmware if you are ever up for it!
>> 
>> 

Re: Oniro or Guix on Zephyr kernel?

[Peter Polidoro]

> https://github.com/guix-zephyr

I actually reserved this organization name on GitHub a while back hoping to someday do what you already did. I am happy to add you to it or transfer it over to you entirely if you want.

I am looking forward to reading the details of what you did, thanks!

Re: Oniro or Guix on Zephyr kernel?

[Mitchell Schmeisser via Development of GNU Guix and the GNU System distribution.]

Hopefully this work can be incorporated into Guix. Initially I did not 
have much hope as even Linux does not make the cut and the embedded 
world is way more friendly to proprietary software/blob sharing. However 
I don't think there are any blobs in the zephyr code base and while 
zephyr itself provides mechanisms for blob sharing, we do not need to 
use these.

If not then I think moving to that domain, or perhaps gitlab given the 
github controversies, would be a great idea.


On 2/25/23 10:19, Peter Polidoro wrote:
>> https://github.com/guix-zephyr
> I actually reserved this organization name on GitHub a while back hoping to someday do what you already did. I am happy to add you to it or transfer it over to you entirely if you want.
>
> I am looking forward to reading the details of what you did, thanks!

Re: Oniro or Guix on Zephyr kernel?

[Mitchell Schmeisser via Development of GNU Guix and the GNU System distribution.]

https://github.com/paperclip4465/guix-embedded-example

Here is an example project which involves a vm and an embedded device.

On 2/25/23 10:19, Peter Polidoro wrote:
>> https://github.com/guix-zephyr
> I actually reserved this organization name on GitHub a while back hoping to someday do what you already did. I am happy to add you to it or transfer it over to you entirely if you want.
>
> I am looking forward to reading the details of what you did, thanks!

Re: Oniro or Guix on Zephyr kernel?

[jbranso]

February 24, 2023 2:35 PM, "Tobias Geerinckx-Rice" <me@tobias.gr> wrote:

> Hi,
> 
> On 2023-02-24 19:36, Mitchell Schmeisser wrote:
> 
>> I don't think the guix daemon is technically > required
>> for the shepherd to boot (being the daemon is a shepherd process
>> itself).
> 
> No, not at all. I think this is a common (and not unreasonable!) assumption we Guixers forget
> exists.
> 
> The ‘Guix daemon’ is not required to run Guix software or Guix System, only for store operations
> (building, GC'ing, …).
> 
> The name ‘guix-build-daemon’ would have been closer to the truth.

Thanks for clarifying.  So, guix system requires a shepherd daemon to be running correct?

That shepherd daemon starts at boot and runs as PID 1 and starts all system services.

Does the guix-build-daemon start lazily or only when needed?  So if I booted up my guix system laptop, and only browsed
the internet, never typed in any "guix " command, would the guix-build-daemon ever run?


> 
> Kind regards,
> 
> T G-R
> 
> Sent from a Web browser. Excuse or enjoy my brevity.

Re: Oniro or Guix on Zephyr kernel?

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1523 bytes --]

On Fri, Feb 24, 2023 at 01:36:06PM -0500, Mitchell Schmeisser via Development of GNU Guix and the GNU System distribution. wrote:
> jbranso@dismail.de writes:
> 
> >> Guix System, 1 Gig of ram, etc, due to the package builder and Guix/Nix daemon? If it was possible
> >> to declare an instance of Guix System that did not include those and only used cross-compiled
> >> packages, could some portion of Guix System function in a similar way as Oniro, or is that a
> >> nonsense question?
> 
> What I understand your question to be is "Can we use Guix to describe an
> embedded operating system which does not run Guix?" and I think the
> answer is probably. I don't think the guix daemon is technically required
> for the shepherd to boot (being the daemon is a shepherd process
> itself).

I can answer this. I built a one-off Guix image of gparted and it
doesn't have the guix-daemon service included.

> I don't think it's a good idea because you need the daemon in order to
> use `guix deploy`. Otherwise you have to make an installation image and
> it can become "involved."

If you were willing to forgo `guix deploy` (and guix install, and really
any package management) and just reflash it every time then the memory
constraints go down to whatever the running programs require.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Oniro or Guix on Zephyr kernel?

[Ryan Sundberg]

[-- Attachment #1.1: Type: text/plain, Size: 1909 bytes --]

I have also used Guix to build a NetBSD image. You can use Guix to build
just about any operating system image imaginable with the right amount
of practice and patience :) A lot of patience in some cases as you are
sure to hit some uncharted territory building whatever it happens to be.

Getting the userspace to run is a bigger task, but you can certainly use
Guix to build for example an embedded OS image with it's default
userspace (not porting the Guix daemon or Shepherd).

--
Sincerely,
Ryan Sundberg

On 2/27/23 11:10 AM, Efraim Flashner wrote:
> On Fri, Feb 24, 2023 at 01:36:06PM -0500, Mitchell Schmeisser via Development of GNU Guix and the GNU System distribution. wrote:
>> jbranso@dismail.de writes:
>>
>>>> Guix System, 1 Gig of ram, etc, due to the package builder and Guix/Nix daemon? If it was possible
>>>> to declare an instance of Guix System that did not include those and only used cross-compiled
>>>> packages, could some portion of Guix System function in a similar way as Oniro, or is that a
>>>> nonsense question?
>>
>> What I understand your question to be is "Can we use Guix to describe an
>> embedded operating system which does not run Guix?" and I think the
>> answer is probably. I don't think the guix daemon is technically required
>> for the shepherd to boot (being the daemon is a shepherd process
>> itself).
> 
> I can answer this. I built a one-off Guix image of gparted and it
> doesn't have the guix-daemon service included.
> 
>> I don't think it's a good idea because you need the daemon in order to
>> use `guix deploy`. Otherwise you have to make an installation image and
>> it can become "involved."
> 
> If you were willing to forgo `guix deploy` (and guix install, and really
> any package management) and just reflash it every time then the memory
> constraints go down to whatever the running programs require.
> 

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: Oniro or Guix on Zephyr kernel?

[Joshua Branson]

Ryan Sundberg <ryan@arctype.co> writes:

> I have also used Guix to build a NetBSD image. You can use Guix to build
> just about any operating system image imaginable with the right amount
> of practice and patience :) A lot of patience in some cases as you are
> sure to hit some uncharted territory building whatever it happens to be.
>
> Getting the userspace to run is a bigger task, but you can certainly use
> Guix to build for example an embedded OS image with it's default
> userspace (not porting the Guix daemon or Shepherd).

Can you post the code or routine that you did for this?  I would love
for guix to run on the OpenBSD kernel, but if guix requires glibc for
most of the features...and porting glibc to OpenBSD might be a far off
goal...

Re: Oniro or Guix on Zephyr kernel?

[Ryan Sundberg]

[-- Attachment #1.1: Type: text/plain, Size: 1664 bytes --]

Hi Joshua, you can find my code to build NetBSD images published here:

https://lab.arctype.co/neobsd/netbsd-guix/

It very much just "research", so no guarantees about it being useful for
anything. But it certainly does boot NetBSD images with the standard
userspace. Work on porting Shepherd was started, but that's where I stopped.

You will also at need to use my branches of:

Netbsd: https://lab.arctype.co/neobsd/netbsd (especially the first
commit, Guix build support, which patches the build script)

Guix https://lab.arctype.co/arctype/guix (Not all my patches have been
accepted or yet been submitted upstream)

If you have any specific questions, please contact me directly
(off-list). There is also other interesting code you are free to explore
in my gitlab :)

--
Sincerely,
Ryan Sundberg

On 3/4/23 9:26 AM, Joshua Branson wrote:
> Ryan Sundberg <ryan@arctype.co> writes:
> 
>> I have also used Guix to build a NetBSD image. You can use Guix to build
>> just about any operating system image imaginable with the right amount
>> of practice and patience :) A lot of patience in some cases as you are
>> sure to hit some uncharted territory building whatever it happens to be.
>>
>> Getting the userspace to run is a bigger task, but you can certainly use
>> Guix to build for example an embedded OS image with it's default
>> userspace (not porting the Guix daemon or Shepherd).
> 
> Can you post the code or routine that you did for this?  I would love
> for guix to run on the OpenBSD kernel, but if guix requires glibc for
> most of the features...and porting glibc to OpenBSD might be a far off
> goal...

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: Oniro or Guix on Zephyr kernel?

[Ludovic Courtès]

Hi,

Peter Polidoro <peter@polidoro.io> skribis:

> Since Guix System can run on both the Linux kernel and the Hurd
> kernel, could it, in theory, also run on the Zephyr kernel?

Only if/when glibc is ported to that kernel.  :-)

As a project, I think it’s important for Guix to focus on the GNU libc.
Attempting to support multiple C libraries, like Nixpkgs does, is really
tricky: platforms with less popular C libraries unavoidably have tons of
package build failures.

Ludo’.