Re: Bootstrapping Zig with no binary blobs: status update and call for packages.

Re: Bootstrapping Zig with no binary blobs: status update and call for packages.

[Ashvith Shetty]

Hi Hilton,

There's a lot of builds happening (perhaps related to the Zig 
sub-versions) when I try to build Zig 0.13 locally to test building a 
package of my choice. Is there a way to navigate this blocker?

Regards,
Ashvith

Bootstrapping Zig with no binary blobs: status update and call for packages.

[Hilton Chain]

Hello Guix!

After working on 'wip-zig-bootstrap' branch for the past month[1], I'm glad to
say it's ready for testing now!

+ Zig is bootstrapped to the current release, 0.13.0!

Greatly thanks to Motiejus!  I cannot find out the way without them finishing
the trickiest part.  They have also published a blog post[2].

You can now use Zig 0.13.0 and everything on wip-zig-bootstrap via
‘guix time-machine’, for example:
--8<---------------cut here---------------start------------->8---
guix time-machine \
        --no-channel-files \
        --branch=wip-zig-bootstrap \
        -- shell zig@0.13
--8<---------------cut here---------------end--------------->8---

Building 0.13 from the beginning of bootstrap chain takes around 5 hours on my
machine.  Efraim is building the branch on berlin so there should be some
substitutes on ci.guix.gnu.org, at least for sources.

Patches for adjusting linking behavior have been applied to all release versions
we currently have (0.9.1, 0.10.1, 0.11.0, 0.12.1, 0.13.0).  They should work
just fine without any extra workaround, please report to [1] if you encounter
any issue.

+ Cross build and Zig package manager support are available in zig-build-system.

zig-build-system now by default installs source under "/src" of package output.
Like in go-build-system, this can be disabled by setting #:install-source? to #f.

Other new arguments, #:skip-build? and #:zig-inputs, like the ones in
cargo-build-system, are available.  Note that packages and sources in
#:zig-inputs must have their output names starting with "zig-".

Example packages are available in (gnu packages zig-xyz) on wip-zig-bootstrap[3].

These new features need testing, please send your favorite packages to
guix-patches@gnu.org, targeting wip-zig-bootstrap and prefixed with "zig-team",
the more the better :)


Cheers,
Hilton
---
[1]: Bootstrapping Zig with no Binary Blobs
     https://issues.guix.gnu.org/74217

[2]: Zig Reproduced Without Binaries
     https://jakstys.lt/2024/zig-reproduced-without-binaries/

[3]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/zig-xyz.scm?h=wip-zig-bootstrap

Re: Bootstrapping Zig with no binary blobs: status update and call for packages.

[Hilton Chain]

Hi Ashvith,

On Mon, 02 Dec 2024 04:39:09 +0800,
Ashvith Shetty wrote:
>
> There's a lot of builds happening (perhaps related to the Zig sub-versions)
> when I try to build Zig 0.13 locally to test building a package of my
> choice. Is there a way to navigate this blocker?

I'm building them on my server so there're substitutes on [1], I didn't mention
it before because it requires extra authorization.

Thanks
---
[1]: https://substitute.boiledscript.com

Re: Bootstrapping Zig with no binary blobs: status update and call for packages.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1030 bytes --]

On Mon, Dec 02, 2024 at 07:19:11AM +0800, Hilton Chain wrote:
> Hi Ashvith,
> 
> On Mon, 02 Dec 2024 04:39:09 +0800,
> Ashvith Shetty wrote:
> >
> > There's a lot of builds happening (perhaps related to the Zig sub-versions)
> > when I try to build Zig 0.13 locally to test building a package of my
> > choice. Is there a way to navigate this blocker?
> 
> I'm building them on my server so there're substitutes on [1], I didn't mention
> it before because it requires extra authorization.
> 
> Thanks
> ---
> [1]: https://substitute.boiledscript.com

Also the substitutes on berlin (ci.guix.gnu.org) might need to be baked
before they're served.  So if it says there aren't substitutes it may be
worth waiting a few minutes and asking again to see if they just hadn't
been requested previously.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Bootstrapping Zig with no binary blobs: status update and call for packages.

[Ashvith Shetty]

[-- Attachment #1: Type: text/plain, Size: 857 bytes --]

Hi Hilton,

I left the build running overnight, and that did the job for me.

Bun is a little tricky to package, as it requires providing a modified
JavaScript core from oven-sh/WebKit <https://github.com/oven-sh/WebKit>,
and then there's a bunch of other dependencies that aren't clearly
specified. To be honest, I am not even sure which tag to start with, would
appreciate some guidance here.

So far, I've packaged waylock. Since I am on Xorg for the time-being, I
would really appreciate it if anyone using Wayland would test this
<https://codeberg.org/Ashvith/guix/src/branch/add-waylock> out - because
the PAM configuration
<https://codeberg.org/ifreund/waylock/src/branch/master/pam.d/waylock> may
have to be modified from system-auth to login - however, I could not get
swaylock working, which has "auth include login" by default.

Regards,
Ashvith

[-- Attachment #2: Type: text/html, Size: 1067 bytes --]

Re: Bootstrapping Zig with no binary blobs: status update and call for packages.

[Hilton Chain]

On Mon, 02 Dec 2024 19:30:38 +0800,
Ashvith Shetty wrote:
>
> [1  <text/plain; UTF-8 (7bit)>]
> I left the build running overnight, and that did the job for me.
>
> Bun is a little tricky to package, as it requires providing a modified
> JavaScript core from oven-sh/WebKit <https://github.com/oven-sh/WebKit>,
> and then there's a bunch of other dependencies that aren't clearly
> specified. To be honest, I am not even sure which tag to start with, would
> appreciate some guidance here.

Looks complex, and has bootstrappability issue it seems.

> So far, I've packaged waylock. Since I am on Xorg for the time-being, I
> would really appreciate it if anyone using Wayland would test this
> <https://codeberg.org/Ashvith/guix/src/branch/add-waylock> out - because
> the PAM configuration
> <https://codeberg.org/ifreund/waylock/src/branch/master/pam.d/waylock> may
> have to be modified from system-auth to login - however, I could not get
> swaylock working, which has "auth include login" by default.

Package works for me.  That file won't be used, screen-locker-service-type will
generate one instead.

Re: Bootstrapping Zig with no binary blobs: status update and call for packages.

[Hilton Chain]

On Tue, 03 Dec 2024 23:37:01 +0800,
Hilton Chain wrote:
>
> On Mon, 02 Dec 2024 19:30:38 +0800,
> Ashvith Shetty wrote:
> >
> > So far, I've packaged waylock. Since I am on Xorg for the time-being, I
> > would really appreciate it if anyone using Wayland would test this
> > <https://codeberg.org/Ashvith/guix/src/branch/add-waylock> out - because
> > the PAM configuration
> > <https://codeberg.org/ifreund/waylock/src/branch/master/pam.d/waylock> may
> > have to be modified from system-auth to login - however, I could not get
> > swaylock working, which has "auth include login" by default.
>
> Package works for me.  That file won't be used, screen-locker-service-type will
> generate one instead.

Applied your waylock package to wip-zig-bootstrap with some adjustments.

Re: Bootstrapping Zig with no binary blobs: status update and call for packages.

[Ludovic Courtès]

Hello Hilton,

Hilton Chain <hako@ultrarare.space> skribis:

> After working on 'wip-zig-bootstrap' branch for the past month[1], I'm glad to
> say it's ready for testing now!

This is quite an achievement, congrats!

Ludo’.

Re: Bootstrapping Zig with no binary blobs: status update and call for packages.

[Hilton Chain]

On Fri, 29 Nov 2024 22:39:10 +0800,
Hilton Chain wrote:
>
> Other new arguments, #:skip-build? and #:zig-inputs, like the ones in
> cargo-build-system, are available.  Note that packages and sources in
> #:zig-inputs must have their output names starting with "zig-".

Amend: #:zig-inputs will be removed in the end.  Now it's recommended to add Zig
dependencies to the standard inputs and propagated-inputs fields, and change
dependency names within package sources instead of embed the name mapping into
build system arguments.

A new procedure 'rename-zig-dependencies' is added for producing the needed
origin snippet.  Examples are available in zig-xyz.scm.

Bootstrapped Zig now available in Guix master.

[Hilton Chain]

Hi Guix,

Zig 0.13.0 is now available in master!

It's worth mentioning that some build steps can be used in shell as well:

Reusing packaged dependencies with `unpack-dependencies' from
`(guix build zig-build-system)':
--8<---------------cut here---------------start------------->8---
# Example: river
git clone https://codeberg.org/river/river && cd river && git checkout v0.3.6
guix shell --development river
echo "((assoc-ref (@ (guix build zig-build-system) %standard-phases) 'unpack-dependencies))" | guix repl -t machine
zig build
--8<---------------cut here---------------end--------------->8---

Additionally with `rename-zig-dependencies' from `(gnu packages zig)':
--8<---------------cut here---------------start------------->8---
# Example: zig-zls@0.13
git clone https://github.com/zigtools/zls && cd zls && git checkout 0.13.0
echo "(eval ((@ (gnu packages zig) rename-zig-dependencies) \
        '((\"diffz\" . \"zig-diffz\") \
          (\"known_folders\" . \"zig-known-folders\"))) \
        (interaction-environment))" | guix repl -t machine
guix shell --development zig-zls@0.13
echo "((assoc-ref (@ (guix build zig-build-system) %standard-phases) 'unpack-dependencies))" | guix repl -t machine
zig build
--8<---------------cut here---------------end--------------->8---


Happy New Year :)

Re: Bootstrapped Zig now available in Guix master.

[Ekaitz Zarraga]

On 2024-12-31 4:39 PM, Hilton Chain wrote:
> Hi Guix,
> 
> Zig 0.13.0 is now available in master!
> 
> It's worth mentioning that some build steps can be used in shell as well:
> 
> Reusing packaged dependencies with `unpack-dependencies' from
> `(guix build zig-build-system)':
> --8<---------------cut here---------------start------------->8---
> # Example: river
> git clone https://codeberg.org/river/river && cd river && git checkout v0.3.6
> guix shell --development river
> echo "((assoc-ref (@ (guix build zig-build-system) %standard-phases) 'unpack-dependencies))" | guix repl -t machine
> zig build
> --8<---------------cut here---------------end--------------->8---
> 
> Additionally with `rename-zig-dependencies' from `(gnu packages zig)':
> --8<---------------cut here---------------start------------->8---
> # Example: zig-zls@0.13
> git clone https://github.com/zigtools/zls && cd zls && git checkout 0.13.0
> echo "(eval ((@ (gnu packages zig) rename-zig-dependencies) \
>          '((\"diffz\" . \"zig-diffz\") \
>            (\"known_folders\" . \"zig-known-folders\"))) \
>          (interaction-environment))" | guix repl -t machine
> guix shell --development zig-zls@0.13
> echo "((assoc-ref (@ (guix build zig-build-system) %standard-phases) 'unpack-dependencies))" | guix repl -t machine
> zig build
> --8<---------------cut here---------------end--------------->8---
> 
> 
> Happy New Year :)

Oh!

Great achievement!

As told in IRC, I think we should write something about this, in the 
line of the Mono bootstrap post. Zig people would enjoy it, and also 
Guix people.

It's a great opportunity to let people know how hard is to bootstrap 
Zig, Rust and some other languages, and make them focus a little bit 
more on the language bootstrappability.

I offer myself as a reviewer for the blogpost, if you decide to go that 
route and write one.

Thanks for making this. I followed this effort closely, even if I didn't 
have the chance to take part.

Today Guix is a little bit more comfortable for me thanks to this.

Happy New Year.

Ekaitz

Re: Bootstrapped Zig now available in Guix master.

[Pjotr Prins]

On Tue, Dec 31, 2024 at 04:57:22PM +0100, Ekaitz Zarraga wrote:
> As told in IRC, I think we should write something about this, in the line of
> the Mono bootstrap post. Zig people would enjoy it, and also Guix people.
> 
> It's a great opportunity to let people know how hard is to bootstrap Zig,
> Rust and some other languages, and make them focus a little bit more on the
> language bootstrappability.

Absolutely! Zig is still not in other distros because of this. Julia
has similar problems. The problem is that language developers live in
their own silo, create a self-hosted compiler with binary blob, and
leave it to the distros to figure it out. Hopefully the bootstrap
can carry on and other distros pick up on it. Zig deserves attention.

Great job.

Pj.

Re: Bootstrapped Zig now available in Guix master.

[Hilton Chain]

On Wed, 01 Jan 2025 13:25:37 +0800,
Pjotr Prins wrote:
>
> On Tue, Dec 31, 2024 at 04:57:22PM +0100, Ekaitz Zarraga wrote:
> > As told in IRC, I think we should write something about this, in the line of
> > the Mono bootstrap post. Zig people would enjoy it, and also Guix people.
> >
> > It's a great opportunity to let people know how hard is to bootstrap Zig,
> > Rust and some other languages, and make them focus a little bit more on the
> > language bootstrappability.
>
> Absolutely! Zig is still not in other distros because of this. Julia
> has similar problems. The problem is that language developers live in
> their own silo, create a self-hosted compiler with binary blob, and
> leave it to the distros to figure it out. Hopefully the bootstrap
> can carry on and other distros pick up on it. Zig deserves attention.

I can try to write a post this month, but please don't have much expectation on
it, I'm in no way good at writing :)

Re: Bootstrapped Zig now available in Guix master.

[Attila Lendvai]

> The problem is that language developers live in
> their own silo, create a self-hosted compiler with binary blob, and
> leave it to the distros to figure it out.

which is baffling to me, because i think bootstrapping is one of the more interesting parts of compiler development.

i had the same experience with the Idris2: i made a full bootstrap chain all the way down from GHC, integrated bootstrapping into the normal build process so that bootstrapping remains constantly tested, and also prepared the Guix side... but then my PR was basically ignored.

-- 
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“To every man is given the key to the gates of heaven; the same key opens the gates of hell.

And so it is with science.”
	— Richard Feynman (1918–1988)

Re: Bootstrapped Zig now available in Guix master.

[Timothy Washington]

[-- Attachment #1: Type: text/plain, Size: 338 bytes --]

Hi there Hilton,

This is a great achievement and I appreciate your work on this. I have my
eye on packaging Ghostty for geeks when I came across your work here. Are
you also eyeing packaging Ghostty as well? If not, then don't mind me. If
so, is there anything a Zig and Guix noob can help with?

Thanks again, for all your effort!

Tim

[-- Attachment #2: Type: text/html, Size: 735 bytes --]

Re: Bootstrapped Zig now available in Guix master.

[Hilton Chain]

Hi Timothy,

On Thu, 02 Jan 2025 09:09:10 +0800,
Timothy Washington wrote:
>
> Hi there Hilton,
>
> This is a great achievement and I appreciate your work on this. I have my
> eye on packaging Ghostty for geeks when I came across your work here. Are
> you also eyeing packaging Ghostty as well? If not, then don't mind me. If
> so, is there anything a Zig and Guix noob can help with?

I have checked it briefly but I'm not going to package it.  There's one package
in an external channel[1], you can try it there for now.  Note its definition is
currently written in a way that cannot be accepted into Guix.

Examples for Zig packages are available in (gnu packages zig-xyz) module[2], I
think zig-build-system can support Ghostty well, but the package itself is a bit
complex, so I would advise against packaging it if you don't have much
experience.

Thanks
---
[1]: https://toys.whereis.social/?search=ghostty
[2]: https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/zig-xyz.scm

Re: Bootstrapped Zig now available in Guix master.

[Timothy Washington]

[-- Attachment #1: Type: text/plain, Size: 3095 bytes --]

Hey Hilton, thanks for the feedback.

These were the general paths I was looking at.


A . Manually
Guix shell with zig
Build regular Ghostty. Here, I'd follow the official direction on Building
Ghostty from Source.
https://ghostty.org/docs/install/build

Build Ghostty with guix + custom build steps. But it looks like this has
already been done.
https://codeberg.org/look/saayix/src/branch/main/saayix/packages/terminals.scm#L31


B. Using Nix

Can we use the Ghostty Nix package? The Ghostty "official build environment
is defined by Nix. You do not need to use Nix to build Ghostty, but the Nix
environment is the environment which runs CI tests and builds release
artifacts, so it is the most reliable way to build Ghostty" (ref).
https://ghostty.org/docs/install/build

The Ghostty "Build from Source / Building with Nix" section describes how
to set up the Nix environment.
https://ghostty.org/docs/install/build#building-with-nix

But Guix comes with a service for using Nix packages. "GNU Guix Reference
Manual / Miscellaneous Services".
https://guix.gnu.org/manual/en/html_node/Miscellaneous-Services.html#index-nix_002dservice_002dtype


C. But you're saying that the Toys external channel already packaged it.
https://toys.whereis.social/?search=ghostty

"Note its definition is currently written in a way that cannot be accepted
into Guix."
I'm fine with that for the moment. I think Ghostty is open source. So we
can play around with it to make the packaging more canonical.

I'll take a look at the examples for Zig packages in the (gnu packages
zig-xyz) module.
https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/zig-xyz.scm

I'm confident that the zig-build-system, or some modification of it, can
support Ghostty. It's just a matter of if I can do it, lol :)

But this sets me on the path. I really appreciate your feedback on this 👏🏿


Cheers
Tim


On Fri, 3 Jan 2025 at 09:41, Hilton Chain <hako@ultrarare.space> wrote:

> Hi Timothy,
>
> On Thu, 02 Jan 2025 09:09:10 +0800,
> Timothy Washington wrote:
> >
> > Hi there Hilton,
> >
> > This is a great achievement and I appreciate your work on this. I have my
> > eye on packaging Ghostty for geeks when I came across your work here. Are
> > you also eyeing packaging Ghostty as well? If not, then don't mind me. If
> > so, is there anything a Zig and Guix noob can help with?
>
> I have checked it briefly but I'm not going to package it.  There's one
> package
> in an external channel[1], you can try it there for now.  Note its
> definition is
> currently written in a way that cannot be accepted into Guix.
>
> Examples for Zig packages are available in (gnu packages zig-xyz)
> module[2], I
> think zig-build-system can support Ghostty well, but the package itself is
> a bit
> complex, so I would advise against packaging it if you don't have much
> experience.
>
> Thanks
> ---
> [1]: https://toys.whereis.social/?search=ghostty
> [2]:
> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/zig-xyz.scm
>

[-- Attachment #2: Type: text/html, Size: 4676 bytes --]