Removing #:skip-build? from the crate importer?

Removing #:skip-build? from the crate importer?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 717 bytes --]

Hi guix,

Often, when new rust package definitions are submitted at guix-
patches@, I see #:skip-build? #false.  Apparently it's added by default
in (guix import cargo), with some exceptions.  However, ‘(guix)Rust
Crates’ states:

   Care should be taken to ensure the correct version of dependencies
are used; to this end we try to refrain from skipping the tests or
using ‘#:skip-build?’ when possible. Of course this is not always
possible [...]

so I respond by asking to remove #:skip-build?.

As such, WDYT of removing #:skip-build? #false from (guix import
crate)?  FWIW, this was added in commit
269c1db41bd82f93c7ae5c62a4969a423e556183 to (guix import crate)?

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Removing #:skip-build? from the crate importer?

[Hartmut Goebel]

Hi,

since rust does not support anything like static or dynamic libraries, 
building (intermediate) crates is useless like a hole in my head. Any 
output on any intermediate crate will just be thrown away.

> Often, when new rust package definitions are submitted at guix-
> patches@, I see #:skip-build? #false.  Apparently it's added by default
> in (guix import cargo), with some exceptions.
The idea behind is to have #:skip-buiild #f for all "top level" crates, 
which are assumed to be programs. Thus, only crates imported recursively 
will get get #:skip-buiild #t. If one imports a single crate, it well 
get #:skip-buiild #f — which is what you experience.

>    However, ‘(guix)Rust
> Crates’ states:
>
>     Care should be taken to ensure the correct version of dependencies
> are used; to this end we try to refrain from skipping the tests or
> using ‘#:skip-build?’ when possible. Of course this is not always
> possible [...]

This text is from 2020-02-17 (written by Effraim) and predates 
269c1db41bd8 (committed 2020-12-02).

While I understand the intention of this, I'm not convinces about it. 
Primary this will lead to a huge wast of time and electrical power - 
just to trash the results. This will not only effect our own build farm, 
but also each user.

Please be aware, that with #:skip-buiild #t, every crate will be build 
again by every other crate using it. So if crate AA is used by B1 and B2 
and C1 depends on B1 and B2, AA will be build 4 times!


> As such, WDYT of removing #:skip-build? #false from (guix import
> crate)?  FWIW, this was added in commit

I would propose the opposite: Keep it and make #t the default.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Compiling rust things without cargo (super WIP POC)

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1671 bytes --]

Hartmut Goebel schreef op do 31-03-2022 om 21:47 [+0200]:
since rust does not support anything like static or dynamic libraries, 
building (intermediate) crates is useless like a hole in my head. Any 
output on any intermediate crate will just be thrown away.

In my experiments, it looks like the rust compiler actually _does_
support static libraries, though perhaps cargo doesn't.

I invite you to take a look at <https://notabug.org/maximed/cargoless-rust-experiments>.
It contains a minimal rust library (libhello) and a minimal 'hello
world'-style application that uses 'libhello'.

To simulate how Guix compiles C software (but here applied to Rust), 
the Makefile does the following:

  * Run 'rustc --crate-type=lib libhello/hello.rs -o out/libhello/lib/libhello.rlib',
    to compile the library and install it in 'out/libhello/lib/libhello.rlib'
    (cf. /gnu/store/...-libhello.../lib/hello.so).

  * Run 'rustc -Lout/libhello/lib hello-app/main.rs -o out/hello-oxygen/bin/hello'
    to compile the application.  By default, rustc will fail because it cannot find
    the library.  However, if -Lout/libhello/lib is passed, then it does find it!

    (cf. LIBRARY_PATH=/gnu/store/.../lib & gcc -L/gnu/store/.../lib)

This is a rather basic example (no transitive dependencies, no test dependencies,
no macros ...), but it seems like there are some possibilities here ...

As a next step, maybe I could try writing a Guix package definition for libhello
and hello-oxygen, gradually making things more complicated (macros, transitive
dependencies, some non-toy Rust dependencies, a Guix build system ...)?

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Compiling rust things without cargo (super WIP POC)

[Brendan Tildesley]

Recently I did the bevy (game engine) hello world tutorial[1].
To facilitate less slow iteration time when developing, it supports 
dynamic linking,
disabling some optimizations, and using the lld linker for fast linking, 
although using lld failed for me.

When running cargo build, only your project is recompiled quickly and 
linked.

I was wondering how this dynamic linking might be facilitated in Guix?


Note: I had to update cargo to 0.60 to compile bevy with edition2021

[1] https://bevyengine.org/learn/book/getting-started/

Re: Compiling rust things without cargo (super WIP POC)

[Ludovic Courtès]

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

> I invite you to take a look at <https://notabug.org/maximed/cargoless-rust-experiments>.
> It contains a minimal rust library (libhello) and a minimal 'hello
> world'-style application that uses 'libhello'.
>
> To simulate how Guix compiles C software (but here applied to Rust), 
> the Makefile does the following:
>
>   * Run 'rustc --crate-type=lib libhello/hello.rs -o out/libhello/lib/libhello.rlib',
>     to compile the library and install it in 'out/libhello/lib/libhello.rlib'
>     (cf. /gnu/store/...-libhello.../lib/hello.so).
>
>   * Run 'rustc -Lout/libhello/lib hello-app/main.rs -o out/hello-oxygen/bin/hello'
>     to compile the application.  By default, rustc will fail because it cannot find
>     the library.  However, if -Lout/libhello/lib is passed, then it does find it!
>
>     (cf. LIBRARY_PATH=/gnu/store/.../lib & gcc -L/gnu/store/.../lib)
>
> This is a rather basic example (no transitive dependencies, no test dependencies,
> no macros ...), but it seems like there are some possibilities here ...

Interesting.

> As a next step, maybe I could try writing a Guix package definition for libhello
> and hello-oxygen, gradually making things more complicated (macros, transitive
> dependencies, some non-toy Rust dependencies, a Guix build system ...)?

I guess the whole question is whether that technique can be made to work
for the vast majority of Rust packages.  I’d suggest working in that
direction: writing a build system as a first step, using it in all the
Rust packages, and analyzing the kinds of problems encountered, with the
goal of estimating the effort it would take to make it work for every
single package.  Easier said than done, I guess.

Overall, I’m afraid Rust packaging is getting out of hands and we’re all
looking elsewhere.  For example, that Rust packages live in their own
separate world means there’s no tooling available, leading to poor QA, a
proliferation of versions of the same packages that never get removed,
and so on.  I think addressing that, for instance with something as I
proposed in <https://issues.guix.gnu.org/53127>, should be high
priority.

Thoughts?

Ludo’.

Re: Compiling rust things without cargo (super WIP POC)

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 709 bytes --]

Ludovic Courtès schreef op vr 01-04-2022 om 11:10 [+0200]:
> Overall, I’m afraid Rust packaging is getting out of hands and we’re all
> looking elsewhere.  For example, that Rust packages live in their own
> separate world means there’s no tooling available, leading to poor QA, a
> proliferation of versions of the same packages that never get removed,
> and so on.  I think addressing that, for instance with something as I
> proposed in <https://issues.guix.gnu.org/53127>, should be high
> priority.

I think I'll look into the cycle issue, maybe cargo.toml can be
modified to not use rust-cfg-il and maybe the solution, if any, could
be used for other cycles.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Compiling rust things without cargo (super WIP POC)

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 755 bytes --]

Ludovic Courtès schreef op vr 01-04-2022 om 11:10 [+0200]:
> I guess the whole question is whether that technique can be made to work
> for the vast majority of Rust packages.  I’d suggest working in that
> direction: writing a build system as a first step, using it in all the
> Rust packages, and analyzing the kinds of problems encountered, with the
> goal of estimating the effort it would take to make it work for every
> single package.  Easier said than done, I guess.

Do you know a ‘real’ Rust applications with few transitive dependencies
(say, 3 or so) with preferably few rust source files?  That would be a
practical target for the ad-hoc Makefile and eventually, the new
antioxidant-build-system.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Compiling rust things without cargo (super WIP POC)

[Hartmut Goebel]

Am 01.04.22 um 12:08 schrieb Maxime Devos:
> Do you know a ‘real’ Rust applications with few transitive dependencies
> (say, 3 or so) with preferably few rust source files?

For my tests I used „roxmltree“, and by searching I just discovered 
https://crates.io/crates/hello_exercism.

HTH

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Compiling rust things without cargo (super WIP POC)

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 407 bytes --]

Maxime Devos schreef op do 31-03-2022 om 22:06 [+0200]:
> As a next step, maybe I could try writing a Guix package definition
> for libhello
> and hello-oxygen, gradually making things more complicated (macros,
> transitive
> dependencies, some non-toy Rust dependencies, a Guix build system
> ...)?

Update: it can now build rust-cfg-if, libhello and rust-hello with
Guix!

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Compiling rust things without cargo (super WIP POC)

[Ludovic Courtès]

Maxime Devos <maximedevos@telenet.be> skribis:

> Maxime Devos schreef op do 31-03-2022 om 22:06 [+0200]:
>> As a next step, maybe I could try writing a Guix package definition
>> for libhello
>> and hello-oxygen, gradually making things more complicated (macros,
>> transitive
>> dependencies, some non-toy Rust dependencies, a Guix build system
>> ...)?
>
> Update: it can now build rust-cfg-if, libhello and rust-hello with
> Guix!

Woohoo, keep up the good work!  :-)

Ludo’.

Building hexyl (a rust app) without cargo, with antioxidant-build-system

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1056 bytes --]

Hi,

antioxidant-build-system can now be used for some ‘real’ software -- it
compiles 'hexyl'.  To test, download
<https://notabug.org/maximed/cargoless-rust-experiments> (commit:
d09fd93750ac6d77e0c85623286b45cf5c3b055b) and run
"guix build -L . -f guix.scm" and then

$ cat guix.scm | /gnu/store/[...]-hexyl-0.8.0/bin/hexyl
> lots of coloured hex output

Some features of antioxidant-build-system:

  * no copying source code of dependencies
  * no compiling dependencies again -- old artifacts are reused
  * all dependencies use the usual package input system
    (native-inputs, inputs, propagated-inputs)

Limitations:

  * no support for linking to arbitrary shared libraries yet
    (only rust deps)
  * makes a few assumptions on the source layout (can be fixed
    by using more info from Cargo.toml)
  * no tests
  * no cross-compilation yet
  * no shared libraries (just replacing 'rlib' by 'dylib' causes problems)
  * code is a bit messy
  * no cdylib yet (probably needed for librsvg)

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Building hexyl (a rust app) without cargo, with antioxidant-build-system

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2291 bytes --]

Maxime Devos schreef op za 02-04-2022 om 17:18 [+0200]:
>   * makes a few assumptions on the source layout (can be fixed
>     by using more info from Cargo.toml)

This has now been partially resolved.  It now looks in Cargo.toml to
determine where Rusts's equivalent of 'configure' (build.rs) is
located, and the Cargo.toml can now override the default src/lib.rs.

Some other improvements that weren't announced previously:

  * The code now automatically detects cycles and reports which
    packages are involved in the cycle.
  * --edition (cf. -std=c11/c99/...) is set appropriately, depending on
    Cargo.toml, fixing some build failures.
  * All default 'features' (in Cargo.toml) are now enabled by default.
  * Package definitions can request non-default features to be built
    anyway.

    A difference from cargo-build-system: features are set in the
    package of the rust crate, not the package using the rust crate.
  * build.rs is now run, which reduces the number of features that need
    to be set manually.
  * proc-macro crates are now named .so instead of .rlib, apparently
    naming them .rlib even though they aren't .rlibs confuses rustc
    or something.
  * #[deny(warnings)] (rust's equivalent of -Werrors) is now ignored
    (FWIW this is done by cargo as well).
  * Conversion from cargo-build-system to antioxidate-build-system is
    now automated by a recursive 'vitaminate/auto' procedure and some
    ad-hoc cycle breaking.

Some things that could perhaps be moved to (upstream) Guix:

  * Sometimes test and build.rs dependencies are listed in
    #:cargo-inputs and #:cargo-development-inputs (e.g.
    rust-quickcheck, rust-cc).  I deleted the test dependencies
    (because they contribute to cycles and for now running tests
    isn't supported anyways).
   
    Maybe they can be moved to #:cargo-development-inputs upstream
    (untested)?  Extending 'check-inputs-should-be-native' would also
    be nice too ...

  * There's a cycle between rust-backtrace-sys and rust-cc can be
    eliminated by deleting build.rs and tweaking Cargo.toml, at cost
    of losing some support for Android (which is not supported by Guix
    except in the sense that Linux is supported).

Greetings,
Maxime

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Building hexyl (a rust app) without cargo, with antioxidant-build-system

[Hartmut Goebel]

Am 05.04.22 um 18:10 schrieb Maxime Devos:
> Some other improvements that weren't announced previously: 

Wow! Impressive!

I'd be eager to tr it with sequoia-openpgp. Please drop me a note when 
you think your work might be able to build that beast.


>    * Package definitions can request non-default features to be built
>      anyway.
>
>      A difference from cargo-build-system: features are set in the
>      package of the rust crate, not the package using the rust crate.

How is this intended to work?

Package 1 has features AAA (= default) and BBB. So a .rlib is build for 
each feature (package1-AAA.rlib, package1-BBB.rlib) Or will one need to 
define two guix packages (package1+aaa and package1+bbb) and make the 
build-system build the respective feature?

I personally would prefer the former. Thus package2 would pick up the 
pre-compiled rlib for the respective feature.

Rational: If there are more features to be combined, the number of 
packages to be build can a order of square. So defining all these 
packages becomes a burden quickly. This is a computer's job :-) The 
build-system could easily build all combinations. suffixing each rlib 
with a short hash over the feature names,


-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Building hexyl (a rust app) without cargo, with antioxidant-build-system

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1723 bytes --]

Hartmut Goebel schreef op wo 06-04-2022 om 17:49 [+0200]:

> [...]
> >     * Package definitions can request non-default features to be built
> >       anyway.
> > 
> >       A difference from cargo-build-system: features are set in the
> >       package of the rust crate, not the package using the rust crate.
> 
> How is this intended to work?
> 
> Package 1 has features AAA (= default) and BBB. So a .rlib is build for 
> each feature (package1-AAA.rlib, package1-BBB.rlib) Or will one need to 
> define two guix packages (package1+aaa and package1+bbb) and make the 
> build-system build the respective feature?
> 
> I personally would prefer the former. Thus package2 would pick up the 
> pre-compiled rlib for the respective feature.
> 
> Rational: If there are more features to be combined, the number of 
> packages to be build can a order of square. So defining all these 
> packages becomes a burden quickly. This is a computer's job :-) The 
> build-system could easily build all combinations. suffixing each rlib 
> with a short hash over the feature names,

My third solution: no package variants, no multiple .rlib variants.
Instead, by default enable the ‘default features’ mentioned in the
Cargo.toml, and (manually) add the extra features to the package
definition of the Rust dependency if they turn out to be needed by the
dependent.

This is like adding --enable-this --enable-that, ... to ffmpeg's
#:configure-flags instead of building a variant of ffmpeg for each
dependent of ffmpeg that needs a different feature set.

So far, I have not encountered incompatible features ... Let's see if
it will become a problem.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Building hexyl (a rust app) without cargo, with antioxidant-build-system

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1682 bytes --]

On Sat, Apr 02, 2022 at 05:18:55PM +0200, Maxime Devos wrote:
> Hi,
> 
> antioxidant-build-system can now be used for some ‘real’ software -- it
> compiles 'hexyl'.  To test, download
> <https://notabug.org/maximed/cargoless-rust-experiments> (commit:
> d09fd93750ac6d77e0c85623286b45cf5c3b055b) and run
> "guix build -L . -f guix.scm" and then
> 
> $ cat guix.scm | /gnu/store/[...]-hexyl-0.8.0/bin/hexyl
> > lots of coloured hex output
> 
> Some features of antioxidant-build-system:
> 
>   * no copying source code of dependencies
>   * no compiling dependencies again -- old artifacts are reused
>   * all dependencies use the usual package input system
>     (native-inputs, inputs, propagated-inputs)
> 
> Limitations:
> 
>   * no support for linking to arbitrary shared libraries yet
>     (only rust deps)
>   * makes a few assumptions on the source layout (can be fixed
>     by using more info from Cargo.toml)
>   * no tests
>   * no cross-compilation yet
>   * no shared libraries (just replacing 'rlib' by 'dylib' causes problems)
>   * code is a bit messy
>   * no cdylib yet (probably needed for librsvg)

Something that might help with that would be to also include the source
of the crate somewhere in the output. Then at the worst we could just
put the rust inputs of librsvg as regular inputs and let librsvg do its
own special build thing. We'd then still keep the build dependency tree
you've got working.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Building hexyl (a rust app) without cargo, with antioxidant-build-system

[raingloom]

On Mon, 30 May 2022 11:23:21 +0300
Efraim Flashner <efraim@flashner.co.il> wrote:

> On Sat, Apr 02, 2022 at 05:18:55PM +0200, Maxime Devos wrote:
> > Hi,
> > 
> > antioxidant-build-system can now be used for some ‘real’ software
> > -- it compiles 'hexyl'.  To test, download
> > <https://notabug.org/maximed/cargoless-rust-experiments> (commit:
> > d09fd93750ac6d77e0c85623286b45cf5c3b055b) and run
> > "guix build -L . -f guix.scm" and then
> > 
> > $ cat guix.scm | /gnu/store/[...]-hexyl-0.8.0/bin/hexyl  
> > > lots of coloured hex output  
> > 
> > Some features of antioxidant-build-system:
> > 
> >   * no copying source code of dependencies
> >   * no compiling dependencies again -- old artifacts are reused
> >   * all dependencies use the usual package input system
> >     (native-inputs, inputs, propagated-inputs)
> > 
> > Limitations:
> > 
> >   * no support for linking to arbitrary shared libraries yet
> >     (only rust deps)
> >   * makes a few assumptions on the source layout (can be fixed
> >     by using more info from Cargo.toml)
> >   * no tests
> >   * no cross-compilation yet
> >   * no shared libraries (just replacing 'rlib' by 'dylib' causes
> > problems)
> >   * code is a bit messy
> >   * no cdylib yet (probably needed for librsvg)  
> 
> Something that might help with that would be to also include the
> source of the crate somewhere in the output. Then at the worst we
> could just put the rust inputs of librsvg as regular inputs and let
> librsvg do its own special build thing. We'd then still keep the
> build dependency tree you've got working.
> 
> 

Maybe put the source in an extra output, not the main "out" one.
Splitting them up later for size reduction will be much more difficult
than doing this from the get-go.

Re: Building hexyl (a rust app) without cargo, with antioxidant-build-system

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1022 bytes --]

Efraim Flashner schreef op ma 30-05-2022 om 11:23 [+0300]:
> Something that might help with that would be to also include the
> source of the crate somewhere in the output. Then at the worst we
> could just put the rust inputs of librsvg as regular inputs and let
> librsvg do its own special build thing. We'd then still keep the
> build dependency tree you've got working.

Maybe, but that kind of defeats the point of

 * no compiling dependencies again -- old artifacts are reused

and librsvg's build system seems to use Cargo, so then we again have
the problem that Cargo wants Windows crates etc. that are useless in
Guix and its overly-pickyness in versioning, we lose the ability to let
crates be shared libraries (not yet implemented, but could be
implemented in antioxidant) ..., so I'd rather just implement cdylib
and patch the build system appropriately.

(It's build system is more complicated than ordinary crates, but the
additional Makefile.am isn't too long).

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Compiling rust things without cargo (super WIP POC)

[Hartmut Goebel]

Am 31.03.22 um 22:06 schrieb Maxime Devos:
> In my experiments, it looks like the rust compiler actually_does_
> support static libraries, though perhaps cargo doesn't.

AFAIU this assumption is correct.


> I invite you to take a look at<https://notabug.org/maximed/cargoless-rust-experiments>.
> It contains a minimal rust library (libhello) and a minimal 'hello
> world'-style application that uses 'libhello'.
Impressive!
> As a next step, maybe I could try writing a Guix package definition for libhello
> and hello-oxygen, gradually making things more complicated (macros, transitive
> dependencies, some non-toy Rust dependencies, a Guix build system ...)?

Here is my challenge :-) 
<https://gitlab.com/sequoia-pgp/sequoia/-/blob/main/openpgp/Cargo.toml>: 
different dependencies per feature, os, target-arch and target-os as 
well as passing on features to dependencies.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Building hexyl (a rust app) without cargo, with antioxidant-build-system

[Brendan Tildesley]

[-- Attachment #1: Type: text/plain, Size: 180 bytes --]

I would have called it car/gone./

Do you believe sidestepping cargo all together like this is a good long 
term strategy?
In particular that importing packages will still be easy.

[-- Attachment #2: Type: text/html, Size: 347 bytes --]

Re: Building hexyl (a rust app) without cargo, with antioxidant-build-system

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 929 bytes --]

Brendan Tildesley schreef op ma 04-04-2022 om 15:10 [+1000]:
>  I would have called it cargone.
>  
>  Do you believe sidestepping cargo all together like this is a good
> long term strategy?  In particular that importing packages will still
> be easy. 

The package definitions look almost the same, except for:

  * #:cargo-inputs and #:cargo-development-inputs are moved to
    'inputs'/'propagated-inputs' and 'native-inputs'
  * For a few packages, #:features ~'(...) needs to be added
  * antioxidant-build-system doesn't look at version numbers,
    so the numbers of variants of packages that needs to be defined
    could perhaps be reduced
  * Cycles are not supported

The fourth point might make things difficult though I have some ideas
on how to resolve it (without simply disabling tests)  (TBI!). 
Otherwise, I don't see much complications with importing packages.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Building hexyl (a rust app) without cargo, with antioxidant-build-system

[raingloom]

On Mon, 04 Apr 2022 11:26:45 +0200
Maxime Devos <maximedevos@telenet.be> wrote:

> Brendan Tildesley schreef op ma 04-04-2022 om 15:10 [+1000]:
> >  I would have called it cargone.
> >  
> >  Do you believe sidestepping cargo all together like this is a good
> > long term strategy?  In particular that importing packages will
> > still be easy.   
> 
> The package definitions look almost the same, except for:
> 
>   * #:cargo-inputs and #:cargo-development-inputs are moved to
>     'inputs'/'propagated-inputs' and 'native-inputs'
>   * For a few packages, #:features ~'(...) needs to be added
>   * antioxidant-build-system doesn't look at version numbers,
>     so the numbers of variants of packages that needs to be defined
>     could perhaps be reduced
>   * Cycles are not supported
> 
> The fourth point might make things difficult though I have some ideas
> on how to resolve it (without simply disabling tests)  (TBI!). 
> Otherwise, I don't see much complications with importing packages.
> 
> Greetings,
> Maxime.

I'm also a bit worried with the trend of Guix trying to duplicate
functionality already present in language package managers and config
file formats. It creates a two-sources-of-truth situation. Trying to
keep one up to date with the other can be an issue, this is why I
didn't create a custom record type for Yggdrasil config files and just
used a generic JSON converter. Which paid off, since there were in fact
changes in the config fields between versions.
Will this build system avoid that issue as well?
I guess if the data it operates on has truly stable semantics, then
writing an independent implementation is not as big a problem, since
once written and debugged, it won't need to change.
(And getting rid of cargo would be nice for Rust dev on Guix.)

Re: Building hexyl (a rust app) without cargo, with antioxidant-build-system

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 4102 bytes --]

raingloom schreef op wo 06-04-2022 om 00:28 [+0200]:
> I'm also a bit worried with the trend of Guix trying to duplicate
> functionality already present in language package managers and config
> file formats. It creates a two-sources-of-truth situation.

For service configurations, there is only one source of truth: the
configuration record.  It just happens to be converted to another
format.  The underlying file format, TOML is not Cargo-specific
(https://toml.io/en/) and parsed with python-toml, presumably python-
toml parses it correctly.  (Whether antioxidant-build-system interprets
it correctly, is of course a different matter.)

Also, Cargo is both a package manager and a build system.  The package
manager part is useless to Guix, we even have to work-around it (see
e.g. patch-cargo-checksums, crate-closure).  Unfortunately, there does
not seem to be a way to only use the build system part of Cargo and
ignore the package manager part, so the build system part has to be
reimplemented ...

> Trying to keep one up to date with the other can be an issue, this is why I
> didn't create a custom record type for Yggdrasil config files and just
> used a generic JSON converter. Which paid off, since there were in fact
> changes in the config fields between versions.

The configuration record fields does not have to support every new
thing, it just haves to be _sufficiently_ up to date for users.  And if
a configuration field that's needed is missing, then it's almost
trivial to add it.

Additionally, some services have an 'extra-content' escape hatch. 
yggrasil-configuration could have one too, then the user could have
both write most things as Scheme (benefit: largely unified interface,
more automation possible) _and_ be able to use the latest features not
yet directly supported by the configuraiton fields, via the escape
hatch.

> Will this build system avoid that issue as well?

antioxidant-build-system is a build system, not a configuration format.

I've found that most Cargo.toml are rather tame.  They often just set:

  * name
  * Some metadata like version, authors, home-page, include, exclude
    ... that antioxidant-build-system doesn't need.
  * sometimes 'build' or 'path'
  * dependencies  / dev-dependencies / build-dependencies, also
    unneeded by antioxidant-build-system (the importer might need them,
    but that's a separate component)

Looking at <https://doc.rust-lang.org/cargo/reference/manifest.html>,
there are some more things are things that antioxidant-build-system
doesn't need:

  * resolver
  * badges, [[bench]]
  * the [patch] and [replace]  --- just use a variant package in the
    package inputs

Now, there are some things that aren't yet supported by antioxidant-
build-system but probably need to be:

  * autobins / autoexamples / autotests / [lib] / [bin] / [example]
    / [links]
  * setting various environment variables before build.rs
    (there's probably some list somewhere ...)

But aside from that, it seems that the essential parts of Cargo.toml
are rather tame and unlikely to be added to.  And if something
essential is added anyway and needs to be supported, then it should be
easy to modify antioxidant-build-system appropriately (long-term)
and/or work-around it by setting some environment variables locally or
maybe some substitute* (short-term).

> I guess if the data it operates on has truly stable semantics, then
> writing an independent implementation is not as big a problem, since
> once written and debugged, it won't need to change.
> (And getting rid of cargo would be nice for Rust dev on Guix.)

AFAICT, the parts important to Guix are sufficiently stable -- not
immutable (sometimes changes do happen).  And even if it was unstable,
it still seems much better than just using Cargo (due to much shorter
build times, less strict about versions, not having to add winapi
crates when compiling for Linux, no copying tarballs around, no
#:cargo-inputs instead of [propagated-]inputs ...).

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]