how does antioxidant work?

how does antioxidant work?

[jgart]

Hi Maxime,

how does antioxidant build system for rust work at a high level?

https://notabug.org/maximed/cargoless-rust-experiments

all best,

jgart

Re: how does antioxidant work?

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1275 bytes --]


On 08-09-2022 03:17, jgart wrote:
> Hi Maxime,
>
> how does antioxidant build system for rust work at a high level?
>
> https://notabug.org/maximed/cargoless-rust-experiments
>
> all best,

There are two parts:

  * The actual build system (antioxidant.scm) -- it reads the Cargo.toml
    to see what the dependencies are, looks for the (compiled!)
    dependencies in the 'inputs' and 'native-inputs', runs build.rs,
    compiles it (with 'rustc') and install the compiled result (sources
    aren't installed).  It also runs tests.
  * The cargo -> antioxidant transformation code.  The procedure
    vitaminate/auto* takes a package, and rewrites it to fit with
    antioxidant -- one of these is moving #:cargo-inputs into 'inputs',
    and (recursively) rewriting these inputs. Sometimes phases need to
    be adjusted, sometimes for cargo there was #:skip-build? but
    antioxidant doesn't do that, and it runs tests, so it sometimes
    turns out tests need to be disabled.

    (This will have to later be changed to _source code_
    transformations, maybe with "guix style", to integrate things into
    Guix proper)

There are also some other small things with some checks for vendoring, 
#:test-options

Greetings,
Maxime.


[-- Attachment #1.1.1.2: Type: text/html, Size: 1887 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: how does antioxidant work?

[jgart]

On Thu, 08 Sep 2022 12:03:36 +0200 Maxime Devos <maximedevos@telenet.be> wrote:
> There are also some other small things with some checks for vendoring, 
> #:test-options

Cool! Will this build system support rust development with guix or it's
only for packaging end user rust apps?

Re: how does antioxidant work?

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 1613 bytes --]



On 10-09-2022 16:50, jgart wrote:
> On Thu, 08 Sep 2022 12:03:36 +0200 Maxime Devos <maximedevos@telenet.be> wrote:
>> There are also some other small things with some checks for vendoring,
>> #:test-options
> 
> Cool! Will this build system support rust development with guix or it's
> only for packaging end user rust apps?

You can already do things like

$ guix build -f file-containing-a-definition-of-the-rust-library.scm

As how antioxidant treats dependencies is similar to, say, 
python-build-system and gnu-build-system, the previously compiled 
dependencies of the library will be reused.

Unlike C, the configure scripts (‘build.rs’ in Rust), when they exist, 
they do very little work.  For many libraries, the compile time is 
usually short (on the order of, say, a most dozen seconds (*), at least 
on my local computer and not counting tests) (**).

As such, a ‘setup package.scm to use a recursive local-file as origin, 
modify source files, do "guix build -f package.scm"’ should be feasible 
(**).  I wouldn't know how it compares to Cargo though, I never used that.

(*) Note: I haven't actually timed it.  Also, if you adjust the 
optimisation level to something lower and non-LTO, it should in theory 
decrease.
(**) With some exceptions, e.g. rust-proptest takes long to compile.

Caveat: the ‘examples’ and ‘benchmarks’ functionality is unlikely to be 
implemented by me (even though they may be useful for development) as 
they don't bring much value to Guix packaging. [insert the usual patches 
welcome here]

Greetings,
Maxime.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

Re: how does antioxidant work?

[jgart]

On Sat, 10 Sep 2022 18:17:50 +0200 Maxime Devos <maximedevos@telenet.be> wrote:
> 
> Caveat: the ‘examples’ and ‘benchmarks’ functionality is unlikely to be 
> implemented by me (even though they may be useful for development) as 
> they don't bring much value to Guix packaging. [insert the usual patches 
> welcome here]

I'm mostly wondering how we would get the rust community to adopt our rust tooling for serious rust development.

Do we expect rust people to use Guix for developing rust projects without cargo?

Are there other package managers in the rust community that people are using besides cargo?

I realize cargo has issues. I don't have a comprehensive list of them but I'm curious to find out more.

Maxime, what have you researched with regards to cargo issues or where
should I start studying up on that to get familiar with why antioxidant
is needed?

Is there a thread that introduces the need for a new rust build system and why?

all best,

jgart

Re: how does antioxidant work?

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 3650 bytes --]



On 10-09-2022 19:38, jgart wrote:
> On Sat, 10 Sep 2022 18:17:50 +0200 Maxime Devos <maximedevos@telenet.be> wrote:
>>
>> Caveat: the ‘examples’ and ‘benchmarks’ functionality is unlikely to be
>> implemented by me (even though they may be useful for development) as
>> they don't bring much value to Guix packaging. [insert the usual patches
>> welcome here]
> 
> I'm mostly wondering how we would get the rust community to adopt our rust tooling for serious rust development.
> 
> Do we expect rust people to use Guix for developing rust projects without cargo?

A next step I had in mind, was to separate some functionality of 
antioxidant-build-system into a Guix-independent 'antioxidant' Guile 
program, that can also be used by other distro's.

Then Rust people could do things like:

$ apt-get install rust-foo ...
$ antioxidant configure
$ antioxidant build
$ antioxidant check
$ antioxidant install

or

$ guix shell -D some-app
$ antioxidant configure
$ antioxidant build
$ antioxidant check
$ antioxidant install

or whatever.  Basically, separate 'package management' from 'building 
software' -- the former needs the latter, and the latter doesn't really 
care how packages are managed as long as it is told where to find the 
dependencies.

I don't think we should expect rust people to use Guix or antioxidant 
though -- it would be nice if rust people found antioxidant and Guix 
useful, but if they use Cargo, that's fine too for us.

(Not recommended though, because of the usual problems with 
single-language package managers and their registries.)

> Are there other package managers in the rust community that people are using besides cargo?

Maybe bazel: https://github.com/mrboatsman/rust_bazel_hello_world (seems 
to be a mix of a package manager and a build system).

Also maybe crane: https://github.com/ipetkov/crane/blob/
>  
> I realize cargo has issues. I don't have a comprehensive list of them but I'm curious to find out more.
> 
> Maxime, what have you researched with regards to cargo issues or where
> should I start studying up on that to get familiar with why antioxidant
> is needed?
> 
> Is there a thread that introduces the need for a new rust build system and why?

There are:

* https://lists.gnu.org/archive/html/guix-devel/2022-03/msg00269.html
* https://lists.gnu.org/archive/html/guix-devel/2022-08/msg00194.html

but I don't think they are good introductions.

Anyway, for Guix, some of the main issues are:

   * overly picky versions requirements -- the rust-cbindgen-VERSION
     crates are (at least, sufficiently) backwards-compatible.
   * dependencies like "rust-windows-..." must be packaged even when not
     compiling for Windows.
   * cargo is inefficient -- if you compile FOO which depends on BAR
     having the feature "a" enabled, and later compile FOO2 which wants
     the feature "b", then IIUC FOO2 is compiled twice, once with "a" and
     then with "b".  How about compiling it once instead, with both "a"
     and "b".
   * cargo-build-system is inefficient -- if you do
     "guix build this-app", and then do "guix build that-app", then the
     compiled dependencies of "this-app" cannot be reused for "that-app".

     Cargo does not seem to have an option to ... not do that.  Even if
     it did, there is the previous issue with 'features'.

     (Crane seems to somehow tell Cargo how to not do that, but it still
     appears to have the feature problem and it heavily depends on
     crates.io or Cargo.lock, not really suitable for distros).

Greetings,
Maxime.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]