bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies

bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies

[ng0]

The cargo build-system should be able to filter out Windows
dependencies.
Ignore blocks with this in Cargo.toml files:
[target.'cfg(windows)'.dependencies]

until the next [ ] starts.

I also found that most *32-sys packages (currently ALL *32-sys
I've seen) are windows specific.
As the build system is simple enough to not complain, you just
have to learn about this while doing the work.

-- 
♥Ⓐ  ng0
PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org

bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies

[zimoun]

Hi,

On Mon, 02 Jan 2017 at 00:46, ng0 <ng0@libertad.pw> wrote:
> The cargo build-system should be able to filter out Windows
> dependencies.
> Ignore blocks with this in Cargo.toml files:
> [target.'cfg(windows)'.dependencies]
>
> until the next [ ] starts.
>
> I also found that most *32-sys packages (currently ALL *32-sys
> I've seen) are windows specific.
> As the build system is simple enough to not complain, you just
> have to learn about this while doing the work.

This bug is really old and the story about cargo has a bit evolved.  Is
it still relevant with the recent additions?

If no, feel free to close it.  And without more comment, I will close it
after the usual 2 weeks delay.


All the best,
simon

bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies

[Hartmut Goebel]

Am 18.12.20 um 20:56 schrieb zimoun:
> Is it still relevant with the recent additions?

I just checked this with sequoia 0.20.0: The package "winapi" is still 
downloaded and compiled - even if obviously not used sicne on Linux.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies

[zimoun]

Hi Hartmut,

On Sat, 19 Dec 2020 at 14:10, Hartmut Goebel <h.goebel@crazy-compilers.com> wrote:
> Am 18.12.20 um 20:56 schrieb zimoun:
>> Is it still relevant with the recent additions?
>
> I just checked this with sequoia 0.20.0: The package "winapi" is still
> downloaded and compiled - even if obviously not used sicne on Linux.

Should the remove of this “winapi” happen at the build-system level or
at the package per package level?

Other said, are the Windows dependencies normalized by Cargo?

All the best,
simon

bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies

[John Soo]

[-- Attachment #1: Type: text/plain, Size: 185 bytes --]

     Hi zimoun,  

  
I think that the configuration phase will fail even if conditional compilation dependencies are missing.    It’s worth a shot though!
  

  
- John
     

[-- Attachment #2: Type: text/html, Size: 232 bytes --]

bug#25327: windows-related dependencies in crates

[Nicolas Graves via Bug reports for GNU Guix]

Hi all,

I've delved into the subject the past week and now have a clearer view
on it :

- windows (and mac) dependencies are not needed when compiling a single
  package. It is possible to bypass it without changing the
  build-system, by a few means :

           - either by using the #:cargo-build-flags argument (doesn't
             work everytime, seems to work only seldomly with
             --target unix)
             In this option, one would also need to modify Cargo.toml
             file before the package phase, since `cargo package`
             vendors all possible dependencies to ensure a package on
             all environments.
             For instance (1) :
             #:phases
             (modify-phases %standard-phases
               (add-before 'package 'unix-cargo-toml
                 (lambda _
                   (delete-file "Cargo.toml")
                   (substitute* "Cargo.toml.orig"
                     (("^mac-notification-sys.*") "")
                     (("^chrono.*") "")
                     (("^winrt-notification.*") ""))
                   (rename-file "Cargo.toml.orig" "Cargo.toml")
                   #t)))

           - either by using the environment variable
             "CARGO_BUILD_RUSTFLAGS"  during the build phase, like that
             for instance (2) :
             #:phases
             (modify-phases %standard-phases
               (add-before 'build 'unix-cargo-toml
                 (setenv
                  "CARGO_BUILD_RUSTFLAGS"
                  (string-append
                   (getenv "CARGO_BUILD_RUSTFLAGS")
                   "--cfg 'target_os=\"linux\"'"))))
             In this case, the package phase would also require the same
             change as in (1). Note that this might not be backward
             compatible further down the line (seen in the cargo
             manual).

For some reason, these two means don't seem to propagate when using such
packages as inputs.

There are also the options to use a snippet or a patch on the source
itself. The snippet on the source could look like (1), or the patch as
its effects, and we would still need to use (2). Haven't experimented
much with that yet.

In any case, all this seems kinda bulky for an edit on an imported base,
and since we have lots of information with `cargo metadata`, I imagine
it would be possible to modify the build-system itself, so that it only
considers `target_os = "linux"` or possibly redox / bsd / dragonfly if
guix considers porting to these platforms.

I have 2 possible ideas :

1) Maybe something like adding a phase after unpack, which would :

- parse `cargo metadata` to isolate only needed dependencies (by maybe
generating and applying a patch ? or maybe there's a cargo command for
generating a Cargo.toml based on modified metadata ?). In any case,
modifying the Cargo.toml to only keep relevant information.

2) Maybe rethinking the package phase entirely instead of relying on
`cargo package --no-verify --no-metadata` which seems to not allow
options without interferring with the Cargo.toml file.

In both cases, we would then need to :
- modify the phase were environment variables are defined to add (2).
- modify the crate import script accordingly (not trivial).

As for the second hypothesis, according to the cargo manual, what `cargo
package` does is the following :

1. Load and check the current workspace, performing some basic checks.
Path dependencies are not allowed unless they have a version key. Cargo
will ignore the path key for dependencies in published
packages. dev-dependencies do not have this restriction.

2. Create the compressed .crate file.
a. The original Cargo.toml file is rewritten and normalized.
b. [patch], [replace], and [workspace] sections are removed from the manifest.
c. Cargo.lock is automatically included if the package contains an
  executable binary or example target. cargo-install(1) will use the
  packaged lock file if the --locked flag is used.
d. A .cargo_vcs_info.json file is included that contains information about
  the current VCS checkout hash if available (not included with
  --allow-dirty).

3. Extract the .crate file and build it to verify it can build.
- This will rebuild your package from scratch to ensure that it can be
  built from a pristine state. The --no-verify flag can be used to skip
  this step.

4. Check that build scripts did not modify any source files.

Step 3 and 4 are actually not performed, since the build-system calls
cargo package with the --no-verify flag. The checks in step 1 doesn't
seem that essential on guix, but I imagine we could replicate
them. Clearly the essential step here is 2, but since we get a crate,
steps a. and b. should actually already be done at the source level, and
step d. is not essential because of origin's field version. Step 2c is
not essential either since Cargo.lock file is actually deleted during
the build system.

I might try to implement these changes, although I do not have enough
time and experience to do that quickly, neither enough horsepower to try
to rebuild everything after that. I included Maxime Devos in the
conversation, would it be possible to fork guix to develop this and try
to rebuild every rust package after that on a guix build farm ? Sorry,
haven't got in touch with guix developers yet, still imposter's
syndrome effect, but implementing this doesn't seem too hard. What would
I need to do to get going on that ?

Cheers,

Nicolas Graves

bug#25327: windows-related dependencies in crates

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 870 bytes --]

Nicolas Graves schreef op do 24-03-2022 om 18:18 [+0100]:
> I included Maxime Devos in the
> conversation, would it be possible to fork guix to develop this and try
> to rebuild every rust package after that on a guix build farm ?

In the past, for some build system changes, separate branches have been
created that are automatically built by ci.guix.gnu.org.  A non-
committer and a committer could then collaborate together (on
guix-devel@gnu.org / guix-patches@gnu.org), though only the committer
can
actually push the changes.  If, at some point, the changes appear to be
‘ready’, the build on ci.guix.gnu.org can be started to build the
substitutes.  If there are no regressions and the builds completed, the
branch can be merged.

TBC, I'm not a committer, but I'd be willing to review FWIW (I don't
understand cargo).

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies

[zimoun]

Hi,

On Thu, 24 Mar 2022 at 18:18, Nicolas Graves <ngraves@ngraves.fr> wrote:

> I might try to implement these changes, although I do not have enough
> time and experience to do that quickly, neither enough horsepower to try
> to rebuild everything after that. I included Maxime Devos in the
> conversation, would it be possible to fork guix to develop this and try
> to rebuild every rust package after that on a guix build farm ? Sorry,
> haven't got in touch with guix developers yet, still imposter's
> syndrome effect, but implementing this doesn't seem too hard. What would
> I need to do to get going on that ?

Just to know what is the status of this bug#25327 [1]?  Well, I have not
followed closely some recent changes about Rust and Guix.  Maybe, some
patches are already around?

Cheers,
simon

1: <http://issues.guix.gnu.org/issue/25327>

bug#25327: cargo build-system should be able to filter out target.cfg(windows) dependencies

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 1286 bytes --]



On 08-10-2022 16:15, zimoun wrote:
> Hi,
> 
> On Thu, 24 Mar 2022 at 18:18, Nicolas Graves <ngraves@ngraves.fr> wrote:
> 
>> I might try to implement these changes, although I do not have enough
>> time and experience to do that quickly, neither enough horsepower to try
>> to rebuild everything after that. I included Maxime Devos in the
>> conversation, would it be possible to fork guix to develop this and try
>> to rebuild every rust package after that on a guix build farm ? Sorry,
>> haven't got in touch with guix developers yet, still imposter's
>> syndrome effect, but implementing this doesn't seem too hard. What would
>> I need to do to get going on that ?
> 
> Just to know what is the status of this bug#25327 [1]?  Well, I have not
> followed closely some recent changes about Rust and Guix.  Maybe, some
> patches are already around?

antioxidant-build-system (*) automatically ignores all dependencies it 
can't find (including Windows dependencies), and as such solves this.

(*) close to 100%, the main remaining problem is lack of support for 
workspaces; I'm currently looking for packaging a topological sorting 
algorithm but it depends on other unpackaged things etc.  Other problems 
should be rather minor.

Greetings,
Maxime.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]