rust: cargo build-system: how to handle libraries

rust: cargo build-system: how to handle libraries

[ng0]

Hi,

I've got about 94 rust packages on my todo list for our
(secushare) current prototype version.

Today I assembled the full list of them and started with the
crate `libc`. This crate comes with no Cargo.toml file, where I
already am aware of the limitation of our cargo build-system.

Before I can start my task now, I have to enhance the cargo
build-system in guix. Now I'm very new to packaging for
rust. When a Cargo.toml is in place, you can find lots of
information very easily in search engines, but without it gets
very difficult.

Please correct me if this is wrong:

We run (or this is the default) cargo with `--bin`, which is
necessary for everything which needs to be compiled and features
a Cargo.toml.

To "build" libraries, we have to explicitly tell cargo to not
run/build/whatever with --bin ?
Or do I have to interprete what I read a while ago "we just need
to copy the libraries to the store" as literally "if we encounter
no Cargo.toml, let's copy everything to the store as is"?

CC'ing David, Eric, and Ludovic as you all were involved to some
degree in the discussion before about cargo and rust.
-- 
♥Ⓐ  ng0
PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org

Re: rust: cargo build-system: how to handle libraries

[David Craven]

> We run (or this is the default) cargo with `--bin`, which is
> necessary for everything which needs to be compiled and features
> a Cargo.toml.

All crates have a Cargo.toml file. The interesting crates are the ones with
a Cargo.lock file.

> To "build" libraries, we have to explicitly tell cargo to not
> run/build/whatever with --bin ?

Building libraries is currently not very useful at the moment, since they are
not meant to be libraries in the C sense, but in the sense that it's a
collection
of source code that can be reused in a rust project. One can create a library
in the C sense, but most crates do not do this and are not intended to be abi
compatible with C.

> Or do I have to interprete what I read a while ago "we just need
> to copy the libraries to the store" as literally "if we encounter
> no Cargo.toml, let's copy everything to the store as is"?

If we encounter no Cargo.lock file it generally means the above.

Re: rust: cargo build-system: how to handle libraries

[ng0]

David Craven <david@craven.ch> writes:

>> We run (or this is the default) cargo with `--bin`, which is
>> necessary for everything which needs to be compiled and features
>> a Cargo.toml.
>
> All crates have a Cargo.toml file. The interesting crates are the ones with
> a Cargo.lock file.
>
>> To "build" libraries, we have to explicitly tell cargo to not
>> run/build/whatever with --bin ?
>
> Building libraries is currently not very useful at the moment, since they are
> not meant to be libraries in the C sense, but in the sense that it's a
> collection
> of source code that can be reused in a rust project. One can create a library
> in the C sense, but most crates do not do this and are not intended to be abi
> compatible with C.
>
>> Or do I have to interprete what I read a while ago "we just need
>> to copy the libraries to the store" as literally "if we encounter
>> no Cargo.toml, let's copy everything to the store as is"?
>
> If we encounter no Cargo.lock file it generally means the above.
>

Thanks for your answer.

But, ouch... note to self, packaging while hanging out with other
people to watch streamed talks sometimes does not work out.
The actual failure is different, the build system (currently)
expects a Cargo.toml everywhere, and the folder `libc-test'
doesn't seem to have one, breaking the build phase.

phase `unpack' succeeded after 0.1 seconds
starting phase `patch-usr-bin-file'
phase `patch-usr-bin-file' succeeded after 0.0 seconds
starting phase `patch-source-shebangs'
patch-shebang: ./ci/docker/arm-linux-androideabi/accept-licenses.sh: warning: no binary for interpreter `expect' found in $PATH
patch-shebang: ./ci/docker/arm-linux-androideabi/install-ndk.sh: changing `/bin/sh' to `/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0/bin/sh'
patch-shebang: ./ci/docker/arm-linux-androideabi/install-sdk.sh: changing `/bin/sh' to `/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0/bin/sh'
patch-shebang: ./ci/dox.sh: changing `/bin/sh' to `/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0/bin/sh'
patch-shebang: ./ci/run.sh: changing `/bin/sh' to `/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0/bin/sh'
phase `patch-source-shebangs' succeeded after 0.0 seconds
starting phase `configure'
phase `configure' succeeded after 0.0 seconds
starting phase `patch-generated-file-shebangs'
patch-shebang: ./ci/docker/arm-linux-androideabi/accept-licenses.sh: warning: no binary for interpreter `expect' found in $PATH
phase `patch-generated-file-shebangs' succeeded after 0.0 seconds
starting phase `build'
error: failed to read `/tmp/guix-build-rust-libc-0.2.18.drv-0/libc-0.2.18/libc-test/Cargo.toml`

Caused by:
  No such file or directory (os error 2)
phase `build' failed after 0.5 seconds
builder for `/gnu/store/wcn4kyhfmp6pi8kdpssdc3k90hkwwhrp-rust-libc-0.2.18.drv' failed with exit code 1
@ build-failed /gnu/store/wcn4kyhfmp6pi8kdpssdc3k90hkwwhrp-rust-libc-0.2.18.drv - 1 builder for `/gnu/store/wcn4kyhfmp6pi8kdpssdc3k90hkwwhrp-rust-libc-0.2.18.drv' failed with exit code 1
guix build: error: build failed: build of `/gnu/store/wcn4kyhfmp6pi8kdpssdc3k90hkwwhrp-rust-libc-0.2.18.drv' failed


-- 
♥Ⓐ  ng0
PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org

Re: rust: cargo build-system: how to handle libraries

[ng0]

ng0 <ng0@libertad.pw> writes:

> David Craven <david@craven.ch> writes:
>
>>> We run (or this is the default) cargo with `--bin`, which is
>>> necessary for everything which needs to be compiled and features
>>> a Cargo.toml.
>>
>> All crates have a Cargo.toml file. The interesting crates are the ones with
>> a Cargo.lock file.
>>
>>> To "build" libraries, we have to explicitly tell cargo to not
>>> run/build/whatever with --bin ?
>>
>> Building libraries is currently not very useful at the moment, since they are
>> not meant to be libraries in the C sense, but in the sense that it's a
>> collection
>> of source code that can be reused in a rust project. One can create a library
>> in the C sense, but most crates do not do this and are not intended to be abi
>> compatible with C.
>>
>>> Or do I have to interprete what I read a while ago "we just need
>>> to copy the libraries to the store" as literally "if we encounter
>>> no Cargo.toml, let's copy everything to the store as is"?
>>
>> If we encounter no Cargo.lock file it generally means the above.
>>
>
> Thanks for your answer.
>
> But, ouch... note to self, packaging while hanging out with other
> people to watch streamed talks sometimes does not work out.
> The actual failure is different, the build system (currently)
> expects a Cargo.toml everywhere, and the folder `libc-test'
> doesn't seem to have one, breaking the build phase.

Of course I meant Cargo.lock, not Cargo.toml ... so I know what I
have to search for, and this makes it a bit easier than the
previous search.

> phase `unpack' succeeded after 0.1 seconds
> starting phase `patch-usr-bin-file'
> phase `patch-usr-bin-file' succeeded after 0.0 seconds
> starting phase `patch-source-shebangs'
> patch-shebang: ./ci/docker/arm-linux-androideabi/accept-licenses.sh: warning: no binary for interpreter `expect' found in $PATH
> patch-shebang: ./ci/docker/arm-linux-androideabi/install-ndk.sh: changing `/bin/sh' to `/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0/bin/sh'
> patch-shebang: ./ci/docker/arm-linux-androideabi/install-sdk.sh: changing `/bin/sh' to `/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0/bin/sh'
> patch-shebang: ./ci/dox.sh: changing `/bin/sh' to `/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0/bin/sh'
> patch-shebang: ./ci/run.sh: changing `/bin/sh' to `/gnu/store/qkw4zrwfybxww8f56nkb6hggxambk89b-bash-4.4.0/bin/sh'
> phase `patch-source-shebangs' succeeded after 0.0 seconds
> starting phase `configure'
> phase `configure' succeeded after 0.0 seconds
> starting phase `patch-generated-file-shebangs'
> patch-shebang: ./ci/docker/arm-linux-androideabi/accept-licenses.sh: warning: no binary for interpreter `expect' found in $PATH
> phase `patch-generated-file-shebangs' succeeded after 0.0 seconds
> starting phase `build'
> error: failed to read `/tmp/guix-build-rust-libc-0.2.18.drv-0/libc-0.2.18/libc-test/Cargo.toml`
>
> Caused by:
>   No such file or directory (os error 2)
> phase `build' failed after 0.5 seconds
> builder for `/gnu/store/wcn4kyhfmp6pi8kdpssdc3k90hkwwhrp-rust-libc-0.2.18.drv' failed with exit code 1
> @ build-failed /gnu/store/wcn4kyhfmp6pi8kdpssdc3k90hkwwhrp-rust-libc-0.2.18.drv - 1 builder for `/gnu/store/wcn4kyhfmp6pi8kdpssdc3k90hkwwhrp-rust-libc-0.2.18.drv' failed with exit code 1
> guix build: error: build failed: build of `/gnu/store/wcn4kyhfmp6pi8kdpssdc3k90hkwwhrp-rust-libc-0.2.18.drv' failed
>
>
> -- 
> ♥Ⓐ  ng0
> PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org

-- 
♥Ⓐ  ng0
PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org

Re: rust: cargo build-system: how to handle libraries

[David Craven]

> Of course I meant Cargo.lock, not Cargo.toml ... so I know what I
> have to search for, and this makes it a bit easier than the
> previous search.

See cargo workspaces:
https://github.com/rust-lang/libc/blob/master/Cargo.toml
https://github.com/rust-lang/rfcs/blob/master/text/1525-cargo-workspace.md

It's a fairly new thing, you'll have to track down the relevant
commits in cargo and check if the cargo version we are using is new
enough and understand it's semantics and how it applies to the cargo
build system.

My recommendation to fix this issue is to not run the tests if there
isn't a Cargo.lock file, but this is likely not going to solve the
underlying cause in this case. You can also package version 0.2.16
which does not use workspaces.

Re: rust: cargo build-system: how to handle libraries

[ng0]

David Craven <david@craven.ch> writes:

>> Of course I meant Cargo.lock, not Cargo.toml ... so I know what I
>> have to search for, and this makes it a bit easier than the
>> previous search.
>
> See cargo workspaces:
> https://github.com/rust-lang/libc/blob/master/Cargo.toml
> https://github.com/rust-lang/rfcs/blob/master/text/1525-cargo-workspace.md

Aha! Thanks :)

> It's a fairly new thing, you'll have to track down the relevant
> commits in cargo and check if the cargo version we are using is new
> enough and understand it's semantics and how it applies to the cargo
> build system.

Is the cargo version attached to the rustc version? In other
words, could I try and package cargo 0.15 (released 6 days ago)
and it will not care about the rustc version?

> My recommendation to fix this issue is to not run the tests if there
> isn't a Cargo.lock file, but this is likely not going to solve the
> underlying cause in this case. You can also package version 0.2.16
> which does not use workspaces.

Are there mechanisms in place in either (our) cargo build-system
or cargo in general to disable tests, or do I have to write this
(if I would run into problems with packaging 0.15 cargo)?

-- 
♥Ⓐ  ng0
PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org

Re: rust: cargo build-system: how to handle libraries

[ng0]

ng0 <ng0@libertad.pw> writes:

> David Craven <david@craven.ch> writes:
>
>>> Of course I meant Cargo.lock, not Cargo.toml ... so I know what I
>>> have to search for, and this makes it a bit easier than the
>>> previous search.
>>
>> See cargo workspaces:
>> https://github.com/rust-lang/libc/blob/master/Cargo.toml
>> https://github.com/rust-lang/rfcs/blob/master/text/1525-cargo-workspace.md
>
> Aha! Thanks :)
>
>> It's a fairly new thing, you'll have to track down the relevant
>> commits in cargo and check if the cargo version we are using is new
>> enough and understand it's semantics and how it applies to the cargo
>> build system.
>
> Is the cargo version attached to the rustc version? In other
> words, could I try and package cargo 0.15 (released 6 days ago)
> and it will not care about the rustc version?

Okay, I understand how we (currently) package rust and cargo, I
try to upgrade rust to 1.14, maybe this helps.

>> My recommendation to fix this issue is to not run the tests if there
>> isn't a Cargo.lock file, but this is likely not going to solve the
>> underlying cause in this case. You can also package version 0.2.16
>> which does not use workspaces.
>
> Are there mechanisms in place in either (our) cargo build-system
> or cargo in general to disable tests, or do I have to write this
> (if I would run into problems with packaging 0.15 cargo)?
>
> -- 
> ♥Ⓐ  ng0
> PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org

-- 
♥Ⓐ  ng0
PGP keys and more: https://n0is.noblogs.org/ http://ng0.chaosnet.org