Packaging rust-analyzer is not necessary.

Packaging rust-analyzer is not necessary.

[Paul Alesius]

Rust analyzer is a language server for the Rust programming language.

I've tried to produce a patch for the latest version, and it has
probably hundreds of dependencies that need to be updated.

In addition to that, rust-analyzer is under heavy development and
there is a release every month. Many people will want to use the
nightly version too.

As long as there is rust and rust-cargo in Guix, then rust-analyzer
can be easily compiled and installed from git with "cargo xtask
install --server" using the rust-cargo system.

I'd suggest that a Guix package for rust-analyzer is not needed,
especially due to the excessive time required to update its package
definition and all of the vendored dependency crates, and focus should
instead be on rust and rust-cargo.

With regards,
- Paul

Re: Packaging rust-analyzer is not necessary.

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 2199 bytes --]

Paul Alesius schreef op vr 25-03-2022 om 09:05 [+0100]:
> I've tried to produce a patch for the latest version, and it has
> probably hundreds of dependencies that need to be updated.

Even if you didn't succeed at updating _all_ dependencies, if you
have patches for some of them, please send them.  It will help people
in the future with updating rust-analyzer or other rust packages.

And is it necessary to update _all_ dependencies?  For comparison,
graphical applications usually compile just fine even if gtk+ or glibc
are somewhat old.

> In addition to that, rust-analyzer is under heavy development and
> there is a release every month. Many people will want to use the
> nightly version too.

For many people, a vaguely recent-ish version would be sufficient.  At
least, that's the case for C, GCC and Clang.

> As long as there is rust and rust-cargo in Guix, then rust-analyzer
> can be easily compiled and installed from git with "cargo xtask
> install --server" using the rust-cargo system.
> 
> I'd suggest that a Guix package for rust-analyzer is not needed,
> especially due to the excessive time required to update its package
> definition and all of the vendored dependency crates,

What is the point of a distribution if it not distributing packages?
Removing a package for being ~two months old and pointing users to
Rust's equivalent of "curl ... | bash -" instead seems a disservice to
users to me.

If I just do ‘cargo xtask install --server", how do I know if it isn't
bundling anything or containing malware?  Modifying the source code
seems also non-trivial, compared to Guix where one can do things like
--with-patch.  What about statelessness, reproducibility, time-
machine, the SWH fallback and sharing substitutes on the local network?

It might be possible to do "cargo xtask install --server", but many
advantages of Guix would be lost.

> and focus should instead be on rust and rust-cargo.

Indeed, e.g. it would be nice to figure out how to eliminate #:skip-
build?, replace #:cargo-inputs by regular inputs, figure out how to
stop having to package multiple versions of the same package.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Packaging rust-analyzer is not necessary.

[John Soo]

Hi Paul And Maxime,

> Even if you didn't succeed at updating _all_ dependencies, if you
> have patches for some of them, please send them.  It will help people
> in the future with updating rust-analyzer or other rust packages.

I had a patchset (here: https://issues.guix.gnu.org/46162) adding rust-analyzer and the rest of the other tools that come part of the rust tree. I think it would not be too much work to add if added as outputs/companion packages to rust itself.

> For many people, a vaguely recent-ish version would be sufficient.  At
> least, that's the case for C, GCC and Clang.
> ...
> It might be possible to do "cargo xtask install --server", but many
> advantages of Guix would be lost.


That worked for me. I was using guix' rust tooling for my job. I much prefer using guix over rustup/cargo. I just had to patch rust a lot and my patches haven't made it in (yet?).

> Indeed, e.g. it would be nice to figure out how to eliminate #:skip-
> build?, replace #:cargo-inputs by regular inputs, figure out how to
> stop having to package multiple versions of the same package.

My gut feeling is still that the functional package managers need to collaborate with the rust/cargo team. Rust itself just does not lend itself well to the model.  It is a shame since we share many of the same goals (reproducibility and reliability come to the top of mind).

Kindly,

John

Improving importers best investment for growing gnu/packages/

[Brendan Tildesley]

[-- Attachment #1: Type: text/plain, Size: 2676 bytes --]

On 25/3/22 18:05, Paul Alesius wrote:

> Rust analyzer is a language server for the Rust programming language.
>
> I've tried to produce a patch for the latest version, and it has
> probably hundreds of dependencies that need to be updated.

[...]

> I'd suggest that a Guix package for rust-analyzer is not needed,
> especially due to the excessive time required to update its package
> definition and all of the vendored dependency crates, and focus should
> instead be on rust and rust-cargo.

I understand how defeating it feels when looking at the countless hundreds of
dependencies that get pulled in, however, I believe this can be solved by
improving the importers in Guix.  I think it requires too much manual labour to
maintain packages curretly, particularly when refresh -u will only update the
package version and not dependencies.

Therefore I'm working on making use of
https://github.com/rust-lang/crates.io-index  to fully import base definitions
all required crates rather than pulling metadata from the internet with each refresh.

Maintaining rust packages then becomes mostly
1. Refresh generated package definitions
2. In a separate file edit any hacks/fix needed argument fields etc
3. Edit cargo-build-system to account for the most common hacks like setting
LIBCLANG_PATH.

It should be easy for a guix user to get all the latest crates that they would
get using cargo.

I think importers can be taken to the next level by going beyond metadata and
actually inspecting the contents of source files. For example grepping build
files it can be determined if pkg-config is used and add it to inputs. an index
of .pc files can automatically detect and add the inputs a package is most
likely looking for. One complaint about Guix is that typically just running
./configure; make on a project will not work, but if a guix can have a Universal
Importer where running guix wave-magic-wand in a directory can inspect files,
determine the build system, required inputs etc and build a program, would that
not be wonderful for programmers? Working on KDE made me feel like I was wasting
too much time manually adding inputs and would be better of writing an importer.

Using rust in a hypothetical, imagine if a rust developer that is otherwise
uninterested in Guix, they just want to build their software and distribute
actually chooses to use Guix not due to idealism but because it:
A. Handles their rust dependecies at least as well as cargo.
B. Handles their non-rust dependencies even better.
C. Improved Guix pack/build can produce distributable packages, stripped of
uneeded files that actually runs on other distributions.

Would that not be awesome?

[-- Attachment #2: Type: text/html, Size: 3401 bytes --]

Re: Improving importers best investment for growing gnu/packages/

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 386 bytes --]

Brendan Tildesley schreef op di 29-03-2022 om 10:50 [+1100]:
> For example grepping build files it can be determined if pkg-config is used
> and add it to inputs. an index of .pc files can automatically detect and add
> the inputs a package is most likely looking for

Not sure what the cargo equivalent of this would be, but this looks
nice and feasible.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Improving importers best investment for growing gnu/packages/

[Brendan Tildesley]

[-- Attachment #1: Type: text/plain, Size: 503 bytes --]

On 29/3/22 19:01, Maxime Devos wrote:

> Brendan Tildesley schreef op di 29-03-2022 om 10:50 [+1100]:
>> For example grepping build files it can be determined if pkg-config is used
>> and add it to inputs. an index of .pc files can automatically detect and add
>> the inputs a package is most likely looking for
> Not sure what the cargo equivalent of this would be, but this looks
> nice and feasible.

That was meant as an example of what Guix could provide that cargo doesn't.

> Greetings,
> Maxime.

[-- Attachment #2: Type: text/html, Size: 1192 bytes --]

Re: Improving importers best investment for growing gnu/packages/

[Liliana Marie Prikler]

Am Dienstag, dem 29.03.2022 um 10:50 +1100 schrieb Brendan Tildesley:
> On 25/3/22 18:05, Paul Alesius wrote:
> > Rust analyzer is a language server for the Rust programming
> > language.
> > 
> > I've tried to produce a patch for the latest version, and it has
> > probably hundreds of dependencies that need to be updated.
> [...]
> > I'd suggest that a Guix package for rust-analyzer is not needed,
> > especially due to the excessive time required to update its package
> > definition and all of the vendored dependency crates, and focus
> > should instead be on rust and rust-cargo.
> 
> I understand how defeating it feels when looking at the countless
> hundreds of dependencies that get pulled in, however, I believe this
> can be solved by improving the importers in Guix.  I think it
> requires too much manual labour to maintain packages curretly,
> particularly when refresh -u will only update the
> package version and not dependencies.
> Therefore I'm working on making use of
> https://github.com/rust-lang/crates.io-index to fully import base
> definitions all required crates rather than pulling metadata from the
> internet with each refresh.
> Maintaining rust packages then becomes mostly
> 1. Refresh generated package definitions
> 2. In a separate file edit any hacks/fix needed argument fields etc
> 3. Edit cargo-build-system to account for the most common hacks like
> setting
> LIBCLANG_PATH.
I'm not sure how this is for Rust packages, but in general, I'd
consider over-reliance on `guix refresh' more harmful than not.  There
has recently been a paper published, which shows this to be the case
for spreadsheet debugging – give people a smell checker without
appropriate hints where it might be flawed and they will be worse at
finding flaws than the control group.  I make a claim, that it's the
same for packaging software.  Our importers should come with a warning
sign attached that they perform the best possible approximation based
on available metadata, but that they might be missing important
information or contain bugs, and that the output needs human
verification.

In general, even though one might claim that this conclusion has been
largely forgone in the case of cargo, npm, et al., human intelligence
is a requirement in human computer interaction.  Trying to take too
much of it away leads to adverse effects rather than "smart systems".

> It should be easy for a guix user to get all the latest crates that
> they would get using cargo.
> I think importers can be taken to the next level by going beyond
> metadata and actually inspecting the contents of source files. For
> example grepping build files it can be determined if pkg-config is
> used and add it to inputs.
That'd probably fine for pkg-config as a single package, but to get
packages to a working state, you'll have to do so for more than just
pkg-config (you list dependencies referenced by pkg-config below, for
example).  Speaking more generally, short of actually running the build
(repeatedly) against all possible combinations of inputs, you will not
obtain a complete or accurate input set (particularly also considering
that some of said inputs might not even be packaged).  And even if you
do, all your machine could then tell you, is that it builds; not
whether it functions as intended.  This also assumes that the source
won't need to be patched to remove icky stuff, no phase needs to be
adjusted, ...

> An index of .pc files can automatically detect and add the inputs a
> package is most likely looking for. One complaint about Guix is that
> typically just running ./configure; make on a project will not work,
> but if a guix can have a Universal Importer where running guix wave-
> magic-wand in a directory can inspect files, determine the build
> system, required inputs etc and build a program, would that
> not be wonderful for programmers? Working on KDE made me feel like I
> was wasting too much time manually adding inputs and would be better
> of writing an importer.
To be fair, scanning pkg-config files sounds like a good idea until you
consider it is but a heuristic and pkg-config names do not easily map
to package names.

> Using rust in a hypothetical, imagine if a rust developer that is
> otherwise uninterested in Guix, they just want to build their
> software and distribute actually chooses to use Guix not due to
> idealism but because it:
> A. Handles their rust dependecies at least as well as cargo.
> B. Handles their non-rust dependencies even better.
> C. Improved Guix pack/build can produce distributable packages,
> stripped of unneeded files that actually runs on other distributions.
> 
> Would that not be awesome?
Hypothetically speaking, apart from the fact that rust inputs are not
inputs because the Rust ecosystem is awful, maintaining a guix.scm on
the side of your Rust package should not be that much more difficult
than maintaining a cargo.toml.  And better yet, you don't need those
god-awful lock files, you could just commit a channels.scm as well if
bit-for-bit reproducibility in CI is a requirement.  You could even
implement a scheme-based toml parser, that establishes a 1:1 mapping
between toml and guix package.  Ironically, Javascript is ahead in that
regard, as someone has already written a Guix package reader in
ECMAScript.  Wisp is also an option for those experiencing terminal
parenthephobia.

IMHO the onus is on Rust to create an ecosystem that does not collapse
if someone removes leftpad.

Re: Improving importers best investment for growing gnu/packages/

[zimoun]

Hi,

On Tue, 29 Mar 2022 at 10:50, Brendan Tildesley <mail@brendan.scot> wrote:

> Therefore I'm working on making use of
> https://github.com/rust-lang/crates.io-index  to fully import base definitions
> all required crates rather than pulling metadata from the internet with each refresh.

From my point of view, the issue is that it requires external Rust tools
and I think such tool will never be part of the Guix dependencies.  Guix
itself cannot depends on Rust-thing, GHC-thing, OCaml-thing,
Julia-thing, etc. because they ease the importers.

And that’s why the current importers parse metadata and somehow
“reimplement” the logic behind the package manager of each language.  I
agree that it is a lot of work and hard to maintain but doing so allow
Guix to have a small footprint (dependency, speaking).

That’s said, how to improve the importers?

Well, Guix provides now GUIX_EXTENSIONS_PATH which allows to extend by
other subcommands.  “guix workflow” is an example, I do not remember how
“guix home” did initially, anyway!

My point is: this mechanism allows to have a package (with many
dependencies) extending Guix itself.  Somehow, it reads

    guix install guix-rusty
    guix rusty <stuff>

where ’guix-rusty’ would be a Guile package (program) using whatever it
needs from the Rust ecosystem (say parse system calls of cargo or
whatever).

Guix is Scheme library after all, let use it! :-)


Cheers,
simon

Re: Packaging rust-analyzer is not necessary.

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 755 bytes --]

John Soo schreef op za 26-03-2022 om 09:33 [-0700]:
> > Indeed, e.g. it would be nice to figure out how to eliminate
> > #:skip-
> > build?, replace #:cargo-inputs by regular inputs, figure out how to
> > stop having to package multiple versions of the same package.
> 
> My gut feeling is still that the functional package managers need to
> collaborate with the rust/cargo team. Rust itself just does not lend
> itself well to the model.  It is a shame since we share many of the
> same goals (reproducibility and reliability come to the top of mind).

(responding to an old post)

FWIW, it turns out that Rust actually works well with Guix, only Cargo
doesn't, see reccent threads about antioxidant-build-system.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]