unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Pinned/fixed versions should be a requirement.
@ 2023-09-05  2:59 Distopico
  2023-09-05 17:15 ` wolf
                   ` (4 more replies)
  0 siblings, 5 replies; 10+ messages in thread
From: Distopico @ 2023-09-05  2:59 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 2112 bytes --]


In my experience using Guix and attempting to make contributions, I've
noticed that the vast majority of times when a library breaks, it's
because one of its dependencies changed version. For instance,
referencing something like `rust-my-lib-1`, where "1" refers to the
semver "1.x" of the package, e.g., "1.0.32", and `rust-foo` depends on
`rust-my-lib == 1.0.32`. However, in some other package got updated to
"1.0.34" so `rust-foo` will break. I've seen this happen a lot with
Haskell and Rust libraries.

Many libraries in different languages don't follow semver, which can
lead to cases like `rust-serde-json`, which, between versions "1.0.97"
and "1.0.98," changed its dependency from `indexmap` "1.x" to "2.x,"
causing several packages like rust-analyzer to break. I've also observed
this in Haskell with packages like "text."

This is problematic because:

    - Over time, it becomes more vulnerable to libraries/packages
      breaking.

    - It makes reproducible software more challenging, as "1.x" can
      encompass many versions.

    - Debugging becomes difficult since that package could be a deep
      dependency in the system package dependency chain, such as
      Rust/Haskell/NPM, etc.

    - It makes it more likely that if a dependency changes, many
      packages will need to be updated/rebuilt due to that change.

For these reasons, I believe that pinned versions should be a
requirement in libraries, always specifying the exact dependency, for
example, `rust-serde-json-1.0.98`.

This brings the following benefits:

    - Fewer packages will be prone to rebuilding when changing the
      definition of a library.

    - Reduced likelihood of libraries/packages breaking.

    - Easier maintenance of packages and libraries without fear of
      breaking others or having to update many.

There could be some potential disadvantages:

    - The list of library versions may grow larger, making it harder to
      detect orphaned or unused versions.

Additionally, I believe that a command to list the dependency tree of a
package would be ideal for easier debugging.

Regards!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 515 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Pinned/fixed versions should be a requirement.
  2023-09-05  2:59 Pinned/fixed versions should be a requirement Distopico
@ 2023-09-05 17:15 ` wolf
  2023-09-07 12:39 ` Pinned " Simon Tournier
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 10+ messages in thread
From: wolf @ 2023-09-05 17:15 UTC (permalink / raw)
  To: Distopico; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 3046 bytes --]

On 2023-09-04 21:59:47 -0500, Distopico wrote:
> 
> In my experience using Guix and attempting to make contributions, I've
> noticed that the vast majority of times when a library breaks, it's
> because one of its dependencies changed version. For instance,
> referencing something like `rust-my-lib-1`, where "1" refers to the
> semver "1.x" of the package, e.g., "1.0.32", and `rust-foo` depends on
> `rust-my-lib == 1.0.32`. However, in some other package got updated to
> "1.0.34" so `rust-foo` will break. I've seen this happen a lot with
> Haskell and Rust libraries.
> 
> Many libraries in different languages don't follow semver, which can
> lead to cases like `rust-serde-json`, which, between versions "1.0.97"
> and "1.0.98," changed its dependency from `indexmap` "1.x" to "2.x,"
> causing several packages like rust-analyzer to break. I've also observed
> this in Haskell with packages like "text."
> 
> This is problematic because:
> 
>     - Over time, it becomes more vulnerable to libraries/packages
>       breaking.
> 
>     - It makes reproducible software more challenging, as "1.x" can
>       encompass many versions.
> 
>     - Debugging becomes difficult since that package could be a deep
>       dependency in the system package dependency chain, such as
>       Rust/Haskell/NPM, etc.
> 
>     - It makes it more likely that if a dependency changes, many
>       packages will need to be updated/rebuilt due to that change.
> 
> For these reasons, I believe that pinned versions should be a
> requirement in libraries, always specifying the exact dependency, for
> example, `rust-serde-json-1.0.98`.
> 
> This brings the following benefits:
> 
>     - Fewer packages will be prone to rebuilding when changing the
>       definition of a library.
> 
>     - Reduced likelihood of libraries/packages breaking.
> 
>     - Easier maintenance of packages and libraries without fear of
>       breaking others or having to update many.
> 
> There could be some potential disadvantages:
> 
>     - The list of library versions may grow larger, making it harder to
>       detect orphaned or unused versions.

I was recently thinking about this, and I think this should be solvable by
introducing a boolean flag (auto-dependency?) to the package definition stating
whether the package was added intentionally, or just as an auto-imported
dependency.  The importer (when running as -r) would set it to #t for all except
the top-level package.

After that we could have a clean up script that would delete all packages that
have this flag set to #t and are not referenced from any packages that have the
flag set to #f.  That should ensure that the list of packages does get cleaned
up eventually.

> 
> Additionally, I believe that a command to list the dependency tree of a
> package would be ideal for easier debugging.
> 
> Regards!

W.

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Pinned versions should be a requirement.
  2023-09-05  2:59 Pinned/fixed versions should be a requirement Distopico
  2023-09-05 17:15 ` wolf
@ 2023-09-07 12:39 ` Simon Tournier
  2023-09-07 15:35   ` Distopico
  2023-09-09 22:50 ` Pinned/fixed " Attila Lendvai
                   ` (2 subsequent siblings)
  4 siblings, 1 reply; 10+ messages in thread
From: Simon Tournier @ 2023-09-07 12:39 UTC (permalink / raw)
  To: Distopico, guix-devel

Hi,

On Mon, 04 Sep 2023 at 21:59, Distopico <distopico@riseup.net> wrote:

> In my experience using Guix and attempting to make contributions, I've
> noticed that the vast majority of times when a library breaks, it's
> because one of its dependencies changed version.

That’s because contributor and/or reviewer are not running

    guix refresh -l foobar

for checking that all the dependants of foobar still build.  Well, there
is no easy solution, although QA is helping.  Note that this points is
not listed in the long list of Katherine,

        Re: How can we decrease the cognitive overhead for contributors?
        Katherine Cox-Buday <cox.katherine.e@gmail.com>
        Wed, 30 Aug 2023 10:11:02 -0600
        id:e47299e8-43f8-aac8-61ba-420daeb88bdd@gmail.com
        https://yhetil.org/guix/e47299e8-43f8-aac8-61ba-420daeb88bdd@gmail.com
        https://lists.gnu.org/archive/html/guix-devel/2023-08



>                                                  For instance,
> referencing something like `rust-my-lib-1`, where "1" refers to the
> semver "1.x" of the package, e.g., "1.0.32", and `rust-foo` depends on
> `rust-my-lib == 1.0.32`. However, in some other package got updated to
> "1.0.34" so `rust-foo` will break. I've seen this happen a lot with
> Haskell and Rust libraries.

Well, from my point of view, the issue depends on the upstream package
ecosystem.  Considering Haskell, we follow LTS, currently

        ;; Latest LTS version compatible with current GHC.
        (define %default-lts-version "20.5")

from the module (guix import stackage).  And note the lint checker,
“guix lint -l”:

    - haskell-stackage: Ensure Haskell packages use Stackage LTS versions


> For these reasons, I believe that pinned versions should be a
> requirement in libraries, always specifying the exact dependency, for
> example, `rust-serde-json-1.0.98`.

In the Subject: of the message, it reads pinned/fixed.  The difference
is:

  + 'pinned': version that rarely changes
  + 'fixed': mainly the ones with security fixes used as grafts

as discussed in [1].  Maybe you already know, it is just in case or for
other potential readers. :-)


> This brings the following benefits:

This issue that you are describing is not new and it often comes up.
The current consensus in order to detect such breaks is to apply patches
to the dedicated branch of the team.

In the case you are reporting, it seems that the changes should go first
to the rust-team branch and that branch should be merges once all
builds.

The workflow is not clearly established and still discussed.  We
collectively came up with a rough first draft at the last Guix Days back
on February.  Here some pointers,

        Discussion notes on releases and branches
        Andreas Enge <andreas@enge.fr>
        Thu, 09 Feb 2023 13:19:28 +0100
        id:Y+Tk0OKTyKKDqqlm@jurong
        https://yhetil.org/guix/Y+Tk0OKTyKKDqqlm@jurong
        https://lists.gnu.org/archive/html/guix-devel/2023-02

        Feature branches (was: 04/09: gnu: mesa: Update to 23.0.3)
        Andreas Enge <andreas@enge.fr>
        Mon, 08 May 2023 18:33:53 +0200
        id:ZFkkcQ2MOpku9YaF@jurong
        https://yhetil.org/guix/ZFkkcQ2MOpku9YaF@jurong
        https://lists.gnu.org/archive/html/guix-devel/2023-05

and the hope is that such workflow should act as your proposal but
without maintaining many variants (regular, /pinned and potential /fixed
for security fixes).


> Additionally, I believe that a command to list the dependency tree of a
> package would be ideal for easier debugging.

Do you mean “guix refresh -l”?

--8<---------------cut here---------------start------------->8---
$ guix refresh -l gmsh
Building the following 3 packages would ensure 4 dependent packages are rebuilt: openfoam-com@2212 python-pygmsh@7.1.17 openfoam-org@10.20230119
$ guix build $(guix refresh -l gmsh | cut -d':' -f2)
    … build all packages impacted by a change in the package gmsh …
--8<---------------cut here---------------end--------------->8---

Cheers,
simon


1: [bug#61078] [PATCH 0/4] Rename '/fixed' by '/pinned'
Simon Tournier <zimon.toutoune@gmail.com>
Thu, 26 Jan 2023 17:42:38 +0100
id:20230126164238.2462377-1-zimon.toutoune@gmail.com
https://yhetil.org/guix/20230126164238.2462377-1-zimon.toutoune@gmail.com
https://issues.guix.gnu.org/msgid/20230126164238.2462377-1-zimon.toutoune@gmail.com


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Pinned versions should be a requirement.
  2023-09-07 12:39 ` Pinned " Simon Tournier
@ 2023-09-07 15:35   ` Distopico
  2023-09-09 10:39     ` Simon Tournier
  0 siblings, 1 reply; 10+ messages in thread
From: Distopico @ 2023-09-07 15:35 UTC (permalink / raw)
  To: Simon Tournier; +Cc: guix-devel

[-- Attachment #1: Type: text/plain, Size: 4125 bytes --]


On 2023-09-07, Simon Tournier <zimon.toutoune@gmail.com> wrote:

> Hi,
>
> On Mon, 04 Sep 2023 at 21:59, Distopico <distopico@riseup.net> wrote:
>
>> In my experience using Guix and attempting to make contributions, I've
>> noticed that the vast majority of times when a library breaks, it's
>> because one of its dependencies changed version.
>
> That’s because contributor and/or reviewer are not running
>
>     guix refresh -l foobar
>
> for checking that all the dependants of foobar still build.  Well, there
> is no easy solution, although QA is helping.  Note that this points is
> not listed in the long list of Katherine,
>
>         Re: How can we decrease the cognitive overhead for contributors?
>         Katherine Cox-Buday <cox.katherine.e@gmail.com>
>         Wed, 30 Aug 2023 10:11:02 -0600
>         id:e47299e8-43f8-aac8-61ba-420daeb88bdd@gmail.com
>         https://yhetil.org/guix/e47299e8-43f8-aac8-61ba-420daeb88bdd@gmail.com
>         https://lists.gnu.org/archive/html/guix-devel/2023-08
>
>
>
>>                                                  For instance,
>> referencing something like `rust-my-lib-1`, where "1" refers to the
>> semver "1.x" of the package, e.g., "1.0.32", and `rust-foo` depends on
>> `rust-my-lib == 1.0.32`. However, in some other package got updated to
>> "1.0.34" so `rust-foo` will break. I've seen this happen a lot with
>> Haskell and Rust libraries.
>
> Well, from my point of view, the issue depends on the upstream package
> ecosystem.  Considering Haskell, we follow LTS, currently
>
>         ;; Latest LTS version compatible with current GHC.
>         (define %default-lts-version "20.5")
>
> from the module (guix import stackage).  And note the lint checker,
> “guix lint -l”:
>
>     - haskell-stackage: Ensure Haskell packages use Stackage LTS versions
>
>

In terms of haskell I notice an incompetence of versions so even GHC are
semver the required version was other and several packages are taking
"text" internal GHC type and no the required package, you can see that
in this patch https://issues.guix.gnu.org/64840

>> For these reasons, I believe that pinned versions should be a
>> requirement in libraries, always specifying the exact dependency, for
>> example, `rust-serde-json-1.0.98`.
>
> In the Subject: of the message, it reads pinned/fixed.  The difference
> is:
>
>   + 'pinned': version that rarely changes
>   + 'fixed': mainly the ones with security fixes used as grafts
>
> as discussed in [1].  Maybe you already know, it is just in case or for
> other potential readers. :-)
>
>
For this case I'm referring mostly to pinned versions as requirement but
for LTS packages fixed could good as well

>> Additionally, I believe that a command to list the dependency tree of a
>> package would be ideal for easier debugging.
>
> Do you mean “guix refresh -l”?
>
> --8<---------------cut here---------------start------------->8---
> $ guix refresh -l gmsh
> Building the following 3 packages would ensure 4 dependent packages are rebuilt: openfoam-com@2212 python-pygmsh@7.1.17 openfoam-org@10.20230119
> $ guix build $(guix refresh -l gmsh | cut -d':' -f2)
>     … build all packages impacted by a change in the package gmsh …
> --8<---------------cut here---------------end--------------->8---
>

I'm referring to something more like `cargo tree --depth=N` or `cabal freeze` to
see all the dependencias like

--8<---------------cut here---------------start------------->8---
my_package v0.1.0 (/gnu/rust.scm)
└── rust-rand v0.7.3
    ├── rust-getrandom v0.1.14
    │   ├── rust-cfg-if v0.1.10
    │   └── rust-libc v0.2.68
    ├── rust-libc v0.2.68 (*)
    ├── rust-rand_chacha v0.2.2
    │   ├── rust-ppv-lite86 v0.2.6
    │   └── rust-rand-core v0.5.1
    │       └── rust-libc v0.1.14 (*) ---->> We can detect this!
    └── rust-rand-core v0.5.1 (*)
[native-inputs]
└── cc v1.0.50
--8<---------------cut here---------------end--------------->8---

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 515 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Pinned versions should be a requirement.
  2023-09-07 15:35   ` Distopico
@ 2023-09-09 10:39     ` Simon Tournier
  0 siblings, 0 replies; 10+ messages in thread
From: Simon Tournier @ 2023-09-09 10:39 UTC (permalink / raw)
  To: Distopico; +Cc: guix-devel

Hi,

On Thu, 07 Sep 2023 at 10:35, Distopico <distopico@riseup.net> wrote:

> In terms of haskell I notice an incompetence of versions so even GHC are
> semver the required version was other and several packages are taking
> "text" internal GHC type and no the required package, you can see that
> in this patch https://issues.guix.gnu.org/64840

Sorry for the late reply on this bug.  I did not take the time for
looking at it since my come back from summer holidays.  I have marked it
since weeks… :-)


> I'm referring to something more like `cargo tree --depth=N` or `cabal freeze` to
> see all the dependencias like
>
> --8<---------------cut here---------------start------------->8---
> my_package v0.1.0 (/gnu/rust.scm)
> └── rust-rand v0.7.3
>     ├── rust-getrandom v0.1.14
>     │   ├── rust-cfg-if v0.1.10
>     │   └── rust-libc v0.2.68
>     ├── rust-libc v0.2.68 (*)
>     ├── rust-rand_chacha v0.2.2
>     │   ├── rust-ppv-lite86 v0.2.6
>     │   └── rust-rand-core v0.5.1
>     │       └── rust-libc v0.1.14 (*) ---->> We can detect this!
>     └── rust-rand-core v0.5.1 (*)
> [native-inputs]
> └── cc v1.0.50
> --8<---------------cut here---------------end--------------->8---

guix graph?

Well, there is no “plain” backend.  Such backend had been discussed in
#43477 [1].

1: https://issues.guix.gnu.org/43477

Cheers,
simon


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Pinned/fixed versions should be a requirement.
  2023-09-05  2:59 Pinned/fixed versions should be a requirement Distopico
  2023-09-05 17:15 ` wolf
  2023-09-07 12:39 ` Pinned " Simon Tournier
@ 2023-09-09 22:50 ` Attila Lendvai
  2023-09-09 23:30 ` Liliana Marie Prikler
  2023-09-27  7:51 ` Nguyễn Gia Phong via Development of GNU Guix and the GNU System distribution.
  4 siblings, 0 replies; 10+ messages in thread
From: Attila Lendvai @ 2023-09-09 22:50 UTC (permalink / raw)
  To: Distopico; +Cc: guix-devel

> For these reasons, I believe that pinned versions should be a
> requirement in libraries, always specifying the exact dependency, for
> example, `rust-serde-json-1.0.98`.


aiming a little higher, we could stop using module-global variables for pointing to packages (aka define-public), and with that eliminate one of the two, orthogonal and inconsistent (https://issues.guix.gnu.org/61292) package repositories, and only use the one-and-only reified package registry.

then, with some more work there could be:

 - a syntax extension to easily point to a package from scheme code;
   something like #^rust-serde-json@1.0.98

 - a "package GC" algorithm that collects and prints
   orphaned/unreferenced packages

 - "semantic" version pointers for situations where multiple versions
   of the same package cannot coexist in one profile, and/or dependent
   packages want to point to e.g. the latest, or the latest in a
   specific major version, or the "picked as current by the
   surrounding environment" (thinking here of things like the web of
   Gnome related packages that must all be compatible with each other,
   and only one instance is allowed to run at once).

this way we could be "rolling ahead" with the package definitions similarly to how derivations in the store are "rolling ahead" and unused ones get GC'd.

it would increase complexity in the sense that contributors would need to constantly keep an eye on moving forward the dependency pointers (unless "latest" or some other semantic reference is used as version for an input).

but in return it would increase resilience: a random update of a package would not break anything else that is downstream on the dependency chain. and i'm not only thinking of build errors here, but also subtle changes at runtime (e.g. how Chromium stopped being able to open the file open/save dialog while xdg-desktop-portal-gtk and xdg-desktop-portal-wlr were installed. their upgade broke Chromium, and it took me quite some time/annoyance to debug this).

i think this would also enable interesting features like being able to have two versions of Gnome alive and runnable in the same profile.

this would also open the path for something else that may or may not be worth it: the files holding the package definitions could be divorced from being full scheme module files, and turned into a text-based package database where algorithms (e.g. package import, GC) could reliably add and delete entries without human intervention (besides recording it as a git commit).

it's a whole other story that we shouldn't store source code as a string of characters, but instead admit that it's a graph, and store/edit it as such. that would implicitly resolve most of this problem, too, but i'm afraid we're not ready for that just yet.

--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“When you change the way you look at things, the things you look at change.”
	— Wayne W. Dyer (1940–2015)



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Pinned/fixed versions should be a requirement.
  2023-09-05  2:59 Pinned/fixed versions should be a requirement Distopico
                   ` (2 preceding siblings ...)
  2023-09-09 22:50 ` Pinned/fixed " Attila Lendvai
@ 2023-09-09 23:30 ` Liliana Marie Prikler
  2023-09-10  1:37   ` Distopico
  2023-09-27  7:51 ` Nguyễn Gia Phong via Development of GNU Guix and the GNU System distribution.
  4 siblings, 1 reply; 10+ messages in thread
From: Liliana Marie Prikler @ 2023-09-09 23:30 UTC (permalink / raw)
  To: Distopico, guix-devel

In this thread: Rust has a broken packaging model, so let's apply that.

Am Montag, dem 04.09.2023 um 21:59 -0500 schrieb Distopico:
> Many libraries in different languages don't follow semver, which can
> lead to cases like `rust-serde-json`, which, between versions
> "1.0.97" and "1.0.98," changed its dependency from `indexmap` "1.x"
> to "2.x," causing several packages like rust-analyzer to break. I've
> also observed this in Haskell with packages like "text."
The thing here is, that cargo itself also relies on semantic
versioning.  In fact, I am befuzzled as to why a dependency on
"indexmap" should affect serde-json's public API and probably so where
the serde folks.  Then again, coming from the GNOME world, libsoup3
wasn't really a silent bomb either.

Btw. note to everyone reading this thread, if you ever consider
updating serde: skip versions [1.0.172, 1.0.185).  Thanks :)

> This is problematic because:
> 
>     - Over time, it becomes more vulnerable to libraries/packages
>       breaking.
> 
>     - It makes reproducible software more challenging, as "1.x" can
>       encompass many versions.
> 
>     - Debugging becomes difficult since that package could be a deep
>       dependency in the system package dependency chain, such as
>       Rust/Haskell/NPM, etc.
> 
>     - It makes it more likely that if a dependency changes, many
>       packages will need to be updated/rebuilt due to that change.
> 
> For these reasons, I believe that pinned versions should be a
> requirement in libraries, always specifying the exact dependency, for
> example, `rust-serde-json-1.0.98`.
This goes contrary to even rust's development model that only forces
lock files onto applications and not libraries.  Now, you make a good
point in that pinned versions save us some trouble, but they can also
trouble on their own.  Rust dependencies are basically glorified
propagated-inputs, but with none of the `guix graph' support, so
they're both incredibly hard to detect with our current tooling *and*
they allow for two pinned versions X and Y to cause a potential
conflict.  Indeed a recipe for fun times :)

I think we need to actually capture these links so that we can more
easily detect potentially critical changes to the rust ecosystem and
stick to our tried and tested recipe of "only touch these ones on
feature branches, mkay?".  Do you know what goes into serde?  I know I
don't.  On that note, does anyone have an ETA for antioxidant?

Cheers

PS: Also consider that software written in Rust may contain bugs that
we need to patch out.  Upgrading a package that adheres to SemVer as it
ought to according to Rust standards is already non-trivial enough. 
Now try that along with writing a sed script to replace it in every
input.  Quickly gets very annoying.


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Pinned/fixed versions should be a requirement.
  2023-09-09 23:30 ` Liliana Marie Prikler
@ 2023-09-10  1:37   ` Distopico
  2023-09-10  5:51     ` Liliana Marie Prikler
  0 siblings, 1 reply; 10+ messages in thread
From: Distopico @ 2023-09-10  1:37 UTC (permalink / raw)
  To: Liliana Marie Prikler; +Cc: guix-devel


On 2023-09-10, Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:

Hi Liliana,

>> This is problematic because:
>> 
>>     - Over time, it becomes more vulnerable to libraries/packages
>>       breaking.
>> 
>>     - It makes reproducible software more challenging, as "1.x" can
>>       encompass many versions.
>> 
>>     - Debugging becomes difficult since that package could be a deep
>>       dependency in the system package dependency chain, such as
>>       Rust/Haskell/NPM, etc.
>> 
>>     - It makes it more likely that if a dependency changes, many
>>       packages will need to be updated/rebuilt due to that change.
>> 
>> For these reasons, I believe that pinned versions should be a
>> requirement in libraries, always specifying the exact dependency, for
>> example, `rust-serde-json-1.0.98`.
> This goes contrary to even rust's development model that only forces
> lock files onto applications and not libraries.  Now, you make a good
> point in that pinned versions save us some trouble, but they can also
> trouble on their own.  Rust dependencies are basically glorified
> propagated-inputs, but with none of the `guix graph' support, so
> they're both incredibly hard to detect with our current tooling *and*
> they allow for two pinned versions X and Y to cause a potential
> conflict.  Indeed a recipe for fun times :)
>
> I think we need to actually capture these links so that we can more
> easily detect potentially critical changes to the rust ecosystem and
> stick to our tried and tested recipe of "only touch these ones on
> feature branches, mkay?".  Do you know what goes into serde?  I know I
> don't.  On that note, does anyone have an ETA for antioxidant?
>
> Cheers
>
> PS: Also consider that software written in Rust may contain bugs that
> we need to patch out.  Upgrading a package that adheres to SemVer as it
> ought to according to Rust standards is already non-trivial enough. 
> Now try that along with writing a sed script to replace it in every
> input.  Quickly gets very annoying.

Beyond Rust, an example of a language/packages ecosystem that does not
follow semantic versioning at all is JavaScript/Npm. Most packages in
node-xyz[1] do not reference a version; they simply use the global
input. For now, the number of npm/node packages is small, but with time,
that could become a problem.

Footnotes:
[1]  https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/node-xyz.scm#n193



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Pinned/fixed versions should be a requirement.
  2023-09-10  1:37   ` Distopico
@ 2023-09-10  5:51     ` Liliana Marie Prikler
  0 siblings, 0 replies; 10+ messages in thread
From: Liliana Marie Prikler @ 2023-09-10  5:51 UTC (permalink / raw)
  To: Distopico; +Cc: guix-devel

Am Samstag, dem 09.09.2023 um 20:37 -0500 schrieb Distopico:
> 
> On 2023-09-10, Liliana Marie Prikler <liliana.prikler@gmail.com>
> wrote:
> > 
> > [bunch of stuff regarding Rust]
> 
> 
> Beyond Rust, an example of a language/packages ecosystem that does
> not follow semantic versioning at all is JavaScript/Npm. Most
> packages in node-xyz[1] do not reference a version; they simply use
> the global input. For now, the number of npm/node packages is small,
> but with time, that could become a problem.
I don't think that pinning versions is going to help us much in the npm
case.  The node packaging model allows inputs to be propagated beyond
reason.  And the response of those gazing into the abyss for any longer
than two seconds is quite often "fuck it, I'm out" [1].  I don't think
we'll get far in packaging node stuff without solving the more
fundamental issue of "how can I replace this JS file with this JS file
and not have my entire tower of dependencies break down?" and that is a
social issue as much as it is a technical one.

Cheers

[1] https://dustycloud.org/blog/javascript-packaging-dystopia/


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: Pinned/fixed versions should be a requirement
  2023-09-05  2:59 Pinned/fixed versions should be a requirement Distopico
                   ` (3 preceding siblings ...)
  2023-09-09 23:30 ` Liliana Marie Prikler
@ 2023-09-27  7:51 ` Nguyễn Gia Phong via Development of GNU Guix and the GNU System distribution.
  4 siblings, 0 replies; 10+ messages in thread
From: Nguyễn Gia Phong via Development of GNU Guix and the GNU System distribution. @ 2023-09-27  7:51 UTC (permalink / raw)
  To: Distopico, guix-devel

[-- Attachment #1: Type: text/plain, Size: 2843 bytes --]

The dependency graph visualization has been discussed
by others more knowledgable of the Guix ecosystem than me,
so I'll focus on titled topic.

On 2023-09-04 at 21:59-05:00, Distopico wrote:
> `rust-my-lib-1`, where "1" refers to the semver "1.x" of the package,
> e.g., "1.0.32", and `rust-foo` depends on `rust-my-lib == 1.0.32`.
> However, in some other package got updated to "1.0.34" so `rust-foo`
> will break.
>
> [...] It makes it more likely that if a dependency changes,
> many packages will need to be updated/rebuilt due to that change.

This is not a bug but a feature, so that a fix can be rolled out
simutaneously to many if not all programs.  Recall the OpenSSL
incidents, and more recently, libwebp.

IMHO as software integrator, distribution maintainers
should provide feedback to upstream to relax version pinning,
preferably with patches.  I know first-hand this is not easy,
but otherwise we might as well just tell users to directly
invoke e.g. Cargo, since packaging without system-wide
integration is just extra hassle without any of the security benefits.

In the alternative world where dependencies are pinned,
distros can also patch each vulnerable version, but delaying
efforts until emergency is hardly wise.

On 2023-09-04 at 21:59-05:00, Distopico wrote:
> Over time, it becomes more vulnerable to libraries/packages breaking.

I'd like to add that this is not wall clock time, but labor time.
This would not be an issue if Guix has enough contributors
(and maintainers to actually apply to patches) to perform integration,
as discussed in the parallel thread.

On 2023-09-04 at 21:59-05:00, Distopico wrote:
> It makes reproducible software more challenging,
> as "1.x" can encompass many versions.

FMIIW, but the Guix way of reproducibility expects
both channel version and derivation name[, version].
With free software philosophy in mind, reproducibility
is not so beneficial without the ease of modification,
which in this context includes changing dependency[ version].

I'm speaking as someone primarily using Nix and seek Guix
for correctness, e.g. not making it impossible to patch
a Go library system-wide.

On 2023-09-04 at 21:59-05:00, Distopico wrote:
> For these reasons, I believe that pinned versions should be a
> requirement in libraries, always specifying the exact dependency, for
> example, `rust-serde-json-1.0.98`.
>
> This brings the following benefits:
>
> - Fewer packages will be prone to rebuilding
>   when changing the definition of a library.
> - Reduced likelihood of libraries/packages breaking.
> - Easier maintenance of packages and libraries
>   without fear of breaking others or having to update many.

I'd like to point out these exact benefits could be achieved
by distributing binary built by upstream.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 248 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2023-09-27  7:51 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-09-05  2:59 Pinned/fixed versions should be a requirement Distopico
2023-09-05 17:15 ` wolf
2023-09-07 12:39 ` Pinned " Simon Tournier
2023-09-07 15:35   ` Distopico
2023-09-09 10:39     ` Simon Tournier
2023-09-09 22:50 ` Pinned/fixed " Attila Lendvai
2023-09-09 23:30 ` Liliana Marie Prikler
2023-09-10  1:37   ` Distopico
2023-09-10  5:51     ` Liliana Marie Prikler
2023-09-27  7:51 ` Nguyễn Gia Phong via Development of GNU Guix and the GNU System distribution.

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).