Magic Wormhole Package Weirdness/Potential Security Issues?

Magic Wormhole Package Weirdness/Potential Security Issues?

[Juliana Sims]

Hey folks,

I tried to update magic-wormhole today and things went super smoothly.  
All I had to do was change the version number.

I didn't even have to change the source hash.

If that strikes you as odd, good!  It should!

To cover all my bases, I pk'd the hash produced by `pypi-uri` and used 
`guix download` to try to fetch the same file and check its hash, only 
to find that `guix download` couldn't find anything at that URL or its 
fallbacks.

To test if things were being exceptionally weird, I switched to pulling 
and building from git, and the build failed, expectedly, probably 
because one of the dependencies (magic-wormhole-transit-relay) was not 
the right version, which was what I had initially expected to happen.

Does anyone know what might be going on here?  Given the intended 
secure nature of this program, I'm concerned there may be something 
malicious happening somewhere along the way.  I would love an 
explanation that quiets that concern.

You can look at the current magic-wormhole package source and play 
around with it yourself to see what I'm talking about.

Best,
Juli

PS I was trying to update all three packages in magic-wormhole.scm, but 
the transit relay in particular requires later versions of twisted and 
autobahn than the other two, which is minorly annoying.  I know twisted 
can't be updated without rebuilding a bunch of stuff, so I don't plan 
to pursue this further for the time being.

Re: Magic Wormhole Package Weirdness/Potential Security Issues?

[Ian Eure]

Hi Juliana,

I’ve observed some similar weirdness in the past when I’ve updated 
versions.  I believe what’s happening is that Guix uses the hash 
to look up the file in a content-addressed store (either the local 
store or SWH), and is lacking verification that the retrieved 
object is the expected one.

  — Ian

Juliana Sims <juli@incana.org> writes:

> Hey folks,
>
> I tried to update magic-wormhole today and things went super 
> smoothly.
> All I had to do was change the version number.
>
> I didn't even have to change the source hash.
>
> If that strikes you as odd, good!  It should!
>
> To cover all my bases, I pk'd the hash produced by `pypi-uri` 
> and used
> `guix download` to try to fetch the same file and check its 
> hash, only
> to find that `guix download` couldn't find anything at that URL 
> or its
> fallbacks.
>
> To test if things were being exceptionally weird, I switched to
> pulling and building from git, and the build failed, expectedly,
> probably because one of the dependencies
> (magic-wormhole-transit-relay) was not the right version, which 
> was
> what I had initially expected to happen.
>
> Does anyone know what might be going on here?  Given the 
> intended
> secure nature of this program, I'm concerned there may be 
> something
> malicious happening somewhere along the way.  I would love an
> explanation that quiets that concern.
>
> You can look at the current magic-wormhole package source and 
> play
> around with it yourself to see what I'm talking about.
>
> Best,
> Juli
>
> PS I was trying to update all three packages in 
> magic-wormhole.scm,
> but the transit relay in particular requires later versions of 
> twisted
> and autobahn than the other two, which is minorly annoying.  I 
> know
> twisted can't be updated without rebuilding a bunch of stuff, so 
> I
> don't plan to pursue this further for the time being.

Re: Magic Wormhole Package Weirdness/Potential Security Issues?

[Troy Figiel]

[-- Attachment #1: Type: text/plain, Size: 1049 bytes --]

Hi Juli,

On Fri, 2024-11-08 at 13:26 -0500, Juliana Sims wrote:
> To cover all my bases, I pk'd the hash produced by `pypi-uri` and
> used 
> `guix download` to try to fetch the same file and check its hash,
> only 
> to find that `guix download` couldn't find anything at that URL or
> its 
> fallbacks.

It seems at some point in between version 0.14.0 and 0.17.0 the name of
the tarball has changed from `magic-wormhole` to `magic_wormhole`.  You
have to change the uri-field accordingly to successfully download the
source code from PyPI.

When building it in the way you describe, the source code cannot be
found on PyPI, so it is pulled from tarballs.nixos.org instead.  It
seems NixOS uses content-addressable storage, so the hash is used to
download the source code and since you have not changed the hash, it
downloads version 0.14.0 again.

Why tarballs.nixos.org is used as a backup, I do not know.  I do not
recall ever having seen this behaviour before.

Hope this helps a bit though!

Best wishes,

Troy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 862 bytes --]

Re: Magic Wormhole Package Weirdness/Potential Security Issues?

[Julien Lepiller]

If you only change the version number, guix will cry to download that version. If it fails, then it relies on the provided hash to fetch from a content-adressed store, such as ci or sohtware heritage. It relies on the hash as the source of truth.

Usually, when updating a package, I alter the hash to make sure it doesn't build the old version. Replace the initial 1 with a 0 or initial 0 with a 1, and you get a wrong hash. If it fails to download the file, you'll notice immediately. Otherwise, it will print the expected hash.

Le 8 novembre 2024 20:18:54 GMT+01:00, Troy Figiel <troy@troyfigiel.com> a écrit :
>Hi Juli,
>
>On Fri, 2024-11-08 at 13:26 -0500, Juliana Sims wrote:
>> To cover all my bases, I pk'd the hash produced by `pypi-uri` and
>> used 
>> `guix download` to try to fetch the same file and check its hash,
>> only 
>> to find that `guix download` couldn't find anything at that URL or
>> its 
>> fallbacks.
>
>It seems at some point in between version 0.14.0 and 0.17.0 the name of
>the tarball has changed from `magic-wormhole` to `magic_wormhole`.  You
>have to change the uri-field accordingly to successfully download the
>source code from PyPI.
>
>When building it in the way you describe, the source code cannot be
>found on PyPI, so it is pulled from tarballs.nixos.org instead.  It
>seems NixOS uses content-addressable storage, so the hash is used to
>download the source code and since you have not changed the hash, it
>downloads version 0.14.0 again.
>
>Why tarballs.nixos.org is used as a backup, I do not know.  I do not
>recall ever having seen this behaviour before.
>
>Hope this helps a bit though!
>
>Best wishes,
>
>Troy

Re: Magic Wormhole Package Weirdness/Potential Security Issues?

[Leo Famulari]

On Fri, Nov 08, 2024 at 08:18:54PM +0100, Troy Figiel wrote:
> Why tarballs.nixos.org is used as a backup, I do not know.  I do not
> recall ever having seen this behaviour before.

The Nix cache has been used since 2016 to mitigate the problem of
package sources disappearing from the net:

https://git.savannah.gnu.org/cgit/guix.git/commit/?id=cd436bf05a8344acf4462f3602e7d360821a902a