bug#69617: guix go import fails on some version tags

bug#69617: guix go import fails on some version tags

[Ryan Barber]

In some cases the "guix import go" command fails when attempting to
checkout the source for a module using a tag which does not exist in
the repo.

Upon further investigation, I have found guix/import/go.scm will use
the version string as tag. While this works most of the time, some
module vendors use a different tagging scheme. For example, the
azure-sdk-for-go repository contains many modules and the version tags
are namespaced by module name.

The tag for version v1.3.0 of azure-sdk-for-go/sdk/storage/azblob is
storage/azblob/v1.3.0.

$ curl -s 'https://proxy.golang.org/github.com/!azure/azure-sdk-for-go/sdk/storage/azblob/@v/v1.3.0.info'
| jq
{
  "Version": "v1.3.0",
  "Time": "2024-02-12T16:20:44Z",
  "Origin": {
    "VCS": "git",
    "URL": "https://github.com/Azure/azure-sdk-for-go",
    "Subdir": "sdk/storage/azblob",
    "Ref": "refs/tags/sdk/storage/azblob/v1.3.0",
    "Hash": "d5dfa9296a115cc5094b14198b7114a64a490994"
  }
}

I have a patch to fix this, but I would like to discuss the approach
before submitting it. Should I reply to this bug report with the
patch?

Here is the backtrace when attempting to run import on storage/azblob

$ guix import go github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
Backtrace:
          14 (primitive-load "/home/rfb/.config/guix/current/bin/guix")
In guix/ui.scm:
   2324:7 13 (run-guix . _)
  2287:10 12 (run-guix-command _ . _)
In guix/scripts/import.scm:
     80:6 11 (guix-import . _)
In ice-9/boot-9.scm:
  1752:10 10 (with-exception-handler _ _ #:unwind? _ # _)
In guix/scripts/import/go.scm:
   116:29  9 (_)
In ice-9/exceptions.scm:
   406:15  8 (go-module->guix-package* . _)
In ice-9/boot-9.scm:
  1752:10  7 (with-exception-handler _ _ #:unwind? _ # _)
In guix/import/go.scm:
   532:19  6 (go-module->guix-package "github.com/Azure/azure-sdk-f…" …)
In guix/git.scm:
    295:4  5 (update-cached-checkout _ #:ref _ #:recursive? _ # _ # _ …)
   281:19  4 (resolve _)
In git/reference.scm:
     60:8  3 (_ _ _)
In git/bindings.scm:
     77:2  2 (raise-git-error _)
In ice-9/boot-9.scm:
  1685:16  1 (raise-exception _ #:continuable? _)
  1683:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1683:16: In procedure raise-exception:
Git error: reference 'refs/tags/v1.3.1' not found

bug#69617: guix go import fails on some version tags

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 2981 bytes --]

On 2024-03-07 10:18:52 -0800, Ryan Barber wrote:
> In some cases the "guix import go" command fails when attempting to
> checkout the source for a module using a tag which does not exist in
> the repo.
>
> Upon further investigation, I have found guix/import/go.scm will use
> the version string as tag. While this works most of the time, some
> module vendors use a different tagging scheme. For example, the
> azure-sdk-for-go repository contains many modules and the version tags
> are namespaced by module name.
>
> The tag for version v1.3.0 of azure-sdk-for-go/sdk/storage/azblob is
> storage/azblob/v1.3.0.
>
> $ curl -s 'https://proxy.golang.org/github.com/!azure/azure-sdk-for-go/sdk/storage/azblob/@v/v1.3.0.info'
> | jq
> {
>   "Version": "v1.3.0",
>   "Time": "2024-02-12T16:20:44Z",
>   "Origin": {
>     "VCS": "git",
>     "URL": "https://github.com/Azure/azure-sdk-for-go",
>     "Subdir": "sdk/storage/azblob",
>     "Ref": "refs/tags/sdk/storage/azblob/v1.3.0",
>     "Hash": "d5dfa9296a115cc5094b14198b7114a64a490994"
>   }
> }
>
> I have a patch to fix this, but I would like to discuss the approach
> before submitting it. Should I reply to this bug report with the
> patch?

I wonder, even if you manage to import it, will go-build-system be able to build
it?  That was what stopped me last time I looked into this, I reached the
conclusion that current go-build-system cannot handle multiple go modules with
overlapping paths.

Out of curiosity, what was your approach to this issue?

>
> Here is the backtrace when attempting to run import on storage/azblob
>
> $ guix import go github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
> Backtrace:
>           14 (primitive-load "/home/rfb/.config/guix/current/bin/guix")
> In guix/ui.scm:
>    2324:7 13 (run-guix . _)
>   2287:10 12 (run-guix-command _ . _)
> In guix/scripts/import.scm:
>      80:6 11 (guix-import . _)
> In ice-9/boot-9.scm:
>   1752:10 10 (with-exception-handler _ _ #:unwind? _ # _)
> In guix/scripts/import/go.scm:
>    116:29  9 (_)
> In ice-9/exceptions.scm:
>    406:15  8 (go-module->guix-package* . _)
> In ice-9/boot-9.scm:
>   1752:10  7 (with-exception-handler _ _ #:unwind? _ # _)
> In guix/import/go.scm:
>    532:19  6 (go-module->guix-package "github.com/Azure/azure-sdk-f…" …)
> In guix/git.scm:
>     295:4  5 (update-cached-checkout _ #:ref _ #:recursive? _ # _ # _ …)
>    281:19  4 (resolve _)
> In git/reference.scm:
>      60:8  3 (_ _ _)
> In git/bindings.scm:
>      77:2  2 (raise-git-error _)
> In ice-9/boot-9.scm:
>   1685:16  1 (raise-exception _ #:continuable? _)
>   1683:16  0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1683:16: In procedure raise-exception:
> Git error: reference 'refs/tags/v1.3.1' not found
>

Have a nice day,
Tomas Volf

--
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#69617: guix go import fails on some version tags

[Ryan Barber]

Hi Tomas,

A little background on what I'm trying to accomplish: I have been
trying to create a package for github.com/kopia/kopia@latest. Kopia is
a multi-cloud encrypted backup thing and with that it has a sprawling
list of dependencies to handle different backends. I've been using 'go
import guix' with the -r option to attempt to recursively collect all
the dependencies. That's where I found the exception with
azure-sdk-for-go.

The approach I came up with is to retrieve the version tag from the
proxy, falling back on the prior method of generating the version tag
if it is not provided. This got me past the azure-sdk-for-go issue,
but now I'm encountering modules where the version ref is not
predictable or listed in the proxy.

I haven't encountered the problem of overlapping paths - yet!

As for my goal of building a package for kopia, I will probably hand
assemble the dependencies.

-Ryan


-Ryan

On Fri, Mar 8, 2024 at 7:25 AM Tomas Volf <~@wolfsden.cz> wrote:
>
> On 2024-03-07 10:18:52 -0800, Ryan Barber wrote:
> > In some cases the "guix import go" command fails when attempting to
> > checkout the source for a module using a tag which does not exist in
> > the repo.
> >
> > Upon further investigation, I have found guix/import/go.scm will use
> > the version string as tag. While this works most of the time, some
> > module vendors use a different tagging scheme. For example, the
> > azure-sdk-for-go repository contains many modules and the version tags
> > are namespaced by module name.
> >
> > The tag for version v1.3.0 of azure-sdk-for-go/sdk/storage/azblob is
> > storage/azblob/v1.3.0.
> >
> > $ curl -s 'https://proxy.golang.org/github.com/!azure/azure-sdk-for-go/sdk/storage/azblob/@v/v1.3.0.info'
> > | jq
> > {
> >   "Version": "v1.3.0",
> >   "Time": "2024-02-12T16:20:44Z",
> >   "Origin": {
> >     "VCS": "git",
> >     "URL": "https://github.com/Azure/azure-sdk-for-go",
> >     "Subdir": "sdk/storage/azblob",
> >     "Ref": "refs/tags/sdk/storage/azblob/v1.3.0",
> >     "Hash": "d5dfa9296a115cc5094b14198b7114a64a490994"
> >   }
> > }
> >
> > I have a patch to fix this, but I would like to discuss the approach
> > before submitting it. Should I reply to this bug report with the
> > patch?
>
> I wonder, even if you manage to import it, will go-build-system be able to build
> it?  That was what stopped me last time I looked into this, I reached the
> conclusion that current go-build-system cannot handle multiple go modules with
> overlapping paths.
>
> Out of curiosity, what was your approach to this issue?
>
> >
> > Here is the backtrace when attempting to run import on storage/azblob
> >
> > $ guix import go github.com/Azure/azure-sdk-for-go/sdk/storage/azblob
> > Backtrace:
> >           14 (primitive-load "/home/rfb/.config/guix/current/bin/guix")
> > In guix/ui.scm:
> >    2324:7 13 (run-guix . _)
> >   2287:10 12 (run-guix-command _ . _)
> > In guix/scripts/import.scm:
> >      80:6 11 (guix-import . _)
> > In ice-9/boot-9.scm:
> >   1752:10 10 (with-exception-handler _ _ #:unwind? _ # _)
> > In guix/scripts/import/go.scm:
> >    116:29  9 (_)
> > In ice-9/exceptions.scm:
> >    406:15  8 (go-module->guix-package* . _)
> > In ice-9/boot-9.scm:
> >   1752:10  7 (with-exception-handler _ _ #:unwind? _ # _)
> > In guix/import/go.scm:
> >    532:19  6 (go-module->guix-package "github.com/Azure/azure-sdk-f…" …)
> > In guix/git.scm:
> >     295:4  5 (update-cached-checkout _ #:ref _ #:recursive? _ # _ # _ …)
> >    281:19  4 (resolve _)
> > In git/reference.scm:
> >      60:8  3 (_ _ _)
> > In git/bindings.scm:
> >      77:2  2 (raise-git-error _)
> > In ice-9/boot-9.scm:
> >   1685:16  1 (raise-exception _ #:continuable? _)
> >   1683:16  0 (raise-exception _ #:continuable? _)
> >
> > ice-9/boot-9.scm:1683:16: In procedure raise-exception:
> > Git error: reference 'refs/tags/v1.3.1' not found
> >
>
> Have a nice day,
> Tomas Volf
>
> --
> There are only two hard things in Computer Science:
> cache invalidation, naming things and off-by-one errors.