[bug#48259] [PATCH 1/2] gnu: Replace go-github-com-golang-protobuf-proto with go-github-com-golang-protobuf

[bug#48259] [PATCH 1/2] gnu: Replace go-github-com-golang-protobuf-proto with go-github-com-golang-protobuf

[Dion Mendel]

[-- Attachment #1: Type: text/plain, Size: 855 bytes --]

Hi guix,

The github.com/golang/protobuf module contains several packages.
Only proto is currently packaged.  This change updates the module from
1.3.1 to 1.5.2 and packages the entire module.

This is needed as I am preparing to package lxd (github.com/lxc/lxd), 
which depends on the unpackaged packages from golang/protobuf.

This change replaces go-github-com-golang-protobuf-proto with 
go-github-com-golang-protobuf.

There are currently two packages that depend on 
go-github-com-golang-protobuf-proto.  I have tested this changeset by 
building both of these packages.

go-github-com-lucas-clemente-quic-go: this is a source only package that 
no other package uses.

syncthing: depends on go-github-com-golang-protobuf-proto via several 
intermediary packages.  After building, I've run some of the syncthing
executables and they appear to work.

[-- Attachment #2: 0001-gnu-Add-go-golang-org-protobuf.patch --]
[-- Type: text/x-diff, Size: 2223 bytes --]

From 47de2a85a1ae1f0502a89d0097d3b36da7d20649 Mon Sep 17 00:00:00 2001
From: Dion Mendel <guix@dm9.info>
Date: Thu, 6 May 2021 23:25:22 +0800
Subject: [PATCH 1/2] gnu: Add go-golang-org-protobuf.

* gnu/packages/golang.scm (go-golang-org-protobuf): New variable.
---
 gnu/packages/golang.scm | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
index 8c3c81ceda..f0fa0e1d36 100644
--- a/gnu/packages/golang.scm
+++ b/gnu/packages/golang.scm
@@ -25,6 +25,7 @@
 ;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;; Copyright © 2021 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2021 Guillaume Le Vaillant <glv@posteo.net>
+;;; Copyright © 2021 Dion Mendel <guix@dm9.info>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -4832,6 +4833,35 @@ data serialization format.")
     (home-page "https://github.com/golang/protobuf")
     (license license:bsd-3)))
 
+(define-public go-golang-org-protobuf
+  (package
+    (name "go-golang-org-protobuf")
+    (version "1.26.0")
+    (home-page "https://github.com/protocolbuffers/protobuf-go")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url home-page)
+                    (commit (string-append "v" version))))
+              (sha256
+               (base32
+                "0xq6phaps6d0vcv13ga59gzj4306l0ki9kikhmb52h6pq0iwfqlz"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "google.golang.org/protobuf"
+       ; Source-only package
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (delete 'build))))
+    (propagated-inputs
+     `(("go-github-com-google-go-cmp-cmp" ,go-github-com-google-go-cmp-cmp)))
+    (synopsis "Go support for Protocol Buffers")
+    (description "This project hosts the Go implementation for protocol buffers, which is a
+language-neutral, platform-neutral, extensible mechanism for serializing
+structured data.")
+    (license license:bsd-3)))
+
 (define-public go-github-com-mattn-go-zglob
   (package
     (name "go-github-com-mattn-go-zglob")
-- 
2.31.1


[-- Attachment #3: 0002-gnu-Replace-go-github-com-golang-protobuf-proto-with.patch --]
[-- Type: text/x-diff, Size: 6298 bytes --]

From 0f029698f12364a475211204498aeb3face1381e Mon Sep 17 00:00:00 2001
From: Dion Mendel <guix@dm9.info>
Date: Thu, 6 May 2021 23:39:16 +0800
Subject: [PATCH 2/2] gnu: Replace go-github-com-golang-protobuf-proto with
 go-github-com-golang-protobuf

The github.com/golang/protobuf module contains several packages.
Only proto is currently packaged.  This change updates the module from
1.3.1 to 1.5.2 and packages the entire module.

* gnu/packages/golang.scm
(go-github-com-golang-protobuf): New variable.
(go-github-com-golang-protobuf-proto): Removed variable.
(go-github-com-lucas-clemente-quic-go)[propagated-inputs]:
Replace go-github-com-golang-protobuf-proto@1.3.1
with go-github-com-golang-protobuf@1.5.2.
* gnu/packages/syncthing.scm
(go-github-com-matttproud-golang-protobuf-extensions-pbutil)
(go-github-com-prometheus-client-golang)
(go-github-com-prometheus-client-model)
(go-github-com-prometheus-common)
[propagated-inputs]: Replace go-github-com-golang-protobuf-proto@1.3.1
with go-github-com-golang-protobuf@1.5.2.
---
 gnu/packages/golang.scm    | 32 ++++++++++++++++++++++----------
 gnu/packages/syncthing.scm | 16 ++++++++--------
 2 files changed, 30 insertions(+), 18 deletions(-)

diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
index f0fa0e1d36..9a689d225b 100644
--- a/gnu/packages/golang.scm
+++ b/gnu/packages/golang.scm
@@ -4237,7 +4237,7 @@ implementation of generics.")
        ("go-github-com-cheekybits-genny" ,go-github-com-cheekybits-genny)
        ("go-github-com-marten-seemann-chacha20" ,go-github-com-marten-seemann-chacha20)
        ("go-github-com-marten-seemann-qtls" ,go-github-com-marten-seemann-qtls)
-       ("go-github-com-golang-protobuf-proto" ,go-github-com-golang-protobuf-proto)))
+       ("go-github-com-golang-protobuf" ,go-github-com-golang-protobuf)))
     (synopsis "QUIC in Go")
     (description "This package provides a Go language implementation of the QUIC
 network protocol.")
@@ -4808,10 +4808,10 @@ test results.")
     (home-page "https://github.com/gotestyourself/gotestsum")
     (license license:asl2.0)))
 
-(define-public go-github-com-golang-protobuf-proto
+(define-public go-github-com-golang-protobuf
   (package
-    (name "go-github-com-golang-protobuf-proto")
-    (version "1.3.1")
+    (name "go-github-com-golang-protobuf")
+    (version "1.5.2")
     (source (origin
               (method git-fetch)
               (uri (git-reference
@@ -4820,16 +4820,28 @@ test results.")
               (file-name (git-file-name name version))
               (sha256
                (base32
-                "15am4s4646qy6iv0g3kkqq52rzykqjhm4bf08dk0fy2r58knpsyl"))))
+                "1mh5fyim42dn821nsd3afnmgscrzzhn3h8rag635d2jnr23r1zhk"))))
     (build-system go-build-system)
     (arguments
-     '(#:import-path "github.com/golang/protobuf/proto"
-       #:unpack-path "github.com/golang/protobuf"
-       ;; Requires unpackaged golang.org/x/sync/errgroup
-       #:tests? #f))
+     '(#:import-path "github.com/golang/protobuf"
+       ; Source-only package
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (delete 'build))))
+    (propagated-inputs
+     `(("go-golang-org-protobuf" ,go-golang-org-protobuf)))
     (synopsis "Go support for Protocol Buffers")
     (description "This package provides Go support for the Protocol Buffers
-data serialization format.")
+data serialization format.
+
+It has been superseded by go-golang-org-protobuf, which contains an updated and
+simplified API, support for protobuf reflection, and many other improvements.
+We recommend that new code use the google.golang.org/protobuf module.
+
+Versions v1.4 and later of go-github-com-golang-protobuf are implemented in
+terms of go-golang-org-protobuf.  Programs which use both modules must use
+at least version v1.4 of this one.")
     (home-page "https://github.com/golang/protobuf")
     (license license:bsd-3)))
 
diff --git a/gnu/packages/syncthing.scm b/gnu/packages/syncthing.scm
index e40f696720..67088ca6ca 100644
--- a/gnu/packages/syncthing.scm
+++ b/gnu/packages/syncthing.scm
@@ -838,8 +838,8 @@ bounds.")
            ;; Source-only package
            (delete 'build))))
       (propagated-inputs
-       `(("go-github-com-golang-protobuf-proto"
-          ,go-github-com-golang-protobuf-proto)))
+       `(("go-github-com-golang-protobuf"
+          ,go-github-com-golang-protobuf)))
       (synopsis "Data model artifacts for Prometheus")
       (description "This package provides data model artifacts for Prometheus.")
       (home-page "https://github.com/prometheus/client_model")
@@ -867,8 +867,8 @@ bounds.")
        '(#:import-path "github.com/matttproud/golang_protobuf_extensions/pbutil"
          #:unpack-path "github.com/matttproud/golang_protobuf_extensions"))
       (propagated-inputs
-       `(("go-github-com-golang-protobuf-proto"
-          ,go-github-com-golang-protobuf-proto)))
+       `(("go-github-com-golang-protobuf"
+          ,go-github-com-golang-protobuf)))
       (synopsis "Streaming Protocol Buffers in Go")
       (description "This package provides various Protocol Buffer
 extensions for the Go language, namely support for record length-delimited
@@ -907,8 +907,8 @@ message streaming.")
            ;; Source-only package
            (delete 'build))))
       (propagated-inputs
-       `(("go-github-com-golang-protobuf-proto"
-          ,go-github-com-golang-protobuf-proto)
+       `(("go-github-com-golang-protobuf"
+          ,go-github-com-golang-protobuf)
          ("go-github-com-matttproud-golang-protobuf-extensions-pbutil"
           ,go-github-com-matttproud-golang-protobuf-extensions-pbutil)
          ("go-github-com-prometheus-client-model"
@@ -968,8 +968,8 @@ system, kernel, and process metrics from the @file{/proc} pseudo file system.")
       (propagated-inputs
        `(("go-github-com-beorn7-perks-quantile"
           ,go-github-com-beorn7-perks-quantile)
-         ("go-github-com-golang-protobuf-proto"
-          ,go-github-com-golang-protobuf-proto)
+         ("go-github-com-golang-protobuf"
+          ,go-github-com-golang-protobuf)
          ("go-github-com-prometheus-client-model"
           ,go-github-com-prometheus-client-model)
          ("go-github-com-prometheus-common"
-- 
2.31.1

[bug#48259] [PATCH 1/2] gnu: Replace go-github-com-golang-protobuf-proto with go-github-com-golang-protobuf

[Leo Famulari]

On Fri, May 07, 2021 at 12:35:11AM +0800, Dion Mendel wrote:
> syncthing: depends on go-github-com-golang-protobuf-proto via several
> intermediary packages.  After building, I've run some of the syncthing
> executables and they appear to work.

Just to clarify, since commit ed3ef756f521a0df8596a88b66f65b7a1ad99252,
Syncthing does not depend on any Go packages, but instead uses its
bundled copies of all its dependencies.

https://git.savannah.gnu.org/cgit/guix.git/commit/?id=ed3ef756f521a0df8596a88b66f65b7a1ad99252

[bug#48259] [PATCH 1/2] gnu: Replace go-github-com-golang-protobuf-proto with go-github-com-golang-protobuf

[Dion Mendel]

On Sat, May 08, 2021 at 11:50:42AM -0400, Leo Famulari wrote:
>
>Just to clarify, since commit ed3ef756f521a0df8596a88b66f65b7a1ad99252,
>Syncthing does not depend on any Go packages, but instead uses its
>bundled copies of all its dependencies.

Is it acceptable to use bundled copies of dependencies?  The manual 
(section 16.6, checklist #6) says otherwise.

If it's okay, this would make packaging lxd much simpler.  I've had to 
package around 90 Go libraries so far.

If using lxd's bundled copies of dependencies would be accepted, then I 
withdraw this patch request.

[bug#48259] [PATCH 1/2] gnu: Replace go-github-com-golang-protobuf-proto with go-github-com-golang-protobuf

[Leo Famulari]

On Sun, May 09, 2021 at 08:37:00PM +0800, Dion Mendel wrote:
> Is it acceptable to use bundled copies of dependencies?  The manual (section
> 16.6, checklist #6) says otherwise.

This item, from the manual section Submitting Patches, says: 

"Make sure the package does not use bundled copies of software already
available as separate packages."

So, already, there is some wiggle room based on whether or not things
are "already available as separate packages."

Using bundled dependnencies is strongly discouraged and you are expected
to try to avoid it.

With Syncthing, there were unfixed security vulnerabilities in our
package, and our go-build-system could not build later versions of
Syncthing. So, until our go-build-system is overhauled [0], we'll use
Syncthing's bundled dependencies. They are all free software anyways.

With Go, the old distro model of "no bundling" is not idiomatic, so the
future of Go in Guix may look different from how it is now. Go software
typically has no concept of "versions" and every program pins its
dependencies at specific Git commits. So, to unbundle everything would
leave us with hundreds or thousands of versions of popular libraries.
We'll have to be creative in how we deal with that.

But for now, we try to unbundle.

> If it's okay, this would make packaging lxd much simpler.  I've had to
> package around 90 Go libraries so far.
> 
> If using lxd's bundled copies of dependencies would be accepted, then I
> withdraw this patch request.

Please, either try packaging them "by hand" or with the Go module
importer [1]. Code reviewers will ask you to explain why you are using the
bundled dependencies.

[0] The work to overhaul go-build-system is progressing.

[1] https://guix.gnu.org/manual/devel/en/html_node/Invoking-guix-import.html#Invoking-guix-import
I guess it would be a command like:
`guix import go github.com/lcx/lxd --recursive -pin-versions`
But, it depends on LXD using the standard Go module mechanisms. If it
doesn't, maybe each top-level dependency can be imported along with its
dependency graph.

[bug#48259] [PATCH 1/2] gnu: Replace go-github-com-golang-protobuf-proto with go-github-com-golang-protobuf

[Dion Mendel]

On Sun, May 09, 2021 at 11:17:27AM -0400, Leo Famulari wrote:
>But for now, we try to unbundle.

Okay, thank you for clarification.

When this changeset passes review and is applied, I'll continue with 
packaging lxd and its dependencies.

(Thanks for pointing me towards guix import).