[bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies.

[bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies.

[Denis 'GNUtoo' Carikli]

The vendor/golang.org/x/net dependency was kept because removing it redults in
a compilation failure:
    %exception #<&invoke-error program: "go" arguments: ("install" "-v" "-x"
     "-ldflags=-s -w" "-trimpath" "github.com/42wim/matterbridge")
     exit-status: 1 term-signal: #f stop-signal: #f>

* gnu/packages/messaging.scm (matterbridge) [source]:
  Add snippet removing the unbundled dependencies.
  [arguments]: Add new unbundled dependencies.

Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
Change-Id: I67afeff8721833d34a54fd52684a4acbcbcaf179
---
 gnu/packages/messaging.scm | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/gnu/packages/messaging.scm b/gnu/packages/messaging.scm
index c482cd469e..86f6ff0e04 100644
--- a/gnu/packages/messaging.scm
+++ b/gnu/packages/messaging.scm
@@ -3302,6 +3302,19 @@ (define-public matterbridge
                     (url "https://github.com/42wim/matterbridge")
                     (commit (string-append "v" version))))
               (file-name (git-file-name name version))
+              (modules '((guix build utils)))
+              (snippet '(for-each delete-file-recursively
+                                  ;; TODO: unbundle the rest as well
+                                  '("vendor/golang.org/x/crypto"
+                                    "vendor/golang.org/x/image"
+                                    "vendor/golang.org/x/mod"
+                                    "vendor/golang.org/x/oauth2"
+                                    "vendor/golang.org/x/sys"
+                                    "vendor/golang.org/x/term"
+                                    "vendor/golang.org/x/text"
+                                    "vendor/golang.org/x/time"
+                                    "vendor/golang.org/x/tools")))
+
               (sha256
                (base32
                 "0939fiy7z53izznfhlr7c6vaskbmkbj3ncb09fzx5dmz9cjngy80"))))
@@ -3309,6 +3322,15 @@ (define-public matterbridge
     (arguments
      (list
       #:import-path "github.com/42wim/matterbridge"))
+    (inputs (list go-golang-org-x-crypto
+                  go-golang-org-x-image
+                  go-golang-org-x-mod
+                  go-golang-org-x-oauth2
+                  go-golang-org-x-sys
+                  go-golang-org-x-term
+                  go-golang-org-x-text
+                  go-golang-org-x-time
+                  go-golang-org-x-tools))
     (synopsis "Bridge together various messaging networks and protocols")
     (description
      "Relays messages between different channels from various
-- 
2.46.0

[bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies.

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 1235 bytes --]


Hi,

Thanks for the initiation of unbundeling process for materbridge!

A general comments for Golang packages which is just final binary, you
may apply "#:install-source? #f" parameter which mean do not place
source of the project into store and just move produced binary.

Also for the final binary all build inputs may be placed to
"native-inputs" they will never be used elsewhere after the install
phase.

While you are keen to unbundle let's pick up as many as we can.  You may
also add some other inputs which are available already:

- filippo.io/edwards25519
- go.uber.org/multierr
- gopkg.in/ini.v1
- gopkg.in/natefinch/lumberjack.v2
- gopkg.in/natefinch/lumberjack.v2
- gopkg.in/yaml.v2
- gopkg.in/yaml.v3
- google.golang.org/protobuf
- github.com/hashicorp ;; all of them are available
- github.com/golang/protobuf
- github.com/fatih/color
- github.com/google/uuid
- github.com/gorilla/websocket
- github.com/pkg/errors

Basically we targeting to have just the first level of dependentices in
"native-inputs" as seen in
<https://github.com/42wim/matterbridge/blob/master/go.mod>.

Please take a look if removing them from vendor and adding them to
native-inputs still keeps the build green.

Thanks for your time.

--
Oleg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

[bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies.

[Denis 'GNUtoo' Carikli]

[-- Attachment #1: Type: text/plain, Size: 3226 bytes --]

On Sun, 27 Oct 2024 20:26:03 +0000
Sharlatan Hellseher <sharlatanus@gmail.com> wrote:

> Thanks for the initiation of unbundeling process for materbridge!
Thanks a lot for the review.

> A general comments for Golang packages which is just final binary, you
> may apply "#:install-source? #f" parameter which mean do not place
> source of the project into store and just move produced binary.
Thanks, I'll do that in a newer patch.

> Also for the final binary all build inputs may be placed to
> "native-inputs" they will never be used elsewhere after the install
> phase.
Right, I just copied a bit what was done in the example I followed, so
that might need to be fixed as well.

Using inputs didn't look right to me but it was done in this way so I
assumed that there was a good reason for it (I tried unbundling long
time ago and it didn't work at the time, but maybe I was just unlucky).

> While you are keen to unbundle let's pick up as many as we can.  You
> may also add some other inputs which are available already:
I was already unsure about the patch I sent. In the v2 should I do it
all at once in a single patch or should I split the work across multiple
patches?

The advantage of the former is that it becomes easier to bisect issues.

The downside is that if the granularity is too small, I fear that we'll
end up with hundreds of patches. 

So I also think it's a good idea to agree on how to make these changes
before doing more and more patches. For instance you could validate one
patch (without necessarily pushing it) so I could use that as a
blueprint for the next ones and not have to redo all the work for so
many patches.

Here I picked something in between where I grouped the unbundling of
dependencies by selecting something they had in common, but I've no
idea if it's the right way to do it.

Also note that I ran the resulting binary, but without feeding it a
configuration. I'll do a more complete test later on as tests are quite
expensive: I need to install Guix on the production server, update it,
build the matterbridge package, and uninstall guix after that to reduce
the attack surface.

> - filippo.io/edwards25519
> - go.uber.org/multierr
> - gopkg.in/ini.v1
> - gopkg.in/natefinch/lumberjack.v2
> - gopkg.in/natefinch/lumberjack.v2
> - gopkg.in/yaml.v2
> - gopkg.in/yaml.v3
> - google.golang.org/protobuf
> - github.com/hashicorp ;; all of them are available
> - github.com/golang/protobuf
> - github.com/fatih/color
> - github.com/google/uuid
> - github.com/gorilla/websocket
> - github.com/pkg/errors
Thanks, I found some of them like the protobuf one but I didn't add
them in my patch yet.

> Basically we targeting to have just the first level of dependentices
> in "native-inputs" as seen in
> <https://github.com/42wim/matterbridge/blob/master/go.mod>.
OK, I didn't look at that yet, I just started looking at what was in
vendor/ with a script and made a patch with the group that is closer to
the language (golang).

> Please take a look if removing them from vendor and adding them to
> native-inputs still keeps the build green.
Thanks for the additional list, I'll definitely do that.

Denis.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies.

[Sharlatan Hellseher]

[-- Attachment #1: Type: text/plain, Size: 1460 bytes --]


Hi Denis,

> I was already unsure about the patch I sent. In the v2 should I do it
> all at once in a single patch or should I split the work across multiple
> patches?

I would suggest to think about the flow, let's say:

* Unbundle as many as possible first, and make it as a single patch
(which should pass build, test, lint etc.)

* Review the massive dependence tree and split it in some sort of
logical blocks/future series which should be self contain and build
individually or when series applied as a whole.

* Start unwind it slowly let's say by 8-12 packages per series, making
sure each series may be apply directly on current master and may be
built.

We may reduce amount of noise to bug-tracker and keep on track with
unbundle process without too much stress and time requiring to review
100++ series

To keep you in a peas of mind - you are not alone in that hard work and
your efforts will be enforced by others who updates/unbunle Golang
packages the more we have the less effort we need to apply to unbunle as
some of the package may be reused.

In short, keep eye on current master and go-team branches and do
rebate/pul often.

I'm waiting for my turne to merge go-team and will continue on the work
with unbundling kubo which will bring about 300 packages.

Some neat-pick give a try to use "guix import go --recursive
<module/name>" on go-team branch which has much better performance than
available in current master.

--
Thanks,
Oleg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]