* [bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies.
@ 2024-10-27 18:12 Denis 'GNUtoo' Carikli
2024-10-27 20:26 ` Sharlatan Hellseher
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Denis 'GNUtoo' Carikli @ 2024-10-27 18:12 UTC (permalink / raw)
To: 74049, sharlatanus; +Cc: Denis 'GNUtoo' Carikli
The vendor/golang.org/x/net dependency was kept because removing it redults in
a compilation failure:
%exception #<&invoke-error program: "go" arguments: ("install" "-v" "-x"
"-ldflags=-s -w" "-trimpath" "github.com/42wim/matterbridge")
exit-status: 1 term-signal: #f stop-signal: #f>
* gnu/packages/messaging.scm (matterbridge) [source]:
Add snippet removing the unbundled dependencies.
[arguments]: Add new unbundled dependencies.
Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
Change-Id: I67afeff8721833d34a54fd52684a4acbcbcaf179
---
gnu/packages/messaging.scm | 22 ++++++++++++++++++++++
1 file changed, 22 insertions(+)
diff --git a/gnu/packages/messaging.scm b/gnu/packages/messaging.scm
index c482cd469e..86f6ff0e04 100644
--- a/gnu/packages/messaging.scm
+++ b/gnu/packages/messaging.scm
@@ -3302,6 +3302,19 @@ (define-public matterbridge
(url "https://github.com/42wim/matterbridge")
(commit (string-append "v" version))))
(file-name (git-file-name name version))
+ (modules '((guix build utils)))
+ (snippet '(for-each delete-file-recursively
+ ;; TODO: unbundle the rest as well
+ '("vendor/golang.org/x/crypto"
+ "vendor/golang.org/x/image"
+ "vendor/golang.org/x/mod"
+ "vendor/golang.org/x/oauth2"
+ "vendor/golang.org/x/sys"
+ "vendor/golang.org/x/term"
+ "vendor/golang.org/x/text"
+ "vendor/golang.org/x/time"
+ "vendor/golang.org/x/tools")))
+
(sha256
(base32
"0939fiy7z53izznfhlr7c6vaskbmkbj3ncb09fzx5dmz9cjngy80"))))
@@ -3309,6 +3322,15 @@ (define-public matterbridge
(arguments
(list
#:import-path "github.com/42wim/matterbridge"))
+ (inputs (list go-golang-org-x-crypto
+ go-golang-org-x-image
+ go-golang-org-x-mod
+ go-golang-org-x-oauth2
+ go-golang-org-x-sys
+ go-golang-org-x-term
+ go-golang-org-x-text
+ go-golang-org-x-time
+ go-golang-org-x-tools))
(synopsis "Bridge together various messaging networks and protocols")
(description
"Relays messages between different channels from various
--
2.46.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies.
2024-10-27 18:12 [bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies Denis 'GNUtoo' Carikli
@ 2024-10-27 20:26 ` Sharlatan Hellseher
2024-10-28 13:00 ` Denis 'GNUtoo' Carikli
2024-10-28 21:56 ` Sharlatan Hellseher
` (2 subsequent siblings)
3 siblings, 1 reply; 6+ messages in thread
From: Sharlatan Hellseher @ 2024-10-27 20:26 UTC (permalink / raw)
To: 74049; +Cc: Denis 'GNUtoo' Carikli
[-- Attachment #1: Type: text/plain, Size: 1235 bytes --]
Hi,
Thanks for the initiation of unbundeling process for materbridge!
A general comments for Golang packages which is just final binary, you
may apply "#:install-source? #f" parameter which mean do not place
source of the project into store and just move produced binary.
Also for the final binary all build inputs may be placed to
"native-inputs" they will never be used elsewhere after the install
phase.
While you are keen to unbundle let's pick up as many as we can. You may
also add some other inputs which are available already:
- filippo.io/edwards25519
- go.uber.org/multierr
- gopkg.in/ini.v1
- gopkg.in/natefinch/lumberjack.v2
- gopkg.in/natefinch/lumberjack.v2
- gopkg.in/yaml.v2
- gopkg.in/yaml.v3
- google.golang.org/protobuf
- github.com/hashicorp ;; all of them are available
- github.com/golang/protobuf
- github.com/fatih/color
- github.com/google/uuid
- github.com/gorilla/websocket
- github.com/pkg/errors
Basically we targeting to have just the first level of dependentices in
"native-inputs" as seen in
<https://github.com/42wim/matterbridge/blob/master/go.mod>.
Please take a look if removing them from vendor and adding them to
native-inputs still keeps the build green.
Thanks for your time.
--
Oleg
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* [bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies.
2024-10-27 20:26 ` Sharlatan Hellseher
@ 2024-10-28 13:00 ` Denis 'GNUtoo' Carikli
0 siblings, 0 replies; 6+ messages in thread
From: Denis 'GNUtoo' Carikli @ 2024-10-28 13:00 UTC (permalink / raw)
To: Sharlatan Hellseher; +Cc: 74049
[-- Attachment #1: Type: text/plain, Size: 3226 bytes --]
On Sun, 27 Oct 2024 20:26:03 +0000
Sharlatan Hellseher <sharlatanus@gmail.com> wrote:
> Thanks for the initiation of unbundeling process for materbridge!
Thanks a lot for the review.
> A general comments for Golang packages which is just final binary, you
> may apply "#:install-source? #f" parameter which mean do not place
> source of the project into store and just move produced binary.
Thanks, I'll do that in a newer patch.
> Also for the final binary all build inputs may be placed to
> "native-inputs" they will never be used elsewhere after the install
> phase.
Right, I just copied a bit what was done in the example I followed, so
that might need to be fixed as well.
Using inputs didn't look right to me but it was done in this way so I
assumed that there was a good reason for it (I tried unbundling long
time ago and it didn't work at the time, but maybe I was just unlucky).
> While you are keen to unbundle let's pick up as many as we can. You
> may also add some other inputs which are available already:
I was already unsure about the patch I sent. In the v2 should I do it
all at once in a single patch or should I split the work across multiple
patches?
The advantage of the former is that it becomes easier to bisect issues.
The downside is that if the granularity is too small, I fear that we'll
end up with hundreds of patches.
So I also think it's a good idea to agree on how to make these changes
before doing more and more patches. For instance you could validate one
patch (without necessarily pushing it) so I could use that as a
blueprint for the next ones and not have to redo all the work for so
many patches.
Here I picked something in between where I grouped the unbundling of
dependencies by selecting something they had in common, but I've no
idea if it's the right way to do it.
Also note that I ran the resulting binary, but without feeding it a
configuration. I'll do a more complete test later on as tests are quite
expensive: I need to install Guix on the production server, update it,
build the matterbridge package, and uninstall guix after that to reduce
the attack surface.
> - filippo.io/edwards25519
> - go.uber.org/multierr
> - gopkg.in/ini.v1
> - gopkg.in/natefinch/lumberjack.v2
> - gopkg.in/natefinch/lumberjack.v2
> - gopkg.in/yaml.v2
> - gopkg.in/yaml.v3
> - google.golang.org/protobuf
> - github.com/hashicorp ;; all of them are available
> - github.com/golang/protobuf
> - github.com/fatih/color
> - github.com/google/uuid
> - github.com/gorilla/websocket
> - github.com/pkg/errors
Thanks, I found some of them like the protobuf one but I didn't add
them in my patch yet.
> Basically we targeting to have just the first level of dependentices
> in "native-inputs" as seen in
> <https://github.com/42wim/matterbridge/blob/master/go.mod>.
OK, I didn't look at that yet, I just started looking at what was in
vendor/ with a script and made a patch with the group that is closer to
the language (golang).
> Please take a look if removing them from vendor and adding them to
> native-inputs still keeps the build green.
Thanks for the additional list, I'll definitely do that.
Denis.
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* [bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies.
2024-10-27 18:12 [bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies Denis 'GNUtoo' Carikli
2024-10-27 20:26 ` Sharlatan Hellseher
@ 2024-10-28 21:56 ` Sharlatan Hellseher
2024-11-05 22:27 ` [bug#74049] [PATCH v2] " Denis 'GNUtoo' Carikli
2024-11-19 12:00 ` bug#74049: [PATCH v1] " Sharlatan Hellseher
3 siblings, 0 replies; 6+ messages in thread
From: Sharlatan Hellseher @ 2024-10-28 21:56 UTC (permalink / raw)
To: 74049; +Cc: Denis 'GNUtoo' Carikli
[-- Attachment #1: Type: text/plain, Size: 1460 bytes --]
Hi Denis,
> I was already unsure about the patch I sent. In the v2 should I do it
> all at once in a single patch or should I split the work across multiple
> patches?
I would suggest to think about the flow, let's say:
* Unbundle as many as possible first, and make it as a single patch
(which should pass build, test, lint etc.)
* Review the massive dependence tree and split it in some sort of
logical blocks/future series which should be self contain and build
individually or when series applied as a whole.
* Start unwind it slowly let's say by 8-12 packages per series, making
sure each series may be apply directly on current master and may be
built.
We may reduce amount of noise to bug-tracker and keep on track with
unbundle process without too much stress and time requiring to review
100++ series
To keep you in a peas of mind - you are not alone in that hard work and
your efforts will be enforced by others who updates/unbunle Golang
packages the more we have the less effort we need to apply to unbunle as
some of the package may be reused.
In short, keep eye on current master and go-team branches and do
rebate/pul often.
I'm waiting for my turne to merge go-team and will continue on the work
with unbundling kubo which will bring about 300 packages.
Some neat-pick give a try to use "guix import go --recursive
<module/name>" on go-team branch which has much better performance than
available in current master.
--
Thanks,
Oleg
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* [bug#74049] [PATCH v2] gnu: matterbridge: Unbundle most golang.org dependencies.
2024-10-27 18:12 [bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies Denis 'GNUtoo' Carikli
2024-10-27 20:26 ` Sharlatan Hellseher
2024-10-28 21:56 ` Sharlatan Hellseher
@ 2024-11-05 22:27 ` Denis 'GNUtoo' Carikli
2024-11-19 12:00 ` bug#74049: [PATCH v1] " Sharlatan Hellseher
3 siblings, 0 replies; 6+ messages in thread
From: Denis 'GNUtoo' Carikli @ 2024-11-05 22:27 UTC (permalink / raw)
To: 74049; +Cc: Denis 'GNUtoo' Carikli
The vendor/golang.org/x/net and vendor/github.com/russross/blackfriday
dependencies were kept because removing any of them results in a compilation
failure:
%exception #<&invoke-error program: "go" arguments: ("install" "-v" "-x"
"-ldflags=-s -w" "-trimpath" "github.com/42wim/matterbridge")
exit-status: 1 term-signal: #f stop-signal: #f>
* gnu/packages/messaging.scm (matterbridge) [source]:
Add snippet removing the unbundled dependencies.
[arguments]: Add new unbundled dependencies.
Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
Change-Id: I67afeff8721833d34a54fd52684a4acbcbcaf179
---
ChangeLog v1->v2:
* Updated copyrights in messaging.scm.
* Unbundled all I could with the existing Guix packages and the commit message
was updated accordingly.
* Added comments to group the dependencies together as there are lots of
dependencies.
---
gnu/packages/messaging.scm | 140 ++++++++++++++++++++++++++++++++++++-
1 file changed, 139 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/messaging.scm b/gnu/packages/messaging.scm
index c482cd469e..f2d7dd90fb 100644
--- a/gnu/packages/messaging.scm
+++ b/gnu/packages/messaging.scm
@@ -29,7 +29,7 @@
;;; Copyright © 2020, 2021 Robert Karszniewicz <avoidr@posteo.de>
;;; Copyright © 2020 Giacomo Leidi <goodoldpaul@autistici.org>
;;; Copyright © 2021 Maxime Devos <maximedevos@telenet.be>
-;;; Copyright © 2021, 2023 Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
+;;; Copyright © 2021, 2023-2024 Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
;;; Copyright © 2021, 2024 Vinicius Monego <monego@posteo.net>
;;; Copyright © 2021 jgart <jgart@dismail.de>
;;; Copyright © 2022 Aleksandr Vityazev <avityazev@posteo.org>
@@ -96,6 +96,9 @@ (define-module (gnu packages messaging)
#:use-module (gnu packages gnupg)
#:use-module (gnu packages golang)
#:use-module (gnu packages golang-build)
+ #:use-module (gnu packages golang-check)
+ #:use-module (gnu packages golang-compression)
+ #:use-module (gnu packages golang-crypto)
#:use-module (gnu packages golang-web)
#:use-module (gnu packages golang-xyz)
#:use-module (gnu packages gperf)
@@ -3302,6 +3305,69 @@ (define-public matterbridge
(url "https://github.com/42wim/matterbridge")
(commit (string-append "v" version))))
(file-name (git-file-name name version))
+ (modules '((guix build utils)))
+ (snippet '(for-each delete-file-recursively
+ ;; TODO: unbundle the rest as well
+ '(;; golang.org
+ "vendor/golang.org/x/crypto"
+ "vendor/golang.org/x/image"
+ "vendor/golang.org/x/mod"
+ "vendor/golang.org/x/oauth2"
+ "vendor/golang.org/x/sys"
+ "vendor/golang.org/x/term"
+ "vendor/golang.org/x/text"
+ "vendor/golang.org/x/time"
+ "vendor/golang.org/x/tools"
+ ;; google.golang.org
+ "vendor/google.golang.org/protobuf/"
+ ;; gopkg.in
+ "vendor/gopkg.in/ini.v1"
+ "vendor/gopkg.in/yaml.v2"
+ "vendor/gopkg.in/yaml.v3"
+ "vendor/gopkg.in/natefinch"
+ ;; filippo.io
+ "vendor/filippo.io"
+ ;; uber.org
+ "vendor/go.uber.org/atomic"
+ "vendor/go.uber.org/multierr"
+ "vendor/go.uber.org/zap"
+ ;; github.com
+ "vendor/github.com/blang"
+ "vendor/github.com/d5"
+ "vendor/github.com/davecgh"
+ "vendor/github.com/dustin"
+ "vendor/github.com/vmihailenco/tagparser"
+ "vendor/github.com/francoispqt"
+ "vendor/github.com/fsnotify"
+ "vendor/github.com/go-asn1-ber"
+ "vendor/github.com/golang"
+ "vendor/github.com/golang-jwt"
+ "vendor/github.com/google/uuid"
+ "vendor/github.com/gorilla/websocket"
+ "vendor/github.com/hashicorp"
+ "vendor/github.com/json-iterator"
+ "vendor/github.com/klauspost"
+ "vendor/github.com/jpillora"
+ "vendor/github.com/kballard"
+ "vendor/github.com/magiconair"
+ "vendor/github.com/mattn/go-colorable"
+ "vendor/github.com/mattn/go-isatty"
+ "vendor/github.com/mattn/go-runewidth"
+ "vendor/github.com/minio/sha256-simd"
+ "vendor/github.com/mitchellh"
+ "vendor/github.com/modern-go"
+ "vendor/github.com/opentracing"
+ "vendor/github.com/pelletier"
+ "vendor/github.com/pkg"
+ "vendor/github.com/rivo"
+ "vendor/github.com/sirupsen"
+ "vendor/github.com/skip2"
+ "vendor/github.com/spf13"
+ "vendor/github.com/stretchr"
+ "vendor/github.com/subosito"
+ "vendor/github.com/valyala/bytebufferpool"
+ "vendor/github.com/pmezard"
+ "vendor/github.com/mgutz/ansi")))
(sha256
(base32
"0939fiy7z53izznfhlr7c6vaskbmkbj3ncb09fzx5dmz9cjngy80"))))
@@ -3309,6 +3375,78 @@ (define-public matterbridge
(arguments
(list
#:import-path "github.com/42wim/matterbridge"))
+ (inputs (list
+ ;; golang.org
+ go-golang-org-x-crypto
+ go-golang-org-x-image
+ go-golang-org-x-mod
+ go-golang-org-x-oauth2
+ go-golang-org-x-sys
+ go-golang-org-x-term
+ go-golang-org-x-text
+ go-golang-org-x-time
+ go-golang-org-x-tools
+ ;; google.golang.org
+ go-google-golang-org-protobuf
+ ;; gopkg.in
+ go-gopkg-in-ini-v1
+ go-gopkg-in-yaml-v2
+ go-gopkg-in-yaml-v3
+ go-gopkg-in-natefinch-lumberjack.v2
+ ;; filippo.io
+ go-filippo-io-edwards25519
+ ;; uber.org
+ go-go-uber-org-atomic
+ go-go-uber-org-multierr
+ go-go-uber-org-zap
+ ;; github.com
+ go-github-com-blang-semver
+ go-github-com-d5-tengo-v2
+ go-github-com-davecgh-go-spew
+ go-github-com-dustin-go-humanize
+ go-github-com-vmihailenco-tagparser
+ go-github-com-francoispqt-gojay
+ go-github-com-fsnotify-fsnotify
+ go-github-com-go-asn1-ber-asn1-ber
+ go-github-com-golang-protobuf
+ go-github-com-golang-jwt-jwt
+ go-github-com-google-uuid
+ go-github-com-gorilla-websocket
+ go-github-com-hashicorp-hcl
+ go-github-com-hashicorp-errwrap
+ go-github-com-hashicorp-golang-lru
+ go-github-com-hashicorp-go-multierror
+ go-github-com-json-iterator-go
+ go-github-com-klauspost-cpuid-v2
+ go-github-com-klauspost-compress
+ go-github-com-jpillora-backoff
+ go-github-com-kballard-go-shellquote
+ go-github-com-magiconair-properties
+ go-github-com-mattn-go-colorable
+ go-github-com-mattn-go-isatty
+ go-github-com-mattn-go-runewidth
+ go-github-com-minio-sha256-simd
+ go-github-com-mitchellh-go-homedir
+ go-github-com-mitchellh-mapstructure
+ go-github-com-modern-go-reflect2
+ go-github-com-modern-go-concurrent
+ go-github-com-opentracing-opentracing-go
+ go-github-com-pelletier-go-toml
+ go-github-com-pelletier-go-toml-v2
+ go-github-com-pkg-errors
+ go-github-com-rivo-uniseg
+ go-github-com-sirupsen-logrus
+ go-github-com-skip2-go-qrcode
+ go-github-com-spf13-pflag
+ go-github-com-spf13-jwalterweatherman
+ go-github-com-spf13-cast
+ go-github-com-spf13-afero
+ go-github-com-spf13-viper
+ go-github-com-stretchr-testify
+ go-github-com-subosito-gotenv
+ go-github-com-valyala-bytebufferpool
+ go-github-com-pmezard-go-difflib
+ go-github-com-mgutz-ansi))
(synopsis "Bridge together various messaging networks and protocols")
(description
"Relays messages between different channels from various
base-commit: 78ffa637313f61a4fec06e5f1e41595f0e79b9eb
--
2.46.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* bug#74049: [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies.
2024-10-27 18:12 [bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies Denis 'GNUtoo' Carikli
` (2 preceding siblings ...)
2024-11-05 22:27 ` [bug#74049] [PATCH v2] " Denis 'GNUtoo' Carikli
@ 2024-11-19 12:00 ` Sharlatan Hellseher
3 siblings, 0 replies; 6+ messages in thread
From: Sharlatan Hellseher @ 2024-11-19 12:00 UTC (permalink / raw)
To: 74049-done
[-- Attachment #1: Type: text/plain, Size: 200 bytes --]
Hi,
I've adjusted the patch a litle by adding embeded file search for
"golang.org/x/net" and fix build with "github.com/russross/blackfriday".
Pushed as cdf968ae86..1dd7a47b65 to master.
--
Oleg
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2024-11-19 12:02 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-10-27 18:12 [bug#74049] [PATCH v1] gnu: matterbridge: Unbundle most golang.org dependencies Denis 'GNUtoo' Carikli
2024-10-27 20:26 ` Sharlatan Hellseher
2024-10-28 13:00 ` Denis 'GNUtoo' Carikli
2024-10-28 21:56 ` Sharlatan Hellseher
2024-11-05 22:27 ` [bug#74049] [PATCH v2] " Denis 'GNUtoo' Carikli
2024-11-19 12:00 ` bug#74049: [PATCH v1] " Sharlatan Hellseher
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).