From mboxrd@z Thu Jan 1 00:00:00 1970 From: Chris Marusich Subject: Re: Packaging Terraform, a Golang package Date: Thu, 20 Dec 2018 01:31:48 -0800 Message-ID: <87bm5g4k8r.fsf@gmail.com> References: <875zvqjjnl.fsf@gmail.com> <875zvpm6d3.fsf@cbaines.net> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:34150) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZugS-0001Vz-La for guix-devel@gnu.org; Thu, 20 Dec 2018 04:32:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gZugP-0003JM-Cg for guix-devel@gnu.org; Thu, 20 Dec 2018 04:32:00 -0500 Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]:33528) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gZugO-0003Gc-QG for guix-devel@gnu.org; Thu, 20 Dec 2018 04:31:57 -0500 Received: by mail-pf1-x433.google.com with SMTP id c123so649101pfb.0 for ; Thu, 20 Dec 2018 01:31:53 -0800 (PST) In-Reply-To: <875zvpm6d3.fsf@cbaines.net> (Christopher Baines's message of "Wed, 19 Dec 2018 23:42:00 +0000") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Christopher Baines , Pierre Neidhardt Cc: guix-devel@gnu.org, 30806@debbugs.gnu.org --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain Content-Transfer-Encoding: quoted-printable Hello Chris and Pierre! Thank you for your tips regarding Golang and Terraform. I've built it - both ad-hoc and using a slightly modified version of Chris' original package definition. I'm still getting used to Golang's build system and our own golang-build-system, so please bear with me! I was able to run the official Terraform build steps on the latest Terraform release by issuing the following commands - but I'm not sure if this actually produced the terraform program: mkdir terraform cd terraform/ wget https://github.com/hashicorp/terraform/archive/v0.11.11.tar.gz tar -xf v0.11.11.tar.gz mkdir -p src/github.com/hashicorp cp -r terraform-0.11.11 src/github.com/hashicorp/terraform guix package -i go make git bash grep findutils which coreutils diffutils -= p .guix-profile mkdir gobin eval "$(guix package --search-paths=3Dexact --profile=3D$(realpath --no-sym= links .guix-profile))" export GOPATH=3D"$(pwd)" export GOBIN=3D"$GOPATH/bin" export PATH=3D"$PATH:$GOBIN" cd src/github.com/hashicorp/terraform/ make tools make Those last two commands are the ones listed in Terraform's README.md. It looks like the final command, "make" will execute the "fmtcheck", "generate", and "test" recipes in that order. Here are the relevant parts of the Makefile: =2D-8<---------------cut here---------------start------------->8--- default: test [...] # test runs the unit tests # we run this one package at a time here because running the entire suite in # one command creates memory usage issues when running in Travis-CI. test: fmtcheck generate go list $(TEST) | xargs -t -n4 go test $(TESTARGS) -timeout=3D2m -parallel= =3D4 [...] # generate runs `go generate` to build the dynamically generated # source files. generate: @which stringer > /dev/null; if [ $$? -ne 0 ]; then \ go get -u golang.org/x/tools/cmd/stringer; \ fi go generate ./... @go fmt command/internal_plugin_list.go > /dev/null [...] fmtcheck: @sh -c "'$(CURDIR)/scripts/gofmtcheck.sh'" =2D-8<---------------cut here---------------end--------------->8--- So, in essence this runs "go generate", "go fmt", and "go test". The script gofmtcheck.sh seems to be a read-only linter to assist the Terraform maintainers in finding badly formatted Golang files. This is slightly different from the installation procedure that our go-build-system runs. When you build the attached package (which packages an older version of terraform for now), our go-build-system runs steps like the following (in order from top to bottom): * Phase: unpack: Make a "src/github.com/hashicorp/terraform" directory. * Phase: setup-environment: set GOPATH to (getcwd) and GOBIN to $out/bin. * Phase: build: go install -v -x '-ldflags=3D-s -w' github.com/hashicorp/te= rraform * Phase: check: go test github.com/hashicorp/terraform All in all, this has raised some questions in my mind: 1) Is it bad that our package definition isn't running "go generate" or "go fmt"? Do you know if "go install" does this for us somehow? I don't think "stringer" or "mockgen" are present in the build environment, but our "go install" invocation seems to build terraform even without them. I wonder if the built terraform is broken in some ways because we didn't run the code generation/formatting steps. 2) After I ran "make" ad-hoc, I couldn't find a built "terraform" executable anywhere. Where is it? Am missing something obvious, or could it be that the official documentation incomplete and I need to ask upstream for advice? 3) The official instructions seem to arbitrarily choose to run the build in parallel, using 4 threads, which means this package won't play nice with build arguments like --cores. I suppose I might need to work with upstream to fix that. This feels so close, and yet so far. Hopefully we only have a little more to do to get Terraform packaged well! Thank you again for your help. =2D-=20 Chris --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-gnu-Add-terraform.patch Content-Transfer-Encoding: quoted-printable From=2071d654e2738c6ef870441d4234632bd30e93c74a Mon Sep 17 00:00:00 2001 From: Chris Marusich Date: Thu, 20 Dec 2018 01:24:59 -0800 Subject: [PATCH] gnu: Add terraform. This patch is slightly modified from Christopher Baines' original patch to accommodate some changes that have taken place recently in the Guix source tree. But it's essentially the same. It is still probably not suitable for inclusion in Guix just yet. * gnu/packages/terraform.scm (terraform): New variable. =2D-- gnu/packages/terraform.scm | 40 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/gnu/packages/terraform.scm b/gnu/packages/terraform.scm index f14b152fd..de2288dda 100644 =2D-- a/gnu/packages/terraform.scm +++ b/gnu/packages/terraform.scm @@ -47,3 +47,43 @@ the inputs and outputs for modules of the Terraform infr= astructure management tool. These can be shown, or written to a file in JSON or Markdown format= s.") (home-page "https://github.com/segmentio/terraform-docs") (license license:expat))) + +(define-public terraform + (package + (name "terraform") + (version "0.11.3") + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/hashicorp/terraform") + (commit (string-append "v" version)))) + (sha256 + (base32 + "0637x7jcm62pdnivmh4rggly6dmlvdh3jpsd1z4vba15gbm203nz")))) + (build-system go-build-system) + (arguments + '(#:import-path "github.com/hashicorp/terraform" + #:phases + (modify-phases %standard-phases + ;; I'm not sure what purpose they serve, but they are readonly, so + ;; they break the reset-gzip-timestamps phase. + (add-after 'install 'delete-test-fixtures + (lambda* (#:key outputs #:allow-other-keys) + ;; If delete-file-recursively fails, it won't throw an except= ion. + ;; However, if it doesn't do its job, the build will fail, so + ;; we'll know one way or another. + (delete-file-recursively + (string-append + (assoc-ref outputs "out") + "/src/github.com/hashicorp/terraform/config/module/test-fix= tures")) + #t))))) + (synopsis "Tool for building and changing computing infrastructure") + (description + "Terraform uses descriptions of infrastructure written in @acronym{HC= L, +Hashicorp Configuration Language} which describe graphs of resources, +including information about dependencies. From this, Terraform can plan and +apply changes to the described resources. + +Terraform uses plugins that provide intergrations to different providers.") + (home-page "https://www.terraform.io/") + (license license:mpl2.0))) =2D-=20 2.20.0 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAlwbYYQACgkQ3UCaFdgi Rp2pVRAAldIZ7HMppNLV19sT0R8pJuFL9MdlTAXrfus1PORIpXXsDyCSbJdkkCgU 0YBSCXMma91dPcABegKNbsN5WdztwMNd1/LailLSTnAQa7NFxAZTKOlrATtq+hY1 aqnUiaTkFSMQ+OKnCPrDJ8DLFqic1HOr72qeHlSu/dMYg5e14bDZQTlm2Z1MV28h OhwevOozyVu7L3mJL/I839U8ysIyEnq3AB+q/+XHn7mwfY727uxM4GNVUzETvJAl OiJTiqxES2Lyit14JHmKIol+lJGD4G2+659n3prodq4g1m0M9Ec0BV4urX1xEaHt e9Aafe0Rx9D7nnRze8L/HNwI8IKS/XmUqBvPWHAAq5IIePAagq3Zvo2BDVYfZFe8 nDcdgB0ViM/mNUa0n7eU7zRAnwcun+Z5DFJCllpkiMNTDr4l57LNnJXuzCuCcSGT 3OHvnUvY7Rgrordhdgv/tSaEM+4L+8GCSptIImW2WNq9aWjURKlOemjitSQCF2kg U8OVM1NoLY4xCFsCyXSA4TXKgKsUuo8bgbTclP7ofYBMKQuEm+aaI4XT2tOhGBAg u4EjBxCR8vreJJob//+CxCr2YzlYvBWvkJPBGNlFgEQj+NOPRRR1ScKDUTxhwbar xPFXrSfIjLUTKGOwEpecKV12WjQI1UL9XKe7J1hwJ4C8rC7ZRJY= =qchI -----END PGP SIGNATURE----- --==-=-=--