* Need some help packaging parts of a scientific workflow @ 2023-03-22 21:29 Kyle Andrews 2023-03-23 21:30 ` Philip McGrath 0 siblings, 1 reply; 3+ messages in thread From: Kyle Andrews @ 2023-03-22 21:29 UTC (permalink / raw) To: help-guix Dear Guix, Part of my scientific workflow involves compiling a small Racket script for a command line program into its executable and placing that on PATH. I had bundled this script inside an R package which made sure it got compiled and everything was correctly configured at library load time. From hanging around here for a bit, I have learned this practice is probably a bad thing for reproducibility. I should separate my support programs out from my R package rather than leaving them bundled in. From reading the documentation a lot, I think the actual compilation step can be done using the "invoke" procedure like so: ``` (invoke "raco" "exe" "{package_location}/custom-shell-tool.rkt") ``` What I'm struggling with the most is understanding all the boilerplate code I need to place around that fundamental call. Below is a text-painting of the state of my misunderstanding: ``` (package (name "custom-shell-tool") (version "1") (source (local-file "package_location")) ; how to refer to local files? (build-system copy-build-system) (arguments (list #:phases #~(modify-phases %standard-phases (add-before 'install 'compile (lambda _ (invoke "raco" "exe" (string-append #$package-folder ; how to refer to the build itself? "custom-shell-tool.rkt")))))) #:install-plan #~'(("custom-shell-tool" "bin/"))) (home-page #f) (synopsis "A custom shell tool needed only for the niche workflows I write") (description "This package is different from the ones in Guix.") (license #f)) ``` Would you please help me translate this text painting into a working package? I'm especially interested in figuring out how I can productively learn to experiment productively with this stuff for myself. Thanks for your help, Kyle ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Need some help packaging parts of a scientific workflow 2023-03-22 21:29 Need some help packaging parts of a scientific workflow Kyle Andrews @ 2023-03-23 21:30 ` Philip McGrath 2023-03-29 4:46 ` Kyle 0 siblings, 1 reply; 3+ messages in thread From: Philip McGrath @ 2023-03-23 21:30 UTC (permalink / raw) To: Kyle, help-guix Hi Kyle, On Wed, Mar 22, 2023, at 5:29 PM, Kyle Andrews wrote: > Dear Guix, > > Part of my scientific workflow involves compiling a small Racket script > for a command line program into its executable and placing that on > PATH. I am always glad to hear of more people using Guix and Racket together! > I had bundled this script inside an R package which made sure it > got compiled and everything was correctly configured at library load > time. > Tangential to your actual question, I think this is not necessarily a terrible practice. There is not much difference between running `my-script` and running `racket -y "path/to/my-script.rkt"`, and, if you do that or `raco make` during the build process of your R package, you'll get compiled files properly. There are tradeoffs to weigh, including support for R users without Guix. But there are also good reasons to decide to separate the Racket script from the R library, so that's what I'll explain below. > From reading the documentation a lot, I think the actual compilation > step can be done using the "invoke" procedure like so: > > ``` > (invoke "raco" "exe" "{package_location}/custom-shell-tool.rkt") > ``` > > What I'm struggling with the most is understanding all the boilerplate > code I need to place around that fundamental call. > I'll start with a working example suitable for `guix build -f`, then answer your specific questions. ``` ;; SPDX-License-Identifier: (CC0-1.0 OR (Apache-2.0 OR MIT)) ;; SPDX-FileCopyrightText: Philip McGrath <philip@philipmcgrath.com> (use-modules (gnu packages racket) (guix build-system copy) (guix gexp) (guix packages)) (package (name "racket-hello") (version "1.0") (source (plain-file "hello.rkt" "#lang racket (displayln '|Hello from Racket!|)")) (inputs (list racket)) (build-system copy-build-system) (arguments (list #:install-plan #~'(("hello" "bin/")) #:phases #~(modify-phases %standard-phases (add-before 'install 'build (lambda args (invoke "raco" "exe" "hello.rkt")))))) (home-page #f) (synopsis "Hello world in Racket") (description "This is a trivial example of using @code{raco exe} with Guix.") (license #f)) ``` In fact this package would be a reasonable candidate for `trivial-build-system`, but I've stuck with `copy-build-system` because the boilerplate for `trivial-build-system` is very different than for all other build systems. Likewise, I'm assuming you know how you want to build your executable, but you might consider the `--launcher` flag for `raco exe` and an explicit call to `raco make`: in particular, it might take use less total disk space than an ELF executable. Note that you do have to use `racket`, not `racket-minimal`, because `racket-minimal` doesn't include the `raco exe` command. > (source > (local-file "package_location")) ; how to refer to local files? In general, `local-file` is the right mechanism; specifics depend on your situation, including how you are expecting your package definition to be used. For a single file, `(local-file "path/to/script.rkt")` is probably what you want, where the path is relative to the Guile file containing the `local-file` expression. For a directory, consider the `#:recursive?` and `#:select?` arguments. > (invoke "raco" "exe" > (string-append > #$package-folder ; how to refer to the build itself? > "custom-shell-tool.rkt")))))) The `unpack` phase from `gnu-build-system` handles this: if the package source is a directory, you're inside a copy of it; if it's a file, you're in a temporary directory containing it. > > I'm especially interested in figuring out how I can productively learn > to experiment productively with this stuff for myself. > Personally, I often insert a phase that calls `error` to stop the build, perhaps printing out interesting values first, and use `guix build --keep-failed` to explore the build environment. -Philip ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Need some help packaging parts of a scientific workflow 2023-03-23 21:30 ` Philip McGrath @ 2023-03-29 4:46 ` Kyle 0 siblings, 0 replies; 3+ messages in thread From: Kyle @ 2023-03-29 4:46 UTC (permalink / raw) To: Philip McGrath, help-guix Thanks, Philip! Your descriptions really made it click for me. Might you give a small example of what you mean by adding an error phase? That sounds like an important debugging strategy, but I can't quite grasp what it would actually look like. I'm curious what kinds of data you are printing out. On March 23, 2023 5:30:11 PM EDT, Philip McGrath <philip@philipmcgrath.com> wrote: >Hi Kyle, > >On Wed, Mar 22, 2023, at 5:29 PM, Kyle Andrews wrote: >> Dear Guix, >> >> Part of my scientific workflow involves compiling a small Racket script >> for a command line program into its executable and placing that on >> PATH. > >I am always glad to hear of more people using Guix and Racket together! > >> I had bundled this script inside an R package which made sure it >> got compiled and everything was correctly configured at library load >> time. >> > >Tangential to your actual question, I think this is not necessarily a terrible practice. There is not much difference between running `my-script` and running `racket -y "path/to/my-script.rkt"`, and, if you do that or `raco make` during the build process of your R package, you'll get compiled files properly. There are tradeoffs to weigh, including support for R users without Guix. But there are also good reasons to decide to separate the Racket script from the R library, so that's what I'll explain below. > >> From reading the documentation a lot, I think the actual compilation >> step can be done using the "invoke" procedure like so: >> >> ``` >> (invoke "raco" "exe" "{package_location}/custom-shell-tool.rkt") >> ``` >> >> What I'm struggling with the most is understanding all the boilerplate >> code I need to place around that fundamental call. >> > >I'll start with a working example suitable for `guix build -f`, then answer your specific questions. > >``` >;; SPDX-License-Identifier: (CC0-1.0 OR (Apache-2.0 OR MIT)) >;; SPDX-FileCopyrightText: Philip McGrath <philip@philipmcgrath.com> > >(use-modules > (gnu packages racket) > (guix build-system copy) > (guix gexp) > (guix packages)) > >(package > (name "racket-hello") > (version "1.0") > (source (plain-file "hello.rkt" > "#lang racket (displayln '|Hello from Racket!|)")) > (inputs (list racket)) > (build-system copy-build-system) > (arguments > (list > #:install-plan #~'(("hello" "bin/")) > #:phases > #~(modify-phases %standard-phases > (add-before 'install 'build > (lambda args > (invoke "raco" "exe" "hello.rkt")))))) > (home-page #f) > (synopsis "Hello world in Racket") > (description > "This is a trivial example of using @code{raco exe} with Guix.") > (license #f)) >``` > >In fact this package would be a reasonable candidate for `trivial-build-system`, but I've stuck with `copy-build-system` because the boilerplate for `trivial-build-system` is very different than for all other build systems. > >Likewise, I'm assuming you know how you want to build your executable, but you might consider the `--launcher` flag for `raco exe` and an explicit call to `raco make`: in particular, it might take use less total disk space than an ELF executable. > >Note that you do have to use `racket`, not `racket-minimal`, because `racket-minimal` doesn't include the `raco exe` command. > >> (source >> (local-file "package_location")) ; how to refer to local files? > >In general, `local-file` is the right mechanism; specifics depend on your situation, including how you are expecting your package definition to be used. For a single file, `(local-file "path/to/script.rkt")` is probably what you want, where the path is relative to the Guile file containing the `local-file` expression. For a directory, consider the `#:recursive?` and `#:select?` arguments. > >> (invoke "raco" "exe" >> (string-append >> #$package-folder ; how to refer to the build itself? >> "custom-shell-tool.rkt")))))) > >The `unpack` phase from `gnu-build-system` handles this: if the package source is a directory, you're inside a copy of it; if it's a file, you're in a temporary directory containing it. > >> >> I'm especially interested in figuring out how I can productively learn >> to experiment productively with this stuff for myself. >> > >Personally, I often insert a phase that calls `error` to stop the build, perhaps printing out interesting values first, and use `guix build --keep-failed` to explore the build environment. > >-Philip ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2023-03-29 4:47 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-03-22 21:29 Need some help packaging parts of a scientific workflow Kyle Andrews 2023-03-23 21:30 ` Philip McGrath 2023-03-29 4:46 ` Kyle
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).