* Need some help packaging parts of a scientific workflow
@ 2023-03-22 21:29 Kyle Andrews
2023-03-23 21:30 ` Philip McGrath
0 siblings, 1 reply; 3+ messages in thread
From: Kyle Andrews @ 2023-03-22 21:29 UTC (permalink / raw)
To: help-guix
Dear Guix,
Part of my scientific workflow involves compiling a small Racket script
for a command line program into its executable and placing that on
PATH. I had bundled this script inside an R package which made sure it
got compiled and everything was correctly configured at library load
time.
From hanging around here for a bit, I have learned this practice is
probably a bad thing for reproducibility. I should separate my support
programs out from my R package rather than leaving them bundled in.
From reading the documentation a lot, I think the actual compilation
step can be done using the "invoke" procedure like so:
```
(invoke "raco" "exe" "{package_location}/custom-shell-tool.rkt")
```
What I'm struggling with the most is understanding all the boilerplate
code I need to place around that fundamental call.
Below is a text-painting of the state of my misunderstanding:
```
(package
(name "custom-shell-tool")
(version "1")
(source
(local-file "package_location")) ; how to refer to local files?
(build-system copy-build-system)
(arguments
(list
#:phases
#~(modify-phases %standard-phases
(add-before 'install 'compile
(lambda _
(invoke "raco" "exe"
(string-append
#$package-folder ; how to refer to the build itself?
"custom-shell-tool.rkt"))))))
#:install-plan
#~'(("custom-shell-tool" "bin/")))
(home-page #f)
(synopsis
"A custom shell tool needed only for the niche workflows I write")
(description "This package is different from the ones in Guix.")
(license #f))
```
Would you please help me translate this text painting into a working
package?
I'm especially interested in figuring out how I can productively learn
to experiment productively with this stuff for myself.
Thanks for your help,
Kyle
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Need some help packaging parts of a scientific workflow
2023-03-22 21:29 Need some help packaging parts of a scientific workflow Kyle Andrews
@ 2023-03-23 21:30 ` Philip McGrath
2023-03-29 4:46 ` Kyle
0 siblings, 1 reply; 3+ messages in thread
From: Philip McGrath @ 2023-03-23 21:30 UTC (permalink / raw)
To: Kyle, help-guix
Hi Kyle,
On Wed, Mar 22, 2023, at 5:29 PM, Kyle Andrews wrote:
> Dear Guix,
>
> Part of my scientific workflow involves compiling a small Racket script
> for a command line program into its executable and placing that on
> PATH.
I am always glad to hear of more people using Guix and Racket together!
> I had bundled this script inside an R package which made sure it
> got compiled and everything was correctly configured at library load
> time.
>
Tangential to your actual question, I think this is not necessarily a terrible practice. There is not much difference between running `my-script` and running `racket -y "path/to/my-script.rkt"`, and, if you do that or `raco make` during the build process of your R package, you'll get compiled files properly. There are tradeoffs to weigh, including support for R users without Guix. But there are also good reasons to decide to separate the Racket script from the R library, so that's what I'll explain below.
> From reading the documentation a lot, I think the actual compilation
> step can be done using the "invoke" procedure like so:
>
> ```
> (invoke "raco" "exe" "{package_location}/custom-shell-tool.rkt")
> ```
>
> What I'm struggling with the most is understanding all the boilerplate
> code I need to place around that fundamental call.
>
I'll start with a working example suitable for `guix build -f`, then answer your specific questions.
```
;; SPDX-License-Identifier: (CC0-1.0 OR (Apache-2.0 OR MIT))
;; SPDX-FileCopyrightText: Philip McGrath <philip@philipmcgrath.com>
(use-modules
(gnu packages racket)
(guix build-system copy)
(guix gexp)
(guix packages))
(package
(name "racket-hello")
(version "1.0")
(source (plain-file "hello.rkt"
"#lang racket (displayln '|Hello from Racket!|)"))
(inputs (list racket))
(build-system copy-build-system)
(arguments
(list
#:install-plan #~'(("hello" "bin/"))
#:phases
#~(modify-phases %standard-phases
(add-before 'install 'build
(lambda args
(invoke "raco" "exe" "hello.rkt"))))))
(home-page #f)
(synopsis "Hello world in Racket")
(description
"This is a trivial example of using @code{raco exe} with Guix.")
(license #f))
```
In fact this package would be a reasonable candidate for `trivial-build-system`, but I've stuck with `copy-build-system` because the boilerplate for `trivial-build-system` is very different than for all other build systems.
Likewise, I'm assuming you know how you want to build your executable, but you might consider the `--launcher` flag for `raco exe` and an explicit call to `raco make`: in particular, it might take use less total disk space than an ELF executable.
Note that you do have to use `racket`, not `racket-minimal`, because `racket-minimal` doesn't include the `raco exe` command.
> (source
> (local-file "package_location")) ; how to refer to local files?
In general, `local-file` is the right mechanism; specifics depend on your situation, including how you are expecting your package definition to be used. For a single file, `(local-file "path/to/script.rkt")` is probably what you want, where the path is relative to the Guile file containing the `local-file` expression. For a directory, consider the `#:recursive?` and `#:select?` arguments.
> (invoke "raco" "exe"
> (string-append
> #$package-folder ; how to refer to the build itself?
> "custom-shell-tool.rkt"))))))
The `unpack` phase from `gnu-build-system` handles this: if the package source is a directory, you're inside a copy of it; if it's a file, you're in a temporary directory containing it.
>
> I'm especially interested in figuring out how I can productively learn
> to experiment productively with this stuff for myself.
>
Personally, I often insert a phase that calls `error` to stop the build, perhaps printing out interesting values first, and use `guix build --keep-failed` to explore the build environment.
-Philip
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: Need some help packaging parts of a scientific workflow
2023-03-23 21:30 ` Philip McGrath
@ 2023-03-29 4:46 ` Kyle
0 siblings, 0 replies; 3+ messages in thread
From: Kyle @ 2023-03-29 4:46 UTC (permalink / raw)
To: Philip McGrath, help-guix
Thanks, Philip! Your descriptions really made it click for me.
Might you give a small example of what you mean by adding an error phase? That sounds like an important debugging strategy, but I can't quite grasp what it would actually look like. I'm curious what kinds of data you are printing out.
On March 23, 2023 5:30:11 PM EDT, Philip McGrath <philip@philipmcgrath.com> wrote:
>Hi Kyle,
>
>On Wed, Mar 22, 2023, at 5:29 PM, Kyle Andrews wrote:
>> Dear Guix,
>>
>> Part of my scientific workflow involves compiling a small Racket script
>> for a command line program into its executable and placing that on
>> PATH.
>
>I am always glad to hear of more people using Guix and Racket together!
>
>> I had bundled this script inside an R package which made sure it
>> got compiled and everything was correctly configured at library load
>> time.
>>
>
>Tangential to your actual question, I think this is not necessarily a terrible practice. There is not much difference between running `my-script` and running `racket -y "path/to/my-script.rkt"`, and, if you do that or `raco make` during the build process of your R package, you'll get compiled files properly. There are tradeoffs to weigh, including support for R users without Guix. But there are also good reasons to decide to separate the Racket script from the R library, so that's what I'll explain below.
>
>> From reading the documentation a lot, I think the actual compilation
>> step can be done using the "invoke" procedure like so:
>>
>> ```
>> (invoke "raco" "exe" "{package_location}/custom-shell-tool.rkt")
>> ```
>>
>> What I'm struggling with the most is understanding all the boilerplate
>> code I need to place around that fundamental call.
>>
>
>I'll start with a working example suitable for `guix build -f`, then answer your specific questions.
>
>```
>;; SPDX-License-Identifier: (CC0-1.0 OR (Apache-2.0 OR MIT))
>;; SPDX-FileCopyrightText: Philip McGrath <philip@philipmcgrath.com>
>
>(use-modules
> (gnu packages racket)
> (guix build-system copy)
> (guix gexp)
> (guix packages))
>
>(package
> (name "racket-hello")
> (version "1.0")
> (source (plain-file "hello.rkt"
> "#lang racket (displayln '|Hello from Racket!|)"))
> (inputs (list racket))
> (build-system copy-build-system)
> (arguments
> (list
> #:install-plan #~'(("hello" "bin/"))
> #:phases
> #~(modify-phases %standard-phases
> (add-before 'install 'build
> (lambda args
> (invoke "raco" "exe" "hello.rkt"))))))
> (home-page #f)
> (synopsis "Hello world in Racket")
> (description
> "This is a trivial example of using @code{raco exe} with Guix.")
> (license #f))
>```
>
>In fact this package would be a reasonable candidate for `trivial-build-system`, but I've stuck with `copy-build-system` because the boilerplate for `trivial-build-system` is very different than for all other build systems.
>
>Likewise, I'm assuming you know how you want to build your executable, but you might consider the `--launcher` flag for `raco exe` and an explicit call to `raco make`: in particular, it might take use less total disk space than an ELF executable.
>
>Note that you do have to use `racket`, not `racket-minimal`, because `racket-minimal` doesn't include the `raco exe` command.
>
>> (source
>> (local-file "package_location")) ; how to refer to local files?
>
>In general, `local-file` is the right mechanism; specifics depend on your situation, including how you are expecting your package definition to be used. For a single file, `(local-file "path/to/script.rkt")` is probably what you want, where the path is relative to the Guile file containing the `local-file` expression. For a directory, consider the `#:recursive?` and `#:select?` arguments.
>
>> (invoke "raco" "exe"
>> (string-append
>> #$package-folder ; how to refer to the build itself?
>> "custom-shell-tool.rkt"))))))
>
>The `unpack` phase from `gnu-build-system` handles this: if the package source is a directory, you're inside a copy of it; if it's a file, you're in a temporary directory containing it.
>
>>
>> I'm especially interested in figuring out how I can productively learn
>> to experiment productively with this stuff for myself.
>>
>
>Personally, I often insert a phase that calls `error` to stop the build, perhaps printing out interesting values first, and use `guix build --keep-failed` to explore the build environment.
>
>-Philip
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2023-03-29 4:47 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-03-22 21:29 Need some help packaging parts of a scientific workflow Kyle Andrews
2023-03-23 21:30 ` Philip McGrath
2023-03-29 4:46 ` Kyle
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).