From: Jason Addison <jraddison@gmail.com> To: guix-science@gnu.org Subject: how to build / pack / distribute / run a simple mpi program Date: Thu, 3 Jun 2021 07:53:52 -0600 [thread overview] Message-ID: <CAA0wQb0sTUNuWhXv=2diUMf27fx=XD9pAjeix5brpKkwAxagBg@mail.gmail.com> (raw) I'm struggling with how to build / pack / distribute / run a simple mpi example from my system with guix to an hpc system without Guix. I know how to accomplish this with Singularity, but I'm not sure what is the best way to do it with Guix. Also, there are pretty good instructions / examples for how to pack / containerize existing guix packages, but I have not been able to find an example for not-already-packaged code. High-level requirements might look something like this: 1- mpich ABI: build against and distribute with mpich. 2- mpi example application, one of the simple examples distributed with openmpi: openmpi/examples/ring_c.c, built like `mpicc -o ring_c ring_c.c`. 3- target systems that use PBSpro batch scheduler. This might not be important, but it might impact how ring_c is launched. Do I need to turn ring_c into its own full-on package and add it to my local guix system and then pack it from there? Should I create an environment, build ring_c with it, and then pack the environment separately, copy the environment pack and the ring_c app to the hpc system and try to run ring_c under the environment pack? I guess I wouldn't really want to have to copy around all of the dev packages if not needed at runtime. With Singularity, you need to use the host system's mpiexec to run the mpi app in your container; I guess so that the host can orchestrate the mpi jobs properly. Is the same true when using a Guix tarball pack? Anyway, it seems like packing one-off or in-development apps to run on non-Guix hpc systems would be a pretty common task and that there is a best practice. I'm probably just not familiar enough with guix to see the obvious. Also, I tried to ask on Guix irc and a package definition file like this was proposed: (package (name "ring_c") (version "1") (source (local-file "ring_c.c")) (build-system trivial-build-system) (arguments `(#:builder (begin (let ((source (assoc-ref %build-inputs "source")) (out (assoc-ref %outputs "out"))) (mkdir-p out) (invoke "mpicc" "-o" (string-append (assoc-ref %outputs "out") "/ring_c") source)))) (home-page "") (synopsis "") (description "") (license #f)) Is this the right approach? I tried to work with it a bit. It failed on "invoke". Is that in a module somewhere? Also, should there be "propagated-inputs" entry with mpich in this? Is there a way to just get the runtime mpich stuff and not include all the build/dev stuff? Thank you very much for your attention. I'm really hoping I can move away from Singularity and transition my work to Guix! -Jason
reply other threads:[~2021-06-03 20:37 UTC|newest] Thread overview: [no followups] expand[flat|nested] mbox.gz Atom feed
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style List information: https://guix.gnu.org/ * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CAA0wQb0sTUNuWhXv=2diUMf27fx=XD9pAjeix5brpKkwAxagBg@mail.gmail.com' \ --to=jraddison@gmail.com \ --cc=guix-science@gnu.org \ --subject='Re: how to build / pack / distribute / run a simple mpi program' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).