unofficial mirror of 
 help / color / mirror / Atom feed
From: Jason Addison <>
Subject: how to build / pack / distribute / run a simple mpi program
Date: Thu, 3 Jun 2021 07:53:52 -0600	[thread overview]
Message-ID: <> (raw)

I'm struggling with how to build / pack / distribute / run a simple
mpi example from my system with guix to an hpc system without Guix.

I know how to accomplish this with Singularity, but I'm not sure what
is the best way to do it with Guix.

Also, there are pretty good instructions / examples for how to pack /
containerize existing guix packages, but I have not been able to find
an example for not-already-packaged code.

High-level requirements might look something like this:
1- mpich ABI: build against and distribute with mpich.
2- mpi example application, one of the simple examples distributed
with openmpi: openmpi/examples/ring_c.c, built like `mpicc -o ring_c
3- target systems that use PBSpro batch scheduler. This might not be
important, but it might impact how ring_c is launched.

Do I need to turn ring_c into its own full-on package and add it to my
local guix system and then pack it from there?

Should I create an environment, build ring_c with it, and then pack
the environment separately, copy the environment pack and the ring_c
app to the hpc system and try to run ring_c under the environment
pack? I guess I wouldn't really want to have to copy around all of the
dev packages if not needed at runtime.

With Singularity, you need to use the host system's mpiexec to run the
mpi app in your container; I guess so that the host can orchestrate
the mpi jobs properly. Is the same true when using a Guix tarball

Anyway, it seems like packing one-off or in-development apps to run on
non-Guix hpc systems would be a pretty common task and that there is a
best practice. I'm probably just not familiar enough with guix to see
the obvious.

Also, I tried to ask on Guix irc and a package definition file like
this was proposed:

  (name "ring_c")
  (version "1")
  (source (local-file "ring_c.c"))
  (build-system trivial-build-system)
        (let ((source (assoc-ref %build-inputs "source"))
                (out (assoc-ref %outputs "out")))
          (mkdir-p out)
          (invoke "mpicc" "-o"
                       (string-append (assoc-ref %outputs "out") "/ring_c")
  (home-page "")
  (synopsis "")
  (description "")
  (license #f))

Is this the right approach? I tried to work with it a bit. It failed
on "invoke". Is that in a module somewhere? Also, should there be
"propagated-inputs" entry with mpich in this? Is there a way to just
get the runtime mpich stuff and not include all the build/dev stuff?

Thank you very much for your attention. I'm really hoping I can move
away from Singularity and transition my work to Guix!


                 reply	other threads:[~2021-06-03 20:37 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='' \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).