From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id wGejKvfCUmFVNAAAgWs5BA (envelope-from ) for ; Tue, 28 Sep 2021 09:23:35 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id R0FXJvfCUmG7EwAA1q6Kng (envelope-from ) for ; Tue, 28 Sep 2021 07:23:35 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id CBC9727D5E for ; Tue, 28 Sep 2021 09:23:34 +0200 (CEST) Received: from localhost ([::1]:42544 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mV7Sf-0004LA-VP for larch@yhetil.org; Tue, 28 Sep 2021 03:23:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46854) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mV7S1-0003PB-4g for guix-science@gnu.org; Tue, 28 Sep 2021 03:22:53 -0400 Received: from korolev.univ-paris7.fr ([2001:660:3301:8000::1:2]:48600) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mV7Rx-0004Si-QH for guix-science@gnu.org; Tue, 28 Sep 2021 03:22:52 -0400 Received: from univ-paris7.fr (cgunivp7.sigu7.jussieu.fr [81.194.16.84]) by korolev.univ-paris7.fr (8.14.4/8.14.4/relay1/82085) with ESMTP id 18S7Mi1A029689; Tue, 28 Sep 2021 09:22:45 +0200 Received: from [81.194.16.82] (account simon.tournier@paris7.jussieu.fr) by univ-paris7.fr (CommuniGate Pro WebUser 4.2.10) with HTTP id 253615875; Tue, 28 Sep 2021 09:22:44 +0200 From: "Simon Tournier" Subject: User & Developer Meetup on Sept. 27th: quick wrap-up To: guix-science@gnu.org X-Mailer: CommuniGate Pro WebUser Interface v.4.2.10 Date: Tue, 28 Sep 2021 09:22:44 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (korolev.univ-paris7.fr [194.254.61.138]); Tue, 28 Sep 2021 09:22:45 +0200 (CEST) X-Miltered: at korolev with ID 6152C2C4.001 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 6152C2C4.001 from cgunivp7.sigu7.jussieu.fr/cgunivp7.sigu7.jussieu.fr/null/univ-paris7.fr/ X-j-chkmail-Score: MSGID : 6152C2C4.001 on korolev.univ-paris7.fr : j-chkmail score : . : R=. U=. O=. B=0.000 -> S=0.000 X-j-chkmail-Status: Ham Received-SPF: pass client-ip=2001:660:3301:8000::1:2; envelope-from=simon.tournier@paris7.jussieu.fr; helo=korolev.univ-paris7.fr X-Spam_score_int: 8 X-Spam_score: 0.8 X-Spam_bar: / X-Spam_report: (0.8 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, XM_LIGHT_HEAVY=2.499 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guix-science@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-science-bounces+larch=yhetil.org@gnu.org Sender: "Guix-Science" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1632813815; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=9SvCY/Fjc09Z9WLJMSokFct63d2gxAH/4YZ0Ip5s2jA=; b=h5VoKjvEo2OQRbSWUYSJtQgdZ+jfcBVwjTeRZSqfmAqUsW2xmwPMzPhOZExnihVxeP8ECr Nivgi09zf7kc5y3z2tsNIpQo8EgVuH+JQI8pj47/eTDV9i00/LX7PQYBgGiYPHD8r+PXni b7iJARPIMPqs2f1SpjaJuvvs/i++LoRfcVIix5gdg+nogFLzjVKBsaWmODxXZxQbkm28pg 9IaJG15GRL1mD8ubUEoA2iXx1t38+V9f4U3Jl+Ul9pEJui/Dc7JULAXh6asAucOmxFeFWd yyb3f7k9tJruNnn/vlxVfW2J0xcU4Y1WK3mgwbm9Ks00SprEZ6wpgTvOASkzZA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1632813815; a=rsa-sha256; cv=none; b=fj1E5uSvqSzjcOpt1fieXlHfdRJB2Ltv+TSOsumWk3qC8SN9UE6b49gXTFJQCXOYLDz+Ps ZiwcLE24wU4svnCTCIdHldW2i+1Ji5DoHcRnWcUNTUN2Yc0pZnUIzIiw6htAZ/993hV3SG BOAKSlbDzTU1A9wdG+9HKZpl0xr92v6TkaEGhnGo7Tvfy7aCbG9KgGfU3WgA1CpykdZ1HC A5TYIb2Q/xD/Ln/XhcA+IzKfKkXEYDmcntYRP0mTjAjgFdHECtdx3Y0tPNvZGHEOIjo5uK nixoVNyXIl2PFcJtXOX8njJaKvDSRI/WWy5CC/Ex43dS68eO7Mic2odD9HCdFQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-science-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-science-bounces@gnu.org X-Migadu-Spam-Score: -2.39 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-science-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-science-bounces@gnu.org X-Migadu-Queue-Id: CBC9727D5E X-Spam-Score: -2.39 X-Migadu-Scanner: scn0.migadu.com X-TUID: P7hUC7vA+k0D Dear, On this Monday 27th, ~10 people met ~2h to map out actions for the coming year about Guix in scientific context from reproducible research to high-performance computing. Here a quick wrap-up! Do not hesitate to drop an email with something you would like to see or feel free to join #guix-hpc on irc.libera.chat to discuss one specific item. Or please go ahead and help to make it happen. :-) * Organizing training sessions and workshops + we need more! + https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/talks how do we give more visibility of this material already there? + sharing material à la Software Carpentry https://github.com/swcarpentry/swcarpentry + "format" (package?) to easily share this material + sharing reading lists (the idea behind is to have a weekly/monthly/? "newsletter") + PRACE-like EU training sessions (Season [summer, winter, etc.] School) + outreach effort + Libraries (jackhill): for archival as well as reproducing library practices * Funding opportunities + EU project participation and grants to fund specific tasks + BIMSB aims to secure funding for PiGx, which uses Guix and Guile. There is a good chance to hire a person to hack on this in the coming year http://bioinformatics.mdc-berlin.de/pigx/ + research topics + funding a few months of development/integration work (Software Heritage) + "keeping in touch", sharing opportunities (share call for fundings on topics: Repro on HPC, Bioinfo, etc.) + join GSoC or other organizations + run mentoring programs under our own organization ? * Long-term archival using Software Heritage and Disarchive + status: "guix lint -c archival" (git-fetch) and https://guix.gnu.org/sources.json (url-fetch) missing tools to archive svn-fetch (and hg-fetch and minor others: CVS, etc.) + Disarchive can be used to "rebuild" tarballs from content (SWH) and metadata (disarchive-DB) https://git.ngyro.com/disarchive PoC: https://git.ngyro.com/disarchive-db/ + Data Service or Cuirass (Berlin)? + Timothy Sample started to create a Disarchive database for previous Guix releases + left to be done: + build/maintain/publish the Disarchive database + archive the database (-> coordinate with Software Heritage) + metrics: what's the archive coverage on Software Heritage? what's the coverage of the Disarchive database? + want to contribute? email Simon Tournier or mailing list * Citing software using Software Heritage IDs and Guix + ReScience has been using SWHIDs for software submitted (submitters must provide the SWHID, by just clicking on the SWH web interface) + "guix show --format=bibtex" to produce a biblatex software style https://www.ctan.org/tex-archive/macros/latex/contrib/biblatex-contrib/biblatex-software + ideal goal: feed Guix with a BibTeX snippet so it starts deploying it + a command to export the state and a list of packages which could be included to a paper as a citation + another command to import this citation and deploy again * “Converting” reproducible/active papers to use Guix + Example: https://rescience.github.io/bibliography/Courtes_2020.html + write "source-code-to-PDF" (PDF or document in a broad sense) + pipelines automated with Guix for other papers out there + task: find candidate papers that can be automated (example: the ReScience collection) + task: organize a hackathon to work collectively on papers? * Packaging machine learning frameworks + status: we have Tensorflow 1.9, Tensorflow-lite 2.x, PyTorch, scikit-learn + we are stuck with Tensorflow 1.9 because that's the last version to provide a CMake build system + Tensorflow blocker: Bazel build system (in Java, cannot be built from source) + Debian package? https://sources.debian.org/src/bazel-bootstrap/3.5.1+ds-3/debian/control/ + Ricardo looked at Bazel-to-CMake converters - cons: not good enough for Tensorflow + idea: use Bazel to produce a "degenerate" build system + task: package more java package dependencies (for Bazel) + Required reading: https://hpc.guix.info/blog/2021/09/whats-in-a-package/ + tensorflow-lite still needs its Python bindings (this should not be too difficult) - cons: tensorflow-lite is "not very useful" + related question: how do we package ML applications? what's the source: the trained model, or the data set? how do we distribute huge data sets? * Packaging (or not packaging) datasets + relevant for ML models but also many other domains + idea: establish contacts with communities working on this question + https://www.datalad.org/ + https://www.pachyderm.com/ data management + git-annex * Programming with GPUs + NVIDIA has a monopoly, CUDA available in the guix-hpc-non-free channel at INRIA + but this very much goes against our goal of building a transparent software stack + how could we make it easier to support "quick-and-dirty" packaging? + binary-build-system from the nonguix channel + example: Zotero + idea: "guix environment --fhs" to provide an FHS-compliant file tree + Possible starting point: https://gitlab.com/pkill-9/guix-packages-free/blob/master/pkill9/services/fhs.scm * Julia packaging and importer + Efraim prepared 100+ packages + Documenter.jl (JS stuff) required by Flux (ML) and many many + patches on the mailing list for package without JS support + Simon started writing an importer + problem: information in the package registry is hard to use as-is; a service by Julia Computing, Inc. helps a bit + https://github.com/JuliaRegistries/General/tree/master/F/Flux requires to parse a lot of TOML and synopsis+description is not there + https://juliahub.com/docs/Flux/QdkVy/0.11.6/pkg.json API with JSON containing almost everything + policy question: can Guix (the importer?) rely on this service? * CPU micro-architecture support with function multi-versioning + function multiversioning, like in Clear Linux + benchmarking to see if it even provides a benefit + prototype: https://gitlab.inria.fr/guix-hpc/function-multi-versioning + task: find candidate code for automatic FMV + benchmarks + gcc-11 supports different x86_64 "micro-architectures" + https://gcc.gnu.org/gcc-11/changes.html + GCC now supports micro-architecture levels defined in the x86-64 psABI via -march=x86-64-v2, -march=x86-64-v3 and -march=x86-64-v4. * Relocatable pack execution on clusters that lack user namespaces + fixing issues when running MPI applications using the "fakechroot" execution engine on big clusters * Run the tutorial FEniCSX and Firedrake cases using Guix-Jupyter + main tasks: package FEniCSX and Firedrake (+ dependencies, mostly Python) + initial work to happen on a channel Contact Paul Garlick * Hosting a list of scientific channels at hpc.guix.info/channels + provide substitutes via Cuirass + discussion about free software and binaries substitutes? We plan to organize a hackathon day soon* to tackle and make progress on one specific item. Stay tuned! *soon: not fixed yet, surely on November or December. All the best