From mboxrd@z Thu Jan 1 00:00:00 1970 From: Josh Marshall Subject: Re: [EXTERNAL]Re: Modifying squashfs usage for `guix pack` for supporting singularity Date: Thu, 12 Mar 2020 14:05:01 +0000 Message-ID: References: , <87v9n9smtp.fsf@gnu.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="_002_BL0PR06MB4673D018CF20B92763C18E0CE9FD0BL0PR06MB4673namp_" Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:59960) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jCOST-00009z-BX for guix-devel@gnu.org; Thu, 12 Mar 2020 10:05:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jCOSQ-0004Rx-Mz for guix-devel@gnu.org; Thu, 12 Mar 2020 10:05:08 -0400 Received: from esa4.hc3102-66.iphmx.com ([139.138.33.123]:65447) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1jCOSQ-0004QA-7i for guix-devel@gnu.org; Thu, 12 Mar 2020 10:05:06 -0400 In-Reply-To: <87v9n9smtp.fsf@gnu.org> Content-Language: en-US List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane-mx.org@gnu.org Sender: "Guix-devel" To: =?Windows-1252?Q?Ludovic_Court=E8s?= Cc: "guix-devel@gnu.org" --_002_BL0PR06MB4673D018CF20B92763C18E0CE9FD0BL0PR06MB4673namp_ MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable Thanks for getting back to me on this. For the options I mentioned in the email, and the one not yet tracked in em= ail since it is only in IRC is "--no-recovery". These are what stood out t= o me when reading the `mksquashfs` documentation ( https://github.com/ploug= her/squashfs-tools/blob/master/USAGE ) and looking at `guix/scripts/pack.sc= m'. The knowledge you're saying about the `guix pack` options is not commu= nicated by `guix pack --list-formats`, which yields ``` $ guix pack --list-formats The supported formats for 'guix pack' are: tarball Self-contained tarball, ready to run on another machine squashfs Squashfs image suitable for Singularity docker Tarball ready for 'docker load' ``` All this seems to be tangential to the fact that `guix pack --format=3Dsqua= shfs ...` fails on CentOS7 for an undescribed reason: ``` $ guix pack --format=3Dsquashfs bash-minimal substitute: /gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23= /bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8) substitute: /gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23= /bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8) substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0% The following derivations will be built: /gnu/store/saar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squashfs.drv /gnu/store/84vj4wgqm46rws1v73h9gc3hr29xdrs6-singularity-environment.sh.d= rv /gnu/store/iwrrg2knk45h4fps88jqfg92ff1x44zi-profile.drv The following profile hooks will be built: /gnu/store/dgh995fyb41gr3il8r2770j2hlf22nzv-fonts-dir.drv /gnu/store/h5dr8a6vpzrk80wrjz586k3axpz9bhqa-manual-database.drv /gnu/store/svyngf4v16q3j8lgsql508573mvc64my-info-dir.drv /gnu/store/svyydyprj7id6gfz6hsifaf7v4g02brj-ca-certificate-bundle.drv building CA certificate bundle... building fonts directory... building directory of Info manuals... building database for manual pages... building /gnu/store/iwrrg2knk45h4fps88jqfg92ff1x44zi-profile.drv... building /gnu/store/84vj4wgqm46rws1v73h9gc3hr29xdrs6-singularity-environmen= t.sh.drv... /gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23/bin/bash: w= arning: setlocale: LC_ALL: cannot change locale (en_US.utf8) /gnu/store/q19l04vd2za80mk1845pz7r8cz29qk43-bash-minimal-4.4.23/bin/bash: w= arning: setlocale: LC_ALL: cannot change locale (en_US.utf8) building /gnu/store/saar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squas= hfs.drv... \builder for `/gnu/store/saar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.= squashfs.drv' failed with exit code 1 build of /gnu/store/saar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squas= hfs.drv failed View build log at '/var/log/guix/drvs/sa/ar594fgxibr6vkrpjq2n2l5mpyycpp-squ= ashfs-pack.gz.squashfs.drv.bz2'. guix pack: error: build of `/gnu/store/saar594fgxibr6vkrpjq2n2l5mpyycpp-squ= ashfs-pack.gz.squashfs.drv' failed ``` According to the OCI description, Singularity's .sif format could be skippe= d, but at my work it is /the/ standard format. It sounds like there should= be in program changes to describe supported container formats, and which s= ystems support which. This is in addition to getting squashfs container fo= rmat working on CentOS7, and applying some changes to more correctly use `m= ksquashfs`. ________________________________________ From: Ludovic Court=E8s Sent: Thursday, March 12, 2020 9:51 AM To: Josh Marshall Cc: guix-devel@gnu.org Subject: [EXTERNAL]Re: Modifying squashfs usage for `guix pack` for support= ing singularity Hi Josh, Apologies for the delay! (In general, it=92s probably safer to report bugs to bug-guix@gnu.org because they=92re tracked and are less likely to fall through the cracks.) Josh Marshall skribis: > I ran into some trouble today when trying to package an image for singula= rity on CentOS7. nxck suggested I go into the cdoe and try adding the "-no= append" option to mksquashfs in 'guix/guix/scripts/pack.scm'. While lookin= g at it, it seems that it performs a multistep process to modify the squash= fs image, which kind of goes against the design of the utility. A SO page = ( https://unix.stackexchange.com/a/459372 ) suggested the full hierarchy be= created and then added in a single invocation. Does anyone have any opini= ons on such a change or useful insights? [...] > It also appears the `-mkfs-time 0`, `-all-time 0`, and `-UTC` arguments h= ave been missing. Could you explain what error you got and what makes you think these options were missing? IME, the images produced by =91guix pack -f squashfs=92 work fine with Singularity 2.x and 3.x. > Docker and Singularity havebeen moving towards support of the Open Contai= ners Initative, which prompts the discussion of if the docker and sinfulari= ty (squashfs) pack options should be depricated in favor of this standard: = https://github.com/opencontainers/image-spec In fact, =91guix pack -f docker=92 produces an OCI image, IIUC. Singularity 3.x introduced the Singularity Image Format (SIF), which is yet another format. AIUI, SIF is the recommended format for Singularity 3.x, isn=92t it? Thanks, Ludo=92. --- The information in this email, including attachments, may be confidential a= nd is intended solely for the addressee(s). If you believe you received thi= s email by mistake, please notify the sender by return email as soon as pos= sible. --_002_BL0PR06MB4673D018CF20B92763C18E0CE9FD0BL0PR06MB4673namp_ Content-Type: application/x-bzip; name="ar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squashfs.drv.bz2" Content-Description: ar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squashfs.drv.bz2 Content-Disposition: attachment; filename="ar594fgxibr6vkrpjq2n2l5mpyycpp-squashfs-pack.gz.squashfs.drv.bz2"; size=1218; creation-date="Thu, 12 Mar 2020 14:01:10 GMT"; modification-date="Thu, 12 Mar 2020 14:01:10 GMT" Content-Transfer-Encoding: base64 QlpoOTFBWSZTWbB/j54AAZtfgECye+//97/r/kr////wUAS+22Z2NpG7o6awzhJImkwmiep6TNI2 pD1PJgmo9Q0eoxqGgAaYnqDIU9AQjaoNEbQj1A0ZAGhiAeoABoEiQiNTTZJ6Kep6T1GmTamgAepo DQ9QaAHqGhw00wQyGmmRkwgGmgDCaNMmABA0EkRGozSaSb0aE2qPFDTRiAABoAD1AHQUhQgWQPk3 D0I+unDqRIX78RAuwhVuBqYWNjGRTpF/Xu4YPjGk6V0UuFgCgGGFJbmCUTWvrT6oghrQBDEvZKqo yOtppSJUk9yNRCic4+Lk6r6ufBoRGrmloM+jkqvK1I3vrt/uh0tOrTD47J2cGWErMTEltae77M+9 tHE8bmxj1dFi4uTk32qv6ZwqqJ7Dy+0PyJRdlGWIJ7gHYtc8krqFLg7mMSqhBi8lb4qu9eir2oV/ ueGCTISFtXinIbpaGNs2joYGYMXPjxviz+R37YQToga3Ot5r8RMw/nkeX2aqyI7So5bHKKRTWs2l SQNOqyV0mAvFZVkNT5oPu5VyarYfQrK+gzQYTISWf/BoJhO3KXHEYoid1G2svdsBtVHbTl0GrGqa bxBj2dw0DGODUSjzQKWgOa45kkO74xyiRdE88DtUMznsrADy2hlQMr58grYCD125DUZyFdOWKBnc Lo4zsmNJtlsQwNs/CbYhBQY3sKBbswSOd1q4Whm6+IkuwKCRxZ4Z6cq8Y185Sph8Gaa/1FK79c/R o0PbwbOaBtt/NkNn39W2ZgZSEH2kiTTGHOyP+fqeSDdzjueOlPu/Z15Hh0jMZGA9Xl6lw9j4QFTv PYM9bbXeW0HuSGo7WI1b3eCa59tq9erWJNGegCiliGgg/g2f1uIZs08cLAww5we9rkiSbza0UsDh 0kDbiNs0ibr5SUiW5AiZyldHvYgIDBgSizCbq5g38cAXabT4yVWyKMboJ3lOvOdCvDxTDFSnVmau lTKyqZlABRDMQYXgCB9U475MQS0K0UjMb6qoD4nQVgEyIFlF5IzBHWaAXnYTG2Ae4mrkgiDtL1S1 aDjTGIW3BA5RSnCHBRDW7kTzBX7VnT4JGpPwLw6FrDArHY2Zmd6LcjzKR7mFzjFuQxVPiL3JxWr9 vEp6pLac90jlUyynPX0iomC6dyDiTToXLjUeMuvLVHPA2GwYBikYRPjVFkXN0juMXrSqjXala7VY zMJeDgFF1upbnPIsVEk3pGeYbKo1VFQSaNAtUOYYhbneqacdw5ioCsKF5OTbgywtvB0bYATEEeRY akl/LKRiFNUXasglUbBysL9GNSSwJyO+ppqw5jDNOktGx63Tw7N/RQIu4VaLBLiHk+czjGUwlLhi mYHRl0zUILMAZFSRhuNaqgjp7/anWZEy9Sg71RwzRIYVDgY775VUgcZFEqDbQVSlRrkdg1PLB4UQ XgNiw3pFx7CeZCY7sDdE4O3+KYUBVjAyE+5WEKoDF2IzIrmF5UKmOlHAE7ETGgnIuJ0J53QJJcay mRrdSwCkcFgKQymEzCTPCnLoLmLYyuWq2wF8denB5wLAsCisnUg8g7xKUcMShvaBRtXz81aq42Aw iWxYVlOIBUazE/8XckU4UJCwf4+e --_002_BL0PR06MB4673D018CF20B92763C18E0CE9FD0BL0PR06MB4673namp_--