While attempting to reproduce the now closed issue <http://issues.guix.gnu.org/30116>, I stumbled upon another annoyance, which is that the patch-and-repack procedure used as part of an origin derivation didn't support single files, so the following failed, for example: <#part type="text/plain" filename="/home/maxim/src/guix-core-updates/repro.scm" disposition=inline description="Script exhibiting problem"> <#/part> The following patch fixes that, allowing to use plain files with any of the already supported compressors, or non-tarball archived that would include a single directory (the same convention as used for our tarballs). Maxim Cournoyer (1): guix: packages: Allow patch-and-repack to work with plain files. guix/packages.scm | 96 +++++++++++++++++++++++++++++++--------------- tests/packages.scm | 87 +++++++++++++++++++++++++++++++++++++++-- 2 files changed, 149 insertions(+), 34 deletions(-) -- 2.29.2
[-- Attachment #1: Type: text/plain, Size: 596 bytes --] Hello, Maxim Cournoyer <maxim.cournoyer@gmail.com> writes: > While attempting to reproduce the now closed issue > <http://issues.guix.gnu.org/30116>, I stumbled upon another annoyance, which > is that the patch-and-repack procedure used as part of an origin derivation > didn't support single files, so the following failed, for example: > > <#part type="text/plain" filename="/home/maxim/src/guix-core-updates/repro.scm" disposition=inline description="Script exhibiting problem"> > <#/part> That didn't work out well :-). Here's the attachment that I meant to send along the cover letter. [-- Attachment #2: repro.scm --] [-- Type: text/plain, Size: 1436 bytes --] ;;; Run script with: ./pre-inst-env guile -e main -s repro.scm (use-modules (gnu packages base) (gnu packages bootstrap) (guix build utils) (guix derivations) (guix download) (guix packages) (guix store)) (define %test-file-uri "https://raw.githubusercontent.com/realgud/realgud/master/realgud/common/bp-image-data.el") (define (main _) (let ((source (origin (method url-fetch) (uri %test-file-uri) (modules '((guix build utils))) (patch-inputs `(("tar" ,%bootstrap-coreutils&co) ("xz" ,%bootstrap-coreutils&co) ("locales" ,glibc-utf8-locales))) (snippet '(begin (with-fluids ((%default-port-encoding "ISO-8859-1") (%default-port-conversion-strategy 'error)) (substitute* "bp-image-data.el" (("something") "something else"))))) (sha256 (base32 "1qpn2zhh2qw579bhgjyxvf670r4kibaxls589hkm2yhwfvsjvs68"))))) (with-store store (build-derivations store (list (run-with-store store (origin->derivation source))))))) [-- Attachment #3: Type: text/plain, Size: 16 bytes --] Thanks, Maxim
Hi!
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
> Before this change, only plain directories, tar or zip archives were supported
> as the source of a package for the GNU build system; anything else would cause
> the unpack phase to fail. Origins relying on snippets would suffer from the
> same problem.
>
> This change adds the support to use files of the following extensions: .gz,
> .Z, .bz2, .lz, and .xz, even when they are not tarballs. Files of unknown
> extensions are treated as uncompressed files and supported as well.
>
> * guix/packages.scm (patch-and-repack): Only add the compressor utility to the
> PATH when the file is compressed. Bind more inputs in the mlet, and use them
> for decompressing single files. Adjust decompression and compression routines.
> [decompression-type]: Return #f when no known compression extension is used.
> [tarball?]: New nested procedure.
> * guix/build/utils.scm (compressor, tarball?): New procedures. Move
> %xz-parallel-args to the new 'compression helpers' section.
> * tests/packages.scm: Add tests. Add missing copyright year for Jan.
> * guix/build/gnu-build-system.scm (first-subdirectory): Return #f when no
> sub-directory was found.
> (unpack): Support more file types, including uncompressed plain files.
> ---
> guix/build/gnu-build-system.scm | 24 ++++++--
> guix/build/utils.scm | 47 ++++++++++-----
> guix/packages.scm | 100 +++++++++++++++++---------------
> guix/tests.scm | 40 ++++++++++++-
> tests/builders.scm | 40 ++++++++++++-
> tests/packages.scm | 69 +++++++++++++++++++++-
> 6 files changed, 247 insertions(+), 73 deletions(-)
How frequent is it for an origin to be a regular file other than an
archive? The underlying question for me is: will this generalization
and increased complexity pay off? WDYT?
There are aspects of the patch that I find welcome regardless, such as
the improved handling of compression helpers.
Thanks,
Ludo’.
Hi Ludovic, Ludovic Courtès <ludo@gnu.org> writes: > Hi! > > Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis: > >> Before this change, only plain directories, tar or zip archives were supported >> as the source of a package for the GNU build system; anything else would cause >> the unpack phase to fail. Origins relying on snippets would suffer from the >> same problem. >> >> This change adds the support to use files of the following extensions: .gz, >> .Z, .bz2, .lz, and .xz, even when they are not tarballs. Files of unknown >> extensions are treated as uncompressed files and supported as well. >> >> * guix/packages.scm (patch-and-repack): Only add the compressor utility to the >> PATH when the file is compressed. Bind more inputs in the mlet, and use them >> for decompressing single files. Adjust decompression and compression routines. >> [decompression-type]: Return #f when no known compression extension is used. >> [tarball?]: New nested procedure. >> * guix/build/utils.scm (compressor, tarball?): New procedures. Move >> %xz-parallel-args to the new 'compression helpers' section. >> * tests/packages.scm: Add tests. Add missing copyright year for Jan. >> * guix/build/gnu-build-system.scm (first-subdirectory): Return #f when no >> sub-directory was found. >> (unpack): Support more file types, including uncompressed plain files. >> --- >> guix/build/gnu-build-system.scm | 24 ++++++-- >> guix/build/utils.scm | 47 ++++++++++----- >> guix/packages.scm | 100 +++++++++++++++++--------------- >> guix/tests.scm | 40 ++++++++++++- >> tests/builders.scm | 40 ++++++++++++- >> tests/packages.scm | 69 +++++++++++++++++++++- >> 6 files changed, 247 insertions(+), 73 deletions(-) > > How frequent is it for an origin to be a regular file other than an > archive? The underlying question for me is: will this generalization > and increased complexity pay off? WDYT? I think consistency is the main driver here. The url-fetch method supports single file sources; it makes sense that the other components handling sources support it as well. It's hard to judge of the popularity of such a feature when it's never been available; but some use cases come to mind such as single Emacs package file. I've made use of such feature for the new texlive-updmap.cfg definition in <http://issues.guix.gnu.org/45870>. > There are aspects of the patch that I find welcome regardless, such as > the improved handling of compression helpers. Great! Thanks for looking at it. Maxim