[bug#45773] [PATCH core-updates 0/1] Allow patch-and-repack to work with plain files.

[bug#45773] [PATCH core-updates 0/1] Allow patch-and-repack to work with plain files.

[Maxim Cournoyer]

While attempting to reproduce the now closed issue
<http://issues.guix.gnu.org/30116>, I stumbled upon another annoyance, which
is that the patch-and-repack procedure used as part of an origin derivation
didn't support single files, so the following failed, for example:

<#part type="text/plain" filename="/home/maxim/src/guix-core-updates/repro.scm" disposition=inline description="Script exhibiting problem">
<#/part>

The following patch fixes that, allowing to use plain files with any of the
already supported compressors, or non-tarball archived that would include a
single directory (the same convention as used for our tarballs).

Maxim Cournoyer (1):
  guix: packages: Allow patch-and-repack to work with plain files.

 guix/packages.scm  | 96 +++++++++++++++++++++++++++++++---------------
 tests/packages.scm | 87 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 149 insertions(+), 34 deletions(-)

-- 
2.29.2

[bug#45773] [PATCH core-updates 0/1] Allow patch-and-repack to work with plain files.

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 596 bytes --]

Hello,

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> While attempting to reproduce the now closed issue
> <http://issues.guix.gnu.org/30116>, I stumbled upon another annoyance, which
> is that the patch-and-repack procedure used as part of an origin derivation
> didn't support single files, so the following failed, for example:
>
> <#part type="text/plain" filename="/home/maxim/src/guix-core-updates/repro.scm" disposition=inline description="Script exhibiting problem">
> <#/part>

That didn't work out well :-).

Here's the attachment that I meant to send along the cover letter.


[-- Attachment #2: repro.scm --]
[-- Type: text/plain, Size: 1436 bytes --]

;;; Run script with: ./pre-inst-env guile -e main -s repro.scm

(use-modules (gnu packages base)
             (gnu packages bootstrap)
             (guix build utils)
             (guix derivations)
             (guix download)
             (guix packages)
             (guix store))

(define %test-file-uri
  "https://raw.githubusercontent.com/realgud/realgud/master/realgud/common/bp-image-data.el")

(define (main _)
  (let ((source (origin
                  (method url-fetch)
                  (uri %test-file-uri)
                  (modules '((guix build utils)))
                  (patch-inputs `(("tar"   ,%bootstrap-coreutils&co)
                                  ("xz"    ,%bootstrap-coreutils&co)
                                  ("locales" ,glibc-utf8-locales)))
                  (snippet
                   '(begin
                      (with-fluids ((%default-port-encoding "ISO-8859-1")
                                    (%default-port-conversion-strategy 'error))
                        (substitute* "bp-image-data.el"
                          (("something")
                           "something else")))))
                  (sha256
                   (base32
                    "1qpn2zhh2qw579bhgjyxvf670r4kibaxls589hkm2yhwfvsjvs68")))))
    (with-store store
      (build-derivations store
                         (list (run-with-store store
                                 (origin->derivation source)))))))

[-- Attachment #3: Type: text/plain, Size: 16 bytes --]


Thanks,

Maxim

[bug#45773] [PATCH core-updates 0/1] Allow patch-and-repack to work with plain files.

[Ludovic Courtès]

Hi!

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Before this change, only plain directories, tar or zip archives were supported
> as the source of a package for the GNU build system; anything else would cause
> the unpack phase to fail.  Origins relying on snippets would suffer from the
> same problem.
>
> This change adds the support to use files of the following extensions: .gz,
> .Z, .bz2, .lz, and .xz, even when they are not tarballs.  Files of unknown
> extensions are treated as uncompressed files and supported as well.
>
> * guix/packages.scm (patch-and-repack): Only add the compressor utility to the
> PATH when the file is compressed.  Bind more inputs in the mlet, and use them
> for decompressing single files.  Adjust decompression and compression routines.
> [decompression-type]: Return #f when no known compression extension is used.
> [tarball?]: New nested procedure.
> * guix/build/utils.scm (compressor, tarball?): New procedures.  Move
> %xz-parallel-args to the new 'compression helpers' section.
> * tests/packages.scm: Add tests.  Add missing copyright year for Jan.
> * guix/build/gnu-build-system.scm (first-subdirectory): Return #f when no
> sub-directory was found.
> (unpack): Support more file types, including uncompressed plain files.
> ---
>  guix/build/gnu-build-system.scm |  24 ++++++--
>  guix/build/utils.scm            |  47 ++++++++++-----
>  guix/packages.scm               | 100 +++++++++++++++++---------------
>  guix/tests.scm                  |  40 ++++++++++++-
>  tests/builders.scm              |  40 ++++++++++++-
>  tests/packages.scm              |  69 +++++++++++++++++++++-
>  6 files changed, 247 insertions(+), 73 deletions(-)

How frequent is it for an origin to be a regular file other than an
archive?  The underlying question for me is: will this generalization
and increased complexity pay off?  WDYT?

There are aspects of the patch that I find welcome regardless, such as
the improved handling of compression helpers.

Thanks,
Ludo’.

[bug#45773] [PATCH core-updates 0/1] Allow patch-and-repack to work with plain files.

[Maxim Cournoyer]

Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Before this change, only plain directories, tar or zip archives were supported
>> as the source of a package for the GNU build system; anything else would cause
>> the unpack phase to fail.  Origins relying on snippets would suffer from the
>> same problem.
>>
>> This change adds the support to use files of the following extensions: .gz,
>> .Z, .bz2, .lz, and .xz, even when they are not tarballs.  Files of unknown
>> extensions are treated as uncompressed files and supported as well.
>>
>> * guix/packages.scm (patch-and-repack): Only add the compressor utility to the
>> PATH when the file is compressed.  Bind more inputs in the mlet, and use them
>> for decompressing single files.  Adjust decompression and compression routines.
>> [decompression-type]: Return #f when no known compression extension is used.
>> [tarball?]: New nested procedure.
>> * guix/build/utils.scm (compressor, tarball?): New procedures.  Move
>> %xz-parallel-args to the new 'compression helpers' section.
>> * tests/packages.scm: Add tests.  Add missing copyright year for Jan.
>> * guix/build/gnu-build-system.scm (first-subdirectory): Return #f when no
>> sub-directory was found.
>> (unpack): Support more file types, including uncompressed plain files.
>> ---
>>  guix/build/gnu-build-system.scm |  24 ++++++--
>>  guix/build/utils.scm            |  47 ++++++++++-----
>>  guix/packages.scm               | 100 +++++++++++++++++---------------
>>  guix/tests.scm                  |  40 ++++++++++++-
>>  tests/builders.scm              |  40 ++++++++++++-
>>  tests/packages.scm              |  69 +++++++++++++++++++++-
>>  6 files changed, 247 insertions(+), 73 deletions(-)
>
> How frequent is it for an origin to be a regular file other than an
> archive?  The underlying question for me is: will this generalization
> and increased complexity pay off?  WDYT?

I think consistency is the main driver here.  The url-fetch method
supports single file sources; it makes sense that the other components
handling sources support it as well.  It's hard to judge of the
popularity of such a feature when it's never been available; but some
use cases come to mind such as single Emacs package file.  I've made use
of such feature for the new texlive-updmap.cfg definition in
<http://issues.guix.gnu.org/45870>.

> There are aspects of the patch that I find welcome regardless, such as
> the improved handling of compression helpers.

Great!  Thanks for looking at it.

Maxim