From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
To: Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>
Cc: 24450@debbugs.gnu.org
Subject: bug#24450: [PATCHv2] Re: pypi importer outputs strange character series in optional dependency case.
Date: Tue, 11 Jun 2019 09:39:48 +0900 [thread overview]
Message-ID: <874l4x550r.fsf@gmail.com> (raw)
In-Reply-To: <idj1s0ialgz.fsf@bimsb-sys02.mdc-berlin.net> (Ricardo Wurmus's message of "Tue, 28 May 2019 13:04:44 +0200")
Hello!
Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:
> Patch number 6:
>
>> From fb0547ef225103c0f8355a7eccc41e0d028f6563 Mon Sep 17 00:00:00 2001
>> From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
>> Date: Thu, 28 Mar 2019 00:26:03 -0400
>> Subject: [PATCH 6/9] import: pypi: Parse wheel METADATA instead of
>> metadata.json.
>
>> With newer Wheel releases, there is no more metadata.json file; the METADATA
>> file should be used instead (see: https://github.com/pypa/wheel/issues/195).
>
>> This change updates our PyPI importer so that it uses the later.
>
> Typo: should be “latter” instead of “later”.
Fixed.
>> * guix/import/pypi.scm (define-module): Remove unnecessary modules and export
>> the PARSE-WHEEL-METADATA method.
>
> Please remove the indentation here. Also, please don’t use “method”
> (because it’s not); use “procedure” instead.
Done. Thanks for fixing my terminology :-).
>> (parse-wheel-metadata): Add method.
>
> Same here.
Done.
>> + (define (requires-dist-header? line)
>> + ;; Return #t if the given LINE is a Requires-Dist header.
>> + (regexp-match? (string-match "^Requires-Dist: " line)))
>> +
>> + (define (requires-dist-value line)
>> + (string-drop line (string-length "Requires-Dist: ")))
>> +
>> + (define (extra? line)
>> + ;; Return #t if the given LINE is an "extra" requirement.
>> + (regexp-match? (string-match "extra == " line)))
>
> The use of “regexp-match?” here isn’t strictly necessary as the return
> value is true-ish anyway.
Done.
>> + (call-with-input-file metadata
>> + (lambda (port)
>> + (let loop ((requirements '()))
>> + (let ((line (read-line port)))
>> + ;; Stop at the first 'Provides-Extra' section: the non-optional
>> + ;; requirements appear before the optional ones.
>> + (if (eof-object? line)
>> + (reverse (delete-duplicates requirements))
>> + (cond
>> + ((and (requires-dist-header? line) (not (extra? line)))
>> + (loop (cons (specification->requirement-name
>> + (requires-dist-value line))
>> + requirements)))
>> + (else
>> + (loop requirements)))))))))
>> +
>
> As before you can simplify the nested let and merge “if” and "cond“.
Oh, I get it now, I think:
--8<---------------cut here---------------start------------->8---
(call-with-input-file metadata
(lambda (port)
(let loop ((requirements '()))
- (let ((line (read-line port)))
- ;; Stop at the first 'Provides-Extra' section: the non-optional
- ;; requirements appear before the optional ones.
- (if (eof-object? line)
- (reverse (delete-duplicates requirements))
- (cond
- ((and (requires-dist-header? line) (not (extra? line)))
- (loop (cons (specification->requirement-name
- (requires-dist-value line))
- requirements)))
- (else
- (loop requirements)))))))))
+ (match (read-line port)
+ (line
+ ;; Stop at the first 'Provides-Extra' section: the non-optional
+ ;; requirements appear before the optional ones.
+ (cond
+ ((eof-object? line)
+ (reverse (delete-duplicates requirements)))
+ ((and (requires-dist-header? line) (not (extra? line)))
+ (loop (cons (specification->requirement-name
+ (requires-dist-value line))
+ requirements)))
+ (else
+ (loop requirements)))))))))
(define (guess-requirements source-url wheel-url archive)
"Given SOURCE-URL, WHEEL-URL and a ARCHIVE of the package, return a list
--8<---------------cut here---------------end--------------->8---
>> (define (read-wheel-metadata wheel-archive)
>> ;; Given WHEEL-ARCHIVE, a ZIP Python wheel archive, return the package's
>> - ;; requirements.
>> + ;; requirements, or #f if the metadata file contained therein couldn't be
>> + ;; extracted.
>> (let* ((dirname (wheel-url->extracted-directory wheel-url))
>> - (json-file (string-append dirname "/metadata.json")))
>> - (and (zero? (system* "unzip" "-q" wheel-archive json-file))
>> - (dynamic-wind
>> - (const #t)
>> - (lambda ()
>> - (call-with-input-file json-file
>> - (lambda (port)
>> - (let* ((metadata (json->scm port))
>> - (run_requires (hash-ref metadata "run_requires"))
>> - (requirements (if run_requires
>> - (hash-ref (list-ref run_requires 0)
>> - "requires")
>> - '())))
>> - (map specification->requirement-name requirements)))))
>> - (lambda ()
>> - (delete-file json-file)
>> - (rmdir dirname))))))
>> + (metadata (string-append dirname "/METADATA")))
>> + (call-with-temporary-directory
>> + (lambda (dir)
>> + (if (zero? (system* "unzip" "-q" wheel-archive "-d" dir metadata))
>> + (parse-wheel-metadata (string-append dir "/" metadata))
>> + (begin
>> + (warning
>> + (G_ "Failed to extract file: ~a from wheel.~%") metadata)
>> + #f))))))
>
> The old approach took care of removing the unpacked archive no matter
> what happened. The new code doesn’t do that.
The temporary directory where the archive is unpacked should be cleared
when leaving upon leaving its scope; the docstring of
"call-with-temporary-directory" says:
"Call PROC with a name of a temporary directory; close the directory and
delete it when leaving the dynamic extent of this call."
>> --- a/tests/pypi.scm
>> +++ b/tests/pypi.scm
>
> Thanks for the tests!
:-)
Maxim
next prev parent reply other threads:[~2019-06-11 0:41 UTC|newest]
Thread overview: 41+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-09-16 20:00 bug#24450: pypi importer outputs strange character series in optional dependency case ng0
2019-03-29 4:24 ` Maxim Cournoyer
2019-06-16 17:02 ` ng0
2019-06-26 4:12 ` Maxim Cournoyer
2019-03-29 4:34 ` bug#24450: [PATCH] " Maxim Cournoyer
2019-03-30 2:12 ` bug#24450: [PATCHv2] " T460s laptop
2019-03-31 14:40 ` bug#24450: [PATCH] " Maxim Cournoyer
2019-04-01 15:28 ` bug#24450: [PATCHv2] " Ludovic Courtès
2019-05-15 11:06 ` Ricardo Wurmus
2019-05-20 4:05 ` bug#24450: [PATCHv2] " Maxim Cournoyer
2019-05-20 15:05 ` Ludovic Courtès
2019-05-22 1:13 ` Maxim Cournoyer
2019-05-27 14:48 ` Ricardo Wurmus
2019-06-10 2:10 ` Maxim Cournoyer
2019-05-27 15:11 ` Ricardo Wurmus
2019-06-10 3:30 ` Maxim Cournoyer
2019-06-10 9:23 ` Ricardo Wurmus
2019-06-16 14:11 ` Maxim Cournoyer
2019-06-17 1:41 ` Ricardo Wurmus
2019-05-27 15:54 ` Ricardo Wurmus
2019-06-10 8:32 ` Maxim Cournoyer
2019-06-10 9:12 ` Ricardo Wurmus
2019-06-16 6:05 ` Maxim Cournoyer
2019-05-27 15:58 ` Ricardo Wurmus
2019-05-28 10:23 ` Ricardo Wurmus
2019-06-10 13:30 ` Maxim Cournoyer
2019-06-10 20:13 ` Ricardo Wurmus
2019-05-28 11:04 ` Ricardo Wurmus
2019-06-11 0:39 ` Maxim Cournoyer [this message]
2019-06-11 11:56 ` Ricardo Wurmus
2019-05-28 13:21 ` Ricardo Wurmus
2019-05-28 14:48 ` Ricardo Wurmus
2019-06-16 5:10 ` Maxim Cournoyer
2019-05-28 14:53 ` Ricardo Wurmus
2019-05-30 2:24 ` Maxim Cournoyer
2019-06-16 5:53 ` Maxim Cournoyer
2019-06-12 3:00 ` Maxim Cournoyer
2019-06-12 6:39 ` Ricardo Wurmus
2019-06-16 14:29 ` Maxim Cournoyer
2019-06-16 14:36 ` bug#24450: [PATCHv3] " Maxim Cournoyer
2019-07-02 1:54 ` Maxim Cournoyer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=874l4x550r.fsf@gmail.com \
--to=maxim.cournoyer@gmail.com \
--cc=24450@debbugs.gnu.org \
--cc=ricardo.wurmus@mdc-berlin.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.