From mboxrd@z Thu Jan 1 00:00:00 1970 From: swedebugia Subject: Re: Quicklisp importer bug in tarball->extract->parse Date: Sat, 27 Apr 2019 08:26:38 +0200 Message-ID: <3f6d08e3-56d4-bc34-944d-b1305b8af37b@riseup.net> References: <874l7fzdbp.fsf@ambrevar.xyz> <87mukwlil1.fsf@ambrevar.xyz> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="WJ2YVLMQ4UM866qjujzgqfhRgIsIY210m" Return-path: Received: from eggs.gnu.org ([209.51.188.92]:58206) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hKGpQ-0005EF-9n for guix-devel@gnu.org; Sat, 27 Apr 2019 02:28:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hKGky-0006U1-OI for guix-devel@gnu.org; Sat, 27 Apr 2019 02:24:17 -0400 Received: from mx1.riseup.net ([198.252.153.129]:36940) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hKGky-0006Sm-C4 for guix-devel@gnu.org; Sat, 27 Apr 2019 02:24:16 -0400 In-Reply-To: <87mukwlil1.fsf@ambrevar.xyz> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Pierre Neidhardt , guix-devel This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --WJ2YVLMQ4UM866qjujzgqfhRgIsIY210m Content-Type: multipart/mixed; boundary="5YnD60PpuoYl21vq4uzrqxiww5L6L5Krr"; protected-headers="v1" From: swedebugia To: Pierre Neidhardt , guix-devel Message-ID: <3f6d08e3-56d4-bc34-944d-b1305b8af37b@riseup.net> Subject: Re: Quicklisp importer bug in tarball->extract->parse References: <874l7fzdbp.fsf@ambrevar.xyz> <87mukwlil1.fsf@ambrevar.xyz> In-Reply-To: <87mukwlil1.fsf@ambrevar.xyz> --5YnD60PpuoYl21vq4uzrqxiww5L6L5Krr Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Hi Thanks a lot for taking the time to look at this. On 2019-04-11 16:31, Pierre Neidhardt wrote: > Hi! >=20 > I've had time to look at it a little. >=20 > If you want to debug, you can, for instance, put a breakpoint in > guess-requirements-from-source,=20 How exactly do you do this? I looked in the guile manual and found the module (system vm trap-state) and tried calling add-trap-at-procedure-call! like this: (add-trap-at-procedure-call! (system* "tar" "xf" tarball "-C" dir "--wildcards" (string-append "*" file-name))))) But it failed with: ;;; (file-name "1am.asd") Backtrace: 13 (apply-smob/1 #) In ice-9/boot-9.scm: 705:2 12 (call-with-prompt _ _ #) In ice-9/eval.scm: 619:8 11 (_ #(#(#))) In ice-9/boot-9.scm: 2312:4 10 (save-module-excursion _) 3831:12 9 (_) In guix/import/quicklisp.scm: 404:3 8 (_) In guix/utils.scm: 618:8 7 (call-with-temporary-output-file _) 632:8 6 (call-with-temporary-directory _) In guix/import/quicklisp.scm: 374:32 5 (_ "/tmp/guix-directory.Z4chBm") In srfi/srfi-1.scm: 592:17 4 (map1 ("1am.asd")) In guix/import/quicklisp.scm: 380:41 3 (_ _) In system/vm/trap-state.scm: 213:15 2 (add-trap-at-procedure-call! 0 _) In system/vm/traps.scm: 137:45 1 (trap-at-procedure-call 0 # #:our-frame? _) In unknown file: 0 (scm-error misc-error #f "~A ~S" ("Not a VM program" 0) #f) ERROR: In procedure scm-error: Not a VM program 0 > There is nothing wrong with find-files: the "dir" directory is empty. > This is because the extraction failed:=20 > check the exit-code of system* before proceeding, you'll see what > happens ;) >=20 > --8<---------------cut here---------------start------------->8--- > (system* "tar" "xf" tarball "-C" dir file-name) > --8<---------------cut here---------------end--------------->8--- >=20 > The above will error out if "file-name" is not found. For 1am: >=20 > Queried file-name: >=20 > "1am.asd" >=20 > Tarball actual file: >=20 > "1am-20141106-git/1am.asd" >=20 > I don't think we can predict the root folder, and more generally > projects have different trees. > So I suggest we extract the whole tarball and proceed from there. So > your code should work by just removing "file-name": >=20 > --8<---------------cut here---------------start------------->8--- > (system* "tar" "xf" tarball "-C" dir) > --8<---------------cut here---------------end--------------->8--- >=20 > Makes sense? Yes. I implemented Ricardos wildcard-suggestion: (system* "tar" "xf" tarball "-C" dir "--wildcards" (string-append "*" file-name)) >=20 > Unrelated comment: If I'm not mistaken, you are fetching the meta-file > and the index-file on every query. I suggest you keep those in memorie= s > as soon as possible (memoize them), this will allow you to iterate much= > faster when developing. I already tried by adding: (memoize (let* ( ;;(name "circular-streams") (name "1am") (source-url (ql-extract 'url name)) (asd (ql-extract 'system-files name))) (peek 'asd asd) (call-with-temporary-output-file (lambda (temp port) (begin (url-fetch source-url temp) (guess-requirements-from-source source-url asd temp) ))))) and (let* ( ;;(name "circular-streams") (name "1am") (source-url (memoize (ql-extract 'url name))) (asd (memoize (ql-extract 'system-files name)))) (peek 'asd asd) (call-with-temporary-output-file (lambda (temp port) (begin (url-fetch source-url temp) (guess-requirements-from-source source-url asd temp) )))) But it still fetches the index-file 2 times regardless where I put it. Any ideas? > It seems that you are on the right track, great job! Thanks for the encouragement! --=20 Cheers Swedebugia --5YnD60PpuoYl21vq4uzrqxiww5L6L5Krr-- --WJ2YVLMQ4UM866qjujzgqfhRgIsIY210m Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEARYIAB0WIQR6IWNlOiLn9hD6a1XPzUNbKAts0gUCXMP2JQAKCRDPzUNbKAts 0ujsAQD/GqUsS9SRC95eKNQnYxuuPskel0RbY4NkwIJGg16GigD+O+bMiJzLxWul Nv298t3N9Zd9+vcabWFAjfWiRS3Xqgc= =Zhi3 -----END PGP SIGNATURE----- --WJ2YVLMQ4UM866qjujzgqfhRgIsIY210m--