From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?UTF-8?Q?Court=C3=A8s?=) Subject: bug#27476: guix pull fails on powerful server Date: Tue, 07 Nov 2017 11:57:10 +0100 Message-ID: <878tfi9x15.fsf__3885.45878599889$1510052317$gmane$org@gnu.org> References: <87h8vvp1q7.fsf@elephly.net> <87377esu1a.fsf@gnu.org> <87k20nz18u.fsf@igalia.com> <87a81jj5gg.fsf@gnu.org> <87bmlyzxj7.fsf@elephly.net> <87shf44ny0.fsf@elephly.net> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:39658) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eC1a4-0006PZ-VW for bug-guix@gnu.org; Tue, 07 Nov 2017 05:58:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eC1Zy-0003D4-SA for bug-guix@gnu.org; Tue, 07 Nov 2017 05:58:09 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:46548) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eC1Zy-0003Cr-Oe for bug-guix@gnu.org; Tue, 07 Nov 2017 05:58:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1eC1Zy-0003hg-FN for bug-guix@gnu.org; Tue, 07 Nov 2017 05:58:02 -0500 Sender: "Debbugs-submit" Resent-Message-ID: In-Reply-To: <87shf44ny0.fsf@elephly.net> (Ricardo Wurmus's message of "Sat, 30 Sep 2017 09:59:03 +0200") List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: Ricardo Wurmus Cc: Andy Wingo , help-guix@gnu.org, 27476@debbugs.gnu.org Hi, Ricardo Wurmus skribis: > After limiting memory with =E2=80=9Culimit -Sv 5000000=E2=80=9D: > > ice-9/threads.scm:289:22: In procedure loop: > ice-9/threads.scm:289:22: Syntax error: > guix/scripts/pull.scm:192:8: >>=3D: >>=3D (bind) used outside of 'with-mo= nad' in form (>>=3D (indirect-root-added latest) (lambda (done) (mlet* %sto= re-monad () (if (and (file-exists? latest) (string=3D? (readlink latest) so= urce-dir)) (begin (display (G_ "Guix already up to date\n")) (return #t)) (= begin (switch-symlinks latest source-dir) (format #t (G_ "updated ~a succes= sfully deployed under `~a'~%") %guix-package-name latest) (return #t)))))) > > Some deprecated features have been used. Set the environment > variable GUILE_WARN_DEPRECATED to "detailed" and rerun the > program to get more information. Set it to "no" to suppress > this message. > builder for `/gnu/store/afji58647yzz7cr9dvlj87sd3ad36lbk-guix-latest.drv'= failed with exit code 1 > guix pull: error: build failed: build of `/gnu/store/afji58647yzz7cr9dvlj= 87sd3ad36lbk-guix-latest.drv' failed > > It always crashes at around 69%. This gave me an idea. With this program: --8<---------------cut here---------------start------------->8--- (use-modules (ice-9 threads) (srfi srfi-1) (guix monads) (guix store) (system base compile)) (compile #f) ;load modules (define threads (unfold (lambda (x) (> x 100)) (lambda (x) (call-with-new-thread (lambda () (while #t (compile '(begin (with-monad %store-monad (>>=3D foo bar (return 3))) (mlet %store-monad ((x y)) (mbegin %store-monad (return x) (return y)))) #:env (current-module) #:from 'scheme #:to 'tree-il))))) 1+ 0)) (for-each join-thread threads) --8<---------------cut here---------------end--------------->8--- I can reproduce the error: --8<---------------cut here---------------start------------->8--- $ ulimit -Sv 2000000 $ guile syntax-parms.scm In ice-9/psyntax.scm: 1678:45 19 (parse _ _ _ _ _ _ _) In ice-9/boot-9.scm: 230:17 18 (map1 (((("placeholder" placeholder) ((#) . #) =E2=80=A6) =E2=80=A6))) In ice-9/psyntax.scm: 1483:23 17 (_ _ _) In ice-9/boot-9.scm: 230:29 16 (map1 (# # (#) (#)) 230:17 15 (map1 (# (#) (# #)) In ice-9/psyntax.scm: 1788:11 14 (lp ((# # () # ((# #)) #) 1678:45 13 (parse _ _ _ _ _ _ _) In ice-9/boot-9.scm: 230:17 12 (map1 (((("placeholder" placeholder) ("l-1dff1b83541ce327-67a3= 671" lexical . #) ("placeho=E2=80=A6" =E2=80=A6) =E2=80=A6) =E2=80=A6))) In ice-9/psyntax.scm: 1678:45 11 (parse _ _ _ _ _ _ _) In ice-9/boot-9.scm: 230:17 10 (map1 (((("placeholder" placeholder) ((#) . #) =E2=80=A6) . #))) In ice-9/psyntax.scm: 2337:44 9 (expand-let _ _ _ #f (hygiene guile-user) # _ =E2=80=A6) 1678:45 8 (parse _ _ _ _ _ _ _) In ice-9/boot-9.scm: 230:17 7 (map1 (((("placeholder" placeholder) ("l-1dff1b83541ce327-67a3= 7b2" lexical . #) ("placeho=E2=80=A6" =E2=80=A6) =E2=80=A6) =E2=80=A6))) In ice-9/psyntax.scm: 1678:45 6 (parse _ _ _ _ _ _ _) In ice-9/boot-9.scm: 230:17 5 (map1 (((("placeholder" placeholder) ((#) . #) =E2=80=A6) =E2=80=A6))) In ice-9/psyntax.scm: 1483:23 4 (_ _ _) In ice-9/boot-9.scm: 230:17 3 (map1 (# # (#) (#=E2=80=A6>)) In ice-9/psyntax.scm: 1406:23 2 (_ _) 1347:32 1 (syntax-type ((# #)) (("p= laceholder" placeholder) ((#) . #) =E2=80=A6) =E2=80=A6) 1558:32 0 (expand-macro # _ _ _ _ _ _) ice-9/psyntax.scm:1558:32: In procedure expand-macro: ice-9/psyntax.scm:1558:32: Syntax error: unknown location: state-return: Wrong number of arguments in form ((%store-= monad %return)) In syntax-parms.scm: 15:17 9 (_) In system/base/compile.scm: 255:6 8 (compile _ #:from _ #:to _ #:env _ #:opts _) 183:32 7 (compile-fold (#) (begin= (with-monad %store-monad (>>=3D =E2=80=A6)) #) =E2=80=A6) In ice-9/boot-9.scm: 2316:4 6 (save-module-excursion #) In language/scheme/compile-tree-il.scm: 31:15 5 (_) In ice-9/psyntax.scm: 1233:22 4 (expand-top-sequence ((begin (with-monad %store-monad (>>=3D f= oo bar (return 3))) (mlet # ((=E2=80=A6)) #))) _ =E2=80=A6) In ice-9/boot-9.scm: 230:17 3 (map1 (# #= )) In ice-9/psyntax.scm: 1611:33 2 (parse (((("placeholder" placeholder) ((#) . #) =E2=80=A6) . #)) =E2=80=A6) 1347:32 1 (syntax-type (>>=3D foo bar (return 3)) (("placeholder" placeh= older) ((#) . #) =E2=80=A6) =E2=80=A6) 1558:32 0 (expand-macro # _ _ _ _ _ _) ice-9/psyntax.scm:1558:32: In procedure expand-macro: ice-9/psyntax.scm:1558:32: Syntax error: unknown location: source expression failed to match any pattern GC Warning: Failed to expand heap by 28770304 bytes GC Warning: Failed to expand heap by 28770304 bytes GC Warning: Failed to expand heap by 14385152 bytes GC Warning: Out of Memory! Heap size: 919 MiB. Returning NULL! Warning: Unwind-only `out-of-memory' exception; skipping pre-unwind handler. --8<---------------cut here---------------end--------------->8--- So it looks like Guile failing badly in ENOMEM conditions. I can=E2=80=99t reproduce this with current Guile =E2=80=98stable-2.2=E2=80= =99, following Andy=E2=80=99s weak-table rewrite=C2=B9, so this might have been a weak-table bug showing = up under memory pressure. With =E2=80=98guix pull=E2=80=99 this was more likely to happen on your man= y-core server than on my laptop because you have more threads and thus much higher memory usage. Ludo=E2=80=99. =C2=B9 https://lists.gnu.org/archive/html/guile-devel/2017-10/msg00051.html https://git.savannah.gnu.org/cgit/guile.git/commit/?h=3Dstable-2.2&id=3Da= 053c0510c4a644f9453166b7b385cf30f6d3a21