From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id mNFaE06O22AJewAAgWs5BA (envelope-from ) for ; Tue, 29 Jun 2021 23:19:10 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id AEIYD06O22DGBgAAB5/wlQ (envelope-from ) for ; Tue, 29 Jun 2021 21:19:10 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8432434A8C for ; Tue, 29 Jun 2021 23:19:09 +0200 (CEST) Received: from localhost ([::1]:34740 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lyL8O-0004YB-4h for larch@yhetil.org; Tue, 29 Jun 2021 17:19:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:32948) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyL8H-0004Xn-Ur for bug-guix@gnu.org; Tue, 29 Jun 2021 17:19:01 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:44255) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lyL8H-0006Mi-Ml for bug-guix@gnu.org; Tue, 29 Jun 2021 17:19:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lyL8H-0002ac-J0 for bug-guix@gnu.org; Tue, 29 Jun 2021 17:19:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#48903: guix substitute: error: TLS error in procedure 'read_from_session_record_port': Error decoding the received TLS packet. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 29 Jun 2021 21:19:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 48903 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Maxim Cournoyer Received: via spool by 48903-submit@debbugs.gnu.org id=B48903.16250015329938 (code B ref 48903); Tue, 29 Jun 2021 21:19:01 +0000 Received: (at 48903) by debbugs.gnu.org; 29 Jun 2021 21:18:52 +0000 Received: from localhost ([127.0.0.1]:55801 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lyL87-0002aE-QZ for submit@debbugs.gnu.org; Tue, 29 Jun 2021 17:18:52 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35058) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lyL83-0002Zu-AP for 48903@debbugs.gnu.org; Tue, 29 Jun 2021 17:18:51 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44294) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lyL7x-00067l-NC; Tue, 29 Jun 2021 17:18:41 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=42562 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyL7x-0007fg-Fu; Tue, 29 Jun 2021 17:18:41 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87bl8hvhgx.fsf@gmail.com> <87wnr05tfz.fsf@gnu.org> <87o8bovm7t.fsf@gmail.com> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 11 Messidor an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Tue, 29 Jun 2021 23:18:39 +0200 In-Reply-To: <87o8bovm7t.fsf@gmail.com> (Maxim Cournoyer's message of "Tue, 29 Jun 2021 09:23:18 -0400") Message-ID: <87sg10pdxs.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 48903@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1625001549; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post; bh=w4OfGP7J09jllefgUb4ed1K0551jrV9P9vSh5LnedGs=; b=dxKTsxIXYi1dHmkq0m3GV3AYkMw1zvHmWoOgaQG312MKWkfVe4Xshf+sPt3ROy+GoMzaqp ikFPzXuI240mhLnt5wXlOPycnujor9CrWGI+5B3FN3hKLkFiqYZqfziWwdPqTH6JvSu5Yy pyJBirQDB5mQPGNCEv4tulsvwyNCaHY8hMHDo1oqkSwJ5cl7aCSHn5KtYtcDUhfwGGduvw SSz9E/bbMvLbXhg5ysSAsvZ4fvqf55zTyxU7MwkywE9yTZ5Vj5znshh1VYRbwpEXceUkbV Hidd+ojiffq/crKq58nRfZ1S/niJJcM9r0G7NvIYhz8ahlxg57YXhLNedP17Qw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1625001549; a=rsa-sha256; cv=none; b=V0PmACgE5GvTIcv9h4z2ZTOFS66T7njOLvO+nYwVC/L2/TVofFx8pQk3kzWGXQW38Myzen gsnsK5g+2rOo/jrwE4GrFIVaIlHPdACBZlRSuQPvWcZFUWVCTwCtf21c9v8lXJPFJ4pjqP avRo3NAZtqwaJ8VaM3bNmLwDls9kh4cAarHmkc1QkGFB8PqNuqxKxsOSPrNcIrynNyFV4p ZKSboGnBQHPvIBDMQzFHqPwRYbFUoah/4ShM3M0YQGH08jXtDE5JIywWTOH+oxN8QNxF8M rABcZhrk7EnHybIILgK2PjJI5Z6rXW8RzC+btHq3Fn1fD75Iu6Qo6lSiPScBFw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Spam-Score: -1.92 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: 8432434A8C X-Spam-Score: -1.92 X-Migadu-Scanner: scn0.migadu.com X-TUID: v4eTaJngwQ/r --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, Maxim Cournoyer skribis: > $ sudo ps -eF | grep guix-daemon > root 25193 216 0 3074 1524 3 Jun28 ? 00:00:00 /gnu/stor= e/vphx2839xv0qj9xwcwrb95592lzrrnx7-guix-1.3.0-3.50dfbbf/bin/guix-daemon 251= 78 guixbuild --max-silent-time 0 --timeout 0 --log-compressio= n none --discover=3Dno --substitute-urls http://127.0.0.1:8080 https://ci.g= uix.gnu.org --max-jobs=3D4--8<---------------cut here---------------end----= ----------->8--- > > I can rather easily (and annoyingly!) trigger the problem (and a few > variations of it, it seems) with something like: > > $ packages=3D$(guix refresh -l protobuf | sed 's/^.*: //') > $ guix build -v3 --keep-going $packages > > For example, running the above, I just got: > > guix build: error: corrupt input while restoring archive from # file 7fc95acfc2a0> > --8<---------------cut here---------------end--------------->8--- > > Does the above commands succeed on the first time on your end? If you > have already lots of things cached, you can try for an architecture you > don't often build for by adding the '--system=3Di686-linux' option; that > should cause a massive amount of downloads, likely to trigger the > problem. Perhaps also try to use --max-jobs=3D4. I=E2=80=99ve tried that, with --max-jobs=3D4, and it fills my disk just fin= e. :-/ > If you have ideas of how to debug this when I hit the issue I'm all ears > :-). The attached patch substitutes a number of store items in a row; run: guix repl -- substitute-stress.scm and it=E2=80=99ll fill /tmp/substitute-test with 200 substitutes, which sho= uld be equivalent to the kind of stress test you had above. It doesn=E2=80=99t crash for me. There are a few =E2=80=9Cerror: no valid = substitute for /gnu/store/=E2=80=A6=E2=80=9D errors, but these are expected: was ask f= or substitutes for 200 packages without first checking whether substitutes are available. Could you run it and report back? You can try with more packages, different substitute URLs, etc. TIA! Ludo=E2=80=99. --=-=-= Content-Type: text/plain Content-Disposition: inline; filename=substitute-stress.scm Content-Description: the stress test (use-modules (guix) (gnu packages) (guix scripts substitute) (guix grafts) (guix build utils) (srfi srfi-1) (ice-9 match) (ice-9 threads)) (define test-directory "/tmp/substitute-test") (define packages ;; Subset of packages for which we request substitutes. (take (fold-packages cons '()) 200)) (define (spawn-substitution-thread input urls) "Spawn a 'guix substitute' thread that reads commands from INPUT and uses URLS as the substitute servers." (call-with-new-thread (lambda () (parameterize ((%reply-file-descriptor #f) (current-input-port input)) (setenv "_NIX_OPTIONS" (string-append "substitute-urls=" (string-join urls))) (let loop () (format (current-error-port) "starting substituter~%") ;; Catch "no valid substitute" errors. (catch 'quit (lambda () (guix-substitute "--substitute")) (const #f)) (unless (eof-object? (peek-char input)) (loop))))))) (match (pipe) ((input . output) (let ((thread (spawn-substitution-thread input %default-substitute-urls))) ;; Remove the test directory. (when (file-exists? test-directory) (for-each make-file-writable (find-files test-directory #:directories? #t)) (delete-file-recursively test-directory)) (mkdir-p test-directory) (parameterize ((%graft? #false)) (with-store store ;; Ask for substitutes for PACKAGES. (for-each (lambda (package n) (define item (run-with-store store (package-file package))) (format output "substitute ~a ~a/~a~%" item test-directory n)) packages (iota (length packages)))) (format #t "sent ~a substitution requests...~%" (length packages)) (close-port output) ;; Wait for substitution to complete. (join-thread thread))))) --=-=-=--