From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id gMJCG9+b3GCn+QAAgWs5BA (envelope-from ) for ; Wed, 30 Jun 2021 18:29:19 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id KNzpFt+b3GDLQQAAbx9fmQ (envelope-from ) for ; Wed, 30 Jun 2021 16:29:19 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id B0D1F1982A for ; Wed, 30 Jun 2021 18:29:18 +0200 (CEST) Received: from localhost ([::1]:53310 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lyd5Q-0001aT-KH for larch@yhetil.org; Wed, 30 Jun 2021 12:29:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:49908) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyd3G-0006s9-7v for bug-guix@gnu.org; Wed, 30 Jun 2021 12:27:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:47041) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lyd3F-0004nR-Vy for bug-guix@gnu.org; Wed, 30 Jun 2021 12:27:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lyd3F-0006yb-Nc for bug-guix@gnu.org; Wed, 30 Jun 2021 12:27:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#48903: guix substitute: error: TLS error in procedure 'read_from_session_record_port': Error decoding the received TLS packet. Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Wed, 30 Jun 2021 16:27:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 48903 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 48903-submit@debbugs.gnu.org id=B48903.162507040126783 (code B ref 48903); Wed, 30 Jun 2021 16:27:01 +0000 Received: (at 48903) by debbugs.gnu.org; 30 Jun 2021 16:26:41 +0000 Received: from localhost ([127.0.0.1]:58585 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lyd2u-0006xu-VE for submit@debbugs.gnu.org; Wed, 30 Jun 2021 12:26:41 -0400 Received: from mail-qk1-f180.google.com ([209.85.222.180]:33570) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lyd2t-0006xh-3J for 48903@debbugs.gnu.org; Wed, 30 Jun 2021 12:26:39 -0400 Received: by mail-qk1-f180.google.com with SMTP id f6so3088922qka.0 for <48903@debbugs.gnu.org>; Wed, 30 Jun 2021 09:26:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=GY6Xgfd4lptmMIISlMTM7lvwqAssU+3rQKhHYcyVNvA=; b=gvCeNHsYLhinc8Hsr3UgSDtaZMQkzeZDNJgZhlvXhYIuI3W1nth0TSOxQ+aZU6TjS4 p2y7iQAMT+n6hAuNPTaiIhXPS3hsrhycxM5Z/T8dHQokl9yjtA2gFFWxSUSrB7iVkW59 pt5Q0O+7+R4LOWc25jp28JnKvpB77srBteoqCz4s7urnd8R008kXyfquD/RzNGODc6Qc iRosS9brWuo03LVIdznGA09TJRpwK6R3LBaIl8YB7g5mm9nEE+6dEHEDh84qgo11Qenw q9QdwNOaghtqwEeqAREnz19QjTaihKoJWgqrO0/whX/nZHTNMJj5VjlXycZB0UcWO7Ma QCRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=GY6Xgfd4lptmMIISlMTM7lvwqAssU+3rQKhHYcyVNvA=; b=IgpyJ7g6NoZMQ1juASI8APcA8ka2JJUre+ceDKNTfExT6qYtY4qxLRhlSB5Z8FhUGZ NkNgGi9cKziZscuUoIs18E2uDS7T1sSx6syLqrez/U4nkSXtzIdhxNrtaDzncenRDNMl nvjuGWXRTGZ56SawmN+b/uYlmpOxNLhiM0KUdqBLotdd3y6INAgXJSIDe65ur+ANrDBj 4zkFy0tA1QUHCTelf+FF8z0wc0ruDLI9sDBti9s832e+i8GKbksvXypTyHd1bjTmcvBA ZH+qmOOFhlUppImwes8dzdcJ2pntE/KNHCXq7kYmOJYfi6CUnpdOiSQA/GWZE7mkeGIQ xKsw== X-Gm-Message-State: AOAM5307Wu5m3Xw/atOSWMilv2B/K82zOKAU+fKKKlruyyxKy58yeQw6 c9zDwiyhv8ICa7UBMQVaggcMpUykevpnpQ== X-Google-Smtp-Source: ABdhPJyvmD5NLQ1bAKAQN5FN/UCeM6wKul6fPXQwT7e+HZSkTNaU0Als3kYxYAHBFUzh9+ac5BET+A== X-Received: by 2002:a37:bce:: with SMTP id 197mr37992113qkl.423.1625070393364; Wed, 30 Jun 2021 09:26:33 -0700 (PDT) Received: from hurd (dsl-150-149.b2b2c.ca. [66.158.150.149]) by smtp.gmail.com with ESMTPSA id y137sm14279173qkb.27.2021.06.30.09.26.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 30 Jun 2021 09:26:32 -0700 (PDT) From: Maxim Cournoyer References: <87bl8hvhgx.fsf@gmail.com> <87wnr05tfz.fsf@gnu.org> <87o8bovm7t.fsf@gmail.com> <87sg10pdxs.fsf@gnu.org> Date: Wed, 30 Jun 2021 12:26:32 -0400 In-Reply-To: <87sg10pdxs.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Tue, 29 Jun 2021 23:18:39 +0200") Message-ID: <871r8j5nev.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 48903@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1625070558; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=GY6Xgfd4lptmMIISlMTM7lvwqAssU+3rQKhHYcyVNvA=; b=UVF77C+PKxy7qnkQheQkwwBOSQLPr5tiGF3insDNBYNrTvHlt+3DGZvLvlRKzZ8LmS0das uRHooH4voAqwJOj3spaGR47Zqxd/bY/Qvuv+58F19MzcEosvS1vZlmx2n5reV6Mfy/bLT9 tPsUUgksI5TVajDrWULDxrD7PBp1fGGnuGnv8doySuTB8k0cAvwYen+eXxCmHgeLeerVn8 CcPO4+VoYoHacGjWWfSLbFSurMX6FR/afaA3HZ4aULFFux8bYYg7AIwdJledH8hVou6UE7 qbfZ8x4ujaYL2QKRG3aFE8eXwP+0in9ik7hrj8dArY3b37wps6S5IOyZpXyrgg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1625070558; a=rsa-sha256; cv=none; b=p1l6aPQc/d+LFbLcEbeMRvrzTd996GYVsTV3BJ4tvRt/X2OWRGFl9p9WmNtv0dU/50mvc/ nffgW/+P0CBOctGAiQ8VfKn4O3nqB2ChteRJj0ht3QQfDTvtefb+AzxQIbhr7L/vlSHTGj JrPHpEd9zRGs46/nAU2K1aKC4HtM4WuRAf+HJCde3ojqifx8N0FOJHixugxS+ZiaKXSVg5 MkHJ2e5djksP6LOouiVMi47jP3cwRo1aVs3ugIr85yjry6xnCS8FC9y+OGArbIH5HBrFpO C8UNm5VAAU4d+WCTSFhNTFbF2S1E2pEMKInbJs6ZBs2QIOeF1ZcgC2tKJ3VT7Q== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20161025 header.b=gvCeNHsY; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Spam-Score: 0.18 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20161025 header.b=gvCeNHsY; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: B0D1F1982A X-Spam-Score: 0.18 X-Migadu-Scanner: scn0.migadu.com X-TUID: VCgsgXYx5i5+ Hello! Ludovic Court=C3=A8s writes: > Hi, > > Maxim Cournoyer skribis: > >> $ sudo ps -eF | grep guix-daemon >> root 25193 216 0 3074 1524 3 Jun28 ? 00:00:00 >> /gnu/store/vphx2839xv0qj9xwcwrb95592lzrrnx7-guix-1.3.0-3.50dfbbf/bin/gui= x-daemon >> 25178 guixbuild --max-silent-time 0 --timeout 0 --log-compression >> none --discover=3Dno --substitute-urls http://127.0.0.1:8080 >> https://ci.guix.gnu.org --max-jobs=3D4--8<---------------cut >> here---------------end--------------->8--- >> >> I can rather easily (and annoyingly!) trigger the problem (and a few >> variations of it, it seems) with something like: >> >> $ packages=3D$(guix refresh -l protobuf | sed 's/^.*: //') >> $ guix build -v3 --keep-going $packages >> >> For example, running the above, I just got: >> >> guix build: error: corrupt input while restoring archive from #> file 7fc95acfc2a0> >> --8<---------------cut here---------------end--------------->8--- >> >> Does the above commands succeed on the first time on your end? If you >> have already lots of things cached, you can try for an architecture you >> don't often build for by adding the '--system=3Di686-linux' option; that >> should cause a massive amount of downloads, likely to trigger the >> problem. Perhaps also try to use --max-jobs=3D4. > > I=E2=80=99ve tried that, with --max-jobs=3D4, and it fills my disk just f= ine. :-/ > >> If you have ideas of how to debug this when I hit the issue I'm all ears >> :-). > > The attached patch substitutes a number of store items in a row; run: > > guix repl -- substitute-stress.scm > > and it=E2=80=99ll fill /tmp/substitute-test with 200 substitutes, which s= hould > be equivalent to the kind of stress test you had above. > > It doesn=E2=80=99t crash for me. There are a few =E2=80=9Cerror: no vali= d substitute > for /gnu/store/=E2=80=A6=E2=80=9D errors, but these are expected: was ask= for > substitutes for 200 packages without first checking whether substitutes > are available. > > Could you run it and report back? > > You can try with more packages, different substitute URLs, etc. > > TIA! > > Ludo=E2=80=99. [...] I've tried with the following modified version which runs multiple threads in parallel (to mimic --max-jobs=3D4 on the daemon), and I've yet to trigger it, although the hard drive is grinding heavily: --8<---------------cut here---------------start------------->8--- (use-modules (guix) (gnu packages) (guix scripts substitute) (guix grafts) (guix build utils) (srfi srfi-1) (ice-9 match) (ice-9 threads)) (define test-directory "/tmp/substitute-test") (define max-jobs 4) (define packages ;; Subset of packages for which we request substitutes. (append (map specification->package '("libreoffice" "ungoogled-chromium" "openjdk" "texmacs")) (take (fold-packages cons '()) 1000))) (define (spawn-substitution-thread input urls) "Spawn a 'guix substitute' thread that reads commands from INPUT and uses URLS as the substitute servers." (call-with-new-thread (lambda () (parameterize ((%reply-file-descriptor #f) (current-input-port input)) (setenv "_NIX_OPTIONS" (string-append "substitute-urls=3D" (string-join urls))) (let loop () (format (current-error-port) "starting substituter~%") ;; Catch "no valid substitute" errors. (catch 'quit (lambda () (guix-substitute "--substitute")) (const #f)) (unless (eof-object? (peek-char input)) (loop))))))) (for-each (lambda (job) (match (pipe) ((input . output) (let ((test-directory* (string-append test-directory "-" (number->string job))) (thread (spawn-substitution-thread input %default-substitute-urls))) ;; Remove the test directory. (when (file-exists? test-directory*) (for-each (lambda (f) (false-if-exception (make-file-writable f))) (find-files test-directory #:directories? #t)) (delete-file-recursively test-directory*)) (mkdir-p test-directory*) (parameterize ((%graft? #false)) (with-store store ;; Ask for substitutes for PACKAGES. (for-each (lambda (package n) (define item (run-with-store store (package-file package))) (format output "substitute ~a ~a/~a~%" item test-directory* n)) packages (iota (length packages)))) (format #t "sent ~a substitution requests...~%" (length packages)) (close-port output) ;; Wait for substitution to complete. (join-thread thread)))))) (iota max-jobs)) --8<---------------cut here---------------end--------------->8--- I wonder if there's something more happening in the real scenario (validating signatures when putting things in the store? or something similar) that may have a role in the failure. That's a tough nut to crack! I'll keep looking for clues. Thanks for your time! Maxim