From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id KFqeNEJTZGO+AQEAbAwnHQ (envelope-from ) for ; Fri, 04 Nov 2022 00:48:18 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id qDOaNEJTZGMgLwAA9RJhRA (envelope-from ) for ; Fri, 04 Nov 2022 00:48:18 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 2A718330EA for ; Fri, 4 Nov 2022 00:48:18 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oqjwS-0001uC-Sj; Thu, 03 Nov 2022 19:48:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqjwK-0001o1-Sw for bug-guix@gnu.org; Thu, 03 Nov 2022 19:48:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oqjwH-0000id-U3 for bug-guix@gnu.org; Thu, 03 Nov 2022 19:48:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1oqjwH-0003K9-Qj for bug-guix@gnu.org; Thu, 03 Nov 2022 19:48:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#58290: guile ssh error on guix deploy Resent-From: Marius Bakke Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 03 Nov 2022 23:48:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 58290 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 58290@debbugs.gnu.org, Andrew Tropin Received: via spool by 58290-submit@debbugs.gnu.org id=B58290.166751924012596 (code B ref 58290); Thu, 03 Nov 2022 23:48:01 +0000 Received: (at 58290) by debbugs.gnu.org; 3 Nov 2022 23:47:20 +0000 Received: from localhost ([127.0.0.1]:51022 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oqjvb-0003H6-7K for submit@debbugs.gnu.org; Thu, 03 Nov 2022 19:47:19 -0400 Received: from eggs.gnu.org ([209.51.188.92]:35260) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oqjvW-0003Go-4Y for 58290@debbugs.gnu.org; Thu, 03 Nov 2022 19:47:17 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqjvP-0000Yz-37; Thu, 03 Nov 2022 19:47:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=/73cpX3sJtXofZ9mR3iYKLjlYHbNyASXEZSibS9d+/o=; b=qoCKLpLMe1IPUy9sdKT9 yDAlORLFYtQXfz25fWPn+RgPzPu44DRIxDxvr70Fh8yqVTs8Nn3aZQbr0DEe7BvB9VZwpoqr7qgHl 35ZyqflkmjQztRCyQyur9Kvf7vyxtY5+uvXsQlleJ+EJ0z4fNd7yUwpzqRzwJk+Esvanir6tCbcle P4NSSZ66udaePASPEd6tEeujSm7Fun9Q6PQDatkJnm/e58RoIOAcbhrXNBfLrDraQJhKDaSGwkj8j 32JJnWbynXMbX3GpLl2sXjCJKy2tDURp091KS7Z1vazKKHOrLFGBOJZgNnYe9KmXD5gVZRuRktL75 LTK+TDdKddBkow==; Received: from [84.214.173.6] (helo=localhost) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oqjvN-00085S-Pk; Thu, 03 Nov 2022 19:47:06 -0400 From: Marius Bakke In-Reply-To: <87y1tdkv1d.fsf@gnu.org> References: <87sfk34zhj.fsf@trop.in> <87bkqm8p5z.fsf@gnu.org> <87o7uec39j.fsf@trop.in> <8735bmsee9.fsf@gnu.org> <87h702uzrb.fsf@gnu.org> <87y1tdkv1d.fsf@gnu.org> Date: Fri, 04 Nov 2022 00:47:02 +0100 Message-ID: <875yfv38jt.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "bug-Guix" Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1667519298; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=/73cpX3sJtXofZ9mR3iYKLjlYHbNyASXEZSibS9d+/o=; b=thMDj1EcG9rpJW9hWMPRyMgnl3uIrmra2W3vtwwkwRPZE5HfkeHf6QiGTdQDeutTU9xuho TDvBJ5/hSoq5oFwO4EK8d5Lj+bEu1USAmtTIHv6M3VgtF5j117xn8ub61+1UAam/ozJiMT RxuRMwuNtrzLU0UR5dLyH8tSMCDRdVHYuxK6lhKSPfrGbyt91z7vwsjVFODlvrL4NVOfPW 3NRAf9VvprTaB1XZJV9Kl89T+TEc4RA2vT1BvZFBk/LNFwU5fTNvZxi8dXvS19h7znQFjE e0T2I/wNzZZINA+lP3WulGX7mxkRpeOjaSWxauFiBGcEoBQNTCqu3VlK8sxqsw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1667519298; a=rsa-sha256; cv=none; b=o8S0WPVdxSmuOS21AlpQgJle+I97bbyZhh1t0VLVHoWrHIAbJu2P+Os1m0yPxh9E01t/uG NWxdGNlThrRJpJiqBLA5cWKUXDSkr9zusDO3mQOFn6p+BfcLRk9HGOuljzfcHhqbzgJlhy E8t1jHA7eid7Itjmf3uCVDiJJHDAMZ3h9dEQ0LO/oVpVD/6VCJznunwFeH5XDAFprQgyQK Jon7vwfz3YFWr+kxH+PxQCks4+H+L1EY2UmyhRYsuBhOKrhxjrqw0EB2+BCk9Dw5AiEkF3 Vw7mEcDy+KshDWeqOzv0L2Qpugz79nf08YYe7/wVpQildEzVNQ2Xm1+HTl12HQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=qoCKLpLM; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.49 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=qoCKLpLM; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 2A718330EA X-Spam-Score: -4.49 X-Migadu-Scanner: scn1.migadu.com X-TUID: /vTo1ghp7/9S --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Ludovic Court=C3=A8s skriver: > Hi, > > Marius Bakke skribis: > >> I have the same problem. Log messages around failure look something >> like this (extracted from above message): >> >> Oct 4 11:51:49 localhost shepherd[1]: Service sshd-2 has been started= .=20 >> Oct 4 11:51:50 localhost sshd[250]: Accepted publickey for bob from 1= 85.70.53.164 port 1915 ssh2: RSA SHA256:/X3jyhyMizAOKOjCfXK5cLN3Pa0vmi7dbQG= +fxK3d3Y >> Oct 4 11:52:28 localhost sshd[252]: error: no more sessions >> Oct 4 11:52:28 localhost shepherd[1]: 1 connection still in use after= sshd-2 termination.=20 >> Oct 4 11:52:28 localhost shepherd[1]: Service sshd-2 has been disable= d.=20 >> >> Then deploy crashes with 'Channel opening failure'. > > =E2=80=9Cno more sessions=E2=80=9D comes from this: > > --8<---------------cut here---------------start------------->8--- > int > session_open(Authctxt *authctxt, int chanid) > { > Session *s =3D session_new(); > debug("session_open: channel %d", chanid); > if (s =3D=3D NULL) { > error("no more sessions"); > return 0; > } > > [...] > > } > --8<---------------cut here---------------end--------------->8--- > > Would be interesting to run sshd in verbose/debug mode and see why we > hit that; it could be because the maximum number of sessions has been > reached or something. I enabled (log-level 'debug) and this is what happens server side: Nov 3 21:48:25 capella sshd[26345]: debug1: permanently_set_uid: 0/0 Nov 3 21:50:27 capella sshd[26115]: debug1: Received SIGCHLD. Nov 3 21:50:27 capella sshd[26115]: debug1: session_by_pid: pid 26345 Nov 3 21:50:27 capella sshd[26115]: debug1: session_exit_message: session = 8 channel 8 pid 26345 Nov 3 21:50:27 capella sshd[26115]: debug1: session_exit_message: release = channel 8 Nov 3 21:50:28 capella sshd[26115]: debug1: server_input_channel_open: cty= pe session rchan 65 win 64000 max 32768 Nov 3 21:50:28 capella sshd[26115]: debug1: input_session_request Nov 3 21:50:28 capella sshd[26115]: debug1: channel 0: new [server-session] Nov 3 21:50:28 capella sshd[26115]: debug1: session_new: session 0 Nov 3 21:50:28 capella sshd[26115]: debug1: session_open: channel 0 Nov 3 21:50:28 capella sshd[26115]: debug1: session_open: session 0: link = with channel 0 Nov 3 21:50:28 capella sshd[26115]: debug1: server_input_channel_open: con= firm session Nov 3 21:50:28 capella sshd[26115]: debug1: server_input_channel_req: chan= nel 0 request exec reply 1 Nov 3 21:50:28 capella sshd[26115]: debug1: session_by_channel: session 0 = channel 0 Nov 3 21:50:28 capella sshd[26115]: debug1: session_input_channel_req: ses= sion 0 req exec Nov 3 21:50:28 capella sshd[26535]: debug1: PAM: reinitializing credentials Nov 3 21:50:28 capella sshd[26535]: debug1: permanently_set_uid: 0/0 Nov 3 21:50:28 capella sshd[26115]: debug1: server_input_channel_open: cty= pe session rchan 66 win 64000 max 32768 Nov 3 21:50:28 capella sshd[26115]: debug1: input_session_request Nov 3 21:50:28 capella sshd[26115]: debug1: channel 10: new [server-sessio= n] Nov 3 21:50:28 capella sshd[26115]: debug1: session_open: channel 10 ["no more sessions" error occurs here, in a different log file] Nov 3 21:50:28 capella sshd[26115]: debug1: session open failed, free chan= nel 10 Nov 3 21:50:28 capella sshd[26115]: debug1: channel 10: free: server-sessi= on, nchannels 11 Nov 3 21:50:28 capella sshd[26115]: debug1: server_input_channel_open: fai= lure session Nov 3 21:50:28 capella sshd[26115]: debug1: channel 0: free: server-sessio= n, nchannels 10 Nov 3 21:50:28 capella sshd[26115]: debug1: channel 1: free: server-sessio= n, nchannels 9 Nov 3 21:50:28 capella sshd[26115]: debug1: channel 2: free: server-sessio= n, nchannels 8 Nov 3 21:50:28 capella sshd[26115]: debug1: channel 3: free: server-sessio= n, nchannels 7 Nov 3 21:50:28 capella sshd[26115]: debug1: channel 4: free: server-sessio= n, nchannels 6 Nov 3 21:50:28 capella sshd[26115]: debug1: channel 5: free: server-sessio= n, nchannels 5 Nov 3 21:50:28 capella sshd[26115]: debug1: channel 6: free: server-sessio= n, nchannels 4 Nov 3 21:50:28 capella sshd[26115]: debug1: channel 7: free: server-sessio= n, nchannels 3 Nov 3 21:50:28 capella sshd[26115]: debug1: channel 8: free: server-sessio= n, nchannels 2 Nov 3 21:50:28 capella sshd[26115]: debug1: channel 9: free: server-sessio= n, nchannels 1 Nov 3 21:50:28 capella sshd[26115]: debug1: do_cleanup I compared this with a successful deploy: Nov 3 21:44:22 capella sshd[25842]: debug1: permanently_set_uid: 0/0 Nov 3 21:46:25 capella sshd[25612]: debug1: Received SIGCHLD. Nov 3 21:46:25 capella sshd[25612]: debug1: session_by_pid: pid 25842 Nov 3 21:46:25 capella sshd[25612]: debug1: session_exit_message: session = 6 channel 6 pid 25842 Nov 3 21:46:25 capella sshd[25612]: debug1: session_exit_message: release = channel 6 Nov 3 21:46:26 capella sshd[25612]: debug1: server_input_channel_open: cty= pe session rchan 65 win 64000 max 32768 Nov 3 21:46:26 capella sshd[25612]: debug1: input_session_request Nov 3 21:46:26 capella sshd[25612]: debug1: channel 0: new [server-session] Nov 3 21:46:26 capella sshd[25612]: debug1: session_new: session 0 Nov 3 21:46:26 capella sshd[25612]: debug1: session_open: channel 0 Nov 3 21:46:26 capella sshd[25612]: debug1: session_open: session 0: link = with channel 0 Nov 3 21:46:26 capella sshd[25612]: debug1: server_input_channel_open: con= firm session Nov 3 21:46:26 capella sshd[25612]: debug1: server_input_channel_req: chan= nel 0 request exec reply 1 Nov 3 21:46:26 capella sshd[25612]: debug1: session_by_channel: session 0 = channel 0 Nov 3 21:46:26 capella sshd[25612]: debug1: session_input_channel_req: ses= sion 0 req exec Nov 3 21:46:26 capella sshd[26032]: debug1: PAM: reinitializing credentials Nov 3 21:46:26 capella sshd[26032]: debug1: permanently_set_uid: 0/0 Nov 3 21:46:26 capella sshd[25612]: debug1: server_input_channel_open: cty= pe session rchan 66 win 64000 max 32768 Nov 3 21:46:26 capella sshd[25612]: debug1: input_session_request Nov 3 21:46:26 capella sshd[25612]: debug1: channel 8: new [server-session] Nov 3 21:46:26 capella sshd[25612]: debug1: session_new: session 8 Nov 3 21:46:26 capella sshd[25612]: debug1: session_open: channel 8 Nov 3 21:46:26 capella sshd[25612]: debug1: session_open: session 8: link = with channel 8 Nov 3 21:46:26 capella sshd[25612]: debug1: server_input_channel_open: con= firm session Nov 3 21:46:26 capella sshd[25612]: debug1: server_input_channel_req: chan= nel 8 request exec reply 1 Nov 3 21:46:26 capella sshd[25612]: debug1: session_by_channel: session 8 = channel 8 Nov 3 21:46:26 capella sshd[25612]: debug1: session_input_channel_req: ses= sion 8 req exec Nov 3 21:46:26 capella sshd[26043]: debug1: PAM: reinitializing credentials Nov 3 21:46:26 capella sshd[26043]: debug1: permanently_set_uid: 0/0 Nov 3 21:46:28 capella sshd[25612]: debug1: chan_shutdown_extended_read: c= hannel 8: (i0 o3 sock -1 wfd -1 efd 13 [read]) Nov 3 21:46:28 capella sshd[25612]: debug1: server_input_channel_open: cty= pe session rchan 67 win 64000 max 32768 Nov 3 21:46:28 capella sshd[25612]: debug1: input_session_request Nov 3 21:46:28 capella sshd[25612]: debug1: channel 9: new [server-session] Nov 3 21:46:28 capella sshd[25612]: debug1: session_new: session 9 [... more channels and stuffs before graceful shutdown ...] I believe the two-minute window before SIGCHLD is waiting for some Shepherd services to time out (long story..!). In my current tests the failure always occur at the very end of deploy: building /gnu/store/19yajyzw4jhnkkz9w0l9gm4az0jxihkc-install-bootloader.scm= .drv... building /gnu/store/m7ngq5gszyswm6sficinz7ghpra30dl4-remote-exp.scm.drv... ;;; [2022/11/03 21:50:28.892606, 0] [GSSH ERROR] Channel opening failure: c= hannel 66 error (2) open failed: # What's interesting is that all of the failed deploys fail to open "channel 10", whereas successful deploys never reach it. All of these are deploying the exact same configuration and commit, so the SSH traffic should nearly identical. I have attached the transcripts of each session (by grepping the PIDs from /var/log/debug). Grepping for session_open yields some inconstencies, but haven't studied them in detail yet. >> Due to the number of SSH connections made by deploy and frequent >> occurence of this problem, I was not able to successfully deploy through >> many attempts. > > Ouch. Normally, thanks to memoization, =E2=80=98guix deploy=E2=80=99 ope= ns only one > session per machine. (I think it works well with the 25 local build > machines behind berlin; it=E2=80=99s also fine for a small set of machine= s I > take care of at work.) Have they been rebooted since the switch to inetd-style sshd? It fails more often than not for me. >> I found that removing the memoizing open-machine-ssh-session* in >> (gnu machine ssh) works around it and can happily deploy again. >> >> (that is, just use 'open-machine-ssh-session' instead) > > Interesting. That gives us a hint. Could you add a =E2=80=98pk=E2=80=99= in > =E2=80=98open-machine-ssh-session*=E2=80=99 to see how many connections i= t opens? Indeed it only opens a single connection. The problem seems to be with multiple "channels" and "sessions" over a single connection. --=-=-= Content-Type: application/gzip Content-Disposition: attachment; filename=deploy-fail.log.gz Content-Transfer-Encoding: base64 H4sICDlQZGMAA2RlcGxveS1mYWlsLmxvZwDtXWtz2zYW/d5fgemXdDt+iNRbs9vZNHGbTBsnE2d2 k+10ODQJW5xQJENSfuTX94KyZIoRiYsL0FJ3+CW2Yulc4AAHOBeXhs/jG8b6zLZmg8msN2Gem/Aw dFmWzf0/7JFlDf+cMZ9fLq+tGQsinvssi73PPM+Ye5XzlPnLJAmi6xnrH7HBd+cqcL/HnhuyG55m QRyxLE8Bh11cvDq2T3rHbxMewffO9MRSQ33PF3HOWZLGeezFj/gAesTS1Q+z+Cq/dVO++WEYXAKY 0zuZnozUwnnxInFz59KNIp7OWBSzhZt785kGZMLThRvxKA/vnYznzjLwZ2w6mZ5OJ332R5Jyd5nP /1TDDIMsd+Zxln/m905+n/BsxtLMPc7mrn08tOyjzQt7ODrinr9+GcEHE/F/AHvMfXs4tKbURsB4 2s6bi1+d384+vj5//YFl0EljYCn3eHDDfSrgZ343Y254HadBPl/AyC7TG170t2BiONICFtzDd/fl CAYoLbC9MAAij3/KOLQ4ZV6QzMVcdHkGrT6+9hb/jkFN8PkTmK3szfMXM/bPYJGEgRfkPxUzOOWZ 0MGMfYVpu/VunXat2nP806p9B9EufpdwL39YaDbzxzl78fKVU8wiA7OxhKY7J1Mupky8zB9W24E9 HUxHY3s6YpchrMOZdnvPz/7729mnCz0p8sjfovTs4wfo/y9vDY7Sup2m+mtmZILI9MDA/GF+HHHq 55cgOvGx45R/WXJYc67itPhPlsZxXkgy8Hix9Hgx7FlAM+x/C57PY58ly0sQn+gYMbqb53yR5KzH rtwgXIJ84Vsi1rvnb4TlCPLADYOvYjKIrnwvuvE9AQl20mJGwQvn/au3Fx9YHrPvf3x/9vL5iw9n L3/Uxfzw4VOBCO9WhFoPmQP0A/ez0jDkYgSTz7BnbG0WyWeYZJfs7OXq5cWr57Cgzt6dXszfXU6+ BO+T/41e5uHVf8Yff3k+/fTz7bl3Nr9K0i8fR19+dpfnFzF1TMToxqmbBmBNoNkra9I77bEf+L/g yz8U0dJ7wd6qu8XmeBWEnJ2KIT49gY+ciibCbvmV+w78OFODv/LZEPZG7hbG8q1z/vb859/fvvhN VehZHqePXf1bEWar4b+Il6EPHjZnYqtdLRvPBPgz9ggs4mbsWUPQZzN2HrNs6c1X7QPZ+gEsuMDj /f8f+Tz3TnewcOIXDO1jykqaNIO3FEmKiFH0I15GwAx9OTHevML1JGJrgjzFvRbuFhZ/SNkKnwGD uv06vy/m6nHqsTvLKv3oKWfbYey8loGdt1vlD3Kh6Vb5bpXvVvk9r/KFUXdh7b5xc+5sWr/OBCJ+ W7wF+hJ4brHCP7xHLYwfO4m7gEAejFo+gzeHIffVMBYxJE9x6sAUCH3nYSuYlbakUkNB8Jf3LEmD G5jt1/AqSWOPZ4qtJpND3anOIhd0CtClwyLYClnIb3jIRidU3DV18GHfCeNrGABBIoNvhXq9MM5U R6NIGGHqiQZnhSi9lPuCCjdUpFl29mD4jEkNThx6J64oERSH10kMXS7mHWUKy8ZXcXqKmcn3te6c RcCuAAzEN6u2gDRWfRJeUZxRKfZodcLqiFMSxw+yRKz3RIRkmcNKIcoYoSN8AQyYqBNsWpiKH7JB n93CxBsNer0ebC93rG+PRxO1kKtYD7jOg1tWrLqsGsp6s2JR+ePhpPkBU1Hs65YA0uYFU9wW1hgP zK2bpwOyaYmo3kSfgfd8TkeuH+U4ugrSxTpcBXdKwIURLVHA1ukQv+MevEjCe1Yt6UmjrHi5vF/H KLFTSwkSdEfbH7HhZdFuNWgB5mTzZe7Ht5HD73Ie+eB+xGayNXV/CHos7hcFVXZssVvYWeALhy8W 5G7i7X9W3es+mSp2vS3ox0FebY/ik2Pj4DM2d7NVhCOW5/cFTR0vZnhRW/0HiNVfEhKz+iP0JRiw UKs/kuit1Z84vbZXfy2QTUt2rf4GR1lv9e9U1fHS8dLx0g4v79c1+ovXv7549fvLanKEpzURZz3w j05f+R0kWQt4ARnjzvExGyDlIXczrjv8e5pTisClWXOVcr55dOgh4BGLHt6Rsepht3aGYrWcoVi6 hqAhQ7FqMhTIH/QzFIuSocgik5jCgtbPS+txrbPHxsERa13HC40XyR6gQGt5D6D2dfceUBofswGq e8Dfa0711Nekh1kj2wOUm6yW5Q7lWa4sJCLLxfKBOuPEjlpTlovFaDzjVANpzHKVkWlZrmU9xRmn PApGq1VKkKAUB0FhpUFTI4SmJCExmpJAKJ0cYdltqhtgMRpPjtRAGusGysj70hTGletoqqdLNqFu YNmtu/J+jSuXRFZV8xihZklIjJoRfAlKbJSapRTsULNNw9hWsxbIpiW71KyMTFQzssVqOkOConKX wdQ4OCZ36XjpeOl4aZ0X2RkAntbyGQC1r7JzYMtsANkZwGHPqaodRu7oiDOAvoG9b8tx2hjHqcG2 rWsIGhynTXSc7ShrpDjqzcoq8WY2QFVZisOCcuukp3xMu/XJQbl1XO5NcevEWdGce5PcOir3PkC3 Xn/SpL0jlKrNfcXJZ6ba3PHS8dLx0j4v7XgKal9lFbue2QCSpzYOfE4R3TriqQ3jbh11PrzP3Ihy PjzQU5bk4zXKGlc7iIRB58FmAsjyYHznFcYaCaqeB0uAFfLgatYibbJa7jBF5A6SkJjcAckHrhaO HLXG3AGJ0VwLVwJpzB2UkYm5AwVXvRZuRKtEsltYlxU1NewdlKZw+ThFU8Rhb87HSZra+XyJMvK+ NIXyOhqaqn8WTF9Ttc+XDPdWC5dEVlWzhVCzJCRGzQi+BCV9lJqlFOxQc9XCIzG21awFsmnJLjUr IxPVjGyxms6QoKga3tQ8OOZUoOOl46XjpXVeZGcAeFrLZwDUvqKfhzcTQHYGcNhzqpqwIHd0xBnA wMDet+U4+xjHqcF2X9cQNDjOPtFxtqIsu1c9n9FSVok3swGqylIclhZr4Ybdun1Qbh2Xe1PcOlGs zbk3ya2jcu8DdOv1J03aO0KphjcxD96y++p46XjpeNmnW6f2FX2DgZkAslr4Yc8poltH1MKNu3XU +fA+c6MWzofbcesWMR3B5sGGAsjyYLO3fg0Rdz7KQmI8M/6OEIRnptz6VZU86dYvLRClGnBLt36R cNVrwG3cX4MFJdSAbbN3DAwRN+nJQiI0JYNY8zjAaEpOwQ5NDWgY25rSAtm0ZJemlJFpmiLhbmlq gNAUlpddmhrokt2gqUGdphC/rS89VRpQasCyyKpqRtwYIguJUTPydoMhSs2U2w2GNIxtNWuBbFqy S83KyEQ1a9w7UK8zJGh95jLYZMN23zw4IhvueOl46XhpnxdJ7qtAazn3pfZ1d+5bGh+zAaq5LxF0 T3OqR9vRB/JTpZGBvW/LcQ4xjlOD7aGuIWhwnEOi42xHWQNiB3crq8Sb2QBVZSmCtlcDNu3WEXeR PaFbx+XeFLdOXMWac2+SW0fl3gfo1mtPmvR3hMfalW33jYO37L46XjpeOl726SmofcXWgA0FkNSA D3xOEd06ogZs3K2jzof3mRu1cD7cjrKGymOjmAebCaCdB6t5ZsSNn0/omVE1YJJnJq4ljTVgmmfG 1ID355lN1ID3uvpTasCGNYW4l+sJNTVqTVPE9W5bU1ogm5bs0pSJvfhJNDVqWVMjXbIbNDWq05SJ W/ZGpBqw2Vv2hoibMmQhMWpG3skzRqmZcsvemIaxrWYtkE1LdqlZGZmoZo27hep1pn2T0+gxG64+ xGsAHJMNd7x0vHS8tM6LLPcl3bJH7uvu3Lc0PmYDVHNf4vDvaU4pUlCaNbJTJUW7IXecY4zj1GB7 rGsIGhznuM5x9vWUJfl4jbKmNg1mt7JKvJkNUFVWdVgkoC3WgKXdUXLrI8QdXLKQGLeO4AufeyNH tDH3RmI0595KIEq5t8YoN7p1vGgVTpqQoKja1WhqHBzjMjpeOl46XlrnpR1PQe0rugZsJoCsBnzY c0oRWKEGrGg3DJ0Pa7BdnxshQSnnw/tQVr9XffhBS1nf5sGGAsjyYMOeGXHT5RN6ZlwNmOKZiWtJ cw2Y5JlRNeC9eWYjNeB9rv6UGrBhTSHuo3pCTU1a05TyZrdLU1ogm5bs0pSJvfhJNDVpWVMTXbIb NDWp0xTipnLpqdKEVAM2e7P7CHFThiwkRs3Im92nKDVTbnav5gWkm921QDYt2aVmZWSimjXuXK/X mfbfHJlssuH+Nw/x6oNjsuGOl46XjpfWeZHlvqS/B0Xu6+7ctzQ+ZgNUc1/i8O9pTlUTFuSOPpGf KlmK0HLLOcVYTg26p7qOoMFyTomWsx1pffMb8FrSKvFmNkBVWorD0mIR2LBdx1zC9XR2HZd8U+w6 cRlrTr5Jdh2VfB+gXa8/atLeEh6LV32r+gyHPnjL9qvjpeOl42WfnoLaV2wR2FAAWRH4sOcU0a4j isDf2HXt26BRJ8Qat0HXZ0f6t0HvPCEe9mb2mC4txMdrpLV1rY4CDDoTNhOgKRMWoBOzpll21yUi pMw0IyDwVWAUBZIqsAJGfRVYGQRdBdYc5VrTTMVVqwIr8IJf/hVAVavALWhKdiPV02rKMiqqyp8U og0SEyjsyg1CDv5LbKpkxHIvpbuz8jytHXfR9mXKaSqjGgoVOuTQUxqyrfYonAJyX448piErXrio gDyUI1ddARIZ8QtCAxryWI7cpyEjDrRtGvIUIRU1ZD+GTIG70TJR+9y752+gWRqfjLMguqYtHAWC DwYxFxBeyn0e5YEbZoqkxoskhfgsXubXMUCB63Rvme/mLrPs3sCaHm3eAy582usNYZV2vTxOWe+k pyrCdbQggm8r0YbjyXi0FW086k2mpXCDyXd/AVZKy1PX0AAA --=-=-= Content-Type: application/gzip Content-Disposition: attachment; filename=deploy-success.log.gz Content-Transfer-Encoding: base64 H4sICDRQZGMAA2RlcGxveS1zdWNjZXNzLmxvZwDtXWtz2zYW/d5fgcmXdDt+iJREPWY3s3m4TaeN k4kzu+l2MhyahC1OKJIhKT/66wtSlkxxSeLeC9BSO/wSW7F0LnCAA9wHfX0e3TA2ZKYxH43mA4u5 TsyDwGFpuvB+N8eWYX6ZM49frq6NOfNDnnksjdyvPEuZc5XxhHmrOPbD6zkbHrHRd+cYuF8j1wnY DU9SPwpZmiUCh11cvD02TwbH72Meiu/t2YmBQ/3Il1HGWZxEWeRGj/gC9Igl6x+m0VV26yR8+8PA vxRg9uBkdmLhzLnRMnYy+9IJQ57MWRixpZO5i7kCZMyTpRPyMAvu7ZRn9sr35mw2nZ3OpkP2e5xw Z5UtvuAwAz/N7EWUZl/5vZ3dxzydsyR1jtOFYx6PDfNo+0J88oi73uZlKD4Y5/8nYI+5Z47Hxow6 CLGepv3u4if7l7PPP5///ImlYpLawBLucv+Ge1TAr/xuzpzgOkr8bLEUK7tKbngx34KJsaUEnHMv vrsvW9BAaYHtBr4g8vhFysWIE+b68SLfiw5PxduPr93lvyOhJvH5E7Fb2buXr+fsn/4yDnzXz14U Ozjhaa6DOftDbNudd6uMaz2e4xfr8R3EuPhdzN3s4aDZ7h/77PWbt3axizTsxhKa6p5MeL5lolX2 cNqOzNloZk3MmcUuA3EOp8rjPT/77y9nv12oSZGH3g6lZ58/ifn/+F7jKm3GqWu+elbGD3UvjNg/ zItCTv38Sogu/9hxwr+tuDhzrqKk+E+WRFFWSNJ3eXH0uJG4swTN4v5b8mwReSxeXQrx5RMjWney jC/jjA3YleMHKyFf8S0R68PLd7nL4We+E/h/5Jshn8qzfBrPCEjiJi12lHhhf3z7/uITyyL27IeP Z29evv509uYHVcxPn34rEMW7kVCbJbMF/YL7eWkZsnwF46/izti5LOKvYpNdsrM365cXb18KzPmH 04vFh8vpN/9j/D/rTRZc/Wfy+ceXs99e3Z67Z4urOPn22fr2ylmdX0TUNclXN0qcxBeuiRj22jUZ nA7Y9/xf4ss/kGjJfc7eerrF5XjlB5yd5kt8eiI+cpoPUdyWf3DPFj9OcfBXHhuLu5E7hWP53j5/ f/7q1/evf8EKPc2i5HGqfynCTBz+62gVeMKHzVh+1a6Pjec5+HP2CJzbTdnzFqPP5+w8YunKXazH J2Tr+eLAFTze//3I55l7WsPCiVcwtI8tKxnSXLylCFJyG8U8olUomKEfJ9qHV3g9cX41iTjFuc69 W3H4i5Ct8DPEou6+zu6LvXqcuOzOMEo/esrddhg3r6Hh5u1P+YM8aPpTvj/l+1N+z6d84ag74uy+ cTJub0e/iQRCflu8RczFd53ihH94D86MF9mxsxSGXLFq2Vy8OQi4h8NYRiJ4ihJbbIHAsx+ugnnp SioNVAj+8p7FiX8jdvu1eBUnkctT5KjJ5FBvqrPQEToV0KVkkbgKWcBveMCsEyruhjrxYc8Oomux ADmJTHybq9cNohS7GkXAKLZePuC0EKWbcC+nwgmQNMtyD5pzTDi4POkdO3mJoEhex5GYcrHvKFtY tr7I7ZnvTL6vc+csFOzmgH7+zXosQhrrOeW+Yp6jQs5onWG18yyJ7flpnJ/3RIR4lYmTIi9jBHbu F4gFy+sE2xEm+Q/ZaMhuxcazRoPBQFwvd2xoTqwpzuTa1gOu/eAtI6su64Gywbw4VH5/yDQ/YCLF vhmJQNq+YMhrYYPxwNxmeCog25Hk1Zvwq+A9W9CRm1c5Cq/8ZLkxV8GdEHDFipYoYJtwiN9xV7yI g3tWLelJrax5ubzf2Cix00gJELRm7I/Y4mUxbhx0Dmani1XmRbehze8yHnrC+8kvk52t+70/YNGw KKiyY4PdiptFfOHiiyFit/ztX6re6z6ZKm69HejHRV5fj/knx9rB52zhpGsLRyzL7guael708II7 /UeA019iEnL6A/SVM2CATn8g0TunP3F77Z7+SiDbkdSd/hpXWe3071XV89Lz0vPSDS8fNzX6i59/ ev321zfV4AhOa5znesQ/KnPldyLIWooXImKsXR+9BhIecCflqsu/pz2FBC7tmquE8+2jQw8Gj1j4 8I6UVZPdyhGK0XGEYqg6BC0RitEUoUw1RCgGKUKRWKYxBQRt3pfG41lnVuWpDg4563peSLzI7gA4 reU7gDrX+jugtD56DVTvgL/WnqreAYAz6WHXyO6A6pBneqPcMSDKlZiERLkSCFyOU0oBIMoFYrTn OFEgrVEuGpkY5VJw8TlOIC/tWiWSTfEgNGvKOihNwTJHFE0pyaHpSCZpqrZuoPHk7FZTIK9cQVMD VbIJdQPhM3ftlQ/rvXKZZayaJ3I1y0wC1AzhK6fEhKhZTkGNmk0axq6alUC2I6lTMxqZpmboiFE6 g4KCYpeRpR0cELv0vPS89Lx0z4skB4CgtZwDoM5Vlgc29BqQ5AAOfE8NaDc6IAcw1HD37XicJsDj VGHbVHUIWjxOk+hxdqOs8USnskq86TVQVVZ1WYx9PeUjs4z11qcAb11iEuKtA/gCx95yCuSxNxSj NfbGgWBib5VVbvXWgSNGZZqgoKBq83CsHRziZfS89Lz0vHTOi8yngNNa9imoc5VV7AZ6DUie2jjw PYUELu0arLdOufvQ+WEVtptjIyAoJT9sqilL8vEGZf2fKwiEAcfBegzI4mD45BFrDQTFx8ESYEQc XI1apEPGxQ4zQOwgMQmJHYB8gGrh0FVrjR2AGK21cBxIa+yARibGDhRcdC1cj1aJZHdwLiM1NR4c lKZg8ThFU8Rlb4/HSZqqfb4EjbwvTYF8HQVNNT4LpkFTTc+XGMO91cIllrFqNgBqlpiEqBnAV07J EKRmKQU1aq668ECMXTUrgWxHUqdmNDJRzcAR43QGBAXV8KYz7eCQrEDPS89Lz0vnvMhyAHBayzkA 6lzBz8PrMSDLARz2nqoGLMAbHZADGGm4+3Y8ziHE41Rge6jqELR4nEOix9mJsiYD5Kq3K6vEm14D VWVVl2W0t1q4xDLWWzcB3rrEJMRbB/AFj72lFABibyBGe+yNAkHF3gqr3OqtA0eMyzQBQUE1vAly 82mq4fW89Lz0vHTOi8yngNNa9tapcwV3MNBjQFYLP+w9hQRG1MKr3jrl7sPnhxXYbo6NgKCU/PA+ lDUxiBOExsGaDMji4H2uNT4OhvvK2KdM9HY/GwN6X8pMQmIHeK8UQOygofsZFKO9Fq6v+xkamRg7 PEnvSygvuCfCuussY0jbQ+A0BegoKDMJ0RS89wRAU8AOGa3PlwAx2uNxFEjr8yVoZKKmKLh4X0eh L0rzs2BAUMrzJYCuBR3VwvV2eRgDOqfITELUDOzyMAKpmdLloerCk7o8KIFsR1KnZjQyUc1d9F+A gkJqeJNh9elIdXBIVqDnpeel56VzXmQ5ADit5RwAda7Q5+E1GZDlAA57T1XdYXjfJlkOoPprPMqd iEYQj1OB7ZGqQ9DicY6IHmc3yhpV8zNKyirxptdAVVnIZemwFq7ZWwf0ZHtCbx0We2voyUbz1omK J8TeB+itN2ealG+ExxrexNR+3YBqeD0vPS89L53z0o1PQZ0r9PfCNRmQ1cIPe08RvXVALVy7tw7K D+8zNuogP9yNssbEcAQcB+sx0MfBsKdMNMcOkA6wTxc7wGrhe4sdlEBaYwc0clexg5ZauBatEsnu 4FzGagrSp+3vEI8rhdJa4/Ha50sOJh7ft6/T/CxYd393wNxbj3SZZayaAZ1TZCYBaobwlVMyhqhZ TkGNmsc0jF01K4FsR1KnZjQyTc3QEaN0BgUF1fAmE+3ggKxAz0vPS89L97xIcgAIWss5AOpcwc/D 6zEgyQEc+J4a0G50QA7A0nD37XicY4DHqcL2WNUhaPE4x00ep1o/U9nHG5Q1rbqCSv1MS7zpNVBV VnVZ9tYjXT4dlLduAXqyyUxCvHWdPdKhK9oWe0MxWmNvHAgm9lZZ5VZvXUtn2OrBrbG388Sqnt1P 0tu556Xnpeele1668Smoc4X+XrgmA5Ja+IHvKSQwohaOJldHfliF7ebYSL1HemN+eC/Kms10Kqsm DtZjQBYHa/aZAZ1Pn9BnBtWAST4z8SxprQHTfGZIDXh/PrOOGvBeT39CDVi3pgD9yZ5QU1ZnmiJ6 EruaUgLZjqROUzru4ifRlNWxpixVsls0ZTVpCtC5XppVskg1YL2d/i1AxxCZSYiagZ3+JyA1Uzr9 V7P4pE7/SiDbkdSpGY1MVLNCD/5mnSn/DRprGw1PjaqbqQ4OiYZ7Xnpeel4650UW+5L+Phh5rvWx b2l99Bqoxr7E5d/TnqoGLMAb3ZJnlZDuhtzjnEA8TgW2J6oOQYvHOSF6nN0oa0hkrV5ZJd70Gqgq C7ksHdaANXvrgF5kT+itw2JvirdOPMXaY2+Stw6KvQ/QW2/ONCnfCI+1q+kAfXZrqV31vPS89Lx0 zks3PgV1rtAasCYDshrwYe8porcOqAFr99ZB+eF9xkbo/LA1N8cKypJ/vEFZI5MGA46D9RhojYMF qKXXZ5Z2/JSblPrMcghEDRhCgawGDMdoqQFjQeA1YLVVbvaZibjIGjCcF8TpDwdF14D1a0ral+tJ NTXtTFPIOdVrSglkO5I6TaGR96WpaceamqqS3aKpaZOmAB3bpVmlKb4GLLeMVbO0U4bcJETNwA73 M5CaKR3uZzSMXTUrgWxHUqdmNDJRzQq955t1BgRtjlymm2jYGuw2ydQCLo2Ge156XnpenoIXWewL p/Ux9qXPtT72La2PXgPV2Je4/HvaU9WABXijT+VZJQMJLXc5ZxCXU4HumapH0OJyzoguZzfS2u2C pyqtEm96DVSlhVyWrorA+t11aROuJ3XXYcE3xV0nHmPtwTfJXQcF3wforjenmpSvhMGjmzGsZoXV wTt2v3peel56XvbpU1DnCiwC6zLQWgQ++D1FdNcBRWD97jooQ7zP6KiDDHE30pqMdUqrJhLWY0Ax Et58zJBvVkIgkCObyGch4MhDOfKEhjxC/g4/HHmM7JQPRwY8IY6MDzfIE+Tf49aa0UDeP6Xqh/Tw xSF7kbgquBOuYtznPrx8J4al8Mko9cNrWlBTIHjihMhyCDfhHg8z3wlSJKnRMk6EfRatsutIQIlj x7llnpM5zDCN0XR2tH2POIZnw4l5xK4cN4sSNjgZYOW9seaH4ttda8OBORgNdqwZxng0GpfsDaff /Ql8PbPK7eoAAA== --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iIUEARYKAC0WIQRNTknu3zbaMQ2ddzTocYulkRQQdwUCY2RS9g8cbWFyaXVzQGdu dS5vcmcACgkQ6HGLpZEUEHeGbAEAsJskXuHr5Vgs3yLpN7xVoiMuCDmXs3cclgHf 0OYUhgcBALAZhvkMmIjCnT7e/cWeRCyuDUvEIOvC6YIboWDmKbQB =ihvL -----END PGP SIGNATURE----- --==-=-=--