From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 2FCB41F54E; Sat, 10 Sep 2022 01:18:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1662772739; bh=SLN3NkP4vwBqpk3y1exXDD/ofWLD+W8Pb0K0JTHOYBI=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=tMm3J8IbE1Lo1J5WivVr1eyf69C7xQ+UbjwOd4uW3IjkzlTyTTNK+3Y2Z3v/wTDC0 gB2NX8g/4Pb2+XI2y8jOBm2nHY62IOsxguv4z5MbQ/YaMZejZKgBTrun8RJtmXXplI JmfvJCueGvuBw230+25boBmkzuV6eWbNJafK34a0= Date: Sat, 10 Sep 2022 01:18:59 +0000 From: Eric Wong To: Ricardo Ribalda Cc: meta@public-inbox.org Subject: [PATCH v2] lei: bail out earlier on IMAP writer failures Message-ID: <20220910011859.M68532@dcvr> References: <20220909174410.M560915@dcvr> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: List-Id: Ricardo Ribalda wrote: > The patch did not seem to have any effect :(, I never get a "IMAP > LastError: " message Yeah, I guess IMAP servers will just shutdown the socket w/o saying anything. At least I didn't get anything from dovecot... The below patch is a refinement of what I posted originally and should stop the process instead of attempting to continue and spew. > On the other hand, the -j worked! I can go up to -j ,15 without any error. Good to know. I wonder if making the default `-j ,4' for IMAP is reasonable if unspecified. That's the default limit for HTTP(S) hosts, and I seem to recall 4 being a reasonable limit for browsers. Thanks for the report and followup! ------8<------ From: Eric Wong Subject: [PATCH] lei: bail out earlier on IMAP writer failures Excessive IMAP connections can overload IMAP servers and cause clients to be disconnected without diagnostic messages. Use $lei->fail on these exceptions to propagate errors to the CLI ASAP to avoid further errors down the line. This ought to make problems more apparent for users using IMAP destinations. Reported-by: Ricardo Ribalda Link: https://public-inbox.org/meta/CANiDSCsDfutAUMBLPZbxdyka+_jnhv+4YNYdL9QPRoC=wNUGCQ@mail.gmail.com/ --- lib/PublicInbox/LeiToMail.pm | 10 +++++++--- lib/PublicInbox/NetReader.pm | 8 +++++++- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/lib/PublicInbox/LeiToMail.pm b/lib/PublicInbox/LeiToMail.pm index 2aa3977e..03cbde3b 100644 --- a/lib/PublicInbox/LeiToMail.pm +++ b/lib/PublicInbox/LeiToMail.pm @@ -310,8 +310,11 @@ sub _imap_write_cb ($$) { my $dedupe = $lei->{dedupe}; $dedupe->prepare_dedupe if $dedupe; my $append = $lei->{net}->can('imap_append'); - my $uri = $self->{uri}; - my $mic = $lei->{net}->mic_get($uri); + my $uri = $self->{uri} // die 'BUG: no {uri}'; + my $mic = $lei->{net}->mic_get($uri) // die <mailbox; $uri->uidvalidity($mic->uidvalidity($folder)); my $lse = $lei->{lse}; # may be undef @@ -749,7 +752,8 @@ sub do_post_auth { $au_peers->[1] = undef; sysread($au_peers->[0], my $barrier1, 1); } - $self->{wcb} = $self->write_cb($lei); + eval { $self->{wcb} = $self->write_cb($lei) }; + $lei->fail($@) if $@; if ($au_peers) { # wait for peer l2m to set write_cb $au_peers->[3] = undef; sysread($au_peers->[2], my $barrier2, 1); diff --git a/lib/PublicInbox/NetReader.pm b/lib/PublicInbox/NetReader.pm index c1af03a3..4de2583e 100644 --- a/lib/PublicInbox/NetReader.pm +++ b/lib/PublicInbox/NetReader.pm @@ -685,7 +685,13 @@ sub mic_get { } my $mic = mic_new($self, $mic_arg, $sec, $uri); $cached //= {}; # invalid placeholder if no cache enabled - $mic && $mic->IsConnected ? ($cached->{$sec} = $mic) : undef; + if ($mic && $mic->IsConnected) { + $cached->{$sec} = $mic; + } else { + warn 'IMAP LastError: ',$mic->LastError, "\n" if $mic; + warn "IMAP errno: $!\n" if $!; + undef; + } } sub imap_each {