From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 5FC2C1F54E; Fri, 9 Sep 2022 17:45:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1662745525; bh=WnJQX1SGDe+LvALzDjAvumnXwc2153NBL1Kb218c4G0=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=dTPKGU8OVuloI2Gd+PoRfLVrsPs+3GrmMre28JpMXe/3WP42zdHL29ReIi4uxAIQ8 Z3tksC9+XedKktmlB84G0fmBiVvilgYXrEx5mq2FlqD8KVxtMyBBWdLD9OjePuwIIZ Mbjpo/9qrqjTRyE1H1Pn+xz9wLo4ObUtZni3RKaY= Date: Fri, 9 Sep 2022 17:44:10 +0000 From: Eric Wong To: Ricardo Ribalda Cc: meta@public-inbox.org Subject: [PATCH] lei: add diagnostics for IMAP writer failures Message-ID: <20220909174410.M560915@dcvr> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: List-Id: Ricardo Ribalda wrote: > Hi > > I am getting a lot of those messages when using lei in imap mode. > > Nonetheless the mail seems to arrive fine to its destination (on the > first run, some of the mail was lost, but for small batches of mails > it seems to work fine). > > It is a valid message or just some red-herring. > > I am using debian testing/ > > Regards! > > # https://lore.kernel.org/all/ limiting to 2022-09-07 10:03 +0000 and newer > # /usr/local/google/home/ribalda/.local/share/lei/store 13/13 > # /usr/bin/curl -Sf -s -d '' > https://lore.kernel.org/all/?x=m&t=1&q=((ribalda)+AND+rt%3A1660039651..)+AND+dt%3A20220907100321.. > 1285740 lei2mail 6 wq_worker: do_post_auth: Can't call method > "uidvalidity" on an undefined value at > /usr/share/perl5/PublicInbox/LeiToMail.pm line 313. > 1285735 lei2mail 1 wq_worker: write_mail: Can't use an undefined value > as a subroutine reference at /usr/share/perl5/PublicInbox/LeiToMail.pm > line 783. > # https://lore.kernel.org/all/ 18/18 > # 14 written to imaps://imap.gmail.com/lei/me (31 matches) I wonder if it's excessive parallelism for gmail's IMAP. I haven't tested IMAP destinations, much... Can you try the patch at the bottom? There's also another patch coming to document the `--jobs|-j' CLI switch for `lei up' and `lei q', but trying `-j ,1' may help you if it's parallelism. Note the comma before `1', it accepts `-j $Q,$W' since $Q is the number of query processes and $W is the number of LeiToMail writers. -------8<------- From: Eric Wong Subject: [PATCH] lei: add diagnostics for IMAP writer failures This may help diagnose the problem with IMAP destinations encountered at: https://public-inbox.org/meta/CANiDSCsDfutAUMBLPZbxdyka+_jnhv+4YNYdL9QPRoC=wNUGCQ@mail.gmail.com/ --- lib/PublicInbox/LeiToMail.pm | 4 ++-- lib/PublicInbox/NetReader.pm | 8 +++++++- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/lib/PublicInbox/LeiToMail.pm b/lib/PublicInbox/LeiToMail.pm index 2aa3977e..bc00b96a 100644 --- a/lib/PublicInbox/LeiToMail.pm +++ b/lib/PublicInbox/LeiToMail.pm @@ -310,8 +310,8 @@ sub _imap_write_cb ($$) { my $dedupe = $lei->{dedupe}; $dedupe->prepare_dedupe if $dedupe; my $append = $lei->{net}->can('imap_append'); - my $uri = $self->{uri}; - my $mic = $lei->{net}->mic_get($uri); + my $uri = $self->{uri} // die 'BUG: no {uri}'; + my $mic = $lei->{net}->mic_get($uri) // die 'BUG: no $mic'; my $folder = $uri->mailbox; $uri->uidvalidity($mic->uidvalidity($folder)); my $lse = $lei->{lse}; # may be undef diff --git a/lib/PublicInbox/NetReader.pm b/lib/PublicInbox/NetReader.pm index c1af03a3..4de2583e 100644 --- a/lib/PublicInbox/NetReader.pm +++ b/lib/PublicInbox/NetReader.pm @@ -685,7 +685,13 @@ sub mic_get { } my $mic = mic_new($self, $mic_arg, $sec, $uri); $cached //= {}; # invalid placeholder if no cache enabled - $mic && $mic->IsConnected ? ($cached->{$sec} = $mic) : undef; + if ($mic && $mic->IsConnected) { + $cached->{$sec} = $mic; + } else { + warn 'IMAP LastError: ',$mic->LastError, "\n" if $mic; + warn "IMAP errno: $!\n" if $!; + undef; + } } sub imap_each {