* [PATCH 0/2] fixes noticed while tracking down fast-import failures @ 2024-03-08 21:05 Eric Wong 2024-03-08 21:05 ` [PATCH 1/2] lei: prevent empty {bytes} field in saved search Eric Wong 2024-03-08 21:05 ` [PATCH 2/2] import: croak (instead of die) on write failures Eric Wong 0 siblings, 2 replies; 3+ messages in thread From: Eric Wong @ 2024-03-08 21:05 UTC (permalink / raw) To: meta I'm still trying to track down the cause of fast-import failures[1], but this series presents some useful fixes regardless. Link: https://public-inbox.org/meta/CAL_JsqK7P4gjLPyvzxNEcYmxT4j6Ah5f3Pz1RqDHxmysTg3aEg@mail.gmail.com/ Eric Wong (2): lei: prevent empty {bytes} field in saved search import: croak (instead of die) on write failures lib/PublicInbox/Import.pm | 2 +- lib/PublicInbox/LeiSearch.pm | 2 ++ lib/PublicInbox/LeiToMail.pm | 1 + lib/PublicInbox/OverIdx.pm | 6 +++++- 4 files changed, 9 insertions(+), 2 deletions(-) ^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH 1/2] lei: prevent empty {bytes} field in saved search 2024-03-08 21:05 [PATCH 0/2] fixes noticed while tracking down fast-import failures Eric Wong @ 2024-03-08 21:05 ` Eric Wong 2024-03-08 21:05 ` [PATCH 2/2] import: croak (instead of die) on write failures Eric Wong 1 sibling, 0 replies; 3+ messages in thread From: Eric Wong @ 2024-03-08 21:05 UTC (permalink / raw) To: meta Noticed while tracking down fast-import crash bug report. Link: https://public-inbox.org/meta/CAL_JsqK7P4gjLPyvzxNEcYmxT4j6Ah5f3Pz1RqDHxmysTg3aEg@mail.gmail.com/ --- lib/PublicInbox/LeiSearch.pm | 2 ++ lib/PublicInbox/LeiToMail.pm | 1 + lib/PublicInbox/OverIdx.pm | 6 +++++- 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/lib/PublicInbox/LeiSearch.pm b/lib/PublicInbox/LeiSearch.pm index 29e3213f..684668c5 100644 --- a/lib/PublicInbox/LeiSearch.pm +++ b/lib/PublicInbox/LeiSearch.pm @@ -103,6 +103,8 @@ sub xoids_for { for my $o (@overs) { my ($id, $prev); while (my $cur = $o->next_by_mid($mid, \$id, \$prev)) { + # {bytes} may be '' from old bug + $cur->{bytes} = 1 if $cur->{bytes} eq ''; next if $cur->{bytes} == 0 || $xoids->{$cur->{blob}}; $git->cat_async($cur->{blob}, \&_cmp_1st, diff --git a/lib/PublicInbox/LeiToMail.pm b/lib/PublicInbox/LeiToMail.pm index a816df6c..dfae29e9 100644 --- a/lib/PublicInbox/LeiToMail.pm +++ b/lib/PublicInbox/LeiToMail.pm @@ -149,6 +149,7 @@ sub git_to_mail { # git->cat_async callback "W: $oid is $type (!= blob)"); $size or return $self->{lei}->child_error(0,"E: $oid is empty"); $smsg->{blob} eq $oid or die "BUG: expected=$smsg->{blob}"; + $smsg->{bytes} ||= $size; $self->{wcb}->($bref, $smsg); }; $self->{lei}->fail("$@ (oid=$oid)") if $@; diff --git a/lib/PublicInbox/OverIdx.pm b/lib/PublicInbox/OverIdx.pm index c9c25828..4f8533f7 100644 --- a/lib/PublicInbox/OverIdx.pm +++ b/lib/PublicInbox/OverIdx.pm @@ -17,6 +17,7 @@ use PublicInbox::MID qw/id_compress mids_for_index references/; use PublicInbox::Smsg qw(subject_normalized); use Compress::Zlib qw(compress); use Carp qw(croak); +use bytes (); # length sub dbh_new { my ($self) = @_; @@ -263,7 +264,10 @@ sub ddd_for ($) { sub add_overview { my ($self, $eml, $smsg) = @_; - $smsg->{lines} = $eml->body_raw =~ tr!\n!\n!; + my $raw = $eml->body_raw; + $smsg->{lines} = $raw =~ tr!\n!\n!; + $smsg->{bytes} //= bytes::length $raw; + undef $raw; my $mids = mids_for_index($eml); my $refs = $smsg->parse_references($eml, $mids); $mids->[0] //= do { ^ permalink raw reply related [flat|nested] 3+ messages in thread
* [PATCH 2/2] import: croak (instead of die) on write failures 2024-03-08 21:05 [PATCH 0/2] fixes noticed while tracking down fast-import failures Eric Wong 2024-03-08 21:05 ` [PATCH 1/2] lei: prevent empty {bytes} field in saved search Eric Wong @ 2024-03-08 21:05 ` Eric Wong 1 sibling, 0 replies; 3+ messages in thread From: Eric Wong @ 2024-03-08 21:05 UTC (permalink / raw) To: meta This allows accurate reporting of the error location and can be made to dump a Perl backtrace via PERL5OPT='-MCarp=verbose'. Noticed while tracking down fast-import failures. Link: https://public-inbox.org/meta/CAL_JsqK7P4gjLPyvzxNEcYmxT4j6Ah5f3Pz1RqDHxmysTg3aEg@mail.gmail.com/ --- lib/PublicInbox/Import.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/PublicInbox/Import.pm b/lib/PublicInbox/Import.pm index 51ddfa7f..a951874b 100644 --- a/lib/PublicInbox/Import.pm +++ b/lib/PublicInbox/Import.pm @@ -85,7 +85,7 @@ sub gfi_start { $self->{io}; } -sub wfail () { die "write to fast-import failed: $!" } +sub wfail () { croak "write to fast-import failed: $!" } sub now_raw () { time . ' +0000' } ^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2024-03-08 21:05 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-03-08 21:05 [PATCH 0/2] fixes noticed while tracking down fast-import failures Eric Wong 2024-03-08 21:05 ` [PATCH 1/2] lei: prevent empty {bytes} field in saved search Eric Wong 2024-03-08 21:05 ` [PATCH 2/2] import: croak (instead of die) on write failures Eric Wong
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).