lei: accessing blob after import requires daemon restart

lei: accessing blob after import requires daemon restart

[Kyle Meyer]

When playing around with lei-import, I was unable to display an mbox for
a just-imported message:

  # starting in an uninitialized state with no externals
  $ curl -d'' -fsS 'https://public-inbox.org/meta/?q=cgit+blob+solver&x=m' \
    | zcat >t.mbox
  $ lei import mboxrd:t.mbox
  $ lei q -f ldjson s:remove
  {"blob":"089cdf1af1d738068494a653532ed01b1844407d","docid":11,...}
  $ lei q -f mboxrd s:remove
  missing 089cdf1af1d738068494a653532ed01b1844407d

It's in the local repo though:

  $ git -C ~/.local/share/lei/store/local/0.git \
    rev-parse --verify 089cdf1af1d738068494a653532ed01b1844407d^{blob}
  089cdf1af1d738068494a653532ed01b1844407d

Killing the daemon and trying again resolves the issue:

  $ lei daemon-kill
  $ lei q -f mboxrd s:remove | head -1
  From 089cdf1af1d738068494a653532ed01b1844407d=99@mboxrd Thu Jan  1 00:00:00 1970

Sorry if I'm reporting a known to-do; with a quick search, I didn't spot
anything on the list or in the code, and it feels like enough of a
corner case to be worth mentioning.

[PATCH] lei_store: populate ALL.git/alternates with new epochs

[Eric Wong]

Kyle Meyer <kyle@kyleam.com> wrote:
> Sorry if I'm reporting a known to-do; with a quick search, I didn't spot
> anything on the list or in the code, and it feels like enough of a
> corner case to be worth mentioning.

Yikes :x  I definitely didn't notice since I'm constantly
restarting the daemon, and the test was insufficient.
Thanks for the bug report, I believe this fixes it:

-----8<-----
Subject: [PATCH] lei_store: populate ALL.git/alternates with new epochs

Since eidx_init updates ALL.git/objects/info/alternates, we need
to ensure new epochs we create from LeiStore->importer exist
before eidx_init writes alternates.

Reported-by: Kyle Meyer <kyle@kyleam.com>
Link: https://public-inbox.org/meta/8735xou0gq.fsf@kyleam.com/
---
 lib/PublicInbox/LeiStore.pm |  4 ++--
 t/lei-import.t              | 13 +++++++++++++
 2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/LeiStore.pm b/lib/PublicInbox/LeiStore.pm
index 546d500b..77601828 100644
--- a/lib/PublicInbox/LeiStore.pm
+++ b/lib/PublicInbox/LeiStore.pm
@@ -192,10 +192,10 @@ sub maildir_keywords {
 
 sub add_eml {
 	my ($self, $eml, @kw) = @_;
-	my $eidx = eidx_init($self);
+	my $im = $self->importer; # may create new epoch
+	my $eidx = eidx_init($self); # writes ALL.git/objects/info/alternates
 	my $oidx = $eidx->{oidx};
 	my $smsg = bless { -oidx => $oidx }, 'PublicInbox::Smsg';
-	my $im = $self->importer;
 	$im->add($eml, undef, $smsg) or return; # duplicate returns undef
 
 	local $self->{current_info} = $smsg->{blob};
diff --git a/t/lei-import.t b/t/lei-import.t
index 46747a91..fa4fc504 100644
--- a/t/lei-import.t
+++ b/t/lei-import.t
@@ -13,6 +13,19 @@ lei_ok([qw(import -f eml -)], undef, { %$lei_opt, 0 => $fh },
 	\'import single file from stdin') or diag $lei_err;
 close $fh;
 lei_ok(qw(q s:boolean), \'search hit after import');
+lei_ok(qw(q s:boolean -f mboxrd), \'blob accessible after import');
+{
+	my $expect = [ eml_load('t/data/0001.patch') ];
+	require PublicInbox::MboxReader;
+	my @cmp;
+	open my $fh, '<', \$lei_out or BAIL_OUT "open :scalar: $!";
+	PublicInbox::MboxReader->mboxrd($fh, sub {
+		my ($eml) = @_;
+		$eml->header_set('Status');
+		push @cmp, $eml;
+	});
+	is_deeply(\@cmp, $expect, 'got expected message in mboxrd');
+}
 lei_ok(qw(import -f eml), 't/data/message_embed.eml',
 	\'import single file by path');
 

Re: [PATCH] lei_store: populate ALL.git/alternates with new epochs

[Kyle Meyer]

Eric Wong writes:

> Thanks for the bug report, I believe this fixes it:
>
> -----8<-----
> Subject: [PATCH] lei_store: populate ALL.git/alternates with new epochs

Confirmed.  Thanks for the very quick fix.