[RFC] pop3: support `?limit=$NUM' parameter in mailbox name

[Eric Wong <e@80x24.org>]

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> Hello:
> 
> I've been playing around with pop3, and I'm wondering if we can improve its
> usability by adding a "last NNN messages" pseudo-folder. Currently, if someone
> wants to access the git mailing list archive via pop3, they have to do the
> following:
> 
> - know that the username should be $(uuidgen)@org.kernel.vger.git.1 (the
>   default username would access slice 0, right? Or is it the last 50,000
>   messages?)

The /\.[0-9]+$/ slice is actually optional for POP3.
`$(uuidgen)@org.kernel.vger.git' alone will get you the latest 50k.

> - wait for their client to retrieve tens of thousands of unread messages on
>   first access

Perhaps 50K is too much?  I figured clients would have a way to
limit that, but I don't really pay attention to POP3 clients...

Patch below adds a `?limit=$NUM' parameter, but I'm not sure if
`?' or `=' are allowed in POP3 mailbox names.  mpop(1) doesn't
complain...  Haven't looked at other POP3 clients.

> - if the remote archive rolls over to the next slice, they have to edit their
>   account info to get new messages (unless I'm wrong about #1)

Yeah, that only applies to IMAP.  IMAP is a pain since connections
can be long-lived and per-connection MSN <=> UID mappings can grow
without bound after more messages arrive.

Perhaps our -imapd can be less nice and forcibly terminate
connections if the most recent window gets too big.

> Perhaps the default could be slightly different:
> 
> - $(uuidgen)@org.kernel.vger.git would start with an empty view (or something
>   like the last 10 messages)

Small numbers would be very unuseful, too, I think...

> - it would only get any new messages added to the archive
> 
> I think this would be a friendlier experience, but not sure how difficult it
> would be to implement. I'm also not 100% sure all my assumptions are correct,
> so please feel free to correct me.

No worries, the POP3 stuff hasn't seen much use.
IMAP's been hammered relentlessly by bots on my server, at least :>

Lightly-tested patch to support ?limit=$NUM

-------8<--------
Subject: [PATCH] pop3: support `?limit=$NUM' parameter in mailbox name

I'm not sure if `?' or `=' are allowed characters in POP3
mailbox names.  In fact, I can't find any information on
valid characters allowed in RFC 1081 nor RFC 1939.

In any case, it seems to work fine with mpop.
---
 lib/PublicInbox/POP3.pm | 18 ++++++++++++------
 xt/pop3d-mpop.t         |  4 ++--
 2 files changed, 14 insertions(+), 8 deletions(-)

diff --git a/lib/PublicInbox/POP3.pm b/lib/PublicInbox/POP3.pm
index d32793e4..4a21ef5e 100644
--- a/lib/PublicInbox/POP3.pm
+++ b/lib/PublicInbox/POP3.pm
@@ -41,6 +41,7 @@ use PublicInbox::IMAP; # for UID slice stuff
 
 use constant {
 	LINE_MAX => 512, # XXX unsure
+	UID_SLICE => PublicInbox::IMAP::UID_SLICE,
 };
 
 # XXX FIXME: duplicated stuff from NNTP.pm and IMAP.pm
@@ -70,20 +71,25 @@ sub cmd_user ($$) {
 	my $user = $1;
 	$user =~ tr/-//d; # most have dashes, some (dbus-uuidgen) don't
 	$user =~ m!\A[a-f0-9]{32}\z!i or return \"-ERR user has no UUID\r\n";
-	my $slice;
-	$mailbox =~ s/\.([0-9]+)\z// and $slice = $1 + 0;
+
+	my $limit = UID_SLICE;
+	$mailbox =~ s/\?limit=([0-9]+)\z// and
+		$limit = $1 > UID_SLICE ? UID_SLICE : $1;
+
+	my $slice = $mailbox =~ s/\.([0-9]+)\z// ? $1 + 0 : undef;
+
 	my $ibx = $self->{pop3d}->{pi_cfg}->lookup_newsgroup($mailbox) //
 		return \"-ERR $mailbox does not exist\r\n";
 	my $uidmax = $ibx->mm(1)->num_highwater // 0;
 	if (defined $slice) {
-		my $max = int($uidmax / PublicInbox::IMAP::UID_SLICE);
+		my $max = int($uidmax / UID_SLICE);
 		my $tip = "$mailbox.$max";
 		return \"-ERR $mailbox.$slice does not exist ($tip does)\r\n"
 			if $slice > $max;
-		$self->{uid_base} = $slice * PublicInbox::IMAP::UID_SLICE;
+		$self->{uid_base} = ($slice * UID_SLICE) + UID_SLICE - $limit;
 		$self->{slice} = $slice;
-	} else { # latest 50K messages
-		my $base = $uidmax - PublicInbox::IMAP::UID_SLICE;
+	} else { # latest $limit messages
+		my $base = $uidmax - $limit;
 		$self->{uid_base} = $base < 0 ? 0 : $base;
 		$self->{slice} = -1;
 	}
diff --git a/xt/pop3d-mpop.t b/xt/pop3d-mpop.t
index fc82bc6b..9da1050c 100644
--- a/xt/pop3d-mpop.t
+++ b/xt/pop3d-mpop.t
@@ -53,7 +53,7 @@ delivery maildir $tmpdir/md
 account default
 host ${\$sock->sockhost}
 port ${\$sock->sockport}
-user $uuid\@$newsgroup
+user $uuid\@$newsgroup?limit=10000
 auth user
 password anonymous
 received_header off
@@ -65,7 +65,7 @@ EOM
 	my $pid = spawn($cmd, undef, { 1 => 2 });
 	$pids{$pid} = $cmd;
 }
-
+diag "mpop is writing to $tmpdir/md ...";
 while (scalar keys %pids) {
 	my $pid = waitpid(-1, 0) or next;
 	my $cmd = delete $pids{$pid} or next;