unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
* [PATCH 00/11] IMAP, NNTP, POP3 golfing
@ 2022-07-23  4:41 Eric Wong
  2022-07-23  4:41 ` [PATCH 01/11] nntp: pass regexp to split() callers Eric Wong
                   ` (10 more replies)
  0 siblings, 11 replies; 12+ messages in thread
From: Eric Wong @ 2022-07-23  4:41 UTC (permalink / raw)
  To: meta

More to come, but duplicate code and data structures are
to be eliminated.

Eric Wong (11):
  nntp: pass regexp to split() callers
  nntp: start adding CRLF to responses natively
  nntp: remove more() wrapper
  ds: support greeting protocols
  ds: move no-op ->zflush to common base class
  ds: move requeue_once
  nntp: listgroup_range_i: remove useless `map' op
  nntp: inline CRLF in all response lines
  ds: share long_step between NNTP and IMAP
  nntp: resolve inboxes immediately on group listings
  imap+nntp: share COMPRESS implementation

 MANIFEST                                      |   3 +-
 lib/PublicInbox/DS.pm                         |  72 ++++
 .../{NNTPdeflate.pm => DSdeflate.pm}          |  15 +-
 lib/PublicInbox/IMAP.pm                       | 102 +----
 lib/PublicInbox/IMAPD.pm                      |   2 +-
 lib/PublicInbox/IMAPdeflate.pm                | 126 ------
 lib/PublicInbox/NNTP.pm                       | 404 +++++++-----------
 lib/PublicInbox/NNTPD.pm                      |   4 +-
 lib/PublicInbox/POP3.pm                       |  80 +---
 t/nntpd.t                                     |   2 +-
 xt/mem-imapd-tls.t                            |   4 +-
 11 files changed, 262 insertions(+), 552 deletions(-)
 rename lib/PublicInbox/{NNTPdeflate.pm => DSdeflate.pm} (91%)
 delete mode 100644 lib/PublicInbox/IMAPdeflate.pm

^ permalink raw reply	[flat|nested] 12+ messages in thread

* [PATCH 01/11] nntp: pass regexp to split() callers
  2022-07-23  4:41 [PATCH 00/11] IMAP, NNTP, POP3 golfing Eric Wong
@ 2022-07-23  4:41 ` Eric Wong
  2022-07-23  4:41 ` [PATCH 02/11] nntp: start adding CRLF to responses natively Eric Wong
                   ` (9 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Eric Wong @ 2022-07-23  4:41 UTC (permalink / raw)
  To: meta

Current implementations of Perl5 don't have optimizations for
single-character field separators.
---
 lib/PublicInbox/NNTP.pm | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index b36722d7..3d304c52 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -1,4 +1,4 @@
-# Copyright (C) 2015-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 #
 # Each instance of this represents a NNTP client socket
@@ -71,7 +71,7 @@ sub new ($$$) {
 sub args_ok ($$) {
 	my ($cb, $argc) = @_;
 	my $tot = prototype $cb;
-	my ($nreq, undef) = split(';', $tot);
+	my ($nreq, undef) = split(/;/, $tot);
 	$nreq = ($nreq =~ tr/$//) - 1;
 	$tot = ($tot =~ tr/$//) - 1;
 	($argc <= $tot && $argc >= $nreq);
@@ -349,7 +349,7 @@ sub cmd_newnews ($$$$;$$) {
 	my $ts = eval { parse_time($date, $time, $gmt) };
 	return r501 if $@;
 	more($self, '230 list of new articles by message-id follows');
-	my ($keep, $skip) = split('!', $newsgroups, 2);
+	my ($keep, $skip) = split(/!/, $newsgroups, 2);
 	ngpat2re($keep);
 	ngpat2re($skip);
 	my @names = grep(!/$skip/, grep(/$keep/,

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 02/11] nntp: start adding CRLF to responses natively
  2022-07-23  4:41 [PATCH 00/11] IMAP, NNTP, POP3 golfing Eric Wong
  2022-07-23  4:41 ` [PATCH 01/11] nntp: pass regexp to split() callers Eric Wong
@ 2022-07-23  4:41 ` Eric Wong
  2022-07-23  4:41 ` [PATCH 03/11] nntp: remove more() wrapper Eric Wong
                   ` (8 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Eric Wong @ 2022-07-23  4:41 UTC (permalink / raw)
  To: meta

With IMAP and POP3, I've started to embed CRLF into constant
response codes to avoid triggering CoW and extra memory traffic
in Perl.

The end goal is to enable more code sharing between IMAP, NNTP,
and POP3 inside one -netd process.
---
 lib/PublicInbox/NNTP.pm | 35 ++++++++++++++---------------------
 1 file changed, 14 insertions(+), 21 deletions(-)

diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 3d304c52..1f9058ca 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -24,9 +24,9 @@ use constant {
 	LINE_MAX => 512, # RFC 977 section 2.3
 	r501 => '501 command syntax error',
 	r502 => '502 Command unavailable',
-	r221 => '221 Header follows',
+	r221 => "221 Header follows\r\n",
 	r224 => '224 Overview information follows (multi-line)',
-	r225 =>	'225 Headers follow (multi-line)',
+	r225 =>	"225 Headers follow (multi-line)\r\n",
 	r430 => '430 No article with that message-id',
 };
 use PublicInbox::Syscall qw(EPOLLIN EPOLLONESHOT);
@@ -82,8 +82,8 @@ sub process_line ($$) {
 	my ($self, $l) = @_;
 	my ($req, @args) = split(/[ \t]+/, $l);
 	return 1 unless defined($req); # skip blank line
-	$req = $self->can('cmd_'.lc($req));
-	return res($self, '500 command not recognized') unless $req;
+	$req = $self->can('cmd_'.lc($req)) //
+		return $self->write(\"500 command not recognized\r\n");
 	return res($self, r501) unless args_ok($req, scalar @args);
 
 	my $res = eval { $req->($self, @args) };
@@ -403,7 +403,7 @@ sub cmd_post ($) {
 
 sub cmd_quit ($) {
 	my ($self) = @_;
-	res($self, '205 closing connection - goodbye!');
+	$self->write(\"205 closing connection - goodbye!\r\n");
 	$self->shutdn;
 	undef;
 }
@@ -663,7 +663,7 @@ sub long_step {
 		$self->requeue if $new_size == 1;
 	} else { # all done!
 		delete $self->{long_cb};
-		res($self, '.');
+		$self->write(\".\r\n");
 		my $elapsed = now() - $t0;
 		my $fd = fileno($self->{sock});
 		out($self, " deferred[$fd] done - %0.6f", $elapsed);
@@ -703,7 +703,7 @@ sub hdr_message_id ($$$) { # optimize XHDR Message-ID [range] for slrnpull.
 		$range = $self->{article} unless defined $range;
 		my $r = get_range($self, $range);
 		return $r unless ref $r;
-		more($self, $xhdr ? r221 : r225);
+		$self->msg_more($xhdr ? r221 : r225);
 		long_response($self, \&hdr_msgid_range_i, @$r);
 	}
 }
@@ -775,7 +775,7 @@ sub hdr_xref ($$$) { # optimize XHDR Xref [range] for rtin
 		$range = $self->{article} unless defined $range;
 		my $r = get_range($self, $range);
 		return $r unless ref $r;
-		more($self, $xhdr ? r221 : r225);
+		$self->msg_more($xhdr ? r221 : r225);
 		long_response($self, \&xref_range_i, @$r);
 	}
 }
@@ -819,7 +819,7 @@ sub hdr_smsg ($$$$) {
 		$range = $self->{article} unless defined $range;
 		my $r = get_range($self, $range);
 		return $r unless ref $r;
-		more($self, $xhdr ? r221 : r225);
+		$self->msg_more($xhdr ? r221 : r225);
 		long_response($self, \&smsg_range_i, @$r, $field);
 	}
 }
@@ -837,7 +837,7 @@ sub do_hdr ($$$;$) {
 	} elsif ($sub =~ /\A:(bytes|lines)\z/) {
 		hdr_smsg($self, $xhdr, $1, $range);
 	} else {
-		$xhdr ? (r221 . "\r\n.") : "503 HDR not permitted on $header";
+		$xhdr ? (r221 . '.') : "503 HDR not permitted on $header";
 	}
 }
 
@@ -867,16 +867,9 @@ sub hdr_mid_prefix ($$$$$) {
 
 sub hdr_mid_response ($$$$$$) {
 	my ($self, $xhdr, $ibx, $n, $mid, $v) = @_;
-	my $res = '';
-	if ($xhdr) {
-		$res .= r221 . "\r\n";
-		$res .= "$mid $v\r\n";
-	} else {
-		$res .= r225 . "\r\n";
-		my $pfx = hdr_mid_prefix($self, $xhdr, $ibx, $n, $mid);
-		$res .= "$pfx $v\r\n";
-	}
-	res($self, $res .= '.');
+	$self->write(($xhdr ? r221.$mid :
+		   r225.hdr_mid_prefix($self, $xhdr, $ibx, $n, $mid)) .
+		" $v\r\n.\r\n");
 	undef;
 }
 
@@ -972,7 +965,7 @@ sub cmd_starttls ($) {
 	return r502 if ($sock->can('accept_SSL') || $self->compressed);
 	my $opt = $self->{nntpd}->{accept_tls} or
 		return '580 can not initiate TLS negotiation';
-	res($self, '382 Continue with TLS negotiation');
+	$self->write(\"382 Continue with TLS negotiation\r\n");
 	$self->{sock} = IO::Socket::SSL->start_SSL($sock, %$opt);
 	$self->requeue if PublicInbox::DS::accept_tls_step($self);
 	undef;

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 03/11] nntp: remove more() wrapper
  2022-07-23  4:41 [PATCH 00/11] IMAP, NNTP, POP3 golfing Eric Wong
  2022-07-23  4:41 ` [PATCH 01/11] nntp: pass regexp to split() callers Eric Wong
  2022-07-23  4:41 ` [PATCH 02/11] nntp: start adding CRLF to responses natively Eric Wong
@ 2022-07-23  4:41 ` Eric Wong
  2022-07-23  4:41 ` [PATCH 04/11] ds: support greeting protocols Eric Wong
                   ` (7 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Eric Wong @ 2022-07-23  4:41 UTC (permalink / raw)
  To: meta

Using PublicInbox::DS->msg_more directly can avoid unnecessary
CoW memory traffic since there's no appending "\r\n".
---
 lib/PublicInbox/NNTP.pm | 40 ++++++++++++++++++++--------------------
 1 file changed, 20 insertions(+), 20 deletions(-)

diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 1f9058ca..e4ca7d14 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -117,7 +117,7 @@ sub cmd_slave ($) { '202 slave status noted' }
 
 sub cmd_xgtitle ($;$) {
 	my ($self, $wildmat) = @_;
-	more($self, '282 list of groups and descriptions follows');
+	$self->msg_more("282 list of groups and descriptions follows\r\n");
 	list_newsgroups($self, $wildmat);
 }
 
@@ -150,7 +150,7 @@ sub list_active_times_i {
 	for my $ngname (@window) {
 		my $ibx = $groups->{$ngname} or next;
 		my $c = eval { $ibx->uidvalidity } // time;
-		more($self, "$ngname $c <$ibx->{-primary_address}>");
+		$self->msg_more("$ngname $c <$ibx->{-primary_address}>\r\n");
 	}
 	scalar(@$groupnames); # continue if there's more
 }
@@ -169,7 +169,7 @@ sub list_newsgroups_i {
 	my $ibx;
 	for my $ngname (@window) {
 		$ibx = $groups->{$ngname} and
-			more($self, "$ngname ".$ibx->description);
+			$self->msg_more("$ngname ".$ibx->description."\r\n");
 	}
 	scalar(@$groupnames); # continue if there's more
 }
@@ -191,10 +191,10 @@ sub cmd_list ($;$$) {
 		$arg = "list_$arg";
 		$arg = $self->can($arg);
 		return r501 unless $arg && args_ok($arg, scalar @args);
-		more($self, '215 information follows');
+		$self->msg_more("215 information follows\r\n");
 		$arg->($self, @args);
 	} else {
-		more($self, '215 list of newsgroups follows');
+		$self->msg_more("215 list of newsgroups follows\r\n");
 		long_response($self, \&list_active_i, [ # copy array
 			@{$self->{nntpd}->{groupnames}} ]);
 	}
@@ -212,7 +212,7 @@ sub listgroup_all_i {
 	my ($self, $num) = @_;
 	my $ary = $self->{ibx}->over(1)->ids_after($num);
 	scalar(@$ary) or return;
-	more($self, join("\r\n", @$ary));
+	$self->msg_more(join("\r\n", @$ary, ''));
 	1;
 }
 
@@ -221,7 +221,7 @@ sub cmd_listgroup ($;$$) {
 	if (defined $group) {
 		my $res = cmd_group($self, $group);
 		return $res if ($res !~ /\A211 /);
-		more($self, $res);
+		$self->msg_more($res .= "\r\n");
 	}
 	$self->{ibx} or return '412 no newsgroup selected';
 	if (defined $range) {
@@ -262,7 +262,7 @@ sub parse_time ($$;$) {
 sub group_line ($$) {
 	my ($self, $ibx) = @_;
 	my ($min, $max) = $ibx->mm(1)->minmax;
-	more($self, "$ibx->{newsgroup} $max $min n");
+	$self->msg_more("$ibx->{newsgroup} $max $min n\r\n");
 }
 
 sub newgroups_i {
@@ -284,7 +284,7 @@ sub cmd_newgroups ($$$;$$) {
 	return r501 if $@;
 
 	# TODO dists
-	more($self, '231 list of new newsgroups follows');
+	$self->msg_more("231 list of new newsgroups follows\r\n");
 	long_response($self, \&newgroups_i, $ts, \(my $i = 0),
 				$self->{nntpd}->{groupnames});
 }
@@ -348,7 +348,7 @@ sub cmd_newnews ($$$$;$$) {
 	my ($self, $newsgroups, $date, $time, $gmt, $dists) = @_;
 	my $ts = eval { parse_time($date, $time, $gmt) };
 	return r501 if $@;
-	more($self, '230 list of new articles by message-id follows');
+	$self->msg_more("230 list of new articles by message-id follows\r\n");
 	my ($keep, $skip) = split(/!/, $newsgroups, 2);
 	ngpat2re($keep);
 	ngpat2re($skip);
@@ -565,15 +565,15 @@ sub blob_cb { # called by git->cat_async via ibx_async_cat
 	my $r = "$code $smsg->{num} <$smsg->{mid}> article retrieved - ";
 	my $eml = PublicInbox::Eml->new($bref);
 	if ($code == 220) {
-		more($self, $r .= 'head and body follow');
+		$self->msg_more($r .= "head and body follow\r\n");
 		msg_hdr_write($eml, $smsg);
 		$self->msg_more("\r\n");
 		msg_body_write($self, $bref);
 	} elsif ($code == 221) {
-		more($self, $r .= 'head follows');
+		$self->msg_more($r .= "head follows\r\n");
 		msg_hdr_write($eml, $smsg);
 	} elsif ($code == 222) {
-		more($self, $r .= 'body follows');
+		$self->msg_more($r .= "body follows\r\n");
 		msg_body_write($self, $bref);
 	} else {
 		$self->close;
@@ -609,7 +609,7 @@ sub cmd_date ($) { '111 '.strftime('%Y%m%d%H%M%S', gmtime(time)) }
 
 sub cmd_help ($) {
 	my ($self) = @_;
-	more($self, '100 help text follows');
+	$self->msg_more("100 help text follows\r\n");
 	'.'
 }
 
@@ -876,7 +876,7 @@ sub hdr_mid_response ($$$$$$) {
 sub xrover_i {
 	my ($self, $beg, $end) = @_;
 	my $h = over_header_for($self->{ibx}, $$beg, 'references');
-	more($self, "$$beg $h") if defined($h);
+	$self->msg_more("$$beg $h\r\n") if defined($h);
 	$$beg++ < $end;
 }
 
@@ -889,7 +889,7 @@ sub cmd_xrover ($;$) {
 	$range = $self->{article} unless defined $range;
 	my $r = get_range($self, $range);
 	return $r unless ref $r;
-	more($self, '224 Overview information follows');
+	$self->msg_more("224 Overview information follows\r\n");
 	long_response($self, \&xrover_i, @$r);
 }
 
@@ -916,7 +916,8 @@ sub cmd_over ($;$) {
 		my ($ibx, $n) = mid_lookup($self, $1);
 		defined $n or return r430;
 		my $smsg = $ibx->over(1)->get_art($n) or return r430;
-		more($self, '224 Overview information follows (multi-line)');
+		$self->msg_more(
+			"224 Overview information follows (multi-line)\r\n");
 
 		# Only set article number column if it's the current group
 		# (RFC 3977 8.3.2)
@@ -952,7 +953,8 @@ sub cmd_xover ($;$) {
 	my $r = get_range($self, $range);
 	return $r unless ref $r;
 	my ($beg, $end) = @$r;
-	more($self, "224 Overview information follows for $$beg to $end");
+	$self->msg_more(
+		"224 Overview information follows for $$beg to $end\r\n");
 	long_response($self, \&xover_i, @$r);
 }
 
@@ -1014,8 +1016,6 @@ sub cmd_xpath ($$) {
 
 sub res ($$) { do_write($_[0], $_[1] . "\r\n") }
 
-sub more ($$) { $_[0]->msg_more($_[1] . "\r\n") }
-
 sub do_write ($$) {
 	my $self = $_[0];
 	my $done = $self->write(\($_[1]));

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 04/11] ds: support greeting protocols
  2022-07-23  4:41 [PATCH 00/11] IMAP, NNTP, POP3 golfing Eric Wong
                   ` (2 preceding siblings ...)
  2022-07-23  4:41 ` [PATCH 03/11] nntp: remove more() wrapper Eric Wong
@ 2022-07-23  4:41 ` Eric Wong
  2022-07-23  4:41 ` [PATCH 05/11] ds: move no-op ->zflush to common base class Eric Wong
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Eric Wong @ 2022-07-23  4:41 UTC (permalink / raw)
  To: meta

We can share some common code between IMAP, NNTP, and POP3
without too much trouble, so cut down our LoC.
---
 lib/PublicInbox/DS.pm   | 18 ++++++++++++++++++
 lib/PublicInbox/IMAP.pm | 23 ++++-------------------
 lib/PublicInbox/NNTP.pm | 25 +++++--------------------
 lib/PublicInbox/POP3.pm | 23 ++++-------------------
 4 files changed, 31 insertions(+), 58 deletions(-)

diff --git a/lib/PublicInbox/DS.pm b/lib/PublicInbox/DS.pm
index bf8c4466..79f7046f 100644
--- a/lib/PublicInbox/DS.pm
+++ b/lib/PublicInbox/DS.pm
@@ -347,6 +347,24 @@ retry:
     $DescriptorMap{$fd} = $self;
 }
 
+# for IMAP, NNTP, and POP3 which greet clients upon connect
+sub greet {
+	my ($self, $sock) = @_;
+	my $ev = EPOLLIN;
+	my $wbuf;
+	if ($sock->can('accept_SSL') && !$sock->accept_SSL) {
+		return CORE::close($sock) if $! != EAGAIN;
+		$ev = PublicInbox::TLS::epollbit() or return CORE::close($sock);
+		$wbuf = [ \&accept_tls_step, $self->can('do_greet')];
+	}
+	new($self, $sock, $ev | EPOLLONESHOT);
+	if ($wbuf) {
+		$self->{wbuf} = $wbuf;
+	} else {
+		$self->do_greet;
+	}
+	$self;
+}
 
 #####################################################################
 ### I N S T A N C E   M E T H O D S
diff --git a/lib/PublicInbox/IMAP.pm b/lib/PublicInbox/IMAP.pm
index 7e695fd8..89a278c1 100644
--- a/lib/PublicInbox/IMAP.pm
+++ b/lib/PublicInbox/IMAP.pm
@@ -36,7 +36,6 @@ use parent qw(PublicInbox::DS);
 use PublicInbox::Eml;
 use PublicInbox::EmlContentFoo qw(parse_content_disposition);
 use PublicInbox::DS qw(now);
-use PublicInbox::Syscall qw(EPOLLIN EPOLLONESHOT);
 use PublicInbox::GitAsyncCat;
 use Text::ParseWords qw(parse_line);
 use Errno qw(EAGAIN);
@@ -99,29 +98,15 @@ undef %FETCH_NEED;
 my $valid_range = '[0-9]+|[0-9]+:[0-9]+|[0-9]+:\*';
 $valid_range = qr/\A(?:$valid_range)(?:,(?:$valid_range))*\z/;
 
-sub greet ($) {
+sub do_greet {
 	my ($self) = @_;
 	my $capa = capa($self);
 	$self->write(\"* OK [$capa] public-inbox-imapd ready\r\n");
 }
 
-sub new ($$$) {
-	my ($class, $sock, $imapd) = @_;
-	my $self = bless { imapd => $imapd }, 'PublicInbox::IMAP_preauth';
-	my $ev = EPOLLIN;
-	my $wbuf;
-	if ($sock->can('accept_SSL') && !$sock->accept_SSL) {
-		return CORE::close($sock) if $! != EAGAIN;
-		$ev = PublicInbox::TLS::epollbit() or return CORE::close($sock);
-		$wbuf = [ \&PublicInbox::DS::accept_tls_step, \&greet ];
-	}
-	$self->SUPER::new($sock, $ev | EPOLLONESHOT);
-	if ($wbuf) {
-		$self->{wbuf} = $wbuf;
-	} else {
-		greet($self);
-	}
-	$self;
+sub new {
+	my (undef, $sock, $imapd) = @_;
+	(bless { imapd => $imapd }, 'PublicInbox::IMAP_preauth')->greet($sock)
 }
 
 sub logged_in { 1 }
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index e4ca7d14..533a0c59 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -29,7 +29,6 @@ use constant {
 	r225 =>	"225 Headers follow (multi-line)\r\n",
 	r430 => '430 No article with that message-id',
 };
-use PublicInbox::Syscall qw(EPOLLIN EPOLLONESHOT);
 use Errno qw(EAGAIN);
 my $ONE_MSGID = qr/\A$MID_EXTRACT\z/;
 my @OVERVIEW = qw(Subject From Date Message-ID References);
@@ -47,25 +46,11 @@ HDR\r
 OVER\r
 COMPRESS DEFLATE\r
 
-sub greet ($) { $_[0]->write($_[0]->{nntpd}->{greet}) };
-
-sub new ($$$) {
-	my ($class, $sock, $nntpd) = @_;
-	my $self = bless { nntpd => $nntpd }, $class;
-	my $ev = EPOLLIN;
-	my $wbuf;
-	if ($sock->can('accept_SSL') && !$sock->accept_SSL) {
-		return CORE::close($sock) if $! != EAGAIN;
-		$ev = PublicInbox::TLS::epollbit() or return CORE::close($sock);
-		$wbuf = [ \&PublicInbox::DS::accept_tls_step, \&greet ];
-	}
-	$self->SUPER::new($sock, $ev | EPOLLONESHOT);
-	if ($wbuf) {
-		$self->{wbuf} = $wbuf;
-	} else {
-		greet($self);
-	}
-	$self;
+sub do_greet ($) { $_[0]->write($_[0]->{nntpd}->{greet}) };
+
+sub new {
+	my ($cls, $sock, $nntpd) = @_;
+	(bless { nntpd => $nntpd }, $cls)->greet($sock)
 }
 
 sub args_ok ($$) {
diff --git a/lib/PublicInbox/POP3.pm b/lib/PublicInbox/POP3.pm
index ec73893c..c368615d 100644
--- a/lib/PublicInbox/POP3.pm
+++ b/lib/PublicInbox/POP3.pm
@@ -33,7 +33,6 @@
 package PublicInbox::POP3;
 use v5.12;
 use parent qw(PublicInbox::DS);
-use PublicInbox::Syscall qw(EPOLLIN EPOLLONESHOT);
 use PublicInbox::GitAsyncCat;
 use PublicInbox::DS qw(now);
 use Errno qw(EAGAIN);
@@ -113,29 +112,15 @@ sub long_response ($$;@) {
 	undef;
 }
 
-sub _greet ($) {
+sub do_greet {
 	my ($self) = @_;
 	my $s = $self->{salt} = sprintf('%x.%x', int(rand(0x7fffffff)), time);
 	$self->write("+OK POP3 server ready <$s\@public-inbox>\r\n");
 }
 
-sub new ($$$) {
-	my ($class, $sock, $pop3d) = @_;
-	my $self = bless { pop3d => $pop3d }, __PACKAGE__;
-	my $ev = EPOLLIN;
-	my $wbuf;
-	if ($sock->can('accept_SSL') && !$sock->accept_SSL) {
-		return CORE::close($sock) if $! != EAGAIN;
-		$ev = PublicInbox::TLS::epollbit() or return CORE::close($sock);
-		$wbuf = [ \&PublicInbox::DS::accept_tls_step, \&_greet ];
-	}
-	$self->SUPER::new($sock, $ev | EPOLLONESHOT);
-	if ($wbuf) {
-		$self->{wbuf} = $wbuf;
-	} else {
-		_greet($self);
-	}
-	$self;
+sub new {
+	my ($cls, $sock, $pop3d) = @_;
+	(bless { pop3d => $pop3d }, $cls)->greet($sock)
 }
 
 # POP user is $UUID1@$NEWSGROUP.$SLICE

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 05/11] ds: move no-op ->zflush to common base class
  2022-07-23  4:41 [PATCH 00/11] IMAP, NNTP, POP3 golfing Eric Wong
                   ` (3 preceding siblings ...)
  2022-07-23  4:41 ` [PATCH 04/11] ds: support greeting protocols Eric Wong
@ 2022-07-23  4:41 ` Eric Wong
  2022-07-23  4:41 ` [PATCH 06/11] ds: move requeue_once Eric Wong
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Eric Wong @ 2022-07-23  4:41 UTC (permalink / raw)
  To: meta

More deduplication, and POP3 never needed it.
---
 lib/PublicInbox/DS.pm   | 2 ++
 lib/PublicInbox/IMAP.pm | 2 --
 lib/PublicInbox/NNTP.pm | 2 --
 lib/PublicInbox/POP3.pm | 2 --
 4 files changed, 2 insertions(+), 6 deletions(-)

diff --git a/lib/PublicInbox/DS.pm b/lib/PublicInbox/DS.pm
index 79f7046f..daef9d89 100644
--- a/lib/PublicInbox/DS.pm
+++ b/lib/PublicInbox/DS.pm
@@ -648,6 +648,8 @@ sub shutdn ($) {
     }
 }
 
+sub zflush {} # overridden by NNTPdeflate and IMAPdeflate
+
 sub dwaitpid ($;$$) {
 	my ($pid, $cb, $arg) = @_;
 	if ($in_loop) {
diff --git a/lib/PublicInbox/IMAP.pm b/lib/PublicInbox/IMAP.pm
index 89a278c1..f4b08eee 100644
--- a/lib/PublicInbox/IMAP.pm
+++ b/lib/PublicInbox/IMAP.pm
@@ -1271,8 +1271,6 @@ sub event_step {
 
 sub compressed { undef }
 
-sub zflush {} # overridden by IMAPdeflate
-
 # RFC 4978
 sub cmd_compress ($$$) {
 	my ($self, $tag, $alg) = @_;
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 533a0c59..f0ee11cb 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -968,8 +968,6 @@ sub cmd_compress ($$) {
 	undef
 }
 
-sub zflush {} # overridden by NNTPdeflate
-
 sub cmd_xpath ($$) {
 	my ($self, $mid) = @_;
 	return r501 unless $mid =~ $ONE_MSGID;
diff --git a/lib/PublicInbox/POP3.pm b/lib/PublicInbox/POP3.pm
index c368615d..fcea4bb9 100644
--- a/lib/PublicInbox/POP3.pm
+++ b/lib/PublicInbox/POP3.pm
@@ -55,8 +55,6 @@ sub out ($$;@) {
 	printf { $self->{pop3d}->{out} } $fmt."\n", @args;
 }
 
-sub zflush {} # noop
-
 sub requeue_once ($) {
 	my ($self) = @_;
 	# COMPRESS users all share the same DEFLATE context.

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 06/11] ds: move requeue_once
  2022-07-23  4:41 [PATCH 00/11] IMAP, NNTP, POP3 golfing Eric Wong
                   ` (4 preceding siblings ...)
  2022-07-23  4:41 ` [PATCH 05/11] ds: move no-op ->zflush to common base class Eric Wong
@ 2022-07-23  4:41 ` Eric Wong
  2022-07-23  4:41 ` [PATCH 07/11] nntp: listgroup_range_i: remove useless `map' op Eric Wong
                   ` (4 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Eric Wong @ 2022-07-23  4:41 UTC (permalink / raw)
  To: meta

It's the same subroutine everywhere.
---
 lib/PublicInbox/DS.pm   | 14 ++++++++++++++
 lib/PublicInbox/IMAP.pm | 22 +++-------------------
 lib/PublicInbox/NNTP.pm | 13 ++-----------
 lib/PublicInbox/POP3.pm | 16 ----------------
 4 files changed, 19 insertions(+), 46 deletions(-)

diff --git a/lib/PublicInbox/DS.pm b/lib/PublicInbox/DS.pm
index daef9d89..f0181b54 100644
--- a/lib/PublicInbox/DS.pm
+++ b/lib/PublicInbox/DS.pm
@@ -650,6 +650,20 @@ sub shutdn ($) {
 
 sub zflush {} # overridden by NNTPdeflate and IMAPdeflate
 
+sub requeue_once {
+	my ($self) = @_;
+	# COMPRESS users all share the same DEFLATE context.
+	# Flush it here to ensure clients don't see
+	# each other's data
+	$self->zflush;
+
+	# no recursion, schedule another call ASAP,
+	# but only after all pending writes are done.
+	# autovivify wbuf.  wbuf may be populated by $cb,
+	# no need to rearm if so: (push returns new size of array)
+	requeue($self) if push(@{$self->{wbuf}}, $self->can('long_step')) == 1;
+}
+
 sub dwaitpid ($;$$) {
 	my ($pid, $cb, $arg) = @_;
 	if ($in_loop) {
diff --git a/lib/PublicInbox/IMAP.pm b/lib/PublicInbox/IMAP.pm
index f4b08eee..18a12564 100644
--- a/lib/PublicInbox/IMAP.pm
+++ b/lib/PublicInbox/IMAP.pm
@@ -563,22 +563,6 @@ sub fetch_body ($;$) {
 	join('', @hold);
 }
 
-sub requeue_once ($) {
-	my ($self) = @_;
-	# COMPRESS users all share the same DEFLATE context.
-	# Flush it here to ensure clients don't see
-	# each other's data
-	$self->zflush;
-
-	# no recursion, schedule another call ASAP,
-	# but only after all pending writes are done.
-	# autovivify wbuf:
-	my $new_size = push(@{$self->{wbuf}}, \&long_step);
-
-	# wbuf may be populated by $cb, no need to rearm if so:
-	$self->requeue if $new_size == 1;
-}
-
 sub fetch_run_ops {
 	my ($self, $smsg, $bref, $ops, $partial) = @_;
 	my $uid = $smsg->{num};
@@ -601,7 +585,7 @@ sub fetch_blob_cb { # called by git->cat_async via ibx_async_cat
 		# it's possible to have TOCTOU if an admin runs
 		# public-inbox-(edit|purge), just move onto the next message
 		warn "E: $smsg->{blob} missing in $ibx->{inboxdir}\n";
-		return requeue_once($self);
+		return $self->requeue_once;
 	} else {
 		$smsg->{blob} eq $oid or die "BUG: $smsg->{blob} != $oid";
 	}
@@ -611,7 +595,7 @@ sub fetch_blob_cb { # called by git->cat_async via ibx_async_cat
 					\&fetch_blob_cb, $fetch_arg);
 	}
 	fetch_run_ops($self, $smsg, $bref, $ops, $partial);
-	$pre ? $self->zflush : requeue_once($self);
+	$pre ? $self->zflush : $self->requeue_once;
 }
 
 sub emit_rfc822 {
@@ -1198,7 +1182,7 @@ sub long_step {
 		$self->close;
 	} elsif ($more) { # $self->{wbuf}:
 		# control passed to ibx_async_cat if $more == \undef
-		requeue_once($self) if !ref($more);
+		$self->requeue_once($self) if !ref($more);
 	} else { # all done!
 		delete $self->{long_cb};
 		my $elapsed = now() - $t0;
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index f0ee11cb..43219b36 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -635,17 +635,8 @@ sub long_step {
 		out($self, " deferred[$fd] aborted - %0.6f", $elapsed);
 		$self->close;
 	} elsif ($more) { # $self->{wbuf}:
-		# COMPRESS users all share the same DEFLATE context.
-		# Flush it here to ensure clients don't see
-		# each other's data
-		$self->zflush;
-
-		# no recursion, schedule another call ASAP, but only after
-		# all pending writes are done.  autovivify wbuf:
-		my $new_size = push(@{$self->{wbuf}}, \&long_step);
-
-		# wbuf may be populated by $cb, no need to rearm if so:
-		$self->requeue if $new_size == 1;
+		# control passed to ibx_async_cat if $more == \undef
+		$self->requeue_once if !ref($more);
 	} else { # all done!
 		delete $self->{long_cb};
 		$self->write(\".\r\n");
diff --git a/lib/PublicInbox/POP3.pm b/lib/PublicInbox/POP3.pm
index fcea4bb9..741b5e58 100644
--- a/lib/PublicInbox/POP3.pm
+++ b/lib/PublicInbox/POP3.pm
@@ -55,22 +55,6 @@ sub out ($$;@) {
 	printf { $self->{pop3d}->{out} } $fmt."\n", @args;
 }
 
-sub requeue_once ($) {
-	my ($self) = @_;
-	# COMPRESS users all share the same DEFLATE context.
-	# Flush it here to ensure clients don't see
-	# each other's data
-	$self->zflush;
-
-	# no recursion, schedule another call ASAP,
-	# but only after all pending writes are done.
-	# autovivify wbuf:
-	my $new_size = push(@{$self->{wbuf}}, \&long_step);
-
-	# wbuf may be populated by $cb, no need to rearm if so:
-	$self->requeue if $new_size == 1;
-}
-
 sub long_step {
 	my ($self) = @_;
 	# wbuf is unset or empty, here; {long} may add to it

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 07/11] nntp: listgroup_range_i: remove useless `map' op
  2022-07-23  4:41 [PATCH 00/11] IMAP, NNTP, POP3 golfing Eric Wong
                   ` (5 preceding siblings ...)
  2022-07-23  4:41 ` [PATCH 06/11] ds: move requeue_once Eric Wong
@ 2022-07-23  4:41 ` Eric Wong
  2022-07-23  4:41 ` [PATCH 08/11] nntp: inline CRLF in all response lines Eric Wong
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Eric Wong @ 2022-07-23  4:41 UTC (permalink / raw)
  To: meta

No need to iterate through the array twice; and this even seems
a hair faster than what I got with commit 726d6e71aee5d974
(nntp: small speed up for multi-line responses, 2020-12-04)
---
 lib/PublicInbox/NNTP.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 43219b36..ab6eb525 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -189,7 +189,7 @@ sub listgroup_range_i {
 	my ($self, $beg, $end) = @_;
 	my $r = $self->{ibx}->mm(1)->msg_range($beg, $end, 'num');
 	scalar(@$r) or return;
-	$self->msg_more(join('', map { "$_->[0]\r\n" } @$r));
+	$self->msg_more(join("\r\n", @$r, ''));
 	1;
 }
 

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 08/11] nntp: inline CRLF in all response lines
  2022-07-23  4:41 [PATCH 00/11] IMAP, NNTP, POP3 golfing Eric Wong
                   ` (6 preceding siblings ...)
  2022-07-23  4:41 ` [PATCH 07/11] nntp: listgroup_range_i: remove useless `map' op Eric Wong
@ 2022-07-23  4:41 ` Eric Wong
  2022-07-23  4:41 ` [PATCH 09/11] ds: share long_step between NNTP and IMAP Eric Wong
                   ` (2 subsequent siblings)
  10 siblings, 0 replies; 12+ messages in thread
From: Eric Wong @ 2022-07-23  4:41 UTC (permalink / raw)
  To: meta

This brings NNTP closer to POP3 and IMAP implementations
to allow CoW avoidance on constants.
---
 lib/PublicInbox/NNTP.pm        | 111 ++++++++++++++-------------------
 lib/PublicInbox/NNTPdeflate.pm |   6 +-
 t/nntpd.t                      |   2 +-
 3 files changed, 51 insertions(+), 68 deletions(-)

diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index ab6eb525..5eb6112c 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -22,20 +22,19 @@ use PublicInbox::Address;
 
 use constant {
 	LINE_MAX => 512, # RFC 977 section 2.3
-	r501 => '501 command syntax error',
-	r502 => '502 Command unavailable',
+	r501 => "501 command syntax error\r\n",
+	r502 => "502 Command unavailable\r\n",
 	r221 => "221 Header follows\r\n",
-	r224 => '224 Overview information follows (multi-line)',
 	r225 =>	"225 Headers follow (multi-line)\r\n",
-	r430 => '430 No article with that message-id',
+	r430 => "430 No article with that message-id\r\n",
 };
 use Errno qw(EAGAIN);
 my $ONE_MSGID = qr/\A$MID_EXTRACT\z/;
 my @OVERVIEW = qw(Subject From Date Message-ID References);
 my $OVERVIEW_FMT = join(":\r\n", @OVERVIEW, qw(Bytes Lines), '') .
-		"Xref:full\r\n.";
+		"Xref:full\r\n.\r\n";
 my $LIST_HEADERS = join("\r\n", @OVERVIEW,
-			qw(:bytes :lines Xref To Cc)) . "\r\n.";
+			qw(:bytes :lines Xref To Cc)) . "\r\n.\r\n";
 my $CAPABILITIES = <<"";
 101 Capability list:\r
 VERSION 2\r
@@ -69,17 +68,16 @@ sub process_line ($$) {
 	return 1 unless defined($req); # skip blank line
 	$req = $self->can('cmd_'.lc($req)) //
 		return $self->write(\"500 command not recognized\r\n");
-	return res($self, r501) unless args_ok($req, scalar @args);
-
+	return $self->write(\r501) unless args_ok($req, scalar @args);
 	my $res = eval { $req->($self, @args) };
 	my $err = $@;
 	if ($err && $self->{sock}) {
 		local $/ = "\n";
 		chomp($l);
 		err($self, 'error from: %s (%s)', $l, $err);
-		$res = '503 program fault - command not performed';
+		$res = \"503 program fault - command not performed\r\n";
 	}
-	defined($res) ? res($self, $res) : 0;
+	defined($res) ? $self->write($res) : 0;
 }
 
 # The keyword argument is not used (rfc3977 5.2.2)
@@ -90,15 +88,15 @@ sub cmd_capabilities ($;$) {
 			$self->{nntpd}->{accept_tls}) {
 		$res .= "STARTTLS\r\n";
 	}
-	$res .= '.';
+	$res .= ".\r\n";
 }
 
 sub cmd_mode ($$) {
 	my ($self, $arg) = @_;
-	uc($arg) eq 'READER' ? '201 Posting prohibited' : r501;
+	uc($arg) eq 'READER' ? \"201 Posting prohibited\r\n" : \r501;
 }
 
-sub cmd_slave ($) { '202 slave status noted' }
+sub cmd_slave ($) { \"202 slave status noted\r\n" }
 
 sub cmd_xgtitle ($;$) {
 	my ($self, $wildmat) = @_;
@@ -205,10 +203,10 @@ sub cmd_listgroup ($;$$) {
 	my ($self, $group, $range) = @_;
 	if (defined $group) {
 		my $res = cmd_group($self, $group);
-		return $res if ($res !~ /\A211 /);
-		$self->msg_more($res .= "\r\n");
+		return $res if ref($res); # error if const strref
+		$self->msg_more($res);
 	}
-	$self->{ibx} or return '412 no newsgroup selected';
+	$self->{ibx} or return \"412 no newsgroup selected\r\n";
 	if (defined $range) {
 		my $r = get_range($self, $range);
 		return $r unless ref $r;
@@ -339,7 +337,7 @@ sub cmd_newnews ($$$$;$$) {
 	ngpat2re($skip);
 	my @names = grep(!/$skip/, grep(/$keep/,
 				@{$self->{nntpd}->{groupnames}}));
-	return '.' unless scalar(@names);
+	return ".\r\n" unless scalar(@names);
 	my $prev = 0;
 	long_response($self, \&newnews_i, \@names, $ts, \$prev);
 }
@@ -348,30 +346,29 @@ sub cmd_group ($$) {
 	my ($self, $group) = @_;
 	my $nntpd = $self->{nntpd};
 	my $ibx = $nntpd->{pi_cfg}->{-by_newsgroup}->{$group} or
-		return '411 no such news group';
+		return \"411 no such news group\r\n";
 	$nntpd->idler_start;
 
 	$self->{ibx} = $ibx;
 	my ($min, $max) = $ibx->mm(1)->minmax;
 	$self->{article} = $min;
 	my $est_size = $max - $min;
-	"211 $est_size $min $max $group";
+	"211 $est_size $min $max $group\r\n";
 }
 
 sub article_adj ($$) {
 	my ($self, $off) = @_;
-	my $ibx = $self->{ibx} or return '412 no newsgroup selected';
-
-	my $n = $self->{article};
-	defined $n or return '420 no current article has been selected';
+	my $ibx = $self->{ibx} // return \"412 no newsgroup selected\r\n";
+	my $n = $self->{article} //
+		return \"420 no current article has been selected\r\n";
 
 	$n += $off;
 	my $mid = $ibx->mm(1)->mid_for($n) // do {
 		$n = $off > 0 ? 'next' : 'previous';
-		return "421 no $n article in this group";
+		return "421 no $n article in this group\r\n";
 	};
 	$self->{article} = $n;
-	"223 $n <$mid> article retrieved - request text separately";
+	"223 $n <$mid> article retrieved - request text separately\r\n";
 }
 
 sub cmd_next ($) { article_adj($_[0], 1) }
@@ -382,8 +379,8 @@ sub cmd_last ($) { article_adj($_[0], -1) }
 sub cmd_post ($) {
 	my ($self) = @_;
 	my $ibx = $self->{ibx};
-	$ibx ? "440 mailto:$ibx->{-primary_address} to post"
-		: '440 posting not allowed'
+	$ibx ? "440 mailto:$ibx->{-primary_address} to post\r\n"
+		: \"440 posting not allowed\r\n"
 }
 
 sub cmd_quit ($) {
@@ -471,22 +468,22 @@ sub art_lookup ($$$) {
 	my $err;
 	if (defined $art) {
 		if ($art =~ /\A[0-9]+\z/) {
-			$err = '423 no such article number in this group';
+			$err = \"423 no such article number in this group\r\n";
 			$n = int($art);
 			goto find_ibx;
 		} elsif ($art =~ $ONE_MSGID) {
 			($ibx, $n) = mid_lookup($self, $1);
 			goto found if $ibx;
-			return r430;
+			return \r430;
 		} else {
-			return r501;
+			return \r501;
 		}
 	} else {
-		$err = '420 no current article has been selected';
+		$err = \"420 no current article has been selected\r\n";
 		$n = $self->{article} // return $err;
 find_ibx:
 		$ibx = $self->{ibx} or
-				return '412 no newsgroup has been selected';
+			return \"412 no newsgroup has been selected\r\n";
 	}
 found:
 	my $smsg = $ibx->over(1)->get_art($n) or return $err;
@@ -494,7 +491,7 @@ found:
 	if ($code == 223) { # STAT
 		set_art($self, $n);
 		"223 $n <$smsg->{mid}> article retrieved - " .
-			"request text separately";
+			"request text separately\r\n";
 	} else { # HEAD | BODY | ARTICLE
 		$smsg->{nntp} = $self;
 		$smsg->{nntp_code} = $code;
@@ -588,20 +585,18 @@ sub cmd_stat ($;$) {
 	art_lookup($self, $art, 223); # art may be msgid
 }
 
-sub cmd_ihave ($) { '435 article not wanted - do not send it' }
+sub cmd_ihave ($) { \"435 article not wanted - do not send it\r\n" }
 
-sub cmd_date ($) { '111 '.strftime('%Y%m%d%H%M%S', gmtime(time)) }
+sub cmd_date ($) { '111 '.strftime('%Y%m%d%H%M%S', gmtime(time))."\r\n" }
 
-sub cmd_help ($) {
-	my ($self) = @_;
-	$self->msg_more("100 help text follows\r\n");
-	'.'
-}
+sub cmd_help ($) { \"100 help text follows\r\n.\r\n" }
 
+# returns a ref on success
 sub get_range ($$) {
 	my ($self, $range) = @_;
-	my $ibx = $self->{ibx} or return '412 no news group has been selected';
-	defined $range or return '420 No article(s) selected';
+	my $ibx = $self->{ibx} //
+		return "412 no news group has been selected\r\n";
+	$range // return "420 No article(s) selected\r\n";
 	my ($beg, $end);
 	my ($min, $max) = $ibx->mm(1)->minmax;
 	if ($range =~ /\A([0-9]+)\z/) {
@@ -615,8 +610,7 @@ sub get_range ($$) {
 	}
 	$beg = $min if ($beg < $min);
 	$end = $max if ($end > $max);
-	return '420 No article(s) selected' if ($beg > $end);
-	[ \$beg, $end ];
+	$beg > $end ? "420 No article(s) selected\r\n" : [ \$beg, $end ];
 }
 
 sub long_step {
@@ -639,7 +633,7 @@ sub long_step {
 		$self->requeue_once if !ref($more);
 	} else { # all done!
 		delete $self->{long_cb};
-		$self->write(\".\r\n");
+		$self->write(\".\r\n"); # TODO get rid of this
 		my $elapsed = now() - $t0;
 		my $fd = fileno($self->{sock});
 		out($self, " deferred[$fd] done - %0.6f", $elapsed);
@@ -813,7 +807,7 @@ sub do_hdr ($$$;$) {
 	} elsif ($sub =~ /\A:(bytes|lines)\z/) {
 		hdr_smsg($self, $xhdr, $1, $range);
 	} else {
-		$xhdr ? (r221 . '.') : "503 HDR not permitted on $header";
+		$xhdr ? (r221.".\r\n") : "503 HDR not permitted on $header\r\n";
 	}
 }
 
@@ -858,9 +852,9 @@ sub xrover_i {
 
 sub cmd_xrover ($;$) {
 	my ($self, $range) = @_;
-	my $ibx = $self->{ibx} or return '412 no newsgroup selected';
+	my $ibx = $self->{ibx} or return \"412 no newsgroup selected\r\n";
 	(defined $range && $range =~ /[<>]/) and
-		return '420 No article(s) selected'; # no message IDs
+		return \"420 No article(s) selected\r\n"; # no message IDs
 
 	$range = $self->{article} unless defined $range;
 	my $r = get_range($self, $range);
@@ -903,8 +897,7 @@ sub cmd_over ($;$) {
 			$smsg->{-orig_num} = $smsg->{num};
 			$smsg->{num} = 0;
 		}
-		$self->msg_more(over_line($self, $ibx, $smsg));
-		'.';
+		over_line($self, $ibx, $smsg).".\r\n";
 	} else {
 		cmd_xover($self, $range);
 	}
@@ -942,7 +935,7 @@ sub cmd_starttls ($) {
 	# RFC 4642 2.2.1
 	return r502 if ($sock->can('accept_SSL') || $self->compressed);
 	my $opt = $self->{nntpd}->{accept_tls} or
-		return '580 can not initiate TLS negotiation';
+		return \"580 can not initiate TLS negotiation\r\n";
 	$self->write(\"382 Continue with TLS negotiation\r\n");
 	$self->{sock} = IO::Socket::SSL->start_SSL($sock, %$opt);
 	$self->requeue if PublicInbox::DS::accept_tls_step($self);
@@ -952,7 +945,7 @@ sub cmd_starttls ($) {
 # RFC 8054
 sub cmd_compress ($$) {
 	my ($self, $alg) = @_;
-	return '503 Only DEFLATE is supported' if uc($alg) ne 'DEFLATE';
+	return "503 Only DEFLATE is supported\r\n" if uc($alg) ne 'DEFLATE';
 	return r502 if $self->compressed;
 	PublicInbox::NNTPdeflate->enable($self);
 	$self->requeue;
@@ -984,18 +977,8 @@ sub cmd_xpath ($$) {
 			push @paths, "$ibx->{newsgroup}/$n";
 		}
 	}
-	return '430 no such article on server' unless @paths;
-	'223 '.join(' ', sort(@paths));
-}
-
-sub res ($$) { do_write($_[0], $_[1] . "\r\n") }
-
-sub do_write ($$) {
-	my $self = $_[0];
-	my $done = $self->write(\($_[1]));
-	return 0 unless $self->{sock};
-
-	$done;
+	return \"430 no such article on server\r\n" unless @paths;
+	'223 '.join(' ', sort(@paths))."\r\n";
 }
 
 sub err ($$;@) {
diff --git a/lib/PublicInbox/NNTPdeflate.pm b/lib/PublicInbox/NNTPdeflate.pm
index 06b4499c..352d4842 100644
--- a/lib/PublicInbox/NNTPdeflate.pm
+++ b/lib/PublicInbox/NNTPdeflate.pm
@@ -1,4 +1,4 @@
-# Copyright (C) 2019-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 
 # RFC 8054 NNTP COMPRESS DEFLATE implementation
@@ -48,10 +48,10 @@ sub enable {
 	my ($in, $err) = Compress::Raw::Zlib::Inflate->new(%IN_OPT);
 	if ($err != Z_OK) {
 		$self->err("Inflate->new failed: $err");
-		$self->res('403 Unable to activate compression');
+		$self->write(\"403 Unable to activate compression\r\n");
 		return;
 	}
-	$self->res('206 Compression active');
+	$self->write(\"206 Compression active\r\n");
 	bless $self, $class;
 	$self->{zin} = $in;
 }
diff --git a/t/nntpd.t b/t/nntpd.t
index cf1c44f8..34e9e1b4 100644
--- a/t/nntpd.t
+++ b/t/nntpd.t
@@ -1,5 +1,5 @@
 #!perl -w
-# Copyright (C) 2015-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 use strict; use v5.10.1; use PublicInbox::TestCommon;
 require_mods(qw(DBD::SQLite));

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 09/11] ds: share long_step between NNTP and IMAP
  2022-07-23  4:41 [PATCH 00/11] IMAP, NNTP, POP3 golfing Eric Wong
                   ` (7 preceding siblings ...)
  2022-07-23  4:41 ` [PATCH 08/11] nntp: inline CRLF in all response lines Eric Wong
@ 2022-07-23  4:41 ` Eric Wong
  2022-07-23  4:41 ` [PATCH 10/11] nntp: resolve inboxes immediately on group listings Eric Wong
  2022-07-23  4:41 ` [PATCH 11/11] imap+nntp: share COMPRESS implementation Eric Wong
  10 siblings, 0 replies; 12+ messages in thread
From: Eric Wong @ 2022-07-23  4:41 UTC (permalink / raw)
  To: meta

It's not actually used by our POP3 code at the moment,
but it may be soon to reduce memory usage when loading
50K smsg objects into memory.
---
 lib/PublicInbox/DS.pm   | 44 +++++++++++++++++++++++++--
 lib/PublicInbox/IMAP.pm | 47 ++---------------------------
 lib/PublicInbox/NNTP.pm | 67 +++++++++--------------------------------
 lib/PublicInbox/POP3.pm | 39 ------------------------
 4 files changed, 58 insertions(+), 139 deletions(-)

diff --git a/lib/PublicInbox/DS.pm b/lib/PublicInbox/DS.pm
index f0181b54..fee31e3d 100644
--- a/lib/PublicInbox/DS.pm
+++ b/lib/PublicInbox/DS.pm
@@ -650,18 +650,56 @@ sub shutdn ($) {
 
 sub zflush {} # overridden by NNTPdeflate and IMAPdeflate
 
+sub long_response_done {} # overridden by Net::NNTP
+
+sub long_step {
+	my ($self) = @_;
+	# wbuf is unset or empty, here; {long} may add to it
+	my ($fd, $cb, $t0, @args) = @{$self->{long_cb}};
+	my $more = eval { $cb->($self, @args) };
+	if ($@ || !$self->{sock}) { # something bad happened...
+		delete $self->{long_cb};
+		my $elapsed = now() - $t0;
+		$@ and $self->err("%s during long response[$fd] - %0.6f",
+				    $@, $elapsed);
+		$self->out(" deferred[$fd] aborted - %0.6f", $elapsed);
+		$self->close;
+	} elsif ($more) { # $self->{wbuf}:
+		# control passed to ibx_async_cat if $more == \undef
+		requeue_once($self) if !ref($more);
+	} else { # all done!
+		delete $self->{long_cb};
+		$self->long_response_done;
+		my $elapsed = now() - $t0;
+		my $fd = fileno($self->{sock});
+		$self->out(" deferred[$fd] done - %0.6f", $elapsed);
+		my $wbuf = $self->{wbuf}; # do NOT autovivify
+		requeue($self) unless $wbuf && @$wbuf;
+	}
+}
+
 sub requeue_once {
 	my ($self) = @_;
 	# COMPRESS users all share the same DEFLATE context.
-	# Flush it here to ensure clients don't see
-	# each other's data
+	# Flush it here to ensure clients don't see each other's data
 	$self->zflush;
 
 	# no recursion, schedule another call ASAP,
 	# but only after all pending writes are done.
 	# autovivify wbuf.  wbuf may be populated by $cb,
 	# no need to rearm if so: (push returns new size of array)
-	requeue($self) if push(@{$self->{wbuf}}, $self->can('long_step')) == 1;
+	requeue($self) if push(@{$self->{wbuf}}, \&long_step) == 1;
+}
+
+sub long_response ($$;@) {
+	my ($self, $cb, @args) = @_; # cb returns true if more, false if done
+	my $sock = $self->{sock} or return;
+	# make sure we disable reading during a long response,
+	# clients should not be sending us stuff and making us do more
+	# work while we are stream a response to them
+	$self->{long_cb} = [ fileno($sock), $cb, now(), @args ];
+	long_step($self); # kick off!
+	undef;
 }
 
 sub dwaitpid ($;$$) {
diff --git a/lib/PublicInbox/IMAP.pm b/lib/PublicInbox/IMAP.pm
index 18a12564..ce0dce0f 100644
--- a/lib/PublicInbox/IMAP.pm
+++ b/lib/PublicInbox/IMAP.pm
@@ -1035,7 +1035,7 @@ sub cmd_uid_fetch ($$$$;@) {
 	my $range_info = range_step($self, \$range_csv);
 	return "$tag $range_info\r\n" if !ref($range_info);
 	uo2m_hibernate($self) if $cb == \&fetch_blob; # slow, save RAM
-	long_response($self, $cb, $tag, [], $range_info, $ops, $partial);
+	$self->long_response($cb, $tag, [], $range_info, $ops, $partial);
 }
 
 sub cmd_fetch ($$$$;@) {
@@ -1050,7 +1050,7 @@ sub cmd_fetch ($$$$;@) {
 	my $range_info = range_step($self, \$range_csv);
 	return "$tag $range_info\r\n" if !ref($range_info);
 	uo2m_hibernate($self) if $cb == \&fetch_blob; # slow, save RAM
-	long_response($self, $cb, $tag, [], $range_info, $ops, $partial);
+	$self->long_response($cb, $tag, [], $range_info, $ops, $partial);
 }
 
 sub msn_convert ($$) {
@@ -1094,7 +1094,7 @@ sub search_common {
 	my ($sql, $range_info) = delete @$q{qw(sql range_info)};
 	if (!scalar(keys %$q)) { # overview.sqlite3
 		$self->msg_more('* SEARCH');
-		long_response($self, \&search_uid_range,
+		$self->long_response(\&search_uid_range,
 				$tag, $sql, $range_info, $want_msn);
 	} elsif ($q = $q->{xap}) {
 		my $srch = $self->{ibx}->isrch or
@@ -1165,35 +1165,6 @@ sub process_line ($$) {
 	$self->write($res);
 }
 
-sub long_step {
-	my ($self) = @_;
-	# wbuf is unset or empty, here; {long} may add to it
-	my ($fd, $cb, $t0, @args) = @{$self->{long_cb}};
-	my $more = eval { $cb->($self, @args) };
-	if ($@ || !$self->{sock}) { # something bad happened...
-		delete $self->{long_cb};
-		my $elapsed = now() - $t0;
-		if ($@) {
-			err($self,
-			    "%s during long response[$fd] - %0.6f",
-			    $@, $elapsed);
-		}
-		out($self, " deferred[$fd] aborted - %0.6f", $elapsed);
-		$self->close;
-	} elsif ($more) { # $self->{wbuf}:
-		# control passed to ibx_async_cat if $more == \undef
-		$self->requeue_once($self) if !ref($more);
-	} else { # all done!
-		delete $self->{long_cb};
-		my $elapsed = now() - $t0;
-		my $fd = fileno($self->{sock});
-		out($self, " deferred[$fd] done - %0.6f", $elapsed);
-		my $wbuf = $self->{wbuf}; # do NOT autovivify
-
-		$self->requeue unless $wbuf && @$wbuf;
-	}
-}
-
 sub err ($$;@) {
 	my ($self, $fmt, @args) = @_;
 	printf { $self->{imapd}->{err} } $fmt."\n", @args;
@@ -1204,18 +1175,6 @@ sub out ($$;@) {
 	printf { $self->{imapd}->{out} } $fmt."\n", @args;
 }
 
-sub long_response ($$;@) {
-	my ($self, $cb, @args) = @_; # cb returns true if more, false if done
-
-	my $sock = $self->{sock} or return;
-	# make sure we disable reading during a long response,
-	# clients should not be sending us stuff and making us do more
-	# work while we are stream a response to them
-	$self->{long_cb} = [ fileno($sock), $cb, now(), @args ];
-	long_step($self); # kick off!
-	undef;
-}
-
 # callback used by PublicInbox::DS for any (e)poll (in/out/hup/err)
 sub event_step {
 	my ($self) = @_;
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 5eb6112c..2a59cbd7 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -122,7 +122,7 @@ sub list_active_i { # "LIST ACTIVE" and also just "LIST" (no args)
 sub list_active ($;$) { # called by cmd_list
 	my ($self, $wildmat) = @_;
 	wildmat2re($wildmat);
-	long_response($self, \&list_active_i, [
+	$self->long_response(\&list_active_i, [
 		grep(/$wildmat/, @{$self->{nntpd}->{groupnames}}) ]);
 }
 
@@ -141,7 +141,7 @@ sub list_active_times_i {
 sub list_active_times ($;$) { # called by cmd_list
 	my ($self, $wildmat) = @_;
 	wildmat2re($wildmat);
-	long_response($self, \&list_active_times_i, [
+	$self->long_response(\&list_active_times_i, [
 		grep(/$wildmat/, @{$self->{nntpd}->{groupnames}}) ]);
 }
 
@@ -160,7 +160,7 @@ sub list_newsgroups_i {
 sub list_newsgroups ($;$) { # called by cmd_list
 	my ($self, $wildmat) = @_;
 	wildmat2re($wildmat);
-	long_response($self, \&list_newsgroups_i, [
+	$self->long_response(\&list_newsgroups_i, [
 		grep(/$wildmat/, @{$self->{nntpd}->{groupnames}}) ]);
 }
 
@@ -178,7 +178,7 @@ sub cmd_list ($;$$) {
 		$arg->($self, @args);
 	} else {
 		$self->msg_more("215 list of newsgroups follows\r\n");
-		long_response($self, \&list_active_i, [ # copy array
+		$self->long_response(\&list_active_i, [ # copy array
 			@{$self->{nntpd}->{groupnames}} ]);
 	}
 }
@@ -210,9 +210,9 @@ sub cmd_listgroup ($;$$) {
 	if (defined $range) {
 		my $r = get_range($self, $range);
 		return $r unless ref $r;
-		long_response($self, \&listgroup_range_i, @$r);
+		$self->long_response(\&listgroup_range_i, @$r);
 	} else { # grab every article number
-		long_response($self, \&listgroup_all_i, \(my $num = 0));
+		$self->long_response(\&listgroup_all_i, \(my $num = 0));
 	}
 }
 
@@ -268,7 +268,7 @@ sub cmd_newgroups ($$$;$$) {
 
 	# TODO dists
 	$self->msg_more("231 list of new newsgroups follows\r\n");
-	long_response($self, \&newgroups_i, $ts, \(my $i = 0),
+	$self->long_response(\&newgroups_i, $ts, \(my $i = 0),
 				$self->{nntpd}->{groupnames});
 }
 
@@ -339,7 +339,7 @@ sub cmd_newnews ($$$$;$$) {
 				@{$self->{nntpd}->{groupnames}}));
 	return ".\r\n" unless scalar(@names);
 	my $prev = 0;
-	long_response($self, \&newnews_i, \@names, $ts, \$prev);
+	$self->long_response(\&newnews_i, \@names, $ts, \$prev);
 }
 
 sub cmd_group ($$) {
@@ -613,46 +613,7 @@ sub get_range ($$) {
 	$beg > $end ? "420 No article(s) selected\r\n" : [ \$beg, $end ];
 }
 
-sub long_step {
-	my ($self) = @_;
-	# wbuf is unset or empty, here; {long} may add to it
-	my ($fd, $cb, $t0, @args) = @{$self->{long_cb}};
-	my $more = eval { $cb->($self, @args) };
-	if ($@ || !$self->{sock}) { # something bad happened...
-		delete $self->{long_cb};
-		my $elapsed = now() - $t0;
-		if ($@) {
-			err($self,
-			    "%s during long response[$fd] - %0.6f",
-			    $@, $elapsed);
-		}
-		out($self, " deferred[$fd] aborted - %0.6f", $elapsed);
-		$self->close;
-	} elsif ($more) { # $self->{wbuf}:
-		# control passed to ibx_async_cat if $more == \undef
-		$self->requeue_once if !ref($more);
-	} else { # all done!
-		delete $self->{long_cb};
-		$self->write(\".\r\n"); # TODO get rid of this
-		my $elapsed = now() - $t0;
-		my $fd = fileno($self->{sock});
-		out($self, " deferred[$fd] done - %0.6f", $elapsed);
-		my $wbuf = $self->{wbuf}; # do NOT autovivify
-		$self->requeue unless $wbuf && @$wbuf;
-	}
-}
-
-sub long_response ($$;@) {
-	my ($self, $cb, @args) = @_; # cb returns true if more, false if done
-
-	my $sock = $self->{sock} or return;
-	# make sure we disable reading during a long response,
-	# clients should not be sending us stuff and making us do more
-	# work while we are stream a response to them
-	$self->{long_cb} = [ fileno($sock), $cb, now(), @args ];
-	long_step($self); # kick off!
-	undef;
-}
+sub long_response_done { $_[0]->write(\".\r\n") } # overrides superclass
 
 sub hdr_msgid_range_i {
 	my ($self, $beg, $end) = @_;
@@ -674,7 +635,7 @@ sub hdr_message_id ($$$) { # optimize XHDR Message-ID [range] for slrnpull.
 		my $r = get_range($self, $range);
 		return $r unless ref $r;
 		$self->msg_more($xhdr ? r221 : r225);
-		long_response($self, \&hdr_msgid_range_i, @$r);
+		$self->long_response(\&hdr_msgid_range_i, @$r);
 	}
 }
 
@@ -746,7 +707,7 @@ sub hdr_xref ($$$) { # optimize XHDR Xref [range] for rtin
 		my $r = get_range($self, $range);
 		return $r unless ref $r;
 		$self->msg_more($xhdr ? r221 : r225);
-		long_response($self, \&xref_range_i, @$r);
+		$self->long_response(\&xref_range_i, @$r);
 	}
 }
 
@@ -790,7 +751,7 @@ sub hdr_smsg ($$$$) {
 		my $r = get_range($self, $range);
 		return $r unless ref $r;
 		$self->msg_more($xhdr ? r221 : r225);
-		long_response($self, \&smsg_range_i, @$r, $field);
+		$self->long_response(\&smsg_range_i, @$r, $field);
 	}
 }
 
@@ -860,7 +821,7 @@ sub cmd_xrover ($;$) {
 	my $r = get_range($self, $range);
 	return $r unless ref $r;
 	$self->msg_more("224 Overview information follows\r\n");
-	long_response($self, \&xrover_i, @$r);
+	$self->long_response(\&xrover_i, @$r);
 }
 
 sub over_line ($$$) {
@@ -924,7 +885,7 @@ sub cmd_xover ($;$) {
 	my ($beg, $end) = @$r;
 	$self->msg_more(
 		"224 Overview information follows for $$beg to $end\r\n");
-	long_response($self, \&xover_i, @$r);
+	$self->long_response(\&xover_i, @$r);
 }
 
 sub compressed { undef }
diff --git a/lib/PublicInbox/POP3.pm b/lib/PublicInbox/POP3.pm
index 741b5e58..60eedea7 100644
--- a/lib/PublicInbox/POP3.pm
+++ b/lib/PublicInbox/POP3.pm
@@ -55,45 +55,6 @@ sub out ($$;@) {
 	printf { $self->{pop3d}->{out} } $fmt."\n", @args;
 }
 
-sub long_step {
-	my ($self) = @_;
-	# wbuf is unset or empty, here; {long} may add to it
-	my ($fd, $cb, $t0, @args) = @{$self->{long_cb}};
-	my $more = eval { $cb->($self, @args) };
-	if ($@ || !$self->{sock}) { # something bad happened...
-		delete $self->{long_cb};
-		my $elapsed = now() - $t0;
-		if ($@) {
-			err($self,
-			    "%s during long response[$fd] - %0.6f",
-			    $@, $elapsed);
-		}
-		out($self, " deferred[$fd] aborted - %0.6f", $elapsed);
-		$self->close;
-	} elsif ($more) { # $self->{wbuf}:
-		# control passed to ibx_async_cat if $more == \undef
-		requeue_once($self) if !ref($more);
-	} else { # all done!
-		delete $self->{long_cb};
-		my $elapsed = now() - $t0;
-		my $fd = fileno($self->{sock});
-		out($self, " deferred[$fd] done - %0.6f", $elapsed);
-		my $wbuf = $self->{wbuf}; # do NOT autovivify
-		$self->requeue unless $wbuf && @$wbuf;
-	}
-}
-
-sub long_response ($$;@) {
-	my ($self, $cb, @args) = @_; # cb returns true if more, false if done
-	my $sock = $self->{sock} or return;
-	# make sure we disable reading during a long response,
-	# clients should not be sending us stuff and making us do more
-	# work while we are stream a response to them
-	$self->{long_cb} = [ fileno($sock), $cb, now(), @args ];
-	long_step($self); # kick off!
-	undef;
-}
-
 sub do_greet {
 	my ($self) = @_;
 	my $s = $self->{salt} = sprintf('%x.%x', int(rand(0x7fffffff)), time);

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 10/11] nntp: resolve inboxes immediately on group listings
  2022-07-23  4:41 [PATCH 00/11] IMAP, NNTP, POP3 golfing Eric Wong
                   ` (8 preceding siblings ...)
  2022-07-23  4:41 ` [PATCH 09/11] ds: share long_step between NNTP and IMAP Eric Wong
@ 2022-07-23  4:41 ` Eric Wong
  2022-07-23  4:41 ` [PATCH 11/11] imap+nntp: share COMPRESS implementation Eric Wong
  10 siblings, 0 replies; 12+ messages in thread
From: Eric Wong @ 2022-07-23  4:41 UTC (permalink / raw)
  To: meta

This prevents potential races between SIGHUP config reloads
while gigantic group listings are streaming, allowing us to
avoid many invalidation checks.

This also reduces send(2) syscalls and avoid Perl internal pad
allocations in a few places where it's not beneficial.  There
might be a slight (0.5%) speedup, but I'm not sure if that's
down to system noise, power/thermal management, or other users
on my VM.
---
 lib/PublicInbox/NNTP.pm  | 127 +++++++++++++++++++--------------------
 lib/PublicInbox/NNTPD.pm |   2 +-
 2 files changed, 62 insertions(+), 67 deletions(-)

diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 2a59cbd7..3929f817 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -108,60 +108,63 @@ sub list_overview_fmt ($) { $OVERVIEW_FMT }
 
 sub list_headers ($;$) { $LIST_HEADERS }
 
-sub list_active_i { # "LIST ACTIVE" and also just "LIST" (no args)
-	my ($self, $groupnames) = @_;
-	my @window = splice(@$groupnames, 0, 100) or return 0;
-	my $ibx;
+sub names2ibx ($;$) {
+	my ($self, $names) = @_;
 	my $groups = $self->{nntpd}->{pi_cfg}->{-by_newsgroup};
-	for my $ngname (@window) {
-		$ibx = $groups->{$ngname} and group_line($self, $ibx);
+	if ($names) { # modify arrayref in-place
+		$_ = $groups->{$_} for @$names;
+		$names; # now an arrayref of ibx
+	} else {
+		my @ret = map { $groups->{$_} } @{$self->{nntpd}->{groupnames}};
+		\@ret;
 	}
-	scalar(@$groupnames); # continue if there's more
+}
+
+sub list_active_i { # "LIST ACTIVE" and also just "LIST" (no args)
+	my ($self, $ibxs) = @_;
+	my @window = splice(@$ibxs, 0, 1000);
+	$self->msg_more(join('', map { group_line($_) } @window));
+	scalar @$ibxs; # continue if there's more
 }
 
 sub list_active ($;$) { # called by cmd_list
 	my ($self, $wildmat) = @_;
 	wildmat2re($wildmat);
-	$self->long_response(\&list_active_i, [
-		grep(/$wildmat/, @{$self->{nntpd}->{groupnames}}) ]);
+	my @names = grep(/$wildmat/, @{$self->{nntpd}->{groupnames}});
+	$self->long_response(\&list_active_i, names2ibx($self, \@names));
 }
 
 sub list_active_times_i {
-	my ($self, $groupnames) = @_;
-	my @window = splice(@$groupnames, 0, 100) or return 0;
-	my $groups = $self->{nntpd}->{pi_cfg}->{-by_newsgroup};
-	for my $ngname (@window) {
-		my $ibx = $groups->{$ngname} or next;
-		my $c = eval { $ibx->uidvalidity } // time;
-		$self->msg_more("$ngname $c <$ibx->{-primary_address}>\r\n");
-	}
-	scalar(@$groupnames); # continue if there's more
+	my ($self, $ibxs) = @_;
+	my @window = splice(@$ibxs, 0, 1000);
+	$self->msg_more(join('', map {
+		my $c = eval { $_->uidvalidity } // time;
+		"$_->{newsgroup} $c <$_->{-primary_address}>\r\n";
+	} @window));
+	scalar @$ibxs; # continue if there's more
 }
 
 sub list_active_times ($;$) { # called by cmd_list
 	my ($self, $wildmat) = @_;
 	wildmat2re($wildmat);
-	$self->long_response(\&list_active_times_i, [
-		grep(/$wildmat/, @{$self->{nntpd}->{groupnames}}) ]);
+	my @names = grep(/$wildmat/, @{$self->{nntpd}->{groupnames}});
+	$self->long_response(\&list_active_times_i, names2ibx($self, \@names));
 }
 
 sub list_newsgroups_i {
-	my ($self, $groupnames) = @_;
-	my @window = splice(@$groupnames, 0, 100) or return 0;
-	my $groups = $self->{nntpd}->{pi_cfg}->{-by_newsgroup};
-	my $ibx;
-	for my $ngname (@window) {
-		$ibx = $groups->{$ngname} and
-			$self->msg_more("$ngname ".$ibx->description."\r\n");
-	}
-	scalar(@$groupnames); # continue if there's more
+	my ($self, $ibxs) = @_;
+	my @window = splice(@$ibxs, 0, 1000);
+	$self->msg_more(join('', map {
+		"$_->{newsgroup} ".$_->description."\r\n"
+	} @window));
+	scalar @$ibxs; # continue if there's more
 }
 
 sub list_newsgroups ($;$) { # called by cmd_list
 	my ($self, $wildmat) = @_;
 	wildmat2re($wildmat);
-	$self->long_response(\&list_newsgroups_i, [
-		grep(/$wildmat/, @{$self->{nntpd}->{groupnames}}) ]);
+	my @names = grep(/$wildmat/, @{$self->{nntpd}->{groupnames}});
+	$self->long_response(\&list_newsgroups_i, names2ibx($self, \@names));
 }
 
 # LIST SUBSCRIPTIONS, DISTRIB.PATS are not supported
@@ -178,8 +181,7 @@ sub cmd_list ($;$$) {
 		$arg->($self, @args);
 	} else {
 		$self->msg_more("215 list of newsgroups follows\r\n");
-		$self->long_response(\&list_active_i, [ # copy array
-			@{$self->{nntpd}->{groupnames}} ]);
+		$self->long_response(\&list_active_i, names2ibx($self));
 	}
 }
 
@@ -242,23 +244,19 @@ sub parse_time ($$;$) {
 	}
 }
 
-sub group_line ($$) {
-	my ($self, $ibx) = @_;
+sub group_line ($) {
+	my ($ibx) = @_;
 	my ($min, $max) = $ibx->mm(1)->minmax;
-	$self->msg_more("$ibx->{newsgroup} $max $min n\r\n");
+	"$ibx->{newsgroup} $max $min n\r\n";
 }
 
 sub newgroups_i {
-	my ($self, $ts, $i, $groupnames) = @_;
-	my $end = $$i + 100;
-	my $groups = $self->{nntpd}->{pi_cfg}->{-by_newsgroup};
-	while ($$i < $end) {
-		my $ngname = $groupnames->[$$i++] // return;
-		my $ibx = $groups->{$ngname} or next; # expired on reload
-		next unless (eval { $ibx->uidvalidity } // 0) > $ts;
-		group_line($self, $ibx);
-	}
-	1;
+	my ($self, $ts, $ibxs) = @_;
+	my @window = splice(@$ibxs, 0, 1000);
+	$self->msg_more(join('', map { group_line($_) } grep {
+		 (eval { $_->uidvalidity } // 0) > $ts
+	} @window));
+	scalar @$ibxs;
 }
 
 sub cmd_newgroups ($$$;$$) {
@@ -268,8 +266,7 @@ sub cmd_newgroups ($$$;$$) {
 
 	# TODO dists
 	$self->msg_more("231 list of new newsgroups follows\r\n");
-	$self->long_response(\&newgroups_i, $ts, \(my $i = 0),
-				$self->{nntpd}->{groupnames});
+	$self->long_response(\&newgroups_i, $ts, names2ibx($self));
 }
 
 sub wildmat2re (;$) {
@@ -304,22 +301,19 @@ sub ngpat2re (;$) {
 }
 
 sub newnews_i {
-	my ($self, $names, $ts, $prev) = @_;
-	my $ngname = $names->[0];
-	if (my $ibx = $self->{nntpd}->{pi_cfg}->{-by_newsgroup}->{$ngname}) {
-		if (my $over = $ibx->over) {
-			my $msgs = $over->query_ts($ts, $$prev);
-			if (scalar @$msgs) {
-				$self->msg_more(join('', map {
-							"<$_->{mid}>\r\n";
-						} @$msgs));
-				$$prev = $msgs->[-1]->{num};
-				return 1; # continue on current group
-			}
+	my ($self, $ibxs, $ts, $prev) = @_;
+	if (my $over = $ibxs->[0]->over) {
+		my $msgs = $over->query_ts($ts, $$prev);
+		if (scalar @$msgs) {
+			$self->msg_more(join('', map {
+						"<$_->{mid}>\r\n";
+					} @$msgs));
+			$$prev = $msgs->[-1]->{num};
+			return 1; # continue on current group
 		}
 	}
-	shift @$names;
-	if (@$names) { # continue onto next newsgroup
+	shift @$ibxs;
+	if (@$ibxs) { # continue onto next newsgroup
 		$$prev = 0;
 		1;
 	} else { # all done, break out of the long_response
@@ -335,11 +329,12 @@ sub cmd_newnews ($$$$;$$) {
 	my ($keep, $skip) = split(/!/, $newsgroups, 2);
 	ngpat2re($keep);
 	ngpat2re($skip);
-	my @names = grep(!/$skip/, grep(/$keep/,
-				@{$self->{nntpd}->{groupnames}}));
-	return ".\r\n" unless scalar(@names);
+	my @names = grep(/$keep/, @{$self->{nntpd}->{groupnames}});
+	@names = grep(!/$skip/, @names);
+	return \".\r\n" unless scalar(@names);
 	my $prev = 0;
-	$self->long_response(\&newnews_i, \@names, $ts, \$prev);
+	$self->long_response(\&newnews_i, names2ibx($self, \@names),
+				$ts, \$prev);
 }
 
 sub cmd_group ($$) {
diff --git a/lib/PublicInbox/NNTPD.pm b/lib/PublicInbox/NNTPD.pm
index 0350830b..6e79f0be 100644
--- a/lib/PublicInbox/NNTPD.pm
+++ b/lib/PublicInbox/NNTPD.pm
@@ -55,7 +55,7 @@ sub refresh_groups {
 			# run in the same process someday.
 		}
 	});
-	$self->{groupnames} = [ sort(keys %$groups) ];
+	@{$self->{groupnames}} = sort(keys %$groups);
 	# this will destroy old groups that got deleted
 	$self->{pi_cfg} = $pi_cfg;
 }

^ permalink raw reply related	[flat|nested] 12+ messages in thread

* [PATCH 11/11] imap+nntp: share COMPRESS implementation
  2022-07-23  4:41 [PATCH 00/11] IMAP, NNTP, POP3 golfing Eric Wong
                   ` (9 preceding siblings ...)
  2022-07-23  4:41 ` [PATCH 10/11] nntp: resolve inboxes immediately on group listings Eric Wong
@ 2022-07-23  4:41 ` Eric Wong
  10 siblings, 0 replies; 12+ messages in thread
From: Eric Wong @ 2022-07-23  4:41 UTC (permalink / raw)
  To: meta

Their code was nearly identical to begin with, so save some
memory in -netd and disk space for all of our tarball/distro
users, at least.

And I seem to have used multiple inheritance successfully, here,
maybe...
---
 MANIFEST                                      |   3 +-
 lib/PublicInbox/DS.pm                         |   4 +-
 .../{NNTPdeflate.pm => DSdeflate.pm}          |  13 +-
 lib/PublicInbox/IMAP.pm                       |  10 +-
 lib/PublicInbox/IMAPD.pm                      |   2 +-
 lib/PublicInbox/IMAPdeflate.pm                | 126 ------------------
 lib/PublicInbox/NNTP.pm                       |  10 +-
 lib/PublicInbox/NNTPD.pm                      |   2 +-
 xt/mem-imapd-tls.t                            |   4 +-
 9 files changed, 26 insertions(+), 148 deletions(-)
 rename lib/PublicInbox/{NNTPdeflate.pm => DSdeflate.pm} (92%)
 delete mode 100644 lib/PublicInbox/IMAPdeflate.pm

diff --git a/MANIFEST b/MANIFEST
index 923f5147..10547351 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -169,6 +169,7 @@ lib/PublicInbox/ContentHash.pm
 lib/PublicInbox/DS.pm
 lib/PublicInbox/DSKQXS.pm
 lib/PublicInbox/DSPoll.pm
+lib/PublicInbox/DSdeflate.pm
 lib/PublicInbox/Daemon.pm
 lib/PublicInbox/DirIdle.pm
 lib/PublicInbox/DummyInbox.pm
@@ -206,7 +207,6 @@ lib/PublicInbox/IMAP.pm
 lib/PublicInbox/IMAPClient.pm
 lib/PublicInbox/IMAPD.pm
 lib/PublicInbox/IMAPTracker.pm
-lib/PublicInbox/IMAPdeflate.pm
 lib/PublicInbox/IMAPsearchqp.pm
 lib/PublicInbox/IPC.pm
 lib/PublicInbox/IdxStack.pm
@@ -295,7 +295,6 @@ lib/PublicInbox/Msgmap.pm
 lib/PublicInbox/MultiGit.pm
 lib/PublicInbox/NNTP.pm
 lib/PublicInbox/NNTPD.pm
-lib/PublicInbox/NNTPdeflate.pm
 lib/PublicInbox/NetNNTPSocks.pm
 lib/PublicInbox/NetReader.pm
 lib/PublicInbox/NetWriter.pm
diff --git a/lib/PublicInbox/DS.pm b/lib/PublicInbox/DS.pm
index fee31e3d..ef483aac 100644
--- a/lib/PublicInbox/DS.pm
+++ b/lib/PublicInbox/DS.pm
@@ -648,8 +648,8 @@ sub shutdn ($) {
     }
 }
 
-sub zflush {} # overridden by NNTPdeflate and IMAPdeflate
-
+sub zflush {} # overridden by DSdeflate
+sub compressed {} # overridden by DSdeflate
 sub long_response_done {} # overridden by Net::NNTP
 
 sub long_step {
diff --git a/lib/PublicInbox/NNTPdeflate.pm b/lib/PublicInbox/DSdeflate.pm
similarity index 92%
rename from lib/PublicInbox/NNTPdeflate.pm
rename to lib/PublicInbox/DSdeflate.pm
index 352d4842..b5208e43 100644
--- a/lib/PublicInbox/NNTPdeflate.pm
+++ b/lib/PublicInbox/DSdeflate.pm
@@ -2,6 +2,7 @@
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 
 # RFC 8054 NNTP COMPRESS DEFLATE implementation
+# RFC 4978 IMAP COMPRESS=DEFLATE extension
 #
 # RSS usage for 10K idle-but-did-something NNTP clients on 64-bit:
 #   TLS + DEFLATE[a] :  1.8 GB  (MemLevel=9, 1.2 GB with MemLevel=8)
@@ -14,14 +15,13 @@
 # [b] - memory-optimized implementation using a global deflate context.
 #       It's less efficient in terms of compression, but way more
 #       efficient in terms of server memory usage.
-package PublicInbox::NNTPdeflate;
+package PublicInbox::DSdeflate;
 use strict;
-use 5.010_001;
-use parent qw(PublicInbox::NNTP);
+use v5.10.1;
 use Compress::Raw::Zlib;
 
 my %IN_OPT = (
-	-Bufsize => PublicInbox::NNTP::LINE_MAX,
+	-Bufsize => 1024,
 	-WindowBits => -15, # RFC 1951
 	-AppendOutput => 1,
 );
@@ -42,21 +42,18 @@ my $zout;
 	$err == Z_OK or die "Failed to initialize zlib deflate stream: $err";
 }
 
-
 sub enable {
 	my ($class, $self) = @_;
 	my ($in, $err) = Compress::Raw::Zlib::Inflate->new(%IN_OPT);
 	if ($err != Z_OK) {
 		$self->err("Inflate->new failed: $err");
-		$self->write(\"403 Unable to activate compression\r\n");
 		return;
 	}
-	$self->write(\"206 Compression active\r\n");
 	bless $self, $class;
 	$self->{zin} = $in;
 }
 
-# overrides PublicInbox::NNTP::compressed
+# overrides PublicInbox::DS::compressed
 sub compressed { 1 }
 
 sub do_read ($$$$) {
diff --git a/lib/PublicInbox/IMAP.pm b/lib/PublicInbox/IMAP.pm
index ce0dce0f..805f1102 100644
--- a/lib/PublicInbox/IMAP.pm
+++ b/lib/PublicInbox/IMAP.pm
@@ -1212,8 +1212,6 @@ sub event_step {
 	$self->requeue unless $pending;
 }
 
-sub compressed { undef }
-
 # RFC 4978
 sub cmd_compress ($$$) {
 	my ($self, $tag, $alg) = @_;
@@ -1223,7 +1221,9 @@ sub cmd_compress ($$$) {
 	# CRIME made TLS compression obsolete
 	# return "$tag NO [COMPRESSIONACTIVE]\r\n" if $self->tls_compressed;
 
-	PublicInbox::IMAPdeflate->enable($self, $tag);
+	PublicInbox::IMAPdeflate->enable($self) or return
+				\"$tag BAD failed to activate compression\r\n";
+	PublicInbox::DS::write($self, \"$tag OK DEFLATE active\r\n");
 	$self->requeue;
 	undef
 }
@@ -1269,4 +1269,8 @@ our @ISA = qw(PublicInbox::IMAP);
 
 sub logged_in { 0 }
 
+package PublicInbox::IMAPdeflate;
+use PublicInbox::DSdeflate;
+our @ISA = qw(PublicInbox::DSdeflate PublicInbox::IMAP);
+
 1;
diff --git a/lib/PublicInbox/IMAPD.pm b/lib/PublicInbox/IMAPD.pm
index d8814324..b24097a2 100644
--- a/lib/PublicInbox/IMAPD.pm
+++ b/lib/PublicInbox/IMAPD.pm
@@ -9,7 +9,7 @@ use v5.10.1;
 use PublicInbox::Config;
 use PublicInbox::ConfigIter;
 use PublicInbox::InboxIdle;
-use PublicInbox::IMAPdeflate; # loads PublicInbox::IMAP
+use PublicInbox::IMAP;
 use PublicInbox::DummyInbox;
 my $dummy = bless { uidvalidity => 0 }, 'PublicInbox::DummyInbox';
 
diff --git a/lib/PublicInbox/IMAPdeflate.pm b/lib/PublicInbox/IMAPdeflate.pm
deleted file mode 100644
index d5929ef2..00000000
--- a/lib/PublicInbox/IMAPdeflate.pm
+++ /dev/null
@@ -1,126 +0,0 @@
-# Copyright (C) 2020-2021 all contributors <meta@public-inbox.org>
-# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
-# TODO: reduce duplication from PublicInbox::NNTPdeflate
-
-# RFC 4978
-package PublicInbox::IMAPdeflate;
-use strict;
-use warnings;
-use 5.010_001;
-use base qw(PublicInbox::IMAP);
-use Compress::Raw::Zlib;
-
-my %IN_OPT = (
-	-Bufsize => 1024,
-	-WindowBits => -15, # RFC 1951
-	-AppendOutput => 1,
-);
-
-# global deflate context and buffer
-my $zbuf = \(my $buf = '');
-my $zout;
-{
-	my $err;
-	($zout, $err) = Compress::Raw::Zlib::Deflate->new(
-		# nnrpd (INN) and Compress::Raw::Zlib favor MemLevel=9,
-		# the zlib C library and git use MemLevel=8 as the default
-		# -MemLevel => 9,
-		-Bufsize => 65536, # same as nnrpd
-		-WindowBits => -15, # RFC 1951
-		-AppendOutput => 1,
-	);
-	$err == Z_OK or die "Failed to initialize zlib deflate stream: $err";
-}
-
-sub enable {
-	my ($class, $self, $tag) = @_;
-	my ($in, $err) = Compress::Raw::Zlib::Inflate->new(%IN_OPT);
-	if ($err != Z_OK) {
-		$self->err("Inflate->new failed: $err");
-		$self->write(\"$tag BAD failed to activate compression\r\n");
-		return;
-	}
-	$self->write(\"$tag OK DEFLATE active\r\n");
-	bless $self, $class;
-	$self->{zin} = $in;
-}
-
-# overrides PublicInbox::NNTP::compressed
-sub compressed { 1 }
-
-sub do_read ($$$$) {
-	my ($self, $rbuf, $len, $off) = @_;
-
-	my $zin = $self->{zin} or return; # closed
-	my $doff;
-	my $dbuf = delete($self->{dbuf}) // '';
-	$doff = length($dbuf);
-	my $r = PublicInbox::DS::do_read($self, \$dbuf, $len, $doff) or return;
-
-	# Workaround inflate bug appending to OOK scalars:
-	# <https://rt.cpan.org/Ticket/Display.html?id=132734>
-	# We only have $off if the client is pipelining, and pipelining
-	# is where our substr() OOK optimization in event_step makes sense.
-	if ($off) {
-		my $copy = $$rbuf;
-		undef $$rbuf;
-		$$rbuf = $copy;
-	}
-
-	# assert(length($$rbuf) == $off) as far as NNTP.pm is concerned
-	# -ConsumeInput is true, so $dbuf is automatically emptied
-	my $err = $zin->inflate($dbuf, $rbuf);
-	if ($err == Z_OK) {
-		$self->{dbuf} = $dbuf if $dbuf ne '';
-		$r = length($$rbuf) and return $r;
-		# nothing ready, yet, get more, later
-		$self->requeue;
-	} else {
-		delete $self->{zin};
-		$self->close;
-	}
-	0;
-}
-
-# override PublicInbox::DS::msg_more
-sub msg_more ($$) {
-	my $self = $_[0];
-
-	# $_[1] may be a reference or not for ->deflate
-	my $err = $zout->deflate($_[1], $zbuf);
-	$err == Z_OK or die "->deflate failed $err";
-	1;
-}
-
-sub zflush ($) {
-	my ($self) = @_;
-
-	my $deflated = $zbuf;
-	$zbuf = \(my $next = '');
-
-	my $err = $zout->flush($deflated, Z_FULL_FLUSH);
-	$err == Z_OK or die "->flush failed $err";
-
-	# We can still let the lower socket layer do buffering:
-	PublicInbox::DS::msg_more($self, $$deflated);
-}
-
-# compatible with PublicInbox::DS::write, so $_[1] may be a reference or not
-sub write ($$) {
-	my $self = $_[0];
-	return PublicInbox::DS::write($self, $_[1]) if ref($_[1]) eq 'CODE';
-
-	my $deflated = $zbuf;
-	$zbuf = \(my $next = '');
-
-	# $_[1] may be a reference or not for ->deflate
-	my $err = $zout->deflate($_[1], $deflated);
-	$err == Z_OK or die "->deflate failed $err";
-	$err = $zout->flush($deflated, Z_FULL_FLUSH);
-	$err == Z_OK or die "->flush failed $err";
-
-	# We can still let the socket layer do buffering:
-	PublicInbox::DS::write($self, $deflated);
-}
-
-1;
diff --git a/lib/PublicInbox/NNTP.pm b/lib/PublicInbox/NNTP.pm
index 3929f817..8ad7adc1 100644
--- a/lib/PublicInbox/NNTP.pm
+++ b/lib/PublicInbox/NNTP.pm
@@ -883,8 +883,6 @@ sub cmd_xover ($;$) {
 	$self->long_response(\&xover_i, @$r);
 }
 
-sub compressed { undef }
-
 sub cmd_starttls ($) {
 	my ($self) = @_;
 	my $sock = $self->{sock} or return;
@@ -903,7 +901,9 @@ sub cmd_compress ($$) {
 	my ($self, $alg) = @_;
 	return "503 Only DEFLATE is supported\r\n" if uc($alg) ne 'DEFLATE';
 	return r502 if $self->compressed;
-	PublicInbox::NNTPdeflate->enable($self);
+	PublicInbox::NNTPdeflate->enable($self) or return
+				\"403 Unable to activate compression\r\n";
+	PublicInbox::DS::write($self, \"206 Compression active\r\n");
 	$self->requeue;
 	undef
 }
@@ -985,4 +985,8 @@ sub busy { # for graceful shutdown in PublicInbox::Daemon:
 	defined($self->{rbuf}) || defined($self->{wbuf})
 }
 
+package PublicInbox::NNTPdeflate;
+use PublicInbox::DSdeflate;
+our @ISA = qw(PublicInbox::DSdeflate PublicInbox::NNTP);
+
 1;
diff --git a/lib/PublicInbox/NNTPD.pm b/lib/PublicInbox/NNTPD.pm
index 6e79f0be..f31d4381 100644
--- a/lib/PublicInbox/NNTPD.pm
+++ b/lib/PublicInbox/NNTPD.pm
@@ -9,7 +9,7 @@ use v5.10.1;
 use Sys::Hostname;
 use PublicInbox::Config;
 use PublicInbox::InboxIdle;
-use PublicInbox::NNTPdeflate; # loads PublicInbox::NNTP
+use PublicInbox::NNTP;
 
 sub new {
 	my ($class) = @_;
diff --git a/xt/mem-imapd-tls.t b/xt/mem-imapd-tls.t
index 8992a6fc..d728ce32 100644
--- a/xt/mem-imapd-tls.t
+++ b/xt/mem-imapd-tls.t
@@ -1,5 +1,5 @@
 #!perl -w
-# Copyright (C) 2020-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 # Idle client memory usage test, particularly after EXAMINE when
 # Message Sequence Numbers are loaded
@@ -221,7 +221,7 @@ package IMAPCdeflate;
 use strict;
 our @ISA;
 use Compress::Raw::Zlib;
-use PublicInbox::IMAPdeflate;
+use PublicInbox::IMAP;
 my %ZIN_OPT;
 BEGIN {
 	@ISA = qw(IMAPC);

^ permalink raw reply related	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2022-07-23  4:41 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-07-23  4:41 [PATCH 00/11] IMAP, NNTP, POP3 golfing Eric Wong
2022-07-23  4:41 ` [PATCH 01/11] nntp: pass regexp to split() callers Eric Wong
2022-07-23  4:41 ` [PATCH 02/11] nntp: start adding CRLF to responses natively Eric Wong
2022-07-23  4:41 ` [PATCH 03/11] nntp: remove more() wrapper Eric Wong
2022-07-23  4:41 ` [PATCH 04/11] ds: support greeting protocols Eric Wong
2022-07-23  4:41 ` [PATCH 05/11] ds: move no-op ->zflush to common base class Eric Wong
2022-07-23  4:41 ` [PATCH 06/11] ds: move requeue_once Eric Wong
2022-07-23  4:41 ` [PATCH 07/11] nntp: listgroup_range_i: remove useless `map' op Eric Wong
2022-07-23  4:41 ` [PATCH 08/11] nntp: inline CRLF in all response lines Eric Wong
2022-07-23  4:41 ` [PATCH 09/11] ds: share long_step between NNTP and IMAP Eric Wong
2022-07-23  4:41 ` [PATCH 10/11] nntp: resolve inboxes immediately on group listings Eric Wong
2022-07-23  4:41 ` [PATCH 11/11] imap+nntp: share COMPRESS implementation Eric Wong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).