unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 15/36] ipc: support Sereal
Date: Thu, 31 Dec 2020 13:51:33 +0000	[thread overview]
Message-ID: <20201231135154.6070-16-e@80x24.org> (raw)
In-Reply-To: <20201231135154.6070-1-e@80x24.org>

Some testing will be needed to see if it's worth the code
and maintenance overhead, but it seems easy-enough to get
working.
---
 lib/PublicInbox/IPC.pm | 29 ++++++++++++++++++++++++-----
 t/ipc.t                |  2 +-
 2 files changed, 25 insertions(+), 6 deletions(-)

diff --git a/lib/PublicInbox/IPC.pm b/lib/PublicInbox/IPC.pm
index c04140ae..0baa218c 100644
--- a/lib/PublicInbox/IPC.pm
+++ b/lib/PublicInbox/IPC.pm
@@ -9,6 +9,29 @@ use v5.10.1;
 use Socket qw(AF_UNIX SOCK_STREAM);
 use Carp qw(confess croak);
 use PublicInbox::Sigfd;
+my ($enc, $dec);
+# ->imports at BEGIN turns serial_*_with_object into custom ops on 5.14+
+# and eliminate method call overhead
+BEGIN {
+	eval {
+		require Sereal::Encoder;
+		require Sereal::Decoder;
+		Sereal::Encoder->import('sereal_encode_with_object');
+		Sereal::Decoder->import('sereal_decode_with_object');
+		($enc, $dec) = (Sereal::Encoder->new, Sereal::Decoder->new);
+	};
+};
+
+if ($enc && $dec) { # should be custom ops
+	*freeze = sub ($) { sereal_encode_with_object $enc, $_[0] };
+	*thaw = sub ($) { sereal_decode_with_object $dec, $_[0], my $ret };
+} else {
+	eval { # some distros have Storable as a separate package from Perl
+		require Storable;
+		Storable->import(qw(freeze thaw));
+		$enc = 1;
+	} // warn("Storable (part of Perl) missing: $@\n");
+}
 
 sub _get_rec ($) {
 	my ($sock) = @_;
@@ -52,11 +75,7 @@ sub ipc_worker_loop ($$) {
 
 sub ipc_worker_spawn ($$$) {
 	my ($self, $ident, $oldset) = @_;
-	eval { require Storable; Storable->import(qw(freeze thaw)); };
-	if ($@) {
-		state $w //= warn "Storable (part of Perl) missing: $@\n";
-		return;
-	}
+	return unless $enc;
 	my $pid = $self->{-ipc_worker_pid};
 	confess "BUG: already spawned PID:$pid" if $pid;
 	confess "BUG: already have worker socket" if $self->{-ipc_sock};
diff --git a/t/ipc.t b/t/ipc.t
index f9c4024b..f3715e2c 100644
--- a/t/ipc.t
+++ b/t/ipc.t
@@ -49,7 +49,7 @@ my $test = sub {
 $test->('local');
 
 SKIP: {
-	require_mods(qw(Storable), 16);
+	require_mods(qw(Storable||Sereal), 16);
 	my $pid = $ipc->ipc_worker_spawn('test worker');
 	ok($pid > 0 && kill(0, $pid), 'worker spawned and running');
 	defined($pid) or BAIL_OUT 'no spawn, no test';

  parent reply	other threads:[~2020-12-31 13:51 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-12-31 13:51 [PATCH 00/36] another round of lei stuff Eric Wong
2020-12-31 13:51 ` [PATCH 01/36] import: respect init.defaultBranch Eric Wong
2020-12-31 13:51 ` [PATCH 02/36] lei_store: use per-machine refname as git HEAD Eric Wong
2020-12-31 13:51 ` [PATCH 03/36] revert "lei_store: use per-machine refname as git HEAD" Eric Wong
2020-12-31 13:51 ` [PATCH 04/36] lei_to_mail: initial implementation for writing mbox formats Eric Wong
2020-12-31 13:51 ` [PATCH 05/36] sharedkv: fork()-friendly key-value store Eric Wong
2020-12-31 13:51 ` [PATCH 06/36] sharedkv: split out index_values Eric Wong
2020-12-31 13:51 ` [PATCH 07/36] lei_to_mail: start atomic and compressed mbox writing Eric Wong
2020-12-31 13:51 ` [PATCH 08/36] mboxreader: new class for reading various mbox formats Eric Wong
2020-12-31 13:51 ` [PATCH 09/36] lei_to_mail: start --augment, dedupe, bz2 and xz Eric Wong
2020-12-31 13:51 ` [PATCH 10/36] lei: implement various deduplication strategies Eric Wong
2020-12-31 13:51 ` [PATCH 11/36] lei_to_mail: lazy-require LeiDedupe Eric Wong
2020-12-31 13:51 ` [PATCH 12/36] lei_to_mail: support for non-seekable outputs Eric Wong
2020-12-31 13:51 ` [PATCH 13/36] lei_to_mail: support Maildir, fix+test --augment Eric Wong
2020-12-31 13:51 ` [PATCH 14/36] ipc: generic IPC dispatch based on Storable Eric Wong
2020-12-31 13:51 ` Eric Wong [this message]
2020-12-31 13:51 ` [PATCH 16/36] lei_store: add ->set_eml, ->add_eml can return smsg Eric Wong
2020-12-31 13:51 ` [PATCH 17/36] lei: rename "extinbox" => "external" Eric Wong
2020-12-31 13:51 ` [PATCH 18/36] mid: use defined-or with `push' for uniqueness check Eric Wong
2020-12-31 13:51 ` [PATCH 19/36] mid: hoist out mids_in sub Eric Wong
2020-12-31 13:51 ` [PATCH 20/36] lei_store: handle messages without Message-ID at all Eric Wong
2020-12-31 13:51 ` [PATCH 21/36] ipc: use shutdown(2), base atfork* callback Eric Wong
2020-12-31 13:51 ` [PATCH 22/36] lei_to_mail: unlink mboxes if not augmenting Eric Wong
2020-12-31 13:51 ` [PATCH 23/36] lei: add --mfolder as an --output alias Eric Wong
2020-12-31 13:51 ` [PATCH 24/36] spawn: move run_die here from PublicInbox::Import Eric Wong
2020-12-31 13:51 ` [PATCH 25/36] init: remove embedded UnlinkMe package Eric Wong
2020-12-31 13:51 ` [PATCH 26/36] t/run: avoid uninitialized var on incomplete test Eric Wong
2020-12-31 13:51 ` [PATCH 27/36] gcf2client: reap process on DESTROY Eric Wong
2020-12-31 13:51 ` [PATCH 28/36] lei_to_mail: open FIFOs O_WRONLY so we block Eric Wong
2020-12-31 13:51 ` [PATCH 29/36] searchidxshard: call DS->Reset at worker start Eric Wong
2020-12-31 13:51 ` [PATCH 30/36] t/ipc.t: test for references via `die' Eric Wong
2020-12-31 13:51 ` [PATCH 31/36] use PublicInbox::DS for dwaitpid Eric Wong
2020-12-31 13:51 ` [PATCH 32/36] syscall: SFD_NONBLOCK can be a constant, again Eric Wong
2020-12-31 13:51 ` [PATCH 33/36] lei: avoid Spawn package when starting daemon Eric Wong
2020-12-31 13:51 ` [PATCH 34/36] avoid calling waitpid from children in DESTROY Eric Wong
2020-12-31 13:51 ` [PATCH 35/36] ds: clobber $in_loop first at reset Eric Wong
2020-12-31 13:51 ` [PATCH 36/36] on_destroy: support PID owner guard Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201231135154.6070-16-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).