[PATCH] pop3d: support fcntl locks on OpenBSD i386

[PATCH] pop3d: support fcntl locks on OpenBSD i386

[Eric Wong]

The packaged Perl on OpenBSD i386 supports 64-bit file offsets
but not 64-bit integer support for 'q' and 'Q' with `pack'.
Since servers aren't likely to require lock files larger than
2 GB (we'd need an inbox with >2 billion messages), we can
workaround the Perl build limitation with explicit padding.

AFAIK File::FcntlLock isn't packaged for OpenBSD, but I can
test i386 OpenBSD on an extremely slow VM.

Big endian support can be done, too, but I have no idea if
there's 32-bit BE users around nowadays...
---
 MANIFEST                 |  1 +
 lib/PublicInbox/POP3D.pm | 28 +++++++++++++++++++++-------
 t/pop3d_lock.t           | 16 ++++++++++++++++
 3 files changed, 38 insertions(+), 7 deletions(-)
 create mode 100644 t/pop3d_lock.t

diff --git a/MANIFEST b/MANIFEST
index 2223cfb4..4c974338 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -575,6 +575,7 @@ t/plack-qp.eml
 t/plack.t
 t/pop3d-limit.t
 t/pop3d.t
+t/pop3d_lock.t
 t/precheck.t
 t/psgi_attach.eml
 t/psgi_attach.t
diff --git a/lib/PublicInbox/POP3D.pm b/lib/PublicInbox/POP3D.pm
index 38e982ee..9c868cfb 100644
--- a/lib/PublicInbox/POP3D.pm
+++ b/lib/PublicInbox/POP3D.pm
@@ -18,18 +18,32 @@ my ($FLOCK_TMPL, @FLOCK_ORDER);
 if ($^O =~ /\A(?:linux|dragonfly)\z/ || $^O =~ /bsd/) {
 	require Config;
 	my $off_t;
+	my @LE_pad = ('', '');
 	my $sz = $Config::Config{lseeksize};
-
-	if ($sz == 8 && eval('length(pack("q", 1)) == 8')) { $off_t = 'q' }
-	elsif ($sz == 4) { $off_t = 'l' }
-	else { warn "sizeof(off_t)=$sz requires File::FcntlLock\n" }
-
+	if ($sz == 8) {
+		if (eval('length(pack("q", 1)) == 8')) {
+			$off_t = 'q';
+		} elsif ($Config::Config{byteorder} == 1234) { # OpenBSD i386
+			$off_t = 'l';
+			@LE_pad = ('@8', '@16');
+		} else { # I have no 32-bit BE machine to test on...
+			warn <<EOM;
+Perl built with 64-bit file support but not 64-bit int (pack("q") support)
+byteorder=$Config::Config{byteorder}
+EOM
+		}
+	} elsif ($sz == 4) {
+		$off_t = 'l';
+	} else {
+		warn "sizeof(off_t)=$sz requires File::FcntlLock\n"
+	}
 	if (defined($off_t)) {
 		if ($^O eq 'linux') {
-			$FLOCK_TMPL = "ss\@8$off_t$off_t\@32";
+			$FLOCK_TMPL = 'ss@8'.$off_t.$LE_pad[0].$off_t.'@32';
 			@FLOCK_ORDER = qw(l_type l_whence l_start l_len);
 		} else { # *bsd including dragonfly
-			$FLOCK_TMPL = "${off_t}${off_t}lss\@256";
+			$FLOCK_TMPL = $off_t.$LE_pad[0].$off_t.$LE_pad[1].
+					'lss@256';
 			@FLOCK_ORDER = qw(l_start l_len l_pid l_type l_whence);
 		}
 	}
diff --git a/t/pop3d_lock.t b/t/pop3d_lock.t
new file mode 100644
index 00000000..fb305f96
--- /dev/null
+++ b/t/pop3d_lock.t
@@ -0,0 +1,16 @@
+# Copyright (C) all contributors <meta@public-inbox.org>
+# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
+use v5.12;
+use PublicInbox::TestCommon;
+require_mods(qw(DBD::SQLite Net::POP3 :fcntl_lock));
+use autodie;
+my $tmpdir = tmpdir;
+require_ok 'PublicInbox::POP3D';
+my $pop3d = bless {}, 'PublicInbox::POP3D';
+open $pop3d->{txn_fh}, '+>>', "$tmpdir/txn.lock";
+use Fcntl qw(F_SETLK F_UNLCK F_WRLCK);
+
+ok $pop3d->_setlk(l_type => F_WRLCK, l_start => 9, l_len => 1),
+	'locked file (check with ktrace/strace)';
+
+done_testing;

Re: [PATCH] pop3d: support fcntl locks on OpenBSD i386

[Štěpán Němec]

On Thu,  1 Feb 2024 00:12:19 +0000
Eric Wong wrote:

> AFAIK File::FcntlLock isn't packaged for OpenBSD,

https://openports.pl/path/devel/p5-File-FcntlLock
https://ftp.openbsd.org/pub/OpenBSD/snapshots/packages/amd64/p5-File-FcntlLock-0.22.tgz

(apparently only added a few months ago, so it's not in any
release yet; should be available starting with 7.5 (due in
May or thereabouts))

-- 
Štěpán

[PATCH v2] pop3d: support fcntl locks on OpenBSD i386

[Eric Wong]

Štěpán Němec <stepnem@smrk.net> wrote:
> Eric Wong wrote:
> 
> > AFAIK File::FcntlLock isn't packaged for OpenBSD,
> 
> https://openports.pl/path/devel/p5-File-FcntlLock
> https://ftp.openbsd.org/pub/OpenBSD/snapshots/packages/amd64/p5-File-FcntlLock-0.22.tgz
> 
> (apparently only added a few months ago, so it's not in any
> release yet; should be available starting with 7.5 (due in
> May or thereabouts))

Thanks for the info.  Updated commit message and fixed
parentheses in warning message below.

Sidenote: 64-bit file offsets in an interpreter without 64-bit
integer support is a strange combination.  I don't understand
why it was chosen by the OpenBSD folks nor why Perl allows it...
Can anybody enlighten me?

-------8<------
Subject: [PATCH v2] pop3d: support fcntl locks on OpenBSD i386

The packaged Perl on OpenBSD i386 supports 64-bit file offsets
but not 64-bit integer support for 'q' and 'Q' with `pack'.
Since servers aren't likely to require lock files larger than
2 GB (we'd need an inbox with >2 billion messages), we can
workaround the Perl build limitation with explicit padding.

File::FcntlLock isn't packaged for OpenBSD <= 7.4 (but should be
in future releases), but I can test i386 OpenBSD on an extremely
slow VM.

Big endian support can be done, too, but I have no idea if
there's 32-bit BE users around nowadays...
---
Range-diff against v1:
1:  0dce97c6 ! 1:  75dab25f pop3d: support fcntl locks on OpenBSD i386
    @@ Commit message
         2 GB (we'd need an inbox with >2 billion messages), we can
         workaround the Perl build limitation with explicit padding.
     
    -    AFAIK File::FcntlLock isn't packaged for OpenBSD, but I can
    -    test i386 OpenBSD on an extremely slow VM.
    +    File::FcntlLock isn't packaged for OpenBSD <= 7.4 (but should be
    +    in future releases), but I can test i386 OpenBSD on an extremely
    +    slow VM.
     
         Big endian support can be done, too, but I have no idea if
         there's 32-bit BE users around nowadays...
    @@ lib/PublicInbox/POP3D.pm: my ($FLOCK_TMPL, @FLOCK_ORDER);
     +			@LE_pad = ('@8', '@16');
     +		} else { # I have no 32-bit BE machine to test on...
     +			warn <<EOM;
    -+Perl built with 64-bit file support but not 64-bit int (pack("q") support)
    ++Perl built with 64-bit file support but not 64-bit int (pack("q")) support.
     +byteorder=$Config::Config{byteorder}
     +EOM
     +		}

 MANIFEST                 |  1 +
 lib/PublicInbox/POP3D.pm | 28 +++++++++++++++++++++-------
 t/pop3d_lock.t           | 16 ++++++++++++++++
 3 files changed, 38 insertions(+), 7 deletions(-)
 create mode 100644 t/pop3d_lock.t

diff --git a/MANIFEST b/MANIFEST
index 2223cfb4..4c974338 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -575,6 +575,7 @@ t/plack-qp.eml
 t/plack.t
 t/pop3d-limit.t
 t/pop3d.t
+t/pop3d_lock.t
 t/precheck.t
 t/psgi_attach.eml
 t/psgi_attach.t
diff --git a/lib/PublicInbox/POP3D.pm b/lib/PublicInbox/POP3D.pm
index 38e982ee..bd440434 100644
--- a/lib/PublicInbox/POP3D.pm
+++ b/lib/PublicInbox/POP3D.pm
@@ -18,18 +18,32 @@ my ($FLOCK_TMPL, @FLOCK_ORDER);
 if ($^O =~ /\A(?:linux|dragonfly)\z/ || $^O =~ /bsd/) {
 	require Config;
 	my $off_t;
+	my @LE_pad = ('', '');
 	my $sz = $Config::Config{lseeksize};
-
-	if ($sz == 8 && eval('length(pack("q", 1)) == 8')) { $off_t = 'q' }
-	elsif ($sz == 4) { $off_t = 'l' }
-	else { warn "sizeof(off_t)=$sz requires File::FcntlLock\n" }
-
+	if ($sz == 8) {
+		if (eval('length(pack("q", 1)) == 8')) {
+			$off_t = 'q';
+		} elsif ($Config::Config{byteorder} == 1234) { # OpenBSD i386
+			$off_t = 'l';
+			@LE_pad = ('@8', '@16');
+		} else { # I have no 32-bit BE machine to test on...
+			warn <<EOM;
+Perl built with 64-bit file support but not 64-bit int (pack("q")) support.
+byteorder=$Config::Config{byteorder}
+EOM
+		}
+	} elsif ($sz == 4) {
+		$off_t = 'l';
+	} else {
+		warn "sizeof(off_t)=$sz requires File::FcntlLock\n"
+	}
 	if (defined($off_t)) {
 		if ($^O eq 'linux') {
-			$FLOCK_TMPL = "ss\@8$off_t$off_t\@32";
+			$FLOCK_TMPL = 'ss@8'.$off_t.$LE_pad[0].$off_t.'@32';
 			@FLOCK_ORDER = qw(l_type l_whence l_start l_len);
 		} else { # *bsd including dragonfly
-			$FLOCK_TMPL = "${off_t}${off_t}lss\@256";
+			$FLOCK_TMPL = $off_t.$LE_pad[0].$off_t.$LE_pad[1].
+					'lss@256';
 			@FLOCK_ORDER = qw(l_start l_len l_pid l_type l_whence);
 		}
 	}
diff --git a/t/pop3d_lock.t b/t/pop3d_lock.t
new file mode 100644
index 00000000..fb305f96
--- /dev/null
+++ b/t/pop3d_lock.t
@@ -0,0 +1,16 @@
+# Copyright (C) all contributors <meta@public-inbox.org>
+# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
+use v5.12;
+use PublicInbox::TestCommon;
+require_mods(qw(DBD::SQLite Net::POP3 :fcntl_lock));
+use autodie;
+my $tmpdir = tmpdir;
+require_ok 'PublicInbox::POP3D';
+my $pop3d = bless {}, 'PublicInbox::POP3D';
+open $pop3d->{txn_fh}, '+>>', "$tmpdir/txn.lock";
+use Fcntl qw(F_SETLK F_UNLCK F_WRLCK);
+
+ok $pop3d->_setlk(l_type => F_WRLCK, l_start => 9, l_len => 1),
+	'locked file (check with ktrace/strace)';
+
+done_testing;