unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
* [PATCH 0/8] NetBSD-related updates
@ 2023-09-09 12:01 Eric Wong
  2023-09-09 12:01 ` [PATCH 1/8] Makefile.PL: check `getconf NPROCESSORS_ONLN', too Eric Wong
                   ` (7 more replies)
  0 siblings, 8 replies; 11+ messages in thread
From: Eric Wong @ 2023-09-09 12:01 UTC (permalink / raw)
  To: meta

It actually packages `p5-highlight' like Debian!

Eric Wong (8):
  Makefile.PL: check `getconf NPROCESSORS_ONLN', too
  ipc: define _SC_NPROCESSORS_ONLN for NetBSD
  ci/run.sh: parameterize BUILD_JOBS TEST_JOBS and TEST_TARGET
  ci/profiles: rewrite in Perl
  update CI helper scripts for NetBSD and `pkgin'
  xap_helper: note __cleanup__ works with C++ exceptions
  xap_helper: use _OPENBSD_SOURCE on NetBSD for reallocarray
  xap_helper: clamp workers to USHRT_MAX

 Makefile.PL                  |  3 +-
 ci/deps.perl                 | 24 +++++++--
 ci/profiles.perl             | 93 ++++++++++++++++++++++++++++++++++
 ci/profiles.sh               | 98 ------------------------------------
 ci/run.sh                    | 14 +++---
 lib/PublicInbox/IPC.pm       | 16 ++++--
 lib/PublicInbox/xap_helper.h | 15 +++++-
 7 files changed, 148 insertions(+), 115 deletions(-)
 create mode 100755 ci/profiles.perl
 delete mode 100755 ci/profiles.sh

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH 1/8] Makefile.PL: check `getconf NPROCESSORS_ONLN', too
  2023-09-09 12:01 [PATCH 0/8] NetBSD-related updates Eric Wong
@ 2023-09-09 12:01 ` Eric Wong
  2023-09-09 12:01 ` [PATCH 2/8] ipc: define _SC_NPROCESSORS_ONLN for NetBSD Eric Wong
                   ` (6 subsequent siblings)
  7 siblings, 0 replies; 11+ messages in thread
From: Eric Wong @ 2023-09-09 12:01 UTC (permalink / raw)
  To: meta

NetBSD and OpenBSD getconf(1) don't accept a leading underscore,
while glibc getconf(1) only accepts the leading underscore
(`_NPROCESSORS_ONLN').  FreeBSD getconf(1) accepts both variants.
---
 Makefile.PL | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Makefile.PL b/Makefile.PL
index 5a5628ba..d0652410 100644
--- a/Makefile.PL
+++ b/Makefile.PL
@@ -196,7 +196,8 @@ WriteMakefile(
 );
 
 sub MY::postamble {
-	my $N = (`{ getconf _NPROCESSORS_ONLN || nproc; } 2>/dev/null` || 1);
+	my $N = (`{ getconf _NPROCESSORS_ONLN || getconf NPROCESSORS_ONLN ||
+		gnproc || nproc; } 2>/dev/null` || 1);
 	$N += 1; # account for sleeps in some tests (and makes an IV)
 	<<EOF;
 PROVE = prove

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 2/8] ipc: define _SC_NPROCESSORS_ONLN for NetBSD
  2023-09-09 12:01 [PATCH 0/8] NetBSD-related updates Eric Wong
  2023-09-09 12:01 ` [PATCH 1/8] Makefile.PL: check `getconf NPROCESSORS_ONLN', too Eric Wong
@ 2023-09-09 12:01 ` Eric Wong
  2023-09-09 12:01 ` [PATCH 3/8] ci/run.sh: parameterize BUILD_JOBS TEST_JOBS and TEST_TARGET Eric Wong
                   ` (5 subsequent siblings)
  7 siblings, 0 replies; 11+ messages in thread
From: Eric Wong @ 2023-09-09 12:01 UTC (permalink / raw)
  To: meta

We'll reorganize this into a hash table for ease-of-reading.
---
 lib/PublicInbox/IPC.pm | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/lib/PublicInbox/IPC.pm b/lib/PublicInbox/IPC.pm
index 528b9133..39021f42 100644
--- a/lib/PublicInbox/IPC.pm
+++ b/lib/PublicInbox/IPC.pm
@@ -451,12 +451,18 @@ sub DESTROY {
 	ipc_worker_stop($self);
 }
 
+# _SC_NPROCESSORS_ONLN = 84 on both Linux glibc and musl,
+# emitted using: $^X devel/sysdefs-list
+my %NPROCESSORS_ONLN = (
+	linux => 84,
+	freebsd => 58,
+	openbsd => 503,
+	netbsd => 1002
+);
+
 sub detect_nproc () {
-	# _SC_NPROCESSORS_ONLN = 84 on both Linux glibc and musl
-	return POSIX::sysconf(84) if $^O eq 'linux';
-	return POSIX::sysconf(58) if $^O eq 'freebsd';
-	return POSIX::sysconf(503) if $^O eq 'openbsd';
-	# TODO: more OSes
+	my $n = $NPROCESSORS_ONLN{$^O};
+	return POSIX::sysconf($n) if defined $n;
 
 	# getconf(1) is POSIX, but *NPROCESSORS* vars are not
 	for (qw(_NPROCESSORS_ONLN NPROCESSORS_ONLN)) {

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 3/8] ci/run.sh: parameterize BUILD_JOBS TEST_JOBS and TEST_TARGET
  2023-09-09 12:01 [PATCH 0/8] NetBSD-related updates Eric Wong
  2023-09-09 12:01 ` [PATCH 1/8] Makefile.PL: check `getconf NPROCESSORS_ONLN', too Eric Wong
  2023-09-09 12:01 ` [PATCH 2/8] ipc: define _SC_NPROCESSORS_ONLN for NetBSD Eric Wong
@ 2023-09-09 12:01 ` Eric Wong
  2023-09-09 12:01 ` [PATCH 4/8] ci/profiles: rewrite in Perl Eric Wong
                   ` (4 subsequent siblings)
  7 siblings, 0 replies; 11+ messages in thread
From: Eric Wong @ 2023-09-09 12:01 UTC (permalink / raw)
  To: meta

Parallezing BUILD_JOBS is usually harmless, but TEST_JOBS can
be problematic when tracking down problems on new platforms.
TEST_TARGET can be `check' or `check-run' for performance.
---
 ci/run.sh | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/ci/run.sh b/ci/run.sh
index 9613943b..1faf92c2 100755
--- a/ci/run.sh
+++ b/ci/run.sh
@@ -1,5 +1,5 @@
 #!/bin/sh
-# Copyright (C) 2019-2021 all contributors <meta@public-inbox.org>
+# Copyright (C) all contributors <meta@public-inbox.org>
 # License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
 set -e
 SUDO=${SUDO-'sudo'} PERL=${PERL-'perl'} MAKE=${MAKE-'make'}
@@ -8,14 +8,16 @@ DO=${DO-''}
 set -x
 if test -f Makefile
 then
-	$DO $MAKE clean
+	$DO $MAKE clean >/dev/null
 fi
+NPROC=${NPROC-$({ getconf _NPROCESSORS_ONLN || getconf NPROCESSORS_ONLN ||
+	gnproc || nproc || echo 2; } 2>/dev/null)}
 
 ./ci/profiles.sh | while read args
 do
 	$DO $SUDO $PERL -w ci/deps.perl $args
 	$DO $PERL Makefile.PL
-	$DO $MAKE
-	$DO $MAKE check
-	$DO $MAKE clean
+	$DO $MAKE -j${BUILD_JOBS-$NPROC}
+	$DO $MAKE -j${TEST_JOBS-1} ${TEST_TARGET-test}
+	$DO $MAKE clean >/dev/null
 done

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 4/8] ci/profiles: rewrite in Perl
  2023-09-09 12:01 [PATCH 0/8] NetBSD-related updates Eric Wong
                   ` (2 preceding siblings ...)
  2023-09-09 12:01 ` [PATCH 3/8] ci/run.sh: parameterize BUILD_JOBS TEST_JOBS and TEST_TARGET Eric Wong
@ 2023-09-09 12:01 ` Eric Wong
  2023-09-09 21:14   ` [SQUASH] manifest update Eric Wong
  2023-09-09 12:01 ` [PATCH 5/8] update CI helper scripts for NetBSD and `pkgin' Eric Wong
                   ` (3 subsequent siblings)
  7 siblings, 1 reply; 11+ messages in thread
From: Eric Wong @ 2023-09-09 12:01 UTC (permalink / raw)
  To: meta

Reading os-release(5) is a bit more painful, now; and still
requires using the shell.  However, sharing code between *BSDs
and being able to use v-strings for version comparisons is much
easier.

Test profiles for *BSDs are also trimmed down and more focused
on portability stuff.
---
 ci/profiles.perl | 97 +++++++++++++++++++++++++++++++++++++++++++++++
 ci/profiles.sh   | 98 ------------------------------------------------
 ci/run.sh        |  2 +-
 3 files changed, 98 insertions(+), 99 deletions(-)
 create mode 100755 ci/profiles.perl
 delete mode 100755 ci/profiles.sh

diff --git a/ci/profiles.perl b/ci/profiles.perl
new file mode 100755
index 00000000..3d67143a
--- /dev/null
+++ b/ci/profiles.perl
@@ -0,0 +1,97 @@
+# Copyright (C) all contributors <meta@public-inbox.org>
+# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
+# Prints OS-specific package profiles to stdout (one per line) to use
+# as command-line args for ci/deps.perl.  Called automatically by ci/run.sh
+eval 'exec perl -wS $0 ${1+"$@"}' # no shebang
+if 0; # running under some shell
+use v5.12;
+our ($ID, $PRETTY_NAME, $VERSION_ID); # same vars as os-release(5)
+my ($release, $version); # from uname
+if ($^O eq 'linux') { # try using os-release(5)
+	for my $f (qw(/etc/os-release /usr/lib/os-release)) {
+		next unless -f $f;
+		my @echo = map {
+			qq{echo "\$"$_" = qq[\$$_];"; }
+		} qw(ID PRETTY_NAME VERSION_ID);
+		# rely on sh(1) to handle interpolation and such:
+		my $vars = `sh -c '. $f; @echo'`;
+		die "sh \$?=$?" if $?;
+		eval $vars;
+		die $@ if $@;
+		$VERSION_ID //= '';
+		$ID //= '';
+		if ($ID eq 'debian') {
+			if ($PRETTY_NAME =~ m!/sid\z!) {
+				$VERSION_ID = 'sid';
+			} else {
+				open my $fh, '<', $f or die "open($f): $!";
+				my $msg = do { local $/; <$fh> };
+				die <<EOM;
+ID=$ID, but no VERSION_ID
+==> $f <==
+$msg
+EOM
+			}
+		}
+		last if $ID ne '' && $VERSION_ID ne '';
+	}
+	$ID = 'linux' if $ID eq ''; # cf. os-release(5)
+} elsif ($^O =~ m!\A(?:free|net|open)bsd\z!) { # TODO: net? dragonfly?
+	$ID = $^O;
+	require POSIX;
+	(undef, undef, $release, $version) = POSIX::uname();
+	$VERSION_ID = lc $release;
+	$VERSION_ID =~ s/[^0-9a-z\.\_\-]//sg; # cf. os-release(5)
+} else { # only support POSIX-like and Free systems:
+	die "$^O unsupported";
+}
+$VERSION_ID //= 0; # numeric? could be 'sid', actually...
+my %MIN_VER = (freebsd => v11, openbsd => v7.3, netbsd => v9.3);
+
+if (defined(my $min_ver = $MIN_VER{$^O})) {
+	my $vstr = eval "v$VERSION_ID";
+	die <<EOM if $vstr lt $min_ver;
+ID=$ID release=$release ($version) too old to support
+EOM
+}
+my $PKG_FMT = do {
+	if ($ID eq 'freebsd') { 'pkg' }
+	# *shrug*, as long as the name doesn't conflict with FreeBSD's
+	elsif ($ID eq 'openbsd') { 'pkg_add' }
+	elsif ($ID =~ m!\A(?:debian|ubuntu)\z!) { 'deb' }
+	elsif ($ID =~ m!\A(?:centos|redhat|fedora)\z!) { 'rpm' }
+	else { die "PKG_FMT undefined for ID=$ID" }
+};
+
+# these package group names and '-' syntax are passed to ci/deps.perl
+my $TASKS = do {
+	if ($ID eq 'freebsd') { <<EOM
+all devtest IO::KQueue-
+all devtest IO::KQueue
+all devtest Inline::C-
+all devtest Inline::C
+EOM
+	} elsif ($ID eq 'debian') { <<EOM
+all devtest
+all devtest Search::Xapian-
+all devtest-
+v2essential
+essential
+essential devtest-
+EOM
+	} elsif ($ID eq 'centos') { <<EOM
+v2essential devtest
+essential devtest
+all Search::Xapian-
+EOM
+	} elsif ($ID eq 'openbsd') { <<EOM
+all devtest Inline::C-
+all devtest Inline::C
+all devtest IO::KQueue-
+all devtest IO::KQueue
+EOM
+	} else { die "TODO: support ID=$ID VERSION_ID=$VERSION_ID" }
+};
+
+$TASKS =~ s/^/$PKG_FMT /gms;
+print $TASKS;
diff --git a/ci/profiles.sh b/ci/profiles.sh
deleted file mode 100755
index 04cefa15..00000000
--- a/ci/profiles.sh
+++ /dev/null
@@ -1,98 +0,0 @@
-#!/bin/sh
-# Copyright (C) all contributors <meta@public-inbox.org>
-# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
-
-# Prints OS-specific package profiles to stdout (one per line) to use
-# as command-line args for ci/deps.perl.  Called automatically by ci/run.sh
-
-# set by os-release(5) or similar
-ID= VERSION_ID=
-case $(uname -o 2>/dev/null || uname -s) in
-GNU/Linux)
-	for f in /etc/os-release /usr/lib/os-release
-	do
-		test -f $f || continue
-		. $f
-
-		# Debian sid (and testing) have no VERSION_ID
-		case $ID--$VERSION_ID in
-		debian--)
-			case $PRETTY_NAME in
-			*/sid) VERSION_ID=sid ;;
-			*)
-				echo >&2 "$ID, but no VERSION_ID"
-				echo >&2 "==> $f <=="
-				cat >&2 $f
-				exit 1
-				;;
-			esac
-			;;
-		esac
-
-		case $ID--$VERSION_ID in
-		*--|--*) continue ;;
-		*) break ;;
-		esac
-	done
-	;;
-FreeBSD)
-	ID=freebsd
-	VERSION_ID=$(uname -r | cut -d . -f 1)
-	test "$VERSION_ID" -lt 11 && {
-		echo >&2 "ID=$ID $(uname -r) too old to support";
-		exit 1
-	}
-	;;
-OpenBSD)
-	ID=openbsd
-	VERSION_ID=$(uname -r | cut -d . -f 1)
-	test "$VERSION_ID" -lt 7 && {
-		echo >&2 "ID=$ID $(uname -r) too old to support";
-		exit 1
-	}
-	;;
-esac
-
-case $ID in
-freebsd) PKG_FMT=pkg ;;
-debian|ubuntu) PKG_FMT=deb ;;
-centos|redhat|fedora) PKG_FMT=rpm ;;
-openbsd) PKG_FMT=pkg_add ;; # unsure about name, but it's not FreeBSD `pkg'
-*) echo >&2 "PKG_FMT undefined for ID=$ID in $0"
-esac
-
-case $ID-$VERSION_ID in
-freebsd-11|freebsd-12) sed "s/^/$PKG_FMT /" <<EOF
-all devtest-
-all devtest IO::KQueue-
-all devtest IO::KQueue
-v2essential
-essential
-essential devtest-
-EOF
-	;;
-debian-sid|debian-9|debian-10) sed "s/^/$PKG_FMT /" <<EOF
-all devtest
-all devtest Search::Xapian-
-all devtest-
-v2essential
-essential
-essential devtest-
-EOF
-	;;
-centos-7) sed "s/^/$PKG_FMT /" <<EOF
-v2essential devtest
-essential devtest
-all Search::Xapian-
-EOF
-	;;
-openbsd-7) sed "s/^/$PKG_FMT /" <<EOF
-all devtest-
-all devtest Inline::C-
-all devtest Inline::C
-v2essential
-essential
-essential devtest-
-EOF
-	;;
-esac
diff --git a/ci/run.sh b/ci/run.sh
index 1faf92c2..8f717508 100755
--- a/ci/run.sh
+++ b/ci/run.sh
@@ -13,7 +13,7 @@ fi
 NPROC=${NPROC-$({ getconf _NPROCESSORS_ONLN || getconf NPROCESSORS_ONLN ||
 	gnproc || nproc || echo 2; } 2>/dev/null)}
 
-./ci/profiles.sh | while read args
+$PERL -w ci/profiles.perl | while read args
 do
 	$DO $SUDO $PERL -w ci/deps.perl $args
 	$DO $PERL Makefile.PL

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 5/8] update CI helper scripts for NetBSD and `pkgin'
  2023-09-09 12:01 [PATCH 0/8] NetBSD-related updates Eric Wong
                   ` (3 preceding siblings ...)
  2023-09-09 12:01 ` [PATCH 4/8] ci/profiles: rewrite in Perl Eric Wong
@ 2023-09-09 12:01 ` Eric Wong
  2023-09-09 12:01 ` [PATCH 6/8] xap_helper: note __cleanup__ works with C++ exceptions Eric Wong
                   ` (2 subsequent siblings)
  7 siblings, 0 replies; 11+ messages in thread
From: Eric Wong @ 2023-09-09 12:01 UTC (permalink / raw)
  To: meta

---
 ci/deps.perl     | 24 +++++++++++++++++++++---
 ci/profiles.perl | 12 ++++--------
 2 files changed, 25 insertions(+), 11 deletions(-)

diff --git a/ci/deps.perl b/ci/deps.perl
index 68b602ef..6bc38735 100755
--- a/ci/deps.perl
+++ b/ci/deps.perl
@@ -67,6 +67,7 @@ my $non_auto = {
 	},
 	perl => {
 		pkg => 'perl5',
+		pkgin => 'perl',
 		pkg_add => [], # Perl is part of OpenBSD base
 	},
 	'Date::Parse' => {
@@ -87,11 +88,13 @@ my $non_auto = {
 	'Search::Xapian' => {
 		pkg => [qw(xapian-core p5-Xapian)],
 		pkg_add => [qw(xapian-core xapian-bindings-perl)],
+		pkgin => [qw(xapian p5-Xapian)],
 		rpm => 'Search::Xapian', # 3rd-party repo
 	},
 	'highlight.pm' => {
 		deb => 'libhighlight-perl',
 		pkg => [],
+		pkgin => 'p5-highlight',
 		rpm => [],
 	},
 
@@ -99,6 +102,7 @@ my $non_auto = {
 	# xapian-delve(1) in public-inbox-cindex(1)
 	'xapian-tools' => {
 		pkg => 'xapian-core',
+		pkgin => 'xapian',
 		rpm => 'xapian-core', # ???
 	},
 
@@ -114,21 +118,29 @@ for (qw(Digest::SHA Encode ExtUtils::MakeMaker IO::Compress Test::Simple)) {
 	$non_auto->{$_} = {
 		deb => 'perl', # libperl5.XX, but the XX varies
 		pkg => 'perl5',
+		pkg_add => [], # perl is in the OpenBSD base system
+		pkgin => 'perl',
 	};
 }
 
+# NetBSD and OpenBSD package names are similar to FreeBSD in most cases
 if ($pkg_fmt eq 'pkg_add') {
 	for my $name (keys %$non_auto) {
 		my $fbsd_pkg = $non_auto->{$name}->{pkg};
-		$fbsd_pkg = [] if ($fbsd_pkg // '') eq 'perl5';
 		$non_auto->{$name}->{pkg_add} //= $fbsd_pkg if $fbsd_pkg;
 	}
+} elsif ($pkg_fmt eq 'pkgin') {
+	for my $name (keys %$non_auto) {
+		my $fbsd_pkg = $non_auto->{$name}->{pkg};
+		$non_auto->{$name}->{pkgin} //= $fbsd_pkg if $fbsd_pkg;
+	}
 }
 
 my %inst_check = (
 	pkg => sub { system(qw(pkg info -q), $_[0]) == 0 },
 	deb => sub { system("dpkg -s $_[0] >/dev/null 2>&1") == 0 },
 	pkg_add => sub { system(qw(pkg_info -q -e), "$_[0]->=0") == 0 },
+	pkgin => sub { system(qw(pkg_info -q -e), $_[0]) == 0 },
 	rpm => sub { system("rpm -qs $_[0] >/dev/null 2>&1") == 0 },
 );
 
@@ -139,7 +151,7 @@ my (@pkg_install, @pkg_remove, %all);
 for my $ary (values %$profiles) {
 	$all{$_} = \@pkg_remove for @$ary;
 }
-if ($^O =~ /\A(?:free|open)bsd\z/) {
+if ($^O =~ /\A(?:free|net|open)bsd\z/) {
 	$all{'IO::KQueue'} = \@pkg_remove;
 }
 $profiles->{all} = [ keys %all ]; # pseudo-profile for all packages
@@ -184,6 +196,12 @@ if ($pkg_fmt eq 'deb') {
 	root(qw(pkg remove -y), @quiet, @pkg_remove) if @pkg_remove;
 	root(qw(pkg install -y), @quiet, @pkg_install) if @pkg_install;
 	root(qw(pkg autoremove -y), @quiet);
+} elsif ($pkg_fmt eq 'pkgin') { # NetBSD
+	my @quiet = $ENV{V} ? ('-'.('V'x$ENV{V})) : ();
+	exclude_uninstalled(\@pkg_remove);
+	root(qw(pkgin -y), @quiet, 'remove', @pkg_remove) if @pkg_remove;
+	root(qw(pkgin -y), @quiet, 'install', @pkg_install) if @pkg_install;
+	root(qw(pkgin -y), @quiet, 'autoremove');
 # TODO: yum / rpm support
 } elsif ($pkg_fmt eq 'rpm') {
 	my @quiet = $ENV{V} ? () : ('-q');
@@ -229,7 +247,7 @@ sub pkg2ospkg {
 		} elsif ($fmt eq 'rpm') {
 			$pkg =~ s/::/-/g;
 			return "perl-$pkg"
-		} elsif ($fmt =~ /\Apkg(?:_add)?\z/) {
+		} elsif ($fmt =~ /\Apkg(?:_add|in)?\z/) {
 			$pkg =~ s/::/-/g;
 			return "p5-$pkg"
 		} else {
diff --git a/ci/profiles.perl b/ci/profiles.perl
index 3d67143a..5b441790 100755
--- a/ci/profiles.perl
+++ b/ci/profiles.perl
@@ -56,7 +56,8 @@ EOM
 }
 my $PKG_FMT = do {
 	if ($ID eq 'freebsd') { 'pkg' }
-	# *shrug*, as long as the name doesn't conflict with FreeBSD's
+	# *shrug*, as long as the (Net|Open)BSD names don't conflict w/ FreeBSD
+	elsif ($ID eq 'netbsd') { 'pkgin' }
 	elsif ($ID eq 'openbsd') { 'pkg_add' }
 	elsif ($ID =~ m!\A(?:debian|ubuntu)\z!) { 'deb' }
 	elsif ($ID =~ m!\A(?:centos|redhat|fedora)\z!) { 'rpm' }
@@ -65,7 +66,8 @@ my $PKG_FMT = do {
 
 # these package group names and '-' syntax are passed to ci/deps.perl
 my $TASKS = do {
-	if ($ID eq 'freebsd') { <<EOM
+	if ($ID =~ /\A(?:free|net|open)bsd\z/) { <<EOM
+all devtest Search::Xapian-
 all devtest IO::KQueue-
 all devtest IO::KQueue
 all devtest Inline::C-
@@ -83,12 +85,6 @@ EOM
 v2essential devtest
 essential devtest
 all Search::Xapian-
-EOM
-	} elsif ($ID eq 'openbsd') { <<EOM
-all devtest Inline::C-
-all devtest Inline::C
-all devtest IO::KQueue-
-all devtest IO::KQueue
 EOM
 	} else { die "TODO: support ID=$ID VERSION_ID=$VERSION_ID" }
 };

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 6/8] xap_helper: note __cleanup__ works with C++ exceptions
  2023-09-09 12:01 [PATCH 0/8] NetBSD-related updates Eric Wong
                   ` (4 preceding siblings ...)
  2023-09-09 12:01 ` [PATCH 5/8] update CI helper scripts for NetBSD and `pkgin' Eric Wong
@ 2023-09-09 12:01 ` Eric Wong
  2023-09-16 13:10   ` [PATCH] xap_helper: test cleanup + throw support in C++ Eric Wong
  2023-09-09 12:01 ` [PATCH 7/8] xap_helper: use _OPENBSD_SOURCE on NetBSD for reallocarray Eric Wong
  2023-09-09 12:01 ` [PATCH 8/8] xap_helper: clamp workers to USHRT_MAX Eric Wong
  7 siblings, 1 reply; 11+ messages in thread
From: Eric Wong @ 2023-09-09 12:01 UTC (permalink / raw)
  To: meta

In case somebody reviewing this code gets curious.
---
 lib/PublicInbox/xap_helper.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/PublicInbox/xap_helper.h b/lib/PublicInbox/xap_helper.h
index 70760367..3f45e1c7 100644
--- a/lib/PublicInbox/xap_helper.h
+++ b/lib/PublicInbox/xap_helper.h
@@ -322,6 +322,9 @@ struct dump_roots_tmp {
 	int root2id_fd;
 };
 
+// n.b. __cleanup__ works fine with C++ exceptions, but not longjmp
+// Only clang and g++ are supported, as AFAIK there's no other
+// relevant Free(-as-in-speech) C++ compilers.
 #define CLEANUP_FBUF __attribute__((__cleanup__(fbuf_ensure)))
 static void fbuf_ensure(void *ptr)
 {

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 7/8] xap_helper: use _OPENBSD_SOURCE on NetBSD for reallocarray
  2023-09-09 12:01 [PATCH 0/8] NetBSD-related updates Eric Wong
                   ` (5 preceding siblings ...)
  2023-09-09 12:01 ` [PATCH 6/8] xap_helper: note __cleanup__ works with C++ exceptions Eric Wong
@ 2023-09-09 12:01 ` Eric Wong
  2023-09-09 12:01 ` [PATCH 8/8] xap_helper: clamp workers to USHRT_MAX Eric Wong
  7 siblings, 0 replies; 11+ messages in thread
From: Eric Wong @ 2023-09-09 12:01 UTC (permalink / raw)
  To: meta

NetBSD prefers reallocarr(3) for predictable zero-sized
allocation behavior; but no other OS seems to have reallocarr(3).
reallocarray(3) appears in by OpenBSD, FreeBSD, glibc, and musl,
so continue to go with that.
---
 lib/PublicInbox/xap_helper.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/PublicInbox/xap_helper.h b/lib/PublicInbox/xap_helper.h
index 3f45e1c7..add2fe8c 100644
--- a/lib/PublicInbox/xap_helper.h
+++ b/lib/PublicInbox/xap_helper.h
@@ -14,6 +14,9 @@
 #ifndef _ALL_SOURCE
 #	define _ALL_SOURCE
 #endif
+#if defined(__NetBSD__) && !defined(_OPENBSD_SOURCE) // for reallocarray(3)
+#	define _OPENBSD_SOURCE
+#endif
 #include <sys/file.h>
 #include <sys/mman.h>
 #include <sys/resource.h>

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH 8/8] xap_helper: clamp workers to USHRT_MAX
  2023-09-09 12:01 [PATCH 0/8] NetBSD-related updates Eric Wong
                   ` (6 preceding siblings ...)
  2023-09-09 12:01 ` [PATCH 7/8] xap_helper: use _OPENBSD_SOURCE on NetBSD for reallocarray Eric Wong
@ 2023-09-09 12:01 ` Eric Wong
  7 siblings, 0 replies; 11+ messages in thread
From: Eric Wong @ 2023-09-09 12:01 UTC (permalink / raw)
  To: meta

This allows us to avoid any integer overflow problems while
having enough room to grow for some future hardware, though it
looks like having hundreds of cores isn't ever going to make
it to typical servers nor workstations.
---
 lib/PublicInbox/xap_helper.h | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/lib/PublicInbox/xap_helper.h b/lib/PublicInbox/xap_helper.h
index add2fe8c..7210c940 100644
--- a/lib/PublicInbox/xap_helper.h
+++ b/lib/PublicInbox/xap_helper.h
@@ -78,6 +78,7 @@ static FILE *orig_err = stderr;
 static int orig_err_fd = -1;
 static void *srch_tree; // tsearch + tdelete + twalk
 static pid_t *worker_pids; // nr => pid
+#define WORKER_MAX USHRT_MAX
 static unsigned long nworker, nworker_hwm;
 static int pipefds[2];
 
@@ -1063,6 +1064,10 @@ static void do_sigchld(void)
 static void do_sigttin(void)
 {
 	if (!alive) return;
+	if (nworker >= WORKER_MAX) {
+		warnx("workers cannot exceed %zu", (size_t)WORKER_MAX);
+		return;
+	}
 	void *p = reallocarray(worker_pids, nworker + 1, sizeof(pid_t));
 	if (!p) {
 		warn("reallocarray");
@@ -1117,7 +1122,7 @@ int main(int argc, char *argv[])
 #ifdef _SC_NPROCESSORS_ONLN
 	long j = sysconf(_SC_NPROCESSORS_ONLN);
 	if (j > 0)
-		nworker = j > UCHAR_MAX ? UCHAR_MAX : j;
+		nworker = j > WORKER_MAX ? WORKER_MAX : j;
 #endif // _SC_NPROCESSORS_ONLN
 
 	// make warn/warnx/err multi-process friendly:
@@ -1130,7 +1135,7 @@ int main(int argc, char *argv[])
 		switch (c) {
 		case 'j':
 			nworker = strtoul(optarg, &end, 10);
-			if (*end != 0 || nworker > USHRT_MAX)
+			if (*end != 0 || nworker > WORKER_MAX)
 				errx(EXIT_FAILURE, "-j %s invalid", optarg);
 			break;
 		case ':':

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [SQUASH] manifest update
  2023-09-09 12:01 ` [PATCH 4/8] ci/profiles: rewrite in Perl Eric Wong
@ 2023-09-09 21:14   ` Eric Wong
  0 siblings, 0 replies; 11+ messages in thread
From: Eric Wong @ 2023-09-09 21:14 UTC (permalink / raw)
  To: meta

---
 MANIFEST | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/MANIFEST b/MANIFEST
index 5964794e..1fe1c7f7 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -113,7 +113,7 @@ certs/.gitignore
 certs/create-certs.perl
 ci/README
 ci/deps.perl
-ci/profiles.sh
+ci/profiles.perl
 ci/run.sh
 contrib/completion/lei-completion.bash
 contrib/css/216dark.css

^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH] xap_helper: test cleanup + throw support in C++
  2023-09-09 12:01 ` [PATCH 6/8] xap_helper: note __cleanup__ works with C++ exceptions Eric Wong
@ 2023-09-16 13:10   ` Eric Wong
  0 siblings, 0 replies; 11+ messages in thread
From: Eric Wong @ 2023-09-16 13:10 UTC (permalink / raw)
  To: meta

Eric Wong <e@80x24.org> wrote:
> In case somebody reviewing this code gets curious.

And just in case we encounter other C++ implementations:
-------8<-------
Subject: [PATCH] xap_helper: test cleanup + throw support in C++

We need to ensure whatever C++ compiler gets used supports this
GNU extension as we expect it to.  clang 13 and 14 and g++ 10
all work as we expect it to.
---
 side note: sadly, longjmp/siglongjmp(3) in POSIX does not trigger
 __cleanup__

 lib/PublicInbox/xap_helper.h | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/lib/PublicInbox/xap_helper.h b/lib/PublicInbox/xap_helper.h
index 2322d062..377ff45a 100644
--- a/lib/PublicInbox/xap_helper.h
+++ b/lib/PublicInbox/xap_helper.h
@@ -578,12 +578,27 @@ static bool cmd_dump_roots(struct req *req)
 	return true;
 }
 
-// internal usage only
+// for test usage only, we need to ensure the compiler supports
+// __cleanup__ when exceptions are thrown
+struct inspect { struct req *req; };
+
+static void inspect_ensure(struct inspect *x)
+{
+	fprintf(x->req->fp[0], "pid=%d has_threadid=%d",
+		(int)getpid(), has_threadid(x->req->srch) ? 1 : 0);
+}
+
 static bool cmd_test_inspect(struct req *req)
 {
-	fprintf(req->fp[0], "pid=%d has_threadid=%d",
-		(int)getpid(), has_threadid(req->srch) ? 1 : 0);
-	return true;
+	__attribute__((__cleanup__(inspect_ensure))) struct inspect x;
+	x.req = req;
+	try {
+		throw Xapian::InvalidArgumentError("test");
+	} catch (Xapian::InvalidArgumentError) {
+		return true;
+	}
+	fputs("this should not be printed", req->fp[0]);
+	return false;
 }
 
 #define CMD(n) { .fn_len = sizeof(#n) - 1, .fn_name = #n, .fn = cmd_##n }

^ permalink raw reply related	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2023-09-16 13:10 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-09-09 12:01 [PATCH 0/8] NetBSD-related updates Eric Wong
2023-09-09 12:01 ` [PATCH 1/8] Makefile.PL: check `getconf NPROCESSORS_ONLN', too Eric Wong
2023-09-09 12:01 ` [PATCH 2/8] ipc: define _SC_NPROCESSORS_ONLN for NetBSD Eric Wong
2023-09-09 12:01 ` [PATCH 3/8] ci/run.sh: parameterize BUILD_JOBS TEST_JOBS and TEST_TARGET Eric Wong
2023-09-09 12:01 ` [PATCH 4/8] ci/profiles: rewrite in Perl Eric Wong
2023-09-09 21:14   ` [SQUASH] manifest update Eric Wong
2023-09-09 12:01 ` [PATCH 5/8] update CI helper scripts for NetBSD and `pkgin' Eric Wong
2023-09-09 12:01 ` [PATCH 6/8] xap_helper: note __cleanup__ works with C++ exceptions Eric Wong
2023-09-16 13:10   ` [PATCH] xap_helper: test cleanup + throw support in C++ Eric Wong
2023-09-09 12:01 ` [PATCH 7/8] xap_helper: use _OPENBSD_SOURCE on NetBSD for reallocarray Eric Wong
2023-09-09 12:01 ` [PATCH 8/8] xap_helper: clamp workers to USHRT_MAX Eric Wong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).