* [PATCH 0/4] git-related updates
@ 2023-01-25 10:18 Eric Wong
2023-01-25 10:18 ` [PATCH 1/4] git: use core.abbrev=no on git 2.31+ Eric Wong
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Eric Wong @ 2023-01-25 10:18 UTC (permalink / raw)
To: meta
core.abbrev=no is now used in preparation for SHA-256 support.
I started working on supporting git 2.36+ --batch-command, but having
to support prior versions will require more refactoring, I think...
2/4 was painful :<
Eric Wong (4):
git: use core.abbrev=no on git 2.31+
process_pipe: warn hackers off using it for bidirectional pipes
git: drop needless ENOENT import
git: reduce delete ops in _destroy
lib/PublicInbox/Git.pm | 42 ++++++++++++++++++++++++----------
lib/PublicInbox/ProcessPipe.pm | 6 +++--
2 files changed, 34 insertions(+), 14 deletions(-)
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/4] git: use core.abbrev=no on git 2.31+
2023-01-25 10:18 [PATCH 0/4] git-related updates Eric Wong
@ 2023-01-25 10:18 ` Eric Wong
2023-01-25 10:18 ` [PATCH 2/4] process_pipe: warn hackers off using it for bidirectional pipes Eric Wong
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2023-01-25 10:18 UTC (permalink / raw)
To: meta
This makes it easier to support SHA-256 inboxes in the future.
Tested with both git 2.30.2 (Debian stable) and 2.39.1
---
lib/PublicInbox/Git.pm | 27 +++++++++++++++++++++++----
1 file changed, 23 insertions(+), 4 deletions(-)
diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index c9ed48be..ff3ac40f 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -16,7 +16,7 @@ use Errno qw(EINTR EAGAIN ENOENT);
use File::Glob qw(bsd_glob GLOB_NOSORT);
use File::Spec ();
use Time::HiRes qw(stat);
-use PublicInbox::Spawn qw(popen_rd spawn);
+use PublicInbox::Spawn qw(popen_rd which);
use PublicInbox::Tmpfile;
use IO::Poll qw(POLLIN);
use Carp qw(croak carp);
@@ -46,7 +46,6 @@ my %GIT_ESC = (
);
my %ESC_GIT = map { $GIT_ESC{$_} => $_ } keys %GIT_ESC;
-
# unquote pathnames used by git, see quote.c::unquote_c_style.c in git.git
sub git_unquote ($) {
return $_[0] unless ($_[0] =~ /\A"(.*)"\z/);
@@ -122,6 +121,26 @@ sub _bidi_pipe {
}
return;
}
+
+ state $EXE_ST = ''; # pack('dd', st_ctime, st_size);
+ my $exe = which('git') // die "git not found in $ENV{PATH}";
+ my @st = stat($exe) or die "stat: $!";
+ my $st = pack('dd', $st[10], $st[7]);
+ state $VER;
+ if ($st ne $EXE_ST) {
+ my $rd = popen_rd([ $exe, '--version' ]);
+ my $v = readline($rd);
+ $v =~ /\b([0-9]+(?:\.[0-9]+){2})/ or die
+ "$exe --version output: $v # unparseable";
+ my @v = split(/\./, $1, 3);
+ $VER = ($v[0] << 24) | ($v[1] << 16) | $v[2];
+ $EXE_ST = $st;
+ }
+
+ # git 2.31.0+ supports -c core.abbrev=no, don't bother with
+ # core.abbrev=64 since not many releases had SHA-256 prior to 2.31
+ my $abbr = $VER < (2 << 24 | 31 << 16) ? 40 : 'no';
+
pipe(my ($out_r, $out_w)) or $self->fail("pipe failed: $!");
my $rdr = { 0 => $out_r, pgid => 0 };
my $gd = $self->{git_dir};
@@ -129,8 +148,8 @@ sub _bidi_pipe {
$rdr->{-C} = $gd;
$gd = $1;
}
- my @cmd = (qw(git), "--git-dir=$gd",
- qw(-c core.abbrev=40 cat-file), $batch);
+ my @cmd = ($exe, "--git-dir=$gd", '-c', "core.abbrev=$abbr",
+ 'cat-file', $batch);
if ($err) {
my $id = "git.$self->{git_dir}$batch.err";
my $fh = tmpfile($id) or $self->fail("tmpfile($id): $!");
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 2/4] process_pipe: warn hackers off using it for bidirectional pipes
2023-01-25 10:18 [PATCH 0/4] git-related updates Eric Wong
2023-01-25 10:18 ` [PATCH 1/4] git: use core.abbrev=no on git 2.31+ Eric Wong
@ 2023-01-25 10:18 ` Eric Wong
2023-01-25 10:18 ` [PATCH 3/4] git: drop needless ENOENT import Eric Wong
2023-01-25 10:18 ` [PATCH 4/4] git: reduce delete ops in _destroy Eric Wong
3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2023-01-25 10:18 UTC (permalink / raw)
To: meta
While most uses of ->DESTROY happens in a predictable order in
long-lived daemons, process teardown on exit is chaotic and not
subject to ordering guarantees, so we must keep both ends of a
`git cat-file --batch*' pipe at the same level in the object
hierarchy.
Drop an old Carp import while I'm in the area.
---
lib/PublicInbox/Git.pm | 1 +
lib/PublicInbox/ProcessPipe.pm | 6 ++++--
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index ff3ac40f..a3813bf2 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -156,6 +156,7 @@ sub _bidi_pipe {
$self->{$err} = $fh;
$rdr->{2} = $fh;
}
+ # see lib/PublicInbox/ProcessPipe.pm for why we don't use that here
my ($in_r, $p) = popen_rd(\@cmd, undef, $rdr);
awaitpid($self->{$pid} = $p, undef);
$self->{"$pid.owner"} = $$;
diff --git a/lib/PublicInbox/ProcessPipe.pm b/lib/PublicInbox/ProcessPipe.pm
index 068631c6..1bc792c4 100644
--- a/lib/PublicInbox/ProcessPipe.pm
+++ b/lib/PublicInbox/ProcessPipe.pm
@@ -1,10 +1,12 @@
# Copyright (C) all contributors <meta@public-inbox.org>
# License: AGPL-3.0+ <https://www.gnu.org/licenses/agpl-3.0.txt>
-# a tied handle for auto reaping of children tied to a pipe, see perltie(1)
+# a tied handle for auto reaping of children tied to a read-only pipe, see perltie(1)
+# DO NOT use this as-is for bidirectional pipes/sockets (e.g. in PublicInbox::Git),
+# both ends of the pipe must be at the same level of the Perl object hierarchy
+# to ensure orderly destruction.
package PublicInbox::ProcessPipe;
use v5.12;
-use Carp qw(carp);
use PublicInbox::DS qw(awaitpid);
sub waitcb { # awaitpid callback
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 3/4] git: drop needless ENOENT import
2023-01-25 10:18 [PATCH 0/4] git-related updates Eric Wong
2023-01-25 10:18 ` [PATCH 1/4] git: use core.abbrev=no on git 2.31+ Eric Wong
2023-01-25 10:18 ` [PATCH 2/4] process_pipe: warn hackers off using it for bidirectional pipes Eric Wong
@ 2023-01-25 10:18 ` Eric Wong
2023-01-25 10:18 ` [PATCH 4/4] git: reduce delete ops in _destroy Eric Wong
3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2023-01-25 10:18 UTC (permalink / raw)
To: meta
I imported it in commit 356439a571c536eaa487031802b436d087113f4f
(gcf2 + extsearch: check for unlinked files on Linux, 2021-09-22)
but never used it.
---
lib/PublicInbox/Git.pm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index a3813bf2..9197ea67 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -12,7 +12,7 @@ use v5.10.1;
use parent qw(Exporter);
use POSIX ();
use IO::Handle; # ->autoflush
-use Errno qw(EINTR EAGAIN ENOENT);
+use Errno qw(EINTR EAGAIN);
use File::Glob qw(bsd_glob GLOB_NOSORT);
use File::Spec ();
use Time::HiRes qw(stat);
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 4/4] git: reduce delete ops in _destroy
2023-01-25 10:18 [PATCH 0/4] git-related updates Eric Wong
` (2 preceding siblings ...)
2023-01-25 10:18 ` [PATCH 3/4] git: drop needless ENOENT import Eric Wong
@ 2023-01-25 10:18 ` Eric Wong
3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2023-01-25 10:18 UTC (permalink / raw)
To: meta
We can avoid some extra returns and branches by just relying on
variadic arguments.
---
lib/PublicInbox/Git.pm | 12 +++++-------
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/lib/PublicInbox/Git.pm b/lib/PublicInbox/Git.pm
index 9197ea67..12f997dc 100644
--- a/lib/PublicInbox/Git.pm
+++ b/lib/PublicInbox/Git.pm
@@ -373,13 +373,11 @@ sub check {
}
sub _destroy {
- my ($self, $rbuf, $in, $out, $pid, $err) = @_;
- delete @$self{($rbuf, $in, $out)};
- delete $self->{$err} if $err; # `err_c'
+ my ($self, $pid, @rest) = @_; # rest = rbuf, in, out, err
+ my ($p) = delete @$self{($pid, @rest)};
# GitAsyncCat::event_step may delete {$pid}
- my $p = delete($self->{$pid}) // return;
- awaitpid($p) if $$ == $self->{"$pid.owner"};
+ awaitpid($p) if defined($p) && $$ == $self->{"$pid.owner"};
}
sub async_abort ($) {
@@ -468,8 +466,8 @@ sub cleanup {
async_wait_all($self);
delete $self->{inflight};
delete $self->{inflight_c};
- _destroy($self, qw(rbuf in out pid));
- _destroy($self, qw(rbuf_c in_c out_c pid_c err_c));
+ _destroy($self, qw(pid rbuf in out));
+ _destroy($self, qw(pid_c rbuf_c in_c out_c err_c));
undef;
}
^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-01-25 10:18 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-01-25 10:18 [PATCH 0/4] git-related updates Eric Wong
2023-01-25 10:18 ` [PATCH 1/4] git: use core.abbrev=no on git 2.31+ Eric Wong
2023-01-25 10:18 ` [PATCH 2/4] process_pipe: warn hackers off using it for bidirectional pipes Eric Wong
2023-01-25 10:18 ` [PATCH 3/4] git: drop needless ENOENT import Eric Wong
2023-01-25 10:18 ` [PATCH 4/4] git: reduce delete ops in _destroy Eric Wong
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).