From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.bugs Subject: bug#46881: 28.0.50; pdumper dumping causes way too many syscalls Date: Wed, 03 Mar 2021 07:51:05 +0200 Message-ID: <83r1kw6b06.fsf@gnu.org> References: Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="3844"; mail-complaints-to="usenet@ciao.gmane.io" Cc: 46881@debbugs.gnu.org To: Pip Cet , Daniel Colascione , Paul Eggert Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Mar 03 06:52:14 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lHKQg-0000sr-KA for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 03 Mar 2021 06:52:14 +0100 Original-Received: from localhost ([::1]:59876 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lHKQf-0005HN-Lt for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 03 Mar 2021 00:52:13 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:59362) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lHKQU-0005Gf-Se for bug-gnu-emacs@gnu.org; Wed, 03 Mar 2021 00:52:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:43430) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lHKQU-0000mO-Lv for bug-gnu-emacs@gnu.org; Wed, 03 Mar 2021 00:52:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lHKQU-0003BC-Kl for bug-gnu-emacs@gnu.org; Wed, 03 Mar 2021 00:52:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 03 Mar 2021 05:52:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46881 X-GNU-PR-Package: emacs Original-Received: via spool by 46881-submit@debbugs.gnu.org id=B46881.161475068312158 (code B ref 46881); Wed, 03 Mar 2021 05:52:02 +0000 Original-Received: (at 46881) by debbugs.gnu.org; 3 Mar 2021 05:51:23 +0000 Original-Received: from localhost ([127.0.0.1]:54974 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHKPq-0003A2-Ul for submit@debbugs.gnu.org; Wed, 03 Mar 2021 00:51:23 -0500 Original-Received: from eggs.gnu.org ([209.51.188.92]:41330) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lHKPo-00039p-46 for 46881@debbugs.gnu.org; Wed, 03 Mar 2021 00:51:21 -0500 Original-Received: from fencepost.gnu.org ([2001:470:142:3::e]:59393) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lHKPh-0000S3-Ue; Wed, 03 Mar 2021 00:51:13 -0500 Original-Received: from 84.94.185.95.cable.012.net.il ([84.94.185.95]:2473 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lHKPf-0003vJ-MQ; Wed, 03 Mar 2021 00:51:13 -0500 In-Reply-To: (message from Pip Cet on Tue, 2 Mar 2021 20:45:04 +0000) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:201254 Archived-At: > From: Pip Cet > Date: Tue, 2 Mar 2021 20:45:04 +0000 > > On Tue, Mar 2, 2021 at 8:35 PM Pip Cet wrote: > > I've looked into the problem, and it seems easy to solve and worth it > > in terms of debuggability and performance. > > Very rough benchmarks, but this seems to be clearly worth it: > > Performance: > With patch: > real 0m3.861s > user 0m3.776s > sys 0m0.085s > > Without patch: > real 0m7.001s > user 0m4.476s > sys 0m2.511s > > Number of syscalls: > With patch: 415442 > Without patch: 2028307 > > > Patch will be attached once this has a bug number. > > And here's the patch. Testing would be very appreciated. > > I'm unsure about the precise usage of dump_off vs ptrdiff_t here; I > don't think it matters, but suggestions, nitpicks, and comments, on > this or any other aspect, would be very appreciated. > From 92ee138852b34ede2f43dd7f93f310fc746bb3bf Mon Sep 17 00:00:00 2001 > From: Pip Cet > Date: Tue, 2 Mar 2021 20:38:23 +0000 > Subject: [PATCH] Prepare pdumper dump file in memory, write it in one go > (Bug#46881) > > * src/pdumper.c (struct dump_context): Add buf, buf_size, max_offset fields. > (grow_buffer): New function. > (dump_write): Use memcpy, not an actual emacs_write. > (dump_seek): Keep track of maximum seen offset. > (Fdump_emacs_portable): Write out the file contents when done. > --- > src/pdumper.c | 20 ++++++++++++++++++-- > 1 file changed, 18 insertions(+), 2 deletions(-) > > diff --git a/src/pdumper.c b/src/pdumper.c > index 337742fda4ade..62ddad8ee5e34 100644 > --- a/src/pdumper.c > +++ b/src/pdumper.c > @@ -473,6 +473,10 @@ dump_fingerprint (char const *label, > { > /* Header we'll write to the dump file when done. */ > struct dump_header header; > + /* Data that will be written to the dump file. */ > + void *buf; > + ptrdiff_t buf_size; > + ptrdiff_t max_offset; > > Lisp_Object old_purify_flag; > Lisp_Object old_post_gc_hook; > @@ -581,6 +585,13 @@ dump_fingerprint (char const *label, > > /* Dump file creation */ > > +static void dump_grow_buffer (struct dump_context *ctx) > +{ > + ctx->buf = xrealloc (ctx->buf, ctx->buf_size = (ctx->buf_size ? > + (ctx->buf_size * 2) > + : 1024 * 1024)); > +} > + > static dump_off dump_object (struct dump_context *ctx, Lisp_Object object); > static dump_off dump_object_for_offset (struct dump_context *ctx, > Lisp_Object object); > @@ -747,8 +758,9 @@ dump_write (struct dump_context *ctx, const void *buf, dump_off nbyte) > eassert (nbyte == 0 || buf != NULL); > eassert (ctx->obj_offset == 0); > eassert (ctx->flags.dump_object_contents); > - if (emacs_write (ctx->fd, buf, nbyte) < nbyte) > - report_file_error ("Could not write to dump file", ctx->dump_filename); > + while (ctx->offset + nbyte > ctx->buf_size) > + dump_grow_buffer (ctx); > + memcpy ((char *)ctx->buf + ctx->offset, buf, nbyte); > ctx->offset += nbyte; > } > > @@ -828,6 +840,8 @@ dump_tailq_pop (struct dump_tailq *tailq) > static void > dump_seek (struct dump_context *ctx, dump_off offset) > { > + if (ctx->max_offset < ctx->offset) > + ctx->max_offset = ctx->offset; > eassert (ctx->obj_offset == 0); > if (lseek (ctx->fd, offset, SEEK_SET) < 0) > report_file_error ("Setting file position", > @@ -4159,6 +4173,8 @@ DEFUN ("dump-emacs-portable", > ctx->header.magic[0] = dump_magic[0]; > dump_seek (ctx, 0); > dump_write (ctx, &ctx->header, sizeof (ctx->header)); > + if (emacs_write (ctx->fd, ctx->buf, ctx->max_offset) < ctx->max_offset) > + report_file_error ("Could not write to dump file", ctx->dump_filename); > > dump_off > header_bytes = header_end - header_start, > -- > 2.30.1 Thanks. Daniel, Paul: any comments? In particular, is it safe to allocate large amounts of memory off the heap while dumping? A couple of places in pdumper.c says some parts of code should call malloc.