unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
From: Alan Mackenzie <acm@muc.de>
To: Gregory Heytings <gregory@heytings.org>
Cc: mattiase@acm.org, Eli Zaretskii <eliz@gnu.org>,
	emacs-devel@gnu.org, larsi@gnus.org, monnier@iro.umontreal.ca
Subject: Re: Time to merge scratch/correct-warning-pos into master, perhaps?
Date: Wed, 26 Jan 2022 19:34:49 +0000	[thread overview]
Message-ID: <YfGiWZUTZGPh8LKH@ACM> (raw)
In-Reply-To: <a7c43577b900578b403e@heytings.org>

Hello, Gregory.

On Wed, Jan 26, 2022 at 18:41:57 +0000, Gregory Heytings wrote:


> > I think we need more measurements in scenarios closer to actual Emacs 
> > usage.  One is intensive display operation -- I think Alan posted 
> > something to that effect.  Another could be starting Gnus to read a 
> > large newsgroup.  Yet another could be reindent a large piece of C or 
> > Python or Lisp code.  Perhaps also "M-x occur" through a large buffer. 
> > Stuff like that -- any command that tends to be used frequently and is 
> > known to take a tangible amount of time.

> The problem is that, IME, such benchmarks do not show anything when the 
> slowdown is not significant enough (> 10%), because the measurements vary 
> a lot depending on external factors.  I wonder how Alan could conclude 
> that there is a slowdown of "less than 1%", when I run his benchmark on my 
> (otherwise unloaded) computer, the running times I get are anywhere 
> between 17s and 20s.  Taking the average of such measures does not really 
> make sense.

No, but the first run in an Emacs session tends to be shorter than
subsequent runs.  I don't know why.  I take it you also saw a very small
difference in the timings for this benchmark between the two versions.
Otherwise you would have said, I think.

The four measurements I got from my benchmark in master (from
benchmark-run) were:
      o - (20.798601883 460 7.728701306)
      o - (20.947118356 295 7.1172684969999995)
      o - (20.941589929 293 7.144901186)
      o - (20.917180235 293 7.136285445000002)
The corresponding four from the (then) branch were
      o - (20.854543266 480 7.691123986)
      o - (21.064465459 320 7.189660959000001)
      o - (21.143813105 318 7.287708998000001)
      o - (21.115932422 318 7.266432223999999)
It is fairly clear the second set are ~1% more than the first set, give
or take unimportant measurement errors.

That's precisely what I meant when I said that the branch was about 1%
slower ON THAT BENCHMARK.  Please note what I said, going back to my
original post if needed.  I did not say "less than" 1%.  I was not
referring to the speed in general.  I was very specific about what I
wrote.

> However, I found one measurement that seems to give reasonably stable 
> results.  On my Debian bookworm computer, with a standard build, on 
> src/xdisp.c, (benchmark-run 100 (occur "a.b")) needs on average (with 10 
> runs) 7.42 seconds (standard deviation 0.04) on 3b33a14380 and 7.66 
> seconds (standard deviation 0.05) on 7922131bb2.  That's a >3% slowdown.

Yes, that's around 3.2%.  Scarcely something you'd notice in an
interactive session, where the difference is around the time it takes to
depress a key on the keyboard.

So we now have a real life measurement which is more than 1%.  It's
nowhere near 10%, though.

-- 
Alan Mackenzie (Nuremberg, Germany).



  parent reply	other threads:[~2022-01-26 19:34 UTC|newest]

Thread overview: 150+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-01-15 18:15 Time to merge scratch/correct-warning-pos into master, perhaps? Alan Mackenzie
2022-01-15 18:24 ` Eli Zaretskii
2022-01-16  8:24   ` Lars Ingebrigtsen
2022-01-16  9:51     ` Po Lu
2022-01-16 13:50       ` Alan Mackenzie
2022-01-16 12:02     ` Lars Ingebrigtsen
2022-01-16 12:06       ` Po Lu
2022-01-16 13:12         ` Lars Ingebrigtsen
2022-01-16 14:02           ` Alan Mackenzie
2022-01-17  0:28           ` Po Lu
2022-01-17  3:28             ` Eli Zaretskii
2022-01-17  3:37               ` Po Lu
2022-01-16 14:00         ` Alan Mackenzie
2022-01-16 12:23       ` Eli Zaretskii
2022-01-16 12:26         ` Lars Ingebrigtsen
2022-01-16 12:36           ` Eli Zaretskii
2022-01-16 14:06         ` Alan Mackenzie
2022-01-16 14:11           ` Lars Ingebrigtsen
2022-01-16 14:21           ` Eli Zaretskii
2022-01-16 14:45             ` Alan Mackenzie
2022-01-16 15:04               ` Eli Zaretskii
2022-01-16 15:26                 ` Alan Mackenzie
2022-01-16 15:43                   ` Eli Zaretskii
2022-01-16 15:50                     ` Alan Mackenzie
2022-01-16 13:47     ` Alan Mackenzie
2022-01-16 14:10       ` Lars Ingebrigtsen
2022-01-16 14:59         ` Stefan Monnier
2022-01-16 14:57     ` Stefan Monnier
2022-01-16 15:04       ` Lars Ingebrigtsen
2022-01-16 15:37         ` Alan Mackenzie
2022-01-16 15:50         ` Mattias Engdegård
2022-01-16 16:18           ` Alan Mackenzie
2022-01-16 16:52             ` Mattias Engdegård
2022-01-16 17:13               ` Stefan Monnier
2022-01-16 17:24                 ` Mattias Engdegård
2022-01-22 12:23         ` Alan Mackenzie
2022-01-22 14:30           ` Lars Ingebrigtsen
2022-01-22 15:09             ` Alan Mackenzie
2022-01-22 15:26               ` Lars Ingebrigtsen
2022-01-22 15:36                 ` Eli Zaretskii
2022-01-22 18:30                   ` Alan Mackenzie
2022-01-23 12:45                     ` Lars Ingebrigtsen
2022-01-22 16:06             ` Mattias Engdegård
2022-01-22 17:02               ` Gregory Heytings
2022-01-22 17:46                 ` Eli Zaretskii
2022-01-22 18:01                   ` Gregory Heytings
2022-01-22 18:12                     ` Eli Zaretskii
2022-01-22 22:36                       ` Gregory Heytings
2022-01-22 22:55                         ` Alan Mackenzie
2022-01-23  6:16                         ` Eli Zaretskii
2022-01-23 21:53                           ` Gregory Heytings
2022-01-24  3:37                             ` Eli Zaretskii
2022-01-24 15:20                               ` Gregory Heytings
2022-01-24 16:47                                 ` Eli Zaretskii
2022-01-24 20:41                                   ` Gregory Heytings
2022-01-25  3:34                                     ` Eli Zaretskii
2022-01-25  8:59                                       ` Gregory Heytings
2022-01-25 11:27                                         ` Alan Mackenzie
2022-01-25 13:27                                           ` Stefan Monnier
2022-01-25 18:27                                             ` Alan Mackenzie
2022-01-25 19:26                                               ` Stefan Monnier
2022-01-25 20:58                                                 ` Alan Mackenzie
2022-01-25 21:27                                                   ` Gregory Heytings
2022-01-26 17:32                                                     ` Alan Mackenzie
2022-01-26 18:59                                                       ` Gregory Heytings
2022-01-26 20:26                                                         ` Alan Mackenzie
2022-01-25 22:11                                                   ` Stefan Monnier
2022-01-25 22:42                                                     ` Óscar Fuentes
2022-01-26  1:08                                                       ` Po Lu
2022-01-26 16:56                                                         ` chad
2022-01-26 17:38                                                           ` Eli Zaretskii
2022-01-26 17:58                                                             ` chad
2022-01-26 18:46                                                         ` Gregory Heytings
2022-01-26 19:47                                                           ` Stefan Monnier
2022-01-26 19:59                                                           ` Alan Mackenzie
2022-01-25 21:18                                                 ` Gregory Heytings
2022-01-25 21:38                                             ` Gregory Heytings
2022-01-25 22:21                                               ` Stefan Monnier
2022-01-26 18:36                                                 ` Gregory Heytings
2022-02-04  0:11                                                 ` Gregory Heytings
2022-02-04 11:57                                                   ` Alan Mackenzie
2022-02-04 12:06                                                     ` Eli Zaretskii
2022-02-04 18:31                                                       ` Alan Mackenzie
2022-02-04 18:54                                                         ` Eli Zaretskii
2022-02-04 19:33                                                           ` Alan Mackenzie
2022-02-04 19:46                                                             ` Eli Zaretskii
2022-02-04 21:24                                                               ` Alan Mackenzie
2022-02-04 22:24                                                                 ` Stefan Monnier
2022-02-04 22:30                                                                   ` Stefan Monnier
2022-02-05  7:28                                                                     ` Eli Zaretskii
2022-02-05 18:04                                                                       ` Stefan Monnier
2022-02-05 18:31                                                                         ` Eli Zaretskii
2022-02-05  8:17                                                                 ` Eli Zaretskii
2022-02-06 11:50                                                                   ` Alan Mackenzie
2022-02-06 11:56                                                                     ` Eli Zaretskii
2022-02-06 18:09                                                                       ` Alan Mackenzie
2022-02-06 18:39                                                                         ` Eli Zaretskii
2022-02-19 16:42                                                                           ` Alan Mackenzie
2022-02-19 17:02                                                                             ` Eli Zaretskii
2022-02-19 17:43                                                                               ` David Engster
2022-02-19 22:10                                                                                 ` Alan Mackenzie
2022-02-20  5:35                                                                                   ` David Engster
2022-02-20 19:45                                                                                     ` Alan Mackenzie
2022-02-20 20:37                                                                                   ` Stefan Monnier
2022-02-20 20:56                                                                                     ` Alan Mackenzie
2022-02-20 23:02                                                                                       ` Stefan Monnier
2022-02-21  0:22                                                                                         ` Óscar Fuentes
2022-02-21  3:31                                                                                           ` Eli Zaretskii
2022-02-21  4:04                                                                                             ` Óscar Fuentes
2022-02-21 12:15                                                                                               ` Eli Zaretskii
2022-02-21 14:55                                                                                                 ` Óscar Fuentes
2022-02-21  3:29                                                                                         ` Eli Zaretskii
2022-02-19 19:01                                                                               ` Stefan Monnier
2022-02-19 19:30                                                                                 ` Eli Zaretskii
2022-02-19 22:03                                                                               ` Alan Mackenzie
2022-02-25 22:29                                                                               ` Alan Mackenzie
2022-02-06 18:40                                                                         ` Eli Zaretskii
2022-02-06 19:03                                                                           ` Eli Zaretskii
2022-02-07 17:36                                                                             ` Andrea Corallo
2022-02-05  6:08                                                   ` Lars Ingebrigtsen
2022-02-05 11:42                                                     ` Alan Mackenzie
2022-02-05 21:31                                                       ` Lars Ingebrigtsen
2022-02-06  7:02                                                         ` Eli Zaretskii
2022-02-06 11:38                                                         ` Alan Mackenzie
2022-02-06 23:14                                                           ` Lars Ingebrigtsen
2022-01-25 21:15                                           ` Gregory Heytings
2022-01-25 21:30                                             ` Andrea Corallo
2022-01-26 18:43                                               ` Gregory Heytings
2022-01-26 21:04                                                 ` Andrea Corallo
     [not found]                                           ` <b0265c41-7ead-4913-667-d0e76a35b3ba@heytings.org>
2022-01-25 21:16                                             ` Gregory Heytings
2022-01-25 12:26                                         ` Eli Zaretskii
2022-01-26 18:41                                           ` Gregory Heytings
2022-01-26 18:59                                             ` Eli Zaretskii
2022-01-26 19:14                                               ` Stefan Monnier
2022-01-26 19:32                                                 ` Eli Zaretskii
2022-01-26 19:34                                             ` Alan Mackenzie [this message]
2022-01-22 18:35                 ` Alan Mackenzie
2022-01-22 18:45                   ` Gregory Heytings
2022-01-22 18:50                     ` Eli Zaretskii
2022-01-22 20:07                       ` Gregory Heytings
2022-01-23  5:32                         ` Eli Zaretskii
2022-01-23 21:44                           ` Gregory Heytings
2022-01-15 22:57 ` Stefan Monnier
2022-01-16  0:27 ` Brahimi Saifullah
2022-01-16 14:53   ` Alan Mackenzie
2022-01-16 16:45     ` Brahimi Saifullah
2022-01-22 11:41       ` Alan Mackenzie
2022-01-22 23:16         ` Brahimi Saifullah
2022-01-23 14:09           ` Alan Mackenzie
2022-01-17  9:38 ` Andrea Corallo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.gnu.org/software/emacs/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YfGiWZUTZGPh8LKH@ACM \
    --to=acm@muc.de \
    --cc=eliz@gnu.org \
    --cc=emacs-devel@gnu.org \
    --cc=gregory@heytings.org \
    --cc=larsi@gnus.org \
    --cc=mattiase@acm.org \
    --cc=monnier@iro.umontreal.ca \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).