bug#24143: 25.1; Windows performace of process-send-string

bug#24143: 25.1; Windows performace of process-send-string

[Bogdan Sirb]

[-- Attachment #1: Type: text/plain, Size: 6253 bytes --]

Hello,

I noticed a performace issue with emacs 25.1 (built with MinGW on
windows 10, but I used the one at
https://sourceforge.net/projects/emacsbinw64/).

I created a small c++ program to read from stdin:

std::cin >> fileSizeStr;
std::streamsize length = std::stoi(fileSizeStr);
char *content = (char *)malloc(length);
std::cin.read(content, length);

and sent data to it with the follogiwn emacs code :

;; added this in the middle of a 162 kb file and eval'd it:
(setq proc (start-process "Test" nil "./test.exe"))
(process-send-string
  proc
  (format "%d\n%s\n" (point-max)
  (buffer-substring (point-min) (point-max))))

I used elp-instrument-function for process-send-string and I got this
results (emacs 25.1 built by me vs emacs 24.5 found in msys2+mingw64
repo, compiled with the same flags):

* emacs 25.1: 0.510064 seconds/call
* emacs 24.5: 0.037496 seconds/call

And for a simple C program with fread(..., stdin) instead of std::cin

* emacs 25.1: 0.547136 seconds/call
* emacs 24.5: 0.031285 seconds/call


In GNU Emacs 25.1.1 (x86_64-w64-mingw32)
 of 2016-08-01 built on CSIRB-DEV
Windowing system distributor 'Microsoft Corp.', version 10.0.10586
Configured using:
 'configure --build=x86_64-w64-mingw32 --with-wide-int=yes
 --with-sound=yes --with-file-notification=yes --without-gpm
 --without-gconf --without-gsettings --without-selinux --with-modules
 'CFLAGS=-march=native -mtune=native -pipe -O3 -fomit-frame-pointer
 -funroll-loops' 'CPPFLAGS=-mtune=native -march=native -DNDEBUG -isystem
 /mingw64/include' 'LDFLAGS=-s -Wl,-s''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS NOTIFY ACL GNUTLS LIBXML2 ZLIB
TOOLKIT_SCROLL_BARS MODULES

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8

Major mode: C/l

Minor modes in effect:
  irony-mode: t
  delete-selection-mode: t
  global-auto-revert-mode: t
  recentf-mode: t
  savehist-mode: t
  save-place-mode: t
  show-paren-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Mark saved where search started
#<process Test>
nil
Quit
#<process Test>
nil
Mark set
Quit
Reusing Dired buffers is now ON

Load-path shadows:
d:/tools/prog/home/csirb/.emacs.d/elpa/seq-2.16/seq hides d:/emacs/share/emacs/25.1/lisp/emacs-lisp/seq

Features:
(shadow sort mail-extr emacsbug message rfc822 mml mml-sec epg mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mail-utils add-log dired+
image-dired image-file dired-x dired-aux dired elp colir color misearch
multi-isearch company-irony flycheck-irony irony-diagnostics
flycheck-tip error-tip notifications dbus xml popup flycheck rx subr-x
irony-cdb-libclang irony-cdb-json pp irony-cdb-clang-complete irony-cdb
irony-completion irony-snippet irony cc-mode cc-fonts cc-guess cc-menus
cc-cmds which-func imenu company-oddmuse company-keywords company-etags
company-gtags company-dabbrev-code company-dabbrev company-files
company-capf company-cmake company-xcode company-clang company-semantic
company-eclim company-template company-css company-nxml company-bbdb
company init init-keys init-prog init-python anaconda-mode pythonic f s
python tramp-sh json map url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util mailcap tramp
tramp-compat tramp-loaddefs trampver ucs-normalize shell init-lisp
init-c cc-styles cc-align cc-engine cc-vars cc-defs git-gutter
init-project projectile ibuf-ext ibuffer init-ido counsel jka-compr
esh-util swiper ivy ffap url-parse auth-source gnus-util mm-util
mail-prsvr password-cache url-vars smex ido init-dired init-editor
mouse-copy undo-tree diff wrap-region dash etags xref cl-seq project
eieio eieio-core cl-macs volatile-highlights tiny org org-macro
org-footnote org-pcomplete pcomplete org-list org-faces org-entities
noutline outline org-version ob-emacs-lisp ob ob-tangle ob-ref ob-lob
ob-table ob-exp org-src ob-keys ob-comint ob-core ob-eval org-compat
org-macs org-loaddefs format-spec find-func cal-menu calendar
cal-loaddefs help-fns which-key init-search iedit iedit-lib anzu
thingatpt grep compile comint ansi-color ring init-completion dabbrev
init-utils init-buffers edmacro kmacro init-advices advice init-sensible
delsel autorevert filenotify recentf tree-widget wid-edit savehist
saveplace init-appearance zenburn-theme paren use-package diminish
bind-key easy-mmode finder-inf info package epg-config seq byte-opt gv
bytecomp byte-compile cl-extra help-mode easymenu cconv cl-loaddefs
pcase cl-lib time-date mule-util tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel dos-w32 ls-lisp disp-table
w32-win w32-vars term/common-win tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote w32notify dbusbind w32 multi-tty
make-network-process emacs)

Memory information:
((conses 16 492637 123968)
 (symbols 56 45068 0)
 (miscs 48 157 220)
 (strings 32 107471 159315)
 (string-bytes 1 3362962)
 (vectors 16 64216)
 (vector-slots 8 990553 6398)
 (floats 8 572 95)
 (intervals 56 2108 72)
 (buffers 976 19)

[-- Attachment #2: Type: text/html, Size: 7133 bytes --]

bug#24143: 25.1; Windows performace of process-send-string

[Eli Zaretskii]

> Date: Wed, 3 Aug 2016 12:28:51 +0000 (UTC)
> From: Bogdan Sirb <bogdans91@yahoo.com>
> 
> I noticed a performace issue with emacs 25.1 (built with MinGW on
> windows 10, but I used the one at
> https://sourceforge.net/projects/emacsbinw64/).
> 
> I created a small c++ program to read from stdin:
> 
> std::cin >> fileSizeStr;
> std::streamsize length = std::stoi(fileSizeStr);
> char *content = (char *)malloc(length);
> std::cin.read(content, length);
> 
> and sent data to it with the follogiwn emacs code :
> 
> ;; added this in the middle of a 162 kb file and eval'd it:
> (setq proc (start-process "Test" nil "./test.exe"))
> (process-send-string
> proc
> (format "%d\n%s\n" (point-max)
> (buffer-substring (point-min) (point-max))))
> 
> I used elp-instrument-function for process-send-string and I got this
> results (emacs 25.1 built by me vs emacs 24.5 found in msys2+mingw64
> repo, compiled with the same flags):
> 
> * emacs 25.1: 0.510064 seconds/call
> * emacs 24.5: 0.037496 seconds/call
> 
> And for a simple C program with fread(..., stdin) instead of std::cin
> 
> * emacs 25.1: 0.547136 seconds/call
> * emacs 24.5: 0.031285 seconds/call
> 
> In GNU Emacs 25.1.1 (x86_64-w64-mingw32)
> of 2016-08-01 built on CSIRB-DEV
> Windowing system distributor 'Microsoft Corp.', version 10.0.10586
> Configured using:
> 'configure --build=x86_64-w64-mingw32 --with-wide-int=yes
> --with-sound=yes --with-file-notification=yes --without-gpm
> --without-gconf --without-gsettings --without-selinux --with-modules
> 'CFLAGS=-march=native -mtune=native -pipe -O3 -fomit-frame-pointer
> -funroll-loops' 'CPPFLAGS=-mtune=native -march=native -DNDEBUG -isystem
> /mingw64/include' 'LDFLAGS=-s -Wl,-s''

The above says how Emacs 25.1 was built, but you didn't give the same
info about Emacs 24.5.  Could you please provide the same info for
that build?

Also, does it help to set w32-pipe-buffer-size in Emacs 25.1 to some
non-default value, like 30000?

bug#24143: 25.1; Windows performace of process-send-string

[Bogdan Sirb]

[-- Attachment #1: Type: text/plain, Size: 2625 bytes --]

Emacs 24.5 was the one found in the repository of msys2 found
https://msys2.github.io/ I also tried the standard one found on emacs website
and the results were the same.

But doing a (setq w32-pipe-buffer-size (* 10 4096)) fixed the problem.

Now I have the following:

* emacs 24.5: 0.037496
* emacs 25.1: 0.040418

But, if I'm doing a (setq w32-pipe-buffer-size (point-max)) to send the whole
file at once once, I get a stunning 0.0010007 per call.

Is there any implications to doing this ?

Thanks!

    On Wednesday, August 3, 2016 7:38 PM, Eli Zaretskii <eliz@gnu.org> wrote:
 

 > Date: Wed, 3 Aug 2016 12:28:51 +0000 (UTC)
> From: Bogdan Sirb <bogdans91@yahoo.com>
> 
> I noticed a performace issue with emacs 25.1 (built with MinGW on
> windows 10, but I used the one at
> https://sourceforge.net/projects/emacsbinw64/).
> 
> I created a small c++ program to read from stdin:
> 
> std::cin >> fileSizeStr;
> std::streamsize length = std::stoi(fileSizeStr);
> char *content = (char *)malloc(length);
> std::cin.read(content, length);
> 
> and sent data to it with the follogiwn emacs code :
> 
> ;; added this in the middle of a 162 kb file and eval'd it:
> (setq proc (start-process "Test" nil "./test.exe"))
> (process-send-string
> proc
> (format "%d\n%s\n" (point-max)
> (buffer-substring (point-min) (point-max))))
> 
> I used elp-instrument-function for process-send-string and I got this
> results (emacs 25.1 built by me vs emacs 24.5 found in msys2+mingw64
> repo, compiled with the same flags):
> 
> * emacs 25.1: 0.510064 seconds/call
> * emacs 24.5: 0.037496 seconds/call
> 
> And for a simple C program with fread(..., stdin) instead of std::cin
> 
> * emacs 25.1: 0.547136 seconds/call
> * emacs 24.5: 0.031285 seconds/call
> 
> In GNU Emacs 25.1.1 (x86_64-w64-mingw32)
> of 2016-08-01 built on CSIRB-DEV
> Windowing system distributor 'Microsoft Corp.', version 10.0.10586
> Configured using:
> 'configure --build=x86_64-w64-mingw32 --with-wide-int=yes
> --with-sound=yes --with-file-notification=yes --without-gpm
> --without-gconf --without-gsettings --without-selinux --with-modules
> 'CFLAGS=-march=native -mtune=native -pipe -O3 -fomit-frame-pointer
> -funroll-loops' 'CPPFLAGS=-mtune=native -march=native -DNDEBUG -isystem
> /mingw64/include' 'LDFLAGS=-s -Wl,-s''

The above says how Emacs 25.1 was built, but you didn't give the same
info about Emacs 24.5.  Could you please provide the same info for
that build?

Also, does it help to set w32-pipe-buffer-size in Emacs 25.1 to some
non-default value, like 30000?


  

[-- Attachment #2: Type: text/html, Size: 4959 bytes --]

bug#24143: 25.1; Windows performace of process-send-string

[Eli Zaretskii]

tags 24143 + notabug
thanks

> Date: Thu, 4 Aug 2016 11:40:20 +0000 (UTC)
> From: Bogdan Sirb <bogdans91@yahoo.com>
> Cc: "24143@debbugs.gnu.org" <24143@debbugs.gnu.org>
> 
> Emacs 24.5 was the one found in the repository of msys2 found
> https://msys2.github.io/ I also tried the standard one found on emacs website
> and the results were the same.

That doesn't tell how it was compiled.  My idea was that when
comparing performance of two versions, one should make sure both
builds used similar compilation switches, otherwise you could see all
kinds of strange disparities.

> But doing a (setq w32-pipe-buffer-size (* 10 4096)) fixed the problem.

Great, so I'm closing this bug.

> Now I have the following:
> 
> * emacs 24.5: 0.037496
> * emacs 25.1: 0.040418
> 
> But, if I'm doing a (setq w32-pipe-buffer-size (point-max)) to send the whole
> file at once once, I get a stunning 0.0010007 per call.
> 
> Is there any implications to doing this ?

It depends on what program is on the other end of the pipe.  Some
programs don't like large writes down the pipe.  See bug#22344, where
such large writes caused a catastrophic failure.  Emacs prefers to err
on the safe side by default, and leave any fine-tuning of the pipe
size to applications that need it.

Thanks.