How to make 'load' fast in window?

How to make 'load' fast in window?

[tumashu]

[-- Attachment #1: Type: text/plain, Size: 182 bytes --]

I use "load" function to load a elisp file which include a 40M hashtable, In linux, it only need 1s, while in window, it need 15s, which is too slowly,
How to speed it up?

Thanks!
 

[-- Attachment #2: Type: text/html, Size: 295 bytes --]

Re: How to make 'load' fast in window?

[Jiege Chen]

tumashu <tumashu@163.com> wrote:
> I use "load" function to load a elisp file which include a 40M hashtable,
> In linux, it only need 1s, while in window, it need 15s, which is too slowly,
How to speed it up?

Thanks!
 

Re: How to make 'load' fast in window?

[Eli Zaretskii]

> From: Jiege Chen <jiegec@qq.com>
> Date: Mon, 13 Jun 2016 05:44:23 +0000 (UTC)
> 
> tumashu <tumashu@163.com> wrote:
> > I use "load" function to load a elisp file which include a 40M hashtable,
> > In linux, it only need 1s, while in window, it need 15s, which is too slowly,
> How to speed it up?

I'd suggest to start by showing some details, like the file in
question and the details of each of the 2 machines where you tried
this.  Also, is the Emacs version the same in both cases, and was it
compiled using the same compiler options?

Re: How to make 'load' fast in window?

[Fabrice Popineau]

Jiege Chen <jiegec <at> qq.com> writes:

> 
> tumashu <tumashu <at> 163.com> wrote:
> > I use "load" function to load a elisp file which include a 40M hashtable,
> > In linux, it only need 1s, while in window, it need 15s, which is too slowly,
> How to speed it up?

On what machine?
Can you share your file?

Regards,

Fabrice

Re:Re: How to make 'load' fast in window?

[tumashu]

[-- Attachment #1: Type: text/plain, Size: 1503 bytes --]

My emacs is GNU Emacs 24.5.1 (i686-pc-mingw32)

I want to convert the content of pyim-bigdict.pyim (A 40M text file) (https://github.com/tumashu/chinese-pyim-bigdict) to a hashtable,
so I use chinese-async to prin1 the hashtable to a el file, which is like:

#+BEGIN_EXAMPLE
(setq pyim-dict-cache #s(hash-table size 1000000 test equal rehash-size 1.5 rehash-threshold 0.8 data ("a" ("哎") "a-a" ("呵呵") "a-h" ("呵呵") ...)))
#+END_EXAMPLE



Then I use load function to load this file, pyim-dict-cache hashtable can be generated too, In my linux machine,  load this file only need 1s,
but in windows machine, this need 15-20s, very slow.

By the way, now I use  eval + read  instead, but seem to slow too.....


download pyim-bigdict-cf210fa066f2fa18c1606eeddfffcf84.el from https://github.com/tumashu/tumashu-temp-files
and eval:


#+BEGIN_EXAMPLE
        (setq pyim-dict-cache
              (with-temp-buffer
                (insert-file-contents "/path/to/pyim-bigdict-cf210fa066f2fa18c1606eeddfffcf84.el")
                (eval (read (current-buffer)))))
#+END_EXAMPLE




At 2016-06-13 22:09:20, "Fabrice Popineau" <fabrice.popineau@gmail.com> wrote:
>Jiege Chen <jiegec <at> qq.com> writes:
>
>> 
>> tumashu <tumashu <at> 163.com> wrote:
>> > I use "load" function to load a elisp file which include a 40M hashtable,
>> > In linux, it only need 1s, while in window, it need 15s, which is too slowly,
>> How to speed it up?
>
>On what machine?
>Can you share your file?
>
>Regards,
>
>Fabrice
>
>

[-- Attachment #2: Type: text/html, Size: 3069 bytes --]

Re: How to make 'load' fast in window?

[Fabrice Popineau]

tumashu <tumashu <at> 163.com> writes:

> download pyim-bigdict-cf210fa066f2fa18c1606eeddfffcf84.el from  
https://github.com/tumashu/tumashu-temp-filesand eval: 
> 
> #+BEGIN_EXAMPLE        (setq pyim-dict-cache              (with-temp-
buffer                (insert-file-contents "/path/to/pyim-bigdict-
cf210fa066f2fa18c1606eeddfffcf84.el")                (eval (read (current-
buffer)))))#+END_EXAMPLE
> 

So, starting from `emacs -Q' on an i7-4702HQ with emacs-25.0.94, windows 10, I 
get a user time of about 4s and the following profiler report:

- command-execute                                                 441  79%
 - call-interactively                                             441  79%
  - funcall-interactively                                         436  78%
   - eval-last-sexp                                               419  75%
    - elisp--eval-last-sexp                                       419  75%
     - eval                                                       213  38%
      - setq                                                      213  38%
       - let                                                      213  38%
        - save-current-buffer                                     213  38%
         - unwind-protect                                         213  38%
          - progn                                                 213  38%
             eval                                                 158  28%
           - insert-file-contents                                  11   1%
            - set-auto-coding                                      11   1%
             - find-auto-coding                                    11   1%
                sgml-html-meta-auto-coding-function                  7   1%
       elisp--eval-last-sexp-print-value                          206  37%
   - execute-extended-command                                      17   3%
    - execute-extended-command--shorter                            16   2%
     - completion-try-completion                                   16   2%
      - completion--nth-completion                                 16   2%
       - completion--some                                          16   2%
        - #<compiled 0x1002489fd>                                  16   2%
         - completion-pcm-try-completion                            9   1%
          - completion-pcm--find-all-completions                    9   1%
             completion-pcm--all-completions                        9   1%
           completion-basic-try-completion                          7   1%
    - sit-for                                                       1   0%
       redisplay                                                    1   0%
  - byte-code                                                       5   0%
   - read-extended-command                                          5   0%
    - completing-read                                               5   0%
     - completing-read-default                                      5   0%
        read-from-minibuffer                                        2   0%
- ...                                                             115  20%
   Automatic GC                                                   115  20%

I'm not sure what could explain such a huge difference with Linux. I don't have 
an easy way right now to run your test on the very same machine with Linux, but 
if someone can do it on the same cu, it would be very welcome.

I see 2 potential explanations:
- memory: allocation, or maybe swap (?, depending on your configuration)
- some transcoding of characters (I don't know much about what emacs does in 
this area and how it differs between Linux and Windows).

You should try emacs 25.1 beta. The memory allocator has been changed and is 
using native win32 memory allocation memory mapped files for buffers. It may be 
more efficient.

Regards,

Fabrice

Re: How to make 'load' fast in window?

[Eli Zaretskii]

> From: Fabrice Popineau <fabrice.popineau@gmail.com>
> Date: Tue, 14 Jun 2016 07:36:19 +0000 (UTC)
> 
> > download pyim-bigdict-cf210fa066f2fa18c1606eeddfffcf84.el from  
> https://github.com/tumashu/tumashu-temp-filesand eval: 
> > 
> > #+BEGIN_EXAMPLE        (setq pyim-dict-cache              (with-temp-
> buffer                (insert-file-contents "/path/to/pyim-bigdict-
> cf210fa066f2fa18c1606eeddfffcf84.el")                (eval (read (current-
> buffer)))))#+END_EXAMPLE
> > 
> 
> So, starting from `emacs -Q' on an i7-4702HQ with emacs-25.0.94, windows 10, I 
> get a user time of about 4s and the following profiler report:

On an i7 system I get 0.6s with Emacs 25.0.95 and 2s with 24.5.  Both
are optimized (-O2) builds.  This is comparable with the GNU/Linux
results, and is what I'd expect.  Unless yours are not -O2, I don't
understand how you get so much slower times.

My guess is that the OP uses an unoptimized build of Emacs on Windows,
or the Windows machine is much slower than the GNU/Linux one (or
both).  There's nothing in this code or the profile you present that
could be significantly slower on Windows; the only potential suspect,
file I/O, takes a negligible fraction of time.

Re: How to make 'load' fast in window?

[Fabrice Popineau]

Eli Zaretskii <eliz <at> gnu.org> writes:

> 
> > From: Fabrice Popineau <fabrice.popineau <at> gmail.com>
> > Date: Tue, 14 Jun 2016 07:36:19 +0000 (UTC)
> > 
> > > download pyim-bigdict-cf210fa066f2fa18c1606eeddfffcf84.el from  
> > https://github.com/tumashu/tumashu-temp-filesand eval: 
> > > 
> > > #+BEGIN_EXAMPLE        (setq pyim-dict-cache              (with-temp-
> > buffer                (insert-file-contents "/path/to/pyim-bigdict-
> > cf210fa066f2fa18c1606eeddfffcf84.el")                (eval (read (current-
> > buffer)))))#+END_EXAMPLE
> > > 
> > 
> > So, starting from `emacs -Q' on an i7-4702HQ with emacs-25.0.94, windows 10, 
I 
> > get a user time of about 4s and the following profiler report:
> 
> On an i7 system I get 0.6s with Emacs 25.0.95 and 2s with 24.5.  Both
> are optimized (-O2) builds.  This is comparable with the GNU/Linux
> results, and is what I'd expect.  Unless yours are not -O2, I don't
> understand how you get so much slower times.

I was running in economy mode power plan :-)

$ time ../../build-emacs-25/src/emacs -Q -batch -l test.el

real    0m0.737s
user    0m0.000s
sys     0m0.015s

So yes, he must be using an unoptimized build.

Fabrice

Re:Re: How to make 'load' fast in window?

[tumashu]

[-- Attachment #1: Type: text/plain, Size: 1590 bytes --]


emacs 24.5 is downloaded from ftp://ftp.gnu.org/gnu/emacs/windows/ , I don't know whether it is enable -02,


May be my machine is too slow, Thinkpad s3-s440

1. i5-4210U  http://ark.intel.com/products/81016/Intel-Core-i5-4210U-Processor-3M-Cache-up-to-2_70-GHz
2. 4G RAM
 







At 2016-06-15 04:39:34, "Fabrice Popineau" <fabrice.popineau@gmail.com> wrote:
>Eli Zaretskii <eliz <at> gnu.org> writes:
>
>> 
>> > From: Fabrice Popineau <fabrice.popineau <at> gmail.com>
>> > Date: Tue, 14 Jun 2016 07:36:19 +0000 (UTC)
>> > 
>> > > download pyim-bigdict-cf210fa066f2fa18c1606eeddfffcf84.el from  
>> > https://github.com/tumashu/tumashu-temp-filesand eval: 
>> > > 
>> > > #+BEGIN_EXAMPLE        (setq pyim-dict-cache              (with-temp-
>> > buffer                (insert-file-contents "/path/to/pyim-bigdict-
>> > cf210fa066f2fa18c1606eeddfffcf84.el")                (eval (read (current-
>> > buffer)))))#+END_EXAMPLE
>> > > 
>> > 
>> > So, starting from `emacs -Q' on an i7-4702HQ with emacs-25.0.94, windows 10, 
>I 
>> > get a user time of about 4s and the following profiler report:
>> 
>> On an i7 system I get 0.6s with Emacs 25.0.95 and 2s with 24.5.  Both
>> are optimized (-O2) builds.  This is comparable with the GNU/Linux
>> results, and is what I'd expect.  Unless yours are not -O2, I don't
>> understand how you get so much slower times.
>
>I was running in economy mode power plan :-)
>
>$ time ../../build-emacs-25/src/emacs -Q -batch -l test.el
>
>real    0m0.737s
>user    0m0.000s
>sys     0m0.015s
>
>So yes, he must be using an unoptimized build.
>
>Fabrice

[-- Attachment #2: Type: text/html, Size: 2631 bytes --]

Re: How to make 'load' fast in window?

[Fabrice Popineau]

tumashu <tumashu <at> 163.com> writes:

> 
> 
> emacs 24.5 is downloaded from ftp://ftp.gnu.org/gnu/emacs/windows/,
> I don't know whether it is enable -O2,

I don't know either.

> May be my machine is too slow, Thinkpad s3-s4401. i5-4210U

No, because you shouldn't see a 40x runtime difference with Linux 
on the same machine.

Fabrice

Re: How to make 'load' fast in window?

[Eli Zaretskii]

> Date: Wed, 15 Jun 2016 10:47:13 +0800 (CST)
> From: tumashu  <tumashu@163.com>
> Cc: "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> 
> emacs 24.5 is downloaded from ftp://ftp.gnu.org/gnu/emacs/windows/ , I don't know whether it is enable -02, 

What is the value of system-configuration-options in that build?  And
what is its value in the GNU/Linux build with which you are comparing?

> May be my machine is too slow, Thinkpad s3-s440
> 
> 1. i5-4210U http://ark.intel.com/products/81016/Intel-Core-i5-4210U-Processor-3M-Cache-up-to-2_70-GHz
> 2. 4G RAM 

Unlikely.  On an i3, I get 0.9s with v25.0.95 and 3.9s with 24.5, so
i5 should be faster than that in a build with similar optimization
options.

Re: How to make 'load' fast in window?

[Fabrice Popineau]

Eli Zaretskii <eliz <at> gnu.org> writes:

> 
> > Date: Wed, 15 Jun 2016 10:47:13 +0800 (CST)
> > From: tumashu  <tumashu <at> 163.com>
> > Cc: "emacs-devel <at> gnu.org" <emacs-devel <at> gnu.org>
> > 
> > emacs 24.5 is downloaded from ftp://ftp.gnu.org/gnu/emacs/windows/ , I don't 
know whether it is enable
> -02, 
> 
> What is the value of system-configuration-options in that build? 

It reports that:

In GNU Emacs 24.5.1 (i686-pc-mingw32)
 of 2015-04-11 on LEG570
Windowing system distributor `Microsoft Corp.', version 6.3.9600
Configured using:
 `configure --prefix=/c/usr --host=i686-pc-mingw32'

This is all I could find.

Fabrice

Re: How to make 'load' fast in window?

[Fabrice Popineau]

[-- Attachment #1: Type: text/plain, Size: 1083 bytes --]

2016-06-15 16:46 GMT+02:00 Eli Zaretskii <eliz@gnu.org>:

> > Date: Wed, 15 Jun 2016 10:47:13 +0800 (CST)
> > From: tumashu  <tumashu@163.com>
> > Cc: "emacs-devel@gnu.org" <emacs-devel@gnu.org>
> >
> > emacs 24.5 is downloaded from ftp://ftp.gnu.org/gnu/emacs/windows/ , I
> don't know whether it is enable -02,
>
> What is the value of system-configuration-options in that build?  And
> what is its value in the GNU/Linux build with which you are comparing?
>
> > May be my machine is too slow, Thinkpad s3-s440
> >
> > 1. i5-4210U
> http://ark.intel.com/products/81016/Intel-Core-i5-4210U-Processor-3M-Cache-up-to-2_70-GHz
> > 2. 4G RAM
>
> Unlikely.  On an i3, I get 0.9s with v25.0.95 and 3.9s with 24.5, so
> i5 should be faster than that in a build with similar optimization
> options.
>

I used the emacs-24.5 build from  ftp://ftp.gnu.org/gnu/emacs/windows/

and I get :

$ time bin/emacs -Q -batch -l
d:/Source/emacs/elisp/tumashu-temp-files/test.el
real    0m3.171s
user    0m0.000s
sys     0m0.015s

So there is definitely something wrong with the 40s reported.

Fabrice

[-- Attachment #2: Type: text/html, Size: 2301 bytes --]

Re: How to make 'load' fast in window?

[Eli Zaretskii]

> From: Fabrice Popineau <fabrice.popineau@gmail.com>
> Date: Wed, 15 Jun 2016 15:15:45 +0000 (UTC)
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> > 
> > > Date: Wed, 15 Jun 2016 10:47:13 +0800 (CST)
> > > From: tumashu  <tumashu <at> 163.com>
> > > Cc: "emacs-devel <at> gnu.org" <emacs-devel <at> gnu.org>
> > > 
> > > emacs 24.5 is downloaded from ftp://ftp.gnu.org/gnu/emacs/windows/ , I don't 
> know whether it is enable
> > -02, 
> > 
> > What is the value of system-configuration-options in that build? 
> 
> It reports that:
> 
> In GNU Emacs 24.5.1 (i686-pc-mingw32)
>  of 2015-04-11 on LEG570
> Windowing system distributor `Microsoft Corp.', version 6.3.9600
> Configured using:
>  `configure --prefix=/c/usr --host=i686-pc-mingw32'
> 
> This is all I could find.

I meant this:

  M-: system-configuration-options RET

Re: How to make 'load' fast in window?

[Fabrice Popineau]

[-- Attachment #1: Type: text/plain, Size: 1174 bytes --]

2016-06-15 17:40 GMT+02:00 Eli Zaretskii <eliz@gnu.org>:

> > From: Fabrice Popineau <fabrice.popineau@gmail.com>
> > Date: Wed, 15 Jun 2016 15:15:45 +0000 (UTC)
> >
> > Eli Zaretskii <eliz <at> gnu.org> writes:
> >
> > >
> > > > Date: Wed, 15 Jun 2016 10:47:13 +0800 (CST)
> > > > From: tumashu  <tumashu <at> 163.com>
> > > > Cc: "emacs-devel <at> gnu.org" <emacs-devel <at> gnu.org>
> > > >
> > > > emacs 24.5 is downloaded from ftp://ftp.gnu.org/gnu/emacs/windows/
> , I don't
> > know whether it is enable
> > > -02,
> > >
> > > What is the value of system-configuration-options in that build?
> >
> > It reports that:
> >
> > In GNU Emacs 24.5.1 (i686-pc-mingw32)
> >  of 2015-04-11 on LEG570
> > Windowing system distributor `Microsoft Corp.', version 6.3.9600
> > Configured using:
> >  `configure --prefix=/c/usr --host=i686-pc-mingw32'
> >
> > This is all I could find.
>
> I meant this:
>
>   M-: system-configuration-options RET
>

Same result:

system-configuration-options is a variable defined in `C source code'.
Its value is "--prefix=/c/usr --host=i686-pc-mingw32"

Documentation:
String containing the configuration options Emacs was built with.

Fabrice

[-- Attachment #2: Type: text/html, Size: 2358 bytes --]

Re: How to make 'load' fast in window?

[Eli Zaretskii]

> From: Fabrice Popineau <fabrice.popineau@gmail.com>
> Date: Wed, 15 Jun 2016 17:49:31 +0200
> Cc: Emacs developers <emacs-devel@gnu.org>
> 
>  M-: system-configuration-options RET
> 
> Same result:
> 
> system-configuration-options is a variable defined in `C source code'.
> Its value is "--prefix=/c/usr --host=i686-pc-mingw32"

Which means the build used the default compiler switches.

Re: How to make 'load' fast in window?

[Feng Shu]

Fabrice Popineau <fabrice.popineau@gmail.com> writes:

> 2016-06-15 16:46 GMT+02:00 Eli Zaretskii <eliz@gnu.org>:
>
>  > Date: Wed, 15 Jun 2016 10:47:13 +0800 (CST)
>  > From: tumashu  <tumashu@163.com>
>  > Cc: "emacs-devel@gnu.org" <emacs-devel@gnu.org>
>  >
>  > emacs 24.5 is downloaded from ftp://ftp.gnu.org/gnu/emacs/windows/ , I don't know whether it is enable
>  -02,
>
>  What is the value of system-configuration-options in that build?  And
>  what is its value in the GNU/Linux build with which you are comparing?
>
>  > May be my machine is too slow, Thinkpad s3-s440
>  >
>  > 1. i5-4210U
>  http://ark.intel.com/products/81016/Intel-Core-i5-4210U-Processor-3M-Cache-up-to-2_70-GHz
>  > 2. 4G RAM
>
>  Unlikely.  On an i3, I get 0.9s with v25.0.95 and 3.9s with 24.5, so
>  i5 should be faster than that in a build with similar optimization
>  options.
>
> I used the emacs-24.5 build from  ftp://ftp.gnu.org/gnu/emacs/windows/
>
> and I get :
>
> $ time bin/emacs -Q -batch -l d:/Source/emacs/elisp/tumashu-temp-files/test.el  
> real    0m3.171s
> user    0m0.000s
> sys     0m0.015s
>
> So there is definitely something wrong with the 40s reported.

I just say 15s, not 40s :-), and I use emacs-25.0.92-i686-w64-mingw32.zip
from http://alpha.gnu.org/gnu/emacs/windows/ , and it seem fast as in linux.


#+BEGIN_EXAMPLE
I use "load" function to load a elisp file which include a 40M
hashtable, In linux, it only need 1s, while in window, it
need 15s, which is too slowly, 
How to speed it up? 
#+END_EXAMPLE

>
> Fabrice

-- 

Re: How to make 'load' fast in window?

[Fabrice Popineau]

Fabrice Popineau <fabrice.popineau <at> gmail.com> writes:

> No, because you shouldn't see a 40x runtime difference with Linux 
> on the same machine.

Linux -> GNU/Linux.

(RMS asked kindly to fix that mistake)

Fabrice