* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode @ 2013-09-06 20:59 Dmitry Gutov 2013-09-06 23:44 ` Xue Fuqiao ` (2 more replies) 0 siblings, 3 replies; 35+ messages in thread From: Dmitry Gutov @ 2013-09-06 20:59 UTC (permalink / raw To: 15294 If I append -*- lexical-binding: t -*- to the first line of js2-mode.el and eval the buffer, parsing a file takes several (3 to 4) times longer. To measure: 0. Install js2-mode, e.g. from GNU ELPA. 1. Save http://mootools.net/download/get/mootools-core-1.4.5-full-nocompat.js to some local directory, open it in Emacs. 2. If it opens in some other mode than js2-mode, M-x js2-mode. 3. Don't touch the keys, wait until the file is parsed (the parsing gets interrupted and rescheduled on input). When redisplay works again, we can re-parse the file again and measure the time it takes. 4. eval-expression (js2-time (js2-reparse t)), wait until the message area displays the amount of seconds the process took. To measure the performance in interpreted mode, open js2-mode.el, M-x eval-buffer and go through the above scenario (1-4). To measure the performance in compiled mode, M-x byte-compile-file, then (load "js2-mode.elc"), then go through the scenario. My measurements: lexical-binding | no | yes | Interpreted | 1 sec | 4.26 sec | Compiled | 0.63 sec | 1.76 sec | -- In GNU Emacs 24.3.50.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.6.4) of 2013-09-06 on axl Bzr revision: 114159 dmantipov@yandex.ru-20130906164012-6j0s6otwkypj9s1z Windowing system distributor `The X.Org Foundation', version 11.0.11303000 System Description: Ubuntu 13.04 ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-06 20:59 bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode Dmitry Gutov @ 2013-09-06 23:44 ` Xue Fuqiao 2013-09-07 3:15 ` Stefan Monnier 2013-09-08 22:32 ` Stefan Monnier 2013-09-10 2:04 ` Stefan Monnier 2 siblings, 1 reply; 35+ messages in thread From: Xue Fuqiao @ 2013-09-06 23:44 UTC (permalink / raw To: 15294-done Fixed in revision 114156. ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-06 23:44 ` Xue Fuqiao @ 2013-09-07 3:15 ` Stefan Monnier 0 siblings, 0 replies; 35+ messages in thread From: Stefan Monnier @ 2013-09-07 3:15 UTC (permalink / raw To: 15294; +Cc: xfq.free, dgutov reopen 15294 thanks > Fixed in revision 114156. No, 114156 fixes another problem (which affected both lexical and dynamic binding code). Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-06 20:59 bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode Dmitry Gutov 2013-09-06 23:44 ` Xue Fuqiao @ 2013-09-08 22:32 ` Stefan Monnier 2013-09-10 2:04 ` Stefan Monnier 2 siblings, 0 replies; 35+ messages in thread From: Stefan Monnier @ 2013-09-08 22:32 UTC (permalink / raw To: Dmitry Gutov; +Cc: 15294 A first likely candidate is the bunch of `catch/throw' in js2-get-token, since catch, unwind-protect and condition-case are all implemented in the bytecode interpreter in ways which don't work well for lexical scoping. Stefan "Just reporting on his little progress" ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-06 20:59 bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode Dmitry Gutov 2013-09-06 23:44 ` Xue Fuqiao 2013-09-08 22:32 ` Stefan Monnier @ 2013-09-10 2:04 ` Stefan Monnier 2013-09-13 3:40 ` Stefan Monnier ` (2 more replies) 2 siblings, 3 replies; 35+ messages in thread From: Stefan Monnier @ 2013-09-10 2:04 UTC (permalink / raw To: Dmitry Gutov; +Cc: 15294 > If I append -*- lexical-binding: t -*- to the first line of js2-mode.el > and eval the buffer, parsing a file takes several (3 to 4) times longer. [ I'm disregarding the interpreted time for now, since the issues are likely to be different. ] In the compiled case, loading js2-mode-lexical.elc and then redefining `js2-get-token' to the value it has in js2-mode-dynamic.elc brings me back the same speed as with js2-mode-dynamic.elc. IOW the slowdown is in js2-get-token. If it indeed comes from the handling of catch/throw, then there are two ways to fix it: - provide a new byte-code for catch which does not require wrapping the body in a closure. This should not be terribly difficult and would benefit all existing code, but it does require changes to the way catch is implemented. - improve the byte-compiler so it can compile away cl-block/cl-return. This should not require new byte-codes, but requires delving "deepish" into the byte-compiler. Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-10 2:04 ` Stefan Monnier @ 2013-09-13 3:40 ` Stefan Monnier 2013-09-13 3:59 ` Drew Adams 2013-09-14 4:20 ` bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode Dmitry Gutov 2013-09-15 0:24 ` Dmitry Gutov 2013-09-18 23:48 ` Stefan Monnier 2 siblings, 2 replies; 35+ messages in thread From: Stefan Monnier @ 2013-09-13 3:40 UTC (permalink / raw To: Dmitry Gutov; +Cc: 15294 > In the compiled case, loading js2-mode-lexical.elc and then redefining > `js2-get-token' to the value it has in js2-mode-dynamic.elc brings me > back the same speed as with js2-mode-dynamic.elc. > IOW the slowdown is in js2-get-token. It seems the slowdown is indeed linked to the way `catch' is handled (indeed, this non-idiomatic ELisp code ends up byte-compiled in a really poor way). The trivial patch below brings the time down from 5s to 2.6s (as compared to 2.1s for the dynamic-binding version). Stefan diff --git a/packages/js2-mode/js2-mode.el b/packages/js2-mode/js2-mode.el index 3568f18..1d76469 100644 --- a/packages/js2-mode/js2-mode.el +++ b/packages/js2-mode/js2-mode.el @@ -5310,10 +5311,10 @@ corresponding number. Otherwise return -1." (defun js2-get-token () "Return next JavaScript token, an int such as js2-RETURN." - (let (c c1 identifier-start is-unicode-escape-start - contains-escape escape-val str result base - is-integer quote-char val look-for-slash continue) - (catch 'return + (catch 'return + (let (c c1 identifier-start is-unicode-escape-start + contains-escape escape-val str result base + is-integer quote-char val look-for-slash continue) (while t ;; Eat whitespace, possibly sensitive to newlines. (setq continue t) ^ permalink raw reply related [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-13 3:40 ` Stefan Monnier @ 2013-09-13 3:59 ` Drew Adams 2013-09-13 4:37 ` Stefan Monnier 2013-09-14 4:20 ` bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode Dmitry Gutov 1 sibling, 1 reply; 35+ messages in thread From: Drew Adams @ 2013-09-13 3:59 UTC (permalink / raw To: Stefan Monnier, Dmitry Gutov; +Cc: 15294 > It seems the slowdown is indeed linked to the way `catch' is handled > (indeed, this non-idiomatic ELisp code ends up byte-compiled in a really > poor way). The trivial patch below brings the time down from 5s to 2.6s > (as compared to 2.1s for the dynamic-binding version). > > - (let (c c1 identifier-start is-unicode-escape-start > - contains-escape escape-val str result base > - is-integer quote-char val look-for-slash continue) > - (catch 'return > + (catch 'return > + (let (c c1 identifier-start is-unicode-escape-start > + contains-escape escape-val str result base > + is-integer quote-char val look-for-slash continue) Not really following this thread, but this caught my eye. Is this something that should be mentioned as a guideline in the manual? Or is the byte compiler likely to be improved in this regard, so users need not be aware of it and take measures manually? ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-13 3:59 ` Drew Adams @ 2013-09-13 4:37 ` Stefan Monnier 2013-09-13 5:45 ` Drew Adams 2013-09-14 0:09 ` Lexical let and setq Michael Welsh Duggan 0 siblings, 2 replies; 35+ messages in thread From: Stefan Monnier @ 2013-09-13 4:37 UTC (permalink / raw To: Drew Adams; +Cc: 15294, Dmitry Gutov > Or is the byte compiler likely to be improved in this regard, so users > need not be aware of it and take measures manually? As indicated in the FIXMEs I added in src/bytecode.c at the time, the current byte codes for unwind-protect, condition-case, and catch are very inefficient for lexical-binding code. I do hope to fix those issues by introducing other byte-codes which will let us generate significantly more efficient code for those constructs, but in 24.1, the priority was to get lexical-binding to work correctly, performance being a secondary concern (tho for most idiomatic Elisp code, the performance tends to be competitive). What people should know is that (let (x y z) ...(setq x ...) ...(setq z ...) ...(setq y ...) is often a bad idea in Elisp, and even more so in lexical-binding code (in some cases, if a variable is immutable it can be handled significantly more efficiently, so the mere existence of a single `setq' on a variable can sometimes slow other chunks of code: in many cases `let' is cheaper than `setq'). Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-13 4:37 ` Stefan Monnier @ 2013-09-13 5:45 ` Drew Adams 2013-09-13 13:01 ` Stefan Monnier 2013-09-14 0:09 ` Lexical let and setq Michael Welsh Duggan 1 sibling, 1 reply; 35+ messages in thread From: Drew Adams @ 2013-09-13 5:45 UTC (permalink / raw To: Stefan Monnier; +Cc: 15294, Dmitry Gutov > > Or is the byte compiler likely to be improved in this regard, so users > > need not be aware of it and take measures manually? > > As indicated in the FIXMEs I added in src/bytecode.c at the time, the > current byte codes for unwind-protect, condition-case, and catch are > very inefficient for lexical-binding code. > > I do hope to fix those issues by introducing other byte-codes which will > let us generate significantly more efficient code for those constructs, > but in 24.1, the priority was to get lexical-binding to work correctly, > performance being a secondary concern (tho for most idiomatic Elisp > code, the performance tends to be competitive). > > What people should know is that > > (let (x y z) > ...(setq x ...) > ...(setq z ...) > ...(setq y ...) > > is often a bad idea in Elisp, and even more so in lexical-binding code > (in some cases, if a variable is immutable it can be handled > significantly more efficiently, so the mere existence of a single `setq' > on a variable can sometimes slow other chunks of code: in many cases > `let' is cheaper than `setq'). Thank you for both answers. Will the second apply even after making the byte-compiler improvements described for the first? (I'm guessing yes.) If so, you might want to mention this guideline in the manual. ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-13 5:45 ` Drew Adams @ 2013-09-13 13:01 ` Stefan Monnier 0 siblings, 0 replies; 35+ messages in thread From: Stefan Monnier @ 2013-09-13 13:01 UTC (permalink / raw To: Drew Adams; +Cc: 15294, Dmitry Gutov >> What people should know is that >> >> (let (x y z) >> ...(setq x ...) >> ...(setq z ...) >> ...(setq y ...) >> >> is often a bad idea in Elisp, and even more so in lexical-binding code >> (in some cases, if a variable is immutable it can be handled >> significantly more efficiently, so the mere existence of a single `setq' >> on a variable can sometimes slow other chunks of code: in many cases >> `let' is cheaper than `setq'). > Thank you for both answers. Will the second apply even after making the > byte-compiler improvements described for the first? (I'm guessing yes.) > If so, you might want to mention this guideline in the manual. Yes. Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* Lexical let and setq 2013-09-13 4:37 ` Stefan Monnier 2013-09-13 5:45 ` Drew Adams @ 2013-09-14 0:09 ` Michael Welsh Duggan 2013-09-14 3:46 ` Stefan Monnier 2013-09-14 21:47 ` Richard Stallman 1 sibling, 2 replies; 35+ messages in thread From: Michael Welsh Duggan @ 2013-09-14 0:09 UTC (permalink / raw To: Stefan Monnier; +Cc: emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: [...] > I do hope to fix those issues by introducing other byte-codes which will > let us generate significantly more efficient code for those constructs, > but in 24.1, the priority was to get lexical-binding to work correctly, > performance being a secondary concern (tho for most idiomatic Elisp > code, the performance tends to be competitive). > > What people should know is that > > (let (x y z) > ...(setq x ...) > ...(setq z ...) > ...(setq y ...) > > is often a bad idea in Elisp, and even more so in lexical-binding code > (in some cases, if a variable is immutable it can be handled > significantly more efficiently, so the mere existence of a single `setq' > on a variable can sometimes slow other chunks of code: in many cases > `let' is cheaper than `setq'). The primary reason I have seen the (let (foo) (setq foo ...)) idiom is in looping code. The way I would normally try to avoid this idiom in most FP languages would be to use recursion (specifically tail recursion, if possible). I know some work was done on implementing efficient tail-recursion in the byte compiler. Has any of that made it onto the trunk yet? -- Michael Welsh Duggan (md5i@md5i.com) ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: Lexical let and setq 2013-09-14 0:09 ` Lexical let and setq Michael Welsh Duggan @ 2013-09-14 3:46 ` Stefan Monnier 2013-09-14 11:13 ` Lars Magne Ingebrigtsen 2013-09-14 21:47 ` Richard Stallman 1 sibling, 1 reply; 35+ messages in thread From: Stefan Monnier @ 2013-09-14 3:46 UTC (permalink / raw To: Michael Welsh Duggan; +Cc: emacs-devel > The primary reason I have seen the (let (foo) (setq foo ...)) idiom is > in looping code. That one is OK, since recursion is not supported efficiently. The problem is when people use the above because they're writing (poor) C code in Elisp (e.g. they begin their functions with a big let declaring all the local vars that they may use later on in the function). Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: Lexical let and setq 2013-09-14 3:46 ` Stefan Monnier @ 2013-09-14 11:13 ` Lars Magne Ingebrigtsen 2013-09-14 14:04 ` Pascal J. Bourguignon 2013-09-15 5:11 ` Stefan Monnier 0 siblings, 2 replies; 35+ messages in thread From: Lars Magne Ingebrigtsen @ 2013-09-14 11:13 UTC (permalink / raw To: emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > That one is OK, since recursion is not supported efficiently. > The problem is when people use the above because they're writing (poor) > C code in Elisp (e.g. they begin their functions with a big let > declaring all the local vars that they may use later on in the > function). I think the most common reason for stashing a lot of variables in a let is to avoid infinite indentation. (let ((a (foo))) (something) (let ((b (something-else))) (more a b) (let ((c (yet-more))) (zot a b c)))) vs (let ((a (foo)) b c) (something) (setq b (something-else)) (more a b) (setq c (yet-more)) (zot a b c)) I kinda think the latter form is sometimes more readable. -- (domestic pets only, the antidote for overdose, milk.) larsi@gnus.org * Lars Magne Ingebrigtsen ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: Lexical let and setq 2013-09-14 11:13 ` Lars Magne Ingebrigtsen @ 2013-09-14 14:04 ` Pascal J. Bourguignon 2013-09-15 5:11 ` Stefan Monnier 1 sibling, 0 replies; 35+ messages in thread From: Pascal J. Bourguignon @ 2013-09-14 14:04 UTC (permalink / raw To: emacs-devel Lars Magne Ingebrigtsen <larsi@gnus.org> writes: > Stefan Monnier <monnier@iro.umontreal.ca> writes: > >> That one is OK, since recursion is not supported efficiently. >> The problem is when people use the above because they're writing (poor) >> C code in Elisp (e.g. they begin their functions with a big let >> declaring all the local vars that they may use later on in the >> function). > > I think the most common reason for stashing a lot of variables in a let > is to avoid infinite indentation. > > (let ((a (foo))) > (something) > (let ((b (something-else))) > (more a b) > (let ((c (yet-more))) > (zot a b c)))) > > vs > > (let ((a (foo)) > b c) > (something) > (setq b (something-else)) > (more a b) > (setq c (yet-more)) > (zot a b c)) > > I kinda think the latter form is sometimes more readable. Err, no. A form with less indentation MAY be more readable, but not when it's a procedural form with a setq every other subform. If something similar and systematicall occurs from one level of indentation to the other, then you can write a macro that will do this uniform something, and have a unindented body: (something-similar (foo) (something) (something-else) (more) (yet-more) (zot)) A simple example would be: (defmacro pipe (input &rest functions) (pipe-aux input functions)) (defun pipe-aux (input functions) (if functions (pipe-aux (list (first functions) input) (rest functions)) input)) (macroexpand '(pipe foo a b c d e f g)) ; --> (g (f (e (d (c (b (a foo))))))) So you just write (pipe foo a b c d e f g) or rather: (pipe foo a b c d e f g) But in case of your let, it is much harder to read, because now you must rebuild the data flow from the variables and setq expressions. The indented form was much clearer, because you can more easily skip a subexpression, knowing for sure that whatever happens to variables in subexpressions can't affect the rest. -- __Pascal Bourguignon__ http://www.informatimago.com/ ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: Lexical let and setq 2013-09-14 11:13 ` Lars Magne Ingebrigtsen 2013-09-14 14:04 ` Pascal J. Bourguignon @ 2013-09-15 5:11 ` Stefan Monnier 1 sibling, 0 replies; 35+ messages in thread From: Stefan Monnier @ 2013-09-15 5:11 UTC (permalink / raw To: emacs-devel > I think the most common reason for stashing a lot of variables in a let > is to avoid infinite indentation. > (let ((a (foo))) > (something) > (let ((b (something-else))) > (more a b) > (let ((c (yet-more))) > (zot a b c)))) > vs > (let ((a (foo)) > b c) > (something) > (setq b (something-else)) > (more a b) > (setq c (yet-more)) > (zot a b c)) > I kinda think the latter form is sometimes more readable. In SML (where `setq' is not an option), the natural way to write it is equivalent to: (let* ((a (foo)) (_ (something)) (b (something-else)) (_ (more a b)) (c (yet-more))) (zot a b c)) -- Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: Lexical let and setq 2013-09-14 0:09 ` Lexical let and setq Michael Welsh Duggan 2013-09-14 3:46 ` Stefan Monnier @ 2013-09-14 21:47 ` Richard Stallman 2013-09-15 5:09 ` Stefan Monnier 1 sibling, 1 reply; 35+ messages in thread From: Richard Stallman @ 2013-09-14 21:47 UTC (permalink / raw To: Michael Welsh Duggan; +Cc: monnier, emacs-devel [ To any NSA and FBI agents reading my email: please consider [ whether defending the US Constitution against all enemies, [ foreign or domestic, requires you to follow Snowden's example. The primary reason I have seen the (let (foo) (setq foo ...)) idiom is in looping code. I have written code that way simply to make it clearer to read. The way I would normally try to avoid this idiom in most FP languages We should not try to avoid it. We should make it work just as efficiently as if it were written the other way. With lexical scope, it is not hard to determine that the lexical variable's value is never used until after the setq. Then it can be compiled as immutable. We suggest Emacs Lisp as a path for non-programmers to learn to program, so we need to encourage styles that are natural. That means loops, not tail recursion. Tail recursion is harder to read. -- Dr Richard Stallman President, Free Software Foundation 51 Franklin St Boston MA 02110 USA www.fsf.org www.gnu.org Skype: No way! That's nonfree (freedom-denying) software. Use Ekiga or an ordinary phone call. ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: Lexical let and setq 2013-09-14 21:47 ` Richard Stallman @ 2013-09-15 5:09 ` Stefan Monnier 2013-09-15 16:54 ` Richard Stallman 0 siblings, 1 reply; 35+ messages in thread From: Stefan Monnier @ 2013-09-15 5:09 UTC (permalink / raw To: Richard Stallman; +Cc: Michael Welsh Duggan, emacs-devel > Tail recursion is harder to read. We'll just agree to disagree here. This is not the place to discuss it. Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: Lexical let and setq 2013-09-15 5:09 ` Stefan Monnier @ 2013-09-15 16:54 ` Richard Stallman 2013-09-15 17:06 ` Stefan Monnier 0 siblings, 1 reply; 35+ messages in thread From: Richard Stallman @ 2013-09-15 16:54 UTC (permalink / raw To: Stefan Monnier; +Cc: mwd, emacs-devel [ To any NSA and FBI agents reading my email: please consider [ whether defending the US Constitution against all enemies, [ foreign or domestic, requires you to follow Snowden's example. If you have in mind something as radical as pushing Emacs Lisp coding towards use of tail recursion, it had better be discussed first. I think that would be a disaster. -- Dr Richard Stallman President, Free Software Foundation 51 Franklin St Boston MA 02110 USA www.fsf.org www.gnu.org Skype: No way! That's nonfree (freedom-denying) software. Use Ekiga or an ordinary phone call. ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: Lexical let and setq 2013-09-15 16:54 ` Richard Stallman @ 2013-09-15 17:06 ` Stefan Monnier 2013-09-16 10:47 ` Richard Stallman 0 siblings, 1 reply; 35+ messages in thread From: Stefan Monnier @ 2013-09-15 17:06 UTC (permalink / raw To: Richard Stallman; +Cc: mwd, emacs-devel > If you have in mind something as radical as pushing Emacs Lisp coding > towards use of tail recursion. I love recursion, but I have no idea where you got the impression that I'd like to do that. Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: Lexical let and setq 2013-09-15 17:06 ` Stefan Monnier @ 2013-09-16 10:47 ` Richard Stallman 0 siblings, 0 replies; 35+ messages in thread From: Richard Stallman @ 2013-09-16 10:47 UTC (permalink / raw To: Stefan Monnier; +Cc: mwd, emacs-devel [ To any NSA and FBI agents reading my email: please consider [ whether defending the US Constitution against all enemies, [ foreign or domestic, requires you to follow Snowden's example. > If you have in mind something as radical as pushing Emacs Lisp coding > towards use of tail recursion. I love recursion, but I have no idea where you got the impression that I'd like to do that. Your previous message seemed to say so. If that was a misunderstanding on my part, we don't need to discuss it. -- Dr Richard Stallman President, Free Software Foundation 51 Franklin St Boston MA 02110 USA www.fsf.org www.gnu.org Skype: No way! That's nonfree (freedom-denying) software. Use Ekiga or an ordinary phone call. ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-13 3:40 ` Stefan Monnier 2013-09-13 3:59 ` Drew Adams @ 2013-09-14 4:20 ` Dmitry Gutov 2013-09-14 14:27 ` Stefan Monnier 1 sibling, 1 reply; 35+ messages in thread From: Dmitry Gutov @ 2013-09-14 4:20 UTC (permalink / raw To: Stefan Monnier; +Cc: 15294 On 13.09.2013 06:40, Stefan Monnier wrote: > It seems the slowdown is indeed linked to the way `catch' is handled > (indeed, this non-idiomatic ELisp code ends up byte-compiled in a really > poor way). What's non-idiomatic about this use of `catch'? > The trivial patch below brings the time down from 5s to 2.6s (as > compared to 2.1s for the dynamic-binding version). Indeed, it does, in the compiled mode (but please don't install it yet, because it'll complicate merging the `arrow-functions' branch(*)). It does not make much of a difference in the interpreted mode. Now that we have eager macro-expansion, I was rather happy that interpreted js2-mode performance is only like 2x worse than when compiled. But 2.6 vs 2.1, it still a noticeable regression. Do you suppose the usage of `setq' is the main contributor? Speaking of byte-code changes, that seems like a fine idea, but ideally the code should work similarly well on Emacs 24.1-24.3. Or we can just hold off on making js2-mode switch to lexical-binding until [long] after 24.4 is out. (*) Would you take a look at it, too? It has quite a few changes in `js2-get-token' and related functions. They also make performing the same change as in your patch more difficult, since I'm actually using the value returned by `catch' before returning from the function. On 13.09.2013 07:37, Stefan Monnier wrote: > the mere existence of a single `setq' > on a variable can sometimes slow other chunks of code: in many cases > `let' is cheaper than `setq'). I see. Does this also extend to `setf' and its defstruct-related functionality? It's all over the js2-mode codebase. ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-14 4:20 ` bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode Dmitry Gutov @ 2013-09-14 14:27 ` Stefan Monnier 2013-09-15 0:11 ` Dmitry Gutov 0 siblings, 1 reply; 35+ messages in thread From: Stefan Monnier @ 2013-09-14 14:27 UTC (permalink / raw To: Dmitry Gutov; +Cc: 15294 >> It seems the slowdown is indeed linked to the way `catch' is handled >> (indeed, this non-idiomatic ELisp code ends up byte-compiled in a really >> poor way). > What's non-idiomatic about this use of `catch'? The non-idiomatic part is the "one big let on top, with lots of setq inside". It's clearly C code in Elisp syntax. > It does not make much of a difference in the interpreted mode. The interpreted performance is affected by completely different factors. My guess for the interpreted case is that there are simply "too many" local variables: the environment is represented by a simple alist, so variable lookup time is proportional to the number of local variables. That fine when there are 5 local variables, but is inefficient when you have 100 (better would be a balanced tree or maybe a hash table). This said, I'm not terribly concerned about it: if you need it to go fast, you should byte-compile the code. And I hope we will be able to get rid of the interpreter in some distantish future. > Now that we have eager macro-expansion, I was rather happy that interpreted > js2-mode performance is only like 2x worse than when compiled. Eager macro-expansion indeed speeds up interpreted code, even though the intention was rather to get one-step closer to the elimination of interpretation. > But 2.6 vs 2.1, it still a noticeable regression. Do you suppose the usage > of `setq' is the main contributor? The problem goes as follows: 1- Because of how the `catch' byte-code works, for a (catch TAG BODY) where BODY refers to some surrounding lexical variables LVARS, the byte-compiler needs to turn the code into something similar to: (let ((body-fun (make-closure LVARS () BODY))) (catch TAG (funcall body-fun))) 2- When a lexical variable is both a- caught in a closure b- not immutable the byte-compiler can't store this variable in the bytecode stack (since the closure can't refer to the bytecode stack directly, but instead stores *copies* of the elements it needs), so it needs to change code like (let ((lvar VAL1)) ... (setq lvar VAL2) ...(lambda () ..lvar..)...) into (let ((lvar (list VAL1))) ... (setcar lvar VAL2) ...(lambda () ..(car lvar)..)...) So if you look at js2-get-token, you'll see that the code does not directly use any closure, but the use of `catch' ends up putting most of the body into various closures. And since all variables are declared outside of the catch but used inside, and they're all modified by `setq', they all end up converted as above, so that every use of such a variable turns into "get the cons cell from the environment, then apply car to it". By moving the let inside the catch, some of those variables end up not being caught by a closure any more, so they don't need to be converted to cons cells, hence the reduction from 5s down to 2.6s. > (*) Would you take a look at it, too? It has quite a few changes in > js2-get-token' and related functions. > They also make performing the same change as in your patch more > difficult, since I'm actually using the value returned by `catch' > before returning from the function. That's not a problem. The rule to follow is simply: sink the `let' bindings closer to their use. You don't need to `let' bind all those vars together in one big `let': you can split this let into various `let's which you can then move deeper into the code. In some cases you'll find that some of those vars don't even need to be `setq'd any more. Note that such a "scope-reduction" can also be done in C and in many cases it's also a good idea to do it in C, tho the impact on performance is much less significant because C doesn't have closures. >> the mere existence of a single `setq' on a variable can sometimes >> slow other chunks of code: in many cases `let' is cheaper than `setq'). > I see. Does this also extend to `setf' and its defstruct-related > functionality? It has to do specifically with `setq' (i.e. modification of plain variables): when `setf' expands to `setq', `setf' is impacted, otherwise no. Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-14 14:27 ` Stefan Monnier @ 2013-09-15 0:11 ` Dmitry Gutov 2013-09-15 5:04 ` Stefan Monnier 2013-09-15 16:54 ` Richard Stallman 0 siblings, 2 replies; 35+ messages in thread From: Dmitry Gutov @ 2013-09-15 0:11 UTC (permalink / raw To: Stefan Monnier; +Cc: 15294 On 14.09.2013 17:27, Stefan Monnier wrote: >>> It seems the slowdown is indeed linked to the way `catch' is handled >>> (indeed, this non-idiomatic ELisp code ends up byte-compiled in a really >>> poor way). >> What's non-idiomatic about this use of `catch'? > > The non-idiomatic part is the "one big let on top, with lots of setq > inside". It's clearly C code in Elisp syntax. Or rather Java code, considering its origins. In general, we have to stay close enough to the SpiderMonkey codebase to be able to port new syntax features easily. You've explained how this is bad when combined with closures, but other than gotchas with `catch' (and `condition-case', I'm assuming), we don't have any higher-order functions in the parser. No lambda forms anywhere, at least. Can the fact that `dotimes', `dolist' and `loop' are advised with `cl--wrap-in-nil-block', which expands into `catch' form, make much of a difference? >> It does not make much of a difference in the interpreted mode. > > The interpreted performance is affected by completely different factors. > My guess for the interpreted case is that there are simply "too many" > local variables: the environment is represented by a simple alist, so > variable lookup time is proportional to the number of local variables. > That fine when there are 5 local variables, but is inefficient when you > have 100 (better would be a balanced tree or maybe a hash table). `js2-get-token' has 13 local variables (*). Which is, while not a little, far from 100. Most of the other functions have fewer than that. Are you counting the global variables, too? The dynamic-binding interpreter has to work with them, too. How is it that much faster? (*) According to Steve's notes, symbol lookup in alists is faster than in Emacs hashtables until 50+ elements. Or was, around the year 2009. > This said, I'm not terribly concerned about it: if you need it to go > fast, you should byte-compile the code. I guess so. The sloppy users who disregarded the instructions to byte-compile the code were actually creating a bad reputation for js2-mode 1-2 years ago, but since package.el is much more popular now, it should be less of a problem. >> But 2.6 vs 2.1, it still a noticeable regression. Do you suppose the usage >> of `setq' is the main contributor? > > The problem goes as follows: ... Thank you for the detailed explanation. There are a few `catch' forms left there, for tags `continue' and `break', used for control flow. So, most of the 0.5s difference left is likely due to them, right? I wonder how hard it'll be to rewrite that without `catch'. >> (*) Would you take a look at it, too? It has quite a few changes in >> js2-get-token' and related functions. > >> They also make performing the same change as in your patch more >> difficult, since I'm actually using the value returned by `catch' >> before returning from the function. > > That's not a problem. The rule to follow is simply: sink the `let' > bindings closer to their use. You don't need to `let' bind all those > vars together in one big `let': you can split this let into various > `let's which you can then move deeper into the code. In some cases > you'll find that some of those vars don't even need to be `setq'd any > more. Thanks, will do. ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-15 0:11 ` Dmitry Gutov @ 2013-09-15 5:04 ` Stefan Monnier 2013-09-15 16:54 ` Richard Stallman 1 sibling, 0 replies; 35+ messages in thread From: Stefan Monnier @ 2013-09-15 5:04 UTC (permalink / raw To: Dmitry Gutov; +Cc: 15294 > Can the fact that `dotimes', `dolist' and `loop' are advised with > cl--wrap-in-nil-block', which expands into `catch' form, make much > of a difference? No, these only expand to `catch' if there's a `return' inside. > `js2-get-token' has 13 local variables (*). Which is, while not a little, > far from 100. Most of the other functions have fewer than that. I don't know how many it takes to be significantly slower than `symbol-value'. > Are you counting the global variables, too? No. But arguments, yes, and every (defvar <var>) between point and BOB as well. > The dynamic-binding interpreter has to work with them, too. How is it > that much faster? Dynamic binding lookup is done by `symbol-value' which is just a field access (plus checking that the var is not special (e.g. buffer-local or a predefined C variable). >>> But 2.6 vs 2.1, it still a noticeable regression. Do you suppose the usage >>> of `setq' is the main contributor? >> The problem goes as follows: ... > Thank you for the detailed explanation. > There are a few `catch' forms left there, for tags `continue' and `break', > used for control flow. So, most of the 0.5s difference left is likely due to > them, right? The problem is not just the use of catch, but the combination of "catch" with all those vars let-bound outside of catch, used inside, and mutated: - make a variable immutable (i.e. remove the setqs on it) and that variable becomes more efficient again. - move the let binding inside the catch, and the var becomes efficient again. - don't use the variable inside the catch, and it becomes efficient again. Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-15 0:11 ` Dmitry Gutov 2013-09-15 5:04 ` Stefan Monnier @ 2013-09-15 16:54 ` Richard Stallman 1 sibling, 0 replies; 35+ messages in thread From: Richard Stallman @ 2013-09-15 16:54 UTC (permalink / raw To: Dmitry Gutov; +Cc: 15294 [ To any NSA and FBI agents reading my email: please consider [ whether defending the US Constitution against all enemies, [ foreign or domestic, requires you to follow Snowden's example. Perhaps we should optimize the implementation of catch in bytecode with some new opcodes. -- Dr Richard Stallman President, Free Software Foundation 51 Franklin St Boston MA 02110 USA www.fsf.org www.gnu.org Skype: No way! That's nonfree (freedom-denying) software. Use Ekiga or an ordinary phone call. ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-10 2:04 ` Stefan Monnier 2013-09-13 3:40 ` Stefan Monnier @ 2013-09-15 0:24 ` Dmitry Gutov 2013-09-15 5:06 ` Stefan Monnier 2013-09-18 23:48 ` Stefan Monnier 2 siblings, 1 reply; 35+ messages in thread From: Dmitry Gutov @ 2013-09-15 0:24 UTC (permalink / raw To: Stefan Monnier; +Cc: 15294 On 10.09.2013 05:04, Stefan Monnier wrote: > If it indeed comes from the handling of catch/throw, then there are two > ways to fix it: > - ... > - improve the byte-compiler so it can compile away cl-block/cl-return. > This should not require new byte-codes, but requires delving "deepish" > into the byte-compiler. This would take care of `cl-block' wrappers, but it would do nothing about explicit uses of `catch', right? ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-15 0:24 ` Dmitry Gutov @ 2013-09-15 5:06 ` Stefan Monnier 0 siblings, 0 replies; 35+ messages in thread From: Stefan Monnier @ 2013-09-15 5:06 UTC (permalink / raw To: Dmitry Gutov; +Cc: 15294 >> If it indeed comes from the handling of catch/throw, then there are two >> ways to fix it: >> - ... >> - improve the byte-compiler so it can compile away cl-block/cl-return. >> This should not require new byte-codes, but requires delving "deepish" >> into the byte-compiler. > This would take care of `cl-block' wrappers, but it would do nothing about > explicit uses of `catch', right? All js2-mode's uses of catch/throw can be replaced with block/return, AFAICT (not that it's useful right now, since they'll just be macroexpanded back to catch/throw). Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-10 2:04 ` Stefan Monnier 2013-09-13 3:40 ` Stefan Monnier 2013-09-15 0:24 ` Dmitry Gutov @ 2013-09-18 23:48 ` Stefan Monnier 2013-09-22 4:56 ` Dmitry Gutov 2 siblings, 1 reply; 35+ messages in thread From: Stefan Monnier @ 2013-09-18 23:48 UTC (permalink / raw To: Dmitry Gutov; +Cc: 15294 I now have a tentative patch which introduces new bytecodes for "catch" which avoid the need to create closures for lexical-binding code. Current results: - dynbind old-bytecode: 0.84s - dynbind new-bytecode: 0.83s - lexbind old-bytecode: 2.14s - lexbind new-bytecode: 0.67s Of course, code compiled with the new bytecodes won't run on non-bleeding edge Emacs. On my "standard test" (i.e. time to recompile all the emacs/lisp files), the new bytecodes make no measurable difference, so `catch' doesn't seem to be used very much there. Similar changes would be welcome for condition-case and unwind-protect, and since adding new bytecodes is something I'd rather not do too often (and since there aren't many unused bytecodes left), I'll try to get a complete patch first to make sure I don't have to change my mind and use different byte codes. Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-18 23:48 ` Stefan Monnier @ 2013-09-22 4:56 ` Dmitry Gutov 2013-10-03 5:00 ` Stefan Monnier 0 siblings, 1 reply; 35+ messages in thread From: Dmitry Gutov @ 2013-09-22 4:56 UTC (permalink / raw To: Stefan Monnier; +Cc: 15294 Stefan Monnier <monnier@IRO.UMontreal.CA> writes: > I now have a tentative patch which introduces new bytecodes for > "catch" which avoid the need to create closures for > lexical-binding code. > > Current results: > - dynbind old-bytecode: 0.84s > - dynbind new-bytecode: 0.83s > - lexbind old-bytecode: 2.14s > - lexbind new-bytecode: 0.67s Looks impressive! > Of course, code compiled with the new bytecodes won't run on > non-bleeding edge Emacs. Sure. > Similar changes would be welcome for condition-case and > unwind-protect, and since adding new bytecodes is something I'd rather > not do too often (and since there aren't many unused bytecodes left), > I'll try to get a complete patch first to make sure I don't have to > change my mind and use different byte codes. There's absolutely no hurry, I'm not going to switch js2-mode to lexical-binding until after 24.4 is out. ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-09-22 4:56 ` Dmitry Gutov @ 2013-10-03 5:00 ` Stefan Monnier 2013-10-04 2:38 ` Dmitry Gutov 0 siblings, 1 reply; 35+ messages in thread From: Stefan Monnier @ 2013-10-03 5:00 UTC (permalink / raw To: Dmitry Gutov; +Cc: 15294 You can try setting byte-compile--use-old-handlers to nil now to see if it fixes your performance problems. Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-10-03 5:00 ` Stefan Monnier @ 2013-10-04 2:38 ` Dmitry Gutov 2013-10-04 13:52 ` Stefan Monnier 0 siblings, 1 reply; 35+ messages in thread From: Dmitry Gutov @ 2013-10-04 2:38 UTC (permalink / raw To: Stefan Monnier; +Cc: 15294 On 03.10.2013 08:00, Stefan Monnier wrote: > You can try setting byte-compile--use-old-handlers to nil now to see if > it fixes your performance problems. Indeed, I see the same peformance improvement you wrote about: 0.60s in byte-compiled dynamic binding mode, and now 0.48s in byte-compiled lexical binding mode. Thanks! Not sure if I should close this issue now or at least wait until byte-compile--use-old-handlers is nil by default. ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-10-04 2:38 ` Dmitry Gutov @ 2013-10-04 13:52 ` Stefan Monnier 2013-10-05 3:27 ` Dmitry Gutov 2014-12-14 12:31 ` Dmitry Gutov 0 siblings, 2 replies; 35+ messages in thread From: Stefan Monnier @ 2013-10-04 13:52 UTC (permalink / raw To: Dmitry Gutov; +Cc: 15294 > Not sure if I should close this issue now or at least wait until > byte-compile--use-old-handlers is nil by default. Your call. I'm not completely sure when we should change it: setting it to nil will mean that pretty much any file compiled with the new Emacs won't run on older Emacsen. I think I'd like to see 24.4 release before making the change. But please do set it in your .emacs ASAP so it gets wider testing. Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-10-04 13:52 ` Stefan Monnier @ 2013-10-05 3:27 ` Dmitry Gutov 2014-12-14 12:31 ` Dmitry Gutov 1 sibling, 0 replies; 35+ messages in thread From: Dmitry Gutov @ 2013-10-05 3:27 UTC (permalink / raw To: Stefan Monnier; +Cc: 15294 On 04.10.2013 16:52, Stefan Monnier wrote: >> Not sure if I should close this issue now or at least wait until >> byte-compile--use-old-handlers is nil by default. > > Your call. I'm not completely sure when we should change it: setting it > to nil will mean that pretty much any file compiled with the new Emacs > won't run on older Emacsen. I think I'd like to see 24.4 release before > making the change. No problem, guess then I'll leave this open until then. I'll do the code reshuffling discussed earlier, but actually having a performance improvement will be a better reason to switch to lexical binding. > But please do set it in your .emacs ASAP so it gets wider testing. Good idea, done. ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2013-10-04 13:52 ` Stefan Monnier 2013-10-05 3:27 ` Dmitry Gutov @ 2014-12-14 12:31 ` Dmitry Gutov 2014-12-14 14:08 ` Stefan Monnier 1 sibling, 1 reply; 35+ messages in thread From: Dmitry Gutov @ 2014-12-14 12:31 UTC (permalink / raw To: Stefan Monnier; +Cc: 15294 Stefan Monnier <monnier@iro.umontreal.ca> writes: >> Not sure if I should close this issue now or at least wait until >> byte-compile--use-old-handlers is nil by default. > won't run on older Emacsen. I think I'd like to see 24.4 release before > making the change. But please do set it in your .emacs ASAP so it gets > wider testing. Just a reminder: I've been running with that setting for a while now, with no problems that I can remember. ^ permalink raw reply [flat|nested] 35+ messages in thread
* bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode 2014-12-14 12:31 ` Dmitry Gutov @ 2014-12-14 14:08 ` Stefan Monnier 0 siblings, 0 replies; 35+ messages in thread From: Stefan Monnier @ 2014-12-14 14:08 UTC (permalink / raw To: Dmitry Gutov; +Cc: 15294-done >>> Not sure if I should close this issue now or at least wait until >>> byte-compile--use-old-handlers is nil by default. >> won't run on older Emacsen. I think I'd like to see 24.4 release before >> making the change. But please do set it in your .emacs ASAP so it gets >> wider testing. > Just a reminder: I've been running with that setting for a while now, So have I. > with no problems that I can remember. Well, I fixed a bug in it a while ago (some issue with "volatile", fixed before the 24.4 release), but other than that, indeed it works fine. The default value has been changed in "master" a couple months ago, already. Stefan ^ permalink raw reply [flat|nested] 35+ messages in thread
end of thread, other threads:[~2014-12-14 14:08 UTC | newest] Thread overview: 35+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-09-06 20:59 bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode Dmitry Gutov 2013-09-06 23:44 ` Xue Fuqiao 2013-09-07 3:15 ` Stefan Monnier 2013-09-08 22:32 ` Stefan Monnier 2013-09-10 2:04 ` Stefan Monnier 2013-09-13 3:40 ` Stefan Monnier 2013-09-13 3:59 ` Drew Adams 2013-09-13 4:37 ` Stefan Monnier 2013-09-13 5:45 ` Drew Adams 2013-09-13 13:01 ` Stefan Monnier 2013-09-14 0:09 ` Lexical let and setq Michael Welsh Duggan 2013-09-14 3:46 ` Stefan Monnier 2013-09-14 11:13 ` Lars Magne Ingebrigtsen 2013-09-14 14:04 ` Pascal J. Bourguignon 2013-09-15 5:11 ` Stefan Monnier 2013-09-14 21:47 ` Richard Stallman 2013-09-15 5:09 ` Stefan Monnier 2013-09-15 16:54 ` Richard Stallman 2013-09-15 17:06 ` Stefan Monnier 2013-09-16 10:47 ` Richard Stallman 2013-09-14 4:20 ` bug#15294: 24.3.50; js2-mode parser is several times slower in lexical-binding mode Dmitry Gutov 2013-09-14 14:27 ` Stefan Monnier 2013-09-15 0:11 ` Dmitry Gutov 2013-09-15 5:04 ` Stefan Monnier 2013-09-15 16:54 ` Richard Stallman 2013-09-15 0:24 ` Dmitry Gutov 2013-09-15 5:06 ` Stefan Monnier 2013-09-18 23:48 ` Stefan Monnier 2013-09-22 4:56 ` Dmitry Gutov 2013-10-03 5:00 ` Stefan Monnier 2013-10-04 2:38 ` Dmitry Gutov 2013-10-04 13:52 ` Stefan Monnier 2013-10-05 3:27 ` Dmitry Gutov 2014-12-14 12:31 ` Dmitry Gutov 2014-12-14 14:08 ` Stefan Monnier
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.