From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Ergus Newsgroups: gmane.emacs.help Subject: Re: Why is Elisp slow? Date: Fri, 3 May 2019 11:58:49 +0200 Message-ID: <20190503095847.5vmkwdwugrqyfj2g@Ergus> References: <20190502131827.GA28987@tuxteam.de> <83k1f8q39o.fsf@gnu.org> <87woj8bqho.fsf@telefonica.net> <83tvecocvv.fsf@gnu.org> <87sgtwboot.fsf@telefonica.net> <83muk4obfd.fsf@gnu.org> <20190502214006.4fdsinp7u5xuqvdv@Ergus> <20190503004416.xfuzzucflp6bxpuz@Ergus> <83ftpwnhsy.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="22790"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: NeoMutt/20180716 Cc: help-gnu-emacs@gnu.org To: Eli Zaretskii Original-X-From: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Fri May 03 23:19:34 2019 Return-path: Envelope-to: geh-help-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hMfad-0012u5-Jr for geh-help-gnu-emacs@m.gmane.org; Fri, 03 May 2019 23:19:31 +0200 Original-Received: from localhost ([127.0.0.1]:37452 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hMUyC-0000nP-Lw for geh-help-gnu-emacs@m.gmane.org; Fri, 03 May 2019 05:59:08 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:44364) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hMUy1-0000n4-Qn for help-gnu-emacs@gnu.org; Fri, 03 May 2019 05:58:58 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hMUy0-00059t-Cz for help-gnu-emacs@gnu.org; Fri, 03 May 2019 05:58:57 -0400 Original-Received: from sonic311-30.consmr.mail.ir2.yahoo.com ([77.238.176.162]:38093) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hMUxz-00056r-VU for help-gnu-emacs@gnu.org; Fri, 03 May 2019 05:58:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aol.com; s=a2048; t=1556877534; bh=q80LeipDzn8v/UGP5h/mBgZAzWj0/nYG4DM3C51iTWM=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From:Subject; b=NYWsZU5YSTvg5tZfCHcfREhyQuE0kLFhgBgOORKj3josXL7PpxprY0OXnyc6uq1l0TLS18FuxkzKJQF00n3xTFoP3CgJa+wAOU8uV0drbWbVuGwTiVZJrYXQguy5NaudJYSXGu2IutyvWq7a97UgyHrtE0xR12i6889ktx78XihESwRiK1xYlAKZlda/WNoMcNgDE37Z3bWqlBxPBE1tP9HPudc7bx8TTEscyxuvcL9XDB1kPwBEKwXnsv3aOeWsSqS31dqJ2qLRFwJ+/eCMVMM1Dz1cNLs7E977tIlbBm+Lpl4DYk+IHjN2QL+izhSlvQcmfWUtCW+KeCclZGXV+g== X-YMail-OSG: hKGIQUMVM1kqpd.V9ta1_uAeP1q.UqDf5_hx4ckZjbsjR1wOCRtcP2v4L1AJnqK E1MhhGBS5HMmpFQXxJ95psCoW2BGQzSGqJX9sjFgQ5pj1OgjlKWgdBfK8jSQ1VzuhhfyBvKLwGyA pI1NXil8GZhb88Dyd0aCFpj2dtczpcn2Xggf_zfaV9JzYJ4xrCy8Sc0RbXVOlr445U50fwvVHTb3 B6_Rf3Rc_L9.9FRaT4ulqzuP.OmaMknnlDIi263j82ink7hwYspkLY_KM3mFtKiCV_D2vEJpNqLm Z0fwQ1mEBiPRB3bdgVbdAgeFIKqILNWJ2JhjxjaIJObsONTXeMJwlmxE9LY1Oa9up0KWP8LpXaJS 8fJLIv702NWv5SI_bF5nVm_Slp1m2ECPygsc2tAKI6cBL1pRFRyUzxcsjezKcdUK29xKslpvWYN2 G3kBeiiybEiSh6b3Xs3U_tIVNsVA1fcyd5reHMovh9urCfSi7Cs9NcVZhloxdQCct68MvubfQ0dX _QqSW9YdfBGb6bU7f96TovnNumRilGezITPgiKlLUSzOJCclvJqblGcx03cR632bCKDy4.N0P6RB DAE4sKbqJT_I5A1s1srw2sE4LIxBmU1JGsI4mawFlH3uCX42rfBaIcAbz29FS71Su3eT8WMqVyZp HxAP3SkIa2WuHKIB67DN1wuT003jjgaziOX3mqQwsgGfhQVo7jAtE7pmsXHR9e84l2_Vp6hgaFok p.w6kKbVHV5NAeZVop39FfU_.4f9BICf7c2XM5NSQMn6Xf.Vt_TdZ6hi9lT4ch5b2uvZDr.CR6yv o94Pt3y9Eb.UHFq5_4d2fl_jE02yfm73p0gWRhUfRh Original-Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.ir2.yahoo.com with HTTP; Fri, 3 May 2019 09:58:54 +0000 Original-Received: from 84.88.50.33 (EHLO Ergus) ([84.88.50.33]) by smtp402.mail.ir2.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 7628731d8c6d11a3f6f0a8274a9edfe2; Fri, 03 May 2019 09:58:52 +0000 (UTC) Content-Disposition: inline In-Reply-To: <83ftpwnhsy.fsf@gnu.org> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x X-Received-From: 77.238.176.162 X-BeenThere: help-gnu-emacs@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Users list for the GNU Emacs text editor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "help-gnu-emacs" Xref: news.gmane.org gmane.emacs.help:120169 Archived-At: On Fri, May 03, 2019 at 10:00:45AM +0300, Eli Zaretskii wrote: >> Date: Fri, 3 May 2019 02:44:16 +0200 >> From: Ergus >> Cc: help-gnu-emacs@gnu.org >> >> Having Guile as a dependency will grow emacs size significantly >> and using it as an external dependency (not usually the emacs way...) >> will require to port Guile to more systems (AFAIK). > >Which systems we care about are not already supported by Guile? > Actually in windows I have a friend who tried it and the experience was not convincing for him. And the native code compiler was in fact tremendously slow in a power9 machine. It felt like a hack in both cases more than a real native application. >> The alternative JIT may be based in libJIT which is not very active >> either... and has serious issues/limitations that has not been solved in >> years. > >There's a branch in the Emacs repository that uses libJIT. It has >problems with some Emacs configurations, which need to be solved in >libJIT. Alas, those problems, reported months ago to the libJIT >developers, are still not fixed. > >More importantly, the libJIT build failed to show any significant >speed-up wrt byte code, so it sounds like maybe the whole idea was >either wrong or its design couldn't possibly provide any gains. Or >maybe we just measured the speed-up in wrong scenarios. > This is not surprising. My work is 80% performance measurement and benchmarking and the real improvements with JIT compilation (in my experience) and specially with libJIT is not as good as many people expect in most of the common scenarios. That's because the generated code is usually very generic (so it does not take advantage of architecture specific features), and strategies like vectorization and branch prediction are very difficult to hint (most of the times impossible). So the only real difference with a bytecode interpreter is the bytecode parsing part, but not too much more. There is also the typical array of struct vs struct of arrays problem, type aligns and the optimal memory management what are also very dark to advice for a JIT compiler. The use of dark types and pointers to functions produces huge overheads and they are extensively used in JIT compilers internally generated code, the impossibility to do proper inlinings, and of course the fact that coming from a higher level language like Lisp we need more dynamic/runtime functionalities like garbage collector and generic function types. Plus the hashtable search costs and the extensively used list searches in Lisp instead of vectors or similar cheaper containers and callbacks in the user code. That's why I support the idea of the Lisp->C compiler more than the JIT at least to use it for the emacs core Lisp functions because gcc is very optimized and migrate the bottleneck back to C code as much as possible. (many of the very used in simple.el for example that are called very frequently) That's the strategy that made python become so successfully (plus the language simplicity). The existence of Cython, Numba + Pure C/Fortran core libraries like numpy and a simple C API to extend it. So when a user implements anything in 100% python; if it has performance issues she only needs to change the python lists with numpy arrays and numpy functions and that's it, the difference will be observed immediately. But if a specific function still represents a problem she can implement it easily in C or use a C library directly with CTypes, Pybind11 or boost, that uses dlopen instead of create processes and parse outputs and writes in numpy arrays/types directly. So in spite of that python's virtual machine is very slow to load, the rest of the execution is not necessarily slow and can be improved as much as the developer desires. Also as there are more users the bytecode part (interpreter and generator) is improved and optimized constantly.