From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Andrea Corallo via "Emacs development discussions." Newsgroups: gmane.emacs.devel Subject: Re: Profiling native-compilation Date: Fri, 23 Apr 2021 08:45:44 +0000 Message-ID: References: <83mttpo4rv.fsf@gnu.org> Reply-To: Andrea Corallo Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="32015"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: emacs-devel@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Apr 23 10:47:25 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lZrTB-00084S-6j for ged-emacs-devel@m.gmane-mx.org; Fri, 23 Apr 2021 10:47:25 +0200 Original-Received: from localhost ([::1]:49846 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lZrTA-0004Vz-AO for ged-emacs-devel@m.gmane-mx.org; Fri, 23 Apr 2021 04:47:24 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:59838) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lZrRk-0003Jt-0Q for emacs-devel@gnu.org; Fri, 23 Apr 2021 04:45:56 -0400 Original-Received: from mx.sdf.org ([205.166.94.24]:65473) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lZrRg-0007eZ-Tr; Fri, 23 Apr 2021 04:45:55 -0400 Original-Received: from mab (ma.sdf.org [205.166.94.33]) by mx.sdf.org (8.15.2/8.14.5) with ESMTPS id 13N8jijC002200 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits) verified NO); Fri, 23 Apr 2021 08:45:45 GMT In-Reply-To: <83mttpo4rv.fsf@gnu.org> (Eli Zaretskii's message of "Fri, 23 Apr 2021 10:10:12 +0300") Received-SPF: pass client-ip=205.166.94.24; envelope-from=akrl@sdf.org; helo=mx.sdf.org X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:268306 Archived-At: Eli Zaretskii writes: > I tried to profile the code involved in native-compilation using > profiler.el, but the results seem to omit most of the functions > involved in that: the percents shown in the resulting profile are very > low, which seems to hint that most of the CPU time is spent in places > that profiler.el cannot identify. > > Is it possible to use profiler.el to produce a meaningful profile of > native-compilation? > If yes, how does one go about setting up such > profiling? And if this is not currently possible, why isn't it, and > can we do something to make the situation better? For example, for at > least two C functions that are important to see in the profile -- GC > and redisplay -- we did arrange to be shown in the profile. So if the > problem is that most of the time is spent inside C code, perhaps we > could do something similar in comp.c? Hi Eli, AFAIK profiling syncronous native compilation with profile.el should be possible but with one caveat that is: we run the final pass in a subprocess so we'll loose visibility on that, OTOH we can consider final being essentially all libgccjit work. Maybe that's the reason why you see low percentages? Have you profiled a `native-compile' invocation? > I think this is important because native-compilation is quite slow, > and at least I don't have a clear idea where most of that time is > spent. Andrea, you said once that most of the time is spent in > libgccjit functions, which I find strange, since those are basically > parts of GCC code, and I don't think I ever saw such slow compilation > times from GCC. Yes I mentioned that but since we changed the layout of generated code for storing immediates GCC got way faster and now the Emacs side is about equally responsible for the time spent in compilation. One simple approach to get an idea of where time is spent in the compiler is to bind `comp-log-time-report' to get a report pass by pass. Running: === (require 'comp) (let ((comp-log-time-report t)) (native-compile ".../emacs2/lisp/emacs-lisp/byte-opt.el")) === I get: Done compiling .../.emacs.d/eln-cache/28.0.50-7f3f9b99/byte-opt-9c5f25f5-d2005e8f.eln Pass comp-spill-lap took: 0.095720s. Pass comp-limplify took: 0.423704s. Pass comp-fwprop took: 3.334509s. Pass comp-call-optim took: 0.023363s. Pass comp-ipa-pure took: 0.000046s. Pass comp-add-cstrs took: 0.222719s. Pass comp-fwprop took: 2.549254s. Pass comp-tco took: 0.000044s. Pass comp-fwprop took: 0.256151s. Pass comp-remove-type-hints took: 0.025859s. Pass comp-final took: 4.268238s. > So I guess there's more here than meets the eye, and > I think we should work on getting a clear idea where the time is > spent, so that we could improve it. For example, if indeed libgccjit > is the main sink, we should ask the respective GCC developers to speed > it up. Agree there is certainly room for ATM unexplored improvements, IIRC consing and GC have a considerable factor in the Emacs side of the compilation. I think would be interesting to sort by priority what I think are the three main ares to work on the native compiler on the post merge phase: compile-time, generated code performance, hardening, documentation. Thanks Andrea