From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Karthik Chikmagalur Newsgroups: gmane.emacs.devel Subject: Understanding filter function calls Date: Sun, 23 Jul 2023 22:46:09 -0700 Message-ID: <87y1j5vp3y.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="35581"; mail-complaints-to="usenet@ciao.gmane.io" To: Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Mon Jul 24 07:47:18 2023 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qNoPe-00092U-2Q for ged-emacs-devel@m.gmane-mx.org; Mon, 24 Jul 2023 07:47:18 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qNoOl-0007WN-DM; Mon, 24 Jul 2023 01:46:23 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qNoOf-0007W9-KK for emacs-devel@gnu.org; Mon, 24 Jul 2023 01:46:18 -0400 Original-Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qNoOc-00055Q-A7 for emacs-devel@gnu.org; Mon, 24 Jul 2023 01:46:16 -0400 Original-Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-668709767b1so2404555b3a.2 for ; Sun, 23 Jul 2023 22:46:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1690177572; x=1690782372; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=lAaz97QhRg9OmIY+cs3tmwbZba5/jI+juHJB1mH1O6I=; b=b3+qUUQgEp62wCjMqyU4QEGOG+vSbEVa9s/lw7t42Xc6nKzFwrpgJ44aBCokMmBXD5 2zCmd7ic+rbFfdoR8aeZn68reo+83LF06+ZAwnXlaX0v4BH2QNpwf9mqUqp3mJHPsQpD VtqPFkFPi3Cn81K5GZAWlmvLMgwHx8rSVF/iTUBhOYxslPqzxduC6Tupf669s1oIkHQ6 lTndbrxPBDMxg9Y0mAw1n4m0eZ5c7XRnvJ2YHpPZLI4vB+YJZuA715tGdnlSLef38t6F q00lS3iL0hOhP7FvEAIbuggMlZp1/Y1WiiqLXcG/t4FjtOB4I05P9LDkUNDaGB96HYa/ yQLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690177572; x=1690782372; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lAaz97QhRg9OmIY+cs3tmwbZba5/jI+juHJB1mH1O6I=; b=PypwO6MgZWplUJTnIg15OynLxrsv9JEPfqqrPpCG7JwVT3CiVbKQQCHr0afn4yVr0W /7msTQmz6UWIBg0H9Eu6v23Cgf6yI4+cgaim0Ovc2qBwgdtPRG8v4M4jqmqbHKwt35Fj jzVvc4l4lzUXXuYO5qXVJuzmKUCSN7x3d7LUG7f5KIkP0Uu38YVJ904DY93OUyqk5m+S ef3RhOkiNw2c65FSeYk6Mxt4geyPp694LlGdx/L+Vtdijz8VYFEsSBV+oq1JDDryZmHM KP2OhAawNV3pf51VZ+JFoKYCzVCtn6XAMH8hAi02d1cAXowsIRyyMBFTR6cZIur0Db/5 lSuQ== X-Gm-Message-State: ABy/qLblYaagiV8Gq/DjNgiYCOg7P316DzKp5adFo9lKlhB5L9GrYMNF nXAyN4jn0atYhl+gO5pOZX9hACctYzw= X-Google-Smtp-Source: APBJJlFwpO4axCI2IxjUTgKMpan4CRwL40dvoaplFyYusvmppCG2wrl9oE/POV0+RResHaw9vjm6BA== X-Received: by 2002:a05:6a00:24ca:b0:682:d2af:218 with SMTP id d10-20020a056a0024ca00b00682d2af0218mr8852754pfv.24.1690177571439; Sun, 23 Jul 2023 22:46:11 -0700 (PDT) Original-Received: from localhost ([2600:8802:5722:9400::91fe]) by smtp.gmail.com with ESMTPSA id q11-20020a638c4b000000b0055b44a901absm7487187pgn.70.2023.07.23.22.46.10 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 23 Jul 2023 22:46:10 -0700 (PDT) Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=karthikchikmagalur@gmail.com; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:308044 Archived-At: I'm testing some code written for the upcoming Org 9.7 for previewing LaTeX fragments. Given a LaTeX environment (or many), the way the code works is: 1. Gather fragments in the buffer 2. Create a TeX file containing this LaTeX 3. Run latex (TeX -> DVI) 4. Run dvisvgm in the LaTeX process sentinel (DVI -> SVG or series of SVGs) 5. Update in-buffer previews as SVGs are generated through the dvisvgm process' filter function. We use a filter function for the dvisvgm process to incrementally update previews as this is much faster on larger runs than waiting for the process sentinel to run. This is different from how LaTeX preview generation has worked in Org mode so far. It's as asynchronous as can be, and somewhat similar to how preview-latex (part of AucTeX works), if you're familiar with that. The problem is that how long preview generation takes is significantly different for different TeXLive versions (i.e. different LaTeX/dvisvgm executables). For example, LaTeX preview generation in an Org file with ~600 fragments takes: | | Preview generation time | |--------------+-------------------------| | TeXLive 2022 | 2.65 secs | | TeXLive 2023 | 4.03 secs | This is with identical code on the Emacs side of things. This difference is NOT explainable as the newer versions of LaTeX or dvisvgm taking longer. When benchmarked individually on the same TeX file -- and without Emacs in the picture -- latex 2022 and latex 2023 (as I'll call them here) take near identical times, as do dvisvgm 2022 and dvisvgm 2023. | | latex run | dvisvgm run | |--------------+-----------------+--------------| | TeXLive 2022 | 253.9 =C2=B1 10.6 ms | 1266 =C2=B1 41 ms | | TeXLive 2023 | 258.9 =C2=B1 15.0 ms | 1298 =C2=B1 15 ms | The stdout from latex and dvisvgm, which the sentinel/filter functions parse, are near identical, and the SVG images are the same sizes. I've controlled every variable I could think to control. - Same Org file. - Same org-latex-preview customizations/settings. - Same Emacs buffers open, etc. - Run `garbage-collect' immediately before benchmarking. - Same background system processes. So why is the TeXLive 2023 run so much slower in Emacs? After profiling with elp and generating a flamegraph, this is the result (png image): https://abode.karthinks.com/share/olp-timing-chart.png You can ignore the disproportionately long duration function calls, this is related to GC, and one additional GC phase during the slower (TeXLive 2023) case cannot explain the discrepancy. Further, sometimes there are more GC phases in the TeXLive 2022 run, but it's still significantly faster. The overall synchronous run times of the code in Emacs for TeXLive 2022 and 2023 are similar (0.77 vs 0.84 s). However the dvisvgm filter function is called quite differently in the two cases. | | call count | total time | average time | |--------------+------------+------------+--------------| | TeXLive 2022 | 25 | 0.77 secs | 31 ms | | TeXLive 2023 | 39 | 0.84 secs | 22 ms | Even though the overall time spent in the filter function is about the same, the TeXLive 2023 dvisvgm run - calls the filter function 39 times instead of 25, where - each run of the filter function processes less stdout, - each run of the filter function takes less time to run, - and crucially, these filter function calls are spaced slightly further apart in time The net result of which is that the overall preview generation process takes much longer to complete. I've tested this multiple times over runs (Org/TeX files) of different sizes, and the pattern is the same. My questions are thus: 1. If the latex/dvisvgm executables from TeXLive 2022/2023 take about the same time, and the stdout (that the filter function sees) is identical, why is Emacs' filter function call behavior different? 2. Is there anything I can do to obtain time signature/behavior like with TeXLive 2022 (see above link)? I would really like to speed up preview generation. -Karthik