* Re: Suppressing native compilation (short and long term) @ 2022-10-23 18:43 Gregor Zattler 0 siblings, 0 replies; 343+ messages in thread From: Gregor Zattler @ 2022-10-23 18:43 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, akrl Hi Eli, * Eli Zaretskii <eliz@gnu.org> [2022-10-23; 20:30 +03]: >> From: Gregor Zattler <telegraph@gmx.net> >> Cc: spwhitton@spwhitton.name, rlb@defaultvalue.org, >> monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, >> akrl@sdf.org >> Date: Sun, 23 Oct 2022 19:08:23 +0200 >> I did A) >> find "/home/grfz/.config/emacs/eln-cache/" "/home/grfz/src/emacs-master-next/native-lisp/" -type f | xargs -r ls -Altr > /tmp/before B) >> <start Emacs again> C) >> while sleep 1 ; do ps -eo pid,tty,stat,user,group,etime,time,cgroup,args fax >> faxme; done # while emacs started this writes a snapshot of the currently running processes every second and accumulates them. D) >> find "/home/grfz/.config/emacs/eln-cache/" "/home/grfz/src/emacs-master-next/native-lisp/" -type f | xargs -r ls -Altr > /tmp/before ^^^^^^ > I guess you mean "after", not "before"? You are right. E) $ diff -aNur before after $ --> No new .eln files were written. >> But file faxme contains 280 lines like this: >> >> /home/grfz/src/emacs-master-next/src/emacs --batch -l /tmp/emacs-async-comp-cl-lib-7Go9da.el > > And there's already a cl-lib-XXXXX.eln file under eln-cache? Yes (see below) G) >> 0 grfz@no:/tmp$ grep -o 'emacs-async-comp-.*' faxme | cut -d - -f 4- | sed -e "s/-[^-]*\.el$//" |sort -u | while read ; do grep -qs $REPLY before || echo $REPLY; done >> 0 grfz@no:/tmp$ This command shows that all the /tmp/emacs-async-comp-cl-lib-7Go9da.el like filenames in the processes reduced to (in this example) "cl-lib-" have a corresponding "cl-lib-XXXXX.eln" file in the .eln cache. H) Actually the command did not ensure that the hit was in the most recent .eln cache directories, the most recent (as an example) cl-lib-XXXXX.eln is -rwxr-xr-x 1 grfz grfz 65832 Okt 19 12:56 /home/grfz/src/emacs-master-next/native-lisp/29.0.50-5a57c71f/cl-lib-8b938900-c76f14d9.eln >> So all these "emacs-async-comp-cl-lib-7Go9da.el" like files >> have corresponding files in the .eln cache. >> >> Is it possible that it takes 150 secs to test the .eln cache? > > "Test" in what sense? Who do you think needs to "test" the cache? Test if an existing .el file needs to be compiled to an .eln file or if a corresponding and up-to-date .eln file for this very .el file already exists. > Anyway, the above is not what I meant. You present some commands and > scripts, and expect me to guess what happens by showing only their > results. It's very hard to analyze a problem that way. > > Instead, please do this: > . find the latest subdirectory of the eln-cache whose name is > 29.0.5-XXXX (where XXXX is some hash) and list is contents I) $ ls -Altrd /home/grfz/.config/emacs/eln-cache/* /home/grfz/src/emacs-master-next/native-lisp/* -rw------- 2 grfz grfz 188 Mai 19 2010 /home/grfz/.config/emacs/eln-cache/CACHEDIR.TAG drwxr-xr-x 2 grfz grfz 36864 Jul 21 22:26 /home/grfz/.config/emacs/eln-cache/29.0.50-50117688/ drwxr-xr-x 2 grfz grfz 36864 Jul 31 11:31 /home/grfz/.config/emacs/eln-cache/29.0.50-9e3d75e3/ drwxr-xr-x 2 grfz grfz 4096 Aug 1 10:47 /home/grfz/.config/emacs/eln-cache/29.0.50-03ee865e/ drwxr-xr-x 2 grfz grfz 20480 Aug 2 21:00 /home/grfz/.config/emacs/eln-cache/29.0.50-f8d87b98/ drwxr-xr-x 2 grfz grfz 4096 Aug 11 13:26 /home/grfz/.config/emacs/eln-cache/28.1.91-6470e6f9/ drwxr-xr-x 2 grfz grfz 12288 Aug 20 17:01 /home/grfz/.config/emacs/eln-cache/29.0.50-548f8cdf/ drwxr-xr-x 2 grfz grfz 20480 Aug 20 19:11 /home/grfz/.config/emacs/eln-cache/29.0.50-b189cc80/ drwxr-xr-x 2 grfz grfz 4096 Aug 24 21:18 /home/grfz/.config/emacs/eln-cache/29.0.50-9d768eff/ drwxr-xr-x 2 grfz grfz 20480 Aug 26 10:43 /home/grfz/.config/emacs/eln-cache/29.0.50-121fd72d/ drwxr-xr-x 2 grfz grfz 4096 Aug 26 20:14 /home/grfz/.config/emacs/eln-cache/29.0.50-11460309/ drwxr-xr-x 2 grfz grfz 20480 Aug 27 11:27 /home/grfz/.config/emacs/eln-cache/29.0.50-00ab4019/ drwxr-xr-x 2 grfz grfz 20480 Sep 3 14:41 /home/grfz/.config/emacs/eln-cache/29.0.50-4cc289c2/ drwxr-xr-x 2 grfz grfz 4096 Sep 13 17:13 /home/grfz/.config/emacs/eln-cache/29.0.50-db6a9ec4/ drwxr-xr-x 2 grfz grfz 36864 Sep 23 19:02 /home/grfz/.config/emacs/eln-cache/29.0.50-54242f81/ drwxr-xr-x 2 grfz grfz 20480 Okt 8 21:09 /home/grfz/.config/emacs/eln-cache/29.0.50-89a6c1cd/ drwxr-xr-x 2 grfz grfz 4096 Okt 8 22:10 /home/grfz/.config/emacs/eln-cache/29.0.50-d0addadc/ drwxr-xr-x 2 grfz grfz 40960 Okt 19 12:01 /home/grfz/.config/emacs/eln-cache/29.0.50-4c1d7d1b/ drwxr-xr-x 2 grfz grfz 20480 Okt 19 13:04 /home/grfz/.config/emacs/eln-cache/29.0.50-1baaebd1/ drwxr-xr-x 3 grfz grfz 28672 Okt 19 14:03 /home/grfz/src/emacs-master-next/native-lisp/29.0.50-5a57c71f/ drwxr-xr-x 2 grfz grfz 4096 Okt 19 14:41 /home/grfz/src/emacs-master-next/native-lisp/29.0.50-9ff4cf87/ drwxr-xr-x 2 grfz grfz 4096 Okt 19 20:06 /home/grfz/.config/emacs/eln-cache/29.0.50-9ff4cf87/ The latest on only shows trampoline files: $ ls -Altr /home/grfz/.config/emacs/eln-cache/29.0.50-9ff4cf87/ total 180 -rwxr-xr-x 1 grfz grfz 16872 Okt 19 15:22 subr--trampoline-73746172742d6b62642d6d6163726f_start_kbd_macro_0.eln* -rwxr-xr-x 1 grfz grfz 16872 Okt 19 15:23 subr--trampoline-656e642d6b62642d6d6163726f_end_kbd_macro_0.eln* -rwxr-xr-x 1 grfz grfz 16872 Okt 19 15:23 subr--trampoline-7965732d6f722d6e6f2d70_yes_or_no_p_0.eln* -rwxr-xr-x 1 grfz grfz 16872 Okt 19 15:23 subr--trampoline-6b696c6c2d627566666572_kill_buffer_0.eln* -rwxr-xr-x 1 grfz grfz 16872 Okt 19 15:33 subr--trampoline-782d666f6375732d6672616d65_x_focus_frame_0.eln* -rwxr-xr-x 1 grfz grfz 16872 Okt 19 19:52 subr--trampoline-61626f72742d7265637572736976652d65646974_abort_recursive_edit_0.eln* -rwxr-xr-x 1 grfz grfz 16872 Okt 19 19:52 subr--trampoline-746f702d6c6576656c_top_level_0.eln* -rwxr-xr-x 1 grfz grfz 16872 Okt 19 19:52 subr--trampoline-6d616b652d70726f63657373_make_process_0.eln* -rwxr-xr-x 1 grfz grfz 16872 Okt 19 20:06 subr--trampoline-6d657373616765_message_0.eln* Surprise, the second to last eln cache directoriy only has one file in it: ls -Altr /home/grfz/src/emacs-master-next/native-lisp/29.0.50-9ff4cf87/ total 40 -rwxr-xr-x 1 grfz grfz 39536 Okt 19 14:41 debug-ce0d397b-01a0d81b.eln* J) But the third to last eln cache directory has more files: $ ls -Altr /home/grfz/src/emacs-master-next/native-lisp/29.0.50-5a57c71f/ total 41744 -rwxr-xr-x 1 grfz grfz 86624 Okt 19 12:21 cconv-3b1f1f98-aafc4e46.eln* -rwxr-xr-x 1 grfz grfz 221592 Okt 19 12:23 byte-opt-9c5f25f5-ca1fc8cd.eln* -rwxr-xr-x 1 grfz grfz 452592 Okt 19 12:23 bytecomp-12882072-a85f7aaf.eln* -rwxr-xr-x 1 grfz grfz 77528 Okt 19 12:24 loaddefs-gen-e8a3ad9c-8046a1f5.eln* -rwxr-xr-x 1 grfz grfz 35288 Okt 19 12:24 radix-tree-669a468d-fa18562f.eln* -rwxr-xr-x 1 grfz grfz 213952 Okt 19 12:24 comp-cstr-ef162ef7-0f1f9b64.eln* -rwxr-xr-x 1 grfz grfz 950544 Okt 19 12:27 comp-7672a6ed-0c77f4fc.eln* -rwxr-xr-x 1 grfz grfz 72832 Okt 19 12:28 titdic-cnv-765ac3be-7e3dd66e.eln* -rwxr-xr-x 1 grfz grfz 37272 Okt 19 12:38 emoji-zwj-4f682c68-6523b72a.eln* -rwxr-xr-x 1 grfz grfz 29080 Okt 19 12:38 charscript-600dca1a-50cd3215.eln* drwxr-xr-x 2 grfz grfz 12288 Okt 19 12:52 preloaded/ -rwxr-xr-x 1 grfz grfz 79000 Okt 19 12:53 eieio-base-3621993d-e4d40210.eln* -rwxr-xr-x 1 grfz grfz 103736 Okt 19 12:53 eieio-0db8d1d4-9439d1d2.eln* -rwxr-xr-x 1 grfz grfz 115320 Okt 19 12:54 db-8199ba49-de500a31.eln* -rwxr-xr-x 1 grfz grfz 47408 Okt 19 12:54 ja-dic-cnv-2eacdaa8-b3b103fd.eln* -rwxr-xr-x 1 grfz grfz 48280 Okt 19 12:54 org-macro-6e7b0632-2b02ba5c.eln* -rwxr-xr-x 1 grfz grfz 163496 Okt 19 12:54 ox-texinfo-e762f923-42b317b1.eln* -rwxr-xr-x 1 grfz grfz 163536 Okt 19 12:55 oc-aec59d52-e1541149.eln* -rwxr-xr-x 1 grfz grfz 143912 Okt 19 12:56 ol-1680a4ec-493f104f.eln* -rwxr-xr-x 1 grfz grfz 65832 Okt 19 12:56 cl-lib-8b938900-c76f14d9.eln* -rwxr-xr-x 1 grfz grfz 554632 Okt 19 12:56 ox-9aa46d10-36fd1809.eln* -rwxr-xr-x 1 grfz grfz 450840 Okt 19 12:57 org-element-763f8d74-c2ed5726.eln* -rwxr-xr-x 1 grfz grfz 103448 Okt 19 12:58 align-da164663-36320772.eln* -rwxr-xr-x 1 grfz grfz 136608 Okt 19 12:59 allout-widgets-0bef4edc-759a8d7d.eln* -rwxr-xr-x 1 grfz grfz 4661816 Okt 19 12:59 ja-dic-283bfd77-a5e4a1e7.eln* -rwxr-xr-x 1 grfz grfz 30200 Okt 19 12:59 ansi-osc-b447f6a8-1801ba7c.eln* -rwxr-xr-x 1 grfz grfz 73632 Okt 19 13:00 ansi-color-75eac800-c8aa485f.eln* -rwxr-xr-x 1 grfz grfz 371920 Okt 19 13:01 allout-54433bcc-e0e759ed.eln* -rwxr-xr-x 1 grfz grfz 109488 Okt 19 13:01 apropos-7c1ecbdf-9650b3d6.eln* -rwxr-xr-x 1 grfz grfz 70240 Okt 19 13:02 array-b56b5eac-ec506bd8.eln* -rwxr-xr-x 1 grfz grfz 58064 Okt 19 13:02 auth-source-pass-c498fbad-499c397b.eln* -rwxr-xr-x 1 grfz grfz 42640 Okt 19 13:02 autoinsert-09f12447-b6d4206c.eln* -rwxr-xr-x 1 grfz grfz 79232 Okt 19 13:04 autorevert-841d6890-8b9bcbc0.eln* -rwxr-xr-x 1 grfz grfz 200848 Okt 19 13:04 arc-mode-bac5621c-4ac9d455.eln* -rwxr-xr-x 1 grfz grfz 52040 Okt 19 13:05 avoid-3a50b57d-a616e3f0.eln* -rwxr-xr-x 1 grfz grfz 180984 Okt 19 13:05 auth-source-49df7eef-1d56c61f.eln* -rwxr-xr-x 1 grfz grfz 99000 Okt 19 13:05 battery-ff1f50e8-f1bc4ce1.eln* -rwxr-xr-x 1 grfz grfz 187672 Okt 19 13:05 bookmark-8667481e-cc4ccd04.eln* -rwxr-xr-x 1 grfz grfz 17088 Okt 19 13:05 cdl-b6e4b7af-883d0dac.eln* -rwxr-xr-x 1 grfz grfz 125120 Okt 19 13:05 bs-87de9342-7fb047b4.eln* -rwxr-xr-x 1 grfz grfz 30224 Okt 19 13:05 chistory-7b662674-200867ad.eln* -rwxr-xr-x 1 grfz grfz 52544 Okt 19 13:06 cmuscheme-673be037-c6eb383e.eln* -rwxr-xr-x 1 grfz grfz 372072 Okt 19 13:06 char-fold-b79b1b8c-7ee3c37f.eln* -rwxr-xr-x 1 grfz grfz 136352 Okt 19 13:06 calculator-1eb85010-340dc1ca.eln* -rwxr-xr-x 1 grfz grfz 116184 Okt 19 13:06 completion-a15d2372-8acb394d.eln* -rwxr-xr-x 1 grfz grfz 56264 Okt 19 13:06 color-9d7980a5-8de89906.eln* -rwxr-xr-x 1 grfz grfz 38832 Okt 19 13:06 cus-dep-6d2e8064-6e7c7f8e.eln* -rwxr-xr-x 1 grfz grfz 69456 Okt 19 13:07 cus-theme-6f9d22e7-94be69bb.eln* -rwxr-xr-x 1 grfz grfz 271296 Okt 19 13:07 comint-faef15ad-db425943.eln* -rwxr-xr-x 1 grfz grfz 34944 Okt 19 13:07 delim-col-6e65f4b3-5c6927c7.eln* -rwxr-xr-x 1 grfz grfz 65896 Okt 19 13:07 dabbrev-ed4e5147-48ba83e7.eln* -rwxr-xr-x 1 grfz grfz 34496 Okt 19 13:07 delsel-1c603c42-f9174b7b.eln* -rwxr-xr-x 1 grfz grfz 127184 Okt 19 13:07 desktop-428f9887-5221ebda.eln* -rwxr-xr-x 1 grfz grfz 61928 Okt 19 13:08 dframe-2a07085b-9ec7b9c5.eln* -rwxr-xr-x 1 grfz grfz 89816 Okt 19 13:08 descr-text-4ed9ee33-6e51a3e5.eln* -rwxr-xr-x 1 grfz grfz 348400 Okt 19 13:08 cus-edit-3cd01345-0ff67bb8.eln* -rwxr-xr-x 1 grfz grfz 78192 Okt 19 13:08 dired-x-a2e5c184-b25c9c36.eln* -rwxr-xr-x 1 grfz grfz 30264 Okt 19 13:08 dirtrack-49e0129c-0551af6d.eln* -rwxr-xr-x 1 grfz grfz 30632 Okt 19 13:08 display-fill-column-indicator-82af8525-87ab4f49.eln* -rwxr-xr-x 1 grfz grfz 43720 Okt 19 13:08 display-line-numbers-1d060f2e-ac11bdb2.eln* -rwxr-xr-x 1 grfz grfz 251832 Okt 19 13:09 dired-aux-1ff8c91a-d5f7e763.eln* -rwxr-xr-x 1 grfz grfz 315872 Okt 19 13:09 dired-6a3ae2bc-07da73b7.eln* -rwxr-xr-x 1 grfz grfz 25680 Okt 19 13:09 double-1384b58f-7d071d40.eln* -rwxr-xr-x 1 grfz grfz 47824 Okt 19 13:09 dom-a7939831-14efc4a3.eln* -rwxr-xr-x 1 grfz grfz 21776 Okt 19 13:09 echistory-286d3fb8-4d42b515.eln* -rwxr-xr-x 1 grfz grfz 30576 Okt 19 13:09 ebuff-menu-ecbd6e97-bff4bb07.eln* -rwxr-xr-x 1 grfz grfz 200600 Okt 19 13:09 doc-view-164df236-dea301f5.eln* -rwxr-xr-x 1 grfz grfz 44400 Okt 19 13:09 ecomplete-d9b6d92f-48f548b1.eln* -rwxr-xr-x 1 grfz grfz 44048 Okt 19 13:09 ehelp-a1fe76fc-c81b17aa.eln* -rwxr-xr-x 1 grfz grfz 25760 Okt 19 13:09 elide-head-f64ec400-b3dc6449.eln* -rwxr-xr-x 1 grfz grfz 30560 Okt 19 13:09 emacs-lock-9dfb7362-56bba4df.eln* -rwxr-xr-x 1 grfz grfz 70720 Okt 19 13:09 elec-pair-9d724d9a-cf155de9.eln* -rwxr-xr-x 1 grfz grfz 17328 Okt 19 13:09 epa-dired-774fd12a-c82884e5.eln* -rwxr-xr-x 1 grfz grfz 46904 Okt 19 13:10 edmacro-74bf45a3-b6df9481.eln* -rwxr-xr-x 1 grfz grfz 43568 Okt 19 13:10 epa-file-d15dd0b4-eb0a4471.eln* -rwxr-xr-x 1 grfz grfz 38736 Okt 19 13:10 epa-mail-bf61c0db-10c78409.eln* -rwxr-xr-x 1 grfz grfz 111520 Okt 19 13:10 epa-bdd8ea1c-902eeb9e.eln* -rwxr-xr-x 1 grfz grfz 35072 Okt 19 13:10 epg-config-78240760-6b96d0a3.eln* -rwxr-xr-x 1 grfz grfz 75080 Okt 19 13:10 epa-ks-e5664732-fb52afb2.eln* -rwxr-xr-x 1 grfz grfz 35080 Okt 19 13:10 expand-9d6a83fd-bc169727.eln* -rwxr-xr-x 1 grfz grfz 30280 Okt 19 13:10 ezimage-53d8406d-3f590e41.eln* -rwxr-xr-x 1 grfz grfz 60872 Okt 19 13:10 face-remap-9d6c47ed-ef6ab2ca.eln* -rwxr-xr-x 1 grfz grfz 74272 Okt 19 13:10 facemenu-cceb809d-28feab5d.eln* -rwxr-xr-x 1 grfz grfz 58232 Okt 19 13:11 filecache-c607f1cc-b015ebc1.eln* -rwxr-xr-x 1 grfz grfz 39304 Okt 19 13:11 fileloop-f8cb1238-1dba446f.eln* -rwxr-xr-x 1 grfz grfz 147456 Okt 19 13:11 ffap-4b3c5789-a74be4a4.eln* -rwxr-xr-x 1 grfz grfz 96744 Okt 19 13:11 filenotify-65939d6e-9aac6817.eln* -rwxr-xr-x 1 grfz grfz 83128 Okt 19 13:11 files-x-59c65c89-ab7c86d0.eln* -rwxr-xr-x 1 grfz grfz 404816 Okt 19 13:11 epg-de089247-9e84ee77.eln* -rwxr-xr-x 1 grfz grfz 188904 Okt 19 13:11 filesets-6b603b3e-1859fd55.eln* -rwxr-xr-x 1 grfz grfz 25760 Okt 19 13:11 find-cmd-c1ed7ad8-0eb680ba.eln* -rwxr-xr-x 1 grfz grfz 38920 Okt 19 13:12 find-dired-b47bcf4a-89f704c0.eln* -rwxr-xr-x 1 grfz grfz 56376 Okt 19 13:12 finder-cfd3373c-4f1b537b.eln* -rwxr-xr-x 1 grfz grfz 39544 Okt 19 13:12 find-lisp-7a0199b9-bf8a387b.eln* -rwxr-xr-x 1 grfz grfz 56784 Okt 19 13:12 find-file-29570019-e91c163b.eln* -rwxr-xr-x 1 grfz grfz 17200 Okt 19 13:12 flow-ctrl-a0b1783a-1e34cedb.eln* -rwxr-xr-x 1 grfz grfz 34288 Okt 19 13:12 foldout-a29a272c-443fd9d0.eln* -rwxr-xr-x 1 grfz grfz 25744 Okt 19 13:12 format-spec-644c0068-b9fc233f.eln* -rwxr-xr-x 1 grfz grfz 114856 Okt 19 13:12 follow-6a4faeee-cd78e901.eln* -rwxr-xr-x 1 grfz grfz 87472 Okt 19 13:12 forms-21950d0a-c751d5ec.eln* -rwxr-xr-x 1 grfz grfz 34640 Okt 19 13:12 help-at-pt-24a2e4cf-eb94cf74.eln* -rwxr-xr-x 1 grfz grfz 112216 Okt 19 13:12 generic-x-3d4b3f79-be7d840a.eln* -rwxr-xr-x 1 grfz grfz 25296 Okt 19 13:12 help-macro-b11d1ebc-ce6478ee.eln* -rwxr-xr-x 1 grfz grfz 131032 Okt 19 13:12 frameset-94504960-582f574c.eln* -rwxr-xr-x 1 grfz grfz 74464 Okt 19 13:13 help-mode-d4dbae3d-942d04d2.eln* -rwxr-xr-x 1 grfz grfz 17336 Okt 19 13:13 hex-util-0952f412-6bf54c1d.eln* -rwxr-xr-x 1 grfz grfz 42040 Okt 19 13:13 hfy-cmap-a102c87f-d37fadac.eln* -rwxr-xr-x 1 grfz grfz 101136 Okt 19 13:13 hexl-eddf9831-49318734.eln* -rwxr-xr-x 1 grfz grfz 78080 Okt 19 13:13 hi-lock-42477945-48eeb072.eln* -rwxr-xr-x 1 grfz grfz 74952 Okt 19 13:13 hilit-chg-61a068fb-c53042c7.eln* -rwxr-xr-x 1 grfz grfz 184240 Okt 19 13:13 help-fns-d233c6e8-63a53820.eln* -rwxr-xr-x 1 grfz grfz 43568 Okt 19 13:13 hl-line-8fa29c14-822de8e8.eln* -rwxr-xr-x 1 grfz grfz 70736 Okt 19 13:13 hippie-exp-9919e80b-c63cbd06.eln* -rwxr-xr-x 1 grfz grfz 230144 Okt 19 13:14 ibuf-ext-5624402a-6d6997f3.eln* -rwxr-xr-x 1 grfz grfz 170632 Okt 19 13:14 htmlfontify-accab028-69f1eafb.eln* -rwxr-xr-x 1 grfz grfz 212960 Okt 19 13:14 ibuffer-c2dc0626-63015081.eln* -rwxr-xr-x 1 grfz grfz 50328 Okt 19 13:14 ibuf-macs-eb4b06c3-7628d5cc.eln* -rwxr-xr-x 1 grfz grfz 60864 Okt 19 13:14 ielm-2a8237b7-74589b82.eln* -rwxr-xr-x 1 grfz grfz 25856 Okt 19 13:14 iimage-13f380a2-2ebdaf0d.eln* -rwxr-xr-x 1 grfz grfz 30264 Okt 19 13:14 image-file-9034ac7e-db87497b.eln* -rwxr-xr-x 1 grfz grfz 91728 Okt 19 13:15 icomplete-92f7cbf4-68920226.eln* -rwxr-xr-x 1 grfz grfz 74640 Okt 19 13:15 imenu-a6693d03-0364eafc.eln* -rwxr-xr-x 1 grfz grfz 131976 Okt 19 13:15 image-mode-f854d2db-fcb7dfeb.eln* -rwxr-xr-x 1 grfz grfz 95608 Okt 19 13:15 info-look-27e24920-f99c60b4.eln* -rwxr-xr-x 1 grfz grfz 34088 Okt 19 13:15 informat-c758f251-bf09288c.eln* -rwxr-xr-x 1 grfz grfz 322568 Okt 19 13:16 ido-ccb260dc-a497d40d.eln* -rwxr-xr-x 1 grfz grfz 52040 Okt 19 13:16 info-xref-df600c94-e0fb13e9.eln* -rwxr-xr-x 1 grfz grfz 25848 Okt 19 13:16 isearchb-14a844f5-9cffa663.eln* -rwxr-xr-x 1 grfz grfz 47904 Okt 19 13:16 jka-compr-40ebe92b-873e7826.eln* -rwxr-xr-x 1 grfz grfz 88568 Okt 19 13:16 json-a90a1eab-350c449d.eln* -rwxr-xr-x 1 grfz grfz 21776 Okt 19 13:16 kermit-847c9fbe-db99bcfa.eln* -rwxr-xr-x 1 grfz grfz 101400 Okt 19 13:16 kmacro-048feaec-e2ae5d33.eln* -rwxr-xr-x 1 grfz grfz 118800 Okt 19 13:16 jsonrpc-e62a9c36-4ac0ad99.eln* -rwxr-xr-x 1 grfz grfz 39072 Okt 19 13:16 loadhist-e9175ed1-a9ab5961.eln* -rwxr-xr-x 1 grfz grfz 331600 Okt 19 13:17 info-ce12c0ca-4d447d17.eln* -rwxr-xr-x 1 grfz grfz 34904 Okt 19 13:17 lpr-06c2a39c-b453b930.eln* -rwxr-xr-x 1 grfz grfz 56936 Okt 19 13:17 locate-bbb85898-15b7a742.eln* -rwxr-xr-x 1 grfz grfz 25976 Okt 19 13:17 master-d6d42554-90ec46ea.eln* -rwxr-xr-x 1 grfz grfz 25808 Okt 19 13:17 macros-51ef7b0a-141955fb.eln* -rwxr-xr-x 1 grfz grfz 21416 Okt 19 13:17 mb-depth-a4691d0b-2aa9169a.eln* -rwxr-xr-x 1 grfz grfz 34616 Okt 19 13:17 midnight-0df4540a-a34daf80.eln* -rwxr-xr-x 1 grfz grfz 30048 Okt 19 13:17 minibuf-eldef-8b1407ef-3edcda03.eln* -rwxr-xr-x 1 grfz grfz 30488 Okt 19 13:17 misc-9bdb6d96-71987c28.eln* -rwxr-xr-x 1 grfz grfz 46776 Okt 19 13:17 md4-234ce043-38a388a1.eln* -rwxr-xr-x 1 grfz grfz 43920 Okt 19 13:17 misearch-3d1286b0-5dfbae2a.eln* -rwxr-xr-x 1 grfz grfz 21512 Okt 19 13:17 mouse-copy-747f314d-a0716908.eln* -rwxr-xr-x 1 grfz grfz 134360 Okt 19 13:17 man-9b8001be-e10db098.eln* -rwxr-xr-x 1 grfz grfz 30272 Okt 19 13:17 mouse-drag-b34287d2-5cd220d2.eln* -rwxr-xr-x 1 grfz grfz 42408 Okt 19 13:17 notifications-bd0b4b34-4d586541.eln* -rwxr-xr-x 1 grfz grfz 83968 Okt 19 13:18 msb-e477e33b-5de02d4c.eln* -rwxr-xr-x 1 grfz grfz 25648 Okt 19 13:18 novice-cc2ef463-3555311a.eln* -rwxr-xr-x 1 grfz grfz 21696 Okt 19 13:18 password-cache-187e4eec-58743954.eln* -rwxr-xr-x 1 grfz grfz 35432 Okt 19 13:18 pcmpl-cvs-743b5ebc-1b2226ef.eln* -rwxr-xr-x 1 grfz grfz 26824 Okt 19 13:18 pcmpl-git-794ef7f3-f17db3ff.eln* -rwxr-xr-x 1 grfz grfz 132416 Okt 19 13:18 outline-afc41f82-860c9481.eln* -rwxr-xr-x 1 grfz grfz 63816 Okt 19 13:18 pcmpl-gnu-5ece415b-b97b3f4c.eln* -rwxr-xr-x 1 grfz grfz 39752 Okt 19 13:18 pcmpl-linux-060b988a-b0dca077.eln* -rwxr-xr-x 1 grfz grfz 220336 Okt 19 13:18 mpc-ebb105de-612804e4.eln* -rwxr-xr-x 1 grfz grfz 63840 Okt 19 13:18 pcmpl-rpm-d791f76b-92b51ed3.eln* -rwxr-xr-x 1 grfz grfz 89448 Okt 19 13:18 pcmpl-unix-e208196d-5c6b0b5f.eln* -rwxr-xr-x 1 grfz grfz 49544 Okt 19 13:18 pcmpl-x-c16436ad-e19e302a.eln* -rwxr-xr-x 1 grfz grfz 53456 Okt 19 13:19 plstore-f04d6393-38883ed2.eln* -rwxr-xr-x 1 grfz grfz 79416 Okt 19 13:19 pixel-scroll-2f9465ae-242e22e4.eln* -rwxr-xr-x 1 grfz grfz 123792 Okt 19 13:19 pcomplete-81dbd8b0-8d6a7537.eln* -rwxr-xr-x 1 grfz grfz 152680 Okt 19 13:19 proced-8fac17dd-0ddbe361.eln* -rwxr-xr-x 1 grfz grfz 130952 Okt 19 13:19 profiler-345cb85e-b5357a8a.eln* -rwxr-xr-x 1 grfz grfz 322056 Okt 19 13:19 printing-8558eaeb-ac24b34d.eln* -rwxr-xr-x 1 grfz grfz 48480 Okt 19 13:20 ps-bdf-e53407b2-3787d715.eln* -rwxr-xr-x 1 grfz grfz 26760 Okt 19 13:20 ps-samp-f57360e6-bbf3fa10.eln* -rwxr-xr-x 1 grfz grfz 94872 Okt 19 13:20 ps-mule-2e9a6c45-d7e24a5c.eln* -rwxr-xr-x 1 grfz grfz 133832 Okt 19 13:20 recentf-3c64dc62-3d5e44b7.eln* -rwxr-xr-x 1 grfz grfz 60328 Okt 19 13:20 registry-bca51796-80e1f6d0.eln* -rwxr-xr-x 1 grfz grfz 94080 Okt 19 13:20 rect-cd288962-0c863cb3.eln* -rwxr-xr-x 1 grfz grfz 52096 Okt 19 13:20 repeat-443831f9-f727a7bb.eln* -rwxr-xr-x 1 grfz grfz 25536 Okt 19 13:20 reposition-5c59bc1d-ea0cf33c.eln* -rwxr-xr-x 1 grfz grfz 34288 Okt 19 13:20 reveal-65b32910-8009efbd.eln* -rwxr-xr-x 1 grfz grfz 21456 Okt 19 13:20 rot13-82cc11d3-341fbb00.eln* -rwxr-xr-x 1 grfz grfz 320752 Okt 19 13:20 ps-print-74a80610-186b597f.eln* -rwxr-xr-x 1 grfz grfz 34848 Okt 19 13:20 rtree-8e3a7d8e-bea31fc8.eln* -rwxr-xr-x 1 grfz grfz 39056 Okt 19 13:21 savehist-b722b772-8de510f4.eln* -rwxr-xr-x 1 grfz grfz 43528 Okt 19 13:21 saveplace-a17120d6-0b02506a.eln* -rwxr-xr-x 1 grfz grfz 26280 Okt 19 13:21 scroll-all-b76288a5-c97066ae.eln* -rwxr-xr-x 1 grfz grfz 26064 Okt 19 13:21 scroll-lock-7af2dd31-80b5e08c.eln* -rwxr-xr-x 1 grfz grfz 65328 Okt 19 13:21 ruler-mode-3c3fd53f-df5fcb4c.eln* -rwxr-xr-x 1 grfz grfz 97672 Okt 19 13:21 shadowfile-a07f492d-dcf2798b.eln* -rwxr-xr-x 1 grfz grfz 131296 Okt 19 13:21 server-0cc44189-7d051c39.eln* -rwxr-xr-x 1 grfz grfz 46928 Okt 19 13:21 skeleton-435acb71-fdc776de.eln* -rwxr-xr-x 1 grfz grfz 109504 Okt 19 13:21 so-long-a68117bf-f2a3256b.eln* -rwxr-xr-x 1 grfz grfz 141176 Okt 19 13:22 shell-57930f41-a0615147.eln* -rwxr-xr-x 1 grfz grfz 56360 Okt 19 13:22 sort-14dd51e7-e6a8ba97.eln* -rwxr-xr-x 1 grfz grfz 17032 Okt 19 13:22 soundex-1b2d2ac2-ba002f59.eln* -rwxr-xr-x 1 grfz grfz 17024 Okt 19 13:22 sqlite-351fdac0-27a5a873.eln* -rwxr-xr-x 1 grfz grfz 34784 Okt 19 13:22 sqlite-mode-83559d5a-767317c6.eln* -rwxr-xr-x 1 grfz grfz 117424 Okt 19 13:23 strokes-19dc35b2-fd9cd395.eln* -rwxr-xr-x 1 grfz grfz 53976 Okt 19 13:23 svg-4c391752-1deded8e.eln* -rwxr-xr-x 1 grfz grfz 263440 Okt 19 13:23 speedbar-2a9b6d1b-a500b18c.eln* -rwxr-xr-x 1 grfz grfz 17120 Okt 19 13:23 tabify-b74f3a50-9f5191d5.eln* -rwxr-xr-x 1 grfz grfz 25864 Okt 19 13:23 talk-dfb156fa-5efb1587.eln* -rwxr-xr-x 1 grfz grfz 98064 Okt 19 13:23 tab-line-e55f541b-044b2bf8.eln* -rwxr-xr-x 1 grfz grfz 61128 Okt 19 13:23 tempo-5756037f-5c44c637.eln* -rwxr-xr-x 1 grfz grfz 136552 Okt 19 13:24 tar-mode-8829ee02-03a59aac.eln* -rwxr-xr-x 1 grfz grfz 368256 Okt 19 13:24 ses-d640128c-4ea056d7.eln* -rwxr-xr-x 1 grfz grfz 30824 Okt 19 13:24 thread-1b59497f-5823bb9c.eln* -rwxr-xr-x 1 grfz grfz 62344 Okt 19 13:24 thingatpt-6fc8a4ab-5c620eb5.eln* -rwxr-xr-x 1 grfz grfz 61416 Okt 19 13:24 time-bbe7023e-418f30bd.eln* -rwxr-xr-x 1 grfz grfz 34968 Okt 19 13:24 timezone-f5339c31-838485ba.eln* -rwxr-xr-x 1 grfz grfz 51744 Okt 19 13:24 tmm-12a7d710-bd9f1f90.eln* -rwxr-xr-x 1 grfz grfz 55280 Okt 19 13:24 time-stamp-20b00c46-359942ab.eln* -rwxr-xr-x 1 grfz grfz 21624 Okt 19 13:24 t-mouse-b934bcc3-6ea766b8.eln* -rwxr-xr-x 1 grfz grfz 61016 Okt 19 13:24 tree-widget-8dccf6ba-aa31b64f.eln* -rwxr-xr-x 1 grfz grfz 262400 Okt 19 13:25 term-fb034019-5137e438.eln* -rwxr-xr-x 1 grfz grfz 60104 Okt 19 13:25 tutorial-2418da58-0d5c21f4.eln* -rwxr-xr-x 1 grfz grfz 26464 Okt 19 13:25 userlock-60e3749c-fd2130e2.eln* -rwxr-xr-x 1 grfz grfz 96448 Okt 19 13:25 type-break-1b71d0e6-73d5d80f.eln* -rwxr-xr-x 1 grfz grfz 70336 Okt 19 13:25 vcursor-f24573d4-c99a7c93.eln* -rwxr-xr-x 1 grfz grfz 74856 Okt 19 13:25 view-faefc6b2-9336d0e6.eln* -rwxr-xr-x 1 grfz grfz 79464 Okt 19 13:25 wdired-01202d01-ece5bc7d.eln* -rwxr-xr-x 1 grfz grfz 34728 Okt 19 13:25 wid-browse-e182a231-e688a13d.eln* -rwxr-xr-x 1 grfz grfz 147024 Okt 19 13:26 whitespace-48717ff3-11a121a8.eln* -rwxr-xr-x 1 grfz grfz 423304 Okt 19 13:26 transient-376febf1-21ae079d.eln* -rwxr-xr-x 1 grfz grfz 71272 Okt 19 13:26 windmove-15827e24-5e608432.eln* -rwxr-xr-x 1 grfz grfz 48536 Okt 19 13:26 winner-c3a8b092-b4847128.eln* -rwxr-xr-x 1 grfz grfz 48072 Okt 19 13:26 xdg-9947111f-7737cb26.eln* -rwxr-xr-x 1 grfz grfz 64824 Okt 19 13:26 xml-78b9f1ba-ca346c9b.eln* -rwxr-xr-x 1 grfz grfz 283392 Okt 19 13:27 wid-edit-5b92861a-20de1749.eln* -rwxr-xr-x 1 grfz grfz 246696 Okt 19 13:27 woman-468654b8-ce2d4a59.eln* -rwxr-xr-x 1 grfz grfz 48080 Okt 19 13:27 xt-mouse-ad97d877-6cd66896.eln* -rwxr-xr-x 1 grfz grfz 26080 Okt 19 13:27 yank-media-62540c94-43b16516.eln* -rwxr-xr-x 1 grfz grfz 125792 Okt 19 13:27 xwidget-9ccb93b3-c50a90c3.eln* -rwxr-xr-x 1 grfz grfz 98920 Okt 19 13:27 calc-aent-1719b1cd-cb2f991d.eln* -rwxr-xr-x 1 grfz grfz 174488 Okt 19 13:29 calcalg3-cdd43fbd-eb7bd16b.eln* -rwxr-xr-x 1 grfz grfz 233368 Okt 19 13:29 calc-alg-5fa19fcf-e239539a.eln* -rwxr-xr-x 1 grfz grfz 83360 Okt 19 13:29 calc-bin-61359665-90237c69.eln* -rwxr-xr-x 1 grfz grfz 338824 Okt 19 13:29 calcalg2-3641d50d-b28bb50c.eln* -rwxr-xr-x 1 grfz grfz 117920 Okt 19 13:30 calc-comb-4d223239-ac552c55.eln* -rwxr-xr-x 1 grfz grfz 48312 Okt 19 13:30 calc-cplx-49f3d288-e0e82bea.eln* -rwxr-xr-x 1 grfz grfz 128984 Okt 19 13:31 calccomp-91a23141-f1acbb70.eln* -rwxr-xr-x 1 grfz grfz 257304 Okt 19 13:31 calc-222b057e-99d16d60.eln* -rwxr-xr-x 1 grfz grfz 104080 Okt 19 13:32 calc-embed-12840cac-ab2cd86f.eln* -rwxr-xr-x 1 grfz grfz 296280 Okt 19 13:32 calc-arith-97da4592-ca2c9343.eln* -rwxr-xr-x 1 grfz grfz 57840 Okt 19 13:33 calc-fin-5313d12c-5ee336ed.eln* -rwxr-xr-x 1 grfz grfz 47816 Okt 19 13:33 calc-frac-13a225bf-a45c3aeb.eln* -rwxr-xr-x 1 grfz grfz 268584 Okt 19 13:34 calc-ext-169a1473-fd44d6be.eln* -rwxr-xr-x 1 grfz grfz 108632 Okt 19 13:34 calc-funcs-41ddedba-66c010cd.eln* -rwxr-xr-x 1 grfz grfz 57576 Okt 19 13:35 calc-help-f82e4a83-c16aaedb.eln* -rwxr-xr-x 1 grfz grfz 34936 Okt 19 13:35 calc-incom-bb13bc57-abb51ab2.eln* -rwxr-xr-x 1 grfz grfz 130000 Okt 19 13:35 calc-graph-f953862d-c98a959c.eln* -rwxr-xr-x 1 grfz grfz 47248 Okt 19 13:35 calc-keypd-5235a410-ba679b1c.eln* -rwxr-xr-x 1 grfz grfz 236264 Okt 19 13:35 calc-forms-0364e2fe-ae86f85d.eln* -rwxr-xr-x 1 grfz grfz 43776 Okt 19 13:36 calc-macs-86f6acaa-ee00de71.eln* -rwxr-xr-x 1 grfz grfz 164760 Okt 19 13:36 calc-lang-77b67574-226c90a4.eln* -rwxr-xr-x 1 grfz grfz 57912 Okt 19 13:36 calc-menu-43d1e6da-d07b5d5b.eln* -rwxr-xr-x 1 grfz grfz 103912 Okt 19 13:36 calc-map-cec1a5a6-9f53b206.eln* -rwxr-xr-x 1 grfz grfz 79000 Okt 19 13:37 calc-misc-0f75b984-58211cbd.eln* -rwxr-xr-x 1 grfz grfz 99392 Okt 19 13:37 calc-mode-3448000a-f206306e.eln* -rwxr-xr-x 1 grfz grfz 215728 Okt 19 13:37 calc-math-5c62dc12-adaa1128.eln* -rwxr-xr-x 1 grfz grfz 51672 Okt 19 13:37 calc-mtx-709af3c9-4c36bd74.eln* -rwxr-xr-x 1 grfz grfz 74928 Okt 19 13:37 calc-nlfit-a260e11b-76d3530d.eln* -rwxr-xr-x 1 grfz grfz 120592 Okt 19 13:38 calc-poly-ff0ca9b3-b3108f6a.eln* -rwxr-xr-x 1 grfz grfz 34120 Okt 19 13:38 calc-rules-162a6cbc-b28539a3.eln* -rwxr-xr-x 1 grfz grfz 42872 Okt 19 13:39 calcsel2-45ef433d-de378d55.eln* -rwxr-xr-x 1 grfz grfz 96872 Okt 19 13:40 calc-sel-92dc8d3c-64be2a50.eln* -rwxr-xr-x 1 grfz grfz 198312 Okt 19 13:40 calc-prog-78e4aeb2-f714efaf.eln* -rwxr-xr-x 1 grfz grfz 80528 Okt 19 13:40 calc-store-a2093015-9b25d5d8.eln* -rwxr-xr-x 1 grfz grfz 66408 Okt 19 13:40 calc-stat-bb7f9833-6d8af6dd.eln* -rwxr-xr-x 1 grfz grfz 26824 Okt 19 13:40 calc-trail-e8bfa5bd-9a0f4a7d.eln* -rwxr-xr-x 1 grfz grfz 30184 Okt 19 13:40 calc-undo-21bf6c23-e7fcd3b8.eln* -rwxr-xr-x 1 grfz grfz 43384 Okt 19 13:40 calc-stuff-781739a1-23a9165d.eln* -rwxr-xr-x 1 grfz grfz 163016 Okt 19 13:40 calc-rewr-0c4bb3e0-58e58a9f.eln* -rwxr-xr-x 1 grfz grfz 74448 Okt 19 13:40 calc-yank-a4aa7301-975f7821.eln* -rwxr-xr-x 1 grfz grfz 64624 Okt 19 13:41 appt-6933a307-3f500bce.eln* -rwxr-xr-x 1 grfz grfz 177520 Okt 19 13:41 calc-vec-8d9dd57c-3c676778.eln* -rwxr-xr-x 1 grfz grfz 189112 Okt 19 13:41 calc-units-12fe779d-de3ff8ab.eln* -rwxr-xr-x 1 grfz grfz 56288 Okt 19 13:41 cal-bahai-caf89889-51cab3a9.eln* -rwxr-xr-x 1 grfz grfz 47984 Okt 19 13:42 cal-coptic-d7417646-2da80eb6.eln* -rwxr-xr-x 1 grfz grfz 68608 Okt 19 13:42 cal-dst-f85a5a15-9c95e4aa.eln* -rwxr-xr-x 1 grfz grfz 108440 Okt 19 13:42 cal-china-28218d30-7928f9a2.eln* -rwxr-xr-x 1 grfz grfz 51904 Okt 19 13:42 cal-french-443613b5-72ea1e9a.eln* -rwxr-xr-x 1 grfz grfz 69624 Okt 19 13:43 cal-html-84e76775-ddb1ca5a.eln* -rwxr-xr-x 1 grfz grfz 256024 Okt 19 13:43 calendar-d19e5c14-866d5903.eln* -rwxr-xr-x 1 grfz grfz 34512 Okt 19 13:43 cal-iso-8ac36734-fc91459c.eln* -rwxr-xr-x 1 grfz grfz 60712 Okt 19 13:43 cal-islam-74de7ee1-4a919494.eln* -rwxr-xr-x 1 grfz grfz 39216 Okt 19 13:43 cal-julian-969ba5f7-1c57247a.eln* -rwxr-xr-x 1 grfz grfz 30088 Okt 19 13:43 cal-menu-9380b697-a0f24163.eln* -rwxr-xr-x 1 grfz grfz 64816 Okt 19 13:44 cal-move-aed18331-70622161.eln* -rwxr-xr-x 1 grfz grfz 153336 Okt 19 13:44 cal-hebrew-32b77d47-480945fd.eln* -rwxr-xr-x 1 grfz grfz 69664 Okt 19 13:44 cal-mayan-43c35bcf-42168f0b.eln* -rwxr-xr-x 1 grfz grfz 21696 Okt 19 13:44 cal-x-8d62ebf5-7d7cc605.eln* -rwxr-xr-x 1 grfz grfz 34736 Okt 19 13:44 cal-persia-39d88853-f564a483.eln* -rwxr-xr-x 1 grfz grfz 101760 Okt 19 13:46 holidays-39d09682-859a6888.eln* -rwxr-xr-x 1 grfz grfz 212920 Okt 19 13:46 cal-tex-9f624404-454ba4e9.eln* -rwxr-xr-x 1 grfz grfz 38912 Okt 19 13:47 iso8601-3903451a-7b67c90d.eln* -rwxr-xr-x 1 grfz grfz 166456 Okt 19 13:47 icalendar-3066204f-9fec696b.eln* -rwxr-xr-x 1 grfz grfz 34864 Okt 19 13:47 parse-time-ca7017be-39be3991.eln* -rwxr-xr-x 1 grfz grfz 58944 Okt 19 13:47 lunar-791aa94f-a3620369.eln* -rwxr-xr-x 1 grfz grfz 264280 Okt 19 13:47 diary-lib-57afc63e-19a7cbdf.eln* -rwxr-xr-x 1 grfz grfz 60192 Okt 19 13:48 time-date-40951a48-f2fbd30f.eln* -rwxr-xr-x 1 grfz grfz 131584 Okt 19 13:48 timeclock-a2d57d06-bbbfe488.eln* -rwxr-xr-x 1 grfz grfz 26128 Okt 19 13:48 cedet-cscope-87f0c476-c02142a9.eln* -rwxr-xr-x 1 grfz grfz 21152 Okt 19 13:48 cedet-e5d89324-af54da04.eln* -rwxr-xr-x 1 grfz grfz 17352 Okt 19 13:48 cedet-files-ec97a64b-70698494.eln* -rwxr-xr-x 1 grfz grfz 26424 Okt 19 13:49 cedet-global-4ae0bafb-c5909a7c.eln* -rwxr-xr-x 1 grfz grfz 26344 Okt 19 13:49 cedet-idutils-e5eca828-03569772.eln* -rwxr-xr-x 1 grfz grfz 111560 Okt 19 13:49 solar-41ca7795-7b23192e.eln* -rwxr-xr-x 1 grfz grfz 77136 Okt 19 13:49 data-debug-01abd8ab-23637260.eln* -rwxr-xr-x 1 grfz grfz 142040 Okt 19 13:49 ede-5a8a99f9-3c1866b0.eln* -rwxr-xr-x 1 grfz grfz 26232 Okt 19 13:49 pulse-35e729a5-4385997e.eln* -rwxr-xr-x 1 grfz grfz 96112 Okt 19 13:49 mode-local-52e88da7-e3e6b204.eln* -rwxr-xr-x 1 grfz grfz 21504 Okt 19 13:49 srecode-f9a0bb35-8d1e660d.eln* -rwxr-xr-x 1 grfz grfz 100024 Okt 19 13:49 semantic-0e6d83f5-373d236b.eln* -rwxr-xr-x 1 grfz grfz 43736 Okt 19 13:50 autoconf-edit-9fac18a2-11513ca0.eln* -rwxr-xr-x 1 grfz grfz 43472 Okt 19 13:50 auto-aad53e20-8f2ff3bd.eln* -rwxr-xr-x 1 grfz grfz 75416 Okt 19 13:50 base-064e31be-14a3d5c4.eln* -rwxr-xr-x 1 grfz grfz 80216 Okt 19 13:50 config-5e70a6cb-c9e05f95.eln* -rwxr-xr-x 1 grfz grfz 57064 Okt 19 13:50 cpp-root-435af696-63bfb39a.eln* -rwxr-xr-x 1 grfz grfz 43816 Okt 19 13:50 custom-d88895eb-e6050aad.eln* -rwxr-xr-x 1 grfz grfz 30952 Okt 19 13:50 detect-c9d7c2b3-95c0f88a.eln* -rwxr-xr-x 1 grfz grfz 30232 Okt 19 13:50 dired-e18206ac-bec97729.eln* -rwxr-xr-x 1 grfz grfz 53192 Okt 19 13:50 emacs-e09048e1-793049b3.eln* -rwxr-xr-x 1 grfz grfz 61720 Okt 19 13:50 files-9f30c50f-3fd74907.eln* -rwxr-xr-x 1 grfz grfz 66408 Okt 19 13:50 linux-eedacb0e-0742f0d4.eln* -rwxr-xr-x 1 grfz grfz 67008 Okt 19 13:51 generic-c019b5ab-d72bf563.eln* -rwxr-xr-x 1 grfz grfz 442640 Okt 19 13:51 todo-mode-595bef2f-75c9cbb3.eln* -rwxr-xr-x 1 grfz grfz 21296 Okt 19 13:51 make-a4d1a972-dbe8735f.eln* -rwxr-xr-x 1 grfz grfz 21936 Okt 19 13:51 makefile-edit-92f94c53-b029ddab.eln* -rwxr-xr-x 1 grfz grfz 66856 Okt 19 13:51 locate-63ce93dd-49c06f90.eln* -rwxr-xr-x 1 grfz grfz 39136 Okt 19 13:51 pconf-e1039d7f-2f89b351.eln* -rwxr-xr-x 1 grfz grfz 34784 Okt 19 13:51 proj-archive-931a920a-7ded753a.eln* -rwxr-xr-x 1 grfz grfz 30408 Okt 19 13:51 proj-aux-6dbd327a-d98fcf8c.eln* -rwxr-xr-x 1 grfz grfz 61856 Okt 19 13:51 proj-comp-2f18b2f6-4d3fc736.eln* -rwxr-xr-x 1 grfz grfz 91488 Okt 19 13:51 pmake-41620d85-b1c90f4f.eln* -rwxr-xr-x 1 grfz grfz 74856 Okt 19 13:51 proj-elisp-9926ccad-66a68f91.eln* -rwxr-xr-x 1 grfz grfz 91888 Okt 19 13:51 proj-3b75ff59-dbc45196.eln* -rwxr-xr-x 1 grfz grfz 52096 Okt 19 13:51 proj-info-e54aa80a-96a42632.eln* -rwxr-xr-x 1 grfz grfz 147144 Okt 19 13:51 project-am-8c1ba5b1-579cb35a.eln* -rwxr-xr-x 1 grfz grfz 30600 Okt 19 13:51 proj-misc-cd3625c9-e7df458d.eln* -rwxr-xr-x 1 grfz grfz 26216 Okt 19 13:51 proj-scheme-e19dcdf2-503c4a73.eln* -rwxr-xr-x 1 grfz grfz 48352 Okt 19 13:52 proj-obj-160a1567-cfdde3d4.eln* -rwxr-xr-x 1 grfz grfz 52352 Okt 19 13:52 proj-prog-104d733c-dba5db4c.eln* -rwxr-xr-x 1 grfz grfz 43248 Okt 19 13:52 proj-shared-e6b4fd1d-1d145605.eln* -rwxr-xr-x 1 grfz grfz 26200 Okt 19 13:52 shell-2dd33f2c-bff6236d.eln* -rwxr-xr-x 1 grfz grfz 31048 Okt 19 13:52 simple-b70c3268-12362a7a.eln* -rwxr-xr-x 1 grfz grfz 30536 Okt 19 13:52 source-a030dba0-865d485b.eln* -rwxr-xr-x 1 grfz grfz 53072 Okt 19 13:52 speedbar-be64e296-c298692a.eln* -rwxr-xr-x 1 grfz grfz 25880 Okt 19 13:52 srecode-5803a755-b642c232.eln* -rwxr-xr-x 1 grfz grfz 30560 Okt 19 13:52 system-7bc2aed2-791db77e.eln* -rwxr-xr-x 1 grfz grfz 30616 Okt 19 13:52 util-fac6ec1c-20e119f0.eln* -rwxr-xr-x 1 grfz grfz 43168 Okt 19 13:52 bovine-2f253aba-d7a490d5.eln* -rwxr-xr-x 1 grfz grfz 39632 Okt 19 13:52 chart-1df0f00d-4cbc69db.eln* -rwxr-xr-x 1 grfz grfz 114520 Okt 19 13:52 analyze-0736102d-0e404dae.eln* -rwxr-xr-x 1 grfz grfz 26768 Okt 19 13:52 db-debug-1b7ffd95-14eb2e22.eln* -rwxr-xr-x 1 grfz grfz 221576 Okt 19 13:53 complete-2b67d14d-529c6816.eln* -rwxr-xr-x 1 grfz grfz 75904 Okt 19 13:53 db-el-4fc170b9-afb4defb.eln* -rwxr-xr-x 1 grfz grfz 101848 Okt 19 13:53 db-ebrowse-54480d61-c988f53d.eln* -rwxr-xr-x 1 grfz grfz 97920 Okt 19 13:53 ctxt-dc25695a-389f480a.eln* -rwxr-xr-x 1 grfz grfz 65416 Okt 19 13:53 db-file-3009064d-efaeeedc.eln* -rwxr-xr-x 1 grfz grfz 62392 Okt 19 13:53 db-javascript-88df227b-61de1a13.eln* -rwxr-xr-x 1 grfz grfz 57832 Okt 19 13:53 db-global-c899a3cf-bb0095c0.eln* -rwxr-xr-x 1 grfz grfz 39320 Okt 19 13:53 db-mode-28bbbf2a-b043681a.eln* -rwxr-xr-x 1 grfz grfz 43264 Okt 19 13:53 db-ref-1cbbe2c8-10294282.eln* -rwxr-xr-x 1 grfz grfz 71016 Okt 19 13:53 debug-1b989c3f-790d7b9d.eln* -rwxr-xr-x 1 grfz grfz 138160 Okt 19 13:53 db-find-f00ee9ff-4fd341c2.eln* -rwxr-xr-x 1 grfz grfz 44968 Okt 19 13:53 decorate-059cc323-065b8f90.eln* -rwxr-xr-x 1 grfz grfz 79440 Okt 19 13:54 db-typecache-81d46fa2-c186a322.eln* -rwxr-xr-x 1 grfz grfz 30552 Okt 19 13:54 dep-77b6feec-52556097.eln* -rwxr-xr-x 1 grfz grfz 34824 Okt 19 13:54 doc-4efe007d-9aad7d0e.eln* -rwxr-xr-x 1 grfz grfz 56728 Okt 19 13:54 ede-grammar-8cc19dfb-913e4127.eln* -rwxr-xr-x 1 grfz grfz 97816 Okt 19 13:54 find-03e42e4a-bd0cc84a.eln* -rwxr-xr-x 1 grfz grfz 73752 Okt 19 13:54 edit-359026ad-0390b97c.eln* -rwxr-xr-x 1 grfz grfz 48088 Okt 19 13:54 fw-fda2d4f2-e46232f6.eln* -rwxr-xr-x 1 grfz grfz 101864 Okt 19 13:55 format-ebbc6838-9a6b7ccf.eln* -rwxr-xr-x 1 grfz grfz 44200 Okt 19 13:55 html-0dfe9b31-0cd90921.eln* -rwxr-xr-x 1 grfz grfz 56712 Okt 19 13:55 ia-6745d2f4-caa6a23a.eln* -rwxr-xr-x 1 grfz grfz 233848 Okt 19 13:55 grammar-7a8b1aed-883a55b3.eln* -rwxr-xr-x 1 grfz grfz 61480 Okt 19 13:55 ia-sb-f610bf29-bf8b45ec.eln* -rwxr-xr-x 1 grfz grfz 274488 Okt 19 13:55 grm-wy-boot-72e0b49d-872491e8.eln* -rwxr-xr-x 1 grfz grfz 53344 Okt 19 13:56 imenu-a1821d64-6331a67c.eln* -rwxr-xr-x 1 grfz grfz 80072 Okt 19 13:56 java-888c7d7a-3b86345f.eln* -rwxr-xr-x 1 grfz grfz 158872 Okt 19 13:56 idle-ddea1a59-ac68dd67.eln* -rwxr-xr-x 1 grfz grfz 65544 Okt 19 13:56 mru-bookmark-392dffc3-349f8408.eln* -rwxr-xr-x 1 grfz grfz 61512 Okt 19 13:56 sb-0ac49e47-5600e7b5.eln* -rwxr-xr-x 1 grfz grfz 126808 Okt 19 13:56 lex-spp-7e2ae6b1-99c6bfed.eln* -rwxr-xr-x 1 grfz grfz 193640 Okt 19 13:56 lex-ce219d01-54a1050c.eln* -rwxr-xr-x 1 grfz grfz 102920 Okt 19 13:58 senator-8eaf8c42-875ae1b7.eln* -rwxr-xr-x 1 grfz grfz 91808 Okt 19 13:58 scope-3d34b0cd-7a18b846.eln* -rwxr-xr-x 1 grfz grfz 85136 Okt 19 13:58 sort-cdd6576f-c7778ba3.eln* -rwxr-xr-x 1 grfz grfz 66064 Okt 19 13:58 symref-6c63732c-db778e72.eln* -rwxr-xr-x 1 grfz grfz 43344 Okt 19 13:58 tag-file-17a3ab0a-bab06b4e.eln* -rwxr-xr-x 1 grfz grfz 43632 Okt 19 13:59 tag-write-5b235879-8b8b5c70.eln* -rwxr-xr-x 1 grfz grfz 167176 Okt 19 14:00 tag-baa603ad-e2302d3a.eln* -rwxr-xr-x 1 grfz grfz 71176 Okt 19 14:00 texi-b4fe6646-53621ac0.eln* -rwxr-xr-x 1 grfz grfz 52280 Okt 19 14:01 util-f6c26979-d4c0f66c.eln* -rwxr-xr-x 1 grfz grfz 74800 Okt 19 14:01 tag-ls-7f8a4456-0f2fc2ff.eln* -rwxr-xr-x 1 grfz grfz 42856 Okt 19 14:02 wisent-786cee7a-9eb33ebf.eln* -rwxr-xr-x 1 grfz grfz 108912 Okt 19 14:02 util-modes-835f6a2c-502a1dc6.eln* -rwxr-xr-x 1 grfz grfz 43128 Okt 19 14:02 complete-8049fcd7-27a541b2.eln* -rwxr-xr-x 1 grfz grfz 47800 Okt 19 14:03 fcn-9a27a4a6-e051d56e.eln* -rwxr-xr-x 1 grfz grfz 64848 Okt 19 14:03 debug-e2a68966-df0ca671.eln* -rwxr-xr-x 1 grfz grfz 56936 Okt 19 14:03 refs-bbf70991-389b9d4b.eln* > . start Emacs > . use 'top' or similar command to see if async compilations are > running which were started by Emacs you run above > . compare the files these async compilations are compiling with what > you see in that latest subdirectory of eln-cache K) This is a list of probably all the files from processes like this one: /home/grfz/src/emacs-master-next/src/emacs --batch -l /tmp/emacs-async-comp-.......el /tmp/emacs-async-comp-ansi-color-pUV33H.el /tmp/emacs-async-comp-ansi-osc-9uy5Z7.el /tmp/emacs-async-comp-async-bytecomp-19BQgz.el /tmp/emacs-async-comp-async-CG3661.el /tmp/emacs-async-comp-auth-source-nLKdu9.el /tmp/emacs-async-comp-avoid-qTHnAl.el /tmp/emacs-async-comp-battery-nR8yrq.el /tmp/emacs-async-comp-bbdb-wNjNnU.el /tmp/emacs-async-comp-bind-key-nT82iR.el /tmp/emacs-async-comp-bookmark-NaKH2G.el /tmp/emacs-async-comp-bytecomp-cOfm1Y.el /tmp/emacs-async-comp-byte-opt-D13On8.el /tmp/emacs-async-comp-calendar-nLweLU.el /tmp/emacs-async-comp-calfw-cal-112gxS.el /tmp/emacs-async-comp-calfw-HgNycP.el /tmp/emacs-async-comp-calfw-ical-1eRC8T.el /tmp/emacs-async-comp-calfw-org-xpzBat.el /tmp/emacs-async-comp-cal-menu-uzBSOE.el /tmp/emacs-async-comp-cconv-XgTAiT.el /tmp/emacs-async-comp-cedet-wW0nUH.el /tmp/emacs-async-comp-cl-lib-NwHLAC.el /tmp/emacs-async-comp-comint-6TAyYC.el /tmp/emacs-async-comp-comp-cstr-yIgiNj.el /tmp/emacs-async-comp-comp-qSvZwH.el /tmp/emacs-async-comp-coolj-PLHerE.el /tmp/emacs-async-comp-cus-edit-lLi8uj.el /tmp/emacs-async-comp-cus-start-THOIFJ.el /tmp/emacs-async-comp-delsel-Qt9mj1.el /tmp/emacs-async-comp-dframe-3hG8KX.el /tmp/emacs-async-comp-diary-lib-a5R3pl.el /tmp/emacs-async-comp-dired-91E78r.el /tmp/emacs-async-comp-dired-async-bbDW10.el /tmp/emacs-async-comp-dired-aux-QNUzoL.el /tmp/emacs-async-comp-doc-view-9gaxyb.el /tmp/emacs-async-comp-dom-zYf21G.el /tmp/emacs-async-comp-easy-repeat-M5fqHV.el /tmp/emacs-async-comp-edmacro-M69ewc.el /tmp/emacs-async-comp-eieio-xPFMh0.el /tmp/emacs-async-comp-epa-jlpMNe.el /tmp/emacs-async-comp-epg-cav6Av.el /tmp/emacs-async-comp-epg-config-6MLDpu.el /tmp/emacs-async-comp-ezimage-cePW4l.el /tmp/emacs-async-comp-fileloop-w6xkto.el /tmp/emacs-async-comp-filenotify-qrZxPs.el /tmp/emacs-async-comp-files-x-wlf2T4.el /tmp/emacs-async-comp-fix-word-k0pNwM.el /tmp/emacs-async-comp-format-spec-IIwMhG.el /tmp/emacs-async-comp-fw-SVCTHV.el /tmp/emacs-async-comp-gcmh-ICqagW.el /tmp/emacs-async-comp-gnus-alias-DSIgPD.el /tmp/emacs-async-comp-helm-adaptive-bc0euJ.el /tmp/emacs-async-comp-helm-buffers-c1OBZP.el /tmp/emacs-async-comp-helm-core-1L6l6w.el /tmp/emacs-async-comp-helm-descbinds-nBVJcV.el /tmp/emacs-async-comp-helm-elisp-9O0NSM.el /tmp/emacs-async-comp-helm-eshell-jMCK3z.el /tmp/emacs-async-comp-helm-eval-5xmr5q.el /tmp/emacs-async-comp-helm-files-ivUhiA.el /tmp/emacs-async-comp-helm-grep-cKmbH7.el /tmp/emacs-async-comp-helm-help-vUbDHI.el /tmp/emacs-async-comp-helm-info-J7puEc.el /tmp/emacs-async-comp-helm-lib-Z1mots.el /tmp/emacs-async-comp-helm-locate-fTFYrR.el /tmp/emacs-async-comp-helm-misc-BIuWXI.el /tmp/emacs-async-comp-helm-mode-dVvqxf.el /tmp/emacs-async-comp-helm-multi-match-iQguey.el /tmp/emacs-async-comp-helm-occur-H54Zxt.el /tmp/emacs-async-comp-helm-regexp-FpbDC4.el /tmp/emacs-async-comp-helm-source-qY20or.el /tmp/emacs-async-comp-helm-tags-vWu94v.el /tmp/emacs-async-comp-helm-types-2bHjYR.el /tmp/emacs-async-comp-helm-utils-GHnDEU.el /tmp/emacs-async-comp-help-mode-yErcaX.el /tmp/emacs-async-comp-hl-line-gBxrcO.el /tmp/emacs-async-comp-holidays-zSdSGL.el /tmp/emacs-async-comp-ibuf-ext-c7AUbB.el /tmp/emacs-async-comp-ibuffer-PFWaNZ.el /tmp/emacs-async-comp-icalendar-TTHuqR.el /tmp/emacs-async-comp-image-mode-laEAEj.el /tmp/emacs-async-comp-imenu-oKJvjT.el /tmp/emacs-async-comp-info-nUAJJc.el /tmp/emacs-async-comp-iso8601-EmlH4S.el /tmp/emacs-async-comp-jka-compr-Ncejiv.el /tmp/emacs-async-comp-json-Qm0QzY.el /tmp/emacs-async-comp-keychain-environment-9bE1aC.el /tmp/emacs-async-comp-key-chord-NfeT60.el /tmp/emacs-async-comp-kmacro-74hYo4.el /tmp/emacs-async-comp-lex-qSD2QJ.el /tmp/emacs-async-comp-ls-lisp-n35tqw.el /tmp/emacs-async-comp-minibuffer-line-k4Hcwv.el /tmp/emacs-async-comp-mode-local-EXHmzg.el /tmp/emacs-async-comp-notmuch-address-yI72IX.el /tmp/emacs-async-comp-notmuch-company-wIfqpI.el /tmp/emacs-async-comp-notmuch-compat-Yv03rR.el /tmp/emacs-async-comp-notmuch-crypto-j48lnj.el /tmp/emacs-async-comp-notmuch-draft-qoOWho.el /tmp/emacs-async-comp-notmuch-hello-q93ftT.el /tmp/emacs-async-comp-notmuch-jump-FrkZJd.el /tmp/emacs-async-comp-notmuch-lib-iAd5zR.el /tmp/emacs-async-comp-notmuch-maildir-fcc-3XMcqn.el /tmp/emacs-async-comp-notmuch-message-CGYlRv.el /tmp/emacs-async-comp-notmuch-Mk1MXH.el /tmp/emacs-async-comp-notmuch-mua-OfK8KY.el /tmp/emacs-async-comp-notmuch-parser-W3d0cV.el /tmp/emacs-async-comp-notmuch-print-AKP3L3.el /tmp/emacs-async-comp-notmuch-show-xhy5jM.el /tmp/emacs-async-comp-notmuch-tag-ty2SaC.el /tmp/emacs-async-comp-notmuch-tree-GIkIm4.el /tmp/emacs-async-comp-notmuch-wash-3bNXz4.el /tmp/emacs-async-comp-ob-comint-leu52R.el /tmp/emacs-async-comp-ob-core-9IF6w2.el /tmp/emacs-async-comp-ob-emacs-lisp-NHr2JM.el /tmp/emacs-async-comp-ob-eval-kdn2CF.el /tmp/emacs-async-comp-ob-exp-uRjsnv.el /tmp/emacs-async-comp-ob-lob-JlPplD.el /tmp/emacs-async-comp-ob-plantuml-iaKkPY.el /tmp/emacs-async-comp-ob-ref-IDHirv.el /tmp/emacs-async-comp-ob-table-NcWkqD.el /tmp/emacs-async-comp-ob-tangle-xvWX3j.el /tmp/emacs-async-comp-oc-gL0Q7V.el /tmp/emacs-async-comp-ol-bbdb-tTsEgm.el /tmp/emacs-async-comp-ol-docview-RNEmM8.el /tmp/emacs-async-comp-ol-eshell-GB2uD4.el /tmp/emacs-async-comp-ol-eww-u1naaW.el /tmp/emacs-async-comp-ol-info-LlKsUV.el /tmp/emacs-async-comp-ol-man-OyKHv2.el /tmp/emacs-async-comp-ol-xHgouj.el /tmp/emacs-async-comp-org-agenda-RnA3rh.el /tmp/emacs-async-comp-org-capture-wQjIom.el /tmp/emacs-async-comp-org-clock-kof9Tb.el /tmp/emacs-async-comp-org-compat-A1486e.el /tmp/emacs-async-comp-org-crypt-NnnZHp.el /tmp/emacs-async-comp-org-ctags-KCGK3l.el /tmp/emacs-async-comp-org-cycle-P3ekty.el /tmp/emacs-async-comp-org-element-nxRVDJ.el /tmp/emacs-async-comp-org-entities-5MCpLH.el /tmp/emacs-async-comp-org-faces-3V9ztT.el /tmp/emacs-async-comp-org-fold-core-kk4sHQ.el /tmp/emacs-async-comp-org-fold-LL85YZ.el /tmp/emacs-async-comp-org-footnote-5bxu5j.el /tmp/emacs-async-comp-org-habit-bsn3xf.el /tmp/emacs-async-comp-org-id-LCwa1b.el /tmp/emacs-async-comp-org-inlinetask-BAJGxw.el /tmp/emacs-async-comp-org-keys-vyNR55.el /tmp/emacs-async-comp-org-list-EMyWLz.el /tmp/emacs-async-comp-org-macro-yNuUMT.el /tmp/emacs-async-comp-org-macs-9LcYMN.el /tmp/emacs-async-comp-org-mouse-PVr4YJ.el /tmp/emacs-async-comp-org-pcomplete-8LTxe1.el /tmp/emacs-async-comp-org-persist-wTY1ec.el /tmp/emacs-async-comp-org-protocol-CstUr4.el /tmp/emacs-async-comp-org-refile-kn1Mqg.el /tmp/emacs-async-comp-org-src-a1yMg3.el /tmp/emacs-async-comp-org-table-ctto0o.el /tmp/emacs-async-comp-org-tempo-RQxJZZ.el /tmp/emacs-async-comp-org-vDvm4j.el /tmp/emacs-async-comp-outline-XIF8wK.el /tmp/emacs-async-comp-parse-time-o7UU9I.el /tmp/emacs-async-comp-password-cache-hZWu1i.el /tmp/emacs-async-comp-pcomplete-3HhI5i.el /tmp/emacs-async-comp-pdf-cache-5aOEpV.el /tmp/emacs-async-comp-pdf-info-DnL1so.el /tmp/emacs-async-comp-pdf-isearch-ZLzJ0i.el /tmp/emacs-async-comp-pdf-misc-RROccj.el /tmp/emacs-async-comp-pdf-occur-FDc2Zt.el /tmp/emacs-async-comp-pdf-tools-iz18hS.el /tmp/emacs-async-comp-pdf-util-XdVNac.el /tmp/emacs-async-comp-pdf-view-LLPvNl.el /tmp/emacs-async-comp-rainbow-delimiters-q5cqNt.el /tmp/emacs-async-comp-savehist-3lwnL5.el /tmp/emacs-async-comp-saveplace-WAXkTV.el /tmp/emacs-async-comp-semantic-t4FM26.el /tmp/emacs-async-comp-server-JA6jnJ.el /tmp/emacs-async-comp-shell-R4LOWu.el /tmp/emacs-async-comp-speedbar-6xZINu.el /tmp/emacs-async-comp-ssh-deploy-oPA4CT.el /tmp/emacs-async-comp-svg-Ud8wup.el /tmp/emacs-async-comp-tablist-filter-kVfrYJ.el /tmp/emacs-async-comp-tablist-Wcu4Bo.el /tmp/emacs-async-comp-tag-cdvAIV.el /tmp/emacs-async-comp-tempo-d1jp5D.el /tmp/emacs-async-comp-thingatpt-75vfGI.el /tmp/emacs-async-comp-time-date-jmTl2i.el /tmp/emacs-async-comp-time-stamp-HHkvFL.el /tmp/emacs-async-comp-timezone-8snfx1.el /tmp/emacs-async-comp-use-package-bind-key-IEkGne.el /tmp/emacs-async-comp-use-package-core-NTc8be.el /tmp/emacs-async-comp-use-package-delight-VkxJon.el /tmp/emacs-async-comp-use-package-diminish-171jJ0.el /tmp/emacs-async-comp-use-package-ensure-wghU4i.el /tmp/emacs-async-comp-util-a74ohp.el /tmp/emacs-async-comp-util-modes-dJtQaT.el /tmp/emacs-async-comp-wcheck-mode-Xg92bL.el /tmp/emacs-async-comp-which-key-MS9s2R.el /tmp/emacs-async-comp-wid-edit-RDFM4w.el /tmp/emacs-async-comp-windmove-S0EC8E.el /tmp/emacs-async-comp-winner-ntjK5a.el /tmp/emacs-async-comp-wisent-wY2J2F.el /tmp/emacs-async-comp-ws-butler-Jie5Lx.el /tmp/emacs-async-comp-xdg-qvyYQs.el /tmp/emacs-async-comp-xml-lU4Geh.el /tmp/emacs-async-comp-xt-mouse-GiY2sT.el /tmp/emacs-async-comp-yank-media-QYt4Fg.el > . show the list of files that were compiled although they already > had corresponding *.eln files in that subdirectory All the files from the list to your last question already have a corresponding file in the third to last eln cache directory (see J)). E.g. /tmp/emacs-async-comp-yank-media-QYt4Fg.el corresponds to yank-media-62540c94-43b16516.eln > . show the list of files that were compiled although they already > had corresponding *.eln files in that subdirectory see K) above > . let the compilations run to completion (use 'top' to see when > there are no more compilations running) > . look in the *Async-native-compile-log* buffer to see if there are > any warning or error messages there; if there are, post them L) There are no warnings or error messages, only this: Compiling /home/grfz/src/emacs-master-next/lisp/wid-edit.el... Compiling /home/grfz/src/emacs-master-next/lisp/emacs-lisp/cl-lib.el... Compiling /home/grfz/src/emacs-master-next/lisp/cus-start.el... Compiling /home/grfz/src/emacs-master-next/lisp/cus-edit.el... Compiling /home/grfz/src/emacs-master-next/lisp/epg-config.el... Compiling /home/grfz/src/emacs-master-next/lisp/epg.el... Compiling /home/grfz/src/emacs-master-next/lisp/emacs-lisp/cconv.el... Compiling /home/grfz/src/emacs-master-next/lisp/emacs-lisp/bytecomp.el... Compiling /home/grfz/src/emacs-master-next/lisp/emacs-lisp/byte-opt.el... Compiling /home/grfz/src/emacs-master-next/lisp/json.el... Compiling /home/grfz/src/emacs-master-next/lisp/password-cache.el... Compiling /home/grfz/src/emacs-master-next/lisp/emacs-lisp/eieio.el... Compiling /home/grfz/src/emacs-master-next/lisp/auth-source.el... Compiling /home/grfz/src/emacs-master-next/lisp/calendar/time-date.el... Compiling /home/grfz/src/emacs-master-next/lisp/epa.el... Compiling /home/grfz/src/emacs-master-next/lisp/help-mode.el... Compiling /home/grfz/src/emacs-master-next/lisp/info.el... Compiling /home/grfz/.config/emacs/elpa-29.0/fix-word-20210319.1414/fix-word.el... Compiling /home/grfz/.config/emacs/elpa-29.0/use-package-20221012.1743/use-package-core.el... Compiling /home/grfz/.config/emacs/elpa-29.0/bind-key-20220910.2157/bind-key.el... Compiling /home/grfz/.config/emacs/elpa-29.0/use-package-20221012.1743/use-package-bind-key.el... Compiling /home/grfz/.config/emacs/elpa-29.0/use-package-20221012.1743/use-package-diminish.el... Compiling /home/grfz/.config/emacs/elpa-29.0/use-package-20221012.1743/use-package-delight.el... Compiling /home/grfz/.config/emacs/elpa-29.0/use-package-20221012.1743/use-package-ensure.el... Compiling /home/grfz/src/emacs-master-next/lisp/delsel.el... Compiling /home/grfz/src/emacs-master-next/lisp/dired.el... Compiling /home/grfz/src/emacs-master-next/lisp/dired-aux.el... Compiling /home/grfz/.config/emacs/elpa-29.0/async-20221014.2225/async.el... Compiling /home/grfz/.config/emacs/elpa-29.0/async-20221014.2225/dired-async.el... Compiling /home/grfz/src/emacs-master-next/lisp/xml.el... Compiling /home/grfz/src/emacs-master-next/lisp/battery.el... Compiling /home/grfz/.config/emacs/elpa-29.0/minibuffer-line-0.1/minibuffer-line.el... Compiling /home/grfz/src/emacs-master-next/lisp/avoid.el... Compiling /home/grfz/src/emacs-master-next/lisp/savehist.el... Compiling /home/grfz/src/emacs-master-next/lisp/format-spec.el... Compiling /home/grfz/src/org-mode/lisp/org-macs.el... Compiling /home/grfz/src/org-mode/lisp/ob-eval.el... Compiling /home/grfz/src/org-mode/lisp/org-compat.el... Compiling /home/grfz/src/org-mode/lisp/org-fold-core.el... Compiling /home/grfz/src/org-mode/lisp/org-fold.el... Compiling /home/grfz/src/org-mode/lisp/org-cycle.el... Compiling /home/grfz/src/org-mode/lisp/ob-core.el... Compiling /home/grfz/src/emacs-master-next/lisp/ansi-color.el... Compiling /home/grfz/src/emacs-master-next/lisp/ansi-osc.el... Compiling /home/grfz/src/emacs-master-next/lisp/comint.el... Compiling /home/grfz/src/org-mode/lisp/ob-comint.el... Compiling /home/grfz/src/org-mode/lisp/oc.el... Compiling /home/grfz/src/org-mode/lisp/org-keys.el... Compiling /home/grfz/src/org-mode/lisp/org-src.el... Compiling /home/grfz/src/org-mode/lisp/ol.el... Compiling /home/grfz/src/org-mode/lisp/ob-tangle.el... Compiling /home/grfz/src/emacs-master-next/lisp/calendar/cal-menu.el... Compiling /home/grfz/src/emacs-master-next/lisp/calendar/calendar.el... Compiling /home/grfz/src/org-mode/lisp/org-table.el... Compiling /home/grfz/src/org-mode/lisp/org.el... Compiling /home/grfz/src/org-mode/lisp/ob-emacs-lisp.el... Compiling /home/grfz/src/org-mode/lisp/ob-exp.el... Compiling /home/grfz/src/org-mode/lisp/ob-table.el... Compiling /home/grfz/src/org-mode/lisp/ob-lob.el... Compiling /home/grfz/src/org-mode/lisp/ob-ref.el... Compiling /home/grfz/src/org-mode/lisp/ob-plantuml.el... Compiling /home/grfz/src/emacs-master-next/lisp/outline.el... Compiling /home/grfz/src/org-mode/lisp/org-entities.el... Compiling /home/grfz/src/org-mode/lisp/org-faces.el... Compiling /home/grfz/src/org-mode/lisp/org-list.el... Compiling /home/grfz/src/emacs-master-next/lisp/pcomplete.el... Compiling /home/grfz/src/org-mode/lisp/org-pcomplete.el... Compiling /home/grfz/src/org-mode/lisp/org-footnote.el... Compiling /home/grfz/src/org-mode/lisp/org-macro.el... Compiling /home/grfz/.config/emacs/elpa-29.0/key-chord-20201222.2030/key-chord.el... Compiling /home/grfz/src/emacs-master-next/lisp/emacs-lisp/comp-cstr.el... Compiling /home/grfz/src/emacs-master-next/lisp/emacs-lisp/comp.el... Compiling /home/grfz/.config/emacs/elpa-29.0/gcmh-20201116.2251/gcmh.el... Compiling /home/grfz/src/org-mode/lisp/ol-bbdb.el... Compiling /home/grfz/src/org-mode/lisp/org-crypt.el... Compiling /home/grfz/src/org-mode/lisp/org-ctags.el... Compiling /home/grfz/src/emacs-master-next/lisp/image-mode.el... Compiling /home/grfz/src/emacs-master-next/lisp/jka-compr.el... Compiling /home/grfz/src/emacs-master-next/lisp/filenotify.el... Compiling /home/grfz/src/emacs-master-next/lisp/doc-view.el... Compiling /home/grfz/src/org-mode/lisp/ol-docview.el... Compiling /home/grfz/src/emacs-master-next/lisp/yank-media.el... Compiling /home/grfz/src/emacs-master-next/lisp/dom.el... Compiling /home/grfz/src/emacs-master-next/lisp/svg.el... Compiling /home/grfz/src/emacs-master-next/lisp/thingatpt.el... Compiling /home/grfz/src/emacs-master-next/lisp/xdg.el... Compiling /home/grfz/src/org-mode/lisp/ol-eww.el... Compiling /home/grfz/src/org-mode/lisp/org-refile.el... Compiling /home/grfz/src/org-mode/lisp/org-agenda.el... Compiling /home/grfz/src/org-mode/lisp/org-habit.el... Compiling /home/grfz/src/org-mode/lisp/org-id.el... Compiling /home/grfz/src/org-mode/lisp/ol-info.el... Compiling /home/grfz/src/org-mode/lisp/org-inlinetask.el... Compiling /home/grfz/src/org-mode/lisp/org-mouse.el... Compiling /home/grfz/src/org-mode/lisp/org-protocol.el... Compiling /home/grfz/src/emacs-master-next/lisp/files-x.el... Compiling /home/grfz/src/org-mode/lisp/ol-eshell.el... Compiling /home/grfz/src/org-mode/lisp/ol-man.el... Compiling /home/grfz/src/emacs-master-next/lisp/hl-line.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-compat.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-lib.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-tag.el... Compiling /home/grfz/src/emacs-master-next/lisp/calendar/diary-lib.el... Compiling /home/grfz/src/emacs-master-next/lisp/calendar/icalendar.el... Compiling /home/grfz/src/notmuch/emacs/coolj.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-wash.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-company.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-parser.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-address.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-maildir-fcc.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-draft.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-message.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-mua.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-crypto.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-show.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-print.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-jump.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-hello.el... Compiling /home/grfz/src/notmuch/emacs/notmuch-tree.el... Compiling /home/grfz/src/notmuch/emacs/notmuch.el... Compiling /home/grfz/src/emacs-master-next/lisp/tempo.el... Compiling /home/grfz/src/org-mode/lisp/org-tempo.el... Compiling /home/grfz/src/org-mode/lisp/org-persist.el... Compiling /home/grfz/src/org-mode/lisp/org-element.el... Compiling /home/grfz/src/emacs-master-next/lisp/edmacro.el... Compiling /home/grfz/src/emacs-master-next/lisp/kmacro.el... Compiling /home/grfz/.config/emacs/elpa-29.0/bbdb-20220706.433/bbdb.el... Compiling /home/grfz/src/emacs-master-next/lisp/timezone.el... Compiling /home/grfz/src/emacs-master-next/lisp/fileloop.el... Compiling /home/grfz/.config/emacs/elpa-29.0/gnus-alias-20150316.42/gnus-alias.el... Compiling /home/grfz/src/emacs-master-next/lisp/imenu.el... Compiling /home/grfz/src/emacs-master-next/lisp/windmove.el... Compiling /home/grfz/src/emacs-master-next/lisp/xt-mouse.el... Compiling /home/grfz/.config/emacs/elpa-29.0/pdf-tools-20221007.1404/pdf-util.el... Compiling /home/grfz/.config/emacs/elpa-29.0/pdf-tools-20221007.1404/pdf-info.el... Compiling /home/grfz/.config/emacs/elpa-29.0/pdf-tools-20221007.1404/pdf-cache.el... Compiling /home/grfz/src/emacs-master-next/lisp/bookmark.el... Compiling /home/grfz/.config/emacs/elpa-29.0/pdf-tools-20221007.1404/pdf-view.el... Compiling /home/grfz/.config/emacs/elpa-29.0/pdf-tools-20221007.1404/pdf-tools.el... Compiling /home/grfz/.config/emacs/elpa-29.0/pdf-tools-20221007.1404/pdf-misc.el... Compiling /home/grfz/.config/emacs/elpa-29.0/pdf-tools-20221007.1404/pdf-isearch.el... Compiling /home/grfz/src/emacs-master-next/lisp/cedet/cedet.el... Compiling /home/grfz/src/emacs-master-next/lisp/cedet/mode-local.el... Compiling /home/grfz/src/emacs-master-next/lisp/cedet/semantic/fw.el... Compiling /home/grfz/src/emacs-master-next/lisp/cedet/semantic/lex.el... Compiling /home/grfz/src/emacs-master-next/lisp/cedet/semantic/tag.el... Compiling /home/grfz/src/emacs-master-next/lisp/cedet/semantic.el... Compiling /home/grfz/src/emacs-master-next/lisp/cedet/semantic/util.el... Compiling /home/grfz/src/emacs-master-next/lisp/cedet/semantic/util-modes.el... Compiling /home/grfz/src/emacs-master-next/lisp/cedet/semantic/wisent.el... Compiling /home/grfz/.config/emacs/elpa-29.0/tablist-20200427.2205/tablist-filter.el... Compiling /home/grfz/.config/emacs/elpa-29.0/tablist-20200427.2205/tablist.el... Compiling /home/grfz/src/emacs-master-next/lisp/ibuffer.el... Compiling /home/grfz/src/emacs-master-next/lisp/ibuf-ext.el... Compiling /home/grfz/.config/emacs/elpa-29.0/pdf-tools-20221007.1404/pdf-occur.el... Compiling /home/grfz/.config/emacs/elpa-29.0/keychain-environment-20180318.2223/keychain-environment.el... Compiling /home/grfz/src/emacs-master-next/lisp/saveplace.el... Compiling /home/grfz/.config/emacs/elpa-29.0/wcheck-mode-2021/wcheck-mode.el... Compiling /home/grfz/.config/emacs/elpa-29.0/ws-butler-20201117.1528/ws-butler.el... Compiling /home/grfz/.config/emacs/elpa-29.0/ssh-deploy-20220126.658/ssh-deploy.el... Compiling /home/grfz/src/org-mode/lisp/org-clock.el... Compiling /home/grfz/src/emacs-master-next/lisp/dframe.el... Compiling /home/grfz/src/emacs-master-next/lisp/ezimage.el... Compiling /home/grfz/src/emacs-master-next/lisp/speedbar.el... Compiling /home/grfz/src/emacs-master-next/lisp/calendar/holidays.el... Compiling /home/grfz/.config/emacs/elpa-29.0/calfw-20180118.45/calfw.el... Compiling /home/grfz/src/org-mode/lisp/org-capture.el... Compiling /home/grfz/.config/emacs/elpa-29.0/calfw-org-20160303.258/calfw-org.el... Compiling /home/grfz/.config/emacs/elpa-29.0/calfw-cal-20170320.1206/calfw-cal.el... Compiling /home/grfz/.config/emacs/elpa-29.0/calfw-ical-20150703.819/calfw-ical.el... Compiling /home/grfz/.config/emacs/elpa-29.0/which-key-20220811.1616/which-key.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-core-20221020.1530/helm-lib.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-core-20221020.1530/helm-multi-match.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-core-20221020.1530/helm-source.el... Compiling /home/grfz/.config/emacs/elpa-29.0/async-20221014.2225/async-bytecomp.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-core-20221020.1530/helm-core.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-types.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-help.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-utils.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-regexp.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-grep.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-locate.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-tags.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-occur.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-buffers.el... Compiling /home/grfz/src/emacs-master-next/lisp/ls-lisp.el... Compiling /home/grfz/src/emacs-master-next/lisp/calendar/iso8601.el... Compiling /home/grfz/src/emacs-master-next/lisp/calendar/parse-time.el... Compiling /home/grfz/src/emacs-master-next/lisp/shell.el... Compiling /home/grfz/src/emacs-master-next/lisp/time-stamp.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-files.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-misc.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-mode.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-adaptive.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-info.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-eval.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-elisp.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-eshell.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-descbinds-20190501.935/helm-descbinds.el... Compiling /home/grfz/src/emacs-master-next/lisp/winner.el... Compiling /home/grfz/.config/emacs/elpa-29.0/rainbow-delimiters-20210515.1254/rainbow-delimiters.el... Compiling /home/grfz/src/emacs-master-next/lisp/server.el... Compiling /home/grfz/.config/emacs/elpa-29.0/easy-repeat-20150516.848/easy-repeat.el... Compilation finished. Compiling /home/grfz/src/emacs-master-next/lisp/international/mule-util.el... Compiling /home/grfz/src/emacs-master-next/lisp/ecomplete.el... Compiling /home/grfz/.config/emacs/elpa-29.0/orgalist-1.13/orgalist.el... Compilation finished. Compiling /home/grfz/src/emacs-master-next/lisp/misearch.el... Compilation finished. Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-net.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-external.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-bookmark.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-x-files.el... Compiling /home/grfz/.config/emacs/elpa-29.0/helm-20221020.1512/helm-for-files.el... Compiling /home/grfz/src/emacs-master-next/lisp/tree-widget.el... Compiling /home/grfz/src/emacs-master-next/lisp/recentf.el... Compiling /home/grfz/src/emacs-master-next/lisp/ffap.el... Compilation finished. > . kill Emacs > . look into that subdirectory of the eln-cache and see whether there > are any new *.eln files in it as result of the compilation There is no file from today. > One reason why Emacs could keep compiling files that already have > *.eln files in the cache is because the existing *.eln files are > outdated or incompatible with the Emacs binary. This happens a lot if > you are using the master branch, because we make significant changes > there frequently, and they require recompilation. You will in those > cases see *.eln files whose base names are the same, but the hashes > are different. This is not the case here since I did not fetch from master since days. Ciao; Gregor -- -... --- .-. . -.. ..--.. ...-.- ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term)
@ 2022-10-23 15:22 Gregor Zattler
2022-10-23 16:07 ` Eli Zaretskii
2022-10-23 17:39 ` Eli Zaretskii
0 siblings, 2 replies; 343+ messages in thread
From: Gregor Zattler @ 2022-10-23 15:22 UTC (permalink / raw)
To: Eli Zaretskii, Sean Whitton; +Cc: rlb, monnier, david, emacs-devel, akrl
Hi Eli,
* Eli Zaretskii <eliz@gnu.org> [2022-10-03; 19:19 +03]:
[... native compilation and laptop battery ..]
> You see, I have my data, and it seems to indicate a very short period
> of initial compilations for a modest consumption of resources. (This
> isn't a laptop, but I'm acutely aware of my system's load at all
> times, and have it on the mode line, so I know what kind of
> computation is going on during that time.) The data I see here
> doesn't look like it should be a problem for anyone, as long as files
> are compiled only as they are used. In my case, for example, I see
> maybe half a dozen *.eln files compiled after the initial startup. I
> expect to see that on any machine where a user has steady usage
> patterns -- the compilation flats out very quickly.
Strangely, that is not what I see. Instead every time I
start Emacs (since a week or so) like this
cd ~/src/emacs-master-next/src; gdb emacs -ex 'set logging file /tmp/gdb.txt' -ex 'set logging on' -ex 'set logging file /tmp/gdb.txt' -ex 'run --fg-daemon'
after roughly 30 seconds I see asynchronous compilation
processes which last at least 120 to most 150 seconds.
While this is the case in htop these compilation processes
are most of the time the top most cpu consumers, in powertop
I see several emacs processes are consuming power right
after the devices of my laptop.
What this means in actual power consumption I cannot say.
What is strange is, that these compilation processes start
over and over again, every time I start Emacs even if I do
not install/upgrade Emacs, nor packages.
I just did this three times in a row and it's a consistent
pattern.
This is GNU Emacs 29.0.50 (build 2, x86_64-pc-linux-gnu,
cairo version 1.16.0) of 2022-10-19
on
Debian bullseye right now with a backported kernel: Linux no
5.18.0-0.deb11.4-amd64 #1 SMP PREEMPT_DYNAMIC Debian
5.18.16-1~bpo11+1 (2022-08-12) x86_64 GNU/Linux
Ciao; Gregor
--
-... --- .-. . -.. ..--.. ...-.-
^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-23 15:22 Gregor Zattler @ 2022-10-23 16:07 ` Eli Zaretskii 2022-10-23 17:08 ` Gregor Zattler 2022-10-23 17:39 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-23 16:07 UTC (permalink / raw) To: Gregor Zattler; +Cc: spwhitton, rlb, monnier, david, emacs-devel, akrl > From: Gregor Zattler <telegraph@gmx.net> > Cc: rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, > emacs-devel@gnu.org, akrl@sdf.org > Date: Sun, 23 Oct 2022 17:22:06 +0200 > > Hi Eli, > * Eli Zaretskii <eliz@gnu.org> [2022-10-03; 19:19 +03]: > [... native compilation and laptop battery ..] > > You see, I have my data, and it seems to indicate a very short period > > of initial compilations for a modest consumption of resources. (This > > isn't a laptop, but I'm acutely aware of my system's load at all > > times, and have it on the mode line, so I know what kind of > > computation is going on during that time.) The data I see here > > doesn't look like it should be a problem for anyone, as long as files > > are compiled only as they are used. In my case, for example, I see > > maybe half a dozen *.eln files compiled after the initial startup. I > > expect to see that on any machine where a user has steady usage > > patterns -- the compilation flats out very quickly. > > Strangely, that is not what I see. Instead every time I > start Emacs (since a week or so) like this > > cd ~/src/emacs-master-next/src; gdb emacs -ex 'set logging file /tmp/gdb.txt' -ex 'set logging on' -ex 'set logging file /tmp/gdb.txt' -ex 'run --fg-daemon' > > after roughly 30 seconds I see asynchronous compilation > processes which last at least 120 to most 150 seconds. > While this is the case in htop these compilation processes > are most of the time the top most cpu consumers, in powertop > I see several emacs processes are consuming power right > after the devices of my laptop. Please tell the details: which files are being compiled, and whether do you see the corresponding *.eln files in the eln-cache afterwards. Once a .eln file is in the eln-cache, the next invocation should not re-compile the same file, it should load the compiled .eln file from the eln-cache. Could it be that your eln-cache is deleted between sessions? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-23 16:07 ` Eli Zaretskii @ 2022-10-23 17:08 ` Gregor Zattler 2022-10-23 17:30 ` Eli Zaretskii 2022-10-23 17:33 ` Gregor Zattler 0 siblings, 2 replies; 343+ messages in thread From: Gregor Zattler @ 2022-10-23 17:08 UTC (permalink / raw) To: Eli Zaretskii; +Cc: spwhitton, rlb, monnier, david, emacs-devel, akrl Hi Eli, * Eli Zaretskii <eliz@gnu.org> [2022-10-23; 19:07 +03]: >> From: Gregor Zattler <telegraph@gmx.net> >> Cc: rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, >> emacs-devel@gnu.org, akrl@sdf.org >> Date: Sun, 23 Oct 2022 17:22:06 +0200 >> Strangely, that is not what I see. Instead every time I >> start Emacs (since a week or so) like this >> >> cd ~/src/emacs-master-next/src; gdb emacs -ex 'set logging file /tmp/gdb.txt' -ex 'set logging on' -ex 'set logging file /tmp/gdb.txt' -ex 'run --fg-daemon' >> >> after roughly 30 seconds I see asynchronous compilation >> processes which last at least 120 to most 150 seconds. >> While this is the case in htop these compilation processes >> are most of the time the top most cpu consumers, in powertop >> I see several emacs processes are consuming power right >> after the devices of my laptop. > > Please tell the details: which files are being compiled, and whether > do you see the corresponding *.eln files in the eln-cache afterwards. > Once a .eln file is in the eln-cache, the next invocation should not > re-compile the same file, it should load the compiled .eln file from > the eln-cache. perhaps I misinterpreted and what I saw was the testing of the .eln cache?: I did find "/home/grfz/.config/emacs/eln-cache/" "/home/grfz/src/emacs-master-next/native-lisp/" -type f | xargs -r ls -Altr > /tmp/before <start Emacs again> while sleep 1 ; do ps -eo pid,tty,stat,user,group,etime,time,cgroup,args fax >> faxme; done # while emacs started find "/home/grfz/.config/emacs/eln-cache/" "/home/grfz/src/emacs-master-next/native-lisp/" -type f | xargs -r ls -Altr > /tmp/before now $ diff -aNur before after $ --> No new .eln files were written. But file faxme contains 280 lines like this: /home/grfz/src/emacs-master-next/src/emacs --batch -l /tmp/emacs-async-comp-cl-lib-7Go9da.el 0 grfz@no:/tmp$ grep -o 'emacs-async-comp-.*' faxme | cut -d - -f 4- | sed -e "s/-[^-]*\.el$//" |sort -u | while read ; do grep -qs $REPLY before || echo $REPLY; done 0 grfz@no:/tmp$ So all these "emacs-async-comp-cl-lib-7Go9da.el" like files have corresponding files in the .eln cache. Is it possible that it takes 150 secs to test the .eln cache? > Could it be that your eln-cache is deleted between sessions? no. Ciao; Gregor -- -... --- .-. . -.. ..--.. ...-.- ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-23 17:08 ` Gregor Zattler @ 2022-10-23 17:30 ` Eli Zaretskii 2022-10-23 17:33 ` Gregor Zattler 1 sibling, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-23 17:30 UTC (permalink / raw) To: Gregor Zattler; +Cc: emacs-devel, akrl > From: Gregor Zattler <telegraph@gmx.net> > Cc: spwhitton@spwhitton.name, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, > akrl@sdf.org > Date: Sun, 23 Oct 2022 19:08:23 +0200 > > > Please tell the details: which files are being compiled, and whether > > do you see the corresponding *.eln files in the eln-cache afterwards. > > Once a .eln file is in the eln-cache, the next invocation should not > > re-compile the same file, it should load the compiled .eln file from > > the eln-cache. > > perhaps I misinterpreted and what I saw was the testing of > the .eln cache?: > > I did > > find "/home/grfz/.config/emacs/eln-cache/" "/home/grfz/src/emacs-master-next/native-lisp/" -type f | xargs -r ls -Altr > /tmp/before > > <start Emacs again> > > while sleep 1 ; do ps -eo pid,tty,stat,user,group,etime,time,cgroup,args fax >> faxme; done # while emacs started > > find "/home/grfz/.config/emacs/eln-cache/" "/home/grfz/src/emacs-master-next/native-lisp/" -type f | xargs -r ls -Altr > /tmp/before ^^^^^^ I guess you mean "after", not "before"? > But file faxme contains 280 lines like this: > > /home/grfz/src/emacs-master-next/src/emacs --batch -l /tmp/emacs-async-comp-cl-lib-7Go9da.el And there's already a cl-lib-XXXXX.eln file under eln-cache? > 0 grfz@no:/tmp$ grep -o 'emacs-async-comp-.*' faxme | cut -d - -f 4- | sed -e "s/-[^-]*\.el$//" |sort -u | while read ; do grep -qs $REPLY before || echo $REPLY; done > 0 grfz@no:/tmp$ > > > So all these "emacs-async-comp-cl-lib-7Go9da.el" like files > have corresponding files in the .eln cache. > > Is it possible that it takes 150 secs to test the .eln cache? "Test" in what sense? Who do you think needs to "test" the cache? Anyway, the above is not what I meant. You present some commands and scripts, and expect me to guess what happens by showing only their results. It's very hard to analyze a problem that way. Instead, please do this: . find the latest subdirectory of the eln-cache whose name is 29.0.5-XXXX (where XXXX is some hash) and list is contents . start Emacs . use 'top' or similar command to see if async compilations are running which were started by Emacs you run above . compare the files these async compilations are compiling with what you see in that latest subdirectory of eln-cache . show the list of files that were compiled although they already had corresponding *.eln files in that subdirectory . let the compilations run to completion (use 'top' to see when there are no more compilations running) . look in the *Async-native-compile-log* buffer to see if there are any warning or error messages there; if there are, post them . kill Emacs . look into that subdirectory of the eln-cache and see whether there are any new *.eln files in it as result of the compilation One reason why Emacs could keep compiling files that already have *.eln files in the cache is because the existing *.eln files are outdated or incompatible with the Emacs binary. This happens a lot if you are using the master branch, because we make significant changes there frequently, and they require recompilation. You will in those cases see *.eln files whose base names are the same, but the hashes are different. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-23 17:08 ` Gregor Zattler 2022-10-23 17:30 ` Eli Zaretskii @ 2022-10-23 17:33 ` Gregor Zattler 2022-10-23 17:34 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Gregor Zattler @ 2022-10-23 17:33 UTC (permalink / raw) To: Eli Zaretskii; +Cc: spwhitton, rlb, monnier, david, emacs-devel, akrl Hi Eli, * Gregor Zattler <telegraph@gmx.net> [2022-10-23; 19:08 +02]: [...] > I did > > find "/home/grfz/.config/emacs/eln-cache/" "/home/grfz/src/emacs-master-next/native-lisp/" -type f | xargs -r ls -Altr > /tmp/before > > <start Emacs again> > > while sleep 1 ; do ps -eo pid,tty,stat,user,group,etime,time,cgroup,args fax >> faxme; done # while emacs started > > find "/home/grfz/.config/emacs/eln-cache/" "/home/grfz/src/emacs-master-next/native-lisp/" -type f | xargs -r ls -Altr > /tmp/before arhgs, that should have been: find "/home/grfz/.config/emacs/eln-cache/" "/home/grfz/src/emacs-master-next/native-lisp/" -type f | xargs -r ls -Altr > /tmp/after > now > > $ diff -aNur before after > $ > > --> No new .eln files were written. > > > But file faxme contains 280 lines like this: > > /home/grfz/src/emacs-master-next/src/emacs --batch -l /tmp/emacs-async-comp-cl-lib-7Go9da.el > > > > 0 grfz@no:/tmp$ grep -o 'emacs-async-comp-.*' faxme | cut -d - -f 4- | sed -e "s/-[^-]*\.el$//" |sort -u | while read ; do grep -qs $REPLY before || echo $REPLY; done > 0 grfz@no:/tmp$ > > > So all these "emacs-async-comp-cl-lib-7Go9da.el" like files > have corresponding files in the .eln cache. > > Is it possible that it takes 150 secs to test the .eln cache? I assumed this is only a test of mtime of files? Ciao; Gregor ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-23 17:33 ` Gregor Zattler @ 2022-10-23 17:34 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-23 17:34 UTC (permalink / raw) To: Gregor Zattler; +Cc: emacs-devel, akrl > From: Gregor Zattler <telegraph@gmx.net> > Cc: spwhitton@spwhitton.name, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, > akrl@sdf.org > Date: Sun, 23 Oct 2022 19:33:05 +0200 > > > Is it possible that it takes 150 secs to test the .eln cache? > > I assumed this is only a test of mtime of files? No, it's much more. But it doesn't take so many seconds, of course. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-23 15:22 Gregor Zattler 2022-10-23 16:07 ` Eli Zaretskii @ 2022-10-23 17:39 ` Eli Zaretskii 2022-10-23 18:55 ` Gregor Zattler 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-23 17:39 UTC (permalink / raw) To: Gregor Zattler; +Cc: emacs-devel, akrl > From: Gregor Zattler <telegraph@gmx.net> > Cc: rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, > emacs-devel@gnu.org, akrl@sdf.org > Date: Sun, 23 Oct 2022 17:22:06 +0200 > > This is GNU Emacs 29.0.50 (build 2, x86_64-pc-linux-gnu, > cairo version 1.16.0) of 2022-10-19 Actually, there was a bug in native compilation that was fixed on that very day. So perhaps update from Git ans see if the problem of repeated compilation goes away. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-23 17:39 ` Eli Zaretskii @ 2022-10-23 18:55 ` Gregor Zattler 2022-10-24 18:35 ` Gregor Zattler 0 siblings, 1 reply; 343+ messages in thread From: Gregor Zattler @ 2022-10-23 18:55 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, akrl Hi Eli, * Eli Zaretskii <eliz@gnu.org> [2022-10-23; 20:39 +03]: >> From: Gregor Zattler <telegraph@gmx.net> >> Cc: rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, >> emacs-devel@gnu.org, akrl@sdf.org >> Date: Sun, 23 Oct 2022 17:22:06 +0200 >> >> This is GNU Emacs 29.0.50 (build 2, x86_64-pc-linux-gnu, >> cairo version 1.16.0) of 2022-10-19 > > Actually, there was a bug in native compilation that was fixed on that > very day. So perhaps update from Git ans see if the problem of > repeated compilation goes away. OK, I'll do so right now. This will take some time, though... Thanks, gregor ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-23 18:55 ` Gregor Zattler @ 2022-10-24 18:35 ` Gregor Zattler 2022-10-24 19:04 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Gregor Zattler @ 2022-10-24 18:35 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel Hi Eli, * Gregor Zattler <telegraph@gmx.net> [2022-10-23; 20:55 +02]: > * Eli Zaretskii <eliz@gnu.org> [2022-10-23; 20:39 +03]: >>> From: Gregor Zattler <telegraph@gmx.net> >>> Cc: rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, >>> emacs-devel@gnu.org, akrl@sdf.org >>> Date: Sun, 23 Oct 2022 17:22:06 +0200 >>> >>> This is GNU Emacs 29.0.50 (build 2, x86_64-pc-linux-gnu, >>> cairo version 1.16.0) of 2022-10-19 >> >> Actually, there was a bug in native compilation that was fixed on that >> very day. So perhaps update from Git ans see if the problem of >> repeated compilation goes away. > > > OK, I'll do so right now. This will take some time, > though... I did as you said, that did the trick, no spurious compilations any more. Thanks. For the record the new version without annoying compilations is: In GNU Emacs 29.0.50 (build 2, x86_64-pc-linux-gnu, cairo version 1.16.0) of 2022-10-24 built on no Repository revision: f7816c94b61f87919afccbedbea5270ca5db4e15 Repository branch: master Ciao; Gregor -- -... --- .-. . -.. ..--.. ...-.- ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-24 18:35 ` Gregor Zattler @ 2022-10-24 19:04 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-24 19:04 UTC (permalink / raw) To: Gregor Zattler; +Cc: emacs-devel, akrl > From: Gregor Zattler <telegraph@gmx.net> > Cc: emacs-devel@gnu.org > Date: Mon, 24 Oct 2022 20:35:18 +0200 > > Hi Eli, > * Gregor Zattler <telegraph@gmx.net> [2022-10-23; 20:55 +02]: > > * Eli Zaretskii <eliz@gnu.org> [2022-10-23; 20:39 +03]: > >>> From: Gregor Zattler <telegraph@gmx.net> > >>> Cc: rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, > >>> emacs-devel@gnu.org, akrl@sdf.org > >>> Date: Sun, 23 Oct 2022 17:22:06 +0200 > >>> > >>> This is GNU Emacs 29.0.50 (build 2, x86_64-pc-linux-gnu, > >>> cairo version 1.16.0) of 2022-10-19 > >> > >> Actually, there was a bug in native compilation that was fixed on that > >> very day. So perhaps update from Git ans see if the problem of > >> repeated compilation goes away. > > > > > > OK, I'll do so right now. This will take some time, > > though... > > I did as you said, that did the trick, no spurious > compilations any more. Thanks. OK, thanks for testing. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term)
@ 2022-09-30 13:13 David Bremner
2022-09-30 13:56 ` Eli Zaretskii
2022-09-30 15:38 ` Stefan Monnier
0 siblings, 2 replies; 343+ messages in thread
From: David Bremner @ 2022-09-30 13:13 UTC (permalink / raw)
To: eliz; +Cc: emacs-devel, akrl, rlb
> The reason stated by Rob was that they want to prevent "writing to
> user's HOME directory". I don't think I understand the rationale well
> enough, and I asked some questions about it. I hope Rob will answer,
> and then we could continue discussing what is reasonable for that use
> case.
For package builds, Debian has the following policy
https://www.debian.org/doc/debian-policy/ch-source.html#main-building-script-debian-rules
in particular
Required targets must not attempt to write outside of the unpacked
source package tree. There are two exceptions. Firstly, the binary
targets may write the binary packages to the parent directory of the
unpacked source package tree. Secondly, required targets may write
to /tmp, /var/tmp and to the directory specified by the TMPDIR
environment variable, but must not depend on the contents of any of
these.
This restriction is intended to prevent source package builds
creating and depending on state outside of themselves, thus
affecting multiple independent rebuilds. In particular, the required
targets must not attempt to write into HOME.
Some additional byte compilation happens at package install time. In my
view the same restrictions should apply there. In addition to the
unintentional sharing of state mentioned in policy, there is also the
issue of cleaning up after a package on uninstall. This is manageable
now because any generated .elc files go in a package specific directory,
which can just be removed on purging the package.
I think just turning off native compilation when HOME is not writeable
would be sensible from a Debian point of view. Emacs did not previously
require a writable HOME (although maybe most users never tested
this). It would be unfortunate if this changed for Emacs with native
compilation support.
David
PS. Please CC me on any replies (that you want me to read). I'm not
subscribed to the list.
PPS. This reply is synthesized from "reply via email" on the archive
page. Apologies in advance for any list etiquette failures.
^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-30 13:13 David Bremner @ 2022-09-30 13:56 ` Eli Zaretskii 2022-09-30 14:33 ` David Bremner ` (3 more replies) 2022-09-30 15:38 ` Stefan Monnier 1 sibling, 4 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-09-30 13:56 UTC (permalink / raw) To: David Bremner; +Cc: emacs-devel, akrl, rlb > From: David Bremner <david@tethera.net> > Cc: emacs-devel@gnu.org, akrl@sdf.org, rlb@defaultvalue.org > Date: Fri, 30 Sep 2022 10:13:56 -0300 > > > > The reason stated by Rob was that they want to prevent "writing to > > user's HOME directory". I don't think I understand the rationale well > > enough, and I asked some questions about it. I hope Rob will answer, > > and then we could continue discussing what is reasonable for that use > > case. > > For package builds, Debian has the following policy > > https://www.debian.org/doc/debian-policy/ch-source.html#main-building-script-debian-rules > > in particular > > Required targets must not attempt to write outside of the unpacked > source package tree. There are two exceptions. Firstly, the binary > targets may write the binary packages to the parent directory of the > unpacked source package tree. Secondly, required targets may write > to /tmp, /var/tmp and to the directory specified by the TMPDIR > environment variable, but must not depend on the contents of any of > these. > > This restriction is intended to prevent source package builds > creating and depending on state outside of themselves, thus > affecting multiple independent rebuilds. In particular, the required > targets must not attempt to write into HOME. > > Some additional byte compilation happens at package install time. This is what I'm asking about: what exactly triggers the compilation? Just installing a package shouldn't do that, only loading it into Emacs should. The other part of what I asked is that if for some reason installation does need to compile files (something that I still need to understand why it happens), there's nothing that hard-codes HOME in the directory list used for that. You can set native-comp-eln-load-path to anything you want, and only the directories in that list will be used. > In my > view the same restrictions should apply there. In addition to the > unintentional sharing of state mentioned in policy, there is also the > issue of cleaning up after a package on uninstall. This is manageable > now because any generated .elc files go in a package specific directory, > which can just be removed on purging the package. See above: you can do the same with *.eln files, if, for some reason, they are produced by the installation. > I think just turning off native compilation when HOME is not writeable > would be sensible from a Debian point of view. Emacs did not previously > require a writable HOME (although maybe most users never tested > this). It would be unfortunate if this changed for Emacs with native > compilation support. Emacs doesn't require a writable HOME, it requires a writable directory to store *.eln files. It doesn't have to be HOME. And once again, I still don't understand why *.eln files are produced at installation time in the first place. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-30 13:56 ` Eli Zaretskii @ 2022-09-30 14:33 ` David Bremner 2022-09-30 15:47 ` Eli Zaretskii 2022-09-30 14:55 ` Sean Whitton ` (2 subsequent siblings) 3 siblings, 1 reply; 343+ messages in thread From: David Bremner @ 2022-09-30 14:33 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, akrl, rlb Eli Zaretskii <eliz@gnu.org> writes: > > This is what I'm asking about: what exactly triggers the compilation? > Just installing a package shouldn't do that, only loading it into > Emacs should. When I talk about "package installation" I mean installing a Debian binary package. This is a more general notion than a package as defined by package.el We have one copy of the .elc files for all users. Because of this, and the cleanup issue I mentioned above, the elc files need to generated either at (Debian) package build time, or at (Debian) package installation time. > The other part of what I asked is that if for some reason installation > does need to compile files (something that I still need to understand > why it happens), there's nothing that hard-codes HOME in the directory > list used for that. You can set native-comp-eln-load-path to anything > you want, and only the directories in that list will be used. Does that restrict where eln files are written? Or just where emacs tries to read? > Emacs doesn't require a writable HOME, it requires a writable > directory to store *.eln files. It doesn't have to be HOME. Fair enough. I tried setting native-compile-target-directory, but that seemed to be ignored by the trampoline stuff. Maybe both variables need to be set. > And once again, I still don't understand why *.eln files are produced > at installation time in the first place. The short version is that we need to run emacs at Debian package build and install time. Byte-compilation is one reason. Running tests at build time is another. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-30 14:33 ` David Bremner @ 2022-09-30 15:47 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-09-30 15:47 UTC (permalink / raw) To: David Bremner; +Cc: emacs-devel, akrl, rlb > From: David Bremner <david@tethera.net> > Cc: emacs-devel@gnu.org, akrl@sdf.org, rlb@defaultvalue.org > Date: Fri, 30 Sep 2022 11:33:29 -0300 > > Eli Zaretskii <eliz@gnu.org> writes: > > > > > This is what I'm asking about: what exactly triggers the compilation? > > Just installing a package shouldn't do that, only loading it into > > Emacs should. > > When I talk about "package installation" I mean installing a Debian > binary package. This is a more general notion than a package as defined > by package.el > > We have one copy of the .elc files for all users. Because of this, and > the cleanup issue I mentioned above, the elc files need to generated > either at (Debian) package build time, or at (Debian) package > installation time. So I understand that this issue is caused by the Debian installation process? If so, the installation process should make sure the *.eln files are written only where you want them to be written. > > The other part of what I asked is that if for some reason installation > > does need to compile files (something that I still need to understand > > why it happens), there's nothing that hard-codes HOME in the directory > > list used for that. You can set native-comp-eln-load-path to anything > > you want, and only the directories in that list will be used. > > Does that restrict where eln files are written? Or just where emacs > tries to read? Both. > > Emacs doesn't require a writable HOME, it requires a writable > > directory to store *.eln files. It doesn't have to be HOME. > > Fair enough. I tried setting native-compile-target-directory, but that > seemed to be ignored by the trampoline stuff. Maybe both variables need > to be set. That's the wrong variable. Please use native-comp-eln-load-path. > > And once again, I still don't understand why *.eln files are produced > > at installation time in the first place. > > The short version is that we need to run emacs at Debian package build > and install time. Byte-compilation is one reason. Running tests at build > time is another. OK, in that case you get to arrange for the environment which will not produce files where you don't want them. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-30 13:56 ` Eli Zaretskii 2022-09-30 14:33 ` David Bremner @ 2022-09-30 14:55 ` Sean Whitton 2022-09-30 16:02 ` Eli Zaretskii 2022-09-30 15:32 ` Stefan Monnier 2022-10-02 5:58 ` tomas 3 siblings, 1 reply; 343+ messages in thread From: Sean Whitton @ 2022-09-30 14:55 UTC (permalink / raw) To: Eli Zaretskii; +Cc: David Bremner, emacs-devel, akrl, rlb Hello, On Fri 30 Sep 2022 at 04:56PM +03, Eli Zaretskii wrote: > This is what I'm asking about: what exactly triggers the compilation? > Just installing a package shouldn't do that, only loading it into > Emacs should. We currently bytecompile during package installation, and would like also to do the native compilation at that time. The main reason for bytecompiling at package installation is that we have logic to purge and recompile anything whose dependency versions have changed, which means that all bytecode installed on Debian systems was generated using known-correct macro definitions. Historically ordinary Emacs processes running in non-batch mode have not guaranteed this; I recall that the Org-mode developers discouraged use of ELPA to install newer versions of Org because of these sorts of issues. A secondary reason is that it makes sense to compile once, system-wide, rather than repeatedly in each user's home directory. It is also nice that you can know everything is already in place once your package is installed, so that I can unplug my laptop once the package manager has exited, and I know that it isn't going to do any CPU-intensive compilation and drain my battery. -- Sean Whitton ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-30 14:55 ` Sean Whitton @ 2022-09-30 16:02 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-09-30 16:02 UTC (permalink / raw) To: Sean Whitton; +Cc: david, emacs-devel, akrl, rlb > From: Sean Whitton <spwhitton@spwhitton.name> > Cc: David Bremner <david@tethera.net>, emacs-devel@gnu.org, akrl@sdf.org, > rlb@defaultvalue.org > Date: Fri, 30 Sep 2022 07:55:08 -0700 > > We currently bytecompile during package installation, and would like > also to do the native compilation at that time. > > The main reason for bytecompiling at package installation is that we > have logic to purge and recompile anything whose dependency versions > have changed, which means that all bytecode installed on Debian systems > was generated using known-correct macro definitions. Historically > ordinary Emacs processes running in non-batch mode have not guaranteed > this; I recall that the Org-mode developers discouraged use of ELPA to > install newer versions of Org because of these sorts of issues. This problem doesn't exist with *.eln files, because Emacs will regenerate new .eln files with different hashes in its name when the source file or the Emacs binary changes. So you do not have to trigger native compilation for these reasons. > A secondary reason is that it makes sense to compile once, system-wide, > rather than repeatedly in each user's home directory. I think this is misguided. If the user never loads a package, there's no reason to produce the *.eln files for it. At best it litters the disk with useless files. At worst, compilation could cause warnings or even errors, which are just an annoyance when the package is not going to be used for a long time, or not at all. These considerations are the reason why *.eln files are compiled JIT and on-demand. It is not a coincidence or lack of thought on our part, it is the result of considering many factors. I think you are trying to deal with *.eln files and with native-compilation in general as if it were a slightly different kind of byte-compilation. That is a mistake: it's a qualitatively different feature with different traits and different user-facing and administrator-facing aspects. > It is also nice that you can know everything is already in place > once your package is installed, so that I can unplug my laptop once > the package manager has exited, and I know that it isn't going to do > any CPU-intensive compilation and drain my battery. The *.eln files should not be part of "everything is in place", because they will be produced when needed. No user involvement or interaction is necessary. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-30 13:56 ` Eli Zaretskii 2022-09-30 14:33 ` David Bremner 2022-09-30 14:55 ` Sean Whitton @ 2022-09-30 15:32 ` Stefan Monnier 2022-09-30 16:06 ` Eli Zaretskii 2022-10-02 5:58 ` tomas 3 siblings, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-09-30 15:32 UTC (permalink / raw) To: Eli Zaretskii; +Cc: David Bremner, emacs-devel, akrl, rlb >> Some additional byte compilation happens at package install time. [ BTW, In Rob's case, we're talking about installation via Debian's `dpkg`, i.e. system-wide installation of ELisp packages, which also causes the .el files to be byte-compiled for&by the currently installed Emacs binary. ] > This is what I'm asking about: what exactly triggers the compilation? > Just installing a package shouldn't do that, only loading it into > Emacs should. Byte-compilation does load files (not the one being compiled, but many others, starting with `bytecomp` of course), so it can trigger native-compilation of some files (including some of the files being byte-compiled, if they `require` each other, which is very common). Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-30 15:32 ` Stefan Monnier @ 2022-09-30 16:06 ` Eli Zaretskii 2022-10-01 16:38 ` Rob Browning 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-09-30 16:06 UTC (permalink / raw) To: Stefan Monnier; +Cc: david, emacs-devel, akrl, rlb > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: David Bremner <david@tethera.net>, emacs-devel@gnu.org, akrl@sdf.org, > rlb@defaultvalue.org > Date: Fri, 30 Sep 2022 11:32:57 -0400 > > Byte-compilation does load files (not the one being compiled, but many > others, starting with `bytecomp` of course), so it can trigger > native-compilation of some files (including some of the files being > byte-compiled, if they `require` each other, which is very common). The solution for this that I'd suggest is to precompile them, so that the package comes with those files already as *.eln. That's what we did in Emacs 28.1 for -nw sessions, which always load the terminal support file from lisp/term/. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-30 16:06 ` Eli Zaretskii @ 2022-10-01 16:38 ` Rob Browning 2022-10-01 16:52 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-01 16:38 UTC (permalink / raw) To: Eli Zaretskii, Stefan Monnier; +Cc: david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > The solution for this that I'd suggest is to precompile them, so that > the package comes with those files already as *.eln. That's what we > did in Emacs 28.1 for -nw sessions, which always load the terminal > support file from lisp/term/. I've finally had a chance to catch up, and this thread to date appears to cover a good bit of the relevant ground. As requested, I'll try to add some additional context that hopefully better explains how and why we arrived at the current situation, asking the questions we're asking. Long ago, we (speaking from the Debian side) came up with the Debian Emacs policy. This was both a policy, and the infrastructure to implement some of it (in the form of the emacsen-common package, and now, additionally, the dh-elpa infrastructure). At the time, XEmacs was still active, and Emacs itself was about to transition from 19 to 20, and my recollection is that there was a nontrivial amount of backward incompatibility with the latter, or at least I recall hearing from people who really didn't want to be forced to upgrade immediately. I'm not completely familiar with those concerns because this was just about the time I started working on Debian's Emacs support. In any case, one of the key goals of that support was to allow Debian to package, and people to install, one or more major versions of Emacs, and/or XEmacs simultaneously. As a result, we added support for "emacsen flavors": emacs20, emacs21, emacs22, xemacs19, etc. (Worth noting that a few years back we did "unversion" the Emacs packages, so that we no longer have emacs27, emacs28, etc., just emacs.) We also wanted to support the creation of "add-on" packages like org, newer versions of gnus, etc., and we wanted each of those packages to be able to work with as many flavors (Emacs or XEmacs) as the add-on package maintainer decided was feasible. We wanted to have only one system-level .elc file for each flavor for each .el file. We did not want to have to build and maintain separate packages for each flavor in the archive, i.e. org-emacs21, org-emacs22, etc., each with a full set of the relevant .elc files. (I'm also not sure that would have worked (easily), given the versioning and dependency concerns described next.) We didn't want to risk breakage from backward-incompatible changes to macros in dependencies, i.e. if the magit package depended on the foo (library) package, we assumed we'd need to rebuild (during package upgrade) all the majit .elc files whenever the foo package changed because foo might have changed a macro backward incompatibly. Since we didn't have any (reasonable?) way to detect the inter-package dependency graph, we just insisted that those be correctly described by the Debian package dependencies. Then at install time, we use that graph to determine what packages to "rebuild" (mostly just the .elc files), and the order in which to rebuild them. We also decided to rebuild *all* the add-on packages (.elc files) for a given flavor whenever the version of the flavor's package changes (e.g. when there's a new Emacs or XEmacs version) because we didn't know whether or not any given release might make that necessary/desirable. Uninstalling an add-on, of course, cleans up all the relevant bits (usually, mostly just the automatically generated .elc files). So when you "apt install elpa-magit", Debian builds all the .elc files and puts them in the right place in /usr for every flavor you have installed. When you "apt install emacs" and it's a new install, Debian builds all of the .elc files for every installed add-on package in dependency order for that flavor. The same thing happens during an emacs package upgrade. Note that we were also thinking of the case where you have a large machine with hundreds of users (as was more often the case when we started). There we didn't want to have N copies of the same file for N users (whether .elc or now .eln). Regarding the writes to HOME, etc. I think David covered that fairly well -- Debian runs emacs to "do things" (build .elcs, run tests, etc.) at package build time, package install time, and during package testing. In all of these cases, we're likely to want to avoid side-effects outside the build/test dir like writing .elc or .eln files to the current user's HOME, whether that's /root/ or /home/*. It sounds like we may be able to accomplish that by redirecting everything to a temp dir, which is likely fine. Hope this helps. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-01 16:38 ` Rob Browning @ 2022-10-01 16:52 ` Eli Zaretskii 2022-10-01 20:42 ` Rob Browning 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-01 16:52 UTC (permalink / raw) To: Rob Browning; +Cc: monnier, david, emacs-devel, akrl > From: Rob Browning <rlb@defaultvalue.org> > Cc: david@tethera.net, emacs-devel@gnu.org, akrl@sdf.org > Date: Sat, 01 Oct 2022 11:38:54 -0500 > > I've finally had a chance to catch up, and this thread to date appears > to cover a good bit of the relevant ground. As requested, I'll try to > add some additional context that hopefully better explains how and why > we arrived at the current situation, asking the questions we're asking. Thanks. > Long ago, we (speaking from the Debian side) came up with the Debian > Emacs policy. This was both a policy, and the infrastructure to > implement some of it (in the form of the emacsen-common package, and > now, additionally, the dh-elpa infrastructure). > > At the time, XEmacs was still active, and Emacs itself was about to > transition from 19 to 20, and my recollection is that there was a > nontrivial amount of backward incompatibility with the latter, or at > least I recall hearing from people who really didn't want to be forced > to upgrade immediately. > > I'm not completely familiar with those concerns because this was just > about the time I started working on Debian's Emacs support. In any > case, one of the key goals of that support was to allow Debian to > package, and people to install, one or more major versions of Emacs, > and/or XEmacs simultaneously. As a result, we added support for > "emacsen flavors": emacs20, emacs21, emacs22, xemacs19, etc. (Worth > noting that a few years back we did "unversion" the Emacs packages, so > that we no longer have emacs27, emacs28, etc., just emacs.) > > We also wanted to support the creation of "add-on" packages like org, > newer versions of gnus, etc., and we wanted each of those packages to be > able to work with as many flavors (Emacs or XEmacs) as the add-on > package maintainer decided was feasible. > > We wanted to have only one system-level .elc file for each flavor for > each .el file. We did not want to have to build and maintain separate > packages for each flavor in the archive, i.e. org-emacs21, org-emacs22, > etc., each with a full set of the relevant .elc files. (I'm also not > sure that would have worked (easily), given the versioning and > dependency concerns described next.) > > We didn't want to risk breakage from backward-incompatible changes to > macros in dependencies, i.e. if the magit package depended on the foo > (library) package, we assumed we'd need to rebuild (during package > upgrade) all the majit .elc files whenever the foo package changed > because foo might have changed a macro backward incompatibly. > > Since we didn't have any (reasonable?) way to detect the inter-package > dependency graph, we just insisted that those be correctly described by > the Debian package dependencies. Then at install time, we use that > graph to determine what packages to "rebuild" (mostly just the .elc > files), and the order in which to rebuild them. > > We also decided to rebuild *all* the add-on packages (.elc files) for a > given flavor whenever the version of the flavor's package changes > (e.g. when there's a new Emacs or XEmacs version) because we didn't know > whether or not any given release might make that necessary/desirable. > > Uninstalling an add-on, of course, cleans up all the relevant bits > (usually, mostly just the automatically generated .elc files). > > So when you "apt install elpa-magit", Debian builds all the .elc files > and puts them in the right place in /usr for every flavor you have > installed. When you "apt install emacs" and it's a new install, Debian > builds all of the .elc files for every installed add-on package in > dependency order for that flavor. The same thing happens during an > emacs package upgrade. > > Note that we were also thinking of the case where you have a large > machine with hundreds of users (as was more often the case when we > started). There we didn't want to have N copies of the same file for N > users (whether .elc or now .eln). Thanks for the background. You should know that the problems you needed to address with the *.elc files don't exist with the *.eln files. The *.eln files have version information of the Emacs to which they "belong" hard-coded into their names. That's why window.el gets compiled into something like window-0d1b8b93-7ef4271a.eln, and lives under a directory named something like 28.2-4fafb146 -- these 3 hashes encode both the Emacs binary and its version, and the original .el file's absolute file name and contents. So there's no way users will have any trouble when multiple Emacs versions are installed or when different users use different versions: the *.eln files are effectively automatically "versioned", and Emacs will only load a .eln file that was compiled for it. Moreover, as soon as some user decides to modify a .el file, the .eln file produced from it will have a different name, and if the Emacs binary changes as result of rebuilding, the new .eln file will be written to a different directory. > Regarding the writes to HOME, etc. I think David covered that fairly > well -- Debian runs emacs to "do things" (build .elcs, run tests, etc.) > at package build time, package install time, and during package testing. > In all of these cases, we're likely to want to avoid side-effects outside > the build/test dir like writing .elc or .eln files to the current user's > HOME, whether that's /root/ or /home/*. It sounds like we may be able > to accomplish that by redirecting everything to a temp dir, which is > likely fine. Yes, by changing native-comp-eln-load-path you should be able to avoid writing to the home directory of the user under whose credentials the installation runs. If you need that. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-01 16:52 ` Eli Zaretskii @ 2022-10-01 20:42 ` Rob Browning 2022-10-02 5:57 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-01 20:42 UTC (permalink / raw) To: Eli Zaretskii; +Cc: monnier, david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > You should know that the problems you needed to address with the *.elc > files don't exist with the *.eln files. The *.eln files have version > information of the Emacs to which they "belong" hard-coded into their > names. Right, I'd noticed that, and while I haven't fully understood the expectations/semantics yet, I'd started to assume much of what you've described. I also assumed we'd need to understand things even better before the Debian support is completely "ready" (right now we're working all this out in unstable). My current expectation is that if it ends up being feasible, we'll probably want to treat the package-related .eln files like the .elc files, and build them during the package install. For example, elpa-magit (the current magit add-on package) will want to generate both the .elc and .eln files for all of its .el files when its turn comes around during installs/upgrades. I say that because if it's a single-user machine and the user invokes "apt install elpa-magit", I'd imagine they're doing that because they want to use magit, in which case it doesn't hurt to get the work out of the way up-front, and it might help in that all the work will be finished at once, so they won't incur costs (battery-consumption, etc.) later, when they might not expect or want it. (Not a critical issue, possibly, but perhaps friendlier.) And if it's a multi-user machine, with a lot of emacs users, at the moment I don't see any reason to want to compile the same file 50 times for 50 users (or even more than just once), incurring the attendant power and storage costs. > Yes, by changing native-comp-eln-load-path you should be able to avoid > writing to the home directory of the user under whose credentials the > installation runs. If you need that. OK, thanks much. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-01 20:42 ` Rob Browning @ 2022-10-02 5:57 ` Eli Zaretskii 2022-10-02 6:10 ` tomas ` (3 more replies) 0 siblings, 4 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 5:57 UTC (permalink / raw) To: Rob Browning; +Cc: monnier, david, emacs-devel, akrl > From: Rob Browning <rlb@defaultvalue.org> > Cc: monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, > akrl@sdf.org > Date: Sat, 01 Oct 2022 15:42:06 -0500 > > My current expectation is that if it ends up being feasible, we'll > probably want to treat the package-related .eln files like the .elc > files, and build them during the package install. For example, > elpa-magit (the current magit add-on package) will want to generate both > the .elc and .eln files for all of its .el files when its turn comes > around during installs/upgrades. > > I say that because if it's a single-user machine and the user invokes > "apt install elpa-magit", I'd imagine they're doing that because they > want to use magit, in which case it doesn't hurt to get the work out of > the way up-front, and it might help in that all the work will be > finished at once, so they won't incur costs (battery-consumption, etc.) > later, when they might not expect or want it. (Not a critical issue, > possibly, but perhaps friendlier.) > > And if it's a multi-user machine, with a lot of emacs users, at the > moment I don't see any reason to want to compile the same file 50 times > for 50 users (or even more than just once), incurring the attendant > power and storage costs. I don't think you should try to second-guess the user who installs a package. They could just want to study the sources, for example. The native compilation is implemented as JIT capability for a reason. We dicussed the various aspects of that for a long time before deciding on what you see today. My advice is not to reject that without very good reasons (and those reasons should probably be communicated to us when found), just because of some analogy with byte compilation, as that analogy breaks on several levels. By using native compilation in its default JIT manner, you are basically using Emacs as the upstream project meant it to be used, which means, among others, that you don't need to fight an uphill battle against various defaults and discover situations that no one expected to happen. The reasons which you mention in favor of AOT native compilation don't sound serious enough: I see no problem in having the compilation happen only when it's needed in those cases. Battery consumption doesn't seem very relevant, because JIT compilation will happen when the system is used, not when it sleeps. And it is entirely not guaranteed that by compiling everything you will save power in the long run, because there are good chances you will be compiling stuff that won't be used for a long time. Without quantitative data of long-term power usage on which to base the conclusions, I don't see why you should a-priori assume that compiling everything from the get-go should use less power. Same goes for disk space by multiple users. All in all, I think JIT compilation strikes a good balance between resources and their actual usage. This also matches my personal experience of using Emacs 28 with native compilation since day one: I never had to look back on the decision not to use AOT compilation of everything. Of course, this is eventually your call. But my recommendation is to try to stick to the default behavior unless it causes actual (as opposed to imaginary or presumed) problems, based on actual usage. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 5:57 ` Eli Zaretskii @ 2022-10-02 6:10 ` tomas 2022-10-02 6:44 ` Eli Zaretskii 2022-10-02 16:53 ` Stefan Monnier ` (2 subsequent siblings) 3 siblings, 1 reply; 343+ messages in thread From: tomas @ 2022-10-02 6:10 UTC (permalink / raw) To: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1107 bytes --] On Sun, Oct 02, 2022 at 08:57:13AM +0300, Eli Zaretskii wrote: > > From: Rob Browning <rlb@defaultvalue.org> > > Cc: monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, > > akrl@sdf.org > > Date: Sat, 01 Oct 2022 15:42:06 -0500 [...] > > And if it's a multi-user machine, with a lot of emacs users, at the > > moment I don't see any reason to want to compile the same file 50 times > > for 50 users (or even more than just once), incurring the attendant > > power and storage costs. > > I don't think you should try to second-guess the user who installs a > package. They could just want to study the sources, for example. That's what apt-get source and friends are. The user can download, build, modify, etc. the sources corresponding to packages. Those are purely user operations, no admin powers needed. But installing a binary pacage on a system does modify the system for all users, so admin powers do make sense there. Nobody is being second-guessed, just the roles separated (again, on a single user system, this might feel artificial). Cheers -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 6:10 ` tomas @ 2022-10-02 6:44 ` Eli Zaretskii 2022-10-02 15:54 ` tomas 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 6:44 UTC (permalink / raw) To: tomas; +Cc: emacs-devel > Date: Sun, 2 Oct 2022 08:10:15 +0200 > From: <tomas@tuxteam.de> > > > I don't think you should try to second-guess the user who installs a > > package. They could just want to study the sources, for example. > > That's what apt-get source and friends are. The user can download, > build, modify, etc. the sources corresponding to packages. Those > are purely user operations, no admin powers needed. > > But installing a binary pacage on a system does modify the system > for all users, so admin powers do make sense there. What do you mean by "binary package" in this context? I believe "package" in this discussion generally alludes to Lisp packages, from ELPA and elsewhere. They aren't "binary" in my book. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 6:44 ` Eli Zaretskii @ 2022-10-02 15:54 ` tomas 2022-10-02 16:02 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: tomas @ 2022-10-02 15:54 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 597 bytes --] On Sun, Oct 02, 2022 at 09:44:16AM +0300, Eli Zaretskii wrote: > > Date: Sun, 2 Oct 2022 08:10:15 +0200 > > From: <tomas@tuxteam.de> [...] > > But installing a binary pacage on a system does modify the system > > for all users, so admin powers do make sense there. > > What do you mean by "binary package" in this context? I believe > "package" in this discussion generally alludes to Lisp packages, from > ELPA and elsewhere. They aren't "binary" in my book. Ah, I see. I was too sloppy. I was talking about Debian packages. Sorry for the confusion. Cheers -- tomás [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 15:54 ` tomas @ 2022-10-02 16:02 ` Eli Zaretskii 2022-10-02 16:13 ` chad 2022-10-02 17:25 ` Rob Browning 0 siblings, 2 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 16:02 UTC (permalink / raw) To: tomas; +Cc: emacs-devel > Date: Sun, 2 Oct 2022 17:54:44 +0200 > From: tomas@tuxteam.de > Cc: emacs-devel@gnu.org > > > > But installing a binary pacage on a system does modify the system > > > for all users, so admin powers do make sense there. > > > > What do you mean by "binary package" in this context? I believe > > "package" in this discussion generally alludes to Lisp packages, from > > ELPA and elsewhere. They aren't "binary" in my book. > > Ah, I see. I was too sloppy. I was talking about Debian packages. That still doesn't explain it to me (I don't use Debian). Can you elaborate what kind of packages are we talking about, in the context of this discussion? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:02 ` Eli Zaretskii @ 2022-10-02 16:13 ` chad 2022-10-02 16:55 ` Eli Zaretskii 2022-10-02 17:25 ` Rob Browning 1 sibling, 1 reply; 343+ messages in thread From: chad @ 2022-10-02 16:13 UTC (permalink / raw) To: Eli Zaretskii; +Cc: tomas, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1127 bytes --] On Sun, Oct 2, 2022 at 12:03 PM Eli Zaretskii <eliz@gnu.org> wrote: > > Ah, I see. I was too sloppy. I was talking about Debian packages. > > That still doesn't explain it to me (I don't use Debian). Can you > elaborate what kind of packages are we talking about, in the context > of this discussion? > Debian supports a user installing, for example, emacs+magit, via Debian packages. This gets the user a stable, known-tested version of emacs plus the package usable on machines that, for another example, cannot or should not connect to the internet. This is less important to developers, but is an important part of the Debian support "contract". For the record: I personally know of situations where such a setup would like to use native-comp and would very much prefer NOT to duplicate either the eln files per-user nor the effort of creating such. Whether or not that situation is important enough to the combination of emacs-devel and debian-maintainers to justify effort on either side is, of course, debatable, but I am highly confident that they exist (at least, did before the pandemic). Hope that helps, ~Chad [-- Attachment #2: Type: text/html, Size: 1557 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:13 ` chad @ 2022-10-02 16:55 ` Eli Zaretskii 2022-10-02 17:13 ` Stefan Monnier 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 16:55 UTC (permalink / raw) To: chad; +Cc: tomas, emacs-devel > From: chad <yandros@gmail.com> > Date: Sun, 2 Oct 2022 12:13:44 -0400 > Cc: tomas@tuxteam.de, emacs-devel@gnu.org > > > Ah, I see. I was too sloppy. I was talking about Debian packages. > > That still doesn't explain it to me (I don't use Debian). Can you > elaborate what kind of packages are we talking about, in the context > of this discussion? > > Debian supports a user installing, for example, emacs+magit, via Debian packages. This gets the user a > stable, known-tested version of emacs plus the package usable on machines that, for another example, > cannot or should not connect to the internet. This is less important to developers, but is an important part of > the Debian support "contract". Which part of the "emacs+magit" package is the "binary package"? The only part of that which could produce *.eln files at installation time is Magit, right? Because Emacs itself already comes with all the preloaded *.eln files, and those aren't what we are talking about, right? And if we are talking about Magit in this example, then how come its being bundled to Emacs change anything of what I said? > For the record: I personally know of situations where such a setup would like to use native-comp and would > very much prefer NOT to duplicate either the eln files per-user nor the effort of creating such. Whether or > not that situation is important enough to the combination of emacs-devel and debian-maintainers to justify > effort on either side is, of course, debatable, but I am highly confident that they exist (at least, did before the > pandemic). Please forgive me, but you cannot expect us to regard such situations seriously as long as you don't describe them. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:55 ` Eli Zaretskii @ 2022-10-02 17:13 ` Stefan Monnier 2022-10-02 17:24 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-02 17:13 UTC (permalink / raw) To: Eli Zaretskii; +Cc: chad, tomas, emacs-devel > Which part of the "emacs+magit" package is the "binary package"? The "binary" Debian package for `elpa-magit` mostly contains Magit's `.el` files plus the doc and a few other things. IOW it's fairly similar to our ELPA tarballs. So it's very close to the source code itself, but it's still separate from "the source" (which you can get via things like `apt source elpa-magit` and will fetch some tarball rather than a `.deb` file) from which the `.deb` is built. > And if we are talking about Magit in this example, then how come its > being bundled to Emacs change anything of what I said? I don't think there's a `emacs+magit` package in Debian. There's an `elpa-magit` package and an `emacs` package. >> For the record: I personally know of situations where such a setup >> would like to use native-comp and would very much prefer NOT to >> duplicate either the eln files per-user nor the effort of creating >> such. Whether or not that situation is important enough to the >> combination of emacs-devel and debian-maintainers to justify effort >> on either side is, of course, debatable, but I am highly confident >> that they exist (at least, did before the pandemic). > > Please forgive me, but you cannot expect us to regard such situations > seriously as long as you don't describe them. We don't have to take their opinion into account when choosing Emacs's defaults. We just need to provide the tools that let them get the behavior they want. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:13 ` Stefan Monnier @ 2022-10-02 17:24 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 17:24 UTC (permalink / raw) To: Stefan Monnier; +Cc: yandros, tomas, emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: chad <yandros@gmail.com>, tomas@tuxteam.de, emacs-devel@gnu.org > Date: Sun, 02 Oct 2022 13:13:59 -0400 > > > Which part of the "emacs+magit" package is the "binary package"? > > The "binary" Debian package for `elpa-magit` mostly contains Magit's > `.el` files plus the doc and a few other things. IOW it's fairly > similar to our ELPA tarballs. That's what I thought. But then I don't see how this is special in the context of this discussion, so that what Emacs users do when they download from ELPA is different from them installing a Debian "binary" package. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:02 ` Eli Zaretskii 2022-10-02 16:13 ` chad @ 2022-10-02 17:25 ` Rob Browning 1 sibling, 0 replies; 343+ messages in thread From: Rob Browning @ 2022-10-02 17:25 UTC (permalink / raw) To: Eli Zaretskii, tomas; +Cc: emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > That still doesn't explain it to me (I don't use Debian). Can you > elaborate what kind of packages are we talking about, in the context > of this discussion? One possible way to look at it -- on a Debian system I think you'd generally apt install emacs # (or gcc-12 or ...) if you just wanted to use emacs or gcc, or... And that would install the binaries, more or less, i.e. roughly like "make install", but with automatic dependency resolution, etc. And note that with many tools that would get you *just* the binaries, i.e. no source at all for say gcc, libc, grep, git, etc. If you wanted to investigate, fix, enhance the packaging itself, then you might apt source emacs which would download the packaging and unpack it in the current directory, as say emacs-28.1+1-3/, and where all the packaging related-changes, including any patches applied to the upstream code, would be clearly visible in a emacs-28.1+1-3/debian/ subdirectory in the emacs source. For many packages these days you might also be able to just clone the Debian packaging archive, e.g. git clone https://salsa.debian.org/rlb/deb-emacs.git which might be preferable if you're more comfortable with git. And finally, if your primary interest is the upstream source, and/or running and/or working on anything other than the versions debian currently provides, I suspect you'd likely want to git clone git://git.savannah.gnu.org/emacs.git i.e. you wouldn't need and might well not want the debian packag(ing|es) at all. Hope this helps. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 5:57 ` Eli Zaretskii 2022-10-02 6:10 ` tomas @ 2022-10-02 16:53 ` Stefan Monnier 2022-10-02 17:16 ` Eli Zaretskii ` (3 more replies) 2022-10-02 17:15 ` Rob Browning 2022-10-02 23:51 ` Sean Whitton 3 siblings, 4 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-02 16:53 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Rob Browning, david, emacs-devel, akrl > I don't think you should try to second-guess the user who installs a > package. They could just want to study the sources, for example. I'm quite happy with the use of JIT as the default in Emacs. But I think I agree with Rob that it makes a lot of sense in the context of Debian to eagerly native-compile the packages when they're installed via APT. In APT there's a clearly distinction between installing the package (which requires admin rights and has non-trivial effects on the whole system) and looking at the source code of the package (this distinction is natural in the context of Debian where many/most packages are written in languages like C, but it naturally carries over to ELisp packages). So if user "just want to study the sources" they wouldn't *install* the package at all. > All in all, I think JIT compilation strikes a good balance between > resources and their actual usage. Yes. But the balance is different in different contexts. FWIW, I'm not convinced it's really useful in Debian's `emacs` package to eagerly native compile all the bundled .elc files, but I think it does make a lot of sense for ELisp packages installed separately. In any case, I'd let Debian's maintainers make their own choices for their own specific needs which are slightly different from ours (where our release tarballs and default config are designed in large part for users who'll compile Emacs themselves and who install third party ELisp packages into their $HOME). Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:53 ` Stefan Monnier @ 2022-10-02 17:16 ` Eli Zaretskii 2022-10-02 18:12 ` Rob Browning 2022-10-02 17:17 ` Lars Ingebrigtsen ` (2 subsequent siblings) 3 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 17:16 UTC (permalink / raw) To: Stefan Monnier; +Cc: rlb, david, emacs-devel, akrl > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Rob Browning <rlb@defaultvalue.org>, david@tethera.net, > emacs-devel@gnu.org, akrl@sdf.org > Date: Sun, 02 Oct 2022 12:53:56 -0400 > > > All in all, I think JIT compilation strikes a good balance between > > resources and their actual usage. > > Yes. But the balance is different in different contexts. Theoretically, yes. But I don't yet see why the particular context described by Rob is different to the degree that it would need special procedures. > FWIW, I'm not convinced it's really useful in Debian's `emacs` package > to eagerly native compile all the bundled .elc files, but I think it > does make a lot of sense for ELisp packages installed separately. Maybe so, but even if that is the decision about the *.elc files, it doesn't automatically follow that the *.eln files should be handled the same. I explained in previous messages why I think so, with enough important differences between them to facilitate rethinking, I hope. > In any case, I'd let Debian's maintainers make their own choices for > their own specific needs which are slightly different from ours (where > our release tarballs and default config are designed in large part for > users who'll compile Emacs themselves and who install third party > ELisp packages into their $HOME). We can only advise them, yes. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:16 ` Eli Zaretskii @ 2022-10-02 18:12 ` Rob Browning 2022-10-02 18:40 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-02 18:12 UTC (permalink / raw) To: Eli Zaretskii, Stefan Monnier; +Cc: david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > Theoretically, yes. But I don't yet see why the particular context > described by Rob is different to the degree that it would need special > procedures. While I'm not sure where we'll end up, I do think others may put more weight on some of the concerns. For example, across the broader world/users-of-debian-and-derivative packages, I suspect there are some who care more about storage space. As mentioned elswhere, we get regular requests to make emacs-nox even smaller. And my eln-cache is currently 40MB, which is storage space we'd thought we might not have to duplicate across the emacs users on a system in the common case (Of course I know the duplication rate would vary depending on which users use which things, but unless the sets were disjoint, there'd be duplication that didn't seem necessary.) But again, in the end, we just wanted to present the broader case for consideration. Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:12 ` Rob Browning @ 2022-10-02 18:40 ` Eli Zaretskii 2022-10-02 18:51 ` Rob Browning 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 18:40 UTC (permalink / raw) To: Rob Browning; +Cc: monnier, david, emacs-devel, akrl > From: Rob Browning <rlb@defaultvalue.org> > Cc: david@tethera.net, emacs-devel@gnu.org, akrl@sdf.org > Date: Sun, 02 Oct 2022 13:12:01 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > > Theoretically, yes. But I don't yet see why the particular context > > described by Rob is different to the degree that it would need special > > procedures. > > While I'm not sure where we'll end up, I do think others may put more > weight on some of the concerns. For example, across the broader > world/users-of-debian-and-derivative packages, I suspect there are some > who care more about storage space. > > As mentioned elswhere, we get regular requests to make emacs-nox even > smaller. And my eln-cache is currently 40MB, which is storage space > we'd thought we might not have to duplicate across the emacs users on a > system in the common case (Of course I know the duplication rate would > vary depending on which users use which things, but unless the sets were > disjoint, there'd be duplication that didn't seem necessary.) IMO, this is actually an argument _against_ compiling everything AOT. Whether the duplication is significant can only be decided based on actual usage figures. It is incorrect to assess this based on the *.elc files, since those are independent of almost everything. There's high probability of wrong decisions based on that analogy. There are many factors that affect compatibility of *.eln files to Emacs binaries; for example, it's enough to add or remove a primitive, and you will need a whol;e new set of *.eln files. Thus, it is quite possible that duplication will be smaller and OTOH waste of disk space due to unnecessarily compiled *.eln files will be higher than you envision. Only practice will show the real situation. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:40 ` Eli Zaretskii @ 2022-10-02 18:51 ` Rob Browning 0 siblings, 0 replies; 343+ messages in thread From: Rob Browning @ 2022-10-02 18:51 UTC (permalink / raw) To: Eli Zaretskii; +Cc: monnier, david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > Whether the duplication is significant can only be decided based on > actual usage figures. It is incorrect to assess this based on the > *.elc files, since those are independent of almost everything. > There's high probability of wrong decisions based on that analogy. > There are many factors that affect compatibility of *.eln files to > Emacs binaries; for example, it's enough to add or remove a primitive, > and you will need a whol;e new set of *.eln files. Thus, it is quite > possible that duplication will be smaller and OTOH waste of disk space > due to unnecessarily compiled *.eln files will be higher than you > envision. Only practice will show the real situation. I think I've probably made this clear elsewhere, but in case not, that wouldn't be the case for Debian specifically. There could/would only be *one* eln tree for the whole system with the current packaging. And that tree would be rebuilt when appropriate, in dependency order (*if* we end up deciding to pursuse this). -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:53 ` Stefan Monnier 2022-10-02 17:16 ` Eli Zaretskii @ 2022-10-02 17:17 ` Lars Ingebrigtsen 2022-10-02 17:28 ` Stefan Monnier 2022-10-02 17:30 ` Eli Zaretskii 2022-10-02 17:37 ` Rob Browning 2022-10-02 23:53 ` Sean Whitton 3 siblings, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-02 17:17 UTC (permalink / raw) To: Stefan Monnier; +Cc: Eli Zaretskii, Rob Browning, david, emacs-devel, akrl Stefan Monnier <monnier@iro.umontreal.ca> writes: > But I think I agree with Rob that it makes a lot of sense in the context > of Debian to eagerly native-compile the packages when they're installed > via APT. I've always assumed that that's what distributions like Debian would do. Which means that there should be an easy way to switch JIT off, but I've just forgotten to add it. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:17 ` Lars Ingebrigtsen @ 2022-10-02 17:28 ` Stefan Monnier 2022-10-02 17:36 ` Lars Ingebrigtsen 2022-10-02 17:30 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-02 17:28 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, Rob Browning, david, emacs-devel, akrl Lars Ingebrigtsen [2022-10-02 19:17:27] wrote: > Stefan Monnier <monnier@iro.umontreal.ca> writes: >> But I think I agree with Rob that it makes a lot of sense in the context >> of Debian to eagerly native-compile the packages when they're installed >> via APT. > I've always assumed that that's what distributions like Debian would > do. Which means that there should be an easy way to switch JIT off, but > I've just forgotten to add it. Switching it off means that the native compilation is never auto-triggered, so it's probably not quite what you meant :-) Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:28 ` Stefan Monnier @ 2022-10-02 17:36 ` Lars Ingebrigtsen 2022-10-02 17:38 ` Eli Zaretskii 2022-10-02 18:17 ` Rob Browning 0 siblings, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-02 17:36 UTC (permalink / raw) To: Stefan Monnier; +Cc: Eli Zaretskii, Rob Browning, david, emacs-devel, akrl Stefan Monnier <monnier@iro.umontreal.ca> writes: > Switching it off means that the native compilation is never > auto-triggered, so it's probably not quite what you meant :-) I'm not quite sure what you mean -- I'm saying that there should be a way for users to switch off native compilation (and for distributions to have native compilation for users switched off). Which was how this thread started: With a request for such a thing. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:36 ` Lars Ingebrigtsen @ 2022-10-02 17:38 ` Eli Zaretskii 2022-10-02 18:17 ` Rob Browning 1 sibling, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 17:38 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: monnier, rlb, david, emacs-devel, akrl > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Eli Zaretskii <eliz@gnu.org>, Rob Browning <rlb@defaultvalue.org>, > david@tethera.net, emacs-devel@gnu.org, akrl@sdf.org > Date: Sun, 02 Oct 2022 19:36:01 +0200 > > I'm not quite sure what you mean -- I'm saying that there should be a > way for users to switch off native compilation (and for distributions to > have native compilation for users switched off). > > Which was how this thread started: With a request for such a thing. Which turned out to be a request for something else: for controlling where the *.eln files are written. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:36 ` Lars Ingebrigtsen 2022-10-02 17:38 ` Eli Zaretskii @ 2022-10-02 18:17 ` Rob Browning 2022-10-02 19:08 ` Lars Ingebrigtsen 1 sibling, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-02 18:17 UTC (permalink / raw) To: Lars Ingebrigtsen, Stefan Monnier; +Cc: Eli Zaretskii, david, emacs-devel, akrl Lars Ingebrigtsen <larsi@gnus.org> writes: > I'm not quite sure what you mean -- I'm saying that there should be a > way for users to switch off native compilation (and for distributions to > have native compilation for users switched off). > > Which was how this thread started: With a request for such a thing. At the top level, we wanted a way to avoid writing to HOME during packaging, testing, installs (in this case, it's the .eln files, now that we've enabled native compilation). That could be handled by some way to turn off native compilation, or by some way to comprehensively divert those writes to another location (e.g. temp dir). Either is fine, though we'd originally thought the former might make things a bit easier. Whether or not we can (or should) try to build system-level (root owned) .eln files along with the .elc files during package installs (as we already do for .elc files) is a separate question, and I think the more controversial one? Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:17 ` Rob Browning @ 2022-10-02 19:08 ` Lars Ingebrigtsen 2022-10-02 19:15 ` Eli Zaretskii ` (2 more replies) 0 siblings, 3 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-02 19:08 UTC (permalink / raw) To: Rob Browning; +Cc: Stefan Monnier, Eli Zaretskii, david, emacs-devel, akrl Rob Browning <rlb@defaultvalue.org> writes: > At the top level, we wanted a way to avoid writing to HOME during > packaging, testing, installs (in this case, it's the .eln files, now > that we've enabled native compilation). > > That could be handled by some way to turn off native compilation, or by > some way to comprehensively divert those writes to another location > (e.g. temp dir). Either is fine, though we'd originally thought the > former might make things a bit easier. Yeah, I think the former is both easier to implement and easier for users. So I'm thinking of introducing a user option like `native-compile-inhibit', which will make Emacs skip the native-comp machinery when loading .elc files. It will default to nil, of course, but perhaps it would be convenient to make it depend on an environment variable like "NATIVE_COMPILE_INHIBIT"? Then users (and the Debian build system) could say "NATIVE_COMPILE_INHIBIT=true emacs ..." when doing testing etc? Would that fit your use case? (This will be for Emacs 29, but you can cherry-pick this for Debian, if that's something you want to do, and it will probably not affect trampolines, since those are necessary for redefining functions.) > Whether or not we can (or should) try to build system-level (root owned) > .eln files along with the .elc files during package installs (as we > already do for .elc files) is a separate question, and I think the more > controversial one? Not controversial from where I'm, er, sitting -- sounds both useful and convenient to me. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 19:08 ` Lars Ingebrigtsen @ 2022-10-02 19:15 ` Eli Zaretskii 2022-10-03 9:12 ` Lars Ingebrigtsen 2022-10-02 20:05 ` Rob Browning 2022-10-05 12:43 ` Andrea Corallo 2 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 19:15 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: rlb, monnier, david, emacs-devel, akrl > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Eli Zaretskii > <eliz@gnu.org>, david@tethera.net, emacs-devel@gnu.org, akrl@sdf.org > Date: Sun, 02 Oct 2022 21:08:15 +0200 > > Rob Browning <rlb@defaultvalue.org> writes: > > > At the top level, we wanted a way to avoid writing to HOME during > > packaging, testing, installs (in this case, it's the .eln files, now > > that we've enabled native compilation). > > > > That could be handled by some way to turn off native compilation, or by > > some way to comprehensively divert those writes to another location > > (e.g. temp dir). Either is fine, though we'd originally thought the > > former might make things a bit easier. > > Yeah, I think the former is both easier to implement and easier for > users. The request that started this discussion was not from users, it was from distributors. If we want to consider providing (yet another) user option for disabling native compilation, then we should: . understand why and in which situations they may need it . what exactly needs to be disabled (e.g.: do you want to disable loading the existing *.eln files?) . understand why the existing options don't already provide solutions I object to introducing new options before we do the above and understand well what we are talking about. > So I'm thinking of introducing a user option like > `native-compile-inhibit', which will make Emacs skip the native-comp > machinery when loading .elc files. It will default to nil, of course, > but perhaps it would be convenient to make it depend on an environment > variable like "NATIVE_COMPILE_INHIBIT"? Then users (and the Debian > build system) could say "NATIVE_COMPILE_INHIBIT=true emacs ..." when > doing testing etc? Would that fit your use case? Their use case is not the use case of Emacs users. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 19:15 ` Eli Zaretskii @ 2022-10-03 9:12 ` Lars Ingebrigtsen 2022-10-03 11:32 ` Lars Ingebrigtsen ` (2 more replies) 0 siblings, 3 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-03 9:12 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rlb, monnier, david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > If we want to consider providing (yet another) user option for > disabling native compilation, then we should: I asked what the user option to disable native compilation was, but didn't get an answer, and here you say "(yet another)", so... what is the current user option to disable native compilation? > . understand why and in which situations they may need it Doing repeatable testing is one obvious situation. > . what exactly needs to be disabled (e.g.: do you want to disable > loading the existing *.eln files?) I don't think anybody has suggested the "e.g." portion, so I'm wondering why you ask? > . understand why the existing options don't already provide > solutions See first paragraph. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 9:12 ` Lars Ingebrigtsen @ 2022-10-03 11:32 ` Lars Ingebrigtsen 2022-10-03 13:07 ` Stefan Monnier 2022-10-03 17:44 ` Eli Zaretskii 2022-10-05 13:18 ` Andrea Corallo 2 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-03 11:32 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rlb, monnier, david, emacs-devel, akrl Lars Ingebrigtsen <larsi@gnus.org> writes: > I asked what the user option to disable native compilation was, but > didn't get an answer, and here you say "(yet another)", so... what is > the current user option to disable native compilation? Looking at the code, I can't see any user options to disable automatic compilation of .elc files. However, there's the confusingly named `native-comp-deferred-compilation' variable which does the trick. And looking at this a bit more, having a user option for this won't quite work, because user options are set in .emacs, and .emacs may trigger loading stuff before handling user options, so it'll have to remain a variable. But should perhaps be renamed to something more understandable, or be deprecated in favour of a `inhibit-native-compilation' variable? In any case, initialising this variable from an environment variable like EMACS_INHIBIT_NATIVE_COMPILATION seems totally trivial. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 11:32 ` Lars Ingebrigtsen @ 2022-10-03 13:07 ` Stefan Monnier 2022-10-03 13:29 ` Lars Ingebrigtsen 2022-10-03 18:21 ` Sean Whitton 0 siblings, 2 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-03 13:07 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, david, emacs-devel, akrl AFAICT so far I've heard of two reasons to "disable automatic native compilation": A) Avoid the "uncontrolled" (mostly in terms of *when* it happens) resource usage associated with it (e.g. battery). B) Avoid writing to $HOME. For (A), I think that (setq native-comp-deferred-compilation nil) seems perfectly sufficient (you can set it early in your (early) init file and if every blue moon some native compilation still happens because of a file loaded before that `setq` it shouldn't be a big deal). For (B) I seem to remember that it's a problem that goes a bit further than just native compilation (we've had bug reports about other files being silently written to ~/.emacs.d in the past and I'd be surprised if there aren't still many such cases, especially if we consider third party packages). So if avoiding all files under $HOME is important, I suspect the only reliable answer is to set $HOME elsewhere (and if you run Emacs under `su`, make sure you use `su -`). This said, we generally do make efforts to try and avoid writing silently to ~/.emacs.d, so maybe we should rethink our choice of ~/.emacs.d/eln-cache as the default location. E.g. maybe when run as root, we should write .eln files somewhere under something like /var/cache? And maybe when not run as root we should favor something underneath ~/.cache? Stefan "who finds the idea of running Emacs as root a bit scary" ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 13:07 ` Stefan Monnier @ 2022-10-03 13:29 ` Lars Ingebrigtsen 2022-10-03 13:42 ` Stefan Monnier 2022-10-03 17:16 ` Eli Zaretskii 2022-10-03 18:21 ` Sean Whitton 1 sibling, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-03 13:29 UTC (permalink / raw) To: Stefan Monnier; +Cc: Eli Zaretskii, rlb, david, emacs-devel, akrl Stefan Monnier <monnier@iro.umontreal.ca> writes: > AFAICT so far I've heard of two reasons to "disable automatic native > compilation": > > A) Avoid the "uncontrolled" (mostly in terms of *when* it happens) > resource usage associated with it (e.g. battery). > > B) Avoid writing to $HOME. > > For (A), I think that (setq native-comp-deferred-compilation nil) seems > perfectly sufficient (you can set it early in your (early) init file and if > every blue moon some native compilation still happens because of a file > loaded before that `setq` it shouldn't be a big deal). I've now introduced the new inhibit-native-compilation variable and deprecated the native-comp-deferred-compilation variable, and also initialised stuff from EMACS_INHIBIT_NATIVE_COMPILATION. (And made trampolines not write to the cache in this case.) > This said, we generally do make efforts to try and avoid writing > silently to ~/.emacs.d, so maybe we should rethink our choice of > ~/.emacs.d/eln-cache as the default location. I think that's a fine location in general, and now there's an understandable and documented way to make that not happen, I think it's even better. > E.g. maybe when run as root, we should write .eln files somewhere under > something like /var/cache? And maybe when not run as root we should > favor something underneath ~/.cache? > > Stefan "who finds the idea of running Emacs as root a bit scary" I run Emacs as root all the time. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 13:29 ` Lars Ingebrigtsen @ 2022-10-03 13:42 ` Stefan Monnier 2022-10-03 14:09 ` Lars Ingebrigtsen 2022-10-05 12:51 ` Andrea Corallo 2022-10-03 17:16 ` Eli Zaretskii 1 sibling, 2 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-03 13:42 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, david, emacs-devel, akrl Lars Ingebrigtsen [2022-10-03 15:29:40] wrote: > Stefan Monnier <monnier@iro.umontreal.ca> writes: >> AFAICT so far I've heard of two reasons to "disable automatic native >> compilation": >> >> A) Avoid the "uncontrolled" (mostly in terms of *when* it happens) >> resource usage associated with it (e.g. battery). >> >> B) Avoid writing to $HOME. >> >> For (A), I think that (setq native-comp-deferred-compilation nil) seems >> perfectly sufficient (you can set it early in your (early) init file and if >> every blue moon some native compilation still happens because of a file >> loaded before that `setq` it shouldn't be a big deal). > > I've now introduced the new inhibit-native-compilation variable and > deprecated the native-comp-deferred-compilation variable, and also > initialised stuff from EMACS_INHIBIT_NATIVE_COMPILATION. > (And made trampolines not write to the cache in this case.) AFAICT for the case (A), we *do* want to save trampolines for the next time around, and those users probably do want to use native compilation (e.g. they would likely set `package-native-compile` to non-nil), just not the deferred kind, so the var name is a bit odd for them. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 13:42 ` Stefan Monnier @ 2022-10-03 14:09 ` Lars Ingebrigtsen 2022-10-03 14:42 ` Stefan Monnier 2022-10-03 17:21 ` Eli Zaretskii 2022-10-05 12:51 ` Andrea Corallo 1 sibling, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-03 14:09 UTC (permalink / raw) To: Stefan Monnier; +Cc: Eli Zaretskii, rlb, david, emacs-devel, akrl Stefan Monnier <monnier@iro.umontreal.ca> writes: > AFAICT for the case (A), we *do* want to save trampolines for the next > time around, and those users probably do want to use native compilation > (e.g. they would likely set `package-native-compile` to non-nil), just > not the deferred kind, so the var name is a bit odd for them. The trampolines are very fast to make, so creating them once per Emacs session doesn't seem like a problem. If that turns out to be an issue, we can tweak the variable. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 14:09 ` Lars Ingebrigtsen @ 2022-10-03 14:42 ` Stefan Monnier 2022-10-03 14:45 ` Lars Ingebrigtsen 2022-10-03 17:21 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-03 14:42 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, david, emacs-devel, akrl Lars Ingebrigtsen [2022-10-03 16:09:35] wrote: > Stefan Monnier <monnier@iro.umontreal.ca> writes: >> AFAICT for the case (A), we *do* want to save trampolines for the next >> time around, > > The trampolines are very fast to make, so creating them once per Emacs > session doesn't seem like a problem. If that turns out to be an issue, > we can tweak the variable. Yes, it's probably not a big deal indeed. But this one is more annoying I think: >> and those users probably do want to use native compilation >> (e.g. they would likely set `package-native-compile` to non-nil), just >> not the deferred kind, so the var name is a bit odd for them. `inhibit-native-compilation` sounds like it really prevents all forms of native compilation, whether "deferred" or not. E.g. going by its name, I'd argue that it's a bug if package.el does native-compile the files when both `package-native-compile` when `inhibit-native-compilation` are non-nil. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 14:42 ` Stefan Monnier @ 2022-10-03 14:45 ` Lars Ingebrigtsen 2022-10-03 14:52 ` Stefan Monnier 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-03 14:45 UTC (permalink / raw) To: Stefan Monnier; +Cc: Eli Zaretskii, rlb, david, emacs-devel, akrl Stefan Monnier <monnier@iro.umontreal.ca> writes: >>> and those users probably do want to use native compilation >>> (e.g. they would likely set `package-native-compile` to non-nil), just >>> not the deferred kind, so the var name is a bit odd for them. > > `inhibit-native-compilation` sounds like it really prevents all forms of > native compilation, whether "deferred" or not. We can expand the variable name, but I couldn't think of a good name. And "deferred" doesn't convey anything. > E.g. going by its name, I'd argue that it's a bug if package.el does > native-compile the files when both `package-native-compile` when > `inhibit-native-compilation` are non-nil. But not by going by the documentation of the variable. 😀 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 14:45 ` Lars Ingebrigtsen @ 2022-10-03 14:52 ` Stefan Monnier 2022-10-03 15:14 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-03 14:52 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, david, emacs-devel, akrl Lars Ingebrigtsen [2022-10-03 16:45:22] wrote: > Stefan Monnier <monnier@iro.umontreal.ca> writes: >>>> and those users probably do want to use native compilation >>>> (e.g. they would likely set `package-native-compile` to non-nil), just >>>> not the deferred kind, so the var name is a bit odd for them. >> `inhibit-native-compilation` sounds like it really prevents all forms of >> native compilation, whether "deferred" or not. > We can expand the variable name, but I couldn't think of a good name. > And "deferred" doesn't convey anything. Lazy? Auto(matic)? Ondemand? >> E.g. going by its name, I'd argue that it's a bug if package.el does >> native-compile the files when both `package-native-compile` when >> `inhibit-native-compilation` are non-nil. > But not by going by the documentation of the variable. 😀 Sorry, my dictionary doesn't know that word. What's "documentation"? Does it have to do with some Norse mythology, maybe? Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 14:52 ` Stefan Monnier @ 2022-10-03 15:14 ` Lars Ingebrigtsen 2022-10-05 12:55 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-03 15:14 UTC (permalink / raw) To: Stefan Monnier; +Cc: Eli Zaretskii, rlb, david, emacs-devel, akrl Stefan Monnier <monnier@iro.umontreal.ca> writes: >> We can expand the variable name, but I couldn't think of a good name. >> And "deferred" doesn't convey anything. > > Lazy? Auto(matic)? Ondemand? `inhibit-lazy-native-compilation' sounds like it makes the compilation less lazy. `inhibit-automatic-native-compilation' might work. > Sorry, my dictionary doesn't know that word. What's "documentation"? > Does it have to do with some Norse mythology, maybe? I think the concept was invented by Loki, the god of mischief and lies. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 15:14 ` Lars Ingebrigtsen @ 2022-10-05 12:55 ` Andrea Corallo 2022-10-05 13:14 ` Lars Ingebrigtsen 2022-10-05 13:55 ` Eli Zaretskii 0 siblings, 2 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 12:55 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Stefan Monnier, Eli Zaretskii, rlb, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Stefan Monnier <monnier@iro.umontreal.ca> writes: > >>> We can expand the variable name, but I couldn't think of a good name. >>> And "deferred" doesn't convey anything. >> >> Lazy? Auto(matic)? Ondemand? > > `inhibit-lazy-native-compilation' sounds like it makes the compilation > less lazy. `inhibit-automatic-native-compilation' might work. Again `inhibit-automatic-native-compilation' does not disable automatic trampoline native compilation :/ BTW I think most of people refers to what was controlled by `native-comp-deferred-compilation' as a jitter mechanism, so I think a better name would have been `inhibit-native-jit-compilation'. I wish this change was not rushed. Best Regards Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 12:55 ` Andrea Corallo @ 2022-10-05 13:14 ` Lars Ingebrigtsen 2022-10-05 13:47 ` Andrea Corallo 2022-10-05 13:55 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 13:14 UTC (permalink / raw) To: Andrea Corallo; +Cc: Stefan Monnier, Eli Zaretskii, rlb, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > Again `inhibit-automatic-native-compilation' does not disable automatic > trampoline native compilation :/ > > BTW I think most of people refers to what was controlled by > `native-comp-deferred-compilation' as a jitter mechanism, so I think a > better name would have been `inhibit-native-jit-compilation'. I wish > this change was not rushed. I don't think `inhibit-native-jit-compilation' conveys anything more to the user than `inhibit-automatic-native-compilation'? Andrea Corallo <akrl@sdf.org> writes: > We can discuss code also on branches, there's no need to install changes > on master to discuss them. > > I'm back now after a long weekend and your proposal of introduciung > `inhibit-native-compilation' is form 2 days and 1 hour ago. Sorry but I > don't want to feel that changes can be rushed into Emacs code and in > order to partecipate to the discussion people can't have some time off. Development takes place on the "master" branch -- code appearing there does not curtail further discussion. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 13:14 ` Lars Ingebrigtsen @ 2022-10-05 13:47 ` Andrea Corallo 0 siblings, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 13:47 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Stefan Monnier, Eli Zaretskii, rlb, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> Again `inhibit-automatic-native-compilation' does not disable automatic >> trampoline native compilation :/ >> >> BTW I think most of people refers to what was controlled by >> `native-comp-deferred-compilation' as a jitter mechanism, so I think a >> better name would have been `inhibit-native-jit-compilation'. I wish >> this change was not rushed. > > I don't think `inhibit-native-jit-compilation' conveys anything more to > the user than `inhibit-automatic-native-compilation'? That's your opinion and I respect it. Still `inhibit-automatic-native-compilation' does *not* disable automatic native compilation but only a mechanism contributing to it, so it's IMO a bad naming decision. > Andrea Corallo <akrl@sdf.org> writes: > >> We can discuss code also on branches, there's no need to install changes >> on master to discuss them. >> >> I'm back now after a long weekend and your proposal of introduciung >> `inhibit-native-compilation' is form 2 days and 1 hour ago. Sorry but I >> don't want to feel that changes can be rushed into Emacs code and in >> order to partecipate to the discussion people can't have some time off. > > Development takes place on the "master" branch -- code appearing there > does not curtail further discussion. I have not said that once code is in master discussion is forbidden. I said that to discuss a change there's *no* requirement to install it in master, especially before sufficient discussion is done on the list for these tricky interfaces. My 2cts are that these mechanisms and changes should be very well thought and participated before being modified. As maintainer of comp.c and related I ask to have this changeset reverted and then we restart thinking again what's the best change (if any) needed here. Thanks Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 12:55 ` Andrea Corallo 2022-10-05 13:14 ` Lars Ingebrigtsen @ 2022-10-05 13:55 ` Eli Zaretskii 2022-10-05 14:08 ` Andrea Corallo 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-05 13:55 UTC (permalink / raw) To: Andrea Corallo; +Cc: larsi, monnier, rlb, david, emacs-devel > From: Andrea Corallo <akrl@sdf.org> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Eli Zaretskii > <eliz@gnu.org>, > rlb@defaultvalue.org, david@tethera.net, emacs-devel@gnu.org > Date: Wed, 05 Oct 2022 12:55:40 +0000 > > I wish this change was not rushed. Me too. Especially since I'm still struggling to understand the rationale of the Debian packagers to request this. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 13:55 ` Eli Zaretskii @ 2022-10-05 14:08 ` Andrea Corallo 0 siblings, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 14:08 UTC (permalink / raw) To: Eli Zaretskii; +Cc: larsi, monnier, rlb, david, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Andrea Corallo <akrl@sdf.org> >> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Eli Zaretskii >> <eliz@gnu.org>, >> rlb@defaultvalue.org, david@tethera.net, emacs-devel@gnu.org >> Date: Wed, 05 Oct 2022 12:55:40 +0000 >> >> I wish this change was not rushed. > > Me too. Especially since I'm still struggling to understand the > rationale of the Debian packagers to request this. Agree and that's not all! As we know native compilations happens automatically for: 1- Trampolines, this we *cannot* disable in any case. 2- Deferred/jit compilation, this is *already* disabled in non interactive sessions! (the way I guess Debian Emacs process is used for package instgallation). So I think this changeset does not help Debian packagers at all but ATM is just more surface for future issues! Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 14:09 ` Lars Ingebrigtsen 2022-10-03 14:42 ` Stefan Monnier @ 2022-10-03 17:21 ` Eli Zaretskii 2022-10-03 17:39 ` Lars Ingebrigtsen 2022-10-05 13:09 ` Andrea Corallo 1 sibling, 2 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 17:21 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: monnier, rlb, david, emacs-devel, akrl > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Eli Zaretskii <eliz@gnu.org>, rlb@defaultvalue.org, david@tethera.net, > emacs-devel@gnu.org, akrl@sdf.org > Date: Mon, 03 Oct 2022 16:09:35 +0200 > > Stefan Monnier <monnier@iro.umontreal.ca> writes: > > > AFAICT for the case (A), we *do* want to save trampolines for the next > > time around, and those users probably do want to use native compilation > > (e.g. they would likely set `package-native-compile` to non-nil), just > > not the deferred kind, so the var name is a bit odd for them. > > The trampolines are very fast to make, so creating them once per Emacs > session doesn't seem like a problem. If that turns out to be an issue, > we can tweak the variable. I think that entire changeset should be reverted. It is not well thought out, and ion some aspects downright dangerous. The environment variable is especially egregious: it will affect all the sub-process Emacsen as well, something that will bite users, a lesson we've learned long ago with the likes of EMACSLOADPATH. Such changes should not be installed without a great deal of careful thought and consideration of the different use cases. Which I tried to explain all the way, but apparently to deaf ears. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 17:21 ` Eli Zaretskii @ 2022-10-03 17:39 ` Lars Ingebrigtsen 2022-10-03 17:52 ` Eli Zaretskii 2022-10-05 13:05 ` Andrea Corallo 2022-10-05 13:09 ` Andrea Corallo 1 sibling, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-03 17:39 UTC (permalink / raw) To: Eli Zaretskii; +Cc: monnier, rlb, david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > I think that entire changeset should be reverted. It is not well > thought out, and ion some aspects downright dangerous. The > environment variable is especially egregious: it will affect all the > sub-process Emacsen as well, something that will bite users, a lesson > we've learned long ago with the likes of EMACSLOADPATH. How is setting EMACS_INHIBIT_NATIVE_COMPILE dangerous? > Such changes should not be installed without a great deal of careful > thought and consideration of the different use cases. Which I tried > to explain all the way, but apparently to deaf ears. The discussion has been going for apparently weeks no with no progress. It's easier to discuss code when there's code to discuss. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 17:39 ` Lars Ingebrigtsen @ 2022-10-03 17:52 ` Eli Zaretskii 2022-10-05 13:05 ` Andrea Corallo 1 sibling, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 17:52 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: monnier, rlb, david, emacs-devel, akrl > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: monnier@iro.umontreal.ca, rlb@defaultvalue.org, david@tethera.net, > emacs-devel@gnu.org, akrl@sdf.org > Date: Mon, 03 Oct 2022 19:39:04 +0200 > > Eli Zaretskii <eliz@gnu.org> writes: > > > I think that entire changeset should be reverted. It is not well > > thought out, and ion some aspects downright dangerous. The > > environment variable is especially egregious: it will affect all the > > sub-process Emacsen as well, something that will bite users, a lesson > > we've learned long ago with the likes of EMACSLOADPATH. > > How is setting EMACS_INHIBIT_NATIVE_COMPILE dangerous? I just explained it, above. > > Such changes should not be installed without a great deal of careful > > thought and consideration of the different use cases. Which I tried > > to explain all the way, but apparently to deaf ears. > > The discussion has been going for apparently weeks no with no progress. "Weeks"? Boy, I guess time really flies when one's enjoying: the first message of this thread was posted less than a week ago: https://lists.gnu.org/archive/html/emacs-devel/2022-09/msg01911.html And it really started just 2 days ago: https://lists.gnu.org/archive/html/emacs-devel/2022-10/msg00015.html But whatever. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 17:39 ` Lars Ingebrigtsen 2022-10-03 17:52 ` Eli Zaretskii @ 2022-10-05 13:05 ` Andrea Corallo 1 sibling, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 13:05 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, monnier, rlb, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Eli Zaretskii <eliz@gnu.org> writes: > >> I think that entire changeset should be reverted. It is not well >> thought out, and ion some aspects downright dangerous. The >> environment variable is especially egregious: it will affect all the >> sub-process Emacsen as well, something that will bite users, a lesson >> we've learned long ago with the likes of EMACSLOADPATH. > > How is setting EMACS_INHIBIT_NATIVE_COMPILE dangerous? > >> Such changes should not be installed without a great deal of careful >> thought and consideration of the different use cases. Which I tried >> to explain all the way, but apparently to deaf ears. > > The discussion has been going for apparently weeks no with no progress. > It's easier to discuss code when there's code to discuss. We can discuss code also on branches, there's no need to install changes on master to discuss them. I'm back now after a long weekend and your proposal of introduciung `inhibit-native-compilation' is form 2 days and 1 hour ago. Sorry but I don't want to feel that changes can be rushed into Emacs code and in order to partecipate to the discussion people can't have some time off. Best Regards Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 17:21 ` Eli Zaretskii 2022-10-03 17:39 ` Lars Ingebrigtsen @ 2022-10-05 13:09 ` Andrea Corallo 1 sibling, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 13:09 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Lars Ingebrigtsen, monnier, rlb, david, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Lars Ingebrigtsen <larsi@gnus.org> >> Cc: Eli Zaretskii <eliz@gnu.org>, rlb@defaultvalue.org, david@tethera.net, >> emacs-devel@gnu.org, akrl@sdf.org >> Date: Mon, 03 Oct 2022 16:09:35 +0200 >> >> Stefan Monnier <monnier@iro.umontreal.ca> writes: >> >> > AFAICT for the case (A), we *do* want to save trampolines for the next >> > time around, and those users probably do want to use native compilation >> > (e.g. they would likely set `package-native-compile` to non-nil), just >> > not the deferred kind, so the var name is a bit odd for them. >> >> The trampolines are very fast to make, so creating them once per Emacs >> session doesn't seem like a problem. If that turns out to be an issue, >> we can tweak the variable. > > I think that entire changeset should be reverted. Definitely agree here. Other than the (IMO) bad naming, is still not clear to me what's exactly the problem is trying to solve. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 13:42 ` Stefan Monnier 2022-10-03 14:09 ` Lars Ingebrigtsen @ 2022-10-05 12:51 ` Andrea Corallo 1 sibling, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 12:51 UTC (permalink / raw) To: Stefan Monnier; +Cc: Lars Ingebrigtsen, Eli Zaretskii, rlb, david, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > Lars Ingebrigtsen [2022-10-03 15:29:40] wrote: >> Stefan Monnier <monnier@iro.umontreal.ca> writes: >>> AFAICT so far I've heard of two reasons to "disable automatic native >>> compilation": >>> >>> A) Avoid the "uncontrolled" (mostly in terms of *when* it happens) >>> resource usage associated with it (e.g. battery). >>> >>> B) Avoid writing to $HOME. >>> >>> For (A), I think that (setq native-comp-deferred-compilation nil) seems >>> perfectly sufficient (you can set it early in your (early) init file and if >>> every blue moon some native compilation still happens because of a file >>> loaded before that `setq` it shouldn't be a big deal). >> >> I've now introduced the new inhibit-native-compilation variable and >> deprecated the native-comp-deferred-compilation variable, and also >> initialised stuff from EMACS_INHIBIT_NATIVE_COMPILATION. >> (And made trampolines not write to the cache in this case.) > > AFAICT for the case (A), we *do* want to save trampolines for the next > time around, and those users probably do want to use native compilation > (e.g. they would likely set `package-native-compile` to non-nil), just > not the deferred kind, so the var name is a bit odd for them. I find this name odd as well. `inhibit-native-compilation' is *not* disabling native compilation for two reasons: 1- One can still inoke it manually 2- Trampoline automatic ocmpilation is still active Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 13:29 ` Lars Ingebrigtsen 2022-10-03 13:42 ` Stefan Monnier @ 2022-10-03 17:16 ` Eli Zaretskii 1 sibling, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 17:16 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: monnier, rlb, david, emacs-devel, akrl > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Eli Zaretskii <eliz@gnu.org>, rlb@defaultvalue.org, david@tethera.net, > emacs-devel@gnu.org, akrl@sdf.org > Date: Mon, 03 Oct 2022 15:29:40 +0200 > > I've now introduced the new inhibit-native-compilation variable and > deprecated the native-comp-deferred-compilation variable, and also > initialised stuff from EMACS_INHIBIT_NATIVE_COMPILATION. > > (And made trampolines not write to the cache in this case.) Thanks a lot for ignoring everything I wrote during the last two days! Makes one wonder what kind of development community this is. And what am I doing here, anyway. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 13:07 ` Stefan Monnier 2022-10-03 13:29 ` Lars Ingebrigtsen @ 2022-10-03 18:21 ` Sean Whitton 2022-10-03 20:43 ` Stefan Monnier 1 sibling, 1 reply; 343+ messages in thread From: Sean Whitton @ 2022-10-03 18:21 UTC (permalink / raw) To: Stefan Monnier Cc: Lars Ingebrigtsen, Eli Zaretskii, rlb, david, emacs-devel, akrl Hello, On Mon 03 Oct 2022 at 09:07AM -04, Stefan Monnier wrote: > AFAICT so far I've heard of two reasons to "disable automatic native > compilation": > > A) Avoid the "uncontrolled" (mostly in terms of *when* it happens) > resource usage associated with it (e.g. battery). > > B) Avoid writing to $HOME. > > For (A), I think that (setq native-comp-deferred-compilation nil) seems > perfectly sufficient (you can set it early in your (early) init file and if > every blue moon some native compilation still happens because of a file > loaded before that `setq` it shouldn't be a big deal). We'd also like to be able to compile and install system-wide -- it would be good to be able to benefit from native compilation without it happening by surrpise. -- Sean Whitton ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 18:21 ` Sean Whitton @ 2022-10-03 20:43 ` Stefan Monnier 0 siblings, 0 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-03 20:43 UTC (permalink / raw) To: Sean Whitton Cc: Lars Ingebrigtsen, Eli Zaretskii, rlb, david, emacs-devel, akrl > We'd also like to be able to compile and install system-wide -- it would > be good to be able to benefit from native compilation without it > happening by surrpise. I haven't heard any specific request in this regard: is there something preventing you from doing that (for your definition of "that")? Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 9:12 ` Lars Ingebrigtsen 2022-10-03 11:32 ` Lars Ingebrigtsen @ 2022-10-03 17:44 ` Eli Zaretskii 2022-10-05 13:18 ` Andrea Corallo 2 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 17:44 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: rlb, monnier, david, emacs-devel, akrl > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, > emacs-devel@gnu.org, akrl@sdf.org > Date: Mon, 03 Oct 2022 11:12:22 +0200 > > Eli Zaretskii <eliz@gnu.org> writes: > > > If we want to consider providing (yet another) user option for > > disabling native compilation, then we should: > > I asked what the user option to disable native compilation was, but > didn't get an answer, and here you say "(yet another)", so... what is > the current user option to disable native compilation? You haven't bothered waiting for my answer, so I won't. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 9:12 ` Lars Ingebrigtsen 2022-10-03 11:32 ` Lars Ingebrigtsen 2022-10-03 17:44 ` Eli Zaretskii @ 2022-10-05 13:18 ` Andrea Corallo 2022-10-05 14:01 ` Lars Ingebrigtsen 2022-10-05 20:37 ` Lars Ingebrigtsen 2 siblings, 2 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 13:18 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Eli Zaretskii <eliz@gnu.org> writes: > >> If we want to consider providing (yet another) user option for >> disabling native compilation, then we should: > > I asked what the user option to disable native compilation was, but > didn't get an answer, and here you say "(yet another)", so... what is > the current user option to disable native compilation? Sorry for being late: `native-comp-deferred-compilation' and `load-no-native'. >> . understand why and in which situations they may need it > > Doing repeatable testing is one obvious situation. I think the two previously mentioned knobs should be sufficient for repeatable testing no? Also repeatable testing are most likely executed in batch mode and... ...surprise surprise deferred compilation is *already* *disabled* in this mode!! I've the impression no one mentioned this small detail in this huge thread, but still we have installed changes to disable a feature for debian pkg installation that is in fact already disabled :( :( Best Regards Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 13:18 ` Andrea Corallo @ 2022-10-05 14:01 ` Lars Ingebrigtsen 2022-10-05 14:17 ` Eli Zaretskii 2022-10-05 14:25 ` Andrea Corallo 2022-10-05 20:37 ` Lars Ingebrigtsen 1 sibling, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 14:01 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > Also repeatable testing are most likely executed in batch mode and... > > ...surprise surprise deferred compilation is *already* *disabled* in > this mode!! Not all testing happens in batch mode. Andrea Corallo <akrl@sdf.org> writes: >> I'm not sure we need to save the trampolines at all (in this >> don't-do-native-compilation configuration) -- Andrea probably knows. >> Andrea, would it be possible to just create and use the trampolines >> without writing them to a file? > > No Yes, it is. (That is, the trampolines get written to /tmp, but can be deleted after loading.) Andrea Corallo <akrl@sdf.org> writes: > That's your opinion and I respect it. Still > `inhibit-automatic-native-compilation' does *not* disable automatic > native compilation but only a mechanism contributing to it, so it's IMO > a bad naming decision. Like I said earlier, if anybody has a better name for the variable, renaming it is fine, but it has to be an improvement. > I have not said that once code is in master discussion is forbidden. I > said that to discuss a change there's *no* requirement to install it in > master, especially before sufficient discussion is done on the list for > these tricky interfaces. My 2cts are that these mechanisms and changes > should be very well thought and participated before being modified. > > As maintainer of comp.c and related I ask to have this changeset > reverted and then we restart thinking again what's the best change (if > any) needed here. I don't see any advantages to doing that -- the changes that are in now seem to work fine for the stated use cases (which are both "don't write to $HOME while testing" and "I want to use a AOT-compiled Emacs, but don't do any further JIT compilation while running Emacs"). ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 14:01 ` Lars Ingebrigtsen @ 2022-10-05 14:17 ` Eli Zaretskii 2022-10-05 14:27 ` Lars Ingebrigtsen 2022-10-05 14:25 ` Andrea Corallo 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-05 14:17 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: akrl, rlb, monnier, david, emacs-devel > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Eli Zaretskii <eliz@gnu.org>, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org > Date: Wed, 05 Oct 2022 16:01:35 +0200 > > > As maintainer of comp.c and related I ask to have this changeset > > reverted and then we restart thinking again what's the best change (if > > any) needed here. > > I don't see any advantages to doing that -- the changes that are in now > seem to work fine for the stated use cases (which are both "don't write > to $HOME while testing" That use case is not yet clear, not at all. > and "I want to use a AOT-compiled Emacs, but don't do any further > JIT compilation while running Emacs"). This use case makes no sense at all. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 14:17 ` Eli Zaretskii @ 2022-10-05 14:27 ` Lars Ingebrigtsen 2022-10-05 16:42 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 14:27 UTC (permalink / raw) To: Eli Zaretskii; +Cc: akrl, rlb, monnier, david, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> and "I want to use a AOT-compiled Emacs, but don't do any further >> JIT compilation while running Emacs"). > > This use case makes no sense at all. It's been requested several times, whether it makes sense to you or not. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 14:27 ` Lars Ingebrigtsen @ 2022-10-05 16:42 ` Eli Zaretskii 2022-10-05 17:08 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-05 16:42 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: akrl, rlb, monnier, david, emacs-devel > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: akrl@sdf.org, rlb@defaultvalue.org, monnier@iro.umontreal.ca, > david@tethera.net, emacs-devel@gnu.org > Date: Wed, 05 Oct 2022 16:27:38 +0200 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> and "I want to use a AOT-compiled Emacs, but don't do any further > >> JIT compilation while running Emacs"). > > > > This use case makes no sense at all. > > It's been requested several times, whether it makes sense to you or not. Where it was requested? Please point me to those requests. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 16:42 ` Eli Zaretskii @ 2022-10-05 17:08 ` Lars Ingebrigtsen 2022-10-05 17:15 ` Eli Zaretskii 2022-10-06 0:40 ` Po Lu 0 siblings, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 17:08 UTC (permalink / raw) To: Eli Zaretskii; +Cc: akrl, rlb, monnier, david, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > Where it was requested? Please point me to those requests. Sorry, I don't have a list -- people have been mentioning this for years. Po Lu seemed to request it just now, though. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 17:08 ` Lars Ingebrigtsen @ 2022-10-05 17:15 ` Eli Zaretskii 2022-10-06 0:41 ` Po Lu 2022-10-06 0:40 ` Po Lu 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-05 17:15 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: akrl, rlb, monnier, david, emacs-devel > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: akrl@sdf.org, rlb@defaultvalue.org, monnier@iro.umontreal.ca, > david@tethera.net, emacs-devel@gnu.org > Date: Wed, 05 Oct 2022 19:08:00 +0200 > > Eli Zaretskii <eliz@gnu.org> writes: > > > Where it was requested? Please point me to those requests. > > Sorry, I don't have a list -- people have been mentioning this for > years. I don't remember even a single request like that. I suspect there's a confusion of sorts. > Po Lu seemed to request it just now, though. He did? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 17:15 ` Eli Zaretskii @ 2022-10-06 0:41 ` Po Lu 0 siblings, 0 replies; 343+ messages in thread From: Po Lu @ 2022-10-06 0:41 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Lars Ingebrigtsen, akrl, rlb, monnier, david, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > He did? Not really, but I build my own Emacs. The difference may be more meaningful for packagers. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 17:08 ` Lars Ingebrigtsen 2022-10-05 17:15 ` Eli Zaretskii @ 2022-10-06 0:40 ` Po Lu 2022-10-06 6:26 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Po Lu @ 2022-10-06 0:40 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, akrl, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Po Lu seemed to request it just now, though. Nope, my solution is to wait the 30 minutes for the fans to subside after startup, every time I update Emacs. I think the end result and time spent is more-or-less the same as a NATIVE_FULL_AOT build of Emacs. Thanks. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 0:40 ` Po Lu @ 2022-10-06 6:26 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-06 6:26 UTC (permalink / raw) To: Po Lu; +Cc: larsi, akrl, rlb, monnier, david, emacs-devel > From: Po Lu <luangruo@yahoo.com> > Cc: Eli Zaretskii <eliz@gnu.org>, akrl@sdf.org, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org > Date: Thu, 06 Oct 2022 08:40:36 +0800 > > Lars Ingebrigtsen <larsi@gnus.org> writes: > > > Po Lu seemed to request it just now, though. > > Nope, my solution is to wait the 30 minutes for the fans to subside > after startup, every time I update Emacs. I'd like to point out that the situation with frequent updates of Emacs that require recompilation of *.eln files is peculiar to Emacs development on the master branch, and will generally rare if ever happen for "normal" users, and even for us developers on the release branch. Like I said before, the pattern of JIT compilation in "normal" usage is that it happens for a few minutes after installing a new version of Emacs, and thereafter happens only rarely. > I think the end result and time spent is more-or-less the same as a > NATIVE_FULL_AOT build of Emacs. I think NATIVE_FULL_AOT takes more time, because it compiles much more than an average user will ever use. (But I don't object to supporting such a build for people who want it.) ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 14:01 ` Lars Ingebrigtsen 2022-10-05 14:17 ` Eli Zaretskii @ 2022-10-05 14:25 ` Andrea Corallo 2022-10-05 14:29 ` Lars Ingebrigtsen 1 sibling, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 14:25 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> Also repeatable testing are most likely executed in batch mode and... >> >> ...surprise surprise deferred compilation is *already* *disabled* in >> this mode!! > > Not all testing happens in batch mode. > > Andrea Corallo <akrl@sdf.org> writes: > >>> I'm not sure we need to save the trampolines at all (in this >>> don't-do-native-compilation configuration) -- Andrea probably knows. >>> Andrea, would it be possible to just create and use the trampolines >>> without writing them to a file? >> >> No > > Yes, it is. (That is, the trampolines get written to /tmp, but can be > deleted after loading.) You asked: "would it be possible to just create and use the trampolines without writing them to a file?" I aswered "no". Of course writing a file into /tmp is still writing to a file. > Andrea Corallo <akrl@sdf.org> writes: > >> That's your opinion and I respect it. Still >> `inhibit-automatic-native-compilation' does *not* disable automatic >> native compilation but only a mechanism contributing to it, so it's IMO >> a bad naming decision. > > Like I said earlier, if anybody has a better name for the variable, > renaming it is fine, but it has to be an improvement. I think `inhibit-jit-native-compilation' is better as I believe users perceive the JIT word related to user code and not internal mechanisms as trampolines. I've the perception that this change was done without the full picture in mind of how the native compiler and his mechanisms works. As a result the current naming is IMO just wrong, and as such is a step backward the original state. As maintainer of the native compiler is my duty to point this out and ask for reversion. >> I have not said that once code is in master discussion is forbidden. I >> said that to discuss a change there's *no* requirement to install it in >> master, especially before sufficient discussion is done on the list for >> these tricky interfaces. My 2cts are that these mechanisms and changes >> should be very well thought and participated before being modified. >> >> As maintainer of comp.c and related I ask to have this changeset >> reverted and then we restart thinking again what's the best change (if >> any) needed here. > > I don't see any advantages to doing that -- the changes that are in now > seem to work fine for the stated use cases (which are both "don't write > to $HOME while testing" and "I want to use a AOT-compiled Emacs, but > don't do any further JIT compilation while running Emacs"). I explained in two other mails that these changesets are orthogonal to what the user asked and don't help them. Also is still not clear why the user asked for this knob. It is *extremely* important that we analyze the usecase *before* introducing new interfaces. The user might ask for a new interface without knowing we can provide or suggest a better solution. It might even already exists!! Developers can't just add mechanisms without a full understanding of the current ones just because there was a user request, even worst without an in deep discussion, sorry this is IMO just recipe for disaster. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 14:25 ` Andrea Corallo @ 2022-10-05 14:29 ` Lars Ingebrigtsen 2022-10-05 14:48 ` Andrea Corallo 2022-10-05 16:43 ` Eli Zaretskii 0 siblings, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 14:29 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > I think `inhibit-jit-native-compilation' is better as I believe users > perceive the JIT word related to user code and not internal mechanisms > as trampolines. It's possible -- I'm not married to the current name. Perhaps we should take a poll? > I've the perception that this change was done without the full picture > in mind of how the native compiler and his mechanisms works. As a > result the current naming is IMO just wrong, and as such is a step > backward the original state. I don't know where you got that perception from. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 14:29 ` Lars Ingebrigtsen @ 2022-10-05 14:48 ` Andrea Corallo 2022-10-05 14:58 ` Lars Ingebrigtsen 2022-10-05 16:43 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 14:48 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> I think `inhibit-jit-native-compilation' is better as I believe users >> perceive the JIT word related to user code and not internal mechanisms >> as trampolines. > > It's possible -- I'm not married to the current name. Perhaps we should > take a poll? > >> I've the perception that this change was done without the full picture >> in mind of how the native compiler and his mechanisms works. As a >> result the current naming is IMO just wrong, and as such is a step >> backward the original state. > > I don't know where you got that perception from. Well to give few examples you were not aware of: the `load-no-native' mechanism, the fact that deferred compilation is not the only mechanism concurring to automatic native compilation (and that's why it was named as such and not just automatic native compilation), the fact that native compilation does not happen in non interactive sessions. There is nothing wrong with that, the native compiler is a complex machine and its interface as well, but still: there's no single aspect of this changeset that I see as an improvement, so as maintainer of the native compiler I ask to have it reverted. Thanks Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 14:48 ` Andrea Corallo @ 2022-10-05 14:58 ` Lars Ingebrigtsen 2022-10-05 15:12 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 14:58 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > Well to give few examples you were not aware of: the `load-no-native' > mechanism, the fact that deferred compilation is not the only mechanism > concurring to automatic native compilation (and that's why it was named > as such and not just automatic native compilation), the fact that native > compilation does not happen in non interactive sessions. I didn't know about the first (because it's badly named and undocumented, as well as totally irrelevant to the discussion in this thread), but I was aware of all the other things here, and I'm not sure why you'd think otherwise. My perception here is that you're mostly angry that somebody else is working on your code -- but that's pretty common. Many people feel proprietary towards code they've written. > There is nothing wrong with that, the native compiler is a complex > machine and its interface as well, but still: there's no single aspect > of this changeset that I see as an improvement, so as maintainer of the > native compiler I ask to have it reverted. Like I said before, the code solves (at least) two real problems, so I'm not in favour of doing so. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 14:58 ` Lars Ingebrigtsen @ 2022-10-05 15:12 ` Andrea Corallo 2022-10-05 15:24 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 15:12 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> Well to give few examples you were not aware of: the `load-no-native' >> mechanism, the fact that deferred compilation is not the only mechanism >> concurring to automatic native compilation (and that's why it was named >> as such and not just automatic native compilation), the fact that native >> compilation does not happen in non interactive sessions. > > I didn't know about the first (because it's badly named and > undocumented, as well as totally irrelevant to the discussion in this > thread), but I was aware of all the other things here, and I'm not sure > why you'd think otherwise. Well because I cannot understand why one would do any of these changes if these mechanisms are known. > My perception here is that you're mostly angry that somebody else is > working on your code -- but that's pretty common. Many people feel > proprietary towards code they've written. This is not the case at all, please trust me, your changeset does two things: 1- change the name a knob, but it goes from a maybe un-intuitive one to just (as explained) a plain wrong one. 2- add a mechanism that (as explained) cannot help with the user request in this discussion at all. These are both not improvements, this is the reason I'm asking for it to be reverted now. Please don't accuse me to feel the ownership on this code, I've never objected to other changes on this done on master as I thought were correct. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 15:12 ` Andrea Corallo @ 2022-10-05 15:24 ` Lars Ingebrigtsen 2022-10-05 15:36 ` Lars Ingebrigtsen 2022-10-05 16:04 ` Andrea Corallo 0 siblings, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 15:24 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > This is not the case at all, please trust me, your changeset does two > things: > > 1- change the name a knob, but it goes from a maybe un-intuitive one to > just (as explained) a plain wrong one. It goes from an un-intuitive (and undocumented) one to an intuitively-named (and documented) one. In my opinion, the old variable is just as wrong as the current one (because it seemed to imply that compilation would be immediate instead of deferred). > 2- add a mechanism that (as explained) cannot help with the user request > in this discussion at all. And I've explained twice now that 2 is wrong -- these changes do exactly what was requested: 1) It allows testing without writing to $HOME. (This has nothing to do with --batch -- testing happens in interactive Emacsen, too.) 2) It allows people to run an AOT Emacs without triggering further compilation. If you have a suggestion for an alternative change that achieves these two things, I'm all ears. (But if your objection is "people shouldn't want those two things", I'm down to just two ears again.) ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 15:24 ` Lars Ingebrigtsen @ 2022-10-05 15:36 ` Lars Ingebrigtsen 2022-10-05 16:10 ` Andrea Corallo 2022-11-05 1:09 ` Juanma Barranquero 2022-10-05 16:04 ` Andrea Corallo 1 sibling, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 15:36 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > 1) It allows testing without writing to $HOME. (This has nothing to > do with --batch -- testing happens in interactive Emacsen, too.) (Which reminds me -- one thing I've been pondering for a while is whether "emacs -Q" should imply having the JIT off or not. We use "-Q" to get a repeatable Emacs for users, so it would make some sense to have -Q now imply `inhibit-automatic-native-compilation'. But I'm not sure. If it does, should it also imply `load-no-native', which should then be renamed to something less awkward and documented? Probably not -- but should there then be yet another twiddle to inhibit from ~/.emacs/eln-cache/? That seems like overkill, probably, but it would be in the spirit of "-Q".) ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 15:36 ` Lars Ingebrigtsen @ 2022-10-05 16:10 ` Andrea Corallo 2022-10-05 16:13 ` Lars Ingebrigtsen 2022-11-05 1:09 ` Juanma Barranquero 1 sibling, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 16:10 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Lars Ingebrigtsen <larsi@gnus.org> writes: > >> 1) It allows testing without writing to $HOME. (This has nothing to >> do with --batch -- testing happens in interactive Emacsen, too.) > > (Which reminds me -- one thing I've been pondering for a while is > whether "emacs -Q" should imply having the JIT off or not. We use "-Q" > to get a repeatable Emacs for users, so it would make some sense to have > -Q now imply `inhibit-automatic-native-compilation'. Noooo please!! -Q has just one single clear meaning, let please not add other implications to it!! The situation would become very quickly unmanageable. And BTW -Q ATM is usefull for testing the native compiler as well :/ :/ Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 16:10 ` Andrea Corallo @ 2022-10-05 16:13 ` Lars Ingebrigtsen 2022-10-05 17:58 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 16:13 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: >> (Which reminds me -- one thing I've been pondering for a while is >> whether "emacs -Q" should imply having the JIT off or not. We use "-Q" >> to get a repeatable Emacs for users, so it would make some sense to have >> -Q now imply `inhibit-automatic-native-compilation'. > > Noooo please!! -Q has just one single clear meaning, let please not add > other implications to it!! The situation would become very quickly > unmanageable. Yes, that one clear meaning is: -Q, --quick Similar to "-q --no-site-file --no-splash". Also, avoid processing X resources. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 16:13 ` Lars Ingebrigtsen @ 2022-10-05 17:58 ` Andrea Corallo 2022-10-05 18:28 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 17:58 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >>> (Which reminds me -- one thing I've been pondering for a while is >>> whether "emacs -Q" should imply having the JIT off or not. We use "-Q" >>> to get a repeatable Emacs for users, so it would make some sense to have >>> -Q now imply `inhibit-automatic-native-compilation'. >> >> Noooo please!! -Q has just one single clear meaning, let please not add >> other implications to it!! The situation would become very quickly >> unmanageable. > > Yes, that one clear meaning is: > > -Q, --quick > Similar to "-q --no-site-file --no-splash". Also, avoid > processing X resources. Exactly, adding complexity involving the execution engine here is IMO just searching for troubles (and having -Q not usable for debugging the execution engine itself). Let's please do not write patches solving issues we never encountered or implementing unrequested features. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 17:58 ` Andrea Corallo @ 2022-10-05 18:28 ` Lars Ingebrigtsen 2022-10-05 23:25 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 18:28 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > Exactly, adding complexity involving the execution engine here is IMO > just searching for troubles (and having -Q not usable for debugging the > execution engine itself). You missed my point. "-Q" doesn't have "one clear meaning" -- it's a mish-mash of stuff, affecting how many things in Emacs works (like Customize, X resources, packages, and so on). ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 18:28 ` Lars Ingebrigtsen @ 2022-10-05 23:25 ` Andrea Corallo 2022-10-05 23:33 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 23:25 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> Exactly, adding complexity involving the execution engine here is IMO >> just searching for troubles (and having -Q not usable for debugging the >> execution engine itself). > > You missed my point. "-Q" doesn't have "one clear meaning" -- it's a > mish-mash of stuff, affecting how many things in Emacs works (like > Customize, X resources, packages, and so on). Still I think none of these as anything to do with the execution engine (and should not). ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 23:25 ` Andrea Corallo @ 2022-10-05 23:33 ` Lars Ingebrigtsen 2022-10-06 0:45 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 23:33 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: >> You missed my point. "-Q" doesn't have "one clear meaning" -- it's a >> mish-mash of stuff, affecting how many things in Emacs works (like >> Customize, X resources, packages, and so on). > > Still I think none of these as anything to do with the execution engine > (and should not). Did I say they did? You claimed that "-Q" has "one clear meaning", and that's false. "-Q" means, sort of, "without any local changes", which is really wishy-washy semantics. Which reminds me of another thing -- was there a reason that --batch implies "avoid most JIT compilation", like it does now, by the way? It's always seemed pretty odd to me, because JITting could well be quite useful when doing batch stuff, and there's no handy way to switch it on when doing --batch. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 23:33 ` Lars Ingebrigtsen @ 2022-10-06 0:45 ` Andrea Corallo 2022-10-06 0:55 ` Lars Ingebrigtsen 2022-10-06 6:33 ` Eli Zaretskii 0 siblings, 2 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-06 0:45 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >>> You missed my point. "-Q" doesn't have "one clear meaning" -- it's a >>> mish-mash of stuff, affecting how many things in Emacs works (like >>> Customize, X resources, packages, and so on). >> >> Still I think none of these as anything to do with the execution engine >> (and should not). > > Did I say they did? No you didn't nor I claimed you did. I'm saying -Q does control a totally different area of Emacs that is not the execution engine. For instance it does not affect bytecode execution, consequentially I don't see why it should affect native code execution. > You claimed that "-Q" has "one clear meaning", and > that's false. "-Q" means, sort of, "without any local changes", which > is really wishy-washy semantics. Sorry for me this meaning is clear. > Which reminds me of another thing -- was there a reason that --batch > implies "avoid most JIT compilation", like it does now, by the way? > It's always seemed pretty odd to me, because JITting could well be quite > useful when doing batch stuff, and there's no handy way to switch it on > when doing --batch. The rational is that "tipically" batch executions are short in time, so spawning there native async compilation would be a waste of cycles if they do not complete in time. I think no one has statistical prove of this, but the rational was at least discussed and I believe agreed here. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 0:45 ` Andrea Corallo @ 2022-10-06 0:55 ` Lars Ingebrigtsen 2022-10-06 6:33 ` Eli Zaretskii 1 sibling, 0 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-06 0:55 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > The rational is that "tipically" batch executions are short in time, so > spawning there native async compilation would be a waste of cycles if > they do not complete in time. I think no one has statistical prove of > this, but the rational was at least discussed and I believe agreed here. Sure, sounds reasonable as a default, at least. But perhaps there should be a way to allow JIT in --batch? It looks like currently the only way is to set `noninteractive', which has wider side effects. Yet Another Variable here would be possible, but perhaps some other mechanism would be nicer. Hm... Perhaps inhibit-automatic-native-compilation should be refashioned into something else, like `native-compilation-types' or something, and would default to 'all in non-batch, but could be 'force to force it even in --batch? (And possibly also 'trampoline to save trampolines without doing any more JIT-it, like Stefan M sort of suggested.) ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 0:45 ` Andrea Corallo 2022-10-06 0:55 ` Lars Ingebrigtsen @ 2022-10-06 6:33 ` Eli Zaretskii 1 sibling, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-06 6:33 UTC (permalink / raw) To: Andrea Corallo; +Cc: larsi, rlb, monnier, david, emacs-devel > From: Andrea Corallo <akrl@sdf.org> > Cc: Eli Zaretskii <eliz@gnu.org>, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org > Date: Thu, 06 Oct 2022 00:45:49 +0000 > > > Which reminds me of another thing -- was there a reason that --batch > > implies "avoid most JIT compilation", like it does now, by the way? > > It's always seemed pretty odd to me, because JITting could well be quite > > useful when doing batch stuff, and there's no handy way to switch it on > > when doing --batch. > > The rational is that "tipically" batch executions are short in time, so > spawning there native async compilation would be a waste of cycles if > they do not complete in time. I think no one has statistical prove of > this, but the rational was at least discussed and I believe agreed here. Yes, we discussed that, and yes, we agreed. And I don't think the decision was wrong. For starters, it would be strange to delay exiting Emacs in batch so it could wait for the async compilation to complete. Moreover, in most cases waiting will not help, since the job of the batch invocation would have been long done anyway, so the produced *.eln files will not be loaded in time to make any difference. We do have batch commands to compile *.eln files explicitly (this is used when a release tarball is built). This was deemed to be enough, at least at the time we discussed these issues. I don't yet see why we'd need to revisit that decision. When does it cause problems? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 15:36 ` Lars Ingebrigtsen 2022-10-05 16:10 ` Andrea Corallo @ 2022-11-05 1:09 ` Juanma Barranquero 2022-11-05 6:44 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Juanma Barranquero @ 2022-11-05 1:09 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Andrea Corallo, Eli Zaretskii, rlb, monnier, david, emacs-devel [-- Attachment #1: Type: text/plain, Size: 825 bytes --] Your idea about -Q inhibiting native comp was obviously not implemented. In my setup, I call `startup-redirect-eln-cache' in init.el because I don't want the cache in my .emacs.d/. However, when I'm using -Q to test something, I do M-x whatever-which-loads-a-module and end up getting an unwanted .emacs.d/eln-cache/ So either a --no-native flag (setting `inhibit-automatic-native-compilation', and perhaps `load-no-native', to t) or having such behavior in -Q would be quite helpful. Yes, I can define an alias to use --eval "(setq inhibit-automatic-native-compilation t)" I already have. Still, I regularly forget it and default to emacs -Q. So, FWIW, I agree with your sentiment that -Q has no single defining goal other than "do not do unnecessary things, please", and inhibiting native comp would fit quite well. [-- Attachment #2: Type: text/html, Size: 1920 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-05 1:09 ` Juanma Barranquero @ 2022-11-05 6:44 ` Eli Zaretskii 2022-11-05 10:12 ` Dr. Arne Babenhauserheide 2022-11-05 11:53 ` Juanma Barranquero 0 siblings, 2 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-11-05 6:44 UTC (permalink / raw) To: Juanma Barranquero; +Cc: larsi, akrl, rlb, monnier, david, emacs-devel > From: Juanma Barranquero <lekktu@gmail.com> > Date: Sat, 5 Nov 2022 02:09:55 +0100 > Cc: Andrea Corallo <akrl@sdf.org>, Eli Zaretskii <eliz@gnu.org>, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org > > In my setup, I call `startup-redirect-eln-cache' in init.el because I don't > want the cache in my .emacs.d/. Why do you need to do that? What's the problem with having the eln-cache under your ~/.emacs.d/? > However, when I'm using -Q to test something, I do > M-x whatever-which-loads-a-module and end up getting an unwanted > .emacs.d/eln-cache/ If this testing is for the purposes of Emacs development, then my suggestion is to have a separate build configured without native compilation support. That's what I do. > So either a --no-native flag (setting `inhibit-automatic-native-compilation', > and perhaps `load-no-native', to t) or having such behavior in -Q would be > quite helpful. > > Yes, I can define an alias to use > > --eval "(setq inhibit-automatic-native-compilation t)" > > I already have. Still, I regularly forget it and default to emacs -Q. So it's just a matter of getting used to using the alias, and will be solved with time. > So, FWIW, I agree with your sentiment that -Q has no single defining goal > other than "do not do unnecessary things, please", and inhibiting native > comp would fit quite well. I disagree. We have too many knobs already, so adding a new one without a good reason would be a mistake in the long run. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-05 6:44 ` Eli Zaretskii @ 2022-11-05 10:12 ` Dr. Arne Babenhauserheide 2022-11-05 11:19 ` Eli Zaretskii 2022-11-06 11:06 ` Max Brieiev 2022-11-05 11:53 ` Juanma Barranquero 1 sibling, 2 replies; 343+ messages in thread From: Dr. Arne Babenhauserheide @ 2022-11-05 10:12 UTC (permalink / raw) To: Eli Zaretskii Cc: Juanma Barranquero, larsi, akrl, rlb, monnier, david, emacs-devel [-- Attachment #1: Type: text/plain, Size: 659 bytes --] Eli Zaretskii <eliz@gnu.org> writes: >> In my setup, I call `startup-redirect-eln-cache' in init.el because I don't >> want the cache in my .emacs.d/. > > Why do you need to do that? What's the problem with having the > eln-cache under your ~/.emacs.d/? My ~/.emacs.d/ is actually versiontracked as part of my repository, because emacs is my build tool to compile org-mode to PDF and I need a reproducible environment so others can use it, too. During building, ~/.emacs.d/ is in the non-writable source directory used by make distcheck. Best wishes, Arne -- Unpolitisch sein heißt politisch sein, ohne es zu merken. draketo.de [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 1125 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-05 10:12 ` Dr. Arne Babenhauserheide @ 2022-11-05 11:19 ` Eli Zaretskii 2022-11-06 11:06 ` Max Brieiev 1 sibling, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-11-05 11:19 UTC (permalink / raw) To: Dr. Arne Babenhauserheide Cc: lekktu, larsi, akrl, rlb, monnier, david, emacs-devel > From: "Dr. Arne Babenhauserheide" <arne_bab@web.de> > Cc: Juanma Barranquero <lekktu@gmail.com>, larsi@gnus.org, akrl@sdf.org, > rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, > emacs-devel@gnu.org > Date: Sat, 05 Nov 2022 11:12:10 +0100 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> In my setup, I call `startup-redirect-eln-cache' in init.el because I don't > >> want the cache in my .emacs.d/. > > > > Why do you need to do that? What's the problem with having the > > eln-cache under your ~/.emacs.d/? > > My ~/.emacs.d/ is actually versiontracked as part of my repository, > because emacs is my build tool to compile org-mode to PDF and I need a > reproducible environment so others can use it, too. > > During building, ~/.emacs.d/ is in the non-writable source directory > used by make distcheck. So what do you do with the *.elc files for the *.el files under ~/.emacs.d/? And anyway, I think your use case is very different from Juanma's, and is easily handled by customizing native-comp-eln-load-path. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-05 10:12 ` Dr. Arne Babenhauserheide 2022-11-05 11:19 ` Eli Zaretskii @ 2022-11-06 11:06 ` Max Brieiev 2022-11-06 16:31 ` Dr. Arne Babenhauserheide 2022-11-07 9:22 ` Andrea Corallo 1 sibling, 2 replies; 343+ messages in thread From: Max Brieiev @ 2022-11-06 11:06 UTC (permalink / raw) To: Dr. Arne Babenhauserheide Cc: Eli Zaretskii, Juanma Barranquero, larsi, akrl, rlb, monnier, david, emacs-devel "Dr. Arne Babenhauserheide" <arne_bab@web.de> writes: > My ~/.emacs.d/ is actually versiontracked as part of my repository, > because emacs is my build tool to compile org-mode to PDF and I need a > reproducible environment so others can use it, too. If Emacs is your build tool, then you probably run it in a batch mode. I think in one of his messages Andrea said that native compiler is disabled when Emacs is run in batch mode. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-06 11:06 ` Max Brieiev @ 2022-11-06 16:31 ` Dr. Arne Babenhauserheide 2022-11-06 18:00 ` Stefan Monnier 2022-11-07 9:22 ` Andrea Corallo 1 sibling, 1 reply; 343+ messages in thread From: Dr. Arne Babenhauserheide @ 2022-11-06 16:31 UTC (permalink / raw) To: Max Brieiev Cc: Eli Zaretskii, Juanma Barranquero, larsi, akrl, rlb, monnier, david, emacs-devel [-- Attachment #1: Type: text/plain, Size: 690 bytes --] Max Brieiev <max.brieiev@gmail.com> writes: > "Dr. Arne Babenhauserheide" <arne_bab@web.de> writes: > >> My ~/.emacs.d/ is actually versiontracked as part of my repository, >> because emacs is my build tool to compile org-mode to PDF and I need a >> reproducible environment so others can use it, too. > > If Emacs is your build tool, then you probably run it in a batch mode. No, I cannot run it in batch mode, because batch mode has different font-locking than visual mode, so html-export is worse. I need a full X11 Emacs (kept working via Xvfb, if there is no X11). Best wishes, Arne -- Unpolitisch sein heißt politisch sein, ohne es zu merken. draketo.de [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 1125 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-06 16:31 ` Dr. Arne Babenhauserheide @ 2022-11-06 18:00 ` Stefan Monnier 0 siblings, 0 replies; 343+ messages in thread From: Stefan Monnier @ 2022-11-06 18:00 UTC (permalink / raw) To: Dr. Arne Babenhauserheide Cc: Max Brieiev, Eli Zaretskii, Juanma Barranquero, larsi, akrl, rlb, david, emacs-devel > No, I cannot run it in batch mode, because batch mode has different > font-locking than visual mode, so html-export is worse. Please report this as a bug. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-06 11:06 ` Max Brieiev 2022-11-06 16:31 ` Dr. Arne Babenhauserheide @ 2022-11-07 9:22 ` Andrea Corallo 2022-11-08 5:51 ` Björn Bidar 1 sibling, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-11-07 9:22 UTC (permalink / raw) To: Dr. Arne Babenhauserheide Cc: Eli Zaretskii, Juanma Barranquero, larsi, rlb, monnier, david, emacs-devel Max Brieiev <max.brieiev@gmail.com> writes: > "Dr. Arne Babenhauserheide" <arne_bab@web.de> writes: > >> My ~/.emacs.d/ is actually versiontracked as part of my repository, >> because emacs is my build tool to compile org-mode to PDF and I need a >> reproducible environment so others can use it, too. > > If Emacs is your build tool, then you probably run it in a batch mode. > > I think in one of his messages Andrea said that native compiler is > disabled when Emacs is run in batch mode. Yep that's correct. BR Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-07 9:22 ` Andrea Corallo @ 2022-11-08 5:51 ` Björn Bidar 2022-11-08 11:23 ` Andrea Corallo 2022-11-08 12:13 ` Eli Zaretskii 0 siblings, 2 replies; 343+ messages in thread From: Björn Bidar @ 2022-11-08 5:51 UTC (permalink / raw) To: Andrea Corallo Cc: Dr. Arne Babenhauserheide, Eli Zaretskii, Juanma Barranquero, larsi, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > Max Brieiev <max.brieiev@gmail.com> writes: > >> "Dr. Arne Babenhauserheide" <arne_bab@web.de> writes: >> >>> My ~/.emacs.d/ is actually versiontracked as part of my repository, >>> because emacs is my build tool to compile org-mode to PDF and I need a >>> reproducible environment so others can use it, too. >> >> If Emacs is your build tool, then you probably run it in a batch mode. >> >> I think in one of his messages Andrea said that native compiler is >> disabled when Emacs is run in batch mode. > > Yep that's correct. But I think it possible to call `native-compile` in batch mode I think. I do this with borg to precompile all packages when I update them. I think doing so is quite useful e.g. for device that not always run on cable but on battery or are otherwise limited in power. Br, Björn ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-08 5:51 ` Björn Bidar @ 2022-11-08 11:23 ` Andrea Corallo 2022-11-08 12:13 ` Eli Zaretskii 1 sibling, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-11-08 11:23 UTC (permalink / raw) To: Björn Bidar Cc: Dr. Arne Babenhauserheide, Eli Zaretskii, Juanma Barranquero, larsi, rlb, monnier, david, emacs-devel Björn Bidar <bjorn.bidar@thaodan.de> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> Max Brieiev <max.brieiev@gmail.com> writes: >> >>> "Dr. Arne Babenhauserheide" <arne_bab@web.de> writes: >>> >>>> My ~/.emacs.d/ is actually versiontracked as part of my repository, >>>> because emacs is my build tool to compile org-mode to PDF and I need a >>>> reproducible environment so others can use it, too. >>> >>> If Emacs is your build tool, then you probably run it in a batch mode. >>> >>> I think in one of his messages Andrea said that native compiler is >>> disabled when Emacs is run in batch mode. >> >> Yep that's correct. > > But I think it possible to call `native-compile` in batch mode I think. > I do this with borg to precompile all packages when I update them. Yes that's correct, only the machanism that automatically triggers async native compilations is disabled but you can still invoke compilations manually. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-08 5:51 ` Björn Bidar 2022-11-08 11:23 ` Andrea Corallo @ 2022-11-08 12:13 ` Eli Zaretskii 1 sibling, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-11-08 12:13 UTC (permalink / raw) To: Björn Bidar Cc: akrl, arne_bab, lekktu, larsi, rlb, monnier, david, emacs-devel > From: Björn Bidar <bjorn.bidar@thaodan.de> > Cc: "Dr. Arne Babenhauserheide" <arne_bab@web.de>, Eli Zaretskii > <eliz@gnu.org>, Juanma Barranquero <lekktu@gmail.com>, larsi@gnus.org, > rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, > emacs-devel@gnu.org > Date: Tue, 08 Nov 2022 07:51:52 +0200 > > Andrea Corallo <akrl@sdf.org> writes: > > >> I think in one of his messages Andrea said that native compiler is > >> disabled when Emacs is run in batch mode. > > > > Yep that's correct. > > But I think it possible to call `native-compile` in batch mode I think. Yes, it is. We actually do that when building a release tarball on the end-user machine. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-05 6:44 ` Eli Zaretskii 2022-11-05 10:12 ` Dr. Arne Babenhauserheide @ 2022-11-05 11:53 ` Juanma Barranquero 2022-11-05 12:44 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Juanma Barranquero @ 2022-11-05 11:53 UTC (permalink / raw) To: Eli Zaretskii; +Cc: larsi, akrl, rlb, monnier, david, emacs-devel [-- Attachment #1: Type: text/plain, Size: 978 bytes --] On Sat, Nov 5, 2022 at 7:45 AM Eli Zaretskii <eliz@gnu.org> wrote: > Why do you need to do that? What's the problem with having the > eln-cache under your ~/.emacs.d/? There's no problem. It's a matter of taste. > If this testing is for the purposes of Emacs development, then my > suggestion is to have a separate build configured without native > compilation support. That's what I do. I prefer to use just one build. > So it's just a matter of getting used to using the alias, and will be > solved with time. Yep. > I disagree. We have too many knobs already, so adding a new one > without a good reason would be a mistake in the long run. In fact, my thinking yesterday was "-Q should stop native compilation...Wait, I bet this was already discussed and rejected", and so I stumbled upon this thread and read or perused its several hundred messages. Believe me, I'm not *proposing* any change. I'm just telling Lars that I agree with him that this fits under -Q. [-- Attachment #2: Type: text/html, Size: 1653 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-05 11:53 ` Juanma Barranquero @ 2022-11-05 12:44 ` Eli Zaretskii 2022-11-05 13:12 ` Juanma Barranquero 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-11-05 12:44 UTC (permalink / raw) To: Juanma Barranquero; +Cc: larsi, akrl, rlb, monnier, david, emacs-devel > From: Juanma Barranquero <lekktu@gmail.com> > Date: Sat, 5 Nov 2022 12:53:42 +0100 > Cc: larsi@gnus.org, akrl@sdf.org, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org > > In fact, my thinking yesterday was "-Q should stop native compilation...Wait, I bet > this was already discussed and rejected", and so I stumbled upon this thread and > read or perused its several hundred messages. Believe me, I'm not *proposing* any > change. I'm just telling Lars that I agree with him that this fits under -Q. Well, we'll have to disagree. The -Q switch is documented as disabling various things that happen at startup, specifically loading stuff that changes the defaults. Native compilation is not in that class, exactly like support for image files or GnuTLS aren't. It is part of the built Emacs, and is thus part of its default operation. I see no reason to change what -Q means, even though some people, for reasons I cannot grasp, consider JIT native compilation to be "unusual". Suppose you start "emacs -Q" where some of the *.el files were already compiled into the corresponding *.eln files, would you then expect "emacs -Q" not to use those *.eln files, and instead to load the *.elc files? If yes, why? If not, how does this differ from when you invoke "emacs -Q" and the *.eln files do not yet exist, but are produced when Emacs loads the corresponding package? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-05 12:44 ` Eli Zaretskii @ 2022-11-05 13:12 ` Juanma Barranquero 2022-11-05 13:35 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Juanma Barranquero @ 2022-11-05 13:12 UTC (permalink / raw) To: Eli Zaretskii; +Cc: larsi, akrl, rlb, monnier, david, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1427 bytes --] On Sat, Nov 5, 2022 at 1:44 PM Eli Zaretskii <eliz@gnu.org> wrote: > Well, we'll have to disagree. The -Q switch is documented as > disabling various things that happen at startup, specifically loading > stuff that changes the defaults. Yeah, well, there's --no-splash ;-) > I see no reason to change what -Q means, even though some people, for > reasons I cannot grasp, consider JIT native compilation to be > "unusual". I don't consider it unusual, except that in my build, if I enter Emacs, load something that triggers native compilation, and exit quickly, the subprocesses crash (I get invitations to "connect with gdb and debug them", which disappear after a few seconds). That had me puzzled for a while. > Suppose you start "emacs -Q" where some of the *.el files were already > compiled into the corresponding *.eln files, would you then expect > "emacs -Q" not to use those *.eln files, and instead to load the *.elc > files? If yes, why? If not, how does this differ from when you > invoke "emacs -Q" and the *.eln files do not yet exist, but are > produced when Emacs loads the corresponding package? Loading them and using them wouldn't be a problem, because it does *not* write anywhere, while producing them does. In my case, they do just where I don't want them to be. As you say, we'll have to agree to disagree. I admit the issue is nuanced and there's no single solution that will please everyone. [-- Attachment #2: Type: text/html, Size: 2363 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-05 13:12 ` Juanma Barranquero @ 2022-11-05 13:35 ` Eli Zaretskii 2022-11-05 14:09 ` Juanma Barranquero 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-11-05 13:35 UTC (permalink / raw) To: Juanma Barranquero; +Cc: larsi, akrl, rlb, monnier, david, emacs-devel > From: Juanma Barranquero <lekktu@gmail.com> > Date: Sat, 5 Nov 2022 14:12:03 +0100 > Cc: larsi@gnus.org, akrl@sdf.org, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org > > > I see no reason to change what -Q means, even though some people, for > > reasons I cannot grasp, consider JIT native compilation to be > > "unusual". > > I don't consider it unusual, except that in my build, if I enter Emacs, load something > that triggers native compilation, and exit quickly, the subprocesses crash (I get > invitations to "connect with gdb and debug them", which disappear after a few seconds). They aren't supposed to crash, and they didn't in Emacs 28. So this should be reported with enough detail for us to be able to fix the crashes. This could be related to bug#58956, btw. > > Suppose you start "emacs -Q" where some of the *.el files were already > > compiled into the corresponding *.eln files, would you then expect > > "emacs -Q" not to use those *.eln files, and instead to load the *.elc > > files? If yes, why? If not, how does this differ from when you > > invoke "emacs -Q" and the *.eln files do not yet exist, but are > > produced when Emacs loads the corresponding package? > > Loading them and using them wouldn't be a problem, because it does *not* write anywhere, > while producing them does. In my case, they do just where I don't want them to be. But we have features who write even under "emacs -Q", don't we? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-11-05 13:35 ` Eli Zaretskii @ 2022-11-05 14:09 ` Juanma Barranquero 0 siblings, 0 replies; 343+ messages in thread From: Juanma Barranquero @ 2022-11-05 14:09 UTC (permalink / raw) To: Eli Zaretskii; +Cc: larsi, akrl, rlb, monnier, david, emacs-devel [-- Attachment #1: Type: text/plain, Size: 777 bytes --] On Sat, Nov 5, 2022 at 2:35 PM Eli Zaretskii <eliz@gnu.org> wrote: > They aren't supposed to crash, and they didn't in Emacs 28. So this > should be reported with enough detail for us to be able to fix the > crashes. This could be related to bug#58956, btw. Yes. I'm waiting to have a more easily reproducible case or a better understanding of how to trigger it. I'll look into #58956. > But we have features who write even under "emacs -Q", don't we? Yes, sure. But if I do "emacs -Q" and then I save a bookmark, I know bookmark.el is going to save a file and I will make sure it is going to .emacs.d/ or wherever else I want it to go. That's not surprising. But I don't usually expect to do M-x bs-show and get something written to disk, in a place I wasn't expecting. [-- Attachment #2: Type: text/html, Size: 1313 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 15:24 ` Lars Ingebrigtsen 2022-10-05 15:36 ` Lars Ingebrigtsen @ 2022-10-05 16:04 ` Andrea Corallo 2022-10-05 16:12 ` Lars Ingebrigtsen 1 sibling, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 16:04 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> This is not the case at all, please trust me, your changeset does two >> things: >> >> 1- change the name a knob, but it goes from a maybe un-intuitive one to >> just (as explained) a plain wrong one. > > It goes from an un-intuitive (and undocumented) one to an > intuitively-named (and documented) one. The documentation topic is orthogonal, you could have improved `native-comp-deferred-compilation' documentation if you wanted. The naming went first to `inhibit-native-compilation', this was really just sooo wrong in so many levels :/ So many as many knobs we have to control and disable different parts of the native compiler. I can't really think of one understanding all this machinery and then picking up this name sorry, I can't. Then the name became `inhibit-automatic-native-compilation' that is fortunately just wrong on one level :/ The original name was the name of one of the two mechanisms concurring to native compilation, that is the reason why it was just not called "automatic". > In my opinion, the old variable > is just as wrong as the current one (because it seemed to imply that > compilation would be immediate instead of deferred). `native-comp-deferred-compilation' implies that native compilation can happen deferred. But anyway this name was reviewed and the name used here countless times for long time. Changing it with zero discussion is just not good cooperation spirit, sorry. You can think of it differently but you just risk to remain alone doing development. >> 2- add a mechanism that (as explained) cannot help with the user request >> in this discussion at all. > > And I've explained twice now that 2 is wrong -- these changes do exactly > what was requested: > > 1) It allows testing without writing to $HOME. (This has nothing to > do with --batch -- testing happens in interactive Emacsen, too.) The user request is for non interactive sessions AFAIU. And still I've to understand exactly what the user wants to solve. Most importantly I feel I'm not alone here. > 2) It allows people to run an AOT Emacs without triggering further > compilation. Sorry as changeset I meant 5fec9182db + f97993ee66. I'm not against e245c4f226. > If you have a suggestion for an alternative change that achieves these > two things, I'm all ears. (But if your objection is "people shouldn't > want those two things", I'm down to just two ears again.) Again 5fec9182db + f97993ee66 are IMO not useful / wrong, they were not discussed and are just a step backward. The best change I can suggest is to revert them now. Thanks Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 16:04 ` Andrea Corallo @ 2022-10-05 16:12 ` Lars Ingebrigtsen 2022-10-05 16:28 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 16:12 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > The naming went first to `inhibit-native-compilation', this was really > just sooo wrong in so many levels :/ So many as many knobs we have to > control and disable different parts of the native compiler. I can't > really think of one understanding all this machinery and then picking up > this name sorry, I can't. Naming is the most difficult thing in computing, after all. The doc string to the variable explained what it does just fine, so I think your understanding needs a check-up. (And I don't appreciated being talked to in this tone, in case you wondered.) >> 1) It allows testing without writing to $HOME. (This has nothing to >> do with --batch -- testing happens in interactive Emacsen, too.) > > The user request is for non interactive sessions AFAIU. And still I've > to understand exactly what the user wants to solve. Most importantly I > feel I'm not alone here. And I'm telling you that's not all that was requested for... the third time? I'm not sure I'm getting any further here by repeating myself, so I think I'll stop. >> 2) It allows people to run an AOT Emacs without triggering further >> compilation. > > Sorry as changeset I meant 5fec9182db + f97993ee66. I'm not against > e245c4f226. > >> If you have a suggestion for an alternative change that achieves these >> two things, I'm all ears. (But if your objection is "people shouldn't >> want those two things", I'm down to just two ears again.) > > Again 5fec9182db + f97993ee66 are IMO not useful / wrong, they were not > discussed and are just a step backward. The best change I can suggest > is to revert them now. You're just repeating that you're against it without proposing an alternative, which is not helpful, so I think we've come to the of the discussion on this point, too. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 16:12 ` Lars Ingebrigtsen @ 2022-10-05 16:28 ` Andrea Corallo 0 siblings, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 16:28 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> The naming went first to `inhibit-native-compilation', this was really >> just sooo wrong in so many levels :/ So many as many knobs we have to >> control and disable different parts of the native compiler. I can't >> really think of one understanding all this machinery and then picking up >> this name sorry, I can't. > > Naming is the most difficult thing in computing, after all. > > The doc string to the variable explained what it does just fine, so I > think your understanding needs a check-up. (And I don't appreciated > being talked to in this tone, in case you wondered.) > >>> 1) It allows testing without writing to $HOME. (This has nothing to >>> do with --batch -- testing happens in interactive Emacsen, too.) >> >> The user request is for non interactive sessions AFAIU. And still I've >> to understand exactly what the user wants to solve. Most importantly I >> feel I'm not alone here. > > And I'm telling you that's not all that was requested for... the third > time? I'm not sure I'm getting any further here by repeating myself, so > I think I'll stop. > >>> 2) It allows people to run an AOT Emacs without triggering further >>> compilation. >> >> Sorry as changeset I meant 5fec9182db + f97993ee66. I'm not against >> e245c4f226. >> >>> If you have a suggestion for an alternative change that achieves these >>> two things, I'm all ears. (But if your objection is "people shouldn't >>> want those two things", I'm down to just two ears again.) >> >> Again 5fec9182db + f97993ee66 are IMO not useful / wrong, they were not >> discussed and are just a step backward. The best change I can suggest >> is to revert them now. > > You're just repeating that you're against it without proposing an > alternative, which is not helpful, so I think we've come to the of the > discussion on this point, too. You asked for an ameliorative change, I gave you one, that is just to revert it. If you don't like it I don't know what to do, I'll not speculate on your psychology as you did with mine ;) Best Regards Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 14:29 ` Lars Ingebrigtsen 2022-10-05 14:48 ` Andrea Corallo @ 2022-10-05 16:43 ` Eli Zaretskii 2022-10-06 0:44 ` Po Lu 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-05 16:43 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: akrl, rlb, monnier, david, emacs-devel > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Eli Zaretskii <eliz@gnu.org>, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org > Date: Wed, 05 Oct 2022 16:29:29 +0200 > > > I've the perception that this change was done without the full picture > > in mind of how the native compiler and his mechanisms works. As a > > result the current naming is IMO just wrong, and as such is a step > > backward the original state. > > I don't know where you got that perception from. It's clear as daylight that this is what happened. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 16:43 ` Eli Zaretskii @ 2022-10-06 0:44 ` Po Lu 2022-10-06 0:56 ` Lars Ingebrigtsen 2022-10-06 6:28 ` Eli Zaretskii 0 siblings, 2 replies; 343+ messages in thread From: Po Lu @ 2022-10-06 0:44 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Lars Ingebrigtsen, akrl, rlb, monnier, david, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Lars Ingebrigtsen <larsi@gnus.org> >> Cc: Eli Zaretskii <eliz@gnu.org>, rlb@defaultvalue.org, >> monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org >> Date: Wed, 05 Oct 2022 16:29:29 +0200 >> >> > I've the perception that this change was done without the full picture >> > in mind of how the native compiler and his mechanisms works. As a >> > result the current naming is IMO just wrong, and as such is a step >> > backward the original state. >> >> I don't know where you got that perception from. > > It's clear as daylight that this is what happened. Since the argument seems to be going nowhere, could we first revert the change in question, and then ask Rob Browning exactly what the problem is with `native-comp-deferred-compilation'? I did not figure that out reading this thread. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 0:44 ` Po Lu @ 2022-10-06 0:56 ` Lars Ingebrigtsen 2022-10-06 6:41 ` Eli Zaretskii 2022-10-06 6:28 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-06 0:56 UTC (permalink / raw) To: Po Lu; +Cc: Eli Zaretskii, akrl, rlb, monnier, david, emacs-devel Po Lu <luangruo@yahoo.com> writes: > Since the argument seems to be going nowhere, could we first revert the > change in question, and then ask Rob Browning exactly what the problem > is with `native-comp-deferred-compilation'? > > I did not figure that out reading this thread. I don't know whether this was Rob's problem, but I found it problematic that there was no way to not write to ~/.emacs.d/eln-cache. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 0:56 ` Lars Ingebrigtsen @ 2022-10-06 6:41 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-06 6:41 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: luangruo, akrl, rlb, monnier, david, emacs-devel > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Eli Zaretskii <eliz@gnu.org>, akrl@sdf.org, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org > Date: Thu, 06 Oct 2022 02:56:54 +0200 > > Po Lu <luangruo@yahoo.com> writes: > > > Since the argument seems to be going nowhere, could we first revert the > > change in question, and then ask Rob Browning exactly what the problem > > is with `native-comp-deferred-compilation'? > > > > I did not figure that out reading this thread. > > I don't know whether this was Rob's problem, but I found it problematic > that there was no way to not write to ~/.emacs.d/eln-cache. But there was such a way: modify native-comp-eln-load-path to have another directory at the front. Which, it seems, is what Rob actually wants, at least in some cases, because they _want_ to keep the *.eln files, just not in the user's home directory. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 0:44 ` Po Lu 2022-10-06 0:56 ` Lars Ingebrigtsen @ 2022-10-06 6:28 ` Eli Zaretskii 2022-10-14 17:53 ` Rob Browning 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-06 6:28 UTC (permalink / raw) To: Po Lu; +Cc: larsi, akrl, rlb, monnier, david, emacs-devel > From: Po Lu <luangruo@yahoo.com> > Cc: Lars Ingebrigtsen <larsi@gnus.org>, akrl@sdf.org, > rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, > emacs-devel@gnu.org > Date: Thu, 06 Oct 2022 08:44:21 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > Since the argument seems to be going nowhere, could we first revert the > change in question, and then ask Rob Browning exactly what the problem > is with `native-comp-deferred-compilation'? I'm still discussing with Rob his needs. I hope to eventually understand their needs. For now my only conclusion is that they need to tweak native-comp-eln-load-path in some situations, to control where the *.eln files are deposited. Which is easy and supported already, AFAIU. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 6:28 ` Eli Zaretskii @ 2022-10-14 17:53 ` Rob Browning 2022-10-14 18:36 ` Stefan Monnier ` (2 more replies) 0 siblings, 3 replies; 343+ messages in thread From: Rob Browning @ 2022-10-14 17:53 UTC (permalink / raw) To: Eli Zaretskii, Po Lu; +Cc: larsi, akrl, monnier, david, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > I'm still discussing with Rob his needs. I hope to eventually > understand their needs. For now my only conclusion is that they need > to tweak native-comp-eln-load-path in some situations, to control > where the *.eln files are deposited. Which is easy and supported > already, AFAIU. Eli, Lars, Andrea, and others, I think I'm mostly caught back up with this thread, and hope to summarize a few things here: - I made a mistake in emphasizing (via the thread subject) a particular "solution" (suppressing native compilation). In the end, from the Debian perspective, for the most immedate issue, we don't need that particular solution (and I wasn't actually set on it at the time). I'd mostly started there because it's *a* possible solution, and one I'd imagined might be easyish to implement (clearly that was naive). But other solutions would be fine too. - For example, redirecting all incidental writes away from HOME would be fine too. - We'd prefer to still be able to set HOME=/does-not-exist, which I assume would mean that we'd need some other way to redirect *all* of the eln file writes (including trampolines). Our practice of setting HOME to a nonexistent directory during some packaging operations allows us to detect unexpected, and erroneous attempts to write to HOME during those processes. - If we are going to have "some other way" to redirect the eln files, then for us an environment variable might be easier, so that we can just export it before we start the relevant build/test/etc. and then it'll affect all invocations of emacs in that environment. I suspect an environment variable might also make it easier to establish the setting "early enough" in the emacs startup process, but don't know that. - If just setting HOME will do the job, and there's no interest here in anything further, then we can probably just do that, or if we ended up needing to, we could likely add our own DEBIAN_... environment variable, and carry a patch, but of course that's less desirable. - As an aside, I suspect Emacs may eventually want to have some way to restore the ability to tolerate an unwritable filesystem. I have more than once in the past launched emacs from an emergency shell to fix a broken host where the filesystem was read-only and /home might not be mounted (and if it were on NFS and there's no network yet, could not be). I'd much rather use emacs there, than /usr/bin/sensible-editor. Of course now I know that I could just set HOME=/tmp and proceed, but will others? Might at least be worth making sure any error messages would lead people to that option. Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 17:53 ` Rob Browning @ 2022-10-14 18:36 ` Stefan Monnier 2022-10-14 19:06 ` Eli Zaretskii 2022-10-15 9:32 ` Lars Ingebrigtsen 2 siblings, 0 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-14 18:36 UTC (permalink / raw) To: Rob Browning; +Cc: Eli Zaretskii, Po Lu, larsi, akrl, david, emacs-devel > - As an aside, I suspect Emacs may eventually want to have some way to > restore the ability to tolerate an unwritable filesystem. If it doesn't work, it's a bug. Please `M-x report-emacs-bug` when you see such problems. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 17:53 ` Rob Browning 2022-10-14 18:36 ` Stefan Monnier @ 2022-10-14 19:06 ` Eli Zaretskii 2022-10-14 21:17 ` Rob Browning 2022-10-15 9:32 ` Lars Ingebrigtsen 2 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-14 19:06 UTC (permalink / raw) To: Rob Browning; +Cc: luangruo, larsi, akrl, monnier, david, emacs-devel > From: Rob Browning <rlb@defaultvalue.org> > Cc: larsi@gnus.org, akrl@sdf.org, monnier@iro.umontreal.ca, > david@tethera.net, emacs-devel@gnu.org > Date: Fri, 14 Oct 2022 12:53:48 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > > I'm still discussing with Rob his needs. I hope to eventually > > understand their needs. For now my only conclusion is that they need > > to tweak native-comp-eln-load-path in some situations, to control > > where the *.eln files are deposited. Which is easy and supported > > already, AFAIU. > > Eli, Lars, Andrea, and others, I think I'm mostly caught back up with > this thread, and hope to summarize a few things here: Thank you for your summary. > - We'd prefer to still be able to set HOME=/does-not-exist, which I > assume would mean that we'd need some other way to redirect *all* of > the eln file writes (including trampolines). > > Our practice of setting HOME to a nonexistent directory during some > packaging operations allows us to detect unexpected, and erroneous > attempts to write to HOME during those processes. This should work for preventing the native-compilation from storing the *.eln files. If it doesn't work for you, please tell the details, and we will investigate. > - If we are going to have "some other way" to redirect the eln files, > then for us an environment variable might be easier, so that we can > just export it before we start the relevant build/test/etc. and then > it'll affect all invocations of emacs in that environment. I > suspect an environment variable might also make it easier to > establish the setting "early enough" in the emacs startup process, > but don't know that. One other way is to change the value of native-comp-eln-load-path. I hesitate to add an environment variable for that, because environment variables are inherited to subprocesses, and so setting a variable for some Emacs process will affect all of its Emacs subprocesses. This was found to be a reason of tricky and hard-to-debug problems when users set EMACSLOADPATH like that, and I presume the same can easily happen with the equivalent variable for *.eln files. So I'd rather not add such a variable unless we find a very good reason. > - As an aside, I suspect Emacs may eventually want to have some way to > restore the ability to tolerate an unwritable filesystem. With respect to writing the *.eln files, Emacs will look along native-comp-eln-load-path for the first directory that is writable, and use that. So you could at least partially handle this case by making sure native-comp-eln-load-path includes at least one writable directory. > I'd much rather use emacs there, than /usr/bin/sensible-editor. Of > course now I know that I could just set HOME=/tmp and proceed, but > will others? I've now documented this method in the ELisp reference manual, in the hope that it will make this more discoverable. Thanks again for your comments and perseverance. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 19:06 ` Eli Zaretskii @ 2022-10-14 21:17 ` Rob Browning 2022-10-15 3:07 ` Stefan Monnier 2022-10-15 6:30 ` Eli Zaretskii 0 siblings, 2 replies; 343+ messages in thread From: Rob Browning @ 2022-10-14 21:17 UTC (permalink / raw) To: Eli Zaretskii; +Cc: luangruo, larsi, akrl, monnier, david, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Rob Browning <rlb@defaultvalue.org> >> - We'd prefer to still be able to set HOME=/does-not-exist, which I >> assume would mean that we'd need some other way to redirect *all* of >> the eln file writes (including trampolines). >> >> Our practice of setting HOME to a nonexistent directory during some >> packaging operations allows us to detect unexpected, and erroneous >> attempts to write to HOME during those processes. > > This should work for preventing the native-compilation from storing > the *.eln files. If it doesn't work for you, please tell the details, > and we will investigate. Hmm, which "this" did you mean? If you meant HOME=/does-not-exist, I thought people had already said that wouldn't work because the attempt to build trampolines would still cause a crash. But I may well have misunderstood. > One other way is to change the value of native-comp-eln-load-path. I > hesitate to add an environment variable for that, because environment > variables are inherited to subprocesses, and so setting a variable for > some Emacs process will affect all of its Emacs subprocesses. This > was found to be a reason of tricky and hard-to-debug problems when > users set EMACSLOADPATH like that, and I presume the same can easily > happen with the equivalent variable for *.eln files. So I'd rather > not add such a variable unless we find a very good reason. For what it's worth, the reason we'd been at least a bit inclined that direction is because if you have hundreds of emacs add-on packages (which I believe we do in Debian), and they all have build and or test suites that vary in any way they like, it could be much more difficult to track down all of the emacs invocations in each project's source to add some argument than it would be to just export an environment variable before running the suite. I suppose we could just shadow emacs in the path with a wrapper that includes the argument(s), assuming few of the projects have their own interfering options, and that they always use the same (or a handful of predictable) relative invocation of emacs, e.g. "emacs" not "/usr/bin/emacs" or... > With respect to writing the *.eln files, Emacs will look along > native-comp-eln-load-path for the first directory that is writable, > and use that. So you could at least partially handle this case by > making sure native-comp-eln-load-path includes at least one writable > directory. Right, though we also have to make sure we can do that sufficiently early in *every* case. > Thanks again for your comments and perseverance. Certainly, appreciate the help. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 21:17 ` Rob Browning @ 2022-10-15 3:07 ` Stefan Monnier 2022-10-15 16:34 ` Rob Browning 2022-10-15 6:30 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-15 3:07 UTC (permalink / raw) To: Rob Browning; +Cc: Eli Zaretskii, luangruo, larsi, akrl, david, emacs-devel > I suppose we could just shadow emacs in the path with a wrapper that > includes the argument(s), assuming few of the projects have their own > interfering options, and that they always use the same (or a handful of > predictable) relative invocation of emacs, e.g. "emacs" not > "/usr/bin/emacs" or... Or put a small chunk of ELisp in Debian's site-start file which sets the eln load path according to some env-var :-) Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 3:07 ` Stefan Monnier @ 2022-10-15 16:34 ` Rob Browning 2022-10-15 23:16 ` Stefan Monnier 0 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-15 16:34 UTC (permalink / raw) To: Stefan Monnier; +Cc: Eli Zaretskii, luangruo, larsi, akrl, david, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >> I suppose we could just shadow emacs in the path with a wrapper that >> includes the argument(s), assuming few of the projects have their own >> interfering options, and that they always use the same (or a handful of >> predictable) relative invocation of emacs, e.g. "emacs" not >> "/usr/bin/emacs" or... > > Or put a small chunk of ELisp in Debian's site-start file which sets the > eln load path according to some env-var :-) I wondered if that would be early enough. I'll have to double-check, but I think people may have found that some of the attempts to change it via the command line may not have been. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 16:34 ` Rob Browning @ 2022-10-15 23:16 ` Stefan Monnier 0 siblings, 0 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-15 23:16 UTC (permalink / raw) To: Rob Browning; +Cc: Eli Zaretskii, luangruo, larsi, akrl, david, emacs-devel Rob Browning [2022-10-15 11:34:14] wrote: > Stefan Monnier <monnier@iro.umontreal.ca> writes: >>> I suppose we could just shadow emacs in the path with a wrapper that >>> includes the argument(s), assuming few of the projects have their own >>> interfering options, and that they always use the same (or a handful of >>> predictable) relative invocation of emacs, e.g. "emacs" not >>> "/usr/bin/emacs" or... >> Or put a small chunk of ELisp in Debian's site-start file which sets the >> eln load path according to some env-var :-) > I wondered if that would be early enough. The site-start file is basically the first chunk of non-builtin ELisp code that is executed except for the `early-init.el` which is loaded even before. Trampolines should not be needed before we load such non-builtin code, so I think the answer is that it's early enough unless the running user doesn't has `early-init.el` which itself needs a trampoline. > I'll have to double-check, but I think people may have found that some > of the attempts to change it via the command line may not have been. I think `--eval` is run too late (after the site-start), indeed. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 21:17 ` Rob Browning 2022-10-15 3:07 ` Stefan Monnier @ 2022-10-15 6:30 ` Eli Zaretskii 2022-10-15 17:00 ` Rob Browning 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-15 6:30 UTC (permalink / raw) To: Rob Browning; +Cc: luangruo, larsi, akrl, monnier, david, emacs-devel > From: Rob Browning <rlb@defaultvalue.org> > Cc: luangruo@yahoo.com, larsi@gnus.org, akrl@sdf.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org > Date: Fri, 14 Oct 2022 16:17:56 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> From: Rob Browning <rlb@defaultvalue.org> > > >> - We'd prefer to still be able to set HOME=/does-not-exist, which I > >> assume would mean that we'd need some other way to redirect *all* of > >> the eln file writes (including trampolines). > >> > >> Our practice of setting HOME to a nonexistent directory during some > >> packaging operations allows us to detect unexpected, and erroneous > >> attempts to write to HOME during those processes. > > > > This should work for preventing the native-compilation from storing > > the *.eln files. If it doesn't work for you, please tell the details, > > and we will investigate. > > Hmm, which "this" did you mean? If you meant HOME=/does-not-exist, I > thought people had already said that wouldn't work because the attempt > to build trampolines would still cause a crash. But I may well have > misunderstood. I meant HOME=/does-not-exist, yes. If that doesn't work (but please test), we should fix that, I think. Please report your findings if that doesn't work. We use that in our own test suite. > > One other way is to change the value of native-comp-eln-load-path. I > > hesitate to add an environment variable for that, because environment > > variables are inherited to subprocesses, and so setting a variable for > > some Emacs process will affect all of its Emacs subprocesses. This > > was found to be a reason of tricky and hard-to-debug problems when > > users set EMACSLOADPATH like that, and I presume the same can easily > > happen with the equivalent variable for *.eln files. So I'd rather > > not add such a variable unless we find a very good reason. > > For what it's worth, the reason we'd been at least a bit inclined that > direction is because if you have hundreds of emacs add-on packages > (which I believe we do in Debian), and they all have build and or test > suites that vary in any way they like, it could be much more difficult > to track down all of the emacs invocations in each project's source to > add some argument than it would be to just export an environment > variable before running the suite. I understand, but don't you have some kind of centralized script or group of scripts that runs the testing? In that case, you could make the Emacs command, or its template, be part of those few scripts, and then the change will be less painful. Please understand my POV: adding an environment variable affects all Emacs users, not just distros that have testing suites. There's much more at stake than your particular testing needs, and many Emacs users know much less about the potential effects of such environment variables than you and me. > I suppose we could just shadow emacs in the path with a wrapper that > includes the argument(s), assuming few of the projects have their own > interfering options, and that they always use the same (or a handful of > predictable) relative invocation of emacs, e.g. "emacs" not > "/usr/bin/emacs" or... Yes, something like that. At least that's what we do in the Emacs's own test suite (which, of course, is much smaller than yours). > > With respect to writing the *.eln files, Emacs will look along > > native-comp-eln-load-path for the first directory that is writable, > > and use that. So you could at least partially handle this case by > > making sure native-comp-eln-load-path includes at least one writable > > directory. > > Right, though we also have to make sure we can do that sufficiently > early in *every* case. Yes, ideally from the same/similar wrapper script that actually invokes Emacs, via the --eval option. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 6:30 ` Eli Zaretskii @ 2022-10-15 17:00 ` Rob Browning 0 siblings, 0 replies; 343+ messages in thread From: Rob Browning @ 2022-10-15 17:00 UTC (permalink / raw) To: Eli Zaretskii; +Cc: luangruo, larsi, akrl, monnier, david, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > I meant HOME=/does-not-exist, yes. If that doesn't work (but please > test), we should fix that, I think. Please report your findings if > that doesn't work. We use that in our own test suite. I've been told that it doesn't because (the person thought) of trampolines. i.e. Emacs can be reliably crashed that way. Though if that's changed since 28.[12], they wouldn't have seen it yet, and there may be patches we should cherry-pick. > I understand, but don't you have some kind of centralized script or > group of scripts that runs the testing? In that case, you could make > the Emacs command, or its template, be part of those few scripts, and > then the change will be less painful. For us, this is about hundreds of upstream trees we don't directly control. How and where does magit invoke emacs during its build or during its tests? Or buttercup, or org, or bbdb, or emacspeak, or ivy, or lsp, or... How do we make sure we affect *all* of the invocations of emacs in all of those? Note here is just a subset things that Debian has packaged that might be affected (i.e. only the ones whose maintainers have decided to keep the Debian tree on salsa in the emacsen-team project): https://salsa.debian.org/emacsen-team > Yes, something like that. At least that's what we do in the Emacs's > own test suite (which, of course, is much smaller than yours). ...as long as "most" invocations are either "emacs" or respect say EMACS, this could work. I have no idea if that's true, but we do already know there are likely to be any number of places we'll have to fix. i.e. someone has told me that this approach will result in "a lot of bugs" (in the debian packages). They already know of places where "some of them try to choose which emacs to use, rather than just 'emacs'". We can handle that, but if we have to patch a lot of the upstream trees, that could be a good bit of work. That's certainly possible, but hopefully that helps give more context about why we were interested in some solution, among the possibilities, that's less potentially invasive. For what it's worth, we're also under a (not too close yet) deadline on the Debian side. We'd really like to have at least emacs 28 in the next stable release, and the first stage of the freeze is currently scheduled for I think January (though Emacs would I think be affected by a slightly later stage). For that to be possible, whatever we end up choosing here will need to be something we can finish accommodating across both the emacs package and all the add-ons by then. Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 17:53 ` Rob Browning 2022-10-14 18:36 ` Stefan Monnier 2022-10-14 19:06 ` Eli Zaretskii @ 2022-10-15 9:32 ` Lars Ingebrigtsen 2022-10-15 16:48 ` Sean Whitton 2022-10-15 17:21 ` Rob Browning 2 siblings, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-15 9:32 UTC (permalink / raw) To: Rob Browning; +Cc: Eli Zaretskii, Po Lu, akrl, monnier, david, emacs-devel Rob Browning <rlb@defaultvalue.org> writes: > - We'd prefer to still be able to set HOME=/does-not-exist, which I > assume would mean that we'd need some other way to redirect *all* of > the eln file writes (including trampolines). I think this should work, but there may be regressions, of course. Let's see... I tried it now, and I got the warning below, but things otherwise seem to work fine: Warning (initialization): Unable to create `user-emacs-directory' (~/.emacs.d/). Any data that would normally be written there may be lost! If you never want to see this message again, customize the variable `user-emacs-directory-warning'. > - If we are going to have "some other way" to redirect the eln files, > then for us an environment variable might be easier, so that we can > just export it before we start the relevant build/test/etc. and then > it'll affect all invocations of emacs in that environment. I > suspect an environment variable might also make it easier to > establish the setting "early enough" in the emacs startup process, > but don't know that. Emacs 29 now has the `inhibit-automatic-native-compilation' variable and and `EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION' environment variables to inhibit writing to ~/.emacs.d/eln-cache... > - As an aside, I suspect Emacs may eventually want to have some way to > restore the ability to tolerate an unwritable filesystem. I have > more than once in the past launched emacs from an emergency shell to > fix a broken host where the filesystem was read-only and /home might > not be mounted (and if it were on NFS and there's no network yet, > could not be). Emacs should work with an unwritable file system, but there's probably code that bugs out in that situation -- but those things should be fixed. If you have a test case that demonstrates the problem, please open a bug report for that, so that we can get fixin'. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 9:32 ` Lars Ingebrigtsen @ 2022-10-15 16:48 ` Sean Whitton 2022-10-16 8:56 ` Lars Ingebrigtsen 2022-10-15 17:21 ` Rob Browning 1 sibling, 1 reply; 343+ messages in thread From: Sean Whitton @ 2022-10-15 16:48 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Rob Browning, Eli Zaretskii, Po Lu, akrl, monnier, david, emacs-devel Hello, On Sat 15 Oct 2022 at 11:32AM +02, Lars Ingebrigtsen wrote: > Emacs 29 now has the `inhibit-automatic-native-compilation' variable and > and `EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION' environment variables > to inhibit writing to ~/.emacs.d/eln-cache... Is it likely to be okay for us to backport those commits to our 28.2? -- Sean Whitton ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 16:48 ` Sean Whitton @ 2022-10-16 8:56 ` Lars Ingebrigtsen 0 siblings, 0 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-16 8:56 UTC (permalink / raw) To: Sean Whitton Cc: Rob Browning, Eli Zaretskii, Po Lu, akrl, monnier, david, emacs-devel Sean Whitton <spwhitton@spwhitton.name> writes: >> Emacs 29 now has the `inhibit-automatic-native-compilation' variable and >> and `EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION' environment variables >> to inhibit writing to ~/.emacs.d/eln-cache... > > Is it likely to be okay for us to backport those commits to our 28.2? Yes. But probably wait a bit -- the semantics aren't set in stone yet. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 9:32 ` Lars Ingebrigtsen 2022-10-15 16:48 ` Sean Whitton @ 2022-10-15 17:21 ` Rob Browning 2022-10-15 17:25 ` Eli Zaretskii 2022-10-16 9:01 ` Lars Ingebrigtsen 1 sibling, 2 replies; 343+ messages in thread From: Rob Browning @ 2022-10-15 17:21 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, Po Lu, akrl, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > I think this should work, but there may be regressions, of course. > Let's see... I tried it now, and I got the warning below, but things > otherwise seem to work fine: > > Warning (initialization): Unable to create `user-emacs-directory' (~/.emacs.d/). > Any data that would normally be written there may be lost! > If you never want to see this message again, > customize the variable `user-emacs-directory-warning'. We just got 28.2 into Debian, and we should likely re-evaluate that version, as compared to 28.1. See if the behaviors have changed. > Emacs should work with an unwritable file system, but there's probably > code that bugs out in that situation -- but those things should be > fixed. > > If you have a test case that demonstrates the problem, please open a bug > report for that, so that we can get fixin'. OK, so if there is (or we come to) an upstream consensus that HOME=/does-not-exist should work (would that be considered a subset of the unwritable filesystem case?), then I think that's likely sufficient for Debian packaging. (That's what we were initially attempting.) And if it doesn't yet work, we're very likely to be able to help track down and fix those cases as we work through all the add-on packages. Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 17:21 ` Rob Browning @ 2022-10-15 17:25 ` Eli Zaretskii 2022-10-16 9:01 ` Lars Ingebrigtsen 1 sibling, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-15 17:25 UTC (permalink / raw) To: Rob Browning; +Cc: larsi, luangruo, akrl, monnier, david, emacs-devel > From: Rob Browning <rlb@defaultvalue.org> > Cc: Eli Zaretskii <eliz@gnu.org>, Po Lu <luangruo@yahoo.com>, akrl@sdf.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org > Date: Sat, 15 Oct 2022 12:21:29 -0500 > > > Emacs should work with an unwritable file system, but there's probably > > code that bugs out in that situation -- but those things should be > > fixed. > > > > If you have a test case that demonstrates the problem, please open a bug > > report for that, so that we can get fixin'. > > OK, so if there is (or we come to) an upstream consensus that > HOME=/does-not-exist should work (would that be considered a subset of > the unwritable filesystem case?), then I think that's likely sufficient > for Debian packaging. (That's what we were initially attempting.) It should work, yes. And note that the message cited here, i.e. > Warning (initialization): Unable to create `user-emacs-directory' (~/.emacs.d/). > Any data that would normally be written there may be lost! > If you never want to see this message again, > customize the variable `user-emacs-directory-warning'. is just a warning, it shouldn't prevent Emacs from running. And yes, features that want to save files under ~/.emacs.d will be unable to do so, and could signal errors, unless you redefine the variables which hold the respective file names to point to some other place. This is the intended behavior, I believe. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 17:21 ` Rob Browning 2022-10-15 17:25 ` Eli Zaretskii @ 2022-10-16 9:01 ` Lars Ingebrigtsen 2022-10-16 16:59 ` Rob Browning 1 sibling, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-16 9:01 UTC (permalink / raw) To: Rob Browning; +Cc: Eli Zaretskii, Po Lu, akrl, monnier, david, emacs-devel Rob Browning <rlb@defaultvalue.org> writes: > OK, so if there is (or we come to) an upstream consensus that > HOME=/does-not-exist should work (would that be considered a subset of > the unwritable filesystem case?), The cases are somewhat different, and may tickle different code paths. For instance, there may be guards in the code where we want to write to, for instance the autosave file, that checks whether the directory exists, and if not, handles that gracefully. But perhaps there's no error handling around actually writing the file, leading Emacs to bug out in bad ways. (Just a random theoretical example.) My guess is that we don't do much testing for the unwritable filesystems case, so there may be many bugs in this area. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-16 9:01 ` Lars Ingebrigtsen @ 2022-10-16 16:59 ` Rob Browning 2022-10-17 10:37 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-16 16:59 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, Po Lu, akrl, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > My guess is that we don't do much testing for the unwritable filesystems > case, so there may be many bugs in this area. Oh, bugs are "fine". Right now, on that front, we just wanted to explain the situation and determine what the expectations are. i.e. if we hit trouble there, from the upstream perspective are we in "don't do that then" territory, or have we found something that might warrant attention? Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-16 16:59 ` Rob Browning @ 2022-10-17 10:37 ` Lars Ingebrigtsen 0 siblings, 0 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-17 10:37 UTC (permalink / raw) To: Rob Browning; +Cc: Eli Zaretskii, Po Lu, akrl, monnier, david, emacs-devel Rob Browning <rlb@defaultvalue.org> writes: > Right now, on that front, we just wanted to explain the situation and > determine what the expectations are. i.e. if we hit trouble there, from > the upstream perspective are we in "don't do that then" territory, or > have we found something that might warrant attention? The latter, definitely. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 13:18 ` Andrea Corallo 2022-10-05 14:01 ` Lars Ingebrigtsen @ 2022-10-05 20:37 ` Lars Ingebrigtsen 2022-10-05 23:40 ` Andrea Corallo 1 sibling, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 20:37 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > Also repeatable testing are most likely executed in batch mode and... > > ...surprise surprise deferred compilation is *already* *disabled* in > this mode!! I forgot to mention that while this is (strictly speaking) true, if the --batch Emacs redefines any built-in functions, the trampoline files will get written to ~/.emacs.d/eln-cache/. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 20:37 ` Lars Ingebrigtsen @ 2022-10-05 23:40 ` Andrea Corallo 2022-10-05 23:46 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 23:40 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> Also repeatable testing are most likely executed in batch mode and... >> >> ...surprise surprise deferred compilation is *already* *disabled* in >> this mode!! > > I forgot to mention that while this is (strictly speaking) true, Not sure what you mean by strictly speaking, I think is just true. > if the > --batch Emacs redefines any built-in functions, the trampoline files > will get written to ~/.emacs.d/eln-cache/. Indeed, and I don't find anything surprising with that. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 23:40 ` Andrea Corallo @ 2022-10-05 23:46 ` Lars Ingebrigtsen 2022-10-06 0:34 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 23:46 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: >>> Also repeatable testing are most likely executed in batch mode and... >>> >>> ...surprise surprise deferred compilation is *already* *disabled* in >>> this mode!! >> >> I forgot to mention that while this is (strictly speaking) true, > > Not sure what you mean by strictly speaking, I think is just true. > >> if the >> --batch Emacs redefines any built-in functions, the trampoline files >> will get written to ~/.emacs.d/eln-cache/. > > Indeed, and I don't find anything surprising with that. I'm sure you don't. But you claimed that --batch was the panacea for repeatable testing -- and it's not, since the second time you run the same test, the trampoline files are created. I.e., not a repeat of the same test. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 23:46 ` Lars Ingebrigtsen @ 2022-10-06 0:34 ` Andrea Corallo 2022-10-06 0:39 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-06 0:34 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >>>> Also repeatable testing are most likely executed in batch mode and... >>>> >>>> ...surprise surprise deferred compilation is *already* *disabled* in >>>> this mode!! >>> >>> I forgot to mention that while this is (strictly speaking) true, >> >> Not sure what you mean by strictly speaking, I think is just true. >> >>> if the >>> --batch Emacs redefines any built-in functions, the trampoline files >>> will get written to ~/.emacs.d/eln-cache/. >> >> Indeed, and I don't find anything surprising with that. > > I'm sure you don't. But you claimed that --batch was the panacea for > repeatable testing Did I?? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 0:34 ` Andrea Corallo @ 2022-10-06 0:39 ` Lars Ingebrigtsen 2022-10-06 1:03 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-06 0:39 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: >>>>> Also repeatable testing are most likely executed in batch mode and... >>>>> >>>>> ...surprise surprise deferred compilation is *already* *disabled* in >>>>> this mode!! [...] >> I'm sure you don't. But you claimed that --batch was the panacea for >> repeatable testing > > Did I?? Yes. *points* ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 0:39 ` Lars Ingebrigtsen @ 2022-10-06 1:03 ` Andrea Corallo 2022-10-06 1:07 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-06 1:03 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >>>>>> Also repeatable testing are most likely executed in batch mode and... >>>>>> >>>>>> ...surprise surprise deferred compilation is *already* *disabled* in >>>>>> this mode!! > > [...] > >>> I'm sure you don't. But you claimed that --batch was the panacea for >>> repeatable testing >> >> Did I?? > > Yes. *points* Where? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 1:03 ` Andrea Corallo @ 2022-10-06 1:07 ` Lars Ingebrigtsen 2022-10-06 1:19 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-06 1:07 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: >>>>>>> Also repeatable testing are most likely executed in batch mode and... >>>>>>> >>>>>>> ...surprise surprise deferred compilation is *already* *disabled* in >>>>>>> this mode!! [...] > Where? There. But if you were just doing another non-sequitur -- that you didn't mean that "deferred compilation" and --batch is related to repeatable testing, that's fine. You were just making conversation and not claiming anything? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 1:07 ` Lars Ingebrigtsen @ 2022-10-06 1:19 ` Andrea Corallo 2022-10-06 1:26 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-06 1:19 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >>>>>>>> Also repeatable testing are most likely executed in batch mode and... >>>>>>>> >>>>>>>> ...surprise surprise deferred compilation is *already* *disabled* in >>>>>>>> this mode!! > > [...] > >> Where? > > There. Oh my! FYI this is what we wrote: > >> . understand why and in which situations they may need it > > > > Doing repeatable testing is one obvious situation. > > I think the two previously mentioned knobs should be sufficient for > repeatable testing no? As you can see I said that the *two* mentioned knobs should be sufficient for test repeatability, not that "--batch is the panacea for repeatable testing". > But if you were just doing another non-sequitur -- that you didn't mean > that "deferred compilation" and --batch is related to repeatable > testing, that's fine. You were just making conversation and not > claiming anything? Sorry are you serious? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 1:19 ` Andrea Corallo @ 2022-10-06 1:26 ` Lars Ingebrigtsen 2022-10-06 1:39 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-06 1:26 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > As you can see I said that the *two* mentioned knobs should be > sufficient for test repeatability, not that "--batch is the panacea for > repeatable testing". The two mentioned knobs are `native-comp-deferred-compilation' and `load-no-native'? And as I've explained, those are not sufficient, because trampolines are written to ~/.emacs.d/eln-cache even those two switched off. And you're being revisionist, because this is what you said: > I think the two previously mentioned knobs should be sufficient for > repeatable testing no? > > Also repeatable testing are most likely executed in batch mode and... > > ...surprise surprise deferred compilation is *already* *disabled* in > this mode!! > > I've the impression no one mentioned this small detail in this huge > thread, but still we have installed changes to disable a feature for > debian pkg installation that is in fact already disabled :( :( The latter is simply not true. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 1:26 ` Lars Ingebrigtsen @ 2022-10-06 1:39 ` Andrea Corallo 2022-10-06 12:07 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-06 1:39 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> As you can see I said that the *two* mentioned knobs should be >> sufficient for test repeatability, not that "--batch is the panacea for >> repeatable testing". > > The two mentioned knobs are `native-comp-deferred-compilation' and > `load-no-native'? yes > And as I've explained, those are not sufficient, > because trampolines are written to ~/.emacs.d/eln-cache even those two > switched off. Sorry in my limited experience in debugging Emacs native code execution those knobs are the one to be used for this case. Sure trampolines add a state (there are probably other states Emacs can generate I'm not aware) but that said I've never had test repeatability issues with trampolines. Anyway I thought we were discussing / trying to solve mainly the user request. Is this about test repeatability? Did we had repeatability issues with native code I'm not aware? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 1:39 ` Andrea Corallo @ 2022-10-06 12:07 ` Lars Ingebrigtsen 0 siblings, 0 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-06 12:07 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, rlb, monnier, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > Anyway I thought we were discussing / trying to solve mainly the user > request. Is this about test repeatability? Did we had repeatability > issues with native code I'm not aware? There were two things requested, and one of them was related to test repeatability (which then also requires not writing anything to eln-cache). It's not a matter of specific issues with native code, but a general principle -- you should be able to run the same test twice and have it trigger all the same code paths. That's just a general maintenance principle. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 19:08 ` Lars Ingebrigtsen 2022-10-02 19:15 ` Eli Zaretskii @ 2022-10-02 20:05 ` Rob Browning 2022-10-02 20:10 ` Lars Ingebrigtsen 2022-10-05 12:43 ` Andrea Corallo 2 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-02 20:05 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Stefan Monnier, Eli Zaretskii, david, emacs-devel, akrl Lars Ingebrigtsen <larsi@gnus.org> writes: > So I'm thinking of introducing a user option like > `native-compile-inhibit', which will make Emacs skip the native-comp > machinery when loading .elc files. It will default to nil, of course, > but perhaps it would be convenient to make it depend on an environment > variable like "NATIVE_COMPILE_INHIBIT"? Then users (and the Debian > build system) could say "NATIVE_COMPILE_INHIBIT=true emacs ..." when > doing testing etc? Would that fit your use case? It might? I'd also suggest adding an EMACS_... prefix to the variable name (of course I'd suggest that for all of any program's variables). Though I'm not positive because if we still really need compilation for the trampolines(?), then we may need to just follow the "redirect to a tempdir" approach, which is also fine, and then the ability to disable native compilation might not be relevant to the packaging. In either case, we might ideally want an EMACS_... var to make propagation a bit easier if that sounds reasonable. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 20:05 ` Rob Browning @ 2022-10-02 20:10 ` Lars Ingebrigtsen 2022-10-05 13:19 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-02 20:10 UTC (permalink / raw) To: Rob Browning; +Cc: Stefan Monnier, Eli Zaretskii, david, emacs-devel, akrl Rob Browning <rlb@defaultvalue.org> writes: > Though I'm not positive because if we still really need compilation for > the trampolines(?), then we may need to just follow the "redirect to a > tempdir" approach, which is also fine, and then the ability to disable > native compilation might not be relevant to the packaging. I'm not sure we need to save the trampolines at all (in this don't-do-native-compilation configuration) -- Andrea probably knows. Andrea, would it be possible to just create and use the trampolines without writing them to a file? > In either case, we might ideally want an EMACS_... var to make > propagation a bit easier if that sounds reasonable. Sure. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 20:10 ` Lars Ingebrigtsen @ 2022-10-05 13:19 ` Andrea Corallo 0 siblings, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 13:19 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Rob Browning, Stefan Monnier, Eli Zaretskii, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Rob Browning <rlb@defaultvalue.org> writes: > >> Though I'm not positive because if we still really need compilation for >> the trampolines(?), then we may need to just follow the "redirect to a >> tempdir" approach, which is also fine, and then the ability to disable >> native compilation might not be relevant to the packaging. > > I'm not sure we need to save the trampolines at all (in this > don't-do-native-compilation configuration) -- Andrea probably knows. > Andrea, would it be possible to just create and use the trampolines > without writing them to a file? No Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 19:08 ` Lars Ingebrigtsen 2022-10-02 19:15 ` Eli Zaretskii 2022-10-02 20:05 ` Rob Browning @ 2022-10-05 12:43 ` Andrea Corallo 2022-10-05 13:16 ` Lars Ingebrigtsen 2 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 12:43 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Rob Browning, Stefan Monnier, Eli Zaretskii, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Rob Browning <rlb@defaultvalue.org> writes: > >> At the top level, we wanted a way to avoid writing to HOME during >> packaging, testing, installs (in this case, it's the .eln files, now >> that we've enabled native compilation). >> >> That could be handled by some way to turn off native compilation, or by >> some way to comprehensively divert those writes to another location >> (e.g. temp dir). Either is fine, though we'd originally thought the >> former might make things a bit easier. > > Yeah, I think the former is both easier to implement and easier for > users. > > So I'm thinking of introducing a user option like > `native-compile-inhibit', which will make Emacs skip the native-comp > machinery when loading .elc files. IIUC this would just control `load-no-native' and `native-comp-deferred-compilation'? I think these two are doing what you suggest here no? Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 12:43 ` Andrea Corallo @ 2022-10-05 13:16 ` Lars Ingebrigtsen 2022-10-05 13:29 ` Andrea Corallo 2022-10-05 14:03 ` Eli Zaretskii 0 siblings, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 13:16 UTC (permalink / raw) To: Andrea Corallo Cc: Rob Browning, Stefan Monnier, Eli Zaretskii, david, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > IIUC this would just control `load-no-native' and > `native-comp-deferred-compilation'? I think these two are doing what > you suggest here no? The latter yes -- nobody has discussed doing anything with `load-no-native' (which is another variable that's difficult to discover due to how it's named, and isn't documented anywhere, either). ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 13:16 ` Lars Ingebrigtsen @ 2022-10-05 13:29 ` Andrea Corallo 2022-10-05 14:03 ` Eli Zaretskii 1 sibling, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-05 13:29 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Rob Browning, Stefan Monnier, Eli Zaretskii, david, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> IIUC this would just control `load-no-native' and >> `native-comp-deferred-compilation'? I think these two are doing what >> you suggest here no? > > The latter yes -- nobody has discussed doing anything with > `load-no-native' I wasn't clear to me what you meant with "native-comp machinery when loading .elc files". > (which is another variable that's difficult to discover due to how > it's named, and isn't documented anywhere, either). It is named like that since before the native comp branch was reviewed and merged, I think you were in that thread. Sorry we all try our best. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 13:16 ` Lars Ingebrigtsen 2022-10-05 13:29 ` Andrea Corallo @ 2022-10-05 14:03 ` Eli Zaretskii 1 sibling, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-05 14:03 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: akrl, rlb, monnier, david, emacs-devel > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Rob Browning <rlb@defaultvalue.org>, Stefan Monnier > <monnier@iro.umontreal.ca>, Eli Zaretskii <eliz@gnu.org>, > david@tethera.net, emacs-devel@gnu.org > Date: Wed, 05 Oct 2022 15:16:31 +0200 > > Andrea Corallo <akrl@sdf.org> writes: > > > IIUC this would just control `load-no-native' and > > `native-comp-deferred-compilation'? I think these two are doing what > > you suggest here no? > > The latter yes -- nobody has discussed doing anything with > `load-no-native' (which is another variable that's difficult to discover > due to how it's named, and isn't documented anywhere, either). Variables that are "hard to discover" are intentionally named like that: we didn't intend them to be discoverable, because we didn't think they should be used except internally. During the last stages of native-comp development, we went through the variables in comp.c and comp.el with Andrea, and renamed those we thought would be useful so that they are easily discoverable. Please don't assume that the names you see are due to omission or negligence or ineptitude. Maybe additional ones should be renamed/aliased to be more discoverable, but before we do that, we need to understand the problems we are solving. That requires careful consideration and discussion, whereas rushing with installing changes in the area where you don't have enough prior knowledge and history ends up wreaking havoc, to say nothing of personal aggravation. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:17 ` Lars Ingebrigtsen 2022-10-02 17:28 ` Stefan Monnier @ 2022-10-02 17:30 ` Eli Zaretskii 2022-10-02 17:38 ` Lars Ingebrigtsen 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 17:30 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: monnier, rlb, david, emacs-devel, akrl > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Eli Zaretskii <eliz@gnu.org>, Rob Browning <rlb@defaultvalue.org>, > david@tethera.net, emacs-devel@gnu.org, akrl@sdf.org > Date: Sun, 02 Oct 2022 19:17:27 +0200 > > Stefan Monnier <monnier@iro.umontreal.ca> writes: > > > But I think I agree with Rob that it makes a lot of sense in the context > > of Debian to eagerly native-compile the packages when they're installed > > via APT. > > I've always assumed that that's what distributions like Debian would > do. Which means that there should be an easy way to switch JIT off, but > I've just forgotten to add it. If everything is already natively-compiled, there's no reason to switch it off: it will simply not happen. OTOH, if for some reason an existing .eln file is incompatible with the user's Emacs (something that can happen easily on a multi-user system), having JIT compilation active is a clear advantage. So there's a net win in having it enabled at all times. In any case, we already have a way of switching off JIT compilation, at least for some situations which we considered. If there are situations which we didn't consider, we should consider them as well, but for that we need to understand the situation in more detail, something that no one presented yet in this discussion. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:30 ` Eli Zaretskii @ 2022-10-02 17:38 ` Lars Ingebrigtsen 2022-10-02 17:48 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-02 17:38 UTC (permalink / raw) To: Eli Zaretskii; +Cc: monnier, rlb, david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > So there's a net win in having it enabled at all times. You may think so, and I may think so, but it should be up to the user (and the distribution). > In any case, we already have a way of switching off JIT compilation, > at least for some situations which we considered. I thought there wasn't a user option to switch it off? I must admit I haven't been paying that much attention to the thread, but if somebody has identified such a user option, then everything's OK. What's it called? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:38 ` Lars Ingebrigtsen @ 2022-10-02 17:48 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 17:48 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: monnier, rlb, david, emacs-devel, akrl > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: monnier@iro.umontreal.ca, rlb@defaultvalue.org, david@tethera.net, > emacs-devel@gnu.org, akrl@sdf.org > Date: Sun, 02 Oct 2022 19:38:01 +0200 > > Eli Zaretskii <eliz@gnu.org> writes: > > > So there's a net win in having it enabled at all times. > > You may think so, and I may think so, but it should be up to the user > (and the distribution). If they present their case clearly, we might change our minds. If they don't present the case clearly, I don't see how we can provide a solution, because disabling JIT compilation is already possible. So someone should explain why what is there is not good enough, and that requires details. > > In any case, we already have a way of switching off JIT compilation, > > at least for some situations which we considered. > > I thought there wasn't a user option to switch it off? There are several, actually, each one doing something slightly different and disabling native compilation on a different level. That's why I'm asking for details: we need to understand whether something is missing and why. For example, "disable native compilation" may or may not be the same as "disable writing *.eln files". ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:53 ` Stefan Monnier 2022-10-02 17:16 ` Eli Zaretskii 2022-10-02 17:17 ` Lars Ingebrigtsen @ 2022-10-02 17:37 ` Rob Browning 2022-10-02 17:44 ` Eli Zaretskii 2022-10-02 23:53 ` Sean Whitton 3 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-02 17:37 UTC (permalink / raw) To: Stefan Monnier, Eli Zaretskii; +Cc: david, emacs-devel, akrl Stefan Monnier <monnier@iro.umontreal.ca> writes: > In any case, I'd let Debian's maintainers make their own choices for > their own specific needs which are slightly different from ours (where > our release tarballs and default config are designed in large part for > users who'll compile Emacs themselves and who install third party > ELisp packages into their $HOME). For what it's worth, we've also had repeated requests for additional variants of emacs. Right now we have emacs-nox (No X), emacs-lucid, and emacs-gtk. The first is often useful for server installs, but we've had requests for something even smaller, say an emacs-min that configures even less, and is even smaller (including the dependency tree), for some situations. Presumably people who would rather be able to easily use emacs in some constrained environments instead of nano, zile, vi, ... etc. We've also for a long time made the emacs-el package (containing all the .el files) optional for similar reasons (constrained environments), but we recently had to start requiring it because (not sure why yet) emacs just crashes in some situations when the .el files aren't available and when native compilation is enabled. If it hasn't already been fixed in 28.2 (haven't tested yet), it'd be nice if we could eventually track that down and then switch emacs-el back to optional (back to "suggested" in Debian dependency parlance). We'll likely try to help with that later, after we get the other native compilation-related issues we've been discussing settled down on the Debian side. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:37 ` Rob Browning @ 2022-10-02 17:44 ` Eli Zaretskii 2022-10-02 18:21 ` Rob Browning 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 17:44 UTC (permalink / raw) To: Rob Browning; +Cc: monnier, david, emacs-devel, akrl > From: Rob Browning <rlb@defaultvalue.org> > Cc: david@tethera.net, emacs-devel@gnu.org, akrl@sdf.org > Date: Sun, 02 Oct 2022 12:37:33 -0500 > > We've also for a long time made the emacs-el package (containing all the > .el files) optional for similar reasons (constrained environments), but > we recently had to start requiring it because (not sure why yet) emacs > just crashes in some situations when the .el files aren't available and > when native compilation is enabled. Loading of natively-compiled files cannot work if the source files aren't accessible, because loading a .eln file involves verifying that it was produced from that particular .el file. (The .el file can be compressed if Emacs was compiled with zlib support.) But Emacs should not "crash" if the *.el files aren't available, it should simply refuse to load any *.eln files and load the *.elc files instead. That produces many warnings, of course, but I hope your users don't consider that "crashing". > If it hasn't already been fixed in 28.2 (haven't tested yet), it'd be > nice if we could eventually track that down and then switch emacs-el > back to optional (back to "suggested" in Debian dependency parlance). It isn't a bug, but intended behavior. If we want to remove this dependency, some non-trivial ideas about reworking the current load procedure should emerge. I don't thin we have any such ideas at this time. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:44 ` Eli Zaretskii @ 2022-10-02 18:21 ` Rob Browning 2022-10-02 18:43 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-02 18:21 UTC (permalink / raw) To: Eli Zaretskii; +Cc: monnier, david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > But Emacs should not "crash" if the *.el files aren't available, it > should simply refuse to load any *.eln files and load the *.elc files > instead. That produces many warnings, of course, but I hope your > users don't consider that "crashing". I believe it was *crashing*. I can't recall if that one was a segfault, or something a bit less drastic, but I'll try to remember to track it down later. > It isn't a bug, but intended behavior. If we want to remove this > dependency, some non-trivial ideas about reworking the current load > procedure should emerge. I don't thin we have any such ideas at this > time. OK, so we should consider that a hard dependency now, i.e. the emacs-el package can't be optional anymore, at least not on architectures where we can enable native compilation, and so probably just "everywhere" for simplicity, if nothing else. Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:21 ` Rob Browning @ 2022-10-02 18:43 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 18:43 UTC (permalink / raw) To: Rob Browning; +Cc: monnier, david, emacs-devel, akrl > From: Rob Browning <rlb@defaultvalue.org> > Cc: monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, > akrl@sdf.org > Date: Sun, 02 Oct 2022 13:21:54 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > > But Emacs should not "crash" if the *.el files aren't available, it > > should simply refuse to load any *.eln files and load the *.elc files > > instead. That produces many warnings, of course, but I hope your > > users don't consider that "crashing". > > I believe it was *crashing*. I can't recall if that one was a segfault, > or something a bit less drastic, but I'll try to remember to track it > down later. If it's a real crash, I'd appreciate bug reports, TIA. > > It isn't a bug, but intended behavior. If we want to remove this > > dependency, some non-trivial ideas about reworking the current load > > procedure should emerge. I don't thin we have any such ideas at this > > time. > > OK, so we should consider that a hard dependency now, i.e. the emacs-el > package can't be optional anymore, at least not on architectures where > we can enable native compilation, and so probably just "everywhere" for > simplicity, if nothing else. More accurately, emacs-el cannot be optional when the installed Emacs supports native compilation. Emacs can still be built without native compilation, and I presume those of your users who want the minimal possible package will want that, since it removes several significant dependencies, like libgccjit itself. When Emacs is built without native compilation, it can work without the *.el files. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:53 ` Stefan Monnier ` (2 preceding siblings ...) 2022-10-02 17:37 ` Rob Browning @ 2022-10-02 23:53 ` Sean Whitton 3 siblings, 0 replies; 343+ messages in thread From: Sean Whitton @ 2022-10-02 23:53 UTC (permalink / raw) To: Stefan Monnier; +Cc: Eli Zaretskii, Rob Browning, david, emacs-devel, akrl Hello, On Sun 02 Oct 2022 at 12:53PM -04, Stefan Monnier wrote: > FWIW, I'm not convinced it's really useful in Debian's `emacs` package > to eagerly native compile all the bundled .elc files, but I think it > does make a lot of sense for ELisp packages installed separately. Could you elaborate on this point? -- Sean Whitton ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 5:57 ` Eli Zaretskii 2022-10-02 6:10 ` tomas 2022-10-02 16:53 ` Stefan Monnier @ 2022-10-02 17:15 ` Rob Browning 2022-10-02 17:25 ` Stefan Monnier 2022-10-02 17:26 ` Eli Zaretskii 2022-10-02 23:51 ` Sean Whitton 3 siblings, 2 replies; 343+ messages in thread From: Rob Browning @ 2022-10-02 17:15 UTC (permalink / raw) To: Eli Zaretskii; +Cc: monnier, david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > Battery consumption doesn't seem very relevant, because JIT > compilation will happen when the system is used, not when it sleeps. > And it is entirely not guaranteed that by compiling everything you > will save power in the long run, because there are good chances you > will be compiling stuff that won't be used for a long time. Without > quantitative data of long-term power usage on which to base the > conclusions, I don't see why you should a-priori assume that compiling > everything from the get-go should use less power. Same goes for disk > space by multiple users. On this particular topic, I was actually just communicating a concern that was communicated to me -- that a user wasn't happy about the unpredictable, compilation spike long after the install on their laptop, (in part due to concerns about power consumption). (Personally, as a user, I'd also prefer to pay the cost up front, most of the time, during package install, and to avoid per-user duplications where possible, but that said, I generally do try to avoid basing Debian packaging decisions solely on my preferences, and I do want to try to favor upstream preferences in general.) -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:15 ` Rob Browning @ 2022-10-02 17:25 ` Stefan Monnier 2022-10-02 18:11 ` Stefan Kangas 2022-10-02 17:26 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-02 17:25 UTC (permalink / raw) To: Rob Browning; +Cc: Eli Zaretskii, david, emacs-devel, akrl > On this particular topic, I was actually just communicating a concern > that was communicated to me -- that a user wasn't happy about the > unpredictable, compilation spike long after the install on their laptop, > (in part due to concerns about power consumption). FWIW, regardless of what Debian decides to do, I think it would make sense to offer some way to configure Emacs so that it native compiles packages a bit more eagerly. E.g. we could offer a configuration option that causes `package-install` to eagerly native compile the installed files. And we could offer a "recompile" command that users can use after upgrading Emacs to eagerly native-(re)compile all the ELPA-installed packages in the user's $HOME. The situation is very different for the bundled packages, since most of them will never be used by most users, so JIT compilation is much more beneficial. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:25 ` Stefan Monnier @ 2022-10-02 18:11 ` Stefan Kangas 2022-10-02 18:26 ` Stefan Monnier 0 siblings, 1 reply; 343+ messages in thread From: Stefan Kangas @ 2022-10-02 18:11 UTC (permalink / raw) To: Stefan Monnier, Rob Browning; +Cc: Eli Zaretskii, david, emacs-devel, akrl Stefan Monnier <monnier@iro.umontreal.ca> writes: > E.g. we could offer a configuration option that causes `package-install` > to eagerly native compile the installed files. Isn't that just `package-native-compile'? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:11 ` Stefan Kangas @ 2022-10-02 18:26 ` Stefan Monnier 2022-10-02 18:57 ` Stefan Kangas 0 siblings, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-02 18:26 UTC (permalink / raw) To: Stefan Kangas; +Cc: Rob Browning, Eli Zaretskii, david, emacs-devel, akrl Stefan Kangas [2022-10-02 18:11:39] wrote: > Stefan Monnier <monnier@iro.umontreal.ca> writes: >> E.g. we could offer a configuration option that causes `package-install` >> to eagerly native compile the installed files. > Isn't that just `package-native-compile'? Oh, it's there! Wee!! Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:26 ` Stefan Monnier @ 2022-10-02 18:57 ` Stefan Kangas 0 siblings, 0 replies; 343+ messages in thread From: Stefan Kangas @ 2022-10-02 18:57 UTC (permalink / raw) To: Stefan Monnier; +Cc: Rob Browning, Eli Zaretskii, david, emacs-devel, akrl Stefan Monnier <monnier@iro.umontreal.ca> writes: >> Isn't that just `package-native-compile'? > > Oh, it's there! > Wee!! BTW, I think nil is the best default for that variable if and only if `native-comp-async-report-warnings-errors' also defaults to nil. Otherwise, users will have a rather scary warning buffer pop up in the middle of trying to do work (or playing `M-x pong', more likely). Maybe we could revisit the latter in time for Emacs 29, or at the latest before switching native compilation to on by default (Emacs 30?). ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:15 ` Rob Browning 2022-10-02 17:25 ` Stefan Monnier @ 2022-10-02 17:26 ` Eli Zaretskii 1 sibling, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 17:26 UTC (permalink / raw) To: Rob Browning; +Cc: monnier, david, emacs-devel, akrl > From: Rob Browning <rlb@defaultvalue.org> > Cc: monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, > akrl@sdf.org > Date: Sun, 02 Oct 2022 12:15:27 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > > Battery consumption doesn't seem very relevant, because JIT > > compilation will happen when the system is used, not when it sleeps. > > And it is entirely not guaranteed that by compiling everything you > > will save power in the long run, because there are good chances you > > will be compiling stuff that won't be used for a long time. Without > > quantitative data of long-term power usage on which to base the > > conclusions, I don't see why you should a-priori assume that compiling > > everything from the get-go should use less power. Same goes for disk > > space by multiple users. > > On this particular topic, I was actually just communicating a concern > that was communicated to me -- that a user wasn't happy about the > unpredictable, compilation spike long after the install on their laptop, > (in part due to concerns about power consumption). That's just something that takes getting used to. > (Personally, as a user, I'd also prefer to pay the cost up front, most > of the time, during package install, and to avoid per-user duplications > where possible, but that said, I generally do try to avoid basing > Debian packaging decisions solely on my preferences, and I do want to > try to favor upstream preferences in general.) Those seem like old habit that die hard, IMO. For users of some programming languages, JIT compilation is routine, and I bet they don't share these habits. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 5:57 ` Eli Zaretskii ` (2 preceding siblings ...) 2022-10-02 17:15 ` Rob Browning @ 2022-10-02 23:51 ` Sean Whitton 2022-10-03 16:19 ` Eli Zaretskii 2022-10-04 0:31 ` Po Lu 3 siblings, 2 replies; 343+ messages in thread From: Sean Whitton @ 2022-10-02 23:51 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Rob Browning, monnier, david, emacs-devel, akrl Hello, On Sun 02 Oct 2022 at 08:57AM +03, Eli Zaretskii wrote: > The reasons which you mention in favor of AOT native compilation don't > sound serious enough: I see no problem in having the compilation > happen only when it's needed in those cases. Battery consumption > doesn't seem very relevant, because JIT compilation will happen when > the system is used, not when it sleeps. And it is entirely not > guaranteed that by compiling everything you will save power in the > long run, because there are good chances you will be compiling stuff > that won't be used for a long time. Without quantitative data of > long-term power usage on which to base the conclusions, I don't see > why you should a-priori assume that compiling everything from the > get-go should use less power. Same goes for disk space by multiple > users. The point with battery consumption is not about running vs. standby. The issue is that while users expect that running apt-get will drain the battery, they expect that once apt-get is done, the only battery-hungry processes are ones they start themselves. Laptop users typically avoid running apt-get without mains power plugged in. If I do an 'apt-get upgrade' then afterwards my CPU will be churning away compiling addon packages, so I can't just unplug. -- Sean Whitton ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 23:51 ` Sean Whitton @ 2022-10-03 16:19 ` Eli Zaretskii 2022-10-03 18:23 ` Sean Whitton 2022-10-04 0:31 ` Po Lu 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 16:19 UTC (permalink / raw) To: Sean Whitton; +Cc: rlb, monnier, david, emacs-devel, akrl > From: Sean Whitton <spwhitton@spwhitton.name> > Cc: Rob Browning <rlb@defaultvalue.org>, monnier@iro.umontreal.ca, > david@tethera.net, emacs-devel@gnu.org, akrl@sdf.org > Date: Sun, 02 Oct 2022 16:51:12 -0700 > > The point with battery consumption is not about running vs. standby. > The issue is that while users expect that running apt-get will drain the > battery, they expect that once apt-get is done, the only battery-hungry > processes are ones they start themselves. How does this work in practice? I mean, what exactly do you mean by "processes they start themselves"? You mean, you know by heart each Emacs command that invokes a subprocess, and avoid to invoke them all unless the laptop is plugged in? I don't believe this is practical, because even "C-x C-f" invokes a a subprocess when you visit a file under VCS (which I guess happens a lot to the likes of you and me). And how can a person avoid all of those commands? What am I missing? > Laptop users typically avoid running apt-get without mains power > plugged in. If I do an 'apt-get upgrade' then afterwards my CPU > will be churning away compiling addon packages, so I can't just > unplug. Did you actually try to see that "churning away" in practice, did you look at the time this typically takes, the resources (battery and otherwise) it consumes, etc.? If so, can you share the data: how many *.eln files were produced in how many minutes, how much battery did that consume? You see, I have my data, and it seems to indicate a very short period of initial compilations for a modest consumption of resources. (This isn't a laptop, but I'm acutely aware of my system's load at all times, and have it on the mode line, so I know what kind of computation is going on during that time.) The data I see here doesn't look like it should be a problem for anyone, as long as files are compiled only as they are used. In my case, for example, I see maybe half a dozen *.eln files compiled after the initial startup. I expect to see that on any machine where a user has steady usage patterns -- the compilation flats out very quickly. I cannot imagine how will your CPU churn out for any significant time after an upgrade. A typical compilation of a single .el file is usually very fast, exceptions are extremely rare. And I barely notice the effect of that on system responsiveness. So I could imagine that many people perhaps base these judgments on something they didn't actually try, but just imagined to themselves, and imagined incorrectly, or based their opinions on some other compilations of different languages in different environments. I urge people to actually try this and see what it does, before making up your minds. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 16:19 ` Eli Zaretskii @ 2022-10-03 18:23 ` Sean Whitton 2022-10-03 18:44 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Sean Whitton @ 2022-10-03 18:23 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rlb, monnier, david, emacs-devel, akrl Hello, On Mon 03 Oct 2022 at 07:19PM +03, Eli Zaretskii wrote: >> From: Sean Whitton <spwhitton@spwhitton.name> >> Cc: Rob Browning <rlb@defaultvalue.org>, monnier@iro.umontreal.ca, >> david@tethera.net, emacs-devel@gnu.org, akrl@sdf.org >> Date: Sun, 02 Oct 2022 16:51:12 -0700 >> >> The point with battery consumption is not about running vs. standby. >> The issue is that while users expect that running apt-get will drain the >> battery, they expect that once apt-get is done, the only battery-hungry >> processes are ones they start themselves. > > How does this work in practice? I mean, what exactly do you mean by > "processes they start themselves"? You mean, you know by heart each > Emacs command that invokes a subprocess, and avoid to invoke them all > unless the laptop is plugged in? I don't believe this is practical, > because even "C-x C-f" invokes a a subprocess when you visit a file > under VCS (which I guess happens a lot to the likes of you and me). > And how can a person avoid all of those commands? What am I missing? Native compilation is significantly more CPU intensive than any of these other things, though. > Did you actually try to see that "churning away" in practice, did you > look at the time this typically takes, the resources (battery and > otherwise) it consumes, etc.? If so, can you share the data: how many > *.eln files were produced in how many minutes, how much battery did > that consume? > > You see, I have my data, and it seems to indicate a very short period > of initial compilations for a modest consumption of resources. (This > isn't a laptop, but I'm acutely aware of my system's load at all > times, and have it on the mode line, so I know what kind of > computation is going on during that time.) The data I see here > doesn't look like it should be a problem for anyone, as long as files > are compiled only as they are used. In my case, for example, I see > maybe half a dozen *.eln files compiled after the initial startup. I > expect to see that on any machine where a user has steady usage > patterns -- the compilation flats out very quickly. I cannot imagine > how will your CPU churn out for any significant time after an upgrade. > A typical compilation of a single .el file is usually very fast, > exceptions are extremely rare. And I barely notice the effect of that > on system responsiveness. > > So I could imagine that many people perhaps base these judgments on > something they didn't actually try, but just imagined to themselves, > and imagined incorrectly, or based their opinions on some other > compilations of different languages in different environments. > > I urge people to actually try this and see what it does, before making > up your minds. I have actually tried it, for more than a year now, but admittedly I do not have hard data. Thank you for sharing yours. -- Sean Whitton ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 18:23 ` Sean Whitton @ 2022-10-03 18:44 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 18:44 UTC (permalink / raw) To: Sean Whitton; +Cc: rlb, monnier, david, emacs-devel, akrl > From: Sean Whitton <spwhitton@spwhitton.name> > Cc: rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, > emacs-devel@gnu.org, akrl@sdf.org > Date: Mon, 03 Oct 2022 11:23:15 -0700 > > > How does this work in practice? I mean, what exactly do you mean by > > "processes they start themselves"? You mean, you know by heart each > > Emacs command that invokes a subprocess, and avoid to invoke them all > > unless the laptop is plugged in? I don't believe this is practical, > > because even "C-x C-f" invokes a a subprocess when you visit a file > > under VCS (which I guess happens a lot to the likes of you and me). > > And how can a person avoid all of those commands? What am I missing? > > Native compilation is significantly more CPU intensive than any of these > other things, though. Do you have data to support that? I just cited mine, and it doesn't sound intensive to me. E.g., when some command invokes Git, Git could decide that it's time to do a background GC, and that will be quite CPU intensive. And yet we all use VC without fear. But I must stop raising the noise level here, because that's what all I write seems to be good for -- making noise that no one takes seriously, starting from Lars. Sorry. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 23:51 ` Sean Whitton 2022-10-03 16:19 ` Eli Zaretskii @ 2022-10-04 0:31 ` Po Lu 2022-10-04 6:57 ` Eli Zaretskii 2022-10-04 23:26 ` Sean Whitton 1 sibling, 2 replies; 343+ messages in thread From: Po Lu @ 2022-10-04 0:31 UTC (permalink / raw) To: Sean Whitton Cc: Eli Zaretskii, Rob Browning, monnier, david, emacs-devel, akrl Sean Whitton <spwhitton@spwhitton.name> writes: > The point with battery consumption is not about running vs. standby. > The issue is that while users expect that running apt-get will drain the > battery, they expect that once apt-get is done, the only battery-hungry > processes are ones they start themselves. Laptop users typically avoid > running apt-get without mains power plugged in. If I do an 'apt-get > upgrade' then afterwards my CPU will be churning away compiling addon > packages, so I can't just unplug. I don't know if that's true with Debian users, but nobody I know of objects to running "dnf install" on battery. After all, installing new packages only takes a short amount of time, certainly not enough to significantly affect battery usage. OTOH, async native compilation is something I would definitely plug in for. That applies to other programs too, notably web browsers. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-04 0:31 ` Po Lu @ 2022-10-04 6:57 ` Eli Zaretskii 2022-10-04 8:37 ` Po Lu 2022-10-04 23:26 ` Sean Whitton 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-04 6:57 UTC (permalink / raw) To: Po Lu; +Cc: spwhitton, rlb, monnier, david, emacs-devel, akrl > From: Po Lu <luangruo@yahoo.com> > Cc: Eli Zaretskii <eliz@gnu.org>, Rob Browning <rlb@defaultvalue.org>, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, > akrl@sdf.org > Date: Tue, 04 Oct 2022 08:31:01 +0800 > > Sean Whitton <spwhitton@spwhitton.name> writes: > > > The point with battery consumption is not about running vs. standby. > > The issue is that while users expect that running apt-get will drain the > > battery, they expect that once apt-get is done, the only battery-hungry > > processes are ones they start themselves. Laptop users typically avoid > > running apt-get without mains power plugged in. If I do an 'apt-get > > upgrade' then afterwards my CPU will be churning away compiling addon > > packages, so I can't just unplug. > > I don't know if that's true with Debian users, but nobody I know of > objects to running "dnf install" on battery. After all, installing new > packages only takes a short amount of time, certainly not enough to > significantly affect battery usage. > > OTOH, async native compilation is something I would definitely plug in > for. Why the difference, I wonder? Async native compilation also takes a short amount of time, as the data I posted indicates. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-04 6:57 ` Eli Zaretskii @ 2022-10-04 8:37 ` Po Lu 2022-10-04 9:25 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Po Lu @ 2022-10-04 8:37 UTC (permalink / raw) To: Eli Zaretskii; +Cc: spwhitton, rlb, monnier, david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > Why the difference, I wonder? Async native compilation also takes a > short amount of time, as the data I posted indicates. I don't know, but the fans spin up during async native compilation and not package installation. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-04 8:37 ` Po Lu @ 2022-10-04 9:25 ` Eli Zaretskii 2022-10-04 9:51 ` Po Lu 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-04 9:25 UTC (permalink / raw) To: Po Lu; +Cc: spwhitton, rlb, monnier, david, emacs-devel, akrl > From: Po Lu <luangruo@yahoo.com> > Cc: spwhitton@spwhitton.name, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, > akrl@sdf.org > Date: Tue, 04 Oct 2022 16:37:10 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > > Why the difference, I wonder? Async native compilation also takes a > > short amount of time, as the data I posted indicates. > > I don't know, but the fans spin up during async native compilation and > not package installation. Maybe you should customize native-comp-async-jobs-number to a small enough value, like 1? The default zero means half of your CPU's execution units. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-04 9:25 ` Eli Zaretskii @ 2022-10-04 9:51 ` Po Lu 2022-10-05 13:58 ` Po Lu 0 siblings, 1 reply; 343+ messages in thread From: Po Lu @ 2022-10-04 9:51 UTC (permalink / raw) To: Eli Zaretskii; +Cc: spwhitton, rlb, monnier, david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > Maybe you should customize native-comp-async-jobs-number to a small > enough value, like 1? The default zero means half of your CPU's > execution units. I will try that and see how it goes, thanks. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-04 9:51 ` Po Lu @ 2022-10-05 13:58 ` Po Lu 2022-10-05 15:02 ` Lars Ingebrigtsen 2022-10-05 16:43 ` Eli Zaretskii 0 siblings, 2 replies; 343+ messages in thread From: Po Lu @ 2022-10-05 13:58 UTC (permalink / raw) To: Eli Zaretskii; +Cc: spwhitton, rlb, monnier, david, emacs-devel, akrl Po Lu <luangruo@yahoo.com> writes: > Eli Zaretskii <eliz@gnu.org> writes: > >> Maybe you should customize native-comp-async-jobs-number to a small >> enough value, like 1? The default zero means half of your CPU's >> execution units. > > I will try that and see how it goes, thanks. That makes the fans less loud (they are still noticable), but it also takes twice as long for the fans to subside, as expected. Hope this data point ends up meaningful. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 13:58 ` Po Lu @ 2022-10-05 15:02 ` Lars Ingebrigtsen 2022-10-05 16:47 ` Eli Zaretskii 2022-10-05 16:43 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-05 15:02 UTC (permalink / raw) To: Po Lu; +Cc: Eli Zaretskii, spwhitton, rlb, monnier, david, emacs-devel, akrl Po Lu <luangruo@yahoo.com> writes: > That makes the fans less loud (they are still noticable), but it also > takes twice as long for the fans to subside, as expected. > > Hope this data point ends up meaningful. Yes. Software is distributed pre-compiled for a reason -- because people run the software on hardware where compiling the software takes a long time. It's entirely reasonable for people to want to have a fully-built native-compiled Emacs installation on their laptops, without making that Emacs do any further JIT compilation. (Except where necessary for trampolines, of course.) This has been requested a number of times over several years now, but these requests have been ignored because apparently "people shouldn't want that". ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 15:02 ` Lars Ingebrigtsen @ 2022-10-05 16:47 ` Eli Zaretskii 2022-10-05 17:59 ` tomas 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-05 16:47 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: luangruo, spwhitton, rlb, monnier, david, emacs-devel, akrl > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Eli Zaretskii <eliz@gnu.org>, spwhitton@spwhitton.name, > rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, > emacs-devel@gnu.org, akrl@sdf.org > Date: Wed, 05 Oct 2022 17:02:45 +0200 > > Po Lu <luangruo@yahoo.com> writes: > > > That makes the fans less loud (they are still noticable), but it also > > takes twice as long for the fans to subside, as expected. > > > > Hope this data point ends up meaningful. > > Yes. > > Software is distributed pre-compiled for a reason -- because people run > the software on hardware where compiling the software takes a long time. > It's entirely reasonable for people to want to have a fully-built > native-compiled Emacs installation on their laptops, without making that > Emacs do any further JIT compilation. (Except where necessary for > trampolines, of course.) Why would people want to have N files compiled, but not the N+1st file? How are the first N files different from the N+1st? > This has been requested a number of times over several years now, but > these requests have been ignored because apparently "people shouldn't > want that". That's an incorrect and unfair accusation, so please stop. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 16:47 ` Eli Zaretskii @ 2022-10-05 17:59 ` tomas 2022-10-05 18:28 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: tomas @ 2022-10-05 17:59 UTC (permalink / raw) To: emacs-devel [-- Attachment #1: Type: text/plain, Size: 2557 bytes --] On Wed, Oct 05, 2022 at 07:47:58PM +0300, Eli Zaretskii wrote: > > From: Lars Ingebrigtsen <larsi@gnus.org> > > Cc: Eli Zaretskii <eliz@gnu.org>, spwhitton@spwhitton.name, > > rlb@defaultvalue.org, monnier@iro.umontreal.ca, david@tethera.net, > > emacs-devel@gnu.org, akrl@sdf.org > > Date: Wed, 05 Oct 2022 17:02:45 +0200 > > > > Po Lu <luangruo@yahoo.com> writes: > > > > > That makes the fans less loud (they are still noticable), but it also > > > takes twice as long for the fans to subside, as expected. > > > > > > Hope this data point ends up meaningful. > > > > Yes. > > > > Software is distributed pre-compiled for a reason -- because people run > > the software on hardware where compiling the software takes a long time. > > It's entirely reasonable for people to want to have a fully-built > > native-compiled Emacs installation on their laptops, without making that > > Emacs do any further JIT compilation. (Except where necessary for > > trampolines, of course.) > > Why would people want to have N files compiled, but not the N+1st > file? How are the first N files different from the N+1st? Perhaps because 1500 <= N <= 2000 (or so) and the N+1st ist just one? Perhaps because in the "normal case", the N+1st won't even happen? The idea of a Debian package is to provide a baseline for those just interested in using that software (with an easy ramp to upgrade to actually hack at the sources and build a modified version). I actually install a few packages from source, those I "personally" care about (Emacs is among them), But I couldn't possibly do it for the > 2000 currently installed on my system. The (Debian) packager's job is to make sure all that stuff works nicely (and as repeatably as possible) together, and still receive the necessary security fixes. Perhaps it would make sense for the Debian Emacs packagers (Rob?) to state their requirements in some more abstract way and work from there. As far as I understand, the wishes are: (a) deliver a package with (all? as many as possible? most? .eln pre-compiled (b) build Emacs in a way that is idempotent (and doesn't change overall system state (c) perhaps run tests (possibly, ideally part of b) Did I forget anything? Cheers -- tomás > > > This has been requested a number of times over several years now, but > > these requests have been ignored because apparently "people shouldn't > > want that". > > That's an incorrect and unfair accusation, so please stop. > > [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 17:59 ` tomas @ 2022-10-05 18:28 ` Eli Zaretskii 2022-10-05 18:56 ` tomas 2022-10-05 19:29 ` Gregory Heytings 0 siblings, 2 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-05 18:28 UTC (permalink / raw) To: tomas; +Cc: emacs-devel > Date: Wed, 5 Oct 2022 19:59:38 +0200 > From: <tomas@tuxteam.de> > > > Why would people want to have N files compiled, but not the N+1st > > file? How are the first N files different from the N+1st? > > Perhaps because 1500 <= N <= 2000 (or so) and the N+1st ist just one? Then it isn't the same N, is it? > Perhaps because in the "normal case", the N+1st won't even happen? Why disable something that will never happen? > The idea of a Debian package is to provide I think the Debian case is not relevant, because they provide all the *.eln files with the package you install (or so I understand). So either the user will only use those packages where *.eln are already provided (in which case there's no reason to disable anything), or they will want to use packages outside of the Debian distribution, in which case I'm asking why they would like to give up on compiling those additional ones. > I actually install a few packages from source, those I "personally" > care about (Emacs is among them), But I couldn't possibly do it for > the > 2000 currently installed on my system. What is "it" in "do it" above? And what does the 2000 number counts here? are you talking about 2000 Emacs Lisp packages that are not bundled with the Emacs distribution and need to be installed separately? Or are you counting some other kind of "packages"? > As far as I understand, the wishes are: > > (a) deliver a package with (all? as many as possible? most? .eln > pre-compiled > (b) build Emacs in a way that is idempotent (and doesn't change > overall system state > (c) perhaps run tests (possibly, ideally part of b) I don't see how disabling JIT compilation is needed for these 3 purposes. In particular, if all the *.eln files are already present, there will be no need to recompile them. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 18:28 ` Eli Zaretskii @ 2022-10-05 18:56 ` tomas 2022-10-05 19:04 ` Eli Zaretskii 2022-10-05 19:29 ` Gregory Heytings 1 sibling, 1 reply; 343+ messages in thread From: tomas @ 2022-10-05 18:56 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 3893 bytes --] On Wed, Oct 05, 2022 at 09:28:55PM +0300, Eli Zaretskii wrote: > > Date: Wed, 5 Oct 2022 19:59:38 +0200 > > From: <tomas@tuxteam.de> > > > > > Why would people want to have N files compiled, but not the N+1st > > > file? How are the first N files different from the N+1st? > > > > Perhaps because 1500 <= N <= 2000 (or so) and the N+1st ist just one? > > Then it isn't the same N, is it? - Base .el(c)s pre-compiled. N. - Additional .el (perhaps written by the user, perhaps downloaded from Emacswiki or what not) (jit) compiled, go to somewhere writable by the user (perhaps somewhere under ~/.emacs.d). 1. Or 2. > > Perhaps because in the "normal case", the N+1st won't even happen? > > Why disable something that will never happen? ...in the normal case. The user should be still capable of tackling the not-that-normal case. > > The idea of a Debian package is to provide > > I think the Debian case is not relevant, because they provide all the > *.eln files with the package you install (or so I understand). So > either the user will only use those packages where *.eln are already > provided (in which case there's no reason to disable anything), or > they will want to use packages outside of the Debian distribution, in > which case I'm asking why they would like to give up on compiling > those additional ones. It seems we are in violent agreement, then. Whatever the user installs "outside" the Debian packaging system is the user's business. It is desirable that it works well together with the Debian-provided baseline (and Debian tries to make that possible). > > I actually install a few packages from source, those I "personally" > > care about (Emacs is among them), But I couldn't possibly do it for > > the > 2000 currently installed on my system. > > What is "it" in "do it" above? Installing software from source. > And what does the 2000 number counts > here? Debian packages: The Gnu compiler toolchain. The mail reader (mutt in my case). A mail transfer agent (Exim). A Web browser. A web server. Lots of different scripting languages. Networking stuff. SSH, rsync, git. Qemu. Cross build tools. X and all those little helpers. R, Octave, I don't know, all that stuff one needs to be happy :-) > are you talking about 2000 Emacs Lisp packages that are not > bundled with the Emacs distribution and need to be installed > separately? Or are you counting some other kind of "packages"? No, Debian packages. > > As far as I understand, the wishes are: > > > > (a) deliver a package with (all? as many as possible? most? .eln > > pre-compiled > > (b) build Emacs in a way that is idempotent (and doesn't change > > overall system state > > (c) perhaps run tests (possibly, ideally part of b) > > I don't see how disabling JIT compilation is needed for these 3 > purposes. In particular, if all the *.eln files are already present, > there will be no need to recompile them. I don't know exactly how the Debian packaging for Emacs proceeds, but I think we are talking about two distinct situations here: (a) the binary install for the end user (for which the target is to end up with (some, most) .eln files pre-compiled and typically in /usr/lib or /usr/share, depending on whether those files are architecture-dependent (I think they are) or not (b) the package build, which happens at the Debian developer's box to build the Debian package to be installed to achieve (a). The question is whether the pre-compiling of the .eln happens at (a) (i.e. package install time) or at (b). It seems Rob has opted for the first (perhaps because there are dependencies which have to be resolved "late"?). Anyway, this "disabling of JIT" would be relevant for (b), if at all (assuming I've been paying attention). Cheers -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 18:56 ` tomas @ 2022-10-05 19:04 ` Eli Zaretskii 2022-10-05 19:40 ` tomas 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-05 19:04 UTC (permalink / raw) To: tomas; +Cc: emacs-devel > Date: Wed, 5 Oct 2022 20:56:00 +0200 > From: tomas@tuxteam.de > Cc: emacs-devel@gnu.org > > > > > Why would people want to have N files compiled, but not the N+1st > > > > file? How are the first N files different from the N+1st? > > > > > > Perhaps because 1500 <= N <= 2000 (or so) and the N+1st ist just one? > > > > Then it isn't the same N, is it? > > - Base .el(c)s pre-compiled. N. > - Additional .el (perhaps written by the user, perhaps > downloaded from Emacswiki or what not) (jit) compiled, > go to somewhere writable by the user (perhaps somewhere > under ~/.emacs.d). 1. Or 2. Then I ask again: why wouldn't the user want to have the addition .el compiled to .eln? > > > Perhaps because in the "normal case", the N+1st won't even happen? > > > > Why disable something that will never happen? > > ...in the normal case. The user should be still capable of tackling > the not-that-normal case. The not-that-normal case being user's own *.el files? Why wouldn't the user want to compile them into *.eln in that user's own eln-cache? Why would that user want to disable native-compilation instead? > > > I actually install a few packages from source, those I "personally" > > > care about (Emacs is among them), But I couldn't possibly do it for > > > the > 2000 currently installed on my system. > > > > What is "it" in "do it" above? > > Installing software from source. > > > And what does the 2000 number counts > > here? > > Debian packages: The Gnu compiler toolchain. The mail reader (mutt in my > case). A mail transfer agent (Exim). A Web browser. A web server. Lots > of different scripting languages. Networking stuff. SSH, rsync, git. > Qemu. Cross build tools. X and all those little helpers. R, Octave, I > don't know, all that stuff one needs to be happy :-) How is this relevant to the issue at hand? Only Emacs comes with *.eln files and can use them. Why are we talking about all the other packages? > > I don't see how disabling JIT compilation is needed for these 3 > > purposes. In particular, if all the *.eln files are already present, > > there will be no need to recompile them. > > I don't know exactly how the Debian packaging for Emacs proceeds, but > I think we are talking about two distinct situations here: > > (a) the binary install for the end user (for which the target is to > end up with (some, most) .eln files pre-compiled and typically > in /usr/lib or /usr/share, depending on whether those files are > architecture-dependent (I think they are) or not > > (b) the package build, which happens at the Debian developer's > box to build the Debian package to be installed to achieve > (a). > > The question is whether the pre-compiling of the .eln happens at > (a) (i.e. package install time) or at (b). It seems Rob has opted > for the first (perhaps because there are dependencies which have > to be resolved "late"?). > > Anyway, this "disabling of JIT" would be relevant for (b), if > at all (assuming I've been paying attention). For case (b) Rob said that the workspace is thrown away after the build, so whether the *.eln files are or aren't built there makes no difference. (And if the package build is done in batch mode, the async compilation is disabled there by default.) So, once again, I see no reason to disable JIT compilation for these use cases. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 19:04 ` Eli Zaretskii @ 2022-10-05 19:40 ` tomas 0 siblings, 0 replies; 343+ messages in thread From: tomas @ 2022-10-05 19:40 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 2824 bytes --] On Wed, Oct 05, 2022 at 10:04:17PM +0300, Eli Zaretskii wrote: > > Date: Wed, 5 Oct 2022 20:56:00 +0200 > > From: tomas@tuxteam.de > > Cc: emacs-devel@gnu.org > > > > > > > Why would people want to have N files compiled, but not the N+1st > > > > > file? How are the first N files different from the N+1st? > > > > > > > > Perhaps because 1500 <= N <= 2000 (or so) and the N+1st ist just one? > > > > > > Then it isn't the same N, is it? > > > > - Base .el(c)s pre-compiled. N. > > - Additional .el (perhaps written by the user, perhaps > > downloaded from Emacswiki or what not) (jit) compiled, > > go to somewhere writable by the user (perhaps somewhere > > under ~/.emacs.d). 1. Or 2. > > Then I ask again: why wouldn't the user want to have the addition .el > compiled to .eln? I think nobody is proposing to prevent the user from doing that. > > > > Perhaps because in the "normal case", the N+1st won't even happen? > > > > > > Why disable something that will never happen? > > > > ...in the normal case. The user should be still capable of tackling > > the not-that-normal case. > > The not-that-normal case being user's own *.el files? Why wouldn't > the user want to compile them into *.eln in that user's own eln-cache? > Why would that user want to disable native-compilation instead? See above. This must be (part of) the misunderstanding. See above my text, perhaps it was unclear: "Additional .el [...] (jit) compiled go to somewhere writable by the user" We are in violent agreement here, I think. > > > > I actually install a few packages from source, those I "personally" > > > > care about [...] > How is this relevant to the issue at hand? Only Emacs comes with > *.eln files and can use them. Why are we talking about all the other > packages? Just trying to explain how the Debian packaging works and what it is good for. Your questions seem to suggest that this is somewhat alien to you (but my impression might be wrong, too). > > > I don't see how disabling JIT compilation is needed for these 3 > > > purposes. In particular, if all the *.eln files are already present, > > > there will be no need to recompile them. [...] > For case (b) Rob said that the workspace is thrown away after the > build, so whether the *.eln files are or aren't built there makes no > difference. (And if the package build is done in batch mode, the > async compilation is disabled there by default.) As I understant, this might work well if it's possible to redirect/ restrict those target directories to specific places, yes. But Rob will know much better than I could. > So, once again, I see no reason to disable JIT compilation for these > use cases. Perhaps we can get away with it, yes. Cheers -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 18:28 ` Eli Zaretskii 2022-10-05 18:56 ` tomas @ 2022-10-05 19:29 ` Gregory Heytings 2022-10-05 19:38 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Gregory Heytings @ 2022-10-05 19:29 UTC (permalink / raw) To: Eli Zaretskii; +Cc: tomas, emacs-devel > > I think the Debian case is not relevant, because they provide all the > *.eln files with the package you install (or so I understand). > I'm not 100% sure what they will do (the version of Emacs distributed by Emacs is at the moment still 27.1), but my guess is that this is not what they will do. There are, in fact, two cases: 1. When a user does "apt install emacs", this actually installs (by default) the "emacs-gtk" package, which contains only the "emacs" binary, and triggers the installation of two other packages: "emacs-common", which contains the precompiled elc files (and the files in etc), and "emacs-bin-common", which contains the emacsclient, etags, ctags and ebrowse binaries. I would guess that in this case, when the user chooses to install an emacs with native compilation enabled (say "emacs-gtk-native"), a third package will be installed, say "emacs-common-native", containing the precompiled eln files. 2. When a user does "apt install elpa-magit" (for example), the package only contains el files. These files are compiled to elc files during installation (and stored in a shared directory, namely /usr/share/emacs/site-lisp). I would guess that, when the installed emacs binary is one with native compilation enabled, these el files will be compiled to eln files during installation, and stored in a shared directory, too. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 19:29 ` Gregory Heytings @ 2022-10-05 19:38 ` Eli Zaretskii 2022-10-05 20:04 ` Gregory Heytings 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-05 19:38 UTC (permalink / raw) To: Gregory Heytings; +Cc: tomas, emacs-devel > Date: Wed, 05 Oct 2022 19:29:45 +0000 > From: Gregory Heytings <gregory@heytings.org> > cc: tomas@tuxteam.de, emacs-devel@gnu.org > > > I think the Debian case is not relevant, because they provide all the > > *.eln files with the package you install (or so I understand). > > I'm not 100% sure what they will do (the version of Emacs distributed by > Emacs is at the moment still 27.1), but my guess is that this is not what > they will do. There are, in fact, two cases: > > 1. When a user does "apt install emacs", this actually installs (by > default) the "emacs-gtk" package, which contains only the "emacs" binary, > and triggers the installation of two other packages: "emacs-common", which > contains the precompiled elc files (and the files in etc), and > "emacs-bin-common", which contains the emacsclient, etags, ctags and > ebrowse binaries. I would guess that in this case, when the user chooses > to install an emacs with native compilation enabled (say > "emacs-gtk-native"), a third package will be installed, say > "emacs-common-native", containing the precompiled eln files. > > 2. When a user does "apt install elpa-magit" (for example), the package > only contains el files. These files are compiled to elc files during > installation (and stored in a shared directory, namely > /usr/share/emacs/site-lisp). I would guess that, when the installed emacs > binary is one with native compilation enabled, these el files will be > compiled to eln files during installation, and stored in a shared > directory, too. I agree with the above. But then why did you say that my description was inaccurate? What you described matches what I wrote perfectly: the end result, after installing Emacs and elpa-magit, is that the *.eln files are available for all the *.el files and stored in shared directories. Whether those *.eln files are produced on the system where the package is packaged or as part of the installation is not important; the important part is that all the *.eln files are there after the installation, and therefore there's no need to disable JIT compilation. And yet Rob says that he thinks there _is_ a need for disabling it. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 19:38 ` Eli Zaretskii @ 2022-10-05 20:04 ` Gregory Heytings 2022-10-06 5:28 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Gregory Heytings @ 2022-10-05 20:04 UTC (permalink / raw) To: Eli Zaretskii; +Cc: tomas, emacs-devel > > I agree with the above. But then why did you say that my description > was inaccurate? > I did not say that, or at least I did not mean to say that. I only tried to clarify something that seemed unclear, namely that eln files are not present in the package, but generated at installation time. It's a subtle difference, but it seems important in the current discussion. > > What you described matches what I wrote perfectly: the end result, after > installing Emacs and elpa-magit, is that the *.eln files are available > for all the *.el files and stored in shared directories. Whether those > *.eln files are produced on the system where the package is packaged or > as part of the installation is not important; the important part is that > all the *.eln files are there after the installation, and therefore > there's no need to disable JIT compilation. And yet Rob says that he > thinks there _is_ a need for disabling it. > If I understand Rob's initial message correctly, this subtle difference is relevant: > > Perhaps relevant in other contexts, but at least in the context of > Debian work, we'd like to be able to suppress all native compilation in > some contexts, or more specifically all writes to HOME (or really, to > avoid any implicit file manipulations[1]). > > One key case is package builds and package installs (whether the emacs > packages themselves, or any of the various emacs add-on packages (e.g. > elpa-*)). > > For package builds, HOME is typically set to /nonexistent (or similar), > and for package installs we don't want the install commands (preinst, > postinst, etc.), which are running as root, to write anything to /root/. > IIUC, what Rob wants is that (1) on Debian systems on which Emacs packages are built, the build process is running as root, and nothing should be written in /root, and (2) on user systems on which elpa-* packages are installed, the installation process, during which el files are compiled to elc and eln files, is running as root, and it should not write anything in /root. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 20:04 ` Gregory Heytings @ 2022-10-06 5:28 ` Eli Zaretskii 2022-10-06 6:35 ` tomas 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-06 5:28 UTC (permalink / raw) To: Gregory Heytings; +Cc: tomas, emacs-devel > Date: Wed, 05 Oct 2022 20:04:34 +0000 > From: Gregory Heytings <gregory@heytings.org> > cc: tomas@tuxteam.de, emacs-devel@gnu.org > > IIUC, what Rob wants is that > > (1) on Debian systems on which Emacs packages are built, the build process > is running as root, and nothing should be written in /root, and For this case, Rob said the workspace is thrown away after the build, so whether the *.eln files are or aren't produced is not relevant. But if, for some reason, the *.eln files could survive the deletion of the workspace, then tweaking native-comp-eln-load-path to have /tmp at the beginning should handle that as well. > (2) on user systems on which elpa-* packages are installed, the > installation process, during which el files are compiled to elc and eln > files, is running as root, and it should not write anything in /root. Since the elpa-* package installation process is supposed to leave the users of the system with ready *.eln files from the packages being installed, the installation process should NOT disable native-compilation. Instead, it should tweak native-comp-eln-load-path so that it includes the shared directory where Debian wants to have the *.eln files of ELPA packages instead or ahead of the user's eln-cache directory. This will produce the *.eln files in the place where Debian wants them, and allow users to use those packages without worrying about compilation. IOW, an option to disable native-compilation is not TRT in this case. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 5:28 ` Eli Zaretskii @ 2022-10-06 6:35 ` tomas 0 siblings, 0 replies; 343+ messages in thread From: tomas @ 2022-10-06 6:35 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Gregory Heytings, emacs-devel [-- Attachment #1: Type: text/plain, Size: 185 bytes --] On Thu, Oct 06, 2022 at 08:28:04AM +0300, Eli Zaretskii wrote: [...] > IOW, an option to disable native-compilation is not TRT in this case. s/TRT/necessary/ ;-) -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 13:58 ` Po Lu 2022-10-05 15:02 ` Lars Ingebrigtsen @ 2022-10-05 16:43 ` Eli Zaretskii 2022-10-06 0:34 ` Po Lu 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-05 16:43 UTC (permalink / raw) To: Po Lu; +Cc: spwhitton, rlb, monnier, david, emacs-devel, akrl > From: Po Lu <luangruo@yahoo.com> > Cc: spwhitton@spwhitton.name, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, > akrl@sdf.org > Date: Wed, 05 Oct 2022 21:58:02 +0800 > > Po Lu <luangruo@yahoo.com> writes: > > > Eli Zaretskii <eliz@gnu.org> writes: > > > >> Maybe you should customize native-comp-async-jobs-number to a small > >> enough value, like 1? The default zero means half of your CPU's > >> execution units. > > > > I will try that and see how it goes, thanks. > > That makes the fans less loud (they are still noticable), but it also > takes twice as long for the fans to subside, as expected. What happens with those fans when you build Emacs with "make -jN"? do they sound the same or louder? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 16:43 ` Eli Zaretskii @ 2022-10-06 0:34 ` Po Lu 2022-10-06 6:21 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Po Lu @ 2022-10-06 0:34 UTC (permalink / raw) To: Eli Zaretskii; +Cc: spwhitton, rlb, monnier, david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > What happens with those fans when you build Emacs with "make -jN"? do > they sound the same or louder? Louder, but building Emacs doesn't happen in the background automatically. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 0:34 ` Po Lu @ 2022-10-06 6:21 ` Eli Zaretskii 2022-10-06 7:11 ` Po Lu 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-06 6:21 UTC (permalink / raw) To: Po Lu; +Cc: spwhitton, rlb, monnier, david, emacs-devel, akrl > From: Po Lu <luangruo@yahoo.com> > Cc: spwhitton@spwhitton.name, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, > akrl@sdf.org > Date: Thu, 06 Oct 2022 08:34:03 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > > What happens with those fans when you build Emacs with "make -jN"? do > > they sound the same or louder? > > Louder, but building Emacs doesn't happen in the background > automatically. It does happen in the background, in the sense that Make launches several processes in parallel, each one of which running in the background. As for "automatically", the JIT compilation is not automatic, either. If Emacs is idle, it will not suddenly decide to compile. Anyway, what would you suggest as a solution for the problem you perceive with JIT native-compilation, which would refrain from being "in the background" and "automatic"? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 6:21 ` Eli Zaretskii @ 2022-10-06 7:11 ` Po Lu 2022-10-06 8:03 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Po Lu @ 2022-10-06 7:11 UTC (permalink / raw) To: Eli Zaretskii; +Cc: spwhitton, rlb, monnier, david, emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > Anyway, what would you suggest as a solution for the problem you > perceive with JIT native-compilation, which would refrain from being > "in the background" and "automatic"? The solution I would propose would be to defer JIT native-compilation until the computer is on AC power, as determined by battery.el. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 7:11 ` Po Lu @ 2022-10-06 8:03 ` Eli Zaretskii 2022-10-06 9:02 ` tomas 2022-10-06 9:52 ` Po Lu 0 siblings, 2 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-06 8:03 UTC (permalink / raw) To: Po Lu; +Cc: emacs-devel, akrl > From: Po Lu <luangruo@yahoo.com> > Cc: spwhitton@spwhitton.name, rlb@defaultvalue.org, > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, > akrl@sdf.org > Date: Thu, 06 Oct 2022 15:11:19 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > > Anyway, what would you suggest as a solution for the problem you > > perceive with JIT native-compilation, which would refrain from being > > "in the background" and "automatic"? > > The solution I would propose would be to defer JIT native-compilation > until the computer is on AC power, as determined by battery.el. We could have such a feature, but how to implement it? If we use a timer for that, the timer itself will drain the battery. And if defer it to the next invocation of the bytecode, we might never compile, because who can guarantee that the laptop is on AC when some arbitrary bytecode is executed? Any other ideas? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 8:03 ` Eli Zaretskii @ 2022-10-06 9:02 ` tomas 2022-10-06 10:13 ` Eli Zaretskii 2022-10-06 9:52 ` Po Lu 1 sibling, 1 reply; 343+ messages in thread From: tomas @ 2022-10-06 9:02 UTC (permalink / raw) To: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1059 bytes --] On Thu, Oct 06, 2022 at 11:03:47AM +0300, Eli Zaretskii wrote: > > From: Po Lu <luangruo@yahoo.com> > > Cc: spwhitton@spwhitton.name, rlb@defaultvalue.org, > > monnier@iro.umontreal.ca, david@tethera.net, emacs-devel@gnu.org, > > akrl@sdf.org > > Date: Thu, 06 Oct 2022 15:11:19 +0800 > > > > Eli Zaretskii <eliz@gnu.org> writes: > > > > > Anyway, what would you suggest as a solution for the problem you > > > perceive with JIT native-compilation, which would refrain from being > > > "in the background" and "automatic"? > > > > The solution I would propose would be to defer JIT native-compilation > > until the computer is on AC power, as determined by battery.el. > > We could have such a feature, but how to implement it? If we use a > timer for that, the timer itself will drain the battery. Guessing from a previous mail by you, the correct way to achieve this is to compiler results to a non-existent directory, right? To me that feels strange, but I could live with that. Can it be documented? Cheers -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 9:02 ` tomas @ 2022-10-06 10:13 ` Eli Zaretskii 2022-10-06 11:49 ` tomas 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-06 10:13 UTC (permalink / raw) To: tomas; +Cc: emacs-devel > Date: Thu, 6 Oct 2022 11:02:44 +0200 > From: <tomas@tuxteam.de> > > > > The solution I would propose would be to defer JIT native-compilation > > > until the computer is on AC power, as determined by battery.el. > > > > We could have such a feature, but how to implement it? If we use a > > timer for that, the timer itself will drain the battery. > > Guessing from a previous mail by you, the correct way to achieve this > is to compiler results to a non-existent directory, right? No, this is a completely different use case. We _can_ disable native-compilation (and had this ability for a while, since Emacs 28 development); the issue here is how to re-enable it again "when computer is on AC power". > To me that feels strange, but I could live with that. Can it be documented? What would you like to be documented, exactly? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 10:13 ` Eli Zaretskii @ 2022-10-06 11:49 ` tomas 2022-10-07 12:40 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: tomas @ 2022-10-06 11:49 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1293 bytes --] On Thu, Oct 06, 2022 at 01:13:40PM +0300, Eli Zaretskii wrote: > > Date: Thu, 6 Oct 2022 11:02:44 +0200 > > From: <tomas@tuxteam.de> > > > > > > The solution I would propose would be to defer JIT native-compilation > > > > until the computer is on AC power, as determined by battery.el. > > > > > > We could have such a feature, but how to implement it? If we use a > > > timer for that, the timer itself will drain the battery. > > > > Guessing from a previous mail by you, the correct way to achieve this > > is to compiler results to a non-existent directory, right? > > No, this is a completely different use case. Now I'm confused. Does the non-existence of a target directory disable compilation or just the writing of the compilation's results? > We _can_ disable native-compilation (and had this ability for a while, > since Emacs 28 development); the issue here is how to re-enable it > again "when computer is on AC power". > > > To me that feels strange, but I could live with that. Can it be documented? > > What would you like to be documented, exactly? that pointing HOME a non-existing directory is the way to either inhibit JIT compilation or writing of the compilation results to disk (depending on the answer above). Cheers -- t [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 11:49 ` tomas @ 2022-10-07 12:40 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-07 12:40 UTC (permalink / raw) To: tomas; +Cc: emacs-devel > Date: Thu, 6 Oct 2022 13:49:44 +0200 > From: tomas@tuxteam.de > Cc: emacs-devel@gnu.org > > On Thu, Oct 06, 2022 at 01:13:40PM +0300, Eli Zaretskii wrote: > > > Date: Thu, 6 Oct 2022 11:02:44 +0200 > > > From: <tomas@tuxteam.de> > > > > > > > > The solution I would propose would be to defer JIT native-compilation > > > > > until the computer is on AC power, as determined by battery.el. > > > > > > > > We could have such a feature, but how to implement it? If we use a > > > > timer for that, the timer itself will drain the battery. > > > > > > Guessing from a previous mail by you, the correct way to achieve this > > > is to compiler results to a non-existent directory, right? > > > > No, this is a completely different use case. > > Now I'm confused. Does the non-existence of a target directory > disable compilation or just the writing of the compilation's > results? ??? What does the existence of the target directory have to do with the use case of laptop running on batteries? > > What would you like to be documented, exactly? > > that pointing HOME a non-existing directory is the way to either > inhibit JIT compilation or writing of the compilation results to > disk (depending on the answer above). That is not something we should advertise, I think, because it shouldn't be needed. We do that in the test suite, but not in order to disable native-compilation. Whether there are valid use cases where users would need to disable native-compilation is still an open question. The use case presented by Po Lu does not require disabling native-compilation, it requires _delaying_ it until some opportune moment. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 8:03 ` Eli Zaretskii 2022-10-06 9:02 ` tomas @ 2022-10-06 9:52 ` Po Lu 2022-10-06 10:17 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Po Lu @ 2022-10-06 9:52 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel, akrl Eli Zaretskii <eliz@gnu.org> writes: > We could have such a feature, but how to implement it? If we use a > timer for that, the timer itself will drain the battery. I think display-battery-mode users (I am one such user) will not agree with that assessment. > And if defer it to the next invocation of the bytecode, we might never > compile, because who can guarantee that the laptop is on AC when some > arbitrary bytecode is executed? We could push it onto a list of files to native compile, the files in which are then compiled once we detect the laptop starts to run on AC power. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 9:52 ` Po Lu @ 2022-10-06 10:17 ` Eli Zaretskii 2022-10-06 10:41 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-06 10:17 UTC (permalink / raw) To: Po Lu; +Cc: emacs-devel, akrl > From: Po Lu <luangruo@yahoo.com> > Cc: emacs-devel@gnu.org, akrl@sdf.org > Date: Thu, 06 Oct 2022 17:52:35 +0800 > > Eli Zaretskii <eliz@gnu.org> writes: > > > We could have such a feature, but how to implement it? If we use a > > timer for that, the timer itself will drain the battery. > > I think display-battery-mode users (I am one such user) will not agree > with that assessment. I didn't invent that, I've heard laptop users complain about timers. But okay, if it's acceptable to run a timer in order to re-enable compilation when AC power is plugged in, I'm okay with such an optional feature. Patches are welcome. > > And if defer it to the next invocation of the bytecode, we might never > > compile, because who can guarantee that the laptop is on AC when some > > arbitrary bytecode is executed? > > We could push it onto a list of files to native compile, the files in > which are then compiled once we detect the laptop starts to run on AC > power. You will see that comp.el already has a queue of files that await compilation. The new feature will newed to plug itself into that mechanism, I think. Unless Andrea has a better idea. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 10:17 ` Eli Zaretskii @ 2022-10-06 10:41 ` Andrea Corallo 2022-10-06 10:54 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-06 10:41 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Po Lu, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Po Lu <luangruo@yahoo.com> >> Cc: emacs-devel@gnu.org, akrl@sdf.org >> Date: Thu, 06 Oct 2022 17:52:35 +0800 >> >> Eli Zaretskii <eliz@gnu.org> writes: >> >> > We could have such a feature, but how to implement it? If we use a >> > timer for that, the timer itself will drain the battery. >> >> I think display-battery-mode users (I am one such user) will not agree >> with that assessment. > > I didn't invent that, I've heard laptop users complain about timers. For my education: why timer are computational expensive? > But okay, if it's acceptable to run a timer in order to re-enable > compilation when AC power is plugged in, I'm okay with such an > optional feature. Patches are welcome. > >> > And if defer it to the next invocation of the bytecode, we might never >> > compile, because who can guarantee that the laptop is on AC when some >> > arbitrary bytecode is executed? >> >> We could push it onto a list of files to native compile, the files in >> which are then compiled once we detect the laptop starts to run on AC >> power. > > You will see that comp.el already has a queue of files that await > compilation. The new feature will newed to plug itself into that > mechanism, I think. Unless Andrea has a better idea. I'd suggest this way as well. BR Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-06 10:41 ` Andrea Corallo @ 2022-10-06 10:54 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-06 10:54 UTC (permalink / raw) To: Andrea Corallo; +Cc: luangruo, emacs-devel > From: Andrea Corallo <akrl@sdf.org> > Cc: Po Lu <luangruo@yahoo.com>, emacs-devel@gnu.org > Date: Thu, 06 Oct 2022 10:41:14 +0000 > > >> I think display-battery-mode users (I am one such user) will not agree > >> with that assessment. > > > > I didn't invent that, I've heard laptop users complain about timers. > > For my education: why timer are computational expensive? They are not, at least not normally. But they interfere with the OS's attempts to conserve power, because Emacs is not idle, as far as the OS is concerned -- it runs some stuff once every so-and-so seconds. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-04 0:31 ` Po Lu 2022-10-04 6:57 ` Eli Zaretskii @ 2022-10-04 23:26 ` Sean Whitton 1 sibling, 0 replies; 343+ messages in thread From: Sean Whitton @ 2022-10-04 23:26 UTC (permalink / raw) To: Po Lu; +Cc: Eli Zaretskii, Rob Browning, monnier, david, emacs-devel, akrl Hello, On Tue 04 Oct 2022 at 08:31AM +08, Po Lu wrote: > Sean Whitton <spwhitton@spwhitton.name> writes: > >> The point with battery consumption is not about running vs. standby. >> The issue is that while users expect that running apt-get will drain the >> battery, they expect that once apt-get is done, the only battery-hungry >> processes are ones they start themselves. Laptop users typically avoid >> running apt-get without mains power plugged in. If I do an 'apt-get >> upgrade' then afterwards my CPU will be churning away compiling addon >> packages, so I can't just unplug. > > I don't know if that's true with Debian users, but nobody I know of > objects to running "dnf install" on battery. After all, installing new > packages only takes a short amount of time, certainly not enough to > significantly affect battery usage. The issue is Debian Emacs addon packages, in particular. Upgrading those kicks off a whole lot of bytecompilation. It's much heavier-weight than usual package upgrades. -- Sean Whitton ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-30 13:56 ` Eli Zaretskii ` (2 preceding siblings ...) 2022-09-30 15:32 ` Stefan Monnier @ 2022-10-02 5:58 ` tomas 2022-10-02 6:42 ` Eli Zaretskii 3 siblings, 1 reply; 343+ messages in thread From: tomas @ 2022-10-02 5:58 UTC (permalink / raw) To: emacs-devel [-- Attachment #1: Type: text/plain, Size: 3188 bytes --] On Fri, Sep 30, 2022 at 04:56:56PM +0300, Eli Zaretskii wrote: > > From: David Bremner <david@tethera.net> [...] > > Some additional byte compilation happens at package install time. > > This is what I'm asking about: what exactly triggers the compilation? > Just installing a package shouldn't do that, only loading it into > Emacs should. I see there two views: the distributor's (represented here by David's) and the jit's (Eli). For Eli, the file system objects are just a long term cache of the jit's work (longer than one Emacs session), for David, since the package's .el (and possibly .elc) are immutable (and of course, the whole Emacs runtime), the corresponding .eln can well be built at package install time. That would avoid each user to have an identical copy of them (and of course, the initial wait). That does make a lot of sense in a multi-user system (see below for more on that). > The other part of what I asked is that if for some reason installation > does need to compile files (something that I still need to understand > why it happens), there's nothing that hard-codes HOME in the directory > list used for that. You can set native-comp-eln-load-path to anything > you want, and only the directories in that list will be used. Yes, but. It's not about HOME, but about a user-writable space. A shared space for .eln would have to be writable by all users if the compilation is to succeed when invoked on behalf of each one. This is a no-no under a multiuser system. Install is invoked as root, so it can put the results of its compilation to a place readable by /all/ users (typically some /usr/share or /usr/lib, depending on whether the results are architecture-independent). > > In my > > view the same restrictions should apply there. In addition to the > > unintentional sharing of state mentioned in policy, there is also the > > issue of cleaning up after a package on uninstall. This is manageable > > now because any generated .elc files go in a package specific directory, > > which can just be removed on purging the package. > > See above: you can do the same with *.eln files, if, for some reason, > they are produced by the installation. > > > I think just turning off native compilation when HOME is not writeable > > would be sensible from a Debian point of view. Emacs did not previously > > require a writable HOME (although maybe most users never tested > > this). It would be unfortunate if this changed for Emacs with native > > compilation support. > > Emacs doesn't require a writable HOME, it requires a writable > directory to store *.eln files. It doesn't have to be HOME. And once > again, I still don't understand why *.eln files are produced at > installation time in the first place. That's all fine, but then users wouldn't profit from the pre-compiled .eln. In a Debian-distributed Emacs (caveat: I'm compiling my own one!) there are .elc in /usr/share for all to use; due to the search path, a user still can install her own in a directory writable by that user and it will take precedence. Can you do the same for .elc? Cheers -- t > [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 5:58 ` tomas @ 2022-10-02 6:42 ` Eli Zaretskii 2022-10-02 15:53 ` tomas 2022-10-02 18:32 ` Rob Browning 0 siblings, 2 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 6:42 UTC (permalink / raw) To: tomas; +Cc: emacs-devel > Date: Sun, 2 Oct 2022 07:58:33 +0200 > From: <tomas@tuxteam.de> > > I see there two views: the distributor's (represented here by David's) > and the jit's (Eli). For Eli, the file system objects are just a long > term cache of the jit's work (longer than one Emacs session), for > David, since the package's .el (and possibly .elc) are immutable (and > of course, the whole Emacs runtime), the corresponding .eln can well > be built at package install time. That would avoid each user to have > an identical copy of them As explained in my other message, I see no reason to make such assumptions. There's no reason to believe that a user who installs a package will immediately start using it. And even it they do, why should we care about some extra disk space or identical files being present? disk space is cheap, while the flexibility this allows (like each user can have a slightly differently-configured Emacs) is non-negligible. > (and of course, the initial wait). There's no wait: a package can be used immediately after loading or requiring it. The replacement of a byte-compiled by natively-compiled code happens transparently and doesn't affect usage nor causes any waits. The advantage of using JIT compilation is that this is how the upstream project meant it to be used, and this is how the defaults are set. Any deviation from the defaults should have a good reason, and I submit that such good reasons can only be based on actual usage, not on theoretical presumptions. My recommendation is to use the default JIT manner until and unless actual problems are reported by users. > That does make a lot of sense in a multi-user system Not to me, it doesn't. It makes _some_ sense, but there are other considerations. > > The other part of what I asked is that if for some reason installation > > does need to compile files (something that I still need to understand > > why it happens), there's nothing that hard-codes HOME in the directory > > list used for that. You can set native-comp-eln-load-path to anything > > you want, and only the directories in that list will be used. > > Yes, but. It's not about HOME, but about a user-writable space. A shared > space for .eln would have to be writable by all users if the compilation > is to succeed when invoked on behalf of each one. This is a no-no under > a multiuser system. ??? What "user-writable space"? If we are talking about install-time compilation, then the *.eln files should be deposited in some centralized place to which the installation process can write. And if we are talking about JIT compilation when the user runs Emacs, then the home directory is perfectly appropriate, and is writable. No one suggested that JIT compilations from "normal" user session should write to shared directories (and if someone did suggest that, IMO this suggestion makes no sense). So I think you confused these two possibilities, because the problem you raise doesn't exist. > Install is invoked as root, so it can put the results of its compilation > to a place readable by /all/ users (typically some /usr/share or /usr/lib, > depending on whether the results are architecture-independent). Exactly. So what is the problem with directories writable by all users? > > Emacs doesn't require a writable HOME, it requires a writable > > directory to store *.eln files. It doesn't have to be HOME. And once > > again, I still don't understand why *.eln files are produced at > > installation time in the first place. > > That's all fine, but then users wouldn't profit from the pre-compiled > .eln. There's no profit, IME. There are only disadvantages: you are in effect fighting against the Emacs defaults, for reasons that are purely theoretical. > In a Debian-distributed Emacs (caveat: I'm compiling my own one!) > there are .elc in /usr/share for all to use; due to the search path, > a user still can install her own in a directory writable by that > user and it will take precedence. > > Can you do the same for .elc? I guess you meant "with .eln files"? Yes, see native-comp-eln-load-path, which was already mentioned here several times. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 6:42 ` Eli Zaretskii @ 2022-10-02 15:53 ` tomas 2022-10-02 16:11 ` Eli Zaretskii 2022-10-02 18:32 ` Rob Browning 1 sibling, 1 reply; 343+ messages in thread From: tomas @ 2022-10-02 15:53 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 3484 bytes --] On Sun, Oct 02, 2022 at 09:42:04AM +0300, Eli Zaretskii wrote: > > Date: Sun, 2 Oct 2022 07:58:33 +0200 > > From: <tomas@tuxteam.de> > > > > I see there two views: the distributor's (represented here by David's) > > and the jit's (Eli) [...] > As explained in my other message, I see no reason to make such > assumptions. There's no reason to believe that a user who installs a > package will immediately start using it [...] (NOTE Eli, I admire your patience :-) As I see from your other post, there is a confusion potential with the word "package". I was talking about Debian binary packages, which are thought for those who want to use them. For those wanting to change or build there are the source packages (which come with a convenient build "kit" enabling you to build the binary package as it comes with Debian, which is a convenient starting point for tinkering). > And even it they do, why > should we care about some extra disk space or identical files being > present? disk space is cheap, while the flexibility this allows (like > each user can have a slightly differently-configured Emacs) is > non-negligible. Definitely. I am not that much concerned about disk usage, more about a uniform "baseline" installation all users can rely on. [...] > The advantage of using JIT compilation is that this is how the > upstream project meant it to be used, and this is how the defaults are > set. Any deviation from the defaults should have a good reason, and I > submit that such good reasons can only be based on actual usage, not > on theoretical presumptions. My recommendation is to use the default > JIT manner until and unless actual problems are reported by users. I see that, and this is the one view I mention above: the .eln are but a JIT cache, and each user (or instance, if there are more than one per user) has its own. > > That does make a lot of sense in a multi-user system > > Not to me, it doesn't. It makes _some_ sense, but there are other > considerations. Two views, as I said :) [HOME, user-writable space] > Exactly. So what is the problem with directories writable by all > users? User separation goes out of the window, and this is one important service the OS provides. To illustrate, one user could put malicious .eln files all other users would execute. > > > Emacs doesn't require a writable HOME, it requires a writable > > > directory to store *.eln files. It doesn't have to be HOME. And once > > > again, I still don't understand why *.eln files are produced at > > > installation time in the first place. > > > > That's all fine, but then users wouldn't profit from the pre-compiled > > .eln. > > There's no profit, IME. There are only disadvantages: you are in > effect fighting against the Emacs defaults, for reasons that are > purely theoretical. I have the impression that some of that reasons are quite practical for Debian packagers. > > In a Debian-distributed Emacs (caveat: I'm compiling my own one!) > > there are .elc in /usr/share for all to use; due to the search path, > > a user still can install her own in a directory writable by that > > user and it will take precedence. > > > > Can you do the same for .elc? > > I guess you meant "with .eln files"? Yes, see > native-comp-eln-load-path, which was already mentioned here several > times. So that might be one part of the way out. Cheers -- tomás [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 15:53 ` tomas @ 2022-10-02 16:11 ` Eli Zaretskii 2022-10-02 16:23 ` chad ` (2 more replies) 0 siblings, 3 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 16:11 UTC (permalink / raw) To: tomas; +Cc: emacs-devel > Date: Sun, 2 Oct 2022 17:53:46 +0200 > From: tomas@tuxteam.de > Cc: emacs-devel@gnu.org > > > The advantage of using JIT compilation is that this is how the > > upstream project meant it to be used, and this is how the defaults are > > set. Any deviation from the defaults should have a good reason, and I > > submit that such good reasons can only be based on actual usage, not > > on theoretical presumptions. My recommendation is to use the default > > JIT manner until and unless actual problems are reported by users. > > I see that, and this is the one view I mention above: the .eln are but > a JIT cache, and each user (or instance, if there are more than one > per user) has its own. Let me be blunt: this is currently _the_only_ view of the Emacs project. After a lot of deliberations, we didn't find any reasons for alternative views. My suggestion is to try our view before concluding that it doesn't fit some situations. > > Exactly. So what is the problem with directories writable by all > > users? > > User separation goes out of the window, and this is one important > service the OS provides. To illustrate, one user could put malicious > .eln files all other users would execute. This is about installation writing files into a shared space on disk, right? If so, this is something for the Debian packagers to figure out, because doing that is their request. And anyway, I don't understand how do *.eln files are different from *.elc files, which are already written to shared disk space upon installation. What am I missing? > > > That's all fine, but then users wouldn't profit from the pre-compiled > > > .eln. > > > > There's no profit, IME. There are only disadvantages: you are in > > effect fighting against the Emacs defaults, for reasons that are > > purely theoretical. > > I have the impression that some of that reasons are quite practical > for Debian packagers. I submit that those reasons were most probably derived from a broken analogy with the *.elc files and with byte-compilation in general. Not from actual usage experience. Native compilation looks deceptively similar to byte compilation, but it isn't. So if producing the *.eln files seems to contradict some Debian rules and procedures, my suggestion is to talk to the upstream project, before inventing solutions, because of 2 considerations: . the problems may not be real ones, only perceived ones . the upstream codebase might already provide solutions > > > In a Debian-distributed Emacs (caveat: I'm compiling my own one!) > > > there are .elc in /usr/share for all to use; due to the search path, > > > a user still can install her own in a directory writable by that > > > user and it will take precedence. > > > > > > Can you do the same for .elc? > > > > I guess you meant "with .eln files"? Yes, see > > native-comp-eln-load-path, which was already mentioned here several > > times. > > So that might be one part of the way out. If one needs it. I don't think they do, and I don't recommend going there. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:11 ` Eli Zaretskii @ 2022-10-02 16:23 ` chad 2022-10-02 16:59 ` Eli Zaretskii 2022-10-02 17:57 ` Rob Browning 2022-10-02 16:27 ` tomas 2022-10-02 23:58 ` Sean Whitton 2 siblings, 2 replies; 343+ messages in thread From: chad @ 2022-10-02 16:23 UTC (permalink / raw) To: Eli Zaretskii; +Cc: tomas, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1761 bytes --] On Sun, Oct 2, 2022 at 12:13 PM Eli Zaretskii <eliz@gnu.org> wrote: > > User separation goes out of the window, and this is one important > > service the OS provides. To illustrate, one user could put malicious > > .eln files all other users would execute. > > This is about installation writing files into a shared space on disk, > right? If so, this is something for the Debian packagers to figure > out, because doing that is their request. And anyway, I don't > understand how do *.eln files are different from *.elc files, which > are already written to shared disk space upon installation. What am I > missing? > At the risk of over-explaining due to message crossing mid-flight: the thing you might be missing is that Debian provides a mechanism for people to install *.elc files in a space shared by all users that is not writable by those users, and there are people that use this provision. Since these are used for *.elc files, it seems highly likely that they will be desired for *.eln files. Even in the face of a theoretical issue like "potential package combinations make it unworkable to pre-build a single set of emacs+emacs-packages", the practical situation is that such combinations exist and are used by Debian users to build a stable base of emacs+emacs-packages that is shared across users who cannot change that shared base (but can, of course, supplement it with their own packages, via site-lisp, user packages, etc.) As a practical goal, there is at least *some* impetus for Debian to provide such a stable base, and to make it as wide as reasonable. The basic mechanism for determining the size of that base is "which emacs-packages are made into debian-packages", (iff I understand correctly; I'm not a Debian expert). ~Chad [-- Attachment #2: Type: text/html, Size: 2215 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:23 ` chad @ 2022-10-02 16:59 ` Eli Zaretskii 2022-10-02 18:35 ` Rob Browning 2022-10-02 17:57 ` Rob Browning 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 16:59 UTC (permalink / raw) To: chad; +Cc: tomas, emacs-devel > From: chad <yandros@gmail.com> > Date: Sun, 2 Oct 2022 12:23:43 -0400 > Cc: tomas@tuxteam.de, emacs-devel@gnu.org > > > User separation goes out of the window, and this is one important > > service the OS provides. To illustrate, one user could put malicious > > .eln files all other users would execute. > > This is about installation writing files into a shared space on disk, > right? If so, this is something for the Debian packagers to figure > out, because doing that is their request. And anyway, I don't > understand how do *.eln files are different from *.elc files, which > are already written to shared disk space upon installation. What am I > missing? > > At the risk of over-explaining due to message crossing mid-flight: the thing you might be missing is that > Debian provides a mechanism for people to install *.elc files in a space shared by all users that is not > writable by those users, and there are people that use this provision. Since these are used for *.elc files, it > seems highly likely that they will be desired for *.eln files. No, I don't think the similar handling makes sense here. The *.elc files are architecture- and configuration-independent, whereas the *.eln files are not. E.g., the same foo.elc could be used by user A who runs Emacs 28 and by user B who runs Emacs 29. But the corresponding *.eln files will be different, even though they were produced from the same foo.el. > Even in the face of a theoretical issue like "potential package combinations make it unworkable to pre-build a > single set of emacs+emacs-packages", the practical situation is that such combinations exist and are used > by Debian users to build a stable base of emacs+emacs-packages that is shared across users who > cannot change that shared base (but can, of course, supplement it with their own packages, via site-lisp, > user packages, etc.) As a practical goal, there is at least *some* impetus for Debian to provide such a stable > base, and to make it as wide as reasonable. The basic mechanism for determining the size of that base is > "which emacs-packages are made into debian-packages", (iff I understand correctly; I'm not a Debian > expert). I don't think this is relevant to the issue at hand. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:59 ` Eli Zaretskii @ 2022-10-02 18:35 ` Rob Browning 2022-10-02 18:54 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-02 18:35 UTC (permalink / raw) To: Eli Zaretskii, chad; +Cc: tomas, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > No, I don't think the similar handling makes sense here. The *.elc > files are architecture- and configuration-independent, whereas the > *.eln files are not. E.g., the same foo.elc could be used by user A > who runs Emacs 28 and by user B who runs Emacs 29. But the > corresponding *.eln files will be different, even though they were > produced from the same foo.el. Right, but for what it's worth, the Debian infrastructure is already set up to, and would, maintain separate .eln files/trees for those two cases, *if* we ever re-versioned the emacs packages. Right now, there's only one GNU Emacs flavor in debian, we don't provide versioned packages/flavors like emacs27 and emacs28 anymore. Though we could return to that again if it were ever deemed sufficiently valuable to people. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:35 ` Rob Browning @ 2022-10-02 18:54 ` Eli Zaretskii 2022-10-02 19:37 ` Rob Browning 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 18:54 UTC (permalink / raw) To: Rob Browning; +Cc: yandros, tomas, emacs-devel > From: Rob Browning <rlb@defaultvalue.org> > Cc: tomas@tuxteam.de, emacs-devel@gnu.org > Date: Sun, 02 Oct 2022 13:35:33 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > > No, I don't think the similar handling makes sense here. The *.elc > > files are architecture- and configuration-independent, whereas the > > *.eln files are not. E.g., the same foo.elc could be used by user A > > who runs Emacs 28 and by user B who runs Emacs 29. But the > > corresponding *.eln files will be different, even though they were > > produced from the same foo.el. > > Right, but for what it's worth, the Debian infrastructure is already set > up to, and would, maintain separate .eln files/trees for those two > cases, *if* we ever re-versioned the emacs packages. Alas, there are many more than just two cases! > Right now, there's only one GNU Emacs flavor in debian, we don't provide > versioned packages/flavors like emacs27 and emacs28 anymore. But a user who installs Emacs 28 doesn't need to nuke the Emacs 27 installation, does he? And the same with a user who wants both emacs-nox, emacs-pgtk, and emacs-lucid (say) installed on the same system. Right? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:54 ` Eli Zaretskii @ 2022-10-02 19:37 ` Rob Browning 0 siblings, 0 replies; 343+ messages in thread From: Rob Browning @ 2022-10-02 19:37 UTC (permalink / raw) To: Eli Zaretskii; +Cc: yandros, tomas, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > But a user who installs Emacs 28 doesn't need to nuke the Emacs 27 > installation, does he? Yes, they do. When the Debian emacs pacakges move from a 27.x version to a 28.x version, upgrading will just move you to 28. You can't have more than one version of emacs installed via the packages right now. (You could in the past (we had emacs25-nox, etc.), but we "unversioned" them for now because, in part, upstream backward compatbility with respect to anything "substantial" has been excellent for a long time.) > And the same with a user who wants both emacs-nox, emacs-pgtk, and > emacs-lucid (say) installed on the same system. Right? That's not allowed (and that in particular never has been in Debian and (I assume) derivatives). e.g. installing emacs-nox will automatically force out/replace emacs-lucid if the former was installed, and then all of the .elcs (and .elns if we go that direction) will be automatically regenerated during the switch. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:23 ` chad 2022-10-02 16:59 ` Eli Zaretskii @ 2022-10-02 17:57 ` Rob Browning 2022-10-02 18:06 ` Lars Ingebrigtsen ` (2 more replies) 1 sibling, 3 replies; 343+ messages in thread From: Rob Browning @ 2022-10-02 17:57 UTC (permalink / raw) To: chad, Eli Zaretskii; +Cc: tomas, emacs-devel chad <yandros@gmail.com> writes: > At the risk of over-explaining due to message crossing mid-flight: the > thing you might be missing is that Debian provides a mechanism for people > to install *.elc files in a space shared by all users that is not > writable by those users, and there are people that use this provision. > Since these are used for *.elc files, it seems highly likely that they will > be desired for *.eln files. To be clear, I think there are (at least) two concerns here. First, from the Debian side, we may need some way to avoid writing files (i.e. the .eln files) to certain locations during testing, installs, etc. That problem may be solved -- via the variable that's already been mentioned. Though note that for reasons we can elaborate on if desired, it might be easier for us if the default for that variable could also be set via a corresponding environment variable, but that's a separate question. (For example, we have relevant emacs-related packages where the upstream build process runs emacs a level or two "down" (subprocess-wise), and so it'd be a lot less invasive if we could just set an environment variable to change the .eln destination, instead of having to figure out how to change each invocation of emacs in that package (and maintain those changes across future upstream versions). A second, and a separable question, is whether Debian should try to maintain system-level (root owned) .eln files the same way it does for .elc files. I consider that an open question, and it could well be that the answer ends up being "no". That's what we're trying to find out, and of course we have to begin by trying to communicate why that might be desirable. So here we are :) It's certainly the case that if the consensus here (among the upstream developers) is that we shouldn't do that, and that future choices/changes aren't likely to take that use case into consideration, then we have our answer. And that's fine. In the end, we just wanted to explain the potential use case(s), and see how the developers felt about it. Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:57 ` Rob Browning @ 2022-10-02 18:06 ` Lars Ingebrigtsen 2022-10-02 18:35 ` Eli Zaretskii 2022-10-02 18:25 ` Stefan Monnier 2022-10-02 18:32 ` Eli Zaretskii 2 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-02 18:06 UTC (permalink / raw) To: Rob Browning; +Cc: chad, Eli Zaretskii, tomas, emacs-devel Rob Browning <rlb@defaultvalue.org> writes: > A second, and a separable question, is whether Debian should try to > maintain system-level (root owned) .eln files the same way it does for > .elc files. I think that makes sense. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:06 ` Lars Ingebrigtsen @ 2022-10-02 18:35 ` Eli Zaretskii 2022-10-02 18:41 ` Rob Browning 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 18:35 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: rlb, yandros, tomas, emacs-devel > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: chad <yandros@gmail.com>, Eli Zaretskii <eliz@gnu.org>, > tomas@tuxteam.de, emacs-devel@gnu.org > Date: Sun, 02 Oct 2022 20:06:11 +0200 > > Rob Browning <rlb@defaultvalue.org> writes: > > > A second, and a separable question, is whether Debian should try to > > maintain system-level (root owned) .eln files the same way it does for > > .elc files. > > I think that makes sense. Why do you think so? A user who installs emacs-gtk and a user who installs emacs-nox will need different *.eln files. So Debian will end up with a separate subdirectory for each configuration they produce and for each Emacs version. That's a lot of files and directories, and high risk that many of them will never be used. Also, a lot of problematic management of those directories. like the decision when to delete them etc. It is much easier to leave that to users. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:35 ` Eli Zaretskii @ 2022-10-02 18:41 ` Rob Browning 2022-10-02 19:00 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-02 18:41 UTC (permalink / raw) To: Eli Zaretskii, Lars Ingebrigtsen; +Cc: yandros, tomas, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > Why do you think so? A user who installs emacs-gtk and a user who > installs emacs-nox will need different *.eln files. You can't install more than one of those at a time in Debian. They're mutually exclusive, at least right now, and have been since the variants were added. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:41 ` Rob Browning @ 2022-10-02 19:00 ` Eli Zaretskii 2022-10-02 19:50 ` Rob Browning 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 19:00 UTC (permalink / raw) To: Rob Browning; +Cc: larsi, yandros, tomas, emacs-devel > From: Rob Browning <rlb@defaultvalue.org> > Cc: yandros@gmail.com, tomas@tuxteam.de, emacs-devel@gnu.org > Date: Sun, 02 Oct 2022 13:41:32 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > > Why do you think so? A user who installs emacs-gtk and a user who > > installs emacs-nox will need different *.eln files. > > You can't install more than one of those at a time in Debian. They're > mutually exclusive, at least right now, and have been since the variants > were added. Even if we are talking about two different users on the same system? IOW, this is a system-wide restriction? Isn't that too harsh? And what about users who make changes to Emacs -- is that a legitimate use case supported by Debian installations? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 19:00 ` Eli Zaretskii @ 2022-10-02 19:50 ` Rob Browning 2022-10-03 17:41 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-02 19:50 UTC (permalink / raw) To: Eli Zaretskii; +Cc: larsi, yandros, tomas, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > Even if we are talking about two different users on the same system? > IOW, this is a system-wide restriction? Isn't that too harsh? The available Debian packages are a balance, intended to cover a broad set of common cases, i.e. Emacs without X, Emacs with the Lucid toolkit (because of, if nothing else, gtk issues), and Emacs with the GTK toolkit. You can only have one of them installed at a time, and you can (currently) only have one major version installed at a time. https://packages.debian.org/search?keywords=emacs We could of course try to accommodate multiple major versions (we did for a good while), and/or multiple simultaneous variants (nox, lucid, gtk), but we'd need to feel like the additional complexity and archive space (multiplied across the architecture-dependent packages (emacs-bin-common, etc.)[1]) was worth it for a large enough audience. [1] https://buildd.debian.org/status/package.php?p=emacs > And what about users who make changes to Emacs -- is that a legitimate > use case supported by Debian installations? I'd say that up to a point you can, and I have symlinked the relevant .el files into a ~/ directory, made sure that it's in my load-path, and then made adjustments, but past a certain point, I'd say that you'd want to switch to building Emacs yourself. Because at that point, you're perhaps no longer in the target audience for the Debian packages, or at least non on that particular machine (i.e. you might be fine with the Debian packages on most of your machines, servers, etc. but want to make much more extensive changes to Emacs on some others). -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 19:50 ` Rob Browning @ 2022-10-03 17:41 ` Eli Zaretskii 2022-10-03 18:17 ` tomas ` (2 more replies) 0 siblings, 3 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 17:41 UTC (permalink / raw) To: Rob Browning; +Cc: larsi, yandros, tomas, emacs-devel [Full disclosure: evidently, none of what I write below matters, so feel free to ignore.] > From: Rob Browning <rlb@defaultvalue.org> > Cc: larsi@gnus.org, yandros@gmail.com, tomas@tuxteam.de, emacs-devel@gnu.org > Date: Sun, 02 Oct 2022 14:50:58 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > > Even if we are talking about two different users on the same system? > > IOW, this is a system-wide restriction? Isn't that too harsh? > > The available Debian packages are a balance, intended to cover a broad > set of common cases, i.e. Emacs without X, Emacs with the Lucid toolkit > (because of, if nothing else, gtk issues), and Emacs with the GTK > toolkit. > > You can only have one of them installed at a time, and you can > (currently) only have one major version installed at a time. > > https://packages.debian.org/search?keywords=emacs That's strange to hear. Even MS-Windows allows per-user variations of PATH and per-use environment variables. It is strange to learn that Debian doesn't. > > And what about users who make changes to Emacs -- is that a legitimate > > use case supported by Debian installations? > > I'd say that up to a point you can, and I have symlinked the relevant > .el files into a ~/ directory, made sure that it's in my load-path, and > then made adjustments, but past a certain point, I'd say that you'd want > to switch to building Emacs yourself. If you want to support even just installing a different minor version, it will already require to recompile all of the *.eln files. But okay, yes, if Debian users live under such severe restrictions, then the case for having user-specific *.eln directories becomes weaker. But I still don't see that it is weaker than disallowing native compilation at run time. And I guess now I'm confused what is it exactly that you'd want to achieve. Do you want to disable native compilation, or do you want to have all *.eln files in a shared location? Because it seems to me you said you wanted both, and I don't see how both could be reconciled. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 17:41 ` Eli Zaretskii @ 2022-10-03 18:17 ` tomas 2022-10-03 18:41 ` Eli Zaretskii 2022-10-03 18:45 ` Stefan Monnier 2022-10-04 0:16 ` Rob Browning 2 siblings, 1 reply; 343+ messages in thread From: tomas @ 2022-10-03 18:17 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Rob Browning, larsi, yandros, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1440 bytes --] On Mon, Oct 03, 2022 at 08:41:22PM +0300, Eli Zaretskii wrote: > [Full disclosure: evidently, none of what I write below matters, so > feel free to ignore.] Now I think this is a bit unfair. You have a string opinion, and I think it shhould be respected, but the others have, too. > > You can only have one of them installed at a time, and you can > > (currently) only have one major version installed at a time. > > > > https://packages.debian.org/search?keywords=emacs > > That's strange to hear. Even MS-Windows allows per-user variations of > PATH and per-use environment variables. It is strange to learn that > Debian doesn't. Note that this is only as far as the Debian packaging system is concerned. Per-user you can have as many Emacsen installed as you want -- the Debian packaging system doesn't know about them. Alternatively, you can have a system-local install (going to /usr/local) besides the Debian-installed one without them interfering (that's the setup I use). This is actually what I love in Debian: you have single applications you care about which you install from souces and take over the hand feeding, and for the rest you get hight quality packaging with pretty low noise. Emacs, for me, happens to be one of those care-and-feeding applications, but for others, the choice might be different. It's them whom the Debian-packaged Emacsen are for. Cheers -- tomás [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 18:17 ` tomas @ 2022-10-03 18:41 ` Eli Zaretskii 2022-10-03 19:02 ` tomas 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 18:41 UTC (permalink / raw) To: tomas; +Cc: rlb, larsi, yandros, emacs-devel > Date: Mon, 3 Oct 2022 20:17:53 +0200 > From: tomas@tuxteam.de > Cc: Rob Browning <rlb@defaultvalue.org>, larsi@gnus.org, yandros@gmail.com, > emacs-devel@gnu.org > > On Mon, Oct 03, 2022 at 08:41:22PM +0300, Eli Zaretskii wrote: > > [Full disclosure: evidently, none of what I write below matters, so > > feel free to ignore.] > > Now I think this is a bit unfair. You have a string opinion, and > I think it shhould be respected, but the others have, too. You misunderstood the intent. > > That's strange to hear. Even MS-Windows allows per-user variations of > > PATH and per-use environment variables. It is strange to learn that > > Debian doesn't. > > Note that this is only as far as the Debian packaging system is > concerned. Per-user you can have as many Emacsen installed as you > want -- the Debian packaging system doesn't know about them. Windows installers allow you to install into a directory of your choice, and some will even set PATH to point to there; if not, you can do it manually. > Alternatively, you can have a system-local install (going to > /usr/local) besides the Debian-installed one without them interfering > (that's the setup I use). I don't understand what that means. Do you mean that you can subvert the general non-support by Debian packages of several Emacs versions installed on the same system? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 18:41 ` Eli Zaretskii @ 2022-10-03 19:02 ` tomas 2022-10-03 19:04 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: tomas @ 2022-10-03 19:02 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rlb, larsi, yandros, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1394 bytes --] On Mon, Oct 03, 2022 at 09:41:16PM +0300, Eli Zaretskii wrote: > > Date: Mon, 3 Oct 2022 20:17:53 +0200 > > From: tomas@tuxteam.de > > Cc: Rob Browning <rlb@defaultvalue.org>, larsi@gnus.org, yandros@gmail.com, > > emacs-devel@gnu.org > > > > On Mon, Oct 03, 2022 at 08:41:22PM +0300, Eli Zaretskii wrote: > > > [Full disclosure: evidently, none of what I write below matters, so > > > feel free to ignore.] > > > > Now I think this is a bit unfair. You have a string opinion, and > > I think it shhould be respected, but the others have, too. > > You misunderstood the intent. Quite possibly. > Windows installers allow you to install into a directory of your > choice, and some will even set PATH to point to there; if not, you can > do it manually. I have no clue about Windows (at least since 1990). I try to keep it that way for reasons. > > Alternatively, you can have a system-local install (going to > > /usr/local) besides the Debian-installed one without them interfering > > (that's the setup I use). > > I don't understand what that means. Do you mean that you can subvert > the general non-support by Debian packages of several Emacs versions > installed on the same system? I don't understand what you mean by "subverting the general non-support", (I somehow feel that it is a loaded term, but I can't be sure). Cheers -- tomás [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 19:02 ` tomas @ 2022-10-03 19:04 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 19:04 UTC (permalink / raw) To: tomas; +Cc: rlb, larsi, yandros, emacs-devel > Date: Mon, 3 Oct 2022 21:02:11 +0200 > From: tomas@tuxteam.de > Cc: rlb@defaultvalue.org, larsi@gnus.org, yandros@gmail.com, > emacs-devel@gnu.org > > > > Alternatively, you can have a system-local install (going to > > > /usr/local) besides the Debian-installed one without them interfering > > > (that's the setup I use). > > > > I don't understand what that means. Do you mean that you can subvert > > the general non-support by Debian packages of several Emacs versions > > installed on the same system? > > I don't understand what you mean by "subverting the general non-support", Rob said they don't support that with Debian packages. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 17:41 ` Eli Zaretskii 2022-10-03 18:17 ` tomas @ 2022-10-03 18:45 ` Stefan Monnier 2022-10-03 18:52 ` Eli Zaretskii 2022-10-04 0:16 ` Rob Browning 2 siblings, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-03 18:45 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Rob Browning, larsi, yandros, tomas, emacs-devel >> You can only have one of them installed at a time, and you can >> (currently) only have one major version installed at a time. >> >> https://packages.debian.org/search?keywords=emacs > > That's strange to hear. Even MS-Windows allows per-user variations of > PATH and per-use environment variables. It is strange to learn that > Debian doesn't. Debian does as well. What Rob is saying is that *if* you want to use the Emacs provided by Debian, then the choice of which flavor (and version) you get will apply system-wide. If that's not good enough for you, then you can install your own build of Emacs in the usual way, and it's very easy to do so. All my Debian boxes come with "Debian's Emacs" installed, and then most of them have a few other versions built locally for my own personal use. > And I guess now I'm confused what is it exactly that you'd want to > achieve. Do you want to disable native compilation, or do you want to > have all *.eln files in a shared location? AFAIK he wants: - To eagerly compile the `.eln` files for the third party ELisp packages installed system-wide using Debian's package manager. - That every time Emacs is run as part of (un)installing/building/testing a Debian package, that Emacs should not write any file under $HOME/ (or ~$REALUSER/ either). -- Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 18:45 ` Stefan Monnier @ 2022-10-03 18:52 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 18:52 UTC (permalink / raw) To: Stefan Monnier; +Cc: rlb, larsi, yandros, tomas, emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Rob Browning <rlb@defaultvalue.org>, larsi@gnus.org, > yandros@gmail.com, tomas@tuxteam.de, emacs-devel@gnu.org > Date: Mon, 03 Oct 2022 14:45:32 -0400 > > > And I guess now I'm confused what is it exactly that you'd want to > > achieve. Do you want to disable native compilation, or do you want to > > have all *.eln files in a shared location? > > AFAIK he wants: > > - To eagerly compile the `.eln` files for the third party ELisp packages > installed system-wide using Debian's package manager. > > - That every time Emacs is run as part of > (un)installing/building/testing a Debian package, that Emacs should > not write any file under $HOME/ (or ~$REALUSER/ either). That's where I'm confused: if *.eln files are installed system-wide, then why would there be any compilation that writes to the home directory, which then causes Rob to wish to disable it? All *.eln files are already available, no? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 17:41 ` Eli Zaretskii 2022-10-03 18:17 ` tomas 2022-10-03 18:45 ` Stefan Monnier @ 2022-10-04 0:16 ` Rob Browning 2022-10-04 9:30 ` Eli Zaretskii 2 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-04 0:16 UTC (permalink / raw) To: Eli Zaretskii; +Cc: larsi, yandros, tomas, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > [Full disclosure: evidently, none of what I write below matters, so > feel free to ignore.] Well, I can't speak for others, but it matters to me (your opinion and the other upstream developers'), or I wouldn't be here, and I'm very glad there's a "here for me to be" because I'm quite fond of Emacs :) > But okay, yes, if Debian users live under such severe restrictions, > then the case for having user-specific *.eln directories becomes > weaker. But I still don't see that it is weaker than disallowing > native compilation at run time. To be clear, I've never been talking about disallowing it for normal user use, and apologies if I made it sound that way. For normal users, if we pursued this, the idea would be that after "apt install emacs" finishes, they'd have a full set of .elc and .eln files corresponding to their version of Emacs, and the .el files they have. Then if anything changes in a way that warrants it, Emacs would (re)compile to their ~/.emacs.d/eln-cache/ as "normal". But for those who don't ever change their .el files (or shadow them), that would never happen, or would only happen for the files that they change/shadow. > And I guess now I'm confused what is it exactly that you'd want to > achieve. Do you want to disable native compilation, or do you want to > have all *.eln files in a shared location? Because it seems to me you > said you wanted both, and I don't see how both could be reconciled. The ability to disable .eln compilation entirely is only for some Debian-specific (though maybe useful for others) special cases, not anything we're proposing for "normal use". And it doesn't have to be "disabling" -- being able to redirect the .eln files to another (temp) dir would be fine too, as long as it wasn't too awkward to arrange for the entire (sub)process tree. Those special cases are (at least): - When building the relevant Debian packages -- for Emacs itself, for add-ons like elpa-magit, etc. Debian forbids package builds from writing outside of the package build directory /tmp, and another tree or two, e.g. HOME is not allowed. Note that this is normally handled by the Debian autobuilders, it's not something users typically do. - During package installs, i.e. whenever Emacs is run during an "apt install emacs" or "apt upgrade emacs", etc., and it may be run many times there, both directly, and indirectly as it rebuilds whichever packages need rebuilding (e.g. add-on packages). - During some autmated Debian tests. Hope this helps -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-04 0:16 ` Rob Browning @ 2022-10-04 9:30 ` Eli Zaretskii 2022-10-05 0:48 ` Rob Browning 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-04 9:30 UTC (permalink / raw) To: Rob Browning; +Cc: larsi, yandros, tomas, emacs-devel > From: Rob Browning <rlb@defaultvalue.org> > Cc: larsi@gnus.org, yandros@gmail.com, tomas@tuxteam.de, emacs-devel@gnu.org > Date: Mon, 03 Oct 2022 19:16:29 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > > [Full disclosure: evidently, none of what I write below matters, so > > feel free to ignore.] > > Well, I can't speak for others, but it matters to me (your opinion and > the other upstream developers'), or I wouldn't be here, and I'm very > glad there's a "here for me to be" because I'm quite fond of Emacs :) I didn't mean you, FWIW. > > And I guess now I'm confused what is it exactly that you'd want to > > achieve. Do you want to disable native compilation, or do you want to > > have all *.eln files in a shared location? Because it seems to me you > > said you wanted both, and I don't see how both could be reconciled. > > The ability to disable .eln compilation entirely is only for some > Debian-specific (though maybe useful for others) special cases, not > anything we're proposing for "normal use". > > And it doesn't have to be "disabling" -- being able to redirect the .eln > files to another (temp) dir would be fine too, as long as it wasn't too > awkward to arrange for the entire (sub)process tree. > > Those special cases are (at least): > > - When building the relevant Debian packages -- for Emacs itself, for > add-ons like elpa-magit, etc. Debian forbids package builds from > writing outside of the package build directory /tmp, and another > tree or two, e.g. HOME is not allowed. Note that this is normally > handled by the Debian autobuilders, it's not something users > typically do. > > - During package installs, i.e. whenever Emacs is run during an "apt > install emacs" or "apt upgrade emacs", etc., and it may be run many > times there, both directly, and indirectly as it rebuilds whichever > packages need rebuilding (e.g. add-on packages). > > - During some autmated Debian tests. The above means (AFAIU) that disabling is not the right solution, because you want the *.eln files to go to the shared location where users could have them loaded when needed. Is that correct, or am I again missing something? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-04 9:30 ` Eli Zaretskii @ 2022-10-05 0:48 ` Rob Browning 2022-10-05 7:39 ` Eli Zaretskii 2022-10-08 17:47 ` Michael Welsh Duggan 0 siblings, 2 replies; 343+ messages in thread From: Rob Browning @ 2022-10-05 0:48 UTC (permalink / raw) To: Eli Zaretskii; +Cc: larsi, yandros, tomas, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > The above means (AFAIU) that disabling is not the right solution, > because you want the *.eln files to go to the shared location where > users could have them loaded when needed. Is that correct, or am I > again missing something? If I understand you, I think the answer may be no? The test and package build cases are (often) isolated, automated, debien-specific processes that happen on debian build machines (as root and chrooted), and the entire workspace is then thrown away. It's never relevant to a "normal" user. The package install/upgrade process is system-wide, runs as root, and is specifically and quite intentionally not per-user. It's intended to establish the static, common baseline for *all* users on the machine until the next package upgrade. Hope this helps -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 0:48 ` Rob Browning @ 2022-10-05 7:39 ` Eli Zaretskii 2022-10-08 17:47 ` Michael Welsh Duggan 1 sibling, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-05 7:39 UTC (permalink / raw) To: Rob Browning; +Cc: larsi, emacs-devel > From: Rob Browning <rlb@defaultvalue.org> > Cc: larsi@gnus.org, yandros@gmail.com, tomas@tuxteam.de, emacs-devel@gnu.org > Date: Tue, 04 Oct 2022 19:48:01 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > > The above means (AFAIU) that disabling is not the right solution, > > because you want the *.eln files to go to the shared location where > > users could have them loaded when needed. Is that correct, or am I > > again missing something? > > If I understand you, I think the answer may be no? > > The test and package build cases are (often) isolated, automated, > debien-specific processes that happen on debian build machines (as root > and chrooted), and the entire workspace is then thrown away. It's never > relevant to a "normal" user. If the workspace is thrown away, you shouldn't care about the *.eln files it produces, right? They will be thrown away together with the workspace. So why bother disabling their production in this case? > The package install/upgrade process is system-wide, runs as root, and is > specifically and quite intentionally not per-user. It's intended to > establish the static, common baseline for *all* users on the machine > until the next package upgrade. When this package install/upgrade process runs, doesn't it include installing the *.eln files that come with the package, and need to be used when the users use the package after the installation? AFAIU, you want to install those *.eln files in a shared location, which is fine and supported: just put them in the same place under /usr/lib/emacs/VERSION/native-lisp/ directory where Emacs installs its precompiled *.eln files. Or, if you want to have the *.eln files from add-on packages to live in a separate place, define where that directory will be, install the package's *.eln files there, and tweak native-comp-eln-load-path to include that additional directory. Again, in this case as well, I see no problem with generation of *.eln files that you'd like to prevent. And yet you are still saying that disabling *.eln generation might be needed. Could you please describe in enough detail when are those unwanted *.eln produced in the two situations you described (test and package build case, and package install/upgrade case), and why are those *.eln files unwanted in that case? Thanks. P.S. I'm sorry to keep this thread alive for so long, but I really don't have a clear idea why you'd want to disable generation of *.eln files, and I think it's important for us to understand your reasons. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-05 0:48 ` Rob Browning 2022-10-05 7:39 ` Eli Zaretskii @ 2022-10-08 17:47 ` Michael Welsh Duggan 2022-10-15 17:39 ` Rob Browning 1 sibling, 1 reply; 343+ messages in thread From: Michael Welsh Duggan @ 2022-10-08 17:47 UTC (permalink / raw) To: Rob Browning; +Cc: Eli Zaretskii, larsi, yandros, tomas, emacs-devel In an effort to try and bridge the understanding gap, I'm going to contribute my current understanding of Debian's requirements, in the hopes that we might be able to identify any misunderstandings. Apologies in advance if this just retreads old ground and doesn't add anything useful to the conversation. From Debian's point of view, there are a few scenarios under which emacs is run: 1) When building emacs (used as part of emacs bootstrap) 2) When installing emacs via dpkg (maybe? Not certain) 3) When building an emacs package (maybe? Not certain) 4) When installing an emacs package via dpkg 5) When a user runs emacs Here are the constraints that I have intuited from the conversation: a) Case 1 builds .eln files, which will be packaged and installed in case 2. b) Cases 1 and 3 normally happen on a Debian build machine. These do *not* want emacs to write anything to $HOME, though writing to a temporary location that will subsequently be thrown away is okay. c) Case 3 might not require running emacs at all, but I can imagine that emacs might be run as part of the build process to auto-generate some files. d) Cases 2, 4, and 5 occur on a Debian user's machine. e) Cases 2 and 4 are run under root (or similar) and should not write to $HOME. f) Case 2 will install the .eln files packaged in case 1 into a world-readable, read-only location. An Emacs run in case 5 will include that location in its native file search path. g) Case 4 might run emacs to build .elc and .eln files for the package's .el files and place them in world-readable, read-only locations. An Emacs run in case 5 will include that location in its native file search path. h) Case 5 should read .eln files from the world-readable, read-only locations mentioned above, when possible, but otherwise should do native compilation and store the generated .eln files in the standard user locations based on $HOME. Open questions: i ) In case 2, are the emacs binaries and the elisp files in the same package, or are they split into different packages? If the latter, which package should contain the .eln files? ii ) Do we want (g) to actually happen? If so, do we want it to happen always, or should this be configurable in the emacs package (dpkg-configure)? iii) Does case 2 also delete files created in (g) and re-generate them using the newly installed Emacs? -- Michael Welsh Duggan (md5i@md5i.com) ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-08 17:47 ` Michael Welsh Duggan @ 2022-10-15 17:39 ` Rob Browning 0 siblings, 0 replies; 343+ messages in thread From: Rob Browning @ 2022-10-15 17:39 UTC (permalink / raw) To: Michael Welsh Duggan; +Cc: Eli Zaretskii, larsi, yandros, tomas, emacs-devel Michael Welsh Duggan <mwd@md5i.com> writes: > 1) When building emacs (used as part of emacs bootstrap) > 2) When installing emacs via dpkg (maybe? Not certain) > 3) When building an emacs package (maybe? Not certain) > 4) When installing an emacs package via dpkg > 5) When a user runs emacs > 1) When building emacs (used as part of emacs bootstrap) "building the emacs package", which we do from an upstream tree usually pretty close to the upstream tag, plus a varying number of patches. Then it's mostly configure, make check, make DESTDIR=... install to build the tree(s) that are packaged. > 2) When installing emacs via dpkg (maybe? Not certain) Yes, "apt install emacs". But that also includes upgrades. The upgrade and install scripts run as root, and should not affect HOME in most, if not all, cases. And note that both install and upgrade will run emacs a bazillion times to rebuild all of the system-level .elc files for all of the installed emacs "add-on" packages in dependency order. > 3) When building an emacs package (maybe? Not certain) Yes, and that will likely run the installed /usr/bin/emacs any number of times, in unpredictable ways, including via any test harnesses/processes the package might have. > 4) When installing an emacs package via dpkg Yes, "apt install elpa-magit", which also includes upgrades, and will invoke emacs to rebuild all of the elc files for the add-on. > 5) When a user runs emacs Yes, but in this case, > g) Case 4 might run emacs to build .elc and .eln files for the package's > .el files and place them in world-readable, read-only locations. An > Emacs run in case 5 will include that location in its native file > search path. Yes, if we proceed with ELN compilation (and don't also decide to start packaging the .elc and .eln files too, rather than building them on-demand -- which may be up for discussion, but would only be viable if the emacs fingerprint only changed very deliberatlely). > Open questions: > > i ) In case 2, are the emacs binaries and the elisp files in the same > package, or are they split into different packages? If the latter, > which package should contain the .eln files? Currently, the elisp files are in a separate emacs-el package because they used to be optional. It sounds like they can't be anymore, so emacs-common now depends on emacs-el, and everything else depends on emacs-common. I'd assume that the .eln files would go in the arch-dependent emacs-bin-common package where other things like ctags, browse, etc. go. Summary seems pretty accurate, overall. Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:57 ` Rob Browning 2022-10-02 18:06 ` Lars Ingebrigtsen @ 2022-10-02 18:25 ` Stefan Monnier 2022-10-02 18:32 ` Eli Zaretskii 2 siblings, 0 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-02 18:25 UTC (permalink / raw) To: Rob Browning; +Cc: chad, Eli Zaretskii, tomas, emacs-devel > Though note that for reasons we can elaborate on if desired, it might be > easier for us if the default for that variable could also be set via a > corresponding environment variable, but that's a separate question. Usually Emacs should obey $HOME so if you can set that to a tmp dir that should let you avoid touching the user's real $HOME. This said, there is some code in Emacs that prefers to use ~<user> over $HOME in some cases (like `su` without the `-`). I'm not sure exactly where this happens, but I suspect that the following lines in `startup.el` are part of the answer: ;; Figure out which user's init file to load, ;; either from the environment or from the options. (setq init-file-user (if noninteractive nil (user-login-name))) ;; If user has not done su, use current $HOME to find .emacs. (and init-file-user (equal init-file-user (user-real-login-name)) (setq init-file-user "")) -- Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:57 ` Rob Browning 2022-10-02 18:06 ` Lars Ingebrigtsen 2022-10-02 18:25 ` Stefan Monnier @ 2022-10-02 18:32 ` Eli Zaretskii 2022-10-02 18:37 ` Lars Ingebrigtsen 2 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 18:32 UTC (permalink / raw) To: Rob Browning; +Cc: yandros, tomas, emacs-devel > From: Rob Browning <rlb@defaultvalue.org> > Cc: tomas@tuxteam.de, emacs-devel@gnu.org > Date: Sun, 02 Oct 2022 12:57:54 -0500 > > Though note that for reasons we can elaborate on if desired, it might be > easier for us if the default for that variable could also be set via a > corresponding environment variable, but that's a separate question. > > (For example, we have relevant emacs-related packages where the upstream > build process runs emacs a level or two "down" (subprocess-wise), and > so it'd be a lot less invasive if we could just set an environment > variable to change the .eln destination, instead of having to figure > out how to change each invocation of emacs in that package (and > maintain those changes across future upstream versions). We could introduce such a variable, similarly to EMACSLOADPATH. But note several important considerations: . unlike with EMACSLOADPATH, the actual place where *.eln files will live is in a subdirectory of any directory in the list, due to the need of having them separately for different Emacs binaries and versions . EMACSLOADPATH is a mixed blessing: if you set it incorrectly, or forget that its value is inherited by subprocesses, you can completely hose an Emacs session, which is why we generally recommend against its use > A second, and a separable question, is whether Debian should try to > maintain system-level (root owned) .eln files the same way it does for > .elc files. > > I consider that an open question, and it could well be that the answer > ends up being "no". That's what we're trying to find out, and of course > we have to begin by trying to communicate why that might be desirable. Given the much more strict requirements for *.eln files wrt the target architecture, certain crucial aspects of the Emacs binary, and the contents and file name of the corresponding source file, my suggestion is to start from "no" and only consider "yes" if you discover good reasons through experience. E.g., my eln-cache directory has no less than 20 subdirectories, each one for a slightly different Emacs version and configuration. That might be somewhat extreme, given that I work on development of and use several versions in parallel, but I wouldn't be surprised to see several subdirectories for each of your users, and even less surprised to see different subdirectories for different users. So I think the case for common compiled Lisp files is much weaker for *.eln files than it is for *.elc files/ > It's certainly the case that if the consensus here (among the upstream > developers) is that we shouldn't do that, and that future > choices/changes aren't likely to take that use case into consideration, > then we have our answer. I only know that I didn't yet hear any good reason for rushing to natively-compile optional Lisp packages. That doesn't mean I'm dead certain there could be no such good reasons, of course, just that I'd like to see them described in enough detail to consider something different from what was envisioned during Emacs 28 development. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:32 ` Eli Zaretskii @ 2022-10-02 18:37 ` Lars Ingebrigtsen 2022-10-02 18:46 ` Rob Browning 2022-10-02 18:57 ` Eli Zaretskii 0 siblings, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-02 18:37 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Rob Browning, yandros, tomas, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > E.g., my eln-cache directory has no less than 20 subdirectories, each > one for a slightly different Emacs version and configuration. You're an Emacs developer, so that's to be expected. But for normal users, the .eln files are neither more nor less specific to an Emacs version than, say, the .pdmp file. If Debian distributes a specific Emacs version, it will be accompanied with the matching .pdmp file -- and the matching .eln files, if that is what Debian decides to do. This is not complicated. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:37 ` Lars Ingebrigtsen @ 2022-10-02 18:46 ` Rob Browning 2022-10-02 18:57 ` Eli Zaretskii 1 sibling, 0 replies; 343+ messages in thread From: Rob Browning @ 2022-10-02 18:46 UTC (permalink / raw) To: Lars Ingebrigtsen, Eli Zaretskii; +Cc: yandros, tomas, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Eli Zaretskii <eliz@gnu.org> writes: > >> E.g., my eln-cache directory has no less than 20 subdirectories, each >> one for a slightly different Emacs version and configuration. > > You're an Emacs developer, so that's to be expected. > > But for normal users, the .eln files are neither more nor less specific > to an Emacs version than, say, the .pdmp file. If Debian distributes a > specific Emacs version, it will be accompanied with the matching .pdmp > file -- and the matching .eln files, if that is what Debian decides to > do. With the current Debian arrangment, there would only ever be *one* system-level .eln tree for the one installed Debian variant (emacs-nox, emacs-lucid, or emacs-gtk). Though of course the dir name hash might/would change during each upgrade, but we already assume we have to rebuild all the .elc files on each upgrade, and do (in dependency order), so that shouldn't be a problem. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:37 ` Lars Ingebrigtsen 2022-10-02 18:46 ` Rob Browning @ 2022-10-02 18:57 ` Eli Zaretskii 2022-10-02 19:01 ` Lars Ingebrigtsen 2022-10-02 19:58 ` Stefan Monnier 1 sibling, 2 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 18:57 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: rlb, yandros, tomas, emacs-devel > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Rob Browning <rlb@defaultvalue.org>, yandros@gmail.com, > tomas@tuxteam.de, emacs-devel@gnu.org > Date: Sun, 02 Oct 2022 20:37:20 +0200 > > Eli Zaretskii <eliz@gnu.org> writes: > > > E.g., my eln-cache directory has no less than 20 subdirectories, each > > one for a slightly different Emacs version and configuration. > > You're an Emacs developer, so that's to be expected. As I wrote (and you elided), I don't expect to see 20 subdirectories on other systems, but I wouldn't be surprised to see 5 or 10. > But for normal users, the .eln files are neither more nor less specific > to an Emacs version than, say, the .pdmp file. If Debian distributes a > specific Emacs version, it will be accompanied with the matching .pdmp > file -- and the matching .eln files, if that is what Debian decides to > do. We are talking about optional packages, not about preloaded Lisp files. The preloaded ones indeed go together with the pdumper file, but the optional ones don't. > This is not complicated. Famous last words. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:57 ` Eli Zaretskii @ 2022-10-02 19:01 ` Lars Ingebrigtsen 2022-10-02 19:03 ` Eli Zaretskii 2022-10-02 19:58 ` Stefan Monnier 1 sibling, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-02 19:01 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rlb, yandros, tomas, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > We are talking about optional packages, not about preloaded Lisp > files. The preloaded ones indeed go together with the pdumper file, > but the optional ones don't. I'm not sure what you mean by "optional packages" -- we're talking about .eln files for the .elc files in the Emacs distribution. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 19:01 ` Lars Ingebrigtsen @ 2022-10-02 19:03 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 19:03 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: rlb, yandros, tomas, emacs-devel > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: rlb@defaultvalue.org, yandros@gmail.com, tomas@tuxteam.de, > emacs-devel@gnu.org > Date: Sun, 02 Oct 2022 21:01:53 +0200 > > Eli Zaretskii <eliz@gnu.org> writes: > > > We are talking about optional packages, not about preloaded Lisp > > files. The preloaded ones indeed go together with the pdumper file, > > but the optional ones don't. > > I'm not sure what you mean by "optional packages" -- we're talking about > .eln files for the .elc files in the Emacs distribution. No, we are not. We are talking about unbundled packages, like Magit. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:57 ` Eli Zaretskii 2022-10-02 19:01 ` Lars Ingebrigtsen @ 2022-10-02 19:58 ` Stefan Monnier 2022-10-02 20:09 ` Stefan Monnier 1 sibling, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-02 19:58 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Lars Ingebrigtsen, rlb, yandros, tomas, emacs-devel > We are talking about optional packages, not about preloaded Lisp > files. The preloaded ones indeed go together with the pdumper file, > but the optional ones don't. Debian always recompiles separately all the .el files in third party packages (installed vi Debian) for all the installed Emacsen. IOW if you have `emacs24`, `xemacs`, and `emacs25` installed, you'll have 3 sets of .elc files for every ELisp package installed via Debian's package manager. So the `.eln` files aren't really more specific than the `.elc` in that setting (except for the situation where you have `emacs-lucid` and `emacs-gtk` and `emacs-nox` installed where these (unnecessarily) require different `.eln` files but will share the same `elc` files). Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 19:58 ` Stefan Monnier @ 2022-10-02 20:09 ` Stefan Monnier 0 siblings, 0 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-02 20:09 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Lars Ingebrigtsen, rlb, yandros, tomas, emacs-devel > So the `.eln` files aren't really more specific than the `.elc` in > that setting (except for the situation where you have `emacs-lucid` and > `emacs-gtk` and `emacs-nox` installed where these (unnecessarily) > require different `.eln` files but will share the same `elc` files). Apparently that exception doesn't apply either, since Debian doesn't allow simultaneously installing those different flavors :-) [ Funny, I never noticed it in all those years using Debian. ] Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:11 ` Eli Zaretskii 2022-10-02 16:23 ` chad @ 2022-10-02 16:27 ` tomas 2022-10-02 17:01 ` Eli Zaretskii 2022-10-02 23:58 ` Sean Whitton 2 siblings, 1 reply; 343+ messages in thread From: tomas @ 2022-10-02 16:27 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 3544 bytes --] On Sun, Oct 02, 2022 at 07:11:52PM +0300, Eli Zaretskii wrote: > > Date: Sun, 2 Oct 2022 17:53:46 +0200 > > From: tomas@tuxteam.de > > Cc: emacs-devel@gnu.org [two views: JIT cache vs. pre-compiled .eln] > Let me be blunt: this is currently _the_only_ view of the Emacs > project. After a lot of deliberations, we didn't find any reasons for > alternative views. My suggestion is to try our view before concluding > that it doesn't fit some situations. Thanks for being blunt :-) My whole intention was to make this difference clear, because I had the impression that there were unspoken assumptions making the discussion unnecessarily difficult. > > > Exactly. So what is the problem with directories writable by all > > > users? > > > > User separation goes out of the window, and this is one important > > service the OS provides. To illustrate, one user could put malicious > > .eln files all other users would execute. > > This is about installation writing files into a shared space on disk, > right? No. I was picking up on your "directories writable by all users". Perhaps I misunderstood and you didn't mean common directories: if so, please ignore. > If so, this is something for the Debian packagers to figure > out, because doing that is their request. And anyway, I don't > understand how do *.eln files are different from *.elc files, which > are already written to shared disk space upon installation. What am I > missing? Perhaps nothing. With the native-comp-eln-load-path, there seems to be a way for Debian to "do it its way"; you don't recommend it (I still don't quite understand), and there are very strong reasons to take your recommendations seriously. > > > > That's all fine, but then users wouldn't profit from the pre-compiled > > > > .eln. > > > > > > There's no profit, IME. There are only disadvantages: you are in > > > effect fighting against the Emacs defaults, for reasons that are > > > purely theoretical. > > > > I have the impression that some of that reasons are quite practical > > for Debian packagers. > > I submit that those reasons were most probably derived from a broken > analogy with the *.elc files and with byte-compilation in general. > Not from actual usage experience. Native compilation looks > deceptively similar to byte compilation, but it isn't. So if > producing the *.eln files seems to contradict some Debian rules and > procedures, my suggestion is to talk to the upstream project, before > inventing solutions, because of 2 considerations: I understand that there is a difference between .elc and .eln (the set of dependencies is significantly bigger in the second case, for one). > > . the problems may not be real ones, only perceived ones > . the upstream codebase might already provide solutions I can understand Debian's position here (yours too). > > > > In a Debian-distributed Emacs [...] > > > > there are .elc in /usr/share for all to use; due to the search path, [...] > > > > Can you do the same for .elc? > > > > > > I guess you meant "with .eln files"? Uh -- yes, sorry. Well spotted. > > > Yes, see native-comp-eln-load-path, which was already mentioned > > > here several times. > > > > So that might be one part of the way out. > > If one needs it. I don't think they do, and I don't recommend going > there. Hm. I don't want to steal your time more, but... if you could illustrate why, I'd eager to learn. Cheers -- tomás [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:27 ` tomas @ 2022-10-02 17:01 ` Eli Zaretskii 2022-10-02 18:37 ` Rob Browning 2022-10-02 20:51 ` tomas 0 siblings, 2 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 17:01 UTC (permalink / raw) To: tomas; +Cc: emacs-devel > Date: Sun, 2 Oct 2022 18:27:05 +0200 > From: tomas@tuxteam.de > Cc: emacs-devel@gnu.org > > > > > Yes, see native-comp-eln-load-path, which was already mentioned > > > > here several times. > > > > > > So that might be one part of the way out. > > > > If one needs it. I don't think they do, and I don't recommend going > > there. > > Hm. I don't want to steal your time more, but... if you could illustrate > why, I'd eager to learn. I already did: the probability of different users to produce different *.eln files from the same *.el sources is high, and disk space is cheap. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:01 ` Eli Zaretskii @ 2022-10-02 18:37 ` Rob Browning 2022-10-02 18:58 ` Eli Zaretskii 2022-10-02 20:51 ` tomas 1 sibling, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-02 18:37 UTC (permalink / raw) To: Eli Zaretskii, tomas; +Cc: emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > I already did: the probability of different users to produce different > *.eln files from the same *.el sources is high As mentioned in another message, not in Debian's infrastructure. It'd be impossible (wrt the shared, system-level .elc and/or .eln files), *if* we ended up deciding to pursue system-level .eln files. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:37 ` Rob Browning @ 2022-10-02 18:58 ` Eli Zaretskii 2022-10-02 19:58 ` Rob Browning 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 18:58 UTC (permalink / raw) To: Rob Browning; +Cc: tomas, emacs-devel > From: Rob Browning <rlb@defaultvalue.org> > Cc: emacs-devel@gnu.org > Date: Sun, 02 Oct 2022 13:37:49 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > > I already did: the probability of different users to produce different > > *.eln files from the same *.el sources is high > > As mentioned in another message, not in Debian's infrastructure. It'd > be impossible (wrt the shared, system-level .elc and/or .eln files), > *if* we ended up deciding to pursue system-level .eln files. Which IMO a significant limitation that is the direct result of such a decision. Right? Whereas leaving it up to the users to produce *.el in the JIT manner lifts this limitation. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:58 ` Eli Zaretskii @ 2022-10-02 19:58 ` Rob Browning 0 siblings, 0 replies; 343+ messages in thread From: Rob Browning @ 2022-10-02 19:58 UTC (permalink / raw) To: Eli Zaretskii; +Cc: tomas, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > Which IMO a significant limitation that is the direct result of such a > decision. Right? Whereas leaving it up to the users to produce *.el > in the JIT manner lifts this limitation. Maybe my other mail about the packaging tradeoffs helps here, but if not, I can try to elaborate further. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 17:01 ` Eli Zaretskii 2022-10-02 18:37 ` Rob Browning @ 2022-10-02 20:51 ` tomas 1 sibling, 0 replies; 343+ messages in thread From: tomas @ 2022-10-02 20:51 UTC (permalink / raw) To: Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 472 bytes --] On Sun, Oct 02, 2022 at 08:01:58PM +0300, Eli Zaretskii wrote: > > Date: Sun, 2 Oct 2022 18:27:05 +0200 > > From: tomas@tuxteam.de > > Cc: emacs-devel@gnu.org [...] > > Hm. I don't want to steal your time more, but... if you could illustrate > > why, I'd eager to learn. > > I already did: the probability of different users to produce different > *.eln files from the same *.el sources is high, and disk space is > cheap. Thanks Cheers -- tomás [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 195 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 16:11 ` Eli Zaretskii 2022-10-02 16:23 ` chad 2022-10-02 16:27 ` tomas @ 2022-10-02 23:58 ` Sean Whitton 2022-10-03 16:24 ` Eli Zaretskii 2 siblings, 1 reply; 343+ messages in thread From: Sean Whitton @ 2022-10-02 23:58 UTC (permalink / raw) To: Eli Zaretskii; +Cc: tomas, emacs-devel Hello, On Sun 02 Oct 2022 at 07:11PM +03, Eli Zaretskii wrote: > I submit that those reasons were most probably derived from a broken > analogy with the *.elc files and with byte-compilation in general. > Not from actual usage experience. Native compilation looks > deceptively similar to byte compilation, but it isn't. So if > producing the *.eln files seems to contradict some Debian rules and > procedures, my suggestion is to talk to the upstream project, before > inventing solutions, because of 2 considerations: > > . the problems may not be real ones, only perceived ones > . the upstream codebase might already provide solutions The battery usage thing is real, from my own user experience. I have to remember not to unplug my laptop until the load indicator in my status bar has dropped back down, else the battery won't last as long as I expect it to. -- Sean Whitton ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 23:58 ` Sean Whitton @ 2022-10-03 16:24 ` Eli Zaretskii 2022-10-03 18:23 ` Sean Whitton 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 16:24 UTC (permalink / raw) To: Sean Whitton; +Cc: tomas, emacs-devel > From: Sean Whitton <spwhitton@spwhitton.name> > Cc: tomas@tuxteam.de, emacs-devel@gnu.org > Date: Sun, 02 Oct 2022 16:58:34 -0700 > > The battery usage thing is real, from my own user experience. > > I have to remember not to unplug my laptop until the load indicator in > my status bar has dropped back down, else the battery won't last as long > as I expect it to. When I start an Emacs session for a new version (which needs to recompile all the packages I use, because my sessions are restored by desktop.el), I see about 100 Lisp files compiled into *.eln inside of 1 minute, and then another bunch of 100 inside another minute (with 10 minutes between these two groups). Does this strike you as a long time to avoid unplugging? I hope your battery can hold more than a few minutes. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 16:24 ` Eli Zaretskii @ 2022-10-03 18:23 ` Sean Whitton 2022-10-03 18:46 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Sean Whitton @ 2022-10-03 18:23 UTC (permalink / raw) To: Eli Zaretskii; +Cc: tomas, emacs-devel Hello, On Mon 03 Oct 2022 at 07:24PM +03, Eli Zaretskii wrote: >> From: Sean Whitton <spwhitton@spwhitton.name> >> Cc: tomas@tuxteam.de, emacs-devel@gnu.org >> Date: Sun, 02 Oct 2022 16:58:34 -0700 >> >> The battery usage thing is real, from my own user experience. >> >> I have to remember not to unplug my laptop until the load indicator in >> my status bar has dropped back down, else the battery won't last as long >> as I expect it to. > > When I start an Emacs session for a new version (which needs to > recompile all the packages I use, because my sessions are restored by > desktop.el), I see about 100 Lisp files compiled into *.eln inside of > 1 minute, and then another bunch of 100 inside another minute (with 10 > minutes between these two groups). Does this strike you as a long > time to avoid unplugging? I hope your battery can hold more than a > few minutes. Sounds like your machine is just a lot better than mine. -- Sean Whitton ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 18:23 ` Sean Whitton @ 2022-10-03 18:46 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 18:46 UTC (permalink / raw) To: Sean Whitton; +Cc: tomas, emacs-devel > From: Sean Whitton <spwhitton@spwhitton.name> > Cc: tomas@tuxteam.de, emacs-devel@gnu.org > Date: Mon, 03 Oct 2022 11:23:46 -0700 > > > When I start an Emacs session for a new version (which needs to > > recompile all the packages I use, because my sessions are restored by > > desktop.el), I see about 100 Lisp files compiled into *.eln inside of > > 1 minute, and then another bunch of 100 inside another minute (with 10 > > minutes between these two groups). Does this strike you as a long > > time to avoid unplugging? I hope your battery can hold more than a > > few minutes. > > Sounds like your machine is just a lot better than mine. It's a 2012 vintage Core i7. And it runs Windows XP, so both process startup and disk I/O are slower than GNU/Linux. Though the main disk is SSD. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 6:42 ` Eli Zaretskii 2022-10-02 15:53 ` tomas @ 2022-10-02 18:32 ` Rob Browning 2022-10-02 18:51 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-02 18:32 UTC (permalink / raw) To: Eli Zaretskii, tomas; +Cc: emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > My recommendation is to use the default JIT manner until and unless > actual problems are reported by users. [...] > There's no profit, IME. There are only disadvantages: you are in > effect fighting against the Emacs defaults, for reasons that are > purely theoretical. If I understand your meaning in both of these cases, I'll just note that for the things we've been discussing here, I believe we've already had complaints/requests from Debian users. Whether that's significant enough to warrant accommodation is another question, but that may not leave the concerns theoretical, strictly speaking. And for what it's worth, I can see both sides of the argument(s), i.e. I can understand why upstream, it could be that on balance, those concerns won't (and maybe shouldn't) be considered sufficient when balanced against other considerations. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:32 ` Rob Browning @ 2022-10-02 18:51 ` Eli Zaretskii 2022-10-02 19:57 ` Rob Browning 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-02 18:51 UTC (permalink / raw) To: Rob Browning; +Cc: tomas, emacs-devel > From: Rob Browning <rlb@defaultvalue.org> > Cc: emacs-devel@gnu.org > Date: Sun, 02 Oct 2022 13:32:02 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > > My recommendation is to use the default JIT manner until and unless > > actual problems are reported by users. > > [...] > > > There's no profit, IME. There are only disadvantages: you are in > > effect fighting against the Emacs defaults, for reasons that are > > purely theoretical. > > If I understand your meaning in both of these cases, I'll just note that > for the things we've been discussing here, I believe we've already had > complaints/requests from Debian users. Whether that's significant > enough to warrant accommodation is another question, but that may not > leave the concerns theoretical, strictly speaking. Please try to understand the nature of the complaints. It is quite possible that users simply use the (broken) analogy with the *.elc files, because they misunderstand the quantitatively new aspects of native-compilation. There's more here than meets the eye, as has been demonstrated even in this discussion. Please don't hesitate to involve us in these discussions with your complaining users, if you think we could help. > And for what it's worth, I can see both sides of the argument(s), i.e. I > can understand why upstream, it could be that on balance, those concerns > won't (and maybe shouldn't) be considered sufficient when balanced > against other considerations. One other aspect that should be kept in mind is complexity. The introduction of the pdumper file in Emacs 27 and of native compilation in Emacs 28 made Emacs deployment and startup code significantly more complex, and as a side effect invalidated some of the deployment methods people used to use, like some "clever" symlinking of the binaries or the directories. We are still learning these consequences (although some of them already caused fixes in the code). In this situation, adding even more possible deployment method that upstream should support risks making the startup code a maintenance burden. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 18:51 ` Eli Zaretskii @ 2022-10-02 19:57 ` Rob Browning 2022-10-03 15:48 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Rob Browning @ 2022-10-02 19:57 UTC (permalink / raw) To: Eli Zaretskii; +Cc: tomas, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: > Please try to understand the nature of the complaints. For what it's worth, one of the complaints was: I do not want emacs suddenly swamping the CPU on my laptop unpredictably. I want all this work (in the common case) done while I know I'm connected to power, before I head out. ...and we have for a very long time, perhaps as long as I've been working with the packaging, had users who are concerned with disk space usage, and ask for smaller package footprints. Those concerns may not carry the day, but they do exist. -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-02 19:57 ` Rob Browning @ 2022-10-03 15:48 ` Eli Zaretskii 2022-10-03 16:39 ` Stefan Monnier 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 15:48 UTC (permalink / raw) To: Rob Browning; +Cc: tomas, emacs-devel > From: Rob Browning <rlb@defaultvalue.org> > Cc: tomas@tuxteam.de, emacs-devel@gnu.org > Date: Sun, 02 Oct 2022 14:57:49 -0500 > > Eli Zaretskii <eliz@gnu.org> writes: > > > Please try to understand the nature of the complaints. > > For what it's worth, one of the complaints was: > > I do not want emacs suddenly swamping the CPU on my laptop > unpredictably. I want all this work (in the common case) done while I > know I'm connected to power, before I head out. > > ...and we have for a very long time, perhaps as long as I've been > working with the packaging, had users who are concerned with disk space > usage, and ask for smaller package footprints. But Emacs does that all the time: there are many features that invoke sub-processes and many more features that write to the disk. I never heard anyone complaining seriously about that, and I'm quite sure many users don't even know which Emacs commands invoke subprocesses under the hood. So I'm not sure these complaints are based on real problems. Did anyone compare the "sudden swamp of the CPU" caused by JIT native compilation with what happens with other commands that invoke subprocesses? If so, did they present some quantitative data? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 15:48 ` Eli Zaretskii @ 2022-10-03 16:39 ` Stefan Monnier 2022-10-03 17:30 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-03 16:39 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Rob Browning, tomas, emacs-devel > But Emacs does that all the time: there are many features that invoke > sub-processes and many more features that write to the disk. I never > heard anyone complaining seriously about that, and I'm quite sure many > users don't even know which Emacs commands invoke subprocesses under > the hood. FWIW, during the stealth jit-lock discussion, several people mentioned the battery impact. And the issue is not subprocesses per se, but it's extra processing that happens outside of the control of the user. > So I'm not sure these complaints are based on real problems. Did > anyone compare the "sudden swamp of the CPU" caused by JIT native > compilation with what happens with other commands that invoke > subprocesses? If so, did they present some quantitative data? Probably not, no. It's likely mostly a question of perception, so if we could make it completely invisible the "problem" would disappear :-) But the fact that lazy native compilation tends to pop up warnings (and to make matters worse, it does so ... without warning) makes it very much visible instead. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 16:39 ` Stefan Monnier @ 2022-10-03 17:30 ` Eli Zaretskii 2022-10-03 18:33 ` Stefan Monnier 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 17:30 UTC (permalink / raw) To: Stefan Monnier; +Cc: rlb, tomas, emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Rob Browning <rlb@defaultvalue.org>, tomas@tuxteam.de, > emacs-devel@gnu.org > Date: Mon, 03 Oct 2022 12:39:44 -0400 > > > But Emacs does that all the time: there are many features that invoke > > sub-processes and many more features that write to the disk. I never > > heard anyone complaining seriously about that, and I'm quite sure many > > users don't even know which Emacs commands invoke subprocesses under > > the hood. > > FWIW, during the stealth jit-lock discussion, several people mentioned the > battery impact. Yes. And jit-stealth is different, in that it takes a much longer time for it to become quiet, because it works in small chinks, and only when Emacs is actually idle. JIT native-compilation is much faster, and also uses several execution units of the CPU in parallel. > And the issue is not subprocesses per se, but it's extra processing that > happens outside of the control of the user. How do you mean "outside of the control of the user"? The user causes it by loading a feature. If no new features are loaded, no native-compilation will happen. How is that different from any other command that uses a subprocess under the hood? > > So I'm not sure these complaints are based on real problems. Did > > anyone compare the "sudden swamp of the CPU" caused by JIT native > > compilation with what happens with other commands that invoke > > subprocesses? If so, did they present some quantitative data? > > Probably not, no. It's likely mostly a question of perception, so if we > could make it completely invisible the "problem" would disappear :-) > But the fact that lazy native compilation tends to pop up warnings (and > to make matters worse, it does so ... without warning) makes it very > much visible instead. On my system, I don't see any warnings. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 17:30 ` Eli Zaretskii @ 2022-10-03 18:33 ` Stefan Monnier 2022-10-03 18:49 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-03 18:33 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rlb, tomas, emacs-devel >> FWIW, during the stealth jit-lock discussion, several people mentioned the >> battery impact. > Yes. And jit-stealth is different, in that it takes a much longer > time for it to become quiet, because it works in small chinks, and > only when Emacs is actually idle. JIT native-compilation is much > faster, and also uses several execution units of the CPU in parallel. Indeed, it's quite different. I think even more important than the above is the fact that it's a one-time thing only (for typical use cases at least), whereas the jit-lock-stealth cost was paid over and over again after every buffer modification. >> And the issue is not subprocesses per se, but it's extra processing that >> happens outside of the control of the user. > How do you mean "outside of the control of the user"? The user causes > it by loading a feature. When a user loads a feature, this request doesn't say explicitly "and native compile the file". And there is no direct way to "just load this package without compiling it". So the compilation is not really under the control of the user. I don't think users know, when they load a feature, if it's the first time they loaded this feature in this version of Emacs, so by and large they can't predict when the compilation does happen. To be clear: I don't think it's a problem. I just think it's part of the reason for the reaction. > How is that different from any other command that uses a subprocess > under the hood? It's different in that for those other commands, the users cannot get the result they want without running that subprocess, so even if they're not fully aware of it, they do have some vague idea it's inevitably going to happen. In contrast, the compilation is not indispensable, (so much so that the past it never happened). >> Probably not, no. It's likely mostly a question of perception, so if we >> could make it completely invisible the "problem" would disappear :-) >> But the fact that lazy native compilation tends to pop up warnings (and >> to make matters worse, it does so ... without warning) makes it very >> much visible instead. > > On my system, I don't see any warnings. That's presumably because you stick to the bundled packages (where we've managed to keep a "no compilation warnings" state for the last few years) or the few third party packages which are careful enough to eliminate all warnings. The rise of ELPA has removed most of the cases where native compilation will miscompile files (because ELPA forces the .el files to be compiled in a "standard" way rather than using ad-hoc Makefile rules which the native compiler cannot and does not try to reproduce), but compilation warnings are still very common (the situation is improving, tho, and arguably the warnings popping up out of the blue during lazy native compilation may prove to be a good way to convince more package authors to clean up their act). Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 18:33 ` Stefan Monnier @ 2022-10-03 18:49 ` Eli Zaretskii 2022-10-03 21:58 ` Stefan Kangas 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-03 18:49 UTC (permalink / raw) To: Stefan Monnier; +Cc: rlb, tomas, emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: rlb@defaultvalue.org, tomas@tuxteam.de, emacs-devel@gnu.org > Date: Mon, 03 Oct 2022 14:33:09 -0400 > > >> And the issue is not subprocesses per se, but it's extra processing that > >> happens outside of the control of the user. > > How do you mean "outside of the control of the user"? The user causes > > it by loading a feature. > > When a user loads a feature, this request doesn't say explicitly "and > native compile the file". And there is no direct way to "just load this > package without compiling it". So the compilation is not really under > the control of the user. > > I don't think users know, when they load a feature, if it's the first > time they loaded this feature in this version of Emacs, so by and large > they can't predict when the compilation does happen. It just takes getting used to, that's all. > > How is that different from any other command that uses a subprocess > > under the hood? > > It's different in that for those other commands, the users cannot get > the result they want without running that subprocess, so even if they're > not fully aware of it, they do have some vague idea it's inevitably > going to happen. > > In contrast, the compilation is not indispensable, (so much so that the > past it never happened). So native-compilation is "guilty" because it can be disabled? ;-) > > On my system, I don't see any warnings. > > That's presumably because you stick to the bundled packages (where we've > managed to keep a "no compilation warnings" state for the last few > years) or the few third party packages which are careful enough to > eliminate all warnings. > > The rise of ELPA has removed most of the cases where native compilation > will miscompile files (because ELPA forces the .el files to be compiled > in a "standard" way rather than using ad-hoc Makefile rules which the > native compiler cannot and does not try to reproduce), but compilation > warnings are still very common (the situation is improving, tho, and > arguably the warnings popping up out of the blue during lazy native > compilation may prove to be a good way to convince more package authors > to clean up their act). So this, too, is extremely temporary, and will disappear soon enough, at least in popular packages. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 18:49 ` Eli Zaretskii @ 2022-10-03 21:58 ` Stefan Kangas 2022-10-04 6:10 ` Eli Zaretskii 2022-10-04 13:30 ` Stefan Monnier 0 siblings, 2 replies; 343+ messages in thread From: Stefan Kangas @ 2022-10-03 21:58 UTC (permalink / raw) To: Eli Zaretskii, Stefan Monnier; +Cc: rlb, tomas, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> compilation warnings are still very common (the situation is >> improving, tho, and arguably the warnings popping up out of the blue >> during lazy native compilation may prove to be a good way to convince >> more package authors to clean up their act). > > So this, too, is extremely temporary, and will disappear soon enough, > at least in popular packages. I don't see that the situation has substantially improved, though I'm of course happy to hear that Stefan M remains optimistic. In far too many cases, it is very hard to get fixes for warnings merged in a timely manner, and it's even harder to get those fixes actually released. Here are some recent examples, to give an idea: https://github.com/deb0ch/emacs-winum/pull/31 https://github.com/lewang/ws-butler/pull/40 https://github.com/Malabarba/aggressive-indent-mode/pull/150 These examples can be multiplied at will. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 21:58 ` Stefan Kangas @ 2022-10-04 6:10 ` Eli Zaretskii 2022-10-04 13:30 ` Stefan Monnier 1 sibling, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-04 6:10 UTC (permalink / raw) To: Stefan Kangas; +Cc: monnier, rlb, tomas, emacs-devel > From: Stefan Kangas <stefankangas@gmail.com> > Date: Mon, 3 Oct 2022 23:58:28 +0200 > Cc: rlb@defaultvalue.org, tomas@tuxteam.de, emacs-devel@gnu.org > > Eli Zaretskii <eliz@gnu.org> writes: > > >> compilation warnings are still very common (the situation is > >> improving, tho, and arguably the warnings popping up out of the blue > >> during lazy native compilation may prove to be a good way to convince > >> more package authors to clean up their act). > > > > So this, too, is extremely temporary, and will disappear soon enough, > > at least in popular packages. > > I don't see that the situation has substantially improved You are too impatient, IMO. This stuff takes time to propagate; Emacs 28 is just beginning to be widely used (otherwise we'd have this discussion many moons ago). We had these problems in Emacs as well, originally. > Here are some recent examples, to give an idea: > > https://github.com/deb0ch/emacs-winum/pull/31 > https://github.com/lewang/ws-butler/pull/40 > https://github.com/Malabarba/aggressive-indent-mode/pull/150 > > These examples can be multiplied at will. Yes, it is easy to think the problem is larger and more significant than it really is. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-03 21:58 ` Stefan Kangas 2022-10-04 6:10 ` Eli Zaretskii @ 2022-10-04 13:30 ` Stefan Monnier 1 sibling, 0 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-04 13:30 UTC (permalink / raw) To: Stefan Kangas; +Cc: Eli Zaretskii, rlb, tomas, emacs-devel > I don't see that the situation has substantially improved, though I'm of > course happy to hear that Stefan M remains optimistic. Seen from the K-side, the glass is half full, but I'm on the M-side where I see it as half-empty. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-30 13:13 David Bremner 2022-09-30 13:56 ` Eli Zaretskii @ 2022-09-30 15:38 ` Stefan Monnier 2022-09-30 17:05 ` David Bremner 1 sibling, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-09-30 15:38 UTC (permalink / raw) To: David Bremner; +Cc: eliz, emacs-devel, akrl, rlb David Bremner [2022-09-30 10:13:56] wrote: > I think just turning off native compilation when HOME is not writeable > would be sensible from a Debian point of view. Emacs did not previously That's not the problem: the problem is when $HOME is writable but doesn't belong to the user (e.g. because the user is root): then Emacs ends up writing files (and creating directories) which then end up being "unwritable" by the owner of $HOME. IIUC this happens very "naturally" with `su` followed by running Emacs, because many version of `su` (contrary to `su -`) preserve the $LOGNAME and Emacs uses ~$LOGNAME as "the HOME" (instead of $HOME) when run as root. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-30 15:38 ` Stefan Monnier @ 2022-09-30 17:05 ` David Bremner 2022-09-30 17:32 ` David Bremner 0 siblings, 1 reply; 343+ messages in thread From: David Bremner @ 2022-09-30 17:05 UTC (permalink / raw) To: Stefan Monnier; +Cc: eliz, emacs-devel, akrl, rlb Stefan Monnier <monnier@iro.umontreal.ca> writes: > David Bremner [2022-09-30 10:13:56] wrote: >> I think just turning off native compilation when HOME is not writeable >> would be sensible from a Debian point of view. Emacs did not previously > > That's not the problem: the problem is when $HOME is writable but doesn't > belong to the user (e.g. because the user is root): then Emacs ends up > writing files (and creating directories) which then end up being > "unwritable" by the owner of $HOME. That is _also_ a problem, but not the one causing many Debian packages to fail to build from source. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-30 17:05 ` David Bremner @ 2022-09-30 17:32 ` David Bremner 0 siblings, 0 replies; 343+ messages in thread From: David Bremner @ 2022-09-30 17:32 UTC (permalink / raw) To: Stefan Monnier; +Cc: eliz, emacs-devel, akrl, rlb David Bremner <david@tethera.net> writes: > Stefan Monnier <monnier@iro.umontreal.ca> writes: > >> David Bremner [2022-09-30 10:13:56] wrote: >>> I think just turning off native compilation when HOME is not writeable >>> would be sensible from a Debian point of view. Emacs did not previously >> >> That's not the problem: the problem is when $HOME is writable but doesn't >> belong to the user (e.g. because the user is root): then Emacs ends up >> writing files (and creating directories) which then end up being >> "unwritable" by the owner of $HOME. > > That is _also_ a problem, but not the one causing many Debian packages > to fail to build from source. Sorry, that came off snappier than intended. You are of course correct that the issue you report is a real problem for users, I've just been up to my elbows for the last week or so dealing with the fallout of the problem I described. d ^ permalink raw reply [flat|nested] 343+ messages in thread
* Suppressing native compilation (short and long term) @ 2022-09-28 1:04 Rob Browning 2022-09-28 11:02 ` Lars Ingebrigtsen ` (2 more replies) 0 siblings, 3 replies; 343+ messages in thread From: Rob Browning @ 2022-09-28 1:04 UTC (permalink / raw) To: emacs-devel Perhaps relevant in other contexts, but at least in the context of Debian work, we'd like to be able to suppress all native compilation in some contexts, or more specifically all writes to HOME (or really, to avoid any implicit file manipulations[1]). One key case is package builds and package installs (whether the emacs packages themselves, or any of the various emacs add-on packages (e.g. elpa-*)). For package builds, HOME is typically set to /nonexistent (or similar), and for package installs we don't want the install commands (preinst, postinst, etc.), which are running as root, to write anything to /root/. It's also been suggested by some users that they might like to have a supported way to disable native compilation (even when it's the system default), in order to avoid the unpredictable cpu and/or battery cost sometimes. Of course, assuming we eventually augment debian's current emacs infrastructure to generate system-level .eln files during (add-on) package installs, in addition to the .elc files it currently produces, much of that concern for debian and its derivatives might recede. In any case, I noticed that there is a no-native-compile variable, and wondered if (in the short term), I might be able to craft a (possibly debian-specific for now) way to disable native compilation across the board. Just to see, I changed no-native-compile to default to (getenv "DEBIAN_EMACS_DISABLE_NATIVE_COMPILATION") instead of nil, and then set that in the environment, but the ebib package tests (which set HOME=/nonexistent) still crashed (as had been originally reported here https://bugs.debian.org/1020191). The crash was in comp-trampoline-compile, and after a bit of investigation I wondered if it might be possible to fix the problem (at least approximately) by adding a no-native-compile clause to the comp-subr-trampoline-install guard like this: (defun comp-subr-trampoline-install (subr-name) "Make SUBR-NAME effectively advice-able when called from native code." (unless (or no-native-compilation ; this is new (null comp-enable-subr-trampolines) (memq subr-name native-comp-never-optimize-functions) (gethash subr-name comp-installed-trampolines-h)) ...)) That did allow the package tests to run successfully. So two questions: - As possibly just a short term, debian-specific fix, do those changes sound plausible, or are there other things we're missing (I might guess yes)? - Longer term, might it make sense to have some emacs-proper way to completely disable native compilation from the outside, either via environment variable, argument, or something else, for situations like this? [1] For add-on packages (e.g. elpa-*, etc.), we want to explicitly build the local .elc files (at least) during the install, but we don't want to write to any other unexpected locations. Thanks -- Rob Browning rlb @defaultvalue.org and @debian.org GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-28 1:04 Rob Browning @ 2022-09-28 11:02 ` Lars Ingebrigtsen 2022-09-28 11:35 ` Eli Zaretskii 2022-09-28 12:52 ` Andrea Corallo 2 siblings, 0 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-09-28 11:02 UTC (permalink / raw) To: Rob Browning; +Cc: emacs-devel, Andrea Corallo Rob Browning <rlb@defaultvalue.org> writes: > It's also been suggested by some users that they might like to have a > supported way to disable native compilation (even when it's the system > default), in order to avoid the unpredictable cpu and/or battery cost > sometimes. I think you can set native-comp-deferred-compilation-deny-list to (".*") to switch this stuff off. That's not the most logical interface, so perhaps there should also be a `inhibit-native-compilation' variable or the like? Andrea probably has some comments; added to the CCs. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-28 1:04 Rob Browning 2022-09-28 11:02 ` Lars Ingebrigtsen @ 2022-09-28 11:35 ` Eli Zaretskii 2022-10-12 20:22 ` Liliana Marie Prikler 2022-09-28 12:52 ` Andrea Corallo 2 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-09-28 11:35 UTC (permalink / raw) To: Rob Browning; +Cc: emacs-devel > From: Rob Browning <rlb@defaultvalue.org> > Date: Tue, 27 Sep 2022 20:04:24 -0500 > > > Perhaps relevant in other contexts, but at least in the context of > Debian work, we'd like to be able to suppress all native compilation in > some contexts, or more specifically all writes to HOME (or really, to > avoid any implicit file manipulations[1]). > > One key case is package builds and package installs (whether the emacs > packages themselves, or any of the various emacs add-on packages > (e.g. elpa-*)). > > For package builds, HOME is typically set to /nonexistent (or similar), > and for package installs we don't want the install commands (preinst, > postinst, etc.), which are running as root, to write anything to /root/. > > It's also been suggested by some users that they might like to have a > supported way to disable native compilation (even when it's the system > default), in order to avoid the unpredictable cpu and/or battery cost > sometimes. Of course, assuming we eventually augment debian's current > emacs infrastructure to generate system-level .eln files during (add-on) > package installs, in addition to the .elc files it currently produces, > much of that concern for debian and its derivatives might recede. > > In any case, I noticed that there is a no-native-compile variable, and > wondered if (in the short term), I might be able to craft a (possibly > debian-specific for now) way to disable native compilation across the > board. > > Just to see, I changed no-native-compile to default to > > (getenv "DEBIAN_EMACS_DISABLE_NATIVE_COMPILATION") > > instead of nil, and then set that in the environment, but the ebib > package tests (which set HOME=/nonexistent) still crashed (as had been > originally reported here https://bugs.debian.org/1020191). > > The crash was in comp-trampoline-compile, and after a bit of > investigation I wondered if it might be possible to fix the problem (at > least approximately) by adding a no-native-compile clause to the > comp-subr-trampoline-install guard like this: > > (defun comp-subr-trampoline-install (subr-name) > "Make SUBR-NAME effectively advice-able when called from native code." > (unless (or no-native-compilation ; this is new > (null comp-enable-subr-trampolines) > (memq subr-name native-comp-never-optimize-functions) > (gethash subr-name comp-installed-trampolines-h)) > ...)) > > That did allow the package tests to run successfully. > > So two questions: > > - As possibly just a short term, debian-specific fix, do those changes > sound plausible, or are there other things we're missing (I might > guess yes)? > > > - Longer term, might it make sense to have some emacs-proper way to > completely disable native compilation from the outside, either via > environment variable, argument, or something else, for situations > like this? I admit that I lost you somewhere near the beginning of your post. I think what's missing here is some more detailed explanation of the problems you are trying to solve. Without them, it is hard to give you intelligent answers. Here are some inline questions I'd like to be answered: > Perhaps relevant in other contexts, but at least in the context of > Debian work, we'd like to be able to suppress all native compilation in > some contexts, or more specifically all writes to HOME (or really, to > avoid any implicit file manipulations[1]). What do you mean by "writes to HOME"? Native-compilation doesn't write to HOME, it writes to directories in native-comp-eln-load-path. So basically, you already have a way to redirect stuff to wherever you want it. > For package builds, HOME is typically set to /nonexistent (or similar), > and for package installs we don't want the install commands (preinst, > postinst, etc.), which are running as root, to write anything to /root/. By "package installs" do you mean when end-users install a prebuilt Emacs? If so, how come that should ever need to write something outside of the installation tree? The Emacs "make install" procedure doesn't, so why do your "package installs" somehow do? Also, AFAIR installing an ELPA package doesn't produce any *.eln files until the first time Emacs 'load's their *.elc files. So why is this a problem? OTOH, installing a package from ELPA does write stuff to the user's directories, so how is the behavior of native-compilation different here? > It's also been suggested by some users that they might like to have a > supported way to disable native compilation (even when it's the system > default), in order to avoid the unpredictable cpu and/or battery cost > sometimes. What do you mean by "disable native compilation" here, and what exactly are the use cases for this? Are we talking about users who installed Emacs with some *.eln files, and from time to time load *.elc files that have no corresponding *.eln files? Or are we talking about something else? If the former, why do these users install Emacs with native-compilation to begin with? More generally, we never expected people who have Emacs with native compilation available to want to disable it. It made no sense to us during development of Emacs 28, and frankly, it still doesn't, at least to me. It is so easy to install Emacs without these capabilities and never look back that we thought providing a build-time option should be enough. (Well, except for MS-Windows users, where more is needed, but that's not your worry.) Therefore, if we need to discuss introduction of run-time features for disabling native-compilation, we need to have a broader view of the relevant use cases and user needs. This includes at least the following aspects that need to be discussed and clarified: . What does "disable native-compilation" mean, exactly? There are several possible interpretations of that. Do people want to prevent Emacs from looking for and loading the *.eln files, and use the *.elc files instead? Or do people only want to prevent Emacs from compiling new *.eln files? Or do people want that *.eln files be produced only by an explicit user command, not transparently and asynchronously? Or something else? . When and why would people want any of the above to be possible? . How and why would downstream distros want to support such capabilities? Armed with this information, we could then see which use cases are something we'd like to support in the core and how. Thanks. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-28 11:35 ` Eli Zaretskii @ 2022-10-12 20:22 ` Liliana Marie Prikler 2022-10-13 5:46 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Liliana Marie Prikler @ 2022-10-12 20:22 UTC (permalink / raw) To: Eli Zaretskii, Rob Browning; +Cc: emacs-devel Hi Eli, Am Mittwoch, dem 28.09.2022 um 14:35 +0300 schrieb Eli Zaretskii: > > What do you mean by "disable native compilation" here, and what > exactly are the use cases for this? Are we talking about users who > installed Emacs with some *.eln files, and from time to time load > *.elc files that have no corresponding *.eln files? Or are we > talking about something else? If the former, why do these users > install Emacs with native-compilation to begin with? > > More generally, we never expected people who have Emacs with native > compilation available to want to disable it. It made no sense to us > during development of Emacs 28, and frankly, it still doesn't, at > least to me. It is so easy to install Emacs without these > capabilities and never look back that we thought providing a > build-time option should be enough. (Well, except for MS-Windows > users, where more is needed, but that's not your worry.) > > Therefore, if we need to discuss introduction of run-time features > for disabling native-compilation, we need to have a broader view of > the relevant use cases and user needs. This includes at least the > following aspects that need to be discussed and clarified: > > . What does "disable native-compilation" mean, exactly? There are > several possible interpretations of that. Do people want to > prevent Emacs from looking for and loading the *.eln files, and > use the *.elc files instead? Or do people only want to prevent > Emacs from compiling new *.eln files? Or do people want that > *.eln files be produced only by an explicit user command, not > transparently and asynchronously? Or something else? > > . When and why would people want any of the above to be possible? > > . How and why would downstream distros want to support such > capabilities? In GNU Guix, we observe certain packages breaking when trying to compile them natively. While some of that is on us for messing up the package descriptions, one could also encounter genuine bugs in native compilation that one wants to work around by disabling it. (For added trouble, whether or not your package breaks could depend on the CPU due to being *native* compilation; thus the "no-native-compile" local variable is insufficient to address this generally.) In GNU Guix, we default to not compiling packages ahead-of-time, instead using a minimal emacs that can only do byte compilation for most packages. Users can however pretty easily switch to ahead-of-time compilation by swapping out the emacs package (using what Guix calls package transformations). This is also important because apart from the current Emacs we typically provide an emacs-next package for upcoming versions, as well as some other variants that the user might want to use instead. Again, we assume that users who want to opt in to native compilation do so via a transformation. In this context, the default of compiling everything that has hitherto only been byte-compiled is an ill-advised choice. Now, there is a chance that the user meant to do this anyway, but I think they rather a) use whatever the distro default is without caring either way or b) actually didn't want this package natively compiled. Due to some packages breaking, we had a lot of b) in our mailing lists in the past few days. As for loading, I think there could be a case made to restrict loading to certain directories managed by "trusted entities", but I think native-comp-eln-load-path already accomplishes this. If one wanted to disable loading altogether, I'm pretty sure one could set it to nil and be done (with perhaps the exception that deferred compilation breaks if there's no path to store binaries in). Thus, I think the issue is really only one of inadvertently launching deferred compilation when all the packages the user *expects* to be natively compiled are already ensured to be compiled ahead-of-time. I admit that this is taking a rather conservative stance, in which the users don't want to have anything natively compiled that they don't know of (and as a side result are happy to run that as bytecode or interpreted code). I hope this answers some of the questions raised. In short, we believe that users might want to control not only from where native shared libraries will be loaded, but also whether to generate them at runtime, and that (distro) package managers should enable them to compile all such shared libraries ahead of time – and better yet, in a reproducible manner, but this hits other bugs I do not want to discuss in detail at the moment. Cheers ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-12 20:22 ` Liliana Marie Prikler @ 2022-10-13 5:46 ` Eli Zaretskii 2022-10-13 19:20 ` Liliana Marie Prikler 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-13 5:46 UTC (permalink / raw) To: Liliana Marie Prikler; +Cc: rlb, emacs-devel > From: Liliana Marie Prikler <liliana.prikler@gmail.com> > Cc: emacs-devel@gnu.org > Date: Wed, 12 Oct 2022 22:22:46 +0200 > > In GNU Guix, we observe certain packages breaking when trying to > compile them natively. While some of that is on us for messing up the > package descriptions, one could also encounter genuine bugs in native > compilation that one wants to work around by disabling it. When you encounter bugs in native compilation, please report them to us, so we could fix them. As of now, we are not aware of any such bugs that were reported and haven't been fixed. So if you still have such problem, please report them ASAP. > (For added > trouble, whether or not your package breaks could depend on the CPU due > to being *native* compilation; thus the "no-native-compile" local > variable is insufficient to address this generally.) Why isn't it sufficient to use no-native-compile? It just means that on some architectures the corresponding file will be loaded as byte-compiled, and thus will be slightly slower (how much slower depends on the code, so if you are worried, my recommendation is first to measure the difference -- you might be surprised). In any case, if no-native-compile doesn't for some reason solve the problem, I don't understand how disabling native compilation will: the latter is a more blunt instrument than the former, e.g., it cannot be applied on the per-file resolution. > In GNU Guix, we default to not compiling packages ahead-of-time, > instead using a minimal emacs that can only do byte compilation for > most packages. Users can however pretty easily switch to ahead-of-time > compilation by swapping out the emacs package (using what Guix calls > package transformations). This is also important because apart from > the current Emacs we typically provide an emacs-next package for > upcoming versions, as well as some other variants that the user might > want to use instead. Again, we assume that users who want to opt in to > native compilation do so via a transformation. Sorry, I'm unfamiliar with this terminology. When you say "ahead-of-time compilation", do you mean native compilation of all the Lisp files before they are loaded for the first time? Or do you mean something else? And what is "swapping out the emacs package", and what is "package transformation"? > In this context, the default of compiling everything that has hitherto > only been byte-compiled is an ill-advised choice. Now, there is a > chance that the user meant to do this anyway, but I think they rather > a) use whatever the distro default is without caring either way or b) > actually didn't want this package natively compiled. Due to some > packages breaking, we had a lot of b) in our mailing lists in the past > few days. If a package is a single file or a small number of files, those users can add the no-native-compile cookies in those files. And again, disabling native compilation is a method that doesn't allow them fine-tuning anyway, so I fail to see how it could help them here. If the problem is real (and I don't yet see it is), we should perhaps discuss its details and invent some new method of disabling compilation with finer granularity. > As for loading, I think there could be a case made to restrict loading > to certain directories managed by "trusted entities", but I think > native-comp-eln-load-path already accomplishes this. Yes, but see below. > If one wanted to disable loading altogether, I'm pretty sure one > could set it to nil and be done (with perhaps the exception that > deferred compilation breaks if there's no path to store binaries > in). I don't think you can set native-comp-eln-load-path to nil. The last entry, the one which points to where the preloaded *.eln files live, must be there, or else Emacs will refuse to start. And at least one other directory should be there as well, because if and when some package advises some primitive, Emacs will need to generate and compile a trampoline .eln file. But yes, if users want to prevent loading from a certain directory, they can remove it from native-comp-eln-load-path, provided that the two necessary entries are still left in the list. > Thus, I think the issue is really only one of inadvertently launching > deferred compilation when all the packages the user *expects* to be > natively compiled are already ensured to be compiled ahead-of-time. This will not happen, so no need to prevent compilation in that case. If a .eln file already exists and is up-to-date, Emacs will not try to compile it again as part of loading the package. If some of the packages aren't natively compiled, against user's expectations, I still see no reason for users to want to disallow JIT compilation, except as a kind of surprise effect due to something they see for the first time. How is JIT compilation different from ahead-of-time compilation? There's nothing really dangerous or harmful in JIT compilation, so all it takes is some getting used to it. Educating your users to get used to it will go a long way towards eliminating the fears, because the dangers aren't real, they are largely imaginary. > I hope this answers some of the questions raised. In short, we believe > that users might want to control not only from where native shared > libraries will be loaded, but also whether to generate them at runtime, > and that (distro) package managers should enable them to compile all > such shared libraries ahead of time – and better yet, in a reproducible > manner, but this hits other bugs I do not want to discuss in detail at > the moment. Thanks for the explanations. I still think the reasons for disabling native compilation are rather weak at best, and the users' requests to allow that based on either bugs that need to be solved or surprise and fears that have no real basis. Moreover, disabling native compilation is a very blunt instrument that cannot be applied better than the existing ones, like no-native-compile (and a few others that we didn't mention; see the defcustom's in comp.el). ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-13 5:46 ` Eli Zaretskii @ 2022-10-13 19:20 ` Liliana Marie Prikler 2022-10-13 20:10 ` Stefan Monnier 2022-10-13 20:22 ` Eli Zaretskii 0 siblings, 2 replies; 343+ messages in thread From: Liliana Marie Prikler @ 2022-10-13 19:20 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rlb, emacs-devel Am Donnerstag, dem 13.10.2022 um 08:46 +0300 schrieb Eli Zaretskii: > > From: Liliana Marie Prikler <liliana.prikler@gmail.com> > > Cc: emacs-devel@gnu.org > > Date: Wed, 12 Oct 2022 22:22:46 +0200 > > > > In GNU Guix, we observe certain packages breaking when trying to > > compile them natively. While some of that is on us for messing up > > the package descriptions, one could also encounter genuine bugs in > > native compilation that one wants to work around by disabling it. > > When you encounter bugs in native compilation, please report them to > us, so we could fix them. As of now, we are not aware of any such > bugs that were reported and haven't been fixed. So if you still have > such problem, please report them ASAP. Of course, that's the intention, but this fix will only make it into the next Emacs release. Thus, if you're between releases, you still need a workaround. A particular candidate known to cause issues with the currently packaged 28.1 is [1]. > > (For added trouble, whether or not your package breaks could depend > > on the CPU due to being *native* compilation; thus the "no-native- > > compile" local variable is insufficient to address this generally.) > > Why isn't it sufficient to use no-native-compile? It just means that > on some architectures the corresponding file will be loaded as > byte-compiled, and thus will be slightly slower (how much slower > depends on the code, so if you are worried, my recommendation is > first to measure the difference -- you might be surprised). Because it'd require a distro-wide fix to address something that e.g. only happens on some AMD CPUs. Why AMD? Because we already had bugs in other languages, where AMD and Intel disagreed about floating points and that caused tests to fail (though admittedly that's not relevant. > In any case, if no-native-compile doesn't for some reason solve the > problem, I don't understand how disabling native compilation will: > the latter is a more blunt instrument than the former, e.g., it > cannot be applied on the per-file resolution. > > > In GNU Guix, we default to not compiling packages ahead-of-time, > > instead using a minimal emacs that can only do byte compilation for > > most packages. Users can however pretty easily switch to ahead-of- > > time compilation by swapping out the emacs package (using what Guix > > calls package transformations). This is also important because > > apart from the current Emacs we typically provide an emacs-next > > package for upcoming versions, as well as some other variants that > > the user might want to use instead. Again, we assume that users > > who want to opt in to native compilation do so via a > > transformation. > > Sorry, I'm unfamiliar with this terminology. When you say > "ahead-of-time compilation", do you mean native compilation of all > the Lisp files before they are loaded for the first time? Or do you > mean something else? Exactly, it's more or less the same as ahead-of-time compilation via package.el, which Emacs supports out of the box. > And what is "swapping out the emacs package", and what is "package > transformation"? Guix users can decide between an Emacs package that only does byte- compilation (emacs-minimal, the default) or native compilation (emacs). They can easily write this by using the command-line switch --with- input=emacs-minimal=emacs while building their Emacs package (e.g. emacs-dash for dash.el). For the record, we could probably also offer transformations, that annotate files with no-native-compile, but this would probably be a little harder to access as a user. What I'm instead hoping for would be a variable that can be set in early-init.el or similar. > > In this context, the default of compiling everything that has > > hitherto only been byte-compiled is an ill-advised choice. Now, > > there is a chance that the user meant to do this anyway, but I > > think they rather a) use whatever the distro default is without > > caring either way or b) actually didn't want this package natively > > compiled. Due to some packages breaking, we had a lot of b) in our > > mailing lists in the past few days. > > If a package is a single file or a small number of files, those users > can add the no-native-compile cookies in those files. This is not trivial in the case where the Elisp code is placed in system-managed storage and thus requires elevated privileges to modify (as is the default in most package managers, I assume). Of course, you can copy the file to your $HOME, but editing it with a broken Emacs is rather painful. We wouldn't want to resort to users using nano ;) > And again, disabling native compilation is a method that doesn't > allow them fine-tuning anyway, so I fail to see how it could help > them here. If the problem is real (and I don't yet see it is), we > should perhaps discuss its details and invent some new method of > disabling compilation with finer granularity. The granularity here is disabling compilation of anything that isn't already compiled – this makes it so that people can still use their Emacs for byte compilation by invoking it with "emacs -Q", they just won't compile anything that their package manager hasn't compiled for them. > > As for loading, I think there could be a case made to restrict > > loading to certain directories managed by "trusted entities", but I > > think native-comp-eln-load-path already accomplishes this. > > Yes, but see below. > > > If one wanted to disable loading altogether, I'm pretty sure one > > could set it to nil and be done (with perhaps the exception that > > deferred compilation breaks if there's no path to store binaries > > in). > > I don't think you can set native-comp-eln-load-path to nil. The last > entry, the one which points to where the preloaded *.eln files live, > must be there, or else Emacs will refuse to start. And at least one > other directory should be there as well, because if and when some > package advises some primitive, Emacs will need to generate and > compile a trampoline .eln file. But yes, if users want to prevent > loading from a certain directory, they can remove it from > native-comp-eln-load-path, provided that the two necessary entries > are still left in the list. I already found this annoying while implementing native compilation as part of our emacs-build-system (i.e. the build system used to compile Emacs packages), and I find it extra questionable that users on traditional distros, where they don't usually get to choose their Emacs, have no means of disabling this loading. Calling back to the earlier point on measuring performance, an easy way to measure performance between bytecode and native code (in a benchmark) would be to simply disable native code loading in one process. But here, it requires two separate builds of Emacs. > > Thus, I think the issue is really only one of inadvertently > > launching deferred compilation when all the packages the user > > *expects* to be natively compiled are already ensured to be > > compiled ahead-of-time. > > This will not happen, so no need to prevent compilation in that case. > If a .eln file already exists and is up-to-date, Emacs will not try > to compile it again as part of loading the package. If some of the > packages aren't natively compiled, against user's expectations, I > still see no reason for users to want to disallow JIT compilation, > except as a kind of surprise effect due to something they see for the > first time. How is JIT compilation different from ahead-of-time > compilation? There's nothing really dangerous or harmful in JIT > compilation, so all it takes is some getting used to it. Educating > your users to get used to it will go a long way towards eliminating > the fears, because the dangers aren't real, they are largely > imaginary. I agree for the most part that JIT compilation is harmless. However, there are some cases in which it isn't. For one, on slower hardware, this can take unreasonable times at startup. While bytecode performance on such machines might too be slow (but perhaps tolerable for the task), ahead-of-time compilation, perhaps with offloading, is preferable. For another, it can cause bugs like [2]. > > I hope this answers some of the questions raised. In short, we > > believe that users might want to control not only from where native > > shared libraries will be loaded, but also whether to generate them > > at runtime and that (distro) package managers should enable them to > > compile all such shared libraries ahead of time – and better yet, > > in a reproducible manner, but this hits other bugs I do not want to > > discuss in detail at the moment. > > Thanks for the explanations. I still think the reasons for disabling > native compilation are rather weak at best, and the users' requests > to allow that based on either bugs that need to be solved or surprise > and fears that have no real basis. Moreover, disabling native > compilation is a very blunt instrument that cannot be applied better > than the existing ones, like no-native-compile (and a few others that > we didn't mention; see the defcustom's in comp.el). Which defcustom? I fear that for all of its customizability, Emacs is currently lacking a good way of disabling native compilation outside of package management. I also tried setting no-native-compile globally, but it seems to only have an effect as a file-local variable. Cheers [1] https://github.com/DarwinAwardWinner/ido-ubiquitous [2] https://issues.guix.gnu.org/issue/57878 ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-13 19:20 ` Liliana Marie Prikler @ 2022-10-13 20:10 ` Stefan Monnier 2022-10-13 20:26 ` Eli Zaretskii 2022-10-15 17:06 ` Sean Whitton 2022-10-13 20:22 ` Eli Zaretskii 1 sibling, 2 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-13 20:10 UTC (permalink / raw) To: Liliana Marie Prikler; +Cc: Eli Zaretskii, rlb, emacs-devel > [2] https://issues.guix.gnu.org/issue/57878 Do we have a bug open on our (Emacs) side for this issue? Could it simply be that the async processes we launch to native-compile some file(s), themselves decide to launch more processes to native compile more files (or maybe even the same files)? Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-13 20:10 ` Stefan Monnier @ 2022-10-13 20:26 ` Eli Zaretskii 2022-10-13 20:57 ` Lars Ingebrigtsen 2022-10-15 17:06 ` Sean Whitton 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-13 20:26 UTC (permalink / raw) To: Stefan Monnier; +Cc: liliana.prikler, rlb, emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Eli Zaretskii <eliz@gnu.org>, rlb@defaultvalue.org, emacs-devel@gnu.org > Date: Thu, 13 Oct 2022 16:10:19 -0400 > > Could it simply be that the async processes we launch to native-compile > some file(s), themselves decide to launch more processes to native > compile more files (or maybe even the same files)? No, it should not happen, because async JIT compilation processes run Emacs in batch mode, and async compilation is disabled in batch mode (for this very reason). ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-13 20:26 ` Eli Zaretskii @ 2022-10-13 20:57 ` Lars Ingebrigtsen 2022-10-13 21:29 ` Lars Ingebrigtsen 2022-10-14 6:08 ` Eli Zaretskii 0 siblings, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-13 20:57 UTC (permalink / raw) To: Eli Zaretskii Cc: Stefan Monnier, liliana.prikler, rlb, emacs-devel, Andrea Corallo Eli Zaretskii <eliz@gnu.org> writes: > No, it should not happen, because async JIT compilation processes run > Emacs in batch mode, and async compilation is disabled in batch mode > (for this very reason). As we've covered many times recently -- yes, but no. .elc -> .eln generation is off, but trampolines are on, and comp.el forks an Emacs to generate those, too, I think? So if the forked Emacs also needs to generate the same trampoline, you have an infinite recursion fork bomb. You'd need a pretty tortured setup to get there, but the build process here seems really tweaked anyway. Let's see if I can reproduce the phenomenon... Ooops! Just crashed my laptop! That's fun. 😀 So don't follow this recipe: # cat /usr/local/share/emacs/29.0.50/site-lisp/site-start.el (fset 'yes-or-no-p 'y-or-n-p) rm -r ~/.emacs.d/eln-cache/; ./src/emacs -Q M-: (fset 'yes-or-no-p 'y-or-n-p) RET Dead laptop (except for the fan that spins up), so handle with caution. So we need a recursion blocker in the trampoline generation function. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-13 20:57 ` Lars Ingebrigtsen @ 2022-10-13 21:29 ` Lars Ingebrigtsen 2022-10-14 22:37 ` Andrea Corallo 2022-10-14 6:08 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-13 21:29 UTC (permalink / raw) To: Eli Zaretskii Cc: Stefan Monnier, liliana.prikler, rlb, emacs-devel, Andrea Corallo Lars Ingebrigtsen <larsi@gnus.org> writes: > So we need a recursion blocker in the trampoline generation function. Or -- since trampolines are really fast to generate and we need to do that synchronously anyway -- we could just avoid forking an Emacs do create the trampoline? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-13 21:29 ` Lars Ingebrigtsen @ 2022-10-14 22:37 ` Andrea Corallo 2022-10-15 3:09 ` Stefan Monnier 2022-10-15 9:24 ` Lars Ingebrigtsen 0 siblings, 2 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-14 22:37 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Eli Zaretskii, Stefan Monnier, liliana.prikler, rlb, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Lars Ingebrigtsen <larsi@gnus.org> writes: > >> So we need a recursion blocker in the trampoline generation function. > > Or -- since trampolines are really fast to generate and we need to do > that synchronously anyway -- we could just avoid forking an Emacs do > create the trampoline? We don't do that because libgccjit (well GCC really) leaks memory. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 22:37 ` Andrea Corallo @ 2022-10-15 3:09 ` Stefan Monnier 2022-10-15 15:06 ` Andrea Corallo 2022-10-15 9:24 ` Lars Ingebrigtsen 1 sibling, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-15 3:09 UTC (permalink / raw) To: Andrea Corallo Cc: Lars Ingebrigtsen, Eli Zaretskii, liliana.prikler, rlb, emacs-devel Andrea Corallo [2022-10-14 22:37:51] wrote: > Lars Ingebrigtsen <larsi@gnus.org> writes: >> Lars Ingebrigtsen <larsi@gnus.org> writes: >>> So we need a recursion blocker in the trampoline generation function. >> Or -- since trampolines are really fast to generate and we need to do >> that synchronously anyway -- we could just avoid forking an Emacs do >> create the trampoline? > We don't do that because libgccjit (well GCC really) leaks memory. But we can do that *in the forked Emacs* (this one will exit after compilation, so leakage is not an issue), so we do fork once but we avoid forking infinitely. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 3:09 ` Stefan Monnier @ 2022-10-15 15:06 ` Andrea Corallo 2022-10-15 16:16 ` Stefan Monnier 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-15 15:06 UTC (permalink / raw) To: Stefan Monnier Cc: Lars Ingebrigtsen, Eli Zaretskii, liliana.prikler, rlb, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > Andrea Corallo [2022-10-14 22:37:51] wrote: >> Lars Ingebrigtsen <larsi@gnus.org> writes: >>> Lars Ingebrigtsen <larsi@gnus.org> writes: >>>> So we need a recursion blocker in the trampoline generation function. >>> Or -- since trampolines are really fast to generate and we need to do >>> that synchronously anyway -- we could just avoid forking an Emacs do >>> create the trampoline? >> We don't do that because libgccjit (well GCC really) leaks memory. > > But we can do that *in the forked Emacs* (this one will exit after > compilation, so leakage is not an issue), so we do fork once but we avoid > forking infinitely. Hi Stefan, by *in the forked Emacs* you mean Emacs just after fork is called or after calling fork+exec to start a fresh session? Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 15:06 ` Andrea Corallo @ 2022-10-15 16:16 ` Stefan Monnier 0 siblings, 0 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-15 16:16 UTC (permalink / raw) To: Andrea Corallo Cc: Lars Ingebrigtsen, Eli Zaretskii, liliana.prikler, rlb, emacs-devel > by *in the forked Emacs* you mean Emacs just after fork is called or > after calling fork+exec to start a fresh session? I wrote "fork" sloppily to mean "spawn a new process". AFAIK we can't rely on `fork` literally because it's not really/efficiently/reliably available under Windows. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 22:37 ` Andrea Corallo 2022-10-15 3:09 ` Stefan Monnier @ 2022-10-15 9:24 ` Lars Ingebrigtsen 2022-10-15 15:02 ` Andrea Corallo 1 sibling, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-15 9:24 UTC (permalink / raw) To: Andrea Corallo Cc: Eli Zaretskii, Stefan Monnier, liliana.prikler, rlb, emacs-devel Andrea Corallo <akrl@sdf.org> writes: >>> So we need a recursion blocker in the trampoline generation function. >> >> Or -- since trampolines are really fast to generate and we need to do >> that synchronously anyway -- we could just avoid forking an Emacs do >> create the trampoline? > > We don't do that because libgccjit (well GCC really) leaks memory. Ah, darn. Then I guess we do have to fork, and then come up with some kind of way to tell that forked Emacs to not recurse any further? (Do you know whether libgccjit might stop leaking memory at some point?) ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 9:24 ` Lars Ingebrigtsen @ 2022-10-15 15:02 ` Andrea Corallo 2022-10-16 8:52 ` Lars Ingebrigtsen 2022-10-19 19:39 ` Andrea Corallo 0 siblings, 2 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-15 15:02 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Eli Zaretskii, Stefan Monnier, liliana.prikler, rlb, emacs-devel, alex.coplan Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >>>> So we need a recursion blocker in the trampoline generation function. >>> >>> Or -- since trampolines are really fast to generate and we need to do >>> that synchronously anyway -- we could just avoid forking an Emacs do >>> create the trampoline? >> >> We don't do that because libgccjit (well GCC really) leaks memory. > > Ah, darn. Then I guess we do have to fork, and then come up with some > kind of way to tell that forked Emacs to not recurse any further? Yes > (Do you know whether libgccjit might stop leaking memory at some point?) Yes, there must be a bug on bugzilla ATM I've no trace of. A colleague of mine did some investigation in the past and discovered IIRC that the GCC driver was the main source of the leakage. I'm Ccing him now hopefully he recalls/knows more than me. Also I think there was some fixing in this area but even in case is fixed completly now we'll have to cope with older version for a while. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 15:02 ` Andrea Corallo @ 2022-10-16 8:52 ` Lars Ingebrigtsen 2022-10-16 9:29 ` Liliana Marie Prikler 2022-10-16 13:45 ` Stefan Monnier 2022-10-19 19:39 ` Andrea Corallo 1 sibling, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-16 8:52 UTC (permalink / raw) To: Andrea Corallo Cc: Eli Zaretskii, Stefan Monnier, liliana.prikler, rlb, emacs-devel, alex.coplan Andrea Corallo <akrl@sdf.org> writes: > Also I think there was some fixing in this area but even in case is > fixed completly now we'll have to cope with older version for a while. Indeed. But hopefully at some point in the future, we'll be able to use libgccjit in-process, which should simplify things a bit, at least. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-16 8:52 ` Lars Ingebrigtsen @ 2022-10-16 9:29 ` Liliana Marie Prikler 2022-10-16 13:45 ` Stefan Monnier 1 sibling, 0 replies; 343+ messages in thread From: Liliana Marie Prikler @ 2022-10-16 9:29 UTC (permalink / raw) To: Lars Ingebrigtsen, Andrea Corallo Cc: Eli Zaretskii, Stefan Monnier, rlb, emacs-devel, alex.coplan Am Sonntag, dem 16.10.2022 um 10:52 +0200 schrieb Lars Ingebrigtsen: > Andrea Corallo <akrl@sdf.org> writes: > > > Also I think there was some fixing in this area but even in case is > > fixed completly now we'll have to cope with older version for a > > while. > > Indeed. > > But hopefully at some point in the future, we'll be able to use > libgccjit in-process, which should simplify things a bit, at least. Also, you could detect whether an in-process libgccjit is available at configure time. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-16 8:52 ` Lars Ingebrigtsen 2022-10-16 9:29 ` Liliana Marie Prikler @ 2022-10-16 13:45 ` Stefan Monnier 2022-10-16 13:47 ` Lars Ingebrigtsen 1 sibling, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-16 13:45 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Andrea Corallo, Eli Zaretskii, liliana.prikler, rlb, emacs-devel, alex.coplan > But hopefully at some point in the future, we'll be able to use > libgccjit in-process, which should simplify things a bit, at least. While an in-process calls to libgccjit make a lot of sense to build trampolines (since we have to wait until the code is generated anyway), but for all deferred compilations proceeding in a forked process seems like a much better option. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-16 13:45 ` Stefan Monnier @ 2022-10-16 13:47 ` Lars Ingebrigtsen 0 siblings, 0 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-16 13:47 UTC (permalink / raw) To: Stefan Monnier Cc: Andrea Corallo, Eli Zaretskii, liliana.prikler, rlb, emacs-devel, alex.coplan Stefan Monnier <monnier@iro.umontreal.ca> writes: > While an in-process calls to libgccjit make a lot of sense to build > trampolines (since we have to wait until the code is generated anyway), > but for all deferred compilations proceeding in a forked process seems > like a much better option. Indeed. But we don't always want the compilation to be deferred. See bug#58509. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 15:02 ` Andrea Corallo 2022-10-16 8:52 ` Lars Ingebrigtsen @ 2022-10-19 19:39 ` Andrea Corallo 1 sibling, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-19 19:39 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Eli Zaretskii, Stefan Monnier, liliana.prikler, rlb, emacs-devel, alex.coplan Andrea Corallo <akrl@sdf.org> writes: > Lars Ingebrigtsen <larsi@gnus.org> writes: > >> Andrea Corallo <akrl@sdf.org> writes: >> >>>>> So we need a recursion blocker in the trampoline generation function. >>>> >>>> Or -- since trampolines are really fast to generate and we need to do >>>> that synchronously anyway -- we could just avoid forking an Emacs do >>>> create the trampoline? >>> >>> We don't do that because libgccjit (well GCC really) leaks memory. >> >> Ah, darn. Then I guess we do have to fork, and then come up with some >> kind of way to tell that forked Emacs to not recurse any further? > > Yes > >> (Do you know whether libgccjit might stop leaking memory at some point?) > > Yes, there must be a bug on bugzilla ATM I've no trace of. A colleague > of mine did some investigation in the past and discovered IIRC that the > GCC driver was the main source of the leakage. I'm Ccing him now > hopefully he recalls/knows more than me. > > Also I think there was some fixing in this area but even in case is > fixed completly now we'll have to cope with older version for a while. Alex reached me out and pointed me to PR63854 [1]. Some fixing happened, but apparently ATM libgccjit is still, unless an external driver is used, leaking quite badly :/ Andrea [1] <https://gcc.gnu.org/bugzilla/show_bug.cgi?id=63854> ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-13 20:57 ` Lars Ingebrigtsen 2022-10-13 21:29 ` Lars Ingebrigtsen @ 2022-10-14 6:08 ` Eli Zaretskii 2022-10-14 23:20 ` Andrea Corallo 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-14 6:08 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: monnier, liliana.prikler, rlb, emacs-devel, akrl > From: Lars Ingebrigtsen <larsi@gnus.org> > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, liliana.prikler@gmail.com, > rlb@defaultvalue.org, emacs-devel@gnu.org, Andrea Corallo <akrl@sdf.org> > Date: Thu, 13 Oct 2022 22:57:51 +0200 > > Eli Zaretskii <eliz@gnu.org> writes: > > > No, it should not happen, because async JIT compilation processes run > > Emacs in batch mode, and async compilation is disabled in batch mode > > (for this very reason). > > As we've covered many times recently -- yes, but no. > > .elc -> .eln generation is off, but trampolines are on, and comp.el > forks an Emacs to generate those, too, I think? So if the forked Emacs > also needs to generate the same trampoline, you have an infinite > recursion fork bomb. Andrea, how can we prevent that? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 6:08 ` Eli Zaretskii @ 2022-10-14 23:20 ` Andrea Corallo 2022-10-15 3:14 ` Stefan Monnier ` (3 more replies) 0 siblings, 4 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-14 23:20 UTC (permalink / raw) To: Eli Zaretskii Cc: Lars Ingebrigtsen, monnier, liliana.prikler, rlb, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Lars Ingebrigtsen <larsi@gnus.org> >> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, liliana.prikler@gmail.com, >> rlb@defaultvalue.org, emacs-devel@gnu.org, Andrea Corallo <akrl@sdf.org> >> Date: Thu, 13 Oct 2022 22:57:51 +0200 >> >> Eli Zaretskii <eliz@gnu.org> writes: >> >> > No, it should not happen, because async JIT compilation processes run >> > Emacs in batch mode, and async compilation is disabled in batch mode >> > (for this very reason). >> >> As we've covered many times recently -- yes, but no. >> >> .elc -> .eln generation is off, but trampolines are on, and comp.el >> forks an Emacs to generate those, too, I think? So if the forked Emacs >> also needs to generate the same trampoline, you have an infinite >> recursion fork bomb. > > Andrea, how can we prevent that? Dumb question: can't we just run the spawned compilation processes with --no-site-file? Other option is to break circularity with an ad-hoc global variable set in the spawned process. I've a cooked patch for that but no energy left to test it tonight. If that's the preferred way I can test and push it tomorrow. Best Regards Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 23:20 ` Andrea Corallo @ 2022-10-15 3:14 ` Stefan Monnier 2022-10-15 6:40 ` Liliana Marie Prikler ` (2 more replies) 2022-10-15 7:04 ` Eli Zaretskii ` (2 subsequent siblings) 3 siblings, 3 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-15 3:14 UTC (permalink / raw) To: Andrea Corallo Cc: Eli Zaretskii, Lars Ingebrigtsen, liliana.prikler, rlb, emacs-devel > Dumb question: can't we just run the spawned compilation processes with > --no-site-file? For trampolines, I guess that should work since they shouldn't depend on local customizations. Of course, a tempting alternative is to resort to "binary hacking", i.e. compile *one* template-trampoline and then generate all every other trampoline by copying that template and patching the right "stuff" into it. That would save us from running the compiler to generate the trampolines (i.e. it would let us behave correctly on Windows even when GCC/libgccjit is not found at run time), but it would force us to write architecture-dependent code to patch the binary template. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 3:14 ` Stefan Monnier @ 2022-10-15 6:40 ` Liliana Marie Prikler 2022-10-15 7:14 ` Eli Zaretskii 2022-10-15 15:10 ` Andrea Corallo 2 siblings, 0 replies; 343+ messages in thread From: Liliana Marie Prikler @ 2022-10-15 6:40 UTC (permalink / raw) To: Stefan Monnier, Andrea Corallo Cc: Eli Zaretskii, Lars Ingebrigtsen, rlb, emacs-devel Am Freitag, dem 14.10.2022 um 23:14 -0400 schrieb Stefan Monnier: > > Of course, a tempting alternative is to resort to "binary hacking", > i.e. compile *one* template-trampoline and then generate all every > other trampoline by copying that template and patching the right > "stuff" into it. That would save us from running the > compiler to generate the trampolines (i.e. it would let us behave > correctly on Windows even when GCC/libgccjit is not found at run > time), but it would force us to write architecture-dependent code to > patch the binary template. Could table jumps work as an architecture-independent solution or is there something else I'm not considering atm? Cheers ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 3:14 ` Stefan Monnier 2022-10-15 6:40 ` Liliana Marie Prikler @ 2022-10-15 7:14 ` Eli Zaretskii 2022-10-15 14:00 ` Stefan Monnier 2022-10-15 15:10 ` Andrea Corallo 2 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-15 7:14 UTC (permalink / raw) To: Stefan Monnier; +Cc: akrl, larsi, liliana.prikler, rlb, emacs-devel > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Eli Zaretskii <eliz@gnu.org>, Lars Ingebrigtsen <larsi@gnus.org>, > liliana.prikler@gmail.com, rlb@defaultvalue.org, emacs-devel@gnu.org > Date: Fri, 14 Oct 2022 23:14:41 -0400 > > > Dumb question: can't we just run the spawned compilation processes with > > --no-site-file? > > For trampolines, I guess that should work since they shouldn't depend on > local customizations. But it will potentially cause other issues, because the environment in which the compiling Emacs sub-process run is different from that of the Emacs session which triggered the compilation? So we could have warnings and errors in the compilation process due to stuff being undefined etc.? Isn't this why we do read the init files in the async sub-process which performs the compilation? > Of course, a tempting alternative is to resort to > "binary hacking", i.e. compile *one* template-trampoline and then > generate all every other trampoline by copying that template and > patching the right "stuff" into it. That would save us from running the > compiler to generate the trampolines (i.e. it would let us behave > correctly on Windows even when GCC/libgccjit is not found at run time), > but it would force us to write architecture-dependent code to patch the > binary template. I don't think we should depend on architecture-dependent code. It would mean we don't support new platforms until someone writes such code for them, for starters. It's against our development tendencies for the past 10 years at least. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 7:14 ` Eli Zaretskii @ 2022-10-15 14:00 ` Stefan Monnier 2022-10-15 14:10 ` Lynn Winebarger 0 siblings, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-15 14:00 UTC (permalink / raw) To: Eli Zaretskii; +Cc: akrl, larsi, liliana.prikler, rlb, emacs-devel >> > Dumb question: can't we just run the spawned compilation processes with >> > --no-site-file? >> For trampolines, I guess that should work since they shouldn't depend on >> local customizations. > But it will potentially cause other issues, because the environment in > which the compiling Emacs sub-process run is different from that of > the Emacs session which triggered the compilation? So we could have > warnings and errors in the compilation process due to stuff being > undefined etc.? Isn't this why we do read the init files in the async > sub-process which performs the compilation? Yes, when compiling actual ELisp files. I'm talking about compiling the trampolines: that's different because it's just compiling a fixed chunk of code *we* write. >> Of course, a tempting alternative is to resort to >> "binary hacking", i.e. compile *one* template-trampoline and then >> generate all every other trampoline by copying that template and >> patching the right "stuff" into it. That would save us from running the >> compiler to generate the trampolines (i.e. it would let us behave >> correctly on Windows even when GCC/libgccjit is not found at run time), >> but it would force us to write architecture-dependent code to patch the >> binary template. > > I don't think we should depend on architecture-dependent code. It > would mean we don't support new platforms until someone writes such > code for them, for starters. It's against our development tendencies > for the past 10 years at least. I know, I know. From a technical point of view, it's The Right Thing to do, I think, tho: it will eliminate all the other issues surrounding trampolines (e.g. disk space to store trampolines, dependence on libgccjit for the creation of those trampolines, fork bombs) and will be much more efficient both in CPU time and memory use (we don't care too much, because trampolines are hopefully not too frequents, so we live with them being quite inefficient in the time to create them (and set them up, via dyn-linking) and in their memory use). But yes, the architecture-dependence is a serious problem, which is why we've so far avoided that solution. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 14:00 ` Stefan Monnier @ 2022-10-15 14:10 ` Lynn Winebarger 2022-10-15 14:29 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Lynn Winebarger @ 2022-10-15 14:10 UTC (permalink / raw) To: Stefan Monnier Cc: Eli Zaretskii, Andrea Corallo, Lars Ingebrigtsen, liliana.prikler, rlb, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1460 bytes --] On Sat, Oct 15, 2022, 10:02 AM Stefan Monnier <monnier@iro.umontreal.ca> wrote: > > > I don't think we should depend on architecture-dependent code. It > > would mean we don't support new platforms until someone writes such > > code for them, for starters. It's against our development tendencies > > for the past 10 years at least. > > I know, I know. From a technical point of view, it's The Right Thing to > do, I think, tho: it will eliminate all the other issues surrounding > trampolines (e.g. disk space to store trampolines, dependence on > libgccjit for the creation of those trampolines, fork bombs) and will be > much more efficient both in CPU time and memory use (we don't care too > much, because trampolines are hopefully not too frequents, so we live > with them being quite inefficient in the time to create them (and set > them up, via dyn-linking) and in their memory use). > > But yes, the architecture-dependence is a serious problem, which is why > we've so far avoided that solution. Didn't you, a couple of years ago, suggest generating generic trampolines (per function signature), compile those with gccjit, then just use them as needed rather than compiling separate trampolines for every subr? Presumably the system would still occasionally need to compile a trampoline for a new signature, but the generic trampoline would not be named so it couldn't cause an infinite recursion by being advised before being compiled. Lynn [-- Attachment #2: Type: text/html, Size: 1976 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 14:10 ` Lynn Winebarger @ 2022-10-15 14:29 ` Eli Zaretskii 2022-10-16 11:55 ` Lynn Winebarger 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-15 14:29 UTC (permalink / raw) To: Lynn Winebarger; +Cc: monnier, akrl, larsi, liliana.prikler, rlb, emacs-devel > From: Lynn Winebarger <owinebar@gmail.com> > Date: Sat, 15 Oct 2022 10:10:23 -0400 > Cc: Eli Zaretskii <eliz@gnu.org>, Andrea Corallo <akrl@sdf.org>, Lars Ingebrigtsen <larsi@gnus.org>, > liliana.prikler@gmail.com, rlb@defaultvalue.org, > emacs-devel <emacs-devel@gnu.org> > > Didn't you, a couple of years ago, suggest generating generic trampolines (per function signature), compile > those with gccjit, then just use them as needed rather than compiling separate trampolines for every subr? > Presumably the system would still occasionally need to compile a trampoline for a new signature, but the > generic trampoline would not be named so it couldn't cause an infinite recursion by being advised before > being compiled. If we want to compile all the trampolines AOT, that is already possible in Emacs 29, one just needs to actually do it when building Emacs. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 14:29 ` Eli Zaretskii @ 2022-10-16 11:55 ` Lynn Winebarger 2022-10-17 7:34 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lynn Winebarger @ 2022-10-16 11:55 UTC (permalink / raw) To: Eli Zaretskii Cc: Stefan Monnier, Andrea Corallo, Lars Ingebrigtsen, liliana.prikler, rlb, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1727 bytes --] On Sat, Oct 15, 2022, 10:29 AM Eli Zaretskii <eliz@gnu.org> wrote: > > From: Lynn Winebarger <owinebar@gmail.com> > > Date: Sat, 15 Oct 2022 10:10:23 -0400 > > Cc: Eli Zaretskii <eliz@gnu.org>, Andrea Corallo <akrl@sdf.org>, Lars > Ingebrigtsen <larsi@gnus.org>, > > liliana.prikler@gmail.com, rlb@defaultvalue.org, > > emacs-devel <emacs-devel@gnu.org> > > > > Didn't you, a couple of years ago, suggest generating generic > trampolines (per function signature), compile > > those with gccjit, then just use them as needed rather than compiling > separate trampolines for every subr? > > Presumably the system would still occasionally need to compile a > trampoline for a new signature, but the > > generic trampoline would not be named so it couldn't cause an infinite > recursion by being advised before > > being compiled. > > If we want to compile all the trampolines AOT, that is already > possible in Emacs 29, one just needs to actually do it when building > Emacs. > The question was about whether trampolines could be designed in such a way as to avoid invoking a separate compiler process without requiring architecture-dependent binary hacking on templates. I believe I was recalling one of Stefan M's suggestions for just this problem from a couple of years ago. Also, my understanding from this conversation is that trampolines may be required at run-time that aren't known at build-time. The issue is when the advised function is somehow called by the compiler while compiling it's own trampoline. Since the call may come from the advised version of some function used by the compiler, it's difficult to limit the risk without controlling both the dump and the startup/configuration files. Lynn [-- Attachment #2: Type: text/html, Size: 2961 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-16 11:55 ` Lynn Winebarger @ 2022-10-17 7:34 ` Andrea Corallo 2022-10-18 22:26 ` Lynn Winebarger 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-17 7:34 UTC (permalink / raw) To: Lynn Winebarger Cc: Eli Zaretskii, Stefan Monnier, Lars Ingebrigtsen, liliana.prikler, rlb, emacs-devel Lynn Winebarger <owinebar@gmail.com> writes: > On Sat, Oct 15, 2022, 10:29 AM Eli Zaretskii <eliz@gnu.org> wrote: > > > From: Lynn Winebarger <owinebar@gmail.com> > > Date: Sat, 15 Oct 2022 10:10:23 -0400 > > Cc: Eli Zaretskii <eliz@gnu.org>, Andrea Corallo <akrl@sdf.org>, Lars Ingebrigtsen <larsi@gnus.org>, > > liliana.prikler@gmail.com, rlb@defaultvalue.org, > > emacs-devel <emacs-devel@gnu.org> > > > > Didn't you, a couple of years ago, suggest generating generic trampolines (per function signature), compile > > those with gccjit, then just use them as needed rather than compiling separate trampolines for every subr? > > Presumably the system would still occasionally need to compile a trampoline for a new signature, but the > > generic trampoline would not be named so it couldn't cause an infinite recursion by being advised before > > being compiled. > > If we want to compile all the trampolines AOT, that is already > possible in Emacs 29, one just needs to actually do it when building > Emacs. > > The question was about whether trampolines could be designed in such a way as to avoid invoking a separate compiler > process without requiring architecture-dependent binary hacking on templates. > I believe I was recalling one of Stefan M's suggestions for just this problem from a couple of years ago. > Also, my understanding from this conversation is that trampolines may be required at run-time that aren't known at > build-time. The list of primitives is known at build time, so is the list of possibly needed trampolines. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-17 7:34 ` Andrea Corallo @ 2022-10-18 22:26 ` Lynn Winebarger 2022-10-19 19:33 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lynn Winebarger @ 2022-10-18 22:26 UTC (permalink / raw) To: Andrea Corallo Cc: Eli Zaretskii, Stefan Monnier, Lars Ingebrigtsen, liliana.prikler, rlb, emacs-devel [-- Attachment #1: Type: text/plain, Size: 629 bytes --] On Mon, Oct 17, 2022, 3:34 AM Andrea Corallo <akrl@sdf.org> wrote: > Lynn Winebarger <owinebar@gmail.com> writes: > > Also, my understanding from this conversation is that trampolines may be > required at run-time that aren't known at > > build-time. > > The list of primitives is known at build time, so is the list of > possibly needed trampolines. Dumb question: if trampolines are only used for primitives from the C source code, could they be generated as C code at build time? That would eliminate the issue entirely and make the semantics of advising primitives not depend on whether native compilation enabled. Lynn [-- Attachment #2: Type: text/html, Size: 1192 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-18 22:26 ` Lynn Winebarger @ 2022-10-19 19:33 ` Andrea Corallo 0 siblings, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-19 19:33 UTC (permalink / raw) To: Lynn Winebarger Cc: Eli Zaretskii, Stefan Monnier, Lars Ingebrigtsen, liliana.prikler, rlb, emacs-devel Lynn Winebarger <owinebar@gmail.com> writes: > On Mon, Oct 17, 2022, 3:34 AM Andrea Corallo <akrl@sdf.org> wrote: > > Lynn Winebarger <owinebar@gmail.com> writes: > > Also, my understanding from this conversation is that trampolines may be required at run-time that aren't known at > > build-time. > > The list of primitives is known at build time, so is the list of > possibly needed trampolines. > > Dumb question: if trampolines are only used for primitives from the C > source code, Trampolines are used for call sites to primitives in native code. They are the reason why native code is fully resilient to primitive redefinition (in contrast to C or bytecode). Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 3:14 ` Stefan Monnier 2022-10-15 6:40 ` Liliana Marie Prikler 2022-10-15 7:14 ` Eli Zaretskii @ 2022-10-15 15:10 ` Andrea Corallo 2022-10-15 16:17 ` Stefan Monnier 2 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-15 15:10 UTC (permalink / raw) To: Stefan Monnier Cc: Eli Zaretskii, Lars Ingebrigtsen, liliana.prikler, rlb, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: >> Dumb question: can't we just run the spawned compilation processes with >> --no-site-file? > > For trampolines, I guess that should work since they shouldn't depend on > local customizations. Of course, a tempting alternative is to resort to > "binary hacking", i.e. compile *one* template-trampoline and then > generate all every other trampoline by copying that template and > patching the right "stuff" into it. That would save us from running the > compiler to generate the trampolines (i.e. it would let us behave > correctly on Windows even when GCC/libgccjit is not found at run time), > but it would force us to write architecture-dependent code to patch the > binary template. I think writing and maintaining arch dependent code to fix and manipulate binaries is really a road we don't want to go down! (a terrified) Andrea :) ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 15:10 ` Andrea Corallo @ 2022-10-15 16:17 ` Stefan Monnier 2022-10-17 7:20 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Stefan Monnier @ 2022-10-15 16:17 UTC (permalink / raw) To: Andrea Corallo Cc: Eli Zaretskii, Lars Ingebrigtsen, liliana.prikler, rlb, emacs-devel Andrea Corallo [2022-10-15 15:10:06] wrote: > Stefan Monnier <monnier@iro.umontreal.ca> writes: >>> Dumb question: can't we just run the spawned compilation processes with >>> --no-site-file? >> >> For trampolines, I guess that should work since they shouldn't depend on >> local customizations. Of course, a tempting alternative is to resort to >> "binary hacking", i.e. compile *one* template-trampoline and then >> generate all every other trampoline by copying that template and >> patching the right "stuff" into it. That would save us from running the >> compiler to generate the trampolines (i.e. it would let us behave >> correctly on Windows even when GCC/libgccjit is not found at run time), >> but it would force us to write architecture-dependent code to patch the >> binary template. > > I think writing and maintaining arch dependent code to fix and > manipulate binaries is really a road we don't want to go down! > > (a terrified) Andrea :) I tend to agree. I just wish we could rely on some other tool to do that for us. Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 16:17 ` Stefan Monnier @ 2022-10-17 7:20 ` Andrea Corallo 0 siblings, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-17 7:20 UTC (permalink / raw) To: Stefan Monnier Cc: Eli Zaretskii, Lars Ingebrigtsen, liliana.prikler, rlb, emacs-devel Stefan Monnier <monnier@iro.umontreal.ca> writes: > Andrea Corallo [2022-10-15 15:10:06] wrote: >> Stefan Monnier <monnier@iro.umontreal.ca> writes: >>>> Dumb question: can't we just run the spawned compilation processes with >>>> --no-site-file? >>> >>> For trampolines, I guess that should work since they shouldn't depend on >>> local customizations. Of course, a tempting alternative is to resort to >>> "binary hacking", i.e. compile *one* template-trampoline and then >>> generate all every other trampoline by copying that template and >>> patching the right "stuff" into it. That would save us from running the >>> compiler to generate the trampolines (i.e. it would let us behave >>> correctly on Windows even when GCC/libgccjit is not found at run time), >>> but it would force us to write architecture-dependent code to patch the >>> binary template. >> >> I think writing and maintaining arch dependent code to fix and >> manipulate binaries is really a road we don't want to go down! >> >> (a terrified) Andrea :) > > I tend to agree. I just wish we could rely on some other tool to do > that for us. Yeah, I'm not aware of such a tool. I guess a similar path lead GCL to maintaining a (now very stale) in tree custom version of binutils. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 23:20 ` Andrea Corallo 2022-10-15 3:14 ` Stefan Monnier @ 2022-10-15 7:04 ` Eli Zaretskii 2022-10-15 15:19 ` Andrea Corallo 2022-10-15 9:25 ` Lars Ingebrigtsen 2022-10-15 14:18 ` Lynn Winebarger 3 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-15 7:04 UTC (permalink / raw) To: Andrea Corallo; +Cc: larsi, monnier, liliana.prikler, rlb, emacs-devel > From: Andrea Corallo <akrl@sdf.org> > Cc: Lars Ingebrigtsen <larsi@gnus.org>, monnier@iro.umontreal.ca, > liliana.prikler@gmail.com, rlb@defaultvalue.org, emacs-devel@gnu.org > Date: Fri, 14 Oct 2022 23:20:43 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> .elc -> .eln generation is off, but trampolines are on, and comp.el > >> forks an Emacs to generate those, too, I think? So if the forked Emacs > >> also needs to generate the same trampoline, you have an infinite > >> recursion fork bomb. > > > > Andrea, how can we prevent that? > > Dumb question: can't we just run the spawned compilation processes with > --no-site-file? I don't think I understand how --no-site-file could help. Are you assuming that such advises could only come from sit-init files? If so, I'm sure they could come from other sources, or what am I missing? > Other option is to break circularity with an ad-hoc global variable set > in the spawned process. I've a cooked patch for that but no energy left > to test it tonight. If that's the preferred way I can test and push it > tomorrow. Maybe it's preferable, but I'm not sure the idea of the change I get from your short description is what you really meant. Can you tell more about that? What ad-hoc variables did you have in mind, and how would we use them in this case to prevent infinite forking of sub-processes? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 7:04 ` Eli Zaretskii @ 2022-10-15 15:19 ` Andrea Corallo 2022-10-15 16:26 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-15 15:19 UTC (permalink / raw) To: Eli Zaretskii; +Cc: larsi, monnier, liliana.prikler, rlb, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Andrea Corallo <akrl@sdf.org> >> Cc: Lars Ingebrigtsen <larsi@gnus.org>, monnier@iro.umontreal.ca, >> liliana.prikler@gmail.com, rlb@defaultvalue.org, emacs-devel@gnu.org >> Date: Fri, 14 Oct 2022 23:20:43 +0000 >> >> Eli Zaretskii <eliz@gnu.org> writes: >> >> >> .elc -> .eln generation is off, but trampolines are on, and comp.el >> >> forks an Emacs to generate those, too, I think? So if the forked Emacs >> >> also needs to generate the same trampoline, you have an infinite >> >> recursion fork bomb. >> > >> > Andrea, how can we prevent that? >> >> Dumb question: can't we just run the spawned compilation processes with >> --no-site-file? > > I don't think I understand how --no-site-file could help. Are you > assuming that such advises could only come from sit-init files? Yes this was the assumption. > If > so, I'm sure they could come from other sources, or what am I missing? I think you are not missing much... as not said. >> Other option is to break circularity with an ad-hoc global variable set >> in the spawned process. I've a cooked patch for that but no energy left >> to test it tonight. If that's the preferred way I can test and push it >> tomorrow. > > Maybe it's preferable, but I'm not sure the idea of the change I get > from your short description is what you really meant. Can you tell > more about that? What ad-hoc variables did you have in mind, and how > would we use them in this case to prevent infinite forking of > sub-processes? We define a new variable say `comp-spawned-compilation' and we set it to t in all spawed compilation sub processes. Then when we need a trampoline: - If it's available we load it. - If it's not and `comp-spawned-compilation' is nil we compile it and load it, otherwise if `comp-spawned-compilation' is t we just do nothing (so we break circularity). WDYT? Bests Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 15:19 ` Andrea Corallo @ 2022-10-15 16:26 ` Eli Zaretskii 2022-10-15 17:19 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-15 16:26 UTC (permalink / raw) To: Andrea Corallo; +Cc: larsi, monnier, liliana.prikler, rlb, emacs-devel > From: Andrea Corallo <akrl@sdf.org> > Cc: larsi@gnus.org, monnier@iro.umontreal.ca, liliana.prikler@gmail.com, > rlb@defaultvalue.org, emacs-devel@gnu.org > Date: Sat, 15 Oct 2022 15:19:40 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> Other option is to break circularity with an ad-hoc global variable set > >> in the spawned process. I've a cooked patch for that but no energy left > >> to test it tonight. If that's the preferred way I can test and push it > >> tomorrow. > > > > Maybe it's preferable, but I'm not sure the idea of the change I get > > from your short description is what you really meant. Can you tell > > more about that? What ad-hoc variables did you have in mind, and how > > would we use them in this case to prevent infinite forking of > > sub-processes? > > We define a new variable say `comp-spawned-compilation' and we set it to > t in all spawed compilation sub processes. Then when we need a > trampoline: > > - If it's available we load it. > - If it's not and `comp-spawned-compilation' is nil we compile it and > load it, otherwise if `comp-spawned-compilation' is t we just do > nothing (so we break circularity). > > WDYT? Yes, this is more-or-less what I thought you had in mind. I guess setting of this new variable for the forked subprocesses will be via the command line? I think it's a fine solution, but either comp-spawned-compilation should be renamed to comp-no-spawn, or its value in the forked Emacs processes should be nil... ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 16:26 ` Eli Zaretskii @ 2022-10-15 17:19 ` Eli Zaretskii 2022-10-17 7:21 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-15 17:19 UTC (permalink / raw) To: akrl; +Cc: larsi, monnier, liliana.prikler, rlb, emacs-devel > Date: Sat, 15 Oct 2022 19:26:53 +0300 > From: Eli Zaretskii <eliz@gnu.org> > Cc: larsi@gnus.org, monnier@iro.umontreal.ca, > liliana.prikler@gmail.com, rlb@defaultvalue.org, emacs-devel@gnu.org > > I think it's a fine solution, but either comp-spawned-compilation > should be renamed to comp-no-spawn, or its value in the forked Emacs > processes should be nil... Btw, instead of inventing yet another variable, we could make native-comp-async-jobs-number accept one more value: -1, to mean "don't run async jobs at all". Your call. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 17:19 ` Eli Zaretskii @ 2022-10-17 7:21 ` Andrea Corallo 2022-10-18 8:52 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-17 7:21 UTC (permalink / raw) To: Eli Zaretskii; +Cc: larsi, monnier, liliana.prikler, rlb, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> Date: Sat, 15 Oct 2022 19:26:53 +0300 >> From: Eli Zaretskii <eliz@gnu.org> >> Cc: larsi@gnus.org, monnier@iro.umontreal.ca, >> liliana.prikler@gmail.com, rlb@defaultvalue.org, emacs-devel@gnu.org >> >> I think it's a fine solution, but either comp-spawned-compilation >> should be renamed to comp-no-spawn, or its value in the forked Emacs >> processes should be nil... > > Btw, instead of inventing yet another variable, we could make > native-comp-async-jobs-number accept one more value: -1, to mean > "don't run async jobs at all". > > Your call. SGTM thanks. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-17 7:21 ` Andrea Corallo @ 2022-10-18 8:52 ` Andrea Corallo 2022-10-18 11:27 ` Lars Ingebrigtsen 2022-10-18 13:55 ` Stefan Monnier 0 siblings, 2 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-18 8:52 UTC (permalink / raw) To: Eli Zaretskii; +Cc: larsi, monnier, liliana.prikler, rlb, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > Eli Zaretskii <eliz@gnu.org> writes: > >>> Date: Sat, 15 Oct 2022 19:26:53 +0300 >>> From: Eli Zaretskii <eliz@gnu.org> >>> Cc: larsi@gnus.org, monnier@iro.umontreal.ca, >>> liliana.prikler@gmail.com, rlb@defaultvalue.org, emacs-devel@gnu.org >>> >>> I think it's a fine solution, but either comp-spawned-compilation >>> should be renamed to comp-no-spawn, or its value in the forked Emacs >>> processes should be nil... >> >> Btw, instead of inventing yet another variable, we could make >> native-comp-async-jobs-number accept one more value: -1, to mean >> "don't run async jobs at all". >> >> Your call. > > SGTM thanks. > > Andrea Okay 1a8015b837 should do the job. I eventually opted for adding `comp-no-spawn' for now, seen the current "debated" situation on what we should offer as interface to the user, I thought was better for now to add a variable supposed to be used for internal use only. We can remove it and link the mechanism to `native-comp-async-jobs-number' afterwards if we decide for it. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-18 8:52 ` Andrea Corallo @ 2022-10-18 11:27 ` Lars Ingebrigtsen 2022-10-18 13:56 ` Andrea Corallo 2022-10-18 13:55 ` Stefan Monnier 1 sibling, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-18 11:27 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, monnier, liliana.prikler, rlb, emacs-devel Andrea Corallo <akrl@sdf.org> writes: >>>> I think it's a fine solution, but either comp-spawned-compilation >>>> should be renamed to comp-no-spawn, or its value in the forked Emacs >>>> processes should be nil... [...] > Okay 1a8015b837 should do the job. I eventually opted for adding > `comp-no-spawn' for now, seen the current "debated" situation on what we > should offer as interface to the user, I thought was better for now to > add a variable supposed to be used for internal use only. We can remove > it and link the mechanism to `native-comp-async-jobs-number' afterwards > if we decide for it. Is this supposed to fix the trampoline fork bomb? (There's a lot of stuff discussed in this thread...) If so, it doesn't, because the variable is set too late. (As Eli suggested in another thread, we should add a command line argument for this, and as I noted, that kind of fits with the stuff discussed in bug#58509.) ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-18 11:27 ` Lars Ingebrigtsen @ 2022-10-18 13:56 ` Andrea Corallo 2022-10-18 18:12 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-18 13:56 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Eli Zaretskii, monnier, liliana.prikler, rlb, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >>>>> I think it's a fine solution, but either comp-spawned-compilation >>>>> should be renamed to comp-no-spawn, or its value in the forked Emacs >>>>> processes should be nil... > > [...] > >> Okay 1a8015b837 should do the job. I eventually opted for adding >> `comp-no-spawn' for now, seen the current "debated" situation on what we >> should offer as interface to the user, I thought was better for now to >> add a variable supposed to be used for internal use only. We can remove >> it and link the mechanism to `native-comp-async-jobs-number' afterwards >> if we decide for it. > > Is this supposed to fix the trampoline fork bomb? (There's a lot of > stuff discussed in this thread...) If so, it doesn't, because the > variable is set too late. (As Eli suggested in another thread, we > should add a command line argument for this, and as I noted, that kind > of fits with the stuff discussed in bug#58509.) Right, I replied in bug#58509 with a patch that should fix the issue. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-18 13:56 ` Andrea Corallo @ 2022-10-18 18:12 ` Lars Ingebrigtsen 0 siblings, 0 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-18 18:12 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, monnier, liliana.prikler, rlb, emacs-devel Andrea Corallo <akrl@sdf.org> writes: >>> Okay 1a8015b837 should do the job. I eventually opted for adding [...] >> Is this supposed to fix the trampoline fork bomb? (There's a lot of >> stuff discussed in this thread...) If so, it doesn't, because the >> variable is set too late. (As Eli suggested in another thread, we >> should add a command line argument for this, and as I noted, that kind >> of fits with the stuff discussed in bug#58509.) > > Right, I replied in bug#58509 with a patch that should fix the issue. Does this mean that 1a8015b837 was not intended to address this issue? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-18 8:52 ` Andrea Corallo 2022-10-18 11:27 ` Lars Ingebrigtsen @ 2022-10-18 13:55 ` Stefan Monnier 1 sibling, 0 replies; 343+ messages in thread From: Stefan Monnier @ 2022-10-18 13:55 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, larsi, liliana.prikler, rlb, emacs-devel > Okay 1a8015b837 should do the job. I eventually opted for adding > `comp-no-spawn' for now, seen the current "debated" situation on what we > should offer as interface to the user, I thought was better for now to > add a variable supposed to be used for internal use only. We can remove Please use "--" in identifiers when they are "for internal use only". Stefan ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 23:20 ` Andrea Corallo 2022-10-15 3:14 ` Stefan Monnier 2022-10-15 7:04 ` Eli Zaretskii @ 2022-10-15 9:25 ` Lars Ingebrigtsen 2022-10-15 15:20 ` Andrea Corallo 2022-10-15 14:18 ` Lynn Winebarger 3 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-15 9:25 UTC (permalink / raw) To: Andrea Corallo; +Cc: Eli Zaretskii, monnier, liliana.prikler, rlb, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > Dumb question: can't we just run the spawned compilation processes with > --no-site-file? Having the `fset' in the site-file was just an example -- there must be fifty ways to put an `fset' into the Emacs startup sequence. > Other option is to break circularity with an ad-hoc global variable set > in the spawned process. I've a cooked patch for that but no energy left > to test it tonight. If that's the preferred way I can test and push it > tomorrow. Would this be an environment variable or an elisp variable? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 9:25 ` Lars Ingebrigtsen @ 2022-10-15 15:20 ` Andrea Corallo 0 siblings, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-15 15:20 UTC (permalink / raw) To: Lars Ingebrigtsen Cc: Eli Zaretskii, monnier, liliana.prikler, rlb, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Andrea Corallo <akrl@sdf.org> writes: > >> Dumb question: can't we just run the spawned compilation processes with >> --no-site-file? > > Having the `fset' in the site-file was just an example -- there must be > fifty ways to put an `fset' into the Emacs startup sequence. I see >> Other option is to break circularity with an ad-hoc global variable set >> in the spawned process. I've a cooked patch for that but no energy left >> to test it tonight. If that's the preferred way I can test and push it >> tomorrow. > > Would this be an environment variable or an elisp variable? Elisp, I've just posted a reply to Eli with some derscription. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 23:20 ` Andrea Corallo ` (2 preceding siblings ...) 2022-10-15 9:25 ` Lars Ingebrigtsen @ 2022-10-15 14:18 ` Lynn Winebarger 3 siblings, 0 replies; 343+ messages in thread From: Lynn Winebarger @ 2022-10-15 14:18 UTC (permalink / raw) To: Andrea Corallo Cc: Eli Zaretskii, Lars Ingebrigtsen, Stefan Monnier, liliana.prikler, rlb, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1592 bytes --] On Fri, Oct 14, 2022, 7:22 PM Andrea Corallo <akrl@sdf.org> wrote: > Eli Zaretskii <eliz@gnu.org> writes: > > >> From: Lars Ingebrigtsen <larsi@gnus.org> > >> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, > liliana.prikler@gmail.com, > >> rlb@defaultvalue.org, emacs-devel@gnu.org, Andrea Corallo < > akrl@sdf.org> > >> Date: Thu, 13 Oct 2022 22:57:51 +0200 > >> > >> Eli Zaretskii <eliz@gnu.org> writes: > >> > >> > No, it should not happen, because async JIT compilation processes run > >> > Emacs in batch mode, and async compilation is disabled in batch mode > >> > (for this very reason). > >> > >> As we've covered many times recently -- yes, but no. > >> > >> .elc -> .eln generation is off, but trampolines are on, and comp.el > >> forks an Emacs to generate those, too, I think? So if the forked Emacs > >> also needs to generate the same trampoline, you have an infinite > >> recursion fork bomb. > > > > Andrea, how can we prevent that? > > Dumb question: can't we just run the spawned compilation processes with > --no-site-file? > > Other option is to break circularity with an ad-hoc global variable set > in the spawned process. I've a cooked patch for that but no energy left > to test it tonight. If that's the preferred way I can test and push it > tomorrow. The cleanest way would be to prepare a dump file specifically for compiling that is known to work, then use that for the async compiler job with any required settings passed in the command line. This would make the system robust when the system dump file is not limited to just the standard build. Lynn [-- Attachment #2: Type: text/html, Size: 2808 bytes --] ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-13 20:10 ` Stefan Monnier 2022-10-13 20:26 ` Eli Zaretskii @ 2022-10-15 17:06 ` Sean Whitton 2022-10-15 17:12 ` Eli Zaretskii 1 sibling, 1 reply; 343+ messages in thread From: Sean Whitton @ 2022-10-15 17:06 UTC (permalink / raw) To: Stefan Monnier; +Cc: Liliana Marie Prikler, Eli Zaretskii, rlb, emacs-devel Hello, On Thu 13 Oct 2022 at 04:10PM -04, Stefan Monnier wrote: >> [2] https://issues.guix.gnu.org/issue/57878 > > Do we have a bug open on our (Emacs) side for this issue? > Could it simply be that the async processes we launch to native-compile > some file(s), themselves decide to launch more processes to native > compile more files (or maybe even the same files)? Just for reference, Debian's filings for this issue: https://bugs.debian.org/1017817 https://bugs.debian.org/1017845 -- Sean Whitton ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 17:06 ` Sean Whitton @ 2022-10-15 17:12 ` Eli Zaretskii 2022-10-15 17:36 ` Sean Whitton 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-15 17:12 UTC (permalink / raw) To: Sean Whitton; +Cc: monnier, liliana.prikler, rlb, emacs-devel > From: Sean Whitton <spwhitton@spwhitton.name> > Cc: Liliana Marie Prikler <liliana.prikler@gmail.com>, Eli Zaretskii > <eliz@gnu.org>, rlb@defaultvalue.org, emacs-devel@gnu.org > Date: Sat, 15 Oct 2022 10:06:03 -0700 > > >> [2] https://issues.guix.gnu.org/issue/57878 > > > > Do we have a bug open on our (Emacs) side for this issue? > > Could it simply be that the async processes we launch to native-compile > > some file(s), themselves decide to launch more processes to native > > compile more files (or maybe even the same files)? > > Just for reference, Debian's filings for this issue: Thanks. > https://bugs.debian.org/1017817 This seems to be about byte-compilation? > https://bugs.debian.org/1017845 This is about trampolines indeed, and I believe we will have a solution VSN. P.S. I wish such problems didn't have to wait since August before they are communicated to us. If we were told back then about the problem, we could have it solved in Emacs 28.2... ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 17:12 ` Eli Zaretskii @ 2022-10-15 17:36 ` Sean Whitton 0 siblings, 0 replies; 343+ messages in thread From: Sean Whitton @ 2022-10-15 17:36 UTC (permalink / raw) To: Eli Zaretskii; +Cc: monnier, liliana.prikler, rlb, emacs-devel Hello, On Sat 15 Oct 2022 at 08:12PM +03, Eli Zaretskii wrote: > This seems to be about byte-compilation? I think it's mistitled, but nvm. >> https://bugs.debian.org/1017845 > > This is about trampolines indeed, and I believe we will have a > solution VSN. Nice. > P.S. I wish such problems didn't have to wait since August before they > are communicated to us. If we were told back then about the problem, > we could have it solved in Emacs 28.2... Yes, sorry about that. -- Sean Whitton ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-13 19:20 ` Liliana Marie Prikler 2022-10-13 20:10 ` Stefan Monnier @ 2022-10-13 20:22 ` Eli Zaretskii 2022-10-14 19:46 ` Liliana Marie Prikler 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-13 20:22 UTC (permalink / raw) To: Liliana Marie Prikler; +Cc: rlb, emacs-devel > From: Liliana Marie Prikler <liliana.prikler@gmail.com> > Cc: rlb@defaultvalue.org, emacs-devel@gnu.org > Date: Thu, 13 Oct 2022 21:20:13 +0200 > > > When you encounter bugs in native compilation, please report them to > > us, so we could fix them. As of now, we are not aware of any such > > bugs that were reported and haven't been fixed. So if you still have > > such problem, please report them ASAP. > Of course, that's the intention, but this fix will only make it into > the next Emacs release. Thus, if you're between releases, you still > need a workaround. If the fix is urgent, why can't you patch the sources when you prepare your distribution? > A particular candidate known to cause issues with the currently > packaged 28.1 is [1]. Where's the description of the actual problem with natively compiling that package? And would you please submit a bug report with the details, if you know them? > > Why isn't it sufficient to use no-native-compile? It just means that > > on some architectures the corresponding file will be loaded as > > byte-compiled, and thus will be slightly slower (how much slower > > depends on the code, so if you are worried, my recommendation is > > first to measure the difference -- you might be surprised). > Because it'd require a distro-wide fix to address something that e.g. > only happens on some AMD CPUs. I'm asking why doing so is a problem? Did you measure the effect on performance and found it to be unacceptable in some cases? > > > In GNU Guix, we default to not compiling packages ahead-of-time, > > > instead using a minimal emacs that can only do byte compilation for > > > most packages. Users can however pretty easily switch to ahead-of- > > > time compilation by swapping out the emacs package (using what Guix > > > calls package transformations). This is also important because > > > apart from the current Emacs we typically provide an emacs-next > > > package for upcoming versions, as well as some other variants that > > > the user might want to use instead. Again, we assume that users > > > who want to opt in to native compilation do so via a > > > transformation. > > > > Sorry, I'm unfamiliar with this terminology. When you say > > "ahead-of-time compilation", do you mean native compilation of all > > the Lisp files before they are loaded for the first time? Or do you > > mean something else? > Exactly, it's more or less the same as ahead-of-time compilation via > package.el, which Emacs supports out of the box. > > > And what is "swapping out the emacs package", and what is "package > > transformation"? > Guix users can decide between an Emacs package that only does byte- > compilation (emacs-minimal, the default) or native compilation (emacs). > They can easily write this by using the command-line switch --with- > input=emacs-minimal=emacs while building their Emacs package (e.g. > emacs-dash for dash.el). OK, so why is this relevant to the issue of disabling? Those who choose ahead-of-time compilation will never see async JIT compilation, and those who selected not to do ahead-of-time will naturally see JIT compilation, as they've chosen. What is the problem here? > > > In this context, the default of compiling everything that has > > > hitherto only been byte-compiled is an ill-advised choice. Now, > > > there is a chance that the user meant to do this anyway, but I > > > think they rather a) use whatever the distro default is without > > > caring either way or b) actually didn't want this package natively > > > compiled. Due to some packages breaking, we had a lot of b) in our > > > mailing lists in the past few days. > > > > If a package is a single file or a small number of files, those users > > can add the no-native-compile cookies in those files. > This is not trivial in the case where the Elisp code is placed in > system-managed storage and thus requires elevated privileges to modify > (as is the default in most package managers, I assume). Of course, you > can copy the file to your $HOME, but editing it with a broken Emacs is > rather painful. Using broken packages is always painful, and native compilation doesn't change that. Packages provided by a distribution and installed into directories where users cannot easily write should be well tested by the distributor. IOW, I don't understand why the upstream project should be held responsible for packages distributed by downstream distributions without testing them well enough to let users use them without pain, and why should the upstream project introduce options to support such "broken" distros. It isn't fair to expect us to solve such problems, because they should be solved elsewhere. > > And again, disabling native compilation is a method that doesn't > > allow them fine-tuning anyway, so I fail to see how it could help > > them here. If the problem is real (and I don't yet see it is), we > > should perhaps discuss its details and invent some new method of > > disabling compilation with finer granularity. > The granularity here is disabling compilation of anything that isn't > already compiled – this makes it so that people can still use their > Emacs for byte compilation by invoking it with "emacs -Q", they just > won't compile anything that their package manager hasn't compiled for > them. That's quite a blunt weapon. Why not tell them to stay with Emacs 27, until the problems are solved, or install Emacs 28 without native compilation? They are similarly drastic solutions, but they are already available and will definitely work. > > I don't think you can set native-comp-eln-load-path to nil. The last > > entry, the one which points to where the preloaded *.eln files live, > > must be there, or else Emacs will refuse to start. And at least one > > other directory should be there as well, because if and when some > > package advises some primitive, Emacs will need to generate and > > compile a trampoline .eln file. But yes, if users want to prevent > > loading from a certain directory, they can remove it from > > native-comp-eln-load-path, provided that the two necessary entries > > are still left in the list. > I already found this annoying while implementing native compilation as > part of our emacs-build-system (i.e. the build system used to compile > Emacs packages), and I find it extra questionable that users on > traditional distros, where they don't usually get to choose their > Emacs, have no means of disabling this loading. You mean, you find the loading of preloaded *.eln files at startup annoying? Then you should know that this is the best solution we found for dumping Emacs with natively-compiled preloaded code. If you know of a better solution that doesn't suffer from any fatal issues we found with the alternatives, please suggest such solutions, and we will definitely consider them. And again, if Emacs with natively-compilation is annoying, by all means offer your users an Emacs built without natively-compilation. This is supported and will continue to be supported for the observable future. > Calling back to the earlier point on measuring performance, an easy > way to measure performance between bytecode and native code (in a > benchmark) would be to simply disable native code loading in one > process. But here, it requires two separate builds of Emacs. As I told earlier, disabling loading of native code made no sense to us while Emacs 28 was in development; it still doesn't. Either one wants native-compilation, or one doesn't. Making Emacs code more complicated and harder to maintain due to features that make no sense to us is a non-starter. I see no problem with having to use a separate build, since building a release tarball takes a minute or so on a modern system. And distros should definitely have a build without native-compilation on offer, for a variety of valid reasons. > I agree for the most part that JIT compilation is harmless. However, > there are some cases in which it isn't. For one, on slower hardware, > this can take unreasonable times at startup. System that are that slow should not install Emacs with native-compilation, plain and simple. My suggestion is to clearly explain this in your README files, so that users could decide which build is for them, exactly like they decide whether they want a GTK build or a Lucid build (or any other of the configurations you offer). > While bytecode performance on such machines might too be slow (but > perhaps tolerable for the task), ahead-of-time compilation, perhaps > with offloading, is preferable. I recommend against this, because it is impossible to rely on AOT installations to never compile at run time. Users cannot rely on that, and should be advised accordingly. > For another, it can cause bugs like [2]. That bug by itself (the cause of massive launching of async subprocesses) was never explored or described in that thread? It seems like the discussion switched to looking for ways of disabling native-compilation right away, without a good understand of what was happening. Or did I mis something? Async compilation by default never launches more subprocesses than half the execution units of the CPU, so what is described there should be carefully investigated and the findings described. The other problem in that discussion, with warnings during async JIT compilation is well-known, was reported several times, and the culprit is always in the 3rd-part packages being compiled, which should be fixed. In any case, those are just warnings in almost all cases, so their only adverse effect is annoyance (that can be suppressed by clicking the button in the message). Again, I see no reason to blame the upstream project for these issues. They should be solved by the offending 3rd-party packages, and the distro should ideally uncover and fix them before they get to users (I presume that you build and compile the add-on packages you offer?). > > Thanks for the explanations. I still think the reasons for disabling > > native compilation are rather weak at best, and the users' requests > > to allow that based on either bugs that need to be solved or surprise > > and fears that have no real basis. Moreover, disabling native > > compilation is a very blunt instrument that cannot be applied better > > than the existing ones, like no-native-compile (and a few others that > > we didn't mention; see the defcustom's in comp.el). > Which defcustom? Begin with those described in the ELisp manual, in the "Native-Compilation Variables" node. And my recommendation is to review _all_ of the defcustoms in comp.el > I fear that for all of its customizability, Emacs is > currently lacking a good way of disabling native compilation outside of > package management. Yes, because, as mentioned, this makes no sense. And disabling native-compilation completely is currently technically impossible (for te same reason: Emacs wasn't designed to support that because it didn't and still doesn't seem needed). > I also tried setting no-native-compile globally, but it seems to > only have an effect as a file-local variable. Yes, as designed. This variable is the equivalent of no-byte-compile, and works very similarly. To summarize: native compilation in a build which supports it is ubiquitous, and is not designed to be disabled except by no-native-compile on a file by file level. If a more general disabling is needed for some reason, users should simply use a build without native-compilation. It's the same as various toolkit builds: if the toolkit is broken or doesn't fit the user's needs, those users should install a build with a different toolkit. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-13 20:22 ` Eli Zaretskii @ 2022-10-14 19:46 ` Liliana Marie Prikler 2022-10-15 8:51 ` Eli Zaretskii 2022-10-15 9:33 ` Lars Ingebrigtsen 0 siblings, 2 replies; 343+ messages in thread From: Liliana Marie Prikler @ 2022-10-14 19:46 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rlb, emacs-devel Am Donnerstag, dem 13.10.2022 um 23:22 +0300 schrieb Eli Zaretskii: > > From: Liliana Marie Prikler <liliana.prikler@gmail.com> > > Cc: rlb@defaultvalue.org, emacs-devel@gnu.org > > Date: Thu, 13 Oct 2022 21:20:13 +0200 > > > > > When you encounter bugs in native compilation, please report them > > > to us, so we could fix them. As of now, we are not aware of any > > > such bugs that were reported and haven't been fixed. So if you > > > still have such problem, please report them ASAP. > > Of course, that's the intention, but this fix will only make it > > into the next Emacs release. Thus, if you're between releases, you > > still need a workaround. > > If the fix is urgent, why can't you patch the sources when you > prepare your distribution? Guix prides itself in being a package manager that can work around many failures (even as the proper workaround to bugs is discussed in mailing lists). The fact, that the solutions to this issue is "compile 28.1 without native-comp" or "use Emacs 27" does not reflect that particularly well. > > A particular candidate known to cause issues with the currently > > packaged 28.1 is [1]. > > Where's the description of the actual problem with natively compiling > that package? And would you please submit a bug report with the > details, if you know them? I am not personally affected, so I can't. I could direct people to the Emacs mailing lists, but it seems people in other threads have already started debugging. Do you still wish me to do so? > > > Why isn't it sufficient to use no-native-compile? It just means > > > that on some architectures the corresponding file will be loaded > > > as byte-compiled, and thus will be slightly slower (how much > > > slower depends on the code, so if you are worried, my > > > recommendation is first to measure the difference -- you might be > > > surprised). > > Because it'd require a distro-wide fix to address something that > > e.g. only happens on some AMD CPUs. > > I'm asking why doing so is a problem? Did you measure the effect on > performance and found it to be unacceptable in some cases? Isn't performance one of the main reasons to use native compilation? Note that I am talking in hypotheticals here when mentioning the AMD thing, i.e. we could very well imagine a performance-critical Emacs package having a native-compilation bug (I imagine those to be particularly likely for those trailing unreleased Emacs versions, though thankfully I don't think we've encountered one so far.) > > > > In GNU Guix, we default to not compiling packages ahead-of- > > > > time, instead using a minimal emacs that can only do byte > > > > compilation for most packages. Users can however pretty easily > > > > switch to ahead-of-time compilation by swapping out the emacs > > > > package (using what Guix calls package transformations). This > > > > is also important because apart from the current Emacs we > > > > typically provide an emacs-next package for upcoming versions, > > > > as well as some other variants that the user might want to use > > > > instead. Again, we assume that users who want to opt in to > > > > native compilation do so via a transformation. > > > > > > Sorry, I'm unfamiliar with this terminology. When you say > > > "ahead-of-time compilation", do you mean native compilation of > > > all the Lisp files before they are loaded for the first time? Or > > > do you mean something else? > > Exactly, it's more or less the same as ahead-of-time compilation > > via package.el, which Emacs supports out of the box. > > > > > And what is "swapping out the emacs package", and what is > > > "package transformation"? > > Guix users can decide between an Emacs package that only does byte- > > compilation (emacs-minimal, the default) or native compilation > > (emacs). > > They can easily write this by using the command-line switch --with- > > input=emacs-minimal=emacs while building their Emacs package (e.g. > > emacs-dash for dash.el). > > OK, so why is this relevant to the issue of disabling? Those who > choose ahead-of-time compilation will never see async JIT > compilation, and those who selected not to do ahead-of-time will > naturally see JIT compilation, as they've chosen. What is the > problem here? The problem is that I can't meaningfully choose the "I don't want JIT for stuff I haven't AOT'd" option, especially not combined with "but I do want to load what I have AOT'd". > > > > In this context, the default of compiling everything that has > > > > hitherto only been byte-compiled is an ill-advised choice. > > > > Now, there is a chance that the user meant to do this anyway, > > > > but I think they rather a) use whatever the distro default is > > > > without caring either way or b) actually didn't want this > > > > package natively compiled. Due to some packages breaking, we > > > > had a lot of b) in our mailing lists in the past few days. > > > > > > If a package is a single file or a small number of files, those > > > users can add the no-native-compile cookies in those files. > > This is not trivial in the case where the Elisp code is placed in > > system-managed storage and thus requires elevated privileges to > > modify (as is the default in most package managers, I assume). Of > > course, you can copy the file to your $HOME, but editing it with a > > broken Emacs is rather painful. > > Using broken packages is always painful, and native compilation > doesn't change that. Using broken packages normally doesn't result in the OOM killer firing off. > Packages provided by a distribution and installed into directories > where users cannot easily write should be well tested by the > distributor. I think you're underestimating the number of breakages that can happen in a rolling release model. Not every distro is as stable as Debian, but the joke's still on you because despite Debian's hard requirements, they still ended up encountering this bug. > IOW, I don't understand why the upstream project should be held > responsible for packages distributed by downstream distributions > without testing them well enough to let users use them without pain, > and why should the upstream project introduce options to support such > "broken" distros. It isn't fair to expect us to solve such problems, > because they should be solved elsewhere. There are limits to testing. When I added native comp to Guix, I gave folks in the mailing lists a heads up to try their own configuration and report bugs. But people don't always read the mailing lists and therefore aren't aware of upcoming changes that may break their system, and I personally can't test every Emacs configuration in existence. > > > And again, disabling native compilation is a method that doesn't > > > allow them fine-tuning anyway, so I fail to see how it could help > > > them here. If the problem is real (and I don't yet see it is), > > > we should perhaps discuss its details and invent some new method > > > of disabling compilation with finer granularity. > > The granularity here is disabling compilation of anything that > > isn't already compiled – this makes it so that people can still use > > their Emacs for byte compilation by invoking it with "emacs -Q", > > they just won't compile anything that their package manager hasn't > > compiled for them. > > That's quite a blunt weapon. Why not tell them to stay with Emacs > 27, until the problems are solved, or install Emacs 28 without native > compilation? That's what they're currently resorting to. Guix already supports this use case. > They are similarly drastic solutions, but they are already available > and will definitely work. Yet they are suboptimal, particularly on traditional distributions, that don't support this use-case well. > > > I don't think you can set native-comp-eln-load-path to nil. The > > > last entry, the one which points to where the preloaded *.eln > > > files live, must be there, or else Emacs will refuse to start. > > > And at least one other directory should be there as well, because > > > if and when some package advises some primitive, Emacs will need > > > to generate and compile a trampoline .eln file. But yes, if > > > users want to prevent loading from a certain directory, they can > > > remove it from native-comp-eln-load-path, provided that the two > > > necessary entries are still left in the list. > > I already found this annoying while implementing native compilation > > as part of our emacs-build-system (i.e. the build system used to > > compile Emacs packages), and I find it extra questionable that > > users on traditional distros, where they don't usually get to > > choose their Emacs, have no means of disabling this loading. > > You mean, you find the loading of preloaded *.eln files at startup > annoying? Then you should know that this is the best solution we > found for dumping Emacs with natively-compiled preloaded code. No, I find it annoying that Emacs supposes it has a writable eln-cache always. This is not the case in typical package manager scenarios and it also isn't the case when users choose to make (parts of) their $HOME read-only, which is a supported configuration in Nix and Guix. I can't think of a good reason why one would want to assume this invariant. > If you know of a better solution that doesn't suffer from any fatal > issues we found with the alternatives, please suggest such solutions, > and we will definitely consider them. I haven't read the discussions around the alternatives, but couldn't you just generate one trampoline per function which you use as soon as it's advised? Also, how come advice isn't breaking byte-compilation in exactly the same manner? > And again, if Emacs with natively-compilation is annoying, by all > means offer your users an Emacs built without natively-compilation. > This is supported and will continue to be supported for the > observable future. > > > Calling back to the earlier point on measuring performance, an easy > > way to measure performance between bytecode and native code (in a > > benchmark) would be to simply disable native code loading in one > > process. But here, it requires two separate builds of Emacs. > > As I told earlier, disabling loading of native code made no sense to > us while Emacs 28 was in development; it still doesn't. Either one > wants native-compilation, or one doesn't. Making Emacs code more > complicated and harder to maintain due to features that make no sense > to us is a non-starter. I see no problem with having to use a > separate build, since building a release tarball takes a minute or so > on a modern system. And distros should definitely have a build > without native-compilation on offer, for a variety of valid reasons. I don't think that asking distros to package every Emacs variant twice is a great idea. At Guix, we prefer to offer the most complete version of a package, so we ship with native compilation enabled. If a user wants a UI, but no native compilation (i.e. neither emacs nor emacs- minimal), they have to roll their own package description. > > > While bytecode performance on such machines might too be slow (but > > perhaps tolerable for the task), ahead-of-time compilation, perhaps > > with offloading, is preferable. > > I recommend against this, because it is impossible to rely on AOT > installations to never compile at run time. Users cannot rely on > that, and should be advised accordingly. But why can't they? > > For another, it can cause bugs like [2]. > > That bug by itself (the cause of massive launching of async > subprocesses) was never explored or described in that thread? It > seems like the discussion switched to looking for ways of disabling > native-compilation right away, without a good understand of what was > happening. Or did I mis something? Async compilation by default > never launches more subprocesses than half the execution units of the > CPU, so what is described there should be carefully investigated and > the findings described. It'd be weird if someone found a counterexample to the above statement. > The other problem in that discussion, with warnings during async JIT > compilation is well-known, was reported several times, and the > culprit is always in the 3rd-part packages being compiled, which > should be fixed. In any case, those are just warnings in almost all > cases, so their only adverse effect is annoyance (that can be > suppressed by clicking the button in the message). I read no such problem in that discussion. Do we read the same thread? > Again, I see no reason to blame the upstream project for these > issues. They should be solved by the offending 3rd-party packages, > and the distro should ideally uncover and fix them before they get to > users (I presume that you build and compile the add-on packages you > offer?). I'd like to tap at the "rolling release distro is not Debian" sign, but again, stable distros like Debian are experiencing issues with native compilation. > > > Thanks for the explanations. I still think the reasons for > > > disabling native compilation are rather weak at best, and the > > > users' requests to allow that based on either bugs that need to > > > be solved or surprise and fears that have no real basis. > > > Moreover, disabling native compilation is a very blunt instrument > > > that cannot be applied better than the existing ones, like no- > > > native-compile (and a few others that we didn't mention; see the > > > defcustom's in comp.el). > > Which defcustom? > > Begin with those described in the ELisp manual, in the > "Native-Compilation Variables" node. And my recommendation is to > review _all_ of the defcustoms in comp.el The only one I found is setting native-comp-speed to -1. Is that the solution? It doesn't appear to be. > > I fear that for all of its customizability, Emacs is > > currently lacking a good way of disabling native compilation > > outside of package management. > > Yes, because, as mentioned, this makes no sense. I think you'll find it does. > And disabling native-compilation completely is currently technically > impossible (for the same reason: Emacs wasn't designed to support > that because it didn't and still doesn't seem needed). > > > I also tried setting no-native-compile globally, but it seems to > > only have an effect as a file-local variable. > > Yes, as designed. This variable is the equivalent of no-byte- > compile, and works very similarly. > > To summarize: native compilation in a build which supports it is > ubiquitous, and is not designed to be disabled except by > no-native-compile on a file by file level. If a more general > disabling is needed for some reason, users should simply use a build > without native-compilation. It's the same as various toolkit builds: > if the toolkit is broken or doesn't fit the user's needs, those users > should install a build with a different toolkit. Pardon my French, but that thinking in and of itself is broken. Native compilation is not a choice in which you pick the one that most suits your fancy from a range of options – it could be that if you allowed the user to choose between libgccjit, clang and some other compilers that shall not be named, not that I recommend you implement this. As such, I think users who do want to use native compilation should get some more say in when, where, and what to compile. Cheers ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 19:46 ` Liliana Marie Prikler @ 2022-10-15 8:51 ` Eli Zaretskii 2022-10-15 15:53 ` Andrea Corallo 2022-10-15 9:33 ` Lars Ingebrigtsen 1 sibling, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-10-15 8:51 UTC (permalink / raw) To: Liliana Marie Prikler, Andrea Corallo; +Cc: rlb, emacs-devel > From: Liliana Marie Prikler <liliana.prikler@gmail.com> > Cc: rlb@defaultvalue.org, emacs-devel@gnu.org > Date: Fri, 14 Oct 2022 21:46:09 +0200 > > > > > When you encounter bugs in native compilation, please report them > > > > to us, so we could fix them. As of now, we are not aware of any > > > > such bugs that were reported and haven't been fixed. So if you > > > > still have such problem, please report them ASAP. > > > Of course, that's the intention, but this fix will only make it > > > into the next Emacs release. Thus, if you're between releases, you > > > still need a workaround. > > > > If the fix is urgent, why can't you patch the sources when you > > prepare your distribution? > Guix prides itself in being a package manager that can work around many > failures (even as the proper workaround to bugs is discussed in mailing > lists). The fact, that the solutions to this issue is "compile 28.1 > without native-comp" or "use Emacs 27" does not reflect that > particularly well. I think this answers a different question. I asked why you cannot patch the Emacs you distribute when you consider a fix to be important enough to not wait until the next Emacs release. My point is that reporting bugs in a timely fashion will help us fix them early on, and you will then have a possibility of backporting the fixes to a released Emacs and distributing an updated package with the fix, if you think that's important enough. > > > A particular candidate known to cause issues with the currently > > > packaged 28.1 is [1]. > > > > Where's the description of the actual problem with natively compiling > > that package? And would you please submit a bug report with the > > details, if you know them? > I am not personally affected, so I can't. I could direct people to the > Emacs mailing lists, but it seems people in other threads have already > started debugging. Do you still wish me to do so? Which threads are you alluding to here? Your [1] is just a reference to ido-completing-read-plus package, and I don't see the description of the problems with native-compilation on that site. So yes, I'd like to hear a description of the problem in that case. > > > > Why isn't it sufficient to use no-native-compile? It just means > > > > that on some architectures the corresponding file will be loaded > > > > as byte-compiled, and thus will be slightly slower (how much > > > > slower depends on the code, so if you are worried, my > > > > recommendation is first to measure the difference -- you might be > > > > surprised). > > > Because it'd require a distro-wide fix to address something that > > > e.g. only happens on some AMD CPUs. > > > > I'm asking why doing so is a problem? Did you measure the effect on > > performance and found it to be unacceptable in some cases? > Isn't performance one of the main reasons to use native compilation? On average, yes. But it depends on what the original Lisp code does. We've found that in some cases the performance gains are minimal, and in at least one very special case we found that native-compilation produces a slightly slower code. Which is why I asked the question above: it is quite possible that the (hopefully, few) packages where you need to avoid native-compilation for now don't gain performance from using native-compilation enough for justify any more elaborate measures. And this is a temporary measure anyway, because those problems will eventually be fixed, whether in the packages themselves or in Emacs core. > Note that I am talking in hypotheticals here when mentioning the AMD > thing, i.e. we could very well imagine a performance-critical Emacs > package having a native-compilation bug (I imagine those to be > particularly likely for those trailing unreleased Emacs versions, > though thankfully I don't think we've encountered one so far.) Let's not be bothered by hypothetical cases until they actually emerge. When there are specific situations where this happens and performance gains from native-compilation are critical, we can always look for specific solutions for those cases, something that is impossible without concrete cases. > > OK, so why is this relevant to the issue of disabling? Those who > > choose ahead-of-time compilation will never see async JIT > > compilation, and those who selected not to do ahead-of-time will > > naturally see JIT compilation, as they've chosen. What is the > > problem here? > The problem is that I can't meaningfully choose the "I don't want JIT > for stuff I haven't AOT'd" option, especially not combined with "but I > do want to load what I have AOT'd". As I already explained, this mode of operation doesn't make sense to me, and is currently not supported for that reason. I fail to see why people would want native-compilation for some parts of Emacs, but not for others. I haven't yet seen a valid use case where that would make sense as the desired, clean, and non-kludgey solution. Only one valid use case was brought p to this date, where it would be desirable to delay JIT native-compilation temporarily: when the user runs a laptop on batteries. We will probably provide a solution for that, which will automatically re-enable JIT compilation when AC power is connected. This would be a clean, non-kludgey solution for that case. None of the problems you describe are of that nature. They all sound like someone wants to arbitrarily disable native-compilation in some cases, but not others, where reasonable solutions already exist. And if you still disagree, then let's agree to disagree, because we are just repeating the same arguments over and over again. > > > > If a package is a single file or a small number of files, those > > > > users can add the no-native-compile cookies in those files. > > > This is not trivial in the case where the Elisp code is placed in > > > system-managed storage and thus requires elevated privileges to > > > modify (as is the default in most package managers, I assume). Of > > > course, you can copy the file to your $HOME, but editing it with a > > > broken Emacs is rather painful. > > > > Using broken packages is always painful, and native compilation > > doesn't change that. > Using broken packages normally doesn't result in the OOM killer firing > off. It could, rarely. And which problem of native-compilation caused the OOM killer? Where is that problem described in enough detail for us to investigate it? Was it reported to the Emacs bug-tracker, and if so, what is the bug number, please? IOW, we'd definitely want to avoid such catastrophic failures, but we need the details to investigate and fix them. I can tell you that I'm using Emacs 28 with JIT native-compilation enabled for the best part of this year, and have yet to see any problems even approaching the one mentioned above. So such problems are quite exceptional, and need to be reported with every possible detail for us to be able to fix them quickly. They are definitely not a reason to disable native-compilation. We generally try to provide at least a workaround for critical problems, once we have enough detail to understand what's going on, so reporting a problem quickly will in many cases yield a quick solution that doesn't hamper unrelated parts of the user's usage patterns. > > Packages provided by a distribution and installed into directories > > where users cannot easily write should be well tested by the > > distributor. > I think you're underestimating the number of breakages that can happen > in a rolling release model. Not every distro is as stable as Debian, > but the joke's still on you because despite Debian's hard requirements, > they still ended up encountering this bug. Sure, that's understandable. But each new problem that is found and reported should cause the corresponding package to be updated with a fix. I don't see why such problems are deemed as reasons to disable native-compilation for the entire Emacs session, or for requirements that they be "fixed" in core. Bugs should be fixed where their root cause is. > > You mean, you find the loading of preloaded *.eln files at startup > > annoying? Then you should know that this is the best solution we > > found for dumping Emacs with natively-compiled preloaded code. > No, I find it annoying that Emacs supposes it has a writable eln-cache > always. The user's home directory should always be writable. This is required by many Emacs features regardless of native-compilation. For example, saving customizations writes to a subdirectory of the user's home directory, as does desktop.el or save-place etc. If this is a problem during installation of packages, which run at root level, the installation procedure can tweak native-comp-eln-load-path to make sure there's a writable directory there, or point HOME to a non-existent directory. > This is not the case in typical package manager scenarios and > it also isn't the case when users choose to make (parts of) their $HOME > read-only, which is a supported configuration in Nix and Guix. Users make ~/.emacs.d/ read-only? Then how do they use all the features, some of which mentioned above, that write to that directory? > I can't think of a good reason why one would want to assume this > invariant. If this use case is supported by pointing the relevant variables, like save-place-file, eshell-directory-name, desktop-dirname, etc., to non-default places, then they can do the same with native-comp-eln-load-path. If this is not what you mean, please describe how Nix and Guix support this use case where parts of $HOME are read-only, and let's see how native-compilation should support it. > > If you know of a better solution that doesn't suffer from any fatal > > issues we found with the alternatives, please suggest such solutions, > > and we will definitely consider them. > I haven't read the discussions around the alternatives, but couldn't > you just generate one trampoline per function which you use as soon as > it's advised? And then re-generate it again each time the advised function is called again? > Also, how come advice isn't breaking byte-compilation in exactly the > same manner? Andrea, can you please answer that? I have only a very general understanding of why trampolines are needed for native-compilation. > > As I told earlier, disabling loading of native code made no sense to > > us while Emacs 28 was in development; it still doesn't. Either one > > wants native-compilation, or one doesn't. Making Emacs code more > > complicated and harder to maintain due to features that make no sense > > to us is a non-starter. I see no problem with having to use a > > separate build, since building a release tarball takes a minute or so > > on a modern system. And distros should definitely have a build > > without native-compilation on offer, for a variety of valid reasons. > I don't think that asking distros to package every Emacs variant twice > is a great idea. At Guix, we prefer to offer the most complete version > of a package, so we ship with native compilation enabled. I think this is a mistake. Native-compilation is not for everyone. It requires GCC and Binutils to be installed, and who says every Emacs user wants that? More generally, when we add optional features, we don't consider whether having them all in the same build will make sense. For example, ImageMagick support has some advantages and some (quite serious, IMO) disadvantages, so always providing it because it's "the most complete version" doesn't necessarily make sense for the users. > > > While bytecode performance on such machines might too be slow (but > > > perhaps tolerable for the task), ahead-of-time compilation, perhaps > > > with offloading, is preferable. > > > > I recommend against this, because it is impossible to rely on AOT > > installations to never compile at run time. Users cannot rely on > > that, and should be advised accordingly. > But why can't they? Trampolines is one reason. I'm sure there are others. Again, we didn't design native-compilation support in Emacs to be switchable on and off at run time, so it's small wonder that it doesn't work reliably. It would be a surprise if it did. > > > For another, it can cause bugs like [2]. > > > > That bug by itself (the cause of massive launching of async > > subprocesses) was never explored or described in that thread? It > > seems like the discussion switched to looking for ways of disabling > > native-compilation right away, without a good understand of what was > > happening. Or did I mis something? Async compilation by default > > never launches more subprocesses than half the execution units of the > > CPU, so what is described there should be carefully investigated and > > the findings described. > It'd be weird if someone found a counterexample to the above statement. I don't understand this comment, sorry. > > The other problem in that discussion, with warnings during async JIT > > compilation is well-known, was reported several times, and the > > culprit is always in the 3rd-part packages being compiled, which > > should be fixed. In any case, those are just warnings in almost all > > cases, so their only adverse effect is annoyance (that can be > > suppressed by clicking the button in the message). > I read no such problem in that discussion. Do we read the same thread? I hope so. I referred to this: https://issues.guix.gnu.org/issue/57878#13 > > Again, I see no reason to blame the upstream project for these > > issues. They should be solved by the offending 3rd-party packages, > > and the distro should ideally uncover and fix them before they get to > > users (I presume that you build and compile the add-on packages you > > offer?). > I'd like to tap at the "rolling release distro is not Debian" sign, but > again, stable distros like Debian are experiencing issues with native > compilation. Once again: no one expects all the issues to be found in advance, but when a new issue in a package is found, I do expect the distro to fix it and publish an updated package. I do not expect the distro to come back to the upstream project and ask for knobs to deal with bugs in 3rd-party packages uncovered by latest Emacs features. > > > Which defcustom? > > > > Begin with those described in the ELisp manual, in the > > "Native-Compilation Variables" node. And my recommendation is to > > review _all_ of the defcustoms in comp.el > The only one I found is setting native-comp-speed to -1. Is that the > solution? It doesn't appear to be. It _is_ a solution, for one class of problems with native-compilation. Another solution is to tweak native-comp-eln-load-path. Yet another one is to temporarily point HOME to another, perhaps non-existent, directory. > > To summarize: native compilation in a build which supports it is > > ubiquitous, and is not designed to be disabled except by > > no-native-compile on a file by file level. If a more general > > disabling is needed for some reason, users should simply use a build > > without native-compilation. It's the same as various toolkit builds: > > if the toolkit is broken or doesn't fit the user's needs, those users > > should install a build with a different toolkit. > Pardon my French, but that thinking in and of itself is broken. Native > compilation is not a choice in which you pick the one that most suits > your fancy from a range of options – it could be that if you allowed > the user to choose between libgccjit, clang and some other compilers > that shall not be named, not that I recommend you implement this. As > such, I think users who do want to use native compilation should get > some more say in when, where, and what to compile. We hear the users and their complaints, and fix stuff that we think belongs to Emacs. This was and will always be the case. In this case, I'm not convinced that the issues you describe justify a new knob in Emacs. You've described several issues which either already have solutions (which you reject), or should be solved elsewhere, not in Emacs. And if that still doesn't convince you, let's agree to disagree. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 8:51 ` Eli Zaretskii @ 2022-10-15 15:53 ` Andrea Corallo 0 siblings, 0 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-15 15:53 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Liliana Marie Prikler, rlb, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Liliana Marie Prikler <liliana.prikler@gmail.com> >> Cc: rlb@defaultvalue.org, emacs-devel@gnu.org >> Date: Fri, 14 Oct 2022 21:46:09 +0200 >> >> > > > When you encounter bugs in native compilation, please report them >> > > > to us, so we could fix them. As of now, we are not aware of any >> > > > such bugs that were reported and haven't been fixed. So if you >> > > > still have such problem, please report them ASAP. >> > > Of course, that's the intention, but this fix will only make it >> > > into the next Emacs release. Thus, if you're between releases, you >> > > still need a workaround. >> > >> > If the fix is urgent, why can't you patch the sources when you >> > prepare your distribution? >> Guix prides itself in being a package manager that can work around many >> failures (even as the proper workaround to bugs is discussed in mailing >> lists). The fact, that the solutions to this issue is "compile 28.1 >> without native-comp" or "use Emacs 27" does not reflect that >> particularly well. > > I think this answers a different question. I asked why you cannot > patch the Emacs you distribute when you consider a fix to be important > enough to not wait until the next Emacs release. My point is that > reporting bugs in a timely fashion will help us fix them early on, and > you will then have a possibility of backporting the fixes to a > released Emacs and distributing an updated package with the fix, if > you think that's important enough. > >> > > A particular candidate known to cause issues with the currently >> > > packaged 28.1 is [1]. >> > >> > Where's the description of the actual problem with natively compiling >> > that package? And would you please submit a bug report with the >> > details, if you know them? >> I am not personally affected, so I can't. I could direct people to the >> Emacs mailing lists, but it seems people in other threads have already >> started debugging. Do you still wish me to do so? > > Which threads are you alluding to here? Your [1] is just a reference > to ido-completing-read-plus package, and I don't see the description > of the problems with native-compilation on that site. So yes, I'd > like to hear a description of the problem in that case. > >> > > > Why isn't it sufficient to use no-native-compile? It just means >> > > > that on some architectures the corresponding file will be loaded >> > > > as byte-compiled, and thus will be slightly slower (how much >> > > > slower depends on the code, so if you are worried, my >> > > > recommendation is first to measure the difference -- you might be >> > > > surprised). >> > > Because it'd require a distro-wide fix to address something that >> > > e.g. only happens on some AMD CPUs. >> > >> > I'm asking why doing so is a problem? Did you measure the effect on >> > performance and found it to be unacceptable in some cases? >> Isn't performance one of the main reasons to use native compilation? > > On average, yes. But it depends on what the original Lisp code does. > We've found that in some cases the performance gains are minimal, and > in at least one very special case we found that native-compilation > produces a slightly slower code. Which is why I asked the question > above: it is quite possible that the (hopefully, few) packages where > you need to avoid native-compilation for now don't gain performance > from using native-compilation enough for justify any more elaborate > measures. And this is a temporary measure anyway, because those > problems will eventually be fixed, whether in the packages themselves > or in Emacs core. > >> Note that I am talking in hypotheticals here when mentioning the AMD >> thing, i.e. we could very well imagine a performance-critical Emacs >> package having a native-compilation bug (I imagine those to be >> particularly likely for those trailing unreleased Emacs versions, >> though thankfully I don't think we've encountered one so far.) > > Let's not be bothered by hypothetical cases until they actually > emerge. When there are specific situations where this happens and > performance gains from native-compilation are critical, we can always > look for specific solutions for those cases, something that is > impossible without concrete cases. > >> > OK, so why is this relevant to the issue of disabling? Those who >> > choose ahead-of-time compilation will never see async JIT >> > compilation, and those who selected not to do ahead-of-time will >> > naturally see JIT compilation, as they've chosen. What is the >> > problem here? >> The problem is that I can't meaningfully choose the "I don't want JIT >> for stuff I haven't AOT'd" option, especially not combined with "but I >> do want to load what I have AOT'd". > > As I already explained, this mode of operation doesn't make sense to > me, and is currently not supported for that reason. I fail to see why > people would want native-compilation for some parts of Emacs, but not > for others. I haven't yet seen a valid use case where that would make > sense as the desired, clean, and non-kludgey solution. > > Only one valid use case was brought p to this date, where it would be > desirable to delay JIT native-compilation temporarily: when the user > runs a laptop on batteries. We will probably provide a solution for > that, which will automatically re-enable JIT compilation when AC power > is connected. This would be a clean, non-kludgey solution for that > case. > > None of the problems you describe are of that nature. They all sound > like someone wants to arbitrarily disable native-compilation in some > cases, but not others, where reasonable solutions already exist. > > And if you still disagree, then let's agree to disagree, because we > are just repeating the same arguments over and over again. > >> > > > If a package is a single file or a small number of files, those >> > > > users can add the no-native-compile cookies in those files. >> > > This is not trivial in the case where the Elisp code is placed in >> > > system-managed storage and thus requires elevated privileges to >> > > modify (as is the default in most package managers, I assume). Of >> > > course, you can copy the file to your $HOME, but editing it with a >> > > broken Emacs is rather painful. >> > >> > Using broken packages is always painful, and native compilation >> > doesn't change that. >> Using broken packages normally doesn't result in the OOM killer firing >> off. > > It could, rarely. > > And which problem of native-compilation caused the OOM killer? Where > is that problem described in enough detail for us to investigate it? > Was it reported to the Emacs bug-tracker, and if so, what is the bug > number, please? > > IOW, we'd definitely want to avoid such catastrophic failures, but we > need the details to investigate and fix them. I can tell you that I'm > using Emacs 28 with JIT native-compilation enabled for the best part > of this year, and have yet to see any problems even approaching the > one mentioned above. So such problems are quite exceptional, and need > to be reported with every possible detail for us to be able to fix > them quickly. They are definitely not a reason to disable > native-compilation. We generally try to provide at least a workaround > for critical problems, once we have enough detail to understand what's > going on, so reporting a problem quickly will in many cases yield a > quick solution that doesn't hamper unrelated parts of the user's usage > patterns. > >> > Packages provided by a distribution and installed into directories >> > where users cannot easily write should be well tested by the >> > distributor. >> I think you're underestimating the number of breakages that can happen >> in a rolling release model. Not every distro is as stable as Debian, >> but the joke's still on you because despite Debian's hard requirements, >> they still ended up encountering this bug. > > Sure, that's understandable. But each new problem that is found and > reported should cause the corresponding package to be updated with a > fix. I don't see why such problems are deemed as reasons to disable > native-compilation for the entire Emacs session, or for requirements > that they be "fixed" in core. Bugs should be fixed where their root > cause is. > >> > You mean, you find the loading of preloaded *.eln files at startup >> > annoying? Then you should know that this is the best solution we >> > found for dumping Emacs with natively-compiled preloaded code. >> No, I find it annoying that Emacs supposes it has a writable eln-cache >> always. > > The user's home directory should always be writable. This is required > by many Emacs features regardless of native-compilation. For example, > saving customizations writes to a subdirectory of the user's home > directory, as does desktop.el or save-place etc. > > If this is a problem during installation of packages, which run at > root level, the installation procedure can tweak > native-comp-eln-load-path to make sure there's a writable directory > there, or point HOME to a non-existent directory. > >> This is not the case in typical package manager scenarios and >> it also isn't the case when users choose to make (parts of) their $HOME >> read-only, which is a supported configuration in Nix and Guix. > > Users make ~/.emacs.d/ read-only? Then how do they use all the > features, some of which mentioned above, that write to that directory? > >> I can't think of a good reason why one would want to assume this >> invariant. > > If this use case is supported by pointing the relevant variables, like > save-place-file, eshell-directory-name, desktop-dirname, etc., to > non-default places, then they can do the same with > native-comp-eln-load-path. If this is not what you mean, please > describe how Nix and Guix support this use case where parts of $HOME > are read-only, and let's see how native-compilation should support it. > >> > If you know of a better solution that doesn't suffer from any fatal >> > issues we found with the alternatives, please suggest such solutions, >> > and we will definitely consider them. >> I haven't read the discussions around the alternatives, but couldn't >> you just generate one trampoline per function which you use as soon as >> it's advised? > > And then re-generate it again each time the advised function is called > again? > >> Also, how come advice isn't breaking byte-compilation in exactly the >> same manner? > > Andrea, can you please answer that? I have only a very general > understanding of why trampolines are needed for native-compilation. The quick answer is that advising or redefining primitive functions breaks byte-code as well. Liliana can try redefining `+' and let us know if it works as expected :/ To be more precise byte-code breaks for redefining each primitive that got a dedicated byte-opcode. Now, the situation is even worst! Redefining primitives fails to take effect for *all* calls to primitives happening from C code. Actually the Elisp manual suggests not to redefine primitives *but*... users kept on doing it since forever and just got used to this bizarre scenario where it does "often" works. When I introduced native compilation being native code the newest into the game it was deemed not acceptable to increase the level of existing incompatibility. Unfortunately Emacs is defined also by what's on the field and not only by what's written the manual, not every behavior or corner case is defined by a spec we comply to. Not supporting this use case would have translated in tons of bug reports from users probably angrier than Liliana. So I added this trampoline mechanism. And yeah, funny enough, as a consequence at present native Lisp (and interpreted Lisp) are the only execution modes supporting for real primitive redefinition! End of the tale :) Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-14 19:46 ` Liliana Marie Prikler 2022-10-15 8:51 ` Eli Zaretskii @ 2022-10-15 9:33 ` Lars Ingebrigtsen 2022-10-15 14:35 ` Liliana Marie Prikler 2022-10-15 15:56 ` Andrea Corallo 1 sibling, 2 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-15 9:33 UTC (permalink / raw) To: Liliana Marie Prikler; +Cc: Eli Zaretskii, rlb, emacs-devel Liliana Marie Prikler <liliana.prikler@gmail.com> writes: > The problem is that I can't meaningfully choose the "I don't want JIT > for stuff I haven't AOT'd" option, especially not combined with "but I > do want to load what I have AOT'd". I haven't followed the Guix part of this thread in detail, but I thought I'd just note that Emacs 29 now has the `inhibit-automatic-native-compilation' variable and and `EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION' environment variables, so this is implemented. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 9:33 ` Lars Ingebrigtsen @ 2022-10-15 14:35 ` Liliana Marie Prikler 2022-10-15 15:56 ` Andrea Corallo 1 sibling, 0 replies; 343+ messages in thread From: Liliana Marie Prikler @ 2022-10-15 14:35 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, emacs-devel Am Samstag, dem 15.10.2022 um 11:33 +0200 schrieb Lars Ingebrigtsen: > Liliana Marie Prikler <liliana.prikler@gmail.com> writes: > > > The problem is that I can't meaningfully choose the "I don't want > > JIT > > for stuff I haven't AOT'd" option, especially not combined with > > "but I > > do want to load what I have AOT'd". > > I haven't followed the Guix part of this thread in detail, but I > thought I'd just note that Emacs 29 now has the > `inhibit-automatic-native-compilation' variable and and > `EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION' environment variables, > so this is implemented. Thanks. I've bumped emacs-next in Guix and tried it; seems to work as advertised. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 9:33 ` Lars Ingebrigtsen 2022-10-15 14:35 ` Liliana Marie Prikler @ 2022-10-15 15:56 ` Andrea Corallo 2022-10-15 16:24 ` Liliana Marie Prikler 2022-10-16 8:55 ` Lars Ingebrigtsen 1 sibling, 2 replies; 343+ messages in thread From: Andrea Corallo @ 2022-10-15 15:56 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Liliana Marie Prikler, Eli Zaretskii, rlb, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Liliana Marie Prikler <liliana.prikler@gmail.com> writes: > >> The problem is that I can't meaningfully choose the "I don't want JIT >> for stuff I haven't AOT'd" option, especially not combined with "but I >> do want to load what I have AOT'd". > > I haven't followed the Guix part of this thread in detail, but I thought > I'd just note that Emacs 29 now has the > `inhibit-automatic-native-compilation' variable and and > `EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION' environment variables, so > this is implemented. I thought this change was just into master "for discussion", are we already suggesting users to integrate it in their infrastructures? BTW AFAIU this usecase was already supported in 28 using `native-comp-deferred-compilation'. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 15:56 ` Andrea Corallo @ 2022-10-15 16:24 ` Liliana Marie Prikler 2022-10-17 7:23 ` Andrea Corallo 2022-10-16 8:55 ` Lars Ingebrigtsen 1 sibling, 1 reply; 343+ messages in thread From: Liliana Marie Prikler @ 2022-10-15 16:24 UTC (permalink / raw) To: Andrea Corallo, Lars Ingebrigtsen; +Cc: Eli Zaretskii, rlb, emacs-devel Am Samstag, dem 15.10.2022 um 15:56 +0000 schrieb Andrea Corallo: > Lars Ingebrigtsen <larsi@gnus.org> writes: > > > Liliana Marie Prikler <liliana.prikler@gmail.com> writes: > > > > > The problem is that I can't meaningfully choose the "I don't want > > > JIT > > > for stuff I haven't AOT'd" option, especially not combined with > > > "but I > > > do want to load what I have AOT'd". > > > > I haven't followed the Guix part of this thread in detail, but I > > thought > > I'd just note that Emacs 29 now has the > > `inhibit-automatic-native-compilation' variable and and > > `EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION' environment variables, > > so > > this is implemented. > > I thought this change was just into master "for discussion", are we > already suggesting users to integrate it in their infrastructures? > BTW AFAIU this usecase was already supported in 28 using > `native-comp-deferred-compilation'. The deferred-compilation switch still ends up writing to $HOME, which the new inhibit one doesn't. Should the former be deprecated in favor of the latter or should it simply take up its meaning? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 16:24 ` Liliana Marie Prikler @ 2022-10-17 7:23 ` Andrea Corallo 2022-10-17 16:59 ` Liliana Marie Prikler 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-17 7:23 UTC (permalink / raw) To: Liliana Marie Prikler; +Cc: Lars Ingebrigtsen, Eli Zaretskii, rlb, emacs-devel Liliana Marie Prikler <liliana.prikler@gmail.com> writes: > Am Samstag, dem 15.10.2022 um 15:56 +0000 schrieb Andrea Corallo: >> Lars Ingebrigtsen <larsi@gnus.org> writes: >> >> > Liliana Marie Prikler <liliana.prikler@gmail.com> writes: >> > >> > > The problem is that I can't meaningfully choose the "I don't want >> > > JIT >> > > for stuff I haven't AOT'd" option, especially not combined with >> > > "but I >> > > do want to load what I have AOT'd". >> > >> > I haven't followed the Guix part of this thread in detail, but I >> > thought >> > I'd just note that Emacs 29 now has the >> > `inhibit-automatic-native-compilation' variable and and >> > `EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION' environment variables, >> > so >> > this is implemented. >> >> I thought this change was just into master "for discussion", are we >> already suggesting users to integrate it in their infrastructures? >> BTW AFAIU this usecase was already supported in 28 using >> `native-comp-deferred-compilation'. > The deferred-compilation switch still ends up writing to $HOME, which > the new inhibit one doesn't. I think Eli explained more than once how to avoid that also without using the new switch. Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-17 7:23 ` Andrea Corallo @ 2022-10-17 16:59 ` Liliana Marie Prikler 2022-10-17 17:07 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Liliana Marie Prikler @ 2022-10-17 16:59 UTC (permalink / raw) To: Andrea Corallo; +Cc: Lars Ingebrigtsen, Eli Zaretskii, rlb, emacs-devel Am Montag, dem 17.10.2022 um 07:23 +0000 schrieb Andrea Corallo: > Liliana Marie Prikler <liliana.prikler@gmail.com> writes: > > > Am Samstag, dem 15.10.2022 um 15:56 +0000 schrieb Andrea Corallo: > > > Lars Ingebrigtsen <larsi@gnus.org> writes: > > > > > > > Liliana Marie Prikler <liliana.prikler@gmail.com> writes: > > > > > > > > > The problem is that I can't meaningfully choose the "I don't > > > > > want JIT for stuff I haven't AOT'd" option, especially not > > > > > combined with "but I do want to load what I have AOT'd". > > > > > > > > I haven't followed the Guix part of this thread in detail, but > > > > I thought I'd just note that Emacs 29 now has the > > > > `inhibit-automatic-native-compilation' variable and and > > > > `EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION' environment > > > > variables, so this is implemented. > > > > > > I thought this change was just into master "for discussion", are > > > we already suggesting users to integrate it in their > > > infrastructures? > > > BTW AFAIU this usecase was already supported in 28 using > > > `native-comp-deferred-compilation'. > > The deferred-compilation switch still ends up writing to $HOME, > > which the new inhibit one doesn't. > > I think Eli explained more than once how to avoid that also without > using the new switch. Now, I'm not following all branches of this discussion, so the chances that I missed something are admittedly high, but the last time I conversed with Eli about this, they said my use case was neither supported nor something they'd consider supporting. Thus, I'd be positively surprised to see it in fact supported. In any case, please do inform me about the solution once everything has settled; I'll need to bump the emacs-next package in Guix, which currently uses Lars' commit. Cheers ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-17 16:59 ` Liliana Marie Prikler @ 2022-10-17 17:07 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-10-17 17:07 UTC (permalink / raw) To: Liliana Marie Prikler; +Cc: akrl, larsi, rlb, emacs-devel > From: Liliana Marie Prikler <liliana.prikler@gmail.com> > Cc: Lars Ingebrigtsen <larsi@gnus.org>, Eli Zaretskii <eliz@gnu.org>, > rlb@defaultvalue.org, emacs-devel@gnu.org > Date: Mon, 17 Oct 2022 18:59:11 +0200 > > > > > > I haven't followed the Guix part of this thread in detail, but > > > > > I thought I'd just note that Emacs 29 now has the > > > > > `inhibit-automatic-native-compilation' variable and and > > > > > `EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION' environment > > > > > variables, so this is implemented. > > > > > > > > I thought this change was just into master "for discussion", are > > > > we already suggesting users to integrate it in their > > > > infrastructures? > > > > BTW AFAIU this usecase was already supported in 28 using > > > > `native-comp-deferred-compilation'. > > > The deferred-compilation switch still ends up writing to $HOME, > > > which the new inhibit one doesn't. > > > > I think Eli explained more than once how to avoid that also without > > using the new switch. > Now, I'm not following all branches of this discussion, so the chances > that I missed something are admittedly high, but the last time I > conversed with Eli about this, they said my use case was neither > supported nor something they'd consider supporting. Thus, I'd be > positively surprised to see it in fact supported. Which use case are you alluding to? Andrea refers to preventing writes to user's HOME directory. That case is supported (in more than one way), and doesn't need the new variable. AFAIU, the use case you were presenting was not about preventing writes to $HOME. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-15 15:56 ` Andrea Corallo 2022-10-15 16:24 ` Liliana Marie Prikler @ 2022-10-16 8:55 ` Lars Ingebrigtsen 2022-10-16 9:27 ` Liliana Marie Prikler 1 sibling, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-16 8:55 UTC (permalink / raw) To: Andrea Corallo; +Cc: Liliana Marie Prikler, Eli Zaretskii, rlb, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > I thought this change was just into master "for discussion", are we > already suggesting users to integrate it in their infrastructures? Yes, I should have mentioned that, as with everything on "master", things may change. But a mechanism more or less like this will exist. Liliana Marie Prikler <liliana.prikler@gmail.com> writes: >> BTW AFAIU this usecase was already supported in 28 using >> `native-comp-deferred-compilation'. > The deferred-compilation switch still ends up writing to $HOME, which > the new inhibit one doesn't. Should the former be deprecated in favor > of the latter or should it simply take up its meaning? It has been deprecated on "master". ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-16 8:55 ` Lars Ingebrigtsen @ 2022-10-16 9:27 ` Liliana Marie Prikler 2022-10-16 9:35 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Liliana Marie Prikler @ 2022-10-16 9:27 UTC (permalink / raw) To: Lars Ingebrigtsen, Andrea Corallo; +Cc: Eli Zaretskii, rlb, emacs-devel Am Sonntag, dem 16.10.2022 um 10:55 +0200 schrieb Lars Ingebrigtsen: > Andrea Corallo <akrl@sdf.org> writes: > > > I thought this change was just into master "for discussion", are we > > already suggesting users to integrate it in their infrastructures? > > Yes, I should have mentioned that, as with everything on "master", > things may change. But a mechanism more or less like this will > exist. > > Liliana Marie Prikler <liliana.prikler@gmail.com> writes: > > > > BTW AFAIU this usecase was already supported in 28 using > > > `native-comp-deferred-compilation'. > > The deferred-compilation switch still ends up writing to $HOME, > > which the new inhibit one doesn't. Should the former be deprecated > > in favor of the latter or should it simply take up its meaning? > > It has been deprecated on "master". But this deprecation may be reverted; I'm simply asking that if it is, then native-comp-deferred-compilation should take up some of the meaning of the switch you've implemented. As a user, I find it somewhat surprising that whatever means I have to inhibit deferred compilation actually inhibit neither deferred compilation nor its side- effects. Cheers ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-16 9:27 ` Liliana Marie Prikler @ 2022-10-16 9:35 ` Lars Ingebrigtsen 2022-10-17 7:30 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-16 9:35 UTC (permalink / raw) To: Liliana Marie Prikler; +Cc: Andrea Corallo, Eli Zaretskii, rlb, emacs-devel Liliana Marie Prikler <liliana.prikler@gmail.com> writes: > But this deprecation may be reverted; I'm simply asking that if it is, > then native-comp-deferred-compilation should take up some of the > meaning of the switch you've implemented. I don't think it's likely that it will be reverted. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-16 9:35 ` Lars Ingebrigtsen @ 2022-10-17 7:30 ` Andrea Corallo 2022-10-17 11:20 ` Lars Ingebrigtsen 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-10-17 7:30 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Liliana Marie Prikler, Eli Zaretskii, rlb, emacs-devel Lars Ingebrigtsen <larsi@gnus.org> writes: > Liliana Marie Prikler <liliana.prikler@gmail.com> writes: > >> But this deprecation may be reverted; I'm simply asking that if it is, >> then native-comp-deferred-compilation should take up some of the >> meaning of the switch you've implemented. > > I don't think it's likely that it will be reverted. You might be a little self-referential here. This change was asked to be reverted by the other head maintainer and by the maintainer of the subsystem in object. BTW how long is the "discussion period" supposed to last? Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-10-17 7:30 ` Andrea Corallo @ 2022-10-17 11:20 ` Lars Ingebrigtsen 0 siblings, 0 replies; 343+ messages in thread From: Lars Ingebrigtsen @ 2022-10-17 11:20 UTC (permalink / raw) To: Andrea Corallo; +Cc: Liliana Marie Prikler, Eli Zaretskii, rlb, emacs-devel Andrea Corallo <akrl@sdf.org> writes: > You might be a little self-referential here. This change was asked to > be reverted by the other head maintainer and by the maintainer of the > subsystem in object. Well, I disagree, so here we are. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-28 1:04 Rob Browning 2022-09-28 11:02 ` Lars Ingebrigtsen 2022-09-28 11:35 ` Eli Zaretskii @ 2022-09-28 12:52 ` Andrea Corallo 2022-09-28 17:04 ` Eli Zaretskii 2 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-09-28 12:52 UTC (permalink / raw) To: Rob Browning; +Cc: emacs-devel Hi Rob, Rob Browning <rlb@defaultvalue.org> writes: > Perhaps relevant in other contexts, but at least in the context of > Debian work, we'd like to be able to suppress all native compilation in > some contexts, or more specifically all writes to HOME (or really, to > avoid any implicit file manipulations[1]). > > One key case is package builds and package installs (whether the emacs > packages themselves, or any of the various emacs add-on packages > (e.g. elpa-*)). > > For package builds, HOME is typically set to /nonexistent (or similar), > and for package installs we don't want the install commands (preinst, > postinst, etc.), which are running as root, to write anything to /root/. > > It's also been suggested by some users that they might like to have a > supported way to disable native compilation (even when it's the system > default), in order to avoid the unpredictable cpu and/or battery cost > sometimes. Of course, assuming we eventually augment debian's current > emacs infrastructure to generate system-level .eln files during (add-on) > package installs, in addition to the .elc files it currently produces, > much of that concern for debian and its derivatives might recede. This is what `native-comp-deferred-compilation' does. Well except for trampolines but this is extremly light as cpu/energy cost. > In any case, I noticed that there is a no-native-compile variable, and > wondered if (in the short term), I might be able to craft a (possibly > debian-specific for now) way to disable native compilation across the > board. > > Just to see, I changed no-native-compile to default to > > (getenv "DEBIAN_EMACS_DISABLE_NATIVE_COMPILATION") > > instead of nil, and then set that in the environment, but the ebib > package tests (which set HOME=/nonexistent) still crashed (as had been > originally reported here https://bugs.debian.org/1020191). > > The crash was in comp-trampoline-compile, and after a bit of > investigation I wondered if it might be possible to fix the problem (at > least approximately) by adding a no-native-compile clause to the > comp-subr-trampoline-install guard like this: > > (defun comp-subr-trampoline-install (subr-name) > "Make SUBR-NAME effectively advice-able when called from native code." > (unless (or no-native-compilation ; this is new > (null comp-enable-subr-trampolines) > (memq subr-name native-comp-never-optimize-functions) > (gethash subr-name comp-installed-trampolines-h)) > ...)) A native compiled Emacs needs to compile a trampoline each time a primitive is redefined or advised. If we disable this mechanism ATM we cannot guarantee Emacs to function correctly. > That did allow the package tests to run successfully. > > So two questions: > > - As possibly just a short term, debian-specific fix, do those changes > sound plausible, or are there other things we're missing (I might > guess yes)? IMO the suggested change is not okay for the reason mentioned above sorry :/ > > - Longer term, might it make sense to have some emacs-proper way to > completely disable native compilation from the outside, either via > environment variable, argument, or something else, for situations > like this? > > [1] For add-on packages (e.g. elpa-*, etc.), we want to explicitly build > the local .elc files (at least) during the install, but we don't > want to write to any other unexpected locations. > > Thanks IIUC the issue is when Emacs is run as root (I didn't know Debian does that for installing packages!). The workaround I'd suggest is to target a temporary HOME and clean it afterwards with its eln-cache (if this is viable). Best Regards Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-28 12:52 ` Andrea Corallo @ 2022-09-28 17:04 ` Eli Zaretskii 2022-09-28 18:49 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-09-28 17:04 UTC (permalink / raw) To: Andrea Corallo; +Cc: rlb, emacs-devel > From: Andrea Corallo <akrl@sdf.org> > Cc: emacs-devel@gnu.org > Date: Wed, 28 Sep 2022 12:52:59 +0000 > > This is what `native-comp-deferred-compilation' does. Well except for > trampolines but this is extremly light as cpu/energy cost. I think if we want to allow people to turn off async compilation, we should provide a way to force native-comp-available-p to return nil. When that function returns nil, Emacs already does react correctly (or at least it's supposed to). But I still would like to understand what kind of use cases are we talking about here, because without that it's impossible to decide whether a given measure will provide a reasonable solution in the long run. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-28 17:04 ` Eli Zaretskii @ 2022-09-28 18:49 ` Andrea Corallo 2022-09-28 19:10 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-09-28 18:49 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rlb, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Andrea Corallo <akrl@sdf.org> >> Cc: emacs-devel@gnu.org >> Date: Wed, 28 Sep 2022 12:52:59 +0000 >> >> This is what `native-comp-deferred-compilation' does. Well except for >> trampolines but this is extremly light as cpu/energy cost. > > I think if we want to allow people to turn off async compilation, we > should provide a way to force native-comp-available-p to return nil. > When that function returns nil, Emacs already does react correctly (or > at least it's supposed to). Yes `native-comp-deferred-compilation' AFAIK does exactly this already, `native-comp-available-p' is to check if the native compiler is available (not necessarily the deferred/async mechanism). Best Regards Andera ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-28 18:49 ` Andrea Corallo @ 2022-09-28 19:10 ` Eli Zaretskii 2022-09-28 21:32 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-09-28 19:10 UTC (permalink / raw) To: Andrea Corallo; +Cc: rlb, emacs-devel > From: Andrea Corallo <akrl@sdf.org> > Cc: rlb@defaultvalue.org, emacs-devel@gnu.org > Date: Wed, 28 Sep 2022 18:49:43 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> From: Andrea Corallo <akrl@sdf.org> > >> Cc: emacs-devel@gnu.org > >> Date: Wed, 28 Sep 2022 12:52:59 +0000 > >> > >> This is what `native-comp-deferred-compilation' does. Well except for > >> trampolines but this is extremly light as cpu/energy cost. > > > > I think if we want to allow people to turn off async compilation, we > > should provide a way to force native-comp-available-p to return nil. > > When that function returns nil, Emacs already does react correctly (or > > at least it's supposed to). > > Yes `native-comp-deferred-compilation' AFAIK does exactly this already, > `native-comp-available-p' is to check if the native compiler is > available (not necessarily the deferred/async mechanism). But then it should disable the trampolines as well, see startup.el. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-28 19:10 ` Eli Zaretskii @ 2022-09-28 21:32 ` Andrea Corallo 2022-09-29 5:56 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-09-28 21:32 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rlb, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Andrea Corallo <akrl@sdf.org> >> Cc: rlb@defaultvalue.org, emacs-devel@gnu.org >> Date: Wed, 28 Sep 2022 18:49:43 +0000 >> >> Eli Zaretskii <eliz@gnu.org> writes: >> >> >> From: Andrea Corallo <akrl@sdf.org> >> >> Cc: emacs-devel@gnu.org >> >> Date: Wed, 28 Sep 2022 12:52:59 +0000 >> >> >> >> This is what `native-comp-deferred-compilation' does. Well except for >> >> trampolines but this is extremly light as cpu/energy cost. >> > >> > I think if we want to allow people to turn off async compilation, we >> > should provide a way to force native-comp-available-p to return nil. >> > When that function returns nil, Emacs already does react correctly (or >> > at least it's supposed to). >> >> Yes `native-comp-deferred-compilation' AFAIK does exactly this already, >> `native-comp-available-p' is to check if the native compiler is >> available (not necessarily the deferred/async mechanism). > > But then it should disable the trampolines as well, see startup.el. Not in my opinion, trampolines are not deferred async compilation. Also as mentioned ATM is not possible to disable trampolines and have a fully working native comp Emacs (if we assume primitives can be redefined). Best Regards Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-28 21:32 ` Andrea Corallo @ 2022-09-29 5:56 ` Eli Zaretskii 2022-09-29 8:18 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-09-29 5:56 UTC (permalink / raw) To: Andrea Corallo; +Cc: rlb, emacs-devel > From: Andrea Corallo <akrl@sdf.org> > Cc: rlb@defaultvalue.org, emacs-devel@gnu.org > Date: Wed, 28 Sep 2022 21:32:13 +0000 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> Yes `native-comp-deferred-compilation' AFAIK does exactly this already, > >> `native-comp-available-p' is to check if the native compiler is > >> available (not necessarily the deferred/async mechanism). > > > > But then it should disable the trampolines as well, see startup.el. > > Not in my opinion, trampolines are not deferred async compilation. > > Also as mentioned ATM is not possible to disable trampolines and have a > fully working native comp Emacs (if we assume primitives can be > redefined). I'm confused. I alluded to this part of startup.el: (when (featurep 'native-compile) (unless (native-comp-available-p) ;; Disable deferred async compilation and trampoline synthesis ;; in this session. This is necessary if libgccjit is not ;; available on MS-Windows, but Emacs was built with ;; native-compilation support. (setq native-comp-deferred-compilation nil comp-enable-subr-trampolines nil)) The last part disables trampolines, AFAIU. So what am I missing here? ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-29 5:56 ` Eli Zaretskii @ 2022-09-29 8:18 ` Andrea Corallo 2022-09-29 9:01 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-09-29 8:18 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rlb, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Andrea Corallo <akrl@sdf.org> >> Cc: rlb@defaultvalue.org, emacs-devel@gnu.org >> Date: Wed, 28 Sep 2022 21:32:13 +0000 >> >> Eli Zaretskii <eliz@gnu.org> writes: >> >> >> Yes `native-comp-deferred-compilation' AFAIK does exactly this already, >> >> `native-comp-available-p' is to check if the native compiler is >> >> available (not necessarily the deferred/async mechanism). >> > >> > But then it should disable the trampolines as well, see startup.el. >> >> Not in my opinion, trampolines are not deferred async compilation. >> >> Also as mentioned ATM is not possible to disable trampolines and have a >> fully working native comp Emacs (if we assume primitives can be >> redefined). > > I'm confused. I alluded to this part of startup.el: > > (when (featurep 'native-compile) > (unless (native-comp-available-p) > ;; Disable deferred async compilation and trampoline synthesis > ;; in this session. This is necessary if libgccjit is not > ;; available on MS-Windows, but Emacs was built with > ;; native-compilation support. > (setq native-comp-deferred-compilation nil > comp-enable-subr-trampolines nil)) > > The last part disables trampolines, AFAIU. So what am I missing here? Hi Eli, yes it does, what do you find confusing about this? This is how I see things, we have two compilation mechanisms: 1- Syncronous native compilation. This can be triggered: 1.1- By the user (calling `native-compile'). 1.2- By Emacs in the need of generating a trampoline. 2- Async/deferred/jit (or how we wanna call it), this is triggered automatically when loafing a .elc with no corresponding .eln. We have two customize to control the two automatic mechanisms: `comp-enable-subr-trampolines' gates 1.2, `native-comp-deferred-compilation' gates 2. Indeed `native-comp-available-p' gates all native compilations. Best Regards Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-29 8:18 ` Andrea Corallo @ 2022-09-29 9:01 ` Eli Zaretskii 2022-09-29 14:32 ` Andrea Corallo 0 siblings, 1 reply; 343+ messages in thread From: Eli Zaretskii @ 2022-09-29 9:01 UTC (permalink / raw) To: Andrea Corallo; +Cc: rlb, emacs-devel > From: Andrea Corallo <akrl@sdf.org> > Cc: rlb@defaultvalue.org, emacs-devel@gnu.org > Date: Thu, 29 Sep 2022 08:18:08 +0000 > > >> >> Yes `native-comp-deferred-compilation' AFAIK does exactly this already, > >> >> `native-comp-available-p' is to check if the native compiler is > >> >> available (not necessarily the deferred/async mechanism). > >> > > >> > But then it should disable the trampolines as well, see startup.el. > >> > >> Not in my opinion, trampolines are not deferred async compilation. > >> > >> Also as mentioned ATM is not possible to disable trampolines and have a > >> fully working native comp Emacs (if we assume primitives can be > >> redefined). > > > > I'm confused. I alluded to this part of startup.el: > > > > (when (featurep 'native-compile) > > (unless (native-comp-available-p) > > ;; Disable deferred async compilation and trampoline synthesis > > ;; in this session. This is necessary if libgccjit is not > > ;; available on MS-Windows, but Emacs was built with > > ;; native-compilation support. > > (setq native-comp-deferred-compilation nil > > comp-enable-subr-trampolines nil)) > > > > The last part disables trampolines, AFAIU. So what am I missing here? > > Hi Eli, > > yes it does, what do you find confusing about this? > > This is how I see things, we have two compilation mechanisms: > > 1- Syncronous native compilation. This can be triggered: > 1.1- By the user (calling `native-compile'). > 1.2- By Emacs in the need of generating a trampoline. > > 2- Async/deferred/jit (or how we wanna call it), this is triggered > automatically when loafing a .elc with no corresponding .eln. > > We have two customize to control the two automatic mechanisms: > `comp-enable-subr-trampolines' gates 1.2, > `native-comp-deferred-compilation' gates 2. > > Indeed `native-comp-available-p' gates all native compilations. So I was asking whether when the user wants to disable native compilations, we should disable both 2 and 1.2. We already do that when Emacs detects at startup that it doesn't have access to libgccjit, so why not do the same when the user wants to disable native compilations for whatever reasons? You seem to be saying that 1.2 cannot be disabled in that case? But if so, how come we do disable it in startup.el? That is the root of my confusion. Thanks. ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-29 9:01 ` Eli Zaretskii @ 2022-09-29 14:32 ` Andrea Corallo 2022-09-29 15:50 ` Eli Zaretskii 0 siblings, 1 reply; 343+ messages in thread From: Andrea Corallo @ 2022-09-29 14:32 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rlb, emacs-devel Eli Zaretskii <eliz@gnu.org> writes: >> From: Andrea Corallo <akrl@sdf.org> >> Cc: rlb@defaultvalue.org, emacs-devel@gnu.org >> Date: Thu, 29 Sep 2022 08:18:08 +0000 >> >> >> >> Yes `native-comp-deferred-compilation' AFAIK does exactly this already, >> >> >> `native-comp-available-p' is to check if the native compiler is >> >> >> available (not necessarily the deferred/async mechanism). >> >> > >> >> > But then it should disable the trampolines as well, see startup.el. >> >> >> >> Not in my opinion, trampolines are not deferred async compilation. >> >> >> >> Also as mentioned ATM is not possible to disable trampolines and have a >> >> fully working native comp Emacs (if we assume primitives can be >> >> redefined). >> > >> > I'm confused. I alluded to this part of startup.el: >> > >> > (when (featurep 'native-compile) >> > (unless (native-comp-available-p) >> > ;; Disable deferred async compilation and trampoline synthesis >> > ;; in this session. This is necessary if libgccjit is not >> > ;; available on MS-Windows, but Emacs was built with >> > ;; native-compilation support. >> > (setq native-comp-deferred-compilation nil >> > comp-enable-subr-trampolines nil)) >> > >> > The last part disables trampolines, AFAIU. So what am I missing here? >> >> Hi Eli, >> >> yes it does, what do you find confusing about this? >> >> This is how I see things, we have two compilation mechanisms: >> >> 1- Syncronous native compilation. This can be triggered: >> 1.1- By the user (calling `native-compile'). >> 1.2- By Emacs in the need of generating a trampoline. >> >> 2- Async/deferred/jit (or how we wanna call it), this is triggered >> automatically when loafing a .elc with no corresponding .eln. >> >> We have two customize to control the two automatic mechanisms: >> `comp-enable-subr-trampolines' gates 1.2, >> `native-comp-deferred-compilation' gates 2. >> >> Indeed `native-comp-available-p' gates all native compilations. > > So I was asking whether when the user wants to disable native > compilations, we should disable both 2 and 1.2. We already do that > when Emacs detects at startup that it doesn't have access to > libgccjit, so why not do the same when the user wants to disable > native compilations for whatever reasons? Ah I see. We could allow for disabling completely native compilation, but that's something different from what those two customizes are for. That said I think that given the very low cost of building trampolines, and the deep impact that their absence would have in the Emacs machinery, we should acctually discourage the user to do so unless some very specific reason is present (I can't think of any ATM). Andrea ^ permalink raw reply [flat|nested] 343+ messages in thread
* Re: Suppressing native compilation (short and long term) 2022-09-29 14:32 ` Andrea Corallo @ 2022-09-29 15:50 ` Eli Zaretskii 0 siblings, 0 replies; 343+ messages in thread From: Eli Zaretskii @ 2022-09-29 15:50 UTC (permalink / raw) To: Andrea Corallo; +Cc: rlb, emacs-devel > From: Andrea Corallo <akrl@sdf.org> > Cc: rlb@defaultvalue.org, emacs-devel@gnu.org > Date: Thu, 29 Sep 2022 14:32:07 +0000 > > That said I think that given the very low cost of building trampolines, > and the deep impact that their absence would have in the Emacs > machinery, we should acctually discourage the user to do so unless some > very specific reason is present (I can't think of any ATM). The reason stated by Rob was that they want to prevent "writing to user's HOME directory". I don't think I understand the rationale well enough, and I asked some questions about it. I hope Rob will answer, and then we could continue discussing what is reasonable for that use case. ^ permalink raw reply [flat|nested] 343+ messages in thread
end of thread, other threads:[~2022-11-08 12:13 UTC | newest] Thread overview: 343+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-10-23 18:43 Suppressing native compilation (short and long term) Gregor Zattler -- strict thread matches above, loose matches on Subject: below -- 2022-10-23 15:22 Gregor Zattler 2022-10-23 16:07 ` Eli Zaretskii 2022-10-23 17:08 ` Gregor Zattler 2022-10-23 17:30 ` Eli Zaretskii 2022-10-23 17:33 ` Gregor Zattler 2022-10-23 17:34 ` Eli Zaretskii 2022-10-23 17:39 ` Eli Zaretskii 2022-10-23 18:55 ` Gregor Zattler 2022-10-24 18:35 ` Gregor Zattler 2022-10-24 19:04 ` Eli Zaretskii 2022-09-30 13:13 David Bremner 2022-09-30 13:56 ` Eli Zaretskii 2022-09-30 14:33 ` David Bremner 2022-09-30 15:47 ` Eli Zaretskii 2022-09-30 14:55 ` Sean Whitton 2022-09-30 16:02 ` Eli Zaretskii 2022-09-30 15:32 ` Stefan Monnier 2022-09-30 16:06 ` Eli Zaretskii 2022-10-01 16:38 ` Rob Browning 2022-10-01 16:52 ` Eli Zaretskii 2022-10-01 20:42 ` Rob Browning 2022-10-02 5:57 ` Eli Zaretskii 2022-10-02 6:10 ` tomas 2022-10-02 6:44 ` Eli Zaretskii 2022-10-02 15:54 ` tomas 2022-10-02 16:02 ` Eli Zaretskii 2022-10-02 16:13 ` chad 2022-10-02 16:55 ` Eli Zaretskii 2022-10-02 17:13 ` Stefan Monnier 2022-10-02 17:24 ` Eli Zaretskii 2022-10-02 17:25 ` Rob Browning 2022-10-02 16:53 ` Stefan Monnier 2022-10-02 17:16 ` Eli Zaretskii 2022-10-02 18:12 ` Rob Browning 2022-10-02 18:40 ` Eli Zaretskii 2022-10-02 18:51 ` Rob Browning 2022-10-02 17:17 ` Lars Ingebrigtsen 2022-10-02 17:28 ` Stefan Monnier 2022-10-02 17:36 ` Lars Ingebrigtsen 2022-10-02 17:38 ` Eli Zaretskii 2022-10-02 18:17 ` Rob Browning 2022-10-02 19:08 ` Lars Ingebrigtsen 2022-10-02 19:15 ` Eli Zaretskii 2022-10-03 9:12 ` Lars Ingebrigtsen 2022-10-03 11:32 ` Lars Ingebrigtsen 2022-10-03 13:07 ` Stefan Monnier 2022-10-03 13:29 ` Lars Ingebrigtsen 2022-10-03 13:42 ` Stefan Monnier 2022-10-03 14:09 ` Lars Ingebrigtsen 2022-10-03 14:42 ` Stefan Monnier 2022-10-03 14:45 ` Lars Ingebrigtsen 2022-10-03 14:52 ` Stefan Monnier 2022-10-03 15:14 ` Lars Ingebrigtsen 2022-10-05 12:55 ` Andrea Corallo 2022-10-05 13:14 ` Lars Ingebrigtsen 2022-10-05 13:47 ` Andrea Corallo 2022-10-05 13:55 ` Eli Zaretskii 2022-10-05 14:08 ` Andrea Corallo 2022-10-03 17:21 ` Eli Zaretskii 2022-10-03 17:39 ` Lars Ingebrigtsen 2022-10-03 17:52 ` Eli Zaretskii 2022-10-05 13:05 ` Andrea Corallo 2022-10-05 13:09 ` Andrea Corallo 2022-10-05 12:51 ` Andrea Corallo 2022-10-03 17:16 ` Eli Zaretskii 2022-10-03 18:21 ` Sean Whitton 2022-10-03 20:43 ` Stefan Monnier 2022-10-03 17:44 ` Eli Zaretskii 2022-10-05 13:18 ` Andrea Corallo 2022-10-05 14:01 ` Lars Ingebrigtsen 2022-10-05 14:17 ` Eli Zaretskii 2022-10-05 14:27 ` Lars Ingebrigtsen 2022-10-05 16:42 ` Eli Zaretskii 2022-10-05 17:08 ` Lars Ingebrigtsen 2022-10-05 17:15 ` Eli Zaretskii 2022-10-06 0:41 ` Po Lu 2022-10-06 0:40 ` Po Lu 2022-10-06 6:26 ` Eli Zaretskii 2022-10-05 14:25 ` Andrea Corallo 2022-10-05 14:29 ` Lars Ingebrigtsen 2022-10-05 14:48 ` Andrea Corallo 2022-10-05 14:58 ` Lars Ingebrigtsen 2022-10-05 15:12 ` Andrea Corallo 2022-10-05 15:24 ` Lars Ingebrigtsen 2022-10-05 15:36 ` Lars Ingebrigtsen 2022-10-05 16:10 ` Andrea Corallo 2022-10-05 16:13 ` Lars Ingebrigtsen 2022-10-05 17:58 ` Andrea Corallo 2022-10-05 18:28 ` Lars Ingebrigtsen 2022-10-05 23:25 ` Andrea Corallo 2022-10-05 23:33 ` Lars Ingebrigtsen 2022-10-06 0:45 ` Andrea Corallo 2022-10-06 0:55 ` Lars Ingebrigtsen 2022-10-06 6:33 ` Eli Zaretskii 2022-11-05 1:09 ` Juanma Barranquero 2022-11-05 6:44 ` Eli Zaretskii 2022-11-05 10:12 ` Dr. Arne Babenhauserheide 2022-11-05 11:19 ` Eli Zaretskii 2022-11-06 11:06 ` Max Brieiev 2022-11-06 16:31 ` Dr. Arne Babenhauserheide 2022-11-06 18:00 ` Stefan Monnier 2022-11-07 9:22 ` Andrea Corallo 2022-11-08 5:51 ` Björn Bidar 2022-11-08 11:23 ` Andrea Corallo 2022-11-08 12:13 ` Eli Zaretskii 2022-11-05 11:53 ` Juanma Barranquero 2022-11-05 12:44 ` Eli Zaretskii 2022-11-05 13:12 ` Juanma Barranquero 2022-11-05 13:35 ` Eli Zaretskii 2022-11-05 14:09 ` Juanma Barranquero 2022-10-05 16:04 ` Andrea Corallo 2022-10-05 16:12 ` Lars Ingebrigtsen 2022-10-05 16:28 ` Andrea Corallo 2022-10-05 16:43 ` Eli Zaretskii 2022-10-06 0:44 ` Po Lu 2022-10-06 0:56 ` Lars Ingebrigtsen 2022-10-06 6:41 ` Eli Zaretskii 2022-10-06 6:28 ` Eli Zaretskii 2022-10-14 17:53 ` Rob Browning 2022-10-14 18:36 ` Stefan Monnier 2022-10-14 19:06 ` Eli Zaretskii 2022-10-14 21:17 ` Rob Browning 2022-10-15 3:07 ` Stefan Monnier 2022-10-15 16:34 ` Rob Browning 2022-10-15 23:16 ` Stefan Monnier 2022-10-15 6:30 ` Eli Zaretskii 2022-10-15 17:00 ` Rob Browning 2022-10-15 9:32 ` Lars Ingebrigtsen 2022-10-15 16:48 ` Sean Whitton 2022-10-16 8:56 ` Lars Ingebrigtsen 2022-10-15 17:21 ` Rob Browning 2022-10-15 17:25 ` Eli Zaretskii 2022-10-16 9:01 ` Lars Ingebrigtsen 2022-10-16 16:59 ` Rob Browning 2022-10-17 10:37 ` Lars Ingebrigtsen 2022-10-05 20:37 ` Lars Ingebrigtsen 2022-10-05 23:40 ` Andrea Corallo 2022-10-05 23:46 ` Lars Ingebrigtsen 2022-10-06 0:34 ` Andrea Corallo 2022-10-06 0:39 ` Lars Ingebrigtsen 2022-10-06 1:03 ` Andrea Corallo 2022-10-06 1:07 ` Lars Ingebrigtsen 2022-10-06 1:19 ` Andrea Corallo 2022-10-06 1:26 ` Lars Ingebrigtsen 2022-10-06 1:39 ` Andrea Corallo 2022-10-06 12:07 ` Lars Ingebrigtsen 2022-10-02 20:05 ` Rob Browning 2022-10-02 20:10 ` Lars Ingebrigtsen 2022-10-05 13:19 ` Andrea Corallo 2022-10-05 12:43 ` Andrea Corallo 2022-10-05 13:16 ` Lars Ingebrigtsen 2022-10-05 13:29 ` Andrea Corallo 2022-10-05 14:03 ` Eli Zaretskii 2022-10-02 17:30 ` Eli Zaretskii 2022-10-02 17:38 ` Lars Ingebrigtsen 2022-10-02 17:48 ` Eli Zaretskii 2022-10-02 17:37 ` Rob Browning 2022-10-02 17:44 ` Eli Zaretskii 2022-10-02 18:21 ` Rob Browning 2022-10-02 18:43 ` Eli Zaretskii 2022-10-02 23:53 ` Sean Whitton 2022-10-02 17:15 ` Rob Browning 2022-10-02 17:25 ` Stefan Monnier 2022-10-02 18:11 ` Stefan Kangas 2022-10-02 18:26 ` Stefan Monnier 2022-10-02 18:57 ` Stefan Kangas 2022-10-02 17:26 ` Eli Zaretskii 2022-10-02 23:51 ` Sean Whitton 2022-10-03 16:19 ` Eli Zaretskii 2022-10-03 18:23 ` Sean Whitton 2022-10-03 18:44 ` Eli Zaretskii 2022-10-04 0:31 ` Po Lu 2022-10-04 6:57 ` Eli Zaretskii 2022-10-04 8:37 ` Po Lu 2022-10-04 9:25 ` Eli Zaretskii 2022-10-04 9:51 ` Po Lu 2022-10-05 13:58 ` Po Lu 2022-10-05 15:02 ` Lars Ingebrigtsen 2022-10-05 16:47 ` Eli Zaretskii 2022-10-05 17:59 ` tomas 2022-10-05 18:28 ` Eli Zaretskii 2022-10-05 18:56 ` tomas 2022-10-05 19:04 ` Eli Zaretskii 2022-10-05 19:40 ` tomas 2022-10-05 19:29 ` Gregory Heytings 2022-10-05 19:38 ` Eli Zaretskii 2022-10-05 20:04 ` Gregory Heytings 2022-10-06 5:28 ` Eli Zaretskii 2022-10-06 6:35 ` tomas 2022-10-05 16:43 ` Eli Zaretskii 2022-10-06 0:34 ` Po Lu 2022-10-06 6:21 ` Eli Zaretskii 2022-10-06 7:11 ` Po Lu 2022-10-06 8:03 ` Eli Zaretskii 2022-10-06 9:02 ` tomas 2022-10-06 10:13 ` Eli Zaretskii 2022-10-06 11:49 ` tomas 2022-10-07 12:40 ` Eli Zaretskii 2022-10-06 9:52 ` Po Lu 2022-10-06 10:17 ` Eli Zaretskii 2022-10-06 10:41 ` Andrea Corallo 2022-10-06 10:54 ` Eli Zaretskii 2022-10-04 23:26 ` Sean Whitton 2022-10-02 5:58 ` tomas 2022-10-02 6:42 ` Eli Zaretskii 2022-10-02 15:53 ` tomas 2022-10-02 16:11 ` Eli Zaretskii 2022-10-02 16:23 ` chad 2022-10-02 16:59 ` Eli Zaretskii 2022-10-02 18:35 ` Rob Browning 2022-10-02 18:54 ` Eli Zaretskii 2022-10-02 19:37 ` Rob Browning 2022-10-02 17:57 ` Rob Browning 2022-10-02 18:06 ` Lars Ingebrigtsen 2022-10-02 18:35 ` Eli Zaretskii 2022-10-02 18:41 ` Rob Browning 2022-10-02 19:00 ` Eli Zaretskii 2022-10-02 19:50 ` Rob Browning 2022-10-03 17:41 ` Eli Zaretskii 2022-10-03 18:17 ` tomas 2022-10-03 18:41 ` Eli Zaretskii 2022-10-03 19:02 ` tomas 2022-10-03 19:04 ` Eli Zaretskii 2022-10-03 18:45 ` Stefan Monnier 2022-10-03 18:52 ` Eli Zaretskii 2022-10-04 0:16 ` Rob Browning 2022-10-04 9:30 ` Eli Zaretskii 2022-10-05 0:48 ` Rob Browning 2022-10-05 7:39 ` Eli Zaretskii 2022-10-08 17:47 ` Michael Welsh Duggan 2022-10-15 17:39 ` Rob Browning 2022-10-02 18:25 ` Stefan Monnier 2022-10-02 18:32 ` Eli Zaretskii 2022-10-02 18:37 ` Lars Ingebrigtsen 2022-10-02 18:46 ` Rob Browning 2022-10-02 18:57 ` Eli Zaretskii 2022-10-02 19:01 ` Lars Ingebrigtsen 2022-10-02 19:03 ` Eli Zaretskii 2022-10-02 19:58 ` Stefan Monnier 2022-10-02 20:09 ` Stefan Monnier 2022-10-02 16:27 ` tomas 2022-10-02 17:01 ` Eli Zaretskii 2022-10-02 18:37 ` Rob Browning 2022-10-02 18:58 ` Eli Zaretskii 2022-10-02 19:58 ` Rob Browning 2022-10-02 20:51 ` tomas 2022-10-02 23:58 ` Sean Whitton 2022-10-03 16:24 ` Eli Zaretskii 2022-10-03 18:23 ` Sean Whitton 2022-10-03 18:46 ` Eli Zaretskii 2022-10-02 18:32 ` Rob Browning 2022-10-02 18:51 ` Eli Zaretskii 2022-10-02 19:57 ` Rob Browning 2022-10-03 15:48 ` Eli Zaretskii 2022-10-03 16:39 ` Stefan Monnier 2022-10-03 17:30 ` Eli Zaretskii 2022-10-03 18:33 ` Stefan Monnier 2022-10-03 18:49 ` Eli Zaretskii 2022-10-03 21:58 ` Stefan Kangas 2022-10-04 6:10 ` Eli Zaretskii 2022-10-04 13:30 ` Stefan Monnier 2022-09-30 15:38 ` Stefan Monnier 2022-09-30 17:05 ` David Bremner 2022-09-30 17:32 ` David Bremner 2022-09-28 1:04 Rob Browning 2022-09-28 11:02 ` Lars Ingebrigtsen 2022-09-28 11:35 ` Eli Zaretskii 2022-10-12 20:22 ` Liliana Marie Prikler 2022-10-13 5:46 ` Eli Zaretskii 2022-10-13 19:20 ` Liliana Marie Prikler 2022-10-13 20:10 ` Stefan Monnier 2022-10-13 20:26 ` Eli Zaretskii 2022-10-13 20:57 ` Lars Ingebrigtsen 2022-10-13 21:29 ` Lars Ingebrigtsen 2022-10-14 22:37 ` Andrea Corallo 2022-10-15 3:09 ` Stefan Monnier 2022-10-15 15:06 ` Andrea Corallo 2022-10-15 16:16 ` Stefan Monnier 2022-10-15 9:24 ` Lars Ingebrigtsen 2022-10-15 15:02 ` Andrea Corallo 2022-10-16 8:52 ` Lars Ingebrigtsen 2022-10-16 9:29 ` Liliana Marie Prikler 2022-10-16 13:45 ` Stefan Monnier 2022-10-16 13:47 ` Lars Ingebrigtsen 2022-10-19 19:39 ` Andrea Corallo 2022-10-14 6:08 ` Eli Zaretskii 2022-10-14 23:20 ` Andrea Corallo 2022-10-15 3:14 ` Stefan Monnier 2022-10-15 6:40 ` Liliana Marie Prikler 2022-10-15 7:14 ` Eli Zaretskii 2022-10-15 14:00 ` Stefan Monnier 2022-10-15 14:10 ` Lynn Winebarger 2022-10-15 14:29 ` Eli Zaretskii 2022-10-16 11:55 ` Lynn Winebarger 2022-10-17 7:34 ` Andrea Corallo 2022-10-18 22:26 ` Lynn Winebarger 2022-10-19 19:33 ` Andrea Corallo 2022-10-15 15:10 ` Andrea Corallo 2022-10-15 16:17 ` Stefan Monnier 2022-10-17 7:20 ` Andrea Corallo 2022-10-15 7:04 ` Eli Zaretskii 2022-10-15 15:19 ` Andrea Corallo 2022-10-15 16:26 ` Eli Zaretskii 2022-10-15 17:19 ` Eli Zaretskii 2022-10-17 7:21 ` Andrea Corallo 2022-10-18 8:52 ` Andrea Corallo 2022-10-18 11:27 ` Lars Ingebrigtsen 2022-10-18 13:56 ` Andrea Corallo 2022-10-18 18:12 ` Lars Ingebrigtsen 2022-10-18 13:55 ` Stefan Monnier 2022-10-15 9:25 ` Lars Ingebrigtsen 2022-10-15 15:20 ` Andrea Corallo 2022-10-15 14:18 ` Lynn Winebarger 2022-10-15 17:06 ` Sean Whitton 2022-10-15 17:12 ` Eli Zaretskii 2022-10-15 17:36 ` Sean Whitton 2022-10-13 20:22 ` Eli Zaretskii 2022-10-14 19:46 ` Liliana Marie Prikler 2022-10-15 8:51 ` Eli Zaretskii 2022-10-15 15:53 ` Andrea Corallo 2022-10-15 9:33 ` Lars Ingebrigtsen 2022-10-15 14:35 ` Liliana Marie Prikler 2022-10-15 15:56 ` Andrea Corallo 2022-10-15 16:24 ` Liliana Marie Prikler 2022-10-17 7:23 ` Andrea Corallo 2022-10-17 16:59 ` Liliana Marie Prikler 2022-10-17 17:07 ` Eli Zaretskii 2022-10-16 8:55 ` Lars Ingebrigtsen 2022-10-16 9:27 ` Liliana Marie Prikler 2022-10-16 9:35 ` Lars Ingebrigtsen 2022-10-17 7:30 ` Andrea Corallo 2022-10-17 11:20 ` Lars Ingebrigtsen 2022-09-28 12:52 ` Andrea Corallo 2022-09-28 17:04 ` Eli Zaretskii 2022-09-28 18:49 ` Andrea Corallo 2022-09-28 19:10 ` Eli Zaretskii 2022-09-28 21:32 ` Andrea Corallo 2022-09-29 5:56 ` Eli Zaretskii 2022-09-29 8:18 ` Andrea Corallo 2022-09-29 9:01 ` Eli Zaretskii 2022-09-29 14:32 ` Andrea Corallo 2022-09-29 15:50 ` Eli Zaretskii
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).