* bug#57684: locked narrowing breaks existing code without an apparent way to repair
@ 2022-09-08 19:37 Paul Pogonyshev
2022-09-08 19:57 ` Gregory Heytings
0 siblings, 1 reply; 17+ messages in thread
From: Paul Pogonyshev @ 2022-09-08 19:37 UTC (permalink / raw)
To: 57684
[-- Attachment #1: Type: text/plain, Size: 1010 bytes --]
"Locked narrowing" added in Emacs 29 cannot be (temporarily) canceled by
code called inside. This in particular breaks Logview (
https://github.com/doublep/logview), because this narrowing is in effect
during fontification and causes Logview to fall down into an infinite loop.
Moreover, variable `restrictions-locked' appears to be not exposed to
Elisp, meaning that this cannot be cleared even with an explicit let-bind.
The mode simply cannot work without an ability to temporarily widen the
text. Macro `logview--std-temporarily-widening' is used 35 times in its
code:
(defmacro logview--std-temporarily-widening (&rest body)
(declare (indent 0) (debug t))
`(save-restriction
(let ((logview--point-min (logview--point-min))
(logview--point-max (logview--point-max)))
(widen)
,@body)))
To make it even harder to debug, Emacs sometimes hangs completely with even
C-g not aborting faulty code (in this case "faulty" because of incompatible
changes in Emacs itself).
Paul
[-- Attachment #2: Type: text/html, Size: 1249 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-08 19:37 bug#57684: locked narrowing breaks existing code without an apparent way to repair Paul Pogonyshev @ 2022-09-08 19:57 ` Gregory Heytings 2022-09-08 20:30 ` Paul Pogonyshev 2022-09-09 16:16 ` Lars Ingebrigtsen 0 siblings, 2 replies; 17+ messages in thread From: Gregory Heytings @ 2022-09-08 19:57 UTC (permalink / raw) To: Paul Pogonyshev; +Cc: 57684 > > "Locked narrowing" added in Emacs 29 cannot be (temporarily) canceled by > code called inside. > This issue is being worked on, see the (not yet finished) feature/improved-locked-narrowing branch. ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-08 19:57 ` Gregory Heytings @ 2022-09-08 20:30 ` Paul Pogonyshev 2022-09-09 1:46 ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors 2022-09-09 16:16 ` Lars Ingebrigtsen 1 sibling, 1 reply; 17+ messages in thread From: Paul Pogonyshev @ 2022-09-08 20:30 UTC (permalink / raw) To: Gregory Heytings; +Cc: 57684 [-- Attachment #1: Type: text/plain, Size: 1015 bytes --] Good to hear. I think in all cases it should be left to the coder to temporarily lift restrictions, even if those are "locked", maybe with strong warnings in documentation. Speed should always come a second priority to functionality. Besides, Logview (one example of what would suffer if widening is impossible) is very fast. As an example of why widening might be needed: fontification may depend on preceding text in the buffer. Maybe standard Emacs fontification code handles that separately, but e.g. Logview (almost) completely replaces the standard code here with a home-grown solution aimed specifically at log files. And actually this was done exactly for speed. Anyway, I hope this is finished soon. Paul On Thu, 8 Sept 2022 at 21:57, Gregory Heytings <gregory@heytings.org> wrote: > > > > > "Locked narrowing" added in Emacs 29 cannot be (temporarily) canceled by > > code called inside. > > > > This issue is being worked on, see the (not yet finished) > feature/improved-locked-narrowing branch. > [-- Attachment #2: Type: text/html, Size: 1529 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-08 20:30 ` Paul Pogonyshev @ 2022-09-09 1:46 ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors 0 siblings, 0 replies; 17+ messages in thread From: Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2022-09-09 1:46 UTC (permalink / raw) To: Paul Pogonyshev; +Cc: Gregory Heytings, 57684 Paul Pogonyshev <pogonyshev@gmail.com> writes: > Good to hear. I think in all cases it should be left to the coder to > temporarily lift restrictions, even if those are "locked", maybe > with strong warnings in documentation. Speed should always > come a second priority to functionality. Besides, Logview (one > example of what would suffer if widening is impossible) is very > fast. > > As an example of why widening might be needed: fontification > may depend on preceding text in the buffer. Maybe standard > Emacs fontification code handles that separately, but e.g. > Logview (almost) completely replaces the standard code here > with a home-grown solution aimed specifically at log files. And > actually this was done exactly for speed. > > Anyway, I hope this is finished soon. For the time being, you can either use an indirect buffer, or run the fontification code in a timer. The former already exists in the wild. ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-08 19:57 ` Gregory Heytings 2022-09-08 20:30 ` Paul Pogonyshev @ 2022-09-09 16:16 ` Lars Ingebrigtsen 2022-09-09 16:20 ` Gregory Heytings 1 sibling, 1 reply; 17+ messages in thread From: Lars Ingebrigtsen @ 2022-09-09 16:16 UTC (permalink / raw) To: Gregory Heytings; +Cc: Paul Pogonyshev, 57684 Gregory Heytings <gregory@heytings.org> writes: > This issue is being worked on, see the (not yet finished) > feature/improved-locked-narrowing branch. Is there much to be done on that branch before merging? ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-09 16:16 ` Lars Ingebrigtsen @ 2022-09-09 16:20 ` Gregory Heytings 2022-09-13 20:54 ` Paul Pogonyshev 0 siblings, 1 reply; 17+ messages in thread From: Gregory Heytings @ 2022-09-09 16:20 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Paul Pogonyshev, 57684 >> This issue is being worked on, see the (not yet finished) >> feature/improved-locked-narrowing branch. > > Is there much to be done on that branch before merging? > I don't know yet. Not "that much" I think, but there are a number of things that need to be fixed/improved. ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-09 16:20 ` Gregory Heytings @ 2022-09-13 20:54 ` Paul Pogonyshev 2022-09-13 21:06 ` Gregory Heytings 2022-09-14 2:34 ` Eli Zaretskii 0 siblings, 2 replies; 17+ messages in thread From: Paul Pogonyshev @ 2022-09-13 20:54 UTC (permalink / raw) To: Gregory Heytings; +Cc: Lars Ingebrigtsen, 57684 [-- Attachment #1: Type: text/plain, Size: 833 bytes --] By the way, today Emacs 29 hung in Magit blaming for me, with C-g doing nothing. Grepping Magit sources suggest it uses the "save-restriction - temporarily widen" more than ten times in various places, 3 of them when blaming. Cannot say for sure that was it, but all the outer symptoms are identical with the hangs in Logview. I really think there must be a way to "widen no matter which locks are installed" - a lot of code seems to depend on that. Paul On Fri, 9 Sept 2022 at 18:20, Gregory Heytings <gregory@heytings.org> wrote: > > >> This issue is being worked on, see the (not yet finished) > >> feature/improved-locked-narrowing branch. > > > > Is there much to be done on that branch before merging? > > > > I don't know yet. Not "that much" I think, but there are a number of > things that need to be fixed/improved. > [-- Attachment #2: Type: text/html, Size: 1246 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-13 20:54 ` Paul Pogonyshev @ 2022-09-13 21:06 ` Gregory Heytings 2022-09-13 21:13 ` Paul Pogonyshev 2022-09-14 2:34 ` Eli Zaretskii 1 sibling, 1 reply; 17+ messages in thread From: Gregory Heytings @ 2022-09-13 21:06 UTC (permalink / raw) To: Paul Pogonyshev; +Cc: Lars Ingebrigtsen, 57684 > > By the way, today Emacs 29 hung in Magit blaming for me, with C-g doing > nothing. Grepping Magit sources suggest it uses the "save-restriction - > temporarily widen" more than ten times in various places, 3 of them when > blaming. Cannot say for sure that was it, but all the outer symptoms are > identical with the hangs in Logview. I really think there must be a way > to "widen no matter which locks are installed" - a lot of code seems to > depend on that. > Yes, we know that, and as I said earlier it will be possible to unlock a locked narrowing. That being said, your description is too vague to draw a conclusion, but given that you tell that Emacs hung in Magit, I'm not quite sure it's locked narrowing that is the culprit. Locked narrowing is currently used only in buffers with very long lines, and only when those buffers are on display. Files with very long lines are typically machine-generated, and not under version control. ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-13 21:06 ` Gregory Heytings @ 2022-09-13 21:13 ` Paul Pogonyshev 2022-09-13 21:35 ` Gregory Heytings 0 siblings, 1 reply; 17+ messages in thread From: Paul Pogonyshev @ 2022-09-13 21:13 UTC (permalink / raw) To: Gregory Heytings; +Cc: Lars Ingebrigtsen, 57684 [-- Attachment #1: Type: text/plain, Size: 1711 bytes --] Well, probably not it then. > Files with very long lines are typically machine-generated Typically, but not always. The Logview hangs for me likely were because the logs I worked with contained long command lines for Java process starting (when you use a huge application with over 100 library JARs, classpath specification can get _really_ long). Actually, in a sense, those are machine-generated too, but the files are meant for human consumption, not e.g. as generated sources that are fed to a compiler. Anyway, I really hope something is done about this soon so that I don't have to switch between Emacs 29 and 28 all the time. Paul On Tue, 13 Sept 2022 at 23:06, Gregory Heytings <gregory@heytings.org> wrote: > > > > > By the way, today Emacs 29 hung in Magit blaming for me, with C-g doing > > nothing. Grepping Magit sources suggest it uses the "save-restriction - > > temporarily widen" more than ten times in various places, 3 of them when > > blaming. Cannot say for sure that was it, but all the outer symptoms are > > identical with the hangs in Logview. I really think there must be a way > > to "widen no matter which locks are installed" - a lot of code seems to > > depend on that. > > > > Yes, we know that, and as I said earlier it will be possible to unlock a > locked narrowing. That being said, your description is too vague to draw > a conclusion, but given that you tell that Emacs hung in Magit, I'm not > quite sure it's locked narrowing that is the culprit. Locked narrowing is > currently used only in buffers with very long lines, and only when those > buffers are on display. Files with very long lines are typically > machine-generated, and not under version control. > [-- Attachment #2: Type: text/html, Size: 2220 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-13 21:13 ` Paul Pogonyshev @ 2022-09-13 21:35 ` Gregory Heytings 0 siblings, 0 replies; 17+ messages in thread From: Gregory Heytings @ 2022-09-13 21:35 UTC (permalink / raw) To: Paul Pogonyshev; +Cc: Lars Ingebrigtsen, 57684 > > Anyway, I really hope something is done about this soon so that I don't > have to switch between Emacs 29 and 28 all the time. > Note that you can safely change long-line-threshold to a somewhat larger value, say 25000, or even 50000. ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-13 20:54 ` Paul Pogonyshev 2022-09-13 21:06 ` Gregory Heytings @ 2022-09-14 2:34 ` Eli Zaretskii 2022-09-14 9:45 ` Paul Pogonyshev 1 sibling, 1 reply; 17+ messages in thread From: Eli Zaretskii @ 2022-09-14 2:34 UTC (permalink / raw) To: Paul Pogonyshev; +Cc: gregory, larsi, 57684 > Cc: Lars Ingebrigtsen <larsi@gnus.org>, 57684@debbugs.gnu.org > From: Paul Pogonyshev <pogonyshev@gmail.com> > Date: Tue, 13 Sep 2022 22:54:07 +0200 > > By the way, today Emacs 29 hung in Magit blaming for me, with C-g doing nothing. Grepping Magit sources > suggest it uses the "save-restriction - temporarily widen" more than ten times in various places, 3 of them > when blaming. Cannot say for sure that was it, but all the outer symptoms are identical with the hangs in > Logview. > > I really think there must be a way to "widen no matter which locks are installed" - a lot of code seems to > depend on that. What does long-line-optimizations-p produce in Magit buffers? If it returns nil, what you see there has nothing to do with locked narrowing, which is only used when buffer text has very long lines. (Magit hides buffer text almost completely, presenting you what is largely overlays and 'display' properties, so you may not be aware of what the actual buffer text looks like.) ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-14 2:34 ` Eli Zaretskii @ 2022-09-14 9:45 ` Paul Pogonyshev 2022-09-14 11:57 ` Eli Zaretskii 0 siblings, 1 reply; 17+ messages in thread From: Paul Pogonyshev @ 2022-09-14 9:45 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Gregory Heytings, Lars Ingebrigtsen, 57684 [-- Attachment #1: Type: text/plain, Size: 1503 bytes --] No, that's not it, I have checked now. Also, blaming is done in normal file buffers, not in the status buffer. By the way, it would really be nice if Emacs could do something about hangs irrespective of what causes that. Even if Elisp code is buggy, Emacs itself should never allow it to fall into an infinite loop and stop responding to C-g, leaving full restart as the only way out. Paul On Wed, 14 Sept 2022 at 04:34, Eli Zaretskii <eliz@gnu.org> wrote: > > Cc: Lars Ingebrigtsen <larsi@gnus.org>, 57684@debbugs.gnu.org > > From: Paul Pogonyshev <pogonyshev@gmail.com> > > Date: Tue, 13 Sep 2022 22:54:07 +0200 > > > > By the way, today Emacs 29 hung in Magit blaming for me, with C-g doing > nothing. Grepping Magit sources > > suggest it uses the "save-restriction - temporarily widen" more than ten > times in various places, 3 of them > > when blaming. Cannot say for sure that was it, but all the outer > symptoms are identical with the hangs in > > Logview. > > > > I really think there must be a way to "widen no matter which locks are > installed" - a lot of code seems to > > depend on that. > > What does long-line-optimizations-p produce in Magit buffers? If it > returns nil, what you see there has nothing to do with locked > narrowing, which is only used when buffer text has very long lines. > > (Magit hides buffer text almost completely, presenting you what is > largely overlays and 'display' properties, so you may not be aware of > what the actual buffer text looks like.) > [-- Attachment #2: Type: text/html, Size: 2121 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-14 9:45 ` Paul Pogonyshev @ 2022-09-14 11:57 ` Eli Zaretskii 2022-09-16 3:38 ` Richard Stallman 0 siblings, 1 reply; 17+ messages in thread From: Eli Zaretskii @ 2022-09-14 11:57 UTC (permalink / raw) To: Paul Pogonyshev; +Cc: gregory, larsi, 57684 > From: Paul Pogonyshev <pogonyshev@gmail.com> > Date: Wed, 14 Sep 2022 11:45:01 +0200 > Cc: Gregory Heytings <gregory@heytings.org>, Lars Ingebrigtsen <larsi@gnus.org>, 57684@debbugs.gnu.org > > By the way, it would really be nice if Emacs could do something about hangs irrespective of what causes > that. Even if Elisp code is buggy, Emacs itself should never allow it to fall into an infinite loop and stop > responding to C-g, leaving full restart as the only way out. I think that's impossible in general, unless we restrict what Lisp programs can do. Every programming language can be used to write a buggy program. However, it should be possible to prevent some cases of such problematic behavior, certainly so when the infloop is caused by our bug. But for that we need to know the details of the specific case in order to investigate. ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-14 11:57 ` Eli Zaretskii @ 2022-09-16 3:38 ` Richard Stallman 2022-09-16 6:04 ` Eli Zaretskii 0 siblings, 1 reply; 17+ messages in thread From: Richard Stallman @ 2022-09-16 3:38 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 57684 [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > Even if Elisp code is buggy, Emacs itself should never allow it to fall into an infinite loop and stop > > responding to C-g, leaving full restart as the only way out. It used to be impossible to have an infinite loop in Emacs Lisp that you could not quit out of. The Lisp interpreter and tye bytecode interpreter both had calls to QUIT in all the loops of Lisp execution. Likewise, all the loops in C code that corresponded to Lisp loops, and might fail to terminate if given circular lists, such as Fmemq, had calls to QUIT. If that is now no longertrue, what made it cease to be true? -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-16 3:38 ` Richard Stallman @ 2022-09-16 6:04 ` Eli Zaretskii 2022-09-17 3:40 ` Richard Stallman 0 siblings, 1 reply; 17+ messages in thread From: Eli Zaretskii @ 2022-09-16 6:04 UTC (permalink / raw) To: rms; +Cc: 57684 > From: Richard Stallman <rms@gnu.org> > Cc: 57684@debbugs.gnu.org > Date: Thu, 15 Sep 2022 23:38:15 -0400 > > > Even if Elisp code is buggy, Emacs itself should never allow it to fall into an infinite loop and stop > > > responding to C-g, leaving full restart as the only way out. > > It used to be impossible to have an infinite loop in Emacs Lisp that > you could not quit out of. The Lisp interpreter and tye bytecode > interpreter both had calls to QUIT in all the loops of Lisp execution. > Likewise, all the loops in C code that corresponded to Lisp loops, and > might fail to terminate if given circular lists, such as Fmemq, > had calls to QUIT. > > If that is now no longertrue, what made it cease to be true? It is still true. The problem exists only with Lisp code that is run when QUITs are either intentionally disabled, or when Lisp is run from the display engine, where QUIT is caught and basically ignored. The particular case which started this thread is of the latter variety. ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-16 6:04 ` Eli Zaretskii @ 2022-09-17 3:40 ` Richard Stallman 2022-09-17 6:33 ` Eli Zaretskii 0 siblings, 1 reply; 17+ messages in thread From: Richard Stallman @ 2022-09-17 3:40 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 57684 [[[ To any NSA and FBI agents reading my email: please consider ]]] [[[ whether defending the US Constitution against all enemies, ]]] [[[ foreign or domestic, requires you to follow Snowden's example. ]]] > It is still true. The problem exists only with Lisp code that is run > when QUITs are either intentionally disabled, or when Lisp is run from > the display engine, where QUIT is caught and basically ignored. The > particular case which started this thread is of the latter variety. I never allowed redisplay to run Lisp code -- it seemed very dangerous. An error there could leave redisplay data structures inconsistent. -- Dr Richard Stallman (https://stallman.org) Chief GNUisance of the GNU Project (https://gnu.org) Founder, Free Software Foundation (https://fsf.org) Internet Hall-of-Famer (https://internethalloffame.org) ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#57684: locked narrowing breaks existing code without an apparent way to repair 2022-09-17 3:40 ` Richard Stallman @ 2022-09-17 6:33 ` Eli Zaretskii 0 siblings, 0 replies; 17+ messages in thread From: Eli Zaretskii @ 2022-09-17 6:33 UTC (permalink / raw) To: rms; +Cc: 57684 > From: Richard Stallman <rms@gnu.org> > Cc: 57684@debbugs.gnu.org > Date: Fri, 16 Sep 2022 23:40:19 -0400 > > > It is still true. The problem exists only with Lisp code that is run > > when QUITs are either intentionally disabled, or when Lisp is run from > > the display engine, where QUIT is caught and basically ignored. The > > particular case which started this thread is of the latter variety. > > I never allowed redisplay to run Lisp code -- it seemed very dangerous. > An error there could leave redisplay data structures inconsistent. That ship sailed a long time ago: Emacs 21 introduced jit-lock.el, which is a way of running font-lock as part of redisplay, on the portion of the buffer which is about to be displayed. This is on by default since Emacs 21. In addition, we have several hooks, notably window-scroll-functions, which redisplay runs when appropriate. Search xdisp.c for "safe_call" to see where and why we call Lisp from redisplay. Yes, it's somewhat dangerous, and caused us nontrivial issues over the years, but I don't think it's possible to go back to not calling Lisp from the display code. ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2022-09-17 6:33 UTC | newest] Thread overview: 17+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2022-09-08 19:37 bug#57684: locked narrowing breaks existing code without an apparent way to repair Paul Pogonyshev 2022-09-08 19:57 ` Gregory Heytings 2022-09-08 20:30 ` Paul Pogonyshev 2022-09-09 1:46 ` Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors 2022-09-09 16:16 ` Lars Ingebrigtsen 2022-09-09 16:20 ` Gregory Heytings 2022-09-13 20:54 ` Paul Pogonyshev 2022-09-13 21:06 ` Gregory Heytings 2022-09-13 21:13 ` Paul Pogonyshev 2022-09-13 21:35 ` Gregory Heytings 2022-09-14 2:34 ` Eli Zaretskii 2022-09-14 9:45 ` Paul Pogonyshev 2022-09-14 11:57 ` Eli Zaretskii 2022-09-16 3:38 ` Richard Stallman 2022-09-16 6:04 ` Eli Zaretskii 2022-09-17 3:40 ` Richard Stallman 2022-09-17 6:33 ` Eli Zaretskii
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).