* bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling
@ 2024-12-08 13:05 rehan malak
2024-12-08 17:37 ` Eli Zaretskii
0 siblings, 1 reply; 11+ messages in thread
From: rehan malak @ 2024-12-08 13:05 UTC (permalink / raw)
To: 74738
[-- Attachment #1: Type: text/plain, Size: 8963 bytes --]
Hi,
I can reproduce systematically this freeze dealing with a 10000lines
Python file :
wget https://raw.githubusercontent.com/hugsy/gef/refs/heads/main/gef.py
emacs -Q gef.py
PageDown several times or scrolling with the mouse
Emacs freezes, Ctrl-g not working, CPU 100%
It works also by searching : Ctrl-s show RET
then Ctrl-s several times
With a minimal .emacs :
(set debug-on-error t)
(set debug-on-quit t)
and before scrolling
M-x profiler-start RET
then in an external terminal
pkill -SIGUSR2 emacs
I get the backtrace :
Debugger entered--entering a function:
* #f(compiled-function () #<bytecode -0x179dcd1db31182ae>)()
syntax-ppss()
python-syntax-context-type()
python-nav-forward-block(-1)
python-nav-backward-block()
python-nav-beginning-of-block()
python-nav-end-of-block()
python-info-statement-ends-block-p()
python-info-end-of-block-p()
python-nav--forward-sexp(1 nil nil)
python-nav-forward-sexp(1)
forward-sexp(1)
up-list(1)
python--font-lock-f-strings(30419)
font-lock-fontify-keywords-region(28914 30419 nil)
font-lock-default-fontify-region(28914 30414 nil)
font-lock-fontify-region(28914 30414)
#f(compiled-function (fun) #<bytecode
-0x8a96d65e1315e71>)(font-lock-fontify-region)
run-hook-wrapped(#f(compiled-function (fun) #<bytecode
-0x8a96d65e1315e71>) font-lock-fontify-region)
jit-lock--run-functions(28914 30414)
jit-lock-fontify-now(28914 30414)
jit-lock-function(28914)
and can look at profiler report
M-x profiler-stop
M-x profiler-report
showing time spent in syntax-ppss
17770 88% - jit-lock-function
17770 88% - jit-lock-fontify-now
17770 88% - jit-lock--run-functions
17770 88% - run-hook-wrapped
17770 88% - #<byte-code-function FA0>
17770 88% - font-lock-fontify-region
17770 88% - font-lock-default-fontify-region
17690 87% - font-lock-fontify-keywords-region
17678 87% - python--font-lock-f-strings
17674 87% - up-list
17674 87% - forward-sexp
17674 87% - python-nav-forward-sexp
17674 87% - python-nav--forward-sexp
9253 45% - python-info-end-of-block-p
9245 45% - python-info-statement-ends-block-p
9221 45% - python-nav-end-of-block
8609 42% - python-nav-beginning-of-block
8557 42% - python-nav-backward-block
8557 42% - python-nav-forward-block
8005 39% - python-syntax-context-type
7997 39% - syntax-ppss
7097 35% parse-partial-sexp
753 3% + #<byte-code-function A23>
376 1% + python-nav-beginning-of-statement
132 0% re-search-backward
24 0% looking-at
28 0% current-indentation
8 0% + python-nav-beginning-of-statement
4 0% python-info-current-line-empty-p
4 0% + python-info-current-line-comment-p
604 2% + python-nav-end-of-statement
4 0% + python-info-current-line-comment-p
4 0% current-indentation
24 0% + python-nav-end-of-statement
8 0% + python-info-end-of-statement-p
8178 40% - python-info-statement-ends-block-p
8150 40% - python-nav-end-of-block
7754 38% - python-nav-beginning-of-block
7738 38% - python-nav-backward-block
7734 38% - python-nav-forward-block
7178 35% - python-syntax-context-type
7174 35% - syntax-ppss
6978 34% parse-partial-sexp
8 0% make-closure
8 0% #<byte-code-function B1E>
4 0% syntax-table
4 0% syntax-ppss--update-stats
412 2% + python-nav-beginning-of-statement
92 0% re-search-backward
44 0% looking-at
12 0% current-indentation
4 0% looking-at
380 1% + python-nav-end-of-statement
16 0% + python-util-forward-comment
28 0% + python-nav-end-of-statement
151 0% + python-nav-end-of-block
28 0% + python-info-beginning-of-block-p
24 0% + python-info-end-of-statement-p
20 0% + python-info-beginning-of-statement-p
8 0% + python-syntax-context-type
8 0% + python-info-statement-starts-block-p
4 0% + syntax-ppss
8 0% + #<byte-code-function 7E0>
4 0% + #<byte-code-function 740>
80 0% + font-lock-fontify-syntactically-region
2279 11% Automatic GC
73 0% + redisplay_internal (C function)
40 0% + command-execute
8 0% + ...
4 0% + mouse--click-1-maybe-follows-link
other people talking about python/profiling/syntax-ppss :
https://www.reddit.com/r/emacs/comments/z0oye9/emacs_freezes_when_opening_a_python_file/
https://www.reddit.com/r/emacs/comments/9h7onq/large_python_files_hanging_with_an_unmatched_quote/
Please tell me how I can help ?
Please tell me if there is a workaround without losing python syntax color ?
Thanks
Rehan
In GNU Emacs 31.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
3.24.43, cairo version 1.18.2) of 2024-12-08 built on cndl
Repository revision: b953bcb17047998c9e41cede7c5e5ffec22209b2
Repository branch: master
System Description: Debian GNU/Linux trixie/sid
Configured using:
'configure --prefix=/home/jean/local/emacs --with-pgtk'
Configured features:
CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG LIBSELINUX
LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PGTK PNG SECCOMP SOUND THREADS
TIFF TOOLKIT_SCROLL_BARS WEBP XIM GTK3 ZLIB
Important settings:
value of $LANG: en_US.UTF-8
value of $XMODIFIERS: @im=ibus
locale-coding-system: utf-8-unix
Major mode: Python
Minor modes in effect:
tooltip-mode: t
global-eldoc-mode: t
eldoc-mode: t
show-paren-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
minibuffer-regexp-mode: t
line-number-mode: t
transient-mark-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
Load-path shadows:
None found.
Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils cl-seq python
rx project byte-opt gv bytecomp byte-compile compat pcase treesit comint
subr-x ansi-osc ring cl-loaddefs cl-lib ansi-color rmc iso-transl
tooltip cconv eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/pgtk-win pgtk-win term/common-win
touch-screen pgtk-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer nadvice seq simple cl-generic indonesian philippine
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help abbrev obarray oclosure
cl-preloaded button loaddefs theme-loaddefs faces cus-face macroexp
files window text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget keymap hashtable-print-readable backquote
threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo gtk pgtk multi-tty move-toolbar
make-network-process emacs)
Memory information:
((conses 16 60995 13527) (symbols 48 7517 0) (strings 32 19122 1493)
(string-bytes 1 515162) (vectors 16 13554)
(vector-slots 8 145472 8433) (floats 8 26 2) (intervals 56 436 0)
(buffers 984 12))
[-- Attachment #2: Type: text/html, Size: 10952 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling
2024-12-08 13:05 bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling rehan malak
@ 2024-12-08 17:37 ` Eli Zaretskii
2024-12-09 14:58 ` rehan malak
2024-12-09 15:04 ` kobarity
0 siblings, 2 replies; 11+ messages in thread
From: Eli Zaretskii @ 2024-12-08 17:37 UTC (permalink / raw)
To: rehan malak, Stefan Monnier, kobarity; +Cc: 74738
> From: rehan malak <rehan.malak@gmail.com>
> Date: Sun, 8 Dec 2024 14:05:36 +0100
>
> I can reproduce systematically this freeze dealing with a 10000lines
> Python file :
>
> wget https://raw.githubusercontent.com/hugsy/gef/refs/heads/main/gef.py
> emacs -Q gef.py
A much smaller reproducer is attached below. Just visiting it freezes
Emacs. But if I change the line marked below:
@classmethod
def is_valid(cls, _: pathlib.Path) -> bool:
raise NotImplementedError
def __str__(self) -> str:
>>>>> return f"{self.name}('{self.path.absolute()}', entry @ {self.entry_point:#x})"
to say this instead:
return self.name
the problem goes away. So it is something in that complex expression
that trips syntax-ppss.
Stefan and kobarity, any suggestions or ideas?
> PageDown several times or scrolling with the mouse
>
> Emacs freezes, Ctrl-g not working, CPU 100%
>
> It works also by searching : Ctrl-s show RET
> then Ctrl-s several times
>
> With a minimal .emacs :
>
> (set debug-on-error t)
> (set debug-on-quit t)
>
> and before scrolling
>
> M-x profiler-start RET
>
> then in an external terminal
>
> pkill -SIGUSR2 emacs
>
> I get the backtrace :
>
> Debugger entered--entering a function:
> * #f(compiled-function () #<bytecode -0x179dcd1db31182ae>)()
> syntax-ppss()
> python-syntax-context-type()
> python-nav-forward-block(-1)
> python-nav-backward-block()
> python-nav-beginning-of-block()
> python-nav-end-of-block()
> python-info-statement-ends-block-p()
> python-info-end-of-block-p()
> python-nav--forward-sexp(1 nil nil)
> python-nav-forward-sexp(1)
> forward-sexp(1)
> up-list(1)
> python--font-lock-f-strings(30419)
> font-lock-fontify-keywords-region(28914 30419 nil)
> font-lock-default-fontify-region(28914 30414 nil)
> font-lock-fontify-region(28914 30414)
> #f(compiled-function (fun) #<bytecode -0x8a96d65e1315e71>)(font-lock-fontify-region)
> run-hook-wrapped(#f(compiled-function (fun) #<bytecode -0x8a96d65e1315e71>) font-lock-fontify-region)
> jit-lock--run-functions(28914 30414)
> jit-lock-fontify-now(28914 30414)
> jit-lock-function(28914)
I see something different, in GDB:
Lisp Backtrace:
"parse-partial-sexp" (0x9f1a788)
"syntax-ppss" (0x9f1a710)
"python-info-line-ends-backslash-p" (0x9f1a6b8)
"python-nav-end-of-statement" (0x9f1a670)
"python-nav-end-of-block" (0x9f1a630)
"python-info-statement-ends-block-p" (0x9f1a610)
"python-info-end-of-block-p" (0x9f1a5c0)
"python-nav--forward-sexp" (0x9f1a548)
"python-nav-forward-sexp" (0x9f1a4f8)
"forward-sexp" (0x9f1a4a8)
"up-list" (0x9f1a438)
"python--font-lock-f-strings" (0x9f1a3a0)
"font-lock-fontify-keywords-region" (0x9f1a308)
"font-lock-default-fontify-region" (0x9f1a2a0)
"font-lock-fontify-region" (0x9f1a230)
0xb9117d0 PVEC_CLOSURE
"run-hook-wrapped" (0x9f1a1c0)
"jit-lock--run-functions" (0x9f1a0e8)
"jit-lock-fontify-now" (0x9f1a058)
"jit-lock-function" (0x5ffba98)
"redisplay_internal (C function)" (0x0)
Here's the file with which I can reproduce the problem?
class FileFormat:
name: str
path: pathlib.Path
entry_point: int
checksec: dict[str, bool]
sections: list[FileFormatSection]
def __init__(self, path: str | pathlib.Path) -> None:
raise NotImplementedError
def __init_subclass__(cls: Type["FileFormat"], **kwargs):
global __registered_file_formats__
super().__init_subclass__(**kwargs)
required_attributes = ("name", "entry_point", "is_valid", "checksec",)
for attr in required_attributes:
if not hasattr(cls, attr):
raise NotImplementedError(f"File format '{cls.__name__}' is invalid: missing attribute '{attr}'")
__registered_file_formats__.add(cls)
return
@classmethod
def is_valid(cls, _: pathlib.Path) -> bool:
raise NotImplementedError
def __str__(self) -> str:
return f"{self.name}('{self.path.absolute()}', entry @ {self.entry_point:#x})"
class Elf(FileFormat):
"""Basic ELF parsing.
Ref:
- http://www.skyfree.org/linux/references/ELF_Format.pdf
- https://refspecs.linuxfoundation.org/elf/elfspec_ppc.pdf
- https://refspecs.linuxfoundation.org/ELF/ppc64/PPC-elf64abi.html
"""
class Class(enum.Enum):
ELF_32_BITS = 0x01
ELF_64_BITS = 0x02
ELF_MAGIC = 0x7f454c46
class Abi(enum.Enum):
X86_64 = 0x3e
X86_32 = 0x03
ARM = 0x28
MIPS = 0x08
POWERPC = 0x14
POWERPC64 = 0x15
SPARC = 0x02
SPARC64 = 0x2b
AARCH64 = 0xb7
RISCV = 0xf3
IA64 = 0x32
M68K = 0x04
class Type(enum.Enum):
ET_RELOC = 1
ET_EXEC = 2
ET_DYN = 3
ET_CORE = 4
class OsAbi(enum.Enum):
SYSTEMV = 0x00
HPUX = 0x01
NETBSD = 0x02
LINUX = 0x03
SOLARIS = 0x06
AIX = 0x07
IRIX = 0x08
FREEBSD = 0x09
OPENBSD = 0x0C
e_magic: int = ELF_MAGIC
e_class: "Elf.Class" = Class.ELF_32_BITS
e_endianness: Endianness = Endianness.LITTLE_ENDIAN
e_eiversion: int
e_osabi: "Elf.OsAbi"
e_abiversion: int
e_pad: bytes
e_type: "Elf.Type" = Type.ET_EXEC
e_machine: Abi = Abi.X86_32
e_version: int
e_entry: int
e_phoff: int
e_shoff: int
e_flags: int
e_ehsize: int
e_phentsize: int
e_phnum: int
e_shentsize: int
e_shnum: int
e_shstrndx: int
path: pathlib.Path
phdrs : list["Phdr"]
shdrs : list["Shdr"]
name: str = "ELF"
__checksec : dict[str, bool]
def __init__(self, path: str | pathlib.Path) -> None:
"""Instantiate an ELF object. A valid ELF must be provided, or an exception will be thrown."""
if isinstance(path, str):
self.path = pathlib.Path(path).expanduser()
elif isinstance(path, pathlib.Path):
self.path = path
else:
raise TypeError
if not self.path.exists():
raise FileNotFoundError(f"'{self.path}' not found/readable, most gef features will not work")
self.__checksec = {}
with self.path.open("rb") as self.fd:
# off 0x0
self.e_magic, e_class, e_endianness, self.e_eiversion = self.read_and_unpack(">IBBB")
if self.e_magic != Elf.ELF_MAGIC:
# The ELF is corrupted, GDB won't handle it, no point going further
raise RuntimeError("Not a valid ELF file (magic)")
self.e_class, self.e_endianness = Elf.Class(e_class), Endianness(e_endianness)
if self.e_endianness != gef.arch.endianness:
warn("Unexpected endianness for architecture")
endian = self.e_endianness
# off 0x7
e_osabi, self.e_abiversion = self.read_and_unpack(f"{endian}BB")
self.e_osabi = Elf.OsAbi(e_osabi)
# off 0x9
self.e_pad = self.read(7)
# off 0x10
e_type, e_machine, self.e_version = self.read_and_unpack(f"{endian}HHI")
self.e_type, self.e_machine = Elf.Type(e_type), Elf.Abi(e_machine)
# off 0x18
if self.e_class == Elf.Class.ELF_64_BITS:
self.e_entry, self.e_phoff, self.e_shoff = self.read_and_unpack(f"{endian}QQQ")
else:
self.e_entry, self.e_phoff, self.e_shoff = self.read_and_unpack(f"{endian}III")
self.e_flags, self.e_ehsize, self.e_phentsize, self.e_phnum = self.read_and_unpack(f"{endian}IHHH")
self.e_shentsize, self.e_shnum, self.e_shstrndx = self.read_and_unpack(f"{endian}HHH")
self.phdrs = []
for i in range(self.e_phnum):
self.phdrs.append(Phdr(self, self.e_phoff + self.e_phentsize * i))
self.shdrs = []
for i in range(self.e_shnum):
self.shdrs.append(Shdr(self, self.e_shoff + self.e_shentsize * i))
return
def read(self, size: int) -> bytes:
return self.fd.read(size)
def read_and_unpack(self, fmt: str) -> tuple[Any, ...]:
size = struct.calcsize(fmt)
data = self.fd.read(size)
return struct.unpack(fmt, data)
def seek(self, off: int) -> None:
self.fd.seek(off, 0)
def __str__(self) -> str:
return f"ELF('{self.path.absolute()}', {self.e_class.name}, {self.e_machine.name})"
def __repr__(self) -> str:
return f"ELF('{self.path.absolute()}', {self.e_class.name}, {self.e_machine.name})"
@property
def entry_point(self) -> int:
return self.e_entry
@classmethod
def is_valid(cls, path: pathlib.Path) -> bool:
return u32(path.open("rb").read(4), e = Endianness.BIG_ENDIAN) == Elf.ELF_MAGIC
@property
def checksec(self) -> dict[str, bool]:
"""Check the security property of the ELF binary. The following properties are:
- Canary
- NX
- PIE
- Fortify
- Partial/Full RelRO.
Return a dict() with the different keys mentioned above, and the boolean
associated whether the protection was found."""
if not self.__checksec:
def __check_security_property(opt: str, filename: str, pattern: str) -> bool:
cmd = [readelf,]
cmd += opt.split()
cmd += [filename,]
lines = gef_execute_external(cmd, as_list=True)
for line in lines:
if re.search(pattern, line):
return True
return False
abspath = str(self.path.absolute())
readelf = gef.session.constants["readelf"]
self.__checksec["Canary"] = __check_security_property("-rs", abspath, r"__stack_chk_fail") is True
has_gnu_stack = __check_security_property("-W -l", abspath, r"GNU_STACK") is True
if has_gnu_stack:
self.__checksec["NX"] = __check_security_property("-W -l", abspath, r"GNU_STACK.*RWE") is False
else:
self.__checksec["NX"] = False
self.__checksec["PIE"] = __check_security_property("-h", abspath, r":.*EXEC") is False
self.__checksec["Fortify"] = __check_security_property("-s", abspath, r"_chk@GLIBC") is True
self.__checksec["Partial RelRO"] = __check_security_property("-l", abspath, r"GNU_RELRO") is True
self.__checksec["Full RelRO"] = self.__checksec["Partial RelRO"] and __check_security_property("-d", abspath, r"BIND_NOW") is True
return self.__checksec
@classproperty
@deprecated("use `Elf.Abi.X86_64`")
def X86_64(cls) -> int: return Elf.Abi.X86_64.value # pylint: disable=no-self-argument
@classproperty
@deprecated("use `Elf.Abi.X86_32`")
def X86_32(cls) -> int : return Elf.Abi.X86_32.value # pylint: disable=no-self-argument
@classproperty
@deprecated("use `Elf.Abi.ARM`")
def ARM(cls) -> int : return Elf.Abi.ARM.value # pylint: disable=no-self-argument
@classproperty
@deprecated("use `Elf.Abi.MIPS`")
def MIPS(cls) -> int : return Elf.Abi.MIPS.value # pylint: disable=no-self-argument
@classproperty
@deprecated("use `Elf.Abi.POWERPC`")
def POWERPC(cls) -> int : return Elf.Abi.POWERPC.value # pylint: disable=no-self-argument
@classproperty
@deprecated("use `Elf.Abi.POWERPC64`")
def POWERPC64(cls) -> int : return Elf.Abi.POWERPC64.value # pylint: disable=no-self-argument
@classproperty
@deprecated("use `Elf.Abi.SPARC`")
def SPARC(cls) -> int : return Elf.Abi.SPARC.value # pylint: disable=no-self-argument
@classproperty
@deprecated("use `Elf.Abi.SPARC64`")
def SPARC64(cls) -> int : return Elf.Abi.SPARC64.value # pylint: disable=no-self-argument
@classproperty
@deprecated("use `Elf.Abi.AARCH64`")
def AARCH64(cls) -> int : return Elf.Abi.AARCH64.value # pylint: disable=no-self-argument
@classproperty
@deprecated("use `Elf.Abi.RISCV`")
def RISCV(cls) -> int : return Elf.Abi.RISCV.value # pylint: disable=no-self-argument
^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling
2024-12-08 17:37 ` Eli Zaretskii
@ 2024-12-09 14:58 ` rehan malak
2024-12-11 14:24 ` kobarity
2024-12-09 15:04 ` kobarity
1 sibling, 1 reply; 11+ messages in thread
From: rehan malak @ 2024-12-09 14:58 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: kobarity, Stefan Monnier, 74738
[-- Attachment #1: Type: text/plain, Size: 14703 bytes --]
Hi,
Deleting all the "#" in the f-string of the gef.py file removes the problem
on my side.
sed -i -e 's/#0{align/0{align/g' gef.py
sed -i -e 's/#0{width/0{width/g' gef.py
sed -i -e 's/#07x}/07x}/g' gef.py
sed -i -e 's/#06x}/06x}/g' gef.py
sed -i -e 's/#04x}/04x}/g' gef.py
sed -i -e 's/#4x}/4x}/g' gef.py
sed -i -e 's/#8x}/8x}/g' gef.py
sed -i -e 's/#10x}/10x}/g' gef.py
sed -i -e 's/#x}/x}/g' gef.py
Your smaller example Eli also contains a f-string with the "#".
This python f-string format specifier is described here:
https://docs.python.org/3/library/string.html#format-specification-mini-language
Is this format specifier supported from the beginning ?
For example :
value = 0xab
print(f"{value:x} is a value")
=> color syntax : *is a value* has the color defined by
font-lock-string-face
while
value = 0xab
print(f"{value:#x} is a value")
=> color are messed up... *is* has color defined by font-lock-keyword-face
and *a value* has color defined by default face
Best,
Rehan
On Sun, Dec 8, 2024 at 6:37 PM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: rehan malak <rehan.malak@gmail.com>
> > Date: Sun, 8 Dec 2024 14:05:36 +0100
> >
> > I can reproduce systematically this freeze dealing with a 10000lines
> > Python file :
> >
> > wget https://raw.githubusercontent.com/hugsy/gef/refs/heads/main/gef.py
> > emacs -Q gef.py
>
> A much smaller reproducer is attached below. Just visiting it freezes
> Emacs. But if I change the line marked below:
>
> @classmethod
> def is_valid(cls, _: pathlib.Path) -> bool:
> raise NotImplementedError
>
> def __str__(self) -> str:
> >>>>> return f"{self.name}('{self.path.absolute()}', entry @
> {self.entry_point:#x})"
>
> to say this instead:
>
> return self.name
>
> the problem goes away. So it is something in that complex expression
> that trips syntax-ppss.
>
> Stefan and kobarity, any suggestions or ideas?
>
> > PageDown several times or scrolling with the mouse
> >
> > Emacs freezes, Ctrl-g not working, CPU 100%
> >
> > It works also by searching : Ctrl-s show RET
> > then Ctrl-s several times
> >
> > With a minimal .emacs :
> >
> > (set debug-on-error t)
> > (set debug-on-quit t)
> >
> > and before scrolling
> >
> > M-x profiler-start RET
> >
> > then in an external terminal
> >
> > pkill -SIGUSR2 emacs
> >
> > I get the backtrace :
> >
> > Debugger entered--entering a function:
> > * #f(compiled-function () #<bytecode -0x179dcd1db31182ae>)()
> > syntax-ppss()
> > python-syntax-context-type()
> > python-nav-forward-block(-1)
> > python-nav-backward-block()
> > python-nav-beginning-of-block()
> > python-nav-end-of-block()
> > python-info-statement-ends-block-p()
> > python-info-end-of-block-p()
> > python-nav--forward-sexp(1 nil nil)
> > python-nav-forward-sexp(1)
> > forward-sexp(1)
> > up-list(1)
> > python--font-lock-f-strings(30419)
> > font-lock-fontify-keywords-region(28914 30419 nil)
> > font-lock-default-fontify-region(28914 30414 nil)
> > font-lock-fontify-region(28914 30414)
> > #f(compiled-function (fun) #<bytecode
> -0x8a96d65e1315e71>)(font-lock-fontify-region)
> > run-hook-wrapped(#f(compiled-function (fun) #<bytecode
> -0x8a96d65e1315e71>) font-lock-fontify-region)
> > jit-lock--run-functions(28914 30414)
> > jit-lock-fontify-now(28914 30414)
> > jit-lock-function(28914)
>
> I see something different, in GDB:
>
> Lisp Backtrace:
> "parse-partial-sexp" (0x9f1a788)
> "syntax-ppss" (0x9f1a710)
> "python-info-line-ends-backslash-p" (0x9f1a6b8)
> "python-nav-end-of-statement" (0x9f1a670)
> "python-nav-end-of-block" (0x9f1a630)
> "python-info-statement-ends-block-p" (0x9f1a610)
> "python-info-end-of-block-p" (0x9f1a5c0)
> "python-nav--forward-sexp" (0x9f1a548)
> "python-nav-forward-sexp" (0x9f1a4f8)
> "forward-sexp" (0x9f1a4a8)
> "up-list" (0x9f1a438)
> "python--font-lock-f-strings" (0x9f1a3a0)
> "font-lock-fontify-keywords-region" (0x9f1a308)
> "font-lock-default-fontify-region" (0x9f1a2a0)
> "font-lock-fontify-region" (0x9f1a230)
> 0xb9117d0 PVEC_CLOSURE
> "run-hook-wrapped" (0x9f1a1c0)
> "jit-lock--run-functions" (0x9f1a0e8)
> "jit-lock-fontify-now" (0x9f1a058)
> "jit-lock-function" (0x5ffba98)
> "redisplay_internal (C function)" (0x0)
>
> Here's the file with which I can reproduce the problem?
>
>
> class FileFormat:
> name: str
> path: pathlib.Path
> entry_point: int
> checksec: dict[str, bool]
> sections: list[FileFormatSection]
>
> def __init__(self, path: str | pathlib.Path) -> None:
> raise NotImplementedError
>
> def __init_subclass__(cls: Type["FileFormat"], **kwargs):
> global __registered_file_formats__
> super().__init_subclass__(**kwargs)
> required_attributes = ("name", "entry_point", "is_valid",
> "checksec",)
> for attr in required_attributes:
> if not hasattr(cls, attr):
> raise NotImplementedError(f"File format '{cls.__name__}'
> is invalid: missing attribute '{attr}'")
> __registered_file_formats__.add(cls)
> return
>
> @classmethod
> def is_valid(cls, _: pathlib.Path) -> bool:
> raise NotImplementedError
>
> def __str__(self) -> str:
> return f"{self.name}('{self.path.absolute()}', entry @
> {self.entry_point:#x})"
>
>
> class Elf(FileFormat):
> """Basic ELF parsing.
> Ref:
> - http://www.skyfree.org/linux/references/ELF_Format.pdf
> - https://refspecs.linuxfoundation.org/elf/elfspec_ppc.pdf
> - https://refspecs.linuxfoundation.org/ELF/ppc64/PPC-elf64abi.html
> """
> class Class(enum.Enum):
> ELF_32_BITS = 0x01
> ELF_64_BITS = 0x02
>
> ELF_MAGIC = 0x7f454c46
>
> class Abi(enum.Enum):
> X86_64 = 0x3e
> X86_32 = 0x03
> ARM = 0x28
> MIPS = 0x08
> POWERPC = 0x14
> POWERPC64 = 0x15
> SPARC = 0x02
> SPARC64 = 0x2b
> AARCH64 = 0xb7
> RISCV = 0xf3
> IA64 = 0x32
> M68K = 0x04
>
> class Type(enum.Enum):
> ET_RELOC = 1
> ET_EXEC = 2
> ET_DYN = 3
> ET_CORE = 4
>
> class OsAbi(enum.Enum):
> SYSTEMV = 0x00
> HPUX = 0x01
> NETBSD = 0x02
> LINUX = 0x03
> SOLARIS = 0x06
> AIX = 0x07
> IRIX = 0x08
> FREEBSD = 0x09
> OPENBSD = 0x0C
>
> e_magic: int = ELF_MAGIC
> e_class: "Elf.Class" = Class.ELF_32_BITS
> e_endianness: Endianness = Endianness.LITTLE_ENDIAN
> e_eiversion: int
> e_osabi: "Elf.OsAbi"
> e_abiversion: int
> e_pad: bytes
> e_type: "Elf.Type" = Type.ET_EXEC
> e_machine: Abi = Abi.X86_32
> e_version: int
> e_entry: int
> e_phoff: int
> e_shoff: int
> e_flags: int
> e_ehsize: int
> e_phentsize: int
> e_phnum: int
> e_shentsize: int
> e_shnum: int
> e_shstrndx: int
>
> path: pathlib.Path
> phdrs : list["Phdr"]
> shdrs : list["Shdr"]
> name: str = "ELF"
>
> __checksec : dict[str, bool]
>
> def __init__(self, path: str | pathlib.Path) -> None:
> """Instantiate an ELF object. A valid ELF must be provided, or an
> exception will be thrown."""
>
> if isinstance(path, str):
> self.path = pathlib.Path(path).expanduser()
> elif isinstance(path, pathlib.Path):
> self.path = path
> else:
> raise TypeError
>
> if not self.path.exists():
> raise FileNotFoundError(f"'{self.path}' not found/readable,
> most gef features will not work")
>
> self.__checksec = {}
>
> with self.path.open("rb") as self.fd:
> # off 0x0
> self.e_magic, e_class, e_endianness, self.e_eiversion =
> self.read_and_unpack(">IBBB")
> if self.e_magic != Elf.ELF_MAGIC:
> # The ELF is corrupted, GDB won't handle it, no point
> going further
> raise RuntimeError("Not a valid ELF file (magic)")
>
> self.e_class, self.e_endianness = Elf.Class(e_class),
> Endianness(e_endianness)
>
> if self.e_endianness != gef.arch.endianness:
> warn("Unexpected endianness for architecture")
>
> endian = self.e_endianness
>
> # off 0x7
> e_osabi, self.e_abiversion =
> self.read_and_unpack(f"{endian}BB")
> self.e_osabi = Elf.OsAbi(e_osabi)
>
> # off 0x9
> self.e_pad = self.read(7)
>
> # off 0x10
> e_type, e_machine, self.e_version =
> self.read_and_unpack(f"{endian}HHI")
> self.e_type, self.e_machine = Elf.Type(e_type),
> Elf.Abi(e_machine)
>
> # off 0x18
> if self.e_class == Elf.Class.ELF_64_BITS:
> self.e_entry, self.e_phoff, self.e_shoff =
> self.read_and_unpack(f"{endian}QQQ")
> else:
> self.e_entry, self.e_phoff, self.e_shoff =
> self.read_and_unpack(f"{endian}III")
>
> self.e_flags, self.e_ehsize, self.e_phentsize, self.e_phnum =
> self.read_and_unpack(f"{endian}IHHH")
> self.e_shentsize, self.e_shnum, self.e_shstrndx =
> self.read_and_unpack(f"{endian}HHH")
>
> self.phdrs = []
> for i in range(self.e_phnum):
> self.phdrs.append(Phdr(self, self.e_phoff +
> self.e_phentsize * i))
>
> self.shdrs = []
> for i in range(self.e_shnum):
> self.shdrs.append(Shdr(self, self.e_shoff +
> self.e_shentsize * i))
> return
>
> def read(self, size: int) -> bytes:
> return self.fd.read(size)
>
> def read_and_unpack(self, fmt: str) -> tuple[Any, ...]:
> size = struct.calcsize(fmt)
> data = self.fd.read(size)
> return struct.unpack(fmt, data)
>
> def seek(self, off: int) -> None:
> self.fd.seek(off, 0)
>
> def __str__(self) -> str:
> return f"ELF('{self.path.absolute()}', {self.e_class.name}, {
> self.e_machine.name})"
>
> def __repr__(self) -> str:
> return f"ELF('{self.path.absolute()}', {self.e_class.name}, {
> self.e_machine.name})"
>
> @property
> def entry_point(self) -> int:
> return self.e_entry
>
> @classmethod
> def is_valid(cls, path: pathlib.Path) -> bool:
> return u32(path.open("rb").read(4), e = Endianness.BIG_ENDIAN) ==
> Elf.ELF_MAGIC
>
> @property
> def checksec(self) -> dict[str, bool]:
> """Check the security property of the ELF binary. The following
> properties are:
> - Canary
> - NX
> - PIE
> - Fortify
> - Partial/Full RelRO.
> Return a dict() with the different keys mentioned above, and the
> boolean
> associated whether the protection was found."""
> if not self.__checksec:
> def __check_security_property(opt: str, filename: str,
> pattern: str) -> bool:
> cmd = [readelf,]
> cmd += opt.split()
> cmd += [filename,]
> lines = gef_execute_external(cmd, as_list=True)
> for line in lines:
> if re.search(pattern, line):
> return True
> return False
>
> abspath = str(self.path.absolute())
> readelf = gef.session.constants["readelf"]
> self.__checksec["Canary"] = __check_security_property("-rs",
> abspath, r"__stack_chk_fail") is True
> has_gnu_stack = __check_security_property("-W -l", abspath,
> r"GNU_STACK") is True
> if has_gnu_stack:
> self.__checksec["NX"] = __check_security_property("-W -l",
> abspath, r"GNU_STACK.*RWE") is False
> else:
> self.__checksec["NX"] = False
> self.__checksec["PIE"] = __check_security_property("-h",
> abspath, r":.*EXEC") is False
> self.__checksec["Fortify"] = __check_security_property("-s",
> abspath, r"_chk@GLIBC") is True
> self.__checksec["Partial RelRO"] =
> __check_security_property("-l", abspath, r"GNU_RELRO") is True
> self.__checksec["Full RelRO"] = self.__checksec["Partial
> RelRO"] and __check_security_property("-d", abspath, r"BIND_NOW") is True
> return self.__checksec
>
> @classproperty
> @deprecated("use `Elf.Abi.X86_64`")
> def X86_64(cls) -> int: return Elf.Abi.X86_64.value # pylint:
> disable=no-self-argument
>
> @classproperty
> @deprecated("use `Elf.Abi.X86_32`")
> def X86_32(cls) -> int : return Elf.Abi.X86_32.value # pylint:
> disable=no-self-argument
>
> @classproperty
> @deprecated("use `Elf.Abi.ARM`")
> def ARM(cls) -> int : return Elf.Abi.ARM.value # pylint:
> disable=no-self-argument
>
> @classproperty
> @deprecated("use `Elf.Abi.MIPS`")
> def MIPS(cls) -> int : return Elf.Abi.MIPS.value # pylint:
> disable=no-self-argument
>
> @classproperty
> @deprecated("use `Elf.Abi.POWERPC`")
> def POWERPC(cls) -> int : return Elf.Abi.POWERPC.value # pylint:
> disable=no-self-argument
>
> @classproperty
> @deprecated("use `Elf.Abi.POWERPC64`")
> def POWERPC64(cls) -> int : return Elf.Abi.POWERPC64.value # pylint:
> disable=no-self-argument
>
> @classproperty
> @deprecated("use `Elf.Abi.SPARC`")
> def SPARC(cls) -> int : return Elf.Abi.SPARC.value # pylint:
> disable=no-self-argument
>
> @classproperty
> @deprecated("use `Elf.Abi.SPARC64`")
> def SPARC64(cls) -> int : return Elf.Abi.SPARC64.value # pylint:
> disable=no-self-argument
>
> @classproperty
> @deprecated("use `Elf.Abi.AARCH64`")
> def AARCH64(cls) -> int : return Elf.Abi.AARCH64.value # pylint:
> disable=no-self-argument
>
> @classproperty
> @deprecated("use `Elf.Abi.RISCV`")
> def RISCV(cls) -> int : return Elf.Abi.RISCV.value # pylint:
> disable=no-self-argument
>
>
[-- Attachment #2: Type: text/html, Size: 19416 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling
2024-12-08 17:37 ` Eli Zaretskii
2024-12-09 14:58 ` rehan malak
@ 2024-12-09 15:04 ` kobarity
1 sibling, 0 replies; 11+ messages in thread
From: kobarity @ 2024-12-09 15:04 UTC (permalink / raw)
To: rehan malak, Eli Zaretskii; +Cc: 74738, Stefan Monnier
rehan malak wrote:
> Please tell me if there is a workaround without losing python syntax color ?
If you can configure Emacs to use tree-sitter, python-ts-mode would
not be affected by this problem.
Eli Zaretskii wrote:
>
> > From: rehan malak <rehan.malak@gmail.com>
> > Date: Sun, 8 Dec 2024 14:05:36 +0100
> >
> > I can reproduce systematically this freeze dealing with a 10000lines
> > Python file :
> >
> > wget https://raw.githubusercontent.com/hugsy/gef/refs/heads/main/gef.py
> > emacs -Q gef.py
It seems to be reproducible after the following commit:
commit 3b3274a85c2f5df21d76d82e0d7740005aa84fdf
Author: Stefan Monnier <monnier@iro.umontreal.ca>
Date: Fri Oct 16 14:03:59 2020 -0400
* lisp/progmodes/python.el: Teach f-strings to `font-lock`
(python--f-string-p, python--font-lock-f-strings): New functions.
(python-font-lock-keywords-maximum-decoration): Use them.
> A much smaller reproducer is attached below. Just visiting it freezes
> Emacs. But if I change the line marked below:
>
> @classmethod
> def is_valid(cls, _: pathlib.Path) -> bool:
> raise NotImplementedError
>
> def __str__(self) -> str:
> >>>>> return f"{self.name}('{self.path.absolute()}', entry @ {self.entry_point:#x})"
>
> to say this instead:
>
> return self.name
>
> the problem goes away. So it is something in that complex expression
> that trips syntax-ppss.
>
> Stefan and kobarity, any suggestions or ideas?
I could not reproduce the freeze, but it took several seconds to visit
the reproducer. It appears to be possible to work around this by
simply removing the format specifier ":#x".
return f"{self.name}('{self.path.absolute()}', entry @ {self.entry_point})"
So there may be a problem with font-lock when there is an f-string
format specifier, but I have not found the root cause yet.
^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling
2024-12-09 14:58 ` rehan malak
@ 2024-12-11 14:24 ` kobarity
2024-12-12 3:49 ` rehan malak
2024-12-13 23:35 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
0 siblings, 2 replies; 11+ messages in thread
From: kobarity @ 2024-12-11 14:24 UTC (permalink / raw)
To: rehan malak, Eli Zaretskii; +Cc: 74738, Stefan Monnier
[-- Attachment #1: Type: text/plain, Size: 1682 bytes --]
rehan malak wrote:
> Deleting all the "#" in the f-string of the gef.py file removes the problem on my side.
> sed -i -e 's/#0{align/0{align/g' gef.py
> sed -i -e 's/#0{width/0{width/g' gef.py
> sed -i -e 's/#07x}/07x}/g' gef.py
> sed -i -e 's/#06x}/06x}/g' gef.py
> sed -i -e 's/#04x}/04x}/g' gef.py
> sed -i -e 's/#4x}/4x}/g' gef.py
> sed -i -e 's/#8x}/8x}/g' gef.py
> sed -i -e 's/#10x}/10x}/g' gef.py
> sed -i -e 's/#x}/x}/g' gef.py
>
> Your smaller example Eli also contains a f-string with the "#".
>
> This python f-string format specifier is described here:
> https://docs.python.org/3/library/string.html#format-specification-mini-language
>
> Is this format specifier supported from the beginning ?
>
> For example :
>
> value = 0xab
> print(f"{value:x} is a value")
>
> => color syntax : is a value has the color defined by font-lock-string-face
>
> while
>
> value = 0xab
> print(f"{value:#x} is a value")
>
> => color are messed up... is has color defined by font-lock-keyword-face and a value has color defined by default face
The mechanism of the hang is still unclear, but I found the cause of
the font-lock malfunction: when using `up-list' to find the
corresponding closing brace for an opening brace in an f-string, if
the format specifier contains "#", the rest of the string is
considered a comment and the search fails. Therefore, it can be
worked around by temporarily binding `parse-sexp-ignore-comments' to
nil, as in the attached patch.
Hangs on gef.py can also be avoided with this patch, but it assumes
that the braces are properly closed. If you remove the closing braces
of expressions containing "#", it will eventually hang.
[-- Attachment #2: 0001-Fix-font-lock-of-Python-f-strings.patch --]
[-- Type: text/plain, Size: 1999 bytes --]
From 697f942d70e40bcd9bd32596313a9a371fffe35c Mon Sep 17 00:00:00 2001
From: kobarity <kobarity@gmail.com>
Date: Wed, 11 Dec 2024 23:21:04 +0900
Subject: [PATCH] Fix font-lock of Python f-strings
* lisp/progmodes/python.el (python--font-lock-f-strings): Bind
'parse-sexp-ignore-comments' to nil so that we can look for
closing braces even if a hash is used in the format specifier.
* test/lisp/progmodes/python-tests.el
(python-font-lock-f-string-1): New test. (Bug#74738)
---
lisp/progmodes/python.el | 4 +++-
test/lisp/progmodes/python-tests.el | 8 ++++++++
2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index cfa3cc59568..827f437013f 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -631,7 +631,9 @@ python--font-lock-f-strings
(forward-char 1) ;Just skip over {{
(let ((beg (match-beginning 0))
(end (condition-case nil
- (progn (up-list 1) (min send (point)))
+ (let ((parse-sexp-ignore-comments))
+ (up-list 1)
+ (min send (point)))
(scan-error send))))
(goto-char end)
(put-text-property beg end 'face nil))))
diff --git a/test/lisp/progmodes/python-tests.el b/test/lisp/progmodes/python-tests.el
index 4cc4040d0ff..ac17c5b1263 100644
--- a/test/lisp/progmodes/python-tests.el
+++ b/test/lisp/progmodes/python-tests.el
@@ -709,6 +709,14 @@ python-font-lock-string-literal-concatenation
(17 . font-lock-operator-face) (18)
(19 . font-lock-string-face))))
+(ert-deftest python-font-lock-f-string-1 ()
+ "Test for bug#74738."
+ (python-tests-assert-faces
+ "print(f\"{value:#x} is a value\")"
+ '((1 . font-lock-builtin-face) (6)
+ (8 . font-lock-string-face) (9)
+ (19 . font-lock-string-face) (31))))
+
\f
;;; Indentation
--
2.43.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling
2024-12-11 14:24 ` kobarity
@ 2024-12-12 3:49 ` rehan malak
2024-12-15 14:26 ` kobarity
2024-12-13 23:35 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
1 sibling, 1 reply; 11+ messages in thread
From: rehan malak @ 2024-12-12 3:49 UTC (permalink / raw)
To: kobarity; +Cc: Eli Zaretskii, Stefan Monnier, 74738
[-- Attachment #1: Type: text/plain, Size: 2064 bytes --]
thanks kobarity, with your patch, I confirm that :
- I can open gef.py and scroll
- I get the correct font-lock-string-face in the f-string after a "#" in
the format
On Wed, Dec 11, 2024 at 3:24 PM kobarity <kobarity@gmail.com> wrote:
> rehan malak wrote:
> > Deleting all the "#" in the f-string of the gef.py file removes the
> problem on my side.
> > sed -i -e 's/#0{align/0{align/g' gef.py
> > sed -i -e 's/#0{width/0{width/g' gef.py
> > sed -i -e 's/#07x}/07x}/g' gef.py
> > sed -i -e 's/#06x}/06x}/g' gef.py
> > sed -i -e 's/#04x}/04x}/g' gef.py
> > sed -i -e 's/#4x}/4x}/g' gef.py
> > sed -i -e 's/#8x}/8x}/g' gef.py
> > sed -i -e 's/#10x}/10x}/g' gef.py
> > sed -i -e 's/#x}/x}/g' gef.py
> >
> > Your smaller example Eli also contains a f-string with the "#".
> >
> > This python f-string format specifier is described here:
> >
> https://docs.python.org/3/library/string.html#format-specification-mini-language
> >
> > Is this format specifier supported from the beginning ?
> >
> > For example :
> >
> > value = 0xab
> > print(f"{value:x} is a value")
> >
> > => color syntax : is a value has the color defined by
> font-lock-string-face
> >
> > while
> >
> > value = 0xab
> > print(f"{value:#x} is a value")
> >
> > => color are messed up... is has color defined by font-lock-keyword-face
> and a value has color defined by default face
>
> The mechanism of the hang is still unclear, but I found the cause of
> the font-lock malfunction: when using `up-list' to find the
> corresponding closing brace for an opening brace in an f-string, if
> the format specifier contains "#", the rest of the string is
> considered a comment and the search fails. Therefore, it can be
> worked around by temporarily binding `parse-sexp-ignore-comments' to
> nil, as in the attached patch.
>
> Hangs on gef.py can also be avoided with this patch, but it assumes
> that the braces are properly closed. If you remove the closing braces
> of expressions containing "#", it will eventually hang.
>
[-- Attachment #2: Type: text/html, Size: 2855 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling
2024-12-11 14:24 ` kobarity
2024-12-12 3:49 ` rehan malak
@ 2024-12-13 23:35 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
1 sibling, 0 replies; 11+ messages in thread
From: Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-12-13 23:35 UTC (permalink / raw)
To: kobarity; +Cc: rehan malak, Eli Zaretskii, 74738
> The mechanism of the hang is still unclear, but I found the cause of
> the font-lock malfunction: when using `up-list' to find the
> corresponding closing brace for an opening brace in an f-string, if
> the format specifier contains "#", the rest of the string is
> considered a comment and the search fails. Therefore, it can be
> worked around by temporarily binding `parse-sexp-ignore-comments' to
> nil, as in the attached patch.
Maybe a cleaner solution would be to locally use a different
syntax-table (where # doesn't start a comment), but I'm not sure it's
worth the trouble. So I installed your patch into `master`, thank you.
Maybe we should also temporarily setup a narrowing so `up-list` can't
look past `send`.
BTW, beside removing the `font-lock-string-face` from the `face`
property, `python--font-lock-f-strings` should probably also add
a special face for the ":<FMT>" thingies found (optionally) at the end.
> Hangs on gef.py can also be avoided with this patch, but it assumes
> that the braces are properly closed. If you remove the closing braces
> of expressions containing "#", it will eventually hang.
Hmm...
Stefan
^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling
2024-12-12 3:49 ` rehan malak
@ 2024-12-15 14:26 ` kobarity
2024-12-15 15:03 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
0 siblings, 1 reply; 11+ messages in thread
From: kobarity @ 2024-12-15 14:26 UTC (permalink / raw)
To: Stefan Monnier, rehan malak; +Cc: Eli Zaretskii, 74738
[-- Attachment #1: Type: text/plain, Size: 1979 bytes --]
rehan malak wrote:
> thanks kobarity, with your patch, I confirm that :
> - I can open gef.py and scroll
> - I get the correct font-lock-string-face in the f-string after a "#" in the format
Thank you for your confirmation.
Stefan Monnier wrote:
> > The mechanism of the hang is still unclear, but I found the cause of
> > the font-lock malfunction: when using `up-list' to find the
> > corresponding closing brace for an opening brace in an f-string, if
> > the format specifier contains "#", the rest of the string is
> > considered a comment and the search fails. Therefore, it can be
> > worked around by temporarily binding `parse-sexp-ignore-comments' to
> > nil, as in the attached patch.
>
> Maybe a cleaner solution would be to locally use a different
> syntax-table (where # doesn't start a comment), but I'm not sure it's
> worth the trouble. So I installed your patch into `master`, thank you.
Thank you. I agree with you about the syntax-table.
> Maybe we should also temporarily setup a narrowing so `up-list` can't
> look past `send`.
>
> BTW, beside removing the `font-lock-string-face` from the `face`
> property, `python--font-lock-f-strings` should probably also add
> a special face for the ":<FMT>" thingies found (optionally) at the end.
>
> > Hangs on gef.py can also be avoided with this patch, but it assumes
> > that the braces are properly closed. If you remove the closing braces
> > of expressions containing "#", it will eventually hang.
>
> Hmm...
Narrowing might improve it, but what about binding
`forward-sexp-function' to nil as well, as in the attached patch? It
seems to resolve the hang caused by the unbalanced braces in
f-strings.
1. wget https://raw.githubusercontent.com/hugsy/gef/refs/heads/main/gef.py
2. sed -i -e 's/\(:#\d*x\)}/\1/g' gef.py
3. emacs -Q gef.py
4. Scroll down the file.
I don't think `python-nav-forward-sexp' is necessary here, since the
inside of the braces should be an expression, not statements.
[-- Attachment #2: 0001-Fix-hangs-caused-by-unbalanced-braces-in-Python-f-st.patch --]
[-- Type: text/plain, Size: 1148 bytes --]
From bb2c1a3e03416532fb97d6fa7ef276c5feebb356 Mon Sep 17 00:00:00 2001
From: kobarity <kobarity@gmail.com>
Date: Sun, 15 Dec 2024 23:02:51 +0900
Subject: [PATCH] Fix hangs caused by unbalanced braces in Python f-strings
* lisp/progmodes/python.el (python--font-lock-f-strings):
Temporarily bind forward-sexp-function to nil when calling
'up-list'. (Bug#74738)
---
lisp/progmodes/python.el | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 5729b12573f..493eab5e611 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -623,7 +623,8 @@ python--font-lock-f-strings
(forward-char 1) ;Just skip over {{
(let ((beg (match-beginning 0))
(end (condition-case nil
- (let ((parse-sexp-ignore-comments))
+ (let ((forward-sexp-function)
+ (parse-sexp-ignore-comments))
(up-list 1)
(min send (point)))
(scan-error send))))
--
2.43.0
^ permalink raw reply related [flat|nested] 11+ messages in thread
* bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling
2024-12-15 14:26 ` kobarity
@ 2024-12-15 15:03 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2024-12-21 9:50 ` Eli Zaretskii
0 siblings, 1 reply; 11+ messages in thread
From: Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-12-15 15:03 UTC (permalink / raw)
To: kobarity; +Cc: rehan malak, Eli Zaretskii, 74738
>> > Hangs on gef.py can also be avoided with this patch, but it assumes
>> > that the braces are properly closed. If you remove the closing braces
>> > of expressions containing "#", it will eventually hang.
>>
>> Hmm...
>
> Narrowing might improve it, but what about binding
> `forward-sexp-function' to nil as well, as in the attached patch?
Oh, of course, yes!
Thanks, pushed!
Stefan
^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling
2024-12-15 15:03 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
@ 2024-12-21 9:50 ` Eli Zaretskii
2024-12-21 14:21 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
0 siblings, 1 reply; 11+ messages in thread
From: Eli Zaretskii @ 2024-12-21 9:50 UTC (permalink / raw)
To: Stefan Monnier; +Cc: kobarity, rehan.malak, 74738
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rehan malak <rehan.malak@gmail.com>, Eli Zaretskii <eliz@gnu.org>,
> 74738@debbugs.gnu.org
> Date: Sun, 15 Dec 2024 10:03:33 -0500
>
> >> > Hangs on gef.py can also be avoided with this patch, but it assumes
> >> > that the braces are properly closed. If you remove the closing braces
> >> > of expressions containing "#", it will eventually hang.
> >>
> >> Hmm...
> >
> > Narrowing might improve it, but what about binding
> > `forward-sexp-function' to nil as well, as in the attached patch?
>
> Oh, of course, yes!
> Thanks, pushed!
Thanks, should we now close this bug?
^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling
2024-12-21 9:50 ` Eli Zaretskii
@ 2024-12-21 14:21 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
0 siblings, 0 replies; 11+ messages in thread
From: Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors @ 2024-12-21 14:21 UTC (permalink / raw)
To: Eli Zaretskii; +Cc: 74738-done, kobarity, rehan.malak
> Thanks, should we now close this bug?
I believe so, yes, closing,
Stefan
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2024-12-21 14:21 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-12-08 13:05 bug#74738: 31.0.50; Freezes in Python-mode on some Python file when searching or scrolling rehan malak
2024-12-08 17:37 ` Eli Zaretskii
2024-12-09 14:58 ` rehan malak
2024-12-11 14:24 ` kobarity
2024-12-12 3:49 ` rehan malak
2024-12-15 14:26 ` kobarity
2024-12-15 15:03 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2024-12-21 9:50 ` Eli Zaretskii
2024-12-21 14:21 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2024-12-13 23:35 ` Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors
2024-12-09 15:04 ` kobarity
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).