bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[David Dynerman]

[-- Attachment #1: Type: text/plain, Size: 23621 bytes --]

Emacs hangs when first opening an org file that contains a python source block. This only occurs the first time the file is visited - if I abort I no longer experience hangs when visiting any org files containing python source blocks. This bug has also been reported to emacs-orgmode (6 Dec 2016, Subject: Emacs hangs while loading org file with python blocks), but the suggestion was to report it upstream to bug-gnu-emacs.

Here is a minimal example of org code (in between <------> markers) to reproduce the hang:

<------->
#+BEGIN_SRC python
  """xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"""
#+END_SRC
#+BEGIN_SRC python
  class x:  
      def x(self):
          """
	  """
          pass
#+END_SRC
<------->

To reproduce:

1) Save the above as python-hang.org
2) Start a new emacs process
3) C-x C-f python-hang.org RET
4) Observe emacs hangs. I have waited up to 30 mins to see if the file loads, without success. Typing C-g unhangs emacs and displays the file.

Other information:

The bug seems related to fontification - during the hang emacs seems stuck in python fontification code. After aborting the load with C-g, occasionally some python blocks will not be correctly fontified. I have attached a profiler report taken during the hang, showing where emacs is stuck. The relevant part of the call stack:

I've attached a profiler report captured during the hang. Here is the relevant part:

	   - org-src-font-lock-fontify-block                    17419  46%
	    - org-font-lock-ensure                              17240  45%
	     - #<compiled 0x41086c73>                           17240  45%
	      - font-lock-default-fontify-buffer                17240  45%
	       - font-lock-fontify-region                       17240  45%
		- font-lock-default-fontify-region              17240  45%
		 - font-lock-fontify-syntactically-region              17240  45%
		  - python-font-lock-syntactic-face-function              17240  45%
		   - python-info-docstring-p                    17240  45%
		    - python-nav-backward-sexp                  17240  45%
		     - python-nav-forward-sexp                  17240  45%
		      - python-nav--forward-sexp                17240  45%
		       - python-info-end-of-block-p              17240  45%
			- python-info-end-of-statement-p              17240  45%
			 - python-nav-end-of-statement              15444  41%
			    syntax-ppss                          8641  22%

I _AM_ able to reproduce this hang in emacs -Q, executing only the following in *scratch*, and then running the reproduction steps above.

(add-to-list 'load-path "/usr/local/share/emacs/site-lisp/org") ;; org from git installed here
(require 'org)

I am using the latest org code from git (release_9.0-132-gd65aa3).

The bug does NOT occur with org 8.2.10.

In addition to experiencing this hang in emacs 26.0.50.1 (where this bug report is generated) I also ran into the bug on Emacs 25.1.1 (x86_64-apple-darwin15.6.0, Carbon Version 157 AppKit 1404.47) of 2016-11-24.

In GNU Emacs 26.0.50.1 (x86_64-apple-darwin15.6.0, NS appkit-1404.47 Version 10.11.6 (Build 15G1108))
 of 2016-12-07 built on CIVIC-TV.local
Repository revision: f41097adf3e7155f9c0db6d90de0aa1623d89232
Windowing system distributor 'Apple', version 10.3.1404
Recent messages:
Created acrlong link.
Created acrfull link.
Created ac link.
Created Ac link.
Created acp link.
Created Acp link.
Created latex link.
Loading /Users/dynerman/personal/dev/emacs/emacs.el (source)...done
Loaded ~/personal/dev/emacs/emacs.el
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --disable-dependency-tracking --disable-silent-rules
 --enable-locallisppath=/usr/local/share/emacs/site-lisp
 --infodir=/usr/local/Cellar/emacs/HEAD-f41097a/share/info/emacs
 --prefix=/usr/local/Cellar/emacs/HEAD-f41097a --without-x --with-xml2
 --without-dbus --with-gnutls --with-imagemagick --with-rsvg --with-ns
 --disable-ns-self-contained'

Configured features:
JPEG RSVG IMAGEMAGICK NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS
NS

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  jabber-activity-mode: t
  diff-auto-refine-mode: t
  helm-mode: t
  async-bytecomp-package-mode: t
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/usr/local/share/emacs/site-lisp/org/ox hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ox
/usr/local/share/emacs/site-lisp/org/ox-texinfo hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ox-texinfo
/usr/local/share/emacs/site-lisp/org/ox-publish hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ox-publish
/usr/local/share/emacs/site-lisp/org/ox-org hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ox-org
/usr/local/share/emacs/site-lisp/org/ox-odt hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ox-odt
/usr/local/share/emacs/site-lisp/org/ox-md hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ox-md
/usr/local/share/emacs/site-lisp/org/ox-man hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ox-man
/usr/local/share/emacs/site-lisp/org/ox-latex hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ox-latex
/usr/local/share/emacs/site-lisp/org/ox-icalendar hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ox-icalendar
/usr/local/share/emacs/site-lisp/org/ox-html hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ox-html
/usr/local/share/emacs/site-lisp/org/ox-beamer hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ox-beamer
/usr/local/share/emacs/site-lisp/org/ox-ascii hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ox-ascii
/usr/local/share/emacs/site-lisp/org/org hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org
/usr/local/share/emacs/site-lisp/org/org-w3m hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-w3m
/usr/local/share/emacs/site-lisp/org/org-version hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-version
/usr/local/share/emacs/site-lisp/org/org-timer hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-timer
/usr/local/share/emacs/site-lisp/org/org-table hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-table
/usr/local/share/emacs/site-lisp/org/org-src hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-src
/usr/local/share/emacs/site-lisp/org/org-rmail hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-rmail
/usr/local/share/emacs/site-lisp/org/org-protocol hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-protocol
/usr/local/share/emacs/site-lisp/org/org-plot hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-plot
/usr/local/share/emacs/site-lisp/org/org-pcomplete hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-pcomplete
/usr/local/share/emacs/site-lisp/org/org-mouse hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-mouse
/usr/local/share/emacs/site-lisp/org/org-mobile hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-mobile
/usr/local/share/emacs/site-lisp/org/org-mhe hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-mhe
/usr/local/share/emacs/site-lisp/org/org-macs hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-macs
/usr/local/share/emacs/site-lisp/org/org-macro hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-macro
/usr/local/share/emacs/site-lisp/org/org-loaddefs hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-loaddefs
/usr/local/share/emacs/site-lisp/org/org-list hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-list
/usr/local/share/emacs/site-lisp/org/org-irc hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-irc
/usr/local/share/emacs/site-lisp/org/org-install hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-install
/usr/local/share/emacs/site-lisp/org/org-inlinetask hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-inlinetask
/usr/local/share/emacs/site-lisp/org/org-info hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-info
/usr/local/share/emacs/site-lisp/org/org-indent hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-indent
/usr/local/share/emacs/site-lisp/org/org-id hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-id
/usr/local/share/emacs/site-lisp/org/org-habit hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-habit
/usr/local/share/emacs/site-lisp/org/org-gnus hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-gnus
/usr/local/share/emacs/site-lisp/org/org-footnote hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-footnote
/usr/local/share/emacs/site-lisp/org/org-feed hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-feed
/usr/local/share/emacs/site-lisp/org/org-faces hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-faces
/usr/local/share/emacs/site-lisp/org/org-eshell hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-eshell
/usr/local/share/emacs/site-lisp/org/org-entities hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-entities
/usr/local/share/emacs/site-lisp/org/org-element hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-element
/usr/local/share/emacs/site-lisp/org/org-docview hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-docview
/usr/local/share/emacs/site-lisp/org/org-datetree hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-datetree
/usr/local/share/emacs/site-lisp/org/org-ctags hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-ctags
/usr/local/share/emacs/site-lisp/org/org-crypt hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-crypt
/usr/local/share/emacs/site-lisp/org/org-compat hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-compat
/usr/local/share/emacs/site-lisp/org/org-colview hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-colview
/usr/local/share/emacs/site-lisp/org/org-clock hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-clock
/usr/local/share/emacs/site-lisp/org/org-capture hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-capture
/usr/local/share/emacs/site-lisp/org/org-bibtex hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-bibtex
/usr/local/share/emacs/site-lisp/org/org-bbdb hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-bbdb
/usr/local/share/emacs/site-lisp/org/org-attach hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-attach
/usr/local/share/emacs/site-lisp/org/org-archive hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-archive
/usr/local/share/emacs/site-lisp/org/org-agenda hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/org-agenda
/usr/local/share/emacs/site-lisp/org/ob hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob
/usr/local/share/emacs/site-lisp/org/ob-tangle hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-tangle
/usr/local/share/emacs/site-lisp/org/ob-table hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-table
/usr/local/share/emacs/site-lisp/org/ob-sqlite hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-sqlite
/usr/local/share/emacs/site-lisp/org/ob-sql hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-sql
/usr/local/share/emacs/site-lisp/org/ob-shen hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-shen
/usr/local/share/emacs/site-lisp/org/ob-screen hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-screen
/usr/local/share/emacs/site-lisp/org/ob-scheme hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-scheme
/usr/local/share/emacs/site-lisp/org/ob-scala hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-scala
/usr/local/share/emacs/site-lisp/org/ob-sass hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-sass
/usr/local/share/emacs/site-lisp/org/ob-ruby hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-ruby
/usr/local/share/emacs/site-lisp/org/ob-ref hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-ref
/usr/local/share/emacs/site-lisp/org/ob-R hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-R
/usr/local/share/emacs/site-lisp/org/ob-python hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-python
/usr/local/share/emacs/site-lisp/org/ob-plantuml hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-plantuml
/usr/local/share/emacs/site-lisp/org/ob-picolisp hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-picolisp
/usr/local/share/emacs/site-lisp/org/ob-perl hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-perl
/usr/local/share/emacs/site-lisp/org/ob-org hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-org
/usr/local/share/emacs/site-lisp/org/ob-octave hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-octave
/usr/local/share/emacs/site-lisp/org/ob-ocaml hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-ocaml
/usr/local/share/emacs/site-lisp/org/ob-mscgen hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-mscgen
/usr/local/share/emacs/site-lisp/org/ob-maxima hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-maxima
/usr/local/share/emacs/site-lisp/org/ob-matlab hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-matlab
/usr/local/share/emacs/site-lisp/org/ob-makefile hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-makefile
/usr/local/share/emacs/site-lisp/org/ob-lob hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-lob
/usr/local/share/emacs/site-lisp/org/ob-lisp hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-lisp
/usr/local/share/emacs/site-lisp/org/ob-lilypond hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-lilypond
/usr/local/share/emacs/site-lisp/org/ob-ledger hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-ledger
/usr/local/share/emacs/site-lisp/org/ob-latex hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-latex
/usr/local/share/emacs/site-lisp/org/ob-keys hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-keys
/usr/local/share/emacs/site-lisp/org/ob-js hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-js
/usr/local/share/emacs/site-lisp/org/ob-java hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-java
/usr/local/share/emacs/site-lisp/org/ob-io hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-io
/usr/local/share/emacs/site-lisp/org/ob-haskell hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-haskell
/usr/local/share/emacs/site-lisp/org/ob-gnuplot hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-gnuplot
/usr/local/share/emacs/site-lisp/org/ob-fortran hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-fortran
/usr/local/share/emacs/site-lisp/org/ob-exp hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-exp
/usr/local/share/emacs/site-lisp/org/ob-eval hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-eval
/usr/local/share/emacs/site-lisp/org/ob-emacs-lisp hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-emacs-lisp
/usr/local/share/emacs/site-lisp/org/ob-dot hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-dot
/usr/local/share/emacs/site-lisp/org/ob-ditaa hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-ditaa
/usr/local/share/emacs/site-lisp/org/ob-css hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-css
/usr/local/share/emacs/site-lisp/org/ob-core hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-core
/usr/local/share/emacs/site-lisp/org/ob-comint hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-comint
/usr/local/share/emacs/site-lisp/org/ob-clojure hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-clojure
/usr/local/share/emacs/site-lisp/org/ob-calc hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-calc
/usr/local/share/emacs/site-lisp/org/ob-C hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-C
/usr/local/share/emacs/site-lisp/org/ob-awk hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-awk
/usr/local/share/emacs/site-lisp/org/ob-asymptote hides /usr/local/Cellar/emacs/HEAD-f41097a/share/emacs/26.0.50/lisp/org/ob-asymptote

Features:
(shadow sort gnus-alias company-oddmuse company-keywords company-etags
etags xref project company-gtags company-dabbrev-code company-dabbrev
company-files company-capf company-cmake company-xcode company-clang
company-semantic company-eclim company-css company-nxml company-bbdb
company-irony company-template irony-completion irony-snippet irony
company-jedi jedi-core python-environment epc ctable concurrent deferred
mail-extr emacsbug sendmail winner helm-command helm-elisp helm-eval
edebug solarized-light-theme solarized color multi-term term disp-table
ehelp org-notmuch ob-ditaa ob-shell ob-C cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs ob-ledger
ob-ipython python tramp-sh dash-functional ob-python org-ref
org-ref-helm-bibtex org-ref-helm helm-bibtex bibtex-completion biblio
biblio-download biblio-dissemin biblio-hal biblio-dblp biblio-crossref
biblio-arxiv timezone biblio-doi biblio-core let-alist url-queue ido
org-ref-core reftex-cite reftex reftex-loaddefs reftex-vars parsebib
org-ref-glossary org-ref-utils org-ref-pdf org-ref-bibtex
org-ref-citeproc hydra lv doi-utils url-http url-auth url-gw nsm
org-bibtex json map bibtex f s dash ox-reveal ox-odt rng-loc rng-uri
rng-parse rng-match rng-dt rng-util rng-pttrn nxml-parse nxml-ns
nxml-enc xmltok nxml-util ox-latex ox-icalendar ox-html table ox-ascii
ox-publish ox jabber jabber-notifications notifications jabber-libnotify
dbus jabber-awesome jabber-osd jabber-wmii jabber-xmessage
jabber-festival jabber-sawfish jabber-ratpoison jabber-tmux
jabber-screen jabber-socks5 jabber-ft-server jabber-si-server
jabber-ft-client jabber-ft-common jabber-si-client jabber-si-common
jabber-feature-neg jabber-truncate jabber-time jabber-autoaway
jabber-vcard-avatars jabber-chatstates jabber-events jabber-vcard
jabber-avatar jabber-activity jabber-watch jabber-modeline
jabber-ahc-presence jabber-ahc jabber-version jabber-ourversion
jabber-muc-nick-completion hippie-exp jabber-browse jabber-search
jabber-register jabber-roster jabber-presence jabber-muc
jabber-bookmarks jabber-private jabber-muc-nick-coloring hexrgb
jabber-widget jabber-disco jabber-chat jabber-history jabber-chatbuffer
jabber-alert jabber-iq jabber-core jabber-console sgml-mode dom ewoc
jabber-keymap jabber-sasl sasl sasl-anonymous sasl-login sasl-plain fsm
jabber-logon jabber-conn srv dns starttls tls gnutls jabber-xml
jabber-menu jabber-util company offlineimap mailq notmuch hl-line
notmuch-message notmuch-hello wid-edit notmuch-tree notmuch-show
notmuch-print notmuch-crypto notmuch-mua notmuch-draft
notmuch-maildir-fcc notmuch-address notmuch-company notmuch-parser
notmuch-wash diff-mode coolj notmuch-query goto-addr icalendar diary-lib
diary-loaddefs notmuch-tag crm notmuch-lib notmuch-version
notmuch-compat message subr-x puny rfc822 mml mailabbrev gmm-utils
mailheader mm-view mml-smime mml-sec epa derived epg gnus-util rmail
rmail-loaddefs mail-utils smime dig mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
helm-mode helm-files rx image-dired dired-x dired-aux ffap thingatpt
helm-buffers helm-elscreen helm-tags helm-bookmark helm-adaptive
helm-info bookmark pp helm-locate helm-grep helm-regexp helm-external
helm-net browse-url xml url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf url-util mailcap helm-utils compile
helm-help helm-types helm helm-source eieio-compat helm-multi-match
helm-lib dired dired-loaddefs helm-config helm-easymenu async-bytecomp
async bash-completion edmacro kmacro theme-changer solar cal-dst cl
exec-path-from-shell org-element avl-tree org org-macro org-footnote
org-pcomplete org-list org-faces org-entities noutline outline
easy-mmode org-version ob-emacs-lisp ob ob-tangle org-src ob-ref ob-lob
ob-table ob-keys ob-exp ob-comint tramp tramp-compat tramp-loaddefs
trampver shell pcomplete parse-time advice comint ansi-color ring
ob-core ob-eval org-compat org-macs org-loaddefs format-spec find-func
cal-menu calendar cal-loaddefs finder-inf tex-site info package
epg-config url-handlers url-parse auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache url-vars seq byte-opt gv bytecomp
byte-compile cl-extra help-mode easymenu cconv cl-loaddefs pcase cl-lib
time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/ns-win ns-win ucs-normalize term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote kqueue cocoa ns
multi-tty make-network-process emacs)

Memory information:
((conses 16 611922 33990)
 (symbols 48 54702 3)
 (miscs 40 135 341)
 (strings 32 139953 16628)
 (string-bytes 1 4454366)
 (vectors 16 76650)
 (vector-slots 8 1130597 32247)
 (floats 8 1183 353)
 (intervals 56 623 0)
 (buffers 976 14))


[-- Attachment #2: fontify-bug-profile --]
[-- Type: application/octet-stream, Size: 2428 bytes --]

+ ...                                                           19562  51%
- redisplay_internal (C function)                               17420  46%
 - jit-lock-function                                            17419  46%
  - jit-lock-fontify-now                                        17419  46%
   - jit-lock--run-functions                                    17419  46%
    - run-hook-wrapped                                          17419  46%
     - #<compiled 0x43af906d>                                   17419  46%
      - font-lock-fontify-region                                17419  46%
       - font-lock-default-fontify-region                       17419  46%
        - font-lock-fontify-keywords-region                     17419  46%
         - org-fontify-meta-lines-and-blocks                    17419  46%
          - org-fontify-meta-lines-and-blocks-1                 17419  46%
           - org-src-font-lock-fontify-block                    17419  46%
            - org-font-lock-ensure                              17240  45%
             - #<compiled 0x41086c73>                           17240  45%
              - font-lock-default-fontify-buffer                17240  45%
               - font-lock-fontify-region                       17240  45%
                - font-lock-default-fontify-region              17240  45%
                 - font-lock-fontify-syntactically-region              17240  45%
                  - python-font-lock-syntactic-face-function              17240  45%
                   - python-info-docstring-p                    17240  45%
                    - python-nav-backward-sexp                  17240  45%
                     - python-nav-forward-sexp                  17240  45%
                      - python-nav--forward-sexp                17240  45%
                       - python-info-end-of-block-p              17240  45%
                        - python-info-end-of-statement-p              17240  45%
                         - python-nav-end-of-statement              15444  41%
                            syntax-ppss                          8641  22%
            + python-mode                                         179   0%
   keymap-canonicalize                                              1   0%
+ command-execute                                                 658   1%
+ timer-event-handler                                               9   0%

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[Glenn Morris]

David Dynerman wrote:

> The bug does NOT occur with org 8.2.10.

Since that's the version included with Emacs, I'm confused as to why
you've been encouraged to report this to bug-gnu-emacs.

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[Glenn Morris]

David Dynerman wrote:

> The bug does NOT occur with org 8.2.10.

Since that's the version included with Emacs, I'm confused as to why
you've been encouraged to report this to bug-gnu-emacs.

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[Clément Pit--Claudel]

[-- Attachment #1.1.1: Type: text/plain, Size: 720 bytes --]

On 2016-12-07 21:08, Glenn Morris wrote:
> David Dynerman wrote:
>> The bug does NOT occur with org 8.2.10.
> 
> Since that's the version included with Emacs, I'm confused as to why
> you've been encouraged to report this to bug-gnu-emacs.

I can reproduce the issue in emacs -Q on master; hence my suggestion to report it here.  Glenn, can you try running the following after downloading the attached file?

   emacs -Q --eval '(setq debug-on-signal t org-src-fontify-natively t)' test.org 

It hangs reproducibly for me.  No idea why the OP can't reproduce it (David, are you sure it doesn't occur with org 8.2.10? Could it be that org-src-fontify-natively isn't enabled by default in 8.2.10?)

Clément.

[-- Attachment #1.1.2: test.org --]
[-- Type: application/vnd.lotus-organizer, Size: 171 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[Clément Pit--Claudel]

[-- Attachment #1.1.1: Type: text/plain, Size: 720 bytes --]

On 2016-12-07 21:08, Glenn Morris wrote:
> David Dynerman wrote:
>> The bug does NOT occur with org 8.2.10.
> 
> Since that's the version included with Emacs, I'm confused as to why
> you've been encouraged to report this to bug-gnu-emacs.

I can reproduce the issue in emacs -Q on master; hence my suggestion to report it here.  Glenn, can you try running the following after downloading the attached file?

   emacs -Q --eval '(setq debug-on-signal t org-src-fontify-natively t)' test.org 

It hangs reproducibly for me.  No idea why the OP can't reproduce it (David, are you sure it doesn't occur with org 8.2.10? Could it be that org-src-fontify-natively isn't enabled by default in 8.2.10?)

Clément.

[-- Attachment #1.1.2: test.org --]
[-- Type: application/vnd.lotus-organizer, Size: 171 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[David Dynerman]

Dear Glenn + bug-gnu-emacs,

Did you try the steps to reproduce? Indeed Clément was right! The bug also reliably reproduces also with org 8.2.10, if you additionally set org-src-fontify-natively to t.

Please let me know if you need any more information for the bug report,
David

Clément Pit--Claudel <clement.pit@gmail.com> writes:

> On 2016-12-07 21:08, Glenn Morris wrote:
>> David Dynerman wrote:
>>> The bug does NOT occur with org 8.2.10.
>> 
>> Since that's the version included with Emacs, I'm confused as to why
>> you've been encouraged to report this to bug-gnu-emacs.
>
> I can reproduce the issue in emacs -Q on master; hence my suggestion to report it here.  Glenn, can you try running the following after downloading the attached file?
>
>    emacs -Q --eval '(setq debug-on-signal t org-src-fontify-natively t)' test.org 
>
> It hangs reproducibly for me.  No idea why the OP can't reproduce it (David, are you sure it doesn't occur with org 8.2.10? Could it be that org-src-fontify-natively isn't enabled by default in 8.2.10?)
>
> Clément.

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[David Dynerman]

Dear Glenn + bug-gnu-emacs,

Did you try the steps to reproduce? Indeed Clément was right! The bug also reliably reproduces also with org 8.2.10, if you additionally set org-src-fontify-natively to t.

Please let me know if you need any more information for the bug report,
David

Clément Pit--Claudel <clement.pit@gmail.com> writes:

> On 2016-12-07 21:08, Glenn Morris wrote:
>> David Dynerman wrote:
>>> The bug does NOT occur with org 8.2.10.
>> 
>> Since that's the version included with Emacs, I'm confused as to why
>> you've been encouraged to report this to bug-gnu-emacs.
>
> I can reproduce the issue in emacs -Q on master; hence my suggestion to report it here.  Glenn, can you try running the following after downloading the attached file?
>
>    emacs -Q --eval '(setq debug-on-signal t org-src-fontify-natively t)' test.org 
>
> It hangs reproducibly for me.  No idea why the OP can't reproduce it (David, are you sure it doesn't occur with org 8.2.10? Could it be that org-src-fontify-natively isn't enabled by default in 8.2.10?)
>
> Clément.

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[npostavs]

tags 25132 confirmed
quit

The problem is that org updates its temporary fontification buffer from
its fontify rules which are called by jit-lock-function, which means
that inhibit-modification-hooks is bound to t.  Therefore, when
org-src-font-lock-fontify-block calls delete-region to remove leftover text from
the previous source block fontification, the `before-change-functions'
are not run.  In this case `syntax-ppss-flush-cache' is the important
function that doesn't get run, so `syntax-propertize--done' is still set
from before and messes up python.el's fontification routines.

  org-src-font-lock-fontify-block(#("python" 0 6 (fontified t)) 19 65)
  org-fontify-meta-lines-and-blocks-1(172)
  org-fontify-meta-lines-and-blocks(172)
  font-lock-fontify-keywords-region(1 172 nil)
  font-lock-default-fontify-region(1 172 nil)
  font-lock-fontify-region(1 172)
  ...
  jit-lock--run-functions(1 172)
  jit-lock-fontify-now(1 501)
  jit-lock-function(1)
  redisplay_internal\ \(C\ function\)()
  redisplay()
  sit-for(2)
  execute-extended-command(nil "25132-test" "25")
  funcall-interactively(execute-extended-command nil "25132-test" "25")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)

(defun org-src-font-lock-fontify-block (lang start end)
  ...
	  (with-current-buffer
	      (get-buffer-create
	       (concat " org-src-fontification:" (symbol-name lang-mode)))
            (delete-region (point-min) (point-max)) ;<-------------- `syntax-propertize--done' not reset here!
            (insert string " ") ;; so there's a final property change
	    (unless (eq major-mode lang-mode) (funcall lang-mode))
            (org-font-lock-ensure)
            ...)
  ...)

(defun jit-lock-function (start)
  ...
  (jit-lock-fontify-now start (+ start jit-lock-chunk-size))
  ...)

(defun jit-lock-fontify-now (&optional start end)
  "Fontify current buffer from START to END.
Defaults to the whole buffer.  END can be out of bounds."
  (with-buffer-prepared-for-jit-lock
    ...))

  (defmacro with-buffer-prepared-for-jit-lock (&rest body)
    "Execute BODY in current buffer, overriding several variables.
Preserves the `buffer-modified-p' state of the current buffer."
    (declare (debug t))
    `(let ((inhibit-point-motion-hooks t))
       (with-silent-modifications ; <------ binds inhibit-modification-hooks to t
         ,@body)))

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[npostavs]

tags 25132 confirmed
quit

The problem is that org updates its temporary fontification buffer from
its fontify rules which are called by jit-lock-function, which means
that inhibit-modification-hooks is bound to t.  Therefore, when
org-src-font-lock-fontify-block calls delete-region to remove leftover text from
the previous source block fontification, the `before-change-functions'
are not run.  In this case `syntax-ppss-flush-cache' is the important
function that doesn't get run, so `syntax-propertize--done' is still set
from before and messes up python.el's fontification routines.

  org-src-font-lock-fontify-block(#("python" 0 6 (fontified t)) 19 65)
  org-fontify-meta-lines-and-blocks-1(172)
  org-fontify-meta-lines-and-blocks(172)
  font-lock-fontify-keywords-region(1 172 nil)
  font-lock-default-fontify-region(1 172 nil)
  font-lock-fontify-region(1 172)
  ...
  jit-lock--run-functions(1 172)
  jit-lock-fontify-now(1 501)
  jit-lock-function(1)
  redisplay_internal\ \(C\ function\)()
  redisplay()
  sit-for(2)
  execute-extended-command(nil "25132-test" "25")
  funcall-interactively(execute-extended-command nil "25132-test" "25")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)

(defun org-src-font-lock-fontify-block (lang start end)
  ...
	  (with-current-buffer
	      (get-buffer-create
	       (concat " org-src-fontification:" (symbol-name lang-mode)))
            (delete-region (point-min) (point-max)) ;<-------------- `syntax-propertize--done' not reset here!
            (insert string " ") ;; so there's a final property change
	    (unless (eq major-mode lang-mode) (funcall lang-mode))
            (org-font-lock-ensure)
            ...)
  ...)

(defun jit-lock-function (start)
  ...
  (jit-lock-fontify-now start (+ start jit-lock-chunk-size))
  ...)

(defun jit-lock-fontify-now (&optional start end)
  "Fontify current buffer from START to END.
Defaults to the whole buffer.  END can be out of bounds."
  (with-buffer-prepared-for-jit-lock
    ...))

  (defmacro with-buffer-prepared-for-jit-lock (&rest body)
    "Execute BODY in current buffer, overriding several variables.
Preserves the `buffer-modified-p' state of the current buffer."
    (declare (debug t))
    `(let ((inhibit-point-motion-hooks t))
       (with-silent-modifications ; <------ binds inhibit-modification-hooks to t
         ,@body)))

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[npostavs]

[-- Attachment #1: Type: text/plain, Size: 735 bytes --]

tags 25132 patch
quit

npostavs@users.sourceforge.net writes:
> The problem is that org updates its temporary fontification buffer from
> its fontify rules which are called by jit-lock-function, which means
> that inhibit-modification-hooks is bound to t.  Therefore, when
> org-src-font-lock-fontify-block calls delete-region to remove leftover text from
> the previous source block fontification, the `before-change-functions'
> are not run.  In this case `syntax-ppss-flush-cache' is the important
> function that doesn't get run, so `syntax-propertize--done' is still set
> from before and messes up python.el's fontification routines.

I think with-silent-modifications should let-bind
inhibit-modification-hooks buffer locally:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch for master --]
[-- Type: text/x-diff, Size: 1171 bytes --]

From da4f1c0338b2b98f97a553568c4b80872484ee97 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 7 Jan 2017 15:47:37 -0500
Subject: [PATCH v1] Inhibit modification hooks buffer locally

`with-silent-modifications' let-binds `inhibit-modification-hooks' to t
globally.  So modifications to other buffers don't trigger modication
hooks.  This causes unexpected results when functions called from
`jit-lock-function' use temporary buffers and modifies them (Bug#25132).

* lisp/subr.el (with-silent-modifications): Bind
inhibit-modification-hooks buffer locally.
---
 lisp/subr.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/subr.el b/lisp/subr.el
index 5377416..fe20d68 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -3298,7 +3298,8 @@ with-silent-modifications
     `(let* ((,modified (buffer-modified-p))
             (buffer-undo-list t)
             (inhibit-read-only t)
-            (inhibit-modification-hooks t))
+            (inhibit-modification-hooks
+             (progn (make-local-variable 'inhibit-modification-hooks) t)))
        (unwind-protect
            (progn
              ,@body)
-- 
2.9.3


[-- Attachment #3: Type: text/plain, Size: 219 bytes --]


Perhaps the other variables it binds should be buffer local as well?

This bug is new in Emacs 25.1, but changing with-silent-modifications is
a bit risky.  Therefore, I propose the following for the emacs-25
branch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: patch for emacs-25 --]
[-- Type: text/x-diff, Size: 1262 bytes --]

From 338aa0c37eba0401616e8e02f0143a57edffd486 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 7 Jan 2017 16:05:19 -0500
Subject: [PATCH v1] Call modification hooks in org-src fontify buffers

* lisp/org/org-src.el (org-src-font-lock-fontify-block): Let-bind
`inhibit-modification-hooks' to nil, since this function can be called
from jit-lock-function which binds that variable to t (Bug#25132).
---
 lisp/org/org-src.el | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lisp/org/org-src.el b/lisp/org/org-src.el
index d01f108..9b66907 100644
--- a/lisp/org/org-src.el
+++ b/lisp/org/org-src.el
@@ -913,8 +913,9 @@ org-src-font-lock-fontify-block
 	  (with-current-buffer
 	      (get-buffer-create
 	       (concat " org-src-fontification:" (symbol-name lang-mode)))
-	    (delete-region (point-min) (point-max))
-	    (insert string " ") ;; so there's a final property change
+	    (let ((inhibit-modification-hooks nil)) ; Avoid Bug#25132.
+              (delete-region (point-min) (point-max))
+              (insert string " ")) ;; so there's a final property change
 	    (unless (eq major-mode lang-mode) (funcall lang-mode))
             (org-font-lock-ensure)
 	    (setq pos (point-min))
-- 
2.9.3

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[npostavs]

[-- Attachment #1: Type: text/plain, Size: 735 bytes --]

tags 25132 patch
quit

npostavs@users.sourceforge.net writes:
> The problem is that org updates its temporary fontification buffer from
> its fontify rules which are called by jit-lock-function, which means
> that inhibit-modification-hooks is bound to t.  Therefore, when
> org-src-font-lock-fontify-block calls delete-region to remove leftover text from
> the previous source block fontification, the `before-change-functions'
> are not run.  In this case `syntax-ppss-flush-cache' is the important
> function that doesn't get run, so `syntax-propertize--done' is still set
> from before and messes up python.el's fontification routines.

I think with-silent-modifications should let-bind
inhibit-modification-hooks buffer locally:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch for master --]
[-- Type: text/x-diff, Size: 1171 bytes --]

From da4f1c0338b2b98f97a553568c4b80872484ee97 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 7 Jan 2017 15:47:37 -0500
Subject: [PATCH v1] Inhibit modification hooks buffer locally

`with-silent-modifications' let-binds `inhibit-modification-hooks' to t
globally.  So modifications to other buffers don't trigger modication
hooks.  This causes unexpected results when functions called from
`jit-lock-function' use temporary buffers and modifies them (Bug#25132).

* lisp/subr.el (with-silent-modifications): Bind
inhibit-modification-hooks buffer locally.
---
 lisp/subr.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/subr.el b/lisp/subr.el
index 5377416..fe20d68 100644
--- a/lisp/subr.el
+++ b/lisp/subr.el
@@ -3298,7 +3298,8 @@ with-silent-modifications
     `(let* ((,modified (buffer-modified-p))
             (buffer-undo-list t)
             (inhibit-read-only t)
-            (inhibit-modification-hooks t))
+            (inhibit-modification-hooks
+             (progn (make-local-variable 'inhibit-modification-hooks) t)))
        (unwind-protect
            (progn
              ,@body)
-- 
2.9.3


[-- Attachment #3: Type: text/plain, Size: 219 bytes --]


Perhaps the other variables it binds should be buffer local as well?

This bug is new in Emacs 25.1, but changing with-silent-modifications is
a bit risky.  Therefore, I propose the following for the emacs-25
branch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: patch for emacs-25 --]
[-- Type: text/x-diff, Size: 1262 bytes --]

From 338aa0c37eba0401616e8e02f0143a57edffd486 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 7 Jan 2017 16:05:19 -0500
Subject: [PATCH v1] Call modification hooks in org-src fontify buffers

* lisp/org/org-src.el (org-src-font-lock-fontify-block): Let-bind
`inhibit-modification-hooks' to nil, since this function can be called
from jit-lock-function which binds that variable to t (Bug#25132).
---
 lisp/org/org-src.el | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lisp/org/org-src.el b/lisp/org/org-src.el
index d01f108..9b66907 100644
--- a/lisp/org/org-src.el
+++ b/lisp/org/org-src.el
@@ -913,8 +913,9 @@ org-src-font-lock-fontify-block
 	  (with-current-buffer
 	      (get-buffer-create
 	       (concat " org-src-fontification:" (symbol-name lang-mode)))
-	    (delete-region (point-min) (point-max))
-	    (insert string " ") ;; so there's a final property change
+	    (let ((inhibit-modification-hooks nil)) ; Avoid Bug#25132.
+              (delete-region (point-min) (point-max))
+              (insert string " ")) ;; so there's a final property change
 	    (unless (eq major-mode lang-mode) (funcall lang-mode))
             (org-font-lock-ensure)
 	    (setq pos (point-min))
-- 
2.9.3

Comments for bug#25132 fix? (Making with-silent-modifications buffer-local)

[npostavs]

Could someone comment on the patches I posted to Bug#25132?

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25132#22

Re: Comments for bug#25132 fix? (Making with-silent-modifications buffer-local)

[John Wiegley]

>>>>> "n" == npostavs  <npostavs@users.sourceforge.net> writes:

n> Could someone comment on the patches I posted to Bug#25132?
n> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25132#22

They look good to me, Noam.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

Re: Comments for bug#25132 fix? (Making with-silent-modifications buffer-local)

[Eli Zaretskii]

> From: John Wiegley <jwiegley@gmail.com>
> Date: Wed, 18 Jan 2017 19:51:59 -0800
> Cc: emacs-devel@gnu.org
> 
> >>>>> "n" == npostavs  <npostavs@users.sourceforge.net> writes:
> 
> n> Could someone comment on the patches I posted to Bug#25132?
> n> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25132#22
> 
> They look good to me, Noam.

Fine with me as well.

Thanks.

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[Dmitry Gutov]

On 08.01.2017 00:20, npostavs@users.sourceforge.net wrote:
> -            (inhibit-modification-hooks t))
> +            (inhibit-modification-hooks
> +             (progn (make-local-variable 'inhibit-modification-hooks) t)))

Are we not worried that inhibit-modificaiton-hooks will become 
buffer-local even after control flow leaves this let*?

If we are not, why not make inhibit-modification-hooks always 
buffer-local instead?

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[Dmitry Gutov]

On 08.01.2017 00:20, npostavs@users.sourceforge.net wrote:
> -            (inhibit-modification-hooks t))
> +            (inhibit-modification-hooks
> +             (progn (make-local-variable 'inhibit-modification-hooks) t)))

Are we not worried that inhibit-modificaiton-hooks will become 
buffer-local even after control flow leaves this let*?

If we are not, why not make inhibit-modification-hooks always 
buffer-local instead?

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[npostavs]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 08.01.2017 00:20, npostavs@users.sourceforge.net wrote:
>> -            (inhibit-modification-hooks t))
>> +            (inhibit-modification-hooks
>> +             (progn (make-local-variable 'inhibit-modification-hooks) t)))
>
> Are we not worried that inhibit-modificaiton-hooks will become
> buffer-local even after control flow leaves this let*?

My feeling is that inhibit-modification-hooks should usually be buffer
local anyway.

> If we are not, why not make inhibit-modification-hooks always
> buffer-local instead?

It would have to be in addition to, because even after doing
(make-variable-buffer-local 'var), (let ((var 'foo))...) still makes a
global binding.  `make-variable-buffer-local' only has effect for
`setq', which I think will hardly ever happen for
`inhibit-modification-hooks'.

Actually, I just grepped for inhibit-modification-hooks and the only
non-let I found is this:

(defun read-passwd (prompt &optional confirm default)
   ...
      (minibuffer-with-setup-hook
          (lambda ()
             ...
             (setq-local inhibit-modification-hooks nil) ;bug#15501.
             ...))
   ...)

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[npostavs]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 08.01.2017 00:20, npostavs@users.sourceforge.net wrote:
>> -            (inhibit-modification-hooks t))
>> +            (inhibit-modification-hooks
>> +             (progn (make-local-variable 'inhibit-modification-hooks) t)))
>
> Are we not worried that inhibit-modificaiton-hooks will become
> buffer-local even after control flow leaves this let*?

My feeling is that inhibit-modification-hooks should usually be buffer
local anyway.

> If we are not, why not make inhibit-modification-hooks always
> buffer-local instead?

It would have to be in addition to, because even after doing
(make-variable-buffer-local 'var), (let ((var 'foo))...) still makes a
global binding.  `make-variable-buffer-local' only has effect for
`setq', which I think will hardly ever happen for
`inhibit-modification-hooks'.

Actually, I just grepped for inhibit-modification-hooks and the only
non-let I found is this:

(defun read-passwd (prompt &optional confirm default)
   ...
      (minibuffer-with-setup-hook
          (lambda ()
             ...
             (setq-local inhibit-modification-hooks nil) ;bug#15501.
             ...))
   ...)

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 1008 bytes --]

On 2017-01-19 19:52, npostavs@users.sourceforge.net wrote:
> because even after doing (make-variable-buffer-local 'var), (let
> ((var 'foo))...) still makes a global binding.
> `make-variable-buffer-local' only has effect for `setq', which I
> think will hardly ever happen for `inhibit-modification-hooks'.

On 2017-01-19 19:52, npostavs@users.sourceforge.net wrote:
> because even after doing (make-variable-buffer-local 'var), (let
> ((var 'foo))...) still makes a global binding.
> `make-variable-buffer-local' only has effect for `setq', which I
> think will hardly ever happen for `inhibit-modification-hooks'.

Hi Noam,

Can you explain a bit more? I'm not sure what you meant.

I tried the following to illustrate your point:

(defvar aa 0)

(with-temp-buffer
  (setq-local aa 1)
  (let ((b1 (current-buffer)))
    (with-temp-buffer
      (let ((aa 2))
        (message "In b2: %S" aa)
        (with-current-buffer b1
          (message "In b1: %S" aa))))))

Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 1008 bytes --]

On 2017-01-19 19:52, npostavs@users.sourceforge.net wrote:
> because even after doing (make-variable-buffer-local 'var), (let
> ((var 'foo))...) still makes a global binding.
> `make-variable-buffer-local' only has effect for `setq', which I
> think will hardly ever happen for `inhibit-modification-hooks'.

On 2017-01-19 19:52, npostavs@users.sourceforge.net wrote:
> because even after doing (make-variable-buffer-local 'var), (let
> ((var 'foo))...) still makes a global binding.
> `make-variable-buffer-local' only has effect for `setq', which I
> think will hardly ever happen for `inhibit-modification-hooks'.

Hi Noam,

Can you explain a bit more? I'm not sure what you meant.

I tried the following to illustrate your point:

(defvar aa 0)

(with-temp-buffer
  (setq-local aa 1)
  (let ((b1 (current-buffer)))
    (with-temp-buffer
      (let ((aa 2))
        (message "In b2: %S" aa)
        (with-current-buffer b1
          (message "In b1: %S" aa))))))

Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[npostavs]

Clément Pit--Claudel <clement.pit@gmail.com> writes:

> On 2017-01-19 19:52, npostavs@users.sourceforge.net wrote:
>> because even after doing (make-variable-buffer-local 'var), (let
>> ((var 'foo))...) still makes a global binding.
>> `make-variable-buffer-local' only has effect for `setq', which I
>> think will hardly ever happen for `inhibit-modification-hooks'.
>
> On 2017-01-19 19:52, npostavs@users.sourceforge.net wrote:
>> because even after doing (make-variable-buffer-local 'var), (let
>> ((var 'foo))...) still makes a global binding.
>> `make-variable-buffer-local' only has effect for `setq', which I
>> think will hardly ever happen for `inhibit-modification-hooks'.
>
> Hi Noam,
>
> Can you explain a bit more? I'm not sure what you meant.
>
> I tried the following to illustrate your point:
>
> (defvar aa 0)
>
> (with-temp-buffer
>   (setq-local aa 1)
>   (let ((b1 (current-buffer)))
>     (with-temp-buffer
>       (let ((aa 2))
>         (message "In b2: %S" aa)
>         (with-current-buffer b1
>           (message "In b1: %S" aa))))))

My point was that the setq-local (or make-local-variable) is required
and that defvar-local (or make-variable-buffer-local) is not enough.

Compare:

(defvar-local bb 0)

(with-temp-buffer
  (let ((b1 (current-buffer)))
    (with-temp-buffer
      (let ((bb 2))
        (message "In b2: %S" bb)
        (with-current-buffer b1
          (message "In b1: %S" bb))))))

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[npostavs]

Clément Pit--Claudel <clement.pit@gmail.com> writes:

> On 2017-01-19 19:52, npostavs@users.sourceforge.net wrote:
>> because even after doing (make-variable-buffer-local 'var), (let
>> ((var 'foo))...) still makes a global binding.
>> `make-variable-buffer-local' only has effect for `setq', which I
>> think will hardly ever happen for `inhibit-modification-hooks'.
>
> On 2017-01-19 19:52, npostavs@users.sourceforge.net wrote:
>> because even after doing (make-variable-buffer-local 'var), (let
>> ((var 'foo))...) still makes a global binding.
>> `make-variable-buffer-local' only has effect for `setq', which I
>> think will hardly ever happen for `inhibit-modification-hooks'.
>
> Hi Noam,
>
> Can you explain a bit more? I'm not sure what you meant.
>
> I tried the following to illustrate your point:
>
> (defvar aa 0)
>
> (with-temp-buffer
>   (setq-local aa 1)
>   (let ((b1 (current-buffer)))
>     (with-temp-buffer
>       (let ((aa 2))
>         (message "In b2: %S" aa)
>         (with-current-buffer b1
>           (message "In b1: %S" aa))))))

My point was that the setq-local (or make-local-variable) is required
and that defvar-local (or make-variable-buffer-local) is not enough.

Compare:

(defvar-local bb 0)

(with-temp-buffer
  (let ((b1 (current-buffer)))
    (with-temp-buffer
      (let ((bb 2))
        (message "In b2: %S" bb)
        (with-current-buffer b1
          (message "In b1: %S" bb))))))

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[Dmitry Gutov]

On 20.01.2017 03:52, npostavs@users.sourceforge.net wrote:

> My feeling is that inhibit-modification-hooks should usually be buffer
> local anyway.

Maybe you're right.

inhibit-read-only, bound nearby, seems to be in the same situation.

>> If we are not, why not make inhibit-modification-hooks always
>> buffer-local instead?
>
> It would have to be in addition to, because even after doing
> (make-variable-buffer-local 'var), (let ((var 'foo))...) still makes a
> global binding.  `make-variable-buffer-local' only has effect for
> `setq', which I think will hardly ever happen for
> `inhibit-modification-hooks'.

You're right, and that sounds a little too complicated for my taste.

So, personally, I'd try to fix the particular instance first. Switching 
buffers inside with-silent-modifications is not a very common usage, I 
think.

Maybe org-src should itself let-bind the aforementioned variable(s) 
where it visits other buffers.

Up to you, of course, since you've already been given the go-ahead for 
the proposed fix.

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[Dmitry Gutov]

On 20.01.2017 03:52, npostavs@users.sourceforge.net wrote:

> My feeling is that inhibit-modification-hooks should usually be buffer
> local anyway.

Maybe you're right.

inhibit-read-only, bound nearby, seems to be in the same situation.

>> If we are not, why not make inhibit-modification-hooks always
>> buffer-local instead?
>
> It would have to be in addition to, because even after doing
> (make-variable-buffer-local 'var), (let ((var 'foo))...) still makes a
> global binding.  `make-variable-buffer-local' only has effect for
> `setq', which I think will hardly ever happen for
> `inhibit-modification-hooks'.

You're right, and that sounds a little too complicated for my taste.

So, personally, I'd try to fix the particular instance first. Switching 
buffers inside with-silent-modifications is not a very common usage, I 
think.

Maybe org-src should itself let-bind the aforementioned variable(s) 
where it visits other buffers.

Up to you, of course, since you've already been given the go-ahead for 
the proposed fix.

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[npostavs]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 20.01.2017 03:52, npostavs@users.sourceforge.net wrote:
>
>> My feeling is that inhibit-modification-hooks should usually be buffer
>> local anyway.
>
> Maybe you're right.
>
> inhibit-read-only, bound nearby, seems to be in the same situation.
>
>>> If we are not, why not make inhibit-modification-hooks always
>>> buffer-local instead?
>>
>> It would have to be in addition to, because even after doing
>> (make-variable-buffer-local 'var), (let ((var 'foo))...) still makes a
>> global binding.  `make-variable-buffer-local' only has effect for
>> `setq', which I think will hardly ever happen for
>> `inhibit-modification-hooks'.
>
> You're right, and that sounds a little too complicated for my taste.
>
> So, personally, I'd try to fix the particular instance
> first. Switching buffers inside with-silent-modifications is not a
> very common usage, I think.
>
> Maybe org-src should itself let-bind the aforementioned variable(s)
> where it visits other buffers.

Yeah, that works, and is my proposal for emacs-25, but I'm still leaning
towards solving this more broadly in with-silent-modifications, probably
also add a mention about this to the inhibit-modification-hooks
docstring.

I think doing the same to inhibit-read-only isn't worth the trouble
because if it happens to be let-bound to t in a buffer where it wasn't
"supposed" to be, the worst that happens is that an error *isn't*
thrown.

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[npostavs]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 20.01.2017 03:52, npostavs@users.sourceforge.net wrote:
>
>> My feeling is that inhibit-modification-hooks should usually be buffer
>> local anyway.
>
> Maybe you're right.
>
> inhibit-read-only, bound nearby, seems to be in the same situation.
>
>>> If we are not, why not make inhibit-modification-hooks always
>>> buffer-local instead?
>>
>> It would have to be in addition to, because even after doing
>> (make-variable-buffer-local 'var), (let ((var 'foo))...) still makes a
>> global binding.  `make-variable-buffer-local' only has effect for
>> `setq', which I think will hardly ever happen for
>> `inhibit-modification-hooks'.
>
> You're right, and that sounds a little too complicated for my taste.
>
> So, personally, I'd try to fix the particular instance
> first. Switching buffers inside with-silent-modifications is not a
> very common usage, I think.
>
> Maybe org-src should itself let-bind the aforementioned variable(s)
> where it visits other buffers.

Yeah, that works, and is my proposal for emacs-25, but I'm still leaning
towards solving this more broadly in with-silent-modifications, probably
also add a mention about this to the inhibit-modification-hooks
docstring.

I think doing the same to inhibit-read-only isn't worth the trouble
because if it happens to be let-bound to t in a buffer where it wasn't
"supposed" to be, the worst that happens is that an error *isn't*
thrown.

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[npostavs]

tags 25132 fixed
close 25132 25.2
quit

npostavs@users.sourceforge.net writes:

> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> So, personally, I'd try to fix the particular instance
>> first. Switching buffers inside with-silent-modifications is not a
>> very common usage, I think.
>>
>> Maybe org-src should itself let-bind the aforementioned variable(s)
>> where it visits other buffers.
>
> Yeah, that works, and is my proposal for emacs-25, but I'm still leaning
> towards solving this more broadly in with-silent-modifications, probably
> also add a mention about this to the inhibit-modification-hooks
> docstring.

I changed my mind.  Bug#25561 reminded me about the "Making <var> local
to <buffer> while let-bound!" message.  My change to
`with-silent-modifications' would trigger that on any nested invocations
of `with-silent-modifications' which seems more likely to happen than
switching buffers.

I've pushed the simpler let-bind in org-src solution [1: ae8264c] to
emacs-25.

1: 2017-01-29 11:01:32 -0500 ae8264c5cccf19d5b25a340a605bf2f07de1577e
  Call modification hooks in org-src fontify buffers

bug#25132: 26.0.50; emacs hangs when loading org file with python source blocks

[npostavs]

tags 25132 fixed
close 25132 25.2
quit

npostavs@users.sourceforge.net writes:

> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> So, personally, I'd try to fix the particular instance
>> first. Switching buffers inside with-silent-modifications is not a
>> very common usage, I think.
>>
>> Maybe org-src should itself let-bind the aforementioned variable(s)
>> where it visits other buffers.
>
> Yeah, that works, and is my proposal for emacs-25, but I'm still leaning
> towards solving this more broadly in with-silent-modifications, probably
> also add a mention about this to the inhibit-modification-hooks
> docstring.

I changed my mind.  Bug#25561 reminded me about the "Making <var> local
to <buffer> while let-bound!" message.  My change to
`with-silent-modifications' would trigger that on any nested invocations
of `with-silent-modifications' which seems more likely to happen than
switching buffers.

I've pushed the simpler let-bind in org-src solution [1: ae8264c] to
emacs-25.

1: 2017-01-29 11:01:32 -0500 ae8264c5cccf19d5b25a340a605bf2f07de1577e
  Call modification hooks in org-src fontify buffers