* Font lock in org+elisp confused with ?\[ @ 2021-04-01 5:43 Tom Gillespie 2021-04-01 7:18 ` Arthur Miller 0 siblings, 1 reply; 9+ messages in thread From: Tom Gillespie @ 2021-04-01 5:43 UTC (permalink / raw) To: arthur.miller, emacs-devel This is a known issue with org babel blocks. It is due to the fact that org babel translates the font locking for the language but not the syntax propertization. Another frequent cause is the bash case statement. The end result is that unmatched parens leak out from the babel blocks and wreak havoc elsewhere in the org file unless you balance out the parens e.g. in a comment. I have a hacked fix for this, but it has horrible performance, especially with line numbers enabled. I think that a proper solution would run arbitrary syntax propertization on subsets of a buffer without having to continually check where those subsets start or end. Best, Tom ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Font lock in org+elisp confused with ?\[ 2021-04-01 5:43 Font lock in org+elisp confused with ?\[ Tom Gillespie @ 2021-04-01 7:18 ` Arthur Miller 2021-04-01 9:04 ` tomas 0 siblings, 1 reply; 9+ messages in thread From: Arthur Miller @ 2021-04-01 7:18 UTC (permalink / raw) To: Tom Gillespie; +Cc: emacs-devel Tom Gillespie <tgbugs@gmail.com> writes: > This is a known issue with org babel blocks. It is due to the fact > that org babel translates the font locking for the language but not > the syntax propertization. Another frequent cause is the bash case > statement. The end result is that unmatched parens leak out from the > babel blocks and wreak havoc elsewhere in the org file unless you > balance out the parens e.g. in a comment. I have a hacked fix for > this, but it has horrible performance, especially with line numbers > enabled. I think that a proper solution would run arbitrary syntax > propertization on subsets of a buffer without having to continually > check where those subsets start or end. Best, > Tom Thanks for the explanation! I don't know how org-babel works, I guess it opens another buffer, font-locks syntax there and copy it over? What was the reason why they don't copy other properties? Performance or something else? ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Font lock in org+elisp confused with ?\[ 2021-04-01 7:18 ` Arthur Miller @ 2021-04-01 9:04 ` tomas 2021-04-01 15:44 ` Arthur Miller 0 siblings, 1 reply; 9+ messages in thread From: tomas @ 2021-04-01 9:04 UTC (permalink / raw) To: Arthur Miller; +Cc: Tom Gillespie, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1646 bytes --] On Thu, Apr 01, 2021 at 09:18:30AM +0200, Arthur Miller wrote: > Tom Gillespie <tgbugs@gmail.com> writes: > > > This is a known issue with org babel blocks. It is due to the fact > > that org babel translates the font locking for the language but not > > the syntax propertization. Another frequent cause is the bash case > > statement. The end result is that unmatched parens leak out from the > > babel blocks and wreak havoc elsewhere in the org file unless you > > balance out the parens e.g. in a comment. I have a hacked fix for > > this, but it has horrible performance, especially with line numbers > > enabled. I think that a proper solution would run arbitrary syntax > > propertization on subsets of a buffer without having to continually > > check where those subsets start or end. Best, > > Tom > > Thanks for the explanation! > > I don't know how org-babel works, I guess it opens another buffer, > font-locks syntax there and copy it over? What was the reason why they > don't copy other properties? Performance or something else? Hm. I've just poked around a bit. If you are talking about babel blocks, I think the entry point is `org-src-font-lock-fontify-block', defined in `org-src.el'. And indeed, this function basically does this (with a lot of corner-case chasing around, as is usual in our trade): (let ((lang-mode (org-src-get-lang-mode lang))) ... (with-current-buffer (get-buffer-create (format " *org-src-fontification:%s*" lang-mode)) ...)) The rest is left to the reader's fantasy, as they say :) Cheers - t [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 198 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Font lock in org+elisp confused with ?\[ 2021-04-01 9:04 ` tomas @ 2021-04-01 15:44 ` Arthur Miller 0 siblings, 0 replies; 9+ messages in thread From: Arthur Miller @ 2021-04-01 15:44 UTC (permalink / raw) To: tomas; +Cc: Tom Gillespie, emacs-devel [-- Attachment #1: Type: text/plain, Size: 2373 bytes --] <tomas@tuxteam.de> writes: > On Thu, Apr 01, 2021 at 09:18:30AM +0200, Arthur Miller wrote: >> Tom Gillespie <tgbugs@gmail.com> writes: >> >> > This is a known issue with org babel blocks. It is due to the fact >> > that org babel translates the font locking for the language but not >> > the syntax propertization. Another frequent cause is the bash case >> > statement. The end result is that unmatched parens leak out from the >> > babel blocks and wreak havoc elsewhere in the org file unless you >> > balance out the parens e.g. in a comment. I have a hacked fix for >> > this, but it has horrible performance, especially with line numbers >> > enabled. I think that a proper solution would run arbitrary syntax >> > propertization on subsets of a buffer without having to continually >> > check where those subsets start or end. Best, >> > Tom >> >> Thanks for the explanation! >> >> I don't know how org-babel works, I guess it opens another buffer, >> font-locks syntax there and copy it over? What was the reason why they >> don't copy other properties? Performance or something else? > > Hm. I've just poked around a bit. If you are talking about babel > blocks, I think the entry point is `org-src-font-lock-fontify-block', > defined in `org-src.el'. > > And indeed, this function basically does this (with a lot of > corner-case chasing around, as is usual in our trade): > > (let ((lang-mode (org-src-get-lang-mode lang))) > ... > (with-current-buffer > (get-buffer-create > (format " *org-src-fontification:%s*" lang-mode)) > ...)) > > The rest is left to the reader's fantasy, as they say :) :-) Left as an exercise to reader's fantasy? I tried to copy 'syntax-table property, but it didn't solve anything. I think the problem is some bad regexp somewhere when it come to indentation, it misses what it looks at somewhere (obviously), but I have no idea where. I am not sure it is org fault. I see gnus is confused too, so it is probably deeper, maybe in elisp mode? I took a profiler snapshot to get a nice stack-trace while moving the cursor just from one place after parenthesis to the closing parenhesis on this line: (if (= (char-before (point)) ?\[) I have attached profiler-repport. I also noticed that the function got called ~1000 times, while there are only ~140 src blocks. [-- Attachment #2: profile-repport --] [-- Type: text/plain, Size: 5475 bytes --] - command-execute 7,784,221 86% - call-interactively 7,784,221 86% - funcall-interactively 7,784,221 86% + helm-M-x 7,689,817 85% - next-line 90,356 1% - line-move 90,356 1% line-move-visual 64,756 0% - org-delete-backward-char 4,048 0% - backward-delete-char 2,832 0% - delete-char 2,832 0% - org-indent-notify-modified-headline 1,024 0% - org-at-heading-p 512 0% outline-on-heading-p 512 0% - jit-lock-after-change 1,024 0% - run-hook-with-args 1,024 0% - font-lock-extend-jit-lock-region-after-change 1,024 0% org-fontify-extend-region 1,024 0% - org-indent-refresh-maybe 784 0% - org-indent-add-properties 784 0% org-at-item-p 704 0% - org-fix-tags-on-the-fly 704 0% org-match-line 704 0% - timer-event-handler 633,847 7% - apply 632,095 7% + #<compiled 0xfe16b9> 627,071 7% - show-paren-function 4,136 0% show-paren--default 4,136 0% - blink-cursor-start 888 0% - blink-cursor--start-timer 888 0% - run-with-timer 888 0% - apply 888 0% - run-at-time 888 0% - timer-set-time 504 0% timer--time-setter 504 0% - timer-activate 336 0% - timer--activate 336 0% timer--time-less-p 336 0% timer-relative-time 48 0% - timer-inc-time 816 0% timer-relative-time 600 0% timer--time-setter 216 0% - timer-activate 720 0% - timer--activate 720 0% timer--time-less-p 720 0% - beacon--post-command 347,992 3% - beacon--maybe-push-mark 347,992 3% beacon--movement-> 347,992 3% + redisplay_internal (C function) 141,823 1% - winner-save-old-configurations 33,088 0% winner-remember 33,088 0% - eldoc-pre-command-refresh-echo-area 4,724 0% - eldoc-message 4,724 0% - eldoc-minibuffer-message 4,724 0% apply 4,724 0% - ... 4,608 0% - helm-create-helm-buffer 4,608 0% - helm-major-mode 4,608 0% - run-mode-hooks 4,608 0% - run-hooks 4,608 0% - wrap-region-global-mode-enable-in-buffers 4,608 0% - turn-on-wrap-region-mode 4,608 0% - wrap-region-mode 4,608 0% - wrap-region-define-wrappers 4,608 0% - mapc 4,608 0% - #<compiled 0x191c53d> 4,608 0% - apply 4,608 0% - wrap-region-add-wrapper 4,608 0% + wrap-region-define-trigger 4,608 0% beacon--record-vars 4,136 0% - eldoc-schedule-timer 24 0% - run-with-idle-timer 24 0% - timer-set-idle-time 24 0% timer--time-setter 24 0% [-- Attachment #3: Type: text/plain, Size: 28 bytes --] > Cheers Happy Eastern! ^ permalink raw reply [flat|nested] 9+ messages in thread
* Font lock in org+elisp confused with ?\[
@ 2021-04-01 4:42 Arthur Miller
2021-04-02 4:23 ` Tim Cross
0 siblings, 1 reply; 9+ messages in thread
From: Arthur Miller @ 2021-04-01 4:42 UTC (permalink / raw)
To: emacs-devel
Is it me or is it a bug?
When in org mode in an elisp block, this seems to confuse syntax
checker:
#+begin_src emacs-lisp
(progn
(if (= (following-char) ?\])
(forward-char -1))
)
#+end_src
Identation seems to think it is one level extra, and it also shows as
error. Same of course when testing for ?\[.
It does evaluate correctly. Ordinary elisp buffer does not have problem
with this, only when in code blocks in org-mode or elisp mode.I can send
in some screenshot with errors if needed.
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: Font lock in org+elisp confused with ?\[ 2021-04-01 4:42 Arthur Miller @ 2021-04-02 4:23 ` Tim Cross 2021-04-02 13:03 ` John Kitchin 0 siblings, 1 reply; 9+ messages in thread From: Tim Cross @ 2021-04-02 4:23 UTC (permalink / raw) To: Arthur Miller; +Cc: Org-mode, Emacs developers [-- Attachment #1: Type: text/plain, Size: 759 bytes --] I'm forwarding this to the emacs-orgmode list as this is something org maintainers probably need to see. On Thu, 1 Apr 2021 at 15:43, Arthur Miller <arthur.miller@live.com> wrote: > > Is it me or is it a bug? > > When in org mode in an elisp block, this seems to confuse syntax > checker: > > #+begin_src emacs-lisp > (progn > (if (= (following-char) ?\]) > (forward-char -1)) > ) > #+end_src > > Identation seems to think it is one level extra, and it also shows as > error. Same of course when testing for ?\[. > > It does evaluate correctly. Ordinary elisp buffer does not have problem > with this, only when in code blocks in org-mode or elisp mode.I can send > in some screenshot with errors if needed. > > > -- regards, Tim -- Tim Cross [-- Attachment #2: Type: text/html, Size: 1296 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Font lock in org+elisp confused with ?\[ 2021-04-02 4:23 ` Tim Cross @ 2021-04-02 13:03 ` John Kitchin 2021-04-02 22:59 ` Tom Gillespie 2021-04-03 18:38 ` Arthur Miller 0 siblings, 2 replies; 9+ messages in thread From: John Kitchin @ 2021-04-02 13:03 UTC (permalink / raw) To: Tim Cross; +Cc: Org-mode, Arthur Miller, Emacs developers [-- Attachment #1: Type: text/plain, Size: 1400 bytes --] This is related to the issues with <> in src blocks. [ and ] have open and close syntactical meanings like < and > do in org files. A similar solution as found in https://emacs.stackexchange.com/questions/50216/org-mode-code-block-parentheses-mismatch seems to work to fix it. John ----------------------------------- Professor John Kitchin (he/him/his) Doherty Hall A207F Department of Chemical Engineering Carnegie Mellon University Pittsburgh, PA 15213 412-268-7803 @johnkitchin http://kitchingroup.cheme.cmu.edu On Fri, Apr 2, 2021 at 12:24 AM Tim Cross <theophilusx@gmail.com> wrote: > I'm forwarding this to the emacs-orgmode list as this is something org > maintainers probably need to see. > > On Thu, 1 Apr 2021 at 15:43, Arthur Miller <arthur.miller@live.com> wrote: > >> >> Is it me or is it a bug? >> >> When in org mode in an elisp block, this seems to confuse syntax >> checker: >> >> #+begin_src emacs-lisp >> (progn >> (if (= (following-char) ?\]) >> (forward-char -1)) >> ) >> #+end_src >> >> Identation seems to think it is one level extra, and it also shows as >> error. Same of course when testing for ?\[. >> >> It does evaluate correctly. Ordinary elisp buffer does not have problem >> with this, only when in code blocks in org-mode or elisp mode.I can send >> in some screenshot with errors if needed. >> >> >> > > -- > regards, > > Tim > > -- > Tim Cross > > [-- Attachment #2: Type: text/html, Size: 2567 bytes --] ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Font lock in org+elisp confused with ?\[ 2021-04-02 13:03 ` John Kitchin @ 2021-04-02 22:59 ` Tom Gillespie 2021-04-03 18:38 ` Arthur Miller 1 sibling, 0 replies; 9+ messages in thread From: Tom Gillespie @ 2021-04-02 22:59 UTC (permalink / raw) To: John Kitchin; +Cc: Tim Cross, Org-mode, Arthur Miller, Emacs developers Reposting my reply to the emacs-devel thread here as well. The hack I mention that has performance issues was derived from John's solution for the <> issue (though the performance issues are all of my own creation). Best, Tom This is a known issue with org babel blocks. It is due to the fact that org babel translates the font locking for the language but not the syntax propertization. Another frequent cause is the bash case statement. The end result is that unmatched parens leak out from the babel blocks and wreak havoc elsewhere in the org file unless you balance out the parens e.g. in a comment. I have a hacked fix for this, but it has horrible performance, especially with line numbers enabled. I think that a proper solution would run arbitrary syntax propertization on subsets of a buffer without having to continually check where those subsets start or end. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: Font lock in org+elisp confused with ?\[ 2021-04-02 13:03 ` John Kitchin 2021-04-02 22:59 ` Tom Gillespie @ 2021-04-03 18:38 ` Arthur Miller 1 sibling, 0 replies; 9+ messages in thread From: Arthur Miller @ 2021-04-03 18:38 UTC (permalink / raw) To: John Kitchin; +Cc: Tim Cross, Org-mode, Emacs developers John Kitchin <jkitchin@andrew.cmu.edu> writes: > This is related to the issues with <> in src blocks. [ and ] have open and close syntactical meanings like < and > do in org files. A similar solution as found in > https://emacs.stackexchange.com/questions/50216/org-mode-code-block-parentheses-mismatch seems to work to fix it. Indeed, the code from SX works, I took the simpler version and converted it to work with square brackets: #+begin_src emacs-lisp (defun org-mode-sqbr-syntax-fix (start end) (when (eq major-mode 'org-mode) (save-excursion (goto-char start) (while (re-search-forward "[]\\[]" end t) (when (get-text-property (point) 'src-block) ;; This is a ?[ or ?] in an org-src block (put-text-property (point) (1- (point)) 'syntax-table (string-to-syntax "_"))))))) (defun org-setup-sqbr-syntax-fix () "Setup for characters ?[ and ?] in source code blocks. Add this function to `org-mode-hook'." (setq syntax-propertize-function 'org-mode-sqbr-syntax-fix) (syntax-propertize (point-max))) (add-hook 'org-mode-hook 'org-setup-sqbr-syntax-fix) #+end_src I have confirmed that code runs when I load my org file, but there seems to be something else than that callback involed, later on, evaluating and changing those properties back. I still see same error in the babel-block. However when I run fix explicitly on that code block (I converted it to interactive "r" for test), then syntax is fine. So it seems there is some other callback involved. Sorry little bit late answer, I thought I sent this a day ago or so, but apparently I didn't. Thanks for the help. ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-04-03 18:38 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-04-01 5:43 Font lock in org+elisp confused with ?\[ Tom Gillespie 2021-04-01 7:18 ` Arthur Miller 2021-04-01 9:04 ` tomas 2021-04-01 15:44 ` Arthur Miller -- strict thread matches above, loose matches on Subject: below -- 2021-04-01 4:42 Arthur Miller 2021-04-02 4:23 ` Tim Cross 2021-04-02 13:03 ` John Kitchin 2021-04-02 22:59 ` Tom Gillespie 2021-04-03 18:38 ` Arthur Miller
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).