unofficial mirror of emacs-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Font lock in org+elisp confused with ?\[
@ 2021-04-01  4:42 Arthur Miller
  2021-04-02  4:23 ` Tim Cross
  0 siblings, 1 reply; 9+ messages in thread
From: Arthur Miller @ 2021-04-01  4:42 UTC (permalink / raw)
  To: emacs-devel


Is it me or is it a bug?

When in org mode in an elisp block, this seems to confuse syntax
checker:

#+begin_src emacs-lisp
(progn
  (if (= (following-char) ?\])
      (forward-char -1))
  )
#+end_src

Identation seems to think it is one level extra, and it also shows as
error. Same of course when testing for ?\[.

It does evaluate correctly. Ordinary elisp buffer does not have problem
with this, only when in code blocks in org-mode or elisp mode.I can send
in some screenshot with errors if needed.




^ permalink raw reply	[flat|nested] 9+ messages in thread

* Font lock in org+elisp confused with ?\[
@ 2021-04-01  5:43 Tom Gillespie
  2021-04-01  7:18 ` Arthur Miller
  0 siblings, 1 reply; 9+ messages in thread
From: Tom Gillespie @ 2021-04-01  5:43 UTC (permalink / raw)
  To: arthur.miller, emacs-devel

This is a known issue with org babel blocks. It is due to the fact
that org babel translates the font locking for the language but not
the syntax propertization. Another frequent cause is the bash case
statement. The end result is that unmatched parens leak out from the
babel blocks and wreak havoc elsewhere in the org file unless you
balance out the parens e.g. in a comment. I have a hacked fix for
this, but it has horrible performance, especially with line numbers
enabled. I think that a proper solution would run arbitrary syntax
propertization on subsets of a buffer without having to continually
check where those subsets start or end. Best,
Tom



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Font lock in org+elisp confused with ?\[
  2021-04-01  5:43 Tom Gillespie
@ 2021-04-01  7:18 ` Arthur Miller
  2021-04-01  9:04   ` tomas
  0 siblings, 1 reply; 9+ messages in thread
From: Arthur Miller @ 2021-04-01  7:18 UTC (permalink / raw)
  To: Tom Gillespie; +Cc: emacs-devel

Tom Gillespie <tgbugs@gmail.com> writes:

> This is a known issue with org babel blocks. It is due to the fact
> that org babel translates the font locking for the language but not
> the syntax propertization. Another frequent cause is the bash case
> statement. The end result is that unmatched parens leak out from the
> babel blocks and wreak havoc elsewhere in the org file unless you
> balance out the parens e.g. in a comment. I have a hacked fix for
> this, but it has horrible performance, especially with line numbers
> enabled. I think that a proper solution would run arbitrary syntax
> propertization on subsets of a buffer without having to continually
> check where those subsets start or end. Best,
> Tom

Thanks for the explanation!

I don't know how org-babel works, I guess it opens another buffer,
font-locks syntax there and copy it over? What was the reason why they
don't copy other properties? Performance or something else?



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Font lock in org+elisp confused with ?\[
  2021-04-01  7:18 ` Arthur Miller
@ 2021-04-01  9:04   ` tomas
  2021-04-01 15:44     ` Arthur Miller
  0 siblings, 1 reply; 9+ messages in thread
From: tomas @ 2021-04-01  9:04 UTC (permalink / raw)
  To: Arthur Miller; +Cc: Tom Gillespie, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 1646 bytes --]

On Thu, Apr 01, 2021 at 09:18:30AM +0200, Arthur Miller wrote:
> Tom Gillespie <tgbugs@gmail.com> writes:
> 
> > This is a known issue with org babel blocks. It is due to the fact
> > that org babel translates the font locking for the language but not
> > the syntax propertization. Another frequent cause is the bash case
> > statement. The end result is that unmatched parens leak out from the
> > babel blocks and wreak havoc elsewhere in the org file unless you
> > balance out the parens e.g. in a comment. I have a hacked fix for
> > this, but it has horrible performance, especially with line numbers
> > enabled. I think that a proper solution would run arbitrary syntax
> > propertization on subsets of a buffer without having to continually
> > check where those subsets start or end. Best,
> > Tom
> 
> Thanks for the explanation!
> 
> I don't know how org-babel works, I guess it opens another buffer,
> font-locks syntax there and copy it over? What was the reason why they
> don't copy other properties? Performance or something else?

Hm. I've just poked around a bit. If you are talking about babel
blocks, I think the entry point is `org-src-font-lock-fontify-block',
defined in `org-src.el'.

And indeed, this function basically does this (with a lot of
corner-case chasing around, as is usual in our trade):

  (let ((lang-mode (org-src-get-lang-mode lang)))
        ...
        (with-current-buffer
            (get-buffer-create
               (format " *org-src-fontification:%s*" lang-mode))
               ...))

The rest is left to the reader's fantasy, as they say :)

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Font lock in org+elisp confused with ?\[
  2021-04-01  9:04   ` tomas
@ 2021-04-01 15:44     ` Arthur Miller
  0 siblings, 0 replies; 9+ messages in thread
From: Arthur Miller @ 2021-04-01 15:44 UTC (permalink / raw)
  To: tomas; +Cc: Tom Gillespie, emacs-devel

[-- Attachment #1: Type: text/plain, Size: 2373 bytes --]

<tomas@tuxteam.de> writes:

> On Thu, Apr 01, 2021 at 09:18:30AM +0200, Arthur Miller wrote:
>> Tom Gillespie <tgbugs@gmail.com> writes:
>> 
>> > This is a known issue with org babel blocks. It is due to the fact
>> > that org babel translates the font locking for the language but not
>> > the syntax propertization. Another frequent cause is the bash case
>> > statement. The end result is that unmatched parens leak out from the
>> > babel blocks and wreak havoc elsewhere in the org file unless you
>> > balance out the parens e.g. in a comment. I have a hacked fix for
>> > this, but it has horrible performance, especially with line numbers
>> > enabled. I think that a proper solution would run arbitrary syntax
>> > propertization on subsets of a buffer without having to continually
>> > check where those subsets start or end. Best,
>> > Tom
>> 
>> Thanks for the explanation!
>> 
>> I don't know how org-babel works, I guess it opens another buffer,
>> font-locks syntax there and copy it over? What was the reason why they
>> don't copy other properties? Performance or something else?
>
> Hm. I've just poked around a bit. If you are talking about babel
> blocks, I think the entry point is `org-src-font-lock-fontify-block',
> defined in `org-src.el'.
>
> And indeed, this function basically does this (with a lot of
> corner-case chasing around, as is usual in our trade):
>
>   (let ((lang-mode (org-src-get-lang-mode lang)))
>         ...
>         (with-current-buffer
>             (get-buffer-create
>                (format " *org-src-fontification:%s*" lang-mode))
>                ...))
>
> The rest is left to the reader's fantasy, as they say :)

:-) Left as an exercise to reader's fantasy?

I tried to copy 'syntax-table property, but it didn't solve anything.

I think the problem is some bad regexp somewhere when it come to
indentation, it misses what it looks at somewhere (obviously),
but I have no idea where. I am not sure it is org fault.

I see gnus is confused too, so it is probably deeper, maybe in elisp
mode?

I took a profiler snapshot to get a nice stack-trace while moving the
cursor just from one place after parenthesis to the closing parenhesis
on this line:

(if (= (char-before (point)) ?\[)

I have attached profiler-repport.

I also noticed that the function got called ~1000 times, while there are
only ~140 src blocks.


[-- Attachment #2: profile-repport --]
[-- Type: text/plain, Size: 5475 bytes --]

- command-execute                                           7,784,221  86%
 - call-interactively                                       7,784,221  86%
  - funcall-interactively                                   7,784,221  86%
   + helm-M-x                                               7,689,817  85%
   - next-line                                                 90,356   1%
    - line-move                                                90,356   1%
       line-move-visual                                        64,756   0%
   - org-delete-backward-char                                   4,048   0%
    - backward-delete-char                                      2,832   0%
     - delete-char                                              2,832   0%
      - org-indent-notify-modified-headline                     1,024   0%
       - org-at-heading-p                                         512   0%
          outline-on-heading-p                                    512   0%
      - jit-lock-after-change                                   1,024   0%
       - run-hook-with-args                                     1,024   0%
        - font-lock-extend-jit-lock-region-after-change         1,024   0%
           org-fontify-extend-region                            1,024   0%
      - org-indent-refresh-maybe                                  784   0%
       - org-indent-add-properties                                784   0%
          org-at-item-p                                           704   0%
    - org-fix-tags-on-the-fly                                     704   0%
       org-match-line                                             704   0%
- timer-event-handler                                         633,847   7%
 - apply                                                      632,095   7%
  + #<compiled 0xfe16b9>                                      627,071   7%
  - show-paren-function                                         4,136   0%
     show-paren--default                                        4,136   0%
  - blink-cursor-start                                            888   0%
   - blink-cursor--start-timer                                    888   0%
    - run-with-timer                                              888   0%
     - apply                                                      888   0%
      - run-at-time                                               888   0%
       - timer-set-time                                           504   0%
          timer--time-setter                                      504   0%
       - timer-activate                                           336   0%
        - timer--activate                                         336   0%
           timer--time-less-p                                     336   0%
         timer-relative-time                                       48   0%
 - timer-inc-time                                                 816   0%
    timer-relative-time                                           600   0%
    timer--time-setter                                            216   0%
 - timer-activate                                                 720   0%
  - timer--activate                                               720   0%
     timer--time-less-p                                           720   0%
- beacon--post-command                                        347,992   3%
 - beacon--maybe-push-mark                                    347,992   3%
    beacon--movement->                                        347,992   3%
+ redisplay_internal (C function)                             141,823   1%
- winner-save-old-configurations                               33,088   0%
   winner-remember                                             33,088   0%
- eldoc-pre-command-refresh-echo-area                           4,724   0%
 - eldoc-message                                                4,724   0%
  - eldoc-minibuffer-message                                    4,724   0%
     apply                                                      4,724   0%
- ...                                                           4,608   0%
 - helm-create-helm-buffer                                      4,608   0%
  - helm-major-mode                                             4,608   0%
   - run-mode-hooks                                             4,608   0%
    - run-hooks                                                 4,608   0%
     - wrap-region-global-mode-enable-in-buffers                4,608   0%
      - turn-on-wrap-region-mode                                4,608   0%
       - wrap-region-mode                                       4,608   0%
        - wrap-region-define-wrappers                           4,608   0%
         - mapc                                                 4,608   0%
          - #<compiled 0x191c53d>                               4,608   0%
           - apply                                              4,608   0%
            - wrap-region-add-wrapper                           4,608   0%
             + wrap-region-define-trigger                       4,608   0%
  beacon--record-vars                                           4,136   0%
- eldoc-schedule-timer                                             24   0%
 - run-with-idle-timer                                             24   0%
  - timer-set-idle-time                                            24   0%
     timer--time-setter                                            24   0%

[-- Attachment #3: Type: text/plain, Size: 28 bytes --]



> Cheers

Happy Eastern! 

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Font lock in org+elisp confused with ?\[
  2021-04-01  4:42 Font lock in org+elisp confused with ?\[ Arthur Miller
@ 2021-04-02  4:23 ` Tim Cross
  2021-04-02 13:03   ` John Kitchin
  0 siblings, 1 reply; 9+ messages in thread
From: Tim Cross @ 2021-04-02  4:23 UTC (permalink / raw)
  To: Arthur Miller; +Cc: Org-mode, Emacs developers

[-- Attachment #1: Type: text/plain, Size: 759 bytes --]

I'm forwarding this to the emacs-orgmode list as this is something org
maintainers probably need to see.

On Thu, 1 Apr 2021 at 15:43, Arthur Miller <arthur.miller@live.com> wrote:

>
> Is it me or is it a bug?
>
> When in org mode in an elisp block, this seems to confuse syntax
> checker:
>
> #+begin_src emacs-lisp
> (progn
>   (if (= (following-char) ?\])
>       (forward-char -1))
>   )
> #+end_src
>
> Identation seems to think it is one level extra, and it also shows as
> error. Same of course when testing for ?\[.
>
> It does evaluate correctly. Ordinary elisp buffer does not have problem
> with this, only when in code blocks in org-mode or elisp mode.I can send
> in some screenshot with errors if needed.
>
>
>

-- 
regards,

Tim

--
Tim Cross

[-- Attachment #2: Type: text/html, Size: 1296 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Font lock in org+elisp confused with ?\[
  2021-04-02  4:23 ` Tim Cross
@ 2021-04-02 13:03   ` John Kitchin
  2021-04-02 22:59     ` Tom Gillespie
  2021-04-03 18:38     ` Arthur Miller
  0 siblings, 2 replies; 9+ messages in thread
From: John Kitchin @ 2021-04-02 13:03 UTC (permalink / raw)
  To: Tim Cross; +Cc: Org-mode, Arthur Miller, Emacs developers

[-- Attachment #1: Type: text/plain, Size: 1400 bytes --]

This is related to the issues with <> in src blocks. [ and ] have open and
close syntactical meanings like < and > do in org files. A similar solution
as found in
https://emacs.stackexchange.com/questions/50216/org-mode-code-block-parentheses-mismatch
seems to work to fix it.

John

-----------------------------------
Professor John Kitchin (he/him/his)
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu



On Fri, Apr 2, 2021 at 12:24 AM Tim Cross <theophilusx@gmail.com> wrote:

> I'm forwarding this to the emacs-orgmode list as this is something org
> maintainers probably need to see.
>
> On Thu, 1 Apr 2021 at 15:43, Arthur Miller <arthur.miller@live.com> wrote:
>
>>
>> Is it me or is it a bug?
>>
>> When in org mode in an elisp block, this seems to confuse syntax
>> checker:
>>
>> #+begin_src emacs-lisp
>> (progn
>>   (if (= (following-char) ?\])
>>       (forward-char -1))
>>   )
>> #+end_src
>>
>> Identation seems to think it is one level extra, and it also shows as
>> error. Same of course when testing for ?\[.
>>
>> It does evaluate correctly. Ordinary elisp buffer does not have problem
>> with this, only when in code blocks in org-mode or elisp mode.I can send
>> in some screenshot with errors if needed.
>>
>>
>>
>
> --
> regards,
>
> Tim
>
> --
> Tim Cross
>
>

[-- Attachment #2: Type: text/html, Size: 2567 bytes --]

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Font lock in org+elisp confused with ?\[
  2021-04-02 13:03   ` John Kitchin
@ 2021-04-02 22:59     ` Tom Gillespie
  2021-04-03 18:38     ` Arthur Miller
  1 sibling, 0 replies; 9+ messages in thread
From: Tom Gillespie @ 2021-04-02 22:59 UTC (permalink / raw)
  To: John Kitchin; +Cc: Tim Cross, Org-mode, Arthur Miller, Emacs developers

Reposting my reply to the emacs-devel thread here as well. The hack I
mention that has performance issues was derived from John's solution
for the <> issue (though the performance issues are all of my own
creation). Best,
Tom

This is a known issue with org babel blocks. It is due to the fact
that org babel translates the font locking for the language but not
the syntax propertization. Another frequent cause is the bash case
statement. The end result is that unmatched parens leak out from the
babel blocks and wreak havoc elsewhere in the org file unless you
balance out the parens e.g. in a comment. I have a hacked fix for
this, but it has horrible performance, especially with line numbers
enabled. I think that a proper solution would run arbitrary syntax
propertization on subsets of a buffer without having to continually
check where those subsets start or end.



^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: Font lock in org+elisp confused with ?\[
  2021-04-02 13:03   ` John Kitchin
  2021-04-02 22:59     ` Tom Gillespie
@ 2021-04-03 18:38     ` Arthur Miller
  1 sibling, 0 replies; 9+ messages in thread
From: Arthur Miller @ 2021-04-03 18:38 UTC (permalink / raw)
  To: John Kitchin; +Cc: Tim Cross, Org-mode, Emacs developers

John Kitchin <jkitchin@andrew.cmu.edu> writes:

> This is related to the issues with <> in src blocks. [ and ] have open and close syntactical meanings like < and > do in org files. A similar solution as found in
> https://emacs.stackexchange.com/questions/50216/org-mode-code-block-parentheses-mismatch seems to work to fix it.

Indeed, the code from SX works, I took the simpler version and converted
it to work with square brackets:

#+begin_src emacs-lisp
             (defun org-mode-sqbr-syntax-fix (start end)
               (when (eq major-mode 'org-mode)
                 (save-excursion
                   (goto-char start)
                   (while (re-search-forward "[]\\[]" end t)
                     (when (get-text-property (point) 'src-block)
                       ;; This is a ?[ or ?] in an org-src block
                       (put-text-property (point) (1- (point))
                                          'syntax-table (string-to-syntax "_")))))))

             (defun org-setup-sqbr-syntax-fix ()
               "Setup for characters ?[ and ?] in source code blocks.
  Add this function to `org-mode-hook'."
               (setq syntax-propertize-function 'org-mode-sqbr-syntax-fix)
               (syntax-propertize (point-max)))

             (add-hook 'org-mode-hook 'org-setup-sqbr-syntax-fix)
#+end_src

I have confirmed that code runs when I load my org file, but there seems
to be something else than that callback involed, later on, evaluating
and changing those properties back. I still see same error in the
babel-block. However when I run fix explicitly on that code block (I
converted it to interactive "r" for test), then syntax is fine. So it
seems there is some other callback involved.

Sorry little bit late answer, I thought I sent this a day ago or so, but
apparently I didn't.

Thanks for the help.



^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2021-04-03 18:38 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-01  4:42 Font lock in org+elisp confused with ?\[ Arthur Miller
2021-04-02  4:23 ` Tim Cross
2021-04-02 13:03   ` John Kitchin
2021-04-02 22:59     ` Tom Gillespie
2021-04-03 18:38     ` Arthur Miller
  -- strict thread matches above, loose matches on Subject: below --
2021-04-01  5:43 Tom Gillespie
2021-04-01  7:18 ` Arthur Miller
2021-04-01  9:04   ` tomas
2021-04-01 15:44     ` Arthur Miller

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).