From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id +CnrBlP3WmS8iAAASxT56A (envelope-from ) for ; Wed, 10 May 2023 03:45:55 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id oOTxBlP3WmSVDwAAauVa8A (envelope-from ) for ; Wed, 10 May 2023 03:45:55 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5B00F3A554 for ; Wed, 10 May 2023 03:45:54 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pwYtB-00011N-2p; Tue, 09 May 2023 21:45:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pwYt8-00011C-1O for emacs-orgmode@gnu.org; Tue, 09 May 2023 21:45:06 -0400 Received: from mail-qt1-x82f.google.com ([2607:f8b0:4864:20::82f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pwYt5-00042P-Nu for emacs-orgmode@gnu.org; Tue, 09 May 2023 21:45:05 -0400 Received: by mail-qt1-x82f.google.com with SMTP id d75a77b69052e-3f393a1fca5so4071021cf.0 for ; Tue, 09 May 2023 18:44:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1683683083; x=1686275083; h=mime-version:message-id:in-reply-to:date:subject:cc:to:from :user-agent:references:from:to:cc:subject:date:message-id:reply-to; bh=Yx5Tn59VvQgjiITv5T+Odc+CAKNvHW6CAhP7V2IGF1k=; b=W0cBvc1zbylsALE1xnUmhOy7a0j1cZiXyUFPNBpMPG3kHLmgSS5/FtAXVPjHBPgZak yfwctlFnAorWxbKTA6eXn5+/wl0QNbRVgxlysfhtvDQ4vrZnc+C08PGbj38gx9LWH1cT u9IUrOd5eqg9A7qOGo2DwhpBoG4MAj0O2UJshV1/1IU9p4W4Av5szGbol4soJGe8Vv8c 571o2yYtMc/Q9o6uMjsJ9oxWrz1pCfosd/cdd8gD4G4MKmJK893rBRMjNncI/oXXiOQu H805240oVBZUmsqxn3K6gey0QI03X7J8mVVHUnCqWNo2vrU763LQwoOfcENWdbKdOs2w YEOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683683083; x=1686275083; h=mime-version:message-id:in-reply-to:date:subject:cc:to:from :user-agent:references:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Yx5Tn59VvQgjiITv5T+Odc+CAKNvHW6CAhP7V2IGF1k=; b=C+DwOsPEpqNbOBm8vHX09S9Ixrm38AJJFTOUF00J8JVw2lY5SfK9Pu3oY4Gn8FAIw2 LrjJg+DgvV8ECTl+MkfQPTaYLxseknqPjBtOF+IoykCjITPkJ7FcMN3AhbLqHjKjzgNa LbRveSJ5gfvnnYHO/7D8oRiuxgnbeV/bPSoQ7Y5nvLMzxzMJRvzTxezbXUaJaKvzC1aU Ggpvf0Gd5CCk//GEQOpTSKij5kLFFA3GgRyWnEBCkoGbXQT912iqY+Noz4sB/Pf7br66 zg5VCbosyKmUdD+PsagBhVKvr8vyaXgv7rVeoDkW+yWeI5uhaKpEAXHK1QmDVSzXBDy0 YwoQ== X-Gm-Message-State: AC+VfDzIYa2qbV0ErqEAd2qV8Knae7emZadb4c2cJUTHpuaW1iKP8axg yqMFRR5TOCjoxJpwSP2ene+ePFHjdWc= X-Google-Smtp-Source: ACHHUZ6UlwJsG6PVeYoUeYyqzhZfBx9VzQ7Y4NRbfllCNDHJOJxIFa0URHWaDPtiLXC/v4vkvVMHgQ== X-Received: by 2002:a05:622a:1807:b0:3f3:98b7:7df with SMTP id t7-20020a05622a180700b003f398b707dfmr5941874qtc.6.1683683082684; Tue, 09 May 2023 18:44:42 -0700 (PDT) Received: from entropy ([2601:241:8c80:25a0::8a55]) by smtp.gmail.com with ESMTPSA id h26-20020ac8777a000000b003eae8609f64sm989733qtu.13.2023.05.09.18.44.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 May 2023 18:44:42 -0700 (PDT) References: <874jpuijpc.fsf@gmail.com> <87y1n6igvo.fsf@localhost> <878rev1q0k.fsf@gmail.com> <877cueonkj.fsf@localhost> User-agent: mu4e 1.8.13; emacs 30.0.50 From: Nathaniel Nicandro To: Ihor Radchenko Cc: emacs-orgmode Subject: Re: [PATCH] Highlight ANSI sequences in the whole buffer (was [PATCH] ANSI color on example blocks and fixed width elements) Date: Tue, 25 Apr 2023 15:33:01 -0500 In-reply-to: <877cueonkj.fsf@localhost> Message-ID: <87zg6dez93.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2607:f8b0:4864:20::82f; envelope-from=nathanielnicandro@gmail.com; helo=mail-qt1-x82f.google.com X-Spam_score_int: 13 X-Spam_score: 1.3 X-Spam_bar: + X-Spam_report: (1.3 / 5.0 requ) BAYES_00=-1.9, DATE_IN_PAST_96_XX=3.405, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1683683154; a=rsa-sha256; cv=none; b=Kl+Cu0eEUa/iHJEdrcMbfG3iTiUHSiCIoqIsYHixrp6cXpL5AsCfSsXvPKY/cqEtM4MoiN kDMRJaArtYyf/B+cMqyWeczG8CYOcl61zCGhbdXUxVSzuGwMStQxUDRFxHC0XrG+SFv42x t85nCrovqcu1aLTMgCJP6XRPUAGDgp6ckhdDjkEE0m0dPsgf95Z3Sok0Wl0nWLaoI6A3in Y4lldfbUbBm6Qfz3Gpbsw+ZMwOQLUxvVYPtYsg5vR348/PTW7dwH7rQXQB3MvS7tMquazh lFGiQfMqRbRFl7vmCXkIs3mVyFAdLKX/H8jANo9Eabc7FlpeTSFLYfNibJQsJQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gmail.com header.s=20221208 header.b=W0cBvc1z; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1683683154; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=c01SEywmiIEf5EK8s3vhSk9RQwNAu8sewxFLs7dkmck=; b=A69rqy4aNinaLNRok1/js98CBEs/gpiatKraizWEN28fMoAgZ+zd3xZm1belUzp8xBqdt7 U6aIbV3HxGYVQsaU36si21Lm6qBXnf4v7jQEnzeaL+Z/BbHrobOkV4LvlvjXOK3Cw6g0ID mFgHSPLXRQ5duo/5Pv49GSO9PhQZZ03/LYWS7/TFf9rzYpJFmMoVlq7+AbXN40ycUsmB1u oxknAFG+cYfUX9kj4Kx/vK3EVK5Kr+DxKwpNIY4ceNSoaQryPw0yD1l+8isbLg4Q3FodQa ek1WQigGiNf94mjFXndDa/UCMO6jHNkdiLLGjJNa7pNwXaOS4uxv+qzYwRehkA== X-Migadu-Spam-Score: -0.21 X-Spam-Score: -0.21 X-Migadu-Queue-Id: 5B00F3A554 X-Migadu-Scanner: scn0.migadu.com Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gmail.com header.s=20221208 header.b=W0cBvc1z; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-TUID: u3CzjS41z4Vd --=-=-= Content-Type: text/plain Ihor Radchenko writes: > Nathaniel Nicandro writes: > >>> Ideally, fontifying ANSI sequences should be fully controlled by users: >>> 1. We may not want to touch src blocks by default, when >>> `org-src-fontify-natively' is set to t. Only, maybe, provide an >>> option. Or you may better publish a minor mode that does this for >>> shell scripts. >>> 2. We may allow all the ANSI sequences to be fontified in the whole >>> buffer. >> >> I've updated my patch to be a combination of (1) and (2), see the >> attached patch. Essentially every sequence is fontified except those in >> source blocks and a minor mode has been created to allow users to >> disable or enable fontification whenever they want. >> >> I've also attached an example Org file with some ANSI sequences in it >> for testing purposes that you can try out. > > Thanks! > >> One issue that remains is how to handle sequences within inline source >> blocks. Those don't have a src-block property so any sequences within >> an inline source block are currently fontified. > > You should not use 'src-block property at all. There are scenarios when > jit-lock defers source block fontification (in particular, when source > block spans beyond the screen) and 'src-block property is not yet > applied. > > Instead, you should query `org-element-at-point' or > `org-element-context'. The attached patch now uses `org-element-at-point' and `org-element-context' to query for the bounds of elements. Note, I've also attached an updated example file which shows that the escape sequences in inline source blocks are now handled similarly to regular source blocks, i.e. they are not fontified. > >> +*** ANSI escape sequences are now highlighted in the whole buffer >> + >> +A new customization ~org-fontify-ansi-sequences~ is available which >> +tells Org to highlight all ANSI sequences in the buffer if non-nil and >> +the new minor mode ~org-ansi-mode~ is enabled. >> + >> +To disable highlighting of the sequences you can either >> +disable ~org-ansi-mode~ or set ~org-fontify-ansi-sequences~ to ~nil~ >> +and =M-x revert-buffer RET=. Doing the latter will disable >> +highlighting of sequences in all newly opened Org buffers whereas >> +doing the former disables highlighting locally to the current buffer. > > Rather than asking to use revert-buffer, we usually suggest M-x > org-mode-restart. Done. > >> +(defun org-fontify-ansi-sequences (limit) >> + "Fontify ANSI sequences." >> + (when (and org-fontify-ansi-sequences org-ansi-mode) >> + (let (end) >> + (while (/= (point) limit) > > Instead of this strict condition and later juggle with > `narrow-to-region', just use the usual (while (< (point) limit) ...). > Done. >> + (cond >> + ((get-text-property (point) 'src-block) > > As I mentioned above, please use `org-element-at-point'. This function > will also give you information about the block boundaries. > >> + (ansi-color-apply-on-region (point) end t) > > We should probably limit ANSI colour pairs to a single Org element. It > does not make much sense to have text in-between the quotes below > coloured: > > #+begin_quote > ... ... > #+end_quote > > > .... > > #+begin_quote > ... ... > #+end_quote > Makes sense. Done. >> + ;; Reset the context before every fontification cycle. This >> + ;; avoids issues where `ansi-color-apply-on-region' attempts to >> + ;; use an old starting point that may be from a different part >> + ;; of the buffer, leading to "wrong side of point" errors. >> + (setq ansi-color-context-region nil) > > This looks fragile. AFAIU, `ansi-color-context-region' is used to track > currently active ANSI colour settings. Since your fontification function > may be called with various LIMITs, depending on what is displayed on the > user screen, the fontification results might be unpredictable for ANSI > defs spanning across multiple screens. > It seems to be safe to reset `ansi-color-context-region' now given that org-element is used to find the bounds of the element at `point'. Although the fontification limits are dependent on screen size, the org-element functions are not and so the bounds used when applying the fontification for the ANSI sequences won't depend on screen size either. Also, re-setting `ansi-color-context-region' has the effect of not propagating previously applied color settings to other Org elements. >> +(defvar org-ansi-colors >> + '(black red green yellow blue purple cyan white)) >> + >> +(defun org-ansi-highlight (beg end seq) >> + (save-excursion >> + (goto-char end) >> + (insert "\e") >> + (insert "[0m") >> + (goto-char beg) >> + (insert "\e") >> + (insert (format "[%sm" seq)))) >> + >> +(defun org-ansi-highlight-foreground (beg end color) >> + "Highlight the foreground between BEG and END with COLOR." >> + (interactive >> + (let ((bounds (car (region-bounds)))) >> + (list (car bounds) (cdr bounds) >> + (completing-read "Color: " org-ansi-colors nil t)))) >> + (let ((n (- (length org-ansi-colors) >> + (length (memq color org-ansi-colors))))) >> + (org-ansi-highlight beg end (+ 30 n)))) >> + >> +(defun org-ansi-highlight-background (beg end color) >> + "Highlight the background between BEG and END with COLOR." >> + (interactive >> + (let ((bounds (car (region-bounds)))) >> + (list (car bounds) (cdr bounds) >> + (completing-read "Color: " org-ansi-colors nil t)))) >> + (let ((n (- (length org-ansi-colors) >> + (length (memq color org-ansi-colors))))) >> + (org-ansi-highlight beg end (+ 40 n)))) > > The above has no relation to fontification and does not belong to Org in > general. Org syntax has no notion of ANSI escapes. We may support them > as a useful feature, but no more. Editing ANSI escapes would make more > sense in shell-script-mode or similar. Removed. > >> + :lighter " OANSI" >> + (cond >> + ((and org-fontify-ansi-sequences org-ansi-mode) >> + (remove-text-properties (point-min) (point-max) '(fontified t)) >> + (font-lock-ensure)) > > Just use `org-restart-font-lock'. > Thanks. Done. >> + (t >> + (dolist (ov (overlays-in (point-min) (point-max))) >> + ;; Attempt to find ANSI specific overlays. See >> + ;; `ansi-color-make-extent'. >> + (when (eq (car-safe (overlay-get ov 'insert-behind-hooks)) >> + 'ansi-color-freeze-overlay) > > This is extremely awkward and relies on internal implementation details > of ansi-color. Moreover, we must avoid overlays, if possible - they do > not scale well. I recommend re-defining `ansi-color-apply-face-function' > to something that uses text properties. Using text properties will also > make restarting font-lock sufficient to clear the fontification. I've re-defined `ansi-color-apply-face-function' as you've said. --=-=-= Content-Type: text/x-org Content-Disposition: attachment; filename=test-ansi.org Content-Transfer-Encoding: 8bit * This is a test :PROPERTIES: :CUSTOM_ID: 123 :END: Of ANSI color sequences #+begin_src python for x in y: print(x + "this is a test") #+end_src : this is a td =testing= In paragraph a ~color sequence~ is here. This is a sequence that covers a block #+begin_example shouldn't be colored #+end_example there should be an end here there is the end. begin  sequence without end #+begin_src python 1 + 1 #+end_src #+begin_quote open  #+end_quote should not be highlighted #+begin_quote close  #+end_quote This is a paragraph src_python{return "testing"} {{{results(=testing=)}}} with multiple inline src_python{return 5*4} {{{results(=20=)}}} source blocks. An inline source block src_python{return 1+ 1 without an end. src_python{return "testing"}. --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-Highlight-ANSI-escape-sequences.patch Content-Description: Patch >From c59d39d76266670200f9cfe70a1e1c2dad04c8bc Mon Sep 17 00:00:00 2001 From: Nathaniel Nicandro Date: Tue, 9 May 2023 19:58:11 -0500 Subject: [PATCH] Highlight ANSI escape sequences * etc/ORG-NEWS: Describe the new feature. * org.el (org-fontify-ansi-sequences): New customization variable and function which does the work of fontifying the sequences. (org-set-font-lock-defaults): Add the `org-fontify-ansi-sequences` function to the font-lock keywords. (org-ansi-mode): New minor mode to enable/disable highlighting of the sequences. Enabled in Org buffers by default. --- etc/ORG-NEWS | 12 ++++++++ lisp/org.el | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+) diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS index b7c88fd..2c28785 100644 --- a/etc/ORG-NEWS +++ b/etc/ORG-NEWS @@ -169,6 +169,18 @@ official [[https://clojure.org/guides/deps_and_cli][Clojure CLI tools]]. The command can be customized with ~ob-clojure-cli-command~. ** New features +*** ANSI escape sequences are now highlighted in the whole buffer + +A new customization ~org-fontify-ansi-sequences~ is available which +tells Org to highlight all ANSI sequences in the buffer if non-nil and +the new minor mode ~org-ansi-mode~ is enabled. + +To disable highlighting of the sequences you can either +disable ~org-ansi-mode~ or set ~org-fontify-ansi-sequences~ to ~nil~ +and =M-x org-mode-restart RET=. Doing the latter will disable +highlighting of sequences in all newly opened Org buffers whereas +doing the former disables highlighting locally to the current buffer. + *** Add support for ~logind~ idle time in ~org-user-idle-seconds~ When Emacs is built with =dbus= support and diff --git a/lisp/org.el b/lisp/org.el index 26d2a86..6742449 100644 --- a/lisp/org.el +++ b/lisp/org.el @@ -81,6 +81,7 @@ (eval-when-compile (require 'gnus-sum)) (require 'calendar) (require 'find-func) (require 'format-spec) +(require 'ansi-color) (condition-case nil (load (concat (file-name-directory load-file-name) @@ -3582,6 +3583,12 @@ (defcustom org-fontify-whole-block-delimiter-line t :group 'org-appearance :type 'boolean) +(defcustom org-fontify-ansi-sequences t + "Non-nil means to highlight ANSI escape sequences." + :group 'org-appearance + :type 'boolean + :package-version '(Org . "9.7")) + (defcustom org-highlight-latex-and-related nil "Non-nil means highlight LaTeX related syntax in the buffer. When non-nil, the value should be a list containing any of the @@ -5543,6 +5550,66 @@ (defun org-fontify-extend-region (beg end _old-len) (cons beg (or (funcall extend "end" "]" 1) end))) (t (cons beg end)))))) +(defvar org-ansi-mode) + +(defun org-fontify-ansi-sequences (limit) + "Fontify ANSI sequences." + (when (and org-fontify-ansi-sequences org-ansi-mode) + (while (< (point) limit) + (let ((el (org-element-at-point)) beg end next) + (pcase (org-element-type el) + (`src-block + (setq beg (org-element-property :end el) + end beg + next end)) + (`headline + (setq beg (org-element-property :begin el) + end (org-element-property :contents-begin el) + next end)) + (`paragraph + ;; Compute the regions of the paragraph excluding inline + ;; source blocks. + (setq beg nil end nil) + (let ((pbeg (org-element-property :begin el)) + (pend (org-element-property :end el))) + (goto-char pbeg) + (push pbeg beg) + (while (re-search-forward + "\\