From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: JD Smith Newsgroups: gmane.emacs.devel Subject: Re: Selective font-locking? Date: Mon, 12 Apr 2021 21:51:28 -0400 Message-ID: <0A6735A5-ABC0-48B0-B7C0-EACF5D85A32B@gmail.com> References: <7A948673-E156-42BA-BA50-E91986908BB5@gmail.com> <65B869A0-CB0B-43C1-90EA-E2B259A74589@gmail.com> Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Content-Type: multipart/alternative; boundary="Apple-Mail=_EB4D4E70-EAA3-4324-A171-3D2CF86CC974" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="19157"; mail-complaints-to="usenet@ciao.gmane.io" Cc: emacs-devel@gnu.org To: Stefan Monnier Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Apr 13 03:52:39 2021 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lW8EI-0004ql-FE for ged-emacs-devel@m.gmane-mx.org; Tue, 13 Apr 2021 03:52:38 +0200 Original-Received: from localhost ([::1]:57576 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lW8EH-0004mt-D0 for ged-emacs-devel@m.gmane-mx.org; Mon, 12 Apr 2021 21:52:37 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:59786) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lW8DH-0004Mf-GB for emacs-devel@gnu.org; Mon, 12 Apr 2021 21:51:35 -0400 Original-Received: from mail-il1-x135.google.com ([2607:f8b0:4864:20::135]:45862) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lW8DE-0000b6-67 for emacs-devel@gnu.org; Mon, 12 Apr 2021 21:51:35 -0400 Original-Received: by mail-il1-x135.google.com with SMTP id e14so1470864ils.12 for ; Mon, 12 Apr 2021 18:51:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=mqocp5RG1tdy4Qdt9of+16fp/yLbwXeMqnc2rFt402Y=; b=Kb8Ec/jlgrUoOFlYFINsTZEgF2wF+WaPjNeLzXSFeOsCv+RDzo4ri//snNMZqvaXKq Jr7ejcAIHso1lEA1+TuFw7DpyW7I+by+RTCS2YmhzMlViiemXPy/XaT8qIG9mI0GvaXY B+CFAd/G0NmVn6PkuuPkIcJF/nXK2K2ey7lbcoMVJB3pbfoX3tsjayjIwLYL725ZbT2C sdwq5Mjco6uq88yOrr7Tvi3vWzOV8WENQxJv2ndHc4XDTpcm+joVtg6zbgp9BFPYzDyt LLE+pYARkl3yWl/XE+Yf11jsB18b2rcLVD++hsJCmHWYrVIEeKy1ZLoW8r2dnjE5cN7Y zqGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=mqocp5RG1tdy4Qdt9of+16fp/yLbwXeMqnc2rFt402Y=; b=DX35b3o0LYteIAL31hXoMQJJDju6ZxmKTYbGiOtYD9sxv2AeGXSzy0arLaz7+cmvuU giFEH8KA0tdFRHIkt2TqBAEjbNT9g9Zm8KQwyM/LCldvAh92kyyy8Tdi5A1ZR5PEIH39 arvQIZyen2WXshe7k+MF9QNdab7iIL/6+97lbktKTuHBcRDhcKhGoc9bNKNG3x4InPoz XPFJ309DaoN0ukDDMWxN9D8E69ZG8Ik/IelJjHrXzUiGnCyf0E75UdCzieI+kDpgnRj1 TcGszwsgL3Iv+O/CHzA0goYnrfUs9NpoWOrVXABNP87pPs5KOEjacThYZrEzexvExmJ5 JK5Q== X-Gm-Message-State: AOAM531fQoeg5Er4/lgtMdaLEeHONegJ1aPH3opEKPywbw/VkiWtQ03+ Dp3u8Wvf2o9+Iyg7FWbbr+o= X-Google-Smtp-Source: ABdhPJxnQrBjr/1amWt+Tmxyi+rMJSkDKor3VtS3j8T+oI5Mn4EsZuaDU3AFJjtppSXR/xAWQXffJQ== X-Received: by 2002:a05:6e02:b2f:: with SMTP id e15mr11961967ilu.255.1618278690382; Mon, 12 Apr 2021 18:51:30 -0700 (PDT) Original-Received: from [192.168.0.118] (cm-134-228-54-223.buckeyecom.net. [134.228.54.223]) by smtp.gmail.com with ESMTPSA id h11sm2001295ilr.84.2021.04.12.18.51.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 12 Apr 2021 18:51:29 -0700 (PDT) In-Reply-To: X-Mailer: Apple Mail (2.3608.120.23.2.4) Received-SPF: pass client-ip=2607:f8b0:4864:20::135; envelope-from=jdtsmith@gmail.com; helo=mail-il1-x135.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:267983 Archived-At: --Apple-Mail=_EB4D4E70-EAA3-4324-A171-3D2CF86CC974 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Thank you! This method works surprisingly well to fontify input at the = end of the buffer with another major-mode=E2=80=99s keywords/syntax. It = seems to be fairly efficient even for long multi-line input text: (defun python-shell-multiline--apply-font-lock (limit) (if-let ((process (get-buffer-process (current-buffer))) (pmark (process-mark))) (if (> limit pmark) (let ((font-lock-keywords = python-shell-multiline-font-lock-keywords) (font-lock-syntactic-face-function #'python-font-lock-syntactic-face-function) (start (max pmark (point)))) (with-syntax-table python-mode-syntax-table (font-lock-flush start limit) (font-lock-ensure start limit)))))) (setq-local font-lock-keywords = '(python-shell-multiline--apply-font-lock) font-lock-keywords-only nil syntax-propertize-function = python-syntax-propertize-function) (setq python-shell-multiline-font-lock-keywords (symbol-value (font-lock-choose-keywords python-font-lock-keywords (font-lock-value-in-major-mode font-lock-maximum-decoration)))) The one curious thing: it definitely requires calls to both = font-lock-flush and font-lock-ensure. Otherwise the input isn=E2=80=99t = fontified. Tracing through, it seems when 'font-lock-mode is enabled, = 'font-lock-flush is set to `jit-lock-refontify`, which simply clears the = =E2=80=98fontified property, presumably expecting it to be noticed and = re-fontified. Not sure if this is the correct/most efficient approach. It seems to me this general technique could be useful for lots of = different =E2=80=9Cmixed fontification buffers=E2=80=9D, simply = rebinding font-lock-keywords/syntax table/etc. as needed for the region = under consideration. It=E2=80=99s much simpler than the round-trip copy = + full buffer re-fontify that many modes use, with no extra buffers to = manage, post-command-hooks, etc. There may be race conditions for = highly dynamic buffers, but it=E2=80=99s working well for this usage = case.=20 Thanks again. > On Apr 11, 2021, at 5:10 PM, Stefan Monnier = wrote: >=20 >> Python-font-lock-keywords is a variable defined in =E2=80=98python.el=E2= =80=99. >> Its value is >> (python-font-lock-keywords-level-1 python-font-lock-keywords-level-1 = python-font-lock-keywords-level-2 = python-font-lock-keywords-maximum-decoration) >>=20 >> Not sure why jit-lock-function would be evaluating it like an sexp. >=20 > `python-font-lock-keywords` does not hold a valid value for use on > `font-lock-keywords` but a value to be used as the first element of > `font-lock-defaults`. This "first element" is used to initialize > `font-lock-keywords` but it depends on things like the > `font-lock-maximum-decoration`. >=20 >=20 > Stefan >=20 >=20 >>> On Apr 11, 2021, at 12:31 PM, Stefan Monnier = wrote: >>>=20 >>>> But then, why bother round-tripping text out to a special-use = buffer anyway, >>>> vs. just letting font-lock operate in-situ in the shell buffer = itself using >>>> python-mode=E2=80=99s fairly simple font-lock-defaults. The only = thing needed to >>>> make this work is asking font-lock to ignore all the text with = =E2=80=98field of >>>> =E2=80=98output? =20 >>>=20 >>> Maybe you can try something like the following? >>>=20 >>> (defvar python--font-lock-keywords ...) >>> (defvar python-font-lock-keywords >>> '(python--apply-font-lock)) >>> (defun python--apply-font-lock (limit) >>> (while (< (point) limit) >>> (let ((next-boundary (find-next-boundary limit))) >>> (if (we-should-skip-this-block) >>> (goto-char next-boundary) >>> (let ((font-lock-keywords python--font-lock-keywords)) >>> (font-lock-ensure (point) limit)))))) >>>=20 >>>=20 >>> -- Stefan >>>=20 >=20 --Apple-Mail=_EB4D4E70-EAA3-4324-A171-3D2CF86CC974 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Thank= you!  This method works surprisingly well to = fontify input at the end of the buffer with another major-mode=E2=80=99s = keywords/syntax.  It seems to be fairly efficient even for long = multi-line input text:

(defun = python-shell-multiline--apply-font-lock (limit)
  (if-let ((process = (get-buffer-process (current-buffer)))
  (pmark = (process-mark)))
  =     (if (> limit pmark)
 (let ((font-lock-keywords = python-shell-multiline-font-lock-keywords)
= (font-lock-syntactic-face-function
= #'python-font-lock-syntactic-face-function)
(start (max pmark = (point))))
  =  (with-syntax-table = python-mode-syntax-table
=      (font-lock-flush start = limit)
  =    (font-lock-ensure start = limit))))))

(setq-local font-lock-keywords = '(python-shell-multiline--apply-font-lock)
  =  font-lock-keywords-only nil
=    syntax-propertize-function = python-syntax-propertize-function)
(setq = python-shell-multiline-font-lock-keywords
      = (symbol-value
  =      (font-lock-choose-keywords = python-font-lock-keywords
= =  (font-lock-value-in-major-mode
  = font-lock-maximum-decoration))))

The one curious thing: = it definitely requires calls to both = font-lock-flush and font-lock-ensure.  Otherwise the input isn=E2=80=99= t fontified.  Tracing through, it seems when 'font-lock-mode is = enabled, 'font-lock-flush is set to `jit-lock-refontify`, which simply = clears the =E2=80=98fontified property, presumably expecting it to be = noticed and re-fontified.  Not sure if this is the correct/most = efficient approach.

It seems to me this general technique = could be useful for lots of different =E2=80=9Cmixed fontification = buffers=E2=80=9D, simply rebinding font-lock-keywords/syntax table/etc. = as needed for the region under consideration. It=E2=80=99s much simpler = than the round-trip copy + full buffer re-fontify that many modes use, = with no extra buffers to manage, post-command-hooks, etc.  There = may be race conditions for highly dynamic buffers, but it=E2=80=99s = working well for this usage case. 

Thanks = again.

On Apr 11, 2021, at 5:10 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

Python-font-lock-keywords = is a variable defined in =E2=80=98python.el=E2=80=99.
Its = value is
(python-font-lock-keywords-level-1 = python-font-lock-keywords-level-1 python-font-lock-keywords-level-2 = python-font-lock-keywords-maximum-decoration)

Not sure why jit-lock-function would be evaluating it like an = sexp.

`python-font-lock-keywords` does not hold a valid value for = use on
`font-lock-keywords` but a value to be used as the = first element of
`font-lock-defaults`.  This "first = element" is used to initialize
`font-lock-keywords` but it = depends on things like the
`font-lock-maximum-decoration`.


=        Stefan


On Apr 11, 2021, at 12:31 PM, Stefan Monnier = <monnier@iro.umontreal.ca> wrote:

But then, why bother = round-tripping text out to a special-use buffer anyway,
vs. = just letting font-lock operate in-situ in the shell buffer itself = using
python-mode=E2=80=99s fairly simple = font-lock-defaults. The only thing needed to
make this = work is asking font-lock to ignore all the text with =E2=80=98field = of
=E2=80=98output?  

Maybe you can try something like the following?

  (defvar = python--font-lock-keywords ...)
  (defvar = python-font-lock-keywords
=     '(python--apply-font-lock))
=   (defun python--apply-font-lock (limit)
=     (while (< (point) limit)
=       (let ((next-boundary = (find-next-boundary limit)))
=         (if = (we-should-skip-this-block)
=             (g= oto-char next-boundary)
=           (let = ((font-lock-keywords python--font-lock-keywords))
=             (f= ont-lock-ensure (point) limit))))))


-- Stefan



= --Apple-Mail=_EB4D4E70-EAA3-4324-A171-3D2CF86CC974--