bug#71698: 29.3; comint--intersect-regions fontifies some output using input function

bug#71698: 29.3; comint--intersect-regions fontifies some output using input function

[JD Smith]

This relates to bug#51940: 29.0.50; [PATCH] Fontification and indentation in M-x shell.

I found a tiny bug in the introduced command `comint--intersect-regions'.  It pertains to the "field-expansion" in the following section, which is responsible for narrowing to alternating regions of input and output:

(let ((beg2 beg1)
      (end2 end1))
  (when (= beg2 beg)
    (setq beg2 (field-beginning beg2))) ; <--- bug here
  (when (= end2 end)
    (setq end2 (field-end end2)))
  ;; Narrow to the whole field surrounding the region
  (narrow-to-region beg2 end2))

The nature of the bug is that `field-beginning' is left-associative, whereas `(get-text-property (point) 'field)` is right associative.  You can demonstrate this to yourself as follows:

(progn
  (insert (concat "\n" (propertize "AAA" 'field 'a) (propertize "B" 'field 'b) (propertize "CCC" 'field 'c)))
  (goto-char (- (point) 4))             ; cursor on 'B'
  (list (get-text-property (point) 'field) (field-beginning) (field-end)
        (buffer-substring-no-properties (field-beginning) (field-end))))
;; gives: (b 356 359 "AAA")

So on the line:

              (setq beg2 (field-beginning beg2))) ; <--- bug here

if beg is at the first character of its field, `field-beginning` will actually return the start of the prior field.  This leads to `comint--intersect-regions' unwittingly including `output' field text in its input fontification narrowing.  I noticed this in a new python interactive mode I'm working on: the prompt itself (i.e. output) was being partially fontified by python-mode.

I believe the correct fix is:

              (setq beg2 (field-beginning (1+ beg2))))

This takes care of the problem here, even for single-character fields (like `boundary').  We know (< beg1 end) from the loop condition, so it's safe to increment beg2.


--- /var/folders/by/19mt78cj63v5_tslqh826g7w0000gn/T/jka-comrWKsIJ	2024-06-21 13:08:59
+++ /var/folders/by/19mt78cj63v5_tslqh826g7w0000gn/T/buffer-content-Q8dmID	2024-06-21 13:08:59
@@ -4122,7 +4122,7 @@
           (let ((beg2 beg1)
                 (end2 end1))
             (when (= beg2 beg)
-              (setq beg2 (field-beginning beg2)))
+              (setq beg2 (field-beginning (1+ beg2))))
             (when (= end2 end)
               (setq end2 (field-end end2)))
             ;; Narrow to the whole field surrounding the region

bug#71698: 29.3; comint--intersect-regions fontifies some output using input function

[JD Smith]

A bit more context, to quote from the manual:

> When the characters before and after POS are part of the same field,
> there is no doubt which field contains POS: the one those characters
> both belong to. When POS is at a boundary between fields, which field
> it belongs to depends on the stickiness of the ‘field’ properties of the
> two surrounding characters (*note Sticky Properties::).  The field whose
> property would be inherited by text inserted at POS is the field that
> contains POS.

Note that comint adds `field' as `rear-nonsticky' on the prompt, but this does not impact the inheritance of the field from the previous character or the behavior of field-beginning (only a front-sticky on the following character would do so).

bug#71698: 29.3; comint--intersect-regions fontifies some output using input function

[miha--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 715 bytes --]

JD Smith <jdtsmith@gmail.com> writes:

> --- /var/folders/by/19mt78cj63v5_tslqh826g7w0000gn/T/jka-comrWKsIJ	2024-06-21 13:08:59
> +++ /var/folders/by/19mt78cj63v5_tslqh826g7w0000gn/T/buffer-content-Q8dmID	2024-06-21 13:08:59
> @@ -4122,7 +4122,7 @@
>            (let ((beg2 beg1)
>                  (end2 end1))
>              (when (= beg2 beg)
> -              (setq beg2 (field-beginning beg2)))
> +              (setq beg2 (field-beginning (1+ beg2))))
>              (when (= end2 end)
>                (setq end2 (field-end end2)))
>              ;; Narrow to the whole field surrounding the region

Thanks. Could you please check if this is related and possibly fixed in
bug#59626 commit 949bc1c72d7.

Best.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]

bug#71698: 29.3; comint--intersect-regions fontifies some output using input function

[JD Smith]

Thanks, I took a look.  commit 949bc1c72d7 I believe solves the same problem, and also fixes my issue.  In fact I think the solutions are equivalent, since the end test will never need to trigger, because for field-end the left associativity works in our favor.  But yours is more explicit.  

Feel free to close.

> On Jun 21, 2024, at 6:30 PM, <miha@kamnitnik.top> <miha@kamnitnik.top> wrote:
> 
> JD Smith <jdtsmith@gmail.com> writes:
> 
> Thanks. Could you please check if this is related and possibly fixed in
> bug#59626 commit 949bc1c72d7.
> 
> Best.

bug#71698: 29.3; comint--intersect-regions fontifies some output using input function

[miha--- via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 656 bytes --]

JD Smith <jdtsmith@gmail.com> writes:

> Thanks, I took a look.  commit 949bc1c72d7 I believe solves the same problem, and also fixes my issue.  In fact I think the solutions are equivalent, since the end test will never need to trigger, because for field-end the left associativity works in our favor.  But yours is more explicit.  
>
> Feel free to close.

Thanks. I'm therefore closing this bug report.

On an unrelated note, my reply to your personal e-mail

    Subject: comint-fontify-input-mode
    Date: Wed, 22 May 2024 14:50:00 -0400

might have ended up in your spam folder, so I kindly ask you to check it
if you are interested.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 861 bytes --]