bug#51437: 29.0.50; The annotated example of a complete working Flymake backend does not work with narrowed buffers

bug#51437: 29.0.50; The annotated example of a complete working Flymake backend does not work with narrowed buffers

[Rudolf Adamkovič via Bug reports for GNU Emacs, the Swiss army knife of text editors]

I have found (yet another) problem with the annotated example of a complete "working" Flymake backend.

When checking a narrowed buffer with errors outside of the narrowed region, I get:

┌────
│ error in process sentinel: flymake--highlight-line: Wrong type argument: integer-or-marker-p, nil
│ error in process sentinel: Wrong type argument: integer-or-marker-p, nil
└────

Looks like `flymake-diag-region' returns `(nil . nil)' and `flymake-make-diagnostic' does not like that.

TL;DR The official example does not work correctly with narrowed buffers.

Rudy


In GNU Emacs 29.0.50 (build 6, x86_64-apple-darwin20.6.0, NS appkit-2022.60 Version 11.5.1 (Build 20G80))
 of 2021-10-24 built on Workstation.local
Repository revision: aea4af5119fdf130f1df7190478a23c6777f92a2
Repository branch: master
Windowing system distributor 'Apple', version 10.3.2022
System Description:  macOS 11.5.1

Configured using:
 'configure --with-json
 PKG_CONFIG_PATH=/usr/local/Cellar/zlib/1.2.11/lib/pkgconfig:/usr/local/lib/pkgconfig:/opt/X11/lib/pkgconfig'

Configured features:
ACL DBUS GIF GLIB GMP GNUTLS JPEG JSON LCMS2 LIBXML2 MODULES NOTIFY
KQUEUE NS PDUMPER PNG RSVG THREADS TIFF TOOLKIT_SCROLL_BARS WEBP XIM
ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Message[Notmuch]

Minor modes in effect:
  flymake-mode: t
  which-key-mode: t
  corfu-global-mode: t
  corfu-mode: t
  vertico-mode: t
  marginalia-mode: t
  global-diff-hl-mode: t
  diff-hl-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  global-hl-todo-mode: t
  hl-todo-mode: t
  savehist-mode: t
  global-subword-mode: t
  subword-mode: t
  save-place-mode: t
  global-auto-revert-mode: t
  delete-selection-mode: t
  mml-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  auto-fill-function: yas--auto-fill
  global-visual-line-mode: t
  visual-line-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
/Users/salutis/.emacs.d/elpa/transient-20211025.1800/transient hides /Users/salutis/src/emacs/nextstep/Emacs.app/Contents/Resources/lisp/transient

Features:
(shadow emacsbug ox-md ox-odt rng-loc rng-uri rng-parse rng-match rng-dt
rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util ox-latex
ox-icalendar org-agenda org-refile ox-html table ox-ascii ox-publish ox
sort bbdb-message sendmail fortune mail-extr notmuch notmuch-tree
notmuch-jump notmuch-hello notmuch-show notmuch-print notmuch-crypto
notmuch-mua notmuch-message notmuch-draft notmuch-maildir-fcc
notmuch-address notmuch-company notmuch-parser notmuch-wash coolj
notmuch-query goto-addr icalendar diary-lib diary-loaddefs notmuch-tag
crm notmuch-lib notmuch-version notmuch-compat pcase hl-line vc-mtn
vc-hg vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs sh-script smie
executable flymake-proc flymake project warnings org-indent image-file
image-converter disp-table char-fold cursor-sensor paredit edmacro bbdb
bbdb-site timezone modus-vivendi-theme modus-operandi-theme modus-themes
which-key corfu orderless consult-vertico vertico marginalia consult
recentf tree-widget kmacro pdf-loader diff-hl log-view pcvs-util vc-dir
ewoc vc diminish yasnippet hl-todo finder-inf org-clock oc-csl citeproc
citeproc-itemgetters citeproc-biblatex citeproc-bibtex parsebib
citeproc-cite citeproc-subbibs citeproc-sort citeproc-name
citeproc-formatters citeproc-number rst compile citeproc-proc
citeproc-disamb citeproc-itemdata citeproc-generic-elements
citeproc-macro citeproc-choose citeproc-date citeproc-context
citeproc-prange citeproc-style citeproc-locale citeproc-term f
citeproc-rt citeproc-lib citeproc-s s let-alist queue dash savehist
ls-lisp cap-words superword subword saveplace autorevert filenotify
delsel elfeed-link elfeed-show elfeed-search elfeed-csv elfeed
elfeed-curl elfeed-log xml-query bookmark pp elfeed-db elfeed-lib vc-git
diff-mode vc-dispatcher org-element avl-tree generator ol-eww eww xdg
url-queue thingatpt mm-url ol-rmail ol-mhe ol-irc ol-info ol-gnus
nnselect gnus-search eieio-opt speedbar ezimage dframe gnus-art mm-uu
mml2015 mm-view mml-smime smime dig gnus-sum shr kinsoku svg dom
gnus-group gnus-undo gnus-start gnus-dbus dbus xml gnus-cloud nnimap
nnmail mail-source utf7 netrc nnoo parse-time gnus-spec gnus-int
gnus-range message rmc puny rfc822 mml mml-sec epa derived epg rfc6068
epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader gnus-win gnus nnheader gnus-util rmail
rmail-loaddefs rfc2047 rfc2045 ietf-drums text-property-search
mail-utils mm-util mail-prsvr wid-edit ol-docview doc-view jka-compr
image-mode exif dired dired-loaddefs ol-bibtex ol-bbdb ol-w3m ol-doi
org-link-doi cl-extra help-mode org ob ob-tangle ob-ref ob-lob ob-table
ob-exp org-macro org-footnote org-src ob-comint org-pcomplete pcomplete
comint ansi-color ring org-list org-faces org-entities noutline outline
easy-mmode org-version ob-emacs-lisp ob-core ob-eval org-table oc-basic
bibtex iso8601 time-date ol rx org-keys oc org-compat advice org-macs
org-loaddefs format-spec find-func cal-menu calendar cal-loaddefs
tex-site slime-autoloads info package browse-url url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-domsuf
url-util mailcap url-handlers url-parse auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs password-cache json map url-vars seq
gv subr-x byte-opt bytecomp byte-compile cconv cl-loaddefs cl-lib
iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/ns-win ns-win ucs-normalize
mule-util term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button
loaddefs faces cus-face macroexp files window text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads dbusbind kqueue cocoa ns
lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 743594 65968)
 (symbols 48 42843 9)
 (strings 32 222286 7052)
 (string-bytes 1 6344437)
 (vectors 16 82119)
 (vector-slots 8 1722047 63625)
 (floats 8 399 286)
 (intervals 56 15568 1600)
 (buffers 992 26))

-- 
"Logic is a science of the necessary laws of thought, without which no employment of the understanding and the reason takes place." -- Immanuel Kant, 1785

Rudolf Adamkovič <salutis@me.com>
Studenohorská 25
84103 Bratislava
Slovakia

[he/him]

bug#51437: 29.0.50; The annotated example of a complete working Flymake backend does not work with narrowed buffers

[Lars Ingebrigtsen]

Rudolf Adamkovič <salutis@me.com> writes:

> When checking a narrowed buffer with errors outside of the narrowed region, I get:
>
> ┌────
> │ error in process sentinel: flymake--highlight-line: Wrong type argument: integer-or-marker-p, nil
> │ error in process sentinel: Wrong type argument: integer-or-marker-p, nil
> └────
>
> Looks like `flymake-diag-region' returns `(nil . nil)' and `flymake-make-diagnostic' does not like that.

Perhaps this is the correct fix?  (João added to the CCs.)

diff --git a/doc/misc/flymake.texi b/doc/misc/flymake.texi
index 309bed7760..1d99342060 100644
--- a/doc/misc/flymake.texi
+++ b/doc/misc/flymake.texi
@@ -798,6 +798,7 @@ An annotated example backend
                        for type = (if (string-match "^warning" msg)
                                       :warning
                                     :error)
+                       when (and beg end)
                        collect (flymake-make-diagnostic source
                                                         beg
                                                         end


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#51437: 29.0.50; The annotated example of a complete working Flymake backend does not work with narrowed buffers

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1494 bytes --]

Might be. I can't test right now, but maybe Rudolf can. If it works then
I'd say commit it.

João

On Fri, Oct 29, 2021, 14:35 Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Rudolf Adamkovič <salutis@me.com> writes:
>
> > When checking a narrowed buffer with errors outside of the narrowed
> region, I get:
> >
> > ┌────
> > │ error in process sentinel: flymake--highlight-line: Wrong type
> argument: integer-or-marker-p, nil
> > │ error in process sentinel: Wrong type argument: integer-or-marker-p,
> nil
> > └────
> >
> > Looks like `flymake-diag-region' returns `(nil . nil)' and
> `flymake-make-diagnostic' does not like that.
>
> Perhaps this is the correct fix?  (João added to the CCs.)
>
> diff --git a/doc/misc/flymake.texi b/doc/misc/flymake.texi
> index 309bed7760..1d99342060 100644
> --- a/doc/misc/flymake.texi
> +++ b/doc/misc/flymake.texi
> @@ -798,6 +798,7 @@ An annotated example backend
>                         for type = (if (string-match "^warning" msg)
>                                        :warning
>                                      :error)
> +                       when (and beg end)
>                         collect (flymake-make-diagnostic source
>                                                          beg
>                                                          end
>
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no
>

[-- Attachment #2: Type: text/html, Size: 2213 bytes --]

bug#51437: 29.0.50; The annotated example of a complete working Flymake backend does not work with narrowed buffers

[Rudolf Adamkovič via Bug reports for GNU Emacs, the Swiss army knife of text editors]

João Távora <joaotavora@gmail.com> writes:

> I can't test right now, but maybe Rudolf can. If it works then
> I'd say commit it.

The patch fixes the errors, but it also causes troubles. Say I use Flymake to check on my English grammar, and say I have 10 errors in my text. I narrow down to a part with 5 errors, fix them, write some new text, and after a while, I widen the buffer. I look through my final text, see that Flymake reports 0 errors, and submit my work to someone. Well, I submitted 5 errors, because Flymake did not re-check when I widened the buffer. Further, Flymake might report incorrect data on narrow. In the example above, it would have said 10 errors after narrowing and then 5 for the same narrowed content on the next re-check. Thus, we should either always check the entire content of the buffer, or we should recheck on narrow and widen, potentially wasting bandwidth for online checks and the like.

Rudy

-- 
"'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and if it were so, it would be; but as it isn't, it ain't. That's logic.'" -- Lewis Carroll, Through the Looking Glass

Rudolf Adamkovič <salutis@me.com>
Studenohorská 25
84103 Bratislava
Slovakia

[he/him]

bug#51437: 29.0.50; The annotated example of a complete working Flymake backend does not work with narrowed buffers

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1826 bytes --]

This is starting to sound like a more generic Flymake limitation/bug than a
documentation bug. I'll have a look when I get home.

Have you experimented with the 'region' arg when reporting back
diagnostics?  Also you can have a look at how the Elisp backend does it?  I
think, though I am not sure, that it works with narrowed buffers.

By the way, Flymake for spell checking is a great idea. Is your backend
available somewhere?

João

On Sat, Oct 30, 2021, 22:15 Rudolf Adamkovič <salutis@me.com> wrote:

> João Távora <joaotavora@gmail.com> writes:
>
> > I can't test right now, but maybe Rudolf can. If it works then
> > I'd say commit it.
>
> The patch fixes the errors, but it also causes troubles. Say I use Flymake
> to check on my English grammar, and say I have 10 errors in my text. I
> narrow down to a part with 5 errors, fix them, write some new text, and
> after a while, I widen the buffer. I look through my final text, see that
> Flymake reports 0 errors, and submit my work to someone. Well, I submitted
> 5 errors, because Flymake did not re-check when I widened the buffer.
> Further, Flymake might report incorrect data on narrow. In the example
> above, it would have said 10 errors after narrowing and then 5 for the same
> narrowed content on the next re-check. Thus, we should either always check
> the entire content of the buffer, or we should recheck on narrow and widen,
> potentially wasting bandwidth for online checks and the like.
>
> Rudy
>
> --
> "'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and if
> it were so, it would be; but as it isn't, it ain't. That's logic.'" --
> Lewis Carroll, Through the Looking Glass
>
> Rudolf Adamkovič <salutis@me.com>
> Studenohorská 25
> 84103 Bratislava
> Slovakia
>
> [he/him]
>

[-- Attachment #2: Type: text/html, Size: 2433 bytes --]

bug#51437: 29.0.50; The annotated example of a complete working Flymake backend does not work with narrowed buffers

[Rudolf Adamkovič via Bug reports for GNU Emacs, the Swiss army knife of text editors]

João Távora <joaotavora@gmail.com> writes:

> This is starting to sound like a more generic Flymake limitation/bug
> than a documentation bug. I'll have a look when I get home.

True.

> Have you experimented with the 'region' arg when reporting back
> diagnostics? Also you can have a look at how the Elisp backend does
> it? I think, though I am not sure, that it works with narrowed
> buffers.

I did not dig that deep yet. :)

> By the way, Flymake for spell checking is a great idea. Is your
> backend available somewhere?

I use Vale to check on my writing (style and spelling) as follows:

┌────
│ (with-eval-after-load 'flymake
│   (my-define-flymake-backend
│    my-flymake-vale
│    (lambda (file) (list "vale" "--output" "line" "--ext" ".md" "--no-wrap"))
│    "^stdin.md:\\([0-9]+\\):\\([0-9]+\\):\\(.*\\)$"
│    (lambda (msg) t)))
└────

I dislike the idea of unnecessary "backend packages", and I think Flymake should empower the user to plug in any CLI tool in a couple of minutes. No packages, no complicated functions with `make-process', and so on. Instead, make simple asynchronous live buffer checking simple. Then, everyone will plug in their linters, unit test runners, style/spell-checkers, and the like. As of now, Flymake fights such a user. For instance, I use the same macro for everything else, including programming:

┌────
│ (with-eval-after-load 'flymake
│     (my-define-flymake-backend
│      my-flymake-swiftlint
│      'my-flymake-swiftlint-command
│      "^<nopath>:\\([0-9]+\\):\\([0-9]+\\): \\(.*\\)$"
│      (lambda (msg) (string-match "^warning" msg))))
└────

TL;DR Vanilla Flymake makes it unnecesserily hard to plug in a simple CLI utility for buffer checking.

Rudy
-- 
"I love deadlines. I love the whooshing noise they make as they go by." -- Douglas Adams, The Salmon of Doubt

Rudolf Adamkovič <salutis@me.com>
Studenohorská 25
84103 Bratislava
Slovakia

[he/him]

bug#51437: 29.0.50; The annotated example of a complete working Flymake backend does not work with narrowed buffers

[Lars Ingebrigtsen]

Rudolf Adamkovič <salutis@me.com> writes:

>> I can't test right now, but maybe Rudolf can. If it works then
>> I'd say commit it.
>
> The patch fixes the errors,

So I've applied the patch to Emacs 29...

> but it also causes troubles. Say I use
> Flymake to check on my English grammar, [...]

But leaving the bug report open to handle the problem more generally.