bug#74890: 31.0.50; (thing-at-point 'string) raises error

bug#74890: 31.0.50; (thing-at-point 'string) raises error

[Jean Louis]

I can see that I cannot run (thing-at-point 'string) safely, though I
cannot exactly determine the condition.

In the buffer I have only this:

Hello

which is string "Hello" and when I place cursor behind "o" and run
(thing-at-point 'string) I am getting the backtrace below. But if I make
one space like "Hello " and place cursor on that empty space in the
buffer, I am getting NIL and no error.  Though I cannot repeat this with
emacs -Q and thus I do not know why is this happening exactly.

I think that  (eq (char-syntax (char-after)) 34) cannot read the char
which is not there "after".

Backtrace:

Debugger entered--Lisp error: (wrong-type-argument characterp nil)
  char-syntax(nil)
  (eq (char-syntax (char-after)) 34)
  (not (eq (char-syntax (char-after)) 34))
  (if (not (eq (char-syntax (char-after)) 34)) (setq syntax (syntax-ppss)) (or (progn (forward-char) (setq syntax (syntax-ppss)) (nth 3 syntax)) (progn (forward-char (- (or nil 1))) (setq syntax (syntax-ppss)) (nth 3 syntax))))
  (let (syntax beg end) (if (not (eq (char-syntax (char-after)) 34)) (setq syntax (syntax-ppss)) (or (progn (forward-char) (setq syntax (syntax-ppss)) (nth 3 syntax)) (progn (forward-char (- (or nil 1))) (setq syntax (syntax-ppss)) (nth 3 syntax)))) (and (nth 3 syntax) (condition-case nil (progn (setq beg (nth 8 syntax)) (setq end (progn (goto-char (nth 8 syntax)) (forward-sexp) (point)))) (error nil)) (cons beg end)))
  (save-excursion (let (syntax beg end) (if (not (eq (char-syntax (char-after)) 34)) (setq syntax (syntax-ppss)) (or (progn (forward-char) (setq syntax (syntax-ppss)) (nth 3 syntax)) (progn (forward-char (- (or nil 1))) (setq syntax (syntax-ppss)) (nth 3 syntax)))) (and (nth 3 syntax) (condition-case nil (progn (setq beg (nth 8 syntax)) (setq end (progn (goto-char ...) (forward-sexp) (point)))) (error nil)) (cons beg end))))
  tap-bounds-of-string-at-point()
  (let ((bounds (tap-bounds-of-string-at-point))) (and bounds (buffer-substring (car bounds) (cdr bounds))))
  tap-string-at-point()
  funcall(tap-string-at-point)
  (prog1 (funcall thing-fn) (constrain-to-field nil opoint))
  (let* ((opoint (point)) (thg (prog1 (funcall thing-fn) (constrain-to-field nil opoint)))) thg)
  (if thing-fn (let* ((opoint (point)) (thg (prog1 (funcall thing-fn) (constrain-to-field nil opoint)))) thg) (let ((bounds (tap-bounds-of-thing-at-point thing syntax-table))) (and bounds (buffer-substring (car bounds) (cdr bounds)))))
  (let* ((thing-fn (or (get thing 'tap-thing-at-point) (get thing 'thing-at-point))) (something (if thing-fn (let* ((opoint (point)) (thg (prog1 ... ...))) thg) (let ((bounds (tap-bounds-of-thing-at-point thing syntax-table))) (and bounds (buffer-substring (car bounds) (cdr bounds))))))) (if (and (stringp something) no-properties) (progn (set-text-properties 0 (length something) nil something))) something)
  thing-at-point(string)
  eval((thing-at-point 'string) t)
  #f(compiled-function () #<bytecode 0x15ba1ac9559f7d5d>)()
  #f(compiled-function () #<bytecode -0x5db3e1955cb81d1>)()
  eval-expression((thing-at-point 'string) nil nil 127)
  funcall-interactively(eval-expression (thing-at-point 'string) nil nil 127)
  command-execute(eval-expression)



In GNU Emacs 31.0.50 (build 4, x86_64-pc-linux-gnu, GTK+ Version
 3.24.38, cairo version 1.16.0) of 2024-12-05 built on lco2
Repository revision: 25b4bf7fcd75564f23b2e60e29e8ff7354186371
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101007
System Description: Debian GNU/Linux 12 (bookworm)

Configured using:
 'configure --with-mailutils --with-native-compilation=yes
 --with-tree-sitter --with-imagemagick'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GSETTINGS HARFBUZZ
IMAGEMAGICK JPEG LCMS2 LIBSELINUX LIBXML2 MODULES NATIVE_COMP NOTIFY
INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP X11 XDBE XIM XINPUT2 XPM GTK3 ZLIB

Important settings:
  value of $LC_ALL: en_US.UTF-8
  value of $LANG: de_DE.UTF-8
  value of $XMODIFIERS: @im=exwm-xim
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel
term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd fontset
image regexp-opt fringe tabulated-list replace newcomment text-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo gtk
x-toolkit xinput2 x multi-tty move-toolbar make-network-process
native-compile emacs)

Memory information:
((conses 16 49894 11144) (symbols 48 5434 0) (strings 32 14122 1285)
 (string-bytes 1 414224) (vectors 16 9179)
 (vector-slots 8 129869 10829) (floats 8 22 2) (intervals 56 247 8)
 (buffers 992 11))

-- 
Jean

bug#74890: Acknowledgement (31.0.50; (thing-at-point 'string) raises error)

[Jean Louis]

Now I realized that this bug is related to Drew's thingatpt+ as when I turned it off, the bug did not appear again.

Drew, do you maybe know how to improve this?

Jean Louis

bug#74890: 31.0.50; (thing-at-point 'string) raises error

[Eli Zaretskii]

> From: Jean Louis <bugs@gnu.support>
> Date: Sun, 15 Dec 2024 11:27:24 +0300
> 
> 
> I can see that I cannot run (thing-at-point 'string) safely, though I
> cannot exactly determine the condition.
> 
> In the buffer I have only this:
> 
> Hello
> 
> which is string "Hello" and when I place cursor behind "o" and run
> (thing-at-point 'string) I am getting the backtrace below. But if I make
> one space like "Hello " and place cursor on that empty space in the
> buffer, I am getting NIL and no error.  Though I cannot repeat this with
> emacs -Q and thus I do not know why is this happening exactly.
> 
> I think that  (eq (char-syntax (char-after)) 34) cannot read the char
> which is not there "after".
> 
> Backtrace:
> 
> Debugger entered--Lisp error: (wrong-type-argument characterp nil)
>   char-syntax(nil)
>   (eq (char-syntax (char-after)) 34)

Please show a complete recipe, preferably starting from "emacs -Q".  I
tried to reproduce this problem, but couldn't, which probably means
some special steps are required to see it.

I also don't understand your claims about "char after", because
there's always something "after" point.

bug#74890: Acknowledgement (31.0.50; (thing-at-point 'string) raises error)

[Stefan Kangas]

Jean Louis <bugs@gnu.support> writes:

> Now I realized that this bug is related to Drew's thingatpt+ as when I
> turned it off, the bug did not appear again.

Thanks, since this bug is not about Emacs, I'm closing it now.

bug#74890: 31.0.50; (thing-at-point 'string) raises error

[Jean Louis]

* Eli Zaretskii <eliz@gnu.org> [2024-12-15 21:32]:
> > Debugger entered--Lisp error: (wrong-type-argument characterp nil)
> >   char-syntax(nil)
> >   (eq (char-syntax (char-after)) 34)
> 
> Please show a complete recipe, preferably starting from "emacs -Q".  I
> tried to reproduce this problem, but couldn't, which probably means
> some special steps are required to see it.
> 
> I also don't understand your claims about "char after", because
> there's always something "after" point.

When I removed loading of Drew's thingatpt+ I could not observe this problem anymore.

I just think it is related to function from that library `tap-bounds-of-string-at-point'

Jean

bug#74890: 31.0.50; (thing-at-point 'string) raises error

[Drew Adams via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> > > Debugger entered--Lisp error: (wrong-type-argument characterp nil)
> > >   char-syntax(nil)
> > >   (eq (char-syntax (char-after)) 34)
> >
> > Please show a complete recipe, preferably starting from "emacs -Q".  I
> > tried to reproduce this problem, but couldn't, which probably means
> > some special steps are required to see it.
> >
> > I also don't understand your claims about "char after", because
> > there's always something "after" point.

No, there's _not_ always something (some text) after
point.  And that, I think, is what this report is about.
From Jean's description, the only text in the buffer is
`Hello', where that `o' char is the last char, and
point is _after_, not on/at, that `o'.

> When I removed loading of Drew's thingatpt+ I could not observe this
> problem anymore.
> 
> I just think it is related to function from that library `tap-bounds-of-
> string-at-point'

Hi Jean,

I see the same thing with `emacs -Q' for Emacs 29.4,
which is the latest Emacs release AFAIK:

Debugger entered--Lisp error: (wrong-type-argument characterp nil)
  thing-at-point-bounds-of-string-at-point()
  bounds-of-thing-at-point(string)
  thing-at-point(string)
  eval-expression((thing-at-point 'string) nil nil 127)
  funcall-interactively(eval-expression (thing-at-point 'string) nil nil 127)
  command-execute(eval-expression)

To me, this behavior makes sense since, per your recipe,
there's _no character_ at point, and for thing-at-point
to tell whether or not _the buffer text at point_
represents a given type of thing or not there must at
least be a char at point!

(You'll get the same error if you try thing-at-point
in an empty buffer - no char at point to test.  I see
that too with Emacs 29.4.)

An alternative behavior - incorrect/poorer IMO - could
conceivably be to return nil.  But to me (and to vanilla
Emacs -Q for all releases I know of), that would be
claiming that _there's text at point_ that does not
represent such a thing.

Your Emacs bug #74890 says you're using an Emacs 31
development version.  Emacs 30 hasn't even been released
yet!  The bleeding edge bleeds, and I'm hoping Emacs 31's
current failure to raise an error in this context is an
ephemeral bug that'll be fixed before release.  (This is a
regression - a backward-incompatible change.  But it may
be intentional.)

If _released_ Emacs 31 thing-at-point changes the behavior
to instead return nil, then I'll have to decide whether to
follow suit.  My feeling, for now at least, is that that's
poor behavior (a bug).  And if I go with that feeling then
at most I'll perhaps provide an option to give users such
(misguided) behavior if they so choose.

To me, it's important that thing-at-point be usable _not_
just to grab some text at point to use as a default value
but - more importantly, if less commonly - to test whether
there's a given type of thing at point, i.e., for
_conditional/filtering_ behavior.  With such an outlook I
think it's important for the testing to be doable and done
on (duh!) text, i.e., a character at point.

That such filtering behavior is important to me is also
why my code returns nil when just _after_ a thing (not
on/at a thing).

One day, vanilla Emacs opted to return a thing that's just
before, not at point.  I disagreed with that change when
it was made, but was overruled.  I think it's due to a
shortsighted opinion that the only use for thing-at-point
is to grab some text for use as a default value - not _at_
point, but _near_ point.  For that use case thingatpt+.el
provides different functions: *-near[est]-point.  Those
functions let you specify what you mean by "near".

When you can repro the bug with `thingatpt+.el', but not
`emacs -Q', in an actual Emacs release (e.g. 31), please
send me a mail with a recipe to repro it.  Thx.

bug#74890: 31.0.50; (thing-at-point 'string) raises error

[Jean Louis]

* Drew Adams <drew.adams@oracle.com> [2024-12-16 03:49]:
> > > > Debugger entered--Lisp error: (wrong-type-argument characterp nil)
> > > >   char-syntax(nil)
> > > >   (eq (char-syntax (char-after)) 34)
> > >
> > > Please show a complete recipe, preferably starting from "emacs -Q".  I
> > > tried to reproduce this problem, but couldn't, which probably means
> > > some special steps are required to see it.
> > >
> > > I also don't understand your claims about "char after", because
> > > there's always something "after" point.
> 
> No, there's _not_ always something (some text) after
> point.  And that, I think, is what this report is about.
> From Jean's description, the only text in the buffer is
> `Hello', where that `o' char is the last char, and
> point is _after_, not on/at, that `o'.

That is right.

> I see the same thing with `emacs -Q' for Emacs 29.4,
> which is the latest Emacs release AFAIK:
> 
> Debugger entered--Lisp error: (wrong-type-argument characterp nil)
>   thing-at-point-bounds-of-string-at-point()
>   bounds-of-thing-at-point(string)
>   thing-at-point(string)
>   eval-expression((thing-at-point 'string) nil nil 127)
>   funcall-interactively(eval-expression (thing-at-point 'string) nil nil 127)
>   command-execute(eval-expression)

I do not have error in default Emacs with -Q with this version:
GNU Emacs 31.0.50 (build 4, x86_64-pc-linux-gnu, GTK+ Version 3.24.38, cairo version 1.16.0) of 2024-12-05

> To me, this behavior makes sense since, per your recipe,
> there's _no character_ at point, and for thing-at-point
> to tell whether or not _the buffer text at point_
> represents a given type of thing or not there must at
> least be a char at point!

I am used to have silent thing-at-point, so I do not expect any
error. I expect the thing to be there or not, as  that is how I am
testing it with my functions. Errors shall be handled with underlying
functions IMHO.

> (You'll get the same error if you try thing-at-point
> in an empty buffer - no char at point to test.  I see
> that too with Emacs 29.4.)

Not on 31.0.50 as I have just tried it.

Or maybe you mean with your library?

> An alternative behavior - incorrect/poorer IMO - could
> conceivably be to return nil.  But to me (and to vanilla
> Emacs -Q for all releases I know of), that would be
> claiming that _there's text at point_ that does not
> represent such a thing.

If the function tests for thing to be "text at point", then yes.

Otherwise, no, as function is testing for something specific, in my
case it was (thing-at-point 'string) where by the default function
without thingatpt+.el does work, and with thingatpt+.el loaded, it
raises errors.

> Your Emacs bug #74890 says you're using an Emacs 31
> development version.  Emacs 30 hasn't even been released
> yet! 

Yes of course, the bug is reported for that version, not earlier.

> The bleeding edge bleeds, and I'm hoping Emacs 31's current failure
> to raise an error in this context is an ephemeral bug that'll be
> fixed before release.  (This is a regression - a
> backward-incompatible change.  But it may be intentional.)

I don't see how that should be bug, and it is actually only raising
error with thingatpt+.el loaded.

(thing-at-point 'string) should give me string or nil

Not error.

I am testing for string, I am not testing if there is text under the
point.

> If _released_ Emacs 31 thing-at-point changes the behavior
> to instead return nil, then I'll have to decide whether to
> follow suit.  My feeling, for now at least, is that that's
> poor behavior (a bug).  And if I go with that feeling then
> at most I'll perhaps provide an option to give users such
> (misguided) behavior if they so choose.

I am expecting you to understand, I am testing  with various functions
`thing-at-point' for specifics, if there is no text, that means that
specific is not there, and why should I get error as user of the function?

It is not error if something is not found!

So the underlying function shall be silently handled for the function
user.

I have been using thingatpt+.el for last 2 years basically, it
remained silent until this special case was discovered, as I was
improving the action button M-RET which verifies many different
things at point.

> To me, it's important that thing-at-point be usable _not_
> just to grab some text at point to use as a default value
> but - more importantly, if less commonly - to test whether
> there's a given type of thing at point, i.e., for
> _conditional/filtering_ behavior.  With such an outlook I
> think it's important for the testing to be doable and done
> on (duh!) text, i.e., a character at point.

It is usable as long as underlying functions do well. 

(thing-at-point 'url) will find URL, and not number for example or
UUID.

So if there is no URL or no UUID, one need not raise errors.

I hope you will come to same thinking.

> That such filtering behavior is important to me is also
> why my code returns nil when just _after_ a thing (not
> on/at a thing).

That is fine approach to me.

> When you can repro the bug with `thingatpt+.el', but not
> `emacs -Q', in an actual Emacs release (e.g. 31), please
> send me a mail with a recipe to repro it.  Thx.

$ emacs -Q

- open empty buffer C-x b akjsndjansjkdn

- evaluate (thing-at-point 'string) and it will return NIL

- I am moving to directory Drew Adams and then evalute:

(add-to-list 'load-path (expand-file-name "."))

- then evaluate (thing-at-point 'string) and it will raise error

-- 
Jean Louis

bug#74890: 31.0.50; (thing-at-point 'string) raises error

[Eli Zaretskii]

> Cc: 74890@debbugs.gnu.org
> Date: Mon, 16 Dec 2024 09:12:47 +0300
> From: Jean Louis <bugs@gnu.support>
> 
> > > > I also don't understand your claims about "char after", because
> > > > there's always something "after" point.
> > 
> > No, there's _not_ always something (some text) after
> > point.  And that, I think, is what this report is about.
> > From Jean's description, the only text in the buffer is
> > `Hello', where that `o' char is the last char, and
> > point is _after_, not on/at, that `o'.
> 
> That is right.

It's wrong, but I cannot afford arguing with you two about this nit.

> > I see the same thing with `emacs -Q' for Emacs 29.4,
> > which is the latest Emacs release AFAIK:
> > 
> > Debugger entered--Lisp error: (wrong-type-argument characterp nil)
> >   thing-at-point-bounds-of-string-at-point()
> >   bounds-of-thing-at-point(string)
> >   thing-at-point(string)
> >   eval-expression((thing-at-point 'string) nil nil 127)
> >   funcall-interactively(eval-expression (thing-at-point 'string) nil nil 127)
> >   command-execute(eval-expression)
> 
> I do not have error in default Emacs with -Q with this version:
> GNU Emacs 31.0.50 (build 4, x86_64-pc-linux-gnu, GTK+ Version 3.24.38, cairo version 1.16.0) of 2024-12-05

Until there's a recipe to reproduce this in "emacs -Q", I maintain
that this is not an Emacs bug, and should be closed.  Which I will do
soon, unless a recipe is posted.

bug#74890: 31.0.50; (thing-at-point 'string) raises error

[Drew Adams via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Until there's a recipe to reproduce this in "emacs -Q", I maintain
> that this is not an Emacs bug, and should be closed.  Which I will do
> soon, unless a recipe is posted.

Please feel free to close this bug.

I thought that this return-nil-no-error behavior
was new in Emacs 30.  But I found that it was
introduced in 28.1.  All Emacs releases 20-27
return nil in the reported scenario (like Emacs
30+ will).

I've changed my mind: I think that's the right
behavior.  I assume that the Emacs 28-29 behavior
wasn't introduced intentionally but was an
unintended regression - fixed in Emacs 30.

[Note that the behavior is to return nil instead
of returning the string that's just before point.
That's the correct behavior, IMO, and it was the
main point I was making earlier: Don't return a
THING that's _before_ point and not _at_ point.
At least for _string_ THINGs, vanilla Emacs is
doing the right thing.]

Jean: I've now adapted my code (thingatpt+.el)
      to use the Emacs 30 definition of
      `thing-at-point-bounds-of-string-at-point'.

      Please download the latest thingatpt+.el.
      and let me know if you see any problems
      with the fix.  Thx.
___

However, since we're talking now about
`thing-at-point-bounds-of-string-at-point':

1. I don't see why it should ever return a non-string
sexp.  E.g., the doc string includes this:

"Prefer the enclosing string, with fallback on sexp at point."

There's no vanilla code that even calls this function,
so that return-a-non-string must have been something
for which a use case was only imagined, and not for
any existing use (?).

(It's of course used by `bound-of-thing-at-point'
when its arg is the symbol `string'.  But grepping
for its name and for "(thing-at-point 'string)"
finds no hits.)

I'm not even sure what kind of non-string sexp this is
meant for.  I tried lists, symbols, and numbers - just
nil was returned, always.

What's the use case for returning a non-string sexp?
IOW, why do we cater to that case, when no string
syntax is involved?

It's the "else" clause of the `if' that I question:
why include it?

;; At the beginning of the string
 (if (let ((ca (char-after)))
       (and ca (eq (char-syntax ca) ?\")))
     (let ((bound (bounds-of-thing-at-point 'sexp)))
	  (and bound
	       (<= (car bound) (point)) (< (point) (cdr bound))
	       bound)))

2. (nit) I don't think it makes sense for the two
functions `thing-at-point-bounds-of-string-at-point'
and `thing-at-point-bounds-of-list-at-point' (and no
other functions) to have this in their doc string:

"[Internal function used by `bounds-of-thing-at-point'.]"

Either all such `thing-at-point-bounds-of-*-at-point'
functions should have it or none should.  I think
none should, because it's misleading.  These _aren't_
internal functions.

No more so than `yank' is an internal function of
`delete-selection'.  They're simply functions `put'
on a THING name such as `string', for use by
`bounds-of-thing-at-point'.  And they can be used
by any code that wants the bounds of a string, list,
etc. directly, and doesn't necessarily want the text
that's bounded.

bug#74890: 31.0.50; (thing-at-point 'string) raises error

[Stefan Kangas]

Drew Adams via "Bug reports for GNU Emacs, the Swiss army knife of text
editors" <bug-gnu-emacs@gnu.org> writes:

>> Until there's a recipe to reproduce this in "emacs -Q", I maintain
>> that this is not an Emacs bug, and should be closed.  Which I will do
>> soon, unless a recipe is posted.
>
> Please feel free to close this bug.

I closed it here:
https://debbugs.gnu.org/74890#16

bug#74890: 31.0.50; (thing-at-point 'string) raises error

[Jean Louis]

* Drew Adams <drew.adams@oracle.com> [2024-12-17 02:39]:
> Jean: I've now adapted my code (thingatpt+.el)
>       to use the Emacs 30 definition of
>       `thing-at-point-bounds-of-string-at-point'.
> 
>       Please download the latest thingatpt+.el.
>       and let me know if you see any problems
>       with the fix.  Thx.

I see no problem now.

-- 
Jean Louis