bug#9300: 24.0.50; `bounds-of-thing-at-point' does not return nil when just after THING

[Drew Adams <drew.adams@oracle.com>]

> Tried it, and yes it does. Otherwise, I wouldn't understand what
> this bug is about.

Sorry, my bad (it's been several years since I filed this).
Yes, it should return nil, as there is NO symbol at point.

> Your reasoning seems valid, however by now this behavior is
> ingrained into my expectations of how thing-at-point should behave.

It is your expectation that is wrong.  There are plenty of uses
of thing-at-point that go far beyond just looking for a default
value of a name near point or trying to complete a name before
(not at) point.

Those other uses include the need to test whether or not there
IS a given THING at point.  The design itself depends on this
difference: Is there a THING at point or not?  The purpose of
thingatpt.el is not only to maximize finding a THING that is
_near_ point.  One purpose is to test whether there IS a THING
AT point.

If your code wants _only_ to complete a name (or other THING)
that is at or _near_ point, then it should temporarily move to
where it really wants to pick up the name (or other THING).
Your code should not depend on an off-by-one bug, however
longstanding.

I am well aware of how useful picking up a name at or _near_
point can be, either to complete it or to use it as a default
value. I see (in emacs-devel) that you are now looking into
picking up a name near point - but you are limiting that to the
same line.

I'm glad someone is finally getting around to this.  FWIW, I
did it in 1996 and have been using it ever since (I proposed
it for Emacs).

But my code does not limit looking to the current line.
Users can control what "near" means using these variables,
which are the max number of chars and lines to search from
point, left-and-right and up-and-down, respectively:

 (tap-)near-point-x-distance - default 50 chars
 (tap-)near-point-y-distance - default 50 lines

These variables are used typically by functions that invoke
`tap-thing/form-nearest-point-with-bounds', and that provide
default text for minibuffer input or text to complete.

`thingatpt+.el' includes these "nearest-point" functions
(with library prefix `tap-'):

 bounds-of-form-nearest-point
 bounds-of-list-nearest-point
 bounds-of-sexp-nearest-point
 bounds-of-symbol-nearest-point
 bounds-of-thing-nearest-point
 color-nearest-point
 color-nearest-point-with-bounds
 find-fn-or-var-nearest-point  (a command)
 form-nearest-point
 form-nearest-point-with-bounds
 list-contents-nearest-point
 list-nearest-point
 list-nearest-point-with-bounds
 list-nearest-point-as-string
 non-nil-symbol-name-nearest-point
 non-nil-symbol-nearest-point
 number-nearest-point
 region-or-word-nearest-point
 region-or-non-nil-symbol-name-nearest-point
 sentence-nearest-point
 sexp-nearest-point
 sexp-nearest-point-with-bounds
 string-contents-nearest-point
 string-nearest-point
 symbol-name-nearest-point
 symbol-nearest-point
 symbol-nearest-point-with-bounds
 thing-nearest-point
 thing-nearest-point-with-bounds
 unquoted-list-nearest-point
 unquoted-list-nearest-point-as-string
 word-nearest-point

So I do have some experience with the kinds of uses of
thing-at-point that you are interested in.  But there are
lots of other uses, as well, which rely on its behavior
being usable in repetitive (iterative, recursive) contexts.

And that means relying on its treating the position after
a thing differently from the position before the thing, wrt
whether or not there is a THING at that position.

If you never make use of thing-at-point, beyond wanting to
pick up a thing at or near point, and you don't care much
about exactly where point is in relation to the thing, then
you won't understand the importance of this bug.

My code that does use thing-at-point repetitively needs the
basic code to distinguish whether there is a thing at point
in a precise way.  It is not enough to just maximize the
possibility of obtaining a thing near point.  The point of
such code is instead to perform operations on occurrences
of a specific THING over a given region of text.

One example is `isearchp-thing', which does incremental
search within THING search contexts.  The contexts are
determined by scanning a region/buffer progressively, and
this depends on basic thing-at-point functions doing the
right thing wrt whether there is a THING at a given point.

The scanning function is `isearchp-thing-scan', and it is
in its code that you will find the comment included in this
bug report:

;; The correct code here is (setq beg thg-end).  However,
;; unless you use my library `thingatpt+.el' or unless Emacs
;; bug #9300 is fixed that will loop forever.  In that case
;; we move forward a char to prevent looping, but that means
;; that the position just after a THING is considered to be
;; covered by the THING (which is incorrect).

(setq beg (if (featurep 'thingatpt+) thg-end (1+ thg-end)))

> This would be a breaking change. For instance, it will make
> (bounds-of-thing-at-point 'symbol) unsuitable for use in a
> completion-at-point-functions element, to compute the first
> two values of the returned list, because during completion 
> you're most often "after" the symbol. And I do use it for
> that purpose in one third-party package.

See previous.  It is the calling code that is wrong, here (and
no need for quoting "after" - you are after, not on, the symbol).
The calling code should instead check `bounds-of-thing-at-point'
at the right position.

There has to be a difference between there being a THING at
point and there not being a THING at point.  And thingatpt.el
is  designed for there not being at THING at point when, well,
there is no THING at point.  At point is not the same as before
point.

(Yes, point is _between_ characters.  But one or the other,
but not both, positions wrt a char that is part of THING needs
to be picked as its start or end.

Especially for uses of thing-at-point that are iterative or
recursive, it is important that either the position after the
last char or the position before the first char not be included
in the THING "at" that position.  This should be obvious, but
is not paid attention to if one's only interest is in grabbing
some text that is _near_ point but might not be exactly _at_
point.

> At the very least, this will change the existing behavior.

Yes - it is a bug fix.  The current behavior is bugged.

> > This is the design of the thingatpt code, and the reason why
> > `<=' instead of `<' is a bug:
> >
> >    the function that is (get THING 'end-op) moves PAST the THING,
> >    so that point is not on the THING.  This is true generally, no
> >    matter the type of THING.
> 
> That's not a quote from thingatpt.el.

It is nevertheless the design (intention), clear from the code.