Re: RE: [External] : Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

Re: RE: [External] : Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

[Jim Porter]

On 4/30/2024 2:10 PM, Drew Adams wrote:
>> I've also fixed a bug in EWW and bug-reference-mode
>> where it would return nil for (thing-at-point 'url)
>> if point was at the *end* of a URL.
> 
> By "at the end" I assume you really mean just
> _after_ a URL, i.e., no longer on/at the URL.
> 
> FWIW, that's actually _superior_ behavior.
> 
> Unfortunately however, Emacs has chosen the
> behavior you describe here:
> 
>> It's now consistent with how 'thing-at-point'
>> works by default.
> 
>> (If you have two consecutive URLs and point
>> is between them...it'll prefer the second one.)
> 
> Which is better!  It's what "at point" means.
[snip]
> See bug #9300, " `bounds-of-thing-at-point'
> does not return nil when just after THING".

I agree overall that your proposed behavior is more correct, and it's 
probably how I'd have implemented 'thing-at-point' if I were doing it 
from scratch. However, I think an even worse outcome than 
"thing-at-point looks at point or before-point" is "sometimes 
thing-at-point just looks at point, and other times it looks at point or 
before-point" (which is what it does today).

I'd even be open to something like a 'thing-at-point-is-strict' defvar 
that people could let-bind as wanted, but I'm not going to *argue* for 
that myself.

Ultimately though, this patch is really just about providing the 
necessary defcustoms for org-mode to be able to use 'thing-at-point' 
(and for Ihor to feel ok about it ;)). Changing 'thing-at-point's 
behavior should probably be handled separately, especially since there'd 
be an uphill battle to revisit the decision in bug#9300.

RE: RE: [External] : Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

[Drew Adams]

Thanks for your reply, Jim.

> On 4/30/2024 2:10 PM, Drew Adams wrote:
> >> I've also fixed a bug in EWW and bug-reference-mode
> >> where it would return nil for (thing-at-point 'url)
> >> if point was at the *end* of a URL.
> >
> > By "at the end" I assume you really mean just
> > _after_ a URL, i.e., no longer on/at the URL.
> >
> > FWIW, that's actually _superior_ behavior.
> >
> > Unfortunately however, Emacs has chosen the
> > behavior you describe here:
> >
> >> It's now consistent with how 'thing-at-point'
> >> works by default.
> >
> >> (If you have two consecutive URLs and point
> >> is between them...it'll prefer the second one.)
> >
> > Which is better!  It's what "at point" means.
> >
> [snip]
> >
> > See bug #9300, " `bounds-of-thing-at-point'
> > does not return nil when just after THING".
> 
> I agree overall that your proposed behavior is more correct, and it's
> probably how I'd have implemented 'thing-at-point' if I were doing it
> from scratch. However, I think an even worse outcome than
> "thing-at-point looks at point or before-point" is "sometimes
> thing-at-point just looks at point, and other times it looks at point or
> before-point" (which is what it does today).

Yes, such inconsistency is arguably worse than
consistent bad behavior.  Arguably worse - and
arguably better.  (That's in the nature of
inconsistency - some will see a glass half
full; others half empty.)

> I'd even be open to something like a 'thing-at-point-is-strict' defvar
> that people could let-bind as wanted, but I'm not going to *argue* for
> that myself.
> 
> Ultimately though, this patch is really just about providing the
> necessary defcustoms for org-mode to be able to use 'thing-at-point'
> (and for Ihor to feel ok about it ;)). Changing 'thing-at-point's
> behavior should probably be handled separately, especially since there'd
> be an uphill battle to revisit the decision in bug#9300.

I hear you.  The behavior should be changed so
that, in general, bounds-of-thing-at-point etc.
return nil when there is _no thing at point_,
including when point is after, including just
after, a thing but not on such a thing.

There can be commands (and noncommand fns) that
return things _near_ point, not only at point.
And "near" can be configurable with an argument.

In particular, they can do what the vanilla fns
currently do: return a thing at OR just before
point.  But the "-at-point" functions shouldn't
do that.  They should do what their names say.

It's important to have such functions.  It's
not just about arguing that strictly at-point
is better than at-or-just-after-point.  The
point (sic) is that currently there's _no way
to determine whether_ there's a thing at point.

That's the real problem - no test for a thing
at a given position.  That means that a whole
slew of possible applications of the feature
are impossible to realize.
___

Along with the fix for this bug, there could
also be a defvar, or even an option, that fuses
the two behaviors (at OR just before), i.e.,
that gives the current, misguided behavior, to
allow existing code or habits compatibility.

It's not hard for Emacs to still DTRT.  It just
takes a decision and admission that the behavior
was misguided and unnecessarily limiting (BIG
time).

RE: Re: [External] : Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 875 bytes --]

Jim wrote:

> At the risk of veering off-topic (I mainly care about adding
> 'bounds-of-thing-at-point-provider-alist' and
> 'forward-thing-provider-alist'), would adding a new optional STRICT
> argument to 'thing-at-point' and friends be an ok resolution for
> everyone? This argument would enable Drew's proposed behavior. That way,
> users get all the nice behavior by default just like today, and
> programmers who require strict correctness in their code also have an
> option.

1+ if STRICT means thing starting at point and without it we may need to
look for the beginning of the thing (lazy point setting)....

/PA
-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 1309 bytes --]

RE: Re: [External] : Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

[Drew Adams]

>> At the risk of veering off-topic (I mainly care about adding
>> 'bounds-of-thing-at-point-provider-alist' and
>> 'forward-thing-provider-alist'), would adding a new optional STRICT
>> argument to 'thing-at-point' and friends be an ok resolution for
>> everyone? This argument would enable Drew's proposed behavior. That way,
>> users get all the nice behavior by default just like today, and
>> programmers who require strict correctness in their code also have an
>> option.
>
> 1+ if STRICT means thing starting at point
> and without it we may need to look for the
> beginning of the thing (lazy point setting)....

Why "starting" at point?  STRICT shouldn't mean
checking whether a THING starts at point.

The "strict" behavior that's needed is checking
_whether_ there is a THING at point, that is,
checking whether the char at point (which really
means just after point) is on/inside a THING.

That's for `bounds-of-thing-at-point' etc.  For
thing-at-point etc., IF there's a THING at point
then that THING is the non-nil value returned.

In terms of implementation, I suggest you take
a look at the code in thingatpt+.el.  Look at
functions `tap-bounds-of-thing-at-point' and
`tap-thing-at-point'.  Just remove the `tap-'
prefix for code that DTRT.  The file Commentary
explains things in detail.

My code adds an optional arg SYNTAX-TABLE, but
you need not bother with that.  Elisp now has
`with-syntax-table', which can be used as a
workaround if there's no such argument.

In the end, `bounds-of-thing-at-point' and
`thing-at-point' should return nil if there's
no THING at the char at point.

For "lax" behavior that corresponds to the
current vanilla behavior, if the strict
behavior would return nil then they return
what the strict behavior would return at
(1- point).

But really such a lax behavior is pretty lame.
What's needed, for trying to grab a THING near
point is not just checking backward one char
but checking backward, forward, up, and down
N chars, where N determines what "near" means.

thingatpt+.el provides two user options,
`tap-near-point-x-distance' and
`tap-near-point-y-distance', the max number of
chars from point to check for a THING (for X:
left and right, for Y: up and down).  Setting
the Y value to zero constrains search to the
same line as point.

Re: Re: [External] : Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

[Pedro Andres Aranda Gutierrez]

[-- Attachment #1: Type: text/plain, Size: 3231 bytes --]

H, Drew
Hmmm... thanks for the clarification. That's going to be extremely helpful
once I'm back on track after the "grading season" ;-) All this started, at
least for me, when I was trying to write a simple interactive function to
copy the contents of the cell the point is in to the kill-ring (and
eventually to the clipboard). I eventually came up with a simple solution.
This make org-mode tables very comfortable for longer Web formularies with
complex data, like bank accounts.

Best, /PA

On Wed, 8 May 2024 at 20:14, Drew Adams <drew.adams@oracle.com> wrote:

> >> At the risk of veering off-topic (I mainly care about adding
> >> 'bounds-of-thing-at-point-provider-alist' and
> >> 'forward-thing-provider-alist'), would adding a new optional STRICT
> >> argument to 'thing-at-point' and friends be an ok resolution for
> >> everyone? This argument would enable Drew's proposed behavior. That way,
> >> users get all the nice behavior by default just like today, and
> >> programmers who require strict correctness in their code also have an
> >> option.
> >
> > 1+ if STRICT means thing starting at point
> > and without it we may need to look for the
> > beginning of the thing (lazy point setting)....
>
> Why "starting" at point?  STRICT shouldn't mean
> checking whether a THING starts at point.
>
> The "strict" behavior that's needed is checking
> _whether_ there is a THING at point, that is,
> checking whether the char at point (which really
> means just after point) is on/inside a THING.
>
> That's for `bounds-of-thing-at-point' etc.  For
> thing-at-point etc., IF there's a THING at point
> then that THING is the non-nil value returned.
>
> In terms of implementation, I suggest you take
> a look at the code in thingatpt+.el.  Look at
> functions `tap-bounds-of-thing-at-point' and
> `tap-thing-at-point'.  Just remove the `tap-'
> prefix for code that DTRT.  The file Commentary
> explains things in detail.
>
> My code adds an optional arg SYNTAX-TABLE, but
> you need not bother with that.  Elisp now has
> `with-syntax-table', which can be used as a
> workaround if there's no such argument.
>
> In the end, `bounds-of-thing-at-point' and
> `thing-at-point' should return nil if there's
> no THING at the char at point.
>
> For "lax" behavior that corresponds to the
> current vanilla behavior, if the strict
> behavior would return nil then they return
> what the strict behavior would return at
> (1- point).
>
> But really such a lax behavior is pretty lame.
> What's needed, for trying to grab a THING near
> point is not just checking backward one char
> but checking backward, forward, up, and down
> N chars, where N determines what "near" means.
>
> thingatpt+.el provides two user options,
> `tap-near-point-x-distance' and
> `tap-near-point-y-distance', the max number of
> chars from point to check for a THING (for X:
> left and right, for Y: up and down).  Setting
> the Y value to zero constrains search to the
> same line as point.
>


-- 
Fragen sind nicht da, um beantwortet zu werden,
Fragen sind da um gestellt zu werden
Georg Kreisler

Headaches with a Juju log:
unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run
a leader-deposed hook here, but we can't yet

[-- Attachment #2: Type: text/html, Size: 4197 bytes --]

Re: Re: [External] : Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

[Jean Louis]

* Drew Adams <drew.adams@oracle.com> [2024-05-08 21:17]:
> >> At the risk of veering off-topic (I mainly care about adding
> >> 'bounds-of-thing-at-point-provider-alist' and
> >> 'forward-thing-provider-alist'), would adding a new optional STRICT
> >> argument to 'thing-at-point' and friends be an ok resolution for
> >> everyone? This argument would enable Drew's proposed behavior. That way,
> >> users get all the nice behavior by default just like today, and
> >> programmers who require strict correctness in their code also have an
> >> option.
> >
> > 1+ if STRICT means thing starting at point
> > and without it we may need to look for the
> > beginning of the thing (lazy point setting)....

I have few functions defining  things at point such as ISO date, or things within various parenthesis. I was of understanding that it is the regular expression that decides what is on the point or not.

Even space before what someone considers object could mean  that user is at the point.

Thus  there is no need to constrain users defining what is thing at point by some switch like STRICT which would basically break what user intended to do.

Just leave it globally available to users to decide how thing at point is defined.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

✡️🛡️ Proudly standing with Israel, a nation rooted in history and culture. Let's condemn hatred and promote understanding.

In support of Richard M. Stallman
https://stallmansupport.org/

RE: Re: [External] : Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

[Drew Adams]

> * Drew Adams <drew.adams@oracle.com> [2024-05-08 21:17]:
> > >> At the risk of veering off-topic (I mainly care about adding
> > >> 'bounds-of-thing-at-point-provider-alist' and
> > >> 'forward-thing-provider-alist'), would adding a new optional STRICT
> > >> argument to 'thing-at-point' and friends be an ok resolution for
> > >> everyone? This argument would enable Drew's proposed behavior. That
> way,
> > >> users get all the nice behavior by default just like today, and
> > >> programmers who require strict correctness in their code also have an
> > >> option.
> > >
> > > 1+ if STRICT means thing starting at point
> > > and without it we may need to look for the
> > > beginning of the thing (lazy point setting)....

Please don't (presumably accidentally) make it
look like you're responding to something I wrote,
but you actually quote two others - nothing at
all from me.  Thx.

> I was of understanding that it is the regular
> expression that decides what is on the point or not.

What regular expression?

Are you thinking of a regular expression that's
checked to see if some particular kind of thing
is at point, e.g. a regexp like the value of a
var such as `thing-at-point-newsgroup-regexp'?

Such a regexp doesn't determine/define what "at
point" means.

The question here is where you test for a thing
at point, i.e., what "at point" means.  It's not
about how you test that.  If your test function
actually checks something at a char that's not
at point, that's its prerogative.  It's about
the generic behavior of Thing At Point.

The char at point (which actually means the char
just after point) is well-defined.  That's what
the generic functions `bounds-of-thing-at-point'
and `thing-at-point' use as starting point.
Then then move forward and backward a THING etc.
to see if there's such a THING at that starting
point.

That movement is customizable (with properties
`forward-op', `beginning-op', `end-op',
`bounds-of-thing-at-point', `thing-at-point',
and in other ways).  But the starting position
is `(point)'; that's where the check is done.

The bug is in `bounds-of-thing-at-point'.  It
should be faithful to its name, and check only
at (point), not at EITHER point OR (1- (point)).

Even if the fix were to always check only at
(1- (point)), instead of (point), it would be
a legitimate fix for the problem caused by the
ambiguity: there's _no way to determine whether_
there's a thing at a given position.

The bugged behavior defeats the successive use
of functions that do something with _the next
thing_ at point when that _next thing butts up
against the previous thing_.

IOW, it technically doesn't matter which single
position is the "origin" for checking: (point)
or (1- (point)) or whatever else.  The bug is
that _no single_ position is used.

However, since "at point" does mean something
in Emacs English, the proper fix is to always
check starting from (point), not (1- (point))
or any other position.

Either always-(point) or always-(1- (point))
would be a fix.  But always-(point) is the
better/proper fix, just because it accords
with Emacs terminology.

Re: Re: [External] : Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

[Jean Louis]

* Drew Adams <drew.adams@oracle.com> [2024-05-15 23:57]:
> > I was of understanding that it is the regular
> > expression that decides what is on the point or not.
> 
> What regular expression?
> 
> Are you thinking of a regular expression that's
> checked to see if some particular kind of thing
> is at point, e.g. a regexp like the value of a
> var such as `thing-at-point-newsgroup-regexp'?
> 
> Such a regexp doesn't determine/define what "at
> point" means.

Maybe I have not call it right. I may explain it on the example:

;;; Thing at point 'iso-date

We assume that rcd-rx-iso-date ➜ "[[:digit:]][[:digit:]][[:digit:]][[:digit:]]-\\(?:0[[:digit:]]\\|1[0-2]\\)-\\(?:[0-2][[:digit:]]\\|3[01]\\)"

(defun rcd-tap-iso-date-start ()
  "Move point to the beginning of the ISO date."
  (when (thing-at-point-looking-at rcd-rx-iso-date)
    (goto-char (match-beginning 0))))

(defun rcd-tap-iso-date-end ()
  "Move point to the end of the ISO date."
  (when (thing-at-point-looking-at rcd-rx-iso-date)
    (goto-char (match-end 0))))

(put 'iso-date 'beginning-op 'rcd-tap-iso-date-start)
(put 'iso-date 'end-op 'rcd-tap-iso-date-end)

The definition of the thing at point is the one deciding what is the thing. And programmer is the one deciding where the thing actually begins. While user may have different view point where the thing at point begins, the actual definition is somewhere in the code and can be found and understood.

I could define that my ISO date is surrounded by spaces " 2024-05-16 " and user may assume that it is not: "2024-05-16", so the definition is in the code and left for programmers to decide how to define it. What if I wish that thing at point is anywhere on the line, regardless where is the cursor? While I did not try it out, I think that is quite feasible to do.

Then why try to make general STRICT, to try to affect specific random or indefinite definitions of code, instead, complaint or suggestion, patch on specific code can be made to make it stricter, or new code may be written to make that what other user wish to achieve.

> That movement is customizable (with properties
> `forward-op', `beginning-op', `end-op',
> `bounds-of-thing-at-point', `thing-at-point',
> and in other ways).  But the starting position
> is `(point)'; that's where the check is done.

Yes, I agree on everything you said with the exception not mentioned, and that is the consideration of programmer. Programmer may say that ISO date below is at point, when cursor or point is at X, and that ISO date shall be extracted or memorized, noted, etc. by follow up functions. It is matter of consideration.

..X.......................... 2024-05-16

ISO date need not be like below X to say that it is "at point" as it is matter of consideration. I see your words that you said the same what I am saying here.

2X24-05-16

> The bug is in `bounds-of-thing-at-point'.  It
> should be faithful to its name, and check only
> at (point), not at EITHER point OR (1- (point)).

1. The thing at point is defined by programmer.

2. What are bounds of thing at point is thus result of definition of programmer.

3. It implies that thing at point may be looking like having spaces or
   being forward or backwards from cursor, and still be considered at
   point, as that is defined by programmer.

I do not consider that a bug.

 2024-05-16

A cursors before the above ISO date is not captured, and after it, is not captured as thing at point.  I have tried it with M-: (thing-at-point 'iso-date), though I believe that other thing at point may be defined so to be sensed close to cursor, and that freedom is fine. What is the "bound" is defined by programmer not by function "bounds-of-thing-at-point".

> Even if the fix were to always check only at
> (1- (point)), instead of (point), it would be
> a legitimate fix for the problem caused by the
> ambiguity: there's _no way to determine whether_
> there's a thing at a given position.

I see that as freedom, and you see as ambiguity. Without looking into
the code and understanding it, isn't it all of the code at some point
ambiguity?

> IOW, it technically doesn't matter which single
> position is the "origin" for checking: (point)
> or (1- (point)) or whatever else.  The bug is
> that _no single_ position is used.

I see that function `thing-at-point-looking-at' is using function `point'. You are here expert, you see some bug there, and I see freedom of considerations.

> However, since "at point" does mean something
> in Emacs English, the proper fix is to always
> check starting from (point), not (1- (point))
> or any other position.

I did not experience that with the ISO date, so I do not see it is bug. Maybe it is, as I forgot what I did to make it work. But I do understand that other things at point are invoked when cursor is before it. And that is freedom to define it, it does not need to be strictly on the point.

My searchable text area maybe a line, not so? And I wish to capture or record that date on the line, while I did not make it so, it may be useful. Point could be anywhere far from the ISO date, and invoking function could capture the date. I find that useful.

> Either always-(point) or always-(1- (point))
> would be a fix.  But always-(point) is the
> better/proper fix, just because it accords
> with Emacs terminology.

Sure, but where is that (1- (point))? I did not find it. I was using this function below, and I do not see that. I tried searching for "(1- )" and did not find that reference, but found something for the URL https://www.EXAMPLE.comX - if I place cursor on X it will detect the URL. But that is because URL thing was defined that way to be detected. Maybe you should complain on that specific function to be changed, instead of trying to change general code in present and in future with STRICT. 

The function below has DISTANCE argument for those settings.

(defun thing-at-point-looking-at (regexp &optional distance)
  "Return non-nil if point is in or just after a match for REGEXP.
Set the match data from the earliest such match ending at or after
point.

Optional argument DISTANCE limits search for REGEXP forward and
back from point."
  (let* ((old (point))
         (beg (if distance (max (point-min) (- old distance)) (point-min)))
         (end (if distance (min (point-max) (+ old distance))))
         prev match)
    (save-excursion
      (goto-char beg)
      (while (and (setq prev (point)
                        match (re-search-forward regexp end t))
                  (< (match-end 0) old))
        (goto-char (match-beginning 0))
        ;; Avoid inflooping when `regexp' matches the empty string.
        (unless (< prev (point)) (forward-char))))
    (and match (<= (match-beginning 0) old (match-end 0)))))

What if I think of visible buffer as thing at point? Why not? Then I wish to capture specific text inside, and no matter where is cursor, it would work. I find that handy, useful.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

✡️🛡️ Proudly standing with Israel, a nation rooted in history and culture. Let's condemn hatred and promote understanding.

In support of Richard M. Stallman
https://stallmansupport.org/

RE: Re: [External] : Re: Adding custom providers for thingatpt.el (was: [PATCH] Add support for 'thing-at-point' to get URL at point)

[Drew Adams]

> > The bug is in `bounds-of-thing-at-point'.  It
> > should be faithful to its name, and check only
> > at (point), not at EITHER point OR (1- (point)).
 
> > Even if the fix were to always check only at
> > (1- (point)), instead of (point), it would be
> > a legitimate fix for the problem caused by the
> > ambiguity: there's _no way to determine whether_
> > there's a thing at a given position.
 
> > IOW, it technically doesn't matter which single
> > position is the "origin" for checking: (point)
> > or (1- (point)) or whatever else.  The bug is
> > that _no single_ position is used.

You're missing the point, even though you quoted
some of it (above).  You didn't quote this part:

  The bugged behavior defeats the _successive_
  use of functions that do something with the
  next thing at point when that next thing
  butts up against the previous thing.

Define a `go-to-next-thing' command that you can
repeat, to go to successive things of given kind,
for any kind of thing.