bug#24969: 26.0.50; number-at-point

bug#24969: 26.0.50; number-at-point

[Andreas Röhler]

After evaluating show-number in buffer of contents below, it should jump 
to char "1" and return them, but returns nil.

;;;

foo-1

(defun show-number ()
   (interactive)
   (goto-char 5)
   (message "%s" (number-at-point)))

;;;


In GNU Emacs 26.0.50.1 (i686-pc-linux-gnu, GTK+ Version 3.14.5) of 
2016-11-15

bug#24969: 26.0.50; number-at-point

[Stephen Berman]

On Sun, 20 Nov 2016 13:47:50 +0100 Andreas Röhler <andreas.roehler@easy-emacs.de> wrote:

> After evaluating show-number in buffer of contents below, it should jump to
> char "1" and return them, but returns nil.
>
> ;;;
>
> foo-1
>
> (defun show-number ()
>   (interactive)
>   (goto-char 5)
>   (message "%s" (number-at-point)))
>
> ;;;

When I try your recipe, point jumps to the - sign and the echo area
displays -1.

> In GNU Emacs 26.0.50.1 (i686-pc-linux-gnu, GTK+ Version 3.14.5) of 2016-11-15

I'm using
GNU Emacs 26.0.50.17 (x86_64-suse-linux-gnu, GTK+ Version 3.14.15)
 of 2016-11-18
(built from commit 0d913da15c094bf596dd685acecf3438228c15cf from master)

Steve Berman

bug#24969: 26.0.50; number-at-point

[Andreas Röhler]

On 20.11.2016 13:53, Stephen Berman wrote:
> On Sun, 20 Nov 2016 13:47:50 +0100 Andreas Röhler <andreas.roehler@easy-emacs.de> wrote:
>
>> After evaluating show-number in buffer of contents below, it should jump to
>> char "1" and return them, but returns nil.
>>
>> ;;;
>>
>> foo-1
>>
>> (defun show-number ()
>>    (interactive)
>>    (goto-char 5)
>>    (message "%s" (number-at-point)))
>>
>> ;;;
> When I try your recipe, point jumps to the - sign
Seems there is a leading whitespace.


>   and the echo area
> displays -1.

Here with that start from minus-sign equally nil is returned.

Thanks

bug#24969: 26.0.50; number-at-point

[npostavs]

tags 24969 unreproducible
quit

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> On 20.11.2016 13:53, Stephen Berman wrote:
>> On Sun, 20 Nov 2016 13:47:50 +0100 Andreas Röhler <andreas.roehler@easy-emacs.de> wrote:
>>
>>> After evaluating show-number in buffer of contents below, it should jump to
>>> char "1" and return them, but returns nil.
>>>
>>> ;;;
>>>
>>> foo-1
>>>
>>> (defun show-number ()
>>>    (interactive)
>>>    (goto-char 5)
>>>    (message "%s" (number-at-point)))
>>>
>>> ;;;
>> When I try your recipe, point jumps to the - sign
> Seems there is a leading whitespace.
>
>
>>   and the echo area
>> displays -1.
>
> Here with that start from minus-sign equally nil is returned.
>
> Thanks

I get -1 here too.

GNU Emacs 26.0.50.15 (x86_64-unknown-linux-gnu, X toolkit) of 2016-11-19

bug#24969: 26.0.50; number-at-point

[Drew Adams]

> tags 24969 unreproducible
> quit

Dunno what this is all about, why some of you are seeing
a number returned (or printed in a message), or why any
of you would expect that to be the case.

In Emacs 25.1 (emacs -Q), `number-at-point' at either
the `-' or the `1' returns nil, for me.  And I do not
see why it should return a number.

`number-at-point' is defined using `form-at-point' with
THING `sexp' and predicate `numberp'.  The sexp picked
up at point is `foo-1', and that fails `numberp'.

What am I missing?  Why should this rightfully return
a number?  I'm guessing that you are all using a more
recent version of `number-at-point' than what is in
Emacs 25.1 (?).  But to me the Emacs 25.1 behavior I
see (i.e., returning nil) is correct.

Did someone change the meaning of `number-at-point'
so that it now picks up a numeral that is not isolated?
If so, why would that be considered proper behavior?
At the very least it is not backward-compatible behavior.

(Hoping I'm just misunderstanding something here.)

bug#24969: 26.0.50; number-at-point

[Noam Postavsky]

On Sun, Nov 20, 2016 at 11:19 AM, Drew Adams <drew.adams@oracle.com> wrote:
>
> In Emacs 25.1 (emacs -Q), `number-at-point' at either
> the `-' or the `1' returns nil, for me.  And I do not
> see why it should return a number.

With 25.1 I get nil, on 26.0.50 I'm getting -1. I believe this is due
to the fix in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24605

bug#24969: 26.0.50; number-at-point

[Drew Adams]

> > In Emacs 25.1 (emacs -Q), `number-at-point' at either
> > the `-' or the `1' returns nil, for me.  And I do not
> > see why it should return a number.
> 
> With 25.1 I get nil, on 26.0.50 I'm getting -1. I believe this is
> due to the fix in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24605

I see.  Dunno what I think about this.  I'd be interested to
hear more from Tino about this.  It clearly is not backward
compatible.  Whether it is an improvement or not, I'm not
sure, in this case.

I guess it depends on what a user would expect of a
"number-at-point" function.  A priori, I don't see why s?he
would expect a non-nil answer if the numeral is embedded in
text that does not delimit a numeral (e.g. non whitespace text).
But maybe it is OK.

Would we expect the same kind of behavior for `sexp-at-point'
if a sexp were not surrounded by chars that delimit a sexp?

In Lisp, at least, there is no number at point, in `foo-2'.
That is, the Lisp parser (reader) would never pick up the
`2' as a number here.

I'm partial to use of thingatpt for Lisp, but I realize that
it is used in other contexts too.  I would probably prefer
that, whatever the context, if the context has a notion of
number and its syntax (numerals) then `number-at-point'
would return a number ONLY when there is really a number
at point, based on the meaning of numeral in that context.

If we could do that well, that would be best, I think.  In
the case of Lisp, it is clear (to me) that returning a number
for `foo-2', with point on the `2', is wrong.

I think we need to think more about this.  One option is
to provide two different functions, one of which does what
has been done in the past and one of which tries to be
smarter along the lines of fixing this bug (but not
necessarily along the lines of the implemented fix).

IOW, I agree with the bug report that `form-at-point'
should - somehow - handle the case where `thing-at-point'
returns a non-string.  There is a bug to be fixed.  But
I'm not convinced that the fix we've implemented is TRT.

I'm open to being convinced otherwise, but this does not
seeme right to me, so far.  It seems like we should be able
to do better, here.

bug#24969: 26.0.50; number-at-point

[Andreas Röhler]

On 20.11.2016 17:58, Drew Adams wrote:
>>> In Emacs 25.1 (emacs -Q), `number-at-point' at either
>>> the `-' or the `1' returns nil, for me.  And I do not
>>> see why it should return a number.
>> With 25.1 I get nil, on 26.0.50 I'm getting -1. I believe this is
>> due to the fix in https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24605
> I see.  Dunno what I think about this.  I'd be interested to
> hear more from Tino about this.  It clearly is not backward
> compatible.  Whether it is an improvement or not, I'm not
> sure, in this case.
>
> I guess it depends on what a user would expect of a
> "number-at-point" function.  A priori, I don't see why s?he
> would expect a non-nil answer if the numeral is embedded in
> text that does not delimit a numeral (e.g. non whitespace text).
> But maybe it is OK.
>
> Would we expect the same kind of behavior for `sexp-at-point'
> if a sexp were not surrounded by chars that delimit a sexp?
>
> In Lisp, at least, there is no number at point, in `foo-2'.
> That is, the Lisp parser (reader) would never pick up the
> `2' as a number here.
>
> I'm partial to use of thingatpt for Lisp, but I realize that
> it is used in other contexts too.

In use here for edit-purposes.
For example raise all numbers in a region - makes it easier sometimes to 
adapt stuff, which doesn't deserve an own template.

Have a first implementation with ar-add-numbers in 
https://github.com/andreas-roehler/werkstatt/blob/master/misc/misc-utils.el 
- just re-writing it.

bug#24969: 26.0.50; number-at-point

[Drew Adams]

> > I guess it depends on what a user would expect of a
> > "number-at-point" function.  A priori, I don't see why s?he
> > would expect a non-nil answer if the numeral is embedded in
> > text that does not delimit a numeral (e.g. non whitespace text).
> > But maybe it is OK.
> >
> > Would we expect the same kind of behavior for `sexp-at-point'
> > if a sexp were not surrounded by chars that delimit a sexp?
> >
> > In Lisp, at least, there is no number at point, in `foo-2'.
> > That is, the Lisp parser (reader) would never pick up the
> > `2' as a number here.
> >
> > I'm partial to use of thingatpt for Lisp, but I realize that
> > it is used in other contexts too.
> 
> In use here for edit-purposes.  For example raise all numbers
> in a region - makes it easier sometimes to adapt stuff, which
> doesn't deserve an own template.

But the question is, "What constitutes a numeral?" in the given
context.  Whatever the context, I would expect some kind of
well-defined delimiting.  In Lisp I would expect what the Lisp
reader would pick up as a number - nothing more.  And that would
exclude picking up `2' within `foo-2'.

> Have a first implementation with ar-add-numbers in
> https://github.com/andreas-roehler/werkstatt/blob/master/misc/misc-
> utils.el - just re-writing it.

There is no `ar-add-numbers' or `add-numbers' in that file
(having downloaded it just now).  Perhaps you meant
`ar-add-to-number-cummulative'?  That is undefined, without
`ar-bounds-of-number-atpt' (not in the file).

Anyway, my point was made above.

bug#24969: 26.0.50; number-at-point

[Andreas Röhler]

On 20.11.2016 22:28, Drew Adams wrote:
>>> I guess it depends on what a user would expect of a
>>> "number-at-point" function.  A priori, I don't see why s?he
>>> would expect a non-nil answer if the numeral is embedded in
>>> text that does not delimit a numeral (e.g. non whitespace text).
>>> But maybe it is OK.
>>>
>>> Would we expect the same kind of behavior for `sexp-at-point'
>>> if a sexp were not surrounded by chars that delimit a sexp?
>>>
>>> In Lisp, at least, there is no number at point, in `foo-2'.
>>> That is, the Lisp parser (reader) would never pick up the
>>> `2' as a number here.
>>>
>>> I'm partial to use of thingatpt for Lisp, but I realize that
>>> it is used in other contexts too.
>> In use here for edit-purposes.  For example raise all numbers
>> in a region - makes it easier sometimes to adapt stuff, which
>> doesn't deserve an own template.
> But the question is, "What constitutes a numeral?" in the given
> context.  Whatever the context, I would expect some kind of
> well-defined delimiting.  In Lisp I would expect what the Lisp
> reader would pick up as a number - nothing more.


The perspective of the lisp-programmer and the user of an editor may be 
different here.
The implementation at progress should pick any valid hex- octal- or 
decimal integer at point.

In a related case even characters are raised here, returning a "b" for 
an "a", an "y" for an "x" etc.
Thus smart-inserting a second (loop)-variable given there exist already 
a first one.


>    And that would
> exclude picking up `2' within `foo-2'.

Not, when for example filenames inside shell-scripts etc. are edited.


>
>> Have a first implementation with ar-add-numbers in
>> https://github.com/andreas-roehler/werkstatt/blob/master/misc/misc-
>> utils.el - just re-writing it.
> There is no `ar-add-numbers' or `add-numbers' in that file
> (having downloaded it just now).  Perhaps you meant
> `ar-add-to-number-cummulative'?  That is undefined, without
> `ar-bounds-of-number-atpt' (not in the file).

Errrm, ar-add-to-number should DTRT, sorry.

bug#24969: 26.0.50; number-at-point

[Drew Adams]

> > But the question is, "What constitutes a numeral?" in the given
> > context.  Whatever the context, I would expect some kind of
> > well-defined delimiting.  In Lisp I would expect what the Lisp
> > reader would pick up as a number - nothing more.
> 
> The perspective of the lisp-programmer and the user of an editor may
> be different here.

Too vague.

> The implementation at progress should pick any valid hex- octal- or
> decimal integer at point.

I disagree.  Users should control whether they want a decimal,
octal, or hex number at point - or any kind of number.  Those
are different things.  In my code I define hex and decimal number
at point functions.  When code wants to get only a decimal number
it does not want to pick up `FACE'.  When code wants an octal
number it does not want `987'.

And nothing prevents a given context from defining a "number"
category.  That's we had, for Lisp numbers.  Other modes are
free to define a number syntax such that the (previous)
implementation of `number-at-point' - or any local,
mode/context-specific implementation, is in charge.

> In a related case even characters are raised here, returning
> a "b" for an "a", an "y" for an "x" etc.

Far too loose and useless.  Thingatpt lets you define any number
of specific kinds of things.  A predefined catch-all that does
lots of things that are not clear is not helpful.

> > In Lisp I would expect what the Lisp reader would pick up
> > as a number - nothing more.  And that would exclude picking
> > up `2' within `foo-2'.
> 
> Not, when for example filenames inside shell-scripts etc. are
> edited.

What part of Lisp reader was not clear?  The (previous)
implementation uses what the current mode considers a number.
It is based on the Lisp reader, but users (or Emacs) can define
other number readings.

bug#24969: 26.0.50; number-at-point

[Tino Calancha]

Drew Adams <drew.adams@oracle.com> writes:

> In Emacs 25.1 (emacs -Q), `number-at-point' at either
> the `-' or the `1' returns nil, for me.  And I do not
> see why it should return a number.
>
> `number-at-point' is defined using `form-at-point' with
> THING `sexp' and predicate `numberp'.  The sexp picked
> up at point is `foo-1', and that fails `numberp'.
From the time when i opened Bug#24605, the 
implementation, in master branch, of `number-at-point'
was different: it changed in commit 786ab4a5 (Bug#8634).
My patch was driven by this implementation.  I didn't notice
that `number-at-point' behaves different in emacs-25.

> What am I missing?  Why should this rightfully return
> a number?  I'm guessing that you are all using a more
> recent version of `number-at-point' than what is in
> Emacs 25.1 (?).  But to me the Emacs 25.1 behavior I
> see (i.e., returning nil) is correct.
>
> Did someone change the meaning of `number-at-point'
> so that it now picks up a numeral that is not isolated?
> If so, why would that be considered proper behavior?
> At the very least it is not backward-compatible behavior.
That's right.  Commit above breaks backward-compatibility.

>In Lisp, at least, there is no number at point, in `foo-2'.
>That is, the Lisp parser (reader) would never pick up the
>`2' as a number here.
The doc string of `list-at-point' clearly says that the list should
be a Lisp list.  In the case of `number-at-point' the doc string just
says 'the number at point', without connection with the Lisp parser.

>IOW, I agree with the bug report that `form-at-point'
>should - somehow - handle the case where `thing-at-point'
>returns a non-string.  There is a bug to be fixed.  But
>I'm not convinced that the fix we've implemented is TRT.
Sure, I am open to alternative fixes.

bug#24969: 26.0.50; number-at-point

[Drew Adams]

> > In Emacs 25.1 (emacs -Q), `number-at-point' at either
> > the `-' or the `1' returns nil, for me.  And I do not
> > see why it should return a number.
> >
> > `number-at-point' is defined using `form-at-point' with
> > THING `sexp' and predicate `numberp'.  The sexp picked
> > up at point is `foo-1', and that fails `numberp'.
>
> From the time when i opened Bug#24605, the
> implementation, in master branch, of `number-at-point'
> was different: it changed in commit 786ab4a5 (Bug#8634).
> My patch was driven by this implementation.  I didn't notice
> that `number-at-point' behaves different in emacs-25.

It behaves the same in Emacs 25.1 as previously.  If something
broke this after Emacs 25.1 then it should be reverted.

> > What am I missing?  Why should this rightfully return
> > a number?  I'm guessing that you are all using a more
> > recent version of `number-at-point' than what is in
> > Emacs 25.1 (?).  But to me the Emacs 25.1 behavior I
> > see (i.e., returning nil) is correct.
> >
> > Did someone change the meaning of `number-at-point'
> > so that it now picks up a numeral that is not isolated?
> > If so, why would that be considered proper behavior?
> > At the very least it is not backward-compatible behavior.
>
> That's right.  Commit above breaks backward-compatibility.

If it returns a number for point on a numeral in the middle
of a symbol name etc. then it breaks not only backward
compatibility - it breaks the very notion of a number at
point.  A number at point should be a number as delimited
and distinguished in the current mode.

The longstanding definition uses Lisp `read', so it distinguishes
a _Lisp_ number.  It uses what Lisp uses to delimit a numeral.

A better implementation of `number-at-point' than what has
always existed would do this:

1. Get (thing-at-point 'sexp)
2. If it is not a string, return nil.
3. Else match it against a regexp that tests for a numeral
   in the current mode/context.  Or use another such test
   other than regexp matching.  If there the mode/context
   defines numeric syntax then perhaps use a function that
   tests that way.
4. For Lisp, the result must coincide with the longstanding
   behavior, one way or another.

Unless `number-at-point' is extended in such a way, it should
simply be restored to what it has always been.  It's behavior
in Lisp should in any case be to return a Lisp number.

I think that its behavior should remain what it was (a Lisp
number), and any more general function that does as just
described (1 to 4) should be given another name.

FWIW, this is how I define `hex-number-at-point'.  It does
not take into account the special syntax of any current mode
or other context.  It gets a sexp at point and matches it
against hex digits.  But (thing-at-point 'sexp) can have
different behavior in different modes/contexts.

(defun number-at-point-hex ()
  "Return the number represented by the hex numeral at point.
Return nil if none is found."
  (let ((strg  (tap-thing-at-point 'sexp)))
    (and (stringp strg)
         (string-match-p "\\`[0-9a-fA-F]+\\'" strg)
         (string-to-number strg 16))))

(put 'hex-number 'bounds-of-thing-at-point 'bounds-of-number-at-point-hex)

(defun bounds-of-number-at-point-hex ()
  "Return bounds of number represented by the hex numeral at point.
Return nil if none is found."
  (and (number-at-point-hex)  (bounds-of-thing-at-point 'sexp)))

> >In Lisp, at least, there is no number at point, in `foo-2'.
> >That is, the Lisp parser (reader) would never pick up the
> >`2' as a number here.
>
> The doc string of `list-at-point' clearly says that the list should
> be a Lisp list.  In the case of `number-at-point' the doc string
> just says 'the number at point', without connection with the Lisp
> parser.

See above.  It has always been a Lisp number - it has always
used `form-at-point' and hence the Lisp function `read'.
Things in thingatpt.el were Lisp from the beginning.  If the
doc doesn't always call out that behavior then it can be updated
to do so.

> >IOW, I agree with the bug report that `form-at-point'
> >should - somehow - handle the case where `thing-at-point'
> >returns a non-string.  There is a bug to be fixed.  But
> >I'm not convinced that the fix we've implemented is TRT.
>
> Sure, I am open to alternative fixes.

See above for one suggestion.  But what is a _general_ number
at point?

Ideally, `number-at-point' should depend on the given context
(e.g. mode - how numerals are recognized and represented in
the given programming language or whatever.  But what about a
default behavior, for a mode that has no notion of numeral?

You can perhaps look to what I've done for decimal and hex
numbers: get a sexp at point and check that it is a string
that matches a regexp that specifies a decimal or hex numeral.
(`sexp' is mode-dependent, of course.)

The definitions I use are pretty simple.  They don't try to
take into account signs or decimal points, for instance.
We can do better, no doubt.  We should have different kinds
of `*-number-at-point' functions, for grabbing numbers that
are decimal integers and natural numbers, numerals in exponential
notation, hex, octal, etc.

But the basic `number-at-point' should be, well basic.  It should
not try to recognize all of the above.  It should just recognize,
for example, decimal numerals that use simple notation.

We could do worse than use Lisp numerals as the basic number
notation, I think.  And in any case, the current mode should
decide what delimits a number.  I do not want to see our
basic `number-at-point' pick up 3 or -3 from the middle of
`foo-3-bar' or -1 from `-'.  Such behavior flies in the face
of what thingatpt is about, IMO.

bug#24969: 26.0.50; number-at-point

[Andreas Röhler]

On 22.11.2016 00:07, Drew Adams wrote:
>>> In Emacs 25.1 (emacs -Q), `number-at-point' at either
>>> the `-' or the `1' returns nil, for me.  And I do not
>>> see why it should return a number.
>>>
>>> `number-at-point' is defined using `form-at-point' with
>>> THING `sexp' and predicate `numberp'.  The sexp picked
>>> up at point is `foo-1', and that fails `numberp'.
>>  From the time when i opened Bug#24605, the
>> implementation, in master branch, of `number-at-point'
>> was different: it changed in commit 786ab4a5 (Bug#8634).
>> My patch was driven by this implementation.  I didn't notice
>> that `number-at-point' behaves different in emacs-25.
> It behaves the same in Emacs 25.1 as previously.  If something
> broke this after Emacs 25.1 then it should be reverted.
>
>>> What am I missing?  Why should this rightfully return
>>> a number?  I'm guessing that you are all using a more
>>> recent version of `number-at-point' than what is in
>>> Emacs 25.1 (?).  But to me the Emacs 25.1 behavior I
>>> see (i.e., returning nil) is correct.
>>>
>>> Did someone change the meaning of `number-at-point'
>>> so that it now picks up a numeral that is not isolated?
>>> If so, why would that be considered proper behavior?
>>> At the very least it is not backward-compatible behavior.
>> That's right.  Commit above breaks backward-compatibility.
> If it returns a number for point on a numeral in the middle
> of a symbol name etc. then it breaks not only backward
> compatibility - it breaks the very notion of a number at
> point.  A number at point should be a number as delimited
> and distinguished in the current mode.
>
> The longstanding definition uses Lisp `read', so it distinguishes
> a _Lisp_ number.  It uses what Lisp uses to delimit a numeral.
>
> A better implementation of `number-at-point' than what has
> always existed would do this:
>
> 1. Get (thing-at-point 'sexp)
> 2. If it is not a string, return nil.
> 3. Else match it against a regexp that tests for a numeral
>     in the current mode/context.  Or use another such test
>     other than regexp matching.  If there the mode/context
>     defines numeric syntax then perhaps use a function that
>     tests that way.
> 4. For Lisp, the result must coincide with the longstanding
>     behavior, one way or another.
>
> Unless `number-at-point' is extended in such a way, it should
> simply be restored to what it has always been.  It's behavior
> in Lisp should in any case be to return a Lisp number.
>

Why? A lisp-number is useful for further computations, for adding them 
for example.
It's not useful when editing the numbers text at point.

As (thing-at-point 'sexp) internally will get the buffer-substring with 
the number's literal, why not make is accessible for edits?

bug#24969: 26.0.50; number-at-point

[Lars Ingebrigtsen]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> After evaluating show-number in buffer of contents below, it should
> jump to char "1" and return them, but returns nil.
>
> ;;;
>
> foo-1
>
> (defun show-number ()
>   (interactive)
>   (goto-char 5)
>   (message "%s" (number-at-point)))

This has returned -1 since Emacs 26.1 (it used to return nil), and I
think -1 is fine here (since we've asked for a number, and that's a
kinda numberish thing).  (I also see the logic in returning nil in Lisp
buffers, but number-at-point is a DWIM-ish command.)

So I think this works as it's supposed, and I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no