bug#24969: 26.0.50; number-at-point

[Drew Adams <drew.adams@oracle.com>]

> > In Emacs 25.1 (emacs -Q), `number-at-point' at either
> > the `-' or the `1' returns nil, for me.  And I do not
> > see why it should return a number.
> >
> > `number-at-point' is defined using `form-at-point' with
> > THING `sexp' and predicate `numberp'.  The sexp picked
> > up at point is `foo-1', and that fails `numberp'.
>
> From the time when i opened Bug#24605, the
> implementation, in master branch, of `number-at-point'
> was different: it changed in commit 786ab4a5 (Bug#8634).
> My patch was driven by this implementation.  I didn't notice
> that `number-at-point' behaves different in emacs-25.

It behaves the same in Emacs 25.1 as previously.  If something
broke this after Emacs 25.1 then it should be reverted.

> > What am I missing?  Why should this rightfully return
> > a number?  I'm guessing that you are all using a more
> > recent version of `number-at-point' than what is in
> > Emacs 25.1 (?).  But to me the Emacs 25.1 behavior I
> > see (i.e., returning nil) is correct.
> >
> > Did someone change the meaning of `number-at-point'
> > so that it now picks up a numeral that is not isolated?
> > If so, why would that be considered proper behavior?
> > At the very least it is not backward-compatible behavior.
>
> That's right.  Commit above breaks backward-compatibility.

If it returns a number for point on a numeral in the middle
of a symbol name etc. then it breaks not only backward
compatibility - it breaks the very notion of a number at
point.  A number at point should be a number as delimited
and distinguished in the current mode.

The longstanding definition uses Lisp `read', so it distinguishes
a _Lisp_ number.  It uses what Lisp uses to delimit a numeral.

A better implementation of `number-at-point' than what has
always existed would do this:

1. Get (thing-at-point 'sexp)
2. If it is not a string, return nil.
3. Else match it against a regexp that tests for a numeral
   in the current mode/context.  Or use another such test
   other than regexp matching.  If there the mode/context
   defines numeric syntax then perhaps use a function that
   tests that way.
4. For Lisp, the result must coincide with the longstanding
   behavior, one way or another.

Unless `number-at-point' is extended in such a way, it should
simply be restored to what it has always been.  It's behavior
in Lisp should in any case be to return a Lisp number.

I think that its behavior should remain what it was (a Lisp
number), and any more general function that does as just
described (1 to 4) should be given another name.

FWIW, this is how I define `hex-number-at-point'.  It does
not take into account the special syntax of any current mode
or other context.  It gets a sexp at point and matches it
against hex digits.  But (thing-at-point 'sexp) can have
different behavior in different modes/contexts.

(defun number-at-point-hex ()
  "Return the number represented by the hex numeral at point.
Return nil if none is found."
  (let ((strg  (tap-thing-at-point 'sexp)))
    (and (stringp strg)
         (string-match-p "\\`[0-9a-fA-F]+\\'" strg)
         (string-to-number strg 16))))

(put 'hex-number 'bounds-of-thing-at-point 'bounds-of-number-at-point-hex)

(defun bounds-of-number-at-point-hex ()
  "Return bounds of number represented by the hex numeral at point.
Return nil if none is found."
  (and (number-at-point-hex)  (bounds-of-thing-at-point 'sexp)))

> >In Lisp, at least, there is no number at point, in `foo-2'.
> >That is, the Lisp parser (reader) would never pick up the
> >`2' as a number here.
>
> The doc string of `list-at-point' clearly says that the list should
> be a Lisp list.  In the case of `number-at-point' the doc string
> just says 'the number at point', without connection with the Lisp
> parser.

See above.  It has always been a Lisp number - it has always
used `form-at-point' and hence the Lisp function `read'.
Things in thingatpt.el were Lisp from the beginning.  If the
doc doesn't always call out that behavior then it can be updated
to do so.

> >IOW, I agree with the bug report that `form-at-point'
> >should - somehow - handle the case where `thing-at-point'
> >returns a non-string.  There is a bug to be fixed.  But
> >I'm not convinced that the fix we've implemented is TRT.
>
> Sure, I am open to alternative fixes.

See above for one suggestion.  But what is a _general_ number
at point?

Ideally, `number-at-point' should depend on the given context
(e.g. mode - how numerals are recognized and represented in
the given programming language or whatever.  But what about a
default behavior, for a mode that has no notion of numeral?

You can perhaps look to what I've done for decimal and hex
numbers: get a sexp at point and check that it is a string
that matches a regexp that specifies a decimal or hex numeral.
(`sexp' is mode-dependent, of course.)

The definitions I use are pretty simple.  They don't try to
take into account signs or decimal points, for instance.
We can do better, no doubt.  We should have different kinds
of `*-number-at-point' functions, for grabbing numbers that
are decimal integers and natural numbers, numerals in exponential
notation, hex, octal, etc.

But the basic `number-at-point' should be, well basic.  It should
not try to recognize all of the above.  It should just recognize,
for example, decimal numerals that use simple notation.

We could do worse than use Lisp numerals as the basic number
notation, I think.  And in any case, the current mode should
decide what delimits a number.  I do not want to see our
basic `number-at-point' pick up 3 or -3 from the middle of
`foo-3-bar' or -1 from `-'.  Such behavior flies in the face
of what thingatpt is about, IMO.