bug#31927: 26.1; Fragment/bookmark part of URL omitted in default offered by `eww'

bug#31927: 26.1; Fragment/bookmark part of URL omitted in default offered by `eww'

[N. Jackson]

When `eww' is invoked interactively with point in an URL, that URL is
offered in the minibuffer as the default web page to open. However, if
the URL has a fragment (aka bookmark) part, the fragment part is ignored
by EWW.

For example:

  0. $ emacs -Q

  1. In scratch type https://en.wikipedia.org/wiki/Midsummer#History

  2. Move point back into the just-typed URL

  3. M-x eww RET

Expected result:
EWW offers https://en.wikipedia.org/wiki/Midsummer#History as the
default.

Actual result:
EWW offers https://en.wikipedia.org/wiki/Midsummer as the default.

Note that the following works just fine, though:

  0. $ emacs -Q

  1. M-x eww RET https://en.wikipedia.org/wiki/Midsummer#History RET

This problem is also present on master
(5583e6460c38c5d613e732934b066421349a5259).

bug#31927: 26.1; Fragment/bookmark part of URL omitted in default offered by `eww'

[Eli Zaretskii]

> From: nljlistbox2@gmail.com (N. Jackson)
> Date: Thu, 21 Jun 2018 11:05:13 -0400
> 
> 
> When `eww' is invoked interactively with point in an URL, that URL is
> offered in the minibuffer as the default web page to open. However, if
> the URL has a fragment (aka bookmark) part, the fragment part is ignored
> by EWW.
> 
> For example:
> 
>   0. $ emacs -Q
> 
>   1. In scratch type https://en.wikipedia.org/wiki/Midsummer#History
> 
>   2. Move point back into the just-typed URL
> 
>   3. M-x eww RET
> 
> Expected result:
> EWW offers https://en.wikipedia.org/wiki/Midsummer#History as the
> default.
> 
> Actual result:
> EWW offers https://en.wikipedia.org/wiki/Midsummer as the default.

You could customize eww-suggest-uris to add, say
thing-at-point-url-at-point.  I think this should give you what you
want.

bug#31927: 26.1; Fragment/bookmark part of URL omitted in default offered by `eww'

[N. Jackson]

At 18:47 +0300 on Thursday 2018-06-21, Eli Zaretskii wrote:
>
>> From: nljlistbox2@gmail.com (N. Jackson)
>> 
>>   1. In scratch type https://en.wikipedia.org/wiki/Midsummer#History
>> 
>>   2. Move point back into the just-typed URL
>> 
>>   3. M-x eww RET
>> 
>> Expected result:
>> EWW offers https://en.wikipedia.org/wiki/Midsummer#History as the
>> default.
>> 
>> Actual result:
>> EWW offers https://en.wikipedia.org/wiki/Midsummer as the default.
>
> You could customize eww-suggest-uris to add, say
> thing-at-point-url-at-point. I think this should give you what you
> want.

Thank you for the suggestion. But, no, that doesn't quite help.

By default the value of `eww-suggest-uris' is `(eww-links-at-point
url-get-url-at-point eww-current-url)'.

When I add `thing-at-point-url-at-point' it's added to the end of the
list and the behaviour is unchanged presumably because
`url-get-url-at-point' comes first.

However if I set `eww-suggest-uris' to just
`(thing-at-point-url-at-point)', then I see the "correct" behaviour --
that is, the fragment/bookmark part of the URL is picked up.

So the question is, why does `thing-at-point-url-at-point' pick up the
fragment/bookmark part or the URL but `url-get-url-at-point' does not?

That is, removing EWW from the picture altogether:

  0. $ emacs -Q

  1. On a new line in scratch type https://en.wikipedia.org/wiki/Midsummer#History

  2. C-a	; Move point to beginning of the just-typed URL.

  3. M-: (thing-at-point-url-at-point) RET
  => https://en.wikipedia.org/wiki/Midsummer#History

  4. M-: (url-get-url-at-point) RET
  => https://en.wikipedia.org/wiki/Midsummer

Is it expected for these two functions to behave differently here?

(And an additional anomaly: In Step 2, if point is just moved back one
character into the just-typed URL (between the `r' and `y' in
"#History") then:

  3. M-: (thing-at-point-url-at-point) RET
  => https://en.wikipedia.org/wiki/Midsummer#History

  4. M-: (url-get-url-at-point) RET
  => nil

).

bug#31927: 26.1; Fragment/bookmark part of URL omitted in default offered by `eww'

[Eli Zaretskii]

> From: nljlistbox2@gmail.com (N. Jackson)
> Cc: 31927@debbugs.gnu.org
> Date: Thu, 21 Jun 2018 19:11:23 -0400
> 
> So the question is, why does `thing-at-point-url-at-point' pick up the
> fragment/bookmark part or the URL but `url-get-url-at-point' does not?
> 
> That is, removing EWW from the picture altogether:
> 
>   0. $ emacs -Q
> 
>   1. On a new line in scratch type https://en.wikipedia.org/wiki/Midsummer#History
> 
>   2. C-a	; Move point to beginning of the just-typed URL.
> 
>   3. M-: (thing-at-point-url-at-point) RET
>   => https://en.wikipedia.org/wiki/Midsummer#History
> 
>   4. M-: (url-get-url-at-point) RET
>   => https://en.wikipedia.org/wiki/Midsummer
> 
> Is it expected for these two functions to behave differently here?

Yes, that's the question.  I hope Lars (CC'ed) could comment on that.

bug#31927: 26.1; Fragment/bookmark part of URL omitted in default offered by `eww'

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>>   1. On a new line in scratch type
>> https://en.wikipedia.org/wiki/Midsummer#History
>> 
>>   2. C-a	; Move point to beginning of the just-typed URL.
>> 
>>   3. M-: (thing-at-point-url-at-point) RET
>>   => https://en.wikipedia.org/wiki/Midsummer#History
>> 
>>   4. M-: (url-get-url-at-point) RET
>>   => https://en.wikipedia.org/wiki/Midsummer
>> 
>> Is it expected for these two functions to behave differently here?
>
> Yes, that's the question.  I hope Lars (CC'ed) could comment on that.

I think the behaviour of `url-get-url-at-point' is suboptimal here.
Perhaps we should just make eww use `thing-at-point-url-at-point'
instead?  And mark `url-get-url-at-point' as obsolete, as it doesn't
seem very actively maintained and duplicates thingatpt functionality.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#31927: 26.1; Fragment/bookmark part of URL omitted in default offered by `eww'

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: nljlistbox2@gmail.com (N. Jackson),  31927@debbugs.gnu.org
> Date: Sun, 22 Jul 2018 15:10:42 +0200
> 
> >>   3. M-: (thing-at-point-url-at-point) RET
> >>   => https://en.wikipedia.org/wiki/Midsummer#History
> >> 
> >>   4. M-: (url-get-url-at-point) RET
> >>   => https://en.wikipedia.org/wiki/Midsummer
> >> 
> >> Is it expected for these two functions to behave differently here?
> >
> > Yes, that's the question.  I hope Lars (CC'ed) could comment on that.
> 
> I think the behaviour of `url-get-url-at-point' is suboptimal here.
> Perhaps we should just make eww use `thing-at-point-url-at-point'
> instead?  And mark `url-get-url-at-point' as obsolete, as it doesn't
> seem very actively maintained and duplicates thingatpt functionality.

Probably.  Unless some functionality really needs to ignore the parts
after the hash, for some reason.

bug#31927: 26.1; Fragment/bookmark part of URL omitted in default offered by `eww'

[Noam Postavsky]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Lars Ingebrigtsen <larsi@gnus.org>
>> Cc: nljlistbox2@gmail.com (N. Jackson),  31927@debbugs.gnu.org
>> Date: Sun, 22 Jul 2018 15:10:42 +0200
>> 
>> >>   3. M-: (thing-at-point-url-at-point) RET
>> >>   => https://en.wikipedia.org/wiki/Midsummer#History
>> >> 
>> >>   4. M-: (url-get-url-at-point) RET
>> >>   => https://en.wikipedia.org/wiki/Midsummer
>> >> 
>> >> Is it expected for these two functions to behave differently here?
>> >
>> > Yes, that's the question.  I hope Lars (CC'ed) could comment on that.
>> 
>> I think the behaviour of `url-get-url-at-point' is suboptimal here.
>> Perhaps we should just make eww use `thing-at-point-url-at-point'
>> instead?  And mark `url-get-url-at-point' as obsolete, as it doesn't
>> seem very actively maintained and duplicates thingatpt functionality.
>
> Probably.  Unless some functionality really needs to ignore the parts
> after the hash, for some reason.

If it's really needed, then it's probably better to parse the url and
explicitly remove unwanted parts.  I notice this bug is pretty similar
to Bug#898 "ffap -> emacs-w3m dropping of #anchor"; it seems pretty
clear that dropping the part after the hash is the wrong thing in web
browsing contexts.

bug#31927: 26.1; Fragment/bookmark part of URL omitted in default offered by `eww'

[N. Jackson]

At 11:57 -0400 on Sunday 2018-07-29, Noam Postavsky wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Lars Ingebrigtsen <larsi@gnus.org>
>>> Cc: nljlistbox2@gmail.com (N. Jackson),  31927@debbugs.gnu.org
>>> Date: Sun, 22 Jul 2018 15:10:42 +0200
>>> 
>>> >>   3. M-: (thing-at-point-url-at-point) RET
>>> >>   => https://en.wikipedia.org/wiki/Midsummer#History
>>> >> 
>>> >>   4. M-: (url-get-url-at-point) RET
>>> >>   => https://en.wikipedia.org/wiki/Midsummer
>>> >> 
>>> >> Is it expected for these two functions to behave
>>> >> differently here?
>>> >
>>> > Yes, that's the question. I hope Lars (CC'ed) could comment
>>> > on that.
>>> 
>>> I think the behaviour of `url-get-url-at-point' is suboptimal
>>> here. Perhaps we should just make eww use
>>> `thing-at-point-url-at-point' instead? And mark
>>> `url-get-url-at-point' as obsolete, as it doesn't seem very
>>> actively maintained and duplicates thingatpt functionality.
>>
>> Probably. Unless some functionality really needs to ignore the
>> parts after the hash, for some reason.
>
> If it's really needed, then it's probably better to parse the
> url and explicitly remove unwanted parts. I notice this bug is
> pretty similar to Bug#898 "ffap -> emacs-w3m dropping of
> #anchor"; it seems pretty clear that dropping the part after the
> hash is the wrong thing in web browsing contexts.

FWIW, in the Emacs codebase at least, `url-get-url-at-point' is
only used twice, both in eww.el:

    $ grep -rHn 'url-get-url-at-point' *
    ChangeLog.3:21009:      * lisp/url/url-util.el (url-get-url-at-point): Add missing group
    lisp/loaddefs.el~:11861:(defvar eww-suggest-uris '(eww-links-at-point url-get-url-at-point eww-current-url) "\
    Binary file lisp/net/eww.elc matches
    lisp/net/eww.el:32:(require 'url-util)                  ; for url-get-url-at-point
--> lisp/net/eww.el:67:    url-get-url-at-point
--> lisp/net/eww.el:77:             url-get-url-at-point
    Binary file lisp/url/url-util.elc matches
    lisp/url/url-util.el:544:(defun url-get-url-at-point (&optional pt)
    lisp/url/ChangeLog.1:742:       (url-get-url-at-point): Don't use eval-when-compile.
    lisp/url/ChangeLog.1:2799:      * url-util.el (url-get-url-at-point): Guard against 'url'
    lisp/url/ChangeLog.1:2859:      * url-util.el (url-get-url-at-point): Allow URLs wrapped in
    lisp/url/ChangeLog.1:2977:      * url-util.el (url-get-url-at-point): Re-integrated.
    lisp/loaddefs.el:11861:(defvar eww-suggest-uris '(eww-links-at-point url-get-url-at-point eww-current-url) "\
    lisp/ldefs-boot.el:11763:(defvar eww-suggest-uris '(eww-links-at-point url-get-url-at-point eww-current-url) "\
    Binary file src/emacs-master-2017-05-31 matches
    Binary file src/emacs matches
    Binary file src/emacs-25.3-2017-09-12 matches
    Binary file src/emacs-head-2017-12-29 matches
    Binary file src/emacs-27.0.50.1 matches
    Binary file src/emacs-master-2016-12-01 matches
    Binary file src/bootstrap-emacs matches

bug#31927: 26.1; Fragment/bookmark part of URL omitted in default offered by `eww'

[N. Jackson]

At 13:16 -0400 on Sunday 2018-07-29, N. Jackson wrote:
>
> FWIW, in the Emacs codebase at least, `url-get-url-at-point' is
> only used twice, both in eww.el:
>
> --> lisp/net/eww.el:67:    url-get-url-at-point
> --> lisp/net/eww.el:77:             url-get-url-at-point

And in emacs/elpa `url-get-url-at-point' is nowhere used.

bug#31927: 26.1; Fragment/bookmark part of URL omitted in default offered by `eww'

[Lars Ingebrigtsen]

"N. Jackson" <nljlistbox2@gmail.com> writes:

> At 13:16 -0400 on Sunday 2018-07-29, N. Jackson wrote:
>>
>> FWIW, in the Emacs codebase at least, `url-get-url-at-point' is
>> only used twice, both in eww.el:
>>
>> --> lisp/net/eww.el:67:    url-get-url-at-point
>> --> lisp/net/eww.el:77:             url-get-url-at-point
>
> And in emacs/elpa `url-get-url-at-point' is nowhere used.

OK; I've now switched eww to thing-at-point-url-at-point and marked the
url.el function as obsolete in Emacs 27.1.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no