unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / Atom feed
* Mummi wishlist: API using Message-ID
@ 2020-09-16 23:51 zimoun
  2020-09-17 10:01 ` Ricardo Wurmus
  0 siblings, 1 reply; 7+ messages in thread
From: zimoun @ 2020-09-16 23:51 UTC (permalink / raw)
  To: Guix Devel

Dear,

Today <issues.guix.gnu.org> is really useful and IMHO better than the
classic Debbugs interface [1].

Bug reports (and patches) are still sent by email, the core of the
Debbugs bug tracker.  However, Mummi does not use the unique identifier
Message-ID as address, AFAICT.  Therefore, it can be tricky to link
between my emails reader/writer and the web interface.

Let take an example.  From my typical use case. :-)

My subscription to bug-guix (or guix-patches) sends me emails and I read
them with emacs-notmuch (but it is not Emacs related, since any good
email reader seems able to do the same).

When I want to refer to one specific message, I have to open my web
browser and go to <issues.guix.gnu.org/1234> then scroll a bit to find
the relevant message, click to the date to get the associated number,
copy the URL and go back to my email writer and finally paste the link.

For example <http://issues.guix.gnu.org/33899#16>.  The number #16 is
really difficult to guess. :-)

Instead, it is easy to get the Message-ID.  (Using emacs-notmuch, only
hit the key ’ci’) Therefore, it could be nice to be able to provide
e.g., the URL:

<issues.guix.gnu.org/HAv_FWZRa_dCADY6U07oX-E2vTrtcxk8ltkpfvTpYCsIh4O5PuItHGIh6w-g5GGmoqVEfkjDUOplpUErdGEPAiFYdrjjDhwd9RJ4OyjjGgY=@hector.link>

redirecting to <http://issues.guix.gnu.org/33899#16>.  And maybe the
current webpage could provide the Message-ID, easy to copy and then
paste in my email reader.


Another (Emacs related) example is emacs-debbugs.
  C-u M-x debbugs-gnu RET guix RET n y
  M-x debbugs-gnu-bugs RET 33899 RET RET
  Read messages

Well, in the relevant message, I run “M-x debbugs->url” and get the
ready-to-be-pasted URL:

<https://yhetil.org/guix-bugs/HAv_FWZRa_dCADY6U07oX-E2vTrtcxk8ltkpfvTpYCsIh4O5PuItHGIh6w-g5GGmoqVEfkjDUOplpUErdGEPAiFYdrjjDhwd9RJ4OyjjGgY=@hector.link>

with the naive Emacs function:

--8<---------------cut here---------------start------------->8---
(defun debbugs->url ()
  (interactive)
  (let ((url "https://yhetil.org/guix-bugs/")
        mid)
    (with-current-buffer gnus-article-buffer
      (gnus-summary-toggle-header 1)
      (setq mid (message-fetch-field "Message-ID"))
      (gnus-summary-toggle-header -1))
    (while (string-match "[<>]" mid)
      (setq mid (replace-match "" t t mid)))
    (kill-new (concat url mid))
    (message mid)))
--8<---------------cut here---------------end--------------->8---



Last and off-topic to the Subject :-), the tool ’public-inbox’ exposes
such API and then it seems easier to refer to relevant message than
going by hand on e.g. <https://lists.gnu.org/archive/html/guix-devel/>
to find it.

An unofficial (but almost? serving org-mode list [2]) example is
demonstrated here:

   <https://yhetil.org/>

Does it make to sense to have something similar,
e.g. <lists.guix.gnu.org> serving the ’public-inbox’?


[1] https://debbugs.gnu.org/cgi/pkgreport.cgi?package=guix;max-bugs=100;base-order=1;bug-rev=1
[2] <https://orgmode.org/list/87r1vd92eg.fsf@bzg.fr/>


All the best,
simon


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Mummi wishlist: API using Message-ID
  2020-09-16 23:51 Mummi wishlist: API using Message-ID zimoun
@ 2020-09-17 10:01 ` Ricardo Wurmus
  2020-09-17 10:38   ` zimoun
  0 siblings, 1 reply; 7+ messages in thread
From: Ricardo Wurmus @ 2020-09-17 10:01 UTC (permalink / raw)
  To: zimoun; +Cc: Guix Devel


zimoun <zimon.toutoune@gmail.com> writes:

> For example <http://issues.guix.gnu.org/33899#16>.  The number #16 is
> really difficult to guess. :-)

Yes, it comes from the number of messages we got from Debbugs.

> Instead, it is easy to get the Message-ID.  (Using emacs-notmuch, only
> hit the key ’ci’) Therefore, it could be nice to be able to provide
> e.g., the URL:
>
> <issues.guix.gnu.org/HAv_FWZRa_dCADY6U07oX-E2vTrtcxk8ltkpfvTpYCsIh4O5PuItHGIh6w-g5GGmoqVEfkjDUOplpUErdGEPAiFYdrjjDhwd9RJ4OyjjGgY=@hector.link>
>
> redirecting to <http://issues.guix.gnu.org/33899#16>.

This is an interesting idea!  I don’t know if it’ll work as a plain URL,
because not all characters of a message id might be usable in a URL (you
may need to URL-encode it and can’t just paste it in the URL bar of your
browser), but it would certainly work as a search query.  The only
problem is that we’re not currently indexing the message ids.

Per bug we index the bug id, the submitter, the authors, the subject(s),
the owner, the severity, the tags, the status, the file name of the
original Debbugs “.log” file (I forgot why!), and all the message
bodies.  All of these map to the very same “document” (in Xapian
parlance), which is a particular “issue”.

I think we could also index the message id (and expose it to the
search), map that to the “issue”, and then find the corresponding anchor
in the HTML later.  We wouldn’t be able to *directly* map the message id
to the actual message in the thread, because the search does not operate
on the concept of messages but only whole bug discussions.  But I think
this would get us very close to what you propose.

> And maybe the
> current webpage could provide the Message-ID, easy to copy and then
> paste in my email reader.

It’s already there but hidden!  Every message is rendered in a
div.message.  Inside of that is div.card > div.card-header > div.details
(hidden) > div.message-id.

With custom CSS you could unhide div.details, and with a custom JS you
could grab the contents of div.message-id on the click of a button.
Most browsers make it a little hard to augment the CSS and/or JS of a
served page, but with a little hacking I’m sure you can extract what you
want.

>
> Another (Emacs related) example is emacs-debbugs.
>   C-u M-x debbugs-gnu RET guix RET n y
>   M-x debbugs-gnu-bugs RET 33899 RET RET
>   Read messages
>
> Well, in the relevant message, I run “M-x debbugs->url” and get the
> ready-to-be-pasted URL:
>
> <https://yhetil.org/guix-bugs/HAv_FWZRa_dCADY6U07oX-E2vTrtcxk8ltkpfvTpYCsIh4O5PuItHGIh6w-g5GGmoqVEfkjDUOplpUErdGEPAiFYdrjjDhwd9RJ4OyjjGgY=@hector.link>
>
> with the naive Emacs function:
>
> --8<---------------cut here---------------start------------->8---
> (defun debbugs->url ()
>   (interactive)
>   (let ((url "https://yhetil.org/guix-bugs/")
>         mid)
>     (with-current-buffer gnus-article-buffer
>       (gnus-summary-toggle-header 1)
>       (setq mid (message-fetch-field "Message-ID"))
>       (gnus-summary-toggle-header -1))
>     (while (string-match "[<>]" mid)
>       (setq mid (replace-match "" t t mid)))
>     (kill-new (concat url mid))
>     (message mid)))
> --8<---------------cut here---------------end--------------->8---
>
>
>
> Last and off-topic to the Subject :-), the tool ’public-inbox’ exposes
> such API and then it seems easier to refer to relevant message than
> going by hand on e.g. <https://lists.gnu.org/archive/html/guix-devel/>
> to find it.

I don’t quite understand how this differs from this:

https://lists.gnu.org/archive/cgi-bin/namazu.cgi?query=%3C86sgbhz3fe.fsf%40gmail.com%3E&submit=Search%21&idxname=guix-devel

Here I search for the id of your message (the one I’m responding to);
that’s the (URL-encoded) value of “query”, “idxname” is for the mailing
list to be searched.

It gives you a list of results, not one message directly, but I think
it’s pretty close. 

-- 
Ricardo


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Mummi wishlist: API using Message-ID
  2020-09-17 10:01 ` Ricardo Wurmus
@ 2020-09-17 10:38   ` zimoun
  2020-09-17 11:33     ` Ricardo Wurmus
  2020-09-18  3:53     ` Kyle Meyer
  0 siblings, 2 replies; 7+ messages in thread
From: zimoun @ 2020-09-17 10:38 UTC (permalink / raw)
  To: Ricardo Wurmus; +Cc: Guix Devel

(Sorry for the misspelling of Mumi in the Subject thread.)

(For the record, reading [1], I discovered [2,3] with similar goals
than Mumi and that the Org-mode community uses public-inbox [4].)

[1] https://bzg.fr/en/org-mode-9.4-is-out-can-you-help.html/
[2] https://updates.orgmode.org/
[3] https://github.com/bzg/woof
[4] https://orgmode.org/list/87r1vd92eg.fsf@bzg.fr/

On Thu, 17 Sep 2020 at 12:00, Ricardo Wurmus <rekado@elephly.net> wrote:
> zimoun <zimon.toutoune@gmail.com> writes:

> > Instead, it is easy to get the Message-ID.  (Using emacs-notmuch, only
> > hit the key ’ci’) Therefore, it could be nice to be able to provide
> > e.g., the URL:
> >
> > <issues.guix.gnu.org/HAv_FWZRa_dCADY6U07oX-E2vTrtcxk8ltkpfvTpYCsIh4O5PuItHGIh6w-g5GGmoqVEfkjDUOplpUErdGEPAiFYdrjjDhwd9RJ4OyjjGgY=@hector.link>
> >
> > redirecting to <http://issues.guix.gnu.org/33899#16>.
>
> This is an interesting idea!  I don’t know if it’ll work as a plain URL,
> because not all characters of a message id might be usable in a URL (you
> may need to URL-encode it and can’t just paste it in the URL bar of your
> browser), but it would certainly work as a search query.  The only
> problem is that we’re not currently indexing the message ids.

I do not know but public-inbox is doing such thing, isn't it?
I mean, the example with emacs-debbugs, my "custom" function and the
service <yhetil.org>:

> > Another (Emacs related) example is emacs-debbugs.
> >   C-u M-x debbugs-gnu RET guix RET n y
> >   M-x debbugs-gnu-bugs RET 33899 RET RET
> >   Read messages
> >
> > Well, in the relevant message, I run “M-x debbugs->url” and get the
> > ready-to-be-pasted URL:
> >
> > <https://yhetil.org/guix-bugs/HAv_FWZRa_dCADY6U07oX-E2vTrtcxk8ltkpfvTpYCsIh4O5PuItHGIh6w-g5GGmoqVEfkjDUOplpUErdGEPAiFYdrjjDhwd9RJ4OyjjGgY=@hector.link>
> >
> > with the naive Emacs function:
> >
> > --8<---------------cut here---------------start------------->8---
> > (defun debbugs->url ()
> >   (interactive)
> >   (let ((url "https://yhetil.org/guix-bugs/")
> >         mid)
> >     (with-current-buffer gnus-article-buffer
> >       (gnus-summary-toggle-header 1)
> >       (setq mid (message-fetch-field "Message-ID"))
> >       (gnus-summary-toggle-header -1))
> >     (while (string-match "[<>]" mid)
> >       (setq mid (replace-match "" t t mid)))
> >     (kill-new (concat url mid))
> >     (message mid)))
> > --8<---------------cut here---------------end--------------->8---

Is it not enough?
The URL,  is it not always ready-to-be-pasted?
The link contains '@' but it works (on my webbrowser at least :-)).


> Per bug we index the bug id, the submitter, the authors, the subject(s),
> the owner, the severity, the tags, the status, the file name of the
> original Debbugs “.log” file (I forgot why!), and all the message
> bodies.  All of these map to the very same “document” (in Xapian
> parlance), which is a particular “issue”.

Thank you for explaining.


> I think we could also index the message id (and expose it to the
> search), map that to the “issue”, and then find the corresponding anchor
> in the HTML later.  We wouldn’t be able to *directly* map the message id
> to the actual message in the thread, because the search does not operate
> on the concept of messages but only whole bug discussions.  But I think
> this would get us very close to what you propose.

Yes.
If the Message-ID is also indexed, the map is almost there.


> > And maybe the
> > current webpage could provide the Message-ID, easy to copy and then
> > paste in my email reader.
>
> It’s already there but hidden!  Every message is rendered in a
> div.message.  Inside of that is div.card > div.card-header > div.details
> (hidden) > div.message-id.
>
> With custom CSS you could unhide div.details, and with a custom JS you
> could grab the contents of div.message-id on the click of a button.
> Most browsers make it a little hard to augment the CSS and/or JS of a
> served page, but with a little hacking I’m sure you can extract what you
> want.

Okish... CSS/JS is not really my cup of tea.  I will try to rehash
your words when giving a look at Mumi source.


> > Last and off-topic to the Subject :-), the tool ’public-inbox’ exposes
> > such API and then it seems easier to refer to relevant message than
> > going by hand on e.g. <https://lists.gnu.org/archive/html/guix-devel/>
> > to find it.
>
> I don’t quite understand how this differs from this:
>
> https://lists.gnu.org/archive/cgi-bin/namazu.cgi?query=%3C86sgbhz3fe.fsf%40gmail.com%3E&submit=Search%21&idxname=guix-devel

Because I did not know. :-)


All the best,
simon


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Mummi wishlist: API using Message-ID
  2020-09-17 10:38   ` zimoun
@ 2020-09-17 11:33     ` Ricardo Wurmus
  2020-09-18  5:44       ` Arun Isaac
  2020-09-18  3:53     ` Kyle Meyer
  1 sibling, 1 reply; 7+ messages in thread
From: Ricardo Wurmus @ 2020-09-17 11:33 UTC (permalink / raw)
  To: zimoun; +Cc: Guix Devel


zimoun <zimon.toutoune@gmail.com> writes:

> (Sorry for the misspelling of Mumi in the Subject thread.)
>
> (For the record, reading [1], I discovered [2,3] with similar goals
> than Mumi and that the Org-mode community uses public-inbox [4].)
>
> [1] https://bzg.fr/en/org-mode-9.4-is-out-can-you-help.html/
> [2] https://updates.orgmode.org/
> [3] https://github.com/bzg/woof
> [4] https://orgmode.org/list/87r1vd92eg.fsf@bzg.fr/
>
> On Thu, 17 Sep 2020 at 12:00, Ricardo Wurmus <rekado@elephly.net> wrote:
>> zimoun <zimon.toutoune@gmail.com> writes:
>
>> > Instead, it is easy to get the Message-ID.  (Using emacs-notmuch, only
>> > hit the key ’ci’) Therefore, it could be nice to be able to provide
>> > e.g., the URL:
>> >
>> > <issues.guix.gnu.org/HAv_FWZRa_dCADY6U07oX-E2vTrtcxk8ltkpfvTpYCsIh4O5PuItHGIh6w-g5GGmoqVEfkjDUOplpUErdGEPAiFYdrjjDhwd9RJ4OyjjGgY=@hector.link>
>> >
>> > redirecting to <http://issues.guix.gnu.org/33899#16>.
>>
>> This is an interesting idea!  I don’t know if it’ll work as a plain URL,
>> because not all characters of a message id might be usable in a URL (you
>> may need to URL-encode it and can’t just paste it in the URL bar of your
>> browser), but it would certainly work as a search query.  The only
>> problem is that we’re not currently indexing the message ids.
>
> I do not know but public-inbox is doing such thing, isn't it?
> I mean, the example with emacs-debbugs, my "custom" function and the
> service <yhetil.org>: […]

It might be fine.  I just don’t know for sure if Message-Ids may contain
characters that are not permitted in URLs.

>> > And maybe the
>> > current webpage could provide the Message-ID, easy to copy and then
>> > paste in my email reader.
>>
>> It’s already there but hidden!  Every message is rendered in a
>> div.message.  Inside of that is div.card > div.card-header > div.details
>> (hidden) > div.message-id.
>>
>> With custom CSS you could unhide div.details, and with a custom JS you
>> could grab the contents of div.message-id on the click of a button.
>> Most browsers make it a little hard to augment the CSS and/or JS of a
>> served page, but with a little hacking I’m sure you can extract what you
>> want.
>
> Okish... CSS/JS is not really my cup of tea.  I will try to rehash
> your words when giving a look at Mumi source.

You can give this a try:

* in Icecat hit F12
* select the Inspector (if it isn’t selected yet)
* in the right column (next to :hov and .cls) there’s a plus icon with
  hover text “Add new rule”; click it
* edit the text to say: “.details { display: block !important }”

This should show you all message ids.  It’s possible to make this
permanent but it’s rather tedious with Icecat/Firefox.

If you just need this for scripting, though, you can parse the HTML and
filter the “div” tag with the “message-id” class.  If scripting is
desired I could probably also offer a JSON endpoint that provides all
information in a format that is easily processed.

-- 
Ricardo


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Mummi wishlist: API using Message-ID
  2020-09-17 10:38   ` zimoun
  2020-09-17 11:33     ` Ricardo Wurmus
@ 2020-09-18  3:53     ` Kyle Meyer
  2020-09-18 16:46       ` Kyle Meyer
  1 sibling, 1 reply; 7+ messages in thread
From: Kyle Meyer @ 2020-09-18  3:53 UTC (permalink / raw)
  To: zimoun; +Cc: Guix Devel

zimoun writes:

> On Thu, 17 Sep 2020 at 12:00, Ricardo Wurmus <rekado@elephly.net> wrote:

>> This is an interesting idea!  I don’t know if it’ll work as a plain URL,
>> because not all characters of a message id might be usable in a URL (you
>> may need to URL-encode it and can’t just paste it in the URL bar of your
>> browser), but it would certainly work as a search query.  The only
>> problem is that we’re not currently indexing the message ids.
>
> I do not know but public-inbox is doing such thing, isn't it?
> I mean, the example with emacs-debbugs, my "custom" function and the
> service <yhetil.org>:
>
[...]
> Is it not enough?
> The URL,  is it not always ready-to-be-pasted?
> The link contains '@' but it works (on my webbrowser at least :-)).

For public-inbox URLs, there is one character that needs to be escaped:
"/" with "%2F" (see <https://yhetil.org/guix-devel/_/text/help/>).

However, in its current form, your custom function has a good chance of
being enough.  Here is the number of message IDs known to
yhetil.org/guix-devel:

    $ echo 'SELECT COUNT(*) FROM msgmap' | \
        sqlite3 -readonly guix-devel-msgmap.sqlite3
    55231

And of those, only 11 have "/" in them:

    $ echo 'SELECT mid FROM msgmap' | \
        sqlite3 -readonly guix-devel-msgmap.sqlite3 | grep -c "/"
    11


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Mummi wishlist: API using Message-ID
  2020-09-17 11:33     ` Ricardo Wurmus
@ 2020-09-18  5:44       ` Arun Isaac
  0 siblings, 0 replies; 7+ messages in thread
From: Arun Isaac @ 2020-09-18  5:44 UTC (permalink / raw)
  To: Ricardo Wurmus, zimoun; +Cc: Guix Devel

[-- Attachment #1: Type: text/plain, Size: 386 bytes --]


> I just don’t know for sure if Message-Ids may contain characters that
> are not permitted in URLs.

According to RFC5322, Message-IDs can contain any ASCII character in the
(decimal) ranges 33-91, 93-126. And, many of these characters are not
permitted in URLs, but perhaps modern browsers automatically percent
encode if the user enters unpermitted characters in the URL.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 524 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Mummi wishlist: API using Message-ID
  2020-09-18  3:53     ` Kyle Meyer
@ 2020-09-18 16:46       ` Kyle Meyer
  0 siblings, 0 replies; 7+ messages in thread
From: Kyle Meyer @ 2020-09-18 16:46 UTC (permalink / raw)
  To: zimoun; +Cc: Guix Devel

Kyle Meyer writes:

> For public-inbox URLs, there is one character that needs to be escaped:
> "/" with "%2F" (see <https://yhetil.org/guix-devel/_/text/help/>).

Correcting myself: I shouldn't take that help page as a complete
description of what needs to be escaped for public-inbox URLs.  It'd be
better to look at what public-inbox's code does itself, particularly
when it generates the path component of the URL for its Archived-At
header.  It feeds the message ID through this function:

,----[ lib/PublicInbox/MID.pm ]
| # RFC3986, section 3.3:
| sub MID_ESC () { '^A-Za-z0-9\-\._~!\$\&\';\(\)\*\+,;=:@' }
| sub mid_escape ($) { uri_escape_utf8($_[0], MID_ESC) }
`----

That code came with 9d1e5fad (www: do not unecessarily escape some chars
in paths, 2016-08-14).

So, if you want to perform the same escaping for your custom Elisp
function that generates public-inbox URLs, you can do something like
this:

    (let ((unreserved-chars
           (append url-unreserved-chars
                   '(?! ?$ ?& ?' ?\( ?\) ?* ?+ ?, ?= ?: ?@ ?\;))))
      (url-hexify-string "88@e.com" unreserved-chars)  ; => "88@e.com"
      (url-hexify-string "8%@e.com" unreserved-chars)  ; => "8%25@e.com"
      (url-hexify-string "8/@e.com" unreserved-chars)) ; => "8%2F@e.com"

Or, if you don't mind the unnecessary escaping (particularly of "@"),
you can just use url-hexify-string with the default set of unreserved
characters.


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2020-09-18 16:55 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-16 23:51 Mummi wishlist: API using Message-ID zimoun
2020-09-17 10:01 ` Ricardo Wurmus
2020-09-17 10:38   ` zimoun
2020-09-17 11:33     ` Ricardo Wurmus
2020-09-18  5:44       ` Arun Isaac
2020-09-18  3:53     ` Kyle Meyer
2020-09-18 16:46       ` Kyle Meyer

unofficial mirror of guix-devel@gnu.org 

This inbox may be cloned and mirrored by anyone:

	git clone --mirror https://yhetil.org/guix-devel/0 guix-devel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 guix-devel guix-devel/ https://yhetil.org/guix-devel \
		guix-devel@gnu.org
	public-inbox-index guix-devel

Example config snippet for mirrors.
Newsgroups are available over NNTP:
	nntp://news.yhetil.org/yhetil.gnu.guix.devel
	nntp://news.gmane.io/gmane.comp.gnu.guix.devel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git