Newsgroups mailing-list gateway broken thread

Newsgroups mailing-list gateway broken thread

[Garreau, Alexandre]

Hi,

After some discussion I’ve realized that in fact mailing traversing the
newsgroups mailing-list gateway receive different message-ids in their
references, which, in my mail user-agent (gnus) break the threads.

I say that because it appears not to break anything on the mailing-list
archive web interface (and to affect most if not all of newsgroup
posters), so I’d like how does it do (does it have mappings of all
correct message-ids to other message-ids?), and how other users of the
mailing-list do, or what can be done to improve the situation.

Thanks for any information.

Re: Newsgroups mailing-list gateway broken thread

[Emanuel Berg]

Garreau, Alexandre wrote:

> After some discussion I’ve realized that in
> fact mailing traversing the newsgroups
> mailing-list gateway receive different
> message-ids in their references, which, in my
> mail user-agent (gnus) break the threads.
>
> I say that because it appears not to break
> anything on the mailing-list archive web
> interface (and to affect most if not all of
> newsgroup posters), so I’d like how does it
> do (does it have mappings of all correct
> message-ids to other message-ids?), and how
> other users of the mailing-list do, or what
> can be done to improve the situation.
>
> Thanks for any information.

(Important note: This post is *not* sent thru
gnu.emacs.help, but from gmane.emacs.help -
like I say below, it is 20 times slower. But:
how do the references look now?)

The most likely explanation is that it is
a glitch in the interface between the newsgroup
and the mailing list.

I always used gnu.emacs.help, only for a short
while I used gmane.emacs.help, but for some
reason it is much slower (to get the summary of
the new posts), so I switched back to
gnu.emacs.help.

I have no idea as to what piece of technology
it is that does the propagation, where it is
located, who runs is, and so on.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Newsgroups mailing-list gateway broken thread

[Garreau, Alexandre]

On 2018-10-18 at 12:05, Emanuel Berg wrote:
> Garreau, Alexandre wrote:
>
>> After some discussion I’ve realized that in
>> fact mailing traversing the newsgroups
>> mailing-list gateway receive different
>> message-ids in their references, which, in my
>> mail user-agent (gnus) break the threads.
>>
>> I say that because it appears not to break
>> anything on the mailing-list archive web
>> interface (and to affect most if not all of
>> newsgroup posters), so I’d like how does it
>> do (does it have mappings of all correct
>> message-ids to other message-ids?), and how
>> other users of the mailing-list do, or what
>> can be done to improve the situation.
>>
>> Thanks for any information.
>
> (Important note: This post is *not* sent thru
> gnu.emacs.help, but from gmane.emacs.help -
> like I say below, it is 20 times slower. But:
> how do the references look now?)

The references are just fine, they correctly references my message.

> The most likely explanation is that it is
> a glitch in the interface between the newsgroup
> and the mailing list.
>
> I always used gnu.emacs.help, only for a short
> while I used gmane.emacs.help, but for some
> reason it is much slower (to get the summary of
> the new posts), so I switched back to
> gnu.emacs.help.

I expect the newsgroups gateway *should* work.  So this is to fix,
rather than suggesting gmane as a workaround (the issue of it being
slower is a separate issue).

> I have no idea as to what piece of technology
> it is that does the propagation, where it is
> located, who runs is, and so on.

I believe this is be on the GNU side, because I saw references referring
to same garbled message-ids, with “mailman” in them: so if different
newsgroup users, such as Emanual Berg and Barry Margolin gets the same
references while using different gateways to usenet, and this one has
“mailman” in it, I think it is not only common to all newsgroups users
(thus coming from GNU), but also coming from either mailman, either
something interfacing with it, I guess.

The question is why.

Re: Newsgroups mailing-list gateway broken thread

[Bob Proulx]

Garreau, Alexandre wrote:
> Emanuel Berg wrote:
> > Garreau, Alexandre wrote:
> >> After some discussion I’ve realized that in
> >> fact mailing traversing the newsgroups
> >> mailing-list gateway receive different
> >> message-ids in their references, which, in my
> >> mail user-agent (gnus) break the threads.

Yes.  When users post to the newsgroup and it is gatewayed into the
mailing list it breaks threading.

> >> I say that because it appears not to break
> >> anything on the mailing-list archive web
> >> interface (and to affect most if not all of

It does break on the web interface too.  It's no different there than
anywhere else.  Messages from the newsgroup break threading and start
a new thread. :-(

> >> newsgroup posters), so I’d like how does it
> >> do (does it have mappings of all correct
> >> message-ids to other message-ids?), and how
> >> other users of the mailing-list do, or what
> >> can be done to improve the situation.

There isn't anything special about the web interface.  Threading is
broken there the same as elsewhere.  Therefore there is no magic to
extract from there to add to your own to fix threading.

> >> Thanks for any information.

> > (Important note: This post is *not* sent thru
> > gnu.emacs.help, but from gmane.emacs.help -
> > like I say below, it is 20 times slower. But:
> > how do the references look now?)
> 
> The references are just fine, they correctly references my message.
...
> I expect the newsgroups gateway *should* work.  So this is to fix,
> rather than suggesting gmane as a workaround (the issue of it being
> slower is a separate issue).

Emanuel, We have talked about this before. :-)

> I believe this is be on the GNU side, because I saw references referring
> to same garbled message-ids, with “mailman” in them: so if different
> newsgroup users, such as Emanual Berg and Barry Margolin gets the same
> references while using different gateways to usenet, and this one has
> “mailman” in it, I think it is not only common to all newsgroups users
> (thus coming from GNU), but also coming from either mailman, either
> something interfacing with it, I guess.
> 
> The question is why.

This is a deja vu as we have discussed this before.  This link jumps
right into some detail from that previous discussion.

 http://lists.gnu.org/archive/html/help-gnu-emacs/2016-06/msg00147.html

Of course the web email address redactor with all of the address
hidden redactions make reading that on the web quite difficult.  One
could ftp down the mailbox and then have the full mailbox of messages.

AFAICT messages through the Stanford newsgroup to email relay do not
include the In-Reply-To header and therefore break threading.  A
message may have the References header with information about previous
messages in the thread causing the message to thread before where it
should thread, and if only one deep them be completely disconnected
from the original thread.

In the intervening two years since the previous discussion nothing
substantial has happened.

Bob

Fixing news-mail gateway [Was: Re: Newsgroups mailing-list gateway broken thread]

[Garreau, Alexandre]

On 2018-10-18 at 12:44, Bob Proulx wrote:
> This is a deja vu as we have discussed this before.  This link jumps
> right into some detail from that previous discussion.
>
>  http://lists.gnu.org/archive/html/help-gnu-emacs/2016-06/msg00147.html
>
> Of course the web email address redactor with all of the address
> hidden redactions make reading that on the web quite difficult.  One
> could ftp down the mailbox and then have the full mailbox of messages.
>
> AFAICT messages through the Stanford newsgroup to email relay do not
> include the In-Reply-To header and therefore break threading.  A
> message may have the References header with information about previous
> messages in the thread causing the message to thread before where it
> should thread, and if only one deep them be completely disconnected
> from the original thread.
>
> In the intervening two years since the previous discussion nothing
> substantial has happened.

So I was right, it is the news-mail interface, and it is in standford,
so now the question is: who to contact? is that GNU people?

Maybe to <action@cs.stanford.edu> as “X-Complaints-To” header suggests?
or <root@usenet.stanford.edu>?

Who to contact about the gateway [Was: Re: Fixing news-mail gateway [Was: Re: Newsgroups mailing-list gateway broken thread]]

[Garreau, Alexandre]

[-- Attachment #1: Type: text/plain, Size: 841 bytes --]

On 2018-10-18 at 21:24, Garreau, Alexandre wrote:
> So I was right, it is the news-mail interface, and it is in standford,
> so now the question is: who to contact? is that GNU people?
>
> Maybe to <action@cs.stanford.edu> as “X-Complaints-To” header suggests?
> or <root@usenet.stanford.edu>?

Is anyone acknowledgable of the gateway mechanism or did the people who
set this up disappeared?  I never used usenet, and am not directly
responsible for this mailing-list administration, nor am I yet that
involved, so I’m a little bit incomfortable asking myself.  Would
someone who actually use newsgroups or even who have special powers on
this mailing-list prefer to try contact the in-charge people?

Otherwise I was planning to send a mail like this one (not yet sent,
this is a draft) to <action@cs.stanford.edu>?


[-- Attachment #2: Type: message/rfc822, Size: 1105 bytes --]

From: "Garreau\, Alexandre" <galex-713@galex-713.eu>
To: help-gnu-emacs@gnu.org
Subject: Who to contact about the gateway [Was: Re: Fixing news-mail gateway [Was: Re: Newsgroups mailing-list gateway broken thread]]

On 2018-10-18 at 21:24, Garreau, Alexandre wrote:
> So I was right, it is the news-mail interface, and it is in standford,
> so now the question is: who to contact? is that GNU people?
>
> Maybe to <action@cs.stanford.edu> as X-Complaints-To header suggests?
> or <root@usenet.stanford.edu>?

Is anyone acknowledgable of the gateway mechanism or did the people who
set this up disappeared?  I never used usenet yet so Im a little bit 

Re: Who to contact about the gateway [Was: Re: Fixing news-mail gateway [Was: Re: Newsgroups mailing-list gateway broken thread]]

[Emanuel Berg]

"Garreau, Alexandre" <galex-713@galex-713.eu> writes:

> Is anyone acknowledgable of the gateway mechanism or
> did the people who set this up disappeared? I never
> used usenet

Using Usenet is very easy. When you add a group in
Gnus, instead of using a Gmane or a mail server, pick
nntp.aioe.org. Then go to the server buffer and open
it. You will see a sad state. But it wasn't always
like that. Will it get better? Ask Father Time.

-- 
underground experts exiled
http://user.it.uu.se/~embe8573

Re: Who to contact about the gateway [Was: Re: Fixing news-mail gateway [Was: Re: Newsgroups mailing-list gateway broken thread]]

[Garreau, Alexandre]

Le 21/10/2018 à 06h11, Emanuel Berg a écrit :
> "Garreau, Alexandre" <galex-713@galex-713.eu> writes:
>
>> Is anyone acknowledgable of the gateway mechanism or
>> did the people who set this up disappeared? I never
>> used usenet
>
> Using Usenet is very easy. When you add a group in
> Gnus, instead of using a Gmane or a mail server, pick
> nntp.aioe.org. Then go to the server buffer and open
> it. You will see a sad state. But it wasn't always
> like that. Will it get better? Ask Father Time.

My primary gnus-select-method was nnnil so I needed to set it back to
(nntp "news") so to do that, but yes it works, and indeed, it is in a
very poor state.  Way worse than I thought, and also way worse than
here: all threads, even with sorting by date activated, are all garbled,
because some times faked root are considered to be back from EPOCH!

I’ll send that mail.

Re: Who to contact about the gateway [Was: Re: Fixing news-mail gateway [Was: Re: Newsgroups mailing-list gateway broken thread]]

[Emanuel Berg]

Garreau, Alexandre wrote:

> My primary gnus-select-method was nnnil so
> I needed to set it back to (nntp "news") so
> to do that, but yes it works, and indeed, it
> is in a very poor state. Way worse than
> I thought, and also way worse than here: all
> threads, even with sorting by date activated,
> are all garbled

Threading works well, at least for me.
Actually everything works well in terms of
technology. What doesn't work, or what is
working worse and worse for each year, is the
"human factor".

And because no newcomers are arriving one can
almost even at this point consider Usenet
dead technology.

Gmane is of course just as good to discuss
technology and computers, FOSS projects, and so
on, but to be able the discuss ANYTHING with
a uniform, text-based interface is probably
a thing of the past.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Fixing news-mail gateway [Was: Re: Newsgroups mailing-list gateway broken thread]

[Nuno Silva]

On 2018-10-18, Garreau, Alexandre wrote:

> On 2018-10-18 at 12:44, Bob Proulx wrote:
>> This is a deja vu as we have discussed this before.  This link jumps
>> right into some detail from that previous discussion.
>>
>>  http://lists.gnu.org/archive/html/help-gnu-emacs/2016-06/msg00147.html
>>
>> Of course the web email address redactor with all of the address
>> hidden redactions make reading that on the web quite difficult.  One
>> could ftp down the mailbox and then have the full mailbox of messages.
>>
>> AFAICT messages through the Stanford newsgroup to email relay do not
>> include the In-Reply-To header and therefore break threading.  A
>> message may have the References header with information about previous
>> messages in the thread causing the message to thread before where it
>> should thread, and if only one deep them be completely disconnected
>> from the original thread.
>>
>> In the intervening two years since the previous discussion nothing
>> substantial has happened.
>
> So I was right, it is the news-mail interface, and it is in standford,
> so now the question is: who to contact? is that GNU people?
>
> Maybe to <action@cs.stanford.edu> as “X-Complaints-To” header suggests?
> or <root@usenet.stanford.edu>?

I might be wrong, but it looks like the In-reply-to header *is* sent,
but the message-ids are being rewritten somewhere, so one message can
end up with two different message-ids, and threading by In-reply-to will
not work as expected. The "Message not available" entries in the web
archive are probably those message-ids.

Alexandre, did you send the post I am replying to through USENET as
well? It has the following In-Reply-To:

In-Reply-To: <20181018121955500722783@bob.proulx.com> (Bob Proulx's message of
	"Thu, 18 Oct 2018 12:44:57 -0600")

-- 
Nuno Silva

Re: Fixing news-mail gateway [Was: Re: Newsgroups mailing-list gateway broken thread]

[Bob Proulx]

Nuno Silva wrote:
> Bob Proulx wrote:
> > AFAICT messages through the Stanford newsgroup to email relay do not
> > include the In-Reply-To header and therefore break threading.  A
> > message may have the References header with information about previous
> > messages in the thread causing the message to thread before where it
> > should thread, and if only one deep them be completely disconnected
> > from the original thread.
> 
> I might be wrong, but it looks like the In-reply-to header *is* sent,
> but the message-ids are being rewritten somewhere, so one message can
> end up with two different message-ids, and threading by In-reply-to will
> not work as expected. The "Message not available" entries in the web
> archive are probably those message-ids.

Hmm...  You are right that at least some mail messages from the
Stanford newsgroup gateway do include In-Reply-To headers.  Most do
not however.  In the last three months of the archive 38 messages that
came through the gateway included In-Reply-To out of 285 messages
total through that gateway with 247 not including it.  At least some
of those not including it were starting a new thread and were not
replies so that normal for those.

This nature of it with some having it and some not having it is a good
clue that I had previously missed.  It might indicate that one of the
newsgroup relays is causing the issue and it depends upon how the
message routed through the newsgroup relays before arriving at
Stanford and getting gatewayed to the mailing list.  It will need more
analysis to see if we can correlate it with one specific host.

Bob

Re: Fixing news-mail gateway [Was: Re: Newsgroups mailing-list gateway broken thread]

[Nuno Silva]

On 2018-10-19, Bob Proulx wrote:

> Nuno Silva wrote:
>> Bob Proulx wrote:
>> > AFAICT messages through the Stanford newsgroup to email relay do not
>> > include the In-Reply-To header and therefore break threading.  A
>> > message may have the References header with information about previous
>> > messages in the thread causing the message to thread before where it
>> > should thread, and if only one deep them be completely disconnected
>> > from the original thread.
>> 
>> I might be wrong, but it looks like the In-reply-to header *is* sent,
>> but the message-ids are being rewritten somewhere, so one message can
>> end up with two different message-ids, and threading by In-reply-to will
>> not work as expected. The "Message not available" entries in the web
>> archive are probably those message-ids.
>
> Hmm...  You are right that at least some mail messages from the
> Stanford newsgroup gateway do include In-Reply-To headers.  Most do
> not however.  In the last three months of the archive 38 messages that
> came through the gateway included In-Reply-To out of 285 messages
> total through that gateway with 247 not including it.  At least some
> of those not including it were starting a new thread and were not
> replies so that normal for those.

And you are right some (most?) don't include In-Reply-To. I checked some
messages that were sent through USENET, but somehow I avoided checking
my own messages. The message you are replying to lacks In-Reply-To (at
least here). (But it has References, which I suppose should be enough to
do threading?)


(A message posted to the group/list through USENET will probably not
break threading on the same server it was posted to.)

> This nature of it with some having it and some not having it is a good
> clue that I had previously missed.  It might indicate that one of the
> newsgroup relays is causing the issue and it depends upon how the
> message routed through the newsgroup relays before arriving at
> Stanford and getting gatewayed to the mailing list.  It will need more
> analysis to see if we can correlate it with one specific host.
>
> Bob
>

-- 
Nuno Silva

Re: Fixing news-mail gateway [Was: Re: Newsgroups mailing-list gateway broken thread]

[Garreau, Alexandre]

On 2018-10-19 at 09:47, Nuno Silva wrote:
> I might be wrong, but it looks like the In-reply-to header *is* sent,
> but the message-ids are being rewritten somewhere, so one message can
> end up with two different message-ids, and threading by In-reply-to
> will not work as expected. The "Message not available" entries in the
> web archive are probably those message-ids.

Yes, as explained before: mailman, or the software directly interfacing
with it, translate all message-ids for the newsgroup side.  All headers
are preserved but this one, afaik (Emanual did show me his headers a
couple days ago).

> Alexandre, did you send the post I am replying to through USENET as
> well?

No, still don’t use it, plan to.

> It has the following In-Reply-To:
>
> In-Reply-To: <20181018121955500722783@bob.proulx.com> (Bob Proulx's message of
> 	"Thu, 18 Oct 2018 12:44:57 -0600")

Yes it has.  However *your* message is shown as a direct answer to a
fake message my Gnus nowadays create, as a child to this same Bob
Proulx’s message, because the last known message-id of your References
is his, mine is rewritten as containing “mailman” stuff as I said before.

PS: I’m pushing forward the reply-to-all convention so to send messages
to non-existing addresses now x) so to convey the semantics of this
message being adressed to you while being public on that mailing-list
newsgroup. nobody will be bothered by extra mail here anyway.

Re: Newsgroups mailing-list gateway broken thread

[Emanuel Berg]

Bob Proulx wrote:

> Messages from the newsgroup break threading
> and start a new thread. :-( [...] There isn't
> anything special about the web interface.
> Threading is broken there the same
> as elsewhere.

Well, except for the newsgroup!

> Emanuel, We have talked about this before.
> :-)

I don't think I understood it 100% that time.

OK, from now on I use gmane.emacs.help instead!
Everyone happy?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Newsgroups mailing-list gateway broken thread

[Nuno Silva]

On 2018-10-19, Emanuel Berg wrote:

> Bob Proulx wrote:
>
>> Messages from the newsgroup break threading
>> and start a new thread. :-( [...] There isn't
>> anything special about the web interface.
>> Threading is broken there the same
>> as elsewhere.
>
> Well, except for the newsgroup!
>
>> Emanuel, We have talked about this before.
>> :-)
>
> I don't think I understood it 100% that time.
>
> OK, from now on I use gmane.emacs.help instead!
> Everyone happy?

But why is the gateway rewriting Message-IDs? 

Your post, seen from USENET, has:

    Message-ID: <mailman.2440.1539944256.1284.help-gnu-emacs@gnu.org>

And on Gmane:

    Message-ID: <86zhvab5r3.fsf@zoho.com>



(This reply is being sent through USENET, but I tried to set
In-Reply-To myself. If that works, it shouldn't break threading.)

-- 
Nuno Silva

Re: Newsgroups mailing-list gateway broken thread

[Emanuel Berg]

Nuno Silva wrote:

> But why is the gateway rewriting Message-IDs?
>
> Your post, seen from USENET, has:
>
>     Message-ID: <mailman.2440.1539944256.1284.help-gnu-emacs@gnu.org>
>
> And on Gmane:
>
>     Message-ID: <86zhvab5r3.fsf@zoho.com>

I don't know. I don't know what kind of program
the "gateway" is or how it gets its data from
either the mailing list or the newsgroup.
I don't know where it is physically or who
maintains it, or if it was just left with the
switch "on" ages ago. Whoever wrote it could be
cutting rock in Siberia for all I know.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Newsgroups mailing-list gateway broken thread

[Bob Proulx]

Nuno Silva wrote:
> But why is the gateway rewriting Message-IDs?

I think that would be a bug.  But it may be a bug only in one of the
newgroup relays.  The list for the replied to message:

  Path: usenet.stanford.edu!goblin2!goblin1!goblin.stu.neva.ru!news.albasani.net!.POSTED!not-for-mail

> Your post, seen from USENET, has:
>
>     Message-ID: <mailman.2440.1539944256.1284.help-gnu-emacs@gnu.org>
>
> And on Gmane:
>
>     Message-ID: <86zhvab5r3.fsf@zoho.com>

I think that is a good clue.  But it may include a race condition.
The message id that includes "mailman" seems like something that could
be inserted by the mail to newsgroup direction of the gateway.  And
why doesn't it include an actual hostname there?  It isn't required.
It is only required that it be unique and any unique string may be
used there.  So that may not be wrong as such.  And it might be the
creation of a news reader creating that message id.  But it feels like
something that needs to be understood in order to continue to make
sense of things.

> (This reply is being sent through USENET, but I tried to set
> In-Reply-To myself. If that works, it shouldn't break threading.)

I think it worked.

What did you try setting it to?  It came to the mailing list with:

  In-Reply-To: <86zhvab5r3.fsf@zoho.com>
  References: <87lg6w43k8.fsf@portable.galex-713.eu> <861s8nefj5.fsf@zoho.com>
          <874ldjtcow.fsf@portable.galex-713.eu>
          <20181018121955500722783@bob.proulx.com>
          <mailman.2440.1539944256.1284.help-gnu-emacs@gnu.org>

It apeared that the message you replied to was:

  Message-ID: <86zhvab5r3.fsf@zoho.com>
  References: <87lg6w43k8.fsf@portable.galex-713.eu> <861s8nefj5.fsf@zoho.com>
          <874ldjtcow.fsf@portable.galex-713.eu>
          <20181018121955500722783@bob.proulx.com>

Seems that the message id should have also been pushed into the
References list too but was not.  But I know you were doing that
manually.  If it were done by software then the message id should be
pushed into References and set as the In-Reply-To.  But it is also
interesting that the References list changed beyond that too.

Bob

Re: Newsgroups mailing-list gateway broken thread

[Nuno Silva]

On 2018-10-19, Bob Proulx wrote:

> Nuno Silva wrote:
>> But why is the gateway rewriting Message-IDs?
>
> I think that would be a bug.  But it may be a bug only in one of the
> newgroup relays.  The list for the replied to message:
>
>   Path: usenet.stanford.edu!goblin2!goblin1!goblin.stu.neva.ru!news.albasani.net!.POSTED!not-for-mail
>
>> Your post, seen from USENET, has:
>>
>>     Message-ID: <mailman.2440.1539944256.1284.help-gnu-emacs@gnu.org>
>>
>> And on Gmane:
>>
>>     Message-ID: <86zhvab5r3.fsf@zoho.com>
>
> I think that is a good clue.  But it may include a race condition.
> The message id that includes "mailman" seems like something that could
> be inserted by the mail to newsgroup direction of the gateway.  And
> why doesn't it include an actual hostname there?  It isn't required.
> It is only required that it be unique and any unique string may be
> used there.  So that may not be wrong as such.  And it might be the
> creation of a news reader creating that message id.  But it feels like
> something that needs to be understood in order to continue to make
> sense of things.
>
>> (This reply is being sent through USENET, but I tried to set
>> In-Reply-To myself. If that works, it shouldn't break threading.)
>
> I think it worked.
>
> What did you try setting it to?  It came to the mailing list with:
>
>   In-Reply-To: <86zhvab5r3.fsf@zoho.com>
>   References: <87lg6w43k8.fsf@portable.galex-713.eu> <861s8nefj5.fsf@zoho.com>
>           <874ldjtcow.fsf@portable.galex-713.eu>
>           <20181018121955500722783@bob.proulx.com>
>           <mailman.2440.1539944256.1284.help-gnu-emacs@gnu.org>

That (above) is exactly what was in the post I sent. I manually added
"In-Reply-To: <86zhvab5r3.fsf@zoho.com>". This should be the Gmane
Message-ID of the post which shows up here (USENET) as
<mailman.2440.1539944256.1284.help-gnu-emacs@gnu.org>.

> It apeared that the message you replied to was:
>
>   Message-ID: <86zhvab5r3.fsf@zoho.com>
>   References: <87lg6w43k8.fsf@portable.galex-713.eu> <861s8nefj5.fsf@zoho.com>
>           <874ldjtcow.fsf@portable.galex-713.eu>
>           <20181018121955500722783@bob.proulx.com>
>
> Seems that the message id should have also been pushed into the
> References list too but was not.  But I know you were doing that
> manually.  If it were done by software then the message id should be
> pushed into References and set as the In-Reply-To.  But it is also
> interesting that the References list changed beyond that too.

It was pushed, but it was the rewritten Message-ID. This is the problem.
For some reason something is changing the Message-IDs. But instead of
changing *all* of them - Message-ID, References and In-Reply-To - (and
in both directions), it appears to be changing the Message-ID only?

-- 
Nuno Silva

Re: Newsgroups mailing-list gateway broken thread

[Garreau, Alexandre]

On 2018-10-19 at 20:38, Nuno Silva wrote:
> It was pushed, but it was the rewritten Message-ID. This is the
> problem.  For some reason something is changing the Message-IDs. But
> instead of changing *all* of them - Message-ID, References and
> In-Reply-To - (and in both directions), it appears to be changing the
> Message-ID only?

Yes, as observed.  However I believe changing message-id, even it it
were consistently across all headers and in both directions, would be
erroneous, as these might be used in extra headers as well, as for
linking, archiving, etc.  One message shouldn’t see its message-id
change that way, whatever the protocol used to transmit it.

Re: Newsgroups mailing-list gateway broken thread

[Garreau, Alexandre]

On 2018-10-19 at 11:33, Nuno Silva wrote:
> (This reply is being sent through USENET, but I tried to set
> In-Reply-To myself. If that works, it shouldn't break threading.)

It looks it did, because you didn’t set the References header correctly
according here.

Re: Newsgroups mailing-list gateway broken thread

[Nuno Silva]

On 2018-10-21, Garreau, Alexandre wrote:

> On 2018-10-19 at 11:33, Nuno Silva wrote:
>> (This reply is being sent through USENET, but I tried to set
>> In-Reply-To myself. If that works, it shouldn't break threading.)
>
> It looks it did, because you didn’t set the References header correctly
> according here.

Yeah, I forgot to do that, sorry!

-- 
Nuno Silva

Re: Newsgroups mailing-list gateway broken thread

[Bob Proulx]

Emanuel Berg wrote:
> OK, from now on I use gmane.emacs.help instead!

Since you are one of the prolific posters that would reduce the number
of broken threads.  But there are also other people who post through
from the newsgroups too.  Those threads will still be broken.  So it
doesn't solve the problem.  It just means we would no longer see it
with your messages.

> Everyone happy?

That is impossible! :-)

Bob

Re: Newsgroups mailing-list gateway broken thread

[Emanuel Berg]

Bob Proulx wrote:

> Since you are one of the prolific posters
> that would reduce the number of broken
> threads. But there are also other people who
> post through from the newsgroups too.
> Those threads will still be broken. So it
> doesn't solve the problem. It just means we
> would no longer see it with your messages.

Yes, I understand that would be the optimal
solution. Not just for you guys but also for
me. Because then I would return to
gnu.emacs.help :)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Newsgroups mailing-list gateway broken thread

[Emanuel Berg]

Bob Proulx wrote:

> In the intervening two years since the
> previous discussion nothing substantial
> has happened.

Last words from my part in the matter is that
two years ago I didn't understand that I broke
threads on a daily basis simply by using the
newsgroup! The reason I didn't understand it
was that in the newsgroup, it didn't happen,
everything looked and looks fine.

But now when I read Proulx' summary [1] - which
I did back then as well, of course -
I understand it, and it is a bit of a mystery
why I didn't understand it even then.

So I'd like to underline that me using the
newsgroup these two years was because 1) me not
understanding the problems it caused, 2) its
speed compared to the equivalent Gmane group.

In my life, there have been a couple of
rebellious acts, some of which I today feel
shame, some pride, and some 50/50, but this
situation in particular was no such act but
pure ignorance. I didn't understand this until
this particular sentence "When users post to
the newsgroup and it is gatewayed into the
mailing list it breaks threading."

Obviously, in the grand scheme of things, all
this is a bubble, but I thought I'd say
it anyway.

[1] http://lists.gnu.org/archive/html/help-gnu-emacs/2016-06/msg00147.html

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Newsgroups mailing-list gateway broken thread

[Emanuel Berg]

Garreau, Alexandre wrote:

> I expect the newsgroups gateway *should*
> work. So this is to fix, rather than
> suggesting gmane as a workaround (the issue
> of it being slower is a separate issue).

(This post I send thru gnu.emacs.help - so now
you can compare it to the Gmane post, if that
interests you!)

Gmane isn't exactly a workaround, either. Or if
it is, it is a HUGE one! Gmane is rather
Usenet 2.0 or Usenet Reloaded. It is the best
of both worlds, the worlds being the
mailing lists and the Usenet newsgroups.

Not all Gmane groups are that much slower by
the way. I don't know why.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Newsgroups mailing-list gateway broken thread

[Garreau, Alexandre]

On 2018-10-18 at 20:37, Emanuel Berg wrote:
> Garreau, Alexandre wrote:
>
>> I expect the newsgroups gateway *should*
>> work. So this is to fix, rather than
>> suggesting gmane as a workaround (the issue
>> of it being slower is a separate issue).
>
> (This post I send thru gnu.emacs.help - so now
> you can compare it to the Gmane post, if that
> interests you!)

Yeah still exactly the same problems as before.

> Gmane isn't exactly a workaround, either. Or if
> it is, it is a HUGE one!

I’m meaning from your side.  You’d *use* gmane *as* a workaround.  Gmane
per se is not a workaround and I agree it is great.

> Gmane is rather Usenet 2.0 or Usenet Reloaded. It is the best of both
> worlds, the worlds being the mailing lists and the Usenet newsgroups.
>
> Not all Gmane groups are that much slower by the way. I don't know
> why.

To me “the best” would be a tight integration of both smtp and nntp
clients/servers and mailing-list software, written in lisp (not python),
and work in P2P, and would primarily use cryptography for
authentication, if available (think DKIM+PGP+SASL+TLS, pick the
available ones).  But gmane is the best that actually exists right now
:)

Re: Newsgroups mailing-list gateway broken thread

[Nuno Silva]

On 2018-10-17, Garreau, Alexandre wrote:

> After some discussion I’ve realized that in fact mailing traversing the
> newsgroups mailing-list gateway receive different message-ids in their
> references, which, in my mail user-agent (gnus) break the threads.
>
> I say that because it appears not to break anything on the mailing-list
> archive web interface (and to affect most if not all of newsgroup
> posters), so I’d like how does it do (does it have mappings of all
> correct message-ids to other message-ids?), and how other users of the
> mailing-list do, or what can be done to improve the situation.

I was looking into this recently, and from what I have seen, threading
is still broken in the web archive, just not as broken as I see it in
Gnus.

There is indeed a mismatch in the references. Gnus here handles this by
starting a new thread. The web archive still keeps the reply in the same
thread, but the "missing" message-id shows up in the tree there as
"Message not available".

Here (through USENET) the message-id for your post is:
Message-ID: <mailman.2336.1539814909.1284.help-gnu-emacs@gnu.org>

-- 
Nuno Silva