aioe (was: Re: Is there a problem with Gmane at the moment?)

aioe (was: Re: Is there a problem with Gmane at the moment?)

[Emanuel Berg]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Emanuel Berg <embe8573@student.uu.se> writes:
>
>> I now start to have problems with gmane.emacs so
>> perhaps this has nothing to do with Gmane but is
>> a problem with the GNU listbots?
>
> Yes.

I just now posted to gnu.emacs.help on nntp.aioe.org
- even tho that newsgroup mirrors the mailing list,
perhaps it has some secondary storage to complement
it, because two posts of mine appeared *instantly*
there while I have been unable to send to the
corresponding mailing list for several days.

-- 
underground experts united .... http://user.it.uu.se/~embe8573
Emacs Gnus Blogomatic ......... http://user.it.uu.se/~embe8573/blogomatic
                   - so far: 30 Blogomatic articles -                   

Re: aioe (was: Re: Is there a problem with Gmane at the moment?)

[Emanuel Berg]

Emanuel Berg <embe8573@student.uu.se> writes:

> I just now posted to gnu.emacs.help on
> nntp.aioe.org - even tho that newsgroup mirrors the
> mailing list, perhaps it has some secondary storage
> to complement it, because two posts of mine
> appeared *instantly* there while I have been unable
> to send to the corresponding mailing list for
> several days.

OK, so no *this* appeared, with only Gmane! But if it
was I queuing problem, this message must have skipped
several others I have sent - well, if anyone can
figure all this out, he is a smarter person than me.
Or older! The devil knows the most, not because he is
the devil, but because he is old, as they say (?) in
the Iberico-Hispanic world. Right now I don't
remember what I wrote in those lost (?) posts, but
most likely it was about technology. But it is all
good. The past is the past. The future will come.
Only the present is up for grabs!

-- 
underground experts united .... http://user.it.uu.se/~embe8573
Emacs Gnus Blogomatic ......... http://user.it.uu.se/~embe8573/blogomatic
                   - so far: 30 Blogomatic articles -                   

Re: aioe (was: Re: Is there a problem with Gmane at the moment?)

[Duncan]

Emanuel Berg posted on Sun, 15 May 2016 01:46:49 +0200 as excerpted:

> Emanuel Berg <embe8573@student.uu.se> writes:
> 
>> I just now posted to gnu.emacs.help on nntp.aioe.org - even tho that
>> newsgroup mirrors the mailing list, perhaps it has some secondary
>> storage to complement it, because two posts of mine appeared
>> *instantly* there while I have been unable to send to the corresponding
>> mailing list for several days.
> 
> OK, so no *this* appeared, with only Gmane! But if it was I queuing
> problem, this message must have skipped several others I have sent -
> well, if anyone can figure all this out, he is a smarter person than me.
> Or older! The devil knows the most, not because he is the devil, but
> because he is old, as they say (?) in the Iberico-Hispanic world. Right
> now I don't remember what I wrote in those lost (?) posts, but most
> likely it was about technology. But it is all good. The past is the
> past. The future will come. Only the present is up for grabs!

This one appeared because it was cross-posted, and the gmane.discuss list 
got it, so it appeared in both.

The others didn't appear as they're (presumably) not cross-posted, at 
least not to an actually working (from gmane?) list.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: aioe (was: Re: Is there a problem with Gmane at the moment?)

[Emanuel Berg]

Duncan <1i5t5.duncan@cox.net> writes:

> This one appeared because it was cross-posted, and
> the gmane.discuss list got it, so it appeared
> in both.
>
> The others didn't appear as they're (presumably)
> not cross-posted, at least not to an actually
> working (from gmane?) list.

Yes, the thought of this explanation "crossed" my
mind, but I couldn't make sense of it!

If I can't post to A, but to B, and then post to
A *and* B, then I can post to A as well as B!

The only way I can think of it is that it is some
illusion provided by Gmane so that when it appears
I read on A, I actually read on B. But the illusion
apparently works for you as, as well!

Is that how it works?

What would happen if I cross-posted to A and B as
above, only I didn't subscribe to B? Would it still
work on A? And if so, would that qualify a magic?

-- 
underground experts united .... http://user.it.uu.se/~embe8573
Emacs Gnus Blogomatic ......... http://user.it.uu.se/~embe8573/blogomatic
                   - so far: 30 Blogomatic articles -                   

Re: aioe (was: Re: Is there a problem with Gmane at the moment?)

[Duncan]

Emanuel Berg posted on Sun, 15 May 2016 10:15:01 +0200 as excerpted:

> Duncan <1i5t5.duncan@cox.net> writes:
> 
>> This one appeared because it was cross-posted, and the gmane.discuss
>> list got it, so it appeared in both.
>>
>> The others didn't appear as they're (presumably) not cross-posted, at
>> least not to an actually working (from gmane?) list.
> 
> Yes, the thought of this explanation "crossed" my mind, but I couldn't
> make sense of it!
> 
> If I can't post to A, but to B, and then post to A *and* B, then I can
> post to A as well as B!
> 
> The only way I can think of it is that it is some illusion provided by
> Gmane so that when it appears I read on A, I actually read on B. But the
> illusion apparently works for you as, as well!
> 
> Is that how it works?

You have to remember that we're dealing with a news server here, and 
while news messages look a lot like mail, there's some differences.

But of course gmane isn't just a news server, it's actually a 
(mailing-)list2news gateway, and some things must be fudged a little bit 
to make one appear as the other.

On a normal news server, a cross-posted message normally appears in all 
groups it was posted to that the news server carries.  When a message 
propagates from one server to another, the headers say what groups it is 
posted to, and those don't change, regardless of whether the news server 
actually carries all of them or not, the server simply ignores groups it 
doesn't carry and puts the message in the groups it does.  (If it doesn't 
carry at least one of them it is of course not interested in the message 
at all, so the message doesn't propagate to that server.)  This way, the 
message can be propagated from a server only carrying some of the groups, 
to another carrying all of the groups, and the message will appear in all 
the groups on the second server, which carries all of them, even tho the 
first server didn't.

This, BTW, is one reason why some news providers entirely drop messages 
with binary attachments that are cross-posted to non-binaries groups.  
The message can't appear in the non-binary groups because it's a binary, 
but it's cross-posted to both binary and non-binary groups that the 
server carries, so the server simply drops it in the binary groups as 
well, to avoid it appearing in the non-binary groups.  Or at least that 
was the explanation.  A lot of users thought that if they could institute 
separate binaries and retention policies for the binary groups vs non-
binary, they could have gotten around it if they wanted to, but...  (Back 
in the day, supernews was the big provider known for this policy that few 
others followed.  But supernews got bought out by IIRC giganews, and I 
don't know if supernews is even around as a separate brand these days, 
and if it is, if it still has that same policy.)

So on a normal news server, the message is propagated once to the news 
server and it can place it in all cross-posted groups it carries based on 
that single propagation.

On gmane, things work rather differently.  First, gmane never posts 
messages as they are posted directly from users.  It forwards them via 
mail to the appropriate mailing list, and only posts the message after it 
gets it back from the mailing list via gmane's own list subscription.

At least, that's the way it works for single-posted messages.

Cross-posted messages get more complicated, first because now, gmane has 
to send the message to multiple lists, separately, if the lists aren't 
all served by the same listserv, as would be the case here.  Each list 
and listserv then processes the message using its own list policies, and 
only if the message isn't blocked either by issues with the listserv 
itself or by list/listserv policies (maybe some lists only take posts 
from subscribers, or consider the post spam for some reason, and block 
it), does it get send back to gmane, along with the other list 
subscribers.

Then when gmane gets the post, it has to figure out what to do with it, 
since it's cross-posted.  As it happens, gmane apparently follows the 
normal news server rules here and after converting the mailing list 
headers back to news, cross-posts the message to all groups/lists it 
carries.

The critical difference here being, gmane may or may not get the message 
again from the other lists it was sent to, while a normal news server 
will only have the message propagated to it once (when multi-peering, a 
normal news server will reject the message as already seen before 
propagation, thus reducing transfer volume between the peers), but it's 
already cross-posted, either way.  So gmane basically throws away the 
cross-posted messages it has already received from other lists, while 
normal news servers don't ever receive them again in the first place as 
they already have them, so don't have to throw them away.


But the real difference with crossposts for gmane users, as opposed to 
single-list-posts, is that single lists posts appear if and only if that 
list is actually working and doesn't block the post, so it gets sent back 
to the subscribers including gmane, while cross-posts will appear on gmane 
on all lists/groups as soon as they're sent back to gmane by just one.

So if there's a broken listserv, as we apparently have here, the only 
posts appearing on that list while it or its server is broken will be 
ones that were cross-posted to some other list, that was /not/ broken at 
the time.

Which is why your cross-post to gmane.discuss appeared almost instantly.  
Because that list happens to be hosted locally by gmane, and when it 
accepted and set out the post, it appeared on gmane in the emacs group as 
well, even tho that server is broken and not returning posts at this time.

> What would happen if I cross-posted to A and B as above, only I didn't
> subscribe to B? Would it still work on A? And if so, would that qualify
> a magic?

It depends on the exact mechanism both gmane and your news client use to 
process cross-posts.  If gmane is actually posting them to all cross-
posted groups, subscription wouldn't matter.  If it's not, and it's just 
your client managing things based on the message headers, then 
subscription would matter and you'd only see it in the otherwise broken 
group if you were subscribed to the unbroken one, because it's your 
client actually doing it, not gmane, and your client could only do it if 
it saw the message in the first place, which it could only do if it got 
the message on a different group.

And yes, FWIW, my client (pan) does handle messages this way, because it 
processes messages by message-id, so as long as that header stays the 
same (as it should unless something's broken somewhere), if a message is 
cross-posted, it will appear in all groups it was posted to once my 
client sees it in just one of them.  Well, it will if it's still in 
cache, anyway.  My client has a tiny default cache, but I have it set to 
effectively unlimited, as I use the cache as a message archive for a 
number of mostly text groups, from gmane and other servers.

I don't know what your client (gnus, based on the headers) does.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: aioe (was: Re: Is there a problem with Gmane at the moment?)

[Emanuel Berg]

Duncan <1i5t5.duncan@cox.net> writes:

> Which is why your cross-post to gmane.discuss
> appeared almost instantly. Because that list
> happens to be hosted locally by gmane, and when it
> accepted and set out the post, it appeared on gmane
> in the emacs group as well, even tho that server is
> broken and not returning posts at this time.
>
> ...
>
> It depends on the exact mechanism both gmane and
> your news client use to process cross-posts.
> If gmane is actually posting them to all
> cross-posted groups, subscription wouldn't matter.

OK, let's see if I understand it:

If Gmane does it, altho one server is down, by
crossposting to a group associated with another
server, that is up, all Gmane users still get it in
the group associated with the server, that is down!

However if the client does it, it only works for
people that also subscribe to the group associated
with the server, that is up!

But what about all other people? They don't get it.
So perhaps this method isn't that good as the post
only reach a subset of the group's subscribers.

Perhaps better to wait for the down server
to resurface?

PS. Yes, I use Gnus.

-- 
underground experts united .... http://user.it.uu.se/~embe8573
Emacs Gnus Blogomatic ......... http://user.it.uu.se/~embe8573/blogomatic
                   - so far: 30 Blogomatic articles -                   

Re: aioe (was: Re: Is there a problem with Gmane at the moment?)

[Duncan]

Emanuel Berg posted on Mon, 16 May 2016 00:43:57 +0200 as excerpted:

> OK, let's see if I understand it:
> 
> If Gmane does it, altho one server is down, by crossposting to a group
> associated with another server, that is up, all Gmane users still get it
> in the group associated with the server, that is down!
> 
> However if the client does it, it only works for people that also
> subscribe to the group associated with the server, that is up!
> 
> But what about all other people? They don't get it.
> So perhaps this method isn't that good as the post only reach a subset
> of the group's subscribers.
> 
> Perhaps better to wait for the down server to resurface?

Correct... AFAIK, anyway.

(I have some experience with news and of course mail and have read and I 
believe understood at a non-implementor's level anyway, many of the news 
and mail RFCs, but I'm not a dev and thus obviously haven't actually 
implemented my own news clients or servers, neither am I a gmane admin so 
I don't have any behind the scenes knowledge there, but with those 
caveats, yes, I believe your summary to be correct as a high-level 
description. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman