Documentation of nnoo.el

Documentation of nnoo.el

[Stefan Monnier]

nnoo.el is in dire need of some documentation.

Could someone add a little bit of:
- something in Commentary: describing which are the main entry points
  and how to use them,
- a docstring for nnoo-definition-alist describing precisely its format,
- a description of the MAP arg to defvoo,
- ...


        Stefan

Re: Documentation of nnoo.el

[Eric Abrahamsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> nnoo.el is in dire need of some documentation.
>
> Could someone add a little bit of:
> - something in Commentary: describing which are the main entry points
>   and how to use them,
> - a docstring for nnoo-definition-alist describing precisely its format,
> - a description of the MAP arg to defvoo,
> - ...

This is a bit nuts, but: are there any technical/policy obstacles to
considering an EIEIO re-write of Gnus? Nnoo.el is re-inventing the
wheel...

Of course, who would actually go and do that is another question
altogether. But in principle?

Yours,
Eric

Re: Documentation of nnoo.el

[Stefan Monnier]

> This is a bit nuts, but: are there any technical/policy obstacles to
> considering an EIEIO re-write of Gnus? Nnoo.el is re-inventing the wheel...

IIUC this wheel didn't exist, or was not widely available back then.
But in any case, I can't answer because I don't understand nnoo.el
(which gets us back to my request for documentation).

> Of course, who would actually go and do that is another question
> altogether. But in principle?

It all depends on how easy it is and what would be the benefits.


        Stefan

Re: Documentation of nnoo.el

[Eric Abrahamsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> This is a bit nuts, but: are there any technical/policy obstacles to
>> considering an EIEIO re-write of Gnus? Nnoo.el is re-inventing the wheel...
>
> IIUC this wheel didn't exist, or was not widely available back then.
> But in any case, I can't answer because I don't understand nnoo.el
> (which gets us back to my request for documentation).

It's probably true EIEIO wasn't quite ready back in the day.

nnoo.el is basically just an object-oriented system replicated with pure
functions. deffoo creates methods, defvoo creates slots. `nnoo-declare'
is kind of the equivalent of `defclass'. `nnoo-change-server' is the
main function for switching servers: it copies the defvoo slot values
for a single server into the respective global variables.

I don't know much more than that because... it's not well documented.

>> Of course, who would actually go and do that is another question
>> altogether. But in principle?
>
> It all depends on how easy it is and what would be the benefits.

In principle it wouldn't be too hard: nnoo tries pretty hard to mimic
OO, so the paths are already in place. In practice, of course, I'm sure
it would be a huge pain.

Potential benefits:

1. Less code -- we could do away with a lot of redundancy.
2. Real object encapsulation. This is the main one in my mind -- right
   now it's easy to end up with variable leakage, and pretty hard to
   find the source of the leakage. Instead of switching global variable
   values as we switch servers, the servers themselves would just get
   passed around.
3. Easier hackage/extensibility. Contributors currently have very hard
   time figuring out how everything works; this would help. Subclassing
   would also make it much easier for people to write new backends, or
   modify existing ones.

Basically it looks like a textbook argument for the benefits of OO
programming.

Doing this would provide opportunities to fix some other odd things
about Gnus. Here's are a couple of suggestions for steps:

1. First, only make backends into classes. This would give us the chance
   to do away with the select-method/secondary-select-method
   distinction, and just have a list of defined servers.
2. Consider turning groups into classes. Group parameters would then be
   object slots. This might not be worth it.
3. Create a "marks-store" class: each server has its own object for
   storing marks. The default one would just write marks to a file.
   Subclasses could save marks to the cloud, etc. This would allow us to
   tackle the .newsrc.eld mess. If you look at bug
   reports/complaints/rants about Gnus, a very large proportion are
   related to marks somehow getting corrupted or out of sync. This would
   be a huge amount of work, though.

Anyway, those are my preliminary thoughts. Maybe someone else will chime
in. Probably to tell me I'm crazy!

Eric

Re: Documentation of nnoo.el

[Rasmus]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> 3. Easier hackage/extensibility. Contributors currently have very hard
>    time figuring out how everything works; this would help. Subclassing
>    would also make it much easier for people to write new backends, or
>    modify existing ones.

It's extremely hard to write patches for Gnus 'cause it's so hard to
figure out what is going on.

And dev. documentation is/docstrings are not the best, though user
documentation is pretty awesome.

Rasmus

-- 
C is for Cookie

Re: Documentation of nnoo.el

[Eric Abrahamsen]

Rasmus <rasmus@gmx.us> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> 3. Easier hackage/extensibility. Contributors currently have very hard
>>    time figuring out how everything works; this would help. Subclassing
>>    would also make it much easier for people to write new backends, or
>>    modify existing ones.
>
> It's extremely hard to write patches for Gnus 'cause it's so hard to
> figure out what is going on.
>
> And dev. documentation is/docstrings are not the best, though user
> documentation is pretty awesome.

Which isn't to say that an EIEIO rewrite would magically result in
better documentation, but if the code were simplified at the same time
that a general overhaul were undertaken... it would make things more
comprehensible.

Re: Documentation of nnoo.el

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> nnoo.el is in dire need of some documentation.
>
> Could someone add a little bit of:
> - something in Commentary: describing which are the main entry points
>   and how to use them,
> - a docstring for nnoo-definition-alist describing precisely its format,
> - a description of the MAP arg to defvoo,
> - ...

Well, there's the "Writing New Back Ends" node in the Gnus manual.  I
think it should cover most of the basics?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Documentation of nnoo.el

[Lars Magne Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> nnoo.el is basically just an object-oriented system replicated with pure
> functions. deffoo creates methods, defvoo creates slots. `nnoo-declare'
> is kind of the equivalent of `defclass'. `nnoo-change-server' is the
> main function for switching servers: it copies the defvoo slot values
> for a single server into the respective global variables.

Yup.  It was written that way (back in...  what?  94?) because there was
some resistance to altering the "interface" defined by previous Gnus
backends.  (See nnnil.el for a basic summary of what that is.)  By
having Gnus switch around variables behind the backends' backs, the
"interface" was preserved while Gnus could go on having multiple servers
of the same type, etc.

I don't know whether that was a good trade-off (keeping the interface
vs. introducing lots of magic), but if I were to write that stuff now, I
wouldn't do it that way.  :-)  I'd pass in explicit object describing
the server state to each backend function, and I wouldn't have any
"global" voo variables at all.

If anybody wants to rewrite all the backends that way -- please go ahead.

> 1. First, only make backends into classes. This would give us the chance
>    to do away with the select-method/secondary-select-method
>    distinction, and just have a list of defined servers.

I think that's a totally orthogonal issue.  We could do that now, but
it'd break the current naming scheme for the groups in Gnus.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Documentation of nnoo.el

[Nikolaus Rath]

On Jul 12 2015, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> nnoo.el is in dire need of some documentation.
>>
>> Could someone add a little bit of:
>> - something in Commentary: describing which are the main entry points
>>   and how to use them,
>> - a docstring for nnoo-definition-alist describing precisely its format,
>> - a description of the MAP arg to defvoo,
>> - ...
>
> Well, there's the "Writing New Back Ends" node in the Gnus manual.  I
> think it should cover most of the basics?

Judging from that documentation, nnoo is only about inheritance, so that
you don't have to redefine identical functions if you want to re-use an
existing backend.

It doesn't explain at all what e.g. nnoo-change-server is good for, or
that nnoo also manages per-server state (which it seems to do, but I'm
not sure).

Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: Documentation of nnoo.el

[Eric Abrahamsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> nnoo.el is basically just an object-oriented system replicated with pure
>> functions. deffoo creates methods, defvoo creates slots. `nnoo-declare'
>> is kind of the equivalent of `defclass'. `nnoo-change-server' is the
>> main function for switching servers: it copies the defvoo slot values
>> for a single server into the respective global variables.
>
> Yup.  It was written that way (back in...  what?  94?) because there was
> some resistance to altering the "interface" defined by previous Gnus
> backends.  (See nnnil.el for a basic summary of what that is.)  By
> having Gnus switch around variables behind the backends' backs, the
> "interface" was preserved while Gnus could go on having multiple servers
> of the same type, etc.
>
> I don't know whether that was a good trade-off (keeping the interface
> vs. introducing lots of magic), but if I were to write that stuff now, I
> wouldn't do it that way.  :-)  I'd pass in explicit object describing
> the server state to each backend function, and I wouldn't have any
> "global" voo variables at all.

That's a lot earlier than I thought!

Does this mean you'd rather not go all the way with EIEIO? Maybe just
vectors or something?

> If anybody wants to rewrite all the backends that way -- please go ahead.

Well that's the real crux, isn't it :) I'm happy to try banging together
a preliminary example of how it might look. I don't think rewriting the
backends would be as much work as adjusting pretty much everything else
to use the new code.

>> 1. First, only make backends into classes. This would give us the chance
>>    to do away with the select-method/secondary-select-method
>>    distinction, and just have a list of defined servers.
>
> I think that's a totally orthogonal issue.  We could do that now, but
> it'd break the current naming scheme for the groups in Gnus.

It is orthogonal, but on the other hand it would take extra effort to
preserve. Do you mean the "nnml+ServerName:GroupName" scheme? I don't
quite see how doing away with the primary/secondary distinction would
affect that...

Eric

Re: Documentation of nnoo.el

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Does this mean you'd rather not go all the way with EIEIO? Maybe just
> vectors or something?

EIEIO would be fine by me...

> Well that's the real crux, isn't it :) I'm happy to try banging together
> a preliminary example of how it might look. I don't think rewriting the
> backends would be as much work as adjusting pretty much everything else
> to use the new code.

Well, every reference to a server variable in the backends would need to
be rewritten as an object access instead, so there would be quite a lot
of churn on the backend side, too.  I kinda thing there wouldn't be as
much on the Gnus side...

>>> 1. First, only make backends into classes. This would give us the chance
>>>    to do away with the select-method/secondary-select-method
>>>    distinction, and just have a list of defined servers.
>>
>> I think that's a totally orthogonal issue.  We could do that now, but
>> it'd break the current naming scheme for the groups in Gnus.
>
> It is orthogonal, but on the other hand it would take extra effort to
> preserve. Do you mean the "nnml+ServerName:GroupName" scheme? I don't
> quite see how doing away with the primary/secondary distinction would
> affect that...

Groups from primary servers aren't prefixed, while groups from
non-primary servers are...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Documentation of nnoo.el

[Eric Abrahamsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Does this mean you'd rather not go all the way with EIEIO? Maybe just
>> vectors or something?
>
> EIEIO would be fine by me...
>
>> Well that's the real crux, isn't it :) I'm happy to try banging together
>> a preliminary example of how it might look. I don't think rewriting the
>> backends would be as much work as adjusting pretty much everything else
>> to use the new code.
>
> Well, every reference to a server variable in the backends would need to
> be rewritten as an object access instead, so there would be quite a lot
> of churn on the backend side, too.  I kinda thing there wouldn't be as
> much on the Gnus side...

Sure -- I guess it's bound to be a whole lot of tweaking no matter what.

>>>> 1. First, only make backends into classes. This would give us the chance
>>>>    to do away with the select-method/secondary-select-method
>>>>    distinction, and just have a list of defined servers.
>>>
>>> I think that's a totally orthogonal issue.  We could do that now, but
>>> it'd break the current naming scheme for the groups in Gnus.
>>
>> It is orthogonal, but on the other hand it would take extra effort to
>> preserve. Do you mean the "nnml+ServerName:GroupName" scheme? I don't
>> quite see how doing away with the primary/secondary distinction would
>> affect that...
>
> Groups from primary servers aren't prefixed, while groups from
> non-primary servers are...

Ah, right. I guess I still don't see why it would be so terrible to lose
that distinction, though...

Taking a step back, there are two general structural alterations I'd
like to suggest. The first is splitting up the .newsrc.eld file a bit:
right now it's sort of kitchen-sink, and feels fragile. If servers were
classes, we could use eieio-persistent to write the server definitions
to one file, and keep marks in a different file (written there by a
different backend). Group definitions/parameters could go in the same
file as the server definitions. This would make the server/group file
somewhat user editable, at least user readable, while the marks backend
would be hands-off. Things like the topic topology could go in a third
file.

The other proposal is the agent. I'm imagining the agent as a mix-in
class for server backends. It would have many of the same methods as the
servers, and would wait in the background until a server signaled a
denied/no-connection/I'm-broken error. The agent would then set the
server's "status" slot to closed or denied, and the agent's methods
would check for that status and take over the action. That would allow
the agent to potentially usurp any server functionality that the server
itself was unable to perform.

Lastly, there's the problem that EIEIO seems to still be in flux, and an
EIEIO version of Gnus would probably only work reliably with Emacs master.

Anyway, all this is totally pie in the sky! But worth discussing, I
think.

Eric

Re: Documentation of nnoo.el

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

>> Groups from primary servers aren't prefixed, while groups from
>> non-primary servers are...
>
> Ah, right. I guess I still don't see why it would be so terrible to lose
> that distinction, though...

It's not terrible -- it's just incompatible.  :-)

> Taking a step back, there are two general structural alterations I'd
> like to suggest. The first is splitting up the .newsrc.eld file a bit:
> right now it's sort of kitchen-sink, and feels fragile.

Actually, I think it's too transparent.  :-)  It almost looks
user-editable, which makes people edit it, and things blow up.  It's
also a cop-out, because we can say "well, just edit the file" instead of
providing proper interfaces to the data.

It's not like Firefox users open Firefox data files in an editor and
start typing away...

> The other proposal is the agent. I'm imagining the agent as a mix-in
> class for server backends. It would have many of the same methods as the
> servers, and would wait in the background until a server signaled a
> denied/no-connection/I'm-broken error. The agent would then set the
> server's "status" slot to closed or denied, and the agent's methods
> would check for that status and take over the action. That would allow
> the agent to potentially usurp any server functionality that the server
> itself was unable to perform.

I think you've pretty much described how the Agent works now.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Documentation of nnoo.el

[Eric Abrahamsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>>> Groups from primary servers aren't prefixed, while groups from
>>> non-primary servers are...
>>
>> Ah, right. I guess I still don't see why it would be so terrible to lose
>> that distinction, though...
>
> It's not terrible -- it's just incompatible.  :-)

Fair enough.

>> Taking a step back, there are two general structural alterations I'd
>> like to suggest. The first is splitting up the .newsrc.eld file a bit:
>> right now it's sort of kitchen-sink, and feels fragile.
>
> Actually, I think it's too transparent.  :-)  It almost looks
> user-editable, which makes people edit it, and things blow up.  It's
> also a cop-out, because we can say "well, just edit the file" instead of
> providing proper interfaces to the data.
>
> It's not like Firefox users open Firefox data files in an editor and
> start typing away...

Well I don't think we disagree here -- .newsrc.eld is a bit dangerous,
for multiple reasons. If marks were stored in a separate file, we could
tell users the file was binary and scare them away from opening it :)

There are already proper interfaces to editing the data, and an EIEIO
version wouldn't change that. It would even add (ugh) a customize
interface for doing the editing, if that's your thing. But at the same
time, the server/group definitions file would be much more
human-readable, if people insisted on looking at it, and much harder to
break in mysterious ways.

>> The other proposal is the agent. I'm imagining the agent as a mix-in
>> class for server backends. It would have many of the same methods as the
>> servers, and would wait in the background until a server signaled a
>> denied/no-connection/I'm-broken error. The agent would then set the
>> server's "status" slot to closed or denied, and the agent's methods
>> would check for that status and take over the action. That would allow
>> the agent to potentially usurp any server functionality that the server
>> itself was unable to perform.
>
> I think you've pretty much described how the Agent works now.  :-)

That's the point! I don't want to change how anything works, just how
it's implemented. All I was saying is that the Agent lends itself very
nicely to generic functions and method composition.

Anyhow, it's maybe not worth trying to have an extended discussion about
this now, when it's all sort of nebulous, and I'm so unlikely to produce
any code right away. Maybe in the next few months, if I come up with a
basic "code sketch", I can post it here or on gnus.general and people
can see what they think?

Yrs,
Eric

Re: Documentation of nnoo.el

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Anyhow, it's maybe not worth trying to have an extended discussion about
> this now, when it's all sort of nebulous, and I'm so unlikely to produce
> any code right away. Maybe in the next few months, if I come up with a
> basic "code sketch", I can post it here or on gnus.general and people
> can see what they think?

Sure; sounds good.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no