Gnus work

Gnus work

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 1976 bytes --]

Hi all,

I've been very slowly reading the Gnus code base, and looking at how to
go about doing some refactoring, cleanups, and a few new features. I
wrote a roadmap which I'm attaching here -- it's not so much an actual
roadmap as a collection of possible changes, as a starting point for
argument. It's also not meant to be a mini bug tracker in Org -- my idea
was that if any of the ideas looked like they were going to be
acceptable, the todo items would turn into real bug reports on debbugs.

In general, I've been working from a few principles:

1. Replace Gnus' homemade mechanisms with equivalent mechanisms that
   have since appeared in core (in many cases Gnus had them first).
2. Add docstrings and comments to minimize bug-hunter bewilderment.
3. Look at unpicking functions so that there's a clearer demarcation
   between code that does server logic (marks and whatnot) and code that
   does UX/presentation stuff.
4. #3 would ideally allow more of the code to be testable.
5. Reduce number of dynamic variables, to cut back on "spooky action at
   a distance". To be honest I don't even know where to start here.

The roadmap is also not meant to address all of Gnus -- for the most
part I'm sticking to groups, servers, and marks. I don't feel
comfortable or competent to mess with other areas like
MIME/threading/washing/network/definitely not security. Some of those
areas already have "shepherds", anyway.

I'd like to make this document publicly accessible and editable (a
branch? Emacswiki?), and see if anyone has anything to contribute, or
otherwise wants to jump in. I think there's a lot of love for Gnus out
there, and my hope is that the roadmap might spur people to pop
up and share their ideas, and maybe offer to pitch in.

Where could I put it?

Eric

PS: No, I don't have any code yet. The one patch I do have touches
nothing but docstrings, code comments, and "Commentary" sections --
stuff I've added over the course of reading the code.



[-- Attachment #2: roadmap.org --]
[-- Type: application/vnd.lotus-organizer, Size: 10633 bytes --]

Re: Gnus work

[Emanuel Berg]

Eric Abrahamsen wrote:

> I've been very slowly reading the Gnus code
> base, and looking at how to go about doing
> some refactoring, cleanups, and a few
> new features.

What strikes me immediately when I read the
Gnus source is that the defuns are often very,
very long. But I'm not suggesting anyone do
something about it for that reason alone.
However if anyone does work on a defun and has
thus acquired a close to complete understanding
of it, it could be considered breaking it up,
possibly repeatedly, as the last step of work,
and then along lines that are as natural
as possible, of course.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Gnus work

[Eric Abrahamsen]

Emanuel Berg <moasen@zoho.com> writes:

> Eric Abrahamsen wrote:
>
>> I've been very slowly reading the Gnus code
>> base, and looking at how to go about doing
>> some refactoring, cleanups, and a few
>> new features.
>
> What strikes me immediately when I read the
> Gnus source is that the defuns are often very,
> very long. But I'm not suggesting anyone do
> something about it for that reason alone.
> However if anyone does work on a defun and has
> thus acquired a close to complete understanding
> of it, it could be considered breaking it up,
> possibly repeatedly, as the last step of work,
> and then along lines that are as natural
> as possible, of course.

I think the main problem is not so much that the defuns are long, but
that they do too much, there's overlap, and there's lots of
work-by-side-effect. A long defun wouldn't be so bad if you could scan
it and feel confident that you knew what it was doing.

What I *do* think is a problem is that variables and functions are named
inconsistently, and often confusingly. It just makes debugging that much
harder. But on the other hand, renaming just for renaming's sake...

Re: Gnus work

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 831 bytes --]


() Eric Abrahamsen <eric@ericabrahamsen.net>
() Wed, 04 Oct 2017 11:32:03 -0700

   3. Look at unpicking functions so that there's a clearer
      demarcation between code that does server logic (marks and
      whatnot) and code that does UX/presentation stuff.

What do you mean by "unpicking functions"?  Should that be read
as ADJ N (functions that have the "unpicking" quality) or V N
(the action of "unpicking" applied to functions), or something
else entirely?  I dig the zen feel of the sentence, anyway.  :-D

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: Gnus work

[Emanuel Berg]

Eric Abrahamsen wrote:

> I think the main problem is not so much that
> the defuns are long, but that they do too
> much, there's overlap, and there's lots of
> work-by-side-effect. A long defun wouldn't be
> so bad if you could scan it and feel
> confident that you knew what it was doing.

More or less. The form always has an impact on
the contents. I think what you describe is in
part a function of the habit of writing too
long functions.

> What I *do* think is a problem is that
> variables and functions are named
> inconsistently, and often confusingly.
> It just makes debugging that much harder.
> But on the other hand, renaming just for
> renaming's sake...

Indeed, small steps while at it.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Gnus work

[Tim Landscheidt]

Eric Abrahamsen <eric@ericabrahamsen.net> wrote:

> I've been very slowly reading the Gnus code base, and looking at how to
> go about doing some refactoring, cleanups, and a few new features. I
> wrote a roadmap which I'm attaching here -- it's not so much an actual
> roadmap as a collection of possible changes, as a starting point for
> argument. It's also not meant to be a mini bug tracker in Org -- my idea
> was that if any of the ideas looked like they were going to be
> acceptable, the todo items would turn into real bug reports on debbugs.

> […]

You, Sir, are my hero :-).  I'm always torn by Gnus: On the
one hand, it offers ample opportunity to do whatever you
want.  On the other hand, if what you want is not described
in the manual or there is not already a suitable code snip-
let out there, reading through the source is a nightmare.  I
do not even have a mental picture where what data is stored
at which point of time.

So I can only wholeheartedly support any effort to change
this :-).  One thing I would prefer, though, is documenting
(and refactoring) before any user-visible changes (and given
how users typically interact with Gnus's internals that can
be even small changes).  It's much easier to review and test
that a change does not break something if it does not
(should not) affect Gnus's outwards behaviour.  For example,
while I agree some parts of Gnus's configuration & Co. are
way too byzantine, changing it in an incompatible way would
effectively say to Gnus users: "We broke your setup, you're
welcome."  So any such changes should come at the end of the
"road".

Tim

Re: Gnus work

[Julien Danjou]

[-- Attachment #1: Type: text/plain, Size: 1198 bytes --]

On Wed, Oct 04 2017, Eric Abrahamsen wrote:

Hi Eric,

> I've been very slowly reading the Gnus code base, and looking at how to
> go about doing some refactoring, cleanups, and a few new features. I
> wrote a roadmap which I'm attaching here -- it's not so much an actual
> roadmap as a collection of possible changes, as a starting point for
> argument. It's also not meant to be a mini bug tracker in Org -- my idea
> was that if any of the ideas looked like they were going to be
> acceptable, the todo items would turn into real bug reports on debbugs.

As a former regular Gnus contributor and as a current and long time Gnus
user, I want to give you my blessing. :-)

They are definitively ton of little code improvement you can make. All
of this technical debt reduction will lead to a better Gnus and a better
development future, so yes please!

I don't know how true it is nowadays, but a lot of what was blocking
improvement back then was support for old Emacs version and XEmacs
support. Now that Gnus is in Emacs source tree, I imagine that might not
applies anymore, so, lucky you! :)

-- 
Julien Danjou
# Free Software hacker
# https://julien.danjou.info

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Gnus work

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> 1. Replace Gnus' homemade mechanisms with equivalent mechanisms that
>    have since appeared in core (in many cases Gnus had them first).

Sounds good.  The backend interface is particularly grungy and could to
with a complete rewrite.  Since there are probably extremely few
out-of-tree backends these days, I think you can massacre the interface
with impunity without thinking about compatibility.

> 2. Add docstrings and comments to minimize bug-hunter bewilderment.

Sounds good.

> 3. Look at unpicking functions so that there's a clearer demarcation
>    between code that does server logic (marks and whatnot) and code that
>    does UX/presentation stuff.

There's a lot of long functions in Gnus, but there's also a reason for
many of them being that way.  The first is that when setting up modes
and stuff, there's a lot of things that just have to be, er, set up.
Replacing a long litany of stuff like

(setq truncate-lines t
      mode-line-format foo)

with

(gnus-set-truncate-lines-to-t)
(gnus-set-mode-line-format foo)

isn't that much clearer.  :-)

(I'm just kidding, but that's kinda the Unclean Bob style of
refactoring).

The other consideration is that Emacs Lisp is really slow.  Splitting
central functions into many component functions will make Gnus even
slower than it already is.  And Gnus is slow.

> 5. Reduce number of dynamic variables, to cut back on "spooky action at
>    a distance". To be honest I don't even know where to start here.

Good luck, but Gnus has a gazillion settings that modify its workings in
many, many ways, and they're all done by variables.  Trimming down that
overload of settings would be nice, but every time I've tried, there's
always somebody that relied on just that single thing you'd think that
nobody would be weird enough to actually use.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Gnus work

[Eric Abrahamsen]

Thien-Thi Nguyen <ttn@gnu.org> writes:

> () Eric Abrahamsen <eric@ericabrahamsen.net>
> () Wed, 04 Oct 2017 11:32:03 -0700
>
>    3. Look at unpicking functions so that there's a clearer
>       demarcation between code that does server logic (marks and
>       whatnot) and code that does UX/presentation stuff.
>
> What do you mean by "unpicking functions"?  Should that be read
> as ADJ N (functions that have the "unpicking" quality) or V N
> (the action of "unpicking" applied to functions), or something
> else entirely?  I dig the zen feel of the sentence, anyway.  :-D

Only when we realize the illusory nature of functions can we make Gnus
go faster? :)