reading the C source of Emacs

reading the C source of Emacs

[Oliver Scholz]

I have just taught myself the very first baby steps of C. I want to
read and understand the C sources of Emacs.

Not surprisingly I am pretty lost. What is a good starting point to
read those sources? Is there any recommended order of reading?

Does anyone have any other recommendations on how to proceed? Can
anyone -- for example -- recommend a shorter and simpler program whose
sources are well documented and that I could study as a training
before I try to read the Emacs sources? I am not interested in C
programming in general, I just want to understand Emacs, if this is
possible for me at all.

    Oliver
-- 
22 Nivôse an 211 de la Révolution
Liberté, Egalité, Fraternité!

Re: reading the C source of Emacs

[David Kastrup]

Oliver Scholz <alkibiades@gmx.de> writes:

> I have just taught myself the very first baby steps of C. I want to
> read and understand the C sources of Emacs.

I have just taught myself the first 2 verb forms of ancient Greek (of
which there are basically {singular,dual,plural} x {1st,2nd,3rd person} x
{indicative,optative,conjunctive,participle,infinitive,imperative} x
{present,past,perfect,past perfect,aorist,future,past future} x
{active,medium,passive} (probably forgot a few here, and of course
you still have to decline your participles).  I want to read and
understand Thukydides.

> Not surprisingly I am pretty lost.

Same here.  You have to be aware that Emacs is basically a _Lisp_
machine with Lisp data structures.  The C code is vary much different
from ordinary C code because it constantly has to access and modify
non-C structures.

> What is a good starting point to read those sources? Is there any
> recommended order of reading?

grep for the Elisp function that you suspect to be the source of your
favorite annoyance/bug.  Fiddle around.  Anyhow: if you are not yet
comfortable with Elisp, start there.  It is not really possible to
understand much of what is going on inside of Emacs without knowing
the Lisp.  There is a short Elisp introduction available in the CVS
version of Emacs.

> Does anyone have any other recommendations on how to proceed? Can
> anyone -- for example -- recommend a shorter and simpler program
> whose sources are well documented and that I could study as a
> training before I try to read the Emacs sources? I am not interested
> in C programming in general, I just want to understand Emacs, if
> this is possible for me at all.

It is pretty difficult.  Emacs is rather complex, and you are
basically are asking for how to learn to play Back partitas on the
violin, and you don't want to learn playing the violin in general.  It
is possible, but it would annoy any prospective teachers and require
access to a basement where you are out of ear reach from your loved
ones.

The Emacs Lisp manual has a section "Emacs Lisp internals".  Required
reading, of course.

I believe that XEmacs might have more extensive internals
documentation, but it could well be that most of it does not apply to
Emacs: XEmacs developers would probably be more thorough documenting
the differences than the similarities between the two.  This is just
un aneducated guess of mine, however, and understanding XEmacs is
certainly a better help to understanding Emacs than, say,
understanding gcc would be.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: reading the C source of Emacs

[Oliver Scholz]

David Kastrup <dak@gnu.org> writes:

> Oliver Scholz <alkibiades@gmx.de> writes:

>> I have just taught myself the very first baby steps of C. I want to
>> read and understand the C sources of Emacs.
>
> I have just taught myself the first 2 verb forms of ancient Greek (of
> which there are basically {singular,dual,plural} x {1st,2nd,3rd person} x
> {indicative,optative,conjunctive,participle,infinitive,imperative} x
> {present,past,perfect,past perfect,aorist,future,past future} x
> {active,medium,passive} (probably forgot a few here, and of course
> you still have to decline your participles).  I want to read and
> understand Thukydides.

Under certain conditions this is not necessarily as impossible as you
seem to suggest. Knowing only two concrete verb forms would hardly
count even as "knowing the first baby steps", but knowing only a few
of them would, however, still imply that you have a very basic
understanding of the overall structure--so to say, the gestalt--of
Greek grammar: that you have to differentiate between forms like the
ones you mentioned above, that many verb forms can be divided into
different parts like prefix, root and suffix, and stuff like that. And
if you were furthermore, for example, familiar with the Homeric
dialect, and for some strange reasons totally ignorant only of
classical Attic Greek, you'd have at least a starting point. Your
knowledge of Homer won't help you much, but you are not _totally_
helpless, because you can make some guesses about the sentence you try
to decipher that are not _too_ wild. You can't sit down and read the
Peloponesian War from the beginning to the end, of course. But
suppose, you have a commentary to the book that tells you some general
information about it, like "This and that is the historical, social,
political and philosophical background of Thukydides; roughly stated
the book deals with ... and covers ... in a way that ...". Now you try
to read a single, short paragraph of the Peloponesian War. Your
commentary would give you some very general information about this
paragraph, like: which its context is, what information it covers, how
it relates to the rest of the book etc.. Then you *do* have a chance
to decipher that paragraph, armed with nothing but your very basic
knowledge, your commentary, a grammar book and a dictionary. You will
have a hard time with each single sentence, but based on your
understanding of the general meaning of that paragraph, you may be
able to make reasonable guesses about their relation to the whole,
isolate the problematic parts, decide whether their understanding is
crucial for understanding what you are looking for, and, if necessary,
do some investigation in your grammar book or in different places or
formulate a specific question to an expert. But you need to have a
vague idea of the whole, if you want to do it this way.

That's my situation: I have a very basic knowledge of the grammar and
I know how to use a dictionary and I try to decipher a paragraph in
the C source of Emacs. But what is this sentence, this chapter, what
is this whole book talking about?

As you do this several times with several paragraphs that are of
particular interest for you, your familiarity with the language in
general and with Thukydides' idiom in particular will slowly (very
slowly) increase. Fama says that H. Schliemann learned Greek by
comparing Greek texts with their German translations. This method as
well as the method I described above are of course among the hardest
ways to learn classical Attic Greek. I you wanted to learn "Greek in
general", one would recommend another approach: "Don't start with
Thukydides, first get an elementary text book, do some exercises, then
start with easier authors ..."

But suppose, you are not interested in Attic prose "in general". Maybe
you actually dislike Attic prose and find most authors boring (but you
really do love Thukydides). Perhaps you are a historian writing a book
about Thukydides, who wants to gain a deeper understanding of certain
selected parts of the Peloponesian War by studying the Greek "source
code". Maybe you are perverse enough to actually *enjoy* puzzle games
like deciphering a sentence of the Peloponesian War without being
fluent in Greek. You would have fun doing it the hard way, but doing
exercises with an elementary text book until you know the whole
grammar would annoy you to death.

Still some expert could advise you to start with a elementary text
book, because she knows how difficult Thukydides is. And if you really
want a "real life" author instead of silly example sentences written
for educational purposes, she could decide that to start with
Thukydides in the way you want is not for mere mortals and advise you
to try your luck with different authors first. She then could give you
some more specific hints: "Don't take Theokritos, that's Doric lyrics
and it won't help you with Attic prose."; "Don't read the
tragedians. Sure, when you finally feel comfortable with them, you are
probably ready for Thukydides, but they are only slightly less
difficult for a beginner than Thukydides, and, worse, they are
difficult in other ways."; "Don't read Homer. Homer is actually rather
easy, if you don't mind consulting a dictionary frequently, but Homer
differs too much from Attic prose."; "Instead Xenophon is a good
choice. Read a lot of Xenophon, if you can deal with it. He is
relatively easy and may serve as a good preparation for Thukydides."
Such information would be invaluable, when you only know the very
basics of Greek and have not read any real author yet. Otherwise it
might happen that you decide to pick up Pindaros as an "easier" author
for you preparational training.

If your goal is to "learn Greek in general" there is no way around
Homer and the tragedians. But if your goal is to understand Thukydides
only, and you are not interested in other authors at all, you may
safely skip some of them.

***

Probably my wording was misleading. I don't pet the idea that I sit
down now and read the sources of Emacs from the beginning to the
end. My idea is indeed rather that I study parts of the code of
particular interest for me, thereby gradually improving my knowledge
of the language and my knowledge of Emacs' internals together.

Basically that's the way I learned Elisp. I recall now that I learned
English in a similar way. I started to learn English by reading a
novel by Walter Scott. My knowledge of the grammar and the vocabulary
was not worth mentioning at that time, but it was enough to make
reasonable guesses about what a particular paragraph is talking
about. Through reading more English novels it gradually became easier
to do so. My English (as well as my Elisp probably) is very bad up
until today, I am afraid, but experience shows that I can help myself
in most situations. As long as I don't want to become a good English
speaker in general and write best seller novels in English, this is
o.k. for me.

My problem with Emacs is that I don't have an overview. Being
unfamiliar with the language C is only one part of the problem on my
side. Being unfamiliar with what is expressed in that language is the
other one. What data structures are especially important? In which
parts is the code divided conceptually? Are there certain layers of
abstraction that I have to keep in mind? WTF is the purpose of the
damn cryptic source file xy? In other words: what is the gestalt of
the internals?

When I have an idea, a mental model of what is happening in the code,
how vague ever -- it may even be partly wrong -- and when I can make
some not too wild guesses based on this mental model about what a
particular function or a data structure is supposed to do, then I have
a chance to decipher the code. There are some explanations of the kind
that I would like to have in some places, for example at the beginning
of xdisp.c. I was half hoping that there are other crucial parts
explained in such a way, and that there is maybe a similar explanation
of how those parts are assembled together. Provided with such a vague
understanding of the rough shape of the general picture, it would be
much easier for me to understand a specific part of it in detail.

[...]
> The Emacs Lisp manual has a section "Emacs Lisp internals".  Required
> reading, of course.

*sound of someone clapping his hand against his forehead*

Silly me! I have forgotten that this chapter exists. Thank you for the
reminder! This carries me a big step further, although it still leaves
many questions open. But I'll dig into it anyways, because I hope that
I will be able to ask some more specific questions in the future.

> I believe that XEmacs might have more extensive internals
> documentation, but it could well be that most of it does not apply to
> Emacs: XEmacs developers would probably be more thorough documenting
> the differences than the similarities between the two.  This is just
> un aneducated guess of mine, however, and understanding XEmacs is
> certainly a better help to understanding Emacs than, say,
> understanding gcc would be.

I'll give it a try. Thank you.

    Oliver
-- 
23 Nivôse an 211 de la Révolution
Liberté, Egalité, Fraternité!

Re: reading the C source of Emacs

[Kai Großjohann]

Oliver Scholz <alkibiades@gmx.de> writes:

> I have just taught myself the very first baby steps of C. I want to
> read and understand the C sources of Emacs.

Wow.  An admirable goal.  I do not understand the C sources of Emacs.
Amazingly, this has not stopped me from doing modifications here and
there.  (Or was it only here?  Yes, I think it was only one -- minor!
-- modification.)

And I agree with David that it might be useful to do an on-demand
kind of learning, rather than trying to just grok everything.

Is there anything that you had in mind modifying in the long term?
Maybe you could turn your plan around: start with the modification
that you wanted to postpone until you understood.
-- 
Ambibibentists unite!

Re: reading the C source of Emacs

[Oliver Scholz]

kai.grossjohann@uni-duisburg.de (Kai Großjohann) writes:

[...]
> Is there anything that you had in mind modifying in the long term?
> Maybe you could turn your plan around: start with the modification
> that you wanted to postpone until you understood.
[...]

Hmm, I haven't thought of that. But it could be an idea. Many of my
wishes are far above my head at the moment, but maybe I could find or
invent something which is sufficently limited to play a bit with it
and to get a feeling for it. Thanks you.

    Oliver
-- 
23 Nivôse an 211 de la Révolution
Liberté, Egalité, Fraternité!

Re: reading the C source of Emacs

[Kai Großjohann]

Oliver Scholz <alkibiades@gmx.de> writes:

> Hmm, I haven't thought of that. But it could be an idea. Many of my
> wishes are far above my head at the moment, but maybe I could find or
> invent something which is sufficently limited to play a bit with it
> and to get a feeling for it. Thanks you.

I don't understand that "above your head" part.  Previously, you
wanted to grok all of the C code.  Now you say grokking only a small
bit to implement something is too difficult.

Well.  Don't be offended by the previous paragraph.  I was trying to
be provocative.  Of course, if what you have in mind is inherently
difficult to do (how to extend doctor.el to pass the Turing test,
say), then just grokking all of the C code of Emacs is easier.  But
if what you have in mind is not inherently so difficult, then I
believe that having something to focus on to guide you through the
code will help a lot.

The feature you want to implement can be your guide through the source
code.

FWIW, C is quite small a language.  So it's not difficult to master
all of the syntax of it, except for some dark areas best left to the
real gurus.  The operator precedence is part of that dark area, and
also the syntax for specifying certain data types.

I recommoned the cdecl program:

cdecl> declare f as pointer to function (int) returning pointer to char
char *(*f)(int )

It accurately describes itself as a translation tool from gibberish
to English and vice versa :-)
-- 
Ambibibentists unite!

Re: reading the C source of Emacs

[Oliver Scholz]

kai.grossjohann@uni-duisburg.de (Kai Großjohann) writes:

> Oliver Scholz <alkibiades@gmx.de> writes:
>
>> Hmm, I haven't thought of that. But it could be an idea. Many of my
>> wishes are far above my head at the moment, but maybe I could find or
>> invent something which is sufficently limited to play a bit with it
>> and to get a feeling for it. Thanks you.
>
> I don't understand that "above your head" part.  Previously, you
> wanted to grok all of the C code.  Now you say grokking only a small
> bit to implement something is too difficult.

Actually I mainly wanted to unterstand the display/redisplay part,
which is not such a small bit.

It seems that I misunderstood you. I understood your advice as:
'Fiddle around with the code first, then try to understand' instead of
'First try to understand, then fiddle around with the code'.

That is not as absurd as it sounds. It shouldn't be too hard to (copy
and) modify a relatively simple DEFUN in the C code slightly. This
will probably lead to a lot of failures, but this is o.k., because
doing so is fun in itself. Based on what I learn this way I could
then carefully start to try to deal with more complex things. I am
already looking for a candidate.

    Oliver
-- 
23 Nivôse an 211 de la Révolution
Liberté, Egalité, Fraternité!

Re: reading the C source of Emacs

[David Kastrup]

Oliver Scholz <alkibiades@gmx.de> writes:

> kai.grossjohann@uni-duisburg.de (Kai Großjohann) writes:
> 
> > Oliver Scholz <alkibiades@gmx.de> writes:
> >
> >> Hmm, I haven't thought of that. But it could be an idea. Many of my
> >> wishes are far above my head at the moment, but maybe I could find or
> >> invent something which is sufficently limited to play a bit with it
> >> and to get a feeling for it. Thanks you.
> >
> > I don't understand that "above your head" part.  Previously, you
> > wanted to grok all of the C code.  Now you say grokking only a small
> > bit to implement something is too difficult.
> 
> Actually I mainly wanted to unterstand the display/redisplay part,
> which is not such a small bit.

Nobody understands that, not even Gerd, and he wrote it.  Go for it:
I have a feeling that the areas you might be interested in could be
those that I want to see improvement.

I can give you a lot of suggestions for image support (which is
abysmally slow, actually), but I am afraid that they would require an
understanding of what is efficient and inefficient...  and that is
probably not some basic knowledge easily come by.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: reading the C source of Emacs

[Stefan Monnier <foo@acm.com>]

> Not surprisingly I am pretty lost. What is a good starting point to
> read those sources? Is there any recommended order of reading?

In my experience, the best way to start understanding a (large) piece of
code, is to look at a small part of it that you expect you should be able
to more or less understand, and once you feel like you understand it
somewhat, change it in an apparently innocuous way.

Most likely it will wreak havoc because you actually completely missed
several important aspects of what the code does, but by looking at how it
breaks, you'll get to understand those things you missed.

Having knowledgeable people at hand makes the experience less
painful, of course.


        Stefan

Re: reading the C source of Emacs

[Oliver Scholz]

[-- Attachment #1: Type: text/plain, Size: 685 bytes --]

Thank you all for you advice. You helped to set me on the track. But
it is indeed a rather difficult read for me. My head dizzles is still
a bit dizzling.

Below are my very first steps in C: a version of `find-if' as a
starter. I'd appreciate any comment. (Did I miss something important,
for example?)

One thing that I noticed is that my `find-if' causes an infinite loop,
when applied to a recursive (?) list, à la:

(setq my-list (list 'alpha 'beta 'gamma))

(setcdr (cddr my-list) my-list)

(find-if (lambda (elt) (eq elt 'zeta)) my-list)

I couldn't think of a remedy for this. How could I solve this problem?
OTOH this happens to `copy-sequence', too. So maybe this is o.k.?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: fns.diff --]
[-- Type: text/x-patch, Size: 1860 bytes --]

Index: src/fns.c
===================================================================
RCS file: /cvsroot/emacs/emacs/src/fns.c,v
retrieving revision 1.327
diff -u -r1.327 fns.c
--- src/fns.c	6 Jan 2003 15:41:17 -0000	1.327
+++ src/fns.c	17 Jan 2003 04:35:26 -0000
@@ -2913,6 +2913,57 @@
 
   return sequence;
 }
+
+#define FIND_IF_1(VARIABLE, PREDICATE, ELEMENT)    \
+       if (! NILP (call1 ((PREDICATE), (ELEMENT)))) \
+	 {                                          \
+	   (VARIABLE) = (ELEMENT);	            \
+	   break;                                   \
+	 }
+
+DEFUN ("find-if", Ffind_if, Sfind_if, 2, 2, 0,
+       doc: /* Return the first element for which PREDICATE returns non-nil. */)
+     (predicate, sequence)
+     Lisp_Object predicate, sequence;
+{
+  Lisp_Object elt = Qnil;
+
+  if (STRINGP (sequence))
+    {
+      int nchars = SCHARS (sequence);
+      int cidx, bidx;
+
+      for (cidx = bidx = 0; cidx < nchars;)
+	{
+	  int c;
+	  FETCH_STRING_CHAR_ADVANCE (c, sequence, cidx, bidx);
+	  FIND_IF_1 (elt, predicate, (make_number (c)));
+	  QUIT;
+	}}
+  else if (VECTORP (sequence))
+    {
+      int i;
+      int len = ASIZE (sequence);
+
+      for (i = 0; i < len; i++)
+	FIND_IF_1 (elt, predicate, (AREF (sequence, i)));
+      QUIT;
+    }
+  else /* It ought to be a list. */
+    {
+      while (! NILP (sequence))
+	{
+	  if (! (CONSP (sequence)))
+	    return wrong_type_argument (Qsequencep, sequence);
+	  else FIND_IF_1 (elt, predicate, (XCAR (sequence)));
+	  sequence = XCDR (sequence);
+	  QUIT;
+	}}
+  return elt;
+}
+
+    
+
 \f
 /* Anything that calls this function must protect from GC!  */
 
@@ -5580,6 +5631,8 @@
   defsubr (&Snconc);
   defsubr (&Smapcar);
   defsubr (&Smapc);
+  //  defsubr (&Sexplode);
+  defsubr (&Sfind_if);
   defsubr (&Smapconcat);
   defsubr (&Sy_or_n_p);
   defsubr (&Syes_or_no_p);

[-- Attachment #3: Type: text/plain, Size: 85 bytes --]


    Oliver
-- 
28 Nivôse an 211 de la Révolution
Liberté, Egalité, Fraternité!

[-- Attachment #4: Type: text/plain, Size: 151 bytes --]

_______________________________________________
Help-gnu-emacs mailing list
Help-gnu-emacs@gnu.org
http://mail.gnu.org/mailman/listinfo/help-gnu-emacs

Re: reading the C source of Emacs

[David Kastrup]

Oliver Scholz <alkibiades@gmx.de> writes:

> Thank you all for you advice. You helped to set me on the track. But
> it is indeed a rather difficult read for me. My head dizzles is still
> a bit dizzling.
> 
> Below are my very first steps in C: a version of `find-if' as a
> starter. I'd appreciate any comment. (Did I miss something important,
> for example?)
> 
> One thing that I noticed is that my `find-if' causes an infinite loop,
> when applied to a recursive (?) list, à la:

There are several tricks for dealing with recursive data structures.
One is to have two traversals active at the same time, where one
traversal progresses just at every second step.  If they should ever
catch up, you have encountered a loop in a data structure.

I doubt this would be worth the trouble: just make sure that C-g will
be able to abort your function should it get stuck.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: reading the C source of Emacs

[Kai Großjohann]

Oliver Scholz <alkibiades@gmx.de> writes:

> One thing that I noticed is that my `find-if' causes an infinite loop,
> when applied to a recursive (?) list, à la:

Many list functions exhibit strange behavior when applied to
non-lists.  The data structure you show is a non-list: the last cdr
pointer of a list should be nil.

I wouldn't worry about it too much.
-- 
Ambibibentists unite!

Re: reading the C source of Emacs

[Janusz S. Bień]

On Sat, 11 Jan 2003  Oliver Scholz <alkibiades@gmx.de> wrote:

> I have just taught myself the very first baby steps of C. I want to
> read and understand the C sources of Emacs.

On Sat, 11 Jan 2003  kai.grossjohann@uni-duisburg.de (Kai Großjohann) wrote:


[...]

> Wow.  An admirable goal.  I do not understand the C sources of Emacs.
> Amazingly, this has not stopped me from doing modifications here and
> there.  (Or was it only here?  Yes, I think it was only one -- minor!
> -- modification.)
> 
> And I agree with David that it might be useful to do an on-demand
> kind of learning, rather than trying to just grok everything.

I support all of this.

Try to understand better the Emacs functions you use. Start with `C-h
k' or `C-h f', then follow the help links to Lisp and C code. Make
sure you have the proper TAGS file and use `M-.' to navigate inside
the program source.

JSB

-- 
                     ,   
dr hab. Janusz S. Bien, prof. UW
Prof. Janusz S. Bien, Warsaw Uniwersity
http://www.orient.uw.edu.pl/~jsbien/

Re: reading the C source of Emacs

[Miles Bader]

On Sat, 11 Jan 2003  Oliver Scholz <alkibiades@gmx.de> wrote:
> And I agree with David that it might be useful to do an on-demand
> kind of learning, rather than trying to just grok everything.

That's for sure.  If you just to try to `grok everything,' you'll
quickly go insane; there's just too much code, and much of it can be
confusing if you just skim it (it isn't the most well-organized code in
the world).

Trying to figure how it does one particular thing is usually (!)
straight-forward though.

-Miles
-- 
80% of success is just showing up.  --Woody Allen