Re: [External] : Re: Shrinking the C core

[Arthur Miller <arthur.miller@live.com>]

"Eric S. Raymond" <esr@thyrsus.com> writes:

> Richard Stallman <rms@gnu.org>:
>> Switching to Common Lisp would cause those painful incompatibilities.
>> It is unacceptable.
>
> I did some checking into this a few weeks ago and even wrote a bit of Lisp to do
> an audit.  There are other reasons I decided to get up to speed on Common Lisp
> recently, and because I have Emacs Lisp engraved on my forebrain this interested
> me in getting a good grasp on how incompatible with SBCL it actally is.
>
> Count of elisp core functions: 1476 
> Total SBCL/elisp function name collisions: 145 
> Primitives identical in elisp and SBCL: 116
> Primitives not known identical: 28
>
> The first two lines are crunched from querying SBCL's symbol list and
> groveling through the Emacs sourcefiles; I can supply the SBCL code I
> write to do this if anybody cares.
>
> The second two lines are from me reading the SBCL documentation a lot;
> I needed immediate motivation to do that and this was a good one.
>
> I was able to write trivial implementations for 15 of those 28 collisions.
> The 14 I didn't implement are these:
>
> (require read-char read random process-status process-plist print
> princ prin1-to-string prin1 load documentation defvar aref)
>
> What this means is that it would, in principle, be possible to write
> an SBCL core that implements all of the Emacs primitives, with only a rather
> small amount of shim code required to make existing elisp code execute
> in an environment where it thinks it sees *all* elisp primitives.
>
> The code would look something like this, where I have deleted all but one emulation
> to showcase setting up the emulation environment.
>
> 	;; Implement functions shared between CL and elisp in an elisp-like way
>
> 	(defun elisp-append (&rest sequences)
> 	  "(append &rest SEQUENCES)
>
> 	  Probably introduced at or before Emacs version 18.
> 	  This function does not change global state, including the match data.
>
> 	Concatenate all the arguments and make the result a list.
> 	The result is a list whose elements are the elements of all the arguments.
> 	Each argument may be a list, vector or string.
> 	The last argument is not copied, just used as the tail of the new list."
> 	  (apply 'append (mapcar
> 			  (lambda (it)
> 			    (if (listp it) it (listp it)))
> 			  sequences))
> 	  )
>
> 	(defmacro within-elisp (form)
> 	  "Evaluate FORM in an environment with elisp behavior."
> 	  `(flet ((append #'emacs-append)
> 		 (delete-file-internal #'elisp-delete-file-internal)
> 		 (eval #'elisp-eval)
> 		 (format #'elisp-format)
> 		 (intern #'elisp-intern)
> 		 (make-hash-table #'elisp-make-hash-table)
> 		 (make-string #'elisp-make-string)
> 		 (rename-file #'elisp-rename-file)
> 		 (sort #'elisp-sort)
> 		 (string #'elisp-string)
> 		 (string-equal #'elisp-string-equal)
> 		 (string-lessp #'elisp-string-lessp)
> 		 (symbol-value #'elisp-symbol-value)
> 		 (type-of #'elisp-type-of)
> 		 (unintern #'elisp-unintern)
> 		 (yes-or-no-p #'elisp-yes-or-no-p)
> 		 )
> 	    ,form)
> 	  )
>
> With this encapsulation, "painful incompatiblities" between elisp and
> a core written in SBCL would never need be visible to anyone who put
> an .el extension on their code to tell the core it should be executed
> using within-elisp.

There is a little bit more than just this; but that one was very clever
to smooth out basic differences. It all depends on the ambition of
course. I do wrap cl funcitons and re-implement some of those that are
missing in CL just to be able to attach the doc strings to symbols, and
counting on sbcl inlining to remove the function call. I am not sure yet
if it is a good strategy, but I can do it relatively automated. Looks
like this:

(defun take (n list)
  "Return the first N elements of LIST.
If N is zero or negative, return nil.
If N is greater or equal to the length of LIST, return LIST (or a copy)."
  ;; todo - this can probably be done without taking the length of the list
  (cl:when (cl:> n 0)
    (cl:let ((l (cl:length list)))
      (when (cl:> n l) (setf n l))
      (cl:subseq list 0 n))))

(defun ntake (n list)
  "Modify LIST to keep only the first N elements.
If N is zero or negative, return nil.
If N is greater or equal to the length of LIST, return LIST unmodified.
Otherwise, return LIST after truncating it."
  (cl:when (cl:> n 0)
    (cl:let ((l list))
      (eli:while (cl:and (cl:> n 1) (cl:cdr l))
             (cl:setf l (cl:cdr l))
             (cl:decf n))
      (cl:rplacd l nil)
      list)))

(declaim (inline nthcdr))
(defun nthcdr (n list)
  "Take cdr N times on LIST, return the result."
  (cl:nthcdr n list))

(declaim (inline nth))
(defun nth (n list)
  "Return the Nth element of LIST.
N counts from zero.  If LIST is not that long, nil is returned."
  (cl:nth n list))

But tere is more than just functions and macros.

The reader needs some help; Emacs uses different syntax for characters
and escapes for example; ? vs #\, but the escape syntax is were a real
hear pulling is. Also Emacs does not have character type but basically
uses whatever int comes out of C and thus Emacs API can do arithmetic
operaions and comparisons on characters whereas in CL they have to use
character specific operators. I have implemented the reader to
understand ? and escapes and to spit out integer codes instead of cl
characters so I don't need to dispatch on each mathematical operation.

There is still more, "markers are numbers" in Emacs, so mathematical
operations are applicable to those as well. I haven't decided if I will
do same as they do in C sources or I'll try to implement static dispatch
for generic functions and have mathematical operations as generic
functions, I have to test how it works, but there are other things like
those that have to be taken care of. And there is more, that is
just the tip of the iceberg :).

I am also talking only about lisp implemented in C core. There weould be
need to do some work on the elisp side too; eieo, cl-lib, defun/defmacro,
stuff like that, but that is probably by far lesser and easier to fix.

> I'm not minimizing the amount of work an SBCL core would take,
> there's a lot of devil in the details of 1476 + 14 primitive
> functions.  Nor am I interested in joining a political argument
> about whether it should be done; I have too much else on my
> plate for that.

Yes, something like that; it is volumous and lots of work, for one
person probably impossible, but it is not impossible to do.