Re: Emacs Lisp's future

["Stephen J. Turnbull" <stephen@xemacs.org>]

Mark H Weaver writes:

 > I would like to change Guile's internal string representation to use a
 > generalization of UTF-8, as used by Emacs.  There are two sticking
 > points that require more thought, however:
 > 
 > * I'm concerned that there are security implications to supporting
 >   the "raw byte" code points.

There are greater security implications to using the full repertoire
of Unicode (so-called confusables for a start).

 >   However, I think this will not be a problem, because the
 >   string<->bytevector conversion procedures could support two modes of
 >   operation: one mode that supports these raw bytes, for use by emacs
 >   and maybe some other purposes (e.g. dealing with POSIX file names),
 >   and another mode that refuses to accept or produce invalid UTF-8,

I'm not sure what you mean by "mode", but this behavior should be
decided for each stream.  IMO Python does this correctly (although its
internal representation is fixed-width, not UTF-8).  Specifically, raw
bytes in I/O streams are treated as errors, and when encountered an
error handler (which is specified stream-by-stream) decides how to
treat them.

Besides the in band "special character" representation, 'strict'
(raise error) must be provided.

Other handlers provided by Python include 'ignore' (drop from the
stream), 'replace' (with a constant replacement character),
'backslashreplace' (replace with a hexadecimal representation such as
"\x1F4A9") and 'xmlreplace' (replace with a character entity) handlers
are provided.

 > Supporting property lists in Scheme raises difficult questions
 > such as:

Difficult, really?

 >  * What should the Scheme procedures 'string=?' and 'equal?' do when
 >    comparing two strings with the equal character sequences but
 >    unequal property lists?

Ignore the property list.

 >  * Should Scheme procedures such as 'substring', 'string-append',
 >    'string-upcase', etc, propagate the associated property list
 >    data?

Ignore the property list.

 >  * Are there security implications to carrying around and possibly
 >    propagating (via Scheme's "substring") extra information that is
 >    effectively invisible to all procedures that have ever been
 >    available in Scheme?

Ignore the property list.

 >  * What should Scheme's 'write' do when applied to a string that
 >    includes a property list?  ('write' is analogous to 'prin1').

Ignore the property list.

There, that wasn't hard, was it?

Scheme itself really needs only to provide a setter and a getter for
the property list, and leave everything else up to Emacs (at first,
anyway).  If you're really worried about the security implications,
provide a interpreter instance switch at invocation time (or even
compile time) so that feature is only available in invocations that
explicitly request it.  This would default to on for Emacs processes,
off for "vanilla" Guile, and other applications that embed special
configurations of Guile can make their own choice.

 > Security concerns are more important for Guile than for Emacs, because
 > Guile is already being used to implement network programs such as web
 > servers, and generally aims to support a much wider range of
 > applications.

Obviously, you don't know Emacs very well.  Emacs is the Swiss Army
knife of the software world.  It is an operating system, a system
library, an application development platform and an application.  (We
also walk dogs.)

 > What do you think?  Do I worry too much [about nothing]? :)

Listen to your Uncle David on this one.  You should treat every
instance of nil as a biohazard.