Re: Python vs Lisp (followups to -tangents)

[Christopher Allan Webber <cwebber@dustycloud.org>]

Random832 writes:

>> The point Richard is making is that Python lacks macros, i.e., you
>> cannot easily write code which writes code.
>> You have to either operate at the level of strings (which is hard to get
>> right) or at the level of AST (which is even harder).
>
> I don't see how operating at the level of AST is harder than
> operating at the level of lists (backquote operates above the
> level of lists; it automatically searches the code you give it
> for placeholders to substitute values in. It probably wouldn't
> be hard to write an equivalent in Python.)

So, I think this thread is getting fairly off topic for the list, but I
can't resist chiming in here.

A project I've contributed to occasionally is Hy, which is sort of a
lisp that uses s-expressions (yes, you do need them if you want
homoiconic properties) which compiles down to Python's AST.  It's pretty
cool, because you can now write macros for Python and do other sorts of
lispy things that are impossible otherwise.  See:

  http://hy.readthedocs.org/en/latest/tutorial.html

Doing this required building a lisp like system on top of Python though.
To add new operations to the language meant making the language
homoiconic in some way, and yes, that meant using Python lists to make
s-expressions.  And yeah, you could compose with Python then.

I think in most ways, Hy was a real lisp, just one that used cpython as
a virtual machine.

However, there are downsides to using a lisp on top of something that
was never intended to be a lisp, which I learned the hard way.  I tried
writing a serious system in Hy, and eventually had to move to Guile
instead.  Why?  Because once you have macros, it becomes very difficult
to tell what line number you're on in (the level of transformations we
did made it so we couldn't keep around that information by the time we
got to the AST, and it was hard to reason about what was going on
there).  This meant debugging was nearly impossible, because when
debugging in Python, line numbers is your primary context for
determining where you are.  I'm not just talking about pdb.set_trace()
(though that too); even tracebacks became impossible to deal with.

Thus Hy is a pretty cool toy to play with, and kind of nice for writing
a quick DSL on top of Python.  And it *has* proven that you could treat
the Python AST as something to compile down to and have macros.  But
even there, it looked like lisp, and even once you had that, underlying
decisions about the implementation you're building on top of can make
things difficult.

Still fun to work on though!  Also a great community and a great route
for Python type people who want to dip their toe into lisp-land.  And a
good chance to learn the above lessons the hard way. :)

 - Chris