Optimising Elisp code

Optimising Elisp code

[Davin Pearson]

Suppose that you have a function:

(defun foo ()
   (bar))

And bar is a function that has no side effects and returns 123 then calling
the function code-optimize (the name of my optimisation routine)

M-x code-optimize on the function foo will result in the following defun 

(defun foo ()
    123)

Re: Optimising Elisp code

[Davin Pearson]

On Friday, October 5, 2018 at 3:15:32 PM UTC+13, Davin Pearson wrote:
> Suppose that you have a function:
> 
> (defun foo ()
>    (bar))
> 
> And bar is a function that has no side effects and returns 123 then calling
> the function code-optimize (the name of my optimisation routine)
> 
> M-x code-optimize on the function foo will result in the following defun 
> 
> (defun foo ()
>     123)

Please could you admise me about whether or not such as function is part of Gnu Emacs or otherwise available as a third party tool.

If not then my code will be reinventing the wheel :-(

Re: Optimising Elisp code

[Emanuel Berg]

Davin Pearson wrote:

> Please could you admise me about whether or
> not such as function is part of Gnu Emacs or
> otherwise available as a third party tool.

Hm, perhaps it will be more educational as for
the purpose and advantages of your tool if you
posted a more live or in-world example of how
you have made use of it?

> If not then my code will be reinventing the
> wheel :-(

Reinventing the wheel is not easy. If you have
done that, cred to you, and it will be
interesting to have a look at it!

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code

[tomas]

[-- Attachment #1: Type: text/plain, Size: 676 bytes --]

On Thu, Oct 04, 2018 at 07:15:30PM -0700, Davin Pearson wrote:
> Suppose that you have a function:
> 
> (defun foo ()
>    (bar))
> 
> And bar is a function that has no side effects and returns 123 then calling
> the function code-optimize (the name of my optimisation routine)
> 
> M-x code-optimize on the function foo will result in the following defun 
> 
> (defun foo ()
>     123)

Have you already looked at the code produced by the elisp compiler? As
in doing "compile-function" and "disassemble-function"?

Have you had a look at eval-and-compile and friends?

Perhaps elisp is already doing (parts of) what you have in mind?

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Optimising Elisp code

[Noam Postavsky]

On Thu, 4 Oct 2018 at 22:20, Davin Pearson <davin.pearson@gmail.com> wrote:
>
> On Friday, October 5, 2018 at 3:15:32 PM UTC+13, Davin Pearson wrote:
> > Suppose that you have a function:
> >
> > (defun foo ()
> >    (bar))
> >
> > And bar is a function that has no side effects and returns 123 then calling
> > the function code-optimize (the name of my optimisation routine)
> >
> > M-x code-optimize on the function foo will result in the following defun
> >
> > (defun foo ()
> >     123)
>
> Please could you admise me about whether or not such as function is part of Gnu Emacs or otherwise available as a third party tool.

Not exactly, but if you use defsubst to define bar (defsubst bar ()
123), then the result of byte-compiling foo will be equivalent to your
optimized version.

Re: Optimising Elisp code

[Barry Margolin]

In article <638fb7dc-6fc5-4645-8793-97a00038a3a8@googlegroups.com>,
 Davin Pearson <davin.pearson@gmail.com> wrote:

> Suppose that you have a function:
> 
> (defun foo ()
>    (bar))
> 
> And bar is a function that has no side effects and returns 123 then calling
> the function code-optimize (the name of my optimisation routine)
> 
> M-x code-optimize on the function foo will result in the following defun 
> 
> (defun foo ()
>     123)

What you're describing is called inline expansion. AFAIK, the Elisp 
compiler doesn't do this automatically.

You can define bar as a macro -- those HAVE to be expanded at compile 
time, since that's how they work.

You can also define an inline function using defsubst. From the Elisp 
manual:

An inline function works just like an ordinary function except for one 
thing: when you compile a call to the function, the function's 
definition is open-coded into the caller.

Making a function inline makes explicit calls run faster. But it also 
has disadvantages. For one thing, it reduces flexibility; if you change 
the definition of the function, calls already inlined still use the old 
definition until you recompile them.

There are more caveats given in the documentation.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Optimising Elisp code

[Emanuel Berg]

Barry Margolin wrote:

> You can also define an inline function using
> defsubst. From the Elisp manual:
>
> An inline function works just like an
> ordinary function except for one thing: when
> you compile a call to the function, the
> function's definition is open-coded into
> the caller.

This rings a bell from the C++ days when/where
you could define a member function as "inline".
That's all I remember, but maybe it was
something similar to this? I think it amounted
to, instead of making a call, that function was
coded into the object itself, and for this
reason it was suited for really short,
simple functions.

Yes, what does it mean that "the function's
definition is open-coded into the caller"?

If the caller is a function, does this mean
instead of calling the other function, the
other function's code is put into the first
function so there isn't a second call but
everything is executed in the first function?

> Making a function inline makes explicit calls
> run faster.

Well, yeah, obviously since there is no second
invocation with just the original function put
onto the stack!

What are "explicit calls"? Regular calls,
right? But then what are implicit calls?
Anonymous functions? Or is an inline function
being executed an implicit call to
that function?

> But it also has disadvantages. For one thing,
> it reduces flexibility; if you change the
> definition of the function, calls already
> inlined still use the old definition until
> you recompile them.

Another good point, but that's obvious as well.

(Hey, for a person who doesn't understand this,
I sure sound confident enough :))

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code

[Emanuel Berg]

> This rings a bell from the C++ days
> when/where you could define a member function
> as "inline". That's all I remember, but maybe
> it was something similar to this? I think it
> amounted to, instead of making a call, that
> function was coded into the object itself,
> and for this reason it was suited for really
> short, simple functions. [...]
>
> If the caller is a function, does this mean
> instead of calling the other function, the
> other function's code is put into the first
> function so there isn't a second call but
> everything is executed in the first function?

OK, in C++ it works like this:

If a member function is defined within the
class definition (not recommended), or if it is
defined outside of it (recommended) with the
word "inline" preceding the return type, then
the function becomes an inline function.

What this means is that when the code is
compiled into machine code, instead of having
one place for the function, and jumping back
and forth every time that function is called,
the machine code for the inline function is
placed, duplicated wherever it is used.

So you get longer machine code, but faster,
since there is just constant execution for the
inline functions, without jumping back
and forth.

When should it be used? If the function is very
short, say 1-3 lines, the jumping back and
forth requires more machine instructions than
the function itself. Then the organizational
gain of having it neatly at one place is
diminished by the speed advantage of not having
to jump back and forth to that place all
the time.

"Make the common case fast, and the complicated
case work" -- M. Yoda

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code

[Óscar Fuentes]

Emanuel Berg <moasen@zoho.com> writes:

> OK, in C++ it works like this:
>
> If a member function is defined within the
> class definition (not recommended), or if it is
> defined outside of it (recommended) with the
> word "inline" preceding the return type, then
> the function becomes an inline function.
>
> What this means is that when the code is
> compiled into machine code, instead of having
> one place for the function, and jumping back
> and forth every time that function is called,
> the machine code for the inline function is
> placed, duplicated wherever it is used.
>
> So you get longer machine code, but faster,
> since there is just constant execution for the
> inline functions, without jumping back
> and forth.
>
> When should it be used? If the function is very
> short, say 1-3 lines, the jumping back and
> forth requires more machine instructions than
> the function itself. Then the organizational
> gain of having it neatly at one place is
> diminished by the speed advantage of not having
> to jump back and forth to that place all
> the time.

Just in case any bystander finds this post: don't rely on anything that is
mentioned above, nor for C++ `inline' keyword nor for code inlining in
general.

Re: Optimising Elisp code

[Emanuel Berg]

Óscar Fuentes wrote:

> Emanuel Berg <moasen@zoho.com> writes:
>
>> OK, in C++ it works like this: If a member
>> function is defined within the class
>> definition (not recommended), or if it is
>> defined outside of it (recommended) with the
>> word "inline" preceding the return type,
>> then the function becomes an inline
>> function. What this means is that when the
>> code is compiled into machine code, instead
>> of having one place for the function, and
>> jumping back and forth every time that
>> function is called, the machine code for the
>> inline function is placed, duplicated
>> wherever it is used. So you get longer
>> machine code, but faster, since there is
>> just constant execution for the inline
>> functions, without jumping back and forth.
>> When should it be used? If the function is
>> very short, say 1-3 lines, the jumping back
>> and forth requires more machine instructions
>> than the function itself. Then the
>> organizational gain of having it neatly at
>> one place is diminished by the speed
>> advantage of not having to jump back and
>> forth to that place all the time.
>
> Just in case any bystander finds this post:
> don't rely on anything that is mentioned
> above, nor for C++ `inline' keyword nor for
> code inlining in general.

I read it in this book:

    @book{cpp-direkt,
      author     = {Jan Skansholm},
      ISBN       = {91-44-47931-X},
      publisher  = {Studentlitteratur},
      title      = {C++ direkt},
      year       = {1996}
    }

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code

[Óscar Fuentes]

Emanuel Berg <moasen@zoho.com> writes:

>> Just in case any bystander finds this post:
>> don't rely on anything that is mentioned
>> above, nor for C++ `inline' keyword nor for
>> code inlining in general.
>
> I read it in this book:
>
>     @book{cpp-direkt,
>       author     = {Jan Skansholm},
>       ISBN       = {91-44-47931-X},
>       publisher  = {Studentlitteratur},
>       title      = {C++ direkt},
>       year       = {1996}
>     }

The key data point there is "1996". But even then things were not as
simple as you described.

Since long time ago C++ "inline" is really about the ODR ("One
Definition Rule"), not about how the compiler generates code.

Re: Optimising Elisp code

[James K. Lowden]

On Fri, 05 Oct 2018 17:04:17 +0200
Emanuel Berg <moasen@zoho.com> wrote:

> "Make the common case fast, and the complicated
> case work" -- M. Yoda

	Complex solutions are slow for small N, and 
	N is almost always small.  
	-- Rob Pike

--jkl

Re: Optimising Elisp code

[Emanuel Berg]

Óscar Fuentes wrote:

>>> Just in case any bystander finds this post:
>>> don't rely on anything that is mentioned
>>> above, nor for C++ `inline' keyword nor for
>>> code inlining in general.
>>
>> I read it in this book:
>>
>>     @book{cpp-direkt,
>>       author     = {Jan Skansholm},
>>       ISBN       = {91-44-47931-X},
>>       publisher  = {Studentlitteratur},
>>       title      = {C++ direkt},
>>       year       = {1996}
>>     }
>
> The key data point there is "1996". But even
> then things were not as simple as
> you described.

It was basically a translation of what it says.

> Since long time ago C++ "inline" is really
> about the ODR ("One Definition Rule"), not
> about how the compiler generates code.

I have now look it up in another book, namely

    @book{c-programming-language,
      author     = {Bjarne Stroustrup},
      ISBN       = {0-201-53992-6},
      publisher  = {Addison Wesley},
      title      = {The C++ Programming Language},
      year       = 1992
    }

here it says much less, with no attempt to
explain what actually "inline" means, still, it
seems to contradict what you say, because it
says the keyword inline is a "hint to the
compiler" to generate the code inline rather
than have the function called the usual way
(page 124).

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code

[Emanuel Berg]

James K. Lowden wrote:

> 	Complex solutions are slow for small N, and 
> 	N is almost always small.  
> 	-- Rob Pike

    Advance too fast, you catch up with death.
    But advance too slow, death catches up with YOU!
    -- The Metabaron

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code

[Emanuel Berg]

> I have now look it up in another book, namely
>
>     @book{c-programming-language,
>       author     = {Bjarne Stroustrup},
>       ISBN       = {0-201-53992-6},
>       publisher  = {Addison Wesley},
>       title      = {The C++ Programming Language},
>       year       = 1992
>     }
>
> here it says much less, with no attempt to
> explain what actually "inline" means, still, it
> seems to contradict what you say, because it
> says the keyword inline is a "hint to the
> compiler" to generate the code inline rather
> than have the function called the usual way
> (page 124).

I was about to say "I can do this all night",
but then I'd be a lier because I'm running out
of C++ books. This is the last one

    @book{small-c-how-to-program,
      author     = {H M Deitel and P J Deitel},
      ISBN       = {0-13-185758-4},
      publisher  = {Pearson},
      title      = {Small C++ How to Program},
      year       = 2005
    }

and it says virtually the same thing: "inline"
advices the compiler to "generate a copy of the
function's code in place" rather than the usual
call. (It also says the compiler can choose not
to do this; page 246.)

But I'm not saying you are wrong. I was once
~fluent in *writing* C++, but I didn't claim
then, and certainly do not claim now, to
understand what goes on under the hood.

To young, promising programmers, two pieces of
advice:

1) don't be a professional programmer; and

2) stay away from these books:

%%%% C++

@book{small-c-how-to-program,
  author     = {H M Deitel and P J Deitel},
  ISBN       = {0-13-185758-4},
  publisher  = {Pearson},
  title      = {Small C++ How to Program},
  year       = 2005
}

@book{c-programming-language,
  author     = {Bjarne Stroustrup},
  ISBN       = {0-201-53992-6},
  publisher  = {Addison Wesley},
  title      = {The C++ Programming Language},
  year       = 1992
}

@book{cpp-direkt,
    author     = {Jan Skansholm},
    ISBN       = {91-44-47931-X},
    publisher  = {Studentlitteratur},
    title      = {C++ direkt},
    year       = 1996
}

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code

[Emanuel Berg]

> C++ blah blah blah

For anyone reading this, don't be confused by
all the C++ talk, please carry on the Elisp
discussion if that is your desire!

As well as the C++ discussion for that matter.
Programming is NEVER off-topic on an
Emacs newsgroup!

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code

[Emanuel Berg]

> I was about to say "I can do this all night",
> but then I'd be a lier because I'm running
> out of C++ books. This is the last one

I was wrong, I have yet another one, but not on
C++, but on C, namely

    @book{vägen-till-c,
      author     = {Ulf Bilting and Jan Skansholm},
      ISBN       = {978-91-44-01468-5},
      publisher  = {Studentlitteratur},
      title      = {Vägen till C},
      year       = 2000
    }

it says, page 279, that into the "new" C99
standard, from C++, they have brought the
"inline" keyword. Then they just say the same
thing, instead of a call, the insertion of
code.

They also say the previous method of getting
somewhat the same stuff has been the use of
macros. By macros, they mean the #preprocessor
replacement of text! So they think the new
"inline" is really good! And one can
appreciate that :)

Now I really need to drink a couple of beers
and ride my bike (bicycle). I haven't written
this much on gnu.emacs.help in ages. Maybe I'm
not getting older after all! That, I'd
appreciate as well...

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code

[Stefan Monnier]

>>> function. What this means is that when the
>>> code is compiled into machine code, instead
[...]
>>> function is called, the machine code for the
>>> inline function is placed, duplicated
                    ^^
Maybe, maybe not: it's up to the compiler, because you won't be able to
tell the difference without looking under the hood.

And for the same reason the compiler can also decide to inline when you
*don't* say `inline`, as long as it doesn't affect the behavior of
the program.

Similarly, the compiler may decide to "outline" a chunk of code (take
a part of your function, move it to a new separate function, and
replace it with a call to that new function), or do all kinds of other
fun stuff, such as compile your C++ to Elisp code and combine it with an
Elisp interpreter.


        Stefan

Re: Optimising Elisp code

[Emanuel Berg]

Stefan Monnier wrote:

> Similarly, the compiler may decide to
> "outline" a chunk of code (take a part of
> your function, move it to a new separate
> function, and replace it with a call to that
> new function), or do all kinds of other fun
> stuff, such as compile your C++ to Elisp code
> and combine it with an Elisp interpreter.

Can you explain what inline means in terms
of Elisp?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Knowing where a function has been used (e.g. for optimizing) [Was: Re: Optimising Elisp code]

[Garreau, Alexandre]

On 2018-10-05 at 10:28, Barry Margolin wrote:
> You can define bar as a macro -- those HAVE to be expanded at compile 
> time, since that's how they work.

But what’s the relationship since here, it’s not?

> You can also define an inline function using defsubst. From the Elisp 
> manual:

> Making a function inline makes explicit calls run faster. But it also 
> has disadvantages. For one thing, it reduces flexibility; if you change 
> the definition of the function, calls already inlined still use the old 
> definition until you recompile them.

So aren’t there any dynamical system that would keep a track of
dependencies so that to allow inlining while recompiling the appropriate
functions when changing definitions of inlined stuff?

> An inline function works just like an ordinary function except for one 
> thing: when you compile a call to the function, the function's 
> definition is open-coded into the caller.

Doesn’t lexical scoping also lead to that behavior?

> What you're describing is called inline expansion. AFAIK, the Elisp 
> compiler doesn't do this automatically.

Does it automatically for lexically scoped function definitions?

Is it only because then the behavior would differ and because there’s
currently no way to, say, tell to defun, each time it redefines
something that has been inlined, to also recompile every functions which
included it?

There’s already, for each currently defined function, a handy way to get
back to where it was defined.  But I always noticed this relation is
one-way: there’s no way, for each currently defined function, to get
back to where it was called.  I always missed it because it would gives
a quick way to look at exemples of how a function is used (examples are
the other great way of learning: inherenthly more imperfect than a
definition but quicker and easier), but I feel like, there, if it
existed, it would also allow to generalize some optimizations that
already are handy for compiled, non-dynamic code (because afaik
recursively do this and other expansions, reducing, simplification,
could lead to making abstraction free of performance overhead, right?).

Re: Optimising Elisp code

[Stefan Monnier]

> What you're describing is called inline expansion.
> AFAIK, the Elisp compiler doesn't do this automatically.

Indeed, it doesn't do it automatically because it doesn't know how to
(automatically) undo it, and it has visible effects w.r.t advice,
debug-on-entry, etc...

> You can define bar as a macro -- those HAVE to be expanded at compile 
> time, since that's how they work.
>
> You can also define an inline function using defsubst. From the Elisp 
> manual:

And also with cl-defsubst, with (declare (compiler-macro ...))
and more recently with define-inline.


        Stefan

Re: Optimising Elisp code

[Garreau, Alexandre]

On 2018-10-06 at 10:40, Stefan Monnier wrote:
>> What you're describing is called inline expansion.
>> AFAIK, the Elisp compiler doesn't do this automatically.
>
> Indeed, it doesn't do it automatically because it doesn't know how to
> (automatically) undo it, and it has visible effects w.r.t advice,
> debug-on-entry, etc...

Why “undo”? do you mean take the byte-compiled form and programmatically
do on it something like find and extract the part that comes from
byte-compiling the inlined function?

Or would “reevaluate every function that where defined with the
(now-redefined) inlined function inside” work too?

Re: Optimising Elisp code

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1234 bytes --]

On Sat, Oct 06, 2018 at 06:55:54PM +0200, Garreau, Alexandre wrote:
> On 2018-10-06 at 10:40, Stefan Monnier wrote:
> >> What you're describing is called inline expansion.
> >> AFAIK, the Elisp compiler doesn't do this automatically.
> >
> > Indeed, it doesn't do it automatically because it doesn't know how to
> > (automatically) undo it, and it has visible effects w.r.t advice,
> > debug-on-entry, etc...
> 
> Why “undo”? [...]

because...

 - when debugging, you want to know that function "foo" is being
   called
 - you want, perhaps, to advice/instrument the function after its
   entrails have been spread across some other byte code
 - because you expect it to be possible to redefine the function
   at run time (the run time system would have to know how to
   excise the inlined and perhaps optimised bits and pieces and
   replace them by something else)

> Or would “reevaluate every function that where defined with the
> (now-redefined) inlined function inside” work too?

Hmm. Not reevaluate, but recompile, and not "inside", but rather
"outside" -- like finding all places where this function was
inlined and do some magic there. Perhaps doable, but not trivial.

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Garreau, Alexandre]

Le 06/10/2018 à 21h24, tomas@tuxteam.de a écrit :
> On Sat, Oct 06, 2018 at 06:55:54PM +0200, Garreau, Alexandre wrote:
>> Or would “reevaluate every function that where defined with the
>> (now-redefined) inlined function inside” work too?
>
> Hmm. Not reevaluate, but recompile, and not "inside", but rather
> "outside" -- like finding all places where this function was
> inlined 

I don’t get how “outside” is the correct form here, as this is precisely
what I was meaning: I mentioned it in my other message titled “Knowing
where a function has been used (e.g. for optimizing)” (message-id: [0],
url: [1]), as I found a such functionality, symetrical to
find-definition, would be quite generally useful for lot of stuff.

> and do some magic there.

What other kind of magic than (maybe recursively)
reevaluating/recompiling could be needed there?

> Perhaps doable, but not trivial.

I have really no precise idea of where and how is stored the place where
each function was defined wrt the related function’s symbol (I tried to
search somewhere ago and found it must be linked with load-history, but,
because of the absence of a such feature ^^, I don’t know where and how
it is updated, without further searching (by grepping?)), but would it
be so hard storing a symbol at the right place (near the place where it
is said where that symbol is defined (and/or other informations about
this symbol)) for each function (why not also variable or even macro?
symbol generically) refered in the function body?

>> Why “undo”? [...]
>
> because...
>
>  - when debugging, you want to know that function "foo" is being
>    called
>  - you want, perhaps, to advice/instrument the function after its
>    entrails have been spread across some other byte code
>  - because you expect it to be possible to redefine the function
>    at run time (the run time system would have to know how to
>    excise the inlined and perhaps optimised bits and pieces and
>    replace them by something else)

So, except maybe wrt debugging (which there should imply reevaluation,
like with edebug-defun), reevaluating (or rather recompiling indeed,
since afaiu then, simple evaluation will never optimize anything, unlike
byte-compiling (right?)).

[0] <9oqtevzzzzzz.yga.xxuns.g6.gal_-_@portable.galex-713.eu>
[1] https://lists.gnu.org/archive/html/help-gnu-emacs/2018-10/msg00068.html

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 2695 bytes --]

On Sat, Oct 06, 2018 at 09:55:09PM +0200, Garreau, Alexandre wrote:
> Le 06/10/2018 à 21h24, tomas@tuxteam.de a écrit :
> > On Sat, Oct 06, 2018 at 06:55:54PM +0200, Garreau, Alexandre wrote:
> >> Or would “reevaluate every function that where defined with the
> >> (now-redefined) inlined function inside” work too?
> >
> > Hmm. Not reevaluate, but recompile, and not "inside", but rather
> > "outside" -- like finding all places where this function was
> > inlined 
> 
> I don’t get how “outside” is the correct form here, as this is precisely
> what I was meaning: I mentioned it in my other message titled “Knowing
> where a function has been used (e.g. for optimizing)” (message-id: [0],
> url: [1]), as I found a such functionality, symetrical to
> find-definition, would be quite generally useful for lot of stuff.

Then I didn't get you right: I gather we both mean the "context" where
the function was "inlined into". If you're lucky, no optimization has
happened *after" you've inlined. Otherwise, you'll have to recompile
the whole context (i.e. parts of the caller).

> > and do some magic there.
> 
> What other kind of magic than (maybe recursively)
> reevaluating/recompiling could be needed there?

In the case of a redefinition of the inlined function you'll have to
find all call sites and do something.
> 
> > Perhaps doable, but not trivial.
> 
> I have really no precise idea of where and how is stored the place where
> each function was defined

Which functions you mean by "each function"? You mean the sites at which
the inlined function is "called" (i.e. inlined), I guess.

Well, I haven't a precise idea either, but for a pretty lucid description
(in the context of Guile, but with pointers to other implementations),
cf

  https://wingolog.org/archives/2018/02/07/design-notes-on-inline-caches-in-guile

> >> Why “undo”? [...]

[...]

> So, except maybe wrt debugging (which there should imply reevaluation,

you mean "recompile"

> like with edebug-defun), reevaluating (or rather recompiling indeed,
> since afaiu then, simple evaluation will never optimize anything, unlike
> byte-compiling (right?)).

Not only debugging, but any kind of redefinition. Emacs lisp is a
dynamic language, and the users expect that when you redefine a
function, the new definition is active; unless the magic is under
user control (you've said "defmacro" or "def-inline", then you know
you've to recompile the callers), the run system has to manage the
mess, i.e. it has to remember all the sites where things have to
be inlined, like in the "inline cache" from the reference above.

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Garreau, Alexandre]

Le 06/10/2018 à 22h27, tomas@tuxteam.de a écrit :
> On Sat, Oct 06, 2018 at 09:55:09PM +0200, Garreau, Alexandre wrote:
>> Le 06/10/2018 à 21h24, tomas@tuxteam.de a écrit :
>> > On Sat, Oct 06, 2018 at 06:55:54PM +0200, Garreau, Alexandre wrote:
>> >> Or would “reevaluate every function that where defined with the
>> >> (now-redefined) inlined function inside” work too?
>> >
>> > Hmm. Not reevaluate, but recompile, and not "inside", but rather
>> > "outside" -- like finding all places where this function was
>> > inlined 
>> 
>> I don’t get how “outside” is the correct form here, as this is precisely
>> what I was meaning: I mentioned it in my other message titled “Knowing
>> where a function has been used (e.g. for optimizing)” (message-id: [0],
>> url: [1]), as I found a such functionality, symetrical to
>> find-definition, would be quite generally useful for lot of stuff.
>
> Then I didn't get you right: I gather we both mean the "context" where
> the function was "inlined into".

Yes.

> If you're lucky, no optimization has happened *after" you've inlined.

Lucky? Why? if it didn’t happen, will you really be able to extract the
inlined part to patch it? won’t recompiling it be simplier?

I like the idea of inline optimization precisely because if done
recursively even on a reasonable level it could sometimes lead to great
expansion/reduction and simplification, not only of one function call.

> Otherwise, you'll have to recompile the whole context (i.e. parts of
> the caller).

How do you recompile only a part? you can do this (I’m pretty ignorant
of elisp’s byte-compiler capabilities)?

>> > and do some magic there.
>> 
>> What other kind of magic than (maybe recursively)
>> reevaluating/recompiling could be needed there?
>
> In the case of a redefinition of the inlined function you'll have to
> find all call sites and do something.

that is, recursively recompiling, as I said, right? “do something” is no
more precise than “do some magic”, and “there” (in both messages)
already meant “at the call sites”.

>> > Perhaps doable, but not trivial.
>> 
>> I have really no precise idea of where and how is stored the place where
>> each function was defined
>
> Which functions you mean by "each function"? You mean the sites at which
> the inlined function is "called" (i.e. inlined), I guess.

Not really, including this, but I was speaking about the find-definition
feature of emacs specifically, since afterwards I talk about a symmetric
feature, that is, store the list of symbols refered by each function, even (or
rather, especially) if it is compiled, in a place linked to each
function definition: so if the functions `f', `g' and `h' refer to `a',
having in something linked to the `a' symbol (an alist, a plist,
anything) a list of these.  So the same way you can find-definition of
`a' and see how and where it was defined (like with C-h f, C-h v… and
xref generally), you could see where it has been used: that’s at the
same time a good way of see the use/meaning of `a' by examples rather
than by definition (if the definition didn’t suffice), and that would
allow to recursively draw some dependency graph showing what needs to be
recompiled if `a' is changed.

Maybe even someone could try some experimental optimizations features
such as, let this `a' be a variable, not a thunk, recompile everything
referring to `a' when you change it (thus leading to the same behavior
of a thunk, without side-effects, and with statical optimization)

> Well, I haven't a precise idea either, but for a pretty lucid description
> (in the context of Guile, but with pointers to other implementations),
> cf
>
>   https://wingolog.org/archives/2018/02/07/design-notes-on-inline-caches-in-guile

I was speaking of something more static, not at eval-time, but at
compile-time.  The idea was to keep a big but exhaustive list of usages
(that would be filled at each definition, I guess the same way as
load-history already is), growing with time (possibly directly linked to
the symbol).

>> >> Why “undo”? [...]
>
> [...]
>
>> So, except maybe wrt debugging (which there should imply reevaluation,
>
> you mean "recompile"

Afaik edebug-defun works by reevaluation everywhere, even for
not-compiled functions.  So I meant reevaluation.  Except, you, indeed,
were speaking about the case I was saying I wasn’t speaking about, that
is debugging an already existing (possibly compiled) function, without
reusing its definition (that is reevaluating, or recompiling if
necessary (because it was already compiled)).

>> like with edebug-defun), reevaluating (or rather recompiling indeed,
>> since afaiu then, simple evaluation will never optimize anything, unlike
>> byte-compiling (right?)).
>
> Not only debugging, but any kind of redefinition. Emacs lisp is a
> dynamic language, and the users expect that when you redefine a
> function, the new definition is active; unless the magic is under
> user control (you've said "defmacro" or "def-inline", then you know
> you've to recompile the callers),

That’s why I’m talking about recompiling.

> the run system has to manage the mess, i.e. it has to remember all the
> sites where things have to be inlined, like in the "inline cache" from
> the reference above.

In wingo’s article he speaks about a cache put *inside* a function,
that’s updated *while* evaluation, depending on it.  I’m speaking of the
statical, pre-compilation case.  With this same information, not
guessed, but exhaustive, and not stored *inside* the function, but in
some place related, be it a global dictionary (hashtable, alist or
plist), or in something related to the symbol (its plist? some modified
symbol?): so you can programatically use this list of locations
typically in xref/help buffers, in some experimental paradigm of
programming, etc.  IC is only an optimization, is VM/evaluation-side, is
non-exhaustive/guessed by nature, and is, afaiu, hidden from the user
point-of-view.

Re: Optimising Elisp code

[Barry Margolin]

In article <86r2h44fqg.fsf@zoho.com>, Emanuel Berg <moasen@zoho.com> 
wrote:

> Barry Margolin wrote:
> 
> > You can also define an inline function using
> > defsubst. From the Elisp manual:
> >
> > An inline function works just like an
> > ordinary function except for one thing: when
> > you compile a call to the function, the
> > function's definition is open-coded into
> > the caller.
> 
> This rings a bell from the C++ days when/where
> you could define a member function as "inline".
> That's all I remember, but maybe it was
> something similar to this? I think it amounted
> to, instead of making a call, that function was
> coded into the object itself, and for this
> reason it was suited for really short,
> simple functions.
> 
> Yes, what does it mean that "the function's
> definition is open-coded into the caller"?
> 
> If the caller is a function, does this mean
> instead of calling the other function, the
> other function's code is put into the first
> function so there isn't a second call but
> everything is executed in the first function?

Yes, that's what "open-coded" means. 

> 
> > Making a function inline makes explicit calls
> > run faster.
> 
> Well, yeah, obviously since there is no second
> invocation with just the original function put
> onto the stack!
> 
> What are "explicit calls"? Regular calls,
> right? But then what are implicit calls?
> Anonymous functions? Or is an inline function
> being executed an implicit call to
> that function?

Calling the function with funcall or apply, e.g.

(funcall some-fun x y z)

The variable some-fun might contain an ordinary function or an inline 
function, but the compiler doesn't know what it will be so it can't 
expand it.

> 
> > But it also has disadvantages. For one thing,
> > it reduces flexibility; if you change the
> > definition of the function, calls already
> > inlined still use the old definition until
> > you recompile them.
> 
> Another good point, but that's obvious as well.
> 
> (Hey, for a person who doesn't understand this,
> I sure sound confident enough :))

Yep, you're picking it up OK.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

On Sat, Oct 06, 2018 at 11:42:12PM +0200, Garreau, Alexandre wrote:

[...]

> Not really, including this, but I was speaking about the find-definition
> feature of emacs specifically, since afterwards I talk about a symmetric
> feature, that is, store the list of symbols refered by each function, even (or
> rather, especially) if it is compiled, in a place linked to each
> function definition: so if the functions `f', `g' and `h' refer to `a',
> having in something linked to the `a' symbol [...]

OK, I roughly got your idea. I'm a bit out of time to discuss those
things at depth (I'd have to do much more reading than I could ATM
to be able to do more than just handwaving), but one last observation
from me: I think what is going to kill you is the inherently dynamic
nature of LISP (think eval: once you reach such a point, potentially
*every* function, even those not yet known at compile time, can be
referenced). This is a trait LISP shares with other languages like
Javascript. Your strategy might make more sense in a context like
Haskell, where the compiler "knows everything" (yeah, sloppy, but
I think you get what I mean).

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Knowing where a function has been used (e.g. for optimizing) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

Garreau, Alexandre wrote:

> (examples are the other
> great way of learning: inherenthly more
> imperfect than a definition but quicker and
> easier)

There should always first be the definition,
then one simple example, then a more
complicated example, then the example that
confirms the rule.

In programming, because computers are
deterministic, example number three shouldn't
exist - but sometimes, it does.

For example, when... err

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

Garreau, Alexandre wrote:

> I have really no precise idea of where and
> how is stored the place where each function
> was defined wrt the related function’s symbol
> (I tried to search somewhere ago and found it
> must be linked with load-history, but,
> because of the absence of a such feature ^^,
> I don’t know where and how it is updated,
> without further searching (by grepping?)),
> but would it be so hard storing a symbol at
> the right place (near the place where it is
> said where that symbol is defined (and/or
> other informations about this symbol)) for
> each function (why not also variable or even
> macro? symbol generically) refered in the
> function body?

Style hint: Don't try to say so many things in
such a hurry! Say one thing. OK, now
that's done.

New paragraph about something else.
Say another thing.

Simple.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

tomas wrote:

> Not only debugging, but any kind of
> redefinition. Emacs lisp is a dynamic
> language, and the users expect that when you
> redefine a function, the new definition is
> active; unless the magic is under user
> control (you've said "defmacro" or
> "def-inline", then you know you've to
> recompile the callers), the run system has to
> manage the mess, i.e. it has to remember all
> the sites where things have to be inlined,
> like in the "inline cache" from the
> reference above.

No one in this thread, unless I missed
something, has made an attempt at defining,
explaining, and showing with examples how
inlining is done and what it means in Elisp:

(defun add-two-digits (a b)
  (+ a b) )
  
(defsubst add-two-digits-inline (i j)
  (+ i j) )

(defun add-it ()
  (add-two-digits 5 10)
  (add-two-digits-inline 15 5) )

This byte-compiles with no warnings/errors.

So what happens then, and what happens when
"add-it" is called?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

tomas wrote:

> OK, I roughly got your idea. I'm a bit out of
> time to discuss those things at depth

We should start by discussing *the basics*.
And I'm not just saying that because I don't
understand the finer points. I'm mean, I...
whatever

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Stefan Monnier]

> (defun add-two-digits (a b)
>   (+ a b) )
>   
> (defsubst add-two-digits-inline (i j)
>   (+ i j) )
>
> (defun add-it ()
>   (add-two-digits 5 10)
>   (add-two-digits-inline 15 5) )
>
> This byte-compiles with no warnings/errors.
>
> So what happens then, and what happens when
> "add-it" is called?

Try M-x disassemble RET add-it RET
to see how the two calls where compiled.


        Stefan

Getting functions usage examples [Was: Re: Knowing where a function has been used (e.g. for optimizing) [Was: Re: Optimising Elisp code]]

[Garreau, Alexandre]

On 2018-10-07 at 14:41, Emanuel Berg wrote:
> Garreau, Alexandre wrote:
>
>> (examples are the other
>> great way of learning: inherenthly more
>> imperfect than a definition but quicker and
>> easier)
>
> There should always first be the definition,

Agree.  Yet sometimes people are bad at abstraction, so when the
abstraction grow complex, in order to *more easily* understand the
example, after one trye, one can begin by getting a guess of what it
does, through an example (with a context, of course, because
side-effects (and imperative programming generally) and complex data
stractures), and then come back at the definition.

> then one simple example, then a more
> complicated example, then the example that
> confirms the rule.

as not everybody needs examples, and ideally we should just keep using
only definitions, nobody (we hope) will waste a lot of precious time
putting 2-3 call examples in the docstrings (especially if it’s
something complex so for each you need to describe a valid context).

Since the function is only dynamically typed, unless you begin
inference-typing or basing on symbols names, you will have a hard time
at automatically generating the simplest meaningful examples.

So having a grasp of “how is this function used in reality” may be quite
useful… and *then*, afterwards, someone could try something more
complex, intuitive and experimental, such as of these (potentially
thousands, or more) examples, automatically sorting out the most simple,
the most idiomatics, the better ones.

Then of course I get this may go from quite straightforward for one
special library function used one or three times around, to grow
especially complex as for “list of functions where `+' / `point' /
`progn' is used”.

And, regard to optimization, again, for these examples (the most used
functions), if one got automatically inlined, then it would require too
much recompiling, then the compiler might figure out it shouldn’t inline
it (unless the programmer accept to do so (maybe it could suggests her
about it (wrt its magnitude of usage through the rest of emacs)?)).

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Garreau, Alexandre]

Le 07/10/2018 à 10h10, tomas@tuxteam.de a écrit :
> On Sat, Oct 06, 2018 at 11:42:12PM +0200, Garreau, Alexandre wrote:
>
> [...]
>
>> Not really, including this, but I was speaking about the find-definition
>> feature of emacs specifically, since afterwards I talk about a symmetric
>> feature, that is, store the list of symbols refered by each function, even (or
>> rather, especially) if it is compiled, in a place linked to each
>> function definition: so if the functions `f', `g' and `h' refer to `a',
>> having in something linked to the `a' symbol [...]
>
> OK, I roughly got your idea. I'm a bit out of time to discuss those
> things at depth (I'd have to do much more reading than I could ATM
> to be able to do more than just handwaving), but one last observation
> from me: I think what is going to kill you is the inherently dynamic
> nature of LISP

some calls will be more interesting to inline than others, or more
expensive, especially as their re-inlining (and recompilation of callee)
might be expensive, so the compiler could use that to choose whether
it’s interesting or not, I guess.

In the case in which it’s interesting, I think that *dynamically*
recompiling is enough.

> (think eval: once you reach such a point, potentially *every*
> function, even those not yet known at compile time, can be
> referenced).

eval will references what it gets as argument, when speaking of
inlining, we’re only speaking about body, not arguments: *here* maybe
(and I’m unsure) dynamic IC could do something, indeed.  But it’s not
what we’re talking about: at the most, if they were in lisp (in emacs
they’re not afaik), we could inline something such as apply and get eval
as a simply recursive function (maybe even tail-recursive in some cases?
or that would imply too much of optimized rearranging of calls
probably…) or something alike that would be useful.  But eval is in C
and if we talk about elisp optimization we only talk about stuff in
lisp:  afaiu there’s no way anything can be inlined inside eval, or eval
be inlined inside anything.

Re: Getting functions usage examples [Was: Re: Knowing where a function has been used (e.g. for optimizing) [Was: Re: Optimising Elisp code]]

[Emanuel Berg]

Garreau, Alexandre wrote:

> as not everybody needs examples, and ideally
> we should just keep using only definitions,
> nobody (we hope) will waste a lot of precious
> time putting 2-3 call examples in the
> docstrings

I was speaking broadly, not about Elisp
docstrings. Like computer books. Learn to do it
all in 21 days. A friend once asked if you
really could learn it all in 21 days. I said,
"yes, if you already know it."

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

Stefan Monnier wrote:

> Try M-x disassemble RET add-it RET
> to see how the two calls where compiled.

OK, here goes:

(defsubst add-it-inline (i j)
  (+ i j) )

(defun add-it (a b)
  (+ a b) )

(defun test-normal ()
  (add-it 1 2) )
;; (test-normal)

;; byte code for test-normal:
;;   args: nil
;; 0       constant  add-it
;; 1       constant  1
;; 2       constant  2
;; 3       call      2
;; 4       return

(defun test-inline ()
  (add-it-inline 10 20) )
;; (test-inline)

;; byte code for test-inline:
;;   args: nil
;; 0       constant  10
;; 1       constant  20
;; 2       varbind   j
;; 3       dup
;; 4       varbind   i
;; 5       varref    j
;; 6       plus
;; 7       unbind    2
;; 8       return

OK, so there is no call, instead the code
is inserted. I suppose this is a glitch in the
Matrix, becuase what I can reCALL this is what
I've been saying this whole time...

But anyway, instead of dwelling on the past,
meditate on this:

;; byte code for add-it:
;;  args: (a b)
;; 0       varref    a
;; 1       varref    b
;; 2       plus      
;; 3       return

So as for machine instructions, the inline
version, "test-inline", has 9.
The "test-normal" has 5. The non-inlined
"add-it" has 4. 5 + 4 = 9. And I've saved the
best forst last: 9 = 9!

So there is no gain as to the number of machine
instructions. So where is the gain? The `call'
instruction specifically, or the implied
funcall overhead not visible in
the disassembly?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

Stefan Monnier wrote:

> Try M-x disassemble RET add-it RET
> to see how the two calls where compiled.

Next question (first was the number of machine
instructions vs. call overhead), next question:
I have 479 `defun's in my Elisp, any general
pointers as how to determine which of those
I should consider inlining?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code [again]

[Garreau, Alexandre]

On 2018-10-07 at 17:59, Emanuel Berg wrote:
> Stefan Monnier wrote:
> So there is no gain as to the number of machine instructions. So where
> is the gain? The `call' instruction specifically, or the implied
> funcall overhead not visible in the disassembly?

A smarter compiler could have optimized the discarded value (since we
don’t use its return value, and the inlined function don’t have any
side-effects (which otherwise would have been preserved)), or could have
simplified the result (I checked and with + 2 then - 2 it seems elisp
compiler don’t do this, while I’m sure gcc can) if something was going
to be done with it.

However, inlining is not an optimization made to gain space or reduce
the number of instruction (even worse: normally it could *increase* the
total amount of instruction and therefore use more memory), it’s here to
gain *speed*.  Not by executing less instructions, but by removing a
jump (the funcall), as in today’s computers, jumps (like function calls,
or conditionals) can increase execution time.  So at the end,
duplicating instructions can lead to faster execution, albeit increased
memory usage.

I kind of forgot why jumps are slow, but afair, either because the cache
being little it’s faster to execute nearby code, either because as
modern cpus try to do as much as possible in the same time (pipeline
instructions optimization, found in today’s superscalar cpu, iirc), they
need to know what will be executed next, and they stop searching what’s
next after a jump…  but that (+ loop unrolling) might be mitigated by
the fact that if you use too much memory at same time, it might not fit
in the smaller and faster cache, and then get slower…  so I guess here
it gets complex and architecture dependant, and becomes the job of
complex and native compilers to guess what is best to do.

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Garreau, Alexandre]

On 2018-10-07 at 18:10, Emanuel Berg wrote:
> Stefan Monnier wrote:
>
>> Try M-x disassemble RET add-it RET
>> to see how the two calls where compiled.
>
> Next question (first was the number of machine
> instructions vs. call overhead), next question:
> I have 479 `defun's in my Elisp, any general
> pointers as how to determine which of those
> I should consider inlining?

I’d say the most called ones, and the ones you don’t mind not being able
to modify once you began to use them: like if you inline `f', then you
use it in hundreds of functions, you won’t be able to just redefun `f'
and get all these functions benefit from the change.  So it’s a matter
of tradeof between speed (if that’s that much important) and dynamism.

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Barry Margolin]

In article <86d0slrb4h.fsf@zoho.com>, Emanuel Berg <moasen@zoho.com> 
wrote:

> Stefan Monnier wrote:
> 
> > Try M-x disassemble RET add-it RET
> > to see how the two calls where compiled.
> 
> Next question (first was the number of machine
> instructions vs. call overhead), next question:
> I have 479 `defun's in my Elisp, any general
> pointers as how to determine which of those
> I should consider inlining?

Only worry about the really small ones, less than 5 lines in the body. 
The more time spent in the function itself, the less significant the 
calling overhead is.

And as another answer said, you should only bother with functions that 
are called very frequently. You're only saving a few milliseconds per 
call, so if you only call the function once an hour the savings is 
insignificant.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Getting functions usage examples [Was: Re: Knowing where a function has been used (e.g. for optimizing) [Was: Re: Optimising Elisp code]]

[Van L]

>> then one simple example, then a more
>> complicated example, then the example that
>> confirms the rule.
> 
> as not everybody needs examples, and ideally we should just keep using
> only definitions

Examples are likely cache hits for 80% of people for bridging their gap.

Re: Optimising Elisp code [again]

[Emanuel Berg]

Garreau, Alexandre wrote:

> I kind of forgot why jumps are slow

Isn't it as simple as all the overhead of
placing the new function on the stack with
return pointers and stuff?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

If you have studied the Gnus source code, you
find that the defuns are insanely long. They go
on all but forever. This is because Gnus is
already slow, and perhaps Elisp is as well, so
they don't want to brake it up into modules
(smaller defuns) because then it would require
the funcall overhead. Perhaps Gnus would
benefit from inlining stuff?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

Barry Margolin wrote:

> And as another answer said, you should only
> bother with functions that are called very
> frequently. You're only saving a few
> milliseconds per call, so if you only call
> the function once an hour the savings
> is insignificant.

Is there a tool to find out which functions are
called the most often or do you do that by
using your mind and instinct alone?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code [again]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 511 bytes --]

On Mon, Oct 08, 2018 at 04:11:14PM +0200, Emanuel Berg wrote:
> Garreau, Alexandre wrote:
> 
> > I kind of forgot why jumps are slow
> 
> Isn't it as simple as all the overhead of
> placing the new function on the stack with
> return pointers and stuff?

And cold caches, and pipeline stalls and...

Modern processors are quite complex beasts, and much of it has to
do with the sad fact that RAM is much, much, MUCH slower than the
CPU, and they try to lie to you about it...

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Garreau, Alexandre]

On 2018-10-08 at 16:18, Emanuel Berg wrote:
> Barry Margolin wrote:
>
>> And as another answer said, you should only
>> bother with functions that are called very
>> frequently. You're only saving a few
>> milliseconds per call, so if you only call
>> the function once an hour the savings
>> is insignificant.
>
> Is there a tool to find out which functions are
> called the most often or do you do that by
> using your mind and instinct alone?

I guess both: knowing what you did you normally should know what
functions are the most “low-level” and “basics” (like “used everywhere”
in loops, or in loops loops, etc.).

But afair, there are also integrated profiling tools for emacs lisp :
(info "(elisp) Profiling")

Naming, and Gnus functions length [Was: Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]]

[Garreau, Alexandre]

On 2018-10-08 at 16:14, Emanuel Berg wrote:
> If you have studied the Gnus source code, you find that the defuns are
> insanely long. They go on all but forever. This is because Gnus is
> already slow, and perhaps Elisp is as well, so they don't want to
> brake it up into modules (smaller defuns) because then it would
> require the funcall overhead. Perhaps Gnus would benefit from inlining
> stuff?

Maybe Gnus functions length are just a question of style.  I guess their
developers are experienced enough to know how to properly use manual
inlining.  And anyway, as, as you pointed it, Gnus can already be
sometimes quite slow, it’s probably more because of I/O than function
call overhead, so this later must be neglictible in comparison.  So Gnus
functions length is probably unrelated.

I put Gnus dev in copy (hoping it’s relevant to the conversation and not
inapropriated to the situation) in case that can help getting more
relevant answers for this.

I personally myself tend to write really big functions, because I like
to have well-named things, and dislike to repeat myself and/or have a
lot of intermediary values and functions with bad names and poor
semantics, so until I come with a proper and nice semantic abstraction I
keep stuff in the same function.

This is unlike some languages like scheme, ocaml, etc. which potentially
requires recursion for loops in some cases, therefore obligating you to
give a name to each loop of your programs (thus potentially resulting in
needing a lot of names, and even potentially bad or complicated ones).

Also unlike pre-algebraic languages which may force you in using lot and
lot of temporary variables, which will inevitably result in either using
very, I find, poor names (such as “double_x = x * 2”), either totally
non-sensical and confusing names such as “A”, “B”, “C”, etc. (TeX is a
notable example where a lot of the loops use recursions, and stuff like
that, you end up needing many names and ending with names with strange
semantics such as “@”, double-“@”, etc. everywhere, resulting in cryptic
code for the unexperienced).

Re: Getting functions usage examples [

[Garreau, Alexandre]

On 2018-10-08 at 11:07, Van L wrote:
>>> then one simple example, then a more
>>> complicated example, then the example that
>>> confirms the rule.
>> 
>> as not everybody needs examples, and ideally we should just keep using
>> only definitions
>
> Examples are likely cache hits for 80% of people for bridging their gap.

Can you be more precise about what do you exactely mean by “bridging
their gap” (to what?) in this context?

Re: Naming, and Gnus functions length [Was: Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]]

[Emanuel Berg]

Garreau, Alexandre wrote:

>> If you have studied the Gnus source code,
>> you find that the defuns are insanely long.
>> They go on all but forever. This is because
>> Gnus is already slow, and perhaps Elisp is
>> as well, so they don't want to brake it up
>> into modules (smaller defuns) because then
>> it would require the funcall overhead.
>> Perhaps Gnus would benefit from
>> inlining stuff?
>
> Maybe Gnus functions length are just
> a question of style. I guess their developers
> are experienced enough to know how to
> properly use manual inlining. And anyway, as,
> as you pointed it, Gnus can already be
> sometimes quite slow, it’s probably more
> because of I/O than function call overhead,
> so this later must be neglictible in
> comparison. So Gnus functions length is
> probably unrelated.

I once said to the Gnus developers, why don't
you break down those age-long functions, to
make it much easier to read and maintain the
code?

They said Gnus is slow, Elisp is slow, and
breaking the defuns up into neat modules would
make it even slower.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Naming, and Gnus functions length [

[Garreau, Alexandre]

On 2018-10-08 at 17:43, Emanuel Berg wrote:
> Garreau, Alexandre wrote:
>>> If you have studied the Gnus source code, you find that the defuns
>>> are insanely long.  They go on all but forever. This is because Gnus
>>> is already slow, and perhaps Elisp is as well, so they don't want to
>>> brake it up into modules (smaller defuns) because then it would
>>> require the funcall overhead.  Perhaps Gnus would benefit from
>>> inlining stuff?
>>
>> Maybe Gnus functions length are just a question of style. I guess
>> their developers are experienced enough to know how to properly use
>> manual inlining. And anyway, as, as you pointed it, Gnus can already
>> be sometimes quite slow, it’s probably more because of I/O than
>> function call overhead, so this later must be neglictible in
>> comparison. So Gnus functions length is probably unrelated.
>
> I once said to the Gnus developers, why don't you break down those
> age-long functions, to make it much easier to read and maintain the
> code?
>
> They said Gnus is slow, Elisp is slow, and breaking the defuns up into
> neat modules would make it even slower.

Seriously? is Gnus so complex (algorithmically at run-time I mean)?
damn, I wouldn’t have figured out…  Then indeed maybe it could… as these
“neat modules” would be only used by Gnus and not all the way around…
but yet: keeping the dynamic behavior (which must be wanted since no
defsubst/define-inline is already used to fix this), while having the
neat modules, if they’re ubiquitous enough, might mean recompiling a big
part of Gnus at each changed function.  So in the thesis of an
automatically-optimizing compiler, maybe the compiler wouldn’t inline
that much…

Re: Naming, and Gnus functions length [

[Emanuel Berg]

Garreau, Alexandre wrote:

> Seriously? is Gnus so complex

Surely you jest! Gnus is probably the most
complex communication facility ever...

Anyway, in the Gnus source, there are
4112 "defun"s, and 122 "defsubst"s.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Naming, and Gnus functions length [Was: Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]]

[Eli Zaretskii]

> From: Emanuel Berg <moasen@zoho.com>
> Date: Mon, 08 Oct 2018 17:43:11 +0200
> 
> I once said to the Gnus developers, why don't
> you break down those age-long functions, to
> make it much easier to read and maintain the
> code?
> 
> They said Gnus is slow, Elisp is slow, and
> breaking the defuns up into neat modules would
> make it even slower.

They are right.

Why is Elisp slow? (was: Re: Naming, and Gnus functions length [Was: Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]])

[Emanuel Berg]

Eli Zaretskii wrote:

>> I once said to the Gnus developers, why
>> don't you break down those age-long
>> functions, to make it much easier to read
>> and maintain the code?
>> 
>> They said Gnus is slow, Elisp is slow, and
>> breaking the defuns up into neat modules
>> would make it even slower.
>
> They are right.

Why is Elisp slow?

Is it funcall overhead that is slow in
particular or is Elisp slow in other aspects
as well?

Is Lisp slow in general? For example, compared
to C? If so, what is the reason for this?

How much faster does it (Elisp) get with the
byte compiler?

How much faster can one get it with inlining,
assuming it is done by a skilled hand?

Are there other speed-up methods other than
inlining. (I mean specifically, not "write
better code".)

Is there like a list of things, functions
perhaps, or data structures, that shouldn't be
used unless necessary, because they are slow?

Is there a tool that I can enable, then use
Emacs a whole day (or night rather), and then
the tool tells me "everything is OK, except for
when you do this, that sticks out as something
that drags down the tempo, so maybe you should
have a look at it"?

Facts for fans: This post contains 12 "?"s!

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

Barry Margolin wrote:

> Only worry about the really small ones, less
> than 5 lines in the body. The more time spent
> in the function itself, the less significant
> the calling overhead is.

OK!

> And as another answer said, you should only
> bother with functions that are called
> very frequently.

How do I know that? Is there a tool to make
a ranking which functions were used the most
during a session?

Excluding the `self-insert-command' and such
functions, of course.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[Garreau, Alexandre]

elisp is mostly interpreted (contrarily to C which is always compiled to
native code, so directly executed by the CPU) and very very dynamic
afaik, not that much optimized at byte-compilation afaiu from what I
observed, and I guess not that much optimized from the VM point of view…

Also aren’t you talking about the tool you’re asking about the, I think,
previously mentioned profiling features of elisp? Like mentioned in
(info "(elisp) Profiling") (you do `M-x profiler-start' to start and
then after a while `M-x profiler-report' to get the relevant
informations).

Re: Optimising Elisp code [again]

[Barry Margolin]

In article <mailman.1852.1539009893.1284.help-gnu-emacs@gnu.org>,
 <tomas@tuxteam.de> wrote:

> On Mon, Oct 08, 2018 at 04:11:14PM +0200, Emanuel Berg wrote:
> > Garreau, Alexandre wrote:
> > 
> > > I kind of forgot why jumps are slow
> > 
> > Isn't it as simple as all the overhead of
> > placing the new function on the stack with
> > return pointers and stuff?
> 
> And cold caches, and pipeline stalls and...
> 
> Modern processors are quite complex beasts, and much of it has to
> do with the sad fact that RAM is much, much, MUCH slower than the
> CPU, and they try to lie to you about it...

Since we're talking about a byte-code interpreter, not machine code, the 
pipeline is probably not relevant.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Barry Margolin]

In article <86d0sknoud.fsf@zoho.com>, Emanuel Berg <moasen@zoho.com> 
wrote:

> > And as another answer said, you should only
> > bother with functions that are called
> > very frequently.
> 
> How do I know that? Is there a tool to make
> a ranking which functions were used the most
> during a session?

Emacs Lisp has a profiler. Read the section on it in the Elisp manual.

In general you shouldn't even be worrying about this unless your code is 
running too slow to begin with. Then you use the profiler to find the 
bottlenecks and try to improve them. Inline coding is likely to be the 
LEAST effective way to improve performance, except for the simplest 
functions.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Why is Elisp slow?

[Emanuel Berg]

Garreau, Alexandre wrote:

> elisp is mostly interpreted (contrarily to
> C which is always compiled to native code, so
> directly executed by the CPU) and very very
> dynamic afaik

Why can't Elisp be compiled as well?

With "dynamic", do you mean for example one can
change a defun, re-eval it, and the next call,
still from the same process, will execute the
new, modified version?

If so, that's a feature we all love and that's
why Elisp is so addicting because it gives
instant feedback. Still, we do it just to
a tiny tiny fraction of mostly our own Elisp.
The rest stays the same.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code [again]

[Emanuel Berg]

Barry Margolin wrote:

> Since we're talking about a byte-code
> interpreter, not machine code, the pipeline
> is probably not relevant.

Cannot one have a three-layer architecture, C,
compiled Lisp, interpretated Lisp?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

Barry Margolin wrote:

> Inline coding is likely to be the LEAST
> effective way to improve performance, except
> for the simplest functions.

If we are talking my personal Elisp - of
course! It will rather be like having an impact
driver in my tool box - it is a good feeling to
have and to understand, but will it improve my
life quality in terms of its functionality,
which obviously is the reason for its
existence? Probably not or at least
very rarely!

If we are talking the insanely long functions
of Gnus, if the funcall overhead is the reason
for that, I don't see why modularization plus
inlining couldn't be one puzzle piece of
the remedy.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[Garreau, Alexandre]

On 2018-10-09 at 08:23, Emanuel Berg wrote:
> Garreau, Alexandre wrote:
>
>> elisp is mostly interpreted (contrarily to
>> C which is always compiled to native code, so
>> directly executed by the CPU) and very very
>> dynamic afaik
>
> Why can't Elisp be compiled as well?

Afaik it can, as any turing-complete language, that just requires hard
work: either writing a full native compiler, or interfacing with one
(like gcc).

> With "dynamic", do you mean for example one can change a defun,
> re-eval it, and the next call, still from the same process, will
> execute the new, modified version?
>
> If so, that's a feature we all love and that's why Elisp is so
> addicting because it gives instant feedback. Still, we do it just to a
> tiny tiny fraction of mostly our own Elisp.  The rest stays the same.

For instance.  There are other details such as garbage collection /
dynamic allocation of everything, having everything on heap, dynamic
typing, etc.  But it’s not going to be the most important I guess.

However being able to change anything (or most things) in emacs is a
stated key feature of it, so it makes an full interpreter and the
presence of almost all (elisp) sources mandatory, as well as a fast
(byte-)compiler.

Even more: ideally, to be able to tweak absolutely everything,
everything should be in lisp and nothing in C (or the strict minimal to
make it portable without having to reinvent the wheel of porting to
different architectures in emacs), but afair currently there is still
much in C because indeed elisp is too slow, and the most important
features are needed to be in C so emacs stays usable.

So more work would be needed to give elisp a native compiler… the most
similar approach I recall are gcl (GNU Common Lisp, common lisp is quite
near to elisp) wanting to become a gcc frontend (it’s already compiling
to C), and Guile proposing to become the elisp engine… as guile is
normally faster because more optimizations and stuff, it should have
made elisp faster, but Guile Emacs is actually slow because, afair,
buffer-local variables, dynamic scoping, and low-level stuff like that
are bad supported, and in the end there is really few work on it (and
Guile is far from being as stable as emacs anyway), so as they want to
implement native compilation (but more-or-less (forgot) independently of
gcc afair) too, that’d make a native compiler for elisp… but as the
dynamic stuff is slow on their VM anyway…

Re: Optimising Elisp code [again]

[Garreau, Alexandre]

On 2018-10-09 at 08:26, Emanuel Berg wrote:
> Barry Margolin wrote:
>
>> Since we're talking about a byte-code interpreter, not machine code,
>> the pipeline is probably not relevant.
>
> Cannot one have a three-layer architecture, C, compiled Lisp,
> interpretated Lisp?

It’s already like this… or what are you talking about otherwise?  We got
C, which interprets directly both byte-compiled and interpreted lisp,
which is all the high-level stuff.

Re: Getting functions usage examples [

[Van L]

>>>> then one simple example, then a more
>>>> complicated example, then the example that
>>>> confirms the rule.
>>> 
>>> as not everybody needs examples, and ideally we should just keep using
>>> only definitions
>> 
>> Examples are likely cache hits for 80% of people for bridging their gap.
> 
> Can you be more precise about what do you exactely mean by “bridging
> their gap” (to what?) in this context?

Most people have a goal wanting to be achieved.

They consult the documentation examples to get their 
fill-in the gap in understanding to get there.

Picture Ikea furniture assembly as a way of getting
the thing done. Some non-technology experts or new comers
learn technology in comic or graphic novel reading form.

Re: Why is Elisp slow?

[Stefan Monnier]

> elisp is mostly interpreted (contrarily to C which is always compiled to
                                                      ^^^
                                                    almost

> native code, so directly executed by the CPU) and very very dynamic
> afaik, not that much optimized at byte-compilation afaiu from what I
> observed, and I guess not that much optimized from the VM point of view…

Emacs's implementation of Elisp is fairly naive and not carefully
optimized.  As a result it's not blazing fast.  This said, on the rare
occasions Elisp was compared to other systems it didn't seem nearly as
slow as I thought.  E.g. I wouldn't be surprised to hear that it's about
as fast as Python (not talking about things like PyPy, tho).


        Stefan

Re: Optimising Elisp code [again]

[Emanuel Berg]

Garreau, Alexandre wrote:

>> Cannot one have a three-layer architecture,
>> C, compiled Lisp, interpretated Lisp?
>
> It’s already like this… or what are you
> talking about otherwise? We got C, which
> interprets directly both byte-compiled and
> interpreted lisp, which is all the
> high-level stuff.

Isn't there an FOSS CL compiler around one can
duplicate/modify?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code [again]

[Garreau, Alexandre]

On 2018-10-09 at 15:28, Emanuel Berg wrote:
> Garreau, Alexandre wrote:
>>> Cannot one have a three-layer architecture,
>>> C, compiled Lisp, interpretated Lisp?
>>
>> It’s already like this… or what are you
>> talking about otherwise? We got C, which
>> interprets directly both byte-compiled and
>> interpreted lisp, which is all the
>> high-level stuff.
>
> Isn't there an FOSS CL compiler around one can
> duplicate/modify?

gcl. It’s GNU, it uses gcc (for now it generates C and compile that with
gcc, but it was planned to make it a gcc frontend a long time ago, but
iso support comes first afair).  The only other compiler I know is the
most used/popular (afaik), sbcl, but it’s not GNU (so it would be less
meaningful since there are a bunch of GNU lisp implementations)… and
clisp has a *VM* too (and its byte-compiler), but I guess it’s better
than emacs’ one.

However if you’re talking about using a cl compiler to host emacs,
*iirc*, rms said cl was too big for emacs which he wanted not too big
(and indeed, unless some modifications are maid, afaiu, it might imply
to make gcc a mandatory dependency of emacs, which is pretty big)…

Interestingly, on my (debian stable) system, gcl [0] appears to be 7,8M
(but there is a 15M gcl static library [3]), emacs [1] 11M, and libguile
[2] 1,6M… so is gcl smaller than emacs (also maybe it is the static that
contain all gcl, and maybe gcl should be bigger when everything is
complete and standard?)? I can’t measure clisp as I didn’t attempted
(yet) to recompile here, and it’s not packaged for the current debian
stable.

Btw, what would it change to make emacs hosted by gcl, compared to by
guile (modulo the obvious facilities of having elisp and cl more near
than elisp and scheme)? would the gcc runtime dep hard to remove? would
that be so big? would that be easier to get buffer-local variables,
dynamic scoping, et al?

[0] resolves to /usr/lib/gcl-2.6.12/unixport/saved_gcl 
[1] /usr/bin/emacs25-x
[2] /usr/lib/i386-linux-gnu/libguile-2.0.so.22.8.1
[3] /usr/lib/gcl-2.6.12/unixport/libgcl.a

Re: Why is Elisp slow?

[Eli Zaretskii]

> From: "Garreau\, Alexandre" <galex-713@galex-713.eu>
> Date: Tue, 09 Oct 2018 10:05:43 +0200
> 
> > Why can't Elisp be compiled as well?
> 
> Afaik it can, as any turing-complete language, that just requires hard
> work: either writing a full native compiler, or interfacing with one
> (like gcc).

Please take a look at the libjit branch in the Emacs repository, it
implements one idea of doing that.

Re: Optimising Elisp code [again]

[Emanuel Berg]

Garreau, Alexandre wrote:

> Btw, what would it change to make emacs
> hosted by gcl

I mean, compile the Lisp that isn't user
defined and unlikely to change - compile for
real, not byte-compile for
sped-up interpretation.

I.e.: fun-and-play Elisp on the top
      compiled Elisp in the middle
      C downmost

?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code [again]

[Garreau, Alexandre]

On 2018-10-09 at 17:26, Emanuel Berg wrote:
> Garreau, Alexandre wrote:
>
>> Btw, what would it change to make emacs
>> hosted by gcl
>
> I mean, compile the Lisp that isn't user
> defined and unlikely to change - compile for
> real, not byte-compile for
> sped-up interpretation.
>
> I.e.: fun-and-play Elisp on the top
>       compiled Elisp in the middle
>       C downmost
>
> ?

We’d get the same problems as mentioned before about loss of dynamism
(you’ll need to recompile everything if changing a core thing, which you
currently don’t need).  Yet the opposite would also be there: some core
functions might be reimplemented in lisp, and then these could be easier
to hack for beginners, or any people uncomfortable with C.

But beside that you would get awesome performances I guess.  Not
*always* like C, because, I guess there are still mallocs and garbage
collection all the way around, types may not be determined as you’d
like, etc.  but especially as elisp is quite imperative (but potentially
less well used, sometimes), you may get quite near performances (I did
hear about 1,2/1,5 more slow than C for compiled elisp).

I’d like to see how the performances of libjit (native jit compiler I
guess) emacs, as previously mentioned, I’ll try it I think.

Re: Optimising Elisp code [again]

[Barry Margolin]

In article <86ftxfm98q.fsf@zoho.com>, Emanuel Berg <moasen@zoho.com> 
wrote:

> Garreau, Alexandre wrote:
> 
> > Btw, what would it change to make emacs
> > hosted by gcl
> 
> I mean, compile the Lisp that isn't user
> defined and unlikely to change - compile for
> real, not byte-compile for
> sped-up interpretation.
> 
> I.e.: fun-and-play Elisp on the top
>       compiled Elisp in the middle
>       C downmost
> 
> ?

The Elisp interpreter is probably tightly intertwined with the Emacs 
editor code. Disentangling them so you can compile Elisp to machine code 
would be a big project, and probably not worth it.

Byte-code interpretation is common in many modern languages, it makes it 
easier for the languages to be portable. Python and PHP both compile to 
byte code rather than machine code. It's generally fast enough for 
interactive applications. For computationally-intensive applications, 
there are often libraries that can be linked in, like Numpy for Python.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Barry Margolin]

In article <86tvlvmxtz.fsf@zoho.com>, Emanuel Berg <moasen@zoho.com> 
wrote:

> If we are talking the insanely long functions
> of Gnus, if the funcall overhead is the reason
> for that, I don't see why modularization plus
> inlining couldn't be one puzzle piece of
> the remedy.

If a function is "insanely long" then inlining will have negligible 
effect. Funcall overhead is only relevant if the function is really 
short, so it spends nearly as much time calling the function as doing 
the actual work of the function, AND you call the function frequently 
enough that this overhead adds up to something significant.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

Barry Margolin wrote:

>> If we are talking the insanely long
>> functions of Gnus, if the funcall overhead
>> is the reason for that, I don't see why
>> modularization plus inlining couldn't be one
>> puzzle piece of the remedy.
>
> If a function is "insanely long" then
> inlining will have negligible effect.
> Funcall overhead is only relevant if the
> function is really short, so it spends nearly
> as much time calling the function as doing
> the actual work of the function, AND you call
> the function frequently enough that this
> overhead adds up to something significant.

The insanely long functions would be shorter if
they, instead of doing one million things,
called other functions to help them with that.
This isn't done because it isn't fast enough.
But if inline works as you describe it, some of
that code could in fact be moved out of the
insanely long function, while still not slowing
it down with funcall overhead.


-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Óscar Fuentes]

Barry Margolin <barmar@alum.mit.edu> writes:

> If a function is "insanely long" then inlining will have negligible 
> effect. Funcall overhead is only relevant if the function is really 
> short, so it spends nearly as much time calling the function as doing 
> the actual work of the function, AND you call the function frequently 
> enough that this overhead adds up to something significant.

Inlining is not only about funcall overhead, although most compilers use
code length as the unique heuristic for deciding when to inline a
function.

Re: Optimising Elisp code [again]

[Óscar Fuentes]

Barry Margolin <barmar@alum.mit.edu> writes:

> The Elisp interpreter is probably tightly intertwined with the Emacs 
> editor code. Disentangling them so you can compile Elisp to machine code 
> would be a big project, and probably not worth it.

http://git.savannah.gnu.org/cgit/emacs.git/log/?h=feature/libjit

https://github.com/tromey/el-compilador

Re: Optimising Elisp code [again]

[Garreau, Alexandre]

On 2018-10-10 at 12:18, Barry Margolin wrote:
> In article <86ftxfm98q.fsf@zoho.com>, Emanuel Berg <moasen@zoho.com> 
> wrote:
>> I mean, compile the Lisp that isn't user
>> defined and unlikely to change - compile for
>> real, not byte-compile for
>> sped-up interpretation.
>> 
>> I.e.: fun-and-play Elisp on the top
>>       compiled Elisp in the middle
>>       C downmost
>> 
>> ?
>
> The Elisp interpreter is probably tightly intertwined with the Emacs 
> editor code. Disentangling them so you can compile Elisp to machine code 
> would be a big project, and probably not worth it.

Why would that need distangling?  Couldn’t editor code be compiled as
well?  I understand things need to stay dynamic and redefinable, but a
large portion would remain unchanged anyway for most of time and the
things that would most likely be changed would be less stable and
less low-level things that wouldn’t break other stuff wouldn’t it?

Re: Optimising Elisp code [again]

[Emanuel Berg]

Óscar Fuentes wrote:

>> The Elisp interpreter is probably tightly
>> intertwined with the Emacs editor code.
>> Disentangling them so you can compile Elisp
>> to machine code would be a big project, and
>> probably not worth it.
>
> http://git.savannah.gnu.org/cgit/emacs.git/log/?h=feature/libjit
>
> https://github.com/tromey/el-compilador

Wonderful! But how does one integrate the
compiled code with the rest of Emacs?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Barry Margolin]

In article <86woqprcdb.fsf@zoho.com>, Emanuel Berg <moasen@zoho.com> 
wrote:

> Barry Margolin wrote:
> 
> >> If we are talking the insanely long
> >> functions of Gnus, if the funcall overhead
> >> is the reason for that, I don't see why
> >> modularization plus inlining couldn't be one
> >> puzzle piece of the remedy.
> >
> > If a function is "insanely long" then
> > inlining will have negligible effect.
> > Funcall overhead is only relevant if the
> > function is really short, so it spends nearly
> > as much time calling the function as doing
> > the actual work of the function, AND you call
> > the function frequently enough that this
> > overhead adds up to something significant.
> 
> The insanely long functions would be shorter if
> they, instead of doing one million things,
> called other functions to help them with that.
> This isn't done because it isn't fast enough.
> But if inline works as you describe it, some of
> that code could in fact be moved out of the
> insanely long function, while still not slowing
> it down with funcall overhead.

Long functions are often long because of complex logic, which can be 
hard to extract into separate functions.

Also, we tend to extract code into separate functions only when there's 
some logic to it -- it has to have a meaningful identity outside the 
function where it exists. If a function can only be used in one place, 
and refers to lots of the variables in that caller, then it's often hard 
to justify splitting it out -- it actually makes things more confusing 
because the modularity doesn't make sense.

I doubt that anyone makes decision about whether to split large 
functions into separate functions based on the overhead of function 
calling.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

Barry Margolin wrote:

> Long functions are often long because of
> complex logic, which can be hard to extract
> into separate functions.
>
> Also, we tend to extract code into separate
> functions only when there's some logic to it
> -- it has to have a meaningful identity
> outside the function where it exists.
> If a function can only be used in one place,
> and refers to lots of the variables in that
> caller, then it's often hard to justify
> splitting it out -- it actually makes things
> more confusing because the modularity doesn't
> make sense.

Complexity isn't reduced by having tons of
stuff in a single function that goes on
forever- on the contrary, this makes it more
difficult to read, understand, and maintain.

But of course the modularity should make sense!

> I doubt that anyone makes decision about
> whether to split large functions into
> separate functions based on the overhead of
> function calling.

This is what the Gnus people told me when
I suggested not writing functions that are
longer and deeper than a novel by Dostoevsky -
they said, ~"Gnus is slow, Elisp is slow,
making neat modules of those defuns will make
it even slower."

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Optimising Elisp code [again]

[Robert Thorpe]

Emanuel Berg <moasen@zoho.com> writes:

> Óscar Fuentes wrote:
>
>>> The Elisp interpreter is probably tightly
>>> intertwined with the Emacs editor code.
>>> Disentangling them so you can compile Elisp
>>> to machine code would be a big project, and
>>> probably not worth it.
>>
>> http://git.savannah.gnu.org/cgit/emacs.git/log/?h=feature/libjit
>>
>> https://github.com/tromey/el-compilador
>
> Wonderful! But how does one integrate the
> compiled code with the rest of Emacs?

Firstly, I think it's worth clarifying how things work now.  Emacs has
two interpreters.  There's the interactive interpreter, that's what's run
when you type elisp straight into Emacs.  It is used to interpret files
ending in ".el".  Then there's the bytecode interpreter.  To use that
you have to compile a program to bytecode first.  It's the bytecode
interpreter that deals with ".elc" files.  The bytecode interpreter is
much faster than the interactive interpreter.  (I expect you know all
this Emanuel, I'm just writing it here for others who are reading who
might not).

The libjit branch works by compiling bytecode into native code.  As I
understand it, this is done at runtime, so the user doesn't have to do
anything.  I think there was another project that did something similar
at compile time.  Compiling bytecode sounds a bit strange, but it makes
sense.  In many cases what the JIT does is emit the same code that
is inside the bytecode interpreter.  In a very simplified sense, it
works like this....  So, the bytecode interpreter is a big switch/case
statement.  Somewhere there is instruction FOO.  When the JIT compiler
sees FOO it inserts into it's output buffer the code that was in the
bytecoding interpreter to deal with that opcode.  There are some
optimizations beyond that.

I think it's a good idea and I hope it will be merged into mainline
Emacs one day.

BR,
Robert Thorpe

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Barry Margolin]

In article <8636tcl05a.fsf@zoho.com>, Emanuel Berg <moasen@zoho.com> 
wrote:

> Barry Margolin wrote:
> 
> > I doubt that anyone makes decision about
> > whether to split large functions into
> > separate functions based on the overhead of
> > function calling.
> 
> This is what the Gnus people told me when
> I suggested not writing functions that are
> longer and deeper than a novel by Dostoevsky -
> they said, ~"Gnus is slow, Elisp is slow,
> making neat modules of those defuns will make
> it even slower."

Sounds like a bullshit rationalization. The real reason is probably that 
refactoring is a PITA.

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Eric Abrahamsen]

Barry Margolin <barmar@alum.mit.edu> writes:

> In article <8636tcl05a.fsf@zoho.com>, Emanuel Berg <moasen@zoho.com> 
> wrote:
>
>> Barry Margolin wrote:
>> 
>> > I doubt that anyone makes decision about
>> > whether to split large functions into
>> > separate functions based on the overhead of
>> > function calling.
>> 
>> This is what the Gnus people told me when
>> I suggested not writing functions that are
>> longer and deeper than a novel by Dostoevsky -
>> they said, ~"Gnus is slow, Elisp is slow,
>> making neat modules of those defuns will make
>> it even slower."
>
> Sounds like a bullshit rationalization. The real reason is probably that 
> refactoring is a PITA.

As someone who's currently trying to refactor Gnus (replacing obarray
hashtables with actual hashtables), I can confirm that it is indeed a
PITA.

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

Barry Margolin wrote:

> Sounds like a bullshit rationalization.
> The real reason is probably that refactoring
> is a PITA.

If so, why did they write the code that way to
begin with? Was everything even slower then or
did they simply not give it much thought?
Refactoring may be difficult but writing
modular code to begin with is obviously much
easier than the opposite as small modules can
be developed and tested individually so
debugging and everything else is much easier.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1260 bytes --]

On Sat, Oct 13, 2018 at 10:51:56PM +0200, Emanuel Berg wrote:
> Barry Margolin wrote:
> 
> > Sounds like a bullshit rationalization.
> > The real reason is probably that refactoring
> > is a PITA.
> 
> If so, why did they write the code that way to
> begin with? [...]

Code "grows", one single (minimum-effort) step by step, following
a path of minimum resistance over a time-changing fractal landscape.

As one colleague of me used to put it (and he's one of the most
gifted programmers I know personally), code grows all by itself,
and the programmer's job is to prune it and to keep the pruning
rate high enough so to not be drowned by the jungle.

Put in more sober terms, usually you start with one idea and a
rough sketch. Most of the time, this sketch is wrong (because
you aren't God), but when it's right enough you start getting
users, so you keep going, and fixing up things as they come.

From time to time the tension between your original vision and
the one you have now (because you're older, or because the folks
pushing the project have changed, or because reality, or your
users, or the weather evolved) is so big that you take the hurdle
of a Big Refactor. Then your users will hate you :-)

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

tomas wrote:

> Code "grows", one single (minimum-effort)
> step by step, following a path of minimum
> resistance over a time-changing
> fractal landscape.
>
> As one colleague of me used to put it (and
> he's one of the most gifted programmers
> I know personally), code grows all by itself,
> and the programmer's job is to prune it and
> to keep the pruning rate high enough so to
> not be drowned by the jungle.

Code grows, but not necessarily by enlonging
the defuns that are already there. Instead,
when some new situation arrives, write yet
another defun to handle it. Then call it from
the original function. That way the original
function grows, but only by a single line.

To do this is much, much simpler than to write
an insanely long defun, and when insanity
becomes a problem, try to "refactor" it!

[This is a general remark; I never did any work
on Gnus so I don't know what explanation holds
the most truth.]

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1470 bytes --]

On Sun, Oct 14, 2018 at 09:10:56PM +0200, Emanuel Berg wrote:
> tomas wrote:
> 
> > Code "grows", one single (minimum-effort)
> > step by step [...]

> Code grows, but not necessarily by enlonging
> the defuns that are already there [...]

Most of the time it does, though: *this* is the
path of least resistance, for small steps.

> when some new situation arrives, write yet
> another defun to handle it. Then call it from
> the original function. That way the original
> function grows, but only by a single line.

On the long term this tends lead to horrible code,
where the defuns reflect all the history of the
situations, as they arrived, not the problem's
structure as it is *now*. But yes, you see something
of that in actual code -- real life is always a
compromise.

> To do this is much, much simpler than to write
> an insanely long defun, and when insanity
> becomes a problem, try to "refactor" it!

I've seen lots of code evolve. I'm an old guy. My
experience seems to contradict your assessment above.

Keeping code clean, always finding the right abstractions
and enforcing them throughout takes a non-trivial amount
of effort. I tend to think that "it's worth it", at least
in terms of self-esteem and developer satisfaction, and
I like to think that, in the long term, it amortises itself
in terms of effort, but for the last one I can't say whether
(or when) it is wishful thinking or not.

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

tomas wrote:

> On the long term this tends lead to horrible
> code, where the defuns reflect all the
> history of the situations, as they arrived,
> not the problem's structure as it is *now*.

Why is this horrible code? Isn't it just
natural that the code reflects how problems
have been solved over time, with changes in
technology, etc.? Isn't it much better to have
this "reflection" in neat modules than in
single functions spanning 3 A4 each?

Besides, if some of these defuns are really
part of history with no relevance for the
present situation, it is much easier to delete
them [1] than to mess with the
original huge functions!

[1] Better yet, delete *the call* to them and
    to move the modules to an out-of-action
    source file, as history might
    repeat itself.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Van L]

> [1] Better yet, delete *the call* to them and
>    to move the modules to an out-of-action
>    source file, as history might
>    repeat itself.

Surely, this is an easy problem for Deep Learning on call-traces to prune.

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Garreau, Alexandre]

On 2018-10-15 at 21:27, Emanuel Berg wrote:
> tomas wrote:
>
>> On the long term this tends lead to horrible
>> code, where the defuns reflect all the
>> history of the situations, as they arrived,
>> not the problem's structure as it is *now*.
>
> Why is this horrible code? Isn't it just
> natural that the code reflects how problems
> have been solved over time, with changes in
> technology, etc.?

No, VCSes a here for that.

> Isn't it much better to have this "reflection" in neat modules than in
> single functions spanning 3 A4 each?

If these modules have no meaning, no, because it will harden the work of
understanding, thus, for instance, of debugging, the function.  While
the same functions may have piece that will cancel or make useless
pieces of other functions, if you just “add up historically”, you may,
if you want to solve the problem as it is *now*, simplify the whole
thing.  And it won’t simply be refactoring by adding functions or
macros, it will be removing stuff.

> Besides, if some of these defuns are really
> part of history with no relevance for the
> present situation, it is much easier to delete
> them [1] than to mess with the
> original huge functions!

> [1] Better yet, delete *the call* to them and
>     to move the modules to an out-of-action
>     source file, as history might
>     repeat itself.

VCS, search tools, or otherwise devs memories, are here for that.

And even if it stayed in the source code anyway people are interested in
functions that are documented and that they use, directly or indirectly.
They might as well not see it if it’s not used.

Re: Knowing where a function has been used (bis) [Was: Re: Optimising Elisp code]

[Emanuel Berg]

Garreau, Alexandre wrote:

>>> On the long term this tends lead to
>>> horrible code, where the defuns reflect all
>>> the history of the situations, as they
>>> arrived, not the problem's structure as it
>>> is *now*.
>>
>> Why is this horrible code? Isn't it just
>> natural that the code reflects how problems
>> have been solved over time, with changes in
>> technology, etc.?
>
> No, VCSes a here for that.

It is a fact of life that code develops and
grows in different directions. Different parts
of the code reflect more so the original idea,
other parts of the code refect more so
situations that arrived without anyone thinking
of them day one. This is completely natural and
not something so you can use the code as
a history book, if anyone now got that idea.

>> Isn't it much better to have this
>> "reflection" in neat modules than in single
>> functions spanning 3 A4 each?
>
> If these modules have no meaning, no, because
> it will harden the work of understanding,
> thus, for instance, of debugging, the
> function. While the same functions may have
> piece that will cancel or make useless pieces
> of other functions, if you just “add up
> historically”, you may, if you want to solve
> the problem as it is *now*, simplify the
> whole thing. And it won’t simply be
> refactoring by adding functions or macros, it
> will be removing stuff.

The modules should have have meaning and
reflect how the problem is solved now. If they,
at some point, become totally obsolete, they
are much easier to drop as modules than to try
to extract that exact portion of the original,
3 A4 long defun, without creating bugs while
doing that.

>> [1] Better yet, delete *the call* to them
>> and to move the modules to an out-of-action
>> source file, as history might repeat itself.
>
> VCS, search tools, or otherwise devs
> memories, are here for that. And even if it
> stayed in the source code anyway people are
> interested in functions that are documented
> and that they use, directly or indirectly.
> They might as well not see it if it’s
> not used.

OK, if you have found a spot where they are the
most visible, go ahead and use it. To me, an
"obsolete-source" directory is fine.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[Van L]

Emanuel Berg <moasen@zoho.com> writes:

> Why can't Elisp be compiled as well?

I came across this tip of an iceberg in the commentary.

#+name: xxx-byte-opt-el
#+begin_src emacs-lisp
  ;; "No matter how hard you try, you can't make a racehorse out of a pig.
  ;; You can, however, make a faster pig."
  ;;
  ;; Or, to put it another way, the Emacs byte compiler is a VW Bug.  This code
  ;; makes it be a VW Bug with fuel injection and a turbocharger...  You're
  ;; still not going to make it go faster than 70 mph, but it might be easier
  ;; to get it there.
#+end_src

If a big advance from an Industry player, such as FB, were to release a
JIT, that the GNU Emacs legal dept. does not object to, does that help?

-- 
© 2019 Van L
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
                             "How did we get to this point?" - William P. Barr

Re: Why is Elisp slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1778 bytes --]

On Thu, May 02, 2019 at 02:49:22PM +1000, Van L wrote:

>  ;; "No matter how hard you try, you can't make a racehorse out of a pig.
>  ;; You can, however, make a faster pig."

[...]

> If a big advance from an Industry player, such as FB, were to release a
> JIT, that the GNU Emacs legal dept. does not object to, does that help?

I, for one, appreciate Emacs much more for other characteristics than
its speed.

Its extremely judicious developer community, which manages advances
at a steady pace (in a 30 year old project) while generally avoiding
disrupting surprises for its regular users is... amazing.

I've yet to run into a serious limitation posed by its "slow" elisp
implementation.

It's the same outside Emacs. For most of my day-to-day problems, I
use known-slow language implementations. For those rare exceptions,
there are other tools.

Of course Common Lispers (and to some extent Schemers, and other
functional programmers [1]) are spoilt by extremely clever implementations,
and this is what the comment's author was hinting at -- but it takes
a wizard to master those tools. It takes a grand wizard to hack them.

The real feat here is that Emacs offers an environment which is still
amenable to non-wizards. In practice, I think, the biggest optimization
potential still exists in making contributors better than in making
the core implementation faster. In that metric, I'd say, Emacs succeeds
very well [2].

Cheers

[1] This is an example of what I'm talking about:
    https://dl.acm.org/citation.cfm?id=3276489

[2] Here I can just give that reference back to you
    "Evolution of Emacs Lisp" by Stefan Monnier & Michael Sperber
    https://www.iro.umontreal.ca/~monnier/hopl-4-emacs-lisp.pdf

-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Why is Elisp slow?

[Marcin Borkowski]

On 2019-05-02, at 09:56, tomas@tuxteam.de wrote:

>> If a big advance from an Industry player, such as FB, were to release a
>> JIT, that the GNU Emacs legal dept. does not object to, does that help?
>
> I, for one, appreciate Emacs much more for other characteristics than
> its speed.

Agreed, but...

> Its extremely judicious developer community, which manages advances
> at a steady pace (in a 30 year old project) while generally avoiding
> disrupting surprises for its regular users is... amazing.
>
> I've yet to run into a serious limitation posed by its "slow" elisp
> implementation.

...well, Org-mode agenda takes almost 30 seconds to generate for me...

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: Why is Elisp slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 534 bytes --]

On Thu, May 02, 2019 at 01:06:03PM +0200, Marcin Borkowski wrote:
> 
> On 2019-05-02, at 09:56, tomas@tuxteam.de wrote:

[on elisp speed]

> > I've yet to run into a serious limitation posed by its "slow" elisp
> > implementation.
> 
> ...well, Org-mode agenda takes almost 30 seconds to generate for me...

Perhaps this is more a case for fine tuning Org mode's algorithms &
data structures? (I don't use agenda, so I don't really know what
I'm talking about. Perhaps all of this is already done?)

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Why is Elisp slow?

[Eli Zaretskii]

> Date: Thu, 2 May 2019 15:18:27 +0200
> From: tomas@tuxteam.de
> Cc: help-gnu-emacs@gnu.org
> 
> > ...well, Org-mode agenda takes almost 30 seconds to generate for me...
> 
> Perhaps this is more a case for fine tuning Org mode's algorithms &
> data structures? (I don't use agenda, so I don't really know what
> I'm talking about. Perhaps all of this is already done?)

It's certainly a reason to report this to the Org developers, and ask
them to speed this up.

(My personal rant to package authors is that they are way too eager to
implement stuff in Lisp which cannot possibly be fast enough or
scalable enough, instead of urging Emacs to provide new C primitives
and core features, or, better, submitting patches to implement such
features in C.  Two cases in point are linum-mode and fci-mode.  End
of rant.)

Re: Why is Elisp slow?

[Marcin Borkowski]

On 2019-05-02, at 15:18, tomas@tuxteam.de wrote:

> On Thu, May 02, 2019 at 01:06:03PM +0200, Marcin Borkowski wrote:
>> 
>> On 2019-05-02, at 09:56, tomas@tuxteam.de wrote:
>
> [on elisp speed]
>
>> > I've yet to run into a serious limitation posed by its "slow" elisp
>> > implementation.
>> 
>> ...well, Org-mode agenda takes almost 30 seconds to generate for me...
>
> Perhaps this is more a case for fine tuning Org mode's algorithms &
> data structures? (I don't use agenda, so I don't really know what
> I'm talking about. Perhaps all of this is already done?)

I don't think so.  I just happen to have several thousand headlines in
my agenda files.  And AFAIK, agenda generation is already pretty
optimized.  So, this might be one (rare, admittedly) example of such
a limitation.

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: Why is Elisp slow?

[Marcin Borkowski]

On 2019-05-02, at 17:34, Eli Zaretskii <eliz@gnu.org> wrote:

>> Date: Thu, 2 May 2019 15:18:27 +0200
>> From: tomas@tuxteam.de
>> Cc: help-gnu-emacs@gnu.org
>> 
>> > ...well, Org-mode agenda takes almost 30 seconds to generate for me...
>> 
>> Perhaps this is more a case for fine tuning Org mode's algorithms &
>> data structures? (I don't use agenda, so I don't really know what
>> I'm talking about. Perhaps all of this is already done?)
>
> It's certainly a reason to report this to the Org developers, and ask
> them to speed this up.

Well, I don't consider it a bug in Orgmode - rather a bug in my workflow
(and I more or less know how to fix that, only it will take me some
time).

> (My personal rant to package authors is that they are way too eager to
> implement stuff in Lisp which cannot possibly be fast enough or
> scalable enough, instead of urging Emacs to provide new C primitives
> and core features, or, better, submitting patches to implement such
> features in C.  Two cases in point are linum-mode and fci-mode.  End
> of rant.)

That is an interesting POV, although I'm not sure if I agree.  IOW,
I fully understand why people might do so.  I, for one, consider myself
pretty fluent in Elisp and hardly literate in C.

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: Why is Elisp slow?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> (My personal rant to package authors is that they are way too eager to
> implement stuff in Lisp which cannot possibly be fast enough or
> scalable enough, instead of urging Emacs to provide new C primitives
> and core features, or, better, submitting patches to implement such
> features in C.  Two cases in point are linum-mode and fci-mode.  End
> of rant.)

On addition to what Marcin Borkowski says, it is a great convenience to
implement something yourself that you can deliver on your own schedule
instead of depending on something that can take years to be available to
your user base. Apart from the extra work of explaining and justifying
what you need and convincing someone to implement it.

(And then, what is "fast enough for me" can be unbearably slow for
others, etc)

Re: Why is Elisp slow?

[Eli Zaretskii]

> From: Marcin Borkowski <mbork@mbork.pl>
> Cc: help-gnu-emacs@gnu.org
> Date: Thu, 02 May 2019 21:13:28 +0200
> 
> > It's certainly a reason to report this to the Org developers, and ask
> > them to speed this up.
> 
> Well, I don't consider it a bug in Orgmode - rather a bug in my workflow

Maybe improvements in Org could make your bug less prominent...

Then again, it could be that there's some trick up Org developers'
sleeve that they could teach you.

In general, anything surprising/inconvenient/buggy should be reported,
you can never know what will come out of that.

> > (My personal rant to package authors is that they are way too eager to
> > implement stuff in Lisp which cannot possibly be fast enough or
> > scalable enough, instead of urging Emacs to provide new C primitives
> > and core features, or, better, submitting patches to implement such
> > features in C.  Two cases in point are linum-mode and fci-mode.  End
> > of rant.)
> 
> That is an interesting POV, although I'm not sure if I agree.  IOW,
> I fully understand why people might do so.  I, for one, consider myself
> pretty fluent in Elisp and hardly literate in C.

Submitting changes in C is a bonus, it's good enough to describe the
use case and ask for primitives to support it.  Someone else might get
motivated.  It could even be very easy, like with fci-mode.

I'm quite sure that some of the bad reputation of Emacs is due to
incredible things people do in Lisp, which then make Emacs look
sluggish and bloated.

Re: Why is Elisp slow?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 02 May 2019 21:33:55 +0200
> 
> On addition to what Marcin Borkowski says, it is a great convenience to
> implement something yourself that you can deliver on your own schedule
> instead of depending on something that can take years to be available to
> your user base. Apart from the extra work of explaining and justifying
> what you need and convincing someone to implement it.

I have no problem with people who code something for their personal
use.  That's none of my concern.

As for convincing: that is not required.  Just put the request in the
open, and be done.

Re: Why is Elisp slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1163 bytes --]

On Thu, May 02, 2019 at 09:13:28PM +0200, Marcin Borkowski wrote:

[...]

> > (My personal rant to package authors is that they are way too eager to
> > implement stuff in Lisp which cannot possibly be fast enough or
> > scalable enough, instead of urging Emacs to provide new C primitives
> > and core features, or, better, submitting patches to implement such
> > features in C.  Two cases in point are linum-mode and fci-mode.  End
> > of rant.)
> 
> That is an interesting POV, although I'm not sure if I agree.  IOW,
> I fully understand why people might do so.  I, for one, consider myself
> pretty fluent in Elisp and hardly literate in C.

This approach is known as "Ousterhout's dichotomy" [1], after John Ousterhout,
author of Tcl (which was conceived as a very simple scripting language
meant to extend existing applications). Tcl grew later a bytecode
interpreter (as did elisp) -- making the C interface significantly
more complex (but the interpreter significantly faster).

Walking the trade-off between simple and efficient is interesting.

Cheers

[1] https://en.wikipedia.org/wiki/Ousterhout%27s_dichotomy

-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Why is Elisp slow?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Thu, 02 May 2019 21:33:55 +0200
>> 
>> On addition to what Marcin Borkowski says, it is a great convenience to
>> implement something yourself that you can deliver on your own schedule
>> instead of depending on something that can take years to be available to
>> your user base. Apart from the extra work of explaining and justifying
>> what you need and convincing someone to implement it.
>
> I have no problem with people who code something for their personal
> use.  That's none of my concern.

The sceneario I described was about a package maintainer, such as Org.

> As for convincing: that is not required.  Just put the request in the
> open, and be done.

That's not a solution if you are package maintainer trying to solve a
performance problem within a given time frame.

Re: Why is Elisp slow?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 02 May 2019 22:12:50 +0200
> 
> > I have no problem with people who code something for their personal
> > use.  That's none of my concern.
> 
> The sceneario I described was about a package maintainer, such as Org.

That was exactly my gripe.  I'm saying that implementing something for
others should be considered more seriously than just "let's code it
because we can".

> > As for convincing: that is not required.  Just put the request in the
> > open, and be done.
> 
> That's not a solution if you are package maintainer trying to solve a
> performance problem within a given time frame.

Of course it is: with the Lisp implementation already in place, you
don't need to wait for anyone or anything.  But requesting a feature
in addition to that does two things: (a) it alerts others, including
Emacs developers, to the need; and (b) it announces load and clear
that the package maintainer is not really happy with the current
solution.  Without such a request, no one will even know that there's
a problem here waiting for a volunteer.

Re: Why is Elisp slow?

[Ergus]

One reason why people implemented everything in elisp instead of
contributing in C (or sending patches) is:

1) in many documents around it says that it is the recommended way to
extend emacs so new users (like me) think that everything is possible
that way.

2) Because emacs didn't have a C api for modules until very recently and
even with that all the infrastructure is Lisp based. (and it still needs
a LOT of documentation) 

3) It was the only available API with documentation and that didn't
require recompile all emacs and they could redistribute, share and
maintain independently without doing all the complex process to
contribute.

Probably (I have a dream) if someone decides to develop a new emacs in
2019 most of the functions and API will be made in pure and clear C (or
any other compiled language), with a full C api that gives the same
access level than what elisp gives in Emacs today (with C lists, arrays
and structs), so it will be not only faster but also simpler to extend
it with other languages like Scheme, Python, Lua, C++, rust and so
on. (There are modules projects going in that direction) And the editor
don't even need to provide a compiler or interpreter for them. (there
will be Guile/gcc/python and so on for that)

But the emacs modules need a lot of time before they can provide the
same power and access than Elisp gives these days, but most of the
infrastructure actually calls indirectly Elisp functions and create Lisp
objects, so there will be not any performance advantage in that part of
the code until it goes back to C or the Lisp->C or the JIT compilers
will be finished.

Vim, for example, added a C api for that reason, but we are talking
about a very different beast here. They don't have their own interpreter
to maintain, no duality with GUI and TUI to maintain, no Lisp objects,
bytecode, debugger... so a very different beast.  



On Thu, May 02, 2019 at 11:20:54PM +0300, Eli Zaretskii wrote:
>> From: ?scar Fuentes <ofv@wanadoo.es>
>> Date: Thu, 02 May 2019 22:12:50 +0200
>>
>> > I have no problem with people who code something for their personal
>> > use.  That's none of my concern.
>>
>> The sceneario I described was about a package maintainer, such as Org.
>
>That was exactly my gripe.  I'm saying that implementing something for
>others should be considered more seriously than just "let's code it
>because we can".
>
>> > As for convincing: that is not required.  Just put the request in the
>> > open, and be done.
>>
>> That's not a solution if you are package maintainer trying to solve a
>> performance problem within a given time frame.
>
>Of course it is: with the Lisp implementation already in place, you
>don't need to wait for anyone or anything.  But requesting a feature
>in addition to that does two things: (a) it alerts others, including
>Emacs developers, to the need; and (b) it announces load and clear
>that the package maintainer is not really happy with the current
>solution.  Without such a request, no one will even know that there's
>a problem here waiting for a volunteer.
>

Re: Why is Elisp slow?

[조성빈]

> Probably (I have a dream) if someone decides to develop a new emacs in
> 2019 most of the functions and API will be made in pure and clear C (or
> any other compiled language), with a full C api that gives the same
> access level than what elisp gives in Emacs today (with C lists, arrays
> and structs), so it will be not only faster but also simpler to extend
> it with other languages like Scheme, Python, Lua, C++, rust and so
> on. (There are modules projects going in that direction) And the editor
> don't even need to provide a compiler or interpreter for them. (there
> will be Guile/gcc/python and so on for that)

This, very much sounds like Guile Emacs. Anyone knows how Guile Emacs is doing?̊̈ It’s very much looks like vaporware these days :-(
Is upstream considering Guile Emacs as a valid solution? 
Is there any development ongoing? (Official or not?)

Re: Why is Elisp slow?

[Ergus]

WARNING: This email has plenty of personal opinions.

On Fri, May 03, 2019 at 08:39:43AM +0900, ????????? wrote:
>
>> Probably (I have a dream) if someone decides to develop a new emacs in
>> 2019 most of the functions and API will be made in pure and clear C (or
>> any other compiled language), with a full C api that gives the same
>> access level than what elisp gives in Emacs today (with C lists, arrays
>> and structs), so it will be not only faster but also simpler to extend
>> it with other languages like Scheme, Python, Lua, C++, rust and so
>> on. (There are modules projects going in that direction) And the editor
>> don't even need to provide a compiler or interpreter for them. (there
>> will be Guile/gcc/python and so on for that)
>
>This, very much sounds like Guile Emacs. Anyone knows how Guile Emacs
>is doing????? It???s very much looks like vaporware these days :-(

Yes, that is the dream (at least the closest we have ever been)

>Is upstream considering Guile Emacs as a valid solution?

I made a similar question some time ago and the answer is that there is
not action. :( In fact there is not too much action in
Guile's development .

Not many projects feel interested in Guile because the weak support it
has so there are few users and few developers (But also because
Lisp-like family languages (common Lisp, Scheme and the others) are a
museum piece for young generations that are more used to Python, Ruby,
javascript, and most of OOP (also because there are more jobs with
that))

Actually I think that these days will be easier to find new C/C++
developers for emacs than Lisp developers. Lisp and Scheme are
beautiful, but they require a different way of thinking and a lot of
time (own experience, I am just starting with it.) One reason why I
can't convince my friends to use Emacs is actually how Lisp scares them
((())()'()).

>Is there any development ongoing? (Official or not?)
>
>
I don't think so :( :( :(. It will require that some core emacs
developers feel more interested in implementing this, because it is a
lot of work... (specially to do it right and keeping the so sacred
backward compatibility) These days only few people know the core parts
of emacs for such a task.

Having Guile as a dependency will grow emacs size significantly
and using it as an external dependency (not usually the emacs way...)
will require to port Guile to more systems (AFAIK). BUt such desition
could benefit very much both projects.

There are even some emacs ports to rust (remacs) but they still have the
elisp as the core languaje. Because most of the code and functionalities
are already in elisp so changing that will be like reimplementing all
emacs from scratch.

(Some time ago there was a lot of effort and time invested in porting C
functions to Elisp in emacs)

So with our actual emacs maybe the JIT or the Lisp->C source to source
is actually the more reliable option in a possible (realistic) future.

Personally I feel the Lisp->C compiler feels like the faster and more
robust solution for me, but it will create a dependency with
binutils... which is for multiple reasons undesired.

The alternative JIT may be based in libJIT which is not very active
either... and has serious issues/limitations that has not been solved in
years.

[1] https://tromey.com/blog/?p=982
[2] https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00393.html

Re: Why is Elisp slow?

[조성빈]

I became curious after reading your email :-)

Since implementing a new language/byte interpreter (Guile, for example) is hard, and making it fast is harder, why doesn't Emacs leverage the Common Lisp environment?̊̈

For example, (since CL and elisp’s syntax is mostly similar) can’t we implement Emacs based on CL (looks like there were ongoing efforts before like Climacs, Hemlock, etc…) and expose elisp functions in a special package named ‘elisp’?̊̈ It would allow elisp compatibility (by running all elisp code in the special elisp package) while giving package writers access to CL libraries. Since legacy cruft can go inside the package ‘elisp’, CL emacs APIs can remove/redesign some old APIs and remove technical debt. This would also greatly accerlate Emacs speed as CL is super fast especially when using with SBCL.

Are there any reasons (that I do not know) why CL was not considered or was considered but rejected as an elisp alternative? 
Are there license problems to build on previous attempts?̊̈ 

IMHO building emacs on CL would be a wonderful idea... and would fix many problems that current Emacs have.

> 2019. 5. 3. 오전 9:44, Ergus <spacibba@aol.com> 작성:
> 
> WARNING: This email has plenty of personal opinions.
> 
>> On Fri, May 03, 2019 at 08:39:43AM +0900, ????????? wrote:
>> 
>>> Probably (I have a dream) if someone decides to develop a new emacs in
>>> 2019 most of the functions and API will be made in pure and clear C (or
>>> any other compiled language), with a full C api that gives the same
>>> access level than what elisp gives in Emacs today (with C lists, arrays
>>> and structs), so it will be not only faster but also simpler to extend
>>> it with other languages like Scheme, Python, Lua, C++, rust and so
>>> on. (There are modules projects going in that direction) And the editor
>>> don't even need to provide a compiler or interpreter for them. (there
>>> will be Guile/gcc/python and so on for that)
>> 
>> This, very much sounds like Guile Emacs. Anyone knows how Guile Emacs
>> is doing????? It???s very much looks like vaporware these days :-(
> 
> Yes, that is the dream (at least the closest we have ever been)
> 
>> Is upstream considering Guile Emacs as a valid solution?
> 
> I made a similar question some time ago and the answer is that there is
> not action. :( In fact there is not too much action in
> Guile's development .
> 
> Not many projects feel interested in Guile because the weak support it
> has so there are few users and few developers (But also because
> Lisp-like family languages (common Lisp, Scheme and the others) are a
> museum piece for young generations that are more used to Python, Ruby,
> javascript, and most of OOP (also because there are more jobs with
> that))
> 
> Actually I think that these days will be easier to find new C/C++
> developers for emacs than Lisp developers. Lisp and Scheme are
> beautiful, but they require a different way of thinking and a lot of
> time (own experience, I am just starting with it.) One reason why I
> can't convince my friends to use Emacs is actually how Lisp scares them
> ((())()'()).
> 
>> Is there any development ongoing? (Official or not?)
> I don't think so :( :( :(. It will require that some core emacs
> developers feel more interested in implementing this, because it is a
> lot of work... (specially to do it right and keeping the so sacred
> backward compatibility) These days only few people know the core parts
> of emacs for such a task.
> 
> Having Guile as a dependency will grow emacs size significantly
> and using it as an external dependency (not usually the emacs way...)
> will require to port Guile to more systems (AFAIK). BUt such desition
> could benefit very much both projects.
> 
> There are even some emacs ports to rust (remacs) but they still have the
> elisp as the core languaje. Because most of the code and functionalities
> are already in elisp so changing that will be like reimplementing all
> emacs from scratch.
> 
> (Some time ago there was a lot of effort and time invested in porting C
> functions to Elisp in emacs)
> 
> So with our actual emacs maybe the JIT or the Lisp->C source to source
> is actually the more reliable option in a possible (realistic) future.
> 
> Personally I feel the Lisp->C compiler feels like the faster and more
> robust solution for me, but it will create a dependency with
> binutils... which is for multiple reasons undesired.
> 
> The alternative JIT may be based in libJIT which is not very active
> either... and has serious issues/limitations that has not been solved in
> years.
> 
> [1] https://tromey.com/blog/?p=982
> [2] https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00393.html

Re: Why is Elisp slow?

[Paul W. Rankin]

On Fri, May 03 2019, Ergus wrote:
> Actually I think that these days will be easier to find new C/C++
> developers for emacs than Lisp developers. Lisp and Scheme are
> beautiful, but they require a different way of thinking and a lot of
> time (own experience, I am just starting with it.) One reason why I
> can't convince my friends to use Emacs is actually how Lisp scares 
> them
> ((())()'()).

Emacs Lisp was the fist programming language I learnt (really as a means 
to an end to make a major mode I wanted). I found Elisp elegant and easy 
to understand; the parens make it clear where each expression begins and 
ends, functions are always the first element, and the documentation is 
incredibly accessible for a noob/hobbyist programmer. I find other 
languages difficult/inaccessible by comparison.

-- 
https://www.paulwrankin.com

Re: Why is Elisp slow?

[Stefan Monnier]

> One reason why people implemented everything in elisp instead of
> contributing in C (or sending patches) is:

Eli's point is quite different (and actually goes well beyond adding
new C primitives): there is a world of Elisp hackers developing great
packages but never engaging with Emacs development at all, in the sense
that they take Emacs as an immutable base whose limitations and bugs are
just obstacles around which to work.

I regularly bump into code "out there" with random hacks to work around
Emacs bugs I've never heard of.

All Eli is saying is: when the Emacs infrastructure isn't as good as
you'd like for your package, please report it as a bug (no need to do
anything more than that).  That doesn't mean we'll necessarily fix those
bugs (sometimes they're hard to fix, or simply nobody is interested in
fixing them), but it helps to know about them, and can guide
future redesigns.


        Stefan

Re: Why is Elisp slow?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Thu, 02 May 2019 22:39:46 -0400
> 
> > One reason why people implemented everything in elisp instead of
> > contributing in C (or sending patches) is:
> 
> Eli's point is quite different (and actually goes well beyond adding
> new C primitives): there is a world of Elisp hackers developing great
> packages but never engaging with Emacs development at all, in the sense
> that they take Emacs as an immutable base whose limitations and bugs are
> just obstacles around which to work.

Well said, thanks.

> I regularly bump into code "out there" with random hacks to work around
> Emacs bugs I've never heard of.

Not just bugs: also limitations and missing features.

> All Eli is saying is: when the Emacs infrastructure isn't as good as
> you'd like for your package, please report it as a bug (no need to do
> anything more than that).  That doesn't mean we'll necessarily fix those
> bugs (sometimes they're hard to fix, or simply nobody is interested in
> fixing them), but it helps to know about them, and can guide
> future redesigns.

Right.

Re: Why is Elisp slow?

[Eli Zaretskii]

> From: 조성빈 <pcr910303@icloud.com>
> Date: Fri, 3 May 2019 08:39:43 +0900
> Cc: Eli Zaretskii <eliz@gnu.org>, help-gnu-emacs@gnu.org
> 
> Is upstream considering Guile Emacs as a valid solution? 

IMO, it is not in a state that it needs ti be considered seriously,
not yet.

Re: Why is Elisp slow?

[Eli Zaretskii]

> Date: Fri, 3 May 2019 02:44:16 +0200
> From: Ergus <spacibba@aol.com>
> Cc: help-gnu-emacs@gnu.org
> 
> Having Guile as a dependency will grow emacs size significantly
> and using it as an external dependency (not usually the emacs way...)
> will require to port Guile to more systems (AFAIK).

Which systems we care about are not already supported by Guile?

> The alternative JIT may be based in libJIT which is not very active
> either... and has serious issues/limitations that has not been solved in
> years.

There's a branch in the Emacs repository that uses libJIT.  It has
problems with some Emacs configurations, which need to be solved in
libJIT.  Alas, those problems, reported months ago to the libJIT
developers, are still not fixed.

More importantly, the libJIT build failed to show any significant
speed-up wrt byte code, so it sounds like maybe the whole idea was
either wrong or its design couldn't possibly provide any gains.  Or
maybe we just measured the speed-up in wrong scenarios.

Re: Why is Elisp slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1717 bytes --]

On Fri, May 03, 2019 at 02:44:16AM +0200, Ergus wrote:
> WARNING: This email has plenty of personal opinions.
> 
> On Fri, May 03, 2019 at 08:39:43AM +0900, ????????? wrote:
> >
> >>Probably (I have a dream [...]

> >This, very much sounds like Guile Emacs. Anyone knows how Guile Emacs
> >is doing????? It???s very much looks like vaporware these days :-(
> 
> Yes, that is the dream (at least the closest we have ever been)
> 
> >Is upstream considering Guile Emacs as a valid solution?
> 
> I made a similar question some time ago and the answer is that there is
> not action. :( In fact there is not too much action in
> Guile's development .
> 
> Not many projects feel interested in Guile [...]

C'm on, folks. Dreaming is OK, but once you wake up, you can
try to exercise your search engine skills. Guile is well and
alive (the Guilers meet every year at FOSDEM, talk videos seem
to be available), Guile's maintainer (Andy Wingo) is doing an
excellent job, and one of the more active Guile projects (Guix)
has just celebrated its 1.0 release.

The official web site is a good starting point:

  https://www.gnu.org/software/guile/

Ah, and Andy Wingo's blog is also recommended reading:

  https://wingolog.org

As to guile-emacs, there are two things here: first, the project
needs hands, it seems. I see four hands out there ;-) Second,
not everyone here likes Scheme (and they might be right), so it
would need some convincing (be sure to read the Emacs history by
Stefan Monnier and Michael Sperber, posted elsewhere in this list).

A good starting point on guile-emacs would be

  https://www.emacswiki.org/emacs/GuileEmacs

So -- lots to do!

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Why is Elisp slow?

[Ergus]

On Fri, May 03, 2019 at 10:00:45AM +0300, Eli Zaretskii wrote:
>> Date: Fri, 3 May 2019 02:44:16 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: help-gnu-emacs@gnu.org
>>
>> Having Guile as a dependency will grow emacs size significantly
>> and using it as an external dependency (not usually the emacs way...)
>> will require to port Guile to more systems (AFAIK).
>
>Which systems we care about are not already supported by Guile?
>

Actually in windows I have a friend who tried it and the experience was
not convincing for him. And the native code compiler was in fact
tremendously slow in a power9 machine. It felt like a hack in both
cases more than a real native application.

>> The alternative JIT may be based in libJIT which is not very active
>> either... and has serious issues/limitations that has not been solved in
>> years.
>
>There's a branch in the Emacs repository that uses libJIT.  It has
>problems with some Emacs configurations, which need to be solved in
>libJIT.  Alas, those problems, reported months ago to the libJIT
>developers, are still not fixed.
>
>More importantly, the libJIT build failed to show any significant
>speed-up wrt byte code, so it sounds like maybe the whole idea was
>either wrong or its design couldn't possibly provide any gains.  Or
>maybe we just measured the speed-up in wrong scenarios.
>
This is not surprising. My work is 80% performance measurement and
benchmarking and the real improvements with JIT compilation (in my
experience) and specially with libJIT is not as good as many people
expect in most of the common scenarios. That's because the generated
code is usually very generic (so it does not take advantage of
architecture specific features), and strategies like vectorization and
branch prediction are very difficult to hint (most of the times
impossible). So the only real difference with a bytecode interpreter is
the bytecode parsing part, but not too much more.

There is also the typical array of struct vs struct of arrays problem, type
aligns and the optimal memory management what are also very dark to
advice for a JIT compiler. The use of dark types and pointers to
functions produces huge overheads and they are extensively used in JIT
compilers internally generated code, the impossibility to do proper
inlinings, and of course the fact that coming from a higher level
language like Lisp we need more dynamic/runtime functionalities like
garbage collector and generic function types. Plus the hashtable search
costs and the extensively used list searches in Lisp instead of vectors
or similar cheaper containers and callbacks in the user code.

That's why I support the idea of the Lisp->C compiler more than the JIT
at least to use it for the emacs core Lisp functions because gcc is very
optimized and migrate the bottleneck back to C code as much as
possible. (many of the very used in simple.el for example that are
called very frequently)

That's the strategy that made python become so successfully (plus the
language simplicity). The existence of Cython, Numba + Pure C/Fortran
core libraries like numpy and a simple C API to extend it. So when a
user implements anything in 100% python; if it has performance issues
she only needs to change the python lists with numpy arrays and numpy
functions and that's it, the difference will be observed
immediately. But if a specific function still represents a problem
she can implement it easily in C or use a C library directly with
CTypes, Pybind11 or boost, that uses dlopen instead of create processes
and parse outputs and writes in numpy arrays/types directly.

So in spite of that python's virtual machine is very slow to load, the
rest of the execution is not necessarily slow and can be improved as
much as the developer desires. Also as there are more users the bytecode
part (interpreter and generator) is improved and optimized constantly.

Re: Why is Elisp slow?

[Ergus]

On Fri, May 03, 2019 at 10:06:56AM +0900, ????????? wrote:
>I became curious after reading your email :-)
>
>Since implementing a new language/byte interpreter (Guile, for example) is hard, and making it fast is harder, why doesn't Emacs leverage the Common Lisp environment?????
>
It is not a language issue I think, but a compiler/interpreter
optimization level.

>For example, (since CL and elisp???s syntax is mostly similar) can???t we implement Emacs based on CL (looks like there were ongoing efforts before like Climacs, Hemlock, etc???) and expose elisp functions in a special package named ???elisp???????? It would allow elisp compatibility (by running all elisp code in the special elisp package) while giving package writers access to CL libraries. Since legacy cruft can go inside the package ???elisp???, CL emacs APIs can remove/redesign some old APIs and remove technical debt. This would also greatly accerlate Emacs speed as CL is super fast especially when using with SBCL.
>
That was the emacs-guile approach. The problem was that nobody finished
it. And if the whole project (means the core and more important
developers) do not decide to go in that direction for a good reason
(like they want to evict the bytecode interpreter from within emacs or
they consider important to use a specific Guile functionality) then it
won't happen. Also, doing big changes like that in emacs core code
becomes a big discussion that takes loooooong time in the mailing list. 

>Are there any reasons (that I do not know) why CL was not considered or was considered but rejected as an elisp alternative?
>Are there license problems to build on previous attempts?????
>
maybe it has to do with this:

https://www.gnu.org/gnu/rms-lisp.en.html

>IMHO building emacs on CL would be a wonderful idea... and would fix many problems that current Emacs have.
>
And create others, it is always a trade of.

Any way

I have to say that comparing to other interpreters around the Elisp is
not the slowest one I have tried by far. Of course it could be way
faster, but some optimization like user code and provide new containers
and datatypes in the library level based in C (and recomend them) code
could potentially make more difference in real user extensions.

>> 2019. 5. 3. ?????? 9:44, Ergus <spacibba@aol.com> ??????:
>>
>> WARNING: This email has plenty of personal opinions.
>>
>>> On Fri, May 03, 2019 at 08:39:43AM +0900, ????????? wrote:
>>>
>>>> Probably (I have a dream) if someone decides to develop a new emacs in
>>>> 2019 most of the functions and API will be made in pure and clear C (or
>>>> any other compiled language), with a full C api that gives the same
>>>> access level than what elisp gives in Emacs today (with C lists, arrays
>>>> and structs), so it will be not only faster but also simpler to extend
>>>> it with other languages like Scheme, Python, Lua, C++, rust and so
>>>> on. (There are modules projects going in that direction) And the editor
>>>> don't even need to provide a compiler or interpreter for them. (there
>>>> will be Guile/gcc/python and so on for that)
>>>
>>> This, very much sounds like Guile Emacs. Anyone knows how Guile Emacs
>>> is doing????? It???s very much looks like vaporware these days :-(
>>
>> Yes, that is the dream (at least the closest we have ever been)
>>
>>> Is upstream considering Guile Emacs as a valid solution?
>>
>> I made a similar question some time ago and the answer is that there is
>> not action. :( In fact there is not too much action in
>> Guile's development .
>>
>> Not many projects feel interested in Guile because the weak support it
>> has so there are few users and few developers (But also because
>> Lisp-like family languages (common Lisp, Scheme and the others) are a
>> museum piece for young generations that are more used to Python, Ruby,
>> javascript, and most of OOP (also because there are more jobs with
>> that))
>>
>> Actually I think that these days will be easier to find new C/C++
>> developers for emacs than Lisp developers. Lisp and Scheme are
>> beautiful, but they require a different way of thinking and a lot of
>> time (own experience, I am just starting with it.) One reason why I
>> can't convince my friends to use Emacs is actually how Lisp scares them
>> ((())()'()).
>>
>>> Is there any development ongoing? (Official or not?)
>> I don't think so :( :( :(. It will require that some core emacs
>> developers feel more interested in implementing this, because it is a
>> lot of work... (specially to do it right and keeping the so sacred
>> backward compatibility) These days only few people know the core parts
>> of emacs for such a task.
>>
>> Having Guile as a dependency will grow emacs size significantly
>> and using it as an external dependency (not usually the emacs way...)
>> will require to port Guile to more systems (AFAIK). BUt such desition
>> could benefit very much both projects.
>>
>> There are even some emacs ports to rust (remacs) but they still have the
>> elisp as the core languaje. Because most of the code and functionalities
>> are already in elisp so changing that will be like reimplementing all
>> emacs from scratch.
>>
>> (Some time ago there was a lot of effort and time invested in porting C
>> functions to Elisp in emacs)
>>
>> So with our actual emacs maybe the JIT or the Lisp->C source to source
>> is actually the more reliable option in a possible (realistic) future.
>>
>> Personally I feel the Lisp->C compiler feels like the faster and more
>> robust solution for me, but it will create a dependency with
>> binutils... which is for multiple reasons undesired.
>>
>> The alternative JIT may be based in libJIT which is not very active
>> either... and has serious issues/limitations that has not been solved in
>> years.
>>
>> [1] https://tromey.com/blog/?p=982
>> [2] https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00393.html
>

Re: Why is Elisp slow?

[조성빈]

>> For example, (since CL and elisp???s syntax is mostly similar) can???t we implement Emacs based on CL (looks like there were ongoing efforts before like Climacs, Hemlock, etc???) and expose elisp functions in a special package named ???elisp???????? It would allow elisp compatibility (by running all elisp code in the special elisp package) while giving package writers access to CL libraries. Since legacy cruft can go inside the package ???elisp???, CL emacs APIs can remove/redesign some old APIs and remove technical debt. This would also greatly accerlate Emacs speed as CL is super fast especially when using with SBCL.
>> 
> That was the emacs-guile approach. The problem was that nobody finished
> it. And if the whole project (means the core and more important
> developers) do not decide to go in that direction for a good reason
> (like they want to evict the bytecode interpreter from within emacs or
> they consider important to use a specific Guile functionality) then it
> won't happen. Also, doing big changes like that in emacs core code
> becomes a big discussion that takes loooooong time in the mailing list. 

The difference between guile and CL is that
* Guile is arguably not popular and not used by many programs.I’ve never heard a program that is popular and uses guile as an extension language. CL is, well, much popular than Guile, and is used by industry-strength programs.
* Guile (which is a scheme) has a radically different syntax with elisp (which means that to implement elisp in guile, one has to hack the byte code interpreter AFAIK). Elisp can be implemented inside CL with full compatibility, which mitigates lots of problems between interfacing between CL code and elisp code.
* I’m pretty sure CL will be much, much faster than guile. CL was optimized for about 30 years or so, and SBCL compiles CL all the way to native code, which is comparable to C. I’m not sure if guile can do that.

>> Are there any reasons (that I do not know) why CL was not considered or was considered but rejected as an elisp alternative?
>> Are there license problems to build on previous attempts?????
>> 
> maybe it has to do with this:
> 
> https://www.gnu.org/gnu/rms-lisp.en.html <https://www.gnu.org/gnu/rms-lisp.en.html>

I read the link; it’s a pity to not consider CL because it isn’t ‘lispy’ enough. 

> 
>> IMHO building emacs on CL would be a wonderful idea... and would fix many problems that current Emacs have.
>> 
> And create others, it is always a trade off.

What problems would CL based Emacs can make?̊̈ Can you show some examples?̊̈ I am genuinely interested.

> 
> Any way
> 
> I have to say that comparing to other interpreters around the Elisp is
> not the slowest one I have tried by far. Of course it could be way
> faster, but some optimization like user code and provide new containers
> and datatypes in the library level based in C (and recomend them) code
> could potentially make more difference in real user extensions.

Well, CL is as fast as C…. 

> 
>>> 2019. 5. 3. ?????? 9:44, Ergus <spacibba@aol.com> ??????:
>>> 
>>> WARNING: This email has plenty of personal opinions.
>>> 
>>>> On Fri, May 03, 2019 at 08:39:43AM +0900, ????????? wrote:
>>>> 
>>>>> Probably (I have a dream) if someone decides to develop a new emacs in
>>>>> 2019 most of the functions and API will be made in pure and clear C (or
>>>>> any other compiled language), with a full C api that gives the same
>>>>> access level than what elisp gives in Emacs today (with C lists, arrays
>>>>> and structs), so it will be not only faster but also simpler to extend
>>>>> it with other languages like Scheme, Python, Lua, C++, rust and so
>>>>> on. (There are modules projects going in that direction) And the editor
>>>>> don't even need to provide a compiler or interpreter for them. (there
>>>>> will be Guile/gcc/python and so on for that)
>>>> 
>>>> This, very much sounds like Guile Emacs. Anyone knows how Guile Emacs
>>>> is doing????? It???s very much looks like vaporware these days :-(
>>> 
>>> Yes, that is the dream (at least the closest we have ever been)
>>> 
>>>> Is upstream considering Guile Emacs as a valid solution?
>>> 
>>> I made a similar question some time ago and the answer is that there is
>>> not action. :( In fact there is not too much action in
>>> Guile's development .
>>> 
>>> Not many projects feel interested in Guile because the weak support it
>>> has so there are few users and few developers (But also because
>>> Lisp-like family languages (common Lisp, Scheme and the others) are a
>>> museum piece for young generations that are more used to Python, Ruby,
>>> javascript, and most of OOP (also because there are more jobs with
>>> that))
>>> 
>>> Actually I think that these days will be easier to find new C/C++
>>> developers for emacs than Lisp developers. Lisp and Scheme are
>>> beautiful, but they require a different way of thinking and a lot of
>>> time (own experience, I am just starting with it.) One reason why I
>>> can't convince my friends to use Emacs is actually how Lisp scares them
>>> ((())()'()).
>>> 
>>>> Is there any development ongoing? (Official or not?)
>>> I don't think so :( :( :(. It will require that some core emacs
>>> developers feel more interested in implementing this, because it is a
>>> lot of work... (specially to do it right and keeping the so sacred
>>> backward compatibility) These days only few people know the core parts
>>> of emacs for such a task.
>>> 
>>> Having Guile as a dependency will grow emacs size significantly
>>> and using it as an external dependency (not usually the emacs way...)
>>> will require to port Guile to more systems (AFAIK). BUt such desition
>>> could benefit very much both projects.
>>> 
>>> There are even some emacs ports to rust (remacs) but they still have the
>>> elisp as the core languaje. Because most of the code and functionalities
>>> are already in elisp so changing that will be like reimplementing all
>>> emacs from scratch.
>>> 
>>> (Some time ago there was a lot of effort and time invested in porting C
>>> functions to Elisp in emacs)
>>> 
>>> So with our actual emacs maybe the JIT or the Lisp->C source to source
>>> is actually the more reliable option in a possible (realistic) future.
>>> 
>>> Personally I feel the Lisp->C compiler feels like the faster and more
>>> robust solution for me, but it will create a dependency with
>>> binutils... which is for multiple reasons undesired.
>>> 
>>> The alternative JIT may be based in libJIT which is not very active
>>> either... and has serious issues/limitations that has not been solved in
>>> years.
>>> 
>>> [1] https://tromey.com/blog/?p=982
>>> [2] https://lists.gnu.org/archive/html/emacs-devel/2018-08/msg00393.html
>> 
> 

Re: Why is Elisp slow?

[Eli Zaretskii]

> Date: Fri, 3 May 2019 11:58:49 +0200
> From: Ergus <spacibba@aol.com>
> Cc: help-gnu-emacs@gnu.org
> 
> >Which systems we care about are not already supported by Guile?
> >
> 
> Actually in windows I have a friend who tried it and the experience was
> not convincing for him.

I don't know what that means.  Guile 2.0.x compiles on Windows, and
(32-bit) binaries are available from the ezwinports site.  Those
binaries are good enough to build GDB with Guile support on Windows.

It's possible that later versions of Guile "need work" to be able to
compile and work well on Windows, but the amount of problems cannot be
too large, given all the fixes that went into 2.0.x.

Re: Why is Elisp slow?

[Eli Zaretskii]

> From: 조성빈 <pcr910303@icloud.com>
> Date: Fri, 3 May 2019 20:52:06 +0900
> Cc: help-gnu-emacs@gnu.org
> 
> * Guile (which is a scheme) has a radically different syntax with elisp (which means that to implement elisp in guile, one has to hack the byte code interpreter AFAIK). Elisp can be implemented inside CL with full compatibility, which mitigates lots of problems between interfacing between CL code and elisp code.

This is not a problem with Guile, because Guile includes a compiler
and interpreter for Emacs Lisp.

Re: Why is Elisp slow?

[Ergus]

On Fri, May 03, 2019 at 08:52:06PM +0900, ????????? wrote:
>
>
>The difference between guile and CL is that * Guile is arguably not
>popular and not used by many programs.I???ve never heard a program that
>is popular and uses guile as an extension language. CL is, well, much
>popular than Guile, and is used by industry-strength programs.

> * Guile (which is a scheme) has a radically different syntax with
>elisp (which means that to implement elisp in guile, one has to hack
>the byte code interpreter AFAIK). Elisp can be implemented inside CL
>with full compatibility, which mitigates lots of problems between
>interfacing between CL code and elisp code.

But guile is in fact developed in the gnu project, so there are the
guile developers (very few but they are) that could potentially
provide and fix any issue. But also the strong part of guile is not the
language itself but the potential of the infrastructure.

>* I???m pretty sure CL will be much, much faster than guile.

Probably, that depends of the compiler not only the language.

>CL was optimized for about 30 years or so, and SBCL compiles CL all the way to native code, which is comparable to C. I???m not sure if guile can do that.
>
Guile has been also optimized (in spite of there is a lot of work to do,
that's true) and yes, it can generate native code, but also
bytecode. But it also offers a lot of functionalities to extend and work
as a glue language which is actually most important in our days.

At the end (in my opinion) if emacs wants to survive other 40 years it
will need to start looking for integration with python, lua, C++, rust,
Ruby and similar languages with a more "promising" future than common
lisp. Because we really need to attract new developers with new ideas,
visions and experiences... and 99% of programmers don't use any lisp
like language.

To make big changes we need more developers, that will not come if we
don't have more users first. And those new users/developers can enforce
the new changes looking at the future and not to backward compatibility.


>> https://www.gnu.org/gnu/rms-lisp.en.html <https://www.gnu.org/gnu/rms-lisp.en.html>
>
>I read the link; it???s a pity to not consider CL because it isn???t ???lispy??? enough.
>
>>
>>> IMHO building emacs on CL would be a wonderful idea... and would fix many problems that current Emacs have.
>>>
>> And create others, it is always a trade off.
>
>What problems would CL based Emacs can make????? Can you show some examples????? I am genuinely interested.
>

Dynamic scoping rules vs exical scoping rules in common lisp is just the
first one I can remember, but maybe someone else will reply you with more details.

>>
>> Any way
>>
>> I have to say that comparing to other interpreters around the Elisp is
>> not the slowest one I have tried by far. Of course it could be way
>> faster, but some optimization like user code and provide new containers
>> and datatypes in the library level based in C (and recomend them) code
>> could potentially make more difference in real user extensions.
>
>Well, CL is as fast as C???.
>

But again, common lisp is just a language. Performance is more associated
with the compiler you use. Cython makes python very fast too for
example AND IT IS PYTHON!!!.  The latest version of the GNU common lisp
compiler (GCL) was in 2014... so its developement goes slower than
Guile.

If we had a native compiler for Elisp it could be as fast as common lisp
(ideally). But in any case it needs to be something that goes embedded
within emacs (otherwise emacs won't be emacs anymore).

Think also in the licenses issues and legal consequences of such strong
dependency for a key part of the editor.

On the other hand it is true that in that case that will be a part of
the code that we won't need to maintain if we rely on other packages.

Let me say something. I am not opposed at all to switch to common lisp,
(or scheme or python, actually I think that at some point it will be
needed to change the core language for something more flexible)
what I mean is that this specific problem is not a language issue, but the
compiler. And I don't think that we can just copy the SBCL code into
emacs for many reasons.

Re: Why is Elisp slow?

[Ergus]

On Fri, May 03, 2019 at 03:44:17PM +0300, Eli Zaretskii wrote:
>> From: ????????? <pcr910303@icloud.com>
>> Date: Fri, 3 May 2019 20:52:06 +0900
>> Cc: help-gnu-emacs@gnu.org
>>
>> * Guile (which is a scheme) has a radically different syntax with elisp (which means that to implement elisp in guile, one has to hack the byte code interpreter AFAIK). Elisp can be implemented inside CL with full compatibility, which mitigates lots of problems between interfacing between CL code and elisp code.
>
>This is not a problem with Guile, because Guile includes a compiler
>and interpreter for Emacs Lisp.
>
Hi Eli, now I am the curious.

If that's already done (which sems to be the harder part), where is the
real problem to migrate to guile? I know there should be many, but what
are the known ones??

Re: Why is Elisp slow?

[조성빈]

Looks like the point of getting Guile Emacs recurs to:

* How much Guile Emacs is polished(seamless integration with Elisp, full compatibility, etc..)
* How much Guile supports other languages such as JS or Lua
* How much Guile manages to be fast

Guile Emacs was developed for at least 15 years or more, right?

How much can Guile Emacs do these in current situation?̊̈
Is Guile Emacs usable day-to-day?


> 2019. 5. 3. 오후 9:51, Ergus <spacibba@aol.com> 작성:
> 
> On Fri, May 03, 2019 at 08:52:06PM +0900, 조성빈 wrote:
> 
>> The difference between Guile and CL is that
>> * Guile is arguably not popular and not used by many programs. I've never heard a program that is popular and uses Guile as an extension language. CL is, well, much more popular than Guile, and is used by industry-strength programs.
> 
>> * Guile (which is a scheme) has a radically different syntax with
>> elisp (which means that to implement elisp in Guile, one has to hack
>> the byte code interpreter AFAIK). Elisp can be implemented inside CL
>> with full compatibility, which mitigates lots of problems between
>> interfacing between CL code and elisp code.
> 
> But Guile is in fact developed in the gnu project, so there are the
> Guile developers (very few but they are) that could potentially
> provide and fix any issue. But also the strong part of Guile is not the
> language itself but the potential of the infrastructure.
> 
>> * I'm pretty sure CL will be much, much faster than Guile.
> 
> Probably, that depends of the compiler not only the language.
> 
>> CL was optimized for about 30 years or so, and SBCL compiles CL all the way to native code, which is comparable to C. I'm not sure if Guile can do that.
> 
> Guile has been also optimized (in spite of there is a lot of work to do,
> that's true) and yes, it can generate native code, but also
> bytecode. But it also offers a lot of functionalities to extend and work
> as a glue language which is actually most important in our days.
> 
> At the end (in my opinion) if emacs wants to survive other 40 years it
> will need to start looking for integration with python, lua, C++, rust,
> Ruby and similar languages with a more "promising" future than common
> lisp. Because we really need to attract new developers with new ideas,
> visions and experiences... and 99% of programmers don't use any lisp
> like language.
> 
> To make big changes we need more developers, that will not come if we
> don't have more users first. And those new users/developers can enforce
> the new changes looking at the future and not to backward compatibility.
> 
> 
>>> https://www.gnu.org/gnu/rms-lisp.en.html <https://www.gnu.org/gnu/rms-lisp.en.html>
>> 
>> I read the link; it's a pity to not consider CL because it isn't "lispy" enough.
>> 
>>> 
>>>> IMHO building emacs on CL would be a wonderful idea... and would fix many problems that current Emacs have.
>>>> 
>>> And create others, it is always a trade off.
>> 
>> What problems would CL based Emacs can make? Can you show some examples? I am genuinely interested.
>> 
> 
> Dynamic scoping rules vs exical scoping rules in common lisp is just the
> first one I can remember, but maybe someone else will reply you with more details.
> 
>>> 
>>> Any way
>>> 
>>> I have to say that comparing to other interpreters around the Elisp is
>>> not the slowest one I have tried by far. Of course it could be way
>>> faster, but some optimization like user code and provide new containers
>>> and datatypes in the library level based in C (and recomend them) code
>>> could potentially make more difference in real user extensions.
>> 
>> Well, CL is as fast as C.
> 
> But again, common lisp is just a language. Performance is more associated
> with the compiler you use. Cython makes python very fast too for
> example AND IT IS PYTHON!!!.  The latest version of the GNU common lisp
> compiler (GCL) was in 2014... so its developement goes slower than
> Guile.
> 
> If we had a native compiler for Elisp it could be as fast as common lisp
> (ideally). But in any case it needs to be something that goes embedded
> within emacs (otherwise emacs won't be emacs anymore).
> 
> Think also in the licenses issues and legal consequences of such strong
> dependency for a key part of the editor.
> 
> On the other hand it is true that in that case that will be a part of
> the code that we won't need to maintain if we rely on other packages.
> 
> Let me say something. I am not opposed at all to switch to common lisp,
> (or scheme or python, actually I think that at some point it will be
> needed to change the core language for something more flexible)
> what I mean is that this specific problem is not a language issue, but the
> compiler. And I don't think that we can just copy the SBCL code into
> emacs for many reasons.

Re: Why is Elisp slow?

[Ergus]

On Fri, May 03, 2019 at 10:16:32PM +0900, ????????? wrote:
>Looks like the point of getting Guile Emacs recurs to:
>
>* How much Guile Emacs is polished(seamless integration with Elisp, full compatibility, etc..)
Maybe here is the actual problem.. I actually asked Eli.

>* How much Guile supports other languages such as JS or Lua
The C api is pretty nice and easy. I have some code done on it. JS is
a different beast. But having C it is straightforward to interface
python and Lua. Actually the Lua C API is much more complicated that
than Guile's.

>* How much Guile manages to be fast

It is decently fine and slowly is getting better.
>
>Guile Emacs was developed for at least 15 years or more, right?
>
>How much can Guile Emacs do these in current situation?????
>Is Guile Emacs usable day-to-day?
>
The problem is maintenance and manpower. The emacs developers are very
busy with bugfixes and there is A LOT of code to maintain/update/fix.

It needs just a voluntary to do the work. I am fine with the C
integration but all Lisp is still a bit esoteric for me. 
>
>> 2019. 5. 3. ?????? 9:51, Ergus <spacibba@aol.com> ??????:
>>
>> On Fri, May 03, 2019 at 08:52:06PM +0900, ????????? wrote:
>>
>>> The difference between Guile and CL is that
>>> * Guile is arguably not popular and not used by many programs. I've never heard a program that is popular and uses Guile as an extension language. CL is, well, much more popular than Guile, and is used by industry-strength programs.
>>
>>> * Guile (which is a scheme) has a radically different syntax with
>>> elisp (which means that to implement elisp in Guile, one has to hack
>>> the byte code interpreter AFAIK). Elisp can be implemented inside CL
>>> with full compatibility, which mitigates lots of problems between
>>> interfacing between CL code and elisp code.
>>
>> But Guile is in fact developed in the gnu project, so there are the
>> Guile developers (very few but they are) that could potentially
>> provide and fix any issue. But also the strong part of Guile is not the
>> language itself but the potential of the infrastructure.
>>
>>> * I'm pretty sure CL will be much, much faster than Guile.
>>
>> Probably, that depends of the compiler not only the language.
>>
>>> CL was optimized for about 30 years or so, and SBCL compiles CL all the way to native code, which is comparable to C. I'm not sure if Guile can do that.
>>
>> Guile has been also optimized (in spite of there is a lot of work to do,
>> that's true) and yes, it can generate native code, but also
>> bytecode. But it also offers a lot of functionalities to extend and work
>> as a glue language which is actually most important in our days.
>>
>> At the end (in my opinion) if emacs wants to survive other 40 years it
>> will need to start looking for integration with python, lua, C++, rust,
>> Ruby and similar languages with a more "promising" future than common
>> lisp. Because we really need to attract new developers with new ideas,
>> visions and experiences... and 99% of programmers don't use any lisp
>> like language.
>>
>> To make big changes we need more developers, that will not come if we
>> don't have more users first. And those new users/developers can enforce
>> the new changes looking at the future and not to backward compatibility.
>>
>>
>>>> https://www.gnu.org/gnu/rms-lisp.en.html <https://www.gnu.org/gnu/rms-lisp.en.html>
>>>
>>> I read the link; it's a pity to not consider CL because it isn't "lispy" enough.
>>>
>>>>
>>>>> IMHO building emacs on CL would be a wonderful idea... and would fix many problems that current Emacs have.
>>>>>
>>>> And create others, it is always a trade off.
>>>
>>> What problems would CL based Emacs can make? Can you show some examples? I am genuinely interested.
>>>
>>
>> Dynamic scoping rules vs exical scoping rules in common lisp is just the
>> first one I can remember, but maybe someone else will reply you with more details.
>>
>>>>
>>>> Any way
>>>>
>>>> I have to say that comparing to other interpreters around the Elisp is
>>>> not the slowest one I have tried by far. Of course it could be way
>>>> faster, but some optimization like user code and provide new containers
>>>> and datatypes in the library level based in C (and recomend them) code
>>>> could potentially make more difference in real user extensions.
>>>
>>> Well, CL is as fast as C.
>>
>> But again, common lisp is just a language. Performance is more associated
>> with the compiler you use. Cython makes python very fast too for
>> example AND IT IS PYTHON!!!.  The latest version of the GNU common lisp
>> compiler (GCL) was in 2014... so its developement goes slower than
>> Guile.
>>
>> If we had a native compiler for Elisp it could be as fast as common lisp
>> (ideally). But in any case it needs to be something that goes embedded
>> within emacs (otherwise emacs won't be emacs anymore).
>>
>> Think also in the licenses issues and legal consequences of such strong
>> dependency for a key part of the editor.
>>
>> On the other hand it is true that in that case that will be a part of
>> the code that we won't need to maintain if we rely on other packages.
>>
>> Let me say something. I am not opposed at all to switch to common lisp,
>> (or scheme or python, actually I think that at some point it will be
>> needed to change the core language for something more flexible)
>> what I mean is that this specific problem is not a language issue, but the
>> compiler. And I don't think that we can just copy the SBCL code into
>> emacs for many reasons.
>
>

Re: Why is Elisp slow?

[Eli Zaretskii]

> Date: Fri, 3 May 2019 14:58:32 +0200
> From: Ergus <spacibba@aol.com>
> Cc: help-gnu-emacs@gnu.org
> 
> >This is not a problem with Guile, because Guile includes a compiler
> >and interpreter for Emacs Lisp.
> >
> Hi Eli, now I am the curious.
> 
> If that's already done (which sems to be the harder part), where is the
> real problem to migrate to guile? I know there should be many, but what
> are the known ones??

There were past discussions about that, I suggest to look them up.

One of the problems I remember is the basic difference in philosophy
regarding raw bytes in text strings: Guile disallows that, and in many
cases signals errors, while ELisp allows that, and Emacs as a project
is unlikely to change that attitude (while Guile developers seemed to
be convinced theirs is the right philosophy).  Since this directly
affects the internal representation of strings, we have a problem
here.  Maybe this is not the most important problem, but until it's
solved, we cannot even think about moving on.

In general, the integration of Guile into Emacs is simply not finished
yet, not even close.  We cannot land such unstable code on master.
And of course, the longer the guile-emacs project is left as an
unmaintained fork, the more bitrot is accrues, for example it doesn't
support latest Emacs features like bignums in ELisp (although Guile
itself does support that).  In short, without an active team working
on it, this will never happen.  (As everything else in Emacs, except
that this feature is very complex and needs a significant effort.)

Re: Why is Elisp slow?

[Eli Zaretskii]

> From: 조성빈 <pcr910303@icloud.com>
> Date: Fri, 3 May 2019 22:16:32 +0900
> Cc: help-gnu-emacs@gnu.org
> 
> Is Guile Emacs usable day-to-day?

I don't think so, but you can try it and find out yourself.

I think in the past the person who worked on guile-emacs posted a
summary of the status and the problems still to be solved, so maybe
look that up.

Re: Why is Elisp slow?

[Óscar Fuentes]

Ergus <spacibba@aol.com> writes:

>>More importantly, the libJIT build failed to show any significant
>>speed-up wrt byte code, so it sounds like maybe the whole idea was
>>either wrong or its design couldn't possibly provide any gains.  Or
>>maybe we just measured the speed-up in wrong scenarios.
>>
> This is not surprising. My work is 80% performance measurement and
> benchmarking and the real improvements with JIT compilation (in my
> experience) and specially with libJIT is not as good as many people
> expect in most of the common scenarios. That's because the generated
> code is usually very generic (so it does not take advantage of
> architecture specific features), and strategies like vectorization and
> branch prediction are very difficult to hint (most of the times
> impossible). So the only real difference with a bytecode interpreter is
> the bytecode parsing part, but not too much more.

On Emacs case, I'm pretty sure whatever advantages comes from good
architecture-specific code, accurate branch prediction, etc are below
the noise level. As you pointed out below, Elisp is a dynamic language
and for turning this

(let ((acc 0))
  (dotimes (i 10)
    (setq acc (+ acc (foo i)))))
    
into this

int acc = 0;
for(int i = 0; i < 10; ++i) {
  acc += foo(i);
}

you need either sophisticated analysis (that, in practice, only works
for the "easy" cases) or annotate the code with type declarations (and
enforce then).

Because otherwise handling variables as containers for generic values is
incompatible with "C-like" performance.

And an Elisp -> C translator does not magically solve this.

Re: Why is Elisp slow?

[Stefan Monnier]

> If that's already done (which sems to be the harder part), where is the
> real problem to migrate to guile? I know there should be many, but what
> are the known ones??

There's also the question of what would be the advantages.


        Stefan

Re: Why is Elisp slow?

[Samuel Wales]

On 5/2/19, Marcin Borkowski <mbork@mbork.pl> wrote:
> ...well, Org-mode agenda takes almost 30 seconds to generate for me...

[replying in general]

org agenda is the slowest part of emacs for me.

it would be *great* if devs could speed it up with c.  i am curious
what would they do?  take a look at the profiles.  maybe speed up
regexps?

a complete rewrite in lisp might or might not speed it up.  it might
or might not allow devs to speed it up with c.  i think a complete
rewrite has been proposed, but dunno if it's going to be done or will
keep compatibility or be known to be likely faster.

i /think/ a rewrite /has already/ been done, with a nice lisp syntax
to replace the existing non-lisp tags search syntax.  but last i heard
it leaves out a small number of features and it isn't faster yet.
iirc part of the reason given for that was that the existing one was
optimized to death.  however, idk if it is a complete rewrite or if
speed was a desideratum.  perhaps it or parts of it could be rewritten
or sped up in c.  dunno.

as for cl, it's so similar to el that it is a natural comparison.
which makes me wonder why a participant in this thread said it was for
wizards?  elisp has buffers, syntax tables, and font locking.

-- 
The Kafka Pandemic

What is misopathy?
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

The disease DOES progress. MANY people have died from it. And ANYBODY
can get it at any time.

Re: Why is Elisp slow?

[Emanuel Berg]

Ergus wrote:

> At the end (in my opinion) if emacs wants to
> survive other 40 years it will need to start
> looking for integration with python, lua,
> C++, rust, Ruby and similar languages with
> a more "promising" future than common lisp.

Maybe there are some modules or interfaces
around that to some degree makes it possible to
use other languages than Lisp the Emacs
extension and configuration language.

However even so I think it'll stay at that.
People won't accept an Emacs w/o Lisp. Its half
the fun of it.

> Ruby and similar languages with a more
> "promising" future than common lisp.
> Because we really need to attract new
> developers with new ideas, visions and
> experiences... and 99% of programmers don't
> use any lisp like language.

IMO there is no need to do anything in
particular to attract anyone. If they are
attracted by Emacs enough, but for some reason
don't want Lisp with it, they will come anyway,
fork it and try (perhaps even succeed) in
having another language replace Lisp.

But even if this happens this breakaway Emacs
will be a very little ship in the Emacs world.

People like Lisp, some even love it! Just take
a look at the [M]ELPA archives and people's
Elisp dotfiles all around the world. (Now it
sounded like I do that, only I don't. But 1)
I have my own dotfiles and 2) I don't have to
see them other guys' to know they are
there. :))

How about Emacs w/ Haskell for example?
Side-effect free Emacs - no matter what button
you push, nothing happens! :)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[Emanuel Berg]

Ergus wrote:

> That's the strategy that made python become
> so successfully (plus the language
> simplicity). The existence of Cython, Numba +
> Pure C/Fortran core libraries like numpy and
> a simple C API to extend it. So when a user
> implements anything in 100% python; if it has
> performance issues she only needs to change
> the python lists with numpy arrays and numpy
> functions and that's it, the difference will
> be observed immediately. But if a specific
> function still represents a problem she can
> implement it easily in C or use a C library
> directly with CTypes, Pybind11 or boost, that
> uses dlopen instead of create processes and
> parse outputs and writes in numpy
> arrays/types directly.

Jesus these days you need a PhD to write
a program.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[Eli Zaretskii]

> From: Samuel Wales <samologist@gmail.com>
> Date: Fri, 3 May 2019 16:34:21 -0700
> Cc: help-gnu-emacs@gnu.org
> 
> org agenda is the slowest part of emacs for me.
> 
> it would be *great* if devs could speed it up with c.  i am curious
> what would they do?  take a look at the profiles.  maybe speed up
> regexps?

A profile of a long run with all its tree branches expanded would be
nice, preferably with the relevant Org Lisp files loaded as *.el, to
increase the accuracy of the profile.  Bonus points for providing Org
file(s) needed to reproduce such a long run, which could serve as
basis for trying various speed-up approaches.

If someone wants to do that, please use the bug-reporting features in
Emacs (so that the relevant system configuration aspects are captured,
and the data is recorded by the issue tracker), and please CC the
relevant Org addresses (mailing list, issue tracker, etc.).

Re: Why is Elisp slow?

[Marcin Borkowski]

On 2019-05-04, at 02:32, Emanuel Berg <moasenwood@zoho.eu> wrote:

> Side-effect free Emacs - no matter what button
> you push, nothing happens! :)

Well, Vim has already taken that niche (though it's not exactly
"nothing", but "nothing with a beep").

;-)

-- 
Marcin Borkowski
http://mbork.pl

Re: Why is Elisp slow?

[Marcin Borkowski]

On 2019-05-02, at 22:00, tomas@tuxteam.de wrote:

> On Thu, May 02, 2019 at 09:13:28PM +0200, Marcin Borkowski wrote:
>
> [...]
>
>> > (My personal rant to package authors is that they are way too eager to
>> > implement stuff in Lisp which cannot possibly be fast enough or
>> > scalable enough, instead of urging Emacs to provide new C primitives
>> > and core features, or, better, submitting patches to implement such
>> > features in C.  Two cases in point are linum-mode and fci-mode.  End
>> > of rant.)
>> 
>> That is an interesting POV, although I'm not sure if I agree.  IOW,
>> I fully understand why people might do so.  I, for one, consider myself
>> pretty fluent in Elisp and hardly literate in C.
>
> This approach is known as "Ousterhout's dichotomy" [1], after John Ousterhout,
> author of Tcl (which was conceived as a very simple scripting language
> meant to extend existing applications). Tcl grew later a bytecode
> interpreter (as did elisp) -- making the C interface significantly
> more complex (but the interpreter significantly faster).
>
> Walking the trade-off between simple and efficient is interesting.

Thanks.  As usual, I learned something new from you...

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: Why is Elisp slow?

[Marcin Borkowski]

On 2019-05-02, at 21:45, Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Marcin Borkowski <mbork@mbork.pl>
>> Cc: help-gnu-emacs@gnu.org
>> Date: Thu, 02 May 2019 21:13:28 +0200
>> 
>> > It's certainly a reason to report this to the Org developers, and ask
>> > them to speed this up.
>> 
>> Well, I don't consider it a bug in Orgmode - rather a bug in my workflow
>
> Maybe improvements in Org could make your bug less prominent...
>
> Then again, it could be that there's some trick up Org developers'
> sleeve that they could teach you.

As it was already said, there's probably not much room for optimization
here - it was raised many times on the Org-mode mailing list.

> In general, anything surprising/inconvenient/buggy should be reported,
> you can never know what will come out of that.
>
>> > (My personal rant to package authors is that they are way too eager to
>> > implement stuff in Lisp which cannot possibly be fast enough or
>> > scalable enough, instead of urging Emacs to provide new C primitives
>> > and core features, or, better, submitting patches to implement such
>> > features in C.  Two cases in point are linum-mode and fci-mode.  End
>> > of rant.)
>> 
>> That is an interesting POV, although I'm not sure if I agree.  IOW,
>> I fully understand why people might do so.  I, for one, consider myself
>> pretty fluent in Elisp and hardly literate in C.
>
> Submitting changes in C is a bonus, it's good enough to describe the
> use case and ask for primitives to support it.  Someone else might get
> motivated.  It could even be very easy, like with fci-mode.
>
> I'm quite sure that some of the bad reputation of Emacs is due to
> incredible things people do in Lisp, which then make Emacs look
> sluggish and bloated.

Possibly.  Still, I guess I never coded something inefficient/big enough
to need the speedup from a potential new C-level feature.  Though I'm
going to remember what you said here, just in case.

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: Why is Elisp slow?

[Ergus]

On Sat, May 04, 2019 at 12:18:29AM +0200, ?scar Fuentes wrote:
>Ergus <spacibba@aol.com> writes:
>
>>>More importantly, the libJIT build failed to show any significant
>>>speed-up wrt byte code, so it sounds like maybe the whole idea was
>>>either wrong or its design couldn't possibly provide any gains.  Or
>>>maybe we just measured the speed-up in wrong scenarios.
>>>
>> This is not surprising. My work is 80% performance measurement and
>> benchmarking and the real improvements with JIT compilation (in my
>> experience) and specially with libJIT is not as good as many people
>> expect in most of the common scenarios. That's because the generated
>> code is usually very generic (so it does not take advantage of
>> architecture specific features), and strategies like vectorization and
>> branch prediction are very difficult to hint (most of the times
>> impossible). So the only real difference with a bytecode interpreter is
>> the bytecode parsing part, but not too much more.
>
>On Emacs case, I'm pretty sure whatever advantages comes from good
>architecture-specific code, accurate branch prediction, etc are below
>the noise level. As you pointed out below, Elisp is a dynamic language
>and for turning this
>
>(let ((acc 0))
>  (dotimes (i 10)
>    (setq acc (+ acc (foo i)))))
>
>into this
>
>int acc = 0;
>for(int i = 0; i < 10; ++i) {
>  acc += foo(i);
>}
>
>you need either sophisticated analysis (that, in practice, only works
>for the "easy" cases) or annotate the code with type declarations (and
>enforce then).
>
>Because otherwise handling variables as containers for generic values is
>incompatible with "C-like" performance.
>
>And an Elisp -> C translator does not magically solve this.
>
That's true. The code quality will be not very good (as what happen in
Android Java native compilers and Cython) but some of the optimizations
in the low level can be applied. That depends of the optimizations
implemented and the information provided to the compilers (both of
them).

A Lisp-C compiler for example can reduce significantly the function call
overheads and callback overheads cause thanks to the Lisp syntax it is
very easy to apply inline optimizations which in C represent a VERY
important improvement. In your example code foo, setq and operator + are
functions called in runtime, which interpreted means go to the symbol
hashtable, find the pointer to the function, interpret the inputs and
execute... compiling that... just think how it can change.

With a simple optimization like having the hardcoded 10 the compiler
will know the number of iterations to execute and the C compiler applies
many good optimizations in those cases.

SO it won't be the same than your second code, but performance could be
in the same order in many cases.

Re: Why is Elisp slow?

[Ergus]

On Fri, May 03, 2019 at 06:57:57PM -0400, Stefan Monnier wrote:
>> If that's already done (which sems to be the harder part), where is the
>> real problem to migrate to guile? I know there should be many, but what
>> are the known ones??
>
>There's also the question of what would be the advantages.
>
>
>        Stefan
Inter-language interaction (which is one of the strong features in
Guile out of the box) and native compilation. Plus some people taking
care of the compiler/interpreter/virtual machine parts and specialized
in that specific section, which is good because we need manpower.

Re: Why is Elisp slow?

[조성빈]

2019. 5. 4. 오후 10:27, Ergus <spacibba@aol.com> 작성:

>> On Sat, May 04, 2019 at 12:18:29AM +0200, ?scar Fuentes wrote:
>> Ergus <spacibba@aol.com> writes:
>> 
>>>> More importantly, the libJIT build failed to show any significant
>>>> speed-up wrt byte code, so it sounds like maybe the whole idea was
>>>> either wrong or its design couldn't possibly provide any gains.  Or
>>>> maybe we just measured the speed-up in wrong scenarios.
>>>> 
>>> This is not surprising. My work is 80% performance measurement and
>>> benchmarking and the real improvements with JIT compilation (in my
>>> experience) and specially with libJIT is not as good as many people
>>> expect in most of the common scenarios. That's because the generated
>>> code is usually very generic (so it does not take advantage of
>>> architecture specific features), and strategies like vectorization and
>>> branch prediction are very difficult to hint (most of the times
>>> impossible). So the only real difference with a bytecode interpreter is
>>> the bytecode parsing part, but not too much more.
>> 
>> On Emacs case, I'm pretty sure whatever advantages comes from good
>> architecture-specific code, accurate branch prediction, etc are below
>> the noise level. As you pointed out below, Elisp is a dynamic language
>> and for turning this
>> 
>> (let ((acc 0))
>> (dotimes (i 10)
>>   (setq acc (+ acc (foo i)))))
>> 
>> into this
>> 
>> int acc = 0;
>> for(int i = 0; i < 10; ++i) {
>> acc += foo(i);
>> }
>> 
>> you need either sophisticated analysis (that, in practice, only works
>> for the "easy" cases) or annotate the code with type declarations (and
>> enforce then).
>> 
>> Because otherwise handling variables as containers for generic values is
>> incompatible with "C-like" performance.
>> 
>> And an Elisp -> C translator does not magically solve this.
>> 
> That's true. The code quality will be not very good (as what happen in
> Android Java native compilers and Cython) but some of the optimizations
> in the low level can be applied. That depends of the optimizations
> implemented and the information provided to the compilers (both of
> them).
> 
> A Lisp-C compiler for example can reduce significantly the function call
> overheads and callback overheads cause thanks to the Lisp syntax it is
> very easy to apply inline optimizations which in C represent a VERY
> important improvement.

I’m pretty sure that it is hard to do that, because lisp is a dynamic language, and it is hard to be sure if that function I(the compiler) wants to inline is really the function that the code writer wants to call.
The transpiled code essentially will be C code that contains a small embedded elisp interpreter, which imposes the same overhead as running elisp code through the elisp engine from Emacs.

> In your example code foo, setq and operator + are
> functions called in runtime, which interpreted means go to the symbol
> hashtable, find the pointer to the function, interpret the inputs and
> execute... compiling that... just think how it can change.

Transpiled C code will have the same overhead.

> 
> With a simple optimization like having the hardcoded 10 the compiler
> will know the number of iterations to execute and the C compiler applies
> many good optimizations in those cases.
> 
> SO it won't be the same than your second code, but performance could be
> in the same order in many cases.

Re: Why is Elisp slow?

[Stefan Monnier]

> Inter-language interaction (which is one of the strong features in
> Guile out of the box)

That's fairly abstract (details matter: e.g. how are you going to use
macros like `define-minor-mode` from non-Elisp languages?). 

Also while I much prefer Scheme to Elisp (or Common-Lisp for that
matter), I'm not looking forward to maintaining Emacs packages in two
(and more) different languages, and other maintainers are probably
imagining such maintenance nightmare with a lot more dread than I.

So again: what are the advantages?

From that point of view, moving Emacs to a Common-Lisp implementation
would make more sense, since there is a chance we can *replace* Elisp
with Common-Lisp in the long-run, rather than adding another language on
the side.

> and native compilation.

That's irrelevant.  I think you mean "efficiency" or something like that.
But that presumes that Elisp-on-Guile is indeed more efficient than
Elisp-on-Emacs.  Is there actual evidence that this is the case?

Also "efficiency" is broad: there's the execution time of some
benchmarks, of course, but there's also the startup time, the VM size,
the heap size, the GC interruptions, the time to load a big package
(e.g. org-mode), ...

Guile would hopefully win on some of those, but by how much?
And how much worse does it make the others?

> Plus some people taking care of the compiler/interpreter/virtual
> machine parts and specialized in that specific section, which is good
> because we need manpower.

AFAIC that's the main goal, indeed.

But last I checked Guile didn't have much manpower either.  And there's
a good chance that tracking Guile development (after switching to Guile)
will also require manpower on Emacs's side.  So it's not clear that it
would turn out to be an advantage.


        Stefan

Re: Why is Elisp slow?

[Eli Zaretskii]

> Date: Sat, 4 May 2019 15:32:18 +0200
> From: Ergus <spacibba@aol.com>
> Cc: help-gnu-emacs@gnu.org
> 
> because we need manpower.

For some reason, I very much doubt that integration with Guile will
add a significant amount of manpower to our team.

Re: Why is Elisp slow?

[Emanuel Berg]

Stefan Monnier wrote:

> Also while I much prefer Scheme to Elisp (or
> Common-Lisp for that matter), I'm not looking
> forward to maintaining Emacs packages in two
> (and more) different languages, and other
> maintainers are probably imagining such
> maintenance nightmare with a lot more dread
> than I.

Except for some other advantages with CL
compared to Elisp, which I take it are there
for natural reasons, is the reason CL is much
faster than Elisp, and even as fast as C,
reportedly, using the Steel Bank compiler - is
the reason CL is much faster than Elisp that CL
can be compiled for real, and there is even
a "a high performance" [1] compiler, SBCL,
readily available to do it, *meanwhile* with
Elisp you're stuck with the mere byte-compiler?

So then the first question, literally [2], as
formulated by a wise but in this case ignorant
man, is:

    Why can't Elisp be compiled as well?

> From that point of view, moving Emacs to
> a Common-Lisp implementation would make more
> sense, since there is a chance we can
> *replace* Elisp with Common-Lisp in the
> long-run, rather than adding another language
> on the side.

I think that'd be great, but instead of having
CL on the side (or "Elisp on the side", if you
will) is it unthinkable to add a module _to CL_
so that _CL can deal with Elisp_ with its
usual toolchain?


[1] http://www.sbcl.org/
[2] http://lists.gnu.org/archive/html/help-gnu-emacs/2019-05/msg00008.html

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[Stefan Monnier]

>     Why can't Elisp be compiled as well?

The question is meaningless because it's premise is wrong: Elisp can
be compiled as well.


        Stefan

Re: Why is Elisp slow?

[Emanuel Berg]

Stefan Monnier wrote:

>> Why can't Elisp be compiled as well?
>
> The question is meaningless because it's
> premise is wrong: Elisp can be compiled
> as well.

Well, someone is edgy all of a sudden?

But very well then, why can't Elisp be compiled
as fast as CL (or C), and then be used on top
of (C) Emacs?

Or if it can, why isn't it?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[조성빈]

2019. 5. 5. 오전 7:56, Stefan Monnier <monnier@iro.umontreal.ca> 작성:

>>    Why can't Elisp be compiled as well?
> 
> The question is meaningless because it's premise is wrong: Elisp can
> be compiled as well.
> 
I’m pretty sure Elisp can’t compile down to raw machine code (like JIT) in current infrastructure. If you’re saying about compiling to bytecode, you’re losing his/her points.
Elisp bytecode is terribly slow :-(
> 
>        Stefan
> 

Re: Why is Elisp slow?

[Ergus]

Hi Stefan:

I mostly agree with you and I get your points but I have some comments about.

On May 4, 2019 4:03:28 PM GMT+02:00, Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:
>> Inter-language interaction (which is one of the strong features in
>> Guile out of the box)
>
>That's fairly abstract (details matter: e.g. how are you going to use
>macros like `define-minor-mode` from non-Elisp languages?). 
>
>Also while I much prefer Scheme to Elisp (or Common-Lisp for that
>matter), I'm not looking forward to maintaining Emacs packages in two
>(and more) different languages, and other maintainers are probably
>imagining such maintenance nightmare with a lot more dread than I.
>
>So again: what are the advantages?
>
I totally agree with the complexity of mixing languages; but if we look around it is very difficult to attract new users and developers if they need to learn a new (old fashion) language only for Emacs (as I have been doing the last months). New programmers generations only know python and some C-like languages but specially OOP and maybe is the moment to think a bit in the future of the project more than in the past. Even if that implies taking some risks and make hard desitions. Just give a look how popular are sublime text or athom, that compared to Emacs those are kid toys.

So at some point it will be needed to make this desitions because the old developers will not live forever. I see the beauty of Lisp, but I am talking more about a human issue here, that is more critical for Emacs right now and for the future.

>From that point of view, moving Emacs to a Common-Lisp implementation
>would make more sense, since there is a chance we can *replace* Elisp
>with Common-Lisp in the long-run, rather than adding another language
>on
>the side.
>
IF we could do so (embed the CL compiler as a submodule within emacs) and this can improve performance (while reduce the amount of code we need to maintain, and the number of field experts we need to reach and attract ourself) I'll totally support such desition. It is not THE solution we need but it will reduce some of the important issues. And will make the project development more dynamic and sutainable. But if we can't take such desition right now is because we have few people to do the work and the new potential users/developers live in a different world with python java rust lua C++ Ruby but also saturated of cheaper (easier to use and that do work without learning curve) alternatives like notepad++, visual studio code, geany, eclipse, kdevelop... and so on. And some of them have put a lot of effort in improving the editing capabilities, programming functionalities and ergonomy more than implementing a pdf reader or games withing the tool. So they only do one thing but they do it right (or good enough for the user general needs) without initial configurations.

For the developers it is also easier to join to those projects because they are hosted on Github/gitlab with a more familiar workflow and interface, no copyright procedure, no mailing list.... and everything in the same please and integrate with a fork based workflow. You can see where I'm going right?

I don't say that all this is better or worst than what we have, it is just what the new generations understand, learn in the universities. Otherwise we are developing Emacs for us and the already existing users only. Ignoring where is the programming world now and where it is going.


>> and native compilation.
>
>That's irrelevant.  I think you mean "efficiency" or something like
>that.
>But that presumes that Elisp-on-Guile is indeed more efficient than
>Elisp-on-Emacs.  Is there actual evidence that this is the case?
>
>Also "efficiency" is broad: there's the execution time of some
>benchmarks, of course, but there's also the startup time, the VM size,
>the heap size, the GC interruptions, the time to load a big package
>(e.g. org-mode), ...
>
>Guile would hopefully win on some of those, but by how much?
>And how much worse does it make the others?
>
>> Plus some people taking care of the compiler/interpreter/virtual
>> machine parts and specialized in that specific section, which is good
>> because we need manpower.
>
>AFAIC that's the main goal, indeed.
>
>But last I checked Guile didn't have much manpower either.  And there's
>a good chance that tracking Guile development (after switching to
>Guile)

Vicious circle no-projects-interested means no users with issues which means no potential developers. Any way I was talking about guile because it was the official gnu language for extensions similar somehow to elisp and with a full platform more or less ready to use without license issues.

>will also require manpower on Emacs's side.  So it's not clear that it
>would turn out to be an advantage.
>
>
>        Stefan

-- 
Enviado desde mi dispositivo Android con K-9 Mail. Por favor, disculpa mi brevedad.

Re: Why is Elisp slow?

[조성빈]

The more I think about Guile Emacs and the Guile project itself, I start to think that it’s not a problem of manpower, but something more fundamental. Is it feasible (or even possible) to make two different high level languages to interoperate in the engine level?̊̈
For example, will it even make sense to have some Elisp macros in scheme land, or even lua land, python land, JS land?̊̈

Even if Emacs has limited success in these problem by e.g. transpiling JS to elisp or doing macro expansion at runtime by the Elisp engine, I would like a JSish API for emacs when writing packages, not a low level wrapper around Elisp APIs.
Same thing for Python, Lua, Scheme, etc...
Different languages introduce different concepts that just don’t match.
What would I do to use a Python module in JS?̊̈ Unless the language itself is designed to be easily interoperable with elisp by superseding concepts (such as Clojure superseding Java), I’m pretty sure many people won’t be satisfied in any kind of implementation. 

CL is one (and presumably the only) case that (at least looks like) supersedes Elisp in concepts. (Actually it would be something more like Elisp designed like that.) It’s a Lisp-2, it has macros, it uses S-exps, etc, etc. It also has the advantage that we can leverage the state-of-art CL compilers that produce insane speeds from such a dynamic language.

2019. 5. 5. 오전 9:43, Ergus <spacibba@aol.com> 작성:

> Hi Stefan:
> 
> I mostly agree with you and I get your points but I have some comments about.
> 
> On May 4, 2019 4:03:28 PM GMT+02:00, Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:
>>> Inter-language interaction (which is one of the strong features in
>>> Guile out of the box)
>> 
>> That's fairly abstract (details matter: e.g. how are you going to use
>> macros like `define-minor-mode` from non-Elisp languages?). 
>> 
>> Also while I much prefer Scheme to Elisp (or Common-Lisp for that
>> matter), I'm not looking forward to maintaining Emacs packages in two
>> (and more) different languages, and other maintainers are probably
>> imagining such maintenance nightmare with a lot more dread than I.
>> 
>> So again: what are the advantages?
> 
> I totally agree with the complexity of mixing languages; but if we look around it is very difficult to attract new users and developers if they need to learn a new (old fashion) language only for Emacs (as I have been doing the last months). New programmers generations only know python and some C-like languages but specially OOP and maybe is the moment to think a bit in the future of the project more than in the past. Even if that implies taking some risks and make hard desitions. Just give a look how popular are sublime text or athom, that compared to Emacs those are kid toys.
> 
> So at some point it will be needed to make this desitions because the old developers will not live forever. I see the beauty of Lisp, but I am talking more about a human issue here, that is more critical for Emacs right now and for the future.
> 
>> From that point of view, moving Emacs to a Common-Lisp implementation
>> would make more sense, since there is a chance we can *replace* Elisp
>> with Common-Lisp in the long-run, rather than adding another language
>> on
>> the side.
>> 
> IF we could do so (embed the CL compiler as a submodule within emacs) and this can improve performance (while reduce the amount of code we need to maintain, and the number of field experts we need to reach and attract ourself) I'll totally support such desition. It is not THE solution we need but it will reduce some of the important issues. And will make the project development more dynamic and sutainable. But if we can't take such desition right now is because we have few people to do the work and the new potential users/developers live in a different world with python java rust lua C++ Ruby but also saturated of cheaper (easier to use and that do work without learning curve) alternatives like notepad++, visual studio code, geany, eclipse, kdevelop... and so on. And some of them have put a lot of effort in improving the editing capabilities, programming functionalities and ergonomy more than implementing a pdf reader or games withing the tool. So they only do one thing but they do it right (or good enough for the user general needs) without initial configurations.
> 
> For the developers it is also easier to join to those projects because they are hosted on Github/gitlab with a more familiar workflow and interface, no copyright procedure, no mailing list.... and everything in the same please and integrate with a fork based workflow. You can see where I'm going right?
> 
> I don't say that all this is better or worst than what we have, it is just what the new generations understand, learn in the universities. Otherwise we are developing Emacs for us and the already existing users only. Ignoring where is the programming world now and where it is going.
> 
> 
>>> and native compilation.
>> 
>> That's irrelevant.  I think you mean "efficiency" or something like
>> that.
>> But that presumes that Elisp-on-Guile is indeed more efficient than
>> Elisp-on-Emacs.  Is there actual evidence that this is the case?
>> 
>> Also "efficiency" is broad: there's the execution time of some
>> benchmarks, of course, but there's also the startup time, the VM size,
>> the heap size, the GC interruptions, the time to load a big package
>> (e.g. org-mode), ...
>> 
>> Guile would hopefully win on some of those, but by how much?
>> And how much worse does it make the others?
>> 
>>> Plus some people taking care of the compiler/interpreter/virtual
>>> machine parts and specialized in that specific section, which is good
>>> because we need manpower.
>> 
>> AFAIC that's the main goal, indeed.
>> 
>> But last I checked Guile didn't have much manpower either.  And there's
>> a good chance that tracking Guile development (after switching to
>> Guile)
> 
> Vicious circle no-projects-interested means no users with issues which means no potential developers. Any way I was talking about guile because it was the official gnu language for extensions similar somehow to elisp and with a full platform more or less ready to use without license issues.
> 
>> will also require manpower on Emacs's side.  So it's not clear that it
>> would turn out to be an advantage.
>> 
>> 
>>       Stefan
> 
> -- 
> Enviado desde mi dispositivo Android con K-9 Mail. Por favor, disculpa mi brevedad.
> 

Re: Why is Elisp slow?

[Stefan Monnier]

>>>    Why can't Elisp be compiled as well?
>> The question is meaningless because it's premise is wrong: Elisp can
>> be compiled as well.
> I’m pretty sure Elisp can’t compile down to raw machine code (like JIT) in
> current infrastructure.

Maybe there is no compiler that does that currently, but that's not
relevant to the question of whether there can be such a compiler.
[ And FWIW, you can probably compile Elisp to "raw" machine code with
  https://sourceforge.net/p/clocc/hg/ci/default/tree/src/cllib/elisp.lisp  ]

> Elisp bytecode is terribly slow :-(

There's been very little effort put into making it fast, indeed.
This said, when we compared it to Guile and some byte-code-based
Common-Lisp (I think it was ECL, but I can't remember) some years ago it
wasn't nearly as bad as I thought.

If someone's interested in speeding it up.  I'd recommend two
approaches:
- Either a JIT based on something like basic-block-versioning (BBV)
- Or a fast byte-code such as those generated by vmgen


        Stefan

Re: Why is Elisp slow?

[Paul W. Rankin]

On Sun, May 05 2019, Ergus wrote:
> I totally agree with the complexity of mixing languages; but if we 
> look around it is very difficult to attract new users and developers 
> if they need to learn a new (old fashion) language only for Emacs (as 
> I have been doing the last months). New programmers generations only 
> know python and some C-like languages but specially OOP and maybe is 
> the moment to think a bit in the future of the project more than in 
> the past. Even if that implies taking some risks and make hard 
> desitions. Just give a look how popular are sublime text or athom, 
> that compared to Emacs those are kid toys.
>
> So at some point it will be needed to make this desitions because the 
> old developers will not live forever. I see the beauty of Lisp, but I 
> am talking more about a human issue here, that is more critical for 
> Emacs right now and for the future.

I don't understand this sentiment. Is there some emergency in Emacs 
development that I'm unaware of? Development seems faster and stronger 
than it ever has in the decade or so I've been using Emacs -- there's a 
new major release every year or so, and with the addition of package.el 
there's the introduction of third-party package archives and the 
explosion that is MELPA.

The possibility of the current Emacs developers all dying out is not 
something anyone needs to worry about for at least another 30-40 years, 
and I'm pretty sure we (the human race) will have vastly different 
concerns by then. But even if we reach a point where there is no one 
left maintaining/developing/using Emacs, it's open source code... 
someone will discover it and if they find it useful or interesting, they 
will continue developing it. Open source code can never really die.

Where does this fear come from that an open source project will die if 
it doesn't keep changing?

> For the developers it is also easier to join to those projects because 
> they are hosted on Github/gitlab with a more familiar workflow and 
> interface, no copyright procedure, no mailing list.... and everything 
> in the same please and integrate with a fork based workflow. You can 
> see where I'm going right?

If a possible contributor has cloned a project repository to their own 
machine and has made some changes, the fork-based workflow requires that 
they: create an account at the origin project's GitHub or GitLab (or the 
project's GitLab instance), create a forked repository there, add the 
fork as a remote on their machine, push the changes, then open a pull 
request.

Once you use a git send-email workflow, this fork-based workflow will 
seem convoluted and unnecessarily centralised. All a contributor need do 
is clone the project repository, commit some changes then run:

    git send-email HEAD^

And send the email to the project's owner/mailing list. No account 
creation necessary. Check out:

    https://git-send-email.io/

Yes the copyright assignment procedure is a deterrent to contributing to 
Emacs/ELPA; this discussion is probably for another thread.

--
https://www.paulwrankin.com

Re: Why is Elisp slow?

[Stefan Monnier]

> the last months). New programmers generations only know python and some
> C-like languages but specially OOP and maybe is the moment to think a bit in
> the future of the project more than in the past. Even if that implies taking

This describes the situation from today just as well as that from 10
years ago, 20 years, and even 30 years ago.

Rewriting all Emacs packages in the language-du-jour every 10 years
doesn't sound very realistic.


        Stefan

Re: Why is Elisp slow?

[Óscar Fuentes]

"Paul W. Rankin" <hello@paulwrankin.com> writes:

>> For the developers it is also easier to join to those projects
>> because they are hosted on Github/gitlab with a more familiar
>> workflow and interface, no copyright procedure, no mailing list....
>> and everything in the same please and integrate with a fork based
>> workflow. You can see where I'm going right?
>
> If a possible contributor has cloned a project repository to their own
> machine and has made some changes, the fork-based workflow requires
> that they: create an account at the origin project's GitHub or GitLab
> (or the project's GitLab instance),

This is an one-off task.

> create a forked repository there,

Same.

> add the fork as a remote on their machine,

Same.

> push the changes, then open a pull request.

Which, in my experience, takes half a minute with the Github UI and
possibly less with Magit's Github plugin.

> Once you use a git send-email workflow, this fork-based workflow will
> seem convoluted and unnecessarily centralised. All a contributor need
> do is clone the project repository, commit some changes then run:
>
>    git send-email HEAD^

You skipped over quite a few issues here. Plus, a branch is easy to
access, in case someone wishes to grab the changes, while fishing mails
on a mailing list requires more setup and constant attention.

> And send the email to the project's owner/mailing list. No account
> creation necessary. Check out:
>
>    https://git-send-email.io/

Yes, that explains the job on a detailed way, and you can see that it is
not as simple as you pretend. Plus, that's the setup for sending
changes, no mention about how to grab them from the ml.

Re: Why is Elisp slow?

[Ergus]

Hi Paul:

On Sun, May 05, 2019 at 03:25:59PM +1000, Paul W. Rankin wrote:
>
>
>I don't understand this sentiment. Is there some emergency in Emacs 
>development that I'm unaware of? Development seems faster and stronger 
>than it ever has in the decade or so I've been using Emacs -- there's 
>a new major release every year or so, and with the addition of 
>package.el there's the introduction of third-party package archives 
>and the explosion that is MELPA.
>
I agree with you in that development is still active. But the grown in
number of users and developers is pretty constant, while the number of
developers and computer users in the world has grow exponentially. But
also there are sections and full parts in the emacs code without expert
to maintain it right now.

Mane old developers has left and deep/infrastructure/architectural big
changes are every time more difficult to do and the tendency goes worst
with time (more and more code).

Just give a look to the rating of the other editors I
mentioned, and their git repository. In fact in the last year, the
articles recommending text editors for programmers rarely include Emacs
while vim is in all of them. Also in my work there are more
than 400 programmers and I am the only Emacs user. Same when I go to
conferences around. So this is a bad statistical signal. Without users
there are not potential contributor.

We don't have a technical problem because emacs is more powerful than
most of the editors in the lists around, but we have a social problem.

Package.el same as modules are the best and more important steps emacs
has made in the last years. It has motivated and organized many things
that were very problematic before, which is very important, but it is
something the GNU/Linux distros figured out like 25 years ago
(aptitude, yast, pacman, synaptic), so we were (and are) late (not
wrong, just late, delayed in time for the decisions).

Just looking at the number of code reviewers we have, it is a reason to
worry. We have much less developers/contributors in emacs than what vim
has these days, and at the same time we have more than 2 times more code
to maintain with more languages and functionalities. 

>
>The possibility of the current Emacs developers all dying out is not 
>something anyone needs to worry about for at least another 30-40 
>years, and I'm pretty sure we (the human race) will have vastly 
>different concerns by then.

Not only dying, but also leaving, getting busy with the years, or
keeping old standards and paradigms, workflows, tools, concerns... and
this is reflected automatically in the project. Emacs needs new blood,
new people with new needs, ideas and concerns.

>But even if we reach a point where there 
>is no one left maintaining/developing/using Emacs, it's open source 
>code... someone will discover it and if they find it useful or 
>interesting, they will continue developing it. Open source code can 
>never really die.
>

The idea (and our reponsability) is not to reach there if we can avoid
that. Otherwise we failed as maintainers/developers. 

>
>Where does this fear come from that an open source project will die if 
>it doesn't keep changing?
>

Is not keep changing, is keep usefulness for its main purpose. What
emacs does as main purpose "edit text" will always gonna be useful. But,
are we being "competitive" in this field? The emacs infrastructure and
philosophy has all the needed for that (and much much muuuuuch more),
but are we using it properly for that purpose?

I think it is the best infrastructure to edit text, but right now we
are not providing the best editor (which should be the main goal). And
we add more and more and more functionalities and code. It is common
to heard that "emacs is a good OS, it only lacks a good editor."
which makes me very angry, but from the current editors standards
point of view it is right.

We don't need to keed changing, we need to do the right changes for
the project long path. FOr example: provide interaction with other
systems, languajes, optimize the compiler or rely in some other one
that already do it well and we trust in, cleanup code and portions of
code not fundamental for the emacs functionality or move them to
elpa/melpa. Standarize a bit more the bindings and function names.

For example the performance issues many people complain about
frequently is just a consequence of the extreme generalization in
functionalities and overheads in the infrastructure added for doing
"everything", but many of those things we barely do well in 2019
standards (some of then were added, and abandoned), so most users use
something different for that (like web browsing, reading PDF, play
tetris, play music, create buffers with buttons, have duality for TUI
and GUI in the same program, support very old architectures that are
inexistent these days) And all this is more and more and more code to
maintain, and more fields of expertise needed in our domain.

Same happens with development, we have files with more than 33000 lines
that we need to maintain, and any change in it is veeeeery complicated
and complex, but most of the code is to give support to some
functionalities very rarely used in practice.

And the infrastructure itself is poorly documented and complex
(specially the C interaction with Lisp from within emacs files.)
Thanks Eli, Stefan and the others who takes care of new users like me,
but that method to introduce developers is not scalable and makes them
to spend a lot of time.

>
>
>If a possible contributor has cloned a project repository to their own 
>machine and has made some changes, the fork-based workflow requires 
>that they: create an account at the origin project's GitHub or GitLab 
>(or the project's GitLab instance), create a forked repository there, 
>add the fork as a remote on their machine, push the changes, then open 
>a pull request.
>
>Once you use a git send-email workflow, this fork-based workflow will 
>seem convoluted and unnecessarily centralised. All a contributor need 
>do is clone the project repository, commit some changes then run:
>
>   git send-email HEAD^
>
>And send the email to the project's owner/mailing list. No account 
>creation necessary. Check out:
>
>   https://git-send-email.io/
>
>Yes the copyright assignment procedure is a deterrent to contributing 
>to Emacs/ELPA; this discussion is probably for another thread.
>
>--
>https://www.paulwrankin.com
>

This is not better than having a fork-merge request workflow. But also
forces to handmake a lot of the steps the new interfaces do for you like
reading the patch and make comments, follow those comments and reply to
specific parts of them, make then visible in the web, automatic
test/benchmark the functionalities, changes, close the requests or deny
them, give access for all that to the users in the future. Group people
in different forks working in the same functionality, bug, issue,
extension; test the code in the fork head automatically before merging
it in master. Is not impossible to do all this, it is just more manual,
time consumming and unfamiliar for 90% of the developers I know.

But also it is somehow dangerous, because actually everybody writes in
the master branch, while in my opinion only the managers (Eli, Stefan,
Alan...) should write there and the rest (me) should write only in the
forks and make pull requests (similar to what we do with branches, but
safer).

Now for the project it looks like:

bug: bugtracker
developement: savannah
developer interaction: mailing list
user interaction: reddit + help mailing list
documentation: within emacs
project main page: gnu.org
contribution: copyright process+mailing list+savannah
questions: another mailing list
packages: elpa
more packages: melpa
user hacks and advices: emacswiki

Re: Why is Elisp slow?

[Stefan Monnier]

> But very well then, why can't Elisp be compiled
> as fast as CL (or C), and then be used on top
> of (C) Emacs?

Same meaningless question.

> Or if it can, why isn't it?

That's the valid question.
AFAIK, the answer is that nobody was motivated enough to do it.


        Stefan

Re: Why is Elisp slow?

[Stefan Monnier]

> The more I think about Guile Emacs and the Guile project itself, I start to
> think that it’s not a problem of manpower, but something more
> fundamental.

Quite right.

> Is it feasible (or even possible) to make two different high
> level languages to interoperate in the engine level?̊̈

Language interoperation is very difficult, and even more so if you want
both languages to be usable "equally" (as opposed to a high-level
language with FFI bindings, say).  The .NET platform aims to do just
that, and note that it's fairly complex and the most successful
languages there were specifically designed/tweaked for that platform.

> For example, will it even make sense to have some Elisp macros in scheme

Interoperation between "similar" languages like Elisp and Scheme can
probably be made to be tolerable (e.g. I guess most Emacs-specific
special forms of Elisp, such as `save-excursion`, could be provided on
the Scheme side without too much trouble (famous last words)), but
macros seem pretty hard to handle in general (i.e. will require manual
work on a macro-by-macro case).

> land, or even lua land, python land, JS land?̊̈

For these, macros aren't the only problem, since there's also the
problem of the different kinds of datatypes used.  What would JS objects
look like in Elisp, and what would Elisp objects look like in JS?

> I would like a JSish API for emacs when writing packages, not a low
> level wrapper around Elisp APIs.

Indeed.

> What would I do to use a Python module in JS?̊̈ Unless the language itself is
> designed to be easily interoperable with elisp by superseding concepts (such
> as Clojure superseding Java), I’m pretty sure many people won’t be satisfied
> in any kind of implementation. 

Exactly.


        Stefan

Re: Why is Elisp slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 477 bytes --]

On Sun, May 05, 2019 at 01:19:08AM +0200, Emanuel Berg wrote:
> Stefan Monnier wrote:
> 
> >> Why can't Elisp be compiled as well?
> >
> > The question is meaningless because it's
> > premise is wrong: Elisp can be compiled
> > as well.
> 
> Well, someone is edgy all of a sudden?

Sorry to put it somewhat bluntly, Emanuel, but my impression
is that you tend to talk much and listen little.

This makes communication... somewhat difficult.

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Why is Elisp slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 553 bytes --]

On Sun, May 05, 2019 at 03:46:01PM +0200, Ergus wrote:
> Hi Paul:
> 
> On Sun, May 05, 2019 at 03:25:59PM +1000, Paul W. Rankin wrote:

[...]

> I agree with you in that development is still active. But the grown in
> number of users and developers is pretty constant, while the number of
> developers and computer users in the world has grow exponentially [...]

Exponential growth is a sign of immaturity. Only the beginning of the
logistic curve [1] looks exponential.

Cheers

[1] https://en.wikipedia.org/wiki/Logistic_curve
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Why is Elisp slow?

[Tadeus Prastowo]

On Sun, May 5, 2019 at 5:51 PM Stefan Monnier
<monnier@iro.umontreal.ca> wrote:> Language interoperation is very
difficult, and even more so if you want
> both languages to be usable "equally" (as opposed to a high-level
> language with FFI bindings, say).  The .NET platform aims to do just
> that, and note that it's fairly complex and the most successful
> languages there were specifically designed/tweaked for that platform.

Out of curiosity, do you mind to elaborate on how it is to be usable
"equally"?  (I think the best way to understand it is by studying the
.NET platform, but maybe you could explain it better).

Thank you very much.

>         Stefan

-- 
Best regards,
Tadeus

Re: Why is Elisp slow?

[Emanuel Berg]

tomas wrote:

>>>> Why can't Elisp be compiled as well?
>>>
>>> The question is meaningless because it's
>>> premise is wrong: Elisp can be compiled
>>> as well.
>> 
>> Well, someone is edgy all of a sudden?
>
> Sorry to put it somewhat bluntly, Emanuel,
> but my impression is that you tend to talk
> much and listen little.

Well, maybe people get the idea I listen
proportionally poorly, just because my talk
speaks for itself?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[조성빈]

나의 iPhone에서 보냄

2019. 5. 6. 오후 4:33, Tadeus Prastowo <tadeus.prastowo@unitn.it> 작성:

> On Sun, May 5, 2019 at 5:51 PM Stefan Monnier
> <monnier@iro.umontreal.ca> wrote:> Language interoperation is very
> difficult, and even more so if you want
>> both languages to be usable "equally" (as opposed to a high-level
>> language with FFI bindings, say).  The .NET platform aims to do just
>> that, and note that it's fairly complex and the most successful
>> languages there were specifically designed/tweaked for that platform.
> 
> Out of curiosity, do you mind to elaborate on how it is to be usable
> "equally"?  (I think the best way to understand it is by studying the
> .NET platform, but maybe you could explain it better).

I would say as ‘to do something in a way that is considered idiomatic in the language’.
For example, some APIs (in general) makes sense when they use C++ templates or Rust `impl` blocks; it doesn’t make sense to enforce type unsafety.
Some APIs makes sense when they are provided as Lisp macros or babel macros; it doesn’t make sense to enforce boilerplate code.

Have you ever done FFI between two high level languages, e.g. Swift and Rust? It’s a real pain, because the interfacing code can’t use any of the language features as they should be defined by their common dominators: the SysV ABI.
You can’t use generics, tuples, lambdas, etc... even though both languages have them.

It’s well, a somewhat similar experience.

> Thank you very much.
> 
>>        Stefan
> 
> -- 
> Best regards,
> Tadeus
> 

Re: Why is Elisp slow?

[Tadeus Prastowo]

On Mon, May 6, 2019 at 11:03 AM 조성빈 <pcr910303@icloud.com> wrote:
>
> 2019. 5. 6. 오후 4:33, Tadeus Prastowo <tadeus.prastowo@unitn.it> 작성:
>
> > On Sun, May 5, 2019 at 5:51 PM Stefan Monnier
> > <monnier@iro.umontreal.ca> wrote:> Language interoperation is very
> > difficult, and even more so if you want
> >> both languages to be usable "equally" (as opposed to a high-level
> >> language with FFI bindings, say).  The .NET platform aims to do just
> >> that, and note that it's fairly complex and the most successful
> >> languages there were specifically designed/tweaked for that platform.
> >
> > Out of curiosity, do you mind to elaborate on how it is to be usable
> > "equally"?  (I think the best way to understand it is by studying the
> > .NET platform, but maybe you could explain it better).
>
> I would say as ‘to do something in a way that is considered idiomatic in the language’.
> For example, some APIs (in general) makes sense when they use C++ templates or Rust `impl` blocks; it doesn’t make sense to enforce type unsafety.
> Some APIs makes sense when they are provided as Lisp macros or babel macros; it doesn’t make sense to enforce boilerplate code.
>
> Have you ever done FFI between two high level languages, e.g. Swift and Rust? It’s a real pain, because the interfacing code can’t use any of the language features as they should be defined by their common dominators: the SysV ABI.
> You can’t use generics, tuples, lambdas, etc... even though both languages have them.
>
> It’s well, a somewhat similar experience.

Okay, I got it now.  In that case, the difficulty of using both
languages "equally" is a natural problem that is found also in, for
example, the English vs. Chinese vs. other natural language vocabulary
mismatches.

Thank you very much for taking the time to explain the things through.

-- 
Best regards,
Tadeus

Re: Why is Elisp slow?

[Ergus]

On Sun, May 05, 2019 at 11:50:16AM -0400, Stefan Monnier wrote:
>> The more I think about Guile Emacs and the Guile project itself, I start to
>> think that it???s not a problem of manpower, but something more
>> fundamental.
>
>Quite right.
>
>> Is it feasible (or even possible) to make two different high
>> level languages to interoperate in the engine level?????
>
>Language interoperation is very difficult, and even more so if you want
>both languages to be usable "equally" (as opposed to a high-level
>language with FFI bindings, say).  The .NET platform aims to do just
>that, and note that it's fairly complex and the most successful
>languages there were specifically designed/tweaked for that platform.
>
It looks more complex because the starting point is a High level
language as Lisp so any port will need many data types conversion and
copies with the corresponding overheads (and ugly code).

All the infrastructure is based in the Lisp Objects, types and data
structures and functions. But IF the base API were designed with basic C
types (int, doubles, char*) and structs, and language agnostic, the
interface with Lua, Python or Ruby will be pretty simple (even Fortran,
or anything else).

That's what vim did and they now support many languages for extending
and create plugins. (I know it is a different system and I am not saying
we have to move in that direction right now.)

But, instead of create lisp object from C the only we need it to create
an interface for the C functions that need access from Lisp and use them
in the interactive or high level lisp specific functions. But the low
level functions will be all exposed in the C library and we could access
then even with dlopen in some cases.

Of course I am not saying that we should develop code inside emacs core
with all the different languages or change language every 10 years. For
me it is fine Elisp (or common lisp even better if that reduces the
complexity of maintaining our on compiler and potentially improves
performance). But to give the user the possibility to create plugins; a
C API is much more useful and simple. And if we want to port some of
those to elisp will be also easier. I specially don't share the choice
of developing core parts (API) in Lisp directly because having them in C
is faster, simpler, more accessible from outside if needed and usually
gives less overheads to the starting time, and GC.

We already have the API with the modules, but if you see most of the
complexity comes from creating and converting lisp types to something
else and do callbacks to lisp functions, and parsing the arguments.

Defun or any other macro could be emulated if we have to, but again,
that's just a complexity of making everything Lisp centric instead of C
centric. 

Bringing a C api and use Lisp for high level or interactive functions
reduces overheads, simplify the C code we have and enables a more
modular design. But I know that try to do that now is not realistic with
the amount of code and people we have. So that's why I was talking about
Guile, but after all it seems not to be realistic either.

So whats the alternative? Keep our interpreter as is and deal with few
developers (and decreasing) and performance issues and a lot of code to
maintain?




>> For example, will it even make sense to have some Elisp macros in scheme
>
>Interoperation between "similar" languages like Elisp and Scheme can
>probably be made to be tolerable (e.g. I guess most Emacs-specific
>special forms of Elisp, such as `save-excursion`, could be provided on
>the Scheme side without too much trouble (famous last words)), but
>macros seem pretty hard to handle in general (i.e. will require manual
>work on a macro-by-macro case).
>
>> land, or even lua land, python land, JS land?????
>
>For these, macros aren't the only problem, since there's also the
>problem of the different kinds of datatypes used.  What would JS objects
>look like in Elisp, and what would Elisp objects look like in JS?
>
>> I would like a JSish API for emacs when writing packages, not a low
>> level wrapper around Elisp APIs.
>
>Indeed.
>
>> What would I do to use a Python module in JS????? Unless the language itself is
>> designed to be easily interoperable with elisp by superseding concepts (such
>> as Clojure superseding Java), I???m pretty sure many people won???t be satisfied
>> in any kind of implementation.
>
>Exactly.
>
>
>        Stefan
>
>

Re: Why is Elisp slow?

[Stefan Monnier]

> That's what vim did and they now support many languages for extending
> and create plugins. (I know it is a different system and I am not saying
> we have to move in that direction right now.)

Yes, it's a very different design.  Elisp was designed to be the
*implementation* language of Emacs (the use of C for some of the
implementation was a necessary evil given the lack of wide availability
of Lisp), rather than just an *extension* language.


        Stefan

Re: Why is Elisp slow?

[조성빈]

(As I am not currently familiar with how the Emacs codebase is structured) How does Emacs do FFI currently? If I recall correctly aren’t there ways to call C functions in elisp? Would it be hard to expose C functions (by compiling to the SysV ABI) and calling them in elisp (to write plugins in another language)? If I understand the current architecture of Vim correctly, Vim is also doing that; it’s impossible to not use VimScript to make plugins, glue code is essential.

> 2019. 5. 6. 오후 9:58, Ergus <spacibba@aol.com> 작성:
> 
> On Sun, May 05, 2019 at 11:50:16AM -0400, Stefan Monnier wrote:
>>> The more I think about Guile Emacs and the Guile project itself, I start to
>>> think that it???s not a problem of manpower, but something more
>>> fundamental.
>> 
>> Quite right.
>> 
>>> Is it feasible (or even possible) to make two different high
>>> level languages to interoperate in the engine level?????
>> 
>> Language interoperation is very difficult, and even more so if you want
>> both languages to be usable "equally" (as opposed to a high-level
>> language with FFI bindings, say).  The .NET platform aims to do just
>> that, and note that it's fairly complex and the most successful
>> languages there were specifically designed/tweaked for that platform.
>> 
> It looks more complex because the starting point is a High level
> language as Lisp so any port will need many data types conversion and
> copies with the corresponding overheads (and ugly code).
> 
> All the infrastructure is based in the Lisp Objects, types and data
> structures and functions. But IF the base API were designed with basic C
> types (int, doubles, char*) and structs, and language agnostic, the
> interface with Lua, Python or Ruby will be pretty simple (even Fortran,
> or anything else).
> 
> That's what vim did and they now support many languages for extending
> and create plugins. (I know it is a different system and I am not saying
> we have to move in that direction right now.)
> 
> But, instead of create lisp object from C the only we need it to create
> an interface for the C functions that need access from Lisp and use them
> in the interactive or high level lisp specific functions. But the low
> level functions will be all exposed in the C library and we could access
> then even with dlopen in some cases.
> 
> Of course I am not saying that we should develop code inside emacs core
> with all the different languages or change language every 10 years. For
> me it is fine Elisp (or common lisp even better if that reduces the
> complexity of maintaining our on compiler and potentially improves
> performance). But to give the user the possibility to create plugins; a
> C API is much more useful and simple. And if we want to port some of
> those to elisp will be also easier. I specially don't share the choice
> of developing core parts (API) in Lisp directly because having them in C
> is faster, simpler, more accessible from outside if needed and usually
> gives less overheads to the starting time, and GC.
> 
> We already have the API with the modules, but if you see most of the
> complexity comes from creating and converting lisp types to something
> else and do callbacks to lisp functions, and parsing the arguments.
> 
> Defun or any other macro could be emulated if we have to, but again,
> that's just a complexity of making everything Lisp centric instead of C
> centric. 
> Bringing a C api and use Lisp for high level or interactive functions
> reduces overheads, simplify the C code we have and enables a more
> modular design. But I know that try to do that now is not realistic with
> the amount of code and people we have. So that's why I was talking about
> Guile, but after all it seems not to be realistic either.
> 
> So whats the alternative? Keep our interpreter as is and deal with few
> developers (and decreasing) and performance issues and a lot of code to
> maintain?
> 
> 
> 
> 
>>> For example, will it even make sense to have some Elisp macros in scheme
>> 
>> Interoperation between "similar" languages like Elisp and Scheme can
>> probably be made to be tolerable (e.g. I guess most Emacs-specific
>> special forms of Elisp, such as `save-excursion`, could be provided on
>> the Scheme side without too much trouble (famous last words)), but
>> macros seem pretty hard to handle in general (i.e. will require manual
>> work on a macro-by-macro case).
>> 
>>> land, or even lua land, python land, JS land?????
>> 
>> For these, macros aren't the only problem, since there's also the
>> problem of the different kinds of datatypes used.  What would JS objects
>> look like in Elisp, and what would Elisp objects look like in JS?
>> 
>>> I would like a JSish API for emacs when writing packages, not a low
>>> level wrapper around Elisp APIs.
>> 
>> Indeed.
>> 
>>> What would I do to use a Python module in JS????? Unless the language itself is
>>> designed to be easily interoperable with elisp by superseding concepts (such
>>> as Clojure superseding Java), I???m pretty sure many people won???t be satisfied
>>> in any kind of implementation.
>> 
>> Exactly.
>> 
>> 
>>       Stefan
>> 
>> 
> 

Re: Why is Elisp slow?

[Óscar Fuentes]

Ergus <spacibba@aol.com> writes:

> Bringing a C api and use Lisp for high level or interactive functions
> reduces overheads, simplify the C code we have and enables a more
> modular design. But I know that try to do that now is not realistic with
> the amount of code and people we have. So that's why I was talking about
> Guile, but after all it seems not to be realistic either.
>
> So whats the alternative? Keep our interpreter as is and deal with few
> developers (and decreasing) and performance issues and a lot of code to
> maintain?

Emacs doesn't suffer from scarcity of Elisp programmers. Emacs lacks C
programmers.

Maybe it is related to how easy it is to get something working on each
language. Elisp has a huge advantage over C on that aspect, not only
because Elisp is more simple and way more expressive, but also because
of the REPL (no edit-compile-run-crash hamster wheel) and because Emacs
is, in practice, a Lisp machine with the associated advantages.

Elisp is a hacker's dream. Not so C.

Re: Why is Elisp slow?

[조성빈]

> 2019. 5. 6. 오후 10:33, Óscar Fuentes <ofv@wanadoo.es> 작성:
> 
> Ergus <spacibba@aol.com> writes:
> 
>> Bringing a C api and use Lisp for high level or interactive functions
>> reduces overheads, simplify the C code we have and enables a more
>> modular design. But I know that try to do that now is not realistic with
>> the amount of code and people we have. So that's why I was talking about
>> Guile, but after all it seems not to be realistic either.
>> 
>> So whats the alternative? Keep our interpreter as is and deal with few
>> developers (and decreasing) and performance issues and a lot of code to
>> maintain?
> 
> Emacs doesn't suffer from scarcity of Elisp programmers. Emacs lacks C
> programmers.

1. Arguably this is because Elisp *is* the extension language of Emacs.
2. Arguable Emacs suffers from both. :-( I have yet to find a good flow (a static type checking variant of JS from facebook) mode. I have yet to find a good debugging package (dap-mode comes close). I have yet to find a good <insert arbitrary package here>. Most Emacs packages have an equivalent one (if possible0 in Atom, VSCode, etc… but there are *lots* of packages that lack in Melpa. 
3. 

> Maybe it is related to how easy it is to get something working on each
> language. Elisp has a huge advantage over C on that aspect, not only
> because Elisp is more simple and way more expressive, but also because
> of the REPL (no edit-compile-run-crash hamster wheel) and because Emacs
> is, in practice, a Lisp machine with the associated advantages.

Also, um… comparing Elisp with C for ‘easy’?̊̈ Maybe comparing with JS, Python, Scheme, CL would be better, and Elisp isn’t always superior. Elisp is more expressive and allows better abstractions than most ALGOL-derived languages, but esoteric function names and scoping rules that default to dynamic scoping, etc… CL is superior than Elisp in almost all viewpoints. Scheme also has superior points to Elisp.

I wouldn’t say that Elisp is the selling point of Emacs; why did Atom succeed much more than Emacs?̊̈
The reason Emacs succeeded is that Emacs allows tweaking the editor to the extreme; Atom was successful for similar reasons albeit using JS instead of Elisp.

> Elisp is a hacker's dream. Not so C.

Lisp was, and is a hacker’s language.
C++, Java is a corporate language. 
C was, and is a hacker’s language.

Re: Why is Elisp slow?

[Ergus]

On Mon, May 06, 2019 at 09:08:03AM -0400, Stefan Monnier wrote:
>> That's what vim did and they now support many languages for extending
>> and create plugins. (I know it is a different system and I am not saying
>> we have to move in that direction right now.)
>
>Yes, it's a very different design.  Elisp was designed to be the
>*implementation* language of Emacs (the use of C for some of the
>implementation was a necessary evil given the lack of wide availability
>of Lisp), rather than just an *extension* language.
>
>
>        Stefan
>
Yes, but then we need to put much more emphasis (and work) improving and
keep good performance in the compiler (which is a much more complex task
than maintaining the editor, and where very few changes are made these
days). Apply the new tools and techniques in compiler sciences and so on
requires a very high level of expertise. OR consider migrate to
something else that someone else maintain like SBCL or whatever the
managers consider better. (I know that only this proposal will open a
sacred war (as it did before (but the problems are the same)) but being
sincere is more valuable than being politically correct to have
progress)

Because many editors around can already do what emacs do (edit text, be
extensible, free, and portable), but they have faster interpreter and
language, better (more familiar or standardized) keybindings, bigger
communities, simpler (in spite of more limited or not) customization and
simpler code or "more pretty/intuitive" UI.

Lisp was the right option 40 years ago because it was probably the most
dynamic language and easier to understand than C for the computer users
that time. But now there are more dynamic and capable languages. And a
language interpreter is not so easy to maintain for a formal/big project
like emacs.

Summing up. In Spanish we say that whoever takes a lot of space, the
less he tightens up (bad translation: Quien mucho abarca poco aprieta).

IN MY OPINION, we should put more attention in the emacs editing
capabilities/functionalities/interface and rely the compiling
functionalities to a compiler we can (with limitations of course)
trust. Even if that means remove the E letter from our language. But
after that big change any compiler released in future will be easier to
use as a module and we could even give the option to the user to
chose.

(I know this sounds a bit heretic) but being a bit more DOTADIW will be
more scalable and maintainable now and in the future. And we can center
the manpower in update the gui, optimizations, cleanup
old/unused/unmaintained code, solve bugreports, standardize keybindings
and so on.

Re: Why is Elisp slow?

[Stefan Monnier]

I totally agree with the sentiment; we'd benefit from "out-sourcing" the
maintenance of the language implementation, but we can't just replace
Elisp with something else, so we need to find another well-maintained
Elisp implementation that's at least as good as that we have now.

AFAIK the options are:
- Keep what we have
- Move to Guile
- Move to CLISP
- Move to SBCL
CLISP is not actively maintained nowadays.
AFAIK SBCL is alive and kicking, but I'm not sure how well it works on
platforms other than GNU/Linux-on-AMD64.

Of course, Guile has the advantage that someone has already spent a fair
bit of time implementing support for Elisp, whereas for CLISP and SBCL
that would be extra work (Elisp is close to a subset of CL but not
quite).

Another approach would be to implement an Elisp-to-JS compiler and
then use one of the heavily-optimized JIT-compilers for JS.
Compiling Elisp to JS should be much easier than compiling to
native code.


        Stefan

Re: Why is Elisp slow?

[조성빈]

> 2019. 5. 7. 오전 2:25, Stefan Monnier <monnier@iro.umontreal.ca> 작성:
> 
> I totally agree with the sentiment; we'd benefit from "out-sourcing" the
> maintenance of the language implementation, but we can't just replace
> Elisp with something else, so we need to find another well-maintained
> Elisp implementation that's at least as good as that we have now.

I’m curious: How likely is this to happen? This would be truly great if happening :-) 

> AFAIK the options are:
> - Keep what we have
> - Move to Guile
> - Move to CLISP
> - Move to SBCL
> CLISP is not actively maintained nowadays.
> AFAIK SBCL is alive and kicking, but I'm not sure how well it works on
> platforms other than GNU/Linux-on-AMD64.

I can confirm that SBCL works great on at least these platforms:
* Arch Linux(GNU userspace) on AMD64 (my friend’s confirmations)
* macOS(Darwin) on AMD64 (I’m currently using macOS)
* Windows 10 on AMD64 (my another friend’s confirmations)
* Raspbian(GNU userspace) on ARM (at least when using SBCL about a year before…)

CCL is similar (actually provides more support on non-GNU platforms).

> Of course, Guile has the advantage that someone has already spent a fair
> bit of time implementing support for Elisp, whereas for CLISP and SBCL
> that would be extra work (Elisp is close to a subset of CL but not
> quite).

Would that extra work outweigh than implementing Guile’s language integration features?
Perhaps an insightful person may know?
May I ask what part of elisp makes implementing in CL hard?

> Another approach would be to implement an Elisp-to-JS compiler and
> then use one of the heavily-optimized JIT-compilers for JS.
> Compiling Elisp to JS should be much easier than compiling to
> native code.

If possible, this would be more than great as we would be able to use the *big* number of JS packages in npm registry out there.

> 
>        Stefan
> 

Re: Why is Elisp slow?

[Stefan Monnier]

> I’m curious: How likely is this to happen?

As long as noone works on it, I'd say 0% likelihood.

> I can confirm that SBCL works great on at least these platforms:
> * Arch Linux(GNU userspace) on AMD64 (my friend’s confirmations)
> * macOS(Darwin) on AMD64 (I’m currently using macOS)
> * Windows 10 on AMD64 (my another friend’s confirmations)
> * Raspbian(GNU userspace) on ARM (at least when using SBCL about a year before…)

[ IIUC of the 4 cases above, at most 2 run the same version, so we'd
  need to make sure the same Emacs version can be compiled against all
  of those versions.  No idea if it would impose a significant extra
  burden or not, but it's something to be considered.  Also the fact
  that the latest release doesn't work on all those platforms is rather
  worrying.  ]

>> Of course, Guile has the advantage that someone has already spent a fair
>> bit of time implementing support for Elisp, whereas for CLISP and SBCL
>> that would be extra work (Elisp is close to a subset of CL but not
>> quite).
> Would that extra work outweigh than implementing Guile’s language
> integration features?

No idea.

> May I ask what part of elisp makes implementing in CL hard?

Haven't thought too much about it, so I don't even know if it would be
hard (the elisp.lisp implementation I mentioned recently shows that
large parts can be done easily enough).
The obvious issue is buffer-local and terminal-local variables.

>> Another approach would be to implement an Elisp-to-JS compiler and
>> then use one of the heavily-optimized JIT-compilers for JS.
>> Compiling Elisp to JS should be much easier than compiling to
>> native code.
>
> If possible, this would be more than great as we would be able to use the
> *big* number of JS packages in npm registry out there.

Note that compiling to JS doesn't *directly* let you access random JS
data-structures and functions any more than implementing Elisp in C lets
you access random C functions and data-structures.
[ language-interoperation, again.  ]


        Stefan

Re: Why is Elisp slow?

[조성빈]

> 2019. 5. 7. 오전 3:08, Stefan Monnier <monnier@iro.umontreal.ca> 작성:
> 
>> I’m curious: How likely is this to happen?
> 
> As long as noone works on it, I'd say 0% likelihood.
> 
>> I can confirm that SBCL works great on at least these platforms:
>> * Arch Linux(GNU userspace) on AMD64 (my friend’s confirmations)
>> * macOS(Darwin) on AMD64 (I’m currently using macOS)
>> * Windows 10 on AMD64 (my another friend’s confirmations)
>> * Raspbian(GNU userspace) on ARM (at least when using SBCL about a year before…)
> 
> [ IIUC of the 4 cases above, at most 2 run the same version, so we'd
>  need to make sure the same Emacs version can be compiled against all
>  of those versions.  No idea if it would impose a significant extra
>  burden or not, but it's something to be considered.  Also the fact
>  that the latest release doesn't work on all those platforms is rather
>  worrying.  ]

Hmm…? I can’t understand :-(
Why can’t Emacs can include a specific version of SBCL’s source (e.g. as a git module) and compile them all together?̊̈
I’m pretty sure SBCL’s platform-specific code is self-contained.

>>> Of course, Guile has the advantage that someone has already spent a fair
>>> bit of time implementing support for Elisp, whereas for CLISP and SBCL
>>> that would be extra work (Elisp is close to a subset of CL but not
>>> quite).
>> Would that extra work outweigh than implementing Guile’s language
>> integration features?
> 
> No idea.
> 
>> May I ask what part of elisp makes implementing in CL hard?
> 
> Haven't thought too much about it, so I don't even know if it would be
> hard (the elisp.lisp implementation I mentioned recently shows that
> large parts can be done easily enough).
> The obvious issue is buffer-local and terminal-local variables.
> 
>>> Another approach would be to implement an Elisp-to-JS compiler and
>>> then use one of the heavily-optimized JIT-compilers for JS.
>>> Compiling Elisp to JS should be much easier than compiling to
>>> native code.
>> 
>> If possible, this would be more than great as we would be able to use the
>> *big* number of JS packages in npm registry out there.
> 
> Note that compiling to JS doesn't *directly* let you access random JS
> data-structures and functions any more than implementing Elisp in C lets
> you access random C functions and data-structures.
> [ language-interoperation, again.  ]

Ah, I was saying libraries in general; libraries like immutable.js or left-pad :-)

> 
>        Stefan

Re: Why is Elisp slow?

[Stefan Monnier]

>> [ IIUC of the 4 cases above, at most 2 run the same version, so we'd
>>  need to make sure the same Emacs version can be compiled against all
>>  of those versions.  No idea if it would impose a significant extra
>>  burden or not, but it's something to be considered.  Also the fact
>>  that the latest release doesn't work on all those platforms is rather
>>  worrying.  ]
>
> Hmm…? I can’t understand :-(
> Why can’t Emacs can include a specific version of SBCL’s source (e.g. as
> a git module) and compile them all together?̊̈

Exactly because the version that can run on ARM is not the same as the
one that can run under Windows, which is not the same as the one that
runs under AMD64.

Of course, maybe I'm confused by the table of available binaries (at
http://sbcl.org/platform-table.html), and in reality the latest version
works fine on all supported platforms.


        Stefan

Re: Why is Elisp slow?

[Ergus]

On May 6, 2019 8:47:12 PM GMT+02:00, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>> [ IIUC of the 4 cases above, at most 2 run the same version, so we'd
>>>  need to make sure the same Emacs version can be compiled against
>all
>>>  of those versions.  No idea if it would impose a significant extra
>>>  burden or not, but it's something to be considered.  Also the fact
>>>  that the latest release doesn't work on all those platforms is
>rather
>>>  worrying.  ]
>>
>> Hmm…? I can’t understand :-(
>> Why can’t Emacs can include a specific version of SBCL’s source (e.g.
>as
>> a git module) and compile them all together?̊̈
>
>Exactly because the version that can run on ARM is not the same as the
>one that can run under Windows, which is not the same as the one that
>runs under AMD64.
>
>Of course, maybe I'm confused by the table of available binaries (at
>http://sbcl.org/platform-table.html), and in reality the latest version
>works fine on all supported platforms.
>
>
>        Stefan

What about just ask to sbcl maintainers about portability? On the other hand, if Emacs doesn't use extremely fancy stuff in the language we could just rely on the system's sbcl version (available in the distro), for gnu/linux systems that will be pretty fine and of course that's something that needs specific tests. But nothing compared to maintain a compiler. 

Stefan if you need to test sbcl in some specific (weird) architecture just send/tell me the tests and the architectured you need to test. (Except windows)  and I can do that.

Btw. Should we move this thread to the developers list? (and start the usual war)

Please don't go for JS stuff... Please... performance is terrible even with jit.

Re: Why is Elisp slow?

[조성빈]

> 2019. 5. 7. 오전 5:23, Ergus <spacibba@aol.com> 작성:
> 
> 
> 
> On May 6, 2019 8:47:12 PM GMT+02:00, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>>> [ IIUC of the 4 cases above, at most 2 run the same version, so we'd
>>>> need to make sure the same Emacs version can be compiled against
>> all
>>>> of those versions.  No idea if it would impose a significant extra
>>>> burden or not, but it's something to be considered.  Also the fact
>>>> that the latest release doesn't work on all those platforms is
>> rather
>>>> worrying.  ]
>>> 
>>> Hmm…? I can’t understand :-(
>>> Why can’t Emacs can include a specific version of SBCL’s source (e.g.
>> as
>>> a git module) and compile them all together?̊̈
>> 
>> Exactly because the version that can run on ARM is not the same as the
>> one that can run under Windows, which is not the same as the one that
>> runs under AMD64.
>> 
>> Of course, maybe I'm confused by the table of available binaries (at
>> http://sbcl.org/platform-table.html), and in reality the latest version
>> works fine on all supported platforms.
>> 
>> 
>>       Stefan
> 
> What about just ask to sbcl maintainers about portability? On the other hand, if Emacs doesn't use extremely fancy stuff in the language we could just rely on the system's sbcl version (available in the distro), for gnu/linux systems that will be pretty fine and of course that's something that needs specific tests. But nothing compared to maintain a compiler. 
> 
> Stefan if you need to test sbcl in some specific (weird) architecture just send/tell me the tests and the architectured you need to test. (Except windows)  and I can do that.
> 
> Btw. Should we move this thread to the developers list? (and start the usual war)
> 
> Please don't go for JS stuff... Please... performance is terrible even with jit.
> 

Um… JS’s performance is pretty good, almost comparable with static languages like Java and C.
I remember a better benchmark, but I couldn’t find that, but look at this: https://attractivechaos.github.io/plb/ <https://attractivechaos.github.io/plb/>
Javascript V8 engine is almost always in about x2 ~ x3 of C.
Compare that to Elisp….uh…which is about comparable with python maybe?

Re: Why is Elisp slow?

[Óscar Fuentes]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> I totally agree with the sentiment; we'd benefit from "out-sourcing" the
> maintenance of the language implementation, but we can't just replace
> Elisp with something else, so we need to find another well-maintained
> Elisp implementation that's at least as good as that we have now.
>
> AFAIK the options are:
> - Keep what we have
> - Move to Guile
> - Move to CLISP
> - Move to SBCL

- Pick an Elisp subset that can be compiled to efficient machine code
  and has a runtime model that is amenable to C. You retain much of the
  benefits of Elisp but gain fast execution and the possibility of using
  a miriad of libraries. Of course, do not replace Elisp with this, use
  it just when it matters.
  
There are several C-with-a-Lisp-coat implementations out there. I have
one (not public) and know that this method works.

> Another approach would be to implement an Elisp-to-JS compiler and
> then use one of the heavily-optimized JIT-compilers for JS.
> Compiling Elisp to JS should be much easier than compiling to
> native code.

Javascript's JITs seem fast because the interpreters are so slow. Those
JITs only provide near-C code efficiency only on selected cases.
Translating Elisp to JS does not inspire much confidence about the
efficiency of the resulting machine code, not to mention the required
work to interface with current Emacs C base and external libraries.

Re: Why is Elisp slow?

[Jean-Christophe Helary]

> On May 7, 2019, at 3:47, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
> Of course, maybe I'm confused by the table of available binaries (at
> http://sbcl.org/platform-table.html), and in reality the latest version
> works fine on all supported platforms.

It looks like the table is not up to date. Version 1.5.2 works fine on the latest version of macOS (intel). I installed it with brew.

Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: Why is Elisp slow?

[Ergus]

Hi again:

Sorry for my ignorance. Syntactically speaking which are the key differences between CL and Elisp you think may produce the bigger issues?


On May 6, 2019 8:47:12 PM GMT+02:00, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>> [ IIUC of the 4 cases above, at most 2 run the same version, so we'd
>>>  need to make sure the same Emacs version can be compiled against
>all
>>>  of those versions.  No idea if it would impose a significant extra
>>>  burden or not, but it's something to be considered.  Also the fact
>>>  that the latest release doesn't work on all those platforms is
>rather
>>>  worrying.  ]
>>
>> Hmm…? I can’t understand :-(
>> Why can’t Emacs can include a specific version of SBCL’s source (e.g.
>as
>> a git module) and compile them all together?̊̈
>
>Exactly because the version that can run on ARM is not the same as the
>one that can run under Windows, which is not the same as the one that
>runs under AMD64.
>
>Of course, maybe I'm confused by the table of available binaries (at
>http://sbcl.org/platform-table.html), and in reality the latest version
>works fine on all supported platforms.
>
>
>        Stefan

-- 
Enviado desde mi dispositivo Android con K-9 Mail. Por favor, disculpa mi brevedad.

Re: Why is Elisp slow?

[Ergus]

Hi again:

Sorry for my ignorance. Syntactically speaking which are the key differences between CL and Elisp you think may produce the bigger issues?


On May 6, 2019 8:47:12 PM GMT+02:00, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>>> [ IIUC of the 4 cases above, at most 2 run the same version, so we'd
>>>  need to make sure the same Emacs version can be compiled against
>all
>>>  of those versions.  No idea if it would impose a significant extra
>>>  burden or not, but it's something to be considered.  Also the fact
>>>  that the latest release doesn't work on all those platforms is
>rather
>>>  worrying.  ]
>>
>> Hmm…? I can’t understand :-(
>> Why can’t Emacs can include a specific version of SBCL’s source (e.g.
>as
>> a git module) and compile them all together?̊̈
>
>Exactly because the version that can run on ARM is not the same as the
>one that can run under Windows, which is not the same as the one that
>runs under AMD64.
>
>Of course, maybe I'm confused by the table of available binaries (at
>http://sbcl.org/platform-table.html), and in reality the latest version
>works fine on all supported platforms.
>
>
>        Stefan

-- 
Enviado desde mi dispositivo Android con K-9 Mail. Por favor, disculpa mi brevedad.
-- 
Enviado desde mi dispositivo Android con K-9 Mail. Por favor, disculpa mi brevedad.

Re: Why is Elisp slow?

[Emanuel Berg]

Óscar Fuentes wrote:

> Emacs doesn't suffer from scarcity of Elisp
> programmers. Emacs lacks C programmers.

What more exactly would those
Emacs C programmers do, if they existed and
were "motivated"?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[조성빈]

> 2019. 5. 7. 오전 5:51, Óscar Fuentes <ofv@wanadoo.es> 작성:
> 
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
>> I totally agree with the sentiment; we'd benefit from "out-sourcing" the
>> maintenance of the language implementation, but we can't just replace
>> Elisp with something else, so we need to find another well-maintained
>> Elisp implementation that's at least as good as that we have now.
>> 
>> AFAIK the options are:
>> - Keep what we have
>> - Move to Guile
>> - Move to CLISP
>> - Move to SBCL
> 
> - Pick an Elisp subset that can be compiled to efficient machine code
>  and has a runtime model that is amenable to C. You retain much of the
>  benefits of Elisp but gain fast execution and the possibility of using
>  a miriad of libraries. Of course, do not replace Elisp with this, use
>  it just when it matters.
> 
> There are several C-with-a-Lisp-coat implementations out there. I have
> one (not public) and know that this method works.

Does this have a solution to interfacing the Elisp subset translated to C and usual Elisp?
It feels like there’s almost no gains compared to using Guile, except that it would be marginally faster...

>> Another approach would be to implement an Elisp-to-JS compiler and
>> then use one of the heavily-optimized JIT-compilers for JS.
>> Compiling Elisp to JS should be much easier than compiling to
>> native code.
> 
> Javascript's JITs seem fast because the interpreters are so slow. Those
> JITs only provide near-C code efficiency only on selected cases.
> Translating Elisp to JS does not inspire much confidence about the
> efficiency of the resulting machine code, not to mention the required
> work to interface with current Emacs C base and external libraries.

Yeah, and the slow interpreters are what Emacs has in itself.

Re: Why is Elisp slow?

[Ergus]

Hi all:

After reading the whole SBCL manual and the internals I have some
comments maybe useful if we decide to go in that direction. (If some of
the core developers starts with this I will be very pleased to work on
that, but I am too new in emacs development to start it myself alone.)

1) + Portability does not seems to be a big issue, they already support
 our architectures (and some extra) and they are putting a lot of effort
 on that. There is work to do, but in general, they already have what we
 need from Common Lisp.

2) + They have documented how to port SBCL to new architectures, and some
of the optimizations they implement, so, if we have a compiler
specialist on board and finally our developers choose not to go for
SBCL, then maybe some of those optimization could be considered for our
Elisp compiler.

3) - There is not C binding for SBCL. To extend it, there is not a
public-documented API and their approach is wrapping the C functions
from Lisp (FFI). I wrote in their mailing list and they redirected me to
the CFFI page.

    + The only advantage (I see) in this approach is that as CFFI has
    been ported to almost all common list compilers/interpreters; if we
    go in that direction for our C code we could potentially use any of
    those and jump from one to another at configuration time. So we
    don't have this problem (of being stock with a "bad" compiler)
    anymore.

    - The overhead produced by wrapping in the high level instead of the
      low level in my experience (python, ruby, Julia) is usually much
      higher than having C bindings. 

    - Writing wrappers in lisp for all our C functions exposed to Lisp I
      think is not an option if we don't create an automatic method. So
      if no one creates a plugin for sbcl to support Elisp, I really
      doubt the feasibility of using SBCL except if many of our current
      core developers agree on that and join effort for that. Else it
      will happen as with guile.

4) + With FFI as they implemented, it will be pretty easy to interface
and call C libraries and functions without needing a C wrapper to expose
it to lisp, so maybe an important part of our C code could be
substituted with Lisp (simpler and smaller) glue code. But also we will
be allowed to use C dynamic libraries directly without all the modules
complexity. (Somehow I feel that not everyone will like that if we
comment it in the developers mailing list :p ).



On Mon, May 06, 2019 at 02:47:12PM -0400, Stefan Monnier wrote:
>>> [ IIUC of the 4 cases above, at most 2 run the same version, so we'd
>>>  need to make sure the same Emacs version can be compiled against all
>>>  of those versions.  No idea if it would impose a significant extra
>>>  burden or not, but it's something to be considered.  Also the fact
>>>  that the latest release doesn't work on all those platforms is rather
>>>  worrying.  ]
>>
>> Hmm???? I can???t understand :-(
>> Why can???t Emacs can include a specific version of SBCL???s source (e.g. as
>> a git module) and compile them all together?????
>
>Exactly because the version that can run on ARM is not the same as the
>one that can run under Windows, which is not the same as the one that
>runs under AMD64.
>
>Of course, maybe I'm confused by the table of available binaries (at
>http://sbcl.org/platform-table.html), and in reality the latest version
>works fine on all supported platforms.
>
>
>        Stefan
>
>

Re: Why is Elisp slow?

[조성빈]

> 2019. 5. 7. 오후 7:49, Ergus <spacibba@aol.com> 작성:
> 
> Hi all:
> 
> After reading the whole SBCL manual and the internals I have some
> comments maybe useful if we decide to go in that direction. (If some of
> the core developers starts with this I will be very pleased to work on
> that, but I am too new in emacs development to start it myself alone.)
> 
> 1) + Portability does not seems to be a big issue, they already support
> our architectures (and some extra) and they are putting a lot of effort
> on that. There is work to do, but in general, they already have what we
> need from Common Lisp.
> 
> 2) + They have documented how to port SBCL to new architectures, and some
> of the optimizations they implement, so, if we have a compiler
> specialist on board and finally our developers choose not to go for
> SBCL, then maybe some of those optimization could be considered for our
> Elisp compiler.
> 
> 3) - There is not C binding for SBCL. To extend it, there is not a
> public-documented API and their approach is wrapping the C functions
> from Lisp (FFI). I wrote in their mailing list and they redirected me to
> the CFFI page.
> 
>   + The only advantage (I see) in this approach is that as CFFI has
>   been ported to almost all common list compilers/interpreters; if we
>   go in that direction for our C code we could potentially use any of
>   those and jump from one to another at configuration time. So we
>   don't have this problem (of being stock with a "bad" compiler)
>   anymore.
>   
>   - The overhead produced by wrapping in the high level instead of the
>     low level in my experience (python, ruby, Julia) is usually much
>     higher than having C bindings. 
>   - Writing wrappers in lisp for all our C functions exposed to Lisp I
>     think is not an option if we don't create an automatic method. So
>     if no one creates a plugin for sbcl to support Elisp, I really
>     doubt the feasibility of using SBCL except if many of our current
>     core developers agree on that and join effort for that. Else it
>     will happen as with guile.

There is an automatic method of generating C FFI bindings from header files.
See https://github.com/rpav/cl-autowrap <https://github.com/rpav/cl-autowrap> :-)
What I was thinking about using CL to support Elisp is to define a new
namespace for symbols (which, in CL terms, is a so-called ‘package’) named ‘elisp’.
All Cl-needed functions/macros are defined in the ‘cl’ package,
which means that you can call any cl function/macro by
prefixing ‘cl:’ in the ‘elisp’ package.
As CL and Elisp has many common things (such as the similarity of
many, many macros like ‘defun’ or ‘defvar’, etc… or that it is a Lisp-2),
we can implement elisp entirely inside CL itself.
No need for developing a plugin specially for SBCL; any CL implementation will do.
For functions defined in C, we can make or generate wrappers to get
passed-in-arguments and dynamically wrap values into a Lisp_Obj C struct
(AFAIU all lisp objects are represented as a struct of type info and some pointers,
 am I right?) and call the underlying function generated by cl-autowrap.

> 4) + With FFI as they implemented, it will be pretty easy to interface
> and call C libraries and functions without needing a C wrapper to expose
> it to lisp, so maybe an important part of our C code could be
> substituted with Lisp (simpler and smaller) glue code. But also we will
> be allowed to use C dynamic libraries directly without all the modules
> complexity. (Somehow I feel that not everyone will like that if we
> comment it in the developers mailing list :p ).
> 
> On Mon, May 06, 2019 at 02:47:12PM -0400, Stefan Monnier wrote:
>>>> [ IIUC of the 4 cases above, at most 2 run the same version, so we'd
>>>> need to make sure the same Emacs version can be compiled against all
>>>> of those versions.  No idea if it would impose a significant extra
>>>> burden or not, but it's something to be considered.  Also the fact
>>>> that the latest release doesn't work on all those platforms is rather
>>>> worrying.  ]
>>> 
>>> Hmm? I can't understand :-(
>>> Why can't Emacs can include a specific version of SBCL's source (e.g. as
>>> a git module) and compile them all together?
>> 
>> Exactly because the version that can run on ARM is not the same as the
>> one that can run under Windows, which is not the same as the one that
>> runs under AMD64.
>> 
>> Of course, maybe I'm confused by the table of available binaries (at
>> http://sbcl.org/platform-table.html), and in reality the latest version
>> works fine on all supported platforms.
>> 
>>       Stefan
>> 
> 

Re: Why is Elisp slow?

[Ergus]

On Tue, May 07, 2019 at 08:51:27PM +0900, ????????? wrote:
>
>
>> 2019. 5. 7. ?????? 7:49, Ergus <spacibba@aol.com> ??????:
>>
>> Hi all:
>>
>> After reading the whole SBCL manual and the internals I have some
>> comments maybe useful if we decide to go in that direction. (If some of
>> the core developers starts with this I will be very pleased to work on
>> that, but I am too new in emacs development to start it myself alone.)
>>
>> 1) + Portability does not seems to be a big issue, they already support
>> our architectures (and some extra) and they are putting a lot of effort
>> on that. There is work to do, but in general, they already have what we
>> need from Common Lisp.
>>
>> 2) + They have documented how to port SBCL to new architectures, and some
>> of the optimizations they implement, so, if we have a compiler
>> specialist on board and finally our developers choose not to go for
>> SBCL, then maybe some of those optimization could be considered for our
>> Elisp compiler.
>>
>> 3) - There is not C binding for SBCL. To extend it, there is not a
>> public-documented API and their approach is wrapping the C functions
>> from Lisp (FFI). I wrote in their mailing list and they redirected me to
>> the CFFI page.
>>
>>   + The only advantage (I see) in this approach is that as CFFI has
>>   been ported to almost all common list compilers/interpreters; if we
>>   go in that direction for our C code we could potentially use any of
>>   those and jump from one to another at configuration time. So we
>>   don't have this problem (of being stock with a "bad" compiler)
>>   anymore.
>>
>>   - The overhead produced by wrapping in the high level instead of the
>>     low level in my experience (python, ruby, Julia) is usually much
>>     higher than having C bindings.
>>   - Writing wrappers in lisp for all our C functions exposed to Lisp I
>>     think is not an option if we don't create an automatic method. So
>>     if no one creates a plugin for sbcl to support Elisp, I really
>>     doubt the feasibility of using SBCL except if many of our current
>>     core developers agree on that and join effort for that. Else it
>>     will happen as with guile.
>
>There is an automatic method of generating C FFI bindings from header files.
>See https://github.com/rpav/cl-autowrap <https://github.com/rpav/cl-autowrap> :-)
>What I was thinking about using CL to support Elisp is to define a new
>namespace for symbols (which, in CL terms, is a so-called ???package???) named ???elisp???.
>All Cl-needed functions/macros are defined in the ???cl??? package,
>which means that you can call any cl function/macro by
>prefixing ???cl:??? in the ???elisp??? package.

This is what I understand as a plugin for SBCL. Because we will need
some steroid for that :p 

>As CL and Elisp has many common things (such as the similarity of
>many, many macros like ???defun??? or ???defvar???, etc??? or that it is a Lisp-2),
>we can implement elisp entirely inside CL itself.
>No need for developing a plugin specially for SBCL; any CL implementation will do.
>For functions defined in C, we can make or generate wrappers to get
>passed-in-arguments and dynamically wrap values into a Lisp_Obj C struct
>(AFAIU all lisp objects are represented as a struct of type info and some pointers,
> am I right?) and call the underlying function generated by cl-autowrap.
>
This is so huge that I don't know where to start looking :(. What I was
looking was a representation of the types in SBCL, but I couldn't find
that.

AFAIK cl-autowrap is to wrap simple C functions with C types and
structs, but Elisp actually has it's own representation of lisp objects
and functions. And the exposed functions already have a unified function
format to store the pointer in the function's hash table. If we use a
wrapper on the hight level we will be packing and unpacking structs
(copies and more copies) constantly and wasting the advantage of having
a similar representation. Or we won't be allowed to pass Elisp object to
CL functions.

So the first to do should be to unify the Elisp types with the CL
representation as much as possible and check the differences we can't
solve implicitly to create an explicit conversion when needed. Because
with guile, after doing the hard work, we found that there was a problem
with strings representations (if I understood right what Stefan
explained)

We also need to find a method to create lisp objects from C to implement
DEFVAR_LISP, DEFSYM and similes in C if we don't want to reimplement big
portions of code (like the display engine for example). There is still
the problem with buffer local variable that we don't have a solution for
that yet either.

Then we can start checking the functions we have that are Elisp agnostic
and can be compiled with SBCL. We could also have another set of
functions we have implemented in C we will not need anymore and finally
there will be functions we need to wrap (like everything in editfns.c
for example)... But again... this is huge.
>
>> 4) + With FFI as they implemented, it will be pretty easy to interface
>> and call C libraries and functions without needing a C wrapper to expose
>> it to lisp, so maybe an important part of our C code could be
>> substituted with Lisp (simpler and smaller) glue code. But also we will
>> be allowed to use C dynamic libraries directly without all the modules
>> complexity. (Somehow I feel that not everyone will like that if we
>> comment it in the developers mailing list :p ).
>>
>> On Mon, May 06, 2019 at 02:47:12PM -0400, Stefan Monnier wrote:
>>>>> [ IIUC of the 4 cases above, at most 2 run the same version, so we'd
>>>>> need to make sure the same Emacs version can be compiled against all
>>>>> of those versions.  No idea if it would impose a significant extra
>>>>> burden or not, but it's something to be considered.  Also the fact
>>>>> that the latest release doesn't work on all those platforms is rather
>>>>> worrying.  ]
>>>>
>>>> Hmm? I can't understand :-(
>>>> Why can't Emacs can include a specific version of SBCL's source (e.g. as
>>>> a git module) and compile them all together?
>>>
>>> Exactly because the version that can run on ARM is not the same as the
>>> one that can run under Windows, which is not the same as the one that
>>> runs under AMD64.
>>>
>>> Of course, maybe I'm confused by the table of available binaries (at
>>> http://sbcl.org/platform-table.html), and in reality the latest version
>>> works fine on all supported platforms.
>>>
>>>       Stefan
>>>
>>
>

Re: Why is Elisp slow?

[Stefan Monnier]

>>   - Writing wrappers in lisp for all our C functions exposed to Lisp I

All those are defined with a "DEFUN" macro on the C side.
Whatever change is needed on this side can likely be made largely
mechanically, so I'm not worried.

Similarly, you'll need to rewrite all the functions/macros like CONSP,
SYMBOLP, FIXNUMP, XCAR, XCDR, make_fixnum, ...  Performance of those
is important.

> What I was thinking about using CL to support Elisp is to define a new
> namespace for symbols (which, in CL terms, is a so-called ‘package’)
> named ‘elisp’.

As I already mentioned, this already exists: elisp.lisp.


        Stefan

Re: Why is Elisp slow?

[조성빈]

> 2019. 5. 7. 오후 9:56, Stefan Monnier <monnier@IRO.UMontreal.CA> 작성:
> 
>>>  - Writing wrappers in lisp for all our C functions exposed to Lisp I
> 
> All those are defined with a "DEFUN" macro on the C side.
> Whatever change is needed on this side can likely be made largely
> mechanically, so I'm not worried.
> 
> Similarly, you'll need to rewrite all the functions/macros like CONSP,
> SYMBOLP, FIXNUMP, XCAR, XCDR, make_fixnum, ...  Performance of those
> is important.

Why would you not use the default CL’s defun, car, cdr, symbol-p, cons-p, etc, etc?

>> What I was thinking about using CL to support Elisp is to define a new
>> namespace for symbols (which, in CL terms, is a so-called ‘package’)
>> named ‘elisp’.
> 
> As I already mentioned, this already exists: elisp.lisp.
> 
> 
>        Stefan
> 

Re: Why is Elisp slow?

[Stefan Monnier]

>> Similarly, you'll need to rewrite all the functions/macros like CONSP,
>> SYMBOLP, FIXNUMP, XCAR, XCDR, make_fixnum, ...  Performance of those
>> is important.
>
> Why would you not use the default CL’s defun, car, cdr, symbol-p,
> cons-p, etc, etc?

I'm talking the work needed to adapt Emacs's C code, e.g:

    DEFUN ("get-buffer-window", Fget_buffer_window, Sget_buffer_window, 0, 2, 0,
           doc: /* Return a window currently displaying BUFFER-OR-NAME, or nil if none.
    BUFFER-OR-NAME may be a buffer or a buffer name and defaults to
    the current buffer.
    
    The optional argument ALL-FRAMES specifies the frames to consider:
    
    - t means consider all windows on all existing frames.
    
    - `visible' means consider all windows on all visible frames.
    
    - 0 (the number zero) means consider all windows on all visible
        and iconified frames.
    
    - A frame means consider all windows on that frame only.
    
    Any other value of ALL-FRAMES means consider all windows on the
    selected frame and no others.  */)
         (Lisp_Object buffer_or_name, Lisp_Object all_frames)
    {
      Lisp_Object buffer;
    
      if (NILP (buffer_or_name))
        buffer = Fcurrent_buffer ();
      else
        buffer = Fget_buffer (buffer_or_name);
    
      if (BUFFERP (buffer))
        return window_loop (GET_BUFFER_WINDOW, buffer, true, all_frames);
      else
        return Qnil;
    }


-- Stefan

Re: Why is Elisp slow?

[Ergus]

On Tue, May 07, 2019 at 09:04:31AM -0400, Stefan Monnier wrote:
>>> Similarly, you'll need to rewrite all the functions/macros like CONSP,
>>> SYMBOLP, FIXNUMP, XCAR, XCDR, make_fixnum, ...  Performance of those
>>> is important.
>>
>> Why would you not use the default CL???s defun, car, cdr, symbol-p,
>> cons-p, etc, etc?
>
>I'm talking the work needed to adapt Emacs's C code, e.g:
>
>    DEFUN ("get-buffer-window", Fget_buffer_window, Sget_buffer_window, 0, 2, 0,
>           doc: /* Return a window currently displaying BUFFER-OR-NAME, or nil if none.
>    BUFFER-OR-NAME may be a buffer or a buffer name and defaults to
>    the current buffer.
>
>    The optional argument ALL-FRAMES specifies the frames to consider:
>
>    - t means consider all windows on all existing frames.
>
>    - `visible' means consider all windows on all visible frames.
>
>    - 0 (the number zero) means consider all windows on all visible
>        and iconified frames.
>
>    - A frame means consider all windows on that frame only.
>
>    Any other value of ALL-FRAMES means consider all windows on the
>    selected frame and no others.  */)
>         (Lisp_Object buffer_or_name, Lisp_Object all_frames)
>    {
>      Lisp_Object buffer;
>
>      if (NILP (buffer_or_name))
>        buffer = Fcurrent_buffer ();
>      else
>        buffer = Fget_buffer (buffer_or_name);
>
>      if (BUFFERP (buffer))
>        return window_loop (GET_BUFFER_WINDOW, buffer, true, all_frames);
>      else
>        return Qnil;
>    }
>
>
>-- Stefan
>
That's why I was wondering about the C binds and the C types
representations in SBCL more than the lisp part of the implementation.

Re: Why is Elisp slow?

[조성빈]

Ahh… I now understood what’s the problem. :-(

> 2019. 5. 7. 오후 10:04, Stefan Monnier <monnier@IRO.UMontreal.CA> 작성:
> 
>>> Similarly, you'll need to rewrite all the functions/macros like CONSP,
>>> SYMBOLP, FIXNUMP, XCAR, XCDR, make_fixnum, ...  Performance of those
>>> is important.
>> 
>> Why would you not use the default CL’s defun, car, cdr, symbol-p,
>> cons-p, etc, etc?
> 
> I'm talking the work needed to adapt Emacs's C code, e.g:
> 
>    DEFUN ("get-buffer-window", Fget_buffer_window, Sget_buffer_window, 0, 2, 0,
>           doc: /* Return a window currently displaying BUFFER-OR-NAME, or nil if none.
>    BUFFER-OR-NAME may be a buffer or a buffer name and defaults to
>    the current buffer.
> 
>    The optional argument ALL-FRAMES specifies the frames to consider:
> 
>    - t means consider all windows on all existing frames.
> 
>    - `visible' means consider all windows on all visible frames.
> 
>    - 0 (the number zero) means consider all windows on all visible
>        and iconified frames.
> 
>    - A frame means consider all windows on that frame only.
> 
>    Any other value of ALL-FRAMES means consider all windows on the
>    selected frame and no others.  */)
>         (Lisp_Object buffer_or_name, Lisp_Object all_frames)
>    {
>      Lisp_Object buffer;
> 
>      if (NILP (buffer_or_name))
>        buffer = Fcurrent_buffer ();
>      else
>        buffer = Fget_buffer (buffer_or_name);
> 
>      if (BUFFERP (buffer))
>        return window_loop (GET_BUFFER_WINDOW, buffer, true, all_frames);
>      else
>        return Qnil;
>    }
> 
> 
> -- Stefan
> 

Re: Why is Elisp slow?

[Ergus]

On Tue, May 07, 2019 at 10:16:42PM +0900, ????????? wrote:

Yes, it is not straight forward, but that's what makes it more
interesting (and useful).

I still think that provide a plugin for SBCL is the best to do it native
and efficient, but I have zero knowledge about the SBCL infrastructure
for that. But the idea of adding it some C bindings is not
crazy. Specially because all the functions have the same signature. We
just need some documentation about the internal data structure
representations in SBCL.

>Ahh??? I now understood what???s the problem. :-(
>
>> 2019. 5. 7. ?????? 10:04, Stefan Monnier <monnier@IRO.UMontreal.CA> ??????:
>>
>>>> Similarly, you'll need to rewrite all the functions/macros like CONSP,
>>>> SYMBOLP, FIXNUMP, XCAR, XCDR, make_fixnum, ...  Performance of those
>>>> is important.
>>>
>>> Why would you not use the default CL???s defun, car, cdr, symbol-p,
>>> cons-p, etc, etc?
>>
>> I'm talking the work needed to adapt Emacs's C code, e.g:
>>
>>    DEFUN ("get-buffer-window", Fget_buffer_window, Sget_buffer_window, 0, 2, 0,
>>           doc: /* Return a window currently displaying BUFFER-OR-NAME, or nil if none.
>>    BUFFER-OR-NAME may be a buffer or a buffer name and defaults to
>>    the current buffer.
>>
>>    The optional argument ALL-FRAMES specifies the frames to consider:
>>
>>    - t means consider all windows on all existing frames.
>>
>>    - `visible' means consider all windows on all visible frames.
>>
>>    - 0 (the number zero) means consider all windows on all visible
>>        and iconified frames.
>>
>>    - A frame means consider all windows on that frame only.
>>
>>    Any other value of ALL-FRAMES means consider all windows on the
>>    selected frame and no others.  */)
>>         (Lisp_Object buffer_or_name, Lisp_Object all_frames)
>>    {
>>      Lisp_Object buffer;
>>
>>      if (NILP (buffer_or_name))
>>        buffer = Fcurrent_buffer ();
>>      else
>>        buffer = Fget_buffer (buffer_or_name);
>>
>>      if (BUFFERP (buffer))
>>        return window_loop (GET_BUFFER_WINDOW, buffer, true, all_frames);
>>      else
>>        return Qnil;
>>    }
>>
>>
>> -- Stefan
>>
>
>

Re: Why is Elisp slow?

[조성빈]

Thinking about this, instead of SBCL (which is super-fast, but imposes restrictions on functions being called from the outside world), why not embed ECL (Embeddable Common Lisp), which 
* is significantly slower than SBCL, about 2~3x slower? but is still much faster than Elisp.
* compiles CL code to C
* has great interoperability with C (have never done this before :-( but based on what I hear, can both freely call CL functions from C and the other way around, generate C inside CL as a ECL extension, etc, etc…)

Looks like if Emacs code has Elisp & C interwined, ECL might be a better shot?

> 2019. 5. 7. 오후 10:14, Ergus <spacibba@aol.com> 작성:
> 
> On Tue, May 07, 2019 at 09:04:31AM -0400, Stefan Monnier wrote:
>>>> Similarly, you'll need to rewrite all the functions/macros like CONSP,
>>>> SYMBOLP, FIXNUMP, XCAR, XCDR, make_fixnum, ...  Performance of those
>>>> is important.
>>> 
>>> Why would you not use the default CL???s defun, car, cdr, symbol-p,
>>> cons-p, etc, etc?
>> 
>> I'm talking the work needed to adapt Emacs's C code, e.g:
>> 
>>   DEFUN ("get-buffer-window", Fget_buffer_window, Sget_buffer_window, 0, 2, 0,
>>          doc: /* Return a window currently displaying BUFFER-OR-NAME, or nil if none.
>>   BUFFER-OR-NAME may be a buffer or a buffer name and defaults to
>>   the current buffer.
>> 
>>   The optional argument ALL-FRAMES specifies the frames to consider:
>> 
>>   - t means consider all windows on all existing frames.
>> 
>>   - `visible' means consider all windows on all visible frames.
>> 
>>   - 0 (the number zero) means consider all windows on all visible
>>       and iconified frames.
>> 
>>   - A frame means consider all windows on that frame only.
>> 
>>   Any other value of ALL-FRAMES means consider all windows on the
>>   selected frame and no others.  */)
>>        (Lisp_Object buffer_or_name, Lisp_Object all_frames)
>>   {
>>     Lisp_Object buffer;
>> 
>>     if (NILP (buffer_or_name))
>>       buffer = Fcurrent_buffer ();
>>     else
>>       buffer = Fget_buffer (buffer_or_name);
>> 
>>     if (BUFFERP (buffer))
>>       return window_loop (GET_BUFFER_WINDOW, buffer, true, all_frames);
>>     else
>>       return Qnil;
>>   }
>> 
>> 
>> -- Stefan
>> 
> That's why I was wondering about the C binds and the C types
> representations in SBCL more than the lisp part of the implementation.

Re: Why is Elisp slow?

[Stefan Monnier]

> embed ECL (Embeddable Common Lisp), which
> * is significantly slower than SBCL, about 2~3x slower? but is still
>   much faster than Elisp.

Last time this discussion came up, ECL seemed like the most promising
option indeed, but the performance was not that impressive compared to
Emacs.  Maybe the situation has changed?
Also in terms of maintenance, it's minimal, so it wouldn't help very
much on the side of manpower.


        Stefan

Re: Why is Elisp slow?

[Ergus]

Personally I think is a bad decision go for ECL. It is unmaintained and
performance is not as good that worth the effort. It will be pretty
much the same than going for GCL.

SBCL is very actively maintained and is free and open source why
couldn't we invest some time doing a plugin for our specific use case??
If we ask the maintainers we could even get some help from them?? Is it
more complex than maintain our interpreter and libraries as we are
doing?? Maybe they have some undocumented functionalities we could take
advantage of. If they have FFI they should have some infrastructure
already there, but hidden for normal users (as python does with CTypes
and the python C API).

Actually the bigger problem I see are the buffer local variables and
similar Emacs specific functionalities. (Maybe because I understand
better C than Lisp)

On Tue, May 07, 2019 at 10:22:39AM -0400, Stefan Monnier wrote:
>> embed ECL (Embeddable Common Lisp), which
>> * is significantly slower than SBCL, about 2~3x slower? but is still
>>   much faster than Elisp.
>
>Last time this discussion came up, ECL seemed like the most promising
>option indeed, but the performance was not that impressive compared to
>Emacs.  Maybe the situation has changed?
>Also in terms of maintenance, it's minimal, so it wouldn't help very
>much on the side of manpower.
>
>
>        Stefan
>
>

Re: Why is Elisp slow?

[Ergus]

Hi:

After some mails in the SBCL mailing list and make some
question-reply-question there; I arrived the conclusion that SBCL is not
an alternative :( I was very wrong.

I'll copy-paste the main answers here just in case someone wants to go
this way in the future. Or in case someone more expert see some possible
workaround for the issues exposed here.

----------------------------------------

Your question as stated is too open-ended for me to discern the exact
intent, but I think's it's something like this: there is much common-lisp
code that people run in emacs, and you'd like not to have to maintain a
common-lisp compiler as well as elisp compiler. Instead you'd like to
retarget the output of the SBCL compiler to produce elisp byte code for
execution.
It's unclear whether you'd expect this to be an out-of-process compiler or
an in-process compiler.  If in-process, then you've got larger problems
than figuring out how to call C.  SBCL is not intended to be a "subsystem"
of other lisps, nor even to allow main() to be anything other than its own
main(), though that limitation has been relaxed a bit.

If out-of-process, then I would imagine that SBCL would operate as a file
compiler only, and not a compiler of arbitrary forms. You need to figure
out how to treat emacs as the virtual machine which the SBCL compiler
targets, and a suitable interface for getting the compiled code out of an
SBCL '.fasl' file and into emacs.   The "machine code" in the '.fasl' file
would actually be emacs byte code. (I'm assuming that you plan to keep
more-or-less the same byte code execution engine)

I don't see why C functions are a particular problem.  SBCL uses C
functions all the time.  In fact it is extremely interoperable with C, but
that's really asking the wrong question imho.
Portability is the real question, since you brought it up. Are you implying
that SBCL's compiler should produce machine code that would run within
emacs? This is highly unlikely; I won't say impossible, but darn near so.
 First of all, SBCL can produce machine code only for a tiny fraction of
the CPUs that emacs supports. Secondly, producing machine code assumes that
the entirety of the runtime (comprised of the memory layout, heap objects,
and support routines) is exactly as SBCL expects it to be - symbols look
like SBCL symbols, special variables get bound in exactly the way that SBCL
expects them to get bound.  SBCL's compiled lisp files are not a portable
abstraction like a '.o' file (or '.so' if you like) that could be loaded
into *any* other program (a C program, Java program, etc) and just run.

But as I said initially, I think you're trying to suggest that you would
produce code that is executed by the emacs lisp engine, not the CPU
directly.
So that would be essentially 1 new architecture that SBCL would have to
support, namely the emacs virtual machine.  That sounds more plausible and
there's no reason to care about the set of CPUs that we can directly
compile for, nor even how the existing architectures would call C code.
You'd have to invent that as part of the retargeting to the emacs vm.

Doug

----------------------------------------

I believe the most viable way to use SBCL for Emacs would be to rethink
the approach: instead of a graphics engine written in C - in addition to
many Lisp functions - and Elisp used to "script" that, with SBCL you no
longer need to have much of that C code for performance reasons.

The entry point to this hypothetical new Emacs would be in Common Lisp,
as well as an Elisp environment that would be mostly macrology on top of
CL (to start with). After that, you could consider rewriting much of the
rendering engine into Lisp too, and leave only a tiny amount of C
strictly for interfacing with the operating system.

I know some people have discussed about turning SBCL into a shared
library but the amount of work would be considerable and with little
gain because I doubt that the SBCL maintainers are interested in
committing to a stable C ABI to the compiler internals - otherwise
embedding SBCL into a C program would be of little use except for maybe
running CL away from the main thread.

Stelian Ionescu

----------------------------------------

I expect portability to be a real issue here. If you *do* want to rely on S=
BCL to generate native machine code for you, and abandon supporting platfor=
ms that SBCL doesn't, that would be OK. This would entail significantly mod=
ifying the Emacs runtime though. The SBCL compiler is married pretty heavil=
y to its runtime (mainly through the GC, which is written in C), and SBCL i=
s a small C program which simply jumps to the Lisp entry point in a large L=
isp image and stays there in its own world, with its own calling convention=
 and object layour, occasionally calling back out to C for operating system=
 tools and GC, which understands Lisp objects and calling conventions. So L=
isp/C interoperation works very well, but as others have stated, it works i=
n a very SBCL-specific manner. I reckon the hard part would be getting the =
C engine for Emacs to fit in the picture, hence the suggestion to rewrite m=
uch of it in Lisp.
But the reality is I sort of doubt that eliminating support for a wide vari=
ety of other architectures is acceptable to Emacs. Hence the suggestion to =
create an Emacs bytecode backend. This would solve the portability problem,=
 at the cost of some efficiency. SBCL tries much harder than Emacs to produ=
ce good code (by utilizing high level optimizations such as constraint (e.g=
. type) propagation), however, so even then the bytecode produced by SBCL w=
ould be superior compared to what Emacs can produce now. A problem with thi=
s is that the compiler really does expect to be targeting a CPU ISA rather =
than something higher level, so you'll have to be clever to figure out how =
to set up the translation from the machine independent intermediate represe=
ntation (IR2), and Elisp bytecode.
Finally, another option to reap the benefits of SBCL's fastish code generat=
ion while not sacrificing portability is to figure out how to make Emacs a =
portable Common Lisp program, through use of portable libraries for foreign=
 code interop, graphics, and Elisp (through macrology, most likely). Then i=
t will run on all platforms that have C compilers through other Lisp implem=
entations which do not target machine code (like ECL, CLISP, etc...) and us=
e SBCL for the platforms it supports. This approach is taken by StumpWM, wh=
ich is fairly similar to Emacs.
=20
Charles

----------------------------------------

On Tue, May 07, 2019 at 10:22:39AM -0400, Stefan Monnier wrote:
>> embed ECL (Embeddable Common Lisp), which
>> * is significantly slower than SBCL, about 2~3x slower? but is still
>>   much faster than Elisp.
>
>Last time this discussion came up, ECL seemed like the most promising
>option indeed, but the performance was not that impressive compared to
>Emacs.  Maybe the situation has changed?
>Also in terms of maintenance, it's minimal, so it wouldn't help very
>much on the side of manpower.
>
>
>        Stefan
>
>

Re: Why is Elisp slow?

[Van L]

Stefan Monnier writes:

> Guile would hopefully win on some of those, but by how much?
>
>>8 [snip]
>>8 [snip]
>> because we need manpower.
>
>8 [snip]
>
> But last I checked Guile didn't have much manpower either.  And there's
> a good chance that tracking Guile development (after switching to Guile)
> will also require manpower on Emacs's side.

Can a big advance in funding boost womanpower participation in making
programming and testing great again? recently, big companies that shal
go nameless have been fined amounts like $USD5B, $USD30B plus, and
another one is enqueued for a $USD5B wet lettuce slap across the wrist.

What if the FSF (1) is given a percentage of those fines? say 2%, to
spray on the next evolution of the Internet's GNU Emacs with GC once
every two weeks.

(1) https://nlnet.nl/NGI/reports/NGI-Study-ISBN-9789279864667.pdf

-- 
© 2019 Van L
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
                               "The pH of lemon juice is 2." - EdX SPU27.1x

Re: Why is Elisp slow?

[조성빈]

I’m just dreaming, but what if Emacs goes the Stumpwm-way and implement everything in CL? There are previous efforts; We might be able to build on them...? 

나의 iPhone에서 보냄

2019. 5. 9. 오후 6:49, Ergus <spacibba@aol.com> 작성:

> Hi:
> 
> After some mails in the SBCL mailing list and make some
> question-reply-question there; I arrived the conclusion that SBCL is not
> an alternative :( I was very wrong.
> 
> I'll copy-paste the main answers here just in case someone wants to go
> this way in the future. Or in case someone more expert see some possible
> workaround for the issues exposed here.
> 
> ----------------------------------------
> 
> Your question as stated is too open-ended for me to discern the exact
> intent, but I think's it's something like this: there is much common-lisp
> code that people run in emacs, and you'd like not to have to maintain a
> common-lisp compiler as well as elisp compiler. Instead you'd like to
> retarget the output of the SBCL compiler to produce elisp byte code for
> execution.
> It's unclear whether you'd expect this to be an out-of-process compiler or
> an in-process compiler.  If in-process, then you've got larger problems
> than figuring out how to call C.  SBCL is not intended to be a "subsystem"
> of other lisps, nor even to allow main() to be anything other than its own
> main(), though that limitation has been relaxed a bit.
> 
> If out-of-process, then I would imagine that SBCL would operate as a file
> compiler only, and not a compiler of arbitrary forms. You need to figure
> out how to treat emacs as the virtual machine which the SBCL compiler
> targets, and a suitable interface for getting the compiled code out of an
> SBCL '.fasl' file and into emacs.   The "machine code" in the '.fasl' file
> would actually be emacs byte code. (I'm assuming that you plan to keep
> more-or-less the same byte code execution engine)
> 
> I don't see why C functions are a particular problem.  SBCL uses C
> functions all the time.  In fact it is extremely interoperable with C, but
> that's really asking the wrong question imho.
> Portability is the real question, since you brought it up. Are you implying
> that SBCL's compiler should produce machine code that would run within
> emacs? This is highly unlikely; I won't say impossible, but darn near so.
> First of all, SBCL can produce machine code only for a tiny fraction of
> the CPUs that emacs supports. Secondly, producing machine code assumes that
> the entirety of the runtime (comprised of the memory layout, heap objects,
> and support routines) is exactly as SBCL expects it to be - symbols look
> like SBCL symbols, special variables get bound in exactly the way that SBCL
> expects them to get bound.  SBCL's compiled lisp files are not a portable
> abstraction like a '.o' file (or '.so' if you like) that could be loaded
> into *any* other program (a C program, Java program, etc) and just run.
> 
> But as I said initially, I think you're trying to suggest that you would
> produce code that is executed by the emacs lisp engine, not the CPU
> directly.
> So that would be essentially 1 new architecture that SBCL would have to
> support, namely the emacs virtual machine.  That sounds more plausible and
> there's no reason to care about the set of CPUs that we can directly
> compile for, nor even how the existing architectures would call C code.
> You'd have to invent that as part of the retargeting to the emacs vm.
> 
> Doug
> 
> ----------------------------------------
> 
> I believe the most viable way to use SBCL for Emacs would be to rethink
> the approach: instead of a graphics engine written in C - in addition to
> many Lisp functions - and Elisp used to "script" that, with SBCL you no
> longer need to have much of that C code for performance reasons.
> 
> The entry point to this hypothetical new Emacs would be in Common Lisp,
> as well as an Elisp environment that would be mostly macrology on top of
> CL (to start with). After that, you could consider rewriting much of the
> rendering engine into Lisp too, and leave only a tiny amount of C
> strictly for interfacing with the operating system.
> 
> I know some people have discussed about turning SBCL into a shared
> library but the amount of work would be considerable and with little
> gain because I doubt that the SBCL maintainers are interested in
> committing to a stable C ABI to the compiler internals - otherwise
> embedding SBCL into a C program would be of little use except for maybe
> running CL away from the main thread.
> 
> Stelian Ionescu
> 
> ----------------------------------------
> 
> I expect portability to be a real issue here. If you *do* want to rely on S=
> BCL to generate native machine code for you, and abandon supporting platfor=
> ms that SBCL doesn't, that would be OK. This would entail significantly mod=
> ifying the Emacs runtime though. The SBCL compiler is married pretty heavil=
> y to its runtime (mainly through the GC, which is written in C), and SBCL i=
> s a small C program which simply jumps to the Lisp entry point in a large L=
> isp image and stays there in its own world, with its own calling convention=
> and object layour, occasionally calling back out to C for operating system=
> tools and GC, which understands Lisp objects and calling conventions. So L=
> isp/C interoperation works very well, but as others have stated, it works i=
> n a very SBCL-specific manner. I reckon the hard part would be getting the =
> C engine for Emacs to fit in the picture, hence the suggestion to rewrite m=
> uch of it in Lisp.
> But the reality is I sort of doubt that eliminating support for a wide vari=
> ety of other architectures is acceptable to Emacs. Hence the suggestion to =
> create an Emacs bytecode backend. This would solve the portability problem,=
> at the cost of some efficiency. SBCL tries much harder than Emacs to produ=
> ce good code (by utilizing high level optimizations such as constraint (e.g=
> . type) propagation), however, so even then the bytecode produced by SBCL w=
> ould be superior compared to what Emacs can produce now. A problem with thi=
> s is that the compiler really does expect to be targeting a CPU ISA rather =
> than something higher level, so you'll have to be clever to figure out how =
> to set up the translation from the machine independent intermediate represe=
> ntation (IR2), and Elisp bytecode.
> Finally, another option to reap the benefits of SBCL's fastish code generat=
> ion while not sacrificing portability is to figure out how to make Emacs a =
> portable Common Lisp program, through use of portable libraries for foreign=
> code interop, graphics, and Elisp (through macrology, most likely). Then i=
> t will run on all platforms that have C compilers through other Lisp implem=
> entations which do not target machine code (like ECL, CLISP, etc...) and us=
> e SBCL for the platforms it supports. This approach is taken by StumpWM, wh=
> ich is fairly similar to Emacs.
> =20
> Charles
> 
> ----------------------------------------
> 
> On Tue, May 07, 2019 at 10:22:39AM -0400, Stefan Monnier wrote:
>>> embed ECL (Embeddable Common Lisp), which
>>> * is significantly slower than SBCL, about 2~3x slower? but is still
>>>  much faster than Elisp.
>> 
>> Last time this discussion came up, ECL seemed like the most promising
>> option indeed, but the performance was not that impressive compared to
>> Emacs.  Maybe the situation has changed?
>> Also in terms of maintenance, it's minimal, so it wouldn't help very
>> much on the side of manpower.
>> 
>> 
>>       Stefan
>> 
>> 
> 

Re: Why is Elisp slow?

[Ergus]

I considered that possibility.

Give a look to the files sizes inside src in Emacs sources. Only xdisp.c, for example, is more than 33000 lines. Projects like remacs are migrating the C code to rust and they have needed years and only moved small portions. Also the lisp compilers aren't as good optimized and mature as GCC for example. And big changes will make the code unfamiliar for our developers.

But we have the scoping differences with common lisp, so that will also force big updates in the lisp code...


On May 10, 2019 5:20:29 AM GMT+02:00, "조성빈" <pcr910303@icloud.com> wrote:
>I’m just dreaming, but what if Emacs goes the Stumpwm-way and implement
>everything in CL? There are previous efforts; We might be able to build
>on them...? 
>
>나의 iPhone에서 보냄
>
>2019. 5. 9. 오후 6:49, Ergus <spacibba@aol.com> 작성:
>
>> Hi:
>> 
>> After some mails in the SBCL mailing list and make some
>> question-reply-question there; I arrived the conclusion that SBCL is
>not
>> an alternative :( I was very wrong.
>> 
>> I'll copy-paste the main answers here just in case someone wants to
>go
>> this way in the future. Or in case someone more expert see some
>possible
>> workaround for the issues exposed here.
>> 
>> ----------------------------------------
>> 
>> Your question as stated is too open-ended for me to discern the exact
>> intent, but I think's it's something like this: there is much
>common-lisp
>> code that people run in emacs, and you'd like not to have to maintain
>a
>> common-lisp compiler as well as elisp compiler. Instead you'd like to
>> retarget the output of the SBCL compiler to produce elisp byte code
>for
>> execution.
>> It's unclear whether you'd expect this to be an out-of-process
>compiler or
>> an in-process compiler.  If in-process, then you've got larger
>problems
>> than figuring out how to call C.  SBCL is not intended to be a
>"subsystem"
>> of other lisps, nor even to allow main() to be anything other than
>its own
>> main(), though that limitation has been relaxed a bit.
>> 
>> If out-of-process, then I would imagine that SBCL would operate as a
>file
>> compiler only, and not a compiler of arbitrary forms. You need to
>figure
>> out how to treat emacs as the virtual machine which the SBCL compiler
>> targets, and a suitable interface for getting the compiled code out
>of an
>> SBCL '.fasl' file and into emacs.   The "machine code" in the '.fasl'
>file
>> would actually be emacs byte code. (I'm assuming that you plan to
>keep
>> more-or-less the same byte code execution engine)
>> 
>> I don't see why C functions are a particular problem.  SBCL uses C
>> functions all the time.  In fact it is extremely interoperable with
>C, but
>> that's really asking the wrong question imho.
>> Portability is the real question, since you brought it up. Are you
>implying
>> that SBCL's compiler should produce machine code that would run
>within
>> emacs? This is highly unlikely; I won't say impossible, but darn near
>so.
>> First of all, SBCL can produce machine code only for a tiny fraction
>of
>> the CPUs that emacs supports. Secondly, producing machine code
>assumes that
>> the entirety of the runtime (comprised of the memory layout, heap
>objects,
>> and support routines) is exactly as SBCL expects it to be - symbols
>look
>> like SBCL symbols, special variables get bound in exactly the way
>that SBCL
>> expects them to get bound.  SBCL's compiled lisp files are not a
>portable
>> abstraction like a '.o' file (or '.so' if you like) that could be
>loaded
>> into *any* other program (a C program, Java program, etc) and just
>run.
>> 
>> But as I said initially, I think you're trying to suggest that you
>would
>> produce code that is executed by the emacs lisp engine, not the CPU
>> directly.
>> So that would be essentially 1 new architecture that SBCL would have
>to
>> support, namely the emacs virtual machine.  That sounds more
>plausible and
>> there's no reason to care about the set of CPUs that we can directly
>> compile for, nor even how the existing architectures would call C
>code.
>> You'd have to invent that as part of the retargeting to the emacs vm.
>> 
>> Doug
>> 
>> ----------------------------------------
>> 
>> I believe the most viable way to use SBCL for Emacs would be to
>rethink
>> the approach: instead of a graphics engine written in C - in addition
>to
>> many Lisp functions - and Elisp used to "script" that, with SBCL you
>no
>> longer need to have much of that C code for performance reasons.
>> 
>> The entry point to this hypothetical new Emacs would be in Common
>Lisp,
>> as well as an Elisp environment that would be mostly macrology on top
>of
>> CL (to start with). After that, you could consider rewriting much of
>the
>> rendering engine into Lisp too, and leave only a tiny amount of C
>> strictly for interfacing with the operating system.
>> 
>> I know some people have discussed about turning SBCL into a shared
>> library but the amount of work would be considerable and with little
>> gain because I doubt that the SBCL maintainers are interested in
>> committing to a stable C ABI to the compiler internals - otherwise
>> embedding SBCL into a C program would be of little use except for
>maybe
>> running CL away from the main thread.
>> 
>> Stelian Ionescu
>> 
>> ----------------------------------------
>> 
>> I expect portability to be a real issue here. If you *do* want to
>rely on S=
>> BCL to generate native machine code for you, and abandon supporting
>platfor=
>> ms that SBCL doesn't, that would be OK. This would entail
>significantly mod=
>> ifying the Emacs runtime though. The SBCL compiler is married pretty
>heavil=
>> y to its runtime (mainly through the GC, which is written in C), and
>SBCL i=
>> s a small C program which simply jumps to the Lisp entry point in a
>large L=
>> isp image and stays there in its own world, with its own calling
>convention=
>> and object layour, occasionally calling back out to C for operating
>system=
>> tools and GC, which understands Lisp objects and calling conventions.
>So L=
>> isp/C interoperation works very well, but as others have stated, it
>works i=
>> n a very SBCL-specific manner. I reckon the hard part would be
>getting the =
>> C engine for Emacs to fit in the picture, hence the suggestion to
>rewrite m=
>> uch of it in Lisp.
>> But the reality is I sort of doubt that eliminating support for a
>wide vari=
>> ety of other architectures is acceptable to Emacs. Hence the
>suggestion to =
>> create an Emacs bytecode backend. This would solve the portability
>problem,=
>> at the cost of some efficiency. SBCL tries much harder than Emacs to
>produ=
>> ce good code (by utilizing high level optimizations such as
>constraint (e.g=
>> . type) propagation), however, so even then the bytecode produced by
>SBCL w=
>> ould be superior compared to what Emacs can produce now. A problem
>with thi=
>> s is that the compiler really does expect to be targeting a CPU ISA
>rather =
>> than something higher level, so you'll have to be clever to figure
>out how =
>> to set up the translation from the machine independent intermediate
>represe=
>> ntation (IR2), and Elisp bytecode.
>> Finally, another option to reap the benefits of SBCL's fastish code
>generat=
>> ion while not sacrificing portability is to figure out how to make
>Emacs a =
>> portable Common Lisp program, through use of portable libraries for
>foreign=
>> code interop, graphics, and Elisp (through macrology, most likely).
>Then i=
>> t will run on all platforms that have C compilers through other Lisp
>implem=
>> entations which do not target machine code (like ECL, CLISP, etc...)
>and us=
>> e SBCL for the platforms it supports. This approach is taken by
>StumpWM, wh=
>> ich is fairly similar to Emacs.
>> =20
>> Charles
>> 
>> ----------------------------------------
>> 
>> On Tue, May 07, 2019 at 10:22:39AM -0400, Stefan Monnier wrote:
>>>> embed ECL (Embeddable Common Lisp), which
>>>> * is significantly slower than SBCL, about 2~3x slower? but is
>still
>>>>  much faster than Elisp.
>>> 
>>> Last time this discussion came up, ECL seemed like the most
>promising
>>> option indeed, but the performance was not that impressive compared
>to
>>> Emacs.  Maybe the situation has changed?
>>> Also in terms of maintenance, it's minimal, so it wouldn't help very
>>> much on the side of manpower.
>>> 
>>> 
>>>       Stefan
>>> 
>>> 
>> 

-- 
Enviado desde mi dispositivo Android con K-9 Mail. Por favor, disculpa mi brevedad.

Re: Why is Elisp slow?

[조성빈]

I understand why this is impossible or near-to-impossible; but...

> 2019. 5. 10. 오후 1:26, Ergus <spacibba@aol.com> 작성:
> 
> I considered that possibility.
> 
> Give a look to the files sizes inside src in Emacs sources. Only xdisp.c, for example, is more than 33000 lines. Projects like remacs are migrating the C code to rust and they have needed years and only moved small portions. Also the lisp compilers aren't as good optimized and mature as GCC for example. And big changes will make the code unfamiliar for our developers.

xdisp.c is a cross-platform (#ifdef-containing) low-level function that handles displaying… C functions like that can just be imported with FFI and be used.
Many CL compilers (such as SBCL, CCL, etc) are actually super-optimized, and emit code as fast as most C compilers as the language was used for a long time;
With some type hints in low-level functions, SBCL emits assembly as fast as C :-)

As for remacs, I’m pretty sure GNU Emacs have much more manpower than Remacs; and Emacs don’t need to migrate all of C code; only the portion that implements Elisp functions.

> But we have the scoping differences with common lisp, so that will also force big updates in the lisp code…

This can be mitigated by implementing Elisp in CL (as we would have to do that) and run the code in the Elisp package.

> On May 10, 2019 5:20:29 AM GMT+02:00, "조성빈" <pcr910303@icloud.com> wrote:
> I’m just dreaming, but what if Emacs goes the Stumpwm-way and implement everything in CL? There are previous efforts; We might be able to build on them...? 
> 
> 나의 iPhone에서 보냄
> 
> 2019. 5. 9. 오후 6:49, Ergus <spacibba@aol.com> 작성:
> 
> Hi:
> 
> After some mails in the SBCL mailing list and make some
> question-reply-question there; I arrived the conclusion that SBCL is not
> an alternative :( I was very wrong.
> 
> I'll copy-paste the main answers here just in case someone wants to go
> this way in the future. Or in case someone more expert see some possible
> workaround for the issues exposed here.
> Your question as stated is too open-ended for me to discern the exact
> intent, but I think's it's something like this: there is much common-lisp
> code that people run in emacs, and you'd like not to have to maintain a
> common-lisp compiler as well as elisp compiler. Instead you'd like to
> retarget the output of the SBCL compiler to produce elisp byte code for
> execution.
> It's unclear whether you'd expect this to be an out-of-process compiler or
> an in-process compiler.  If in-process, then you've got larger problems
> than figuring out how to call C.  SBCL is not intended to be a "subsystem"
> of other lisps, nor even to allow main() to be anything other than its own
> main(), though that limitation has been relaxed a bit.
> 
> If out-of-process, then I would imagine that SBCL would operate as a file
> compiler only, and not a compiler of arbitrary forms. You need to figure
> out how to treat emacs as the virtual machine which the SBCL compiler
> targets, and a suitable interface for getting the compiled code out of an
> SBCL '.fasl' file and into emacs.   The "machine code" in the '.fasl' file
> would actually be emacs byte code. (I'm assuming that you plan to keep
> more-or-less the same byte code execution engine)
> 
> I don't see why C functions are a particular problem.  SBCL uses C
> functions all the time.  In fact it is extremely interoperable with C, but
> that's really asking the wrong question imho.
> Portability is the real question, since you brought it up. Are you implying
> that SBCL's compiler should produce machine code that would run within
> emacs? This is highly unlikely; I won't say impossible, but darn near so.
> First of all, SBCL can produce machine code only for a tiny fraction of
> the CPUs that emacs supports. Secondly, producing machine code assumes that
> the entirety of the runtime (comprised of the memory layout, heap objects,
> and support routines) is exactly as SBCL expects it to be - symbols look
> like SBCL symbols, special variables get bound in exactly the way that SBCL
> expects them to get bound.  SBCL's compiled lisp files are not a portable
> abstraction like a '.o' file (or '.so' if you like) that could be loaded
> into *any* other program (a C program, Java program, etc) and just run.
> 
> But as I said initially, I think you're trying to suggest that you would
> produce code that is executed by the emacs lisp engine, not the CPU
> directly.
> So that would be essentially 1 new architecture that SBCL would have to
> support, namely the emacs virtual machine.  That sounds more plausible and
> there's no reason to care about the set of CPUs that we can directly
> compile for, nor even how the existing architectures would call C code.
> You'd have to invent that as part of the retargeting to the emacs vm.
> 
> Doug
> I believe the most viable way to use SBCL for Emacs would be to rethink
> the approach: instead of a graphics engine written in C - in addition to
> many Lisp functions - and Elisp used to "script" that, with SBCL you no
> longer need to have much of that C code for performance reasons.
> 
> The entry point to this hypothetical new Emacs would be in Common Lisp,
> as well as an Elisp environment that would be mostly macrology on top of
> CL (to start with). After that, you could consider rewriting much of the
> rendering engine into Lisp too, and leave only a tiny amount of C
> strictly for interfacing with the operating system.
> 
> I know some people have discussed about turning SBCL into a shared
> library but the amount of work would be considerable and with little
> gain because I doubt that the SBCL maintainers are interested in
> committing to a stable C ABI to the compiler internals - otherwise
> embedding SBCL into a C program would be of little use except for maybe
> running CL away from the main thread.
> 
> Stelian Ionescu
> I expect portability to be a real issue here. If you *do* want to rely on S=
> BCL to generate native machine code for you, and abandon supporting platfor=
> ms that SBCL doesn't, that would be OK. This would entail significantly mod=
> ifying the Emacs runtime though. The SBCL compiler is married pretty heavil=
> y to its runtime (mainly through the GC, which is written in C), and SBCL i=
> s a small C program which simply jumps to the Lisp entry point in a large L=
> isp image and stays there in its own world, with its own calling convention=
> and object layour, occasionally calling back out to C for operating system=
> tools and GC, which understands Lisp objects and calling conventions. So L=
> isp/C interoperation works very well, but as others have stated, it works i=
> n a very SBCL-specific manner. I reckon the hard part would be getting the =
> C engine for Emacs to fit in the picture, hence the suggestion to rewrite m=
> uch of it in Lisp.
> But the reality is I sort of doubt that eliminating support for a wide vari=
> ety of other architectures is acceptable to Emacs. Hence the suggestion to =
> create an Emacs bytecode backend. This would solve the portability problem,=
> at the cost of some efficiency. SBCL tries much harder than Emacs to produ=
> ce good code (by utilizing high level optimizations such as constraint (e.g=
> . type) propagation), however, so even then the bytecode produced by SBCL w=
> ould be superior compared to what Emacs can produce now. A problem with thi=
> s is that the compiler really does expect to be targeting a CPU ISA rather =
> than something higher level, so you'll have to be clever to figure out how =
> to set up the translation from the machine independent intermediate represe=
> ntation (IR2), and Elisp bytecode.
> Finally, another option to reap the benefits of SBCL's fastish code generat=
> ion while not sacrificing portability is to figure out how to make Emacs a =
> portable Common Lisp program, through use of portable libraries for foreign=
> code interop, graphics, and Elisp (through macrology, most likely). Then i=
> t will run on all platforms that have C compilers through other Lisp implem=
> entations which do not target machine code (like ECL, CLISP, etc...) and us=
> e SBCL for the platforms it supports. This approach is taken by StumpWM, wh=
> ich is fairly similar to Emacs.
> =20
> Charles
> On Tue, May 07, 2019 at 10:22:39AM -0400, Stefan Monnier wrote:
> embed ECL (Embeddable Common Lisp), which
> * is significantly slower than SBCL, about 2~3x slower? but is still
>  much faster than Elisp.
> 
> Last time this discussion came up, ECL seemed like the most promising
> option indeed, but the performance was not that impressive compared to
> Emacs.  Maybe the situation has changed?
> Also in terms of maintenance, it's minimal, so it wouldn't help very
> much on the side of manpower.
> 
> 
>       Stefan
> 
> 
> 
> 
> 
> 
> -- 
> Enviado desde mi dispositivo Android con K-9 Mail. Por favor, disculpa mi brevedad.

Re: Why is Elisp slow?

[Van L]

> Yes, it's a very different design.  Elisp was designed to be the
> *implementation* language of Emacs (the use of C for some of the
> implementation was a necessary evil given the lack of wide availability
> of Lisp), rather than just an *extension* language.

Is it possible to drop the C and re-introduce the missing microcodes
from the Lisp Machine for the full lispy experience on a CuBox Pulse? (1)

(1) https://www.solid-run.com/nxp-family/cubox-pulse/

-- 
© 2019 Van L
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
                               "The pH of lemon juice is 2." - EdX SPU27.1x

Re: Why is Elisp slow?

[Van L]

> Lisp was, and is a hacker’s language.
> C++, Java is a corporate language. 
> C was, and is a hacker’s language.

Is it possible to compare programming languages to fighter jets at trade
shows demonstrating what can be done w/wo the capability.  For example,
F-22's vectored thrust, F-35's lackof.  They fly their signature flight
paths.  When RMS says CL isn't lispy does that mean no vectored thrust?

-- 
© 2019 Van L
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
                               "The pH of lemon juice is 2." - EdX SPU27.1x

Re: Why is Elisp slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 233 bytes --]

On Fri, May 10, 2019 at 06:26:57AM +0200, Ergus wrote:
> I considered that possibility.
> 
> [...] Also the lisp compilers aren't as good optimized and mature
> as GCC for example [...]

Hmmm. Citation needed.

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Why is Elisp slow?

[Van L]

Ergus writes:

> Actually I think that these days will be easier to find new C/C++
> developers for emacs than Lisp developers. Lisp and Scheme are
> beautiful, but they require a different way of thinking and a lot of
> time (own experience, I am just starting with it.) One reason why I
> can't convince my friends to use Emacs is actually how Lisp scares them
> ((())()'()).

Alice wasn't afraid of the cat in Wonderland.

-- 
© 2019 Van L
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
                  "The iPhone was a really great invention." - Elon Musk

Re: Why is Elisp slow?

[Michael Heerdegen]

Van L <van@scratch.space> writes:

> > ((())()'()).

I wonder how this would look like in C/C++.

Michael.

Lisp, C, and C++ (was: Re: Why is Elisp slow?)

[Emanuel Berg]

Van L wrote:

>> Lisp was, and is a hacker’s language. C++,
>> Java is a corporate language. C was, and is
>> a hacker’s language.
>
> [...]

Who said that? I must'a missed that message and
Van L didn't provide an attribution.

Anyway I agree both Lisp and C are hacker
languages! [figure: 1] They are just different
hacker personalities.

And those personalities can coexist in the same
hacker, for that matter. Just like I enjoyed
"Battle Angel Alita" (2019) just as much as
"Everest" (2015) despite them not being in the
same genre [OT list: 2].

As for Java, I agree it is a corporate language
- or whatever is the appropriate term, but it
isn't a hacker language for sure.

C++ tho I don't know! I think it is a hacker
language as well. A lot of the bad rep it has
taken is because of people's adversity to OO,
which, in turn, is a reaction to the
super OO hype that once was, and also because
of people using it in crazy ways (all but
infinite inheritance trees etc) But, 1) OO is
not necessarily bad, it depends how you use it
as with (almost) everything else; and 2) while
huge in the 90s, most C++ programmers of today
don't put half the emphasis on OO, neither
orally or in practice.


[1] https://dataswamp.org/~incal/pics/lang.png
[2] https://dataswamp.org/~incal/FILM

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[Emanuel Berg]

Michael Heerdegen wrote:

>> ((())()'()).
>
> I wonder how this would look like in C/C++.

That question does not compute, as C and C++
are two different languages :P

Actually, it doesn't look much in Lisp either -
it doesn't even evaluate.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Lisp, C, and C++ (was: Re: Why is Elisp slow?)

[Emanuel Berg]

> A lot of the bad rep it has taken is because
> of people's adversity to OO [...]

*aversion

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[Emanuel Berg]

If it is possible to have Elisp as fast as CL
or even as fast as C (!), the only problem is
no one has been motivated to do it! ???

Are there not young people and university
students who would love such a challenge?

Or for that matter experienced people who would
love to tinker with it once in a while?

??

I'm sure someone already does it, in some form,
somewhere on the planet! If he reads this,
hello to YOU! Impressive but "as fast as C" was
an exaggeration, right?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[Stefan Monnier]

> Are there not young people and university
> students who would love such a challenge?

There are, but they usually consider that applying those ideas to
Javascript/Python/younameit is more worthwhile than applying it
to Elisp.


        Stefan

Re: Why is Elisp slow?

[Emanuel Berg]

Stefan Monnier wrote:

>> Are there not young people and university
>> students who would love such a challenge?
>
> There are, but they usually consider that
> applying those ideas to
> Javascript/Python/younameit is more
> worthwhile than applying it to Elisp.

Is there a tutorial to get them going? I only
read "Compiler Design" at the university, which
means I'm not qualified. But I'm sure you are.
It doesn't have to be a long article. Like
this [1] is enough. If you or anyone else does
it, I can help with the edit, only... if you do
it, I think you'd do all that as well.


[1] https://dataswamp.org/~incal/cols/www/COLORS

-- 
underground experts united
http://user.it.uu.se/~embe8573

Is Elisp really that slow? (was: Why is Elisp slow?)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 2542 bytes --]

On Sat, May 11, 2019 at 02:38:58AM +0200, Emanuel Berg wrote:
> If it is possible to have Elisp as fast as CL
> or even as fast as C (!), the only problem is
> no one has been motivated to do it! ???
> 
> Are there not young people and university
> students who would love such a challenge?

One last attempt: this whole thread is based on the premise
that there is one clear definition of slow. Yet, there isn't.

Back in the 1980s, the (micro-) computers were so slow, that
access to main memory wasn't a problem at all. I know of one
design where two Z80 ran off the same static RAM, taking turns
to access it.

Somewhat later, RAM was so slow that we had to interpose several
layers of cache between it and the CPU (that is still the situation
nowadays, which brought upon us Meltdown and Spectre).

A tad later, CPUs stopped getting faster at sequential execution.
Making pipelines deeper and things stopped being fun, and transistors
are at the current limit of how fast they can do. But, we can make
them cheaper and smaller (and perhaps somewhat less greedy on
electrons), so packing more CPUs on the chip is becoming more
fashionable [1].

If you want to be really fast these days, you have a gang of
fairly dumb processors all doing the same instruction at a
time (SIMD). It's weird to program that: if you take a branch,
you have to decide for one side, and throw away those members
of your gang which went the other way (well, you throw their
results away, actually). Perhaps pick the second branch up
in a second pass.

Of course, the definition of "fast software" changes radically
when your bedrock moves that quickly. The second "generation"
above made cache-efficient algorithms interesting (somehow
a rebirth of the 70ies, where you had very limited RAM and
somewhat bigger external memory). In the third generation,
parallelizable algorithms became more interesting (did you
know that bubble sort, the laughing stock of CS, parallelizes
pretty well? If you have more processors than time, this one
isn't that bad). The fourth generation... well. Interestingly,
functional languages help in making fast programs for those
strange beasts without getting crazy.

What I don't like about this thread is that "fast" is being
treated as the only metrics (it isn't) and as if it were a
metrics at all (it isn't either).

Cheers

[1] Sometimes you see that coming and bet too early on it.
    Alas, Sun is no more:
    https://en.wikipedia.org/wiki/Sun_Niagara 

-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[조성빈]

I understand your points; but IMHO Elisp is not doing well in all kinds of aspects, Isn’t it?̊̈

나의 iPhone에서 보냄

2019. 5. 11. 오후 4:32, tomas@tuxteam.de 작성:

>> On Sat, May 11, 2019 at 02:38:58AM +0200, Emanuel Berg wrote:
>> If it is possible to have Elisp as fast as CL
>> or even as fast as C (!), the only problem is
>> no one has been motivated to do it! ???
>> 
>> Are there not young people and university
>> students who would love such a challenge?
> 
> One last attempt: this whole thread is based on the premise
> that there is one clear definition of slow. Yet, there isn't.
> 
> Back in the 1980s, the (micro-) computers were so slow, that
> access to main memory wasn't a problem at all. I know of one
> design where two Z80 ran off the same static RAM, taking turns
> to access it.
> 
> Somewhat later, RAM was so slow that we had to interpose several
> layers of cache between it and the CPU (that is still the situation
> nowadays, which brought upon us Meltdown and Spectre).
> 
> A tad later, CPUs stopped getting faster at sequential execution.
> Making pipelines deeper and things stopped being fun, and transistors
> are at the current limit of how fast they can do. But, we can make
> them cheaper and smaller (and perhaps somewhat less greedy on
> electrons), so packing more CPUs on the chip is becoming more
> fashionable [1].
> 
> If you want to be really fast these days, you have a gang of
> fairly dumb processors all doing the same instruction at a
> time (SIMD). It's weird to program that: if you take a branch,
> you have to decide for one side, and throw away those members
> of your gang which went the other way (well, you throw their
> results away, actually). Perhaps pick the second branch up
> in a second pass.
> 
> Of course, the definition of "fast software" changes radically
> when your bedrock moves that quickly. The second "generation"
> above made cache-efficient algorithms interesting (somehow
> a rebirth of the 70ies, where you had very limited RAM and
> somewhat bigger external memory). In the third generation,
> parallelizable algorithms became more interesting (did you
> know that bubble sort, the laughing stock of CS, parallelizes
> pretty well? If you have more processors than time, this one
> isn't that bad). The fourth generation... well. Interestingly,
> functional languages help in making fast programs for those
> strange beasts without getting crazy.
> 
> What I don't like about this thread is that "fast" is being
> treated as the only metrics (it isn't) and as if it were a
> metrics at all (it isn't either).
> 
> Cheers
> 
> [1] Sometimes you see that coming and bet too early on it.
>    Alas, Sun is no more:
>    https://en.wikipedia.org/wiki/Sun_Niagara 
> 
> -- tomás

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 564 bytes --]

On Sat, May 11, 2019 at 04:42:44PM +0900, 조성빈 wrote:
> I understand your points; but IMHO Elisp is not doing well in all kinds of aspects, Isn’t it?̊̈

(quoting myself):

> > What I don't like about this thread is that "fast" is being
> > treated as the only metrics (it isn't)

Accessibility/hackability: elisp is doing pretty well in this
one (and efficient compilation does put a burden on that).

>                                      and as if it were a
> > metrics at all (it isn't either).

Show us the benchmarks.

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[조성빈]

2019. 5. 11. 오후 4:57, tomas@tuxteam.de 작성:

>> On Sat, May 11, 2019 at 04:42:44PM +0900, 조성빈 wrote:
>> I understand your points; but IMHO Elisp is not doing well in all kinds of aspects, Isn’t it?̊̈
> 
> (quoting myself):
> 
>>> What I don't like about this thread is that "fast" is being
>>> treated as the only metrics (it isn't)
> 
> Accessibility/hackability: elisp is doing pretty well in this
> one (and efficient compilation does put a burden on that).

Javascript? Common Lisp? PyPy(Implementation of Python)?

>>                                     and as if it were a
>>> metrics at all (it isn't either).
> 
> Show us the benchmarks.
> 
> Cheers
> -- t

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 805 bytes --]

On Sat, May 11, 2019 at 11:30:55PM +0900, 조성빈 wrote:
> 
> 2019. 5. 11. 오후 4:57, tomas@tuxteam.de 작성:
> 
> >> On Sat, May 11, 2019 at 04:42:44PM +0900, 조성빈 wrote:
> >> I understand your points; but IMHO Elisp is not doing well in all kinds of aspects, Isn’t it?̊̈
> > 
> > (quoting myself):
> > 
> >>> What I don't like about this thread is that "fast" is being
> >>> treated as the only metrics (it isn't)
> > 
> > Accessibility/hackability: elisp is doing pretty well in this
> > one (and efficient compilation does put a burden on that).
> 
> Javascript? Common Lisp? PyPy(Implementation of Python)?

What's the last time you hacked the innards of those?

;-)

IOW: I was talking also about accessibility/hackability "down the
stack".

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[Emanuel Berg]

tomas wrote:

> Show us the benchmarks.

How do people do benchmarks when it comes to
comparing the speed of different
computer languages?

Do you solve a wide/varied set of problems (the
same set) with different languages and then
compare the wall clock execution time?

Perhaps using an atomic clock in
a sterile environment?

Now, while that would be interesting to do,
I think it's pretty clear Elisp would benefit
from more speed. Maybe it is not slow for its
purposes, but it ain't fast either, sure is
my impression.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 2719 bytes --]

On Sun, May 12, 2019 at 01:09:09AM +0200, Emanuel Berg wrote:
> tomas wrote:
> 
> > Show us the benchmarks.
> 
> How do people do benchmarks when it comes to
> comparing the speed of different
> computer languages?

My favourite search engine recommends [1], [2].

> Do you solve a wide/varied set of problems (the
> same set) with different languages and then
> compare the wall clock execution time?

More or less, yes. See references.

> Perhaps using an atomic clock in
> a sterile environment?

Clock accuracy/precision/resolution will be the least of your problems
in a cross-language benchmark. Different languages exist for a reason,
and carry with them a whole set of different cultures, so they'll tend
to express a problem in subtly different way (leading to subtly differing
semantics). How do you account for the fact that some problem expressed
in C won't do array bounds checking, while in Java it will do them
dynamically all the time (costing runtime), whereas in some smart
functional language the compiler will do most of that work most of
the time, due to advanced compiler magic? How do you account for the
fact that in language A the programmer will take 100 times longer to
master the language, but one-tenth of the time to write a program?
The fact that this compiler will produce runtimes twice as fast, but
ten times as fat, and take 50 times the compile time?

Yeah. Numbers for illustration only.

> Now, while that would be interesting to do,
> I think it's pretty clear Elisp would benefit
> from more speed.

My whole point is that this is /not/ so clear. Emphatically so.
Speed comes at a price (did you do your research about Spectre
and Meltdown?). Perhaps Elisp has found an optimum in this tradeoff,
for the job it's supposed to do.

Well, actually this optimum point shifts around with time, since
the environment Emacs lives in changes (people using it, the kind
of tasks it is put to, but also the hardware it lives on).

Actually I'm convinced Emacs is tracking this optimum pretty nicely,
since it is looking back to 34 years of life (and still receives a
steady stream of patches [3]!). Few pieces of software can say that
of them.

>                 Maybe it is not slow for its
> purposes,

...and that is exactly the point!

>           but it ain't fast either, sure is
> my impression.

I haven't the time currently to illustrate my point more, sorry. Do
your experiments, find out for yourself.

Cheers

[1] https://en.wikipedia.org/wiki/The_Computer_Language_Benchmarks_Game
[2] https://benchmarksgame-team.pages.debian.net/benchmarksgame/
[3] http://git.savannah.gnu.org/cgit/emacs.git

-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[Emanuel Berg]

tomas wrote:

> Different languages exist for a reason, and
> carry with them a whole set of different
> cultures, so they'll tend to express
> a problem in subtly different way (leading to
> subtly differing semantics). How do you
> account for the fact that some problem
> expressed in C won't do array bounds
> checking, while in Java it will do them
> dynamically all the time (costing runtime),
> whereas in some smart functional language the
> compiler will do most of that work most of
> the time, due to advanced compiler magic?
> How do you account for the fact that in
> language A the programmer will take 100 times
> longer to master the language, but one-tenth
> of the time to write a program? The fact that
> this compiler will produce runtimes twice as
> fast, but ten times as fat, and take 50 times
> the compile time?

One program will finish execution with
a correct result at time X and another at
time Y and it will not be X = Y. So one
language, with associated tools, is faster!

If one language has a better compiler, a better
programmer, and so on, that's part of the game.

You can't have a better computer, that's all.

> Clock accuracy/precision/resolution will be
> the least of your problems in
> a cross-language benchmark.

How about a zero-gravity chamber then?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[조성빈]

2019. 5. 12. 오후 4:54, tomas@tuxteam.de 작성:

>> On Sun, May 12, 2019 at 01:09:09AM +0200, Emanuel Berg wrote:
>> tomas wrote:
>> 
>>> Show us the benchmarks.
>> 
>> How do people do benchmarks when it comes to
>> comparing the speed of different
>> computer languages?
> 
> My favourite search engine recommends [1], [2].
> 
>> Do you solve a wide/varied set of problems (the
>> same set) with different languages and then
>> compare the wall clock execution time?
> 
> More or less, yes. See references.
> 
>> Perhaps using an atomic clock in
>> a sterile environment?
> 
> Clock accuracy/precision/resolution will be the least of your problems
> in a cross-language benchmark. Different languages exist for a reason,
> and carry with them a whole set of different cultures, so they'll tend
> to express a problem in subtly different way (leading to subtly differing
> semantics). How do you account for the fact that some problem expressed
> in C won't do array bounds checking, while in Java it will do them
> dynamically all the time (costing runtime), whereas in some smart
> functional language the compiler will do most of that work most of
> the time, due to advanced compiler magic? How do you account for the
> fact that in language A the programmer will take 100 times longer to
> master the language, but one-tenth of the time to write a program?
> The fact that this compiler will produce runtimes twice as fast, but
> ten times as fat, and take 50 times the compile time?

So, you would compare languages that are ‘safe and dynamic enough’ as Elisp.
E.g. Python, JS, CL, Ruby, etc.

> Yeah. Numbers for illustration only.
> 
>> Now, while that would be interesting to do,
>> I think it's pretty clear Elisp would benefit
>> from more speed.
> 
> My whole point is that this is /not/ so clear. Emphatically so.
> Speed comes at a price (did you do your research about Spectre
> and Meltdown?). Perhaps Elisp has found an optimum in this tradeoff,
> for the job it's supposed to do.

Speed comes at a price, but as we see Pypy, JS, CL, and other high languages that are fast, we can see that it is /so/ clear that we can pursue more speed without sacrificing the language’s expressiveness, dynamicness, etc.

> Well, actually this optimum point shifts around with time, since
> the environment Emacs lives in changes (people using it, the kind
> of tasks it is put to, but also the hardware it lives on).
> 
> Actually I'm convinced Emacs is tracking this optimum pretty nicely,
> since it is looking back to 34 years of life (and still receives a
> steady stream of patches [3]!). Few pieces of software can say that
> of them.

Well, look at Atom and VS Code which is using the V8 engine (through node) to execute JS to tweak the editor. 
Now compare that to Emacs.
And Atom gets criticized by it’s slow bootup time, etc...
It’s pretty sure Emacs can benefit from some speed...

>>                Maybe it is not slow for its
>> purposes,
> 
> ...and that is exactly the point!
> 
>>          but it ain't fast either, sure is
>> my impression.
> 
> I haven't the time currently to illustrate my point more, sorry. Do
> your experiments, find out for yourself.
> 
> Cheers
> 
> [1] https://en.wikipedia.org/wiki/The_Computer_Language_Benchmarks_Game
> [2] https://benchmarksgame-team.pages.debian.net/benchmarksgame/
> [3] http://git.savannah.gnu.org/cgit/emacs.git
> 
> -- tomás

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[Eli Zaretskii]

> From: 조성빈 <pcr910303@icloud.com>
> Date: Sun, 12 May 2019 18:46:20 +0900
> Cc: help-gnu-emacs@gnu.org
> 
> Well, look at Atom and VS Code which is using the V8 engine (through node) to execute JS to tweak the editor. 
> Now compare that to Emacs.

This is meaningless, unless you describe the use case for comparison
(and while at that, how about some timings instead of hand-waving?).

Most stuff Emacs routinely does is something the other editors could
only dream about, so by and large you are comparing apples to peanuts.

And while I'm here, allow me a few comments regarding this thread's
topic:

 . Speed of ELisp is a valid topic for discussion, but it has very
   little, if anything to do with whether Emacs as a whole is or isn't
   slow.  Why? because speed-critical stuff is not supposed to be
   implemented in Lisp in Emacs, that is not how Emacs is designed and
   implemented.  If you find some operation that is slow in Emacs, it
   just means it might either use more efficient algorithms, or should
   have some of it recoded in C.

 . The speed of various Emacs operations is always a trade-off between
   how fast we want Emacs to be, on the one hand, and how flexible and
   controllable from Lisp we want it to be, OTOH.  When neither better
   algorithms nor reimplementation in C are likely to speed up some
   operation, it means we decided to give up some speed in order to
   have more control and flexibility.  There's nothing wrong with
   that, IMO.

 . Some Emacs operations are slow because no one felt the itch to sit
   down and speed them up -- here's your chance to do something about
   that.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> Most stuff Emacs routinely does is something the other editors could
> only dream about, so by and large you are comparing apples to peanuts.

Examples?

Emacs is amazing, but as far as productivity goes, at least as a
programmer's editor for popular languages, it is far behind current
contenders.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sun, 12 May 2019 16:45:47 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Most stuff Emacs routinely does is something the other editors could
> > only dream about, so by and large you are comparing apples to peanuts.
> 
> Examples?

  M-: (add-text-properties 1 2 '(display "Look, ma!")) RET

> Emacs is amazing, but as far as productivity goes, at least as a
> programmer's editor for popular languages, it is far behind current
> contenders.

You've changed the subject, yes?  This was never about productivity.

I agree that our situation with IDE features is quite bleak, but
that's entirely unrelated to the issues discussed in this thread, and
its reasons IMO have nothing to do with either ELisp speed or
flexibility of Emacs in general.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> Emacs is amazing, but as far as productivity goes, at least as a
>> programmer's editor for popular languages, it is far behind current
>> contenders.
>
> You've changed the subject, yes?  This was never about productivity.
>
> I agree that our situation with IDE features is quite bleak, but
> that's entirely unrelated to the issues discussed in this thread, and
> its reasons IMO have nothing to do with either ELisp speed or
> flexibility of Emacs in general.

It has a lot to do. Analyzing source code is cpu-intensive, because it
is algorithmically complex and lots of data needs to be processed.

The ill-fated Semantic package, despite all the effort invested on
implementing speed-ups, suffered badly from this and AFAIK it was one of
the main causes of its failure.

Even if you defer the actual analysis to an external tool (which is the
sane thing to do, BTW) the amount of data to be processed on the Emacs
end can be quite large, more than enough to introduce perceptible delays
that spoil the experience.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sun, 12 May 2019 17:46:54 +0200
> 
> > I agree that our situation with IDE features is quite bleak, but
> > that's entirely unrelated to the issues discussed in this thread, and
> > its reasons IMO have nothing to do with either ELisp speed or
> > flexibility of Emacs in general.
> 
> It has a lot to do. Analyzing source code is cpu-intensive, because it
> is algorithmically complex and lots of data needs to be processed.
> 
> The ill-fated Semantic package, despite all the effort invested on
> implementing speed-ups, suffered badly from this and AFAIK it was one of
> the main causes of its failure.
> 
> Even if you defer the actual analysis to an external tool (which is the
> sane thing to do, BTW) the amount of data to be processed on the Emacs
> end can be quite large, more than enough to introduce perceptible delays
> that spoil the experience.

It makes very little sense to me to build a case about bad performance
on a package that is incomplete, not very actively maintained, and
whose features are nowhere near what is needed for a good modern IDE.
It can maybe serve as an opportunity to let off steam, perhaps that's
what you wanted.

Or are you saying that this situation is due to the fact that ELisp is
a bad implementation language?

Re: Is Elisp really that slow?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Sun, 12 May 2019 17:46:54 +0200
>> 
>> > I agree that our situation with IDE features is quite bleak, but
>> > that's entirely unrelated to the issues discussed in this thread, and
>> > its reasons IMO have nothing to do with either ELisp speed or
>> > flexibility of Emacs in general.
>> 
>> It has a lot to do. Analyzing source code is cpu-intensive, because it
>> is algorithmically complex and lots of data needs to be processed.
>> 
>> The ill-fated Semantic package, despite all the effort invested on
>> implementing speed-ups, suffered badly from this and AFAIK it was one of
>> the main causes of its failure.
>> 
>> Even if you defer the actual analysis to an external tool (which is the
>> sane thing to do, BTW) the amount of data to be processed on the Emacs
>> end can be quite large, more than enough to introduce perceptible delays
>> that spoil the experience.
>
> It makes very little sense to me to build a case about bad performance
> on a package that is incomplete, not very actively maintained, and
> whose features are nowhere near what is needed for a good modern IDE.

Maybe this is because they hit some walls, like slowness on Elisp.

> It can maybe serve as an opportunity to let off steam, perhaps that's
> what you wanted.

When I started with Emacs around 20 years ago, whenever something was
incorrectly indented or fontified on C++ code I was writing it indicated
an error on my part. It was way ahead of any other editor. Nowadays CC
Mode can't handle C++ code styles and constructs that became available
shortly afterwards the period I mentioned. For languages that became
popular since the 1990's Emacs is barely adequate on some cases and a
non-starter on others.

I'm certainly frustrated because this is a self-inflicted problem.
Emacs' high-governance (and I mean "really high", you know what I mean)
forced some very bad decisions upon the project (and not only on Emacs)
which put us all, developers and users, on a really difficult position.
It is terrible that now I have to choose between a superb editor that is
dumb about what I'm writing and an average editor that is imbricated
with the language's tooling and thus greatly helps me to be more
productive.

As someone that swears for Emacs, yes, I need to let off some steam
whenever I see how certain topics are discussed.

> Or are you saying that this situation is due to the fact that ELisp is
> a bad implementation language?

Elisp is really good, I don't even dare to propose a migration to
another language.

I have on my to-do list for this year to grab Tromey's JIT branch and
migrate it to LLVM and see what the optimization passes can squeeze,
possibly with some simple static analysis and, if I'm in the mood, type
annotations on Elisp code.

Just for fun, because I know on advance that it is unacceptable on Emacs
repo.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sun, 12 May 2019 20:37:08 +0200
> 
> When I started with Emacs around 20 years ago, whenever something was
> incorrectly indented or fontified on C++ code I was writing it indicated
> an error on my part. It was way ahead of any other editor. Nowadays CC
> Mode can't handle C++ code styles and constructs that became available
> shortly afterwards the period I mentioned. For languages that became
> popular since the 1990's Emacs is barely adequate on some cases and a
> non-starter on others.
> 
> I'm certainly frustrated because this is a self-inflicted problem.
> Emacs' high-governance (and I mean "really high", you know what I mean)
> forced some very bad decisions upon the project (and not only on Emacs)
> which put us all, developers and users, on a really difficult position.

If you want to explain the problems of CC Mode by some "governance",
you are dead wrong, AFAIK.  The real reasons, IMO, are entirely
different.  At the very least, it was never proven that this has
anything to do with the problems we have.  Anyone who tracks
emacs-devel could easily formulate an alternative hypothesis, which
IMO is much more probable.

In general, problems in Emacs are rarely if ever due to management
(which doesn't really exist), they are almost always due to lack of
manpower, expertise, talent, free time, motivation, etc.

> I have on my to-do list for this year to grab Tromey's JIT branch and
> migrate it to LLVM and see what the optimization passes can squeeze,
> possibly with some simple static analysis and, if I'm in the mood, type
> annotations on Elisp code.

The JIT branch doesn't really work, and I have yet to see any speedup
from it in the configurations where it does compile.  So from my POV,
that experiment is a failure, and (having spent more than one or two
hours on making it work better) I see no reason why the failure could
be fixed by using LLVM, because libjit doesn't use any compiler in the
first place.

So, here too, I see a very weak connection, to say the least, between
your suspected reasons and the reality as I know it.

Re: Is Elisp really that slow?

[Stefan Monnier]

>> I agree that our situation with IDE features is quite bleak, but
>> that's entirely unrelated to the issues discussed in this thread, and
>> its reasons IMO have nothing to do with either ELisp speed or
>> flexibility of Emacs in general.
>
> It has a lot to do. Analyzing source code is cpu-intensive, because it
> is algorithmically complex and lots of data needs to be processed.

That's part of the problem, but I think it's a minor part of it: most
successful IDEs talk directly to the compiler instead of reimplementing
their own language-implementation-front-end (that requires changes to
the compiler, admittedly, but that's what the `I` is all about).

IOW besides performance, another big challenge is having to
(re)implement a complete language front-end.

I think that's why no effort in this direction took off until LSP:
without LSP, providing IDE-like features in Emacs requires either
hacking your compiler so it gives Emacs the relevant info (this has
been done for a few languages where Emacs users represent a large part
of the population (Lisp and Agda come to mind), but it's rather rare),
or reimplementing a front-end which is hard to justify (and requires
a lot of up-front work before you get any benefit at all).

Most major modes have just enough manpower to get a highlighter and
basic indenter up and running with incremental improvements after that.

Of course, the fact that performance may get in the way makes this even
less attractive.

> Even if you defer the actual analysis to an external tool (which is the
> sane thing to do, BTW)

Indeed.

> the amount of data to be processed on the Emacs end can be quite
> large, more than enough to introduce perceptible delays that spoil
> the experience.

With LSP front-ends using the C-implemented json parser present in recent
Emacsen, this should hopefully be less problematic, but indeed this is
a part where we need to put some of the code in C in order to get good
enough performance.

And the single-threadedness is an additional issue in this respect.


        Stefan

Re: Is Elisp really that slow?

[Stefan Monnier]

> I have on my to-do list for this year to grab Tromey's JIT branch and
> migrate it to LLVM and see what the optimization passes can squeeze,
> possibly with some simple static analysis and, if I'm in the mood, type
> annotations on Elisp code.

Elisp's current implementation is sufficiently naive that it should be
"easy" to make it significantly faster at least on some problems.

This said, it's also easy to overlook the fact that in most
circumstances time is spent running within functions implemented in
C and no amount of clever compilation will help (and clever compilation
can easily end up counterproductive because we spend a lot of resources
(heap space, CPU time, you name it) trying to optimize code that's
largely irrelevant, so the end result is slower startup and no speedup,
or worse).

If you want to do a JIT thingy that speeds up Elisp "in general" then
you need to make sure it does a good job eliminating redundant type
checks and things like that.  You'll probably easily get good speed ups
for the usual tight loops this way.  I encourage people to try something
like basic block versioning for that.

But I'm not sure such a "faster Elisp" will make a big difference to the
kind of code one can write in Elisp.  More specifically, I think it'll
take a lot of effort for a JIT to be good enough that we can implement
the json printer/parser in Elisp instead of C.

So another alternative is to make it possible to write "Elisp-ish" code
that's specifically tailored for compilation to efficient code and
that's specifically marked as such.  This has 2 very significant
advantages:
- the compiler can take its time because it's only applied to those
  specific cases where we (presumably) know it's worthwhile.  We may
  even run the compiler ahead of time rather than in a JIT fashion so it
  doesn't impact startup.
- the compiler doesn't need to accept all Elisp code and reproduce all
  its intricate details faithfully.  It can even signal errors and burp
  on some constructs that it doesn't want to support.
Then again, maybe this "Elisp-ish" could be fairly far removed
from Elisp.  E.g. it could even be plain old C (tho something in
between is likely more interesting).


        Stefan

Re: Is Elisp really that slow?

[Óscar Fuentes]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I have on my to-do list for this year to grab Tromey's JIT branch and
>> migrate it to LLVM and see what the optimization passes can squeeze,
>> possibly with some simple static analysis and, if I'm in the mood, type
>> annotations on Elisp code.
>
> Elisp's current implementation is sufficiently naive that it should be
> "easy" to make it significantly faster at least on some problems.

I'm not interested on making Elisp 30% faster. I want something that can
replace C for most purposes.

> This said, it's also easy to overlook the fact that in most
> circumstances time is spent running within functions implemented in
> C and no amount of clever compilation will help

Exactly. Elisp acts as glue code for the C base. This plus the fact that
many elisp primitives reduce to calls to C functions explains the
observed lack of impact of the JIT.

>(and clever compilation
> can easily end up counterproductive because we spend a lot of resources
> (heap space, CPU time, you name it) trying to optimize code that's
> largely irrelevant, so the end result is slower startup and no speedup,
> or worse).

Agreed.

> If you want to do a JIT thingy that speeds up Elisp "in general" then
> you need to make sure it does a good job eliminating redundant type
> checks and things like that.  You'll probably easily get good speed ups
> for the usual tight loops this way.  I encourage people to try something
> like basic block versioning for that.
>
> But I'm not sure such a "faster Elisp" will make a big difference to the
> kind of code one can write in Elisp.  More specifically, I think it'll
> take a lot of effort for a JIT to be good enough that we can implement
> the json printer/parser in Elisp instead of C.
>
> So another alternative is to make it possible to write "Elisp-ish" code
> that's specifically tailored for compilation to efficient code and
> that's specifically marked as such.  This has 2 very significant
> advantages:
> - the compiler can take its time because it's only applied to those
>   specific cases where we (presumably) know it's worthwhile.  We may
>   even run the compiler ahead of time rather than in a JIT fashion so it
>   doesn't impact startup.
> - the compiler doesn't need to accept all Elisp code and reproduce all
>   its intricate details faithfully.  It can even signal errors and burp
>   on some constructs that it doesn't want to support.
> Then again, maybe this "Elisp-ish" could be fairly far removed
> from Elisp.  E.g. it could even be plain old C (tho something in
> between is likely more interesting).

I've been advocating this Elisp-ish C for a long time. Even on your
stint as maintainer I offered my implementation of my own C-with-sexps
language.

I'm interested on islotating certain parts of Elisp, add some stuff to
help the optimizers and feed it to LLVM. Ideally, an Elisp hacker
working within those limits could implement some cpu-intensive code that
would perform similar to C with minimal retraining. At the same time,
the language must retain enough expressiveness to bring significant
advantages on development over C. defmacro is a must.

Re: Is Elisp really that slow?

[Samuel Wales]

can this be sped up?

               (while (re-search-forward from nil t)
                 (replace-match to t))

Re: Is Elisp really that slow?

[Emanuel Berg]

Eli Zaretskii wrote:

> I agree that our situation with IDE features
> is quite bleak

What IDE features are those? (Not
a rhetorical question.)

Actually in the literal sense of the word
(acronym) I think Emacs is the _ultimate_ IDE
for doing anything and everything with
a computer.

I couldn't stand having one program for HTML,
one for LaTeX, one for Lisp, and so on. And if
they (the dedicated IDEs) have special features
which we don't in our major modes, isn't this
also a question of ~"nobody has been motivated
to do it"?

And if nobody has or is, why care/complain
about it? Let it be, man.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[Emanuel Berg]

Óscar Fuentes wrote:

> When I started with Emacs around 20 years
> ago, whenever something was incorrectly
> indented or fontified on C++ code I was
> writing it indicated an error on my part.
> It was way ahead of any other editor.
> Nowadays CC Mode can't handle C++ code styles
> and constructs that became available shortly
> afterwards the period I mentioned.
> For languages that became popular since the
> 1990's Emacs is barely adequate on some cases
> and a non-starter on others.
>
> [...] It is terrible that now I have to
> choose between a superb editor that is dumb
> about what I'm writing and an average editor
> that is imbricated with the language's
> tooling and thus greatly helps me to be
> more productive.

OK, I see what you mean. Perhaps I was wrong in
my pervious message just now. You want it but
you can't improve it because you are busy
_using_ it. Not everyone are derailed like me
and can thus do whatever they want for no
money. Well, that ain't no good, what
you describe.

> I'm certainly frustrated because this is
> a self-inflicted problem.
> Emacs' high-governance (and I mean "really
> high", you know what I mean) forced some very
> bad decisions upon the project (and not only
> on Emacs) which put us all, developers and
> users, on a really difficult position.

What is this about? Please explain.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[Emanuel Berg]

Eli Zaretskii wrote:

> In general, problems in Emacs are rarely if
> ever due to management (which doesn't really
> exist), they are almost always due to lack of
> manpower, expertise, talent, free time,
> motivation, etc.

Perhaps one could strive to make the general
atmosphere more welcoming and attractive to
newcomers and young programmers... as
a collective effort, I mean.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[Emanuel Berg]

Stefan Monnier wrote:

> So another alternative is to make it possible
> to write "Elisp-ish" code that's specifically
> tailored for compilation to efficient code
> and that's specifically marked as such.
> This has 2 very significant advantages:
>
> - the compiler can take its time because it's
>   only applied to those specific cases where
>   we (presumably) know it's worthwhile.
>   We may even run the compiler ahead of time
>   rather than in a JIT fashion so it doesn't
>   impact startup.
>
> - the compiler doesn't need to accept all
>   Elisp code and reproduce all its intricate
>   details faithfully. It can even signal
>   errors and burp on some constructs that it
>   doesn't want to support.
>
> Then again, maybe this "Elisp-ish" could be
> fairly far removed from Elisp. E.g. it could
> even be plain old C (tho something in between
> is likely more interesting).

Yes, I thought about this as well! That would
be so cool and, as you say, interesting!
Only one would keep the Lisp syntax, right?
Just remove the compilation trouble spots and
see what's left. Probably quite a lot! And then
with the project started, people would get to
them trouble spots/areas and make them smaller
and smaller until some would disappear even!

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Why is Elisp slow?

[Van L]

> Michael Heerdegen writes:
> 
>>> ((())()'()).
> 
> I wonder how this would look like in C/C++.

You may find with luck an image archive (1)
of obfuscatory C/C++ source code ascii art.

(1) https://www.space.com/rosetta-comet-67p-huge-photo-archive.html

Re: Is Elisp really that slow?

[Stefan Monnier]

> can this be sped up?
>
>                (while (re-search-forward from nil t)
>                  (replace-match to t))

That's a perfect example where no matter how many man-years you invest
into a super-duper fancy compiler for Elisp, the above code will not be
affected one bit, because all the time is normally spent within
`re-search-forward` and `replace-match` which are already implemented
in C.

As for whether it can be sped up: yes, no doubt it can.  But how to
speed it up will depend on `from`, on the density of matches in the
text, and on the current major-mode, mostly (the complexity of `from`
impacts the amount of time spent in re-search-forward and which
approaches can reduce it, while the major-mode impacts the overhead of
modifying the buffer (via before/after-change-functions)).

I'd expect that in most cases, there won't be many matches so most of
the time is spent in re-search-forward.  This is already pretty fast in
the case where the pattern is simple, but if the pattern is complex, our
regexp-engine can be somewhere between not very efficient and
pathologically slow.  Fixing this would be great (even better if we can
do it by re-using an existing regexp implementation), especially if we
can do it without slowing down the simple case too much.


        Stefan

Re: Is Elisp really that slow?

[Emanuel Berg]

Stefan Monnier wrote:

>> can this be sped up?
>>
>>  (while (re-search-forward from nil t)
>>    (replace-match to t))
>
> That's a perfect example where no matter how
> many man-years you invest into a super-duper
> fancy compiler for Elisp, the above code will
> not be affected one bit, because all the time
> is normally spent within `re-search-forward`
> and `replace-match` which are already
> implemented in C.

How about this [code last]? I just wrote it.

Comments on code also welcome, as always, but
then perhaps a subject change is called for so
not to enrage anyone for being "OT" (...?).

The goal was

    - case sensitive regexp search

    - interactively, whole buffer if not
      region, otherwise region

    - from Lisp, whole buffer, if no optional
      args delimiting

What in particular will slow it down save for
the same old regexp matcher?

And how do you know what is slow and what
is fast?

And should you really think about that when you
write Elisp?

Anyway:


(defun case-sensitive-regexp-search-and-replace (regexp to-string &optional start stop)
  (interactive
   `(,(read-from-minibuffer "regexp: ")
     ,(read-from-minibuffer "replace: ")
     ,@(if (use-region-p) (list (region-beginning) (region-end))
         (list (point-min) (point-max))) ))
  (save-excursion
    (let ((beg (or start (point-min)))
          (end (or stop  (point-max))) )
      (goto-char beg)
      (let ((case-fold-search nil))
        (while (re-search-forward regexp end t)
          (replace-match to-string) )))))
(defalias 'cs-replace #'case-sensitive-regexp-search-and-replace)


File: https://dataswamp.org/~incal/emacs-init/edit.el

--
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[Emanuel Berg]

Stefan Monnier wrote:

>> can this be sped up?
>>
>>                (while (re-search-forward from nil t)
>>                  (replace-match to t))
>
> That's a perfect example where no matter how
> many man-years you invest into a super-duper
> fancy compiler for Elisp, the above code will
> not be affected one bit, because all the time
> is normally spent within `re-search-forward`
> and `replace-match` which are already
> implemented in C.

Wait, if so, isn't this the solution to the
whole problem?

1) Identify what is slow in Elisp.

2) Rewrite it in C?

What am I missing?

Is there some things that are slow _and_ cannot
easily be rewritten in C?

Or is there an Elisp-to-C transition penalty?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Emanuel Berg <moasenwood@zoho.eu>
> Date: Mon, 13 May 2019 03:27:54 +0200
> 
> Eli Zaretskii wrote:
> 
> > I agree that our situation with IDE features
> > is quite bleak
> 
> What IDE features are those? (Not
> a rhetorical question.)

Out of the important features listed in

  https://en.wikipedia.org/wiki/Integrated_development_environment

we completely lack support for code completion and refactoring. Our
debugging support is only up to speed with GDB, which means we are
limited in the programming languages we can support reasonably well.
(And even gdb-mi is just good, e.g., it cannot display complex
structures unless you install Python plugins; and its development
stagnated).  Our code search capabilities are rudimentary, and don't
support well modern languages like Java and C++.  So 4 out of 6
important features are either missing or have serious deficiencies.

Sure, add-on packages are available to provide some of the missing
features, but they don't always work well together, and ion any case,
Emacs should have these features built-in, if we want (and we do want)
to claim the title of programmer's editor.

> Actually in the literal sense of the word
> (acronym) I think Emacs is the _ultimate_ IDE
> for doing anything and everything with
> a computer.

In theory.  But not in practice, not at this time.

> And if nobody has or is, why care/complain
> about it? Let it be, man.

Because we want to be "the ultimate IDE".

Re: Is Elisp really that slow?

[Emanuel Berg]

Eli Zaretskii wrote:

> Out of the important features listed in
>
>   https://en.wikipedia.org/wiki/Integrated_development_environment
>
> we completely lack support for code
> completion and refactoring. Our debugging
> support is only up to speed with GDB, which
> means we are limited in the programming
> languages we can support reasonably well.

OK, so if I understand you correctly we are not
happy to be just an editor that writes code and
then relies on other program to the rest, like
compiling and debugging, only we do actually
want that, but we want it to be
more integrated?

And we want more language-specific features
like I guess Java has with Eclipse or C# with
.NET/Mono?

And we also disregard things like Dired, Gnus,
ERC, Emacs-w3m, and what have you, which are
a great help when developing... if you don't
talk to much instead of working, that is :$

We also disregard that when you work on
a project there are often several languages
involved, documentation, a home page, and so on
and we have all that all in one house with the
same finger-habits, help system, and
extension/configuration interface.

I think the whole (point) with Emacs is that it
is ready for everything, with much the same
approach. Want to learn a new language?
You can start immediately, as you don't have to
learn a new editor or "IDE" first!

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[John Yates]

On Mon, May 13, 2019 at 10:38 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> Our> debugging support is only up to speed with GDB ...

Separate from the display limitations Eli mentions a bit later
I would argue that the experience of debugging using gdb-mi with
gdb-many-windows set is quite painful.  At work various of my
team members use gdb via VSCode.  On unit and package tests the
performance is comparable.  Once one progresses to the company's
enormous full application the performance difference is startling.

/john

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: John Yates <john@yates-sheets.org>
> Date: Mon, 13 May 2019 11:02:41 -0400
> Cc: help-gnu-emacs@gnu.org
> 
> I would argue that the experience of debugging using gdb-mi with
> gdb-many-windows set is quite painful.

So even what I thought was still "good enough" isn't, really...
Hardly surprising, given that it's unmaintained for the last few
years.

> At work various of my
> team members use gdb via VSCode.  On unit and package tests the
> performance is comparable.  Once one progresses to the company's
> enormous full application the performance difference is startling.

Is this perhaps due to parsing the GDB/MI output?  Did you try to
profile that case?  If it's due to parsing, maybe switching it to use
the built-in JSON available with Emacs 27 could help (assuming that
profiling points to that as a hot spot)?

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Emanuel Berg <moasenwood@zoho.eu>
> Date: Mon, 13 May 2019 17:00:01 +0200
> 
> Eli Zaretskii wrote:
> 
> > Out of the important features listed in
> >
> >   https://en.wikipedia.org/wiki/Integrated_development_environment
> >
> > we completely lack support for code
> > completion and refactoring. Our debugging
> > support is only up to speed with GDB, which
> > means we are limited in the programming
> > languages we can support reasonably well.
> 
> OK, so if I understand you correctly we are not
> happy to be just an editor that writes code and
> then relies on other program to the rest, like
> compiling and debugging, only we do actually
> want that, but we want it to be
> more integrated?

No, you misunderstood.  We want these features to be as programmers
expect nowadays.  Whether some of the features depend on other
programs or not is immaterial, but if they do, the respective Emacs
front-ends should provide what the users expect.

GDB is not an IDE, it's a debugging engine.  It's okay to make an IDE
whose debugging facilities use GDB as their engine, but the front end
needs to provide the functionalities expected from debugging function
in an IDE.  And if GDB doesn't support some popular language, like
Python, we need an alternative for those who want to develop Python
programs; we don't want to tell them to go to the shell and invoke pdb
as a console program.

> And we want more language-specific features
> like I guess Java has with Eclipse or C# with
> .NET/Mono?

They are not language-specific features, they are in general features
needed in any language.

> And we also disregard things like Dired, Gnus,
> ERC, Emacs-w3m, and what have you, which are
> a great help when developing... if you don't
> talk to much instead of working, that is :$
> 
> We also disregard that when you work on
> a project there are often several languages
> involved, documentation, a home page, and so on
> and we have all that all in one house with the
> same finger-habits, help system, and
> extension/configuration interface.

No, we don't disregard anything.  But having Dired or the other
niceties is little comfort if what you need is to refactor your code
or debug it.

> I think the whole (point) with Emacs is that it
> is ready for everything, with much the same
> approach. Want to learn a new language?
> You can start immediately, as you don't have to
> learn a new editor or "IDE" first!

What if I don't want to learn a language, but to program in it?
I want the tools programmers nowadays expect to have from an
environment that claims to be for programmers.

Re: Is Elisp really that slow?

[Van L]

Emanuel Berg writes:

> Eli Zaretskii writes:
>
> Actually in the literal sense of the word
> (acronym) I think Emacs is the _ultimate_ IDE
> for doing anything and everything with
> a computer.
>
>8 [snip]
>
> And if nobody has or is, why care/complain
> about it? Let it be, man.

There's the `Crit 1' need for a new
interdisciplinarians's field, you name it
`machine behavior,' that affords Emacs the opportunity to be the fittest
naturally in the niche.

-- 
© 2019 VanL
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
                    "I had to learn through trial and error" - Takao Morita

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 13.05.2019 17:38, Eli Zaretskii wrote:
> Out of the important features listed in
> 
>    https://en.wikipedia.org/wiki/Integrated_development_environment
> 
> we completely lack support for code completion and refactoring.

JFYI: Eglot, published in GNU ELPA, is a an adequate LSP client, and it 
integrates with Company (for completion) and Flymake (to show syntax 
errors, missing imports, etc). Most of the time the user just needs to 
install a corresponding language server program.

It doesn't offer as comprehensive set of refactorings as the 
contemporary Java IDEs, but it does support some.

> Our code search capabilities are rudimentary, and don't
> support well modern languages like Java and C++.

See above. There are language servers for Java and C++.

> Sure, add-on packages are available to provide some of the missing
> features, but they don't always work well together, and ion any case,
> Emacs should have these features built-in, if we want (and we do want)
> to claim the title of programmer's editor.

I think we'll get there after we figure out how to bundle some ELPA 
packages together with the core for releases.

Re: Is Elisp really that slow?

[Paul W. Rankin]

On Tue, May 14 2019, Eli Zaretskii wrote:

> Emacs should have these features built-in, if we want (and we do want) 
> to claim the title of programmer's editor.

<aside>
Just because it sometimes gets overlooked, I think it's worth not losing 
sight of Emacs as a text editor for non-programmers also.
</aside>

--
https://www.paulwrankin.com

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 498 bytes --]

On Mon, May 13, 2019 at 04:33:12PM +0200, Emanuel Berg wrote:

[...]

> 1) Identify what is slow in Elisp.
> 
> 2) Rewrite it in C?

3) Write a regexp-to-assembler compiler. Bonus points if that runs
on the GPU, using its SIMD features.

(Cuz the slow parts upthread were the regexp parts anyway, which
are implemented in a sub-language independent of Elisp and embedded
in C).

Caveat: the regexp compiler already does parts of this. Perhaps
not always optimally.

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Emanuel Berg]

Eli Zaretskii wrote:

>> OK, so if I understand you correctly we are
>> not happy to be just an editor that writes
>> code and then relies on other program to the
>> rest, like compiling and debugging, only we
>> do actually want that, but we want it to be
>> more integrated?
>
> No, you misunderstood. We want these features
> to be as programmers expect nowadays.
> Whether some of the features depend on other
> programs or not is immaterial, but if they
> do, the respective Emacs front-ends should
> provide what the users expect.
>
> GDB is not an IDE, it's a debugging engine.
> It's okay to make an IDE whose debugging
> facilities use GDB as their engine, but the
> front end needs to provide the
> functionalities expected from debugging
> function in an IDE.

Yes, that's what I meant. I never did lots'a
debugging myself so perhaps I used the
incorrect/misleading words. But that's what
I meant.

> And if GDB doesn't support some popular
> language, like Python, we need an alternative
> for those who want to develop Python
> programs; we don't want to tell them to go to
> the shell and invoke pdb as
> a console program.

Again, that's what I thought I said, or wanted
to say - you are fine with using external
programs to do things, which I think is 100%
the right thing to do (to be), only you want it
integrated with Emacs.

Exactly what kind of integration, I don't know,
as I lack experience doing that even as a user,
_also_ I certainly don't feel there is
_anything wrong at all_ to "to go to the shell
and invoke pdb as a console program." I do such
things all day every day and feel no need to
change that.

>> And we want more language-specific features
>> like I guess Java has with Eclipse or C#
>> with .NET/Mono?
>
> They are not language-specific features, they
> are in general features needed in any language.

OK, FTR "needed" isn't a word to my liking...
But what I mean is they are language-specific
in terms of how one would go about and
implement them. font-lock and `forward-char'
and much of Emacs isn't.

And actually, _some_ of them features must
really be language-specific also in the sense
you refer to (which, I agree, is the correct
literal interpretation of what I said) - right?
Because I can't imagine you want all the same
little helpers for every single language
there is?

>> And we also disregard things like Dired,
>> Gnus, ERC, Emacs-w3m, and what have you,
>> which are a great help when developing... if
>> you don't talk to much instead of working,
>> that is :$
>> 
>> We also disregard that when you work on
>> a project there are often several languages
>> involved, documentation, a home page, and so
>> on and we have all that all in one house
>> with the same finger-habits, help system,
>> and extension/configuration interface.
>
> No, we don't disregard anything. But having
> Dired or the other niceties is little comfort
> if what you need is to refactor your code or
> debug it.

Right, but when we compare our editor to the
supposedly superior ones [1], I mean. They are
perhaps better IDEs in terms of
_a single language_, but we are the better
(the best?) "IDE" in terms of the
whole computer!

No doubt they will have an advantage for their
particular language as they put all their
effort into that and only that. I think that is
completely natural: a young guy can be in
terrific shape mentally and physically but not
one such guy on the whole geosphere can be the
best in every single Olympic discipline.
Impossible!

>> I think the whole (point) with Emacs is that
>> it is ready for everything, with much the
>> same approach. Want to learn a new language?
>> You can start immediately, as you don't have
>> to learn a new editor or "IDE" first!
>
> What if I don't want to learn a language, but
> to program in it? I want the tools
> programmers nowadays expect to have from an
> environment that claims to be
> for programmers.

I never used those tools so I don't know what
I'm missing. Programming in Emacs I've done in
some 20ish languages with no problems - not
form Emacs, at least :)

Sure, I didn't do it with a deadline, a 40+h
a work week, a deadline, and a I don't know how
many digit salary. And this is probably part of
the difference. I don't really care about being
the best. I want to be something I like and
enjoy. Emacs is certainly good enough for that.
But I respect your efforts and different POV,
for sure, and God willing I can find a spot to
help you some day.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[Emanuel Berg]

Paul W. Rankin wrote:

>> Emacs should have these features built-in,
>> if we want (and we do want) to claim the
>> title of programmer's editor.
>
> Just because it sometimes gets overlooked,
> I think it's worth not losing sight of Emacs
> as a text editor for non-programmers also.

No one is forgetting about that, here the topic
is the speed of Elisp and the lack of some
programmer editor's features tho :)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[Emanuel Berg]

tomas wrote:

>> 1) Identify what is slow in Elisp.
>> 
>> 2) Rewrite it in C?
>
> 3) Write a regexp-to-assembler compiler.
> Bonus points if that runs on the GPU, using
> its SIMD features.

Please DIY Mr. Zerolo, personally I'd be happy
to try out translating Elisp into C.

I could start with some of my own Elisp,
I mean, I certainly have enough of it...

I know how to write C in general, but how does
one integrate it into Emacs?

Also, I suppose you just don't write the same
Elisp program, only this time in C, but instead
there are certain building blocks that should
correspond to certain things? Not all of it,
but to some extent?

The easiest way to understand this is probably
to see one Elisp defun, and then see the
C version, and then if there is a table
somewhere with the specific building blocks
and conventions all mentioned?

Or if there's a tutorial ~"How to become an
Emacs C programmer" for those who already know,
imperfectly of course but still, Elisp and C?

At this point I'm not talking about changing
actual Emacs, I just want to try it out for my
own purposes.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[Stefan Monnier]

Óscar Fuentes <ofv@wanadoo.es> writes:
> I've been advocating this Elisp-ish C for a long time.  Even on your
> stint as maintainer I offered my implementation of my own C-with-sexps
> language.

Damn!  That doesn't ring a bell :-(

Óscar Fuentes <ofv@wanadoo.es> writes:
> I'm interested on islotating certain parts of Elisp, add some stuff to
> help the optimizers and feed it to LLVM. Ideally, an Elisp hacker
> working within those limits could implement some cpu-intensive code that
> would perform similar to C with minimal retraining. At the same time,
> the language must retain enough expressiveness to bring significant
> advantages on development over C. defmacro is a must.

Emanuel Berg <moasenwood@zoho.eu> writes:
> Yes, I thought about this as well! That would
> be so cool and, as you say, interesting!
> Only one would keep the Lisp syntax, right?

Of course, there's the question of how what this language should look
like, but I think more importantly, there's the question of how its
compiler is used:
- can any random Emacs user download your elisp-ish-c code and run it
  with the same ease as if it were written in Elisp?
- I.e. does it require re-starting Emacs?
  Does it require installing a C compiler or some such external tool?
  Does it work on all the architectures supported by Emacs?
- can incorrect elisp-ish-c code cause your Emacs to seg-fault?


        Stefan

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1637 bytes --]

On Tue, May 14, 2019 at 03:21:54PM +0200, Emanuel Berg wrote:
> tomas wrote:
> 
> >> 1) Identify what is slow in Elisp.
> >> 
> >> 2) Rewrite it in C?
> >
> > 3) Write a regexp-to-assembler compiler.
> > Bonus points if that runs on the GPU, using
> > its SIMD features.
> 
> Please DIY Mr. Zerolo, personally I'd be happy
> to try out translating Elisp into C.

But regexp match /was/ the bottleneck in the code sample
you posted, whose core loop is:

  (while (re-search-forward regexp end t)
          (replace-match to-string) )

(go benchmark it, Emacs offers great facilities for that).
So no amount of Elisp -> C translation or other Elisp
speedup will make your code noticeably faster.

[...]

> I know how to write C in general, but how does
> one integrate it into Emacs?

Again, Emacs is happy to tell you. Just do C-f (aka
describe-function), enter re-search-forward <ENTER> and
you get a small blurb:

    re-search-forward is an interactive built-in function in
    ‘C source code’.

    (re-search-forward REGEXP &optional BOUND NOERROR COUNT)

    Probably introduced at or before Emacs version 19.29.

    Search forward from point for regular expression REGEXP.
    [...more interesting text here...]

Notice how this ‘C source code’ above is a link? Follow it,
either by putting point on it and hitting ENTER or by mouse-
clicking on it, and presto, you are in the middle of a C
function written to be called from Elisp (you compile your
Emacs from source code, do you?).

In the manual, there is "E7. Writing Emacs Primitives".

Have fun...

cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Óscar Fuentes]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Of course, there's the question of how what this language should look
> like, but I think more importantly, there's the question of how its
> compiler is used:
> - can any random Emacs user download your elisp-ish-c code and run it
>   with the same ease as if it were written in Elisp?

See below.

> - I.e. does it require re-starting Emacs?

I hope not.

>   Does it require installing a C compiler or some such external tool?

No. Just LLVM as a library.

>   Does it work on all the architectures supported by Emacs?

It would work on all architectures supported by LLVM.

> - can incorrect elisp-ish-c code cause your Emacs to seg-fault?

For code that the user can download or write himself, this is
undesirable and therefore automatic checks must be added. For code
written by the Emacs developers and meant to compete with C, explicit
checks make sense.

One thing that cannot be left out from this C-ish Elisp is garbage
collection, else the language would have a very different runtime model.
This is probably the most difficult part because GC is something that
you must add on top of LLVM.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Emanuel Berg <moasenwood@zoho.eu>
> Date: Tue, 14 May 2019 13:54:15 +0200
> 
> _also_ I certainly don't feel there is
> _anything wrong at all_ to "to go to the shell
> and invoke pdb as a console program."

Modern debugging front-ends show multiple windows every time the
debuggee stops: a window with the source, another with local
variables, yet another with callstack (a.k.a. "backtrace"), one more
with program input/output, optionally threads and registers, etc.
Displaying that on the console requires the user to type separate
commands for each display, which is time consuming, and also pushes
the other stuff off the visible portion of the console window (so you
need to retype the same commands, or scroll the window with the mouse,
to see that again).  So using a debugger from a console is much less
convenient, when you deal with a complex program, and users nowadays
expect much more from an IDE.

> >> And we want more language-specific features
> >> like I guess Java has with Eclipse or C#
> >> with .NET/Mono?
> >
> > They are not language-specific features, they
> > are in general features needed in any language.
> 
> OK, FTR "needed" isn't a word to my liking...
> But what I mean is they are language-specific
> in terms of how one would go about and
> implement them. font-lock and `forward-char'
> and much of Emacs isn't.

Actually, font-lock needs to be customized for each language, and
there are commands that move by functions, blocks, and other lexical
units, which also need to understand the language to some extent.
IOW, someone must write the code to support each language with the
infrastructure provided by Emacs; if we didn't write that, we cannot
claim we support the language "because the tools are there".

> Because I can't imagine you want all the same
> little helpers for every single language
> there is?

Not sure what you mean by "all the same little helpers", but in
general every language needs its own variety of the same "helpers",
yes.

> > No, we don't disregard anything. But having
> > Dired or the other niceties is little comfort
> > if what you need is to refactor your code or
> > debug it.
> 
> Right, but when we compare our editor to the
> supposedly superior ones [1], I mean. They are
> perhaps better IDEs in terms of
> _a single language_, but we are the better
> (the best?) "IDE" in terms of the
> whole computer!

This argument doesn't fly with users, because newcomers usually need
just one language, but they need a good support for it.  They will go
away if we don't have it, and telling them we support a dozen others
will not make them change their minds.

To keep Emacs alive and kicking for the observable future, we need to
be sure we will have enough developers and contributors.  And since
developers and contributors start as users, we want to attract new
users.  If we fail to attract them, we will quite literally lose our
future.

> I never used those tools so I don't know what
> I'm missing. Programming in Emacs I've done in
> some 20ish languages with no problems - not
> form Emacs, at least :)
> 
> Sure, I didn't do it with a deadline, a 40+h
> a work week, a deadline, and a I don't know how
> many digit salary. And this is probably part of
> the difference.

I think a more important factor is the size of the project you work
on.

Re: Is Elisp really that slow?

[Emanuel Berg]

Eli Zaretskii wrote:

> Modern debugging front-ends show multiple
> windows every time the debuggee stops:
> a window with the source, another with local
> variables, yet another with callstack (a.k.a.
> "backtrace"), one more with program
> input/output, optionally threads and
> registers, etc. Displaying that on the
> console requires the user to type separate
> commands for each display, which is time
> consuming, and also pushes the other stuff
> off the visible portion of the console window
> (so you need to retype the same commands, or
> scroll the window with the mouse, to see that
> again). So using a debugger from a console is
> much less convenient, when you deal with
> a complex program, and users nowadays expect
> much more from an IDE.

OK, point taken. Users, newcomers, aren't
impressed by Emacs. The don't dig the console
and they use the mouse. And they want
everything integrated, all tho, which is
amusing, integration is limited to
a single language.

(Maybe MS IDEs can do all of C#, VB(A), and
MS Access now that I think about it...?)

> Actually, font-lock needs to be customized
> for each language, and there are commands
> that move by functions, blocks, and other
> lexical units, which also need to understand
> the language to some extent. IOW, someone
> must write the code to support each language
> with the infrastructure provided by Emacs; if
> we didn't write that, we cannot claim we
> support the language "because the tools are
> there".

OK, now it gets totally confused %) The
different language major modes are the absolute
_backbone_ of Emacs as a programmer editor,
indispensible! Obviously we want them for every
conceivable language to be as good as possible!
Setting up font-lock and indentation for
a programming language major mode (which even
I have done BTW) isn't what I thought we
were talking about rather much more advanced
stuff, the refactoring stuff (how ever that
works), integrating the debugger and build
process, and so on, and as for
language-specifics, qualitatively different
stuff, not just configuring and tweaking with
the same old interface!
<http://user.it.uu.se/~embe8573/fps/>

> This argument doesn't fly with users, because
> newcomers usually need just one language, but
> they need a good support for it. They will go
> away if we don't have it, and telling them we
> support a dozen others will not make them
> change their minds.
>
> To keep Emacs alive and kicking for the
> observable future, we need to be sure we will
> have enough developers and contributors.
> And since developers and contributors start
> as users, we want to attract new users. If we
> fail to attract them, we will quite literally
> lose our future.

OK, good point.

>> I never used those tools so I don't know
>> what I'm missing. Programming in Emacs I've
>> done in some 20ish languages with no
>> problems - not form Emacs, at least :)
>> 
>> Sure, I didn't do it with a deadline, a 40+h
>> a work week, a deadline, and a I don't know
>> how many digit salary. And this is probably
>> part of the difference.
>
> I think a more important factor is the size
> of the project you work on.

Well, obviously you worked on much bigger
projects than I, but I did a university project
(check out the URL in my signature). The report
- compiled LaTeX and Biblatex, which I wrote in
Emacs - is 153 pages. I used C++, Lisp, zsh,
gnuplot, groff, and pic(1), and wrote all of
that in Emacs. Even the Makefile and several
textfiles :)

So how large should a project be before Emacs
becomes insufficient? A Quake? If so, then no,
regretfully I wasn't part of the team...

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Emanuel Berg <moasenwood@zoho.eu>
> Date: Tue, 14 May 2019 19:05:21 +0200
> 
> So how large should a project be before Emacs
> becomes insufficient?

It depends on the language, and I don't really know where to draw the
line.  On my daytime job I deal with C++ projects that have a few
million lines of code, and Emacs OOB is only borderline useful there.

Re: Is Elisp really that slow?

[Ergus]

Hi,

Sorry I lost the most recent messages because the mail engine disables
my account very often.

>
>OK, point taken. Users, newcomers, aren't
>impressed by Emacs. The don't dig the console
>and they use the mouse. And they want
>everything integrated, all tho, which is
>amusing, integration is limited to
>a single language.
>
Not only that, see below.
>
>(Maybe MS IDEs can do all of C#, VB(A), and
>MS Access now that I think about it...?)
>
Yes they can, VS code supports many of them, butt especially the most
popular.
>
>OK, now it gets totally confused %) The
>different language major modes are the absolute
>_backbone_ of Emacs as a programmer editor,
>indispensible! Obviously we want them for every
>conceivable language to be as good as possible!
>Setting up font-lock and indentation for
>a programming language major mode (which even
>I have done BTW) isn't what I thought we
>were talking about rather much more advanced
>stuff, the refactoring stuff (how ever that
>works), integrating the debugger and build
>process, and so on, and as for
>language-specifics, qualitatively different
>stuff, not just configuring and tweaking with
>the same old interface!
><http://user.it.uu.se/~embe8573/fps/>
>
This is something even advanced compared to a real integration we need
like unify commands bindings, interfaces and functionalities names.

I go every time to the same topic here, but all the time I only receive
complains and strong answers with strong feelings from the older
users. We NEED to update the interface (unify the bindings for all the
languages, unify packages with similar functionalities, delete unused
functionalities.) It is not a should, it is a must. 

To mention just one example: It does not make sense that C-c C-c
comments the current lines in C-mode, but sends the current sexep to
terminal in other modes, or send the messages in others. So emacs
somehow behaves as a different tool/editor for 3 different modes, so the
"unified bindings/behavior" is not an advantage anymore.

Or that we provide several options (or packages) to add parents with 300
customizable variables but a very bad default behavior. The existence of
spacemacs ergoemacs and other similar customization is a user's scream
for better defaults (I am not telling to ennable all what spacemacs
does, but we have functionalities that the users will never discover if
they don't go in the ddeep parts of the manual).

Usually the old users (who deserve the legacy behaviors), are also the
most skilled ones, so it is easier for them to add some lines to their
config, than for new users that we don't want to scare the first day
with Lisp.

>
>> This argument doesn't fly with users, because
>> newcomers usually need just one language, but
>> they need a good support for it. They will go
>> away if we don't have it, and telling them we
>> support a dozen others will not make them
>> change their minds.
>>
Sometimes new users also mix languages, but the worst supported ones are
the newer languages (Lua, Julia, Ruby, Python, C++ 11+, Rust) Which are
also what they need more often.

>> To keep Emacs alive and kicking for the
>> observable future, we need to be sure we will
>> have enough developers and contributors.
>> And since developers and contributors start
>> as users, we want to attract new users. If we
>> fail to attract them, we will quite literally
>> lose our future.
>
>OK, good point.

But also there is the fact that we are spending a lot of
effort/work/manpower in specific use cases and fancy functionalities
(web browsing, pdf reader, image shower) instead of looking and
prioritizing the general and basic editor functionalities (faster
movements commands, default bindings for comment/uncomment, select whole
line, infrastructure performance). So going to the specifics instead of
the generals.

AFAIR emacs started (and became popular) cloning the popular
functionalities in other editors and making them accessible with simple
macros. 

We just need to open sublime or athom or VS code and look all the
functionalities they provide for EDITING TEXT 2 clicks (or key binds)
far. That's the real reason of their success. Basic functionality out
of the box.

What happens now is that if a user wants a simple editor with
indentation support and syntax highlight for multiple languages, they go
for vim (it is there already, is small and has many commands for editing
text quickly, and the learning curve is similar than for emacs and much
faster/responsive). Else, if they want something advanced, then they go
for an ide or a simpler (more familiar) editor that they can start using
without reading or configuring anything, geany, athom, sublime, visual
studio code. But if the project is big with autotools or cmake they just
go for a bigger ide like kdevelor, qtcreator, eclipse, NetBeans with a
better autocompletion, debugger, compiler, packer.

Lets say for example: sublime is extensible with a simpler language
(than (lisp)) that many people use these days, it is pretty by default,
and supports many languages. It works good enough, the code is on
github, the issues, pull requests and collaboration in general is
without an arcane mailing list, and a familiar fork-joing approach. So
from the point of view of a 20 yo developers "why to us emacs?" (It is a
rhetoric context question, please don't reply to it in your answers)

So the good thing is that emacs can provide all this, but is doesn't to
have all that working it needs years (literally) of
configurations/packages. Is that what a user will chose having easier
alternatives?

>
>
>Well, obviously you worked on much bigger
>projects than I, but I did a university project
>(check out the URL in my signature). The report
>- compiled LaTeX and Biblatex, which I wrote in
>Emacs - is 153 pages. I used C++, Lisp, zsh,
>gnuplot, groff, and pic(1), and wrote all of
>that in Emacs. Even the Makefile and several
>textfiles :)
>
>So how large should a project be before Emacs
>becomes insufficient? A Quake? If so, then no,
>regretfully I wasn't part of the team...

The problem is that (in my opinion) we lost the center of what emacs
should be and where are the practical-sustainable-maintainable limits a
long time ago (the difference between "we can" vs "we should").

The program grow an grow and grow and the
unused/old/substituted/unsuccess functionalities weren't deleted.

(We added the GUI over the TUI mixing everything and adding support for
every single detail we could imagine, multi os support forced to
implement everything generic.) And as a plus the Elisp language and
interpreter, two more things to maintain and update.

To maintain-support all that with (every time) less people, we have
abandoned (not improved) the editor-ide functionalities, and the world
continued moving in the mean time. (Also some needed changes have been
delayed and arrived veeeery late because "we shouldn't convert emacs in
the other editors" or similar opinions.

So, as usual in technology, other products filled the hole thinking in
the final user and not in the developers. So, in spite of our product is
better we don't find users for it because we don't know how to present
it to the new market.

Re: Is Elisp really that slow?

[Emanuel Berg]

tomas wrote:

>>>> 1) Identify what is slow in Elisp.
>>>> 
>>>> 2) Rewrite it in C?
>>>
>>> 3) Write a regexp-to-assembler compiler.
>>>    Bonus points if that runs on the GPU,
>>>    using its SIMD features.
>> 
>> Please DIY Mr. Zerolo, personally I'd be
>> happy to try out translating Elisp into C.
>
> But regexp match /was/ the bottleneck in the
> code sample you posted, whose core loop is:
>
>   (while (re-search-forward regexp end t)
>           (replace-match to-string) )
>
> (go benchmark it, Emacs offers great
> facilities for that). So no amount of Elisp
> -> C translation or other Elisp speedup will
> make your code noticeably faster.

:) I believe you! It is not that. Personally,
I don't really care that much for Emacs' speed.
I don't really care if other people use it or
not [1]. I mean, I'd _like_ it to be faster and for
more people to use it. But it doesn't bug me on
a daily basis, if you get me.

So from a personal POV, I would be
interesting/fun to write some of my Elisp into
C, and if I get good enough at it, maybe
I could even do it for the project.

You know my Elisp, hardly brilliant but
capable, and I think my C would be good again
if I picked it up after not doing it for
many years.

So the practical issue is how to do it not in
the Elisp sense, not in the C sense, but in the
_Elisp-to-C sense in the context of Emacs_.
And for that purpose, the Elisp I posted and
you referred to (thank you) is too complicated!
Baby steps first.

So, how do I write this in C with correct
conventions?

(defun hello-elisp-world ()
  "This docstring is for display purposes only."
  (interactive)
  (message "hello Elisp world!")
  )
;; (hello-elisp-world) ; eval me, otherwise you don't know what it does

Them, where do I put it (assuming it gets its
own file for test purposes) and do I need to
change the Makefile or build process in general
to make it work?

> Notice how this ‘C source code’ above is
> a link? Follow it, either by putting point on
> it and hitting ENTER or by mouse- clicking on
> it, and presto, you are in the middle of
> a C function written to be called from Elisp

Yes, I know about that, of course :) I think
you suspected that, but thanks anyway as I'm
not the only one reading this...

> (you compile your Emacs from source code, do
> you?).

I don't because I didn't get Emacs-w3m to work
last time I tried, and I always prefered the
repos, anyway. But installation of Emacs itself
was amazingly smooth. I solved the problem with
Emacs-w3m with "apt pinning", which I described
here: [2]

If your comment refers to the Emacs C source,
one can set that up even tho one has Emacs from
the repos. Apparently now it isn't for me, as
it doesn't work, but I have done it before, so
I know it's possible :) If your comment refers
to ~"make a manual install and start mucking
around", I can do that as well :)

> In the manual, there is "E7. Writing Emacs
> Primitives".

You mean this one:

    (info "(elisp) Writing Emacs Primitives")

?


[1] "I don't really care if other people use it
    or not" - this sounds like... something
    that isn't my impression, that other people
    don't use it! Because almost _every_
    programmer I meet on the street and we
    start to talk about our stuff, they use
    some Unix system, most often Linux,
    sometimes BSD. "What editor do you use?"
    "Emacs." ~2/3 times. Only perhaps they can
    read my mind and give me the answer I so
    much desire... Wait! I just said I didn't
    care! What the heck am I talking about?!?

[2] http://lists.gnu.org/archive/html/help-gnu-emacs/2019-04/msg00261.html

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[Emanuel Berg]

Eli Zaretskii wrote:

>> So how large should a project be before
>> Emacs becomes insufficient?
>
> It depends on the language, and I don't
> really know where to draw the line. On my
> daytime job I deal with C++ projects that
> have a few million lines of code, and Emacs
> OOB is only borderline useful there.

And what makes it not useful is the lack of
refactoring, and poor integration with
debugging and the build process?

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[Emanuel Berg]

Ergus wrote:

> I go every time to the same topic here, but
> all the time I only receive complains and
> strong answers with strong feelings from the
> older users.

Well, if you ever feel you get an unpleasant
answer from me, tell me immediately, thru an
off-list mail if necessary.

> We NEED to update the interface (unify the
> bindings for all the languages, unify
> packages with similar functionalities, delete
> unused functionalities.) It is not a should,
> it is a must.

Hey, man, relax. We don't need to do anything
and we certainly must not. We need food and
clothes and shelter. If you feel that what you
mention is so important, start working on it
today! It is much more rewarding and less
frustrating that trying to convince others you
are right. But as/if you do, please do continue
to tell us all about it, of course.

> To mention just one example: It does not make
> sense that C-c C-c comments the current lines
> in C-mode, but sends the current sexep to
> terminal in other modes, or send the messages
> in others. So emacs somehow behaves as
> a different tool/editor for 3 different
> modes, so the "unified bindings/behavior" is
> not an advantage anymore.

Well, Emacs isn't consistent in that sense.
I think it never will be. I don't see a huge
advantage having it consistent either, if that
is even possible, actually I see many
disadvantages to it. If the `C-c C-c', which is
a short and close (i.e. fast and good)
shortcut, if that is always to do the same
thing, many modes that do not do that _at all_
will be one short, close, and fast shortcut,
eh, "short" (... :)). Besides, people are used
to the shortcut! Changing them to have
potential future generation of Emacs users have
a more consistent interface will just make tons
of people, who already use Emacs every second
day, unhappy.

> Or that we provide several options (or
> packages) to add parents with 300
> customizable variables but a very bad default
> behavior. The existence of spacemacs
> ergoemacs and other similar customization is
> a user's scream for better defaults (I am not
> telling to ennable all what spacemacs does,
> but we have functionalities that the users
> will never discover if they don't go in the
> ddeep parts of the manual).

Find a spot and start working, is again my
piece of advice! Start small and post on
gmane.emacs.devel and see what happens.
Keep posting here as well tho. It is much
appreciated if you ever got the impression
it isn't.

> Usually the old users (who deserve the legacy
> behaviors), are also the most skilled ones,
> so it is easier for them to add some lines to
> their config, than for new users that we
> don't want to scare the first day with Lisp.

Not all new users are that afraid! Some think
Lisp is awesome and that is a big reason they
came to and stayed with Emacs. What one can do
with Lisp, in terms of Emacs, but also in terms
of Lisp itself, and actually even the very
language, Lisp, itself! Emacs user's
contribution to the Lisp world is huge.

> Sometimes new users also mix languages, but
> the worst supported ones are the newer
> languages (Lua, Julia, Ruby, Python, C++ 11+,
> Rust) Which are also what they need
> more often.

Again, pick your favorite of the ones you
mention and start working on what you think
is lacking!

> But also there is the fact that we are
> spending a lot of effort/work/manpower in
> specific use cases and fancy functionalities
> (web browsing, pdf reader, image shower)
> instead of looking and prioritizing the
> general and basic editor functionalities

OK, let me tell you straight off without being
unpleasant, I hope, this is a complete dead end
for you. Don't go there.

Because we *want* Dired, Emacs-w3m, Gnus, ERC,
and everything else! Just as much as we want
the programming modes.

There is no contradiction. On the
... contrary :)

> What happens now is that if a user wants
> a simple editor with indentation support and
> syntax highlight for multiple languages

I'm not following? We have that.

> The program grow an grow and grow and the
> unused/old/substituted/unsuccess
> functionalities weren't deleted.

IMO things that work should _never_ be deleted.
If it isn't used, like ever, perhaps move it to
ELPA. But don't delete it.

> And as a plus the Elisp language and
> interpreter, two more things to maintain
> and update.

Again, it is there because we _want_ it!

> So, in spite of our product is better [...]

Is it? The way you talk of it one would think
you wouldn't think so? But I agree, of
course :)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 3624 bytes --]

On Wed, May 15, 2019 at 01:25:25PM +0200, Emanuel Berg wrote:
> tomas wrote:

[...]

> :) I believe you! It is not that. Personally,
> I don't really care that much for Emacs' speed.

Me neither. But some might...

> I don't really care if other people use it or
> not [1]. I mean, I'd _like_ it to be faster and for
> more people to use it. But it doesn't bug me on
> a daily basis, if you get me.
> 
> So from a personal POV, I would be
> interesting/fun to write some of my Elisp into
> C, and if I get good enough at it, maybe
> I could even do it for the project.
> 
> You know my Elisp, hardly brilliant but
> capable, and I think my C would be good again
> if I picked it up after not doing it for
> many years.
> 
> So the practical issue is how to do it not in
> the Elisp sense, not in the C sense, but in the
> _Elisp-to-C sense in the context of Emacs_.
> And for that purpose, the Elisp I posted and
> you referred to (thank you) is too complicated!
> Baby steps first.
> 
> So, how do I write this in C with correct
> conventions?
> 
> (defun hello-elisp-world ()
>   "This docstring is for display purposes only."
>   (interactive)
>   (message "hello Elisp world!")
>   )
> ;; (hello-elisp-world) ; eval me, otherwise you don't know what it does
> 
> Them, where do I put it (assuming it gets its
> own file for test purposes) and do I need to
> change the Makefile or build process in general
> to make it work?

A bit pressed at the moment to provide you with
a working example, but just following the forward-char
pattern might get you far. But see below...

> > Notice how this ‘C source code’ above is
> > a link? Follow it, either by putting point on
> > it and hitting ENTER or by mouse- clicking on
> > it, and presto, you are in the middle of
> > a C function written to be called from Elisp
> 
> Yes, I know about that, of course :) I think
> you suspected that, but thanks anyway as I'm
> not the only one reading this...

Great :)

The point I wanted to make is that Emacs has been
very good at keeping itself hackable, and this seems
a more important metrics than raw speed, given Emacs's
"mission".

> > (you compile your Emacs from source code, do
> > you?).
> 
> I don't because I didn't get Emacs-w3m to work
> last time I tried, and I always prefered the
> repos, anyway. But installation of Emacs itself
> was amazingly smooth. I solved the problem with
> Emacs-w3m with "apt pinning", which I described
> here: [2]

Hm. I see. Did you file a bug report?

> If your comment refers to the Emacs C source,
> one can set that up even tho one has Emacs from
> the repos. Apparently now it isn't for me, as
> it doesn't work, but I have done it before, so
> I know it's possible :) If your comment refers
> to ~"make a manual install and start mucking
> around", I can do that as well :)

I think I dimly remember something in this list.

> > In the manual, there is "E7. Writing Emacs
> > Primitives".
> 
> You mean this one:
> 
>     (info "(elisp) Writing Emacs Primitives")

Yes. Following this has the downside that you end
up with a "modified Emacs". Thus, perhaps "E8 Writing
Dynamically-Loaded Modules", where your C functions
end up in a separate library, loaded by Emacs at
run time, is more interesting.

That said, it is always a trade-off: stuff written
in Elisp is far more hackable, so writing extensions
in C takes a lot of consideration and a good interface
design (how could people want to use my new function?),
given the "hackability cost".

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 15.05.2019 2:54, Ergus wrote:
> To mention just one example: It does not make sense that C-c C-c
> comments the current lines in C-mode, but sends the current sexep to
> terminal in other modes, or send the messages in others.

Indeed.

Maybe you should submit a bug report about that (or see if one is 
already open).

Re: Is Elisp really that slow?

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 15.05.2019 2:54, Ergus wrote:
>> To mention just one example: It does not make sense that C-c C-c
>> comments the current lines in C-mode, but sends the current sexep to
>> terminal in other modes, or send the messages in others.
>
> Indeed.
>
> Maybe you should submit a bug report about that (or see if one is
> already open).

As C-Mode has no associated terminal nor it composes messages, the
request would amount to "unbind C-c C-c", something no current or future
C/C++ user would benefit from.

OTOH C-Mode supports M-; (comment-dwim) which is an standard method of
(un)commenting code in Emacs.

Pretending that every mode conforms to the same rules is
counterproductive, because there are vast differences among them. We
shall strive for pragmatism, not for bureaucracy.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 15.05.2019 16:25, Óscar Fuentes wrote:
> OTOH C-Mode supports M-; (comment-dwim) which is an standard method of
> (un)commenting code in Emacs.

Right. So why the other binding?

> Pretending that every mode conforms to the same rules is
> counterproductive, because there are vast differences among them.

We also have different bindings for similar things in similar modes.

> We
> shall strive for pragmatism, not for bureaucracy.

Sure, but I don't understand your negative reaction in this case.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 15.05.2019 16:25, Óscar Fuentes wrote:
>> OTOH C-Mode supports M-; (comment-dwim) which is an standard method of
>> (un)commenting code in Emacs.
>
> Right. So why the other binding?

comment-dwin != comment-region

>> Pretending that every mode conforms to the same rules is
>> counterproductive, because there are vast differences among them.
>
> We also have different bindings for similar things in similar modes.

In principle, fixing those would be beneficial. OTOH, forcing existing
users to adapt just for the cause of coherence with modes they don't use
is not desirable either.

>> We shall strive for pragmatism, not for bureaucracy.
>
> Sure, but I don't understand your negative reaction in this case.

I think I already explained the issue, but I'll put it this way: what
you would do about C-c C-c on C-Mode and how that would help their
users?

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 15.05.2019 17:18, Óscar Fuentes wrote:

> comment-dwin != comment-region

Is the latter better frequently enough? If yes, why don't we have it in 
other modes?

> In principle, fixing those would be beneficial. OTOH, forcing existing
> users to adapt just for the cause of coherence with modes they don't use
> is not desirable either.

It's an investment in the future. Future users, at least. Or existing 
users who use some major modes and would benefit from consistency when 
trying out new ones. E.g. I don't use CC Mode often, and when I do, it's 
a whole new universe.

> I think I already explained the issue, but I'll put it this way: what
> you would do about C-c C-c on C-Mode and how that would help their
> users?

Pretty much what you suggested. Remove the binding and mention that in 
NEWS, together with the existing alternative.

Some users might be upset, most won't care, and some will be educated 
about the better alternative. The binding would also be freed for some 
feature in the future, like C-REPL maybe.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Emanuel Berg <moasenwood@zoho.eu>
> Date: Wed, 15 May 2019 13:27:31 +0200
> 
> > It depends on the language, and I don't
> > really know where to draw the line. On my
> > daytime job I deal with C++ projects that
> > have a few million lines of code, and Emacs
> > OOB is only borderline useful there.
> 
> And what makes it not useful is the lack of
> refactoring, and poor integration with
> debugging and the build process?

Debugging works well enough, but code search is very simplistic
(brings a lot of false positives, because it's basically dumb text
search, not sensitive enough to the language), and refactoring doesn't
work at all, because we have no facilities for that.  For small
projects, you could use 'A' or 'Q' in Dired, but that quickly becomes
unworkable in large projects.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> I think I already explained the issue, but I'll put it this way: what
>> you would do about C-c C-c on C-Mode and how that would help their
>> users?
>
> Pretty much what you suggested. Remove the binding and mention that in
> NEWS, together with the existing alternative.

You forgot to mention how removing C-c C-c would help CC-Mode users.

And, as already mentioned, comment-dwim is not an alternative to
comment-region. They do different things.

> Some users might be upset, most won't care, and some will be educated
> about the better alternative. The binding would also be freed for some
> feature in the future, like C-REPL maybe.

So we have certain downsides in exchange for hypothetical future
advantages.

Different modes have different requirements and it is natural that the
same bindings do specific things on each case.

Following your logic, as we have some modes where C-c C-c sends text to
a terminal and others where it is used to indicate that the current
edition has ended (and others where it interrupts current execution, as
in Eshell) we should decide that the binding will do one and only one
thing, and remove it from all other modes where that thing does not
exists.

At the end, we deplete the available bindings from each mode just for
the cause of coherence.

Doesn't look like an improvement to me.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> Date: Wed, 15 May 2019 01:54:12 +0200
> From: Ergus <spacibba@aol.com>
> 
> But also there is the fact that we are spending a lot of
> effort/work/manpower in specific use cases and fancy functionalities
> (web browsing, pdf reader, image shower) instead of looking and
> prioritizing the general and basic editor functionalities (faster
> movements commands, default bindings for comment/uncomment, select whole
> line, infrastructure performance).

There's no "we".  Each developer and contributor does what they think
is important/interesting/fun for them.  You can only call for
implementing some important functionality, but have no real
instruments to make that happen, except do it yourself (which might
not be possible due to lack of knowledge/talent/time).  Whether people
will follow your lead is entirely up to them.  Case in point: IDE
features.  We know for a long time we lag in this department, and
there were numerous calls for working on the related features.  The
result is before your eyes.

> AFAIR emacs started (and became popular) cloning the popular
> functionalities in other editors and making them accessible with simple
> macros. 

Actually, Emacs pioneered many features that didn't exist at all in
other editors.

Re: Is Elisp really that slow?

[Stefan Monnier]

> As C-Mode has no associated terminal nor it composes messages, the
> request would amount to "unbind C-c C-c",

That's right.

In my book, prog-mode should bind C-c C-c to `compile` and C-mode has no
need to have its own binding for it (because I can't think of a better
way to "compile/run" C code than via `compile`).  And it definitely
doesn't need a specific binding for comment-region since M-; already
covers this need.

> something no current or future C/C++ user would benefit from.

They would because the bindings would be more uniform across modes.

> Pretending that every mode conforms to the same rules is
> counterproductive, because there are vast differences among them.

Pretending that every detail of every major mode is unique is similarly
counter-productive.

> We shall strive for pragmatism, not for bureaucracy.

This is not about bureaucracy.  It's actually about streamlining the
overall system (by removing an unneeded chunk of code from CC-mode in
this case).

> comment-dwin != comment-region

I don't think that justifies having both bind to such short
key-sequences, nor does it justify binding comment-region to
a key-sequence usually bound to something completely unrelated.

> In principle, fixing those would be beneficial. OTOH, forcing existing
> users to adapt just for the cause of coherence with modes they don't use
> is not desirable either.

Short term pain for longer term gain.
There's no question that it's a tradeoff.

This particular C-c C-c binding in CC-mode has been on the "short term
gain for long term pain" for more than 10 years.


        Stefan

Re: Is Elisp really that slow?

[Óscar Fuentes]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> As C-Mode has no associated terminal nor it composes messages, the
>> request would amount to "unbind C-c C-c",
>
> That's right.
>
> In my book, prog-mode should bind C-c C-c to `compile`

That sounds good.

>> Pretending that every mode conforms to the same rules is
>> counterproductive, because there are vast differences among them.
>
> Pretending that every detail of every major mode is unique is similarly
> counter-productive.

Nobody argued for that.

About the pain associated with change: I started using Emacs for coding
C++. When years later I migrated to a new, lispy language, which mode I
derived from elisp-mode, guess what keybinding I copied from CC-Mode.

Changing long-established things that are deeply engraved in our muscle
memory is not easy, and on that aspect I can sympathize with the CC-Mode
maintainer.

I, personally, would accept your suggested change, adapt my config and
be done with it, but it is natural that others would react with
irritation.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 15.05.2019 18:14, Óscar Fuentes wrote:

>> Pretty much what you suggested. Remove the binding and mention that in
>> NEWS, together with the existing alternative.
> 
> You forgot to mention how removing C-c C-c would help CC-Mode users.

They, in particular, won't become confused when they try 'C-c C-c' in a 
different major mode, and it either fails to work, or does something 
unexpected to them.

> And, as already mentioned, comment-dwim is not an alternative to
> comment-region. They do different things.

You ignore, like, half the things I write. I am out of this discussion.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> Case in point: IDE features. We know for a long time we lag in this
> department, and there were numerous calls for working on the related
> features. The result is before your eyes.

Take a look outside Savannah and you will see that quite a lot of work
was done on that regard, at least for C++ IDE features.

The key part of the above paragraph is "outside Savannah".

>> AFAIR emacs started (and became popular) cloning the popular
>> functionalities in other editors and making them accessible with simple
>> macros. 
>
> Actually, Emacs pioneered many features that didn't exist at all in
> other editors.

So true.

Re: Is Elisp really that slow?

[Stefan Monnier]

> And, as already mentioned, comment-dwim is not an alternative to
> comment-region. They do different things.

I beg to differ.

> At the end, we deplete the available bindings from each mode just for
> the cause of coherence.

comment-region is not specific to C-mode, so if we want to bind it (to
C-c C-c or something else), it should be done elsewhere
(e.g. prog-mode).

The way things work in Emacs is that major-modes are written by users.
Those don't directly care about distinguishing what is really specific
to this major-mode from what is rather a personal preference or what
would be a generally useful functionality: they just want to get their
major mode to behave in a particular way.

Then, over time, as this major mode and its behavior is exposed to other
users, its maintainer(s) learn to distinguish between these differences,
so they remove some settings altogether (which proved to be personal
preferences of the author, typical examples being comment-indent,
indent-tabs-mode, word-syntax for the _ character, ...), and repackage
others as separate packages usable in different major modes
(e.g. tab-always-indent, electric-indent-mode, comment-region, smie,
...).

This second step is not necessarily in the best short-term interest of
users of this mode (not only because change is annoying but also
because often as feature are made usable in several modes they end up
dropping some very specific ad-hoc behavior which is too hard to
support in a generic way, ...), but it is the best long term interest of
Emacs as a whole.


        Stefan

Re: Is Elisp really that slow?

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 15.05.2019 18:14, Óscar Fuentes wrote:
>
>>> Pretty much what you suggested. Remove the binding and mention that in
>>> NEWS, together with the existing alternative.
>>
>> You forgot to mention how removing C-c C-c would help CC-Mode users.
>
> They, in particular, won't become confused when they try 'C-c C-c' in
> a different major mode, and it either fails to work, or does something
> unexpected to them.

AFAIK, modes where C-c C-c does something do not support the concept of
commenting regions. elisp-mode, for instance.

>> And, as already mentioned, comment-dwim is not an alternative to
>> comment-region. They do different things.
>
> You ignore, like, half the things I write. I am out of this discussion.

I do not understand your frustration.

The heuristics of comment-dwim not always gives the desired results. If
I know that I want to comment or uncomment a region, I prefer a method
that will do precisely that instead of something that tries to guess my
intention and can end doing the opposite thing depending on the presence
of commented-out text somewhere on the region.

Unless that by "better alternative" you referred to something else.

Re: Is Elisp really that slow?

[Stefan Monnier]

> I, personally, would accept your suggested change, adapt my config and
> be done with it, but it is natural that others would react with
> irritation.

I know it's natural.
I also get annoyed by some of those changes (including some that I install).
But maintainers have to look further than that (although they have to
take it into account).


        Stefan

Re: Is Elisp really that slow?

[Óscar Fuentes]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> On 15.05.2019 18:14, Óscar Fuentes wrote:
>>
>>>> Pretty much what you suggested. Remove the binding and mention that in
>>>> NEWS, together with the existing alternative.
>>>
>>> You forgot to mention how removing C-c C-c would help CC-Mode users.
>>
>> They, in particular, won't become confused when they try 'C-c C-c' in
>> a different major mode, and it either fails to work, or does something
>> unexpected to them.
>
> AFAIK, modes where C-c C-c does something do not support the concept of
> commenting regions. elisp-mode, for instance.

That was half-baked.

An hypothetical C/C++ hacker is unlikely to try C-c C-c for
commenting-out a region on an interactive mode because he will first
learn that that binding is using for sending the code to the subprocess
and, furthermore, commenting-out regions is not something that one would
do as one of his first operations when one starts using those modes.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 15.05.2019 18:51, Óscar Fuentes wrote:

> AFAIK, modes where C-c C-c does something do not support the concept of
> commenting regions. elisp-mode, for instance.

If you meant ielm (elisp-mode has no C-c C-c), then C-; works there and 
even does something. REPLs can have multiline inputs too.

>>> And, as already mentioned, comment-dwim is not an alternative to
>>> comment-region. They do different things.
>>
>> You ignore, like, half the things I write. I am out of this discussion.
> 
> I do not understand your frustration.

Because, even with the paragraph below, you haven't addressed my 
response on that subject. And it was a one-line response, FFS.

Anyway, Stefan has written a more expanded message on the same subject.

> The heuristics of comment-dwim not always gives the desired results. If
> I know that I want to comment or uncomment a region, I prefer a method
> that will do precisely that instead of something that tries to guess my
> intention and can end doing the opposite thing depending on the presence
> of commented-out text somewhere on the region.

I don't see you arguing for adding a comment-region binding in other 
major modes.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 15.05.2019 19:05, Óscar Fuentes wrote:
> An hypothetical C/C++ hacker is unlikely to try C-c C-c for
> commenting-out a region on an interactive mode

I didn't just mean interactive modes. Other modes, where C-c C-c is 
simply unbound.

> because he will first
> learn that that binding

Unlikely, but never mind.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> The heuristics of comment-dwim not always gives the desired results. If
>> I know that I want to comment or uncomment a region, I prefer a method
>> that will do precisely that instead of something that tries to guess my
>> intention and can end doing the opposite thing depending on the presence
>> of commented-out text somewhere on the region.
>
> I don't see you arguing for adding a comment-region binding in other
> major modes.

I can show you my .emacs.el instead.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 15 May 2019 17:40:24 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Case in point: IDE features. We know for a long time we lag in this
> > department, and there were numerous calls for working on the related
> > features. The result is before your eyes.
> 
> Take a look outside Savannah and you will see that quite a lot of work
> was done on that regard, at least for C++ IDE features.
> 
> The key part of the above paragraph is "outside Savannah".

"Outside Savannah" is not relevant when we talk about Emacs.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 15.05.2019 19:12, Óscar Fuentes wrote:
> I can show you my .emacs.el instead.

That's not very relevant.

We are talking about the default bindings and what's best for Emacs 
users overall.

Re: Is Elisp really that slow?

[Tadeus Prastowo]

On Wed, May 15, 2019 at 6:14 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Óscar Fuentes <ofv@wanadoo.es>
> > Date: Wed, 15 May 2019 17:40:24 +0200
> >
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> > > Case in point: IDE features. We know for a long time we lag in this
> > > department, and there were numerous calls for working on the related
> > > features. The result is before your eyes.
> >
> > Take a look outside Savannah and you will see that quite a lot of work
> > was done on that regard, at least for C++ IDE features.
> >
> > The key part of the above paragraph is "outside Savannah".
>
> "Outside Savannah" is not relevant when we talk about Emacs.

Assuming that the authors have no problem assigning their copyrights
to FSF, what prevents the outside work from being integrated into
Emacs in Savannah to improve Emacs's C++ IDE features?

-- 
Best regards,
Tadeus

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 497 bytes --]

On Wed, May 15, 2019 at 06:23:10PM +0200, Tadeus Prastowo wrote:

[...]

> Assuming that the authors have no problem assigning their copyrights
> to FSF, what prevents the outside work from being integrated into
> Emacs in Savannah to improve Emacs's C++ IDE features?

You can't make working code by piling things up. You need some
"organizational principle" and some quality management. That's
the hard work maintainers do. Sometimes they have to reject patches.

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Tadeus Prastowo <tadeus.prastowo@unitn.it>
> Date: Wed, 15 May 2019 18:23:10 +0200
> 
> > "Outside Savannah" is not relevant when we talk about Emacs.
> 
> Assuming that the authors have no problem assigning their copyrights
> to FSF, what prevents the outside work from being integrated into
> Emacs in Savannah to improve Emacs's C++ IDE features?

Under those assumptions, and if that work doesn't use any non-free
software, nothing.

RE: Is Elisp really that slow?

[Drew Adams]

> > And, as already mentioned, comment-dwim is not an
> > alternative to comment-region. They do different
> > things.
> 
> I beg to differ.

And your begged difference, which presumably shows
that they do NOT do different things, is what? ;-)

I have nothing special to say here about `C-c C-c'
or `cc-mode'.  But I will say I agree with Oscar
that `comment-dwim' and `comment-region' have
different behaviors - so neither is a substitute
for the other.

In particular, `comment-region' lets you also
uncomment, including unnest, comments.

I use both, and I bind each to a short key sequence.

More precisely, I leave `comment-dwim' on `M-;'
and I bind `C-x C-;' to a command similar to
`comment-region' but that I find more useful:
`comment-region-lines'.  It comments/uncomments
whole lines.

(So it too differs from `comment-region' - so it
too is not an exact "replacement".)

As for key bindings:

Although I haven't bothered to change the binding of
`M-;', I think it's a shame and a waste to sacrifice
such a nice, short, _repeatable_ key sequence for a
command that does _nothing_ when you repeat it.

Far better to use `M-;' for some command that keeps
doing something when repeated (just hold down `M-;').
That's what Emacs should do eventually, IMHO.  No
urgency, but someday, when we find a really useful
repeatable command...

As for `comment-dwim': Since I use `C-x C-;'
(`comment-region-lines') for block commenting and
uncommenting, I never really use `M-;' for anything
other than an end-of-line comment.

`M-;' used to be bound to a command that did only
that: `indent-for-comment'.  And since that's all
I really use `M-;' for, the rest of `comment-dwim'
is, yes, wasted and replaceable by `comment-region'
or my `comment-region-line'.  `M-;' for eol comment,
`C-x C-;' for commenting/uncommenting lines.

(defun comment-region-lines (beg end &optional arg)
  "Like `comment-region', but comment/uncomment whole lines."
  (interactive "*r\nP")
  (when (> beg end)
    (setq beg  (prog1 end (setq end  beg))))
  (let ((bol  (save-excursion
                (goto-char beg)
                (line-beginning-position)))
        (eol  (save-excursion
                (goto-char end)
                (if (bolp) (point) (line-end-position)))))
    (comment-region bol eol arg)))

Re: Is Elisp really that slow?

[John Yates]

On Wed, May 15, 2019 at 11:56 AM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:
>
> I know it's natural.
> I also get annoyed by some of those changes (including some that I
install).
> But maintainers have to look further than that (although they have to
> take it into account).

I have been using emacs since 1992 (Lucid emacs at Apollo Computer).
I am approaching 70 and still happy to change habits.  I am far more
concerned about emacs attracting new converts and new development
talent than humoring crotchety current users.  (Does anyone really
think that they will abandon the tribe?)  All reasonably well thought
out changes that make emacs more regular and easier for newbies to
master have my vote.

/john

Re: Is Elisp really that slow?

[Ergus]

On Wed, May 15, 2019 at 01:30:56PM -0400, John Yates wrote:
>On Wed, May 15, 2019 at 11:56 AM Stefan Monnier <monnier@iro.umontreal.ca>
>wrote:
>>
>> I know it's natural.
>> I also get annoyed by some of those changes (including some that I
>install).
>> But maintainers have to look further than that (although they have to
>> take it into account).
>
>I have been using emacs since 1992 (Lucid emacs at Apollo Computer).
>I am approaching 70 and still happy to change habits.  I am far more
>concerned about emacs attracting new converts and new development
>talent than humoring crotchety current users.  (Does anyone really
>think that they will abandon the tribe?)  All reasonably well thought
>out changes that make emacs more regular and easier for newbies to
>master have my vote.
>
That's the attitude that makes emacs survive. 

>/john

Re: Is Elisp really that slow?

[Ergus]

On Wed, May 15, 2019 at 10:29:42AM -0700, Drew Adams wrote:
>> > And, as already mentioned, comment-dwim is not an
>> > alternative to comment-region. They do different
>> > things.
>>
>> I beg to differ.
>
>And your begged difference, which presumably shows
>that they do NOT do different things, is what? ;-)
>
>I have nothing special to say here about `C-c C-c'
>or `cc-mode'.  But I will say I agree with Oscar
>that `comment-dwim' and `comment-region' have
>different behaviors - so neither is a substitute
>for the other.
>
>In particular, `comment-region' lets you also
>uncomment, including unnest, comments.
>

To no waste bindings we could just modify comment-region to accept
negative prefix to uncomment. So C-- M-; uncoments region, and M-;
comments it. But now there will come another user saying that the dwim
version is better... we will never agree in anything this way.

>I use both, and I bind each to a short key sequence.
>
>More precisely, I leave `comment-dwim' on `M-;'
>and I bind `C-x C-;' to a command similar to
>`comment-region' but that I find more useful:
>`comment-region-lines'.  It comments/uncomments
>whole lines.
>
>(So it too differs from `comment-region' - so it
>too is not an exact "replacement".)
>
>As for key bindings:
>
>Although I haven't bothered to change the binding of
>`M-;', I think it's a shame and a waste to sacrifice
>such a nice, short, _repeatable_ key sequence for a
>command that does _nothing_ when you repeat it.
>
>Far better to use `M-;' for some command that keeps
>doing something when repeated (just hold down `M-;').
>That's what Emacs should do eventually, IMHO.  No
>urgency, but someday, when we find a really useful
>repeatable command...
>
>As for `comment-dwim': Since I use `C-x C-;'
>(`comment-region-lines') for block commenting and
>uncommenting, I never really use `M-;' for anything
>other than an end-of-line comment.
>
>`M-;' used to be bound to a command that did only
>that: `indent-for-comment'.  And since that's all
>I really use `M-;' for, the rest of `comment-dwim'
>is, yes, wasted and replaceable by `comment-region'
>or my `comment-region-line'.  `M-;' for eol comment,
>`C-x C-;' for commenting/uncommenting lines.
>
>(defun comment-region-lines (beg end &optional arg)
>  "Like `comment-region', but comment/uncomment whole lines."
>  (interactive "*r\nP")
>  (when (> beg end)
>    (setq beg  (prog1 end (setq end  beg))))
>  (let ((bol  (save-excursion
>                (goto-char beg)
>                (line-beginning-position)))
>        (eol  (save-excursion
>                (goto-char end)
>                (if (bolp) (point) (line-end-position)))))
>    (comment-region bol eol arg)))
>

This is the king of features I would really support. Enable line or
region by default. There is a package for that in melpa y find very
useful.

Re: Is Elisp really that slow?

[Ergus]

On Wed, May 15, 2019 at 04:18:26PM +0200, �scar Fuentes wrote:
>Dmitry Gutov <dgutov@yandex.ru> writes:
>
>> On 15.05.2019 16:25, �scar Fuentes wrote:
>>> OTOH C-Mode supports M-; (comment-dwim) which is an standard method of
>>> (un)commenting code in Emacs.
>>
>> Right. So why the other binding?
>
>comment-dwin != comment-region
>
>>> Pretending that every mode conforms to the same rules is
>>> counterproductive, because there are vast differences among them.
>>
>> We also have different bindings for similar things in similar modes.
>
>In principle, fixing those would be beneficial. OTOH, forcing existing
>users to adapt just for the cause of coherence with modes they don't use
>is not desirable either.
>
>>> We shall strive for pragmatism, not for bureaucracy.
>>
>> Sure, but I don't understand your negative reaction in this case.
>
>I think I already explained the issue, but I'll put it this way: what
>you would do about C-c C-c on C-Mode and how that would help their
>users?
>
>
Oscar changes and sooner or latter they will arrive the uniformity is
good for the users and for the developers. It can simplify code and
documentation, reduce learning curve, improve external packages avoiding
potential colapses with specific modes behaviors... and the only thin
is that some users get annoyed on the beginning, but doing it well the
advantage (short, medium and long term) is more than evident.

Re: Is Elisp really that slow?

[Ergus]

On Wed, May 15, 2019 at 05:45:48PM +0300, Dmitry Gutov wrote:
>On 15.05.2019 17:18, �scar Fuentes wrote:
>
>>comment-dwin != comment-region
>
>Is the latter better frequently enough? If yes, why don't we have it 
>in other modes?
>
>>In principle, fixing those would be beneficial. OTOH, forcing existing
>>users to adapt just for the cause of coherence with modes they don't use
>>is not desirable either.
>
>It's an investment in the future. Future users, at least. Or existing 
>users who use some major modes and would benefit from consistency when 
>trying out new ones. E.g. I don't use CC Mode often, and when I do, 
>it's a whole new universe.
>
+1000 points to this comment from my side.

>>I think I already explained the issue, but I'll put it this way: what
>>you would do about C-c C-c on C-Mode and how that would help their
>>users?
>
>Pretty much what you suggested. Remove the binding and mention that in 
>NEWS, together with the existing alternative.
>
>Some users might be upset, most won't care, and some will be educated 
>about the better alternative. The binding would also be freed for some 
>feature in the future, like C-REPL maybe.
>

RE: Is Elisp really that slow?

[Drew Adams]

> >I agree with Oscar that `comment-dwim' and `comment-region'
> >have different behaviors - so neither is a substitute
> >for the other.
> >
> >In particular, `comment-region' lets you also
> >uncomment, including unnest, comments.
> 
> To no waste bindings we could just modify comment-region to accept
> negative prefix to uncomment. So C-- M-; uncoments region, and M-;
> comments it. 

I don't get your point.  `comment-region' already
uncomments, with plain `C-u'.

> But now there will come another user saying that the dwim
> version is better... we will never agree in anything this way.

The point was that both `M-;' and `comment-region'
are useful, and that neither substitutes for the other.
It doesn't matter which one someone think is "better".
They're both available.

> >I use both, and I bind each to a short key sequence.
> >
> >More precisely, I leave `comment-dwim' on `M-;'
> >and I bind `C-x C-;' to a command similar to
> >`comment-region' but that I find more useful:
> >`comment-region-lines'.  It comments/uncomments
> >whole lines.
> >
> >(So it too differs from `comment-region' - so it
> >too is not an exact "replacement".)
> >
> >As for key bindings:
> >
> >Although I haven't bothered to change the binding of
> >`M-;', I think it's a shame and a waste to sacrifice
> >such a nice, short, _repeatable_ key sequence for a
> >command that does _nothing_ when you repeat it.
> >
> >Far better to use `M-;' for some command that keeps
> >doing something when repeated (just hold down `M-;').
> >That's what Emacs should do eventually, IMHO.  No
> >urgency, but someday, when we find a really useful
> >repeatable command...
> >
> >As for `comment-dwim': Since I use `C-x C-;'
> >(`comment-region-lines') for block commenting and
> >uncommenting, I never really use `M-;' for anything
> >other than an end-of-line comment.
> >
> >`M-;' used to be bound to a command that did only
> >that: `indent-for-comment'.  And since that's all
> >I really use `M-;' for, the rest of `comment-dwim'
> >is, yes, wasted and replaceable by `comment-region'
> >or my `comment-region-line'.  `M-;' for eol comment,
> >`C-x C-;' for commenting/uncommenting lines.
> >
> >(defun comment-region-lines (beg end &optional arg)
> >  "Like `comment-region', but comment/uncomment whole lines."
> >  (interactive "*r\nP")
> >  (when (> beg end)
> >    (setq beg  (prog1 end (setq end  beg))))
> >  (let ((bol  (save-excursion
> >                (goto-char beg)
> >                (line-beginning-position)))
> >        (eol  (save-excursion
> >                (goto-char end)
> >                (if (bolp) (point) (line-end-position)))))
> >    (comment-region bol eol arg)))
> 
> This is the king of features I would really support. Enable line or
> region by default. There is a package for that in melpa y find very
> useful.

I proposed it long ago to emacs-devel.  Other,
similar commands were also discussed.  Emacs-devel
decided not to go there (by adding such a command
or by giving it - or even `comment-region' - a
binding by default).  Apparently it was thought
that `M-;' is sufficient.  Giving something a DWIM
name makes it universally useful, I guess. ;-)

Well, `comment-region' still does have a "key"
binding in vanilla Emacs (at least in some modes),
but not a _keyboard_ key binding:

comment-region is on <menu-bar> <emacs-lisp> <comment-region>

Re: Is Elisp really that slow?

[Ergus]

On Wed, May 15, 2019 at 12:53:43PM -0700, Drew Adams wrote:
>> >I agree with Oscar that `comment-dwim' and `comment-region'
>> >have different behaviors - so neither is a substitute
>> >for the other.
>> >
>> >In particular, `comment-region' lets you also
>> >uncomment, including unnest, comments.
>>
>> To no waste bindings we could just modify comment-region to accept
>> negative prefix to uncomment. So C-- M-; uncoments region, and M-;
>> comments it.
>
>I don't get your point.  `comment-region' already
>uncomments, with plain `C-u'.
>
>> But now there will come another user saying that the dwim
>> version is better... we will never agree in anything this way.
>
>The point was that both `M-;' and `comment-region'
>are useful, and that neither substitutes for the other.
>It doesn't matter which one someone think is "better".
>They're both available.
>
>> >I use both, and I bind each to a short key sequence.
>> >
>> >More precisely, I leave `comment-dwim' on `M-;'
>> >and I bind `C-x C-;' to a command similar to
>> >`comment-region' but that I find more useful:
>> >`comment-region-lines'.  It comments/uncomments
>> >whole lines.
>> >
>> >(So it too differs from `comment-region' - so it
>> >too is not an exact "replacement".)
>> >
>> >As for key bindings:
>> >
>> >Although I haven't bothered to change the binding of
>> >`M-;', I think it's a shame and a waste to sacrifice
>> >such a nice, short, _repeatable_ key sequence for a
>> >command that does _nothing_ when you repeat it.
>> >
>> >Far better to use `M-;' for some command that keeps
>> >doing something when repeated (just hold down `M-;').
>> >That's what Emacs should do eventually, IMHO.  No
>> >urgency, but someday, when we find a really useful
>> >repeatable command...
>> >
>> >As for `comment-dwim': Since I use `C-x C-;'
>> >(`comment-region-lines') for block commenting and
>> >uncommenting, I never really use `M-;' for anything
>> >other than an end-of-line comment.
>> >
>> >`M-;' used to be bound to a command that did only
>> >that: `indent-for-comment'.  And since that's all
>> >I really use `M-;' for, the rest of `comment-dwim'
>> >is, yes, wasted and replaceable by `comment-region'
>> >or my `comment-region-line'.  `M-;' for eol comment,
>> >`C-x C-;' for commenting/uncommenting lines.
>> >
>> >(defun comment-region-lines (beg end &optional arg)
>> >  "Like `comment-region', but comment/uncomment whole lines."
>> >  (interactive "*r\nP")
>> >  (when (> beg end)
>> >    (setq beg  (prog1 end (setq end  beg))))
>> >  (let ((bol  (save-excursion
>> >                (goto-char beg)
>> >                (line-beginning-position)))
>> >        (eol  (save-excursion
>> >                (goto-char end)
>> >                (if (bolp) (point) (line-end-position)))))
>> >    (comment-region bol eol arg)))
>>
>> This is the king of features I would really support. Enable line or
>> region by default. There is a package for that in melpa y find very
>> useful.
>
>I proposed it long ago to emacs-devel.  Other,
>similar commands were also discussed.  Emacs-devel
>decided not to go there (by adding such a command
>or by giving it - or even `comment-region' - a
>binding by default).  Apparently it was thought
>that `M-;' is sufficient.  Giving something a DWIM
>name makes it universally useful, I guess. ;-)
>
>Well, `comment-region' still does have a "key"
>binding in vanilla Emacs (at least in some modes),
>but not a _keyboard_ key binding:
>
But that's the point. The common features in emacs should have the same
base behavior independently of the mode (comment region, hungry deletion
indent region, send to terminal (compile and execute)/ send mail/ commit)

>comment-region is on <menu-bar> <emacs-lisp> <comment-region>

Re: Is Elisp really that slow?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> > "Outside Savannah" is not relevant when we talk about Emacs.

If people are implementing the features but can't be integrated in stock
Emacs, the Savannah boundary becomes relevant.

>> Assuming that the authors have no problem assigning their copyrights
>> to FSF, what prevents the outside work from being integrated into
>> Emacs in Savannah to improve Emacs's C++ IDE features?
>
> Under those assumptions, and if that work doesn't use any non-free
> software, nothing.

Unfortunately, this is not true.

Re: Is Elisp really that slow?

[Ergus]

On Wed, May 15, 2019 at 04:06:42PM +0300, Dmitry Gutov wrote:
>On 15.05.2019 2:54, Ergus wrote:
>>To mention just one example: It does not make sense that C-c C-c
>>comments the current lines in C-mode, but sends the current sexep to
>>terminal in other modes, or send the messages in others.
>
>Indeed.
>
>Maybe you should submit a bug report about that (or see if one is 
>already open).
>
I could report that, I could even try to fix that myself and hope Alan
accepts the fix, but the problem is the social attitude about these kind
of changes.

Sometimes a benevolent dictator (or a less democratic management) is
needed to make things work. And I can't pretend to help that direction
because I am probably the less experiences and with least contributions
to the project.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 15.05.2019 23:15, Ergus wrote:
> I could report that, I could even try to fix that myself and hope Alan
> accepts the fix, but the problem is the social attitude about these kind
> of changes.
> 
> Sometimes a benevolent dictator (or a less democratic management) is
> needed to make things work. And I can't pretend to help that direction
> because I am probably the less experiences and with least contributions
> to the project.

M-x report-emacs-bug

You describe the problem and your personal opinion, and leave it at 
that. That's the contribution.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 15.05.2019 19:12, Óscar Fuentes wrote:
>> I can show you my .emacs.el instead.
>
> That's not very relevant.
>
> We are talking about the default bindings and what's best for Emacs
> users overall.

I'm way less sure than other participants on this thread about what's
best for Emacs users overall.

And on the CC-Mode C-c C-c subtopic, I put my C++ programmer's hat and
wonder how that makes Emacs better for me, appart from disrupting my
muscle memory and forcing me to adapt (1). Those improvements on
uniformity amount to nothing when you use a mode for serious work. They
are nice when read on a blog post promoting Emacs though.

1. I emphasize that I have no problem at all adapting to changes, I'm
   just providing my point of view about how much of an improvement this
   specific change would be for somebody that uses CC-Mode on a regular
   basis.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 15.05.2019 23:23, Óscar Fuentes wrote:

> And on the CC-Mode C-c C-c subtopic, I put my C++ programmer's hat and
> wonder how that makes Emacs better for me, appart from disrupting my
> muscle memory and forcing me to adapt (1). Those improvements on
> uniformity amount to nothing when you use a mode for serious work.

The irritation you're going to feel will likewise be minor. You'll put 
your own preferences into the init script and forget about the issue by 
the next day.

> 1. I emphasize that I have no problem at all adapting to changes, I'm
>     just providing my point of view about how much of an improvement this
>     specific change would be for somebody that uses CC-Mode on a regular
>     basis.

I think everybody and their dog have already agreed that the short-term 
effect on existing users who only do C++ would likely be negative, if small.

Re: Is Elisp really that slow?

[Ergus]

On Wed, May 15, 2019 at 05:14:18PM +0200, �scar Fuentes wrote:
>Dmitry Gutov <dgutov@yandex.ru> writes:
>
>>> I think I already explained the issue, but I'll put it this way: what
>>> you would do about C-c C-c on C-Mode and how that would help their
>>> users?
>>
>> Pretty much what you suggested. Remove the binding and mention that in
>> NEWS, together with the existing alternative.
>
>You forgot to mention how removing C-c C-c would help CC-Mode users.
>
>And, as already mentioned, comment-dwim is not an alternative to
>comment-region. They do different things.
>
I agree that they are not the same, but I think also that users use one
or the other in their workflow, so wasting 2 short (comfortable)
bindings for the same action is not the right to do in any case. Also
short commands like M-; should be prefixes (like C-c or C-x).

In the unification process that's something that may be organized too:
if the short bindings must provide simple commands like comment-region,
or if they will provide more heuristic versions with dwim. And if the
opposite actions will be prefixed with C-u or C--

>> Some users might be upset, most won't care, and some will be educated
>> about the better alternative. The binding would also be freed for some
>> feature in the future, like C-REPL maybe.
>
>So we have certain downsides in exchange for hypothetical future
>advantages.
>
>Different modes have different requirements and it is natural that the
>same bindings do specific things on each case.
>

Comment region is common to all languages, and send to terminal (or
compile and execute) is common to most languages.
 
>Following your logic, as we have some modes where C-c C-c sends text to
>a terminal and others where it is used to indicate that the current
>edition has ended (and others where it interrupts current execution, as
>in Eshell) we should decide that the binding will do one and only one
>thing, and remove it from all other modes where that thing does not
>exists.
>
So... I am scared now to be sincere here... but yes. That's better (and
more logical) than confusing and scare the users. Any way the bindings
must be associated with primitive actions (most primitive and abstract
the best because they will be applicable to more possible actions in
more options) so the derived modes just need to re-implement the action
but if they rebind the primitive in their config they still will keep
uniformity.
>
>At the end, we deplete the available bindings from each mode just for
>the cause of coherence.
>
But we won't have any conflict with external packages in some modes and
not in others because the actions and the reserved bindings will be
clear..
>
>Doesn't look like an improvement to me.
>
>
If everything is organized in advance it will be, actually it may
simplify and reduce a lot of redundant code, reduce the manual (good for
developers and the users because will need to read less to start
working), avoid conflicts in the mailing list (and discussions like
this). Help external developers to provide better packages at leas more
organized and standard without conflicts with the internal
functionalities.

Re: Is Elisp really that slow?

[Ergus]

On Wed, May 15, 2019 at 11:21:11PM +0300, Dmitry Gutov wrote:
>On 15.05.2019 23:15, Ergus wrote:
>>I could report that, I could even try to fix that myself and hope Alan
>>accepts the fix, but the problem is the social attitude about these kind
>>of changes.
>>
>>Sometimes a benevolent dictator (or a less democratic management) is
>>needed to make things work. And I can't pretend to help that direction
>>because I am probably the less experiences and with least contributions
>>to the project.
>
>M-x report-emacs-bug
>
>You describe the problem and your personal opinion, and leave it at 
>that. That's the contribution.
>
I know, but the problem is something I can even fix, but if there is not
agreement about how to fix it, or even if it should be changed, so it
will be just a bug more but like whole-line-or-region behavior, initial
script for new users, better defaults, indent-with-tabs align with
spaces, the problem is really there, the attitude about core changes.

Look how long this thread became with a single example I mention. And we
still don't have solution for it.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 15.05.2019 23:57, Ergus wrote:
> I know, but the problem is something I can even fix, but if there is not
> agreement about how to fix it, or even if it should be changed, so it
> will be just a bug more but like whole-line-or-region behavior, initial
> script for new users, better defaults, indent-with-tabs align with
> spaces, the problem is really there, the attitude about core changes.

What are you trying to accomplish with this email?

Re: Is Elisp really that slow?

[Ergus]

On Wed, May 15, 2019 at 07:14:10PM +0300, Eli Zaretskii wrote:
>> From: �scar Fuentes <ofv@wanadoo.es>
>> Date: Wed, 15 May 2019 17:40:24 +0200
>>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>> > Case in point: IDE features. We know for a long time we lag in this
>> > department, and there were numerous calls for working on the related
>> > features. The result is before your eyes.
>>
>> Take a look outside Savannah and you will see that quite a lot of work
>> was done on that regard, at least for C++ IDE features.
>>
>> The key part of the above paragraph is "outside Savannah".
>
>"Outside Savannah" is not relevant when we talk about Emacs.
>
Hi Eli:

We are far from becoming a toxic community, but new/different ideas are
not really very welcome and sometimes the arguments are like "it has
always been like that", "old users don't want that change".

The conservative attitude in emacs development group (apart from the
technical obstacles like the workflow and the paperwork and so on) gives
the idea of a closed environment (that actually it is from the
development point of view because we are like frozen in the past) where
only very experts are welcome (that's the vision people have from
outside) so very few melpa developers are really interested to face all
those issues to contribute.

Re: Is Elisp really that slow?

[Ergus]

On Thu, May 16, 2019 at 12:00:32AM +0300, Dmitry Gutov wrote:
>On 15.05.2019 23:57, Ergus wrote:
>>I know, but the problem is something I can even fix, but if there is not
>>agreement about how to fix it, or even if it should be changed, so it
>>will be just a bug more but like whole-line-or-region behavior, initial
>>script for new users, better defaults, indent-with-tabs align with
>>spaces, the problem is really there, the attitude about core changes.
>
>What are you trying to accomplish with this email?

That there are many pending bugs already to add one more for something
that others consider a feature and don't want to be change. 

Re: Is Elisp really that slow?

[Emanuel Berg]

tomas wrote:

> The point I wanted to make is that Emacs has
> been very good at keeping itself hackable,
> and this seems a more important metrics than
> raw speed, given Emacs's "mission".

To make Emacs less hackable isn't allowed :)

And Emacs in general is already fast. I do
things much faster than most people who use
modern GUIs. I type faster, I switch between
programs faster, everything is more integrated,
and so on - it is fast!

That doesn't mean I do more productive stuff
for every given computer hour. Sometimes it
works the opposite way, one discovers some
detail, gets an idea, and spend an hour on some
thing which really doesn't matter. But it is
fun and rewarding, so in a way it does. It is
meaningless, but if it's meaningless, one might
as well do it :)

But if we return to (in your words)
"raw speed", I like that as well, so it ain't
to dismiss the discussion.

(BTW did the subject change or am I tripping?!?)

> Hm. I see. Did you file a bug report?

I don't remember the details but it was solved
with the help from the gmane.emacs.w3m
guys, yes.

>> You mean this one:
>> 
>>     (info "(elisp) Writing Emacs Primitives")
>
> Yes. Following this has the downside that you
> end up with a "modified Emacs".

Right, but as you still need to recompile it,
you have no choice anyway, you must get the
source and build it manually, right?

But, one can use the repo Emacs for everyday
purposes, and then have one other to experiment
on.

Perhaps that gets tangled up in the initial
phase but I'm sure it gets worked
out eventually.

> Thus, perhaps "E8 Writing Dynamically-Loaded
> Modules", where your C functions end up in
> a separate library, loaded by Emacs at run
> time, is more interesting.

Aha, so you can do that! Wonderful!

Only... I don't find it?

;; DNC
(info "(emacs) Writing Dynamically-Loaded Modules")
(info "(elisp) Writing Dynamically-Loaded Modules")

Where is it?

And what is this "E8" notation?

> That said, it is always a trade-off: stuff
> written in Elisp is far more hackable, so
> writing extensions in C takes a lot of
> consideration and a good interface design
> (how could people want to use my new
> function?), given the "hackability cost".

Well, I've written enough Elisp by now anyway.
Time to move on :)

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg]

Óscar Fuentes wrote:

> So we have certain downsides in exchange for
> hypothetical future advantages.
>
> Different modes have different requirements
> and it is natural that the same bindings do
> specific things on each case.
>
> Following your logic, as we have some modes
> where C-c C-c sends text to a terminal and
> others where it is used to indicate that the
> current edition has ended (and others where
> it interrupts current execution, as in
> Eshell) we should decide that the binding
> will do one and only one thing, and remove it
> from all other modes where that thing does
> not exists.
>
> At the end, we deplete the available bindings
> from each mode just for the cause
> of coherence.
>
> Doesn't look like an improvement to me.

Agree 100%!

_Nothing_ that spans across a huge area of
<whatever> is *ever* consistent. Go by bike
thru the city at night and look for it.
You won't find it. And not just because the
city is dark at night!

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg]

Ergus wrote:

> But that's the point. The common features in
> emacs should have the same base behavior
> independently of the mode (comment region,
> hungry deletion indent region, send to
> terminal (compile and execute)/ send mail/
> commit)

It is desirable but not something to worry too
much about. Better be creative doing more
creative stuff.

Here is a list of bike tires [1]. As you see,
there are three different systems, or four
actually, and one has an extension one might
say, so let's settle for 4.5 systems. And all
those tires? Does it really make sense to have
one 56-559 and one 54-559 tire? Perhaps not!
But it's reality. And Emacs is also part
of reality.

But before we get lost in a principal
discussion... Isn't what you mention the case
already to a large extent?

When exactly is what key doing something
totally unexpected in your opinion?


[1] https://dataswamp.org/~incal/bike/TIRE

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg]

Ergus wrote:

> So... I am scared now to be sincere here...
> but yes. That's better (and more logical)
> than confusing and scare the users.

Again this talk about scaring the users.
Are computer people so darn afraid and fragile
these days? Without boasting, let me boast that
when I started Emacs the very first time,
I wasn't the least scared, in fact I thought it
was awesome, including the Lisp part, and
I have used it ever since.

> If everything is organized in advance it will
> be, actually it may simplify and reduce a lot
> of redundant code, reduce the manual (good
> for developers and the users because will
> need to read less to start working), avoid
> conflicts in the mailing list (and
> discussions like this). Help external
> developers to provide better packages at leas
> more organized and standard without conflicts
> with the internal functionalities.

??? All that will happen from changing a bunch
of keybindings?

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Ergus]

>Ergus wrote:
>
>> But that's the point. The common features in
>> emacs should have the same base behavior
>> independently of the mode (comment region,
>> hungry deletion indent region, send to
>> terminal (compile and execute)/ send mail/
>> commit)
>
>It is desirable but not something to worry too
>much about. Better be creative doing more
>creative stuff.
>
>Here is a list of bike tires [1]. As you see,
>there are three different systems, or four
>actually, and one has an extension one might
>say, so let's settle for 4.5 systems. And all
>those tires? Does it really make sense to have
>one 56-559 and one 54-559 tire? Perhaps not!
>But it's reality. And Emacs is also part
>of reality.
>
>But before we get lost in a principal
>discussion... Isn't what you mention the case
>already to a large extent?
>
There are already many attempts to do that (cua mode, evil, ergoemacs,
god-mode) so I am not for sure the first with these concerns but they
fail because of the incompatibilities and conflicts with other packages.

Finally the most of the new users that wants to use the terminal are
going to vim or vim like systems like spacemacs. It is not because
modal editing is better, it is because after the initial learning curve
they can just deduce most of the actions withing the schema. Also
because vim removed the bugs and inconsistencies existent in vi in spite
of many users used to exploit them.

So in the context of your example, our tire is 55 (incompatible with
everything else by far), but also it works in a special kind of bike
that needs a 50 tire on Mondays and Tuesdays, but is the user who needs
to do the changes. We need to maintain our own bikes, wheels, the
material, and as we don't have practical arguments about why we keep
that system in many cases except that it is because it is compatible
with the previous bikes we produced the last 40 years and the old users
are use to them and they already know how to fix them.

But the new users can find spare parts for the other systems anywhere,
and specialized personal in the other technologies, and they don't need
to be alert about the day of the week... So for him is an obvious
choice. Our tire is better, because it is the only tire that can change
size 2 times a week, but for him it does not represent an advantage.

>When exactly is what key doing something
>totally unexpected in your opinion?
>
>
>[1] https://dataswamp.org/~incal/bike/TIRE

-- 

Re: Is Elisp really that slow?

[Ergus]

> > So... I am scared now to be sincere here...
> > but yes. That's better (and more logical)
> > than confusing and scare the users.
> 
> Again this talk about scaring the users.
> Are computer people so darn afraid and fragile
> these days? Without boasting, let me boast that
> when I started Emacs the very first time,
> I wasn't the least scared, in fact I thought it
> was awesome, including the Lisp part, and
> I have used it ever since.
> 
Times has changes, developers too, alternative software grows like
mushrooms. What we used to do exceptionally better than the rest, now is
provided more or less by many other applications. The user experience is
different now and the needs too. Projects are bigger and more complex,
they mix languages constantly (just thin in Javascript+PHP+ASP+SQL+CSS),
languages are more complex... The choice to use a tool is based on how
more productive and easy to use it is, not in how powerful it is after
3000 lines of configuration and reading 5000 pages manual and learning
another programming  language.

Also there is a baseline already that the 99% of the users are use do
things differently (copy, paste, mouse wheel behavior, rectangular
selection, save, search, undo-redo) and the users don't want (and it
actually does not make too much sense) switch to M-w C-y and so on if
they already know a method that works for their browser, their games and
their chats.

So yes, they feel scared when they type emacs and they don't know how to
exit, or how to copy, paste, or search text with C-f, or save with
C-s... Suppose that even after that they persist and after 2 hours
reading a manual and a tutorial they try to comment a line in Python
mode with C-c C-c because they saw that it works for a c program in
stack overflow, or try to copy text from the minibuffer the same way they
do in the main buffer.

The user needs to see advantage over other systems/applications to feel
attracted enough to spend the initial time that emacs requires and deall
with the uncommon issues. In 1992 there were less alternatives around
and most of them were uglier, notably more limited or most
expensive. But in 2019 there are cheaper, prettier and free
alternatives. They are not as powerful as emacs, but they are more
ergonomic and do the work and include small details to make the life
easier||simpler||faster out of the box.
> 
> > If everything is organized in advance it will
> > be, actually it may simplify and reduce a lot
> > of redundant code, reduce the manual (good
> > for developers and the users because will
> > need to read less to start working), avoid
> > conflicts in the mailing list (and
> > discussions like this). Help external
> > developers to provide better packages at leas
> > more organized and standard without conflicts
> > with the internal functionalities.
> 
> ??? All that will happen from changing a bunch
> of keybindings?

If all the section modes in the manuals could remove the repeated (non
specific) commands and bindings that now they have to specify because
everything is disordered from their sections, the modes don't have to
include the explicit binding from them, The users don't need to learn
different commands in every different mode and the packages developers
had a clear set of reserved bindings to avoid. Yes all this will in some
degree happen.

But as you can see, it is not "a bunch" of keybindings. Ans something I
am pretty sure will never happen.

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 749 bytes --]

On Thu, May 16, 2019 at 01:02:36AM +0200, Emanuel Berg wrote:
> tomas wrote:

[...]

> > Thus, perhaps "E8 Writing Dynamically-Loaded
> > Modules", where your C functions end up in
> > a separate library, loaded by Emacs at run
> > time, is more interesting.
> 
> Aha, so you can do that! Wonderful!
> 
> Only... I don't find it?

Ah, sorry. Dynamically loaded modules were added to Emacs on version
25, I think. What version are you using?

> And what is this "E8" notation?

This is just the section numbering.

> > That said, it is always a trade-off: stuff
> > written in Elisp is far more hackable [...]

> Well, I've written enough Elisp by now anyway.
> Time to move on :)

Enjoy the landscape :-)

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Noam Postavsky]

On Thu, 16 May 2019 at 02:48, <tomas@tuxteam.de> wrote:

> > > Thus, perhaps "E8 Writing Dynamically-Loaded
> > > Modules", where your C functions end up in
> > > a separate library, loaded by Emacs at run
> > > time, is more interesting.
> >
> > Aha, so you can do that! Wonderful!
> >
> > Only... I don't find it?
>
> Ah, sorry. Dynamically loaded modules were added to Emacs on version
> 25, I think.

The docs were added only in Emacs 26, I believe.

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 758 bytes --]

On Thu, May 16, 2019 at 05:37:03AM -0400, Noam Postavsky wrote:
> On Thu, 16 May 2019 at 02:48, <tomas@tuxteam.de> wrote:
> 
> > > > Thus, perhaps "E8 Writing Dynamically-Loaded
> > > > Modules", where your C functions end up in
> > > > a separate library, loaded by Emacs at run
> > > > time, is more interesting.
> > >
> > > Aha, so you can do that! Wonderful!
> > >
> > > Only... I don't find it?
> >
> > Ah, sorry. Dynamically loaded modules were added to Emacs on version
> > 25, I think.
> 
> The docs were added only in Emacs 26, I believe.

Oh, I see. Emanuel: in the meantime, here's the online ref:

  https://www.gnu.org/software/emacs/manual/html_node/elisp/Writing-Dynamic-Modules.html#Writing-Dynamic-Modules

Enjoy
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Wed, 15 May 2019 22:09:32 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > "Outside Savannah" is not relevant when we talk about Emacs.
> 
> If people are implementing the features but can't be integrated in stock
> Emacs, the Savannah boundary becomes relevant.

Not for this thread, which deals specifically with what's available in
Emacs OOB.

> >> Assuming that the authors have no problem assigning their copyrights
> >> to FSF, what prevents the outside work from being integrated into
> >> Emacs in Savannah to improve Emacs's C++ IDE features?
> >
> > Under those assumptions, and if that work doesn't use any non-free
> > software, nothing.
> 
> Unfortunately, this is not true.

Yes, it is.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Emanuel Berg <moasenwood@zoho.eu>
> Date: Thu, 16 May 2019 01:02:36 +0200
> 
> > Thus, perhaps "E8 Writing Dynamically-Loaded
> > Modules", where your C functions end up in
> > a separate library, loaded by Emacs at run
> > time, is more interesting.
> 
> Aha, so you can do that! Wonderful!
> 
> Only... I don't find it?
> 
> ;; DNC
> (info "(emacs) Writing Dynamically-Loaded Modules")
> (info "(elisp) Writing Dynamically-Loaded Modules")
> 
> Where is it?

Get a newer Emacs, it's a recent addition to the manual.

> And what is this "E8" notation?

Appendix E, chapter 8.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Wed, 15 May 2019 22:09:32 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> > "Outside Savannah" is not relevant when we talk about Emacs.
>> 
>> If people are implementing the features but can't be integrated in stock
>> Emacs, the Savannah boundary becomes relevant.
>
> Not for this thread, which deals specifically with what's available in
> Emacs OOB.

I'm sure the authors of those packages would be happy to see them
integrated on Emacs but...

>> >> Assuming that the authors have no problem assigning their copyrights
>> >> to FSF, what prevents the outside work from being integrated into
>> >> Emacs in Savannah to improve Emacs's C++ IDE features?
>> >
>> > Under those assumptions, and if that work doesn't use any non-free
>> > software, nothing.
>> 
>> Unfortunately, this is not true.
>
> Yes, it is.

... they work on top of Clang which, as you very well know, exclude them
from being incorporated in Emacs.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Thu, 16 May 2019 15:40:02 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Óscar Fuentes <ofv@wanadoo.es>
> >> Date: Wed, 15 May 2019 22:09:32 +0200
> >> 
> >> Eli Zaretskii <eliz@gnu.org> writes:
> >> 
> >> >> > "Outside Savannah" is not relevant when we talk about Emacs.
> >> 
> >> If people are implementing the features but can't be integrated in stock
> >> Emacs, the Savannah boundary becomes relevant.
> >
> > Not for this thread, which deals specifically with what's available in
> > Emacs OOB.
> 
> I'm sure the authors of those packages would be happy to see them
> integrated on Emacs but...

It frequently is not enough to be happy about including something,
there are other factors.

> >> >> Assuming that the authors have no problem assigning their copyrights
> >> >> to FSF, what prevents the outside work from being integrated into
> >> >> Emacs in Savannah to improve Emacs's C++ IDE features?
> >> >
> >> > Under those assumptions, and if that work doesn't use any non-free
> >> > software, nothing.
> >> 
> >> Unfortunately, this is not true.
> >
> > Yes, it is.
> 
> ... they work on top of Clang which, as you very well know, exclude them
> from being incorporated in Emacs.

It does?

Re: Is Elisp really that slow?

[Eli Zaretskii]

> Date: Wed, 15 May 2019 23:09:24 +0200
> From: Ergus <spacibba@aol.com>
> Cc: help-gnu-emacs@gnu.org
> 
> We are far from becoming a toxic community, but new/different ideas are
> not really very welcome and sometimes the arguments are like "it has
> always been like that", "old users don't want that change".

Really?  You've just went through a process of proposing and
implementing a new feature -- did you feel your idea was "not really
welcome"?

> The conservative attitude in emacs development group (apart from the
> technical obstacles like the workflow and the paperwork and so on) gives
> the idea of a closed environment (that actually it is from the
> development point of view because we are like frozen in the past) where
> only very experts are welcome (that's the vision people have from
> outside) so very few melpa developers are really interested to face all
> those issues to contribute.

Is this your experience from implementing the fill-column indicator?
If so, how do you explain that you, as a relative non-expert, were
welcome in that case?

Re: Is Elisp really that slow?

[Stefan Monnier]

> So from a personal POV, I would be
> interesting/fun to write some of my Elisp into
> C, and if I get good enough at it, maybe
> I could even do it for the project.

FWIW, compiling existing Elisp to C code is not really interesting in
the sense that if it exists as Elisp code it's usually because it's not
very speed-sensitive.

What is more interesting is to take existing C code and see if it can be
rewritten into a kind of Elisp (or other language that has similar
dynamic properties, most importantly the ability to replace code at
run-time) that can be compiled back to efficient code.

After all, the main problem with the part of Emacs that's written in
C is that you can't change it from your ~/.emacs, contrary to all the
Elisp code.


        Stefan

Re: Is Elisp really that slow?

[Ergus]

On Thu, May 16, 2019 at 05:12:22PM +0300, Eli Zaretskii wrote:
>> Date: Wed, 15 May 2019 23:09:24 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: help-gnu-emacs@gnu.org
>>

Hi Eli. Sorry, maybe this email sounds stronger than the intention.

>> We are far from becoming a toxic community, but new/different ideas are
>> not really very welcome and sometimes the arguments are like "it has
>> always been like that", "old users don't want that change".
>
>Really?  You've just went through a process of proposing and
>implementing a new feature -- did you feel your idea was "not really
>welcome"?
>
Actually fill-column-indicator was something that many people were using
(with the package) but it doesn't affect the global behavior at
all. Actually I think that most of the users won't even know that the
functionality will be there probably until emacs 30 when the
fill-column-indicator package just stop working or so. But also it was
an addition. Adding features to emacs is something easy from the mailing
list point of view. Actually we something is added that solves N
problems that nobody complained before (as fci) then automatically
appears some mails claiming that it must solve P other issues to no body
have complained either.

But when we mention (propose, suggest) to do any change in the defaults
or remove obsolete functionalities to promote new ones, or change old
features to add more modern ones... the mailing list starts becoming
crazy. Just give a look what happen when I just mention the C-c C-c
example. So at the end everything goes in and nothing goes out... that's
unmaintainable YKWIM.

>> The conservative attitude in emacs development group (apart from the
>> technical obstacles like the workflow and the paperwork and so on) gives
>> the idea of a closed environment (that actually it is from the
>> development point of view because we are like frozen in the past) where
>> only very experts are welcome (that's the vision people have from
>> outside) so very few melpa developers are really interested to face all
>> those issues to contribute.
>
>Is this your experience from implementing the fill-column indicator?
>If so, how do you explain that you, as a relative non-expert, were
>welcome in that case?
>
I put explicitly: that's the vision people have from outside.

Actually I was surprised that you were son kind, and patient with all
the emails and questions. But again fci does not affect anyone that does
not enable it, and actually the reddit post just received a couple of
comments. So actually it is a specific functionality that maybe
shouldn't have been added because there was not interest on it.

But any proposals that potentially change any default or break any
backward compatibility or even correct previous mistakes or update some
behavior are actually like a bomb in the mailing list.

Just give a look how long it took to enable delete-selection-mode or
transient-mark-mode since they were implemented and proposed the first
time. But also the new behaviors as they need to be optional, many
functions need to check a lot of conditional to work in all the
situations.

Just give a look how many good packages are in melpa, but not in elpa
in spite of they are totally free with no dependencies and their
maintainers are very active, and the communities too. (elpy, slime,
helm, company, swiper-avy-counsel...).

Re: Is Elisp really that slow?

[Eli Zaretskii]

> Date: Thu, 16 May 2019 18:14:08 +0200
> From: Ergus <spacibba@aol.com>
> Cc: help-gnu-emacs@gnu.org
> 
> >> We are far from becoming a toxic community, but new/different ideas are
> >> not really very welcome and sometimes the arguments are like "it has
> >> always been like that", "old users don't want that change".
> >
> >Really?  You've just went through a process of proposing and
> >implementing a new feature -- did you feel your idea was "not really
> >welcome"?
> >
> Actually fill-column-indicator was something that many people were using
> (with the package) but it doesn't affect the global behavior at
> all.

It's a reimplementation in core of a popular feature.  I think you are
wrong: I'm sure quite a few people will be happy to see it alive and
much faster than the Lisp implementation that is no longer maintained.

> But when we mention (propose, suggest) to do any change in the defaults
> or remove obsolete functionalities to promote new ones, or change old
> features to add more modern ones... the mailing list starts becoming
> crazy.

Emacs is an old program, with users who stay with it for 25 and more
years.  Changing old defaults or making other incompatible changes
definitely annoys at least some of those users, especially since quite
a few people don't track the development, and don't see the changes
until years later, sometimes even after skipping several releases
in-between.  This isn't theory, this is our (and my personal) actual
experience from watching Emacs development.  When someone comes up and
asks why we made a certain change in how Emacs worked for decades, we
should have a good reason, and if we don't, we shouldn't make the
change "just because we can".  That's the price of maintaining and
developing an old and stable program.

Personally, I wish people would invest much more time and efforts in
adding new features, and would stop futzing with existing features.
For starters, the former is much more useful for our users than the
latter.  Also, I see many times how a change in existing code to make
some minor improvement introduces bugs, which sometimes are more
significant than the "fixed" problem -- this is expected in a complex
stable program, and is a clear sign that changes of existing code long
since entered the area which engineers call "the limit cycle" --
oscillations that don't converge, i.e. the overall code quality is
quasi-constant.  IOW, we are wasting our own resources for little or
no gain.

> Just give a look how many good packages are in melpa, but not in elpa

If you are saying that those packages are in MELPA because they were
rejected to be included in Emacs or ELPA, then I'd be very surprised
to learn that this is true.  I think the vast majority of those
packages started up with MELPA and their developers never tried to
submit the packages to be included in Emacs.

Re: Is Elisp really that slow?

[Ergus]

Hi Eli:

I agree a little bit too much here. But... 

>
>Emacs is an old program, with users who stay with it for 25 and more
>years.  Changing old defaults or making other incompatible changes
>definitely annoys at least some of those users, especially since quite
>a few people don't track the development, and don't see the changes
>until years later, sometimes even after skipping several releases
>in-between.  This isn't theory, this is our (and my personal) actual
>experience from watching Emacs development.  When someone comes up and
>asks why we made a certain change in how Emacs worked for decades, we
>should have a good reason, and if we don't, we shouldn't make the
>change "just because we can".  That's the price of maintaining and
>developing an old and stable program.
>
My point of view since the beginning of my first email ever in the
mailing list was based in 3 things:

1) Some design choices and limitations in the actual emacs were imposed
due to technology 40 years ago. (keyboard bindings, coding system,
screen resolutions, memory available and CPU, network latency, storage
capacity) but most of them are not issues anymore (at least not in the
emacs scales) so we are limiting the potential of emacs due to issues
that disappeared long time ago.

2) Many new features and functionalities (pure editing oriented) have
been introduced/proved/improved in other editors, and they have proven
to be useful (move line, initial configuration, associative bindings,
specific key combinations like in cua-mode, undo/redo and many
others).

An important part of those functionalities are can be implemented with
Elisp in one afternoon probably because they are just details and some
are already available as external packages, but the changes never arrive
because there are not available (practical) keybindings or because of
backward compatibility.

3) The development is not focused in the first thing that a user needs
when she opens emacs: provide the most comfortable and useful TEXT
EDITOR.

If emacs as TEXT EDITOR does not convince them (just the first try,
without config, without reading the manual/tutorial/documentation), then
they will not even try any other functionality. I think that you are one
of the few in this list who sees the importance if attracting new
users/developers. Unlike vim; emacs is not in the gnu/linux distros, it
is slower and bigger... so we need to offer some advantage on the first
try over the others to keep the users.

For example, many people use nano just because the basic actions are in
a button bar, so they don't get stocked and they can do the basic stuff
quick and fast, no installation no config needed.

Look at the spacemacs community and how many members are there wrapping
emacs functionalities to behave like in vim...

>Personally, I wish people would invest much more time and efforts in
>adding new features, and would stop futzing with existing features.
>For starters, the former is much more useful for our users than the
>latter.  Also, I see many times how a change in existing code to make
>some minor improvement introduces bugs, which sometimes are more
>significant than the "fixed" problem -- this is expected in a complex
>stable program, and is a clear sign that changes of existing code long
>since entered the area which engineers call "the limit cycle" --
>oscillations that don't converge, i.e. the overall code quality is
>quasi-constant.  IOW, we are wasting our own resources for little or
>no gain.
>
I would support that the functionalities go in elpa (I say elpa because
it won't need an initial configuration to be used, just list-packages
and install). Because there will be big chances that some other better
alternatives substitute it (ido -> helm -> avy; isearch -> swiper, etags
-> gtags, vc -> magit, ace -> avy) and looking tto the past... nothing
will be removed never ever from the core emacs code.

Also improve the modules API and support to create packages with C as
with elisp. But add the less possible functionalities in the core if
they are not directly editor related (basic ones) or general enough
(infrastructure / API).
>
>> Just give a look how many good packages are in melpa, but not in elpa
>
>If you are saying that those packages are in MELPA because they were
>rejected to be included in Emacs or ELPA, then I'd be very surprised
>to learn that this is true.  I think the vast majority of those
>packages started up with MELPA and their developers never tried to
>submit the packages to be included in Emacs.
>
No, I am saying that there is a reason why they made their work, and
they maintain it, but they don't spend time to put it in the official
repository.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Can't resist on commenting two points:

Ergus <spacibba@aol.com> writes:

> 3) The development is not focused in the first thing that a user needs
> when she opens emacs: provide the most comfortable and useful TEXT
> EDITOR.

For many people this is Notepad.

> If emacs as TEXT EDITOR does not convince them (just the first try,
> without config, without reading the manual/tutorial/documentation), then
> they will not even try any other functionality.

Indeed, Emacs has a non-negligible learning curve. Other editors do a
lot out of the box in a familiar way, but then you hit the wall. Emacs
is lacking in out-of-the-box functionality for modern programming
languages but otherwise, as a pure editor, it has no walls.

> I think that you are one
> of the few in this list who sees the importance if attracting new
> users/developers. Unlike vim; emacs is not in the gnu/linux distros, it
> is slower and bigger...

Are you sure? emacs -Q is instant here, vim grew quite a bit and,
anyway, who cares about a few dozen MB of difference when some of the
modern contenders use GB of *RAM* to work?

> so we need to offer some advantage on the first
> try over the others to keep the users.

Emacs provides some advantages, but they are not apparent until you
experience them. That's a problem for people grown on a culture of
instant gratification. Emacs appeals to certain type of users who
understand that gains require efforts. The really big problem is that
Emacs no longer compete on areas were it used to bring the largest
gains. Other editors largely surpassed Emacs' gains while requiring less
effort.

Re: Is Elisp really that slow?

[Ergus]

On Thu, May 16, 2019 at 10:50:45PM +0200, �scar Fuentes wrote:
>Can't resist on commenting two points:
>
>Ergus <spacibba@aol.com> writes:
>
>> 3) The development is not focused in the first thing that a user needs
>> when she opens emacs: provide the most comfortable and useful TEXT
>> EDITOR.
>
>For many people this is Notepad.
>
Or notepad++, or sublime text. 
>
>> If emacs as TEXT EDITOR does not convince them (just the first try,
>> without config, without reading the manual/tutorial/documentation), then
>> they will not even try any other functionality.
>
>Indeed, Emacs has a non-negligible learning curve. Other editors do a
>lot out of the box in a familiar way, but then you hit the wall. Emacs
>is lacking in out-of-the-box functionality for modern programming
>languages but otherwise, as a pure editor, it has no walls.
>
I agree 50% here. One thing is to use specific emacs functionalities
tricks, but another one is when you don't know how to start, because the
workflow is orthogonal to 90% of everything else, or you look for the
word copy and paste in the tutorial you only find kill and yank.

The first impression is actually the most important. And if a user don't
see any advantage the first 10 minutes, we lost him.
>
>> I think that you are one
>> of the few in this list who sees the importance if attracting new
>> users/developers. Unlike vim; emacs is not in the gnu/linux distros, it
>> is slower and bigger...
>
>Are you sure? emacs -Q is instant here, vim grew quite a bit and,

But with the -Q option you will have precisely all the defaults that
most of the people end changing because we don't do for them, without
extra plugins or extensions... so... when fast no advantages...

>anyway, who cares about a few dozen MB of difference when some of the
>modern contenders use GB of *RAM* to work?
>

There is actually a thread complaining about the emacs size in windows 

>
>> so we need to offer some advantage on the first
>> try over the others to keep the users.
>
>Emacs provides some advantages, but they are not apparent until you
>experience them.

That has nothing to  do with the fact that the apparent ones could be
better and they are not because of inertia.

>That's a problem for people grown on a culture of
>instant gratification. Emacs appeals to certain type of users who
>understand that gains require efforts.

Here agree, but the spectrum must be opened. We are not in 1990 anymore. 

>The really big problem is that
>Emacs no longer compete on areas were it used to bring the largest
>gains. Other editors largely surpassed Emacs' gains while requiring less
>effort.
>

This is exactly the key of all my point. We can't continue patching and
putting works around to this issue and we can't compete and win in all
the fields we cover, so at least we must be centered in one of them, the
most important one. Compete in the main goal, the main objective... And
as simple and logical for the users as possible. It looks illogical for
me that fill-columns-indicator or line-numbers came into emacs in
version 26.1 and 27.0 being so basic functionalities for any editor. Or
that we don't have at least a native-simple undo-redo (even if it needs
an extra line to be enables in the configuration file). Or that there
were so many discussions about enabling transient-mark-mode, or
delete-selection-mode.

Re: Is Elisp really that slow?

[Emanuel Berg]

Eli Zaretskii wrote:

> Debugging works well enough, but code search
> is very simplistic (brings a lot of false
> positives, because it's basically dumb text
> search, not sensitive enough to the
> language), and refactoring doesn't work at
> all, because we have no facilities for that.
> For small projects, you could use 'A' or 'Q'
> in Dired, but that quickly becomes unworkable
> in large projects.

OK, thanks!

By refactoring, here, I take it you mean
a program that is parsing the code, analyzing
it, and breaking it up, to have neat building
blocks instead of huge functions, but with the
same functionality, and all this without or
with little human intervention?

Well, that sounds... advanced! So this is
something modern IDEs do? :O

To have that for all our programming modes
sounds like a enormous challenge!

The only way I can see it work in practice is
if the refactoring logic could be separated and
be the same for all languages, so one would
just have to have parsing language-specific and
then hook it to the refactoring engine...

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Óscar Fuentes]

Ergus <spacibba@aol.com> writes:

>>anyway, who cares about a few dozen MB of difference when some of the
>>modern contenders use GB of *RAM* to work?
>>
>
> There is actually a thread complaining about the emacs size in windows 

I was one of the complainers. The problem was that the zip file
contained debug info and a duplicated large executable, it had nothing
to do with the topic of this thread.

Re: Is Elisp really that slow?

[Emanuel Berg]

Eli Zaretskii wrote:

> You can only call for implementing some
> important functionality, but have no real
> instruments to make that happen, except do it
> yourself (which might not be possible due to
> lack of knowledge/talent/time).

Heck, this kind of comment... *shakes my head*

It is very possible! Just do it! Start today!

_Knowledge_ is accumulated.

There is plenty enough _talent_.

And if you care about _time_, take your
paleo-computer, carefully pull out all the
cables, then throw it out your window! _Or_
disintegrate it with a Multicogan! This will
solve any problems you have experienced with
time, in an instant...

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Jean-Christophe Helary]

> On May 17, 2019, at 5:50, Óscar Fuentes <ofv@wanadoo.es> wrote:
> 
>> so we need to offer some advantage on the first
>> try over the others to keep the users.
> 
> Emacs provides some advantages, but they are not apparent until you
> experience them. That's a problem for people grown on a culture of
> instant gratification. Emacs appeals to certain type of users who
> understand that gains require efforts.

I find that comment extremely condescending.

If "instant gratification" means finding a common ground on which one can get started right away, then I'm all for it.

Considering the state of affairs, emacs seems first to appeal to people who want to give priority to free software, at the *cost* of ease of use.

Access to free software should never be the sole privilege of "users who understand that gains require efforts". Quite the opposite.

> The really big problem is that Emacs no longer compete on areas were it used to bring the largest gains. Other editors largely surpassed Emacs' gains while requiring less effort.

Eli earlier clearly identified a number of areas where emacs required huge and totally undue efforts to get the thing to work as expected in the 21st century. 


Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: Is Elisp really that slow?

[Óscar Fuentes]

Jean-Christophe Helary <brandelune@gmail.com> writes:

>> On May 17, 2019, at 5:50, Óscar Fuentes <ofv@wanadoo.es> wrote:
>> 
>>> so we need to offer some advantage on the first
>>> try over the others to keep the users.
>> 
>> Emacs provides some advantages, but they are not apparent until you
>> experience them. That's a problem for people grown on a culture of
>> instant gratification. Emacs appeals to certain type of users who
>> understand that gains require efforts.
>
> I find that comment extremely condescending.
>
> If "instant gratification" means finding a common ground on which one
> can get started right away, then I'm all for it.

"Instant gratification" means wanting things that require no learning
nor practicing nor understanding to be effectively used *right* *now*.

It would be doubleplusgood if Emacs could be one of those things but,
alas, it is obvious that text editors still are on the class of things
that require certain effort to be used effectively. Maybe Emacs requires
a bit more effort at the beginning, but it pays off... at least on text
manipulation tasks.

> Considering the state of affairs, emacs seems first to appeal to
> people who want to give priority to free software, at the *cost* of
> ease of use.

From 1985 to 2010 (give or take a few years and discounting Java and
some other modern language) Emacs was the best programmer's editor on
the "by hackers, for hackers" category. I suppose that most current
users come from that period.

> Access to free software should never be the sole privilege of "users
> who understand that gains require efforts". Quite the opposite.

Free Software is not a factor *today*, because most competitors are Free
Software too. Even Visual Studio Code is MIT-licensed.

OTOH, we have the vim phenomenon. An "old thing" which is way more
peculiar than Emacs, but with a growing user base. Those who point out
the dificulties of new users to copy and paste or to save text to a file
with Emacs, should ponder how vim has no problem requiring training for
doing the most basic thing a text editor is supposed to do.

Maybe, just maybe, having "kill & yank" instead "copy & paste" is not
the cause of Emacs' lack of appeal to the new generations.

Re: Is Elisp really that slow?

[Jean-Christophe Helary]

> On May 17, 2019, at 11:26, Óscar Fuentes <ofv@wanadoo.es> wrote:
> 
> Jean-Christophe Helary <brandelune@gmail.com> writes:
> 
>> If "instant gratification" means finding a common ground on which one
>> can get started right away, then I'm all for it.
> 
> "Instant gratification" means wanting things that require no learning
> nor practicing nor understanding to be effectively used *right* *now*.

There is another word for that (and the earlier wheel metaphor may have escaped you), it is call "standardization".

> It would be doubleplusgood if Emacs could be one of those things but,
> alas, it is obvious that text editors still are on the class of things
> that require certain effort to be used effectively.

Please. Text editing is the most common task by at least an order of magnitude in the IT world. Even for kids.

> Maybe Emacs requires a bit more effort at the beginning, but it pays off... at least on text manipulation tasks.

Except that no. Check Eli's list of areas where emacs has fallen way behind. There are all related to advanced text manipulation.

> From 1985 to 2010 (give or take a few years and discounting Java and
> some other modern language) Emacs was the best programmer's editor on
> the "by hackers, for hackers" category. I suppose that most current
> users come from that period.

Because they don't fit your narrative ? I've started trying emacs in the mid 90's and the best pro editor I could find then that I could make sense of was BBEdit. I'm still using it when I have no time to uncover emacs' arbitrary idiosyncrasies. In fact, this discussion makes me realize that the only reason I use emacs is because it is a lisp environment and so I don't have to wait for developers to develop something for me and I can write a few lines of lisp myself if needed. Just like I write a few lines of AppleScript to solve trivial issues on my Mac instead of buying a $5 hack (but I could write AppleScript to automate BBEdit, except that it is less elegant than lisp).

>> Access to free software should never be the sole privilege of "users
>> who understand that gains require efforts". Quite the opposite.
> 
> Free Software is not a factor *today*, because most competitors are Free
> Software too. Even Visual Studio Code is MIT-licensed.

Ummm. So you agree that emacs sucks until you reach a given yet undefined enlightenment point and that free software is not a factor, so why not plainly declare that emacs is for a self proclaimed elite and then move on?

> OTOH, we have the vim phenomenon. An "old thing" which is way more
> peculiar than Emacs,

No. Unless "peculiar" means "consistent".

> but with a growing user base. Those who point out
> the dificulties of new users to copy and paste or to save text to a file
> with Emacs, should ponder how vim has no problem requiring training for
> doing the most basic thing a text editor is supposed to do.

"consistency"

> Maybe, just maybe, having "kill & yank" instead "copy & paste" is not
> the cause of Emacs' lack of appeal to the new generations.

If that's your conclusion, you must have missed a few mails in the thread. Eli clarified that a long time ago.

Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: Is Elisp really that slow?

[Ergus]

On Fri, May 17, 2019 at 04:26:26AM +0200, �scar Fuentes wrote:
>Jean-Christophe Helary <brandelune@gmail.com> writes:
>
>>> On May 17, 2019, at 5:50, �scar Fuentes <ofv@wanadoo.es> wrote:
>>>
>>>> so we need to offer some advantage on the first
>>>> try over the others to keep the users.
>>>
>>> Emacs provides some advantages, but they are not apparent until you
>>> experience them. That's a problem for people grown on a culture of
>>> instant gratification. Emacs appeals to certain type of users who
>>> understand that gains require efforts.
>>
>> I find that comment extremely condescending.
>>
>> If "instant gratification" means finding a common ground on which one
>> can get started right away, then I'm all for it.
>
>"Instant gratification" means wanting things that require no learning
>nor practicing nor understanding to be effectively used *right* *now*.
>

The vim phenomenon is the prove that our problem is not "Instant
gratification". Vim made many changes in the interface to make all the
commands more coherent or shorter. And removed the errors existing in vi
even when some people used them as features. There is an interesting
article about that.

Of course there were complains. There are always complains.

>It would be doubleplusgood if Emacs could be one of those things but,
>alas, it is obvious that text editors still are on the class of things
>that require certain effort to be used effectively. Maybe Emacs requires
>a bit more effort at the beginning, but it pays off... at least on text
>manipulation tasks.
>
The problem comes when people needs to configure everything because the
defaults are terrible, they rely on some packages and then they can't use
another machine (because emacs is not there, or does not have their
configuration). So the effort invested and the training, confines them
and worth nothing outside their machine.
>
>> Considering the state of affairs, emacs seems first to appeal to
>> people who want to give priority to free software, at the *cost* of
>> ease of use.
>
>From 1985 to 2010 (give or take a few years and discounting Java and
>some other modern language) Emacs was the best programmer's editor on
>the "by hackers, for hackers" category. I suppose that most current
>users come from that period.
>
Agree. But future is coming.
>
>> Access to free software should never be the sole privilege of "users
>> who understand that gains require efforts". Quite the opposite.
>
>Free Software is not a factor *today*, because most competitors are Free
>Software too. Even Visual Studio Code is MIT-licensed.
>
I will no go in the free vs open source here. But no, most of the
editors are open source, not free. 
>
>OTOH, we have the vim phenomenon. An "old thing" which is way more
>peculiar than Emacs, but with a growing user base. Those who point out
>the dificulties of new users to copy and paste or to save text to a file
>with Emacs, should ponder how vim has no problem requiring training for
>doing the most basic thing a text editor is supposed to do.
>
Vim is actually my starting point. It is the only editor that worth to
compare with emacs.

It is easy to explain and I have refereed to this in many maaaaany other
emails:

1) vim is there in all the GNU/Linux distros.

2) It works the same in all the systems, in all the languages, even the
default color themes are better by default.

3) The keybinds (apart from the insert-escape) are easier, more
ergonomic and logically composable.

4) It is extremely responsive and fast to open-close workflows. No
lagging, or delays, no server configuration needed.

5) The important editing commands are usually only 1 key far. We can
make a simple comparison:

Move forward: C-n -> j
Move forward 3 lines: M-3 C-n -> 3j (look sparsity in the keyboard)
Copy 3 lines and return to point position:
C-SPC C-SPC C-a C-SPC M-3 M-w C-u C-SPC C-u C-SPC -> 3yy

Plus:
hjkl are way more ergonomic than what we have (you can type them
with one hand, while the other types the prefix, but they are also
together, so going 3 lines up 5 letters left is way faster and easier) 

Plus:
In vim the user can enter complete lines/commands/functions:

:e file
:8,10 s/search/replace/g

which is more intuitive and familiar for terminal users. And those
commands are also composable.

Plus:
There are no conflicts with the modified inputs and the terminals, so
they have more keybindings to use.

6) They do one thing and do it well. Editing functionalities have
priority (for example column indicator or line numbers were added very
long time ago.) 

7) I understand it is also much simpler than emacs in functionalities,
but that is a benefit from the maintenance and update point of
view. They don't need to maintain an interpreter, their own language, a
graphical and a terminal interface, different modes for every
programming language, wrapper functions for terminal commands like grep
(or version control functionalities) a browser, file manager, a server
interface and client, a network infrastructure... This also means that
the number of programmers and expert fields they need to maintain all
the code is also smaller.

Let me say:

I am NOT making apology of vim, I am just pointing how are they doing
and why they success more with a "worst product" because it is not a
mystery.

I should also highlight that emacs can do (almost) all this: (spacemacs,
evil-mode) or in its own way: (ergoemacs, god-mode, hydra) but we still
enforce the old way. And we don't promote enough some packages and
functionalities that are not imported, but created exclusively for emacs
and could make the difference (like avy).

>
>Maybe, just maybe, having "kill & yank" instead "copy & paste" is not
>the cause of Emacs' lack of appeal to the new generations.
>

It is one more.

Re: Is Elisp really that slow?

[Ergus]

On Fri, May 17, 2019 at 01:05:12PM +0900, Jean-Christophe Helary wrote:
>
>
>> On May 17, 2019, at 11:26, �scar Fuentes <ofv@wanadoo.es> wrote:
>>
>> Jean-Christophe Helary <brandelune@gmail.com> writes:
>>
>>> If "instant gratification" means finding a common ground on which one
>>> can get started right away, then I'm all for it.
>>
>> "Instant gratification" means wanting things that require no learning
>> nor practicing nor understanding to be effectively used *right* *now*.
>
>There is another word for that (and the earlier wheel metaphor may have escaped you), it is call "standardization".
>
Agree. 
>
>> It would be doubleplusgood if Emacs could be one of those things but,
>> alas, it is obvious that text editors still are on the class of things
>> that require certain effort to be used effectively.
>
>Please. Text editing is the most common task by at least an order of magnitude in the IT world. Even for kids.
>
Also agree 100%.
>
>> Maybe Emacs requires a bit more effort at the beginning, but it pays off... at least on text manipulation tasks.
>
>Except that no. Check Eli's list of areas where emacs has fallen way behind. There are all related to advanced text manipulation.
>
And standardization was mentioned by Stefan as important too.
>
>> From 1985 to 2010 (give or take a few years and discounting Java and
>> some other modern language) Emacs was the best programmer's editor on
>> the "by hackers, for hackers" category. I suppose that most current
>> users come from that period.
>
>Because they don't fit your narrative ? I've started trying emacs in the mid 90's and the best pro editor I could find then that I could make sense of was BBEdit. I'm still using it when I have no time to uncover emacs' arbitrary idiosyncrasies. In fact, this discussion makes me realize that the only reason I use emacs is because it is a lisp environment and so I don't have to wait for developers to develop something for me and I can write a few lines of lisp myself if needed. Just like I write a few lines of AppleScript to solve trivial issues on my Mac instead of buying a $5 hack (but I could write AppleScript to automate BBEdit, except that it is less elegant than lisp).
>
That is sadly a very common only reason people use Emacs.

But for new users, lisp looks like ancient Cyrillic. So I don't expect
that in the future it will be a reason many users chose emacs. And there
is also the fact that other editors offer somehow the same flexibility
with Java Script or Python, which are more familiar and popular these
days.
>
>>> Access to free software should never be the sole privilege of "users
>>> who understand that gains require efforts". Quite the opposite.
>>
>> Free Software is not a factor *today*, because most competitors are Free
>> Software too. Even Visual Studio Code is MIT-licensed.
>
>Ummm. So you agree that emacs sucks until you reach a given yet undefined enlightenment point and that free software is not a factor, so why not plainly declare that emacs is for a self proclaimed elite and then move on?
>
Better no comment this, but I agree that this is the feeling.
>
>> OTOH, we have the vim phenomenon. An "old thing" which is way more
>> peculiar than Emacs,
>
>No. Unless "peculiar" means "consistent".
>
It is actually a prove that being consistent can success, even when not
following the standards in the rest of the world. Also that programmers
prefer to use their logic (compose binding commands) instead of their
memory (memorize bindings) to interact with the editor.
>
>> but with a growing user base. Those who point out
>> the dificulties of new users to copy and paste or to save text to a file
>> with Emacs, should ponder how vim has no problem requiring training for
>> doing the most basic thing a text editor is supposed to do.
>
>"consistency"
>
Exactly
>
>> Maybe, just maybe, having "kill & yank" instead "copy & paste" is not
>> the cause of Emacs' lack of appeal to the new generations.
>
>If that's your conclusion, you must have missed a few mails in the thread. Eli clarified that a long time ago.
>
>Jean-Christophe Helary
>-----------------------------------------------
>http://mac4translators.blogspot.com @brandelune
>
>
>

Re: Is Elisp really that slow?

[Eli Zaretskii]

> Date: Thu, 16 May 2019 22:23:27 +0200
> From: Ergus <spacibba@aol.com>
> Cc: help-gnu-emacs@gnu.org
> 
> My point of view since the beginning of my first email ever in the
> mailing list was based in 3 things:
> 
> 1) Some design choices and limitations in the actual emacs were imposed
> due to technology 40 years ago. (keyboard bindings, coding system,
> screen resolutions, memory available and CPU, network latency, storage
> capacity) but most of them are not issues anymore (at least not in the
> emacs scales) so we are limiting the potential of emacs due to issues
> that disappeared long time ago.
> 
> 2) Many new features and functionalities (pure editing oriented) have
> been introduced/proved/improved in other editors, and they have proven
> to be useful (move line, initial configuration, associative bindings,
> specific key combinations like in cua-mode, undo/redo and many
> others).
> 
> An important part of those functionalities are can be implemented with
> Elisp in one afternoon probably because they are just details and some
> are already available as external packages, but the changes never arrive
> because there are not available (practical) keybindings or because of
> backward compatibility.

It is very hard to discuss this stuff in a useful manner when your
arguments are so abstract.  In general, the (alleged) reason for the
current defaults to be rooted in some obsolete technology, if that
indeed is the case, doesn't necessarily mean those defaults are
invalid today.  The discussion of the defaults should be on a case by
case basis.  I can assure you that if the _only_ reason for having a
default is that alternatives don't work well on obsolete systems,
there's no chance in the world someone will be reasonably against
changing them.  IME, usually there are other significant reasons, such
as incompatibility with Emacs conventions, disruption of keyboard-only
workflows, etc.

OTOH, many, if not all of the features you say were proven in other
editors we already provide in Emacs.  If you are saying it took too
long for Emacs to adopt them, then this is an argument about the past,
which is again not useful IME.

So if you want to have a useful discussion about these matters, I
suggest to come up with specific contemporary examples of features
that illustrate your POV.  Otherwise, these just empty slogans for me,
sorry.

> 3) The development is not focused in the first thing that a user needs
> when she opens emacs: provide the most comfortable and useful TEXT
> EDITOR.

Since you didn't say what TEXT EDITOR should entail, it is again very
hard to discuss this claim.  From my POV, we do provide a text editor
OOB: you have the usual arrow keys, PgUp/PgDn, copy/paste by mouse, a
menu bar and a tool bar with items most users will expect, scroll
bars, simple edit commands like delete-char bound to keys users expect
them to be, F1 for Help, RET which indents when appropriate, etc.

If you are saying that these basic features are not enough, then a
useful start would be to come up with a list of additional features
that new users are looking for (but please don't try to make your
point by deliberately making a list of features we don't have; this
list should be the result of either polling users or some kind of
survey of popular text editors).  We could then discuss their
introduction into Emacs in an intelligent way.

> If emacs as TEXT EDITOR does not convince them (just the first try,
> without config, without reading the manual/tutorial/documentation), then
> they will not even try any other functionality. I think that you are one
> of the few in this list who sees the importance if attracting new
> users/developers.

We will not attract new users by providing just a text editor.  There
are way too many of them out there, and it's very hard, to say the
least, to be significantly better just in that class.  We must find
our own niches, and be one of the top players in those niches.  One of
those niches should IMO be multi-language IDEs, but there are others.

> For example, many people use nano just because the basic actions are in
> a button bar, so they don't get stocked and they can do the basic stuff
> quick and fast, no installation no config needed.

Emacs has the same.

> Look at the spacemacs community and how many members are there wrapping
> emacs functionalities to behave like in vim...

Doesn't prove anything to me, sorry.  People who like spacemacs should
be free to use spacemacs, but it tells nothing about those who don't,
of whom there's quite a large number, AFAIU.

> >Personally, I wish people would invest much more time and efforts in
> >adding new features, and would stop futzing with existing features.
> >For starters, the former is much more useful for our users than the
> >latter.  Also, I see many times how a change in existing code to make
> >some minor improvement introduces bugs, which sometimes are more
> >significant than the "fixed" problem -- this is expected in a complex
> >stable program, and is a clear sign that changes of existing code long
> >since entered the area which engineers call "the limit cycle" --
> >oscillations that don't converge, i.e. the overall code quality is
> >quasi-constant.  IOW, we are wasting our own resources for little or
> >no gain.
> >
> I would support that the functionalities go in elpa (I say elpa because
> it won't need an initial configuration to be used, just list-packages
> and install).

Before we decide where the functionality will go, we should have it in
the first place.  It currently doesn't exist anywhere.  That's the
main problem; the location of the packages is secondary.

> Also improve the modules API and support to create packages with C as
> with elisp.

The main IDE features cannot be implemented in modules.  They can use
some external support functionality via the module interface, but the
features themselves must be in Lisp.  We don't even have a coherent
framework for such features at this time.  CEDET was supposed to be
it, but judging by the fact that no one is developing it, and, on the
contrary, what little development we have in the IDE area is bypassing
CEDET, I'm beginning to think that maybe that decision was a de-facto
mistake.

> >> Just give a look how many good packages are in melpa, but not in elpa
> >
> >If you are saying that those packages are in MELPA because they were
> >rejected to be included in Emacs or ELPA, then I'd be very surprised
> >to learn that this is true.  I think the vast majority of those
> >packages started up with MELPA and their developers never tried to
> >submit the packages to be included in Emacs.
> >
> No, I am saying that there is a reason why they made their work, and
> they maintain it, but they don't spend time to put it in the official
> repository.

People are welcome to scan MELPA packages and suggest to their
developers to submit them to Emacs.  If indeed the reason is what you
say (and I sincerely doubt that), we will have many of them in
Emacs/ELPA in no time.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Emanuel Berg <moasenwood@zoho.eu>
> Date: Fri, 17 May 2019 01:19:42 +0200
> 
> By refactoring, here, I take it you mean
> a program that is parsing the code, analyzing
> it, and breaking it up, to have neat building
> blocks instead of huge functions, but with the
> same functionality, and all this without or
> with little human intervention?

Refactoring is a large topic.  I suggest to do some Internet search
for something like "tools for refactoring programs".  One of the hits
will be this 10-year old paper, which shows some of what I meant in
practice:

  https://pdfs.semanticscholar.org/ce71/00df509f37e4d1c1fc9fa50f2b98ba035ae8.pdf

> Well, that sounds... advanced! So this is
> something modern IDEs do? :O

Some of them, yes.  A good IDE should support at least some of that.

> To have that for all our programming modes
> sounds like a enormous challenge!

The simplest parts can be based on only rudimentary analysis of the
language, something that we already have in every supported language.

> The only way I can see it work in practice is
> if the refactoring logic could be separated and
> be the same for all languages, so one would
> just have to have parsing language-specific and
> then hook it to the refactoring engine...

Yes.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 16.05.2019 23:50, Óscar Fuentes wrote:
> Emacs provides some advantages, but they are not apparent until you
> experience them. That's a problem for people grown on a culture of
> instant gratification. Emacs appeals to certain type of users who
> understand that gains require efforts.

I strongly disagree: as somebody who like instant gratification, I see 
that Emacs provides significant advantages to us: fast ability to see 
how it works and why, to change its behavior in an instance, to 
experiment without recompiling/restarting/whatever.

Whose are the things that VS Code and friends do not provide. Vim 
neither, not to the same extent.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 5:26, Óscar Fuentes wrote:
> OTOH, we have the vim phenomenon. An "old thing" which is way more
> peculiar than Emacs, but with a growing user base.

Without reading the rest of the replies, I also wanted to say this right 
away:

Vim is consistent. What most people seem to like is a small set of rules 
that guide its behavior and the predictable set of features that can be 
gained from it (aside from plugins, of course).

Well, that and Vim Golf.

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 2038 bytes --]

On Thu, May 16, 2019 at 10:50:45PM +0200, Óscar Fuentes wrote:
> Can't resist on commenting two points:
> 
> Ergus <spacibba@aol.com> writes:
> 
> > 3) The development is not focused in the first thing that a user needs
> > when she opens emacs: provide the most comfortable and useful TEXT
> > EDITOR.
> 
> For many people this is Notepad.

+1 :-)

> > If emacs as TEXT EDITOR does not convince them (just the first try,
> > without config, without reading the manual/tutorial/documentation), then
> > they will not even try any other functionality.
> 
> Indeed, Emacs has a non-negligible learning curve. Other editors do a
> lot out of the box in a familiar way [...]

Yes.

I've been in this software business for 30+ years, and with the arrival
of "commodity computing", I've observed a strong anti-pattern: designs
tend to cater to the buyer and not to the user.

I'll explain: If you decide which product to "buy", you haven't the
time/resources to become proficient with that product: you decide on
first impressions. But once you use that product for years, you'll
"need" other features, which perhaps don't stick out at first sight.

There are two mechanisms pushing that anti-pattern forward:

1. In bigger corporations those taking the decision which software
  to "buy" (usually in the monetary sense) won't be those
  paid (and thus more or less blackmailed) to use it. Management
  goes "oh, shiny" and workers go "oh, no!".

2. In general, when you personally "buy" a piece of software
  (in the monetary, or in the general sense), you often have
  no idea on what you need, so you go "oh shiny" again, and
  sink considerable effort into fine-tuning your interface to
  that software. Re-tuning is so expensive that you better not
  think of it (as a long-time vi user, I mostly moved to Emacs
  about ten years ago: it /was/ expensive.

Cheers

[1] "buy" in a very general sense: it might cost money or not,
   but it'll cost learning, dedication and commitment.
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Eli Zaretskii]

> Date: Fri, 17 May 2019 07:52:02 +0200
> From: Ergus <spacibba@aol.com>
> Cc: help-gnu-emacs@gnu.org
> 
> The problem comes when people needs to configure everything because the
> defaults are terrible

Please don't exaggerate like that, it makes the discussion more
emotional and less useful.  Our defaults are not "terrible" by any
account, certainly not when expressed in such a general form.

> Vim [...] is the only editor that worth to compare with emacs.

I think you are wrong here.  There are VSCode, Atom, and a few others.

> It is easy to explain and I have refereed to this in many maaaaany other
> emails:
> 
> 1) vim is there in all the GNU/Linux distros.

Not much we can do about that.  Feel free to lobby distros to include
Emacs.  IMO, if something is related to 40-year old decision, it is
this one.

> 2) It works the same in all the systems, in all the languages, even the
> default color themes are better by default.

Emacs also works the same on all systems.  As for "better default
colors", that debatable at best.  It's a matter of taste, and
psychologically we tend to favor our first experience: old habits die
hard.

> 3) The keybinds (apart from the insert-escape) are easier, more
> ergonomic and logically composable.
> 
> 4) It is extremely responsive and fast to open-close workflows. No
> lagging, or delays, no server configuration needed.
> 
> 5) The important editing commands are usually only 1 key far. We can
> make a simple comparison:
> 
> Move forward: C-n -> j
> Move forward 3 lines: M-3 C-n -> 3j (look sparsity in the keyboard)
> Copy 3 lines and return to point position:
> C-SPC C-SPC C-a C-SPC M-3 M-w C-u C-SPC C-u C-SPC -> 3yy
> 
> Plus:
> hjkl are way more ergonomic than what we have (you can type them
> with one hand, while the other types the prefix, but they are also
> together, so going 3 lines up 5 letters left is way faster and easier) 

If you want to argue that Emacs should be a dual-mode editor like the
vi family, and that this is your "future", then we will never agree.
One of the revolutions that Emacs brought to text editing was its lack
of modality, where you just type text to have it inserted.  Dual-mode
editors are a thing of the past in my eyes, a remnant from the old
days when editors didn't have the real-time display, where you needed
to have commands like "move N lines from the current one, then delete
M lines" etc.  Please don't even get me (and others) started on this
"feature".

> Plus:
> In vim the user can enter complete lines/commands/functions:
> 
> :e file
> :8,10 s/search/replace/g

How is this different from Emacs's "C-x C-f" or M-% in the selected
region?  Are you saying that it's easier to remember ":e" than "C-x
C-f"??

And you contradict yourself here: simple text editors all provide
operations on the selected portion of text.  Emacs does that, while
vim tells the users to remember the cryptic "N,M s/foo/bar/g" stuff
that goes back to the 50-year old ed(1) and sed(1) editors!

> which is more intuitive and familiar for terminal users.

"terminal users"? who are those?  what's a "terminal"?  Are we really
going to target 70-year old curmudgeons that still work on a
'terminal"?  why? because vim does that?

Forgive me, but this is all backwards!  Modern users want first and
foremost GUI editors, because you can do in GUI display stuff that is
unimaginable for text-mode environments.

> Plus:
> There are no conflicts with the modified inputs and the terminals, so
> they have more keybindings to use.

Yesh, and you pay by having two modes.  No, thanks.

> 6) They do one thing and do it well. Editing functionalities have
> priority (for example column indicator or line numbers were added very
> long time ago.) 

Line numbers are a _must_ in vim, because so many commands _require_
you to name the line numbers.  See your example above.  That Emacs
originally didn't have line numbers is because you don't actually
_need_ them so much.

> 7) I understand it is also much simpler than emacs in functionalities,
> but that is a benefit from the maintenance and update point of
> view. They don't need to maintain an interpreter, their own language, a
> graphical and a terminal interface, different modes for every
> programming language, wrapper functions for terminal commands like grep
> (or version control functionalities) a browser, file manager, a server
> interface and client, a network infrastructure... This also means that
> the number of programmers and expert fields they need to maintain all
> the code is also smaller.

This also means that you can never have a MUA in vim, or an IDE, or a
debugger front-end or anything even remotely similar to Org.  Let the
people who want simplicity at a price of a limited editor use vim.  It
is not them that I think we should attract.

> I am NOT making apology of vim, I am just pointing how are they doing
> and why they success more with a "worst product" because it is not a
> mystery.

We don't _want_ success of the vim type.  That's the wrong type of
success for Emacs.

> >Maybe, just maybe, having "kill & yank" instead "copy & paste" is not
> >the cause of Emacs' lack of appeal to the new generations.
> >
> 
> It is one more.

Excuse me, but this is nonsense.  Our menu and tool bar say copy/paste
for at least 20 years, as do the manuals.  I wonder when people will
stop beating this dead horse.

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 2229 bytes --]

On Fri, May 17, 2019 at 10:28:00AM +0900, Jean-Christophe Helary wrote:
> 
> 
> > On May 17, 2019, at 5:50, Óscar Fuentes <ofv@wanadoo.es> wrote:
> > 
> >> so we need to offer some advantage on the first
> >> try over the others to keep the users.
> > 
> > Emacs provides some advantages, but they are not apparent until you
> > experience them. That's a problem for people grown on a culture of
> > instant gratification. Emacs appeals to certain type of users who
> > understand that gains require efforts.
> 
> I find that comment extremely condescending.
> 
> If "instant gratification" means finding a common ground on which one can get started right away, then I'm all for it.
> 
> Considering the state of affairs, emacs seems first to appeal to people who want to give priority to free software, at the *cost* of ease of use.

Condescending seems to go both ways, mind you.

> Access to free software should never be the sole privilege of "users who understand that gains require efforts". Quite the opposite.

Fully agreed here.

> > The really big problem is that Emacs no longer compete on areas were it used to bring the largest gains. Other editors largely surpassed Emacs' gains while requiring less effort.
> 
> Eli earlier clearly identified a number of areas where emacs required huge and totally undue efforts to get the thing to work as expected in the 21st century. 

Yes, and to achieve that, we gotta work together. It doesn't help
to insinuate intentions as you did above ;-)

Point is -- perhaps there's a tradeoff between initial ease-of-use
efficient for experienced users and the way from "here" to "there"?

Perhaps it takes more effort to come up with a software which is
good at all three departments?

I'm convinced that conventional software has promoted patterns which
tend to keep users in dependency (much more so in the Web "application"
space[1], because there it's strategic for your business), and that
those patterns infiltrate our way of seeing things -- therefore they
end up in free software too, for no reason.

Cheers

[1] Actually, those have names, like, e.g. Dark Patterns
   https://en.wikipedia.org/wiki/Dark_pattern

-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1814 bytes --]

On Fri, May 17, 2019 at 08:08:58AM +0200, Ergus wrote:
> On Fri, May 17, 2019 at 01:05:12PM +0900, Jean-Christophe Helary wrote:

[...]

> >Please. Text editing is the most common task by at least an order of magnitude in the IT world. Even for kids.
> >
> Also agree 100%.

I disagree, at least by 87%

Text editing can mean just that. Notepad.

It can mean that I can syntax-highlight a program text and get
documentation on a function.

It can also mean that I can keep clickable links from one text file
to other files. On the same computer. On another computer accessible
via ssh. On my mail inbox.

It can mean that I can execute snippets of a bash, C, Perl program
in my text and have their results inserted somewhere else (invaluable
to document little programs).

But it can also mean that I can load a "mostly UTF-8" file and my
editor doesn't choke on it. More so: If I do a local change and
save, the other parts end up identical to the input.

Or that I can load a megabite-big mostly-binary file and it takes
a second or so (I'll venture a guess: if you try that with Eclipse,
lights go out -- hence the name ;-D.

If things look funny at that point, you do M-x hexl-mode, and get
another view at things.

This all is Emacs to me. I get *very nervous* when I hear excited
people muttering "don't worry: we'll refactor all that for you!".

Now: this was all a bit tongue-in-cheek, and progress never happens
if no one pushes; I guess Emacs wouldn't have come so far without
folks like you, but understand that some (me!) appreciate Emacs
for its careful (perhaps sometimes slow) approach at changing
things (another example in this department is the Linux kernel:
"you don't break user space" is nearly dogma there, and for a
reason).

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 17 May 2019 11:54:50 +0300
> 
> Vim is consistent.

So is Emacs.  Inconsistencies in key bindings are rare exceptions in
Emacs.  Even the tutorial makes the point of explaining the rules for
consistent keybindings in basic editing commands.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 16.05.2019 0:17, Ergus wrote:
> That there are many pending bugs already to add one more for something
> that others consider a feature and don't want to be change.

That's not an answer to the question.

I can only repeat my advice. Venting emotions is healthy and all, but 
the main vehicle with which we drive change is bug reports.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 12:36, Eli Zaretskii wrote:
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Fri, 17 May 2019 11:54:50 +0300
>>
>> Vim is consistent.
> 
> So is Emacs.  Inconsistencies in key bindings are rare exceptions in
> Emacs.  Even the tutorial makes the point of explaining the rules for
> consistent keybindings in basic editing commands.

In the basics, sure. But I wouldn't still be using Emacs if it only 
provided basics.

We pride ourselves on being more than just an editor. But the more 
features a program provides, the more important it is to have a logical 
common structure and similar underlying principles under all those 
features. Or how else is a person supposed to remember how to use all of 
them?

Anyway, you seem to be agreeing with me, but this discussion started 
from Ergus giving an example of one such inconsistency. And I have yet 
to see you state a solid opinion about it in particular.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 17 May 2019 14:09:45 +0300
> 
> Anyway, you seem to be agreeing with me, but this discussion started 
> from Ergus giving an example of one such inconsistency. And I have yet 
> to see you state a solid opinion about it in particular.

A single example doesn't yet make a point.  To prove we have a
problem, much more than one example will be needed.

As for my opinion on this, I make a point of not posting "me too"
responses, if I don't have something more intelligent to add to the
discussion.  Especially a discussion as long as this one.

Re: Is Elisp really that slow?

[Ergus]

On Fri, May 17, 2019 at 12:01:54PM +0300, Eli Zaretskii wrote:
>> Date: Fri, 17 May 2019 07:52:02 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: help-gnu-emacs@gnu.org
>>
>> The problem comes when people needs to configure everything because the
>> defaults are terrible
>
>Please don't exaggerate like that, it makes the discussion more
>emotional and less useful.  Our defaults are not "terrible" by any
>account, certainly not when expressed in such a general form.
>
Sorry then. But I am getting tired.
>
>> Vim [...] is the only editor that worth to compare with emacs.
>
>I think you are wrong here.  There are VSCode, Atom, and a few others.
>
Yes, but they are very different and limited in many sense. VSCode
extensibility is very limited. And they don't work in terminal, which is
my base starting point (because some limitations comes from there)
>
>> It is easy to explain and I have refereed to this in many maaaaany other
>> emails:
>>
>> 1) vim is there in all the GNU/Linux distros.
>
>Not much we can do about that.  Feel free to lobby distros to include
>Emacs.  IMO, if something is related to 40-year old decision, it is
>this one.
>
I did actually some time ago and some distributions just replied that
the community was too small and the package too big compared to vim, but
it could be easily installed from repositories bla bla bla.
>
>> 2) It works the same in all the systems, in all the languages, even the
>> default color themes are better by default.
>
>Emacs also works the same on all systems.  As for "better default
>colors", that debatable at best.  It's a matter of taste, and
>psychologically we tend to favor our first experience: old habits die
>hard.
>
The gui could look much modern with a couple of lines in the
configuration. But the first impression now is like a program in windows
95.

This is something that an emacs user changes in 10 minutes probably, but
for a new user is not trivial.

The gui interface is not important at all for me. This is one of the
very simple changes that could take 5 minutes to do and 3 months arguing
in the mailing list.
>
>> 3) The keybinds (apart from the insert-escape) are easier, more
>> ergonomic and logically composable.
>>
>> 4) It is extremely responsive and fast to open-close workflows. No
>> lagging, or delays, no server configuration needed.
>>
>> 5) The important editing commands are usually only 1 key far. We can
>> make a simple comparison:
>>
>> Move forward: C-n -> j
>> Move forward 3 lines: M-3 C-n -> 3j (look sparsity in the keyboard)
>> Copy 3 lines and return to point position:
>> C-SPC C-SPC C-a C-SPC M-3 M-w C-u C-SPC C-u C-SPC -> 3yy
>>
>> Plus:
>> hjkl are way more ergonomic than what we have (you can type them
>> with one hand, while the other types the prefix, but they are also
>> together, so going 3 lines up 5 letters left is way faster and easier)
>
>If you want to argue that Emacs should be a dual-mode editor like the
>vi family, and that this is your "future", then we will never agree.
>One of the revolutions that Emacs brought to text editing was its lack
>of modality, where you just type text to have it inserted.  Dual-mode
>editors are a thing of the past in my eyes, a remnant from the old
>days when editors didn't have the real-time display, where you needed
>to have commands like "move N lines from the current one, then delete
>M lines" etc.  Please don't even get me (and others) started on this
>"feature".
>
I don't use vim due to the modes. I literally hate the modes. But on the
other hand npbf are too sparse and between some keys that modify the
buffer when Ctrl is hold (ojd).

My real proposes (if any, which is not actually) would be to change
those basic ones with jkli, or asdw (it is just an example). Delete
redundant bindings like ESC = Alt for prefixes, M-4 = C-u, or the
numerical prefixes with alt and C and keep only one. Join similar
"opposite" commands like C-o and C-j, or comment uncomment to exploit
negative prefix for one of them (so we free a bind and standardize
somehow, except C-d and DEL). Reserve one prefix only for user specific
functions and recommend the packages not to use that. Enable some extra
verbose features when there is not any configuration file and not using
-Q (because it is a hint that probably it is a new user or an old user
in a foreign machine) (I mean which-key for example)

>> Plus:
>> In vim the user can enter complete lines/commands/functions:
>>
>> :e file
>> :8,10 s/search/replace/g
>
>How is this different from Emacs's "C-x C-f" or M-% in the selected
>region?  Are you saying that it's easier to remember ":e" than "C-x
>C-f"??
>
Once we entered C-x C-f (which is actually longer and with no
associative relation) it is not possible to change mind, go back and do
the equivalent to C-x C-v or C-x 4 f (without C-g reenter the new bind
and reenter the file). With a ":command argument" it is more familiar
for terminal users and is possible to add modifiers to the commands (to
easy standarization and reduce memorized bindings)... Add finally an undo
and redo, even if it is optional. but have some binding for both.
>
>And you contradict yourself here: simple text editors all provide
>operations on the selected portion of text.  Emacs does that, while
>vim tells the users to remember the cryptic "N,M s/foo/bar/g" stuff
>that goes back to the 50-year old ed(1) and sed(1) editors!
>
I don't complain about our approach in replace (with transient-mark-mode
of course, and there are suggestions there) it was just an example of a
very complex command that vim users love for some reason but it is
composed by many pieces that work independently with logic, no memory.
>
>> which is more intuitive and familiar for terminal users.
>
>"terminal users"? who are those?  what's a "terminal"?  Are we really
>going to target 70-year old curmudgeons that still work on a
>'terminal"?  why? because vim does that?
>
>Forgive me, but this is all backwards!  Modern users want first and
>foremost GUI editors, because you can do in GUI display stuff that is
>unimaginable for text-mode environments.
>
Actually all vim users are terminal users, so they are not quite a few
these days, specially on GNU/Linux. Server management, remote access,
embedded systems, and core hardware pieces with Linux kernels (routers)
raspberry pi and single board computers, High performance clusters don't
have GUI. The terminal emulators are still the most popular and
important part for GNU/Linux users. And those systems actually are more
used in servers than desktop. And looks like it will be like that for a
long time.

The gui support is a good move, but most of our potential (short-medium
time) users are terminal dependents.

Packages like i3wm, bash, zsh and low level GNU/Linux distributions
(like arch) are very popular.
>
>> Plus:
>> There are no conflicts with the modified inputs and the terminals, so
>> they have more keybindings to use.
>
>Yesh, and you pay by having two modes.  No, thanks.
>
I know, I am just pointing out one of their advantages over our design
limitations. I really hate the modes.
>
>> 6) They do one thing and do it well. Editing functionalities have
>> priority (for example column indicator or line numbers were added very
>> long time ago.)
>
>Line numbers are a _must_ in vim, because so many commands _require_
>you to name the line numbers.  See your example above.  That Emacs
>originally didn't have line numbers is because you don't actually
>_need_ them so much.
>
For programming it is a must, actually. Compilation errors and warnings
are much simple to find that way. Also, going to a line is the kind of
command that must have only a 2 keys binding by default (and probably a
behavior like goto-line-preview by default)
>
>> 7) I understand it is also much simpler than emacs in functionalities,
>> but that is a benefit from the maintenance and update point of
>> view. They don't need to maintain an interpreter, their own language, a
>> graphical and a terminal interface, different modes for every
>> programming language, wrapper functions for terminal commands like grep
>> (or version control functionalities) a browser, file manager, a server
>> interface and client, a network infrastructure... This also means that
>> the number of programmers and expert fields they need to maintain all
>> the code is also smaller.
>
>This also means that you can never have a MUA in vim, or an IDE, or a
>debugger front-end or anything even remotely similar to Org.  Let the
>people who want simplicity at a price of a limited editor use vim.  It
>is not them that I think we should attract.
>
I mention this because there is too much code to maintain and of course
we can't do the best in all those fields. There are external packages
(compilers, text web browsers) that at some point we will need to trust
them and rely functionalities on them. And other functionalities could
be unified (like grep, rgrep and so on to a single general function to
call all shell commands and produce the same outputs (we already have
that, but still specialize to produce occur like buffers))
>
>> I am NOT making apology of vim, I am just pointing how are they doing
>> and why they success more with a "worst product" because it is not a
>> mystery.
>
>We don't _want_ success of the vim type.  That's the wrong type of
>success for Emacs.
>
Why?
>
>> >Maybe, just maybe, having "kill & yank" instead "copy & paste" is not
>> >the cause of Emacs' lack of appeal to the new generations.
>> >
>>
>> It is one more.
>
>Excuse me, but this is nonsense.  Our menu and tool bar say copy/paste
>for at least 20 years, as do the manuals.  I wonder when people will
>stop beating this dead horse.
>
In reply to the other email... I proposed to do a survey and
functionalities interest question a long time ago in the developers
mailing list. I even proposed some initial questions to do. Sent the
link of some pages to publish that and finally nobody cared.

Re: Is Elisp really that slow?

[Ergus]

Sorry for starting this, this is exactly the kind of discussions I try
to avoid at any price because at the end there are a lot of emails that
finally solve nothing because there is never an agreement.

I just mentioned AN EXAMPLE. So please, as none of these issues will be
solved, just try to forget this threads (which started as a question
about using a common lisp interpreter to improve performance) and
continue working with emacs as you do normally.

Else just think what a new user would like to see in a editor (or what
they don't find in emacs) and just make a list (please open mind). And
publish that somewhere that people can vote for them.

We have this discussions so often because we only get the developer's and
old users opinion. But not a general public survey to measure global
opinions. 


On Fri, May 17, 2019 at 03:04:17PM +0300, Eli Zaretskii wrote:
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Fri, 17 May 2019 14:09:45 +0300
>>
>> Anyway, you seem to be agreeing with me, but this discussion started
>> from Ergus giving an example of one such inconsistency. And I have yet
>> to see you state a solid opinion about it in particular.
>
>A single example doesn't yet make a point.  To prove we have a
>problem, much more than one example will be needed.
>
>As for my opinion on this, I make a point of not posting "me too"
>responses, if I don't have something more intelligent to add to the
>discussion.  Especially a discussion as long as this one.
>

Re: Is Elisp really that slow?

[Eli Zaretskii]

> Date: Fri, 17 May 2019 14:35:51 +0200
> From: Ergus <spacibba@aol.com>
> Cc: help-gnu-emacs@gnu.org
> 
> >> Vim [...] is the only editor that worth to compare with emacs.
> >
> >I think you are wrong here.  There are VSCode, Atom, and a few others.
> >
> Yes, but they are very different and limited in many sense. VSCode
> extensibility is very limited. And they don't work in terminal, which is
> my base starting point (because some limitations comes from there)

We want to do all these editors do, and also all the rest.  That's one
of the Emacs's sale points.

Currently, we have many more features, but in the IDE department we
lag after those "limited" alternatives.  We need to catch up.

> >> 1) vim is there in all the GNU/Linux distros.
> >
> >Not much we can do about that.  Feel free to lobby distros to include
> >Emacs.  IMO, if something is related to 40-year old decision, it is
> >this one.
> >
> I did actually some time ago and some distributions just replied that
> the community was too small and the package too big compared to vim, but
> it could be easily installed from repositories bla bla bla.

Is an Emacs installation really significantly larger than a Vim
installation nowadays?

> The gui could look much modern with a couple of lines in the
> configuration. But the first impression now is like a program in windows
> 95.

AFAIK, the GUI is modeled after GTK applications.

But feel free to suggest how to "modernize" it (on emacs-devel,
preferably), I don't think we had such a discussion in a long while.

> This is something that an emacs user changes in 10 minutes probably, but
> for a new user is not trivial.

If many users change the GUI appearance, then we should see if those
changes should be made by default.  Hard to say without knowing the
details.  FWIW, most posts about "my favorite config" simply turn off
most GUI features, and we cannot learn much from those.

> The gui interface is not important at all for me. This is one of the
> very simple changes that could take 5 minutes to do and 3 months arguing
> in the mailing list.

Then I guess you are in a minority.  Certainly among newcomers.

> >If you want to argue that Emacs should be a dual-mode editor like the
> >vi family, and that this is your "future", then we will never agree.
> >One of the revolutions that Emacs brought to text editing was its lack
> >of modality, where you just type text to have it inserted.  Dual-mode
> >editors are a thing of the past in my eyes, a remnant from the old
> >days when editors didn't have the real-time display, where you needed
> >to have commands like "move N lines from the current one, then delete
> >M lines" etc.  Please don't even get me (and others) started on this
> >"feature".
> >
> I don't use vim due to the modes. I literally hate the modes. But on the
> other hand npbf are too sparse and between some keys that modify the
> buffer when Ctrl is hold (ojd).

In that case, I don't even understand why you brought that up.  Not
having modes comes with a price, and IMO it is not too high.  Once we
all are on the same page about this basic fact, we should stop wasting
time discussing it.  Arguing for a change in basic bindings like C-f,
"C-x C-f", etc. is a non-starter, the community will never agree to
something like that.

> My real proposes (if any, which is not actually) would be to change
> those basic ones with jkli, or asdw (it is just an example).

This is not going to fly, no way.

> Delete redundant bindings like ESC = Alt for prefixes, M-4 = C-u, or
> the numerical prefixes with alt and C and keep only one.

This will cause more keystrokes in some cases.  E.g., if the binding
is Meta-something, the M-digit binding is very handy, and the same
with commands that are Control-something.

> Join similar "opposite" commands like C-o and C-j, or comment
> uncomment to exploit negative prefix for one of them

This is not ergonomic for frequent commands (witness how you exempt
DEL from this scheme), because typing the prefix too frequently is
painful.

Anyway, which other editor does that?  They all have different
commands to DO and to un-DO.

> Reserve one prefix only for user specific functions and recommend
> the packages not to use that.

That's C-c, which we already have.

> Enable some extra verbose features when there is not any
> configuration file and not using -Q (because it is a hint that
> probably it is a new user or an old user in a foreign machine) (I
> mean which-key for example)

What is which-key.

The idea to be more verbose is an interesting one, but I don't yet see
what kind of implementation do you envision.  For starters, where will
the extra text come from, and where and under what conditions will it
be displayed?

> Actually all vim users are terminal users

Then I don't think we should be too interested in those, when we
describe modernization of Emacs.

> so they are not quite a few these days, specially on
> GNU/Linux. Server management, remote access, embedded systems, and
> core hardware pieces with Linux kernels (routers) raspberry pi and
> single board computers, High performance clusters don't have
> GUI. The terminal emulators are still the most popular and important
> part for GNU/Linux users. And those systems actually are more used
> in servers than desktop. And looks like it will be like that for a
> long time.

Emacs does support text-mode terminals, and it does that very well.
But it makes no sense to build the UI on the assumption that most of
our users work on a TTY.  In particular, if we ever catch up on the
IDE front, because text-mode IDEs cannot have many important features
considered a must-have for an IDE.

> The gui support is a good move, but most of our potential (short-medium
> time) users are terminal dependents.

That should be backed up by some statistics.  I don't think it's true,
but even if it is, it just is one result of us lagging behind in the
IDE department.

> >Line numbers are a _must_ in vim, because so many commands _require_
> >you to name the line numbers.  See your example above.  That Emacs
> >originally didn't have line numbers is because you don't actually
> >_need_ them so much.
> >
> For programming it is a must, actually.

I disagree.

> Compilation errors and warnings are much simple to find that way.

Our compilation-related features make that entirely unneeded.

> Also, going to a line is the kind of command that must have only a 2
> keys binding by default (and probably a behavior like
> goto-line-preview by default)

Why would one need to go to a line by its number, when you have
fast-movement commands like "C-u C-u C-n"?

> >This also means that you can never have a MUA in vim, or an IDE, or a
> >debugger front-end or anything even remotely similar to Org.  Let the
> >people who want simplicity at a price of a limited editor use vim.  It
> >is not them that I think we should attract.
> >
> I mention this because there is too much code to maintain and of course
> we can't do the best in all those fields.

That's the price, yes.  But removing valuable features to make the
maintenance easier is IMO the wrong way of solving this problem.

> >We don't _want_ success of the vim type.  That's the wrong type of
> >success for Emacs.
> >
> Why?

Because we are a different kind of beast, see above.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 15:04, Eli Zaretskii wrote:

>> Anyway, you seem to be agreeing with me, but this discussion started
>> from Ergus giving an example of one such inconsistency. And I have yet
>> to see you state a solid opinion about it in particular.
> 
> A single example doesn't yet make a point.  To prove we have a
> problem, much more than one example will be needed.

What point would I be trying to prove? That Emacs is outdated are people 
shouldn't be using it anymore?

I want us to *work toward* making Emacs more consistent, across all of 
its features. An example of it being non-consistent is not an argument, 
it's an opportunity to make it better.

> As for my opinion on this, I make a point of not posting "me too"
> responses, if I don't have something more intelligent to add to the
> discussion.  Especially a discussion as long as this one.

Since you're the person to make the ultimate choice, your silence should 
be taken to mean you don't see a reason to change things.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 16:13, Eli Zaretskii wrote:
> We want to do all these editors do, and also all the rest.  That's one
> of the Emacs's sale points.
> 
> Currently, we have many more features, but in the IDE department we
> lag after those "limited" alternatives.  We need to catch up.

We are actually okay in the IDE department, with the help of the 
community, even if you don't like to consider it to be a part of Emacs.

We really are much better in that regard these days than we were even a 
couple years ago. A lot of it is thanks to VS Code and LSP, of course, 
but not all.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> Date: Fri, 17 May 2019 14:56:57 +0200
> From: Ergus <spacibba@aol.com>
> Cc: help-gnu-emacs@gnu.org
> 
> Sorry for starting this, this is exactly the kind of discussions I try
> to avoid at any price because at the end there are a lot of emails that
> finally solve nothing because there is never an agreement.

It is up to us whether this discussion will lead somewhere
constructive.

> I just mentioned AN EXAMPLE. So please, as none of these issues will be
> solved

If this is just a single example, then yes, we should probably forget
about it.  OTOH, if there are many such examples, we should not
forget.  Inconsistency in keybindings is baaad, mkay?  But a single
inconsistent binding, or a small number of them, are not yet a problem
that needs our attention.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 17 May 2019 16:17:56 +0300
> 
> On 17.05.2019 15:04, Eli Zaretskii wrote:
> 
> >> Anyway, you seem to be agreeing with me, but this discussion started
> >> from Ergus giving an example of one such inconsistency. And I have yet
> >> to see you state a solid opinion about it in particular.
> > 
> > A single example doesn't yet make a point.  To prove we have a
> > problem, much more than one example will be needed.
> 
> What point would I be trying to prove? That Emacs is outdated are people 
> shouldn't be using it anymore?

No, that we have lots of inconsistencies in keybindings (or
elsewhere).

> > As for my opinion on this, I make a point of not posting "me too"
> > responses, if I don't have something more intelligent to add to the
> > discussion.  Especially a discussion as long as this one.
> 
> Since you're the person to make the ultimate choice, your silence should 
> be taken to mean you don't see a reason to change things.

I hope now it is clear that would be a wrong interpretation.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 17 May 2019 16:22:43 +0300
> 
> > Currently, we have many more features, but in the IDE department we
> > lag after those "limited" alternatives.  We need to catch up.
> 
> We are actually okay in the IDE department, with the help of the 
> community, even if you don't like to consider it to be a part of Emacs.

I don't think I agree, as I hear a lot of complaints about the
difficulties to set up an environment for working on this or that
language.  Even if all the parts of the solution are already in place
(and I don't think they are), the difficulty in bringing them to work
together in a coherent way is something that needs fixing, IMO.

> We really are much better in that regard these days than we were even a 
> couple years ago.

If so, it's good to know.  But I don't think we are quite there yet.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 16:35, Eli Zaretskii wrote:
> No, that we have lots of inconsistencies in keybindings (or
> elsewhere).

That would be a more difficult discussion, probably calling for more 
changes across the board. We may go there another time, but not today, 
at least not as far as I'm concerned.

> I hope now it is clear that would be a wrong interpretation.

Why not fix at least one thing, then? It wouldn't change much by itself, 
but it would send a good message.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Jean-Christophe Helary <brandelune@gmail.com> writes:

>> On May 17, 2019, at 11:26, Óscar Fuentes <ofv@wanadoo.es> wrote:
>> 
>> Jean-Christophe Helary <brandelune@gmail.com> writes:
>> 
>>> If "instant gratification" means finding a common ground on which one
>>> can get started right away, then I'm all for it.
>> 
>> "Instant gratification" means wanting things that require no learning
>> nor practicing nor understanding to be effectively used *right* *now*.
>
> There is another word for that (and the earlier wheel metaphor may
> have escaped you), it is call "standardization".

Where is that standard?

>> It would be doubleplusgood if Emacs could be one of those things but,
>> alas, it is obvious that text editors still are on the class of things
>> that require certain effort to be used effectively.
>
> Please. Text editing is the most common task by at least an order of
> magnitude in the IT world.

So what?

> Even for kids.

Well, I understand now. Your standard ends in CUA.

It is obvious that you and me have vast differences on what we expect
from a text editor. I doubt that we can have a productive discussion, so
I'll stop here.

Re: Is Elisp really that slow?

[Óscar Fuentes]

<tomas@tuxteam.de> writes:

> On Fri, May 17, 2019 at 08:08:58AM +0200, Ergus wrote:
>> On Fri, May 17, 2019 at 01:05:12PM +0900, Jean-Christophe Helary wrote:
>
> [...]
>
>> >Please. Text editing is the most common task by at least an order of
>> >magnitude in the IT world. Even for kids.
>> >
>> Also agree 100%.
>
> I disagree, at least by 87%

[snip]

Well said.

If somebody reads this thread knowing nothing about text editors, he
will likely think that Emacs is a mess of arbitrary hacks and the other
editors are models of consistency and elegance. Well, truth is that
those editors are half-assed hodgepodges of accumulated afterthoughts.

They allow you to do basic things in an ineffective way, but as they are
menu- and dialog-based many people are happy because those basic
features are easily discoverable and usable without training. Then, when
you need to do a search&replace that goes beyond the dumb thing that the
dialog offers, well, bad luck. (In my experience, the possibility that
better things could exist don't cross the mind of many users.)

Oh, about saying that Vim is consistent (while implying that Emacs is
not)... that's a good joke :-)

Re: Is Elisp really that slow?

[Óscar Fuentes]

Ergus <spacibba@aol.com> writes:

>>Free Software is not a factor *today*, because most competitors are Free
>>Software too. Even Visual Studio Code is MIT-licensed.
>>
> I will no go in the free vs open source here. But no, most of the
> editors are open source, not free. 

Open Source is Free Software, at least for licenses like MIT that
respects the user's esseential freedoms as defined by the GNU manifesto:

https://www.gnu.org/philosophy/open-source-misses-the-point.html

As for the other points on your message, I can't go into lenghty
discussions now but I agree on everything Eli said on his response.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 17:01, Óscar Fuentes wrote:
> Then, when
> you need to do a search&replace that goes beyond the dumb thing that the
> dialog offers, well, bad luck.

What do you do to search&replace in all files across the project?

Re: Is Elisp really that slow?

[Óscar Fuentes]

Ergus <spacibba@aol.com> writes:

> In reply to the other email... I proposed to do a survey and
> functionalities interest question a long time ago in the developers
> mailing list. I even proposed some initial questions to do. Sent the
> link of some pages to publish that and finally nobody cared.

If you come with *results* of a representative survey, I'm pretty sure
that Emacs developers will study it with great attention.

But keep in mind that probing opinions is a tricky art. Recruiting
reprentative populations and making the right questions is quite
complex.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Fri, 17 May 2019 16:43:14 +0300
> 
> On 17.05.2019 16:35, Eli Zaretskii wrote:
> > No, that we have lots of inconsistencies in keybindings (or
> > elsewhere).
> 
> That would be a more difficult discussion

You said you thought the competition was more popular because they are
more consistent.  We need evidence to support that, if we are to
believe that this is a reason.  And if it is a reason, we should work
to eliminate those inconsistencies.

Thus, a single example certainly is not enough to identify a general
problem that could explain why we are less popular.

> Why not fix at least one thing, then?

I didn't say I objected to such a change.  So this question is not for
me to answer.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 17.05.2019 12:36, Eli Zaretskii wrote:
>>> From: Dmitry Gutov <dgutov@yandex.ru>
>>> Date: Fri, 17 May 2019 11:54:50 +0300
>>>
>>> Vim is consistent.

That was not my recollection when I learned it, nor is my impression
when I use it daily as an Evil user and as an occasional "true vim"
user.

I mean, vim has certain rules that work consistently, but not everything
that vim can do is subject to consistent rules. The ":" interface, for
instance. And even on the supposedly consistent interface there are
surprising details such as dw giving different results than vwd.

>> So is Emacs.  Inconsistencies in key bindings are rare exceptions in
>> Emacs.  Even the tutorial makes the point of explaining the rules for
>> consistent keybindings in basic editing commands.
>
> In the basics, sure. But I wouldn't still be using Emacs if it only
> provided basics.
>
> We pride ourselves on being more than just an editor. But the more
> features a program provides, the more important it is to have a
> logical common structure and similar underlying principles under all
> those features. Or how else is a person supposed to remember how to
> use all of them?

I doubt anybody is interested on using all of Emacs features. Not even
10% of them. Consistence for consintence's sake is useless. On every
case (and Emacs is many editors within an editor) there are local
optimums and those should prevail over global consistence.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 17.05.2019 17:01, Óscar Fuentes wrote:
>> Then, when
>> you need to do a search&replace that goes beyond the dumb thing that the
>> dialog offers, well, bad luck.
>
> What do you do to search&replace in all files across the project?

What I do is invoke a searcher such as "ag" or vc-git-grep and then
wgrep to operate on the resulting buffer. This has the advantage that I
can simply do a replace-string or a more detailed manual edition on a
case-by-case basis with the associated file on a side window for
context, etc.

RE: Is Elisp really that slow?

[Drew Adams]

> I've been in this software business for 30+ years, and with the arrival
> of "commodity computing", I've observed a strong anti-pattern: designs
> tend to cater to the buyer and not to the user.
> 
> I'll explain: If you decide which product to "buy", you haven't the
> time/resources to become proficient with that product: you decide on
> first impressions. But once you use that product for years, you'll
> "need" other features, which perhaps don't stick out at first sight.
> 
> There are two mechanisms pushing that anti-pattern forward:
> 
> 1. In bigger corporations those taking the decision which software
>   to "buy" (usually in the monetary sense) won't be those
>   paid (and thus more or less blackmailed) to use it. Management
>   goes "oh, shiny" and workers go "oh, no!".
> 
> 2. In general, when you personally "buy" a piece of software
>   (in the monetary, or in the general sense), you often have
>   no idea on what you need, so you go "oh shiny" again, and
>   sink considerable effort into fine-tuning your interface to
>   that software. Re-tuning is so expensive that you better not
>   think of it (as a long-time vi user, I mostly moved to Emacs
>   about ten years ago: it /was/ expensive.
> 
> Cheers
> 
> [1] "buy" in a very general sense: it might cost money or not,
>    but it'll cost learning, dedication and commitment.

And in a later msg:

> I'm convinced that conventional software has promoted patterns
> which tend to keep users in dependency (much more so in the Web
> "application" space[1], because there it's strategic for your
> business), and that those patterns infiltrate our way of seeing
> things -- therefore they end up in free software too, for no reason.

+1000.  Thank you for putting it so clearly.

user != chooser

Of course users may first choose.  But they also must then use.
Design for choosers ("sales" in some sense) is not, in itself,
design for users and use.

OTOH, a bazaar does have this advantage over a cathedral:
masses of user/developers attracted (in whatever way initially
- click-bait/"oh-shiny" or not) do sometimes improve a thing
that might have started out rudimentary or poorly designed.

Do Photoshop developers spend a huge amount of their energy
worrying about dumbing it down to commodity-sell it to every
Joe & Jane?  There may be some of that, but I doubt that it
is a strong/main consideration for the product design, in
particular for making the product as usable, useful, and
powerful as possible.

RE: Is Elisp really that slow?

[Drew Adams]

> The gui could look much modern with a couple of lines in the
> configuration. But the first impression now is like a program in windows
> 95.

"Modern" look-&-feel changes...  Imagine if hair or
clothing styles were stuck in the "modern" of 1975, 1985,
or 1995.  More important than making something look and
feel "modern" once and for all is helping it be _easily
changeable_, so it can be made (by users) to feel any way
(modern, retro, whatever) at any point in time.

Yes, it would be good if it were easier to change some of
the look-&-feel aspects of Emacs that are baked in C code.
To make that possible there would no doubt need to be more
people working on that aspect.

The argument that just attracting more users will provide
users who would contribute to that effort is weak.  Maybe,
maybe not.  What can be said is that if there are _no_ new
users then that effort won't be extended - duh.

For those aspects that users can change in _Lisp_, Emacs
shines in its ability to change look-&-feel, and other
behavior, so that it does _not_ get locked into today's
(and so yesterday's) idea of what "modern" is like.

The problem is that there are some basic look-&-feel
aspects that cannot be easily changed by users - they
are built in C and in some cases are platform-dependent.
Improving _that_ requires additional development energy,
and often quite a lot of expertise.

User-developers to work on that are, yes, sorely needed.

> My real proposes (if any, which is not actually) would be to change
> those basic ones with jkli, or asdw (it is just an example). Delete
> redundant bindings like ESC = Alt for prefixes, M-4 = C-u, or the
> numerical prefixes with alt and C and keep only one. Join similar
> "opposite" commands like C-o and C-j, or comment uncomment to exploit
> negative prefix for one of them (so we free a bind and standardize
> somehow, except C-d and DEL). Reserve one prefix only for user specific
> functions and recommend the packages not to use that.

Talk about changing default key bindings etc. to attract
new users is, in general, a waste of time.  That's _not_
the problem Emacs has wrt "modernism", as Eli and others
have pointed out.

Gaining a few great hackers to help with the C level (or
other "low-level" design/implementation) is a different
story - Emacs could really benefit from that.

RE: Is Elisp really that slow?

[Drew Adams]

> We have this discussions so often because we only get the developer's and
> old users opinion. But not a general public survey to measure global
> opinions.

The opinions of Emacs developers and experienced Emacs
user are the most helpful opinions, including in terms
of guiding where Emacs can and should go.

The opinions of non-users and non-developers of Emacs
are useful mainly in terms of discovering their
misconceptions and their current, often limited or
unfortunate experience with applications only somewhat
similar to Emacs (editors and other).

We have this discussion, especially about key bindings,
so often because there are _lots_ of non-users of Emacs
who discover Emacs, realize it's something powerful and
potentially even more powerful, but who have ingrained
key-binding etc. habits that they imagine are superior.

Eli nailed it when he gave his list of the kinds of
things Emacs really needs to improve its usefulness,
especially for programmers (e.g. IDE features).  It's
NOT at all about key bindings and such.

Perhaps most (nearly all?) newbies to Emacs initially
feel that the key bindings and interactions they were
used to before Emacs are superior or easier.  That's
been the case since Day One.

In general (and no, not always), they've been wrong
about that.  When they've been right they've just
customized their Emacs to do what they prefer.  And
in some cases Emacs has itself ended up changing
default behavior along the lines of what they've
implemented as, initially, personal customizations.

Yes, we took decades to turn on `transient-mark-mode'
by default.  And yes, we still haven't turned on
`delete-selection-mode' by default (something I'm in
favor of, FWIW).  So what?  It's so easy to turn
something on or redefine something in your init file.
The argument that Emacs should change to reflect,
out of the box, what most non-Emacs users are used to
is super-flawed, and hollow.

RE: Is Elisp really that slow?

[Drew Adams]

> > 1) vim is there in all the GNU/Linux distros.
> 
> Not much we can do about that.  Feel free to lobby distros to include
> Emacs.  IMO, if something is related to 40-year old decision, it is
> this one.

Yes.  Unix distributions did usually include both vi and
Emacs.  Why wouldn't GNU/Linux distributions include both?

Re: Is Elisp really that slow?

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Fri, 17 May 2019 16:22:43 +0300
>> 
>> > Currently, we have many more features, but in the IDE department we
>> > lag after those "limited" alternatives.  We need to catch up.
>> 
>> We are actually okay in the IDE department, with the help of the 
>> community, even if you don't like to consider it to be a part of Emacs.
>
> I don't think I agree, as I hear a lot of complaints about the
> difficulties to set up an environment for working on this or that
> language.  Even if all the parts of the solution are already in place
> (and I don't think they are), the difficulty in bringing them to work
> together in a coherent way is something that needs fixing, IMO.

Those problems exists but, unfortunately, what we could do to fix them
is limited. The complexity is not on setting-up Emacs itself, but on
installing and integrating the required *external dependencies* with the
corresponding Emacs package. It is very similar to setting Emacs on
Windows to work with grep, etc but with the added problem that the
external tools are not simple executable files that you must put on the
right place, but huge frameworks that comes on several variations
depending on their version and publisher.

Re: Is Elisp really that slow?

[Ken Goldman]

On 5/12/2019 10:45 AM, Óscar Fuentes wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
> 
>> Most stuff Emacs routinely does is something the other editors could
>> only dream about, so by and large you are comparing apples to peanuts.
> 
> Examples?
> 
> Emacs is amazing, but as far as productivity goes, at least as a
> programmer's editor for popular languages, it is far behind current
> contenders.

I was hooked on emacs the first time I saw keyboard macros.

Is there another editor that can spell check code?

I appreciate that it's multi-OS and multi-language.

I use emacs for bulk coding.  However, I use a language and OS specific 
IDE for debugging.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 16.05.2019 23:50, Óscar Fuentes wrote:
>> Emacs provides some advantages, but they are not apparent until you
>> experience them. That's a problem for people grown on a culture of
>> instant gratification. Emacs appeals to certain type of users who
>> understand that gains require efforts.
>
> I strongly disagree: as somebody who like instant gratification, I see
> that Emacs provides significant advantages to us: fast ability to see
> how it works and why, to change its behavior in an instance, to
> experiment without recompiling/restarting/whatever.

But that type of instant gratification require previous effort (learning
that doing those things is possible, to begin with.) The instant
gratification that I refer to consists on installing the éditeur de
texte du jour, "oh, looks nice", poke it for a minute and decide that it
is okay for you and be happy thereafter, or until you eventually become
tired/annoyed enough and then select a replacement by the same criteria.

Re: Is Elisp really that slow?

[Ergus]

On Fri, May 17, 2019 at 04:39:34PM +0300, Eli Zaretskii wrote:
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Fri, 17 May 2019 16:22:43 +0300
>>
>> > Currently, we have many more features, but in the IDE department we
>> > lag after those "limited" alternatives.  We need to catch up.
>>
>> We are actually okay in the IDE department, with the help of the
>> community, even if you don't like to consider it to be a part of Emacs.
>
>I don't think I agree, as I hear a lot of complaints about the
>difficulties to set up an environment for working on this or that
>language.  Even if all the parts of the solution are already in place
>(and I don't think they are), the difficulty in bringing them to work
>together in a coherent way is something that needs fixing, IMO.
>
For this issue spaceemacs has the so called sets. They don't reinvent
the wheel, they just integrate the packages that the community created
as a different/independent pieces. This is not optimal, but at least
is more scalable to maintain and modular. 

>> We really are much better in that regard these days than we were even a
>> couple years ago.
>
>If so, it's good to know.  But I don't think we are quite there yet.
>
Part of the problems we have to integrate environments is the collapse
of keybindings and overlap of functionalities. Just an EXAMPLE: We have
ycm, irony and rtags for the same purpose, and now lsp just came out. So
there are 3 pieces of code very complex and important but with few
people each. Same with anaconda-jedi-elpy, flycheck-flymake.

A technical question (a bit out of topic, but somehow related):

1) Why the core emacs decided to include flymake instead of flycheck
that is more modular and supports more languages?

2) Does it makes sense to make the keybindings case sensitive?

Re: Is Elisp really that slow?

[Noam Postavsky]

On Fri, 17 May 2019 at 11:40, Ergus <spacibba@aol.com> wrote:

> 1) Why the core emacs decided to include flymake instead of flycheck
> that is more modular and supports more languages?

The original flycheck author (who still holds the copyright on a large
portion of its code) strongly objects to including flycheck in Emacs.

https://github.com/flycheck/flycheck/issues/1177#issuecomment-267445833

Re: Is Elisp really that slow?

[Eli Zaretskii]

> Date: Fri, 17 May 2019 17:39:50 +0200
> From: Ergus <spacibba@aol.com>
> Cc: help-gnu-emacs@gnu.org
> 
> 1) Why the core emacs decided to include flymake instead of flycheck
> that is more modular and supports more languages?

Legal issues, AFAIR.

> 2) Does it makes sense to make the keybindings case sensitive?

No, not IMO.

Re: Is Elisp really that slow?

[Stefan Huchler]

Hello guys,

Eli Zaretskii <eliz@gnu.org> writes:

> If you want to argue that Emacs should be a dual-mode editor like the
> vi family, and that this is your "future", then we will never agree.
> One of the revolutions that Emacs brought to text editing was its lack
> of modality, where you just type text to have it inserted.  Dual-mode
> editors are a thing of the past in my eyes, a remnant from the old
> days when editors didn't have the real-time display, where you needed
> to have commands like "move N lines from the current one, then delete
> M lines" etc.  Please don't even get me (and others) started on this
> "feature".

I was reading your conversation and was mostly on your side till here. I
have to disagree with you at this point, sure many of this do things 10
times commands are very seldom useful, not never but seldom.

But that has nothing to do with modal vs non modal. Modal is more
efficiant, something simple as copy / paste is not only less keys to
press but literally more healthy, pressing c + reposition + v instead of
M-k + reposition + C-Y is just better for your fingers.

And for programmers 50% of commands you do as programmer are not basic
text manipulation like insert/delete key. So 50% of the
keystrokes/combitations you use on emacs are a attack on your health,
and are more likely to result in a error, hitting a long C-c C-q or even
longer commands is not so easy, even you don't get pain in your fingers.

Btw it's not only pain where many emacs famous hackers became problems
with, it's also getting faster tired.

http://ergoemacs.org/emacs/command-frequency.html

But I have to admit that I use the paredit commands or at least some of
em not so much for navigation but for manipulation. It kind of makes
sense and they are mostly 1-2 modifier and a arrow key. They are easy to
remember and that is a plus of course, I just think that the GNU project
should prioratise the health of their Users more than the
rememberability.

Btw there is a middle ground, look at xah-fly-keys, besides the modal
mode it also has a mode that depending on the keyboard can use key
combinations that are no keychords, so you don't have to hold a
modifier, which is what hurts your pinky.

So I hit my easy accessable Menu key and then afterwards the c key for
copying as example, the holding of keys and then stretching to other
keys is what hurts the pinky so much and why C-c C-f as example is a
horrible command.

Sadly we have to accept the current default keyboard layout therefor the
default settings in xah-fly keys is that space is the command key and
that only in command mode so for that you need the modal mode.

But we should not forget that the old emacs defaults were made for more
ergonomic keyboards that today we don't have anymore:

http://ergoemacs.org/emacs/emacs_kb_shortcuts_pain.html

They had the Ctrl and meta/alt key switched. So witch a bit of flexing
you could press the ctrl key with your thumb and not with your pinky.

So that the keyboard shortcuts are optimised for A rememberability and B
old lisps keyboards that 99.999% of the users don't use is surely not a
good default, I am sorry.

A way to solve that would maybe be intergrating something like ergoemacs
or xah-fly-keys and if you start emacs without config ask do you want
modal or classic mode, or something along that line.

Heck switching Control and Meta key in all keyboard configurations would
probably better the situation a lot. Because that was what the first
developer wanted back then with their lisp keyboards, but because of
backward compatibility the sitution became worse because the keyboard
changed and no emacs developer reacted to it.

> And you contradict yourself here: simple text editors all provide
> operations on the selected portion of text.  Emacs does that, while
> vim tells the users to remember the cryptic "N,M s/foo/bar/g" stuff
> that goes back to the 50-year old ed(1) and sed(1) editors!

I agree on that

> Excuse me, but this is nonsense.  Our menu and tool bar say copy/paste
> for at least 20 years, as do the manuals.  I wonder when people will
> stop beating this dead horse.

Yet how do you remember M-k and C-y because you know it means kill/yank,
else this shortcut makes no sense, so at least if you prioratise
rememberability over the health of the developers be consequent, if you
call it copy and paste make it C-c or M-c and for paste C-p. Then you
would at least be consequent.

That C-p is bound to previous line is no real reason because you also
have they arrow keys bound to it (which also is easier to remember), and
that C-c is for extentions their shortcut beginning also don't matter
because that problem is solved in cua-mode just add a small timer if you
are not fast enough to add the second part it's copy if you are fast
enough you can do your long shortcut.

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 632 bytes --]

On Fri, May 17, 2019 at 05:16:59PM +0300, Dmitry Gutov wrote:
> On 17.05.2019 17:01, Óscar Fuentes wrote:
> >Then, when
> >you need to do a search&replace that goes beyond the dumb thing that the
> >dialog offers, well, bad luck.
> 
> What do you do to search&replace in all files across the project?

  http://ergoemacs.org/emacs/find_replace_inter.html

Enjoy.

One of the things Emacs does very well is to integrate this
seemingly disparate mess pretty seamlessly, if you want.

E.g. Org-mode + Tramp: links pointing to files accessible
by some protocol (e.g. ssh or ftp or...). And so on.

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Óscar Fuentes]

<tomas@tuxteam.de> writes:

>> What do you do to search&replace in all files across the project?
>
>   http://ergoemacs.org/emacs/find_replace_inter.html
>
> Enjoy.

I guess that Dmitry knows the xref method fairly well :-)

(Doex

Re: Is Elisp really that slow?

[Óscar Fuentes]

Óscar Fuentes <ofv@wanadoo.es> writes:

> I guess that Dmitry knows the xref method fairly well :-)
>
> (Doex

Hit Send too fast. I was going to ask if it is possible to freely edit
an *xref* buffer and then apply the changes just like wgrep does. The
existence of xref-query-replace-in-results seems to indicate that the
answer is negative...

Re: Is Elisp really that slow?

[Ergus]

On Fri, May 17, 2019 at 06:29:44PM +0200, Stefan Huchler wrote:
>Hello guys,
>
>Eli Zaretskii <eliz@gnu.org> writes:
>
>> If you want to argue that Emacs should be a dual-mode editor like the
>> vi family, and that this is your "future", then we will never agree.
>> One of the revolutions that Emacs brought to text editing was its lack
>> of modality, where you just type text to have it inserted.  Dual-mode
>> editors are a thing of the past in my eyes, a remnant from the old
>> days when editors didn't have the real-time display, where you needed
>> to have commands like "move N lines from the current one, then delete
>> M lines" etc.  Please don't even get me (and others) started on this
>> "feature".
>
>I was reading your conversation and was mostly on your side till here. I
>have to disagree with you at this point, sure many of this do things 10
>times commands are very seldom useful, not never but seldom.
>
>But that has nothing to do with modal vs non modal. Modal is more
>efficiant, something simple as copy / paste is not only less keys to
>press but literally more healthy, pressing c + reposition + v instead of
>M-k + reposition + C-Y is just better for your fingers.
>
>And for programmers 50% of commands you do as programmer are not basic
>text manipulation like insert/delete key. So 50% of the
>keystrokes/combitations you use on emacs are a attack on your health,
>and are more likely to result in a error, hitting a long C-c C-q or even
>longer commands is not so easy, even you don't get pain in your fingers.
>
>Btw it's not only pain where many emacs famous hackers became problems
>with, it's also getting faster tired.
>
>http://ergoemacs.org/emacs/command-frequency.html
>
>But I have to admit that I use the paredit commands or at least some of
>em not so much for navigation but for manipulation. It kind of makes
>sense and they are mostly 1-2 modifier and a arrow key. They are easy to
>remember and that is a plus of course, I just think that the GNU project
>should prioratise the health of their Users more than the
>rememberability.
>
>Btw there is a middle ground, look at xah-fly-keys, besides the modal
>mode it also has a mode that depending on the keyboard can use key
>combinations that are no keychords, so you don't have to hold a
>modifier, which is what hurts your pinky.
>
>So I hit my easy accessable Menu key and then afterwards the c key for
>copying as example, the holding of keys and then stretching to other
>keys is what hurts the pinky so much and why C-c C-f as example is a
>horrible command.
>
>Sadly we have to accept the current default keyboard layout therefor the
>default settings in xah-fly keys is that space is the command key and
>that only in command mode so for that you need the modal mode.
>
>But we should not forget that the old emacs defaults were made for more
>ergonomic keyboards that today we don't have anymore:
>
>http://ergoemacs.org/emacs/emacs_kb_shortcuts_pain.html
>
>They had the Ctrl and meta/alt key switched. So witch a bit of flexing
>you could press the ctrl key with your thumb and not with your pinky.
>
>So that the keyboard shortcuts are optimised for A rememberability and B
>old lisps keyboards that 99.999% of the users don't use is surely not a
>good default, I am sorry.
>
I have experienced the issue that when adding a new functionality (to
isearch) there is not an available/standard method to assign a binding
for it. But also I ended discovering a new set of shortcuts I didn't
know. 

>A way to solve that would maybe be intergrating something like ergoemacs
>or xah-fly-keys and if you start emacs without config ask do you want
>modal or classic mode, or something along that line.
>
That could be something very interesting.

I am not proposing to make all the ergoemacs changes, but there are
interesting ideas there.

But ergoemacs actually failed because all the modules and packages bind
the default keys to the "expected" action. So for example next-line is
hardcoded to anything that moves one line down in helm, counsel, emms,
and so on. So at the end the user never gets a real 100% ergoemacs
experience. Similar happens in evil mode or even in cua-mode that is
integrated inside emacs. One never gets the 100% of the experience.

>Heck switching Control and Meta key in all keyboard configurations would
>probably better the situation a lot. Because that was what the first
>developer wanted back then with their lisp keyboards, but because of
>backward compatibility the sitution became worse because the keyboard
>changed and no emacs developer reacted to it.
>
>> And you contradict yourself here: simple text editors all provide
>> operations on the selected portion of text.  Emacs does that, while
>> vim tells the users to remember the cryptic "N,M s/foo/bar/g" stuff
>> that goes back to the 50-year old ed(1) and sed(1) editors!
>
>I agree on that
>
>> Excuse me, but this is nonsense.  Our menu and tool bar say copy/paste
>> for at least 20 years, as do the manuals.  I wonder when people will
>> stop beating this dead horse.
>
>Yet how do you remember M-k and C-y because you know it means kill/yank,
>else this shortcut makes no sense, so at least if you prioratise
>rememberability over the health of the developers be consequent, if you
>call it copy and paste make it C-c or M-c and for paste C-p. Then you
>would at least be consequent.
>
>That C-p is bound to previous line is no real reason because you also
>have they arrow keys bound to it (which also is easier to remember), and
>that C-c is for extentions their shortcut beginning also don't matter
>because that problem is solved in cua-mode just add a small timer if you
>are not fast enough to add the second part it's copy if you are fast
>enough you can do your long shortcut.
>
Personally I won't like to use the arrow keys, but if that could release
some bindings for a more standard behavior and simplify the access to
some functionalities (like get indentation), I will respect the
decision.

Another opportunity to release and access some similar commands affected
by the lack of bindings is to unify isearch with replace as isearch-mode
already have its own map. for a dynamic behavior there is the iedit
package. and multi-cursors.

swiper actually uses C-s for search, but C-n and C-p for next and
previous candidate, and with a bind to C-c m enables multiple cursors
(to replace all) and as C-r will be free, then it could be used to
switch from isearch to query-replace (as M-r switches to regex search).

Please, don't tell me that this won't fly; I already know.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 20:05, Óscar Fuentes wrote:

> Hit Send too fast. I was going to ask if it is possible to freely edit
> an *xref* buffer and then apply the changes just like wgrep does.

Unfortunately, no. But it should be possible to implement in the future.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 19:37, tomas@tuxteam.de wrote:
> One of the things Emacs does very well is to integrate this
> seemingly disparate mess pretty seamlessly, if you want.

What if you just want to replace 'foo' with 'bar'? Don't you think it 
requires too many steps?

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 19:50, Óscar Fuentes wrote:
> I guess that Dmitry knows the xref method fairly well:-)

The one I was thinking of myself is project-find-regexp and then 
pressing 'r'. But xref is a fairly recent development.

And wgrep not only in Emacs core, it's not even in ELPA. Which seems 
unfortunate for such basic functionality.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 17:50, Óscar Fuentes wrote:

>> On 17.05.2019 17:01, Óscar Fuentes wrote:
>>> Then, when
>>> you need to do a search&replace that goes beyond the dumb thing that the
>>> dialog offers, well, bad luck.
>>
>> What do you do to search&replace in all files across the project?
> 
> What I do is invoke a searcher such as "ag" or vc-git-grep and then
> wgrep to operate on the resulting buffer. This has the advantage that I
> can simply do a replace-string or a more detailed manual edition on a
> case-by-case basis with the associated file on a side window for
> context, etc.

A lot of the time, you don't need detailed editing for matches, though.

And the "dumb thing" the popular editors offer serves this case (as well 
as many others) actually very well and better than Emacs. So there's no 
reason to feel all smug and content, in my opinion.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 18:22, Óscar Fuentes wrote:
> But that type of instant gratification require previous effort (learning
> that doing those things is possible, to begin with.)

Still, that doesn't mean that we're free from making the amount of 
effort this takes arbitrarily hard, or that we shouldn't make it easier 
when possible.

> The instant
> gratification that I refer to consists on installing the éditeur de
> texte du jour, "oh, looks nice", poke it for a minute and decide that it
> is okay for you and be happy thereafter, or until you eventually become
> tired/annoyed enough and then select a replacement by the same criteria.

That seems like a strawman. A lot of smart, thoughtful professionals use 
other editors as well.

Re: Is Elisp really that slow?

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 17.05.2019 17:50, Óscar Fuentes wrote:
>
>>> On 17.05.2019 17:01, Óscar Fuentes wrote:
>>>> Then, when
>>>> you need to do a search&replace that goes beyond the dumb thing that the
>>>> dialog offers, well, bad luck.
>>>
>>> What do you do to search&replace in all files across the project?
>>
>> What I do is invoke a searcher such as "ag" or vc-git-grep and then
>> wgrep to operate on the resulting buffer. This has the advantage that I
>> can simply do a replace-string or a more detailed manual edition on a
>> case-by-case basis with the associated file on a side window for
>> context, etc.
>
> A lot of the time, you don't need detailed editing for matches, though.
>
> And the "dumb thing" the popular editors offer serves this case (as
> well as many others) actually very well and better than Emacs. So
> there's no reason to feel all smug and content, in my opinion.

If those other editors do the simple thing better than Emacs and you
decided that *I* don't need the not-so-simple thing a lot of the time...
well, it is difficult to keep arguing, isn't it?

Re: Is Elisp really that slow?

[Ergus]

On Fri, May 17, 2019 at 04:13:53PM +0300, Eli Zaretskii wrote:
>
>
>Is an Emacs installation really significantly larger than a Vim
>installation nowadays?
>
In arch Linux:

39M     /var/cache/pacman/pkg/emacs-26.1-2-x86_64.pkg.tar.xz

1.5M    /var/cache/pacman/pkg/vim-8.1.1186-1-x86_64.pkg.tar.xz
5.6M    /var/cache/pacman/pkg/vim-runtime-8.1.1186-1-x86_64.pkg.tar.xz

>> The gui interface is not important at all for me. This is one of the
>> very simple changes that could take 5 minutes to do and 3 months arguing
>> in the mailing list.
>
>Then I guess you are in a minority.  Certainly among newcomers.
>
Most Linux users are very terminal lovers. Actually many mac users like
mac because there is a bash shell, and even MS added the BashOnWindows
for a reason. 
>
>
>In that case, I don't even understand why you brought that up.  Not
>having modes comes with a price, and IMO it is not too high.  Once we
>all are on the same page about this basic fact, we should stop wasting
>time discussing it.  Arguing for a change in basic bindings like C-f,
>"C-x C-f", etc. is a non-starter, the community will never agree to
>something like that.
>
I know, but still those they are uncomfortable distributed on the
keyboard. Now that the TKL keyboards are becoming popular again is not
a good idea to rely on the arrow keys too much.

>> My real proposes (if any, which is not actually) would be to change
>> those basic ones with jkli, or asdw (it is just an example).
>
>This is not going to fly, no way.
>
I know that's why the "(if any, which is not actually)" was added.

>> Delete redundant bindings like ESC = Alt for prefixes, M-4 = C-u, or
>> the numerical prefixes with alt and C and keep only one.
>
>This will cause more keystrokes in some cases.  E.g., if the binding
>is Meta-something, the M-digit binding is very handy, and the same
>with commands that are Control-something.
>
I still don't know any user that uses the numeric arguments for common
commands. But I know this won't change either.
>
>> Join similar "opposite" commands like C-o and C-j, or comment
>> uncomment to exploit negative prefix for one of them
>
>This is not ergonomic for frequent commands (witness how you exempt
>DEL from this scheme), because typing the prefix too frequently is
>painful.
>
>
>Anyway, which other editor does that?  They all have different
>commands to DO and to un-DO.
>
They use C-z and C-S-z for undo-redo. That's why I asked in other email
about case sensitive bindings. Same for many others.

>> Reserve one prefix only for user specific functions and recommend
>> the packages not to use that.
>
>That's C-c, which we already have.
>

But some modes sets commands there like message mode, so the user needs
to care not reassign any in all the modes he uses. And sometimes there
are collapses (personal experience)

>> Enable some extra verbose features when there is not any
>> configuration file and not using -Q (because it is a hint that
>> probably it is a new user or an old user in a foreign machine) (I
>> mean which-key for example)
>
>What is which-key.
>
https://github.com/justbur/emacs-which-key

>The idea to be more verbose is an interesting one, but I don't yet see
>what kind of implementation do you envision.  For starters, where will
>the extra text come from, and where and under what conditions will it
>be displayed?
>
https://github.com/justbur/emacs-which-key
https://github.com/abo-abo/hydra

These are good starter points. Abo-abo actually holds a copyright
already. And all his packages modernize emacs a lot.

>> Actually all vim users are terminal users
>
>Then I don't think we should be too interested in those, when we
>describe modernization of Emacs.
>
I disagree, specially considering that the GNU/Linux OS is our main
destination. Else we need to center more in MSW (no coherent with free
software promotion, but also, in that case, the cua-mode must be
default, because C-x C-c C-v C-z are the standard there for
everything. And we won't convince any average MSW user to change that.
>
>
>> The gui support is a good move, but most of our potential
>> (short-medium time) users are terminal dependents.
>
>That should be backed up by some statistics.  I don't think it's true,
>but even if it is, it just is one result of us lagging behind in the
>IDE department.
>
We just need to consider the statistics of the users who use emacs
(including spacemacs) and what OS they use. Linux for desktop is sadly a
fail in the short medium time, but the number of GNU/Linux users keeps
growing (very slowly, but growing). And the most popular distributions
are not the most fancy-desktop-oriented ones. At the moment we can't
compite as an IDE for comercial software, so the probabilities that a 20
yo MS user chose emacs over VS or androidStudio are very low compared to
a Linux user that is developing backend, embedded systems or HPC. And
also the door for windows is opening with Azure and BashForLinux.
>
>> >Line numbers are a _must_ in vim, because so many commands _require_
>> >you to name the line numbers.  See your example above.  That Emacs
>> >originally didn't have line numbers is because you don't actually
>> >_need_ them so much.
>> >
>> For programming it is a must, actually.
>
>I disagree.
>
This is a deadlock ;p
>
>> Compilation errors and warnings are much simple to find that way.
>
>Our compilation-related features make that entirely unneeded.
>
If we use only C, yes, but in PHP (where an interpreter server produces
a log), Python mode (that the script is sent to an external terminal) or
Latex projects (built in latexmk), or fancy compilers like Rust... in
general we end up opening a terminal (in or outside emacs) an building
by hand. CMake project are difficult to set up as they use out of
sources compilation for everything and in tramp mode the remote
compilation usually needs to set up environment or lmod modules
dynamically ... The compilation engine can't be optimized/configured for
all those cases. If we do for 300 use cases there will be 300 more at
the moment.
>
>> Also, going to a line is the kind of command that must have only a 2
>> keys binding by default (and probably a behavior like
>> goto-line-preview by default)
>
>Why would one need to go to a line by its number, when you have
>fast-movement commands like "C-u C-u C-n"?
>
This is a joke right? ;) xdisp.c is a good example: long file, long
functions, lisp variables in the button and macros on the top.

Another example is how imenu fails in C++ mode when there are classes
and namespaces in a long file. So for going to a function isearch or
line numbers are the only alternatives.
>
>> >This also means that you can never have a MUA in vim, or an IDE, or a
>> >debugger front-end or anything even remotely similar to Org.  Let the
>> >people who want simplicity at a price of a limited editor use vim.  It
>> >is not them that I think we should attract.
>> >
>> I mention this because there is too much code to maintain and of course
>> we can't do the best in all those fields.
>
>That's the price, yes.  But removing valuable features to make the
>maintenance easier is IMO the wrong way of solving this problem.
>
It is the only really scalable solution I can see in the medium-long
term. And it is not applicable to everything of course.

But maintain everything, add new complex features to compete with
specialized applications already available that has been corrected and
upgraded among years I don't think will never work (Unix principles
imposes here, do one thing and do it right). We will not create a better
Lisp compiler that specialized projects that center all their effort on
that. Or gtags or a shell. It is impossible and unneeded.
>
>> >We don't _want_ success of the vim type.  That's the wrong type of
>> >success for Emacs.
>> >
>> Why?
>
>Because we are a different kind of beast, see above.
>

Re: Is Elisp really that slow?

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 17.05.2019 18:22, Óscar Fuentes wrote:
>> But that type of instant gratification require previous effort (learning
>> that doing those things is possible, to begin with.)
>
> Still, that doesn't mean that we're free from making the amount of
> effort this takes arbitrarily hard, or that we shouldn't make it
> easier when possible.

Sure.

>> The instant
>> gratification that I refer to consists on installing the éditeur de
>> texte du jour, "oh, looks nice", poke it for a minute and decide that it
>> is okay for you and be happy thereafter, or until you eventually become
>> tired/annoyed enough and then select a replacement by the same criteria.
>
> That seems like a strawman. A lot of smart, thoughtful professionals
> use other editors as well.

That's true. It is also true that most people who work with keyboards
don't do a thorough research and then use the best... they just go for
what's at hand, with bad consequences 20 years down the path.

See, I'm not saying that Emacs is perfect or the users of other editors
are lazy fools or wathever, I'm just saying that converging towards
popular editors is not good per se. I started with Borland IDEs, then
Visual Studio and recently Android Studio. No matter their popularity,
they are rudimentary editors compared to Emacs.

Re: Is Elisp really that slow?

[Eli Zaretskii]

> Date: Fri, 17 May 2019 21:24:29 +0200
> From: Ergus <spacibba@aol.com>
> Cc: help-gnu-emacs@gnu.org

This discussion is going nowhere useful.  So I'm going to respond just
to a couple of minor issues, and let's just stop and agree to disagree
about the rest, okay?  (Personally, I'm amazed you use Emacs with such
ideas in your head, excuse my being blunt.)

> >> Join similar "opposite" commands like C-o and C-j, or comment
> >> uncomment to exploit negative prefix for one of them
> >
> >This is not ergonomic for frequent commands (witness how you exempt
> >DEL from this scheme), because typing the prefix too frequently is
> >painful.
> >
> >
> >Anyway, which other editor does that?  They all have different
> >commands to DO and to un-DO.
> >
> They use C-z and C-S-z for undo-redo.

I didn't mean "undo" literally, I meant "do-SOMETHING" and
"UN-do-SOMETHING".  Every editor I've met has different commands for
action and counter-action, while you say we should have a single
command that does both.

Of course the fact that undo and redo have different keys only
confirms my point, and refutes yours.

> >> Reserve one prefix only for user specific functions and recommend
> >> the packages not to use that.
> >
> >That's C-c, which we already have.
> 
> But some modes sets commands there like message mode

This is a tangent: you said "reserve one prefix", and I pointed out
that we already did.

> >> Compilation errors and warnings are much simple to find that way.
> >
> >Our compilation-related features make that entirely unneeded.
> >
> If we use only C, yes, but in PHP (where an interpreter server produces
> a log), Python mode (that the script is sent to an external terminal) or
> Latex projects (built in latexmk), or fancy compilers like Rust... in
> general we end up opening a terminal (in or outside emacs) an building
> by hand. CMake project are difficult to set up as they use out of
> sources compilation for everything and in tramp mode the remote
> compilation usually needs to set up environment or lmod modules
> dynamically ...

That just means we need to extend our support for those other
languages.

> The compilation engine can't be optimized/configured for
> all those cases. If we do for 300 use cases there will be 300 more at
> the moment.

I disagree with this, I see no reason not to extend our support to
more languages as needed.  We are doing that constantly.

> >> Also, going to a line is the kind of command that must have only a 2
> >> keys binding by default (and probably a behavior like
> >> goto-line-preview by default)
> >
> >Why would one need to go to a line by its number, when you have
> >fast-movement commands like "C-u C-u C-n"?
> >
> This is a joke right? ;) xdisp.c is a good example: long file, long
> functions, lisp variables in the button and macros on the top.

And you go there by line numbers?  Meaning that you remember by heart
the line numbers of those places?  Me, I use M-. instead, and
occasionally M-C-s and M-> (a.k.a. C-END).  I don't really care on
which line number these variables and functions live.

> Another example is how imenu fails in C++ mode when there are classes
> and namespaces in a long file. So for going to a function isearch or
> line numbers are the only alternatives.

No, the alternative is to improve the features that fail to find what
they are supposed to.

> >That's the price, yes.  But removing valuable features to make the
> >maintenance easier is IMO the wrong way of solving this problem.
> >
> It is the only really scalable solution I can see in the medium-long
> term.

I see at least one more: make the developer team larger.  Which is
actually happening, albeit slowly.

> Unix principles imposes here, do one thing and do it right

That's not applicable to Emacs, since Emacs is not a tool, it's a
programming and text-editing environment.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 22:23, Óscar Fuentes wrote:

> If those other editors do the simple thing better than Emacs and you
> decided that *I* don't need the not-so-simple thing a lot of the time...
> well, it is difficult to keep arguing, isn't it?

I'm saying there's no need to feel smug, given the current situation. We 
should be able to do both the simple thing, and not-so-simple thing, 
better. Or at least, reasonably well.

And if we do the simple thing well, that's one less reason for anybody 
new to Emacs to shrug and give up.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 18:07, Óscar Fuentes wrote:

>>> We are actually okay in the IDE department, with the help of the
>>> community, even if you don't like to consider it to be a part of Emacs.
>>
>> I don't think I agree, as I hear a lot of complaints about the
>> difficulties to set up an environment for working on this or that
>> language.  Even if all the parts of the solution are already in place
>> (and I don't think they are), the difficulty in bringing them to work
>> together in a coherent way is something that needs fixing, IMO.
> 
> Those problems exists but, unfortunately, what we could do to fix them
> is limited. The complexity is not on setting-up Emacs itself, but on
> installing and integrating the required *external dependencies* with the
> corresponding Emacs package. It is very similar to setting Emacs on
> Windows to work with grep, etc but with the added problem that the
> external tools are not simple executable files that you must put on the
> right place, but huge frameworks that comes on several variations
> depending on their version and publisher.

The difficulty varies with the language and environment, but generally 
the steps can be:

- Install eglot and company.
- Install the appropriate language server program for the language 
you're going to be working in. For C/C++ all options are based on Clang, 
I think. Here's an incomplete list: 
https://github.com/joaotavora/eglot/blob/5f629ebfb680f43849ca894c7166a2b54ff5ba50/eglot.el#L81 
https://langserver.org/ also has a table.

You may have to add it to PATH.

- M-x global-company-mode (add it to the init script later).
- Open a file inside your project and M-x eglot.

Wait for the server to start and enjoy the completion/code navigation 
and documentation lookup features. Some refactorings as well, where 
supported.

A lot of the current user complains can be attributed to the fact that 
there is no single officially blessed, documented way to get IDE 
features in Emacs, but the above is a solid option.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 17.05.2019 17:36, Eli Zaretskii wrote:

> You said you thought the competition was more popular because they are
> more consistent.  We need evidence to support that, if we are to
> believe that this is a reason.

I think you're asking too much. The only way to undoubtedly prove 
Emacs's popularity is hurt by inconsistency is to create a mass A/B 
trial with volunteers spending time with different editors, as well as 
with a "fixed" version of Emacs, for some definition of "fixed". And 
then observing the behavior of al participants over the years.

The only things we can really do is to make the improvements (or, say, 
particular changes, to call them neutrally) and then somehow observe 
whether that improves things over time.

I never said that the competition was more popular *only because* it was 
more consistent, because there are a lot of things at play: installation 
base (for Vim), lots of marketing (mostly VS Code lately, but Atom too), 
market capture, certain features, of course. But user friendliness a 
large part of that. I think most people would agree that the user of a 
familiar editing paradigm is a serious advantage in terms of popularity.

We're probably not going to change our keybindings wholesale, but 
improving consistency can at least help learn and become comfortable 
with them faster.

And technically, changing key bindings is an easy task, so that wouldn't 
require too much actual work. I might also improve our image a little 
within the existing community, where the prevailing thinking is, 
unfortunately, that we're afraid to change long-standing behaviors, even 
if it's ultimately for the better.

> And if it is a reason, we should work
> to eliminate those inconsistencies.

So my vote is to simply work toward better UI, one step at a time. As 
long as the total capability of the features that we provide is not 
diminished, I do not think most existing users would really become 
upset. Similar to how John Yates put, it's going to take more than some 
changed/removed key bindings to drive away old and experienced users.

> Thus, a single example certainly is not enough to identify a general
> problem that could explain why we are less popular.

It's not the question I've been trying to answer.

>> Why not fix at least one thing, then?
> 
> I didn't say I objected to such a change.  So this question is not for
> me to answer.
Who then? Alan, who never saw this thread? Would you say that somebody 
should finally file the bug report?

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 18 May 2019 19:54:45 +0300
> 
> So my vote is to simply work toward better UI, one step at a time.

Is that different from what we do now?  If so, in what way?

> >> Why not fix at least one thing, then?
> > 
> > I didn't say I objected to such a change.  So this question is not for
> > me to answer.
> Who then? Alan, who never saw this thread? Would you say that somebody 
> should finally file the bug report?

If filing a bug will make a difference, yes.  Of course, we can fix
problems even without a filed bug, it isn't unheard of.

Re: Is Elisp really that slow?

[Dmitry Gutov]

On 18.05.2019 20:17, Eli Zaretskii wrote:

>> So my vote is to simply work toward better UI, one step at a time.
> 
> Is that different from what we do now?  If so, in what way?

In that somebody mentioned a specific issue, and we're not fixing it 
yet. Instead we're talking about global problems.

> If filing a bug will make a difference, yes.  Of course, we can fix
> problems even without a filed bug, it isn't unheard of.

OK, what should I do? You're the boss. I can probably easily fix it with 
a commit and push. Should I?

Re: Is Elisp really that slow?

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> If filing a bug will make a difference, yes.  Of course, we can fix
>> problems even without a filed bug, it isn't unheard of.
>
> OK, what should I do? You're the boss. I can probably easily fix it
> with a commit and push. Should I?

What you pretend is to completely ignore the maintainer of a prominent
package to force your very personal preference and implement a change
that, by your own admission, harms the users of that package without
benefiting Emacs in a tangible way other than "being uniform".

That's toxic.

FYI, Org-mode also assigns C-c C-c to something that does not align to
your schema. As does Eshell. And M-x shell. And Magit.

Please stop with this nonsense.

Re: Is Elisp really that slow?

[Emanuel Berg]

Ergus wrote:

> We are far from becoming a toxic community,
> but new/different ideas are not really very
> welcome and sometimes the arguments are like
> "it has always been like that", "old users
> don't want that change".
>
> The conservative attitude in emacs
> development group (apart from the technical
> obstacles like the workflow and the paperwork
> and so on) gives the idea of a closed
> environment (that actually it is from the
> development point of view because we are like
> frozen in the past) where only very experts
> are welcome (that's the vision people have
> from outside) so very few melpa developers
> are really interested to face all those
> issues to contribute.

While I don't like quotes that aren't actual
quotes (e.g., who exactly said "it has always
been like that"? NB rhetorical question) -
again, while I don't like that sort'a stuff,
I *do* have to agree I get this/your general
vibe as well!

I don't consider myself an Emacs expert - far
from it. But I've been here for 10+ years, so
I'm happy with my Emacs and my skill level.

But this place still doesn't feel like home!
That is strange.

I'm not sure your analysis is correct why this
is tho, but heck, it might be, or part of it
at least.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[조성빈]

> 2019. 5. 19. 오전 8:54, Óscar Fuentes <ofv@wanadoo.es> 작성:
> 
> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
>>> If filing a bug will make a difference, yes.  Of course, we can fix
>>> problems even without a filed bug, it isn't unheard of.
>> 
>> OK, what should I do? You're the boss. I can probably easily fix it
>> with a commit and push. Should I?
> 
> What you pretend is to completely ignore the maintainer of a prominent
> package to force your very personal preference and implement a change
> that, by your own admission, harms the users of that package without
> benefiting Emacs in a tangible way other than "being uniform”.

What you pretend is to completely ignore the importance of coherence and force your very personal preference, and harm users of that package by making them surprised by not being uniform.

> That's toxic.
> 
> FYI, Org-mode also assigns C-c C-c to something that does not align to
> your schema. As does Eshell. And M-x shell. And Magit.

And, that’s exactly the point! Emacs *is* inconsistent, and we should try to eliminate them (hopefully before a bigger user base appears).
There’s a point where any kind of big software platform has an HIG, like Apple macOS, iOS, or XDG, Gnome, Qt, e.g… and I would appreciate if we can make an HIG for Emacs packages.
Thinks like keybindings, UI, messages, etc. 

> Please stop with this nonsense.

Re: Is Elisp really that slow?

[Eli Zaretskii]

On May 19, 2019 1:43:15 AM GMT+03:00, Dmitry Gutov <dgutov@yandex.ru> wrote:
> On 18.05.2019 20:17, Eli Zaretskii wrote:
> 
> >> So my vote is to simply work toward better UI, one step at a time.
> > 
> > Is that different from what we do now?  If so, in what way?
> 
> In that somebody mentioned a specific issue, and we're not fixing it 
> yet. Instead we're talking about global problems.
> 
> > If filing a bug will make a difference, yes.  Of course, we can fix
> > problems even without a filed bug, it isn't unheard of.
> 
> OK, what should I do? You're the boss. I can probably easily fix it
> with 
> a commit and push. Should I?

As usual, please post a proposed patch with the ratiobale either to emacs-devel or (preferable) as bug report, and let's take it from there.

Re: Is Elisp really that slow?

[Óscar Fuentes]

조성빈 <pcr910303@icloud.com> writes:

>> FYI, Org-mode also assigns C-c C-c to something that does not align to
>> your schema. As does Eshell. And M-x shell. And Magit.
>
> And, that’s exactly the point! Emacs *is* inconsistent,

No, it isn't.

Somehow you decided that using C-c C-c with the same semantics across
unrelated modes is desirable because consistence. Furthermore, you
decided which semantics is the right one.

Well, this is wrong.

Why those modes that you take as models for C-c C-c use precisely that
binding for sending text to an inferior process? Because it is the most
used action and, hence, it is natural to use a keybinding that is fast
and easy to type.

That's the same reason why Org-mode uses C-c C-c for its most frequent
actions, although you pretend to remove that convenience because
"consistency"... when in fact you'll be reducing consistency!

Likewise, (E)shell and CC-Mode use C-c C-c because it is easy to relate
to the associated action *and* because it is easy to type.

Thus, you also would reduce Emacs' ergonomics and mnemonics because of
your false idea of consistency.

It is important to understand why things are as they are before acting
as if those who made them were incompetent and the world is depending on
us to fix their glaring mistakes.

Re: Is Elisp really that slow?

[Van L]

Emanuel Berg writes:

> Ergus wrote:
> I don't consider myself an Emacs expert -
> far from it. But I've been here for 10+
> years, so I'm happy with my Emacs and my
> skill level.  But this place still
> doesn't feel like home!  That is strange.

Is it possible to have the best of all possible
worlds?

For conservatives, a winter release of old gold keybindings.
For the free radicals, a spring release with modernizations.

When I use a long M-x sequence, a shortcut suggestion appears. It disappears before I can catch it. Can it stay for 30 seconds? Can there be an instant interactive override to set it whatever you like? Evolutionary programming of popular custom keybindings collected at upstream and put thru obstacle course competition is one way of composing a spring release.

-- 
© 2019 Van L
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
                             "The interface is a nightmare." - Brendan Schaub

Re: Is Elisp really that slow?

[조성빈]

2019. 5. 19. 오후 5:16, Van L <van@scratch.space> 작성:

> Emanuel Berg writes:
> 
>> Ergus wrote:
>> I don't consider myself an Emacs expert -
>> far from it. But I've been here for 10+
>> years, so I'm happy with my Emacs and my
>> skill level.  But this place still
>> doesn't feel like home!  That is strange.
> 
> Is it possible to have the best of all possible
> worlds?
> 
> For conservatives, a winter release of old gold keybindings.
> For the free radicals, a spring release with modernizations.

What if having a compatibility-mode that can be activated by something like:
```elisp
(classic-keybindings-mode 1)
```
and refine the default keybindings to be more consistent/mnemonic? People who miss the old keybindings will be elisp-proficient; Adding 1 s-exp to the init file won’t be a barrier.
For the refined keybindings, Spacemacs can provide a good starting point.

> When I use a long M-x sequence, a shortcut suggestion appears. It disappears before I can catch it. Can it stay for 30 seconds? Can there be an instant interactive override to set it whatever you like?

I would like a semi-AI that suggests interactive functions based on key presses or actions the user performs... `You can use C-e (goto-end-line) to perform 12 keystokes you performed.'
Saying about discoverability, I would like a context-sensitive right-click mouse menu, something like Microsoft Office. Most newcomers are familiar with finding functionality with the mouse; and it isn’t intuitive to find new keybindings/functions that Emacs provide to boost productivity. (Actually, that’s one of my problems; how should I find new functions...?)

> Evolutionary programming of popular custom keybindings collected at upstream and put thru obstacle course competition is one way of composing a spring release.
> 
> -- 
> © 2019 Van L
> gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
>                             "The interface is a nightmare." - Brendan Schaub
> 

Re: Is Elisp really that slow?

[Van L]

조성빈 <pcr910303@icloud.com> writes:

> 2019. 5. 19. 오후 5:16, Van L <van@scratch.space> 작성:
>
> For the refined keybindings, Spacemacs can provide a good starting point.

A bikeshedding of well worn paths of keybinding combinations is required
to justify the new starting point.

> I would like a semi-AI that suggests interactive functions based on

Maybe helm or ivy package does that.

> it isn’t intuitive to find new keybindings/functions that Emacs

One way is `C-h b' and think differently.

-- 
© 2019 Van L
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
                             "The interface is a nightmare." - Brendan Schaub

Re: Is Elisp really that slow?

[Ergus]

On Sun, May 19, 2019 at 07:35:58PM +0900, 조성빈 wrote:
>
>2019. 5. 19. 오후 5:16, Van L <van@scratch.space> 작성:
>
>> Emanuel Berg writes:
>>
>>> Ergus wrote:
>>> I don't consider myself an Emacs expert -
>>> far from it. But I've been here for 10+
>>> years, so I'm happy with my Emacs and my
>>> skill level.  But this place still
>>> doesn't feel like home!  That is strange.
>>
>> Is it possible to have the best of all possible
>> worlds?
>>
>> For conservatives, a winter release of old gold keybindings.
>> For the free radicals, a spring release with modernizations.
>
>What if having a compatibility-mode that can be activated by something like:
>```elisp
>(classic-keybindings-mode 1)
>```
>and refine the default keybindings to be more consistent/mnemonic? People who miss the old keybindings will be elisp-proficient; Adding 1 s-exp to the init file won’t be a barrier.
>For the refined keybindings, Spacemacs can provide a good starting point.
>
This is interesting. (of course spacemacs not in evil mode).

I will just tell an idea I had (before knowing how things work in emacs,
I actually thought it was implemented in that way)

If we create a single file with the full list of general bindings as
variables for all the main modes like

;; Basics

C-a begin-line-binding
C-e end-line-binding 
C-b go-backward-line-binding 
C-d delete-forward-binding
...

;; Prog mode
C-c C-c whatever-you-decide-binding

;; Minibuffer mode
bla bla bla
...

And so on.

All the derived modes can be changed to use the variables instead of
hard-code the bindings in multiple files.

this:
(define-key c-mode-base-map "\C-d" 'c-delete-forward)
will be:
(define-key c-mode-base-map delete-forward-binding 'c-delete-forward)

This is similar to use remap, but with some advantages:

1) The bindings will be organized in a single file and the collisions
will be exposed easily. And fixed for all of them.

2) In case a mode adds a new functionality and it needs a binding, it t
in the global map, it will be clear if some others already have a
similar one and will bind to the same using the same criteria. If the
binding has never been used, it will be added to the list for future
modes that wants the same functionality.

3) Old users will be minimally affected because the starting criteria
are the actual bindings.

4) Implementation of test modes (like ergoemacs or evil-mode) and future
modifications will require less effort and the final experience will be
less hacky than now..

5) If a user personalize a binding (changing the value of one of these
variables) the changes will apply to all the packages and modes she
uses in his section.

6) If the user wants to use remap or hard-code the binding (as now) the
actual behavior will be unchanged.

7) Packages like elpy, irony, or company will also use the "binding-list",
so the user will only learn the most specific bindings (if any) so
external packages will be standardized with minimal effort.

8) Future bindings changes that the community agree will not require
changes in the files of all the modes. With the potential side effects.

I know that proposing this here is a crazy idea, but the intention is
not to start a crazy war again, If it is crazy (for technical reasons)
just ignore it. Actually I just mention it because it seems to me like
something extremely simple-advantageous and maintainable. With minimal
affections to the user space.

But remember that we have a limited number of bindings so managing them
better is important if we want to find place for new functionalities.

In any case I will leave this threads as it is not going anywhere (as
usual with this topics) and what is obvious for me seems to be heretic
for the rest. But you are free to consider the idea if it has anything
useful for emacs..


>
>> When I use a long M-x sequence, a shortcut suggestion appears. It disappears before I can catch it. Can it stay for 30 seconds? Can there be an instant interactive override to set it whatever you like?
>
>I would like a semi-AI that suggests interactive functions based on key presses or actions the user performs... `You can use C-e (goto-end-line) to perform 12 keystokes you performed.'
>Saying about discoverability, I would like a context-sensitive right-click mouse menu, something like Microsoft Office. Most newcomers are familiar with finding functionality with the mouse; and it isn’t intuitive to find new keybindings/functions that Emacs provide to boost productivity. (Actually, that’s one of my problems; how should I find new functions...?)
>
>> Evolutionary programming of popular custom keybindings collected at upstream and put thru obstacle course competition is one way of composing a spring release.
>>
>> --
>> © 2019 Van L
>> gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
>>                             "The interface is a nightmare." - Brendan Schaub
>>
>
>

Re: Is Elisp really that slow?

[Ergus]

On Sun, May 19, 2019 at 07:35:58PM +0900, 조성빈 wrote:
>
>2019. 5. 19. 오후 5:16, Van L <van@scratch.space> 작성:
>
>> Emanuel Berg writes:
>>
>>> Ergus wrote:
>>> I don't consider myself an Emacs expert -
>>> far from it. But I've been here for 10+
>>> years, so I'm happy with my Emacs and my
>>> skill level.  But this place still
>>> doesn't feel like home!  That is strange.
>>
>> Is it possible to have the best of all possible
>> worlds?
>>
>> For conservatives, a winter release of old gold keybindings.
>> For the free radicals, a spring release with modernizations.
>
>What if having a compatibility-mode that can be activated by something like:
>```elisp
>(classic-keybindings-mode 1)
>```
>and refine the default keybindings to be more consistent/mnemonic? People who miss the old keybindings will be elisp-proficient; Adding 1 s-exp to the init file won’t be a barrier.
>For the refined keybindings, Spacemacs can provide a good starting point.
>
>> When I use a long M-x sequence, a shortcut suggestion appears. It disappears before I can catch it. Can it stay for 30 seconds? Can there be an instant interactive override to set it whatever you like?
>
>I would like a semi-AI that suggests interactive functions based on key presses or actions the user performs... `You can use C-e (goto-end-line) to perform 12 keystokes you performed.'
>Saying about discoverability, I would like a context-sensitive right-click mouse menu, something like Microsoft Office. Most newcomers are familiar with finding functionality with the mouse; and it isn’t intuitive to find new keybindings/functions that Emacs provide to boost productivity. (Actually, that’s one of my problems; how should I find new functions...?)
>
Ohh, Have you seen/used which-key? counsel/ivy list them, but which-key
does the same with the bindings. Spacemacs uses it.

>> Evolutionary programming of popular custom keybindings collected at upstream and put thru obstacle course competition is one way of composing a spring release.
>>
>> --
>> © 2019 Van L
>> gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
>>                             "The interface is a nightmare." - Brendan Schaub
>>
>
>

Re: Is Elisp really that slow?

[Ergus]

On Sun, May 19, 2019 at 10:48:26PM +1000, Van L wrote:
>조성빈 <pcr910303@icloud.com> writes:
>
>> 2019. 5. 19. 오후 5:16, Van L <van@scratch.space> 작성:
>>
>> For the refined keybindings, Spacemacs can provide a good starting point.
>
>A bikeshedding of well worn paths of keybinding combinations is required
>to justify the new starting point.
>

Actual spacemacs keybindings are based in mode editing ideas, so we
can't use them as they are OOB, but the concepts and ideas they use to
organize the bindings are very interesting and intuitive.

>
>> I would like a semi-AI that suggests interactive functions based on
>
>Maybe helm or ivy package does that.
>
>> it isn’t intuitive to find new keybindings/functions that Emacs
>
>One way is `C-h b' and think differently.
>
There is which-key. I can't recommend it enough to new users. I knew
about it very late (after 2 years or so) but, for starters, the learning
curve with it could be way easier.

>-- 
>© 2019 Van L
>gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
>                             "The interface is a nightmare." - Brendan Schaub
>
>

Re: Is Elisp really that slow?

[Ergus]

On Fri, May 17, 2019 at 11:12:39PM +0300, Eli Zaretskii wrote:
>> Date: Fri, 17 May 2019 21:24:29 +0200
>> From: Ergus <spacibba@aol.com>
>> Cc: help-gnu-emacs@gnu.org
>
>This discussion is going nowhere useful.  So I'm going to respond just
>to a couple of minor issues, and let's just stop and agree to disagree
>about the rest, okay?  (Personally, I'm amazed you use Emacs with such
>ideas in your head, excuse my being blunt.)
>
>> >> Join similar "opposite" commands like C-o and C-j, or comment
>> >> uncomment to exploit negative prefix for one of them
>> >
>> >This is not ergonomic for frequent commands (witness how you exempt
>> >DEL from this scheme), because typing the prefix too frequently is
>> >painful.
>> >
>> >
>> >Anyway, which other editor does that?  They all have different
>> >commands to DO and to un-DO.
>> >
>> They use C-z and C-S-z for undo-redo.
>
>I didn't mean "undo" literally, I meant "do-SOMETHING" and
>"UN-do-SOMETHING".  Every editor I've met has different commands for
>action and counter-action, while you say we should have a single
>command that does both.
>
>Of course the fact that undo and redo have different keys only
>confirms my point, and refutes yours.
>
Actually my proposal in this point was just to find a way to release
some bindings. It was not even a real proposal at all, just an attempt.

What is clear is that all the bindings are set and we talk about new
features without binding (or a plan to bind them). Y made all the
possible questions, case sensitive bindings, release some of them, make
a C-a go to indent and then to beginning of line... anything to have
some more place without create longer bindings.

>> >> Reserve one prefix only for user specific functions and recommend
>> >> the packages not to use that.
>> >
>> >That's C-c, which we already have.
>>
>> But some modes sets commands there like message mode
>
>This is a tangent: you said "reserve one prefix", and I pointed out
>that we already did.
>

So why C-mode, message modes and others bind commands there. Then the
user does not have a reserved place confident to no create collisions. 


>> >> Compilation errors and warnings are much simple to find that way.
>> >
>> >Our compilation-related features make that entirely unneeded.
>> >
>> If we use only C, yes, but in PHP (where an interpreter server produces
>> a log), Python mode (that the script is sent to an external terminal) or
>> Latex projects (built in latexmk), or fancy compilers like Rust... in
>> general we end up opening a terminal (in or outside emacs) an building
>> by hand. CMake project are difficult to set up as they use out of
>> sources compilation for everything and in tramp mode the remote
>> compilation usually needs to set up environment or lmod modules
>> dynamically ...
>
>That just means we need to extend our support for those other
>languages.
>
That's the perfect choice (utopic), with the actual number of
maintainers, ignoring whats already in melpa (that is a lot of community
work and time improving) I think I am pessimist about the
materialization.
>
>> The compilation engine can't be optimized/configured for
>> all those cases. If we do for 300 use cases there will be 300 more at
>> the moment.
>
>I disagree with this, I see no reason not to extend our support to
>more languages as needed.  We are doing that constantly.
>
The new methods standards and systems complexity grows faster and in
many directions that what we can maintain updated enough in the core. It
is not a theoretical problem, it is a practical limitation.  
>
>> >> Also, going to a line is the kind of command that must have only a 2
>> >> keys binding by default (and probably a behavior like
>> >> goto-line-preview by default)
>> >
>> >Why would one need to go to a line by its number, when you have
>> >fast-movement commands like "C-u C-u C-n"?
>> >
>> This is a joke right? ;) xdisp.c is a good example: long file, long
>> functions, lisp variables in the button and macros on the top.
>
>And you go there by line numbers?  Meaning that you remember by heart
>the line numbers of those places?  Me, I use M-. instead, and
>occasionally M-C-s and M-> (a.k.a. C-END).  I don't really care on
>which line number these variables and functions live.
>
OK, different workflows. 

>> Another example is how imenu fails in C++ mode when there are classes
>> and namespaces in a long file. So for going to a function isearch or
>> line numbers are the only alternatives.
>
>No, the alternative is to improve the features that fail to find what
>they are supposed to.
>
That's true, but in the mean time that's what we have and some bugs need
years to be solved. This in fact is a consequence of the C++ support
limitations we were mentioning before in this threads. 
>
>> >That's the price, yes.  But removing valuable features to make the
>> >maintenance easier is IMO the wrong way of solving this problem.
>> >
>> It is the only really scalable solution I can see in the medium-long
>> term.
>
>I see at least one more: make the developer team larger.  Which is
>actually happening, albeit slowly.
>
>> Unix principles imposes here, do one thing and do it right
>
>That's not applicable to Emacs, since Emacs is not a tool, it's a
>programming and text-editing environment.
>

Re: Is Elisp really that slow?

[Noam Postavsky]

On Sun, 19 May 2019 at 09:38, Ergus <spacibba@aol.com> wrote:

> >> >> Reserve one prefix only for user specific functions and recommend
> >> >> the packages not to use that.
> >> >
> >> >That's C-c, which we already have.
> >>
> >> But some modes sets commands there like message mode
> >
> >This is a tangent: you said "reserve one prefix", and I pointed out
> >that we already did.
> >
>
> So why C-mode, message modes and others bind commands there. Then the
> user does not have a reserved place confident to no create collisions.

Please read https://www.gnu.org/software/emacs/manual/html_node/elisp/Key-Binding-Conventions.html
aka (elisp) Key Binding Conventions.

RE: Is Elisp really that slow?

[Drew Adams]

> When I use a long M-x sequence, a shortcut suggestion appears. It disappears
> before I can catch it. Can it stay for 30 seconds? Can there be an instant
> interactive override to set it whatever you like?

`C-h v suggest-key-bindings':

 suggest-key-bindings is a variable defined in `simple.el'.
 Its value is t

 Documentation:
 Non-nil means show the equivalent key-binding when M-x command has one.
 The value can be a length of time to show the message for.
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 If the value is non-nil and not a number, we wait 2 seconds.

 You can customize this variable.

Re: Is Elisp really that slow?

[조성빈]

> 2019. 5. 19. 오후 10:00, Ergus <spacibba@aol.com> 작성:
> 
> On Sun, May 19, 2019 at 07:35:58PM +0900, 조성빈 wrote:
>> 
>> 2019. 5. 19. 오후 5:16, Van L <van@scratch.space> 작성:
>> 
>>> Emanuel Berg writes:
>>> 
>>>> Ergus wrote:
>>>> I don't consider myself an Emacs expert -
>>>> far from it. But I've been here for 10+
>>>> years, so I'm happy with my Emacs and my
>>>> skill level.  But this place still
>>>> doesn't feel like home!  That is strange.
>>> 
>>> Is it possible to have the best of all possible
>>> worlds?
>>> 
>>> For conservatives, a winter release of old gold keybindings.
>>> For the free radicals, a spring release with modernizations.
>> 
>> What if having a compatibility-mode that can be activated by something like:
>> ```elisp
>> (classic-keybindings-mode 1)
>> ```
>> and refine the default keybindings to be more consistent/mnemonic? People who miss the old keybindings will be elisp-proficient; Adding 1 s-exp to the init file won’t be a barrier.
>> For the refined keybindings, Spacemacs can provide a good starting point.
>> 
> This is interesting. (of course spacemacs not in evil mode).

Yeah, of course evil’s keybindings can’t be a starting point; I was saying that spacemacs classify most keybindings in a pretty consistent way; we can be inspired by that.

> I will just tell an idea I had (before knowing how things work in emacs,
> I actually thought it was implemented in that way)
> 
> If we create a single file with the full list of general bindings as
> variables for all the main modes like
> 
> ;; Basics
> 
> C-a begin-line-binding
> C-e end-line-binding C-b go-backward-line-binding C-d delete-forward-binding
> ...
> 
> ;; Prog mode
> C-c C-c whatever-you-decide-binding
> 
> ;; Minibuffer mode
> bla bla bla
> ...
> 
> And so on.
> 
> All the derived modes can be changed to use the variables instead of
> hard-code the bindings in multiple files.
> 
> this:
> (define-key c-mode-base-map "\C-d" 'c-delete-forward)
> will be:
> (define-key c-mode-base-map delete-forward-binding 'c-delete-forward)
> 
> This is similar to use remap, but with some advantages:
> 
> 1) The bindings will be organized in a single file and the collisions
> will be exposed easily. And fixed for all of them.
> 
> 2) In case a mode adds a new functionality and it needs a binding, it t
> in the global map, it will be clear if some others already have a
> similar one and will bind to the same using the same criteria. If the
> binding has never been used, it will be added to the list for future
> modes that wants the same functionality.
> 
> 3) Old users will be minimally affected because the starting criteria
> are the actual bindings.
> 
> 4) Implementation of test modes (like ergoemacs or evil-mode) and future
> modifications will require less effort and the final experience will be
> less hacky than now..
> 
> 5) If a user personalize a binding (changing the value of one of these
> variables) the changes will apply to all the packages and modes she
> uses in his section.
> 
> 6) If the user wants to use remap or hard-code the binding (as now) the
> actual behavior will be unchanged.
> 
> 7) Packages like elpy, irony, or company will also use the "binding-list",
> so the user will only learn the most specific bindings (if any) so
> external packages will be standardized with minimal effort.
> 
> 8) Future bindings changes that the community agree will not require
> changes in the files of all the modes. With the potential side effects.

This, at least for me, feels like a fantastic idea :-)
IMHO the point of this is that we can now have a list of functions that are common between modes; which will help users having a coherent experience and provides package authors a minimum start point to provide a pretty-good experience.

> I know that proposing this here is a crazy idea, but the intention is
> not to start a crazy war again, If it is crazy (for technical reasons)
> just ignore it. Actually I just mention it because it seems to me like
> something extremely simple-advantageous and maintainable. With minimal
> affections to the user space.
> 
> But remember that we have a limited number of bindings so managing them
> better is important if we want to find place for new functionalities.
> 
> In any case I will leave this threads as it is not going anywhere (as
> usual with this topics) and what is obvious for me seems to be heretic
> for the rest. But you are free to consider the idea if it has anything
> useful for emacs..
> 
> 
>> 
>>> When I use a long M-x sequence, a shortcut suggestion appears. It disappears before I can catch it. Can it stay for 30 seconds? Can there be an instant interactive override to set it whatever you like?
>> 
>> I would like a semi-AI that suggests interactive functions based on key presses or actions the user performs... `You can use C-e (goto-end-line) to perform 12 keystokes you performed.'
>> Saying about discoverability, I would like a context-sensitive right-click mouse menu, something like Microsoft Office. Most newcomers are familiar with finding functionality with the mouse; and it isn’t intuitive to find new keybindings/functions that Emacs provide to boost productivity. (Actually, that’s one of my problems; how should I find new functions...?)
>> 
>>> Evolutionary programming of popular custom keybindings collected at upstream and put thru obstacle course competition is one way of composing a spring release.
>>> 
>>> --
>>> © 2019 Van L
>>> gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
>>>                            "The interface is a nightmare." - Brendan Schaub

Re: Is Elisp really that slow?

[Stefan Huchler]

Óscar Fuentes <ofv@wanadoo.es> writes:

> Why those modes that you take as models for C-c C-c use precisely that
> binding for sending text to an inferior process? Because it is the most
> used action and, hence, it is natural to use a keybinding that is fast
> and easy to type.
>
> That's the same reason why Org-mode uses C-c C-c for its most frequent
> actions, although you pretend to remove that convenience because
> "consistency"... when in fact you'll be reducing consistency!
>
> Likewise, (E)shell and CC-Mode use C-c C-c because it is easy to relate
> to the associated action *and* because it is easy to type.
>
> Thus, you also would reduce Emacs' ergonomics and mnemonics because of
> your false idea of consistency.
>
> It is important to understand why things are as they are before acting
> as if those who made them were incompetent and the world is depending on
> us to fix their glaring mistakes.

I replaced as example C-c C-c in org mode (edit-special) with Menu + d +
d, the problem is that vanilla Emacs doesn't allow that kind of bindings
as far as I know:

(:map xah-fly-e-keymap
	  ("e" . (lambda () (interactive)
		   (if org-src-mode (org-edit-src-exit)
		     (org-edit-special)))))

But I have no standard keyboard, but the fly-keys can also be accessed
with Space + d + d. (in command mode)

And it's e in the code because I use dvorak. d is easier to hit as c,
and C is not more rememberable then d it's completely randomly chosen.

Also I would argue that most people press C-c C-c with left control
instead of the ergonomic way to use the right control therefor you train
people to use unergonomic keychords.

But the main problem is not the char you use but that you have to hold
this modifiers, which "reduces ergonomics" as compromise I would also
be ok with having some sort of sticky keys? and you press once Ctrl
release then press c c.

But this is not even a optional feature in emacs:
https://www.emacswiki.org/emacs/StickyModifiers

And I would argue it should be the default behaviour, but it should at
least be a opiton. So don't claim it has to do with mnemonics or
ergonomics it has historic reasons not more not less. I would also argue
that consistence is not that important.

But another suggestion for that, if C-c C-c is meant as shortcut for a
"important-function" why not have a binding for "do-important-action" or
"do-major-action" and depending on mode that functions calls the
important function of the mode. So that the user can choose globaly a
keybinding for that and don't has to do that for 80 modes
seperately. and the developer of the mode just somewhere sets which
function is bound to C-c C-c by setting:
(setq mode-important-function 'compile...)

That would give at least so much consistancy that you don't have to
change the keybinding 50 times in 50 modes if you want to change it, and
in some where I was to lazy yet I still have to press C-c C-c and in
some I press my Menu + e + e. 

Re: Is Elisp really that slow?

[Stefan Monnier]

>> Vim is consistent.
> So is Emacs.

In terms of key-bindings, Emacs's scheme is not as regular as VI.

While we can argue that the various VI emulators are just trying to
provide for those users who like VI, the number of other "alternative
set of keybindings" (god-mode and several others) out there shows that
there's a need for something else.

I think one of the main differences (besides the fact that it's modal,
obviously) is that VI has prefix commands like `d` which Emacs lacks.
That helps make things regular/orthogonal.

`other-frame-window` is one package that tries to add such a prefix
command (actually a pair of such) to Emacs and it's not trivial to
implement.  It'd be good to extend Emacs to provide better support
for that.


        Stefan

Re: Is Elisp really that slow?

[Óscar Fuentes]

Stefan Huchler <stefan.huchler@mail.de> writes:

> I replaced as example C-c C-c in org mode (edit-special) with Menu + d +
> d, the problem is that vanilla Emacs doesn't allow that kind of bindings
> as far as I know:
>
> (:map xah-fly-e-keymap
> 	  ("e" . (lambda () (interactive)
> 		   (if org-src-mode (org-edit-src-exit)
> 		     (org-edit-special)))))
>
> But I have no standard keyboard, but the fly-keys can also be accessed
> with Space + d + d. (in command mode)
>
> And it's e in the code because I use dvorak. d is easier to hit as c,
> and C is not more rememberable then d it's completely randomly chosen.
>
> Also I would argue that most people press C-c C-c with left control
> instead of the ergonomic way to use the right control therefor you train
> people to use unergonomic keychords.

I'm afraid that this is hard to fix from Emacs. People (myself included)
tend to chord in unhealthy ways. The problem is similar to postural
injuries: you feel comfortable but you are harming yourself on the long
term. The real solution is to train people on ergonomics. Admitedly,
Emacs is not a model of good practices on this regards.

> But the main problem is not the char you use but that you have to hold
> this modifiers, which "reduces ergonomics" as compromise I would also
> be ok with having some sort of sticky keys? and you press once Ctrl
> release then press c c.

I'll argue that holding "C-c C-c" is more ergonomic that "C-c c". As for
sticky keys, maybe you can activate them with your OS/desktop
environment?

> But this is not even a optional feature in emacs:
> https://www.emacswiki.org/emacs/StickyModifiers
>
> And I would argue it should be the default behaviour, but it should at
> least be a opiton. So don't claim it has to do with mnemonics or
> ergonomics it has historic reasons not more not less. I would also argue
> that consistence is not that important.

I never claimed that C-c C-c was chosen because it is ergonomic on a
global sense, but because is ergonomic given the imposed constraints
which require the C-c prefix for major modes.

> But another suggestion for that, if C-c C-c is meant as shortcut for a
> "important-function" why not have a binding for "do-important-action" or
> "do-major-action" and depending on mode that functions calls the
> important function of the mode. So that the user can choose globaly a
> keybinding for that and don't has to do that for 80 modes
> seperately. and the developer of the mode just somewhere sets which
> function is bound to C-c C-c by setting:
> (setq mode-important-function 'compile...)
>
> That would give at least so much consistancy that you don't have to
> change the keybinding 50 times in 50 modes if you want to change it, and
> in some where I was to lazy yet I still have to press C-c C-c and in
> some I press my Menu + e + e. 

This is one of those ideas that sounds good when proposed but are hard
to implement. You will face obstacles as, for instance, the fact that
key bindings are mentioned on the manual files as well all over the
websphere on questions and answers, etc. Then you have the problem of
collisions, etc.

This is an area that I'm interested on and, on the past, I proposed some
steps towards better discoverability and ergonomy and volunteered to do
the boring, lengthy, clerical part of the job, but no Emacs hacker was
interested enough to do the technical part.

I also pondered similar ideas to what others proposed on this thread
about some sort of "semantic" UI but it is a very hard problem, moreover
if you try to implement something solid that could be defended with
rigorous arguments instead of something that others will view as a
collection of personal quirks.

Re: Is Elisp really that slow?

[조성빈]

> 2019. 5. 20. 오전 3:35, Stefan Monnier <monnier@iro.umontreal.ca> 작성:
> 
>>> Vim is consistent.
>> So is Emacs.
> 
> In terms of key-bindings, Emacs's scheme is not as regular as VI.
> 
> While we can argue that the various VI emulators are just trying to
> provide for those users who like VI, the number of other "alternative
> set of keybindings" (god-mode and several others) out there shows that
> there's a need for something else.
> 
> I think one of the main differences (besides the fact that it's modal,
> obviously) is that VI has prefix commands like `d` which Emacs lacks.
> That helps make things regular/orthogonal.

The fact that vim allows composing commands make vim ‘feel’ consistent (and which is a good thing) because vim doesn’t have to rely on ad-hoc conventions.
Vim changes the convention to the ‘text object’ concept, which reduces the number of keybindings and make different keys have no (or little) relationship between each other, which means the key don’t have to satisfy a certain condition to feel ‘consistent’.
I’m actually using [composable.el](https://github.com/paldepind/composable.el) currently, and also found [objed](https://github.com/clemera/objed) while surfing the web yesterday.
Would it be hard to have composable bindings as an emacs default keybinding scheme?

> `other-frame-window` is one package that tries to add such a prefix
> command (actually a pair of such) to Emacs and it's not trivial to
> implement.  It'd be good to extend Emacs to provide better support
> for that.
> 
> 
>        Stefan
> 
> 

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1026 bytes --]

On Sun, May 19, 2019 at 08:40:23PM +0200, Óscar Fuentes wrote:
> Stefan Huchler <stefan.huchler@mail.de> writes:

[...]

> > Also I would argue that most people press C-c C-c with left control
> > instead of the ergonomic way to use the right control therefor you train
> > people to use unergonomic keychords.
> 
> I'm afraid that this is hard to fix from Emacs. People (myself included)
> tend to chord in unhealthy ways. The problem is similar to postural
> injuries: you feel comfortable but you are harming yourself on the long
> term. The real solution is to train people on ergonomics. Admitedly,
> Emacs is not a model of good practices on this regards.

I think one of the biggest "failures" in this monster thread is, IMHO,
the failure to realize that different people are... different.

I keep reading things like "ergonomic", "consistent", etc. as if the
authors had the ultimate yardstick for that.

Go tell a pro violinist that the way she holds her violin is unergonomic.

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 710 bytes --]

On Sun, May 19, 2019 at 02:35:17PM -0400, Stefan Monnier wrote:
> >> Vim is consistent.
> > So is Emacs.
> 
> In terms of key-bindings, Emacs's scheme is not as regular as VI.

Dunno. For me, it's "regular enough". And I came from ten years of
first vi, then elvis, then vim.

I think there are difficult tradeoffs to take, and Emacs is taking
them pretty well, overall.

To arrive at "better" (default!) key bindings we'd have first to
develop an idea which is "consensual" enough to serve as that, then
carefully move in that direction (kind of what *you* have been doing
all that years, so thank you for that :-)

So I think this would be more a social task at first.

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Emanuel Berg]

Ergus wrote:

> So in the context of your example, our tire
> is 55 (incompatible with everything else by
> far), but also it works in a special kind of
> bike that needs a 50 tire on Mondays and
> Tuesdays, but is the user who needs to do the
> changes. We need to maintain our own bikes,
> wheels, the material, and as we don't have
> practical arguments about why we keep that
> system in many cases except that it is
> because it is compatible with the previous
> bikes we produced the last 40 years and the
> old users are use to them and they already
> know how to fix them.
>
> But the new users can find spare parts for
> the other systems anywhere, and specialized
> personal in the other technologies, and they
> don't need to be alert about the day of the
> week... So for him is an obvious choice.
> Our tire is better, because it is the only
> tire that can change size 2 times a week, but
> for him it does not represent an advantage.

I don't recognize this analogy. I think Emacs
is consistent enough. Sure, I did a lot of
configuration but I don't recall doing much of
that because Emacs was/is inconsistent in terms
of its interface. Obviously, if you look for
inconsistencies, you'll find them. How can you
not? Just think about how many files, modes,
packages, and so on Emacs consists of, and how
many different people worked on it for decades.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

RE: Is Elisp really that slow?

[Drew Adams]

> I think one of the biggest "failures" in this monster thread is, IMHO,
> the failure to realize that different people are... different.
> 
> I keep reading things like "ergonomic", "consistent", etc. as if the
> authors had the ultimate yardstick for that.
> 
> Go tell a pro violinist that the way she holds her violin is unergonomic.

+1

Re: Is Elisp really that slow?

[Óscar Fuentes]

<tomas@tuxteam.de> writes:

> I think one of the biggest "failures" in this monster thread is, IMHO,
> the failure to realize that different people are... different.
>
> I keep reading things like "ergonomic", "consistent", etc. as if the
> authors had the ultimate yardstick for that.
>
> Go tell a pro violinist that the way she holds her violin is unergonomic.

Musculoskeletal & nervous injuries are common among musicians, pro and
not-pro. Not so common as among keyboard users, because the former, as
part of their learning, are instructed about correct postures and
exercises.

Ergonomics is a well established discipline. Not exact, but mature
enough to say with a very high level of confidence that certain
practices are harmful.

Re: Is Elisp really that slow?

[Van L]

Ergus writes:

> On Sun, May 19, 2019 at 07:35:58PM +0900, 조성빈 wrote:
>>
>>2019. 5. 19. 오후 5:16, Van L 작성:
>>
>>> Emanuel Berg writes:
>>>
>>For the refined keybindings, Spacemacs can provide a good starting point.
>>
>>> [snip]
>>
>> Most
>> newcomers are familiar with finding functionality with the mouse;
>> and it isn’t intuitive to find new keybindings/functions that Emacs
>> provide to boost productivity. (Actually, that’s one of my problems;
>> how should I find new functions...?)
>>
> Ohh, Have you seen/used which-key? counsel/ivy list them, but which-key
> does the same with the bindings. Spacemacs uses it.

Thank you for suggesting the which-key-mode. Awesome.

A possible starting point for transitioning from the mouse to boosting
keyboard productivity is for the right third of the keyboard under the
delete-key to do arrow-key navigation and block/line/page/def jumping
through text by hyper or super key combos on the eight keys around RET
and SHIFT. Sending the right hand all the way to the bottom right corner
for the arrow keys requires an entire hand move, again, to get back.

When Elon Musk ships the Neural Lace Link (NLL) you can be the Dilbert CEO
and think the Strategic Consultant to move the mouse to have things done
for you.

-- 
© 2019 Van L
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
    "Once you get to that elite level, we are all the same." - Pat Farmer

Re: Is Elisp really that slow?

[Van L]

Drew Adams writes:

>> When I use a long M-x sequence, a shortcut suggestion appears. It disappears
>> before I can catch it. Can it stay for 30 seconds? Can there be an instant
>> interactive override to set it whatever you like?
>
> `C-h v suggest-key-bindings':
>
>  The value can be a length of time to show the message for.

>  If the value is non-nil and not a number, we wait 2 seconds.
>
>  You can customize this variable.

Thank you. I didn't know the sequence to give C-h v.

The 1 second wait for which-key's idle feels way longer than the 2
seconds here for this. I guess this is due to the resource load on task
focus when you are going from A to B and are surprised by
suggest-key-bindings's advertisement.

-- 
© 2019 Van L
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
    "Once you get to that elite level, we are all the same." - Pat Farmer

Re: Is Elisp really that slow?

[Van L]

<tomas@tuxteam.de> writes:

> Go tell a pro violinist that the way she holds her violin is unergonomic.

And, follow-up with a leading question on the ergonomy of the erhu.

-- 
© 2019 Van L
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
    "Once you get to that elite level, we are all the same." - Pat Farmer

Re: Is Elisp really that slow?

[Emanuel Berg]

Ergus wrote:

> In 1992 there were less alternatives around
> and most of them were uglier, notably more
> limited or most expensive. But in 2019 there
> are cheaper, prettier and free alternatives.
> They are not as powerful as emacs, but they
> are more ergonomic

I didn't start using Emacs in 1992 - you did?!
- but rather around 2008-10-17, that is
10y 7m 5d, or if you'd like, 3869 days
ago :) [1]

Emacs is free, and free of charge, and the
prettiest software ever. [2]

As for ergonomics the configuration
possibilities makes Emacs the most ergonomic
software as well. The mouse-free style of most
Emacs' hackers is also more ergonomic than the
"click and play" of much other software.

Emacs would be even more ergonomic
out-of-the-box if the default background color
was black and the cursor didn't blink by
default. (At least in the GUI Emacs, that _is_
an Emacs thing, right? In the Linux VTs, one
has to disable that outside of Emacs.)


[1] line 43 @ https://dataswamp.org/~incal/conf/.zsh/time
[2] https://dataswamp.org/~incal/figures/emacs/emacs-cpp.png

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg]

tomas wrote:

> Oh, I see. Emanuel: in the meantime, here's
> the online ref:
>
>   https://www.gnu.org/software/emacs/manual/html_node/elisp/Writing-Dynamic-Modules.html#Writing-Dynamic-Modules

Thanks, I have been reading them for a couple
of days now at a snail's pace. Perhaps it is
time for me to change style a bit and not just
"do" things but try to understand them as
well...

Also, in [1] it says

    Most of the functions described below
    became available in Emacs 25, the first
    Emacs release that supported
    dynamic modules.

And Emacs 25 is what I use! Amazing - this must
be the first time ever I and the time around me
keep an even pace!


[1] https://www.gnu.org/software/emacs/manual/html_node/elisp/Module-Values.html

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg]

Eli Zaretskii wrote:

>> [...] what is this "E8" notation?
>
> Appendix E, chapter 8.

This notation is better:

    (info "(elisp) Integer Basics")

Here is a little helper, I have it bound to
"s", as in source.

(defun info-copy-current-node-name-elisp ()
  "E.g. kill-ring this line: (info \"(gnus) Mail Source Specifiers\")"
  (interactive)
  (Info-copy-current-node-name 0) ) ; [1]


[1] line 18 @ https://dataswamp.org/~incal/emacs-init/my-info.el

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

RE: Is Elisp really that slow?

[Drew Adams]

> Here is a little helper, I have it bound to
> "s", as in source.
> (defun info-copy-current-node-name-elisp ()
>   "..."
>   (interactive)
>   (Info-copy-current-node-name 0))

Good one.

Somewhat related: This command is available in
Info+ (bound in Info to `G').  I use it to
quickly get the URL to paste into web pages etc.

(defun Info-goto-node-web (node &optional flip-new-win)
  "Use `browse-url' to go to Info node NODE using a Web browser.
With a prefix arg, reverse the effect of option
option`browse-url-new-window-flag'.

NODE is the name of a node in the GNU Emacs or Elisp manual.
Alternatively, NODE can have the form (MANUAL)NODE, where MANUAL is
\"emacs\" or \"elisp\" and NODE is the name of the node in that
manual.  Empty NODE in (MANUAL) defaults to the `Top' node."
  (interactive
   (list (Info-read-node-name "Go to node: " Info-current-node)
         current-prefix-arg))
  (require 'browse-url)
  (unless Info-current-file
    (error "This command must be invoked from Info"))
  (browse-url (Info-url-for-node node) 
              (list (if flip-new-win
                        (not browse-url-new-window-flag)
                      browse-url-new-window-flag))))

https://www.emacswiki.org/emacs/InfoPlus

Re: Is Elisp really that slow?

[Eli Zaretskii]

> From: Emanuel Berg <moasenwood@zoho.eu>
> Date: Thu, 23 May 2019 16:28:35 +0200
> 
> Eli Zaretskii wrote:
> 
> >> [...] what is this "E8" notation?
> >
> > Appendix E, chapter 8.
> 
> This notation is better:
> 
>     (info "(elisp) Integer Basics")

It's a question of taste, and other people can legitimately have
different tastes.  For example, if someone prefers to read the manual
in PDF format, your notation is useless.

Re: Is Elisp really that slow?

[Robert Thorpe]

Stefan Huchler <stefan.huchler@mail.de> writes:

> But the main problem is not the char you use but that you have to hold
> this modifiers, which "reduces ergonomics" as compromise I would also
> be ok with having some sort of sticky keys? and you press once Ctrl
> release then press c c.
>
> But this is not even a optional feature in emacs:
> https://www.emacswiki.org/emacs/StickyModifiers

In Emacs sticky keys aren't so simple.  The one you give is a fairly
good example.

The keybindings of the form "C-c C-something" are used by the current
mode.  So, C-mode enables "C-c C-c", for example.  The bindings of the
form "C-c something" are for the user to define.  Where the "something"
isn't control, of course.

How is this to work with sticky key?  Does the user have to press ctrl
then c and then ctrl again to switch off the stickiness before pressing
c the second time?  A timeout could also be used, but that brings it's
own problems.

BR,
Robert Thorpe

Re: Is Elisp really that slow?

[Robert Thorpe]

Emanuel Berg <moasenwood@zoho.eu> writes:
> I don't consider myself an Emacs expert - far
> from it. But I've been here for 10+ years, so
> I'm happy with my Emacs and my skill level.
>
> But this place still doesn't feel like home!
> That is strange.
>
> I'm not sure your analysis is correct why this
> is tho, but heck, it might be, or part of it
> at least.

I think is mainly because of the way you use this mailing list.

You seem to think of it as something more than a help mailing list.  I
think lots of other people think of it strictly as a help list.

I expect that some of your mails would work better elsewhere.  Perhaps
on a blog or on Reddit.

BR,
Robert Thorpe

Re: Is Elisp really that slow?

[Stefan Huchler]

Robert Thorpe <rt@robertthorpeconsulting.com> writes:

> The keybindings of the form "C-c C-something" are used by the current
> mode.  So, C-mode enables "C-c C-c", for example.  The bindings of the
> form "C-c something" are for the user to define.  Where the "something"
> isn't control, of course.
>
> How is this to work with sticky key?  Does the user have to press ctrl
> then c and then ctrl again to switch off the stickiness before pressing
> c the second time?  A timeout could also be used, but that brings it's
> own problems.

No you would change it to Ctrl + c + c.

You only need the timeout if you use both my ideas I mentioned at the
same time. So CUA mode uses a timeout, but it does not use sticky keys.

So if you press C-c it runs a small timeout and if you don't press
something else that like again C-c it uses it as copy command like
notepad/gedit would do.

But if you don't need this double use of C-c you could do it without a
timeout. and Just do the Ctrl + c + c and skip the second Ctrl.

You barely find a mode that has both defined. C-c c and C-c C-c.

But that is when you limit the bindings to C-c as start. so I güss here
in Message mode C-c C-f s stands for change subject and append and C-c
C-f C-s stands for move to subject.

Both would be come down to
Ctrl + c + f + s in my suggestion.

So you would need to press the Ctrl again but that would be still more
ergonomic than keeping the key pressed I think, because holding and
streating the finger is still less ergonomic.

My solution for C-c C-c in that mode and in most is to bind it to:
<menu> m	message-send-and-exit

But I have no standard keyborad :D

But we do it with Shift the same way, if I would write TEST I press left
shift + t + rshift + e + lshift + s + lshit + t, maybe I hold it for the
last 2 letters but I use always the one on the oposite site, because
that is more ergonomic.

And now ask yourself do you ergonomically correctly press left control +
c or right control, most including me use left, which is not ergonomic,
but it's faster and deep in our muscle memory.

But to make it short with that minimal backward compatible change yes
you would press ctrl again and I don't see a need for a timeout.

The only problem would be that you have to implement it for every os and
that might not be totally trivial. xah-fly-keys does that with menu or
whatever key you define but menu is no modifier key it's harder to do
with modifier keys because the US normaly first waits after you press a
modifier what follows and sents that compiled thing as keysignal to the
process, as far as I know.

So describe key doesn't even register that I press Ctrl till I press a
non modifier key. But if xemacs could do it emacs should be able, too.

And I mean gnome can do it too. and Emacs is also only a desktop
environment (exwm) :D

Re: Is Elisp really that slow?

[Van L]

Emanuel Berg writes:
> Amazing - this must be the
> first time ever I and the
> time around me keep an
> even pace!

Welcome to the simulation.

-- 
© 2019 Van L
gpg using EEF2 37E9 3840 0D5D 9183  251E 9830 384E 9683 B835
          "I don't believe any anything that the media say." - Ali Abdelaziz

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[Xavier Maillard]

> From: Emanuel Berg <moasenwood@zoho.eu>
> Date: Sun, 12 May 2019 01:09:09 +0200
> 
> Now, while that would be interesting to do,
> I think it's pretty clear Elisp would benefit
> from more speed. Maybe it is not slow for its
> purposes, but it ain't fast either, sure is
> my impression.

I have read pretty much all this thread and as Tomas said it quite
well, there is more than speed. Speaking for myself, speed is not of
any concern here. It is fast enough for everything. Other aspects are
of much priority for me: hackability is one of them, accessibility,
portability, stability, etc.

What would you do with more speed that you do not do today ?

-- 
Xavier Maillard                      
e/j:xavier@maillard.im                w:www.maillard.im
m: 06 49 60 48 56

GPG:                           9983 DCA1 1FAC 8DA7 653A
                               F9AA BA49 09B7 8F04 DE1B

Re: Is Elisp really that slow?

[Robert Thorpe]

Stefan Huchler <stefan.huchler@mail.de> writes:
>
> You barely find a mode that has both defined. C-c c and C-c C-c.

You don't understand my point.  You should not be able to find *any*
mode that defines C-c c.  That's because C-c c is in the *users* range
of keybindings.

All keybindings of the form "C-c something" are reserved for setting by
the user.  Let's say you have a command that doesn't have a key.  You
want to assign it to a key.  You're supposed to use a keybinding
beginning with C-c and then an alphabetic key afterwards (e.g. C-c a C-c
w C-c p, etc).  It's the same if you write your own elisp command and
want to assign that to a key.  Many users assign lots of of their own
keys.

See: (info "(elisp) Key Binding Conventions").

The problem with your plan is that the effect it would have on this.
Either the user key-range would disappear or it would become very clumsy
to use.  I think the ability to define your own keys is an essential
feature of Emacs.

BR,
Robert Thorpe

Re: Is Elisp really that slow?

[Stefan Huchler]

Robert Thorpe <rt@robertthorpeconsulting.com> writes:

> Stefan Huchler <stefan.huchler@mail.de> writes:
>>
>> You barely find a mode that has both defined. C-c c and C-c C-c.
>
> You don't understand my point.  You should not be able to find *any*
> mode that defines C-c c.  That's because C-c c is in the *users* range
> of keybindings.

Ahh I thought a while it was meant everything that starts with C-c which
would also include C-c C-c as example that explains much :D.


> Many users assign lots of of their own
> keys.

Yes I do that too, but not limited to C-c *. But I understand your
problem now with Sticky keys, well then you would have to press C again
if you want to use C-c C-c as example, Problem solved.

> The problem with your plan is that the effect it would have on this.
> Either the user key-range would disappear or it would become very clumsy
> to use.  I think the ability to define your own keys is an essential
> feature of Emacs.

Of course that's important I just don't see how it's clumsy? Let's say
you want or have defined C-c c as a action just do C + c c instead. And
if you want to use C-c C-c you press C + c + C + c.

Nothing changes except you don't have to hold C, you could even have
both paralell, if you press c while you pressed C it's the normal C-c if
you release it in between it counts as the same, but you need to release
it and press it again if you want the sticky C-c C-c or you hold it
through both C-c-c.

I see no real conflict, the only "Problem" I see is that you can't press
C once then press c twice and expect C-c C-c. The stickyness only persists
for 1 letter afterwards.

Does that make sense?

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Eli Zaretskii wrote:

>> We are far from becoming a toxic community,
>> but new/different ideas are not really very
>> welcome and sometimes the arguments are like
>> "it has always been like that", "old users
>> don't want that change".
>
> Really? You've just went through a process of
> proposing and implementing a new feature --
> did you feel your idea was "not really
> welcome"?
>
>> The conservative attitude in emacs
>> development group (apart from the technical
>> obstacles like the workflow and the
>> paperwork and so on) gives the idea of
>> a closed environment (that actually it is
>> from the development point of view because
>> we are like frozen in the past) where only
>> very experts are welcome (that's the vision
>> people have from outside) so very few melpa
>> developers are really interested to face all
>> those issues to contribute.
>
> Is this your experience from implementing the
> fill-column indicator? If so, how do you
> explain that you, as a relative non-expert,
> were welcome in that case?

I think the tone of this very reply confirms he
is right.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Stefan Monnier wrote:

>> So from a personal POV, I would be
>> interesting/fun to write some of my Elisp
>> into C, and if I get good enough at it,
>> maybe I could even do it for the project.
>
> FWIW, compiling existing Elisp to C code is
> not really interesting in the sense that if
> it exists as Elisp code it's usually because
> it's not very speed-sensitive.

Yeah, I suspected that much. Still, some Elisp
I'm sure I can find somewhere that is
worth translating?

If not, I'll just translate some of my own
stuff. Here, I wonder how to determine what to
optimize, tho? Obviously it should be something
simple that is used often and can benefit from
a speed-up. Is there an automated way to answer
any of those three questions? Or do you
actually have to use your _brain_ and not just
your fingers to do computer work?

Or maybe I'll write something all new in C...

> What is more interesting is to take existing
> C code and see if it can be rewritten into
> a kind of Elisp (or other language that has
> similar dynamic properties, most importantly
> the ability to replace code at run-time) that
> can be compiled back to efficient code.

That's beyond my capabilities at the moment,
I'm afraid...

> After all, the main problem with the part of
> Emacs that's written in C is that you can't
> change it from your ~/.emacs, contrary to all
> the Elisp code.

Is _that_ the problem with Emacs' C? I thought
there wasn't any problem with Emacs' C, we just
needed more to make the program faster?
For simple, fundamental stuff that people don't
configure anyway because of their
low-level-of-complexity nature? You're saying
all of that is already done?

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Eli Zaretskii wrote:

> Emacs is an old program, with users who stay
> with it for 25 and more years. Changing old
> defaults or making other incompatible changes
> definitely annoys at least some of those
> users, especially since quite a few people
> don't track the development, and don't see
> the changes until years later, sometimes even
> after skipping several releases in-between.
> This isn't theory, this is our (and my
> personal) actual experience from watching
> Emacs development. When someone comes up and
> asks why we made a certain change in how
> Emacs worked for decades, we should have
> a good reason, and if we don't, we shouldn't
> make the change "just because we can".
> That's the price of maintaining and
> developing an old and stable program.
>
> Personally, I wish people would invest much
> more time and efforts in adding new features,
> and would stop futzing with existing
> features. For starters, the former is much
> more useful for our users than the latter.
> Also, I see many times how a change in
> existing code to make some minor improvement
> introduces bugs, which sometimes are more
> significant than the "fixed" problem -- this
> is expected in a complex stable program, and
> is a clear sign that changes of existing code
> long since entered the area which engineers
> call "the limit cycle" -- oscillations that
> don't converge, i.e. the overall code quality
> is quasi-constant. IOW, we are wasting our
> own resources for little or no gain.

This is all true, not just with Emacs but
with everything.

Fiddle with details up to a point, then stop.

Obey tradition. What has worked will work and
there is no reason to fix it for the sake of
it, for esthetical reasons or to make it
more modern. If it ain't broke etc.

If you want to be a radical, do new stuff and
add them next to the old stuff. Then have
users, including yourself, decide what to use
and when.

This is so much more the right and easy thing
to do in a computer program!

While in a bike repair shop for example, there
are almost always practical considerations that
are much worse to deal with. If you acquire
a new machine or piece of equipment (a bench
grinder, truing stand, or whatever), you can't
just put it at the optimal place because there
are already a bunch of other stuff there!
The new stuff may be better in 9/10 cases, but
what about the 10th case? Also, the old stuff,
everyone knows how to use, so for some time, it
will still be better in practice!

With a computer program, you don't have to deal
with such issues. Just add more great stuff
whenever possible, w/o removing any of the old
that 1) works and 2) is useful!

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Eli Zaretskii]

> Date: Sat, 25 May 2019 06:42:06 +0200
> From: Emanuel Berg via help-gnu-emacs <help-gnu-emacs@gnu.org>
> 
> I think the tone of this very reply confirms he
> is right.

What "tone" is that?  Those were simple questions, asked in good
faith, using polite wording.  I quite clearly disagree with the
opinion that "new/different ideas are not really very welcome", but
disagreement should not be taken as bad attitude, especially when
backed up by facts.  Disagreement and its expressions are legitimate
in discussions, and serve an important purpose of helping to approach
the truth by mentioning different aspects and POVs.

On the contrary, claims such as yours are IMO harming the discussions
because they make people think twice before posting their legitimate
opinions, lest they will be accused of having a "tone".

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Ergus wrote:

> 2) Many new features and functionalities
> (pure editing oriented) have been
> introduced/proved/improved in other editors,
> and they have proven to be useful (move line,
> initial configuration, associative bindings,
> specific key combinations like in cua-mode,
> undo/redo and many others).

I don't know what "move line" or "associative
bindings" are ... but if it is just about
editing and keyboard-to-function behavior, I'm
pretty sure we have it.

As for `cua-mode', even I have that, and
I don't want it, so it must be there to
begin with.

But I do agree the initial config is very
unsexy. Especially the GUI Emacs looks awful
out of the box, to be honest. Obviously you can
get rid of all that stuff easily enough, then
get a bright-on-black theme with a more radical
syntax highlighting, have the cursor stop
blinking, and a couple of other things I don't
remember right now. That's a good start and
should greatly improve the first impression.
I don't know why it isn't like this to
begin with.

> 3) The development is not focused in the
> first thing that a user needs when she opens
> emacs: provide the most comfortable and
> useful TEXT EDITOR.

Emacs is an interface/tool to operate the
_entire computer_. Stop thinking of it as
a text editor only. It isn't true - far from it
- so I don't know why you do it. And it will
never happen, no matter what you or anyone else
say in CAPITAL LETTERS. The only way it will
happen if you fork it and remove tons and tons
of useful code. You can do that, of course.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[조성빈 via help-gnu-emacs]

Oops, I sent this as a personal mail first :-(
Resent...

2019. 5. 25. 오후 5:22, Emanuel Berg via help-gnu-emacs <help-gnu-emacs@gnu.org> 작성:

> Ergus wrote:
> 
>> 2) Many new features and functionalities
>> (pure editing oriented) have been
>> introduced/proved/improved in other editors,
>> and they have proven to be useful (move line,
>> initial configuration, associative bindings,
>> specific key combinations like in cua-mode,
>> undo/redo and many others).
> 
> I don't know what "move line" or "associative
> bindings" are ... but if it is just about
> editing and keyboard-to-function behavior, I'm
> pretty sure we have it.
> 
> As for `cua-mode', even I have that, and
> I don't want it, so it must be there to
> begin with.
> 
> But I do agree the initial config is very
> unsexy. Especially the GUI Emacs looks awful
> out of the box, to be honest. Obviously you can
> get rid of all that stuff easily enough, then
> get a bright-on-black theme with a more radical
> syntax highlighting, have the cursor stop
> blinking, and a couple of other things I don't
> remember right now. That's a good start and
> should greatly improve the first impression.
> I don't know why it isn't like this to
> begin with.

This is so true... Why is the defaults so bad?̊̈
Just having a default theme will be much better...

>> 3) The development is not focused in the
>> first thing that a user needs when she opens
>> emacs: provide the most comfortable and
>> useful TEXT EDITOR.
> 
> Emacs is an interface/tool to operate the
> _entire computer_. Stop thinking of it as
> a text editor only. It isn't true - far from it
> - so I don't know why you do it. And it will
> never happen, no matter what you or anyone else
> say in CAPITAL LETTERS. The only way it will
> happen if you fork it and remove tons and tons
> of useful code. You can do that, of course.

I would say, Emacs is an tool that abstracts operating a computer into text editing; which makes text-editing capabilities one of the most important part of Emacs. Text editing is *the* interface of the tool, Emacs.

> -- 
> underground experts united
> http://user.it.uu.se/~embe8573
> https://dataswamp.org/~incal
> 
> 

Re: Is Elisp really that slow?

[Robert Thorpe]

Stefan Huchler <stefan.huchler@mail.de> writes:
>
> Does that make sense?

I think it does.  So, there will be no timeout.  Instead Ctrl is sticky
until the next key is pressed, but not after that.  I could see that
working.

BR,
Robert Thorpe

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Óscar Fuentes wrote:

> The really big problem is that Emacs no
> longer compete on areas were it used to bring
> the largest gains. Other editors largely
> surpassed Emacs' gains while requiring
> less effort.

:)

Big problem!

Perhaps time to be content that Emacs appeals
to certain users, and will do so even in
the future.

I guess in particular those who are appealed to
Lisp and those who look for a one-program
interface to the whole computer, and those who
enjoy and have time to do endless tweaking with
their software and want to be able to do so in
an easy and unified way...

And if it doesn't, well nothing lasts forever.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Ergus wrote:

> I agree 50% here. One thing is to use
> specific emacs functionalities tricks, but
> another one is when you don't know how to
> start, because the workflow is orthogonal to
> 90% of everything else, or you look for the
> word copy and paste in the tutorial you only
> find kill and yank.

Yes, we have some terminology of our own.
That's nothing to be ashamed of. Perhaps one
can explain it somewhere if this hasn't
happened already, like a "Feel confused by
Emacs terminology? Here is a crib for YOU" type
of document or part of the manual.

But obey tradition is all one can do. It is
what one wants to do. Try to change it - trust
me, it never works, anywhere.

> The first impression is actually the most
> important. And if a user don't see any
> advantage the first 10 minutes, we lost him.

You have again so low esteem of the user.
Why didn't WE quit after 10 minutes? Are we so
much better than everyone else?

> This is exactly the key of all my point.
> We can't continue patching and putting works
> around to this issue and we can't compete and
> win in all the fields we cover, so at least
> we must be centered in one of them, the most
> important one.

We want it ALL. That's the whole idea with
Emacs. We want programming, but also HTML/CSS,
LaTeX and Biblatex, the shell, the file system,
Emacs-w3m, man pages (a pager _and_ a mode to
edit groff), Gnus, ERC, Makefile, and
everything else people are using Emacs for
*every day*.

Emacs hackers all around the world are working
on all of this, and much, much more, every day.
You will never, ever be able to get them to
focus on "the most important" field, whatever
that is.

The diversity (plethora of activities) is the
_strength_, not the/a problem.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

RE: Is Elisp really that slow?

[Drew Adams]

> Yes, we have some terminology of our own.
> That's nothing to be ashamed of. Perhaps one
> can explain it somewhere if this hasn't
> happened already, like a "Feel confused by
> Emacs terminology? Here is a crib for YOU" type
> of document or part of the manual.


https://www.emacswiki.org/emacs/EmacsJargon

https://www.emacswiki.org/emacs/Glossary

https://www.gnu.org/software/emacs/manual/html_node/emacs/Glossary.html

https://www.emacswiki.org/emacs/CategoryGlossary

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Jean-Christophe Helary wrote:

>> Emacs provides some advantages, but they are
>> not apparent until you experience them.
>> That's a problem for people grown on
>> a culture of instant gratification.
>> Emacs appeals to certain type of users who
>> understand that gains require efforts.
>
> I find that comment extremely condescending.

Well, maybe not "extremely", but as I said many
times by now, little faith seems to be put in
people coming to Emacs for the first time.

Actually so little, one might start to suspect
we ourselves are very unhappy with our
current state!

Hint: Just change the initial config so it
looks good (cool)! ~Half the problem with
newcomers is solved right there.

> If "instant gratification" means finding
> a common ground on which one can get started
> right away, then I'm all for it.

It doesn't mean that.

> Considering the state of affairs, emacs seems
> first to appeal to people who want to give
> priority to free software, at the *cost* of
> ease of use.

*laughter*

I can assure you I could barely spell to
free software when I started to use Emacs and
the political aspects were totally irrelevant.

There was no "cost". It was just a cool program
and I wanted to do more and more with it.
And that's exactly what happened.

Why this can't happen to a newcomers, some guy
or girl around the world starting Emacs for the
first time _today_, why this cannot happen to
him or her as well I don't understand.
Please enlighten me! Why can't it happen?
What's the difference?

> Access to free software should never be the
> sole privilege of "users who understand that
> gains require efforts". Quite the opposite.

... :) ?

> Eli earlier clearly identified a number of
> areas where emacs required huge and totally
> undue efforts to get the thing to work as
> expected in the 21st century.

Oh, no! More efforts! And huge at that!
And what will happen after those efforts?
Will we then win? Editor Champions of
the World?

Stop it. Or continue, rather. The journey _is_
the goal!

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Óscar Fuentes wrote:

> From 1985 to 2010 (give or take a few years
> and discounting Java and some other modern
> language) Emacs was the best programmer's
> editor on the "by hackers, for hackers"
> category. I suppose that most current users
> come from that period.

I started using Emacs somewhere around
2011-06-11. Because I knew I did another thing
then, with either nano or Emacs. So either
I stopped using nano then, or I begun using
Emacs then ... (wait, that didn't make
any sense!)

So obviously, it pleases me, with your
give-or-take-a-few-years play, I am included in
the period! Even then, my hacker intuition
was impacable!

Now... "Emacs was the best programmer's editor
on the 'by hackers, for hackers' category" -

1) When did this stop, give or take
   a few years?

2) What editors/IDEs pushed Emacs down
   the ladder?

3) Is it possible to summarize in three or four
   sentences, what they offer that we lack?
   Factorization, and instead of just basic
   modes for a programming language P, tons of
   features and helpers for that particular
   language, is what I've heard so far.
   That doesn't sound so impressive - but it
   might just be, or there's more.

Now two other questions:

1) Are we, or are we not counting Emacs-w3m,
   Gnus, ERC, the shell, [M]ELPA, the
   man pages, the file system (Dired), and
   everything else I've mentioned many times by
   now?

   Or do the other editors provide that as well
   - in different forms, of course?

   They don't, right? So we are actually not
   counting huge parts of Emacs, when we
   compare Emacs to other editors? Parts that
   are actually really beneficial when
   writing programs?

2) Do the other editors support tons of
   languages, like Emacs does, and not just the
   Emacs favorites like Lisp but Cobol, Pascal,
   ML, seekwell, LaTeX, zsh, and so on? As well
   as all the configuration modes?

   Or are they just better, perhaps much
   better, at one or two or three languages,
   e.g. C++ or C#/VB(A)/MS SQL Server?

> Maybe, just maybe, having "kill & yank"
> instead "copy & paste" is not the cause of
> Emacs' lack of appeal to the new generations.

Of course it isn't.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Jean-Christophe Helary wrote:

> Because they don't fit your narrative ?
> I've started trying emacs in the mid 90's and
> the best pro editor I could find then that
> I could make sense of was BBEdit. I'm still
> using it when I have no time to uncover
> emacs' arbitrary idiosyncrasies.

Oh, yeah? Like when?

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Ergus wrote:

> But for new users, lisp looks like ancient
> Cyrillic.

Again this disrespect for the new user. LISP is
from 1958, CL is from 1984 and Elisp is
from 1985. [1]

Anyone here wants to raise their hand "I was
there 1958, and LISP felt just right with the
times!" I was there in 1984 and 1985, but my
interests then weren't programming.

I discovered Lisp much later and it didn't feel
like ancient Cyrillic, or paleo-Etruscan for
that matter. It felt interesting, awesome, and
practical, and that's why I went with it (to
a large extent). Despite never getting rich or
famous, nor a Lisp expert, I don't regret it
for a microsecond.

To paraphrase tomás, that is

$ echo $(( 1 * 10**(-6) )) seconds
9.9999999999999995e-07 seconds

So it is pretty clear for these computations
that you again underestimate people, big time.
LISP (or Lisp) wasn't on everyone's lips,
lispers or not, when I found it!

When I found it thru Emacs, I had never heard
of it! Yet I liked it instantly! I wasn't in
the least intimidated, on the contrary,
I was attracted, just like light is to
a celestial body!

Again, why can't this happen again?
Young people today are smarter and more
dedicated than I was - well, perhaps not, now
that I think about it :) Well, smarter and more
dedicated in a different way, at least.

But in any case, they are smart and dedicated
enough to not be scared away by the ultimate
terror, ancient Lisp!

It doesn't work like that. Young people are
*brave*!

> It is actually a prove that being consistent
> can success, even when not following the
> standards in the rest of the world.

Consistency for large projects are virtually
impossible. And Emacs is not large, it is
enormous. I can't keep my one room bike repair
shop "consistent" for long, and it's just me
and ~10 other guys being active there!
Just forget about it. Do the best you can, yes,
then live with it. Don't be all Asperger about
it. It is a dead end.

Part of success is being organized, yes, like
Amundsen on the pole. But a much bigger part is
being prepared, dedicated, brave, knowledgeable,
and confident. Having your equipment in good
order gives you some advantage, and some
confidence for that matter, but it ain't enough
and never will be. You don't reach the pole by
counting <handlovremmar> (no idea what they are
called in English) - you don't reach to pole by
counting <handlovremmar> and putting them
i orders of size, weight and color. You get to
the pole by trying them on and experiencing
first hand if they are too long, too short,
needs to be of a different material, or are
just right. And if some other guys has it the
other way around, and even has his own name for
them, so be it. Learn his name as well and
let's be off already.


[1] https://dataswamp.org/~incal/COMP-HIST

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Xavier Maillard]

> Date: Mon, 27 May 2019 16:30:05 +0200
> From: Emanuel Berg via help-gnu-emacs <help-gnu-emacs@gnu.org>
> 
> Óscar Fuentes wrote:
> 
> > The really big problem is that Emacs no
> > longer compete on areas were it used to bring
> > the largest gains. Other editors largely
> > surpassed Emacs' gains while requiring
> > less effort.
> 
> :)
> 
> Big problem!

GNU Emacs has no need to be competitive IMO. As I am okay living right
into emacs buffers, I do not need to compare it with competitors.

YMMV.

> I guess in particular those who are appealed to
> Lisp and those who look for a one-program
> interface to the whole computer, and those who
> enjoy and have time to do endless tweaking with
> their software and want to be able to do so in
> an easy and unified way...

I, for years, constantly was tweaking my .emacs for hours and
hours. As to my surprise, I did not get the job done ;) THat was quite
fun but, in the end, that was pretty conter-productive.

My new mantra: resist to the extension appeal ! Limit yourself to the
bare minimum. Surprinsingly, it works.

I doubt powerusers have tons of them installed and worst, I even doubt
their .emacs are huge. That's why having good default and stick with
them is so important and thus, having stable standards is so important
to old men like me ;)

-- 
Xavier Maillard                      
e/j:xavier@maillard.im                w:www.maillard.im
m: 06 52 18 63 43 (old)
m: 06 49 60 48 56 (NEW)

GPG:                           9983 DCA1 1FAC 8DA7 653A
                               F9AA BA49 09B7 8F04 DE1B

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Eli Zaretskii wrote:

> Not much we can do about that. Feel free to
> lobby distros to include Emacs. IMO, if
> something is related to 40-year old decision,
> it is this one.

Hell, I went to Computer School (completed my
first course 2008-10-17, 10y 7m 13d (3877d)
from now. At that point, Emacs was not only
included, there was also an introductory course
to computers which I didn't do because I had no
respect for it, _but_ they handed out a booklet
called "Gula Luntan" (the yellow tome, actually
it wasn't that hefty) but anyway _in that
booklet_ which I read two or three times were
whole sections on Emacs! Now, at the same
system, when I type 'emacs' and hit RET it
isn't to be found. I do

$ find / -name \*emacs\* 2> /dev/null | wc -l

and get nothing! Becuase computation still
hasn't found every-thig!

I do get tons' of Emacs stuff, tho...

> Emacs also works the same on all systems.
> As for "better default colors", that
> debatable at best. It's a matter of taste,
> and psychologically we tend to favor our
> first experience: old habits die hard.

And the correct taste is: 1) non-blinking
cursor, 2) black background, and 2) use all
colors that contrast as much to the background
as to each other as possible.

normal                         bright
   bk  r   g   y  bl   m   c   w  bk    r   g   y  bl   m   c   w
r  0 255   0 240 150 225   0 200 140  255   0 255 175 255 140 255
g  0 100 200 240 150 150 230 200 140  125 230 175 175 125 255 230
b  0 100   0   0 255   0 230 200 140  125   0   0 255 255 255 190

>> which is more intuitive and familiar for
>> terminal users.
>
> "terminal users"? who are those?
> what's a "terminal"? Are we really going to
> target 70-year old curmudgeons that still work
> on a 'terminal"? why? because vim does that?

We. Because he meant "terminal emulator" or
"virtual terminal" users, which I'm sure you
fathomed just as well.

> Excuse me, but this is nonsense. Our menu and
> tool bar say copy/paste for at least 20
> years, as do the manuals. I wonder when
> people will stop beating this dead horse.

Remove the menu and toolbar, and/or rename that
other stuff back to kill/yank. That will make
it cooler and more attractive.


(menu-bar-mode 0) ; do this
(menu-bar-mode 1) ; don't do this

-- 
underground experts unijted
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Re: Is Elisp really that slow?
Óscar Fuentes wrote:

> (In my experience, the possibility that
> better things could exist don't cross the
> mind of many users.)

But finding things in Emacs isn't always
so easy. Often I get a feeling, either this
exists already or how come in hell I'm the first
person to think of such a basic thing?

I know about the on-line help (on-line = not on
paper :)), about `info', about `apropos' with
`apropos-command' and `apropos-value' and
all that. Still, when you are in the workflow,
and you don't want to leave it to find the real
deal (which you think, but cannot _know_
already exists) you just write Elisp yourself.

Here are two examples. This one I wrote as
a pretty novice Lisper:

(require 'dired)
(defun su-edit ()
  "Edit the current buffer file as superuser."
  (interactive)
  (let*((window-start (window-start))
        (point        (point))
        (mark         (when mark-active (mark)))
        (path         (or dired-directory
                          (file-truename (buffer-file-name)) ))
        (sudo-path    (sudo-root-path path)) )
    (find-alternate-file sudo-path)
    (when mark (set-mark mark))
    (goto-char point)
    (when dired-directory (dired-previous-line 1))
    (set-window-start nil window-start) ; the selected WINDOW
    )) ; [1]

The `sudo-root-path path' is here: [2]

This OTOH I wrote just this week or so:

(defun case-sensitive-regexp-search-and-replace (regexp to-string &optional start stop)
  (interactive
   `(,(read-from-minibuffer "regexp: ")
     ,(read-from-minibuffer "replace: ")
     ,@(if (use-region-p) (list (region-beginning) (region-end))
         (list (point-min) (point-max))) ))
  (save-excursion
    (let ((beg (or start (point-min)))
          (end (or stop  (point-max))) )
      (goto-char beg)
      (let ((case-fold-search nil))
        (while (re-search-forward regexp end t)
          (replace-match to-string t) )))))
(defalias 'cs-replace #'case-sensitive-regexp-search-and-replace) ; [3]

In both "cases" ... :) ... I'm pretty sure
there are canonical ways to do both because
they would seem so fundamental.

As for the quality of the code and possible
bugs, actually I'm more confident with
"su-edit" because I used that so many times,
despite the code doesn't really look good,
than with "cs-replace", having used that only
a few couple of times thus far. But as always,
suggestions/corrections are welcome, even when
the code serves to illustrate a detached
phenomena...


[1] line 64 @ https://dataswamp.org/~incal/emacs-init/edit.el
[2] https://dataswamp.org/~incal/emacs-init/sudo-user-path.el
[3] line 8, same URL as in [1]

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Eli Zaretskii wrote:

> OTOH, many, if not all of the features you
> say were proven in other editors we already
> provide in Emacs. If you are saying it took
> too long for Emacs to adopt them, then this
> is an argument about the past, which is again
> not useful IME.

That's exactly right: the past is the past and
nothing can be done about it. The future will
come when it comes, no doubt. So only the
present is up for grabs! So get as good and
meaningful a grip as you can, I'd say.

> We will not attract new users by providing
> just a text editor. There are way too many of
> them out there, and it's very hard, to say
> the least, to be significantly better just in
> that class. We must find our own niches, and
> be one of the top players in those niches.

But: we have already found our own niche, and
that is to be *everything*. This explains why
we aren't the best at every little programming
language out there. That would mean to be
superhuman - which (except for the X-Men)
is impossible.

> It is very hard to discuss this stuff in
> a useful manner when your arguments are so
> abstract. In general, the (alleged) reason
> for the current defaults to be rooted in some
> obsolete technology, if that indeed is the
> case, doesn't necessarily mean those defaults
> are invalid today. The discussion of the
> defaults should be on a case by case basis.

It is much easier that way, most definitely.

> [...] disruption of keyboard-only
> workflows, etc.

And that in particular we never want
to disrupt! :)

> Since you didn't say what TEXT EDITOR should
> entail, it is again very hard to discuss this
> claim. From my POV, we do provide a text
> editor OOB: you have the usual arrow keys,
> PgUp/PgDn, copy/paste by mouse, a menu bar
> and a tool bar with items most users will
> expect, scroll bars, simple edit commands
> like delete-char bound to keys users expect
> them to be, F1 for Help, RET which indents
> when appropriate, etc.

...? Is this the definition of a text editor?
If so, perhaps we should come up with another
category name for Emacs...

> The main IDE features cannot be implemented
> in modules. They can use some external
> support functionality via the module
> interface, but the features themselves must
> be in Lisp.

...? Cannot there be Lisp modules? What
"module interface" or type of module are we
talking, exactly?

> We don't even have a coherent framework for
> such features at this time. CEDET was
> supposed to be it, but judging by the fact
> that no one is developing it, and, on the
> contrary, what little development we have in
> the IDE area is bypassing CEDET, I'm
> beginning to think that maybe that decision
> was a de-facto mistake.

CEDET - ah, yes, I've heard of that (now).
It means "Collection of Emacs Development
Environment Tools" [1]

> People are welcome to scan MELPA packages and
> suggest to their developers to submit them to
> Emacs.

Shouldn't stuff from MELPA be moved to ELPA
first before it is moved into Emacs? :)


[1] http://cedet.sourceforge.net

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Dmitry Gutov wrote:

> Vim is consistent. What most people seem to
> like is a small set of rules that guide its
> behavior and the predictable set of features
> that can be gained from it (aside from
> plugins, of course).

I don't know if Vim is consistent or not, but
if it is, that shouldn't come as a surprise
because Vim is a regular program, and as such,
perhaps a very good one, but still not
a _software world_ like Emacs.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Eli Zaretskii wrote:

>> Vim is consistent.
>
> So is Emacs. Inconsistencies in key bindings
> are rare exceptions in Emacs. Even the
> tutorial makes the point of explaining the
> rules for consistent keybindings in basic
> editing commands.

Emacs is not consistent in the consistent sense
of the word. There are conventions and that's
all good but there are many examples where
Emacs consistency is inconsistent.

Here's one from the holster. Compare:

`yes-or-no-p'

and

`getenv'

Shouldn't that last one be renamed `get-env' to
make it consistent?

Answer: No. (But it is still inconsistent.)

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Dmitry Gutov wrote:

> I strongly disagree: as somebody who like
> instant gratification, I see that Emacs
> provides significant advantages to us: fast
> ability to see how it works and why, to
> change its behavior in an instance, to
> experiment without
> recompiling/restarting/whatever.

Interesting...

Perhaps many, if not most people like their
gratification instantly, it is rather the
gratification in particular that differs?

This

> change its behavior in an instance, to
> experiment without
> recompiling/restarting/whatever.

also appealed to me in a huge way BTW.
And I think that appeal came pretty instantly.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Drew Adams wrote:

> OTOH, a bazaar does have this advantage over
> a cathedral: masses of user/developers
> attracted (in whatever way initially -
> click-bait/"oh-shiny" or not) do sometimes
> improve a thing that might have started out
> rudimentary or poorly designed.

The bazaar is (or has been at least) far
superior to the cathedral but that is a general
statement/observation.

If newcomers to Emacs think it looks unsexy and
that it lacks IDE features, and no one in the
Emacs community does anything about it (save
for the unsexy thing, which they do at their
own desks and dont propagate save for a random
theme on MELPA no one cares to install let
alone bring in as the new standard) then no, to
Emacs and in the unsexy/IDE case it doesn't
matter how much superior the bazaar model in
general is.

But then the question is, why do the Emacs
people act this way? To make Emacs look good
isn't difficult. But the IDE thing? The reason
it hasn't happened is it is to difficult for
the layman Emacs Elisp hacker, and those who
could possibly pull it off are either to busy
with other things or thinks it is too big
a commitment to do on a voluntary-basis?

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Eli Zaretskii wrote:

> A single example doesn't yet make a point.
> To prove we have a problem, much more than
> one example will be needed.

I don't think trivial to minor inconsistencies
in a project as big as the Emacs world is
a problem. On the contrary, they are natural
and to have them corrected for the sake of
imposing consistency everywhere would be a big
effort, with a very small reward, and with the
almost certain side-effect of introducing bugs
and breaking old things in the process.

> As for my opinion on this, I make a point of
> not posting "me too" responses, if I don't
> have something more intelligent to add to the
> discussion. Especially a discussion as long
> as this one.

I agree. I also make a point of that. At the
very least when the discussion's length has
reached a certain point.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Eli Zaretskii wrote:

> Is an Emacs installation really significantly
> larger than a Vim installation nowadays?

On my RPi3 with Raspbian 9, 'aptitude show'
tells me that the "Uncompressed Size" for the
vim pack is 1,921 k, for emacs25 it is 12.8 M.

>>> Line numbers are a _must_ in vim, because
>>> so many commands _require_ you to name the
>>> line numbers. See your example above.
>>> That Emacs originally didn't have line
>>> numbers is because you don't actually
>>> _need_ them so much.
>>>
>> For programming it is a must, actually.
>
> I disagree.

Are we talking about being able to tell what
line point is at, to go to a specific line,
either thru a button or a manual command, and
so on?

If so, isn't that like one of the most basic
feature any editor whatsoever should have?

Is this a new feature to Emacs?!

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Eli Zaretskii]

> Date: Fri, 31 May 2019 17:50:46 +0200
> From: Emanuel Berg via help-gnu-emacs <help-gnu-emacs@gnu.org>
> 
> Eli Zaretskii wrote:
> 
> > Is an Emacs installation really significantly
> > larger than a Vim installation nowadays?
> 
> On my RPi3 with Raspbian 9, 'aptitude show'
> tells me that the "Uncompressed Size" for the
> vim pack is 1,921 k, for emacs25 it is 12.8 M.

Vim I have here weighs in at 24MB, so your Vim is not really a full
package.

> >>> Line numbers are a _must_ in vim, because
> >>> so many commands _require_ you to name the
> >>> line numbers. See your example above.
> >>> That Emacs originally didn't have line
> >>> numbers is because you don't actually
> >>> _need_ them so much.
> >>>
> >> For programming it is a must, actually.
> >
> > I disagree.
> 
> Are we talking about being able to tell what
> line point is at, to go to a specific line,
> either thru a button or a manual command, and
> so on?

We are talking about displaying line numbers for each line, at the
beginning of that line.

> Is this a new feature to Emacs?!

linum.el was part of Emacs since about forever.  There's now (since
Emacs 26) a built-in display feature that replaces linum.

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Ergus wrote:

> Else just think what a new user would like to
> see in a editor (or what they don't find in
> emacs) and just make a list (please open
> mind). And publish that somewhere that people
> can vote for them.

The only thing I can think of is make it look
as cool and polished as possible. Get away with
GNU propaganda and everything redundant.

It should be like a calm, warm, futuristic
underground base in dim light but with bright
colors. A couple of cockpit-style chairs and
some big scientific work areas in a huge, oval
room, with some tools and equipment hinting in
the background on neat shelves and lockers.

This is an amazing place. Now do whatever you
want with it.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Eli Zaretskii wrote:

> You said you thought the competition was more
> popular because they are more consistent.

Emacs may be inconsistent in spots but not to
the point that people will choose other
software because of that. If so, we are dealing
with people who are ... special.

"The competition" must also be consistent
beyond belief for their's to contrast to our's
to the point people will choose them rather
than us!


PS. I put "the competition" between quotation
    marks not only because it was in a quote,
    but also because ... yeah, what is "the
    competition", exactly? In this whole thread
    I think I've heard of .NET/Mono, Eclipse,
    and Vim, all of which I don't consider
    competitors as they do different things,
    mostly, or in a very different way at
    least. Also nano and notepad has been
    mentioned, two strong competitors to Emacs
    indeed :) And ed and sed, which is us
    leaving the realm of sanity IMO. So again,
    what is the competition exactly?
    Do tell, and I'll try it out with some of
    my own source right away!

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Jean Louis]

* Emanuel Berg via help-gnu-emacs <help-gnu-emacs@gnu.org> [2019-05-31 21:04]:
> Ergus wrote:
> 
> > Else just think what a new user would like to
> > see in a editor (or what they don't find in
> > emacs) and just make a list (please open
> > mind). And publish that somewhere that people
> > can vote for them.
> 
> The only thing I can think of is make it look
> as cool and polished as possible. Get away with
> GNU propaganda and everything redundant.

Getting rid of GNU propaganda would mean to drop
successful actions, as that is how GNU Emacs and
plethora of free software came to existence.

Jean

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Óscar Fuentes wrote:

> I doubt anybody is interested on using all of
> Emacs features. Not even 10% of them.
> Consistence for consintence's sake is
> useless. On every case (and Emacs is many
> editors within an editor) there are local
> optimums and those should prevail over
> global consistence.

"Local optimums ... should prevail over
global consistence."

Not bad :)

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Stefan Huchler]

Emanuel Berg via help-gnu-emacs <help-gnu-emacs@gnu.org> writes:

> Are we talking about being able to tell what
> line point is at, to go to a specific line,
> either thru a button or a manual command, and
> so on?
>
> If so, isn't that like one of the most basic
> feature any editor whatsoever should have?

Showing lines sure goto line yes should be there but do I find it
important? Not really, because you rarely remember a line number in a
big file, you either make bookmarks or move with the search ability.

And on the screen you are on movement oven forward-block / forward-line
(word / sexp-forward) should not be almost everytime better.

If you have a descent big file you have 4 digit line numbers easy, so
you have to press a key for goto-line + 4 digits + enter 6 keys.

show me a position you can't reach with 6 keys with forward/backward
block/line. Also it's not only the amount if you want 6 blocks forward
you can press the same key 6 times which is easier then typing in a
number with 6 different numbers, and if you want to go to a specific
point in a line search is another good alternative 1 button for search
start 2. 2 3 4 letters 3 enter.

So moving through typing in line numbers for cursor movement seems to me
very seldom a good way to operate.

Re: Is Elisp really that slow?

[Marcin Borkowski]

On 2019-06-02, at 12:49, Stefan Huchler <stefan.huchler@mail.de> wrote:

> Emanuel Berg via help-gnu-emacs <help-gnu-emacs@gnu.org> writes:
>
>> Are we talking about being able to tell what
>> line point is at, to go to a specific line,
>> either thru a button or a manual command, and
>> so on?
>>
>> If so, isn't that like one of the most basic
>> feature any editor whatsoever should have?
>
> Showing lines sure goto line yes should be there but do I find it
> important? Not really, because you rarely remember a line number in a
> big file, you either make bookmarks or move with the search ability.
>
> [...]
>
> So moving through typing in line numbers for cursor movement seems to me
> very seldom a good way to operate.

Fair enough, but please remember that there are also other use cases.
I sometimes need to tell my teammate to look at a specific place in the
code.  Giving the line number is often the easiest way.

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: Is Elisp really that slow?

[Stefan Huchler]

Marcin Borkowski <mbork@mbork.pl> writes:

> Fair enough, but please remember that there are also other use cases.
> I sometimes need to tell my teammate to look at a specific place in the
> code.  Giving the line number is often the easiest way.
>
> Best,

Good Point didn't think about that.

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Drew Adams wrote:

>> Not much we can do about that. Feel free to
>> lobby distros to include Emacs. IMO, if
>> something is related to 40-year old
>> decision, it is this one.
>
> Yes. Unix distributions did usually include
> both vi and Emacs. Why wouldn't GNU/Linux
> distributions include both?

What's the "40-year old decision"?

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Drew Adams wrote:

> Eli nailed it when he gave his list of the
> kinds of things Emacs really needs to improve
> its usefulness, especially for programmers
> (e.g. IDE features). It's NOT at all about
> key bindings and such.

But we just heard Emacs does have IDE features,
only not in vanilla Emacs or even the the
[M]ELPAs? So what Emacs "needs" (yuk, that
word) isn't IDE features as much as ways of
integrating those that are already there, in
the wild, and then keeping up the good work
improving them even further?


"The truth is out there"

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Ken Goldman wrote:

> I was hooked on emacs the first time I saw
> keyboard macros.

I consider keyboard macros poor man's
programming as everything you can do with
keyboard macros, you can do with programming,
but not the other way around.

I've seen people use them to align like three
or four liens or something. One should work on
ones typing and cursor moving speed instead of
doing a macro for that, IMO. And if there are
too many lines, Elisp is better than a macro
IMO as the Elisp is a helpful building block
to have. Maybe the same situation arrives
again, only with some detail changed? If you
keep your Elisp, you can solve that as well
with a minimal change to a duplicate function.

> Is there another editor that can spell
> check code?

I certainly think so :)

> I appreciate that it's multi-OS and
> multi-language.

I don't know what editors the other guys are
talking about, the so called competition.
But there are many multi-OS editors.
Multi-language, I don't know. I can only think
of Vim and .NET/Mono which I think are
C#/VB(A)/MS SQL Server and possibly some more.

> I use emacs for bulk coding. [...]

Bulk coding:

    The process of Coding all members of
    a group of Documents (identified, for
    example, by Deduplication,
    Near-Deduplication, Email Threading, or
    Clustering) based on the review of only one
    or a few members of the group.
    Also referred to as Bulk Tagging. [1]

Okaaay...?


[1] https://www.edrm.net/glossary/bulk-coding/
    
-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Óscar Fuentes wrote:

> The instant gratification that I refer to
> consists on installing the éditeur de texte
> du jour, "oh, looks nice", poke it for
> a minute and decide that it is okay for you
> and be happy thereafter, or until you
> eventually become tired/annoyed enough and
> then select a replacement by the
> same criteria.

Yes, I think that is like the definition of the
"instant gratification" mindset :)

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Dmitry Gutov wrote:

> A lot of smart, thoughtful professionals use
> other editors as well.

I'm sure you're right, but again, what editors
are those?

I'd like to try them as well :)

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Óscar Fuentes wrote:

> See, I'm not saying that Emacs is perfect or
> the users of other editors are lazy fools or
> wathever, I'm just saying that converging
> towards popular editors is not good per se.
> I started with Borland IDEs, then Visual
> Studio [...] No matter their popularity, they
> are rudimentary editors compared to Emacs.

Ah, right!

Borland and Visual Studio!

But that was a long time ago! We are not
comparing us now, with them then, right?

(Not that I doubt Emacs was better even then,
of course.)

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Drew Adams wrote:

>>>> Not much we can do about that. Feel free
>>>> to lobby distros to include Emacs. IMO, if
>>>> something is related to 40-year old
>>>> decision, it is this one.
>>>
>>> Yes. Unix distributions did usually include
>>> both vi and Emacs. Why wouldn't GNU/Linux
>>> distributions include both?
>> 
>> What's the "40-year old decision"?
>
> That was Eli who spoke about such a decision.
> Presumably he meant a decision, or decisions,
> by those in charge of GNU/Linux distributions
> not to include Emacs.

Yes, but what decision was that? In 1979?

$ echo $(( 2019 - 40 ))
1979

But GNU is from 1983, and Linux from 1991. [1]


[1] https://dataswamp.org/~incal/COMP-HIST

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Stefan Huchler wrote:

> But that has nothing to do with modal vs non
> modal. Modal is more efficiant, something
> simple as copy / paste is not only less keys
> to press but literally more healthy, pressing
> c + reposition + v instead of M-k +
> reposition + C-Y is just better for
> your fingers.

I'm not following the "reposition" stuff,
but...

If you have your hands/fingers, on at asdf and
jkl; (and thumbs on SPC) on a modern keyboard,
where the keyboard manufacturer's Alt is our
Meta, then M-k requires moving your left thumb,
in my case, ~3.5cm horizontally, and your right
hand, not at all, as the middle finger is
already at `k' position.

As for C-y, on your right hand, this requires
a move with your index finger from `j' to
`y', which is again 3.5cm, but now to the NE
which makes a tiny bit more strain to it.
On the other hand... :) ... on the left hand,
moving your pinky finger from `a' to Control is
5cm SSE, which I agree isn't optimal.

As you say later on this was designed for
keyboards where Control where at another place,
and it can be arranged (in the Linux
VT/ttys/the console as well as in X) to have
Control and CAPS change places. This is is not
a bad idea - I tried it once, but by then I was
so used to the position of Control and besides
I had rewired CAPS as well (see my HP, URL in
signature; search for CAPS) so it was too late
for me. That said, I don't think C-y is
_that_ bad!

> And for programmers 50% of commands you do as
> programmer are not basic text manipulation
> like insert/delete key.

If you mean the manufacturer's insert/delete,
they mean you move you hands away from the
asdfjkl; position which reduces speed. If you
mean C-d and DEL, I dont' think they are bad.

> hitting a long C-c C-q or even longer
> commands is not so easy, even you don't get
> pain in your fingers.

`C-c C-q' is undefined in many modes. If you
don't like it for the modes where it is and
where you use it, why don't you redefine it?
It's simple.

> I just think that the GNU project should
> prioratise the health of their Users more
> than the rememberability.

Yes, here I agree 100%.

> So I hit my easy accessable Menu key

? ... what is the "Menu key"? :O

> and then afterwards the c key for copying as
> example, the holding of keys and then
> stretching to other keys is what hurts the
> pinky so much and why C-c C-f as example is
> a horrible command.

Same comment as for `C-c C-q'.

> Yet how do you remember M-k and C-y because
> you know it means kill/yank, else this
> shortcut makes no sense, so at least if you
> prioratise rememberability over the health of
> the developers be consequent, if you call it
> copy and paste make it C-c or M-c and for
> paste C-p. Then you would at least
> be consequent.

Mnemonic keys should never be prioritized over
ergonomics. But again, I don't think M-k is
bad, and if C-y is bad, not by much. Also,
don't forget to take into consideration every
other cursor/point movement and editing command
which we can do with a keystroke, but other
editors require one to use the mouse to (maybe)
do something similar!

> That C-p is bound to previous line is no real
> reason because you also have they arrow keys

The arrow keys!? Oh no, C-p is much better.
The arrow keys require you to move your right
hand from jkl;. Instead, it is better to think
of ijkl as arrow keys (i = up, k = down, j =
left, and l = right). I have set up tons of
keys to work on that, scrolling not the
least:

    https://dataswamp.org/~incal/emacs-init/scroll.el

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Ergus wrote:

> I have experienced the issue that when adding
> a new functionality (to isearch) there is not
> an available/standard method to assign
> a binding for it. But also I ended
> discovering a new set of shortcuts
> I didn't know.

If you feel like you run out of shortcuts, you
can define a new prefix key, e.g.:

    (define-prefix-command          'C-o-prefix)
    (global-set-key         "\C-o"  'C-o-prefix)

> But ergoemacs actually failed because all the
> modules and packages bind the default keys to
> the "expected" action. So for example
> next-line is hardcoded to anything that moves
> one line down in helm, counsel, emms, and so
> on. So at the end the user never gets a real
> 100% ergoemacs experience. Similar happens in
> evil mode or even in cua-mode that is
> integrated inside emacs. One never gets the
> 100% of the experience.

If you set it up 100% as you'd like it, mode
for mode, you'll get it.

> Please, don't tell me that this won't fly;
> I already know.

It will fly if you do it.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Ergus wrote:

> I still don't know any user that uses the
> numeric arguments for common commands.

Well, now you know one who uses them all
the time.

I even have several functions I've written
myself that do different things, not just
numerically, depending on if they receive it
or not.

> They use C-z and C-S-z for undo-redo.
> That's why I asked in other email about case
> sensitive bindings. Same for many others.

I have M-0 for `undo' and for redo I hit SPC
and then do `undo' twice. (Perhaps I should
bind a redo to C-0 which I never thought of but
apparently has undefined ATM...)

> We just need to consider the statistics of
> the users who use emacs (including spacemacs)
> and what OS they use. Linux for desktop is
> sadly a fail

Linux (GNU/Linux/X/GNOME|KDE|whatever) as
a desktop is a huge success. Nowadays with the
smartphones etc the desktop itself may be in
trouble but that's another issue.

To be honest, I never cared much for it myself,
I got enough of it during my Mac Plus "Finder"
days, and later WinXP.

Emacs and a terminal emulator is much
better IMO.

> in the short medium time, but the number of
> GNU/Linux users keeps growing (very slowly,
> but growing). And the most popular
> distributions are not the most
> fancy-desktop-oriented ones.

? ... ever heard of Ubuntu?

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Eli Zaretskii wrote:

>> Unix principles imposes here, do one thing
>> and do it right
>
> That's not applicable to Emacs, since Emacs
> is not a tool, it's a programming and
> text-editing environment.

And howcome that is that not a tool? A Swiss
army knife and a combination plier is not
a tool either, as they contain different tools?
A toolbox is not a tool, as you can put
different tools in it? A milling machine is not
a tool, as you can use it to make yet another
tool? Maybe even a tool to improve the milling
machine itself?

No, Emacs is a tool.

Rather, Emacs is the exception that confirms
the Unix rule...

> I didn't mean "undo" literally, I meant
> "do-SOMETHING" and "UN-do-SOMETHING".
> Every editor I've met has different commands
> for action and counter-action, while you say
> we should have a single command that
> does both.

One can do that with the single `undo' command,
as I described in (I think) my previous post.
(I'm sure I'm not the first/only one doing that
and probably not the last either.)

> And you go there by line numbers?

Line numbers, here meaning the functionality,
not a vertical ruler by the side, is very
useful - when compiling, when communicating,
when editing. Indispensible I'd say (in the
context of and advanced editor, of course).

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

X Y Z wrote:

> And, that's exactly the point! Emacs *is*
> inconsistent, and we should try to eliminate
> them (hopefully before a bigger user base
> appears).

We should always strive for it but there is no
need to burn down the house in order to kill
the rats.

> There's a point where any kind of big
> software platform has an HIG, like Apple
> macOS, iOS, or XDG, Gnome, Qt, e.g... and
> I would appreciate if we can make an HIG for
> Emacs packages. Thinks like keybindings, UI,
> messages, etc.

We have HIGs as well. Check out, for example

    (info "(elisp) Coding Conventions")
    (info "(elisp) Packaging")
    
and I'm sure there are better examples still,
in particular WRT keybindings.

BTW:

XDG = X Desktop Group
<https://www.quora.com/What-does-xdg-mean-literally-in-linux>

HIG = Human Interface Guidelines
<https://www.allacronyms.com/HIG/computer>

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Óscar Fuentes wrote:

> Why those modes that you take as models for
> C-c C-c use precisely that binding for
> sending text to an inferior process?
> Because it is the most used action and,
> hence, it is natural to use a keybinding that
> is fast and easy to type.
>
> That's the same reason why Org-mode uses
> C-c C-c for its most frequent actions,
> although you pretend to remove that
> convenience because "consistency"... when in
> fact you'll be reducing consistency!

That's exactly right.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Stefan Huchler wrote:

> But another suggestion for that, if C-c C-c
> is meant as shortcut for
> a "important-function" why not have a binding
> for "do-important-action" or
> "do-major-action" and depending on mode that
> functions calls the important function of the
> mode. So that the user can choose globaly
> a keybinding for that and don't has to do
> that for 80 modes seperately. and the
> developer of the mode just somewhere sets
> which function is bound to C-c C-c by
> setting: (setq mode-important-function
> 'compile...)

Well, there are many functions that could be
considered the important one. People will have
different ideas what is the one to use.
Instead of configuring keys, people will start
configuring what function is the
"important-function".

Besides, most people don't use 80 modes. And if
those who do are really unhappy with keystrokes
in all 80, so be it. It is easy to change the
keys for one mode, and once one has mastered
that black art, to do it for the next 79 should
be a snap. Also, one doesn't have to do it all
in one evening.

Rather, make one adjustment every day and ten
years later you still have interesting things
to do :)

> Also I would argue that most people press C-c
> C-c with left control instead of the
> ergonomic way to use the right control
> therefor you train people to use
> unergonomic keychords.

? ... why is the right control more ergonomic?
It is farther away so at least on my keyboard
I have to move my entire right hand out of
typing position to reach it, which with the
right hand fingers just get slightly of their
asdf marks.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Van L wrote:

> Is it possible to have the best of all
> possible worlds?
>
> For conservatives, a winter release of old
> gold keybindings. For the free radicals,
> a spring release with modernizations.

Possible.

> When I use a long M-x sequence, a shortcut
> suggestion appears. It disappears before
> I can catch it. Can it stay for 30 seconds?

That should be configurable easily if you find
the variable - go fish!

> Can there be an instant interactive override
> to set it whatever you like?
> Evolutionary programming of popular custom
> keybindings collected at upstream and put
> thru obstacle course competition is one way
> of composing a spring release.

Hm, perhaps you should put your literary
talents to use in a fancy quarterly
"Emacs World" magazine? I'd read it, for sure.
Only perhaps you need an editor, to make the
material a bit more readily digestible. (Here,
I'm not volunteering tho.) That said, don't
stop writing here for the love of it...

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

X Y Z wrote:

> I would like a semi-AI that suggests
> interactive functions based on key presses
> [...]

Better to learn them yourself by heart IMO.
This comes quickly if you use Emacs every day.

> Saying about discoverability, I would like
> a context-sensitive right-click mouse menu,
> something like Microsoft Office.
> Most newcomers are familiar with finding
> functionality with the mouse; and it isn’t
> intuitive to find new keybindings/functions
> that Emacs provide to boost productivity.
> (Actually, that’s one of my problems; how
> should I find new functions...?)

Easy:

  1) Throw the mouse out the window
  2) (menu-bar-mode 0)
  3) C-h f something-related RET

Also: `apropos', and for interactive functions
in particular: `apropos-command'

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

X Y Z wrote:

> I would like a semi-AI that suggests
> interactive functions based on key presses
> [...]

Better to learn them yourself by heart IMO.
This comes quickly if you use Emacs every day.

> Saying about discoverability, I would like
> a context-sensitive right-click mouse menu,
> something like Microsoft Office.
> Most newcomers are familiar with finding
> functionality with the mouse; and it isn’t
> intuitive to find new keybindings/functions
> that Emacs provide to boost productivity.
> (Actually, that’s one of my problems; how
> should I find new functions...?)

Easy:

  1) Throw the mouse out the window
  2) (menu-bar-mode 0)
  3) C-h f something-related RET

Also:

  `apropos', and for interactive functions in
  particular: `apropos-command'

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

RE: Is Elisp really that slow?

[Drew Adams]

> > When I use a long M-x sequence, a shortcut
> > suggestion appears. It disappears before
> > I can catch it. Can it stay for 30 seconds?
> 
> That should be configurable easily if you find
> the variable - go fish!

https://emacs.stackexchange.com/a/48763/105



C-h v echo-keystrokes:

echo-keystrokes is a variable defined in C source code.

Its value is 1

Documentation:

Nonzero means echo unfinished commands after this many seconds of pause.

The value may be integer or floating point. If the value is zero, don't echo at all.

You can customize this variable.


---

(You really should get your email fixed, BTW, so that
Reply All replies also to help-gnu-emacs and not just
to your private email address.  I had to substitute
the mailing list for the latter by hand.)

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Ergus wrote:

> There is which-key.

... where? :S

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

X Y Z wrote:

>> this: (define-key c-mode-base-map "\C-d"
>> 'c-delete-forward) will be: (define-key
>> c-mode-base-map delete-forward-binding
>> 'c-delete-forward)
>
> This, at least for me, feels like a fantastic
> idea :-)

The idea is a classic computing idea where the
method is to reduce complexity by squeezing in
yet another layer in between.

Here, I'm hesitant because there really is no
complexity to assigning keys to
different modes.

But I might be wrong, so do it! Is what I would
have done. The worst thing that can happen (if
you succeed) is that you have it on your local
Emacs and it makes you happy now and then, and
no one else cares, neither positively or
negatively, really.



             "Do it today - in a different way!"
                - Stuck on Replay, Scooter, 2010
   <https://www.youtube.com/watch?v=VlpNp2HBNUE>

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Van L wrote:

> A possible starting point for transitioning
> from the mouse to boosting keyboard
> productivity is for the right third of the
> keyboard under the delete-key to do arrow-key
> navigation and block/line/page/def jumping
> through text by hyper or super key combos on
> the eight keys around RET and SHIFT.
> Sending the right hand all the way to the
> bottom right corner for the arrow keys
> requires an entire hand move, again, to
> get back.

Yes, only I think the arrow keys should be
where the right hand already is, or should be
for touch typing, namely at ijkl where i = up,
k = down, j = left, and l = right.

My first computer was a Mac Plus in the late
80s, and there were tons of little games,
Lode Runner and such, and all those used ijkl
like this. And it works great for Emacs as
well! I have M-i "scroll-up-1" and M-k
"scroll-down-1", with M-I "scroll-up-pane" and
M-K "scroll-down-pane" [1] _everywhere_, in
every mode. Awesome stuff!

Now in a recent post I said one should throw
the mouse away. Good idea! But there are
exception where the mouse is helpful, in
application that are visual in nature.
For example GIS and map applications. If we
assume the user isn't a southpaw, so [s]he uses
the mouse with the right hand, the arrow keys
should then not be as above but instead the
wsad keys. Just like in Quake, or Dark Castle,
if we stick to the Mac Plus: aim with the
mouse, move with the keys.


[1] https://dataswamp.org/~incal/emacs-init/scroll.el

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Ergus wrote:

> What is clear is that all the bindings are
> set and we talk about new features without
> binding (or a plan to bind them). Y made all
> the possible questions, case sensitive
> bindings, release some of them, make a C-a go
> to indent and then to beginning of line...
> anything to have some more place without
> create longer bindings.

New prefix keys are by definition a tiny bit
longer, yes, but if placed sensibly they can be
really, really fast and ergonomic.

So don't be afraid of 'em.

Just try, and you'll fly, as the barrel racer
puts it.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Stefan Monnier wrote:

> I think one of the main differences (besides
> the fact that it's modal, obviously) is that
> VI has prefix commands like `d` which Emacs
> lacks. That helps make things
> regular/orthogonal.

But we also have it: `define-prefix-command'

Care to explain the difference? Pretty please :)

Also, why should it be capitalized, "VI"?

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Óscar Fuentes wrote:

> Ergonomics is a well established discipline.
> Not exact, but mature enough to say with
> a very high level of confidence that certain
> practices are harmful.

The best ergonomic thing I ever did was to get
a projector and have Emacs displayed on a wall.
That way, you keep your back and head straight.

I actually think this has improved my posture
AFK as well - people always tell me I have good
posture!

Once I entered a muay Thai fight club and when
I addressed the club owner (who is a champion,
let be in a small sport, small sport here that
is), when I addressed the owner with some
practical question, he immediately straightened
his whole body and filled his chest with air!

A muay Thai champ! I didn't know Emacs could
do that :)

The second best thing I did was to get rid of
the mouse. I did both some ten years ago and
the reason was then I had severe eye problems.

The eye problems wasn't because of computer
use, but because of shampoo getting into my
eyes! But I didn't know that, and it still hurt
the most when I did computers (I did CS at the
university at that time) so naturally I thought
it was the computers that caused it, so
I started configure everything to limit the
strain, and the two biggest changes was the
projector and the mouse.

Now, eye problems are gone but I stick with
almost everything I did back then, as it makes
sense regardless IMO.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Eli Zaretskii wrote:

> It's a question of taste, and other people
> can legitimately have different tastes.
> For example, if someone prefers to read the
> manual in PDF format, your notation
> is useless.

Right... *read* the manual. I always forget
about that!

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Robert Thorpe wrote:

>> But the main problem is not the char you use
>> but that you have to hold this modifiers,
>> which "reduces ergonomics" as compromise
>> I would also be ok with having some sort of
>> sticky keys? and you press once Ctrl release
>> then press c c. But this is not even
>> a optional feature in emacs:
>> https://www.emacswiki.org/emacs/StickyModifiers
>
> In Emacs sticky keys aren't so simple.
> The one you give is a fairly good example.

I have sticky keys here (I think?) [1] and
I didn't even know that was their official
name. I use them to navigate the filesystem but
yes, they can be used to invoke functions
as well.

So for example, to go to that particular file,
I hit

C-j e n

e = Emacs/Elisp file
n = navigate [the filesystem]

As you see, mnemonic which is all-but necessary
in this case since there are so many files.
Another example, to go to

    ~/.emacs.d/emacs-init/gnus/message-my.el [2]

I hit

C-j e g M

e = again Emacs/Elisp file
g = Gnus
M = message

(lowercase m is another file)


[1] https://dataswamp.org/~incal/emacs-init/navigate-fs-keys.el
[2] https://dataswamp.org/~incal/emacs-init/gnus/message-my.el

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Xavier Maillard]

> Date: Wed, 05 Jun 2019 06:44:09 +0200
> From: Emanuel Berg via help-gnu-emacs <help-gnu-emacs@gnu.org>
>
> > They use C-z and C-S-z for undo-redo.
> > That's why I asked in other email about case
> > sensitive bindings. Same for many others.
> 
> I have M-0 for `undo' and for redo I hit SPC
> and then do `undo' twice. (Perhaps I should
> bind a redo to C-0 which I never thought of but
> apparently has undefined ATM...)

I do not quite get it, you do M-0 for undo and SPC-M-0-M-0 for redo,
right ?

> Emacs and a terminal emulator is much
> better IMO.

100% agree. This is pretty much all I need here. For some times, I was
even in the linux console with no X server. Pro: it is super stable,
damn fast and smooth. Cons: I was being chambered by friends and
family :)

-- 
Xavier Maillard                      
e/j:xavier@maillard.im                w:www.maillard.im
m: 06 52 18 63 43 (old)
m: 06 49 60 48 56 (NEW)

GPG:                           9983 DCA1 1FAC 8DA7 653A
                               F9AA BA49 09B7 8F04 DE1B

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Robert Thorpe wrote:

> I think is mainly because of the way you use
> this mailing list.
>
> You seem to think of it as something more
> than a help mailing list. I think lots of
> other people think of it strictly as
> a help list.
>
> I expect that some of your mails would work
> better elsewhere. Perhaps on a blog or
> on Reddit.

Had I known there was no computer culture
whatsoever I would never have done any of this
to begin with.

Taking pride in elitist one-dimensionalness and
the prevailing notion that computer people
should themselves be and act as computers -
it is _in-human_.

Or wait... maybe it isn't elitism or even
a notion... maybe people really *are* like
that? Now had I known _that_, not only would
I never have done any of this, I would have
turned and run as fast as I possibly could,
without ever looking back!

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Eli Zaretskii wrote:

>> I think the tone of this very reply confirms
>> he is right.
>
> What "tone" is that?

Monotone and haughty.

> On the contrary, claims such as yours are IMO
> harming the discussions because they make
> people think twice before posting their
> legitimate opinions, lest they will be
> accused of having a "tone".

It's a good thing then you come to their
defense, then, so they dare voice their minds
without fear.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Xavier Maillard wrote:

> GNU Emacs has no need to be competitive IMO.
> As I am okay living right into emacs buffers,
> I do not need to compare it with competitors.

Me neither.

> I, for years, constantly was tweaking my
> .emacs for hours and hours. As to my
> surprise, I did not get the job done ;) THat
> was quite fun but, in the end, that was
> pretty conter-productive.

Same for me, only I'm not sure if I really had
done anything better had I not done it.
Maybe I'd have done 20 or so "would have been
the next Quake" only none were ever finished.
The stuff I did with Emacs at least I use every
day, it was/is fun to do, and I take happiness
and pride from that, and productivity as well,
when I do real projects - those have become
fewer and farther in-between tho, but I don't
blame Elisp for this.

> I doubt powerusers have tons of them
> installed and worst, I even doubt their
> .emacs are huge. [...]

It depends what the definition of poweruser is.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Eli Zaretskii wrote:

> Vim I have here weighs in at 24MB, so your
> Vim is not really a full package.

Both sizes refer to compressed packages.
The packs rely one other things, which rely on
other things, and ultimately everything relies
on the whole computer, so I suppose Emacs and
Vim are actually equally big.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Jean Louis wrote:

> Getting rid of GNU propaganda would mean to
> drop successful actions, as that is how GNU
> Emacs and plethora of free software came
> to existence.

Reduce the "in your face" nature of it, maybe
I should have said. In the beginning, it felt
like every time you missed a keystroke up came
documents telling everything you didn't want to
know about GNU.

And:

(setq inhibit-startup-screen t)
(setq inhibit-startup-echo-area-message "incal")

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Stefan Huchler wrote:

> If you have a descent big file you have 4
> digit line numbers [...]

Why would you want a file that is 9999 lines
long, and that's still just a "descent big
file"? How big are the really big files? :O

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Marcin Borkowski wrote:

> I sometimes need to tell my teammate to look
> at a specific place in the code. Giving the
> line number is often the easiest way.

You don't say.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Xavier Maillard wrote:

> I do not quite get it, you do M-0 for undo
> and SPC-M-0-M-0 for redo, right ?

Right, only it doesn't have to to be SPC in
particular. But SPC seems to "do" the least,
altho I suppose it is just another char like
any other technically speaking.

(characterp ?\ ) ; t

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[Emanuel Berg via help-gnu-emacs]

Xavier Maillard wrote:

> What would you do with more speed that you do
> not do today ?

Because there isn't more speed, your brain
doesn't produce ideas what to do with it.
At least mine doesn't.

But OK, if there was unlimited speed I'd do
a space simulator game where you traveled
across the galaxies all the while controlling,
setting up, and upgrading the space ship with
Emacs Lisp only.

Including the crew:

    (setq number-of-pushups-every-day 45)
    (setq hire-women-only t)

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Eli Zaretskii]

> Date: Fri, 07 Jun 2019 20:34:30 +0200
> From: Emanuel Berg via help-gnu-emacs <help-gnu-emacs@gnu.org>
> 
> Eli Zaretskii wrote:
> 
> >> I think the tone of this very reply confirms
> >> he is right.
> >
> > What "tone" is that?
> 
> Monotone and haughty.

It was none of that.  They were simple and polite words.  I suggest
you ask a friend to give you a second opinion about that.

> > On the contrary, claims such as yours are IMO
> > harming the discussions because they make
> > people think twice before posting their
> > legitimate opinions, lest they will be
> > accused of having a "tone".
> 
> It's a good thing then you come to their
> defense, then, so they dare voice their minds
> without fear.

People should be free to speak their minds without requiring any
"defense".

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[Samuel Wales]

On 5/11/19, tomas@tuxteam.de <tomas@tuxteam.de> wrote:
> On Sat, May 11, 2019 at 02:38:58AM +0200, Emanuel Berg wrote:
>> If it is possible to have Elisp as fast as CL
>> or even as fast as C (!), the only problem is
>> no one has been motivated to do it! ???

> One last attempt: this whole thread is based on the premise
> that there is one clear definition of slow. Yet, there isn't.

well, for me as a mere user, i just want elisp [per se] to run faster.
maybe that has 1000 definitions, but the 3 things that i have read in
this thread wfm as PERFECT targets for speeding up and would make
emacs even better for me.

  1] org mode timestamp [aka "daily/weekly"] agenda to complete faster
for a 2 day [that is my normal span] agenda.  if this has more than
one definition, then i impetuously choose all.  :]  n.b. it could in
principle be mostly spread to 1 core per org agenda file as i doubt
it's teh sorting and displaying that takes up the time.  [not
intending to set cat among pigeons on that last one.]

  2] while re-search-forward do replace-match [iirc] [my question
which stefan nicely addressed] to work faster.  this is a tight loop
called frequently.

  3] elisp to be much faster so that new ideas can be done.

by "per se" i mean, perhaps confusingly, mere users program in a lisp,
preferably elisp, as opposed to a strange new syntax or c-like stuff
or elisp-like c.

for my part, i'll gladly program in cl if it is faster [quite similar
to elisp and standard], or even in scheme if needed for tight loops
[different from elisp but lisp].  i will not be thinking about
anything lower level than lisp.  unless somebody says use this
construct.

-- 
The Kafka Pandemic

What is misopathy?
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

The disease DOES progress. MANY people have died from it. And ANYBODY
can get it at any time.

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[Samuel Wales]

On 6/7/19, Samuel Wales <samologist@gmail.com> wrote:
> anything lower level than lisp.  unless somebody says use this
> construct.

for clarity i mean, merely, like, that great blog on efficiency that
said the cl-loop construct is fast.  [i like fp, but i also like
cl-loop.]

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1550 bytes --]

On Fri, Jun 07, 2019 at 05:26:10PM -0700, Samuel Wales wrote:

[...]

> well, for me as a mere user, i just want elisp [per se] to run faster.
> maybe that has 1000 definitions, but the 3 things that i have read in
> this thread wfm as PERFECT targets for speeding up and would make
> emacs even better for me.

Buy a faster computer, then :-)

>   1] org mode timestamp [aka "daily/weekly"] agenda to complete faster
> for a 2 day [...]

Dunno about that. Perhaps Org has sub-optimal algorithms there?
Why don't you get your hands dirty and profile the whole thing?

>   2] while re-search-forward do replace-match [iirc] [my question
> which stefan nicely addressed] to work faster.  this is a tight loop
> called frequently.

AFAIR the bottleneck here wasn't Elisp at all, but regexp match,
which is written totally... in C (actually there's a mini-language
layer in-between). So the totally wrong target for this thread:
Elisp ain't slow here because it isn't in the equation at all.

Do I remember incorrectly?

>   3] elisp to be much faster so that new ideas can be done.
> 
> by "per se" i mean, perhaps confusingly, mere users program in a lisp,
> preferably elisp, as opposed to a strange new syntax or c-like stuff
> or elisp-like c.

And this is so generic that it's best served by a generic programming
language, which Elisp isn't, and for a good reason.

If you pose a question, you should be ready to expect that the
answer is... "it depends". That's how learning starts :-)

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Robert Thorpe]

Emanuel Berg via help-gnu-emacs <help-gnu-emacs@gnu.org> writes:

> Robert Thorpe wrote:
>
>> I think is mainly because of the way you use
>> this mailing list.
>>
>> You seem to think of it as something more
>> than a help mailing list. I think lots of
>> other people think of it strictly as
>> a help list.
>>
>> I expect that some of your mails would work
>> better elsewhere. Perhaps on a blog or
>> on Reddit.
>
> Had I known there was no computer culture
> whatsoever I would never have done any of this
> to begin with.
>
> Taking pride in elitist one-dimensionalness and
> the prevailing notion that computer people
> should themselves be and act as computers -
> it is _in-human_.
>
> Or wait... maybe it isn't elitism or even
> a notion... maybe people really *are* like
> that? Now had I known _that_, not only would
> I never have done any of this, I would have
> turned and run as fast as I possibly could,
> without ever looking back!

What I'm saying here is about context.

I have nothing against computer culture.  I think it's fairly
interesting, and I sometimes talk about it too.

The point is though, it's not what *this list* is about.  This list is
much more focused on help, as it says in the name.  There are other
forums that are much more loose.

Take Reddit, for example.  Lots of your more off-the-wall mails would be
appreciated on Emacs Reddit.  That's because it's not a forum focused on
help.  By posting off-the-wall stuff here you're irritating others and
also needlessly frustrating yourself.

BR,
Robert Thorpe

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[Samuel Wales]

On 6/8/19, tomas@tuxteam.de <tomas@tuxteam.de> wrote:
> On Fri, Jun 07, 2019 at 05:26:10PM -0700, Samuel Wales wrote:
>> well, for me as a mere user, i just want elisp [per se] to run faster.
>> maybe that has 1000 definitions, but the 3 things that i have read in
>> this thread wfm as PERFECT targets for speeding up and would make
>> emacs even better for me.
>
> Buy a faster computer, then :-)

i was going to add somethning like "without buying a supercomputer"
but figured nobody would say to get a new computer.

an osx help channel was once asked by a disabled person what to do to
increase font sizes for the icons.  these were not configurable at
that time.  therefore their response was that there was nothing wrong
with osx, and the obvious solution was always for the user to get new
glasses.  this included all disabled users of all types.

i suspect that if they were configurable, the answer would have been
that obviously they needed to be configurable.

the topic of the thread is elisp speed.  it's true that if a
significantly faster single-core-throughput computer existed and
everybody could have access to it, that would solve the problem.  but
i don't think it is the question that most of us are discussing in
this particular case.  it definitely is not the quesiton i was
addressing.

>>   1] org mode timestamp [aka "daily/weekly"] agenda to complete faster
>> for a 2 day [...]
>
> Dunno about that. Perhaps Org has sub-optimal algorithms there?
> Why don't you get your hands dirty and profile the whole thing?

we discussed this on this thread.  it is optimized to death.  it could
perhaps be rewritten but has not been and that is a large task.  and
no, i am not capable of it.  and no, that is not the question in this
particular case.  the question is, can we speed it up, as the topic of
the thread states, by speeding up elisp.  for example with a faster re
matcher.

also, take a look at the profile.  it's kind of flat.  there isn't
anything that obviously takes up the time.

>
>>   2] while re-search-forward do replace-match [iirc] [my question
>> which stefan nicely addressed] to work faster.  this is a tight loop
>> called frequently.
>
> AFAIR the bottleneck here wasn't Elisp at all, but regexp match,
> which is written totally... in C (actually there's a mini-language
> layer in-between). So the totally wrong target for this thread:
> Elisp ain't slow here because it isn't in the equation at all.
>
> Do I remember incorrectly?

i don't know.  i know your feeling.  the thread talks about
keybindings too and i struggle to figure out that relevance.

to me, you remember incorrectly, but i could be wrong.

i thoght th thread was abuot speeding uyp elisp.  improving the c core
or the bytecode or whatever seems to me a perfect speedup.

but i am a mere user.

>
>>   3] elisp to be much faster so that new ideas can be done.
> And this is so generic that it's best served by a generic programming
> language, which Elisp isn't, and for a good reason.

well, what do i know.  i thought perhaps c could be sped up, the core
could replaced with a really optimized lisp, a fwe new elisp
constructs with opimized c like a new re engine could be introduced,
etc.

>
> If you pose a question, you should be ready to expect that the
> answer is... "it depends". That's how learning starts :-)

i think i am regressing.

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 833 bytes --]

On Sat, Jun 08, 2019 at 02:00:23PM -0700, Samuel Wales wrote:
> On 6/8/19, tomas@tuxteam.de <tomas@tuxteam.de> wrote:
> > On Fri, Jun 07, 2019 at 05:26:10PM -0700, Samuel Wales wrote:
> >> well, for me as a mere user, i just want elisp [per se] to run faster.
> >> maybe that has 1000 definitions, but the 3 things that i have read in
> >> this thread wfm as PERFECT targets for speeding up and would make
> >> emacs even better for me.
> >
> > Buy a faster computer, then :-)
> 
> i was going to add somethning like "without buying a supercomputer"
> but figured nobody would say to get a new computer.

[...]

Yeah, I know. That's why I put a smiley in there. Single-thread speed
hasn't advanced for a while (on commodity hardware, at least) and it
seems it'll stay so for yet another while.

Cheers
-- tomás

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Óscar Fuentes]

tomas@tuxteam.de writes:

> On Sat, Jun 08, 2019 at 02:00:23PM -0700, Samuel Wales wrote:
>> On 6/8/19, tomas@tuxteam.de <tomas@tuxteam.de> wrote:
>> > On Fri, Jun 07, 2019 at 05:26:10PM -0700, Samuel Wales wrote:
>> >> well, for me as a mere user, i just want elisp [per se] to run faster.
>> >> maybe that has 1000 definitions, but the 3 things that i have read in
>> >> this thread wfm as PERFECT targets for speeding up and would make
>> >> emacs even better for me.
>> >
>> > Buy a faster computer, then :-)
>> 
>> i was going to add somethning like "without buying a supercomputer"
>> but figured nobody would say to get a new computer.
>
> [...]
>
> Yeah, I know. That's why I put a smiley in there. Single-thread speed
> hasn't advanced for a while (on commodity hardware, at least) and it
> seems it'll stay so for yet another while.

Yep, we can't count on the hardware guys anymore to resolve our
performance problems, we are forced to improve what we have if we wish
to broaden what our tools can do on terms of data volume and/or
complexity.

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[Emanuel Berg via help-gnu-emacs]

tomas wrote:

> And this is so generic that it's best served
> by a generic programming language, which
> Elisp isn't, and for a good reason.

Well, I guess it depends what you mean. To me,
Elisp is a domain-specific dialect of Lisp.
And Lisp is as generic as it can be. It is the
Pythagorean theorem theorem of computing, as
Pascal Bourguignon would have put it (long time
I heard from him BTW). [1]

So is Elisp Elisp or is it Lisp with an E?
(NB not Ecstacy.)

It is rather the "ungenericness" of the domain
that makes it ungeneric, if so. And Emacs is
pretty generic, I'd say?

Or what do you mean, more specifically?

But in practice if someone I meet in the
city center has a computer problem, I don't say
"solve it with Elisp". So I suppose you are
right, still...


[1] PJB has a homepage here:
    <https://www.informatimago.com>
    But no e-mail what I can see anywhere.
    I always like to CC people when I mention
    them, so I put this here to show I at least
    made a small effort to find his e-mail.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

Óscar Fuentes wrote:

> Yep, we can't count on the hardware guys
> anymore to resolve our performance problems,
> we are forced to improve what we have if we
> wish to broaden what our tools can do on
> terms of data volume and/or complexity.

And even if we could count on other people to
do everything for us, we wouldn't want
to anyway.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[Eli Zaretskii]

> From: Samuel Wales <samologist@gmail.com>
> Date: Sat, 8 Jun 2019 14:00:23 -0700
> Cc: help-gnu-emacs@gnu.org
> 
> the topic of the thread is elisp speed.  it's true that if a
> significantly faster single-core-throughput computer existed and
> everybody could have access to it, that would solve the problem.  but
> i don't think it is the question that most of us are discussing in
> this particular case.  it definitely is not the quesiton i was
> addressing.

The literal question of whether Lisp can be sped up was answered
already: what we have is probably as fast as it can be.  Minor
improvements are still possible, of course, but they are unlikely to
change the situation drastically, where "drastically" meand something
like 30% at least.  We've hit the wall.

That is why there are "out of the box" ideas that float around, like
JIT byte-compilation, compilation to native code instead of byte code,
etc.  Personally, I'm not sure they will bring a significant speedup,
but since none of those projects was finished yet, I think we should
wait until some of them do before concluding whether any of them show
a direction we should go.

> >>   1] org mode timestamp [aka "daily/weekly"] agenda to complete faster
> >> for a 2 day [...]
> >
> > Dunno about that. Perhaps Org has sub-optimal algorithms there?
> > Why don't you get your hands dirty and profile the whole thing?
> 
> we discussed this on this thread.  it is optimized to death.  it could
> perhaps be rewritten but has not been and that is a large task.  and
> no, i am not capable of it.  and no, that is not the question in this
> particular case.  the question is, can we speed it up, as the topic of
> the thread states, by speeding up elisp.  for example with a faster re
> matcher.

But the re matcher is written in C, and is also optimized to death.
Replacing it with another regexp engine is a huge task, certainly not
smaller than rewriting the Org agenda code, because Emacs has some
unusual requirements from the regexp code (which is why we use a
forked version of an old glibc regexp code, and didn't yet succeed to
migrate to the current glibc code).

Re: Is Elisp really that slow?

[Tomas Nordin]

Robert Thorpe <rt@robertthorpeconsulting.com> writes:

> Emanuel Berg via help-gnu-emacs <help-gnu-emacs@gnu.org> writes:
>
>> Robert Thorpe wrote:
>>
>>> I think is mainly because of the way you use
>>> this mailing list.
>>>
>>> You seem to think of it as something more
>>> than a help mailing list. I think lots of
>>> other people think of it strictly as
>>> a help list.
>>>
>>> I expect that some of your mails would work
>>> better elsewhere. Perhaps on a blog or
>>> on Reddit.
>>
>> Had I known there was no computer culture
>> whatsoever I would never have done any of this
>> to begin with.
>>
>> Taking pride in elitist one-dimensionalness and
>> the prevailing notion that computer people
>> should themselves be and act as computers -
>> it is _in-human_.
>>
>> Or wait... maybe it isn't elitism or even
>> a notion... maybe people really *are* like
>> that? Now had I known _that_, not only would
>> I never have done any of this, I would have
>> turned and run as fast as I possibly could,
>> without ever looking back!
>
> What I'm saying here is about context.
>
> I have nothing against computer culture.  I think it's fairly
> interesting, and I sometimes talk about it too.
>
> The point is though, it's not what *this list* is about.  This list is
> much more focused on help, as it says in the name.  There are other
> forums that are much more loose.
>
> Take Reddit, for example.  Lots of your more off-the-wall mails would be
> appreciated on Emacs Reddit.  That's because it's not a forum focused on
> help.  By posting off-the-wall stuff here you're irritating others and
> also needlessly frustrating yourself.

No irritation observed over here. On the contrary, without the rantings
of Emanuel this list would be OK, but not at all as interesting as it
is with it. Even with some irretation, makes the list more alive.

Best regards
--
Tomas

Re: Is Elisp really that slow?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 797 bytes --]

On Sun, Jun 09, 2019 at 03:37:20PM +0200, Tomas Nordin wrote:
> Robert Thorpe <rt@robertthorpeconsulting.com> writes:

[...]

> > Take Reddit, for example.  Lots of your more off-the-wall mails would be
> > appreciated on Emacs Reddit.  That's because it's not a forum focused on
> > help.  By posting off-the-wall stuff here you're irritating others and
> > also needlessly frustrating yourself.
> 
> No irritation observed over here. On the contrary, without the rantings
> of Emanuel this list would be OK, but not at all as interesting as it
> is with it. Even with some irretation, makes the list more alive.

Seconded. Emanuel sometimes manages to overflow my input buffer, then
I have to flush it (sorry, Emanuel!), but I rather prefer a less sterile
list.

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Is Elisp really that slow?

[Emanuel Berg via help-gnu-emacs]

tomas wrote:

> Seconded. Emanuel sometimes manages to
> overflow my input buffer, then I have to
> flush it (sorry, Emanuel!), but I rather
> prefer a less sterile list.

No apologizes or explanation needed...

But: where have all the people gone that used
to ask questions about Emacs here, often
several, every day?

If people don't do that, nothing good will ever
happen, I'm afraid, and it doesn't matter what
_we_ do.

Either they do that (look for Emacs support)
somewhere else or there _are_ less people
coming to Emacs today than say 10 years ago. (I
say this (10) because this is when came,
roughly, and I think I found this
list/newsgroup all but instantly.)

I have no data to back this up, and I did the
so called ranting even then, perhaps even more,
but even so, I think I was a much, much bigger
help to "the OP" then than I am today :( Which
besides the depressing aspect is an irony, as
then I knew much less!

And, before anyone points it out, I said
a couple of things, people said a couple of
things about newcomers etc etc - and while
those couple of things might be true, or have
some truth to them, who knows, really? -
regardless of whatever, I don't think for
a minute *that* is the explanation!

People don't come here as they used to to
because either they find support somewhere
else, or the user base has decreased. Maybe the
SX site, which (almost) everyone was against
when it appeared, but I said it would be great,
has come back to haunt me - personally - as
altho I was right, I'd never join the
artificial perfection "reputation hunt" myself.
I very much don't like it and consider it
"reducing", if that is idiomatic English?
(Doesn't sound like it.)

I already know my reputation is ambiguous and
I have no problem with that. It's just who
I am. And I don't want to join some competition
and have people reward or punish me or even
edit my stuff if it doesn't conform to the
"computer people should behave like computers"
paradigm. I just want to be me, here as well as
everywhere else!

OK, now I've talked so much about myself in the
last couple of paragraphs that I hope you still
remember what I wrote even earlier, that
I don't think this is about us. Because I don't
think it works like that.

The Third International (the ML, Communist one)
set up an intricate net of militant parties all
over the world. Number of successful
revolutions: 0

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal

Re: Is Elisp really that slow?

[Xavier Maillard]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Sat, 08 Jun 2019 23:44:15 +0200
> 
> tomas@tuxteam.de writes:
> 
> > On Sat, Jun 08, 2019 at 02:00:23PM -0700, Samuel Wales wrote:
> >> On 6/8/19, tomas@tuxteam.de <tomas@tuxteam.de> wrote:
> >> > On Fri, Jun 07, 2019 at 05:26:10PM -0700, Samuel Wales wrote:
> >> >> well, for me as a mere user, i just want elisp [per se] to run faster.
> >> >> maybe that has 1000 definitions, but the 3 things that i have read in
> >> >> this thread wfm as PERFECT targets for speeding up and would make
> >> >> emacs even better for me.
> >> >
> >> > Buy a faster computer, then :-)
> >> 
> >> i was going to add somethning like "without buying a supercomputer"
> >> but figured nobody would say to get a new computer.
> >
> > [...]
> >
> > Yeah, I know. That's why I put a smiley in there. Single-thread speed
> > hasn't advanced for a while (on commodity hardware, at least) and it
> > seems it'll stay so for yet another while.
> 
> Yep, we can't count on the hardware guys anymore to resolve our
> performance problems, we are forced to improve what we have if we wish
> to broaden what our tools can do on terms of data volume and/or
> complexity.

I would say that it is fortunate. I did not buy the argument to let
the hardware optimize all the sucky software.

At least, this is going to be fun and all we need need is skilled
people (which I am not ;) ).

        - xma

GPG:  BA4909B7 8F04DE1B

Re: Is Elisp really that slow?

[Ergus via help-gnu-emacs]

Hi Stefan:

I agree with you about the prefix. After some suggestions in private
(from this same thread) I have been using the composable package [1] and
it works like a charm.

I didn't use it before because I didn't know about it (it is in
melpa). But it really improves the emacs editing experience a lot
without affecting the default behavior. It has the best of modal editing
(the consistency) but without the annoying modes.

There are some missing details, and things that can be improved, but I
see a lot of potential there. Probably this package (or a similar
design/reimplementation) may finish with the arguments about vim's modal
editing vs emacs memorize command and inconsistencies. Because it
provides the best of both worlds. It is compatible with transient and
delete-selection modes.

The same applies to some commands and functions that will be not needed
anymore (or needed to remember) as explained in the project's Readme.

[1] https://github.com/paldepind/composable.el


On Sun, May 19, 2019 at 02:35:17PM -0400, Stefan Monnier wrote:
>>> Vim is consistent.
>> So is Emacs.
>
>In terms of key-bindings, Emacs's scheme is not as regular as VI.
>
>While we can argue that the various VI emulators are just trying to
>provide for those users who like VI, the number of other "alternative
>set of keybindings" (god-mode and several others) out there shows that
>there's a need for something else.
>
>I think one of the main differences (besides the fact that it's modal,
>obviously) is that VI has prefix commands like `d` which Emacs lacks.
>That helps make things regular/orthogonal.
>
>`other-frame-window` is one package that tries to add such a prefix
>command (actually a pair of such) to Emacs and it's not trivial to
>implement.  It'd be good to extend Emacs to provide better support
>for that.
>
>
>        Stefan
>
>

Re: Is Elisp really that slow?

[Óscar Fuentes]

Ergus via help-gnu-emacs <help-gnu-emacs@gnu.org> writes:

> Hi Stefan:
>
> I agree with you about the prefix. After some suggestions in private
> (from this same thread) I have been using the composable package [1] and
> it works like a charm.
>
> I didn't use it before because I didn't know about it (it is in
> melpa). But it really improves the emacs editing experience a lot
> without affecting the default behavior. It has the best of modal editing
> (the consistency) but without the annoying modes.
>
> There are some missing details, and things that can be improved, but I
> see a lot of potential there. Probably this package (or a similar
> design/reimplementation) may finish with the arguments about vim's modal
> editing vs emacs memorize command and inconsistencies. Because it
> provides the best of both worlds. It is compatible with transient and
> delete-selection modes.
>
> The same applies to some commands and functions that will be not needed
> anymore (or needed to remember) as explained in the project's Readme.
>
> [1] https://github.com/paldepind/composable.el

That indeed is an interesting package, but keep in mind that what you
consider "annoying modes" on Vi(m) is the Right Thing for other users.
Specifically, those modes allow to avoid the Control-Meta Emacs hell.

The demographics of Evil's users consists on Vim emigrees looking for a
more powerful Vim, Emacs users who came to think that Vim's model is
more efficient and/or elegant (some of those could find composable.el
interesting) and Emacs veterans trying to alleviate their RSI-related
maladies.

Re: Is Elisp really that slow? (was: Why is Elisp slow?)

[Samuel Wales]

On 6/8/19, Eli Zaretskii <eliz@gnu.org> wrote:
> The literal question of whether Lisp can be sped up was answered
> already: what we have is probably as fast as it can be.  Minor

> That is why there are "out of the box" ideas that float around, like
> JIT byte-compilation, compilation to native code instead of byte code,

to me those are valid attempts to answer the literal question.

Re: Is Elisp really that slow?

[Ergus via help-gnu-emacs]

On June 11, 2019 3:37:00 PM GMT+02:00, "Óscar Fuentes" <ofv@wanadoo.es> wrote:
>Ergus via help-gnu-emacs <help-gnu-emacs@gnu.org> writes:
>
>> Hi Stefan:
>>
>> I agree with you about the prefix. After some suggestions in private
>> (from this same thread) I have been using the composable package [1]
>and
>> it works like a charm.
>>
>> I didn't use it before because I didn't know about it (it is in
>> melpa). But it really improves the emacs editing experience a lot
>> without affecting the default behavior. It has the best of modal
>editing
>> (the consistency) but without the annoying modes.
>>
>> There are some missing details, and things that can be improved, but
>I
>> see a lot of potential there. Probably this package (or a similar
>> design/reimplementation) may finish with the arguments about vim's
>modal
>> editing vs emacs memorize command and inconsistencies. Because it
>> provides the best of both worlds. It is compatible with transient and
>> delete-selection modes.
>>
>> The same applies to some commands and functions that will be not
>needed
>> anymore (or needed to remember) as explained in the project's Readme.
>>
>> [1] https://github.com/paldepind/composable.el
>
>That indeed is an interesting package, but keep in mind that what you
>consider "annoying modes" on Vi(m) is the Right Thing for other users.
>Specifically, those modes allow to avoid the Control-Meta Emacs hell.
>

For users that like modes there is evil and similar modes. I was just referring to an approach to improve-simplify editing and memory usage keeping the modeless behavior.

This is actually my main concern (and complain) about emacs bindings. That the inconsistencies forces to abuse of memory (most commands are bind where there is space and  it's needed to read a documentation to know what the prefix will do) or make very long commands combinations for simple frequent tasks (like copy a whole line or 3 consecutive words)

>The demographics of Evil's users consists on Vim emigrees looking for a
>more powerful Vim, Emacs users who came to think that Vim's model is
>more efficient and/or elegant (some of those could find composable.el
>interesting) and Emacs veterans trying to alleviate their RSI-related
>maladies.

I think composable.el works very fine now but it needs some small improvements in order to make more people happy and provide a 100% standard experience (Display useful information in the modeline for example). Maybe (I have a dream) something similar to composable will be included in emacs (like vile or CUA mode). But this really looks to me like a right approach: it keeps the modeless philosophy, reduces commands to remember and inconsistencies, it is a very original solution/improvement to the key bindings limitation issues, and reduces some binding combinations to the half while keeping the backward compatibility and simplicity.

Re: Is Elisp really that slow?

[Jean Louis]

* Emanuel Berg via help-gnu-emacs <help-gnu-emacs@gnu.org> [2019-05-30 14:09]:
> (require 'dired)
> (defun su-edit ()
>   "Edit the current buffer file as superuser."
>   (interactive)
>   (let*((window-start (window-start))
>         (point        (point))
>         (mark         (when mark-active (mark)))
>         (path         (or dired-directory
>                           (file-truename (buffer-file-name)) ))
>         (sudo-path    (sudo-root-path path)) )
>     (find-alternate-file sudo-path)
>     (when mark (set-mark mark))
>     (goto-char point)
>     (when dired-directory (dired-previous-line 1))
>     (set-window-start nil window-start) ; the selected WINDOW
>     )) ; [1]

let*: Symbol’s function definition is void: sudo-root-path

Then I found out that tramp is offering it with C-x C-f /sudo://etc/asound.conf