New maintainer

New maintainer

[Stefan Monnier]

So, now that I stepped down, we need to find a new maintainer (or a new
maintainer-team).

Don't be afraid: it's a fun job.  Oldtimers take care of a lot of the
work, and it's not like you need to know everything about Emacs's
internals or anything (e.g. after all these years, the redisplay code is
still very much foreign to me).

My experience co-maintaining with Yidong was very good, so I'd
recommend that.

Of course, we'd also welcome people volunteering to take charge of
particular sub-tasks, so as to reduce the overall load of the
maintainer.  E.g. taking care of GNU ELPA.

But we still need a head maintainer, whose task is mostly to keep an eye
on the general direction, can make a final decision when we can't reach
an agreement (of course, we could also delegate that task to
/dev/random), and to be the official contact point with the FSF.

If you're still not sue the job is for you, think about all the side
benefits, such as the fact that you can get as many copies of Emacs as
you want *for free*, and you even get ssh access to fencepost.gnu.org!


        Stefan "I'm not in emacs-devel right now, so keep m in the Cc"

Re: New maintainer

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 1150 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> So, now that I stepped down, we need to find a new maintainer (or a new
> maintainer-team).
>
> Don't be afraid: it's a fun job.  Oldtimers take care of a lot of the
> work, and it's not like you need to know everything about Emacs's
> internals or anything (e.g. after all these years, the redisplay code is
> still very much foreign to me).
>
> My experience co-maintaining with Yidong was very good, so I'd
> recommend that.
>
> Of course, we'd also welcome people volunteering to take charge of
> particular sub-tasks, so as to reduce the overall load of the
> maintainer.  E.g. taking care of GNU ELPA.

Hi Stefan,

As I told you, I would gladly volunteer, but:

- I would only have ~6 hours per week to dedicate to Emacs, which seems
  to be too little considering the task.
- Many parts of the codebase are completely foreign to me (including
  most of the C code).
- Being relatively new, I probably don't have enough experience with the
  community.

That said, I would like to give a hand to the new maintainer(s), so working
on some sub-tasks would be a good fit I think.

Cheers,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

Re: New maintainer

[John Wiegley]

>>>>> Nicolas Petton <nicolas@petton.fr> writes:

> - I would only have ~6 hours per week to dedicate to Emacs, which seems to
> be too little considering the task.

Stefan, can you briefly describe what the time commitments would be for anyone
seeking to become am maintainer?  6 hours a week would also be a bit much for
me as well, so perhaps a little more clarification would be helpful.

John

Re: New maintainer

[Eli Zaretskii]

> From: Nicolas Petton <nicolas@petton.fr>
> Date: Tue, 29 Sep 2015 17:26:11 +0200
> 
> That said, I would like to give a hand to the new maintainer(s), so working
> on some sub-tasks would be a good fit I think.

One task that I think needs attention from as many people as possible
is reviewing the patches and working on bug reports, in the areas
where you consider yourself knowledgeable enough.

TIA

Re: New maintainer

[Aurélien Aptel]

On Thu, Oct 1, 2015 at 8:27 AM, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Nicolas Petton <nicolas@petton.fr>
>> Date: Tue, 29 Sep 2015 17:26:11 +0200
>>
>> That said, I would like to give a hand to the new maintainer(s), so working
>> on some sub-tasks would be a good fit I think.
>
> One task that I think needs attention from as many people as possible
> is reviewing the patches and working on bug reports, in the areas
> where you consider yourself knowledgeable enough.

Like Stefan, I think the debbugs UI could be improved. I really hate
browsing it, I get lost in all the listing, and don't know which bugs
are important.

Re: New maintainer

[Eli Zaretskii]

> Date: Thu, 1 Oct 2015 16:13:31 +0200
> From: Aurélien Aptel <aurelien.aptel+emacs@gmail.com>
> Cc: Nicolas Petton <nicolas@petton.fr>, Stefan Monnier <monnier@iro.umontreal.ca>, 
> 	Emacs development discussions <emacs-devel@gnu.org>
> 
> > One task that I think needs attention from as many people as possible
> > is reviewing the patches and working on bug reports, in the areas
> > where you consider yourself knowledgeable enough.
> 
> Like Stefan, I think the debbugs UI could be improved. I really hate
> browsing it, I get lost in all the listing, and don't know which bugs
> are important.

There's a debbugs package in ELPA, did you try it?

Re: New maintainer

[Dmitry Gutov]

On 10/01/2015 05:13 PM, Aurélien Aptel wrote:

> Like Stefan, I think the debbugs UI could be improved. I really hate
> browsing it, I get lost in all the listing, and don't know which bugs
> are important.

That might be one of the tasks for the new maintainer (or alternatively, 
finding the person willing to do that job).

FWIW, I think almost anything would be better than Debbugs (Bugzilla, 
for instance). But I don't have enough spare time to work on a 
replacement in the near future.

Re: New maintainer

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> FWIW, I think almost anything would be better than Debbugs (Bugzilla, for
> instance). But I don't have enough spare time to work on a replacement in
> the near future.

The only reason I like Bugzilla (which I use for Ledger) is due to the Java UI
Deskzilla, which is freely available for those working on Open Source
projects. It makes working with Bugzilla quick, and working offline is quite
well-done.

Other than that, I've actively used GitHub, Redmine and Trac, and out of all
those, perhaps GitHub is the easiest to work with, despite having the smallest
feature set.

John

Re: New maintainer

[Dmitry Gutov]

On 10/01/2015 09:35 PM, John Wiegley wrote:

> The only reason I like Bugzilla (which I use for Ledger) is due to the Java UI
> Deskzilla, which is freely available for those working on Open Source
> projects. It makes working with Bugzilla quick, and working offline is quite
> well-done.

Last time the discussion of bug trackers came up, certain people stated 
a strong preference for bug trackers that have an Emacs interface (like 
the debbugs package in ELPA). And AFAIK RMS always required that the bug 
tracker could be used entirely over email.

> Other than that, I've actively used GitHub, Redmine and Trac, and out of all
> those, perhaps GitHub is the easiest to work with, despite having the smallest
> feature set.

Yup. It also will never be an option for Emacs.

Though one might look into using GitLab, to be hosted at FSF premises.

Re: New maintainer

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

>> The only reason I like Bugzilla (which I use for Ledger) is due to the Java
>> UI Deskzilla, which is freely available for those working on Open Source
>> projects. It makes working with Bugzilla quick, and working offline is
>> quite well-done.

> Last time the discussion of bug trackers came up, certain people stated a
> strong preference for bug trackers that have an Emacs interface (like the
> debbugs package in ELPA). And AFAIK RMS always required that the bug tracker
> could be used entirely over email.

I can understand that.

>> Other than that, I've actively used GitHub, Redmine and Trac, and out of
>> all those, perhaps GitHub is the easiest to work with, despite having the
>> smallest feature set.

> Yup. It also will never be an option for Emacs.

Yeah, I'll be surprised if they continue to be around as long as Emacs will be
anyway. :)

John

Re: New maintainer

[Barry Warsaw]

[-- Attachment #1: Type: text/plain, Size: 486 bytes --]

On Oct 01, 2015, at 10:18 PM, Dmitry Gutov wrote:

>Last time the discussion of bug trackers came up, certain people stated a
>strong preference for bug trackers that have an Emacs interface (like the
>debbugs package in ELPA). And AFAIK RMS always required that the bug tracker
>could be used entirely over email.

There's also Roundup which is used by the Python project.  It's very email
friendly and has a good web ui too.

http://roundup-tracker.org/

Cheers,
-Barry

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: New maintainer

[Dmitry Gutov]

Hi Barry,

On 10/06/2015 02:37 AM, Barry Warsaw wrote:

> There's also Roundup which is used by the Python project.  It's very email
> friendly and has a good web ui too.
>
> http://roundup-tracker.org/

The feature list seems to check all the boxes, although the web ui seems 
a bit bare-bones in terms of looks, compared to certain other options.

But even so, control over email makes it a serious contender. Not sure 
what are the next steps (guess it's up to the new maintainer(s)), but 
deploying it somewhere we all could take a look would be a good choice. 
Next steps would be doing the same for a few other alternatives, 
creating a migration script, and an Emacs interface for it (probably by 
forking the debbugs package).

Re: New maintainer

[Stefan Monnier]

> FWIW, I think almost anything would be better than Debbugs (Bugzilla, for
> instance).

The grass is always greener on the other side.


        Stefan

Re: New maintainer

[Przemysław Wojnowski]

Maybe it would be a good time to consider creating a Roadmap with vision of
Emacs future and Product Backlog that would prioritize things?

It's just a thought for consideration. I'm not in a position to tell others
what to do.

Cheers,
Przemysław

W dniu 29.09.2015 o 08:28, Stefan Monnier pisze:
> So, now that I stepped down, we need to find a new maintainer (or a new
> maintainer-team).
>
> Don't be afraid: it's a fun job.  Oldtimers take care of a lot of the
> work, and it's not like you need to know everything about Emacs's
> internals or anything (e.g. after all these years, the redisplay code is
> still very much foreign to me).
>
> My experience co-maintaining with Yidong was very good, so I'd
> recommend that.
>
> Of course, we'd also welcome people volunteering to take charge of
> particular sub-tasks, so as to reduce the overall load of the
> maintainer.  E.g. taking care of GNU ELPA.
>
> But we still need a head maintainer, whose task is mostly to keep an eye
> on the general direction, can make a final decision when we can't reach
> an agreement (of course, we could also delegate that task to
> /dev/random), and to be the official contact point with the FSF.
>
> If you're still not sue the job is for you, think about all the side
> benefits, such as the fact that you can get as many copies of Emacs as
> you want *for free*, and you even get ssh access to fencepost.gnu.org!
>
>
>          Stefan "I'm not in emacs-devel right now, so keep m in the Cc"
>

Re: New maintainer

[John Wiegley]

>>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:

> So, now that I stepped down, we need to find a new maintainer (or a new
> maintainer-team).

I'd like to self-nominate for that role, Stefan. I've been contributing to
Emacs since 1994, and have loved it all the while. Emacs Lisp remains a very
enjoyable language to write certain types of code in.

Some things I'd like to see happen to Emacs are more efficiency, closing bugs,
and wider adoption of some of our newest features, like lexical scoping. That
said, I'm also excited by new prospects, and wonder what can be done in the
area of concurrency (in some form), a new language under the hood (Guile?),
etc.

Emacs is my favorite application, by far, and the one I spend the most time
in, both professionally and personally. It's my programming environment,
E-mail reader, IRC client, task manager, note taker, and occasional shell. I'm
hoping it will still be the best choice for these things after twenty _more_
years of use, and perhaps as head maintainer I could help keep things moving
in that direction.

John

Re: New maintainer

[Andreas Röhler]

Am 29.09.2015 um 23:46 schrieb John Wiegley:
>>>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>> So, now that I stepped down, we need to find a new maintainer (or a new
>> maintainer-team).
> I'd like to self-nominate for that role, Stefan. I've been contributing to
> Emacs since 1994, and have loved it all the while. Emacs Lisp remains a very
> enjoyable language to write certain types of code in.
>
> Some things I'd like to see happen to Emacs are more efficiency, closing bugs,
> and wider adoption of some of our newest features, like lexical scoping. That
> said, I'm also excited by new prospects, and wonder what can be done in the
> area of concurrency (in some form), a new language under the hood (Guile?),
> etc.
>
> Emacs is my favorite application, by far, and the one I spend the most time
> in, both professionally and personally. It's my programming environment,
> E-mail reader, IRC client, task manager, note taker, and occasional shell. I'm
> hoping it will still be the best choice for these things after twenty _more_
> years of use, and perhaps as head maintainer I could help keep things moving
> in that direction.
>
> John
>
>

Hi John,

knowing you as author of some really awesome libraries,

not just http://www.ledger-cli.org

would be interesting to see you heading the beast.

BTW your github account is hard to find by your name --sorry being 
stupid--, may you provide it?

Thanks Stefan for all the work done,

Andreas

Re: New maintainer

[Tassilo Horn]

Andreas Röhler <andreas.roehler@online.de> writes:

> BTW your github account is hard to find by your name --sorry being
> stupid--, may you provide it?

http://lmgtfy.com/?q=John+Wiegley+github&l=1

;-)

Bye,
Tassilo

Re: New maintainer

[Xue Fuqiao]

On Thu, Oct 1, 2015 at 2:12 PM, Andreas Röhler
<andreas.roehler@online.de> wrote:
> knowing you as author of some really awesome libraries,

I agree.  John is a very smart Emacs hacker.  IIRC he is the author of
(at least) align.el, cal-bahai.el, eudcb-mab.el, isearchb.el,
pcmpl-cvs.el, pcomplete.el, remember.el, timeclock.el, Eshell, and the
`use-package' macro.  He has also contributed to Org and some other
Emacs packages.  I think most Emacs users have used his code.  Besides
Emacs Lisp, he's also quite familiar with many other languages, such as
Common Lisp, C++, Java, and Haskell.

> BTW your github account is hard to find by your name --sorry being stupid--,
> may you provide it?

It's here: https://github.com/jwiegley

Re: New maintainer

[Mathieu Lirzin]

Xue Fuqiao <xfq.free@gmail.com> writes:

> On Thu, Oct 1, 2015 at 2:12 PM, Andreas Röhler
> <andreas.roehler@online.de> wrote:
>> knowing you as author of some really awesome libraries,
>
> I agree.  John is a very smart Emacs hacker.  IIRC he is the author of
> (at least) align.el, cal-bahai.el, eudcb-mab.el, isearchb.el,
> pcmpl-cvs.el, pcomplete.el, remember.el, timeclock.el, Eshell, and the
> `use-package' macro.  He has also contributed to Org and some other
> Emacs packages.  I think most Emacs users have used his code.  Besides
> Emacs Lisp, he's also quite familiar with many other languages, such as
> Common Lisp, C++, Java, and Haskell.

What about Ethical skills?  I would argue that technical skills are not
sufficient especially for maintaining a major GNU package like Emacs.
Using MacOSX & iOS as main operating systems and Hangout/Skype for
communications, seems incompatible with the role to me.

--
Mathieu Lirzin

Re: New maintainer

[John Wiegley]

>>>>> Mathieu Lirzin <mthl@openmailbox.org> writes:

> What about Ethical skills? I would argue that technical skills are not
> sufficient especially for maintaining a major GNU package like Emacs. Using
> MacOSX & iOS as main operating systems and Hangout/Skype for communications,
> seems incompatible with the role to me.

I also disagree with the spirit of the GPL, vocally in fact. If the requisite
for maintaining Emacs is that one use (GNU/)Linux and espouse the philosophies
of RMS, that is not me.

However: are you looking for someone to act as an arm of the FSF, or do you
want the best possible Emacs?

John

Re: New maintainer

[Yoni Rabkin]

> However: are you looking for someone to act as an arm of the FSF, or
> do you want the best possible Emacs?

You've manufactured that distinction artificially in order to make
people think that the two are mutually exclusive. Please don't do
that. How can it be the best possible GNU/Emacs if it doesn't respect my
freedom?

-- 
   "Cut your own wood and it will warm you twice"

Re: New maintainer

[John Wiegley]

>>>>> Yoni Rabkin <yoni@rabkins.net> writes:

> You've manufactured that distinction artificially in order to make people
> think that the two are mutually exclusive. Please don't do that. How can it
> be the best possible GNU/Emacs if it doesn't respect my freedom?

I have contrasted attending to the politics of the FSF with the technical
needs of Emacs, and asked which one was primary in the search for a
maintainer. That is all.

>>>>> David Kastrup <dak@gnu.org> writes:

> I wouldn't go as far as calling this "ethical skills" but yes, it seems like
> a cultural mismatch that would appear likely to cause considerable friction
> in choosing consistent priorities for ongoing development.

It's possible this is true. I don't seek to change anything about Emacs from a
legal standpoint, but I would be more vocal in wanting better support for non-
GNU platforms, such as OS X.

If this makes me an undesirable candidate, that's quite understandable. But
know that first and foremost I care about *Emacs*, and not the politics of the
FSF. If that's more important than technical considerations, let the
maintainer be chosen accordingly.

John

Re: New maintainer

[Dmitry Gutov]

On 10/02/2015 03:38 AM, John Wiegley wrote:

> It's possible this is true. I don't seek to change anything about Emacs from a
> legal standpoint, but I would be more vocal in wanting better support for non-
> GNU platforms, such as OS X.

Better than GNU platforms? The only relevant "political" hurdle there 
AFAIK is that we don't add features that work on proprietary platforms, 
but don't work (or work considerably worse) on GNU platforms.

Re: New maintainer

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> Better than GNU platforms? The only relevant "political" hurdle there AFAIK
> is that we don't add features that work on proprietary platforms, but don't
> work (or work considerably worse) on GNU platforms.

No, not better than GNU. Just not red-headed step-child worse. :)

I would not want to invest time in features that only a subset of the
community would care to maintain. But those features that should available
everywhere, should function equivalently well on all supported platforms.

John

Re: New maintainer

[Dmitry Gutov]

On 10/02/2015 03:49 AM, John Wiegley wrote:

> I would not want to invest time in features that only a subset of the
> community would care to maintain. But those features that should available
> everywhere, should function equivalently well on all supported platforms.

That shouldn't be a problem. But there's no need to be "more vocal", 
then - just roll up the sleeves and go at it. :)

Re: New maintainer

[David Kastrup]

"John Wiegley" <johnw@newartisans.com> writes:

>>>>>> Yoni Rabkin <yoni@rabkins.net> writes:
>
>> You've manufactured that distinction artificially in order to make people
>> think that the two are mutually exclusive. Please don't do that. How can it
>> be the best possible GNU/Emacs if it doesn't respect my freedom?
>
> I have contrasted attending to the politics of the FSF with the technical
> needs of Emacs, and asked which one was primary in the search for a
> maintainer. That is all.
>
>>>>>> David Kastrup <dak@gnu.org> writes:
>
>> I wouldn't go as far as calling this "ethical skills" but yes, it
>> seems like a cultural mismatch that would appear likely to cause
>> considerable friction in choosing consistent priorities for ongoing
>> development.
>
> It's possible this is true. I don't seek to change anything about
> Emacs from a legal standpoint, but I would be more vocal in wanting
> better support for non- GNU platforms, such as OS X.

You could start by telling Apple to make its terms and conditions for
XCode compatible with creating and distributing software without
requiring a native OSX installation (which is not feasible when
targetting multiple platforms with crosscompilation).

Because that's kind of a bummer.  Where is the point in promoting a
portable compiler chain ike Clang/LLVM if you are precluded from using
it for, well, porting stuff to OSX?

The necessity for maintaining a _natively_ compiled port of OSX is a
really big setback for supporting a diverse set of platforms and leads
to a fragmentation of efforts.

Getting ideological about this saves oneself from sinking an arbitrary
amount of effort in endeavors that are constantly under threats outside
of the developers' control and can be killed at the whim of parties not
interested in free software.

I don't want an Emacs maintainer ultimately banking on the goodwill of
Apple regarding the invested efforts.  That's not an inspiring work
environment.

-- 
David Kastrup

Re: New maintainer

[John Wiegley]

>>>>> David Kastrup <dak@gnu.org> writes:

> Getting ideological about this saves oneself from sinking an arbitrary
> amount of effort in endeavors that are constantly under threats outside of
> the developers' control and can be killed at the whim of parties not
> interested in free software.
>
> I don't want an Emacs maintainer ultimately banking on the goodwill of Apple
> regarding the invested efforts. That's not an inspiring work environment.

I'm bitten by this _regularly_ in the Nix world, so I'm not seeking to tether
anyone to the developer churn happening over at Apple. Just this week I had to
downgrade to XCode 6.4, because 7.0 broke my entire world.

My OS X story for Emacs is mostly this: Give Mitsuharu Yamamoto whatever help
he needs. His "Mac port" variant of Emacs is all I could ask for in terms of
OS X support right now. I'd like to see it modernized under Cocoa, if at all
possible, or promoted to a build flavor in the master branch.

I'm not talking about upheaval here, just equal footing, and paying attention
to OS X (and other platform) gripes as they arise -- like the slow sub-process
bug in Cocoa Emacs that has been a show-stopper for some of us for several
years now.

You'll find I'm upfront with my opinions, but it's not my desire to alienate
*anyone* from participating, joyfully, in making Emacs a great community and
development experience.

John

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > My OS X story for Emacs is mostly this: Give Mitsuharu Yamamoto
  > whatever help he needs. His "Mac port" variant of Emacs is all I
  > could ask for in terms of OS X support right now. I'd like to see
  > it modernized under Cocoa, if at all possible, or promoted to a
  > build flavor in the master branch.

That would go against the goals of the GNU Project, both practically
and in principle.  It would be a very strong commitment to a system
that exemplifies the injustice we aim to get rid of.  It would be
utterly backwards.

  > I'm not talking about upheaval here, just equal footing,

It would be wrong and harmful to give MacOS an "equal footing".
Our goal is to replace nonfree systems (and nonfree software in
general), not to enhance them.

We measure improvement of GNU Emacs in terms of what it can do as part
of the GNU system.  What happens on MacOS or Windows does not count.

See the section "Platforms to Support", in Information for Maintainers
of GNU Software.

We do include support for MacOS and Windows, to the extent people
develop such support and it isn't a problem to include; but we reject
any obligation to support them.  Making that an obligation would
legitimize those systems -- and go against our goal, which is to beat
them -- and divert effort to something that doesn't count.

If the Emacs maintainers rejected features that work only on GNU-like
systems, saying "You must add support for Windows and MacOS before we
can install this," that would pressure our contributors to use
proprietary systems (it is unethical even to suggest people use
them!).  That requirement would hold back contribution from developers
that don't use them.  It would thus impede the improvement of GNU
Emacs (which means, making it function better in GNU).

Thus, it is unacceptable to require Windows or MacOS support before
installing contributions.  A contribution only HAS to work on GNU (but
it should be conditionalized so it does not break Emacs on the other
platforms it doesn't support), but we should try to keep it working on
*BSD since that is usually easy.  As for Windows and MacOS support, we
can integrate that if and when someone provides it.

There is nothing wrong with an Emacs maintainer's writing code to
support for Windows or MacOS.  However, if the maintainers have
limited time for Emacs, spending much time supporting secondary
platforms could leave the Emacs maintainers' main job starved for
time.  That would be a practical problem, if it happens.  Perhaps
it won't happen.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Andreas Röhler]

Am 03.10.2015 um 03:37 schrieb Richard Stallman:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>    > My OS X story for Emacs is mostly this: Give Mitsuharu Yamamoto
>    > whatever help he needs. His "Mac port" variant of Emacs is all I
>    > could ask for in terms of OS X support right now. I'd like to see
>    > it modernized under Cocoa, if at all possible, or promoted to a
>    > build flavor in the master branch.
>
> That would go against the goals of the GNU Project, both practically
> and in principle.  It would be a very strong commitment to a system
> that exemplifies the injustice we aim to get rid of.  It would be
> utterly backwards.
>
>    > I'm not talking about upheaval here, just equal footing,
>
> It would be wrong and harmful to give MacOS an "equal footing".
> Our goal is to replace nonfree systems (and nonfree software in
> general), not to enhance them.
>
> We measure improvement of GNU Emacs in terms of what it can do as part
> of the GNU system.  What happens on MacOS or Windows does not count.
>
> See the section "Platforms to Support", in Information for Maintainers
> of GNU Software.
>
> We do include support for MacOS and Windows, to the extent people
> develop such support and it isn't a problem to include; but we reject
> any obligation to support them.  Making that an obligation would
> legitimize those systems -- and go against our goal, which is to beat
> them -- and divert effort to something that doesn't count.
>
> If the Emacs maintainers rejected features that work only on GNU-like
> systems, saying "You must add support for Windows and MacOS before we
> can install this," that would pressure our contributors to use
> proprietary systems (it is unethical even to suggest people use
> them!).  That requirement would hold back contribution from developers
> that don't use them.  It would thus impede the improvement of GNU
> Emacs (which means, making it function better in GNU).
>
> Thus, it is unacceptable to require Windows or MacOS support before
> installing contributions.  A contribution only HAS to work on GNU (but
> it should be conditionalized so it does not break Emacs on the other
> platforms it doesn't support), but we should try to keep it working on
> *BSD since that is usually easy.  As for Windows and MacOS support, we
> can integrate that if and when someone provides it.
>
> There is nothing wrong with an Emacs maintainer's writing code to
> support for Windows or MacOS.  However, if the maintainers have
> limited time for Emacs, spending much time supporting secondary
> platforms could leave the Emacs maintainers' main job starved for
> time.  That would be a practical problem, if it happens.  Perhaps
> it won't happen.
>

Hi Richard,

+1

Thanks for clarification.

Any response from Eli WRT acting as co-maintainer?

Would be great to see both in office ASAP.

Cheers,

Andreas

Re: New maintainer

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> We measure improvement of GNU Emacs in terms of what it can do as part of
> the GNU system. What happens on MacOS or Windows does not count.

I can see how this provides clear focus for the FSF in supporting Emacs, no
argument here.

> We do include support for MacOS and Windows, to the extent people develop
> such support and it isn't a problem to include; but we reject any obligation
> to support them. Making that an obligation would legitimize those systems --
> and go against our goal, which is to beat them -- and divert effort to
> something that doesn't count.

OK, as long as the FSF isn't actively getting in the way of such support, to
intentionally cripple those systems to make GNU seem better. GNU should *be*
better if it is to succeed, not position itself as the best of the worst.

> If the Emacs maintainers rejected features that work only on GNU-like
> systems, saying "You must add support for Windows and MacOS before we can
> install this," that would pressure our contributors to use proprietary
> systems (it is unethical even to suggest people use them!). That requirement
> would hold back contribution from developers that don't use them. It would
> thus impede the improvement of GNU Emacs (which means, making it function
> better in GNU).

I don't recognize your authority to tell me what is and is not ethical,
Richard, and think you should stop using words like "injustice" and
"inethical" in your presentations. Not everyone agrees with you, so calling
them wrong to paint yourself as right does little service to your cause.

If you present the benefits and virtue of GNU-like systems, it gives weight to
your message. But standing out by demonizing opponents is a horse that
politicians have beat to death, and has never, ever led to lasting success.

> Thus, it is unacceptable to require Windows or MacOS support before
> installing contributions. A contribution only HAS to work on GNU (but it
> should be conditionalized so it does not break Emacs on the other platforms
> it doesn't support), but we should try to keep it working on *BSD since that
> is usually easy. As for Windows and MacOS support, we can integrate that if
> and when someone provides it.
>
> There is nothing wrong with an Emacs maintainer's writing code to support
> for Windows or MacOS. However, if the maintainers have limited time for
> Emacs, spending much time supporting secondary platforms could leave the
> Emacs maintainers' main job starved for time. That would be a practical
> problem, if it happens. Perhaps it won't happen.

Again, I can't argue with this as FSF policy. Only that, since I use OS X
daily, this is the Emacs I experience, and it colors my perception of how well
Emacs is doing. If the experience is great on GNU systems, but awful on OS X,
I'll see this as meaning that changes needing to be made. Conversely, I won't
notice degradation on GNU systems owing to cross-platform changes, and would
require those using such systems to inform me.

So there are probably much better candidates than I for achieving the best
"GNU Emacs", instead of the broader definition of "Emacs for all supported
platforms" that I have in mind. Note too that the latter is very likely what
most Emacs users in the world think of when they think of a "better Emacs"; I
imagine only a subset of those using Emacs do so to promote GNU more than to
edit files.

Eli Zaretskii wrote:

> So, given this seemingly unsolvable contradiction, what do you, the crowd,
> expect of the new maintainer? What "job description", in addition to what
> Richard wrote, would you propose if you were tasked with the job of finding
> the candidates? E.g., how many hours a week should that person be available
> for working on Emacs? Which talents and personal traits should he or she
> possess? Etc. etc.

I think this is a great next step. We need a proper "job description",
including objectives, responsibilities, expectations, and what should define a
successful tenure.

John

Re: New maintainer

[David Kastrup]

"John Wiegley" <johnw@newartisans.com> writes:

> I don't recognize your authority to tell me what is and is not
> ethical, Richard, and think you should stop using words like
> "injustice" and "inethical" in your presentations. Not everyone agrees
> with you, so calling them wrong to paint yourself as right does little
> service to your cause.
>
> If you present the benefits and virtue of GNU-like systems, it gives
> weight to your message. But standing out by demonizing opponents is a
> horse that politicians have beat to death, and has never, ever led to
> lasting success.

Uh, it's the fundamental tenet of the GNU project.  Free Software's
value does not rest on relative benefits and virtues but on the
fundamental software freedoms.  The stance of the FSF fundamentally
differs from that of the "Open Source" software camp which tries to
paint Free Software as a superior development model, consequently giving
a nod to proprietary software as long as it is successful.

With that stance, Free Software would never have gotten off the ground
since it by necessity started inferior to existing proprietary
solutions.

The whole point of GNU is the non-acceptance of software denying the
users the fundamental software freedoms.  This constitutes a moral
judgment and as such is indistinguishable from "demonizing opponents" or
at the very least damning their actions.

So it would be pointless to ask for change on that front.  It's what
makes this show ultimately tick.

-- 
David Kastrup

Re: New maintainer

[John Wiegley]

>>>>> David Kastrup <dak@gnu.org> writes:

> Uh, it's the fundamental tenet of the GNU project. Free Software's value
> does not rest on relative benefits and virtues but on the fundamental
> software freedoms. The stance of the FSF fundamentally differs from that of
> the "Open Source" software camp which tries to paint Free Software as a
> superior development model, consequently giving a nod to proprietary
> software as long as it is successful.

Thanks for clarifying that, David, I very much appreciate your tone of
presentation.

> The whole point of GNU is the non-acceptance of software denying the users
> the fundamental software freedoms. This constitutes a moral judgment and as
> such is indistinguishable from "demonizing opponents" or at the very least
> damning their actions.

Then I respectfully withdraw myself as a candidate for maintainer. Damning by
implication is one thing; setting out to defame other organizations in order
to make one's own appear the standard of virtue is something else entirely,
and I do not wish to be associated with such methods.

Thanks to all for their supporting words and encouragement, and to the FSF for
having this frank and open discussion with me on the issues that matter.

John

Re: New maintainer

[David Kastrup]

"John Wiegley" <johnw@newartisans.com> writes:

>>>>>> David Kastrup <dak@gnu.org> writes:
>
>> Uh, it's the fundamental tenet of the GNU project. Free Software's value
>> does not rest on relative benefits and virtues but on the fundamental
>> software freedoms. The stance of the FSF fundamentally differs from that of
>> the "Open Source" software camp which tries to paint Free Software as a
>> superior development model, consequently giving a nod to proprietary
>> software as long as it is successful.
>
> Thanks for clarifying that, David, I very much appreciate your tone of
> presentation.
>
>> The whole point of GNU is the non-acceptance of software denying the users
>> the fundamental software freedoms. This constitutes a moral judgment and as
>> such is indistinguishable from "demonizing opponents" or at the very least
>> damning their actions.
>
> Then I respectfully withdraw myself as a candidate for
> maintainer. Damning by implication is one thing; setting out to defame
> other organizations in order to make one's own appear the standard of
> virtue is something else entirely, and I do not wish to be associated
> with such methods.
>
> Thanks to all for their supporting words and encouragement, and to the
> FSF for having this frank and open discussion with me on the issues
> that matter.

Let me make clear here that I do not speak for the FSF and am neither a
member nor a representative of it.  So nothing I say constitutes any
official statement by the FSF.  I merely spent enough time discussing
such matters that I believe myself to rarely end up far off the mark.

-- 
David Kastrup

Re: New maintainer

[Andreas Röhler]

Am 03.10.2015 um 21:26 schrieb John Wiegley:
>>>>>> David Kastrup <dak@gnu.org> writes:
>> Uh, it's the fundamental tenet of the GNU project. Free Software's value
>> does not rest on relative benefits and virtues but on the fundamental
>> software freedoms. The stance of the FSF fundamentally differs from that of
>> the "Open Source" software camp which tries to paint Free Software as a
>> superior development model, consequently giving a nod to proprietary
>> software as long as it is successful.
> Thanks for clarifying that, David, I very much appreciate your tone of
> presentation.
>
>> The whole point of GNU is the non-acceptance of software denying the users
>> the fundamental software freedoms. This constitutes a moral judgment and as
>> such is indistinguishable from "demonizing opponents" or at the very least
>> damning their actions.
> Then I respectfully withdraw myself as a candidate for maintainer. Damning by
> implication is one thing; setting out to defame other organizations in order
> to make one's own appear the standard of virtue is something else entirely,
> and I do not wish to be associated with such methods.
>
> Thanks to all for their supporting words and encouragement, and to the FSF for
> having this frank and open discussion with me on the issues that matter.
>
> John
>
>


Don't think a moral is 'indistinguishable from demonizing opponents", as 
David writes. That's a misguided pseudo-religous approach. Also AFAIK 
it's not the declared FSF policy.

Cheers,

Andreas

Re: New maintainer

[David Kastrup]

Andreas Röhler <andreas.roehler@online.de> writes:

> Am 03.10.2015 um 21:26 schrieb John Wiegley:
>>>>>>> David Kastrup <dak@gnu.org> writes:
>>
>>> The whole point of GNU is the non-acceptance of software denying the
>>> users the fundamental software freedoms. This constitutes a moral
>>> judgment and as such is indistinguishable from "demonizing
>>> opponents" or at the very least damning their actions.

>> Then I respectfully withdraw myself as a candidate for
>> maintainer. Damning by implication is one thing; setting out to
>> defame other organizations in order to make one's own appear the
>> standard of virtue is something else entirely,

And not at all what I have been saying.

>> and I do not wish to be associated with such methods.
>>
>> Thanks to all for their supporting words and encouragement, and to
>> the FSF for having this frank and open discussion with me on the
>> issues that matter.
>
> Don't think a moral is 'indistinguishable from demonizing opponents",
> as David writes. That's a misguided pseudo-religous approach. Also
> AFAIK it's not the declared FSF policy.

<URL:https://www.gnu.org/philosophy/free-software-even-more-important.html>

    Since 1983, the Free Software Movement has campaigned for computer
    users' freedom—for users to control the software they use, rather
    than vice versa. When a program respects users' freedom and
    community, we call it “free software.”

    We also sometimes call it “libre software” to emphasize that we're
    talking about liberty, not price. Some proprietary (nonfree)
    programs, such as Photoshop, are very expensive; others, such as
    Flash Player, are available gratis—but that's a minor detail. Either
    way, they give the program's developer power over the users, power
    that no one should have.

    Those two nonfree programs have something else in common: they are
    both malware. That is, both have functionalities designed to
    mistreat the user. Proprietary software nowadays is often malware
    because the developers' power corrupts them.

    With free software, the users control the program, both individually
    and collectively. So they control what their computers do (assuming
    those computers are loyal and do what the users' programs tell them
    to do).

    With proprietary software, the program controls the users, and some
    other entity (the developer or “owner”) controls the program. So the
    proprietary program gives its developer power over its users. That
    is unjust in itself, and tempts the developer to mistreat the users
    in other ways.

I don't think that I am wide off the mark with regard to the statement
I actually made rather than John's interpretation of it.

-- 
David Kastrup

Re: New maintainer

[John Wiegley]

>>>>> David Kastrup <dak@gnu.org> writes:

>>> Damning by implication is one thing; setting out to defame other
>>> organizations in order to make one's own appear the standard of virtue is
>>> something else entirely,

> And not at all what I have been saying.

Maybe not what you've been saying here, but what the FSF has been doing. The
way RMS talks about Apple and other companies, using words like unethical,
corrupted, evil; I can't get behind that sort of rhetoric.

But I knew I had differences with RMS before this, so that's not changed. I
also know that I love Emacs and want to help it remain my editor of choice for
a long time to come. If I can serve Emacs independently, from RMS and his
agenda, I'm happy to do so, whether it benefits the FSF's cause or not.

What RMS may not know is that I respect his spirit, the intentions behind his
actions, and the effect he has had on our world as a whole. Free software in
general (if not "libre" software) is without a doubt more real today because
of him, and I've directly benefited from those efforts, both personally and
professionally. So the FSF always has some credit in my bank, so to speak. If
GNU/Linux could be an awesome OS with awesome applications, I wouldn't use my
OS X machine anymore.

What bothers me are the socialistic aspects involved, the _way_ RMS demonizes
other approaches to licensing (eerily similar to how socialism demonizes other
political philosophies), and the way he talks about "freedom" -- while what is
meant is specifically the freedom of the user/consumer, and not the freedom of
the developer/producer. These are reasons I do not support his path as leading
toward true liberty. I _do_ want to true liberty, and for all software to be
as free as RMS could ever want to to be; I just don't believe in getting there
this way.

John

Re: New maintainer

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 422 bytes --]

On Sat, Oct 3, 2015 at 10:44 PM, John Wiegley <johnw@newartisans.com> wrote:

> What bothers me are the socialistic aspects involved, the _way_ RMS
demonizes
> other approaches to licensing (eerily similar to how socialism demonizes
other
> political philosophies),

Perhaps we should leave non-software-related politics out of this
discussion. Reading "socialistic" as a pejorative could be a bit jarring
for some of us.

[-- Attachment #2: Type: text/html, Size: 571 bytes --]

Re: New maintainer

[John Wiegley]

>>>>> Juanma Barranquero <lekktu@gmail.com> writes:

> Perhaps we should leave non-software-related politics out of this
> discussion. Reading "socialistic" as a pejorative could be a bit jarring for
> some of us.

Agreed, Juanma, with my apologies. I had meant to imply certain historical
experiments with negative outcomes, but not to erase the positive aspects of
socialism altogether. I'll refrain from such examples in the future.

John

Re: New maintainer

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Sat, 3 Oct 2015 23:49:16 +0200
> 
> On Sat, Oct 3, 2015 at 10:44 PM, John Wiegley <johnw@newartisans.com> wrote:
> 
> > What bothers me are the socialistic aspects involved, the _way_ RMS demonizes
> > other approaches to licensing (eerily similar to how socialism demonizes
> other
> > political philosophies),
> 
> Perhaps we should leave non-software-related politics out of this discussion.

I agree.

Please also keep in mind that AFAIK a possibility exists to more or
less split these two parts, by nominating a "steering committee" for
the project, which will act as the FSF representative and deal with
most of these issues.  Several projects (GCC and GDB, to give just 2
examples) already have that, although the split is not very strong
there (because there's no need, IIUC).

Richard, please correct me if I'm wrong in my understanding.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I say that nonfree software licenses are an injustice, but that is not
"demonizing".  I focus criticism mainly on the programs rather than
the people that develop them.

As regards non-copyleft free licenses, I say they are ethically
acceptible but weak.  A free program under a pushover license is
better than no free program at all.  This is not demonization; it is
praise with caveat.  It is not glowing praise, but nonetheless praise.
See http://gnu.org/licenses/license-list.html.

See http://gnu.org/copyleft/ for more explanation of this issue.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Jean-Christophe Helary]

> On Oct 4, 2015, at 05:44, John Wiegley <johnw@newartisans.com> wrote:

> What bothers me are the socialistic aspects involved, the _way_ RMS demonizes
> other approaches to licensing (eerily similar to how socialism demonizes other
> political philosophies),

You've not been watching the news in the past 100 years. Socialism is the philosophy that has been the most demonised. Your comment only shows a gross misunderstanding of political history and of the aspirations of the people who gave their lives so that more people could achieve freedom from economic serfdom regardless of what self-proclaimed tyrants did or said. We're not on Fox News here are we?

Jean-Christophe Helary

Re: New maintainer

[John Wiegley]

>>>>> Jean-Christophe Helary <jean.christophe.helary@gmail.com> writes:

> You've not been watching the news in the past 100 years. Socialism is the
> philosophy that has been the most demonised. Your comment only shows a gross
> misunderstanding of political history and of the aspirations of the people
> who gave their lives so that more people could achieve freedom from economic
> serfdom regardless of what self-proclaimed tyrants did or said. We're not on
> Fox News here are we?

Not even sure what to say to that Jean-Christophe, except that here is not the
place. I fully retract my own demonization of socialism per se, so let's back
away from this particular cliff...

John

Re: New maintainer

[Lennart Borgman]

[-- Attachment #1: Type: text/plain, Size: 1239 bytes --]

On Sun, Oct 4, 2015 at 12:53 AM, John Wiegley <johnw@newartisans.com> wrote:

> >>>>> Jean-Christophe Helary <jean.christophe.helary@gmail.com> writes:
>
> > You've not been watching the news in the past 100 years. Socialism is the
> > philosophy that has been the most demonised. Your comment only shows a
> gross
> > misunderstanding of political history and of the aspirations of the
> people
> > who gave their lives so that more people could achieve freedom from
> economic
> > serfdom regardless of what self-proclaimed tyrants did or said. We're
> not on
> > Fox News here are we?
>
> Not even sure what to say to that Jean-Christophe, except that here is not
> the
> place. I fully retract my own demonization of socialism per se, so let's
> back
> away from this particular cliff...
>
> John
>
>
When developing software together with other people I have always found it
saves a lot of time to focus on the intention with the code, what it is
supposed to do. I usually only look at the details if I have too. ;-)

Maybe discussions could learn a bit from this? (Even if they are very
different. We can't run tests for example. But we can take the discussions
further by trying to find the positive aspects of other peoples view.)

[-- Attachment #2: Type: text/html, Size: 2262 bytes --]

Re: New maintainer

[Andreas Röhler]

Am 03.10.2015 um 22:10 schrieb David Kastrup:
> Andreas Röhler <andreas.roehler@online.de> writes:
>
>> Am 03.10.2015 um 21:26 schrieb John Wiegley:
>>>>>>>> David Kastrup <dak@gnu.org> writes:
>>>> The whole point of GNU is the non-acceptance of software denying the
>>>> users the fundamental software freedoms. This constitutes a moral
>>>> judgment and as such is indistinguishable from "demonizing
>>>> opponents" or at the very least damning their actions.
>>> Then I respectfully withdraw myself as a candidate for
>>> maintainer. Damning by implication is one thing; setting out to
>>> defame other organizations in order to make one's own appear the
>>> standard of virtue is something else entirely,
> And not at all what I have been saying.
>
>>> and I do not wish to be associated with such methods.
>>>
>>> Thanks to all for their supporting words and encouragement, and to
>>> the FSF for having this frank and open discussion with me on the
>>> issues that matter.
>> Don't think a moral is 'indistinguishable from demonizing opponents",
>> as David writes. That's a misguided pseudo-religous approach. Also
>> AFAIK it's not the declared FSF policy.
> <URL:https://www.gnu.org/philosophy/free-software-even-more-important.html>
>
>      Since 1983, the Free Software Movement has campaigned for computer
>      users' freedom—for users to control the software they use, rather
>      than vice versa. When a program respects users' freedom and
>      community, we call it “free software.”
>
>      We also sometimes call it “libre software” to emphasize that we're
>      talking about liberty, not price. Some proprietary (nonfree)
>      programs, such as Photoshop, are very expensive; others, such as
>      Flash Player, are available gratis—but that's a minor detail. Either
>      way, they give the program's developer power over the users, power
>      that no one should have.
>
>      Those two nonfree programs have something else in common: they are
>      both malware. That is, both have functionalities designed to
>      mistreat the user. Proprietary software nowadays is often malware
>      because the developers' power corrupts them.
>
>      With free software, the users control the program, both individually
>      and collectively. So they control what their computers do (assuming
>      those computers are loyal and do what the users' programs tell them
>      to do).
>
>      With proprietary software, the program controls the users, and some
>      other entity (the developer or “owner”) controls the program. So the
>      proprietary program gives its developer power over its users. That
>      is unjust in itself, and tempts the developer to mistreat the users
>      in other ways.
>
> I don't think that I am wide off the mark with regard to the statement
> I actually made rather than John's interpretation of it.
>

Sorry, can't read anything which justifies or encourages "demonizing 
opponents" or "at the very least damning their actions." Consider your 
conclusion not just wrong but contra-productive. The liberation effort 
of the soviets died from these kind of treatment.

Re: New maintainer

[David Kastrup]

Andreas Röhler <andreas.roehler@online.de> writes:

> Am 03.10.2015 um 22:10 schrieb David Kastrup:
>> Andreas Röhler <andreas.roehler@online.de> writes:
>>
>>> Am 03.10.2015 um 21:26 schrieb John Wiegley:
>>>>>>>>> David Kastrup <dak@gnu.org> writes:
>>>>> The whole point of GNU is the non-acceptance of software denying the
>>>>> users the fundamental software freedoms. This constitutes a moral
>>>>> judgment and as such is indistinguishable from "demonizing
>>>>> opponents" or at the very least damning their actions.
>>>> Then I respectfully withdraw myself as a candidate for
>>>> maintainer. Damning by implication is one thing; setting out to
>>>> defame other organizations in order to make one's own appear the
>>>> standard of virtue is something else entirely,
>> And not at all what I have been saying.
>>
>>>> and I do not wish to be associated with such methods.
>>>>
>>>> Thanks to all for their supporting words and encouragement, and to
>>>> the FSF for having this frank and open discussion with me on the
>>>> issues that matter.
>>> Don't think a moral is 'indistinguishable from demonizing opponents",
>>> as David writes. That's a misguided pseudo-religous approach. Also
>>> AFAIK it's not the declared FSF policy.
>> <URL:https://www.gnu.org/philosophy/free-software-even-more-important.html>
>>
>>      Since 1983, the Free Software Movement has campaigned for computer
>>      users' freedom—for users to control the software they use, rather
>>      than vice versa. When a program respects users' freedom and
>>      community, we call it “free software.”
>>
>>      We also sometimes call it “libre software” to emphasize that we're
>>      talking about liberty, not price. Some proprietary (nonfree)
>>      programs, such as Photoshop, are very expensive; others, such as
>>      Flash Player, are available gratis—but that's a minor detail. Either
>>      way, they give the program's developer power over the users, power
>>      that no one should have.
>>
>>      Those two nonfree programs have something else in common: they are
>>      both malware. That is, both have functionalities designed to
>>      mistreat the user. Proprietary software nowadays is often malware
>>      because the developers' power corrupts them.
>>
>>      With free software, the users control the program, both individually
>>      and collectively. So they control what their computers do (assuming
>>      those computers are loyal and do what the users' programs tell them
>>      to do).
>>
>>      With proprietary software, the program controls the users, and some
>>      other entity (the developer or “owner”) controls the program. So the
>>      proprietary program gives its developer power over its users. That
>>      is unjust in itself, and tempts the developer to mistreat the users
>>      in other ways.
>>
>> I don't think that I am wide off the mark with regard to the statement
>> I actually made rather than John's interpretation of it.
>>
>
> Sorry, can't read anything which justifies or encourages "demonizing
> opponents" or "at the very least damning their actions." Consider your
> conclusion not just wrong but contra-productive. The liberation effort
> of the soviets died from these kind of treatment.

Shrug.  I was quoting John's choice of words.  He stated:

    I don't recognize your authority to tell me what is and is not
    ethical, Richard, and think you should stop using words like
    "injustice" and "inethical" in your presentations. Not everyone
    agrees with you, so calling them wrong to paint yourself as right
    does little service to your cause.

    If you present the benefits and virtue of GNU-like systems, it gives
    weight to your message. But standing out by demonizing opponents is
    a horse that politicians have beat to death, and has never, ever led
    to lasting success.

In the end, it will be up to him to decide whether the paragraphs
I quoted are what he calls "demonizing opponents".  I considered it
likely that they were of the kind he was talking about and stated the
reasons for which I considered it unrealistic of John to expect any
change in that regard.

-- 
David Kastrup

Re: New maintainer

[John Wiegley]

>>>>> John Wiegley <johnw@newartisans.com> writes:

> Then I respectfully withdraw myself as a candidate for maintainer. Damning
> by implication is one thing; setting out to defame other organizations in
> order to make one's own appear the standard of virtue is something else
> entirely, and I do not wish to be associated with such methods.

It's been pointed out off-list that I was a bit too hasty in responding to
David's comments as if he represented the FSF in any way.

So I will withdraw my withdrawal, and say that if the opinions of the FSF, and
the maintenance of Emacs by someone like myself, can co-exist without
involving ethical compromise by either one of us, I'm happy for us to work
together.

The most important thing, even beyond the technical refinement of Emacs, is
the fostering of a supportive and active user community. Perhaps this means a
collaboritive maintainship, as suggested by Eli, though I'm still not sure
that is the wisest course of action. Happy to debate that in another thread,
though, if multiple maintainers should be chosen. If they are people I can
work with, then I'm certain a happy outcome can be found. If not, that alone
is proof we shouldn't be trying to lead together.

John

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > So I will withdraw my withdrawal, and say that if the opinions of the FSF, and
  > the maintenance of Emacs by someone like myself, can co-exist without
  > involving ethical compromise by either one of us, I'm happy for us to work
  > together.

I hope that we can do do that, because you would bring great ability
to the job.

We have a deep disagreement, and neither of us is likely to change
position on the issue.  However, that disagreement does not
necessarily mean you can't be a maintainer, because all the GNU
Project really needs from a maintainer is to engage to carry out the
maintainer's responsibility.  So it is comes down to your decision
about that.

In practice, the parts of the responsibility that you'd need to
reconcile with your views are (1) accepting changes that work only on
GNU or GNU-like systems without reluctance, so as not to hold back the
advance of Emacs on those systems with contributions from people who
may not use or care about Windows or MacOS, and (2) and cooperating
with the things we do to present the GNU Project's positions in the
Emacs distribution itself (for instance, some material in etc and
doc).

I think it would be best to have two maintainers or three maintainers.
Stefan and Yidong worked well together as a team.  The fact that they
were two rather than three didn't hinder them from deciding.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> We have a deep disagreement, and neither of us is likely to change position
> on the issue. However, that disagreement does not necessarily mean you can't
> be a maintainer, because all the GNU Project really needs from a maintainer
> is to engage to carry out the maintainer's responsibility. So it is comes
> down to your decision about that.

> In practice, the parts of the responsibility that you'd need to reconcile
> with your views are (1) accepting changes that work only on GNU or GNU-like
> systems without reluctance, so as not to hold back the advance of Emacs on
> those systems with contributions from people who may not use or care about
> Windows or MacOS, and (2) and cooperating with the things we do to present
> the GNU Project's positions in the Emacs distribution itself (for instance,
> some material in etc and doc).

Thanks, Richard, that's really quite reasonable.

There is one technical matter that's been brought to my attention that I would
like to get your thoughts on:

In Emacs' future, I'd like to improve the way that external processes are used
to inform Emacs about the syntax and semantics of buffer contents. For
example, building completion lists, indexing code locations, looking up or
rendering documentation, etc.

I would like to achieve this in a general way, such that both GPL and non-GPL
software can be freely chosen by the Emacs user. For example, in C++ mode, one
could use either GCC or Clang (or both) to determine type information. It's
fine with me if only GCC support is actively endorsed, so long as the Clang
solution is equally feasible -- provided volunteers want to contribute to it,
and without detracting from the GCC support.

Some have told me off-list that you might be reluctant to allow even the
possibility of Clang being used in this scenario, since it might encourage use
of non-GNU software should its support prove superior. Since that doesn't seem
quite in keeping with the rational tone I read in your e-mail, I wanted to
check with you directly.

I can appreciate wanting Emacs to be a flagship GNU project, and to ensure
that GNU-favored technologies are not in any way damaged by supporting non-GNU
solutions. What I wonder is, if support for either option can be made purely a
matter of the user's choosing, without affecting Emacs' ability to make use of
GNU solutions, would this be opposed?

John

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I would like to achieve this in a general way, such that both GPL and non-GPL
  > software can be freely chosen by the Emacs user. For example, in C++ mode, one
  > could use either GCC or Clang (or both) to determine type information. It's
  > fine with me if only GCC support is actively endorsed, so long as the Clang
  > solution is equally feasible -- provided volunteers want to contribute to it,
  > and without detracting from the GCC support.

If it really works usefully with GCC -- if that is not just a
theoretical idea -- then I won't object to its supporting other
compilers as well.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> If it really works usefully with GCC -- if that is not just a theoretical
> idea -- then I won't object to its supporting other compilers as well.

That's great to hear, Richard.

Any solution we choose should never preclude the opportunity for GCC to
outshine other choices. The main thing is that it is GCC's responsibility to
be better, and not Emacs' to prevent better options from being chosen, simply
to accommodate a lack of progress by GCC.

John

Re: New maintainer

[Dmitry Gutov]

On 10/05/2015 10:20 PM, John Wiegley wrote:

> Any solution we choose should never preclude the opportunity for GCC to
> outshine other choices. The main thing is that it is GCC's responsibility to
> be better, and not Emacs' to prevent better options from being chosen, simply
> to accommodate a lack of progress by GCC.

I'm assuming that's exactly what Richard meant by "theoretical idea". At 
least, it's how similar discussions went in the past.

Re: New maintainer

[Jay Belanger]

> If it really works usefully with GCC -- if that is not just a theoretical
> idea -- then I won't object to its supporting other compilers as well.

> On 10/05/2015 10:20 PM, John Wiegley wrote:

>> Any solution we choose should never preclude the opportunity for GCC to
>> outshine other choices. The main thing is that it is GCC's responsibility to
>> be better, and not Emacs' to prevent better options from being chosen, simply
>> to accommodate a lack of progress by GCC.

Dmitry Gutov <dgutov@yandex.ru> writes:

> I'm assuming that's exactly what Richard meant by "theoretical
> idea". At least, it's how similar discussions went in the past.

Maybe I'm misreading it, but it doesn't sound like what Richard meant at
all.  I read it as the features have to actually work with GCC ("not just a
theoretical idea") to be included.

Re: New maintainer

[Dmitry Gutov]

On 10/05/2015 10:31 PM, Jay Belanger wrote:

> Maybe I'm misreading it, but it doesn't sound like what Richard meant at
> all.  I read it as the features have to actually work with GCC ("not just a
> theoretical idea") to be included.

The issue is that GCC, in its current state, doesn't provide a certain 
set a features Emacs can take advantage of, that Clang does.

Re: New maintainer

[Jay Belanger]

>> Maybe I'm misreading it, but it doesn't sound like what Richard meant at
>> all.  I read it as the features have to actually work with GCC ("not just a
>> theoretical idea") to be included.
>
> The issue is that GCC, in its current state, doesn't provide a certain
> set a features Emacs can take advantage of, that Clang does.

That's the point.  It sounded like Richard was saying that in that case,
Emacs shouldn't take advantage of it.  How else could

> If it really works usefully with GCC -- if that is not just a theoretical
> idea -- then I won't object to its supporting other compilers as well.

be interpreted?

Re: New maintainer

[John Wiegley]

>>>>> Jay Belanger <jay.p.belanger@gmail.com> writes:

>>> Maybe I'm misreading it, but it doesn't sound like what Richard meant at
>>> all. I read it as the features have to actually work with GCC ("not just a
>>> theoretical idea") to be included.
>> 
>> The issue is that GCC, in its current state, doesn't provide a certain
>> set a features Emacs can take advantage of, that Clang does.

> That's the point. It sounded like Richard was saying that in that case,
> Emacs shouldn't take advantage of it. How else could

>> If it really works usefully with GCC -- if that is not just a theoretical
>> idea -- then I won't object to its supporting other compilers as well.

> be interpreted?

I interpret him as meaning that the support should not favor non-GCC compilers
in any way, not that GCC should determine the least upper bound on
functionality.

For example, an automobile can be driven by any able-bodied individual. It
does not favor one person over another, because it is adjustable. Both my wife
and I can drive the same cars, always.

A car does, however, favor ability. A better driver will drive any car better
than a worse driver. This does not preclude the worse driver from studying and
becoming better, however. It does not "build in" any advantage that makes one
driver better, no matter what the other driver does.

To me, Emacs is a vehicle for content, and external processes sometimes guide
or drive that content, such as GCC or Clang within a C++ buffer. Emacs should
be adjustable to allow either one to be used. Certainly one may provide better
functionality than the other, but Emacs itself should not favor one over the
other. If GCC ends up excelling Clang as a compiler, its Emacs support should
be capable of excelling Clang as well, without any change necessary from Emacs
to allow this.

If, on the other hand, Clang offered some clever API that was specific to
Clang, and we built support for that API directly into Emacs to allow a better
experience for Clang users, this is what Richard would not allow, according to
what I read.

John

Re: New maintainer

[David Engster]

John Wiegley writes:
> I interpret him as meaning that the support should not favor non-GCC compilers
> in any way, not that GCC should determine the least upper bound on
> functionality.

I'd suggest to read the past discussion on this to better understand
Richard's position. There's a good summary at LWN:

http://lwn.net/Articles/629259/

-David

Re: New maintainer

[John Wiegley]

>>>>> David Engster <deng@randomsample.de> writes:

> I'd suggest to read the past discussion on this to better understand
> Richard's position. There's a good summary at LWN:

> http://lwn.net/Articles/629259/

Stefan wrote:

> My understanding is that you're opposed to GCC providing this useful info
> because that info would need to be complete enough to be usable as input to
> a proprietary compiler backend.

Isn't crippling the output of GCC, to prevent use by proprietary vendors, a
direct example of limiting *our* freedom, as users who want access to that
information to improve our use of Emacs (or other tools)? Making such
information available does not make GCC or Emacs in any way more proprietary
or freedom-destroying. If anything, it is liberating the information known to
these applications, so that it can be more widely applied.

Richard, can you please clarify? I can appreciate not wanting to support,
favor, or even recommend, proprietary systems. But the discussion I'm reading
at that link feels different from this.

John

Re: New maintainer

[Jay Belanger]

"John Wiegley" <johnw@newartisans.com> writes:

> If, on the other hand, Clang offered some clever API that was specific
> to Clang, and we built support for that API directly into Emacs to
> allow a better experience for Clang users, this is what Richard would
> not allow, according to what I read.

Ah.  This sounds different than what you wrote before:

>> The main thing is that it is GCC's responsibility to be better, and
>> not Emacs' to prevent better options from being chosen, simply to
>> accommodate a lack of progress by GCC.

I took this to mean if Clang offered something that Emacs could support,
then it would be GCC's responsibility to add it.  But I'm probably
misunderstanding something.

Thanks for the answer.

Re: New maintainer

[David Kastrup]

Jay Belanger <jay.p.belanger@gmail.com> writes:

> "John Wiegley" <johnw@newartisans.com> writes:
>
>> If, on the other hand, Clang offered some clever API that was specific
>> to Clang, and we built support for that API directly into Emacs to
>> allow a better experience for Clang users, this is what Richard would
>> not allow, according to what I read.
>
> Ah.  This sounds different than what you wrote before:
>
>>> The main thing is that it is GCC's responsibility to be better, and
>>> not Emacs' to prevent better options from being chosen, simply to
>>> accommodate a lack of progress by GCC.
>
> I took this to mean if Clang offered something that Emacs could
> support, then it would be GCC's responsibility to add it.

If consistent with the technical possibilities and steering decisions.
Yes.

But it is Emacs' responsibility to wait until GCC does that before
supporting it on other platforms.

-- 
David Kastrup

Re: New maintainer

[David Kastrup]

"John Wiegley" <johnw@newartisans.com> writes:

>>>>>> Jay Belanger <jay.p.belanger@gmail.com> writes:
>
>>>> Maybe I'm misreading it, but it doesn't sound like what Richard meant at
>>>> all. I read it as the features have to actually work with GCC ("not just a
>>>> theoretical idea") to be included.
>>> 
>>> The issue is that GCC, in its current state, doesn't provide a certain
>>> set a features Emacs can take advantage of, that Clang does.
>
>> That's the point. It sounded like Richard was saying that in that case,
>> Emacs shouldn't take advantage of it. How else could
>
>>> If it really works usefully with GCC -- if that is not just a theoretical
>>> idea -- then I won't object to its supporting other compilers as well.
>
>> be interpreted?
>
> I interpret him as meaning that the support should not favor non-GCC
> compilers in any way, not that GCC should determine the least upper
> bound on functionality.

That interpretation is not supported by the mailing list history.  In
particular, there were (and still are) conscious restrictions to the
amount of information GCC may provide in a generic format and/or via a
generic plugin interface since such generic mechanisms define the
"application as a whole" border beyond which copyright and consequently
the GPL does not reach.  If Emacs now provided functionality using Clang
that has been intentionally omitted from or restricted in GCC, this
would be self-defeating.

So yes, in that respect GCC _does_ determine the least upper bound of
functionality.  This has by far been the most heatedly debated influence
of GPL-supporting strategic decisions on this mailing list in the last
years and has even be covered in "Linux Weekly News" several times
(searching for "Emacs" in their archives will likely pull up
references).  The way to make progress in such areas may thus be the
most frustrating part of Emacs maintainership as it requires pushing for
changes with GCC in lockstep, and in this case not just functional
changes (which already provide a challenge and thus can be expected to
move forward rather slower than faster) but also strategic direction
changes.

If you want to go there, it might be an idea starting with the
Ada-specific GCC fork which probably went under the radar with its
option exporting the parse tree.  Working on a good proof-of-concept
using that is probably the most convincing argument you can make.

The good news is likely that this is about the most you can expect in
the area of influence of the GPL and associated policies on Emacs, so if
you make yourself acquainted with the happenings there, you have seen
the practical side of the limitations due to licensing.

-- 
David Kastrup

Re: New maintainer

[Karl Fogel]

"John Wiegley" <johnw@newartisans.com> writes:
>I interpret him as meaning that the support should not favor non-GCC compilers
>in any way, not that GCC should determine the least upper bound on
>functionality.

Just to confirm what others have pointed out:

Given the context and past discussions, I think you would better assume that Richard meant "If GCC doesn't *actually* support the feature, then Emacs shouldn't add support for that feature just because Clang does."  I think at the very least the criterion would be that an actual patch to GCC must exist, even if no release of GCC includes it yet.  

That's just a guess though.  It's an open question whether the requirement would be that the FSF version (i.e., what we would call the "official" version) of GCC must support the feature, or whether it would be sufficient for the feature to be supported in a publicly available patch to GCC.  I hope the latter, since that's exactly the point at which Emacs's corresponding support would no longer be merely "theoretical" with respect to GCC.

> Isn't crippling the output of GCC, to prevent use by proprietary
> vendors, a direct example of limiting *our* freedom, as users who want
> access to that information to improve our use of Emacs (or other
> tools)? Making such information available does not make GCC or Emacs
> in any way more proprietary or freedom-destroying. If anything, it is
> liberating the information known to these applications, so that it can
> be more widely applied.

What one group chooses to do to their copy of a free software program can *never* interfere with others' freedom w.r.t. that program, because those others are always free to do whatever they want with their own copy.  If the FSF chooses not to add a feature to GCC, that doesn't interfere with your freedom.  It may interfere with your convenience, but respecting people's freedom does not require supplying them exactly the thing they want, it merely requires not interfering with their ability to procure what they want by whatever means are available to them.

The FSF can't "cripple" GCC.  It can only cripple *its version* of GCC.  You and anyone else are free to make a non-crippled version of GCC, and that's what freedom means :-).

Best,
-Karl

Re: New maintainer

[David Kastrup]

Karl Fogel <kfogel@red-bean.com> writes:

> "John Wiegley" <johnw@newartisans.com> writes:
>>I interpret him as meaning that the support should not favor non-GCC compilers
>>in any way, not that GCC should determine the least upper bound on
>>functionality.
>
> Just to confirm what others have pointed out:
>
> Given the context and past discussions, I think you would better
> assume that Richard meant "If GCC doesn't *actually* support the
> feature, then Emacs shouldn't add support for that feature just
> because Clang does."  I think at the very least the criterion would be
> that an actual patch to GCC must exist, even if no release of GCC
> includes it yet.
>
> That's just a guess though.  It's an open question whether the
> requirement would be that the FSF version (i.e., what we would call
> the "official" version) of GCC must support the feature, or whether it
> would be sufficient for the feature to be supported in a publicly
> available patch to GCC.  I hope the latter, since that's exactly the
> point at which Emacs's corresponding support would no longer be merely
> "theoretical" with respect to GCC.

I should think that the requirement would be that the patch would be
acceptable into the FSF version.  Reasons for non-acceptance would be a
lack of copyright assignment, or a mismatch with FSF policies.

It's really not speculative how this pans out: just look up the recent
GCC/AST/Smartcompletion spat on the Emacs developer list.

-- 
David Kastrup

Re: New maintainer

[John Wiegley]

>>>>> Karl Fogel <kfogel@red-bean.com> writes:

> Given the context and past discussions, I think you would better assume that
> Richard meant "If GCC doesn't *actually* support the feature, then Emacs
> shouldn't add support for that feature just because Clang does." I think at
> the very least the criterion would be that an actual patch to GCC must
> exist, even if no release of GCC includes it yet.

OK, there are a few details here, and I'm hoping Richard will clarify. Let's
assume some feature X that one might want of a compiler. There are a few ways
GCC might relate to this feature:

  1. It has X, and we can expose it to Emacs.
  2. It has X, but does not provide it in a useful way, because doing so is
     against FSF policy.
  3. It could have X, but doesn't yet.
  4. It will never have X, since providing it would be prohibitively
     expensive, or against policy.

The question is, assuming Clang falls into first category, what is the
situation for Emacs?

  A. Emacs is only allowed to provide the feature for GCC, and must wait until
     GCC makes it available (if ever).

  B. Emacs can only offer the feature for other compilers too, but only once
     it is able to offer it for GCC. This means we are blocked on GCC
     development before we can support other compilers.

  C. If Emacs can support the feature in a _general_ fashion -- so that GCC
     could just as easily be supported as Clang -- then Clang support is
     allowed before GCC support, assuming Clang has it and GCC doesn't (or
     might never).

  D. Emacs is allowed to directly support Clang features that GCC never will,
     because this makes Emacs a better editor.

I'm pretty sure D is out, based on RMS' past comments. I also think A is out.
My question is whether Emacs project policy is B, C, or something more.

John

Re: New maintainer

[Karl Fogel]

[-- Attachment #1: Type: text/plain, Size: 991 bytes --]

On Tue, Oct 6, 2015 at 5:59 PM, John Wiegley <johnw@newartisans.com> wrote:

>   A. Emacs is only allowed to provide the feature for GCC, and must wait
> until
>      GCC makes it available (if ever).
>
>   B. Emacs can only offer the feature for other compilers too, but only
> once
>      it is able to offer it for GCC. This means we are blocked on GCC
>      development before we can support other compilers.
>
>   C. If Emacs can support the feature in a _general_ fashion -- so that GCC
>      could just as easily be supported as Clang -- then Clang support is
>      allowed before GCC support, assuming Clang has it and GCC doesn't (or
>      might never).
>
>   D. Emacs is allowed to directly support Clang features that GCC never
> will,
>      because this makes Emacs a better editor.
>
> I'm pretty sure D is out, based on RMS' past comments. I also think A is
> out.
> My question is whether Emacs project policy is B, C, or something more.


I believe it is (B).

Best,
-K

[-- Attachment #2: Type: text/html, Size: 1420 bytes --]

Re: New maintainer

[Phillip Lord]

John Wiegley <johnw@newartisans.com> writes:
> The question is, assuming Clang falls into first category, what is the
> situation for Emacs?
>
>   A. Emacs is only allowed to provide the feature for GCC, and must wait until
>      GCC makes it available (if ever).
>
>   B. Emacs can only offer the feature for other compilers too, but only once
>      it is able to offer it for GCC. This means we are blocked on GCC
>      development before we can support other compilers.
>
>   C. If Emacs can support the feature in a _general_ fashion -- so that GCC
>      could just as easily be supported as Clang -- then Clang support is
>      allowed before GCC support, assuming Clang has it and GCC doesn't (or
>      might never).
>
>   D. Emacs is allowed to directly support Clang features that GCC never will,
>      because this makes Emacs a better editor.
>
> I'm pretty sure D is out, based on RMS' past comments. I also think A is out.
> My question is whether Emacs project policy is B, C, or something more.

It's worth mentioning that we have been through all of this before. More
over there was a very clear difference of opinion between Richard and
the current (if retiring) Emacs maintainer.

https://lists.gnu.org/archive/html/emacs-devel/2015-02/msg00498.html
https://lists.gnu.org/archive/html/emacs-devel/2015-01/msg00171.html

If I may be so bold as to speak for Richard and Stefan the answers were
B and D respectively.

I mention this not to stir up old arguments, but simply to point out
that these arguments have not been resolved in the past. While I am
hopeful that they will be resolved in the future, I suspect that trying
to sort this issue out now is a side-track, which should not block
discussion of the maintainership.

Phil

Re: New maintainer

[John Wiegley]

>>>>> Phillip Lord <phillip.lord@russet.org.uk> writes:

> I mention this not to stir up old arguments, but simply to point out that
> these arguments have not been resolved in the past. While I am hopeful that
> they will be resolved in the future, I suspect that trying to sort this
> issue out now is a side-track, which should not block discussion of the
> maintainership.

I realize we're on the 1000th round of this discussion, but I've not been
directly involved in it before, and it has a direct bearing on my willingness
to maintain Emacs.

Guiding a project's technical future requires devotion and enthusiasm, and a
certain degree of freedom. If the directions I want to take Emacs in are going
to be consistently hampered by the "needs of freedom", this will cause me to
lose all such energy.

I'm going be the one at conferences, talking to users, saying, "Yes, we know;
yes, it's a great idea; yes, it should be there; yes, I even want to do it
myself, yesterday; but talk to me in ten years when GCC has gotten around to
providing what we need."

I'm beginning to think GNU Emacs will need someone who also cares about the
freedom argument first, and the technical needs second, because I'm very much
concerned I would chomping at the bit to move forward, and unable to for
reasons I don't necessarily agree with.

John

Re: New maintainer

[Phillip Lord]

John Wiegley <johnw@newartisans.com> writes:

>>>>>> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>> I mention this not to stir up old arguments, but simply to point out that
>> these arguments have not been resolved in the past. While I am hopeful that
>> they will be resolved in the future, I suspect that trying to sort this
>> issue out now is a side-track, which should not block discussion of the
>> maintainership.
>
> I realize we're on the 1000th round of this discussion, but I've not been
> directly involved in it before, and it has a direct bearing on my willingness
> to maintain Emacs.
>
> Guiding a project's technical future requires devotion and enthusiasm, and a
> certain degree of freedom. If the directions I want to take Emacs in are going
> to be consistently hampered by the "needs of freedom", this will cause me to
> lose all such energy.

Yes, that is a possibility indeed. To be honest, though, I think the
"consistently hampered" concerns is misplaced. In fact, the GCC argument
came about because of (Richard's) interpretation of the needs of freedom
wrt GCC, rather than Emacs per se.

From my own perspective, as a very long term Emacs user, the issue
doesn't impact me directly because I don't write very much C or use
either GCC or clang (as dev tools -- obviously I build with them).

My own take on the issue, though, is that worrying now about what *may*
cause you to lose energy in the future is self-defeating. I hope that
you take up the role, I think that you will bring energy to it, and I
hope that you find it to be a rewarding and valuable experience.


> I'm going be the one at conferences, talking to users, saying, "Yes, we know;
> yes, it's a great idea; yes, it should be there; yes, I even want to do it
> myself, yesterday; but talk to me in ten years when GCC has gotten around to
> providing what we need."
>
> I'm beginning to think GNU Emacs will need someone who also cares about the
> freedom argument first, and the technical needs second, because I'm very much
> concerned I would chomping at the bit to move forward, and unable to for
> reasons I don't necessarily agree with.

I don't think that this is the case. It's just the case that the moral
discussions raise more noise than the technical ones, simply because
more people know about something about them.

For instance, while this (long) thread has been happening, Stefan has
been his useful helpful self in helping me with a small change to the
undo system. In total, that's resulted in maybe 10 emails over a month,
compared to the 10 emails an hour on the ethics of exposing GCC/clang
ASTs. The evolution of Emacs is affected as much, I think, by
combination of these small changes as anything.

In short, I wouldn't judge the needs of Emacs by the volume of email.
And, in an attempt to not add to that volume, I shall say no more on
this thread, other than to wish any future maintainer(s) luck whoever
they may be.

Phil

Re: New maintainer

[David Kastrup]

"John Wiegley" <johnw@newartisans.com> writes:

>>>>>> Phillip Lord <phillip.lord@russet.org.uk> writes:
>
>> I mention this not to stir up old arguments, but simply to point out that
>> these arguments have not been resolved in the past. While I am hopeful that
>> they will be resolved in the future, I suspect that trying to sort this
>> issue out now is a side-track, which should not block discussion of the
>> maintainership.
>
> I realize we're on the 1000th round of this discussion, but I've not been
> directly involved in it before, and it has a direct bearing on my willingness
> to maintain Emacs.
>
> Guiding a project's technical future requires devotion and enthusiasm, and a
> certain degree of freedom. If the directions I want to take Emacs in are going
> to be consistently hampered by the "needs of freedom", this will cause me to
> lose all such energy.
>
> I'm going be the one at conferences, talking to users, saying, "Yes, we know;
> yes, it's a great idea; yes, it should be there; yes, I even want to do it
> myself, yesterday; but talk to me in ten years when GCC has gotten around to
> providing what we need."

No, that will not be the case.  "when GCC has gotten around to providing
what we need" suggests that GCC is holding things up.  They are equally
"stomping at the bits", meaning that there have been previous attempts
of writing out the annotated syntax tree or similarly generally useful
stuff that could feed other functionality based on GCC but not falling
under the GPL with regard to "the whole work" criterion because of using
a generic interface.

So for developing this kind of functionality in lockstep with GCC
developers using specialized (namely non-generic) interfaces, you'll
likely find a path leading forward.  The more generally useful such
interfaces will get, the more it will require seeking for an adaption of
FSF policies to the realities you are eager to create.

You won't be turning this ship on a dime.  But I also don't think that
it wouldn't budge when you try getting the pieces in place.  But the
pieces would have to start with working with GCC developers and Richard
open-ended on the kind of needed functionality rather than being based
off Clang's existing state.

> I'm beginning to think GNU Emacs will need someone who also cares
> about the freedom argument first, and the technical needs second,
> because I'm very much concerned I would chomping at the bit to move
> forward, and unable to for reasons I don't necessarily agree with.

As I said: I don't think you'll be unable to move forward but you'll
need to coordinate that movement with the other horses in the equipage
(sorry, your analogy).

-- 
David Kastrup

Re: New maintainer

[Stephen J. Turnbull]

John Wiegley writes:

 > I'm beginning to think GNU Emacs will need someone who also cares
 > about the freedom argument first, and the technical needs second,
 > because I'm very much concerned I would chomping at the bit to move
 > forward, and unable to for reasons I don't necessarily agree with.

I wouldn't worry about that if I were you.  The principle itself
bothers me a heck of a lot more than the exercise of it does.

In the twenty years I've been following Emacs development, I can
remember only four occasions where Richard has deliberately sacrificed
significant improvements to Emacs in the name of promoting either
software freedom or the GNU Project: TRAMP, Bazaar, DSOs, and now use
of the AST exported by LLVM.

Of course, the TRAMP mistake has long since been corrected, and the
Bazaar fiasco is a thing of the past.  The no-DSO policy has been
rescinded recently, and work is actively proceeding on adding that
feature.  LLVM?  "This, too, will pass."

Bottom line: just thinking about it is frustrating, yes, but
"consistently hampered" (your words) in getting work done?  No.

Re: New maintainer

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> John Wiegley writes:
>
>  > I'm beginning to think GNU Emacs will need someone who also cares
>  > about the freedom argument first, and the technical needs second,
>  > because I'm very much concerned I would chomping at the bit to move
>  > forward, and unable to for reasons I don't necessarily agree with.
>
> I wouldn't worry about that if I were you.  The principle itself
> bothers me a heck of a lot more than the exercise of it does.
>
> In the twenty years I've been following Emacs development, I can
> remember only four occasions where Richard has deliberately sacrificed
> significant improvements to Emacs in the name of promoting either
> software freedom or the GNU Project: TRAMP, Bazaar, DSOs, and now use
> of the AST exported by LLVM.
>
> Of course, the TRAMP mistake has long since been corrected, and the
> Bazaar fiasco is a thing of the past.  The no-DSO policy has been
> rescinded recently, and work is actively proceeding on adding that
> feature.  LLVM?  "This, too, will pass."

To put more precision to it: usually the point of time where the
strategy changes is when it has become pointless.  Since LLVM already
outputs annotated syntax trees, blocking GCC from that kind of
interoperation is not going to achieve a useful purpose for the GNU
project at the current point of time.

So I don't expect that restriction to stick around for all that much
longer in the current form: after all, we don't have anything to gain
from people putting LLVM into their applications rather than GCC even if
we had preferred such applications to fall under the GPL because of
tight integration with GCC.

-- 
David Kastrup

Re: New maintainer

[Stephen J. Turnbull]

David Kastrup writes:
 > "Stephen J. Turnbull" <stephen@xemacs.org> writes:

 > > LLVM?  "This, too, will pass."
 > 
 > To put more precision to it: usually the point of time where the
 > strategy changes is when it has become pointless.

I don't think that's true.  First of all, all of the examples I
mentioned were considered pointless when first implemented, except to
satisfy Richard's extreme risk aversion.  Nobody else could see any
danger to compare to the gain.  Second, history says otherwise.

- TRAMP: added when free SSHv2 became available.  (Today, lsh is a
  logical operator, not a secure shell.)

- Bazaar: abandoned when it became clear that not only was it GNU in
  name only, it was a project in name only, and annoying (admittedly,
  not crippling) bugs continued to surface and not be fixed.
  Unfamiliarity with it and distaste for its opinionated approach to
  VC were genuine barriers to new contributors (not terribly high, but
  more real than any danger cited in support of these policies).

- DSOs: permitted when the leagle eagles OK'ed the strategy
  (ironically, originated by the Linux kernel IIRC) of requiring a
  call to a "I am licensed under GPL" API before enabling a module for
  GCC, and Richard judged that the same arguments applied to Emacs.

 > Since LLVM already outputs annotated syntax trees, blocking GCC
 > from that kind of interoperation is not going to achieve a useful
 > purpose for the GNU project at the current point of time.

I agree, as I'm sure you already guessed, but here Richard isn't
concerned with useful purposes, he worries that part of GCC itself can
be suborned by a proprietary system.  Currently RTL and the like are
*internal* formats that can be saved to disk, but they are not "data"
that is output to be consumed by *arbitrary* other Works (in the
copyright sense), and therefore exempt from the GPL, which (as a bare
license) can make no restriction on how you use the data output from a
program.  They're internal data structures, subject to copyright even
though produced by a program, that happen to reside on disk rather
than in memory.  IIUC, this principle was tested in the NeXT (Apple?) 
Objective C case and found dependable (enough).

So GCC is is directly constrained, but this has a knock-on effect on
Emacs because support for this feature provided by LLVM would give
LLVM-based development a competitive advantage over GCC-based
development, so support is (for now) prohibited.

 > So I don't expect that restriction to stick around for all that
 > much longer in the current form: after all, we don't have anything
 > to gain from people putting LLVM into their applications rather
 > than GCC even if we had preferred such applications to fall under
 > the GPL because of tight integration with GCC.

That was true when Richard announced that he was putting this idea on
hold while he studied the technical and legal aspects of outputting
CAST[1] so that Emacs can use it but even GCC's own backend can't
consume it.  I think you misunderstand the strength of his
determination not to yield a nanometer to proprietary software.

I believe that, from John's point of view, the risk is real that
Richard will indeed insist on CAST, which will hamper attempts to
extend use of compiler-generated ASTs from completion (or whatever the
initial application is) to refactoring and so on, miring them in
further discussions of the kind that happened earlier for every
extension.

I do, however, want to emphasize that this kind of prior restraint on
Emacs development is *extremely* rare.

Footnotes: 
[1]  Crippled Annotated Syntax Trees.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'm beginning to think GNU Emacs will need someone who also cares about the
  > freedom argument first, and the technical needs second, because I'm very much
  > concerned I would chomping at the bit to move forward, 

It's a question of which direction is "forward".
I want to move forward on the GNU Project, of which GNU Emacs
is a part.

Most of the time, making Emacs better to use is exactly the way Emacs
can move the GNU Project forward, but once in a rare while there's an
exception.  Someone posted a list of four cases that have occurred
in GNU Emacs over three decades.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Dmitry Gutov]

On 10/05/2015 11:16 PM, Jay Belanger wrote:

> That's the point.  It sounded like Richard was saying that in that case,
> Emacs shouldn't take advantage of it.  How else could

Right. Since the GCC support would only be "theoretical".

>> If it really works usefully with GCC -- if that is not just a theoretical
>> idea -- then I won't object to its supporting other compilers as well.
>
> be interpreted?

Works usefully with GCC *now*, or whenever the feature in question is 
introduced.

While I personally disagree with that stance, it's good to understand 
clearly the current state of this issue.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Any solution we choose should never preclude the opportunity for GCC to
  > outshine other choices.

That's true, but it is too weak; we are failing to communicate.  The
issue is not about what is "precluded" but rather about what actually
works.  The point is, when the feature is introduced in Emacs, it
should really work with GCC as well as it does with any competitor to
GCC.

What I want to avoid is that Emacs starts encouraging people to switch
from GCC to some competitor.

This issue is not about free vs proprietary software.  I am not saying
that competitors to a GNU package are unjust or bad -- that isn't
necessarily so.  The pertinent point is that they are _competitors_.
The goal if the GNU Project is for GNU to win the competition.  Each
GNU package is a part of the GNU system, and should contribute to the
success of the GNU Project.

Thus, each GNU package should encourage people to run other GNU
packages rather than their competitors -- even competitors which are
free software.

This is a practical question.  It's about practical effect, not some
legalistic criterion.

Thus, it wouldn't be good enough if there is a GCC patch we don't
endorse, that makes it do the job in question -- because, practically
speaking, that wouldn't change the effect much.  The support should be
in the real GCC sources.

On the other hand, it would be good enough for that to be in the
development GCC repository in the master or root.  We don't need to
wait for it to be officially released.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> That's true, but it is too weak; we are failing to communicate. The issue is
> not about what is "precluded" but rather about what actually works. The
> point is, when the feature is introduced in Emacs, it should really work
> with GCC as well as it does with any competitor to GCC.

Hi Richard,

Thank you for the clarification, the picture is becoming much clearer. I
really appreciate the time you've taken to reiterate these positions for the
millionth time.

There is one point I am having a hard time with. I'm feeling as though my
Emacs experience (as a user) is being sacrificed at someone else's altar.

The idea, if I understand it, is that you don't want Emacs' C++ support to
allow Clang to beat GCC, because then people would naturally choose Clang who
care more about getting things done, than they do about software freedom. In
effect, Emacs is being used to keep people within the free software agenda, by
making Clang no more appealing than GCC.

This troubles me. I can see that for you, the freedom idea is much more
important than the technical idea. You'd rather we stick with GCC until the
cows come home, so long as it leads to a freer world.

Meanwhile, there are those among us who don't share your ideals to the same
extent. We'd prefer an editor that lets us get things done faster, better,
leaving us with free time to... produce more free software.

I can't help but think that unless the FSF has more to offer than its ideals,
its technical decisions are going to render it obsolete. Progress waits for no
man, and the world is changing more and more rapidly. There is a reason Clang
is eating GCC's lunch: because the needs of a larger community demand a better
free compiler.

Emacs is still a fantastic editor, but it's old and its age is showing. If we
remain competitive, it could stay awesome for another 30 years; but if we
avoid progress to further non-technical agendas, I think it will drive people
AWAY from the GNU project, not bind them more tightly to it.

John

Re: New maintainer

[David Kastrup]

John Wiegley <johnw@newartisans.com> writes:

> I can't help but think that unless the FSF has more to offer than its
> ideals, its technical decisions are going to render it obsolete.

The FSF never had substantially more to offer than its ideals.  All the
rest has been provided by volunteers and donations.  While Richard has
thrown a substantial amount of his own work and money and time into it
at the start, in the overall scheme that's not much more than a
catalyst.

> Progress waits for no man, and the world is changing more and more
> rapidly. There is a reason Clang is eating GCC's lunch: because the
> needs of a larger community demand a better free compiler.

The FSF has no control over the direction of Clang, technical or with
regard to licensing.  A whole lot of software is "eating GNU's lunch" in
a number of technical categories.  GNU started out with everybody
"eating its lunch".  The mission of the GNU project is to provide a
coherent whole with freedoms that cannot be subverted.

Apple's XCode environment is based on a free compiler, Clang, but is
licensed in a way where you may not run it on anything but Apple
computers.  _That's_ how you really eat the lunch of free software.
Having Emacs integrate with XCode for developing code in a manner that
cannot be done with the GNU system would be self-defeating.

It's the point of the GPL to be hard to subvert against the cause of
free software.  But the GPL is not a philosophical authority but a legal
tool.  Software licensed under it can be used according to our goals or
against them.  Where the only uses are weakening our cause, there is no
point in being the front-runner.  Everybody may fork Emacs (or just
provide his own packages) who wants to work on goals not helping the GNU
project, but there is no point in the core of Emacs relying on resources
and the blessing of the FSF to do so.

> Emacs is still a fantastic editor, but it's old and its age is
> showing. If we remain competitive, it could stay awesome for another
> 30 years; but if we avoid progress to further non-technical agendas, I
> think it will drive people AWAY from the GNU project, not bind them
> more tightly to it.

That argument is more than 30 years old, and many parts of the GNU
project have taken second place to other software a whole lot of the
time.  But the front leaders wither and die and get replaced by others.
GNU sticks around.  Emacs sticks around.  Its largest traditional
competitor is "vi" and it is factually gone, replaced by the most
popular free vi clone of the decade (currently vim).

Yes, we'll not end up in first place on technical merit lots of the time
because ending up in first place is not the first priority.  The first
priority is to provide a free cohesive system with essential parts
nominally and effectively under the GPL so that its use as a building
block will lead to more systems honoring and providing software freedom.

Taking custody of that may be a nuisance if you don't care or even
disagree.  But even though it's an essential part of the job, it should
not turn out a permanent distraction.  And if it does, one should try
finding a solution or compromise that manages to serve the conflicting
priorities better.

-- 
David Kastrup

Re: New maintainer

[joakim]

David Kastrup <dak@gnu.org> writes:

> John Wiegley <johnw@newartisans.com> writes:
>
>> I can't help but think that unless the FSF has more to offer than its
>> ideals, its technical decisions are going to render it obsolete.
>
> The FSF never had substantially more to offer than its ideals.  All the
> rest has been provided by volunteers and donations.  While Richard has
> thrown a substantial amount of his own work and money and time into it
> at the start, in the overall scheme that's not much more than a
> catalyst.
>
>> Progress waits for no man, and the world is changing more and more
>> rapidly. There is a reason Clang is eating GCC's lunch: because the
>> needs of a larger community demand a better free compiler.
>
> The FSF has no control over the direction of Clang, technical or with
> regard to licensing.  A whole lot of software is "eating GNU's lunch" in
> a number of technical categories.  GNU started out with everybody
> "eating its lunch".  The mission of the GNU project is to provide a
> coherent whole with freedoms that cannot be subverted.
>
> Apple's XCode environment is based on a free compiler, Clang, but is
> licensed in a way where you may not run it on anything but Apple
> computers.  _That's_ how you really eat the lunch of free software.
> Having Emacs integrate with XCode for developing code in a manner that
> cannot be done with the GNU system would be self-defeating.

As I sometimes get bitten by this in my day job, I would just like to
add that a lot of people I meet that are not even interested in free
software, find this aspect of the Apple development stack appalling.

While this is not my personal position, you can use the GNU tools and
not really care about the freedom, but you still benefit from it. You
can use free software library and all you need to do is provide the
source for your users. You can deploy code like this with no worries, as
long as you comply with the GPL, which isn't really hard.

Then when you are writing, say, an ios client, the situation is
different. The developers I work with quite often don't think that Apple
software is especially fascinating. Still, they can't choose not to use Apple
tools in the build chain. You have to have Apple hardware, which isn't
suitable in a datacenter. You can't choose to not use Xcode and
whatever.

With this I just wanted to add another datapoint.  


>
> It's the point of the GPL to be hard to subvert against the cause of
> free software.  But the GPL is not a philosophical authority but a legal
> tool.  Software licensed under it can be used according to our goals or
> against them.  Where the only uses are weakening our cause, there is no
> point in being the front-runner.  Everybody may fork Emacs (or just
> provide his own packages) who wants to work on goals not helping the GNU
> project, but there is no point in the core of Emacs relying on resources
> and the blessing of the FSF to do so.
>
>> Emacs is still a fantastic editor, but it's old and its age is
>> showing. If we remain competitive, it could stay awesome for another
>> 30 years; but if we avoid progress to further non-technical agendas, I
>> think it will drive people AWAY from the GNU project, not bind them
>> more tightly to it.
>
> That argument is more than 30 years old, and many parts of the GNU
> project have taken second place to other software a whole lot of the
> time.  But the front leaders wither and die and get replaced by others.
> GNU sticks around.  Emacs sticks around.  Its largest traditional
> competitor is "vi" and it is factually gone, replaced by the most
> popular free vi clone of the decade (currently vim).
>
> Yes, we'll not end up in first place on technical merit lots of the time
> because ending up in first place is not the first priority.  The first
> priority is to provide a free cohesive system with essential parts
> nominally and effectively under the GPL so that its use as a building
> block will lead to more systems honoring and providing software freedom.
>
> Taking custody of that may be a nuisance if you don't care or even
> disagree.  But even though it's an essential part of the job, it should
> not turn out a permanent distraction.  And if it does, one should try
> finding a solution or compromise that manages to serve the conflicting
> priorities better.

-- 
Joakim Verona

Re: New maintainer

[Karl Fogel]

John, your points below are a continuation of a very interesting tactical (and philosophical) debate... that was already held on this list fairly recently, and that eventually went somewhat off-topic for this list even then :-).

While the questions you raise are worth discussing for the GNU project as a whole, I think Richard has made it clear that the position of the FSF regarding Emacs' development priorities is definitely not going to be changed as part of this new maintainer discussion.

So if at this point you were to say that, as far as being an Emacs maintainer goes, you understand and are willing to abide by the priorities Richard has articulated, even though you disagree with them, that would settle an important question.  Or if you don't think you can abide by them, in your work as maintainer, that would be useful to know too (though it would probably result in you not being the maintainer).

As you know, I hope you can abide by them and be the maintainer (or a co-maintainer) because I think you'd be terrific at it.  But it's your call.  I merely urge clarity; Richard has made his position clear, so if you do too then things can move forward one way or the other.

Best,
-K

John Wiegley <johnw@newartisans.com> writes:
>Hi Richard,
>
>Thank you for the clarification, the picture is becoming much clearer. I
>really appreciate the time you've taken to reiterate these positions for the
>millionth time.
>
>There is one point I am having a hard time with. I'm feeling as though my
>Emacs experience (as a user) is being sacrificed at someone else's altar.
>
>The idea, if I understand it, is that you don't want Emacs' C++ support to
>allow Clang to beat GCC, because then people would naturally choose Clang who
>care more about getting things done, than they do about software freedom. In
>effect, Emacs is being used to keep people within the free software agenda, by
>making Clang no more appealing than GCC.
>
>This troubles me. I can see that for you, the freedom idea is much more
>important than the technical idea. You'd rather we stick with GCC until the
>cows come home, so long as it leads to a freer world.
>
>Meanwhile, there are those among us who don't share your ideals to the same
>extent. We'd prefer an editor that lets us get things done faster, better,
>leaving us with free time to... produce more free software.
>
>I can't help but think that unless the FSF has more to offer than its ideals,
>its technical decisions are going to render it obsolete. Progress waits for no
>man, and the world is changing more and more rapidly. There is a reason Clang
>is eating GCC's lunch: because the needs of a larger community demand a better
>free compiler.
>
>Emacs is still a fantastic editor, but it's old and its age is showing. If we
>remain competitive, it could stay awesome for another 30 years; but if we
>avoid progress to further non-technical agendas, I think it will drive people
>AWAY from the GNU project, not bind them more tightly to it.
>
>John

Re: New maintainer

[John Wiegley]

>>>>> Karl Fogel <kfogel@red-bean.com> writes:

> So if at this point you were to say that, as far as being an Emacs
> maintainer goes, you understand and are willing to abide by the priorities
> Richard has articulated, even though you disagree with them, that would
> settle an important question. Or if you don't think you can abide by them,
> in your work as maintainer, that would be useful to know too (though it
> would probably result in you not being the maintainer).

I'm pretty sure that if it came up in a significant way, I wouldn't be able to
stand by it. The insistence on Bazaar over Git, for example, caused me to stop
contributing to Emacs a few years back. And I've been unhappy about the DSO
situation since around 1999. Very glad to see these two getting resolved!

What I wonder is whether Richard and I could reach a compromise if it happens
while we're working together. I'm not saying everything has to go my way; but
if I see something that needs to happen for the sake of users, would we be
able to find an alternate path? I'm not sure this can be answered in a mailing
list thread. It depends on what rapport develops between me and the FSF.

I'll be in the Boston area at the beginning of December; I'd like to stop by
the offices of the FSF, and sit down with Richard to talk about these things
face to face. That would answer more for me than any amount of debate.

If that timeframe is too long for a decision, I'm willing to help out Emacs
until it becomes a real problem. There are plenty of areas where no
disagreement exists at all.

John

Re: New maintainer

[David Kastrup]

"John Wiegley" <johnw@newartisans.com> writes:

> I'll be in the Boston area at the beginning of December; I'd like to
> stop by the offices of the FSF, and sit down with Richard to talk
> about these things face to face. That would answer more for me than
> any amount of debate.

I consider that an excellent idea.

-- 
David Kastrup

Re: New maintainer

[Eli Zaretskii]

> From: "John Wiegley" <johnw@newartisans.com>
> Date: Thu, 08 Oct 2015 17:14:02 -0700
> 
> >>>>> Karl Fogel <kfogel@red-bean.com> writes:
> 
> > So if at this point you were to say that, as far as being an Emacs
> > maintainer goes, you understand and are willing to abide by the priorities
> > Richard has articulated, even though you disagree with them, that would
> > settle an important question. Or if you don't think you can abide by them,
> > in your work as maintainer, that would be useful to know too (though it
> > would probably result in you not being the maintainer).
> 
> I'm pretty sure that if it came up in a significant way, I wouldn't be able to
> stand by it. The insistence on Bazaar over Git, for example, caused me to stop
> contributing to Emacs a few years back. And I've been unhappy about the DSO
> situation since around 1999. Very glad to see these two getting resolved!

It's too bad, IMO, that you evidently assign so much importance to
issues with which you disagree.  Cooperation is about finding the
areas of agreement, which are certainly vast in this case, and using
them for the common good, rather than poking at the few disagreements.

> What I wonder is whether Richard and I could reach a compromise if it happens
> while we're working together. I'm not saying everything has to go my way; but
> if I see something that needs to happen for the sake of users, would we be
> able to find an alternate path?

It's a question no one will be able to answer in advance.  These
issues are decided on a case by case basis, depending on the balance
of advantages and disadvantages in each specific case.  Sometimes your
suggestion could be accepted even without a compromise, sometimes
there could be a compromise, and sometimes no compromise will be
possible.

You will have to decide up front whether a possibility of no
compromise in some situations is something you will be able to live
with.  This is something that IMO sheds light not only on your
relations with Richard and the FSF, but also on your relations with
other contributors here, because such situations will arise there as
well.

> If that timeframe is too long for a decision, I'm willing to help out Emacs
> until it becomes a real problem. There are plenty of areas where no
> disagreement exists at all.

Exactly.  And my question is: why not concentrate on those areas, and
simply "bypass" (a.k.a. "ignore") the rest?

Re: New maintainer

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> It's too bad, IMO, that you evidently assign so much importance to issues
> with which you disagree. Cooperation is about finding the areas of
> agreement, which are certainly vast in this case, and using them for the
> common good, rather than poking at the few disagreements.
> 
> You will have to decide up front whether a possibility of no compromise in
> some situations is something you will be able to live with. This is
> something that IMO sheds light not only on your relations with Richard and
> the FSF, but also on your relations with other contributors here, because
> such situations will arise there as well.
> 
> [...] And my question is: why not concentrate on those areas, and simply
> "bypass" (a.k.a. "ignore") the rest?

Eli, the time I have free to spend on open source projects is very limited,
and extremely precious to me. There are a hundred more things I want to do,
than can do, on any given day. So to make a commitment of this extent, I must
be confident in what I'm getting myself into. Not just the technical content,
or even the political content, but the "spirit" of those who are in charge of
decisions that could make my life as a contributor miserable, should they
choose.

My questions are somewhat pinpointed in this thread, because I'm using them to
bring out characteristics from people that I can't determine just by asking
their friendly opinions. I've been extremely impressed with Richard throughout
this transaction, and frankly that has mattered more to me than anything
that's been said. Seeing how a thread progresses when an issue is put under a
microscope like this, tells me more about the community than it does about the
issue. The issue itself, in fact, has only become greyer -- as often happens
-- since I'm coming to appreciate the complexities of the problem.

Once I have faith, I do "ignore" the rest, and you won't have to worry about
me arguing every point. But if you do worry, then vote -1, and I'll completely
understand.

John

Re: New maintainer

[Eli Zaretskii]

> From: John Wiegley <johnw@newartisans.com>
> Date: Fri, 09 Oct 2015 10:25:36 -0700
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > It's too bad, IMO, that you evidently assign so much importance to issues
> > with which you disagree. Cooperation is about finding the areas of
> > agreement, which are certainly vast in this case, and using them for the
> > common good, rather than poking at the few disagreements.
> > 
> > You will have to decide up front whether a possibility of no compromise in
> > some situations is something you will be able to live with. This is
> > something that IMO sheds light not only on your relations with Richard and
> > the FSF, but also on your relations with other contributors here, because
> > such situations will arise there as well.
> > 
> > [...] And my question is: why not concentrate on those areas, and simply
> > "bypass" (a.k.a. "ignore") the rest?
> 
> Eli, the time I have free to spend on open source projects is very limited,
> and extremely precious to me. There are a hundred more things I want to do,
> than can do, on any given day.

Same here.

> So to make a commitment of this extent, I must be confident in what
> I'm getting myself into. Not just the technical content, or even the
> political content, but the "spirit" of those who are in charge of
> decisions that could make my life as a contributor miserable, should
> they choose.

All understandable and agreed.  I have no problem with the questions
you ask.  I just responded to one specific thing you wrote that is not
a question at all:

> I'm pretty sure that if it came up in a significant way, I wouldn't be able to
> stand by it.

That's pretty black-and-white, no gray areas there.  I do hope that
this kind of confrontation will happen less in Emacs development than
it happened until now.  Maybe I'm naïve.

Re: New maintainer

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

>> I'm pretty sure that if it came up in a significant way, I wouldn't be able
>> to stand by it.

> That's pretty black-and-white, no gray areas there. I do hope that this kind
> of confrontation will happen less in Emacs development than it happened
> until now. Maybe I'm naïve.

I'm willing to hope right along with you. :) There are some decisions I can't
brook, but based on what I've seen from everyone here, I'm becoming more and
more hopeful that it won't ever be a real problem.

John

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > What I wonder is whether Richard and I could reach a compromise if
  > it happens while we're working together. I'm not saying everything
  > has to go my way; but if I see something that needs to happen for
  > the sake of users, would we be able to find an alternate path? I'm
  > not sure this can be answered in a mailing list thread. It depends
  > on what rapport develops between me and the FSF.

I think we can work together this way.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

   > Emacs is being used to keep people within the free software agenda,

Since people are free to change a free program, our decisions of what
to include or not include can't "keep" people anywhere.  The most we
can do is decide in which direction to lead people, where to
help/encourage them to go.

Of course we should develop GNU Emacs so as to support the free
software agenda.  That's been its purpose since I started it in 1984.
That's the purpose of the GNU system as a whole, too.

  > This troubles me. I can see that for you, the freedom idea is much more
  > important than the technical idea. You'd rather we stick with GCC until the
  > cows come home, so long as it leads to a freer world.

Naturally -- because I think freedom is more important than technical
progress.  Proprietary software offers plenty of technical "progress",
but since I won't surrender my freedom to use it, as far as I'm
concerned it is no progress at all.

If I had valued technical advances over freedom in 1984, instead of
developing GNU Emacs and GCC and GDB I would have gone to work for
AT&T and improved its nonfree software.  What a big head start I could
have got!

If we subordinate our freedom to technical advance, companies will
easily be able to lead us away from freedom.  Companies make
multi-year plans to part users from their freedom.  (Clang has
benefited from such a plan.)  We can't carry out plans the way
companies can, but we do need to think about where we want to end up.

  > avoid progress to further non-technical agendas, I think it will drive people
  > AWAY from the GNU project, not bind them more tightly to it.

I don't think we can enable the GNU system to succeed more by
declaring "Each package for itself!"  To stand against capable rivals
-- some of which are not merely competitors, but intentional
opposition -- GNU packages need to work and stand together.

We also need that cooperation in order to give the GNU system a better
IDE.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Tom]

Richard Stallman <rms <at> gnu.org> writes:
> 
> Naturally -- because I think freedom is more important than technical
> progress.  Proprietary software offers plenty of technical "progress",
> but since I won't surrender my freedom to use it, as far as I'm
> concerned it is no progress at all.


Then there is no sense talking about competing with IDEs. IDEs provide
cutting edge features in the areas of completion and refactoring and
most users will only choose Emacs over an IDE if it provides a
a comparable level of these features.

Most users won't switch just because of the freedom aspect. They will
switch if Emacs is at least as good in these areas as their IDEs,
so if competing is the goal then catching up with popular IDEs
technically is unaviodable.

Re: New maintainer

[David Kastrup]

Tom <adatgyujto@gmail.com> writes:

> Richard Stallman <rms <at> gnu.org> writes:
>> 
>> Naturally -- because I think freedom is more important than technical
>> progress.  Proprietary software offers plenty of technical "progress",
>> but since I won't surrender my freedom to use it, as far as I'm
>> concerned it is no progress at all.
>
> Then there is no sense talking about competing with IDEs.

Under the premise that Emacs' only user is Richard, this is not a
competition.

> IDEs provide cutting edge features in the areas of completion and
> refactoring and most users will only choose Emacs over an IDE if it
> provides a a comparable level of these features.

Emacs will rarely ever be "cutting edge" but that does not mean that
there is no point in improving it.  And the features with which
proprietary software is competing among its ilk are certainly something
worthwhile to consider for such improvement.

> Most users won't switch just because of the freedom aspect. They will
> switch if Emacs is at least as good in these areas as their IDEs, so
> if competing is the goal then catching up with popular IDEs
> technically is unaviodable.

Competing is not the goal.  Improving Emacs is.  There are a number of
areas in which Emacs is so much different than other systems that there
is no competition at all.  But that does not mean that those areas are
not worth improving for improvement's sake.

-- 
David Kastrup

Re: New maintainer

[Stephen J. Turnbull]

Tom writes:

 > Richard Stallman <rms <at> gnu.org> writes:
 > > 
 > > Naturally -- because I think freedom is more important than technical
 > > progress.  Proprietary software offers plenty of technical "progress",
 > > but since I won't surrender my freedom to use it, as far as I'm
 > > concerned it is no progress at all.
 > 
 > Then there is no sense talking about competing with IDEs. IDEs provide
 > cutting edge features in the areas of completion and refactoring and
 > most users will only choose Emacs over an IDE if it provides a
 > a comparable level of these features.

I disagree with Richard's approach to the problem of advocating
freedom.[1]

But let's not be extreme, especially not in the face of the GNU/Linux
experience.  We *can* do, we have done, the "impossible".  While Emacs
is the best at what it does, admittedly it caters to a particularly
refined taste.  But Apache and Linux prove that not only can we do the
impossible, we can do it better than well-funded corporations,
sometimes.

 > Most users won't switch just because of the freedom aspect.

True.  But so what?  A few will switch for freedom alone (eg, Richard
Stallman ;-), a few more will switch for freedom and 10% of the
proprietary alternative's functionality.  Let's see how high we can
push the comparison coefficient on functionality (without sacrificing
the freedom), and see how many come.  We all love dog food, right?


Footnotes: 
[1]  I managed to suppress the temptation to explain my viewpoint and
left only this footnote.  But I couldn't resist leaving the
footnote.... ;-)

Re: New maintainer

[Artur Malabarba]

(Back on the topic of new maintainers)

I only know John Wiegley by reputation and by code (both of which are
very good), but putting that together with the conversations on the
current topic, it looks to me like he would make a very good
maintainer.

I know Eli from many other topics on this list, and I also think he
would make a good addition to the maintainer team. My only concern
would be that the burn-out effect might cause him to cease
contributing. Eli is among our most active contributors and I shiver
in fear at the thought of losing his contributions.

Best,
Artur

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > So I will withdraw my withdrawal, and say that if the opinions of the FSF, and
  > the maintenance of Emacs by someone like myself, can co-exist without
  > involving ethical compromise by either one of us, I'm happy for us to work
  > together.

I am very glad.  However, another message reminded me that the
maintainer's responsibility includes some other practical tasks that
relate to the GNU Project's philosophy, that I forgot about when
writing my previous message.  I had better correct that omission now.

They include (1) making sure we have copyright papers from substantial
contributors, (2) keeping track of authorship of contributions, and
(3) on rare occasions implementing or maintaining code to support
either the GPL requirements (as in the case of dynamic linking) or
informing people about the philosophy (such as the command C-h g)

I can't be absolutely sure I haven't omitted anything else.  If there
are any other such tasks, they would in any case be practical tasks,
like those above.

I am making everything clear and explicit, modulo the imperfections
of my memory.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> They include (1) making sure we have copyright papers from substantial
> contributors, (2) keeping track of authorship of contributions, and (3) on
> rare occasions implementing or maintaining code to support either the GPL
> requirements (as in the case of dynamic linking) or informing people about
> the philosophy (such as the command C-h g)

Understood, and not arguments from me here. Since Emacs is an GNU project, it
should remain consistent with the needs of GNU.

John

Re: New maintainer

[Jens K. Loewe]

Ah, finally an interesting discussion here.

> The whole point of GNU is the non-acceptance of software denying
> the users the fundamental software freedoms.

So GNU does not stand *for* something but *against* something? GNU's
whole point is not to be better than others but to point out that others
are worse? That reminds me of a kindergarten.

Furthermore, shouldn't it be the right of a free user to use a non-free
system at his own will? I don't use GNU for the sole reason that I get
my things done fast on other platforms, but I solve a lot of problems
with FLOSS software even on closed platforms. Am I a bad guy now?

I actually considered suggesting myself as a maintainer just to
contribute at least something to Emacs. I never wrote
"upstream" code for Emacs but I do that team development thing for
money. I guess as a Windows and BSD user I'm out of the game though - I
disagree with some of the points the GPL makes and I'll always prefer
the BSD infrastructure to GNU; but I really like working with
Emacs. However, it seems that the Emacs product is less important than
the Emacs "philosophy" here. No wonders that you can't find a
volunteer for ethical management on a technical mailing list.

Sigh.

JKL

--
I could contain traces of nuts.

Re: New maintainer

[Tassilo Horn]

jens.k.loewe@googlemail.com (Jens K. Loewe) writes:

>> The whole point of GNU is the non-acceptance of software denying the
>> users the fundamental software freedoms.
>
> So GNU does not stand *for* something but *against* something? GNU's
> whole point is not to be better than others but to point out that
> others are worse? That reminds me of a kindergarten.

The mission of GNU is to give users the ability to do all their
computing needs with software that respects their freedom.  Raising
awareness that software has also non-technical aspects is also a point
of that.

> Furthermore, shouldn't it be the right of a free user to use a
> non-free system at his own will?  I don't use GNU for the sole reason
> that I get my things done fast on other platforms, but I solve a lot
> of problems with FLOSS software even on closed platforms.  Am I a bad
> guy now?

Obviously not.  A user should have the ultimate freedom to use whatever
he prefers, and of course, we advocate for making users aware of freedom
being an important factor.  But from the point of view of GNU, when the
possibilities he can choose from don't include a free alternative,
that's an unacceptable situation where GNU must take action.  That could
mean providing a free alternative or advocating against that thing in
case there cannot be a free alternative (e.g., for DRM there cannot be a
free alternative).

> However, it seems that the Emacs product is less important than the
> Emacs "philosophy" here.

It's actually very simple: there won't be a technical feature which
relies on or works best with proprietary components because that would
encourage users to give up some of their freedom for technical merits.

Bye,
Tassilo

Re: New maintainer

[David Kastrup]

jens.k.loewe@googlemail.com (Jens K. Loewe) writes:

> Ah, finally an interesting discussion here.
>
>> The whole point of GNU is the non-acceptance of software denying
>> the users the fundamental software freedoms.
>
> So GNU does not stand *for* something but *against* something?

"Justice" and "fairness" and "freedom" are things that stand
fundamentally against letting things run their natural course,
downwards.

> GNU's whole point is not to be better than others but to point out
> that others are worse?

Nonsense.  GNU's point is to stand for software that is and remains free
to share, modify, run, and study.  It does not do so in a vacuum: it
would not be necessary to take a stand if there was an overall trend to
respect user and developer freedom.  Taking a stand does not happen in a
vacuum.

> That reminds me of a kindergarten.
>
> Furthermore, shouldn't it be the right of a free user to use a
> non-free system at his own will?

It is the definition of a non-free system that you cannot use it at your
own will in many respects.  At any rate, freedom does not mean a
different prescribed path but a choice.  How you use the choice is yours
but it is important that you have the choice in the first place.

The difference between a home and a prison is not how often you go
outside, but whether you can.

The point of the GNU project is to support free software.

If you are to manage a vegetarian fair and your idea of improvement
focuses on the sorely missing hot dog stands and you think a show
tannery a great addition, you are out of your depth.  That's different
from hiring bouncers who keep everybody out wearing leather shows or
stuff.

But a project manager is not managing the visitors, he is managing the
booths and stalls.  And regardless of whether he or she likes a nice
juicy steak at home, a steak hut does not belong on the fair.  And if
his main motivation for organizing the fair was getting the best steak
hut far and wide, he is a mismatch for the fair, even if 90% of all
visitors happen to eat meat at home.

GNU has a message, and as a core project manager one cannot afford to
ignore it.

A church custodian does not need to be devout, but it won't be
acceptable to celebrate orgies in the church either.  He needs to be
aware what the subject of his job and its sensibilities are and deal
respectably with it.

> I don't use GNU for the sole reason that I get my things done fast on
> other platforms, but I solve a lot of problems with FLOSS software
> even on closed platforms. Am I a bad guy now?

Sounds like a utilitarian to me.  I'm not sure why so many people insist
they deserve special praise for going the path of least resistance and
why they consider people offensive who try doing better than that.

I get this sort of defensive-aggressiveness a lot.  I'm not interested
in the life stories and explanations of people who need to convince
themself of something by talking down my life choices.

> I actually considered suggesting myself as a maintainer just to
> contribute at least something to Emacs. I never wrote "upstream" code
> for Emacs but I do that team development thing for money. I guess as a
> Windows and BSD user I'm out of the game though - I disagree with some
> of the points the GPL makes and I'll always prefer the BSD
> infrastructure to GNU; but I really like working with Emacs. However,
> it seems that the Emacs product is less important than the Emacs
> "philosophy" here. No wonders that you can't find a volunteer for
> ethical management on a technical mailing list.

Nobody is looking for an "ethical manager" here.  The FSF has one
already.  But even a technical manager needs to be aware of the
principles of the GNU project and deal with the tasks responsibly that
those imply for the technical management.  Stuff like making sure that
the respective copyright assignments for contributions are in place,
even if neither the manager nor the contributor believes in them, making
sure that the policies with regard to software only available for
specific platforms are heeded, and even making sure that stuff like
plugin interfaces is done in a manner consistent with the discussions
with Richard and other key persons.

That can mean that a technical manager not invested into GNU's
philosophy will likely have to deal with a few things he considers
technically awkward.  That's hopefully a minuscule part of the job, but
when it does occur it still needs to be done in a responsible and
responsive manner.

-- 
David Kastrup

Re: New maintainer

[Jens K. Loewe]

David Kastrup schrob am 04. Okt. 2015 um 17:49 Uhr dies:

> "Justice" and "fairness" and "freedom" are things that stand
> fundamentally against letting things run their natural course,
> downwards.

So you say it's wrong to let the user work with the best tool for his
job if it's not free?

> It is the definition of a non-free system that you cannot use it at your
> own will in many respects.

This implies that every user has the will to know how his preferred tool
does the job he wants to use it for. This is wrong. Let's take the
famous example "write a letter": Why do you think your Grandma has to
know the internals of Microsoft Word when Microsoft Word is the tool
that makes her finish any letter faster than LibO or OOo?

> The point of the GNU project is to support free software.

That sounds different and better from "the point is to fight non-free
software", do you agree?

> If you are to manage a vegetarian fair and your idea of improvement
> focuses on the sorely missing hot dog stands and you think a show
> tannery a great addition, you are out of your depth.

I use GNU tools (well, at least Emacs) on non-free systems and I
strongly disagree with your interpretation. I don't say "GNU should make
more Windows software" and I even agree with the ethical disapproval of
Apple's products, *but* I say that a focus on so-called "free systems"
is implying a restriction. Not being actively interested in maintaining
compatibility with Windows means a lock-in for non-Windows systems.

Being able to use GNU tools on non-GNU systems often leads to the
thought that switching the OS would not be much effort.

> And if his main motivation for organizing the fair was getting the
> best steak hut far and wide, he is a mismatch for the fair, even if
> 90% of all visitors happen to eat meat at home.

The best free tool is a tool that does not require you to change your
operating system, do you agree?

> A church custodian does not need to be devout, but it won't be
> acceptable to celebrate orgies in the church either.

No one implied to do so. None of the aspirants for the GNU Emacs
maintainership suggested to start a Windows party on this mailing list,
did we?

There's a huge difference between maintaining a software and living an
idea. Can a software only be good when its maintainer has /the right
mind/?

> Nobody is looking for an "ethical manager" here.  The FSF has one
> already.  But even a technical manager needs to be aware of the
> principles of the GNU project and deal with the tasks responsibly that
> those imply for the technical management.

I know the difference between agreement and awareness. I'm pretty sure
that even I could follow the rules of the GPL and the GNU manifesto
without necessarily agreeing to all of them. Managing a task is
different from doing that task.

I can enforce a license which I don't comply with for my own stuff when
it is a part of my job (which it would be in this case). I guess that
applies to everyone here.

> That can mean that a technical manager not invested into GNU's
> philosophy will likely have to deal with a few things he considers
> technically awkward.

Which is OK as long as it does not have a major influence in what he
does IMO.

JKL


-- 
I could contain traces of nuts.

Re: New maintainer

[John Wiegley]

>>>>> Jens K Loewe <jens.k.loewe@googlemail.com> writes:

> So you say it's wrong to let the user work with the best tool for his job if
> it's not free?

Hi Jens,

I'm not sure emacs-devel is still the place to continue this discussion,
especially since we haven't found a new maintainer yet, and that was the
objective for this thread.

I believe the FSF has other forums where you could raise this concern, if
someone from that group could point them out.

John

Re: New maintainer

[David Kastrup]

jens.k.loewe@googlemail.com (Jens K. Loewe) writes:

> David Kastrup schrob am 04. Okt. 2015 um 17:49 Uhr dies:
>
>> The point of the GNU project is to support free software.
>
> That sounds different and better from "the point is to fight non-free
> software", do you agree?

Since there enough users who do not care one bit, you don't get one
without the other.  The playfield is tilted.

The whole point of the GPL is to reject non-free software rather than
not care any which way like the BSD licenses do.

>> If you are to manage a vegetarian fair and your idea of improvement
>> focuses on the sorely missing hot dog stands and you think a show
>> tannery a great addition, you are out of your depth.
>
> I use GNU tools (well, at least Emacs) on non-free systems and I
> strongly disagree with your interpretation. I don't say "GNU should
> make more Windows software" and I even agree with the ethical
> disapproval of Apple's products, *but* I say that a focus on so-called
> "free systems" is implying a restriction.

Oh definitely.  And the GPL is restricting software to be used only as a
building piece for free software.  That's its sole point.

> Not being actively interested in maintaining compatibility with
> Windows means a lock-in for non-Windows systems.
>
> Being able to use GNU tools on non-GNU systems often leads to the
> thought that switching the OS would not be much effort.

Look, Windows 10 contains keyloggers that record any key combinations of
you and send them to Redmond to "make your computing experience more
enjoyable".  They reserve the right to sell every information they gain
about you for any purpose.  The operating system is malware to a degree
where the most money-grabbing software company in the world does not
want you to pay a dime for it (likely because then you'd have some
minimal rights) and will force the upgrade on users of older Windows
systems.

People for whom such a system is a serious consideration do not care
about freedom one bit.  Bending over backwards for them so that they can
keep getting screwed over is not going to make them change their mind.
We don't really have more convincing arguments to change from Windows or
MacOSX to offer than the Windows EULA and privacy agreements and MacOSX
licenses.  If you can read through all that and wholeheartedly state
"I agree", then we don't have anything to offer, really.

For every new Windows system, the EULA has become more ridiculous.
MacOS was comparatively benign in that regard for a long time since the
hardware (and to some degree the not-easily-replaceable firmware ROMs)
served as a sort of dongle, not necessitating additional thumbscrews
over what copyright already provided.

But right now by far the most important consideration has to be to make
GNU software work well in connection with free systems since the unfree
systems are so far off the chart that catering for them is not providing
a free work environment in sufficiently tangible respects.

>> And if his main motivation for organizing the fair was getting the
>> best steak hut far and wide, he is a mismatch for the fair, even if
>> 90% of all visitors happen to eat meat at home.
>
> The best free tool is a tool that does not require you to change your
> operating system, do you agree?

The best vegetarian dinner is a sideplate to a juicy steak, do you
agree?

>> A church custodian does not need to be devout, but it won't be
>> acceptable to celebrate orgies in the church either.
>
> No one implied to do so. None of the aspirants for the GNU Emacs
> maintainership suggested to start a Windows party on this mailing
> list, did we?

The issue here was actually MacOSX, resulting in a reminder of the rule
that no functionality is to be introduced for non-free operating systems
that is not available for GNU/Linux.

> There's a huge difference between maintaining a software and living an
> idea. Can a software only be good when its maintainer has /the right
> mind/?

You seem to be fond of flogging that strawman.  I repeat: the maintainer
does not need to have certain personal beliefs but he must be able to
understand and follow the rules governing and underlying the project.

>> That can mean that a technical manager not invested into GNU's
>> philosophy will likely have to deal with a few things he considers
>> technically awkward.
>
> Which is OK as long as it does not have a major influence in what he
> does IMO.

In some respects, it will.  That makes it harder to find a person for
what amounts to an unpaid position since it means that the only person
doing such a job will have strong reasons of his own.  And when those
reasons are partly in conflict with the FSF's goals, that takes an
impact on the satisfaction one can derive from such a job as one's
metric for doing it well might not be in complete concord with the FSF's
metric.

-- 
David Kastrup

Re: New maintainer

[Jens K. Loewe]

David Kastrup schrob am 04. Okt. 2015 um 23:34 Uhr dies:

> Since there enough users who do not care one bit, you don't get one
> without the other.  The playfield is tilted.

I guess the FSF is in a position where they can shuffle the playfield if
they'd want to.

> Oh definitely.  And the GPL is restricting software to be used only as
> a building piece for free software.  That's its sole point.

So the actual point of this was that a GNU Emacs maintainer has to
enforce software restrictions to users? That sounds wrong.

> Look, Windows 10 contains keyloggers that record any key combinations of
> you and send them to Redmond to "make your computing experience more
> enjoyable".

You might have missed it, but Windows 10 *actually asks you* if you want
to enable this feature during the first set-up steps, so it's basically
opt-in. People who propagate the right to choose should appreciate this
IMO.

> People for whom such a system is a serious consideration do not care
> about freedom one bit.

This is not valid. I like my freedom, I use quite a couple of different
operating systems, both free and non-free. The free ones are mostly used
for technical reasons indeed, but I still do most of my work on
Windows. Not because Windows was free or something, but because I can
get my things done with it. Switching to another main OS would force me
to develop an entirely new workflow - which would decrease my
productivity.

I still don't think I don't care about freedom. I might repeat myself: I
am an active contributor to and user of some FLOSS projects (including
some you might know), I just happen to have the /perfect/ workflow on
Windows. Don't condemn me for wishing to use a tool (my computer) as a
tool and not as a church.

> If you can read through all that and wholeheartedly state "I agree",
> then we don't have anything to offer, really.

How many people have read Canonical's terms of service in the first
place?

> The best vegetarian dinner is a sideplate to a juicy steak, do you
> agree?

I like steak, so no complaints on this point... ;-)

> The issue here was actually MacOSX, resulting in a reminder of the rule
> that no functionality is to be introduced for non-free operating systems
> that is not available for GNU/Linux.

Which is a pretty OK rule, I just wanted you to consider that having a
"broken by design" Windows or OSX version of a GNU tool will rather make
people drop the tool than their operating system.

>> Which is OK as long as it does not have a major influence in what he
>> does IMO.
>
> In some respects, it will.

Which would be those respects?

> That makes it harder to find a person for
> what amounts to an unpaid position since it means that the only person
> doing such a job will have strong reasons of his own.

The reason I suggested myself was that I wanted to contribute something
to GNU Emacs because I love it and its community and I wish that I can
give you something back. I guess that reason is strong enough and not in
conflict with anything?

Regards.

JKL


-- 
I could contain traces of nuts.

Re: New maintainer

[Dmitry Gutov]

On 10/05/2015 01:28 AM, Jens K. Loewe wrote:

> Which is a pretty OK rule, I just wanted you to consider that having a
> "broken by design" Windows or OSX version of a GNU tool will rather make
> people drop the tool than their operating system.

Why don't we stop this thread of discussion right here?

Emacs is highly unlikely to become "broken by design" on Windows or OS X 
while there are volunteers willing to contribute their time to that purpose.

The only "breakage" that's likely to occur is Emacs can have features 
not working there, while being functional on GNU platforms. That will 
also be predicated on whether a feature in question needs special 
support for the OS in question, and whether there are volunteers 
interested in contributing that support.

Re: New maintainer

[David Kastrup]

jens.k.loewe@googlemail.com (Jens K. Loewe) writes:

> David Kastrup schrob am 04. Okt. 2015 um 23:34 Uhr dies:
>
>> Since there enough users who do not care one bit, you don't get one
>> without the other.  The playfield is tilted.
>
> I guess the FSF is in a position where they can shuffle the playfield
> if they'd want to.

Nonsense.  They can just level their corner.

>> Oh definitely.  And the GPL is restricting software to be used only
>> as a building piece for free software.  That's its sole point.
>
> So the actual point of this was that a GNU Emacs maintainer has to
> enforce software restrictions to users? That sounds wrong.

Shrug.  I don't see the point in providing a target for you letting you
twist words around.  You obviously have an agenda from which you don't
want to get deterred.  The GPL works by posing restrictions to the
manner in which software is subverted, and a GNU project maintainer is
expected to keep projects in a state where the GPL is effective in
ensuring software freedom.

This won't get better or worse by you trying to put absurd spins on it.

>> Look, Windows 10 contains keyloggers that record any key combinations
>> of you and send them to Redmond to "make your computing experience
>> more enjoyable".
>
> You might have missed it, but Windows 10 *actually asks you* if you
> want to enable this feature during the first set-up steps, so it's
> basically opt-in. People who propagate the right to choose should
> appreciate this IMO.

You might have missed it, but Windows 10 goes ahead nevertheless.  So
far testers have not been able to find any settings that would not send
a continuous string of data related to keypresses to Microsoft servers.

>> People for whom such a system is a serious consideration do not care
>> about freedom one bit.
>
> I still don't think I don't care about freedom. I might repeat myself:
> I am an active contributor to and user of some FLOSS projects
> (including some you might know), I just happen to have the /perfect/
> workflow on Windows.

Perfect means that there is nothing wrong with it for you, so "do not
care about freedom one bit" seems to be a pretty good description.

> Don't condemn me for wishing to use a tool (my computer) as a tool and
> not as a church.

I don't condemn you.  I state that you don't care for freedom one bit.
And you state that perfection for you does not involve freedom.  And
that people for which this is different are using their computer as a
church.  Which presumably is something bad.

>> If you can read through all that and wholeheartedly state "I agree",
>> then we don't have anything to offer, really.
>
> How many people have read Canonical's terms of service in the first
> place?

Uh, there's not all that much in there if you don't actually enter
service contracts.  Their contributor agreements are more on the
debatable side, but they don't concern end user freedom all that much.

>> The best vegetarian dinner is a sideplate to a juicy steak, do you
>> agree?
>
> I like steak, so no complaints on this point... ;-)

But you do understand that you have to leave that preference at home
when you agree to organize a vegetarian fair or you'll be the wrong
person for the job?

-- 
David Kastrup

Re: New maintainer

[Jens K. Loewe]

David Kastrup schrob am 05. Okt. 2015 um 01:20 Uhr dies:

> Nonsense.  They can just level their corner.

They have a loyal fanbase though. (Don't take this word too seriously
please.)

> You obviously have an agenda from which you don't want to get
> deterred.

My so-called "agenda" is to try to understand what I /might/ have got
wrong in order to be able to keep up with the discussions. I just don't
consider something -- not even "this or that is freedom" -- to be right
just because someone said so. Sorry.

> a GNU project maintainer is expected to keep projects in a state where
> the GPL is effective in ensuring software freedom.

Which, again, is OK and perfectly understandable for me.

> You might have missed it, but Windows 10 goes ahead nevertheless.  So
> far testers have not been able to find any settings that would not
> send a continuous string of data related to keypresses to Microsoft
> servers.

Yes, I have missed that yet. Thank you.

> Perfect means that there is nothing wrong with it for you, so "do not
> care about freedom one bit" seems to be a pretty good description.

This is the difference between using a computer as a tool and using a
computer as a church again. I know that Windows does not let me have all
the freedom I could want to (this is one of the reasons why I own other
machines with other operating systems too), it's just that I, sometimes,
don't need to have (e.g.) a way to patch my Windows kernel. Sometimes I
just want to use it as it was meant to be used.

I'd happily drop Windows when someone invents a free operating system
which provides adequate alternatives to the proprietary software I use.

> I don't condemn you.  I state that you don't care for freedom one bit.
> And you state that perfection for you does not involve freedom.  And
> that people for which this is different are using their computer as a
> church.  Which presumably is something bad.

Well, yes. A computer is a tool to me, like a hammer is a tool. I use a
hammer to drive a nail into the wall, I use a computer to communicate
with people (in a couple of ways, admittedly). When I need something to
drive a nail into a wall, it does not matter much to me if I can replace
its head with a different head as long as it gets its task done.

It's still wrong that I don't care for freedom, I'm just skeptical about
enforcing freedom as /enforcing/ is quite the opposite of
/freeing/. But, again, that does not mean that I come with the "wrong
mind". My personal choice of software is a result of my freedom to
choose, and you won't ever see my advocating a closed system unless for
a very good reason.

> But you do understand that you have to leave that preference at home
> when you agree to organize a vegetarian fair or you'll be the wrong
> person for the job?

Yes, of course. I never said I wouldn't. However, as of now, I'm here as
a non-representative user, so everything I write is not a part of a
job. If a job involved advocating the GPL, I'd follow the rules.

JKL


-- 
I could contain traces of nuts.

Re: New maintainer

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Date: Mon, 05 Oct 2015 01:20:16 +0200
> Cc: emacs-devel@gnu.org
> 
> >> Look, Windows 10 contains keyloggers that record any key combinations
> >> of you and send them to Redmond to "make your computing experience
> >> more enjoyable".
> >
> > You might have missed it, but Windows 10 *actually asks you* if you
> > want to enable this feature during the first set-up steps, so it's
> > basically opt-in. People who propagate the right to choose should
> > appreciate this IMO.
> 
> You might have missed it, but Windows 10 goes ahead nevertheless.  So
> far testers have not been able to find any settings that would not send
> a continuous string of data related to keypresses to Microsoft servers.

Google did, among its first few hits:

  http://www.express.co.uk/life-style/science-technology/603524/Windows-10-Microsoft-Key-Logger-Record-Privacy
  http://arstechnica.com/information-technology/2015/08/windows-10-doesnt-offer-much-privacy-by-default-heres-how-to-fix-it/

Re: New maintainer

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Date: Mon, 05 Oct 2015 01:20:16 +0200
>> Cc: emacs-devel@gnu.org
>> 
>> >> Look, Windows 10 contains keyloggers that record any key combinations
>> >> of you and send them to Redmond to "make your computing experience
>> >> more enjoyable".
>> >
>> > You might have missed it, but Windows 10 *actually asks you* if you
>> > want to enable this feature during the first set-up steps, so it's
>> > basically opt-in. People who propagate the right to choose should
>> > appreciate this IMO.
>> 
>> You might have missed it, but Windows 10 goes ahead nevertheless.  So
>> far testers have not been able to find any settings that would not send
>> a continuous string of data related to keypresses to Microsoft servers.
>
> Google did, among its first few hits:
>
>   http://www.express.co.uk/life-style/science-technology/603524/Windows-10-Microsoft-Key-Logger-Record-Privacy
>   http://arstechnica.com/information-technology/2015/08/windows-10-doesnt-offer-much-privacy-by-default-heres-how-to-fix-it/

Ah, but turning those settings off does not really suffice.

<URL:http://arstechnica.com/information-technology/2015/08/even-when-told-not-to-windows-10-just-cant-stop-talking-to-microsoft/>

-- 
David Kastrup

Re: New maintainer

[Artur Malabarba]

[-- Attachment #1: Type: text/plain, Size: 185 bytes --]

There's a lot of non-maintainer-related discussion here. Everyone please
(at least) branch these discussions off into their own subjects and stop
drowning the  "New maintainer" thread.

[-- Attachment #2: Type: text/html, Size: 219 bytes --]

Re: New maintainer

[Michael Westbom]

[-- Attachment #1: Type: text/plain, Size: 509 bytes --]

I second the motion to move unrelated topics of discussion to a new thread or different forum—fascinating as they may be (I hope, wherever the discussion lands next, that I may continue to follow).



-- 
Michael Westbom

On October 5, 2015 at 2:56:46 AM, Artur Malabarba (bruce.connor.am@gmail.com) wrote:

There's a lot of non-maintainer-related discussion here. Everyone please (at least) branch these discussions off into their own subjects and stop drowning the  "New maintainer" thread.


[-- Attachment #2: Type: text/html, Size: 4141 bytes --]

Re: New maintainer

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Cc: jens.k.loewe@googlemail.com,  emacs-devel@gnu.org
> Date: Mon, 05 Oct 2015 08:15:41 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: David Kastrup <dak@gnu.org>
> >> Date: Mon, 05 Oct 2015 01:20:16 +0200
> >> Cc: emacs-devel@gnu.org
> >> 
> >> You might have missed it, but Windows 10 goes ahead nevertheless.  So
> >> far testers have not been able to find any settings that would not send
> >> a continuous string of data related to keypresses to Microsoft servers.
> >
> > Google did, among its first few hits:
> >
> >   http://www.express.co.uk/life-style/science-technology/603524/Windows-10-Microsoft-Key-Logger-Record-Privacy
> >   http://arstechnica.com/information-technology/2015/08/windows-10-doesnt-offer-much-privacy-by-default-heres-how-to-fix-it/
> 
> Ah, but turning those settings off does not really suffice.
> 
> <URL:http://arstechnica.com/information-technology/2015/08/even-when-told-not-to-windows-10-just-cant-stop-talking-to-microsoft/>

Which includes further advice on how to turn the other stuff off.

Look, this particular point of yours is simply invalid: for any
problem encountered on Windows, there are always solutions and/or
workarounds that work.  It sometimes takes time to discover them
(Windows 10 was released only 2.5 months ago), but eventually they do
surface.  The enormously large number of people using Windows and the
basic user desire to solve problems they bump into, coupled with the
Internet and the efficiency of the search engines, makes any such
malware-like features easily bypassed for those who don't want them.

The basis for the Free Software movement and the GPL, as I understand
it, was, and still is, that denying free access to the software
sources prevents the free flow and exchange of ideas, which is
immoral, unethical, and detrimental to progress, and whose only
justification is greed and the desire to increase profits.  The
supposedly easier way of fixing bugs and adding features by the user
herself is IMO a beneficial side-effect, and is today limited to
adding features (as workarounds for bugs will usually be found by
googling).  The free flow of ideas argument is still very much valid,
although it, too, is not black-and-white anymore, because there's
enormous amount of information out there, both in books and in
articles, that describes the inner workings of Windows (don't know
about MacOS, OS X etc.) to astonishing depth and detail.

Bottom line, I submit that representing the split as black-and-white
chasm-like one is nowadays a misrepresentation.  Things are more like
shades of gray, not in the least due to the fact that the modern
societies are much more open than they were in 1980s.  Which goes a
long way towards explaining why people don't easily see the core of
the GPL's rationale and quite a few are sincerely confused about the
Free Software movement's main principles and ideas.  It also makes our
job explaining those ideas quite a bit harder.  But trying to make it
easier by representing the issues as black-and-white is not TRT, IME,
and it will always fail given an intelligent enough opponent who knows
her ground.  You (not you personally, David) can even be accused in
trying to con your audience by false statements, which then might have
far-reaching effect on our argumentation in general.

Re: New maintainer

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Cc: jens.k.loewe@googlemail.com,  emacs-devel@gnu.org
>> Date: Mon, 05 Oct 2015 08:15:41 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> From: David Kastrup <dak@gnu.org>
>> >> Date: Mon, 05 Oct 2015 01:20:16 +0200
>> >> Cc: emacs-devel@gnu.org
>> >> 
>> >> You might have missed it, but Windows 10 goes ahead nevertheless.  So
>> >> far testers have not been able to find any settings that would not send
>> >> a continuous string of data related to keypresses to Microsoft servers.
>> >
>> > Google did, among its first few hits:
>> >
>> >   http://www.express.co.uk/life-style/science-technology/603524/Windows-10-Microsoft-Key-Logger-Record-Privacy
>> >   http://arstechnica.com/information-technology/2015/08/windows-10-doesnt-offer-much-privacy-by-default-heres-how-to-fix-it/
>> 
>> Ah, but turning those settings off does not really suffice.
>> 
>> <URL:http://arstechnica.com/information-technology/2015/08/even-when-told-not-to-windows-10-just-cant-stop-talking-to-microsoft/>
>
> Which includes further advice on how to turn the other stuff off.

Have we been reading the same article?  "We've asked Microsoft if there
is any way to disable this additional communication or information about
what its purpose is." ... "if Web searching and Cortana are disabled, we
suspect that the inference that most people would make is that searching
the Start menu wouldn't hit the Internet at all. But it does. The
traffic could be innocuous, but the inclusion of a machine ID gives it a
suspicious appearance."

> Look, this particular point of yours is simply invalid: for any
> problem encountered on Windows, there are always solutions and/or
> workarounds that work.  It sometimes takes time to discover them
> (Windows 10 was released only 2.5 months ago), but eventually they do
> surface.  The enormously large number of people using Windows and the
> basic user desire to solve problems they bump into, coupled with the
> Internet and the efficiency of the search engines, makes any such
> malware-like features easily bypassed for those who don't want them.

Do you or don't you agree that there is a point for an operating system
respecting the user's choices, privacy and control?  Because we are
talking here about the need of a prospective Emacs manager to heed the
policies designed to ensuring that such a system remains available in
future and cannot be watered down easily.  Even while the availability
of such systems depressingly appears to have very little impact on
users' choices, it's the GNU mission to make sure that for those few who
actually care, the choice remains.

With Emacs being a core component of GNU, the maintainer needs to accept
that he is not independently responsible for Emacs but also for GNU.

> But trying to make it easier by representing the issues as
> black-and-white is not TRT, IME, and it will always fail given an
> intelligent enough opponent who knows her ground.  You (not you
> personally, David) can even be accused in trying to con your audience
> by false statements, which then might have far-reaching effect on our
> argumentation in general.

I wish.  Unfortunately, the world is moving away from shades of grey and
rather thoroughly into contrasts increasingly indistinguishable from
black-and-white.  The idea that systems like GNU were needed to keep the
big players in check by offering a free alternative has mostly failed.
It does not help that MacOSX is these days built based on a free but
non-copylefted platform.  And even the Android universe built around a
GPLed kernel is at best a thoroughly mixed blessing and its effective
level of freedom is much more determined by the marketing department of
Google than by the workers on the free software base they are employing.

Oh, and Google, in the course of changing its company name has also
chosen to drop its motto "do no evil" in favor of the fuzzier and less
absolute "do the right thing".

That is the general situation where the FSF with the GNU project tries
to carve out and maintain a harbor of freedom and Emacs is one of the
few core products and resources it has to work with, and so it is
important that the maintainer will respect the aspects of his work
concerned with that.  It's not like they push themselves in the
foreground that often.

In my opinion, _particularly_ if you care about recommending and working
with Windows and MacOSX, you should be concerned about a solid
counterweight that those systems may be held against in their current
downward spiral.  That decline can only be slowed if people start caring
about their rights and privacy, and a viable alternative is the most
important asset for that.

-- 
David Kastrup

Re: New maintainer

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Cc: jens.k.loewe@googlemail.com,  emacs-devel@gnu.org
> Date: Mon, 05 Oct 2015 09:36:30 +0200
> 
> >> Ah, but turning those settings off does not really suffice.
> >> 
> >> <URL:http://arstechnica.com/information-technology/2015/08/even-when-told-not-to-windows-10-just-cant-stop-talking-to-microsoft/>
> >
> > Which includes further advice on how to turn the other stuff off.
> 
> Have we been reading the same article?

Yes.  It's a long article, not just the 2 sentences you've cited.

> "We've asked Microsoft if there is any way to disable this
> additional communication or information about what its purpose is."
> ... "if Web searching and Cortana are disabled, we suspect that the
> inference that most people would make is that searching the Start
> menu wouldn't hit the Internet at all. But it does. The traffic
> could be innocuous, but the inclusion of a machine ID gives it a
> suspicious appearance."

"We suspect", "could be innocuous", "suspicious appearance", etc. here
mean that the authors don't really know themselves whether there is a
problem.

In any case, wait a bit, and more information will surely surface
about disabling that, if there is indeed a problem.

> Do you or don't you agree that there is a point for an operating system
> respecting the user's choices, privacy and control?

I do, but the GPL in my reading is not about privacy and choices.
There could be a GPL'ed program that doesn't care about privacy and
gives the users little control on its operation.  (It would be a bad
program, IMO.)

Protecting privacy and giving users more control are all good
principles for building user-friendly software, and I very much hope
Emacs and the rest of the GNU project will do that.  But they have
nothing to do with software freedom of the GPL itself, IMO.  Lumping
them together just muddies the waters.

> Because we are
> talking here about the need of a prospective Emacs manager to heed the
> policies designed to ensuring that such a system remains available in
> future and cannot be watered down easily.  Even while the availability
> of such systems depressingly appears to have very little impact on
> users' choices, it's the GNU mission to make sure that for those few who
> actually care, the choice remains.
> 
> With Emacs being a core component of GNU, the maintainer needs to accept
> that he is not independently responsible for Emacs but also for GNU.

All true and agreed.  All I'm saying is that the specific point you
were trying to make regarding malware-type misfeatures of Windows 10
is not relevant to this.

> > But trying to make it easier by representing the issues as
> > black-and-white is not TRT, IME, and it will always fail given an
> > intelligent enough opponent who knows her ground.  You (not you
> > personally, David) can even be accused in trying to con your audience
> > by false statements, which then might have far-reaching effect on our
> > argumentation in general.
> 
> I wish.  Unfortunately, the world is moving away from shades of grey and
> rather thoroughly into contrasts increasingly indistinguishable from
> black-and-white.

I strongly disagree, but I don't want to argue about that, certainly
not here.

> if you care about recommending and working with Windows and MacOSX

I don't.

Re: New maintainer

[Michael Westbom]

[-- Attachment #1: Type: text/plain, Size: 612 bytes --]

I’m new to this mailing list and happen to find it in a morass of maintainership discussion quite by coincidence; weigh my comments as you will.

I’d like to offer a solid +1 to Mr. Wiegley’s maintainership of Emacs.

After having witnessed his conduct on this thread, as well as being familiar with his name through seeing it on the by-line of many an excellent work over the last couple years (in other words, about as long as I’ve been using Emacs), I believe his depth of knowledge, clarity of vision, and capacity for technical writing are well suited to the task.



-- 
Michael Westbom

[-- Attachment #2: Type: text/html, Size: 6950 bytes --]

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I do, but the GPL in my reading is not about privacy and choices.
  > There could be a GPL'ed program that doesn't care about privacy and
  > gives the users little control on its operation.

Free software does not guarantee the absence of malfeatures,
but free software puts the users in control of the program
which enables the user community to fix any malfeatures.
This in turn reduces the temptation to introduce malfeatures.

See http://gnu.org/philosophy/free-software-even-more-important.html
and fsf.org/tedx.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Ah, but turning those settings off does not really suffice.

  > <URL:http://arstechnica.com/information-technology/2015/08/even-when-told-not-to-windows-10-just-cant-stop-talking-to-microsoft/>

For more information about known malware functionalities
in many proprietary programs see http://gnu.org/proprietary/.

If you find an error in that information, or if you know about other
cases to add, please report it to webmasters@gnu.org.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Ricardo Wurmus]

Jens K. Loewe <jens.k.loewe@googlemail.com> writes:

>> The issue here was actually MacOSX, resulting in a reminder of the rule
>> that no functionality is to be introduced for non-free operating systems
>> that is not available for GNU/Linux.
>
> Which is a pretty OK rule, I just wanted you to consider that having a
> "broken by design" Windows or OSX version of a GNU tool will rather make
> people drop the tool than their operating system.

Just a data point: the frustrating experience of using Emacs on MacOS
has resulted in many complaints from a user about ... MacOS.  The user
is considering to move to a system that doesn’t arbitrarily restrict
freedom to the degree that it is done on MacOS.

(This is in the context of the world of bioinformatics, in which Windows
is hardly important, Macs are often used and complained about, and GNU
is the system that is actually supported by the local IT department.)

A tool is hardly “broken by design” when it works better on a particular
kind of system.  I rather think that it is a matter of degree: if the
tool is hardly usable at all, it probably would be dropped without a
second thought; if it can be used, even if not to the full extent as it
is useful on a free system, it can be enough to tip the scales in favour
of the free system.

~~ vegetarian Ricardo

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

We don't need to debate about whether in principle GNU Emacs should
have support for MacOS.  There is a general GNU Project policy about
this.

The policy is non-GNU systems are secondary, and lower priority than
the GNU system, but we are glad to include support for them in GNU
packages if users contribute the necessary code -- provided that code
isn't a maintenance problem for us.

The maintenainers of any particular package are the ones who judge
whether that code is a maintenance problem, since they are the ones
it would be a problem for.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

This list is for developing GNU Emacs.  I state the GNU Project's
philosophy and rules here when that's pertinent to developing Emacs.

If you want to argue about the GNU Project's basic philosophy, we have
another list for that: gnu-misc-discuss@gnu.org.  Please take your
arguments there.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I think you've put it rightly and clearly.  We don't exclude people
who use proprietary software, or even people who develope proprietary
software, from using Emacs or even from developing Emacs.

I will mention one abstract subpoint where I disagree with what you
said.

  >   At any rate, freedom does not mean a
  > different prescribed path but a choice.

I think that defining freedom as "choice" is a mistaken approach.  All
else being equal, more freedom generally means you have more choices,
but defining freedom in terms of the number of choices leads to
paradoxes.

I define freedom as "having control of one's own life".
Freedom in one's computing means having control over how it is done,
which requires control over the software that does it.

Your church analogy was a good one, but let's make sure no one
misunderstands it as more than an analogy.  The Church of Emacs is
pure humor.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> I define freedom as "having control of one's own life". Freedom in one's
> computing means having control over how it is done, which requires control
> over the software that does it.

Even those of us who are fine using proprietary software have been bitten by
its lack of freedom. Why, I can't even run OS X on hardware of my own
choosing!

Let me be clear on the question of freedom: while I have disagreements with
you, I also don't consider any existing alternative to be ideal. I see pros
and cons in all of them. I choose BSD3 for my own software because it
represents the least commitment to any specific path.

So while I may disagree with the GPL as "The Way", I'm not opposed to helping
it succeed a little bit either. I think all these experiments need more time,
before we'll discover what truly helps our society progress in the ways we all
want it to.

> Your church analogy was a good one, but let's make sure no one
> misunderstands it as more than an analogy. The Church of Emacs is pure
> humor.

The church analogy, qua analogy, was pretty excellent. Point taken!

John

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Furthermore, shouldn't it be the right of a free user to use a non-free
  > system at his own will?

We have never tried to people from running nonfree programs.
You seem to be attacking something that didn't come from us.

  > I guess as a Windows and BSD user I'm out of the game though - I
  > disagree with some of the points the GPL makes and I'll always prefer
  > the BSD infrastructure to GNU; but I really like working with
  > Emacs.

We are happy to accept contributions from BSD users -- I am sure you
have seen this -- and BSD is sufficiently similar to GNU that it
should not be an impediment in practice.

It is simply a matter of whether you wish to contribute.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[David Reitter]

John,

> If the experience is great on GNU systems, but awful on OS X,
> I'll see this as meaning that changes needing to be made. Conversely, I won't
> notice degradation on GNU systems owing to cross-platform changes, and would
> require those using such systems to inform me.

Do you consider downstream projects a legitimate way of providing that experience on the basis of GPL principles without diluting the overall mission of the FSF and the GNU project?

By providing support and stability, the upstream project (Emacs) can foster an ecosystem that is ultimately a good advertisement to free software.  Downstream projects can reach a wider audience than the GNU-only system can, and they can incorporate libraries and functionalities that the upstream project is unwilling to adopt for philosophical reasons.  With the right relationship to these projects, some improvements can also be ported back and thus benefit everybody.

Re: New maintainer

[John Wiegley]

>>>>> David Reitter <david.reitter@gmail.com> writes:

> Do you consider downstream projects a legitimate way of providing that
> experience on the basis of GPL principles without diluting the overall
> mission of the FSF and the GNU project?

Can you give an example?  I don't fully understand your question.

> By providing support and stability, the upstream project (Emacs) can foster
> an ecosystem that is ultimately a good advertisement to free software.
> Downstream projects can reach a wider audience than the GNU-only system can,
> and they can incorporate libraries and functionalities that the upstream
> project is unwilling to adopt for philosophical reasons. With the right
> relationship to these projects, some improvements can also be ported back
> and thus benefit everybody.

Sounds nice in principle, but an example would help me to be clear on what
that might look like in practice.

John

Re: New maintainer

[David Reitter]

On Oct 5, 2015, at 6:21 PM, John Wiegley <johnw@newartisans.com> wrote:
> 
>>>>>> David Reitter <david.reitter@gmail.com> writes:
> 
>> Do you consider downstream projects a legitimate way of providing that
>> experience on the basis of GPL principles without diluting the overall
>> mission of the FSF and the GNU project?
> 
> Can you give an example?  I don't fully understand your question.

I’m sorry, I thought that was obvious.

Since 2005, the Aquamacs project has been publishing its own Emacs distribution for the Mac, based on a recent stable or development-stage version of Emacs 22, 23, 24, and soon 25.    The version we distribute is patched at Lisp and C levels, and it comes packaged with a great deal of useful packages, including ESS and AUCTeX.  Last time I checked, we had about 12-14,000 regular users (not just installs).

It’s a downstream project implementing features that do not conform to all of GNU’s and the FSF’s policies, and addressing a non-free (but quite open) platform.  I think it still promotes freedom among those who choose this platform.

D

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It’s a downstream project implementing features that do not
  > conform to all of GNU’s and the FSF’s policies, and addressing a
  > non-free

I have nothing against downstream projects in general.  That might be
a good way to handle MacOS support, if maintaining it in Emacs itself
is inconvenient.  (I seem to recall that that is the case, but I am
not sure.)

You mention that it doesn't follow all the GNU and FSF policies.
That might be very bad, or might not matter much, depending on the details 
not stated.

        (but quite open) platform.

One more illustration that "open" is insufficient.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Dmitry Gutov]

On 10/01/2015 11:46 PM, John Wiegley wrote:

> However: are you looking for someone to act as an arm of the FSF, or do you
> want the best possible Emacs?

I imagine you have to have a certain amount of respect for FSF policies.

Being a mouthpiece for RMS is not required, but liking GPL would help. 
We have enough drama here as it is.

Re: New maintainer

[Mathieu Lirzin]

John Wiegley <johnw@newartisans.com> writes:

> However: are you looking for someone to act as an arm of the FSF, or do you
> want the best possible Emacs?

As already answered by someone else, things are not incompatible.
Moreover it is precisely the purpose of the work done through the GNU
project, which is promoting freedom by technical contributions.

I just think you can make the best possible Emacs without being a
maintainer of it. ;)

--
Mathieu Lirzin

Re: New maintainer

[David Kastrup]

John Wiegley <johnw@newartisans.com> writes:

>>>>>> Mathieu Lirzin <mthl@openmailbox.org> writes:
>
>> What about Ethical skills? I would argue that technical skills are not
>> sufficient especially for maintaining a major GNU package like Emacs. Using
>> MacOSX & iOS as main operating systems and Hangout/Skype for communications,
>> seems incompatible with the role to me.
>
> I also disagree with the spirit of the GPL, vocally in fact. If the
> requisite for maintaining Emacs is that one use (GNU/)Linux and
> espouse the philosophies of RMS, that is not me.
>
> However: are you looking for someone to act as an arm of the FSF, or
> do you want the best possible Emacs?

Well, the GPL is what makes sure that I actually have the right to get
the best possible Emacs once it is distributed anywhere.  A lot of "best
possible Emacsen" lie by the wayside, starting with Gosling Emacs and
arguably ending with XEmacs.

Now Stephen Turnbull, the current XEmacs maintainer for longer than any
of his Emacsen colleagues with the possible exception of RMS, is not
making a point of "disagreeing with the spirit of the GPL" at all even
though it's sort of foisted onto XEmacs.  It's more like they are
driving XEmacs under different work parameters than Emacs is driven,
with different conclusions from the same set of principles.

I don't think that "vocally disagreeing with the spirit of the GPL"
would provide a maintainership retaining whatever it was that has
enabled Emacs to claw back its way to the front time and again.  There
have been a number of heated discussions between RMS and various
maintainers, but they did not concern the "spirit of the GPL" as much as
they did the best choices to make in order to get the most from it.

I wouldn't go as far as calling this "ethical skills" but yes, it seems
like a cultural mismatch that would appear likely to cause considerable
friction in choosing consistent priorities for ongoing development.

-- 
David Kastrup

Re: New maintainer

[Stephen J. Turnbull]

First, I second John's self-nomination.  I suspect it's a no-op given
his "disagreement with the spirit of the GPL", but FWIW I think he
would be an excellent Emacs maintainer.

I've reordered David's post to suit my own purposes, but the
attributions are all correct.

 > >>>>>> Mathieu Lirzin <mthl@openmailbox.org> writes:

 > >> What about Ethical skills?

IMO, John's ethical skills are unquestionable.  However, his
observations of the world and certain fundamental postulates lead him
to different conclusions about what is and isn't ethically required.

 > >> I would argue that technical skills are not sufficient
 > >> especially for maintaining a major GNU package like Emacs. Using
 > >> MacOSX & iOS as main operating systems and Hangout/Skype for
 > >> communications, seems incompatible with the role to me.

I disagree, as John evidently does.  By the way, if I didn't know he
would have self-nominated if he could, I would nominate Eli Zaretskii
as maintainer.  Do you have a problem with him?

 > John Wiegley <johnw@newartisans.com> writes:

 > > I also disagree with the spirit of the GPL, vocally in fact. If the
 > > requisite for maintaining Emacs is that one use (GNU/)Linux and
 > > espouse the philosophies of RMS, that is not me.

I wouldn't require use of the GNU OS, but I'm sorry if I am to
understand that you can't at least defend gnu.org/philosophy in
public.  I think that disqualifies you:

 > > However: are you looking for someone to act as an arm of the FSF,
 > > or do you want the best possible Emacs?

I would assume Richard is looking for an arm of the FSF, as he has
done in the past.  AIUI, a great Emacs must be first, a tool for
advancing software freedom, and second, the best tool for its
applications that it can be.  I have other reasons for not
volunteering myself (lack of technical qualifications and time being
the most easily mentioned), but I would have no trouble with
enthusiastically defending the FSF position *in public* when *wearing
my GNU Emacs maintainer hat*.  That's probably not good enough for
Richard, but I can't imagine anything less being acceptable.

David Kastrup writes:

 > Well, the GPL is what makes sure that I actually have the right to
 > get the best possible Emacs once it is distributed anywhere.  A lot
 > of "best possible Emacsen" lie by the wayside, starting with
 > Gosling Emacs and arguably ending with XEmacs.
 > 
 > Now Stephen Turnbull, the current XEmacs maintainer for longer than any
 > of his Emacsen colleagues with the possible exception of RMS, is not
 > making a point of "disagreeing with the spirit of the GPL" at all

That's too strong.  What I cannot disagree with is use of GPL in a
project that chooses it understanding the ramifications, nor in that
project's downstream.  Richard is a visionary, but he also understands
the practical implications of GPL as well as anybody, and he considers
the principle well worth the problems (and in some cases things that
others consider problems he considers benefits).

As long as Emacs is firmly in the GPL camp, I have no problem
defending the GPL on Emacs lists.  I'd rather let RMS do it
(authoritative), or you or Stefan or Eli do it (more credible), but
occasionally I've stepped up when nobody else does (especially on
technical legal issues).  And I defend the GPL (without my fingers
crossed :-) on XEmacs lists, and not just because it was foisted on
us.  (I can't do the same for the FDL, but that's another matter.)

What I do elsewhere is quite another matter.

The religious principle that software freedom is an inalienable human
right that Jefferson somehow forgot to include in the Bill of Rights
is what I disagree with.  I'm Jeffersonian -- when it comes to freedom,
"The tree of liberty must be refreshed from time to time with the
blood of patriots and tyrants."  We need to all hang together lest
they hang us separately (well, that was Franklin).

 > even though it's sort of foisted onto XEmacs.  It's more like they
 > are driving XEmacs under different work parameters than Emacs is
 > driven, with different conclusions from the same set of principles.
 > 
 > I don't think that "vocally disagreeing with the spirit of the GPL"
 > would provide a maintainership retaining whatever it was that has
 > enabled Emacs to claw back its way to the front time and again.

I don't think that has much to do with the periodic resurgence of
Emacs.  Stefan and Yidong mostly left advocacy of any kind up to RMS.
Stefan himself openly disagreed with RMS over support for non-copyleft
software.  Emacs's GPL being a fact, I would hope John would keep his
mouth shut about his disagreement while wearing his GNU Emacs
maintainer hat (not limited to participation in Emacs lists).

 > I wouldn't go as far as calling this "ethical skills" but yes, it
 > seems like a cultural mismatch that would appear likely to cause
 > considerable friction in choosing consistent priorities for ongoing
 > development.

I don't see that as a problem, really.  Choice of license and choice
of development directions are quite orthogonal, especially since the
GPL allows you to suck in software from any GPL-compatible project.

The real question is whether a "cultural mismatch" that was openly
displayed would reduce enthusiasm for work on Emacs among developers.
I have to admit I believe it would, for the same reasons I believe it
has been an advantage for Emacs in attracting and keeping developers
vis-a-vis XEmacs.  People have stronger loyalty to organizations that
espouse their ethical principles openly.  And it's easier to cooperate
with people "of your culture" because you're "getting your signals
crossed" much less frequently.

I personally don't think it would matter all that much that it would
turn the tide in the opposite direction if Emacs had another
existential crisis, but YMMV.

Sincere regards to all Emacs'ers,

Steve

Re: New maintainer

[John Wiegley]

>>>>> Stephen J Turnbull <stephen@xemacs.org> writes:

> Emacs's GPL being a fact, I would hope John would keep his mouth shut about
> his disagreement while wearing his GNU Emacs maintainer hat (not limited to
> participation in Emacs lists).

I like this, Stephen. Yes, Emacs is, and likely always will be, GPL-licensed.
All my own contributions to Emacs have been under the GPL -- in order to be
compatible -- even those developments not distributed with Emacs, like the
Emacs Chess client, or use-package.

Whether I believe the GPL is a right path socially is, I think, a separate
matter from maintaining Emacs. If asked to speak about Emacs, I will speak
about Emacs. If asked to speak about the GPL, I would make it clear that my
opinions have nothing to do with Emacs, or the FSF, or any other connection
people might falsely draw due to my involvement with an FSF-sponsored project.

However, if you look at the past ten years or so of my online communications,
I've adopted a personal path of not speaking *against* anything, but rather
*for* the things that I believe in. I've come to believe that constructive
positivity is, in the end, a more fruitful path toward change than destructive
negativity -- however much I may dislike certain things in the world.

I mainly brought this up so that people would know where I stand in my
thinking, and not be surprised in the future if anyone should read all of my
old blog articles.

I'm not here to take the GPL away from anyone. I have always supported the
freedom to choose whatever path you want in life, if it does not encroach on
the freedoms of others. If the GPL is that path for you, I would defend your
right to choose it, even if I do not necessarily agree with the path you've
chosen per se.

John

Re: New maintainer

[Christopher Allan Webber]

John Wiegley writes:

>>>>>> Stephen J Turnbull <stephen@xemacs.org> writes:
>
>> Emacs's GPL being a fact, I would hope John would keep his mouth shut about
>> his disagreement while wearing his GNU Emacs maintainer hat (not limited to
>> participation in Emacs lists).
>
> I like this, Stephen. Yes, Emacs is, and likely always will be, GPL-licensed.
> All my own contributions to Emacs have been under the GPL -- in order to be
> compatible -- even those developments not distributed with Emacs, like the
> Emacs Chess client, or use-package.

This seems pretty reasonable to me, and I do think John would be an
excellent maintainer of Emacs.  Though there may be some philosophical
differences, it seems that John is willing to work within the FSF's
needs and requirements there.

John certainly knows the field can have strong guidance.  I'd look
forward to a John Wiegley maintained emacs.

 - Chris

Re: New maintainer

[David Kastrup]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> I disagree, as John evidently does.  By the way, if I didn't know he
> would have self-nominated if he could, I would nominate Eli Zaretskii
> as maintainer.  Do you have a problem with him?

I'd support his nomination and I don't see any current candidate that
I'd feel more comfortable with out of the box.  However, I have the
impression that Eli is more comfortable setting accents than directions,
allowing him to voice opinions stronger than if were he responsible for
setting the overall direction and atmosphere.  I'd still trust him to
work on balancing his and the project needs but am not sure how happy
he'd be with it.

-- 
David Kastrup

RE: New maintainer

[Drew Adams]

> > By the way, if I didn't know he
> > would have self-nominated if he could, I would nominate Eli Zaretskii
> > as maintainer.  Do you have a problem with him?
> 
> I'd support his nomination and I don't see any current candidate that
> I'd feel more comfortable with out of the box.  However, I have the
> impression that Eli is more comfortable setting accents than directions,
> allowing him to voice opinions stronger than if were he responsible for
> setting the overall direction and atmosphere.  I'd still trust him to
> work on balancing his and the project needs but am not sure how happy
> he'd be with it.

I too hope that Eli considers this possibility seriously.

Re: New maintainer

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 259 bytes --]

On Fri, Oct 2, 2015 at 4:07 AM, Stephen J. Turnbull <stephen@xemacs.org>
wrote:

> By the way, if I didn't know he
> would have self-nominated if he could, I would nominate Eli Zaretskii
> as maintainer.

Perhaps we could get Eli to accept co-maintainership?

[-- Attachment #2: Type: text/html, Size: 406 bytes --]

Re: New maintainer

[Rasmus]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Fri, Oct 2, 2015 at 4:07 AM, Stephen J. Turnbull <stephen@xemacs.org>
> wrote:
>
>> By the way, if I didn't know he
>> would have self-nominated if he could, I would nominate Eli Zaretskii
>> as maintainer.
>
> Perhaps we could get Eli to accept co-maintainership?

That would indeed be very cool!

Rasmus

-- 
And I faced endless streams of vendor-approved Ikea furniture. . .

Re: New maintainer

[Karl Fogel]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:
>First, I second John's self-nomination.  I suspect it's a no-op given
>his "disagreement with the spirit of the GPL", but FWIW I think he
>would be an excellent Emacs maintainer.

An important question for John is how he would feel about RMS overriding technical decisions -- that is, decisions John might feel are purely technical -- for philosophical reasons.  Can John live with that happening occasionally?  Because it surely would.

(I heartily second Stephen's second, fwiw.  John would be a great maintainer, and while I disagree with him about the GPL, I don't think that bears much on the question of whether he can maintain Emacs well, given everything else he's said here.)

Re: New maintainer

[John Wiegley]

>>>>> Karl Fogel <kfogel@red-bean.com> writes:

> An important question for John is how he would feel about RMS overriding
> technical decisions -- that is, decisions John might feel are purely
> technical -- for philosophical reasons. Can John live with that happening
> occasionally? Because it surely would.

It has happened before: for example, Bazaar vs. Git, and not allowing dynamic
loading of libraries. Yet both of these positions changed over time. I can't
think of a "political over technical" decision right now that bothers me; and
the fact that those that did finally resolved themselves, gives me faith in
being able to work with the FSF. So yes, I can live with it happening
occasionally.

I further think that working alongside Eli would be nice, if he's up for it.
Having an active co-maintainer would relieve some of the time pressure of this
position, since I can't give equal focus every week.

John

Re: New maintainer

[Karl Fogel]

"John Wiegley" <johnw@newartisans.com> writes:
>> An important question for John is how he would feel about RMS overriding
>> technical decisions -- that is, decisions John might feel are purely
>> technical -- for philosophical reasons. Can John live with that happening
>> occasionally? Because it surely would.
>
>It has happened before: for example, Bazaar vs. Git, and not allowing dynamic
>loading of libraries. Yet both of these positions changed over time. I can't
>think of a "political over technical" decision right now that bothers me; and
>the fact that those that did finally resolved themselves, gives me faith in
>being able to work with the FSF. So yes, I can live with it happening
>occasionally.

...and that answers my question! :-)  Thank you.

>I further think that working alongside Eli would be nice, if he's up for it.
>Having an active co-maintainer would relieve some of the time pressure of this
>position, since I can't give equal focus every week.

+1 to both the general and the specific portions of that.

Re: New maintainer

[Rustom Mody]

[-- Attachment #1: Type: text/plain, Size: 271 bytes --]

On Fri, Oct 2, 2015 at 10:32 PM, John Wiegley <johnw@newartisans.com> wrote:

>
> I further think that working alongside Eli would be nice, if he's up for
> it.


A big YES from me also for this.
And thanks Stefan for showing the way
And rms for making this world for us

[-- Attachment #2: Type: text/html, Size: 636 bytes --]

Re: New maintainer

[Óscar Fuentes]

Rustom Mody <rustompmody@gmail.com> writes:

> On Fri, Oct 2, 2015 at 10:32 PM, John Wiegley <johnw@newartisans.com> wrote:
>
>>
>> I further think that working alongside Eli would be nice, if he's up for
>> it.
>
>
> A big YES from me also for this.

Maintainers tend to burn out and eventually cease their contribution as
developers. So I wish Eli does not become a maintainer.

OTOH, appointing as maintainer someone who is not on the group of the
most active developers but respected by the community is a good deal:
that person will contribute more to Emacs than he usually does and when
he goes away the Emacs day-to-day development does not suffer.

Re: New maintainer

[Dmitry Gutov]

On 10/03/2015 03:03 PM, Óscar Fuentes wrote:

> OTOH, appointing as maintainer someone who is not on the group of the
> most active developers but respected by the community is a good deal:
> that person will contribute more to Emacs than he usually does and when
> he goes away the Emacs day-to-day development does not suffer.

+1.

Although if Eli accepts co-maintainersip only with intention to add the 
political duties of the maintainer (plus being the FSF contact) to his 
normal ones, that shouldn't be too bad.

Re: New maintainer

[Przemysław Wojnowski]

> Maintainers tend to burn out and eventually cease their contribution as
> developers.
IMHO this is very important issue to address, because with them Emacs
looses active developers and a lot of knowledge.

What can be changed in this role to not burn out people?

IMHO one problem might be that, due to lack of roadmap and clear
priorities, a lot of different things are developed at the same time,
which springs even more topics on emacs-devel. Going through this flood
and deciding on them must be tiring. Selecting top 10 (X) from the
roadmap would constrain topics range and make it simpler to manage.
Also it would clarify for developers what to focus on.

Another thing is that a maintainer doesn't have to code review every
single change (search for "the cost of perfectionism"). It's enough if
some other developer would do that.
A developer that wrote a feature can just write tests and send an email
to emacs-devel "XXX - ready for Code Review".

I don't have experience as Emacs maintainer, so I may be wrong. But I
do have some experience as a team lead and have to deal with similar
issues.

Re: New maintainer

[Eli Zaretskii]

> From: Przemysław Wojnowski <esperanto@cumego.com>
> Date: Sat, 3 Oct 2015 22:04:19 +0200
> 
> IMHO one problem might be that, due to lack of roadmap and clear
> priorities, a lot of different things are developed at the same time,

I don't think you can prevent that in a project as multi-faceted and
multi-discipline as Emacs, nor do I think you should want to.

> which springs even more topics on emacs-devel. Going through this flood
> and deciding on them must be tiring. Selecting top 10 (X) from the
> roadmap would constrain topics range and make it simpler to manage.
> Also it would clarify for developers what to focus on.

People work on the itches they want to scratch.  I think an attempt to
tell them what to do and what not to will be futile at best, and at
worst will make some go away.

> Another thing is that a maintainer doesn't have to code review every
> single change (search for "the cost of perfectionism"). It's enough if
> some other developer would do that.

Assuming those other developers indeed do that.  We have too few who
do.

> A developer that wrote a feature can just write tests and send an email
> to emacs-devel "XXX - ready for Code Review".

They already do.  Then they have to ping us weeks later.

Re: New maintainer

[Przemysław Wojnowski]

W dniu 04.10.2015 o 08:26, Eli Zaretskii pisze:
>> IMHO one problem might be that, due to lack of roadmap and clear
>> priorities, a lot of different things are developed at the same time,
>
> I don't think you can prevent that in a project as multi-faceted and
> multi-discipline as Emacs, nor do I think you should want to.
>
[...]
> People work on the itches they want to scratch.  I think an attempt to
> tell them what to do and what not to will be futile at best, and at
> worst will make some go away.

But still a roadmap, with prioritized features, would help to choose
"the itches they want to scratch". Also it would help newcomers find
out what to do.

Anyway, it was just a proposition.

The point is to find out how not too loose maintainers. If they would
step down and stay as developers would be great.
Maybe "exit interview" would help. ;-)

Re: New maintainer

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Przemysław Wojnowski <esperanto@cumego.com>
> Date: Sun, 4 Oct 2015 09:03:14 +0200
> 
> W dniu 04.10.2015 o 08:26, Eli Zaretskii pisze:
> >> IMHO one problem might be that, due to lack of roadmap and clear
> >> priorities, a lot of different things are developed at the same time,
> >
> > I don't think you can prevent that in a project as multi-faceted and
> > multi-discipline as Emacs, nor do I think you should want to.
> >
> [...]
> > People work on the itches they want to scratch.  I think an attempt to
> > tell them what to do and what not to will be futile at best, and at
> > worst will make some go away.
> 
> But still a roadmap, with prioritized features, would help to choose
> "the itches they want to scratch". Also it would help newcomers find
> out what to do.

Of course, it will.  It's a good thing to have that.  I just don't
think it could relieve the workload of the head maintainer and the
resulting burn-out, which is what you were suggesting, AFAIU.

Re: New maintainer

[Przemysław Wojnowski]

>> W dniu 04.10.2015 o 08:26, Eli Zaretskii pisze:
>>>> IMHO one problem might be that, due to lack of roadmap and clear
>>>> priorities, a lot of different things are developed at the same time,
>>>
>>> I don't think you can prevent that in a project as multi-faceted and
>>> multi-discipline as Emacs, nor do I think you should want to.
I'm not saying that people should stop work on things that would not be
on a roadmap or in the next release. My point was that there should be a 
vision of Emacs in 5-10-15 years (what we would like it to be?), a 
roadmap based on that - list of features that bring us closer to that 
vision, etc. All written down.

Based on that maintainers would be able to plan releases with features 
from a roadmap - with consensus with developers. Maybe not many features 
would make it into the next release. Maybe just one of them. But it 
would make it clear what we want and were are we going (the vision). It 
would also make developers to focus on the next release.

For new developers clear goals are very important. Nobody wants to work 
on a project that goes nowhere. There always have to be some goal.

For example I see Emacs in 5 years with strong tests that immediately 
catch bugs and make refactoring fun. To achieve that I would add a goal 
that can be started right now, especially by newcomers:
1. Write unit tests to learn how Emacs works.
It's clear, very easy to do, very good for newcomers, and brings value
to Emacs. :-)


Anyway, the beauty of Agile Software Development is its adaptability.
Such teams try different things to improve their development process.
Things that don't work are refined or rejected. It's like evolution. 
IMHO Emacs community could try to apply such process. :-)

> I just don't
> think it could relieve the workload of the head maintainer and the
> resulting burn-out, which is what you were suggesting, AFAIU.
IMHO working on a concrete release would constraint number of topics
and decisions to make, which would relieve the workload.

Here are other ideas:
1. Constraint maintenance term (for example 3 years)
Such people wouldn't be exploited and maybe would step down to
development. Of course, after that time such person can decide that
everything is ok and can be a maintainer for another X years.

IMHO having ex-maintainers still in community would help subsequent
maintainers with making hard decisions. So it would also relieve the
workload.

2. Cut off-topics and end with action items.
Cutting off-topics should be done be everyone on the list. It creates a
flood of, maybe interesting, but irrelevant to the main topic, messages.

Unrelated messages make it very hard to follow the main thread.

Even this topic has a number of unrelated threads (politics, keylogger,
mac os, compiler support, etc.). How that help to find a "New
maintainer"?


Sorry for late reply. :-|

Cheers,
Przemysław

Re: New maintainer

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Przemysław Wojnowski <esperanto@cumego.com>
> Date: Tue, 6 Oct 2015 23:58:33 +0200
> 
> >> W dniu 04.10.2015 o 08:26, Eli Zaretskii pisze:
> >>>> IMHO one problem might be that, due to lack of roadmap and clear
> >>>> priorities, a lot of different things are developed at the same time,
> >>>
> >>> I don't think you can prevent that in a project as multi-faceted and
> >>> multi-discipline as Emacs, nor do I think you should want to.
> I'm not saying that people should stop work on things that would not be
> on a roadmap or in the next release. My point was that there should be a 
> vision of Emacs in 5-10-15 years (what we would like it to be?), a 
> roadmap based on that - list of features that bring us closer to that 
> vision, etc. All written down.

I already agreed that it would be a good thing to have such a
roadmap.  But it's entirely not easy to come up with.

It's not that we didn't try before.  The best result we could come up
with is in etc/TODO.  It's an undeservedly forgotten resource.

> Based on that maintainers would be able to plan releases with features 
> from a roadmap - with consensus with developers. Maybe not many features 
> would make it into the next release. Maybe just one of them.

This assumes that there will be some sufficient correlation between
the roadmap and the significant features being developed each year.
However, such an assumption will only hold if the roadmap is
coordinated with the existing developers before it becomes official.
Otherwise, you have only luck on your side, which IME is not a good
plan.

> [roadmap] would make it clear what we want and were are we going
> (the vision). It would also make developers to focus on the next
> release.

That's the hope, but it won't happen by itself, IME.

For this reason, we have been doing until now the exact opposite:
decided on a major release when enough significant new features became
available.  I cannot say it worked badly, btw.  You can review the
major releases and see for yourself.

> Nobody wants to work on a project that goes nowhere. There always
> have to be some goal.

I don't think there could be _a_ goal for Emacs.  It will always be
quite a few significant ones, and then many more less significant, but
still important ones.  In this sense, there's no single direction in
which Emacs is, or should be going.

> For example I see Emacs in 5 years with strong tests that immediately 
> catch bugs and make refactoring fun. To achieve that I would add a goal 
> that can be started right now, especially by newcomers:
> 1. Write unit tests to learn how Emacs works.
> It's clear, very easy to do, very good for newcomers, and brings value
> to Emacs. :-)

And it's already in etc/TODO, not very far from its beginning.

Besides, "more tests" is hardly a development direction, it's a means
to an end.

> Anyway, the beauty of Agile Software Development is its adaptability.
> Such teams try different things to improve their development process.
> Things that don't work are refined or rejected. It's like evolution. 
> IMHO Emacs community could try to apply such process. :-)

Reality check: we are not a team, in the Agile development sense.

> > I just don't
> > think it could relieve the workload of the head maintainer and the
> > resulting burn-out, which is what you were suggesting, AFAIU.
> IMHO working on a concrete release would constraint number of topics
> and decisions to make, which would relieve the workload.

I don't believe we will be able to constrain contributors to "working
on a release".  Just watch the pressure we have every time we declare
a feature freeze to cut a release branch and end the freeze.

> Here are other ideas:
> 1. Constraint maintenance term (for example 3 years)

No need.  Someone who feels burnt out will step down.  Someone who
don't, and does a good job, should be allowed to proceed.

> 2. Cut off-topics and end with action items.
> Cutting off-topics should be done be everyone on the list. It creates a
> flood of, maybe interesting, but irrelevant to the main topic, messages.

This is not a job with bosses and employees.  There are no means for
anyone here, including the head maintainer, to shut anyone up or force
them to stop discussing something.  Trying to do that wastes energy
and does little else.

> Unrelated messages make it very hard to follow the main thread.

Yes, liberal democracy is the worst of all regimes, except for all the
rest.

> Even this topic has a number of unrelated threads (politics, keylogger,
> mac os, compiler support, etc.). How that help to find a "New
> maintainer"?

There's nothing you can do against that.

Re: New maintainer

[Malk'Zameth]

[-- Attachment #1: Type: text/plain, Size: 5890 bytes --]

Hi all,

Pardon my ignorance and, I presume my raging naïveté possible only out of a
lack of the whole context, here
(if answering me would derail the subject, please ignore)

1 - the GCCvsClang issue touches features of Emacs itself (impacting 100%
of Emacsers) or just people using GCC/Clang for dev?

2 - If the latter: If we where to move CC-mode from the emacs core to Elpa
would that cut the debate from the core Emacs point of view?

we have an amazing module/package system now right? And probably the C devs
are no longer the majority ?

3 - On more abstract level: If we where, hypothetically, to slim down the
Core Emacs as much as possible and rely heavily on the packaging system
itself: what contention points between Freedom and Technical Evolution
would remain on the core itself?

Thanks,

Romeu


On Wed, Oct 7, 2015 at 5:27 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> > Cc: emacs-devel@gnu.org
> > From: Przemysław Wojnowski <esperanto@cumego.com>
> > Date: Tue, 6 Oct 2015 23:58:33 +0200
> >
> > >> W dniu 04.10.2015 o 08:26, Eli Zaretskii pisze:
> > >>>> IMHO one problem might be that, due to lack of roadmap and clear
> > >>>> priorities, a lot of different things are developed at the same
> time,
> > >>>
> > >>> I don't think you can prevent that in a project as multi-faceted and
> > >>> multi-discipline as Emacs, nor do I think you should want to.
> > I'm not saying that people should stop work on things that would not be
> > on a roadmap or in the next release. My point was that there should be a
> > vision of Emacs in 5-10-15 years (what we would like it to be?), a
> > roadmap based on that - list of features that bring us closer to that
> > vision, etc. All written down.
>
> I already agreed that it would be a good thing to have such a
> roadmap.  But it's entirely not easy to come up with.
>
> It's not that we didn't try before.  The best result we could come up
> with is in etc/TODO.  It's an undeservedly forgotten resource.
>
> > Based on that maintainers would be able to plan releases with features
> > from a roadmap - with consensus with developers. Maybe not many features
> > would make it into the next release. Maybe just one of them.
>
> This assumes that there will be some sufficient correlation between
> the roadmap and the significant features being developed each year.
> However, such an assumption will only hold if the roadmap is
> coordinated with the existing developers before it becomes official.
> Otherwise, you have only luck on your side, which IME is not a good
> plan.
>
> > [roadmap] would make it clear what we want and were are we going
> > (the vision). It would also make developers to focus on the next
> > release.
>
> That's the hope, but it won't happen by itself, IME.
>
> For this reason, we have been doing until now the exact opposite:
> decided on a major release when enough significant new features became
> available.  I cannot say it worked badly, btw.  You can review the
> major releases and see for yourself.
>
> > Nobody wants to work on a project that goes nowhere. There always
> > have to be some goal.
>
> I don't think there could be _a_ goal for Emacs.  It will always be
> quite a few significant ones, and then many more less significant, but
> still important ones.  In this sense, there's no single direction in
> which Emacs is, or should be going.
>
> > For example I see Emacs in 5 years with strong tests that immediately
> > catch bugs and make refactoring fun. To achieve that I would add a goal
> > that can be started right now, especially by newcomers:
> > 1. Write unit tests to learn how Emacs works.
> > It's clear, very easy to do, very good for newcomers, and brings value
> > to Emacs. :-)
>
> And it's already in etc/TODO, not very far from its beginning.
>
> Besides, "more tests" is hardly a development direction, it's a means
> to an end.
>
> > Anyway, the beauty of Agile Software Development is its adaptability.
> > Such teams try different things to improve their development process.
> > Things that don't work are refined or rejected. It's like evolution.
> > IMHO Emacs community could try to apply such process. :-)
>
> Reality check: we are not a team, in the Agile development sense.
>
> > > I just don't
> > > think it could relieve the workload of the head maintainer and the
> > > resulting burn-out, which is what you were suggesting, AFAIU.
> > IMHO working on a concrete release would constraint number of topics
> > and decisions to make, which would relieve the workload.
>
> I don't believe we will be able to constrain contributors to "working
> on a release".  Just watch the pressure we have every time we declare
> a feature freeze to cut a release branch and end the freeze.
>
> > Here are other ideas:
> > 1. Constraint maintenance term (for example 3 years)
>
> No need.  Someone who feels burnt out will step down.  Someone who
> don't, and does a good job, should be allowed to proceed.
>
> > 2. Cut off-topics and end with action items.
> > Cutting off-topics should be done be everyone on the list. It creates a
> > flood of, maybe interesting, but irrelevant to the main topic, messages.
>
> This is not a job with bosses and employees.  There are no means for
> anyone here, including the head maintainer, to shut anyone up or force
> them to stop discussing something.  Trying to do that wastes energy
> and does little else.
>
> > Unrelated messages make it very hard to follow the main thread.
>
> Yes, liberal democracy is the worst of all regimes, except for all the
> rest.
>
> > Even this topic has a number of unrelated threads (politics, keylogger,
> > mac os, compiler support, etc.). How that help to find a "New
> > maintainer"?
>
> There's nothing you can do against that.
>
>
>

[-- Attachment #2: Type: text/html, Size: 8214 bytes --]

Re: New maintainer

[Eli Zaretskii]

> From: "Malk'Zameth" <malk@zameth.org>
> Date: Wed, 7 Oct 2015 18:47:39 +0200
> Cc: emacs-devel@gnu.org
> 
> 1 - the GCCvsClang issue touches features of Emacs itself (impacting 100% of
> Emacsers) or just people using GCC/Clang for dev?

Anyone who wants an IDE for languages supported by GCC (not just C).

> 2 - If the latter: If we where to move CC-mode from the emacs core to Elpa
> would that cut the debate from the core Emacs point of view?

No, because ELPA is for all practical purposes part of Emacs.

> we have an amazing module/package system now right? And probably the C devs are
> no longer the majority ?

GCC supports more than just C, and C++ is still a very popular
development language.

> 3 - On more abstract level: If we where, hypothetically, to slim down the Core
> Emacs as much as possible and rely heavily on the packaging system itself: what
> contention points between Freedom and Technical Evolution would remain on the
> core itself?

All of it.

Re: New maintainer

[Artur Malabarba]

I'm not going to dive into GCC/Clang discussion, but I think I can
answer your questions.

2015-10-07 17:47 GMT+01:00 Malk'Zameth wrote:

> 1 - the GCCvsClang issue touches features of Emacs itself (impacting 100% of
> Emacsers) or just people using GCC/Clang for dev?

IIUC, People using GCC/Clang.

> 2 - If the latter: If we where to move CC-mode from the emacs core to Elpa
> would that cut the debate from the core Emacs point of view?

No. GNU Elpa is part of Emacs in pretty much every sense. It's not
(currently) bundled in the tarball, but other than that it _is_ Emacs
(and is bound by all the same rules).

> we have an amazing module/package system now right? And probably the C devs
> are no longer the majority ?

I have no numbers, but my guess is that no single language holds the
majority of the Emacs user base. Emacs has a pretty wide spread from
what I can tell.

But, given my answer above, this is besides the point. :-)

> 3 - On more abstract level: If we where, hypothetically, to slim down the
> Core Emacs as much as possible and rely heavily on the packaging system
> itself: what contention points between Freedom and Technical Evolution would
> remain on the core itself?

Don't know, but it's besides the point again. Any contention points
moved to Elpa would still be contention points.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > 2 - If the latter: If we where to move CC-mode from the emacs core to Elpa
  > would that cut the debate from the core Emacs point of view?

The issue is one of substance, not just labeling.  Moving CC mode
(which is used for various languages, not just C) would not change
the substance.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Przemysław Wojnowski]

W dniu 07.10.2015 o 17:27, Eli Zaretskii pisze:
> It's not that we didn't try before.  The best result we could come up
> with is in etc/TODO.  It's an undeservedly forgotten resource.
Didn't know that such file exist. Thanks!
It definitely should have more exposure, especially "Simple tasks"
section. How about mentioning it in etc/CONTRIBUTE?.

What I found strange is that the following file has different content:
http://www.gnu.org/software/emacs/CONTRIBUTE

>> Based on that maintainers would be able to plan releases with features
>> from a roadmap - with consensus with developers. Maybe not many features
>> would make it into the next release. Maybe just one of them.
>
> This assumes that there will be some sufficient correlation between
> the roadmap and the significant features being developed each year.
> However, such an assumption will only hold if the roadmap is
> coordinated with the existing developers before it becomes official.
Yes. Keep in mind that nothing is welded and can change for various 
reason, the roadmap too. But at least it's a plan (starting point).

>> [roadmap] would make it clear what we want and were are we going
>> (the vision). It would also make developers to focus on the next
>> release.
>
> That's the hope, but it won't happen by itself, IME.
>
> For this reason, we have been doing until now the exact opposite:
> decided on a major release when enough significant new features became
> available.  I cannot say it worked badly, btw.  You can review the
> major releases and see for yourself.
Maybe it's a good approach.

>> Nobody wants to work on a project that goes nowhere. There always
>> have to be some goal.
>
> I don't think there could be _a_ goal for Emacs.  It will always be
> quite a few significant ones, and then many more less significant, but
> still important ones.  In this sense, there's no single direction in
> which Emacs is, or should be going.
Well, IMHO some goals could be defined, for example "Provide IDE
features for language X (maybe unified) to make a programmer
productive". To achieve that some smaller goals would need to be
achieved, for example:
- provide project support (notion of project, maybe unified project API
   for different languages, debugger, etc.)
- provide refactoring tools for language(s) X (Y/Z)
- etc.

It could be divided into yet smaller goals until they finish in "Simple
tasks" section . ;-)

>> [...] strong tests

>> 1. Write unit tests to learn how Emacs works.
>> It's clear, very easy to do, very good for newcomers, and brings value
>> to Emacs. :-)
It was not a joke. Writing tests is a very good way to learn how things
work.

> Besides, "more tests" is hardly a development direction, it's a means
> to an end.
I've written "strong tests", which is not the same. But to be clear,
the goal is to have enough test to change Emacs safely, without a fear
of introducing new bugs (of course that will always happen, but with
"strong tests" they are very limited). Such tests allow to build
deployment pipeline and be able to release Emacs on every commit that
pass the pipeline. IMHO this can be a goal.

>
>> Anyway, the beauty of Agile Software Development is its adaptability.
>> Such teams try different things to improve their development process.
>> Things that don't work are refined or rejected. It's like evolution.
>> IMHO Emacs community could try to apply such process. :-)
>
> Reality check: we are not a team, in the Agile development sense.
Yes. The point here is constant improvement. Being open on new things ,
trying them and adapting what works. It doesn't have to be constrained
only to the Agile teams.

>>> I just don't
>>> think it could relieve the workload of the head maintainer and the
>>> resulting burn-out, which is what you were suggesting, AFAIU.
>> IMHO working on a concrete release would constraint number of topics
>> and decisions to make, which would relieve the workload.
>
> I don't believe we will be able to constrain contributors to "working
> on a release".  Just watch the pressure we have every time we declare
> a feature freeze to cut a release branch and end the freeze.
Maybe you're right. It was just an idea.

>> Here are other ideas:
>> 1. Constraint maintenance term (for example 3 years)
> No need.  Someone who feels burnt out will step down.  Someone who
> don't, and does a good job, should be allowed to proceed.
Maybe. It was just an idea how not to loose valuable people. Maybe
someone else will have a better one.

>> 2. Cut off-topics and end with action items.
>> Cutting off-topics should be done be everyone on the list. It creates a
>> flood of, maybe interesting, but irrelevant to the main topic, messages.
>
> This is not a job with bosses and employees.  There are no means for
> anyone here, including the head maintainer, to shut anyone up or force
> them to stop discussing something.  Trying to do that wastes energy
> and does little else.
Sorry. Seems that there's a misunderstanding. I don't want to "shut up"
anyone, but just ask for moving "off-topics" to new threads - "Please
discuss X in a separate thread."

>> Unrelated messages make it very hard to follow the main thread.
>> Even this topic has a number of unrelated threads (politics, keylogger,
>> mac os, compiler support, etc.). How that help to find a "New
>> maintainer"?
>
> There's nothing you can do against that.
Well, I can ask to discuss other topics in a new threads. Being polite
doesn't hurt.

Cheers,
Przemysław

Re: New maintainer

[Eli Zaretskii]

> From: Przemysław Wojnowski <esperanto@cumego.com>
> Date: Wed, 7 Oct 2015 20:49:39 +0200
> Cc: emacs-devel@gnu.org
> 
> W dniu 07.10.2015 o 17:27, Eli Zaretskii pisze:
> > It's not that we didn't try before.  The best result we could come up
> > with is in etc/TODO.  It's an undeservedly forgotten resource.
> Didn't know that such file exist. Thanks!
> It definitely should have more exposure, especially "Simple tasks"
> section. How about mentioning it in etc/CONTRIBUTE?.

It is already, indirectly, via a pointer to (info "(emacs)Contributing").

> What I found strange is that the following file has different content:
> http://www.gnu.org/software/emacs/CONTRIBUTE

The documentation Web site tracks the released versions, not the
development master branch.  See etc/CONTRIBUTE in Emacs 24.5.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

A roadmap is a useful thing for the maintainers to develop.
That doesn't mean they should reject improvements that are
outside the roadmap.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

One thing the maintainer might do, when people launch many tangents on
the list, is to pick one topic to discuss first and tell people to
save the others for later.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

GNU Emacs is part of the GNU Project, which is a technical project
with a specific political aim: software freedom for all users.  This
goal includes all the software they use, not just the specific programs
we develop.

The GNU Emacs maintainer's responsibility is to take charge of Emacs
on behalf of the GNU Project, and produce the best possible GNU Emacs
-- which means, the one that advances our aim the most.

Mostly, making Emacs better is a matter of practical improvements, but
there are some exceptions.  The maintainer's responsibility includes
some tasks to support the GPL, both practically and politically.  It
includes getting copyright papers from contributors so we can enforce
the GPL.  It includes making sure dynamic loading resists GPL
violation.  It includes putting some GNU Project political statements
into Emacs.  It includes making sure nothing in Emacs disagrees with
them.  An Emacs maintainer has to be willing to undertake this part of
the responsibility as well as the politically neutral bulk of the
responsibility.

The maintainer's job does not include personal political statements.
Maintainers don't have to say they agree with the GNU Project's
political positions, they just have to implement them wholeheartedly.

However, a maintainer shouldn't publicly oppose our positions.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Fri, 02 Oct 2015 21:36:28 -0400
> Cc: emacs-devel@gnu.org
> 
> GNU Emacs is part of the GNU Project, which is a technical project
> with a specific political aim: software freedom for all users.  This
> goal includes all the software they use, not just the specific programs
> we develop.
> 
> The GNU Emacs maintainer's responsibility is to take charge of Emacs
> on behalf of the GNU Project, and produce the best possible GNU Emacs
> -- which means, the one that advances our aim the most.
> 
> Mostly, making Emacs better is a matter of practical improvements, but
> there are some exceptions.  The maintainer's responsibility includes
> some tasks to support the GPL, both practically and politically.  It
> includes getting copyright papers from contributors so we can enforce
> the GPL.  It includes making sure dynamic loading resists GPL
> violation.  It includes putting some GNU Project political statements
> into Emacs.  It includes making sure nothing in Emacs disagrees with
> them.  An Emacs maintainer has to be willing to undertake this part of
> the responsibility as well as the politically neutral bulk of the
> responsibility.
> 
> The maintainer's job does not include personal political statements.
> Maintainers don't have to say they agree with the GNU Project's
> political positions, they just have to implement them wholeheartedly.
> 
> However, a maintainer shouldn't publicly oppose our positions.

Thanks for that summary, Richard.

In my personal experience, what you describe is not the most important
part of the decision a candidate or a nominee should make before
agreeing to volunteer.  He or she should indeed make sure they are OK
with all of the above, but I believe most of us are anyway.  The
practical implications of the above are more or less no-brainers:
performing the few implied duties is simple, even mechanical, and some
of them will rarely if ever be needed at all, except in some extreme
and improbable situation (like if someone declares they no longer want
to be part of the GNU Project, forks Emacs and takes most of the
contributors with them).  Yet another part is just agreement to some
principles, and has not practical implications beyond the fact of the
agreement itself.

IOW, this, what I call "the FSF side", of being the maintainer is not
the hard part of the decision to become one, nor something most of the
others should consider when they decide whether a candidate gets their
vote of confidence.  The hard part, IMO, is what does it mean "to
produce the best possible Emacs"?  What's the translation of this to
everyday's practice?

Perhaps we should consider this part of the job description before we
start nominating candidates and volunteering.

Emacs is so large that its maintenance is IMO qualitatively different
from almost any other package out there.  There are maybe a dozen
distinct large areas of expertise in the C core, dozens of such areas
in core Lisp infrastructures, and hundreds of them on the application
level.  Each one of these comes with its own non-trivial
domain-specific knowledge, its own algorithms, its own do's and
dont's.  No single person nor a small (2-3) number of people could
ever cover all that turf in any reasonable way.

By contrast, a head maintainer seems to be expected to be the final
authority for making decisions on changes in any particular area, and
also on design and implementation of both significant and
insignificant new features.

So, given this seemingly unsolvable contradiction, what do you, the
crowd, expect of the new maintainer?  What "job description", in
addition to what Richard wrote, would you propose if you were tasked
with the job of finding the candidates?  E.g., how many hours a week
should that person be available for working on Emacs?  Which talents
and personal traits should he or she possess?  Etc. etc.

I'm not writing this to solicit a barrage of suggestions for such a
job description, so please hold off your fast-typing fingers.  I'm
writing this to suggest that each one of us should take a moment to
think about these expectations, and decide who or what would we want
that next "maintainer" to be.  That's because I'm not sure all of
those who participated in this thread have the same things in mind
when they express their opinions about who might be a good maintainer.
Then look around and see if there are any persons who fit the bill
here.  Then you could perhaps provide some rationale for why you think
this or that person could be a good candidate.

HTH

Re: New maintainer

[Xue Fuqiao]

On Sat, Oct 3, 2015 at 4:04 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> Emacs is so large that its maintenance is IMO qualitatively different
> from almost any other package out there.  There are maybe a dozen
> distinct large areas of expertise in the C core, dozens of such areas
> in core Lisp infrastructures, and hundreds of them on the application
> level.  Each one of these comes with its own non-trivial
> domain-specific knowledge, its own algorithms, its own do's and
> dont's.  No single person nor a small (2-3) number of people could
> ever cover all that turf in any reasonable way.
>
> By contrast, a head maintainer seems to be expected to be the final
> authority for making decisions on changes in any particular area, and
> also on design and implementation of both significant and
> insignificant new features.
>
> So, given this seemingly unsolvable contradiction, what do you, the
> crowd, expect of the new maintainer?  What "job description", in
> addition to what Richard wrote, would you propose if you were tasked
> with the job of finding the candidates?  E.g., how many hours a week
> should that person be available for working on Emacs?  Which talents
> and personal traits should he or she possess?  Etc. etc.

The head maintainer should make policy about the general direction of
Emacs, and should make final decisions.

Since one single maintainer or a small team can't "cover all that turf",
the decision-making can be distributed to a range of participants.  I
think a (documented) decision-making policy would be nice.  When a new
idea, feature, patch, etc. turns up in the mailing lists (most likely on
emacs-devel), voting, for example, can be helpful for gauging the
community's general sentiment, or help the head maintainer make a
decision if he or she is not familiar with the domain-specific knowledge
of a package/area.  It doesn't have to have a binding force.  Voting can
be done with numbers (+1, -1, 0), and a -1 vote should include an
alternative proposal or a detailed explanation of the reasons for the
vote.  Of course, to provide an opportunity for participants from all
over the world, there should be a minimum voting duration (e.g., 72
hours).  But IMHO using a vote tracking system like Debian's devotee[1]
is overkill.

Or, the head maintainer only makes a decision when:
* it needs urgent action
* nobody else has responsibility (perhaps judged by `admin/MAINTAINERS'
  and/or the `Maintainer' header in a file)
* we can't reach an agreement

As for "job description", in addition to what Richard wrote, I think the
head maintainer should also:
* know the overall situation of Emacs well
* be familiar enough with Emacs Lisp, C, and Git
* be comfortable with Debbugs (or be willing to find a good alternative
  that can still be controlled with email and write import scripts from
  Debbugs and assist its deployment inside the FSF infrastructure)
* be comfortable with Texinfo
* (at least) have some basic understanding of the GNU build system
* keep an eye on GNU ELPA commits, not only Emacs core

Moreover, IMO if an Emacs developer nominate herself as a candidate head
maintainer, she should summarize her plans for her term.  For example,
if I self-nominate for that role, I'll write something about how to make
Emacs easier to contribute, the release schedule, when to
obsolete/deprecate features, how to improve the bug-fixing workflow and
the testing infrastructure, how to cooperate/communicate better with
popular third-party packages (like Magit/CIDER/Projectile), and write a
draft feature roadmap (dynload/modules, concurrency, new extension
languages, FFI, i18n, GUI builder, etc.).

[1] https://www.debian.org/vote/

Re: New maintainer

[David Engster]

Eli Zaretskii writes:
> So, given this seemingly unsolvable contradiction, what do you, the
> crowd, expect of the new maintainer?

That he actually wants the job. Since we seem to be in the 'zero
candidates' phase again, I have a hunch that this will be the biggest
hurdle. I don't think that formulating a job description will be a good
way of getting people to step up as candidates, as it will most likely
look pretty daunting.

-David

Re: New maintainer

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: rms@gnu.org,  johnw@newartisans.com,  emacs-devel@gnu.org
> Date: Sat, 03 Oct 2015 21:47:58 +0200
> 
> Eli Zaretskii writes:
> > So, given this seemingly unsolvable contradiction, what do you, the
> > crowd, expect of the new maintainer?
> 
> That he actually wants the job.

What job is that?  What does it entail?

> I don't think that formulating a job description will be a good way
> of getting people to step up as candidates, as it will most likely
> look pretty daunting.

So you expect someone to volunteer without knowing what's expected
from them?  I'd say "good luck with that", but I very much hope to be
proved wrong.

Re: New maintainer

[David Engster]

Eli Zaretskii writes:
>> From: David Engster <deng@randomsample.de>
>> Cc: rms@gnu.org,  johnw@newartisans.com,  emacs-devel@gnu.org
>> Date: Sat, 03 Oct 2015 21:47:58 +0200
>> 
>> Eli Zaretskii writes:
>> > So, given this seemingly unsolvable contradiction, what do you, the
>> > crowd, expect of the new maintainer?
>> 
>> That he actually wants the job.
>
> What job is that?  What does it entail?

I think the new head maintainer should already have an idea what it
entails, simply by following emacs-devel and the bug tracker and
contributing to Emacs. It's not like Stefan did his work in secret.

-David

Re: New maintainer

[John Wiegley]

>>>>> David Engster <deng@randomsample.de> writes:

> I think the new head maintainer should already have an idea what it entails,
> simply by following emacs-devel and the bug tracker and contributing to
> Emacs. It's not like Stefan did his work in secret.

Very good point.

John

Re: New maintainer

[Mathieu Lirzin]

Richard Stallman <rms@gnu.org> writes:

> The GNU Emacs maintainer's responsibility is to take charge of Emacs
> on behalf of the GNU Project, and produce the best possible GNU Emacs
> -- which means, the one that advances our aim the most.
                                        ^^^
> Mostly, making Emacs better is a matter of practical improvements, but
> there are some exceptions.  The maintainer's responsibility includes
> some tasks to support the GPL, both practically and politically.  It
> includes getting copyright papers from contributors so we can enforce
                                                         ^^
> the GPL.  It includes making sure dynamic loading resists GPL
> violation.  It includes putting some GNU Project political statements
> into Emacs.  It includes making sure nothing in Emacs disagrees with
> them.  An Emacs maintainer has to be willing to undertake this part of
> the responsibility as well as the politically neutral bulk of the
> responsibility.
>
> The maintainer's job does not include personal political statements.
> Maintainers don't have to say they agree with the GNU Project's
> political positions, they just have to implement them wholeheartedly.
>
> However, a maintainer shouldn't publicly oppose our positions.
                                                  ^^^

My overall feeling of your description of the Emacs maintainer's role is
that it consists of practical improvements with political obligations,
and no vocal disagreement with “us”.  This gives me the impression that
you consider GNU package maintainers as separate from the GNU project
itself.  But who is that “We” if it doesn't include the maintainers?

--
Mathieu Lirzin

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

    > But who is that “We” if it doesn't include the maintainers?

I founded the GNU Project in 1983 with a specific technical goal and a
specific political purpose.  I am its leader, though others help
manage it.  I say "we" when I speak for the project rather than for
myself personally.

GNU package maintainers undertake to maintain specific packages
on behalf of the GNU Project.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Using MacOSX & iOS as main operating systems and Hangout/Skype for
  > communications, seems incompatible with the role to me.

The Emacs maintainers should never ask contributors to use any of
those things.  What they personally use is a different issue.
It is very desirable for the main Emacs maintainer to use GNU/Linux,
since that is the principal target platform for GNU Emacs.
However, a person who uses some other system can still be a good
comaintainer.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Lennart Borgman]

[-- Attachment #1: Type: text/plain, Size: 1258 bytes --]

On Tue, Sep 29, 2015 at 11:46 PM, John Wiegley <johnw@newartisans.com>
wrote:

> >>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
> > So, now that I stepped down, we need to find a new maintainer (or a new
> > maintainer-team).
>
> I'd like to self-nominate for that role, Stefan. I've been contributing to
> Emacs since 1994, and have loved it all the while. Emacs Lisp remains a
> very
> enjoyable language to write certain types of code in.
>
> Some things I'd like to see happen to Emacs are more efficiency, closing
> bugs,
> and wider adoption of some of our newest features, like lexical scoping.
> That
> said, I'm also excited by new prospects, and wonder what can be done in the
> area of concurrency (in some form), a new language under the hood (Guile?),
> etc.
>
> Emacs is my favorite application, by far, and the one I spend the most time
> in, both professionally and personally. It's my programming environment,
> E-mail reader, IRC client, task manager, note taker, and occasional shell.
> I'm
> hoping it will still be the best choice for these things after twenty
> _more_
> years of use, and perhaps as head maintainer I could help keep things
> moving
> in that direction.
>
> John
>
>
I am glad to see you want to do the job!

[-- Attachment #2: Type: text/html, Size: 2291 bytes --]

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Being the Emacs maintainer (or a comaintainer) is a different job from
developing Emacs (although normally the maintainer also participates
in development).  The maintainer is in charge of Emacs development on
behalf of the GNU Project.  The maintainer's job is to manage the
development, not necessarily to do it.

I think that two maintainers would be ideal, but three could work.
More than that would be difficult as it would be hard for them
to make decisions together.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[David De La Harpe Golden]

On 02/10/15 03:24, Richard Stallman wrote:
 > I think that two maintainers would be ideal, but three could work.
 > More than that would be difficult as it would be hard for them
 > to make decisions together.
 >

Regardin two vs three, note a Triumvirate can be better in that 
particular respect. If each person has one "vote", and they're deciding 
on a binary issue, then two people can deadlock, three can't.


1	2	result
No	No	No
No	Yes	Civil War
Yes	No	Civil War
Yes	Yes	Yes


1	2	3	result
No	No	No	No
No	No	Yes	No
No	Yes	No	No
No	Yes	Yes	Yes
Yes	No	No	No
Yes	No	Yes	Yes
Yes	Yes	No	Yes
Yes	Yes	Yes	Yes

Re: New maintainer

[Eli Zaretskii]

> Date: Sat, 03 Oct 2015 19:37:46 +0100
> From: David De La Harpe Golden <david@harpegolden.net>
> 
> Regardin two vs three, note a Triumvirate can be better in that 
> particular respect. If each person has one "vote", and they're deciding 
> on a binary issue, then two people can deadlock, three can't.

Why do you assume that majority voting is the only possible way of
making decisions?

Re: New maintainer

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> Why do you assume that majority voting is the only possible way of making
> decisions?

It's the only way to break a tie if there is vehement difference of opinion. I
didn't say it was the only way to make decisions.

John

Re: New maintainer

[John Wiegley]

>>>>> John Wiegley <johnw@newartisans.com> writes:

> It's the only way to break a tie if there is vehement difference of opinion.
> I didn't say it was the only way to make decisions.

There's another reason not to use a Cabal-of-3, though, Eli: it would mean
always waiting until you could consult with the other 2 before making
pronouncements about direction on the mailing list. Otherwise, you risk the
possibility of deciding something that the other two don't agree with. This
would add unnecessary inertia to the process, in my opinion.

I say this having participated in group-driven scenarios before. It does slow
things down, since you have to communicate first with the group (either on or
off list), and then with the community. And there's the matter of presenting a
united front, being clear on what is personal opinion and what is the opinion
of the group, etc.

John

Re: New maintainer

[Eli Zaretskii]

> From: John Wiegley <johnw@newartisans.com>
> Date: Sat, 03 Oct 2015 12:12:42 -0700
> 
> >>>>> John Wiegley <johnw@newartisans.com> writes:
> 
> > It's the only way to break a tie if there is vehement difference of opinion.
> > I didn't say it was the only way to make decisions.
> 
> There's another reason not to use a Cabal-of-3, though, Eli: it would mean
> always waiting until you could consult with the other 2 before making
> pronouncements about direction on the mailing list. Otherwise, you risk the
> possibility of deciding something that the other two don't agree with. This
> would add unnecessary inertia to the process, in my opinion.

What's the rush?  This isn't some military operation we are planning,
where any delay might cause a catastrophe.

Besides, I see no reason to believe that any of the maintainers would
become incommunicado for long periods without an advance warning.
Being available on short notice should be part of the job description.

> I say this having participated in group-driven scenarios before. It does slow
> things down, since you have to communicate first with the group (either on or
> off list), and then with the community. And there's the matter of presenting a
> united front, being clear on what is personal opinion and what is the opinion
> of the group, etc.

The good feeling of being part of the team and having your opinions
heard and seriously considered, rather than dismissed, might be worth
the delay.

Re: New maintainer

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> The good feeling of being part of the team and having your opinions heard
> and seriously considered, rather than dismissed, might be worth the delay.

Perhaps so, I could be convinced I'm wrong on this point.

John

Re: New maintainer

[Eli Zaretskii]

> From: "John Wiegley" <johnw@newartisans.com>
> Date: Sat, 03 Oct 2015 12:08:21 -0700
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Why do you assume that majority voting is the only possible way of making
> > decisions?
> 
> It's the only way to break a tie if there is vehement difference of
> opinion.

No, it isn't.  It's just one way.  One obvious alternative is to look
for a compromise that everyone can live with.  And there are others.

Re: New maintainer

[John Wiegley]

>>>>> David De La Harpe Golden <david@harpegolden.net> writes:

> Regardin two vs three, note a Triumvirate can be better in that particular
> respect. If each person has one "vote", and they're deciding on a binary
> issue, then two people can deadlock, three can't.

I appreciate the logic of this, but I think real leadership means having a
head maintainer, and a supporting co-maintainer, so that the head can always
have final decision on matters relating to direction. Otherwise, you risk
inconsistency or disgruntlement when something truly important to one
maintainer is voted down by the other two.

In short, you either trust the person you're giving primary reins to, or you
do not. Making it a rule-by-committee is not necessarily going to give you a
"three minds are better than one" result.

John

Re: New maintainer

[Eli Zaretskii]

> From: "John Wiegley" <johnw@newartisans.com>
> Date: Sat, 03 Oct 2015 11:59:12 -0700
> 
> I think real leadership means having a head maintainer, and a
> supporting co-maintainer, so that the head can always have final
> decision on matters relating to direction.

How would such an arrangement differ from having just that head as a
single maintainer?  What can the co-maintainer do that the rest of us
cannot?

Re: New maintainer

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> How would such an arrangement differ from having just that head as a single
> maintainer? What can the co-maintainer do that the rest of us cannot?

The co-maintainer is usually given full maintainership over pieces of the
puzzle he (or she) has expertise with, until such time that the head
maintainer feels a unified direction is no longer being pursued. If there is
commonality of thought between them, they typically act in concert and most
people wouldn't realize that one of them has final decision.

Ensuring that one person sets the tone and vision for progress ensures that
things are never paralyzed by in-fighting or disagreement. If the
co-maintainer has issues with the maintainer, he resigns; if the maintainer
has issues with the co-maintainer, he asks him to step down.

John

Re: New maintainer

[Eli Zaretskii]

> From: John Wiegley <johnw@newartisans.com>
> Date: Sat, 03 Oct 2015 12:19:57 -0700
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > How would such an arrangement differ from having just that head as a single
> > maintainer? What can the co-maintainer do that the rest of us cannot?
> 
> The co-maintainer is usually given full maintainership over pieces of the
> puzzle he (or she) has expertise with, until such time that the head
> maintainer feels a unified direction is no longer being pursued.

That's the situation with every non-maintainer here: they are free to
do whatever they feel like in the areas they are interested in, with
the head maintainer keeping an eye on their commits and asking them to
make changes where he/she doesn't like the results.

I don't see how what you describe is any different.

> If there is commonality of thought between them, they typically act
> in concert and most people wouldn't realize that one of them has
> final decision.

Of course, they will realize: if nothing else, that fact is announced
up front.  And even if someone misses that announcement, it becomes
crystal clear very soon.

Anyway, if there are never any differences of opinions (and I think
it's naïve to expect that), then you have in effect a single person,
not 2 or 3.  In which case there's no real meaning to being the head,
is there?

> Ensuring that one person sets the tone and vision for progress ensures that
> things are never paralyzed by in-fighting or disagreement. If the
> co-maintainer has issues with the maintainer, he resigns; if the maintainer
> has issues with the co-maintainer, he asks him to step down.

I don't think this could ever work well in a project such as Emacs.
How can the head set the tone and vision, when he/she is not expert
enough in at least a few of the core areas?  If you want to set the
tone and vision in the development of the area of my expertise --
let's take the support for bidirectional editing as a good example --
don't you need me to first teach you enough about that, so you could
make up your own mind, instead of just trusting me?  And if you are
afraid of "issues" between us (i.e. you don't really trust me 100%),
why would you believe that I'll make an unbiased presentation of what
you need to learn, rather than bias it a bit to ensure that you agree
with me?

I think this method will encourage in-fighting and "bad blood", not
play them down.

Re: New maintainer

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> I don't think this could ever work well in a project such as Emacs. How can
> the head set the tone and vision, when he/she is not expert enough in at
> least a few of the core areas? If you want to set the tone and vision in the
> development of the area of my expertise -- let's take the support for
> bidirectional editing as a good example -- don't you need me to first teach
> you enough about that, so you could make up your own mind, instead of just
> trusting me? And if you are afraid of "issues" between us (i.e. you don't
> really trust me 100%), why would you believe that I'll make an unbiased
> presentation of what you need to learn, rather than bias it a bit to ensure
> that you agree with me?

I'm not sure it's worth derailing this thread to argue these things. Let them
find some new maintainer(s), and those candidates can work out with the FSF
whatever arrangement they prefer.

John

Re: New maintainer

[Andreas Röhler]

Am 03.10.2015 um 22:04 schrieb John Wiegley:
>>>>>> Eli Zaretskii <eliz@gnu.org> writes:
>> I don't think this could ever work well in a project such as Emacs. How can
>> the head set the tone and vision, when he/she is not expert enough in at
>> least a few of the core areas? If you want to set the tone and vision in the
>> development of the area of my expertise -- let's take the support for
>> bidirectional editing as a good example -- don't you need me to first teach
>> you enough about that, so you could make up your own mind, instead of just
>> trusting me? And if you are afraid of "issues" between us (i.e. you don't
>> really trust me 100%), why would you believe that I'll make an unbiased
>> presentation of what you need to learn, rather than bias it a bit to ensure
>> that you agree with me?
> I'm not sure it's worth derailing this thread to argue these things. Let them
> find some new maintainer(s), and those candidates can work out with the FSF
> whatever arrangement they prefer.
>
> John
>
>

Hi Eli,

doubt if there is anyone now knowing all the basic code which runs Emacs.

OTOH maintainership --while requiring technical knowledge-- basically is 
decision making, ruling out at cases presented by the parties.

The ability to preserve some coolness even in heated debates seems much 
more important than technical knowledge in detail.

Cheers,

Andreas

Re: New maintainer

[Eli Zaretskii]

> Date: Sun, 04 Oct 2015 10:18:04 +0200
> From: Andreas Röhler <andreas.roehler@online.de>
> CC: Eli Zaretskii <eliz@gnu.org>, John Wiegley <johnw@newartisans.com>
> 
> doubt if there is anyone now knowing all the basic code which runs Emacs.
> 
> OTOH maintainership --while requiring technical knowledge-- basically is 
> decision making, ruling out at cases presented by the parties.
> 
> The ability to preserve some coolness even in heated debates seems much 
> more important than technical knowledge in detail.

All true and agreed, but _some_ minimally useful degree of technical
knowledge is required for the decision making.  How can the head
maintainer make such decisions where he/she lacks that?

And even if he/she does have the minimal knowledge, can't it be that
it's not enough to make correct decisions in cases where the correct
alternative is not clear-cut, and the decision needs some intuition
(which is impossible without good knowledge and experience)?

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > All true and agreed, but _some_ minimally useful degree of technical
  > knowledge is required for the decision making.

We can't expect the impossible.  A perfect Emacs maintainer would be
familiar with all the code in Emacs.  We won't find a person like
that, but Stefan has shown that a less-than-perfect maintainer can do
a good job.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: andreas.roehler@online.de, johnw@newartisans.com,
> 	emacs-devel@gnu.org
> Date: Mon, 05 Oct 2015 13:05:48 -0400
> 
>   > All true and agreed, but _some_ minimally useful degree of technical
>   > knowledge is required for the decision making.
> 
> We can't expect the impossible.  A perfect Emacs maintainer would be
> familiar with all the code in Emacs.  We won't find a person like
> that, but Stefan has shown that a less-than-perfect maintainer can do
> a good job.

No disagreement here, neither in principle nor wrt Stefan's work.

The issue is merely how to organize the maintainership, and how to
define the division of responsibilities with c-maintainers, if there
will be such.

Re: New maintainer

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> The issue is merely how to organize the maintainership, and how to define
> the division of responsibilities with c-maintainers, if there will be such.

This is a great question, and one I've been pondering myself, since the most
pressing variable for me in all of this is time.

Where I think I can contribute best is the bigger picture, or "meta issues":
weighing in on technical discussions, making higher-level decisions about
technical direction, keeping an eye on user experience within the community
and the quality of Emacs resources, coordinating volunteers, ensuring proper
legal forms are maintained, liaising with the FSF, and assisting other
maintainers so they don't burnout and receive the help they need.

What I probably don't have enough time for is giving all the issues, code
submissions, and discussions on emacs-devel, the depth and refinement they
deserve. This is where I've noticed Eli (and certainly Stefan) doing an
excellent job. I'm quite impressed by their energy and involvement. I would
need such people to make this job even possible within the constraints of my
life.

I also think that detail-level maintenance is far more likely to induce
burnout, since the flood at that level can be intense. Seeing the number of
replies by Eli and Stefan in the past few weeks has been impressive to say the
least, and requires a special kind of interest and energy to maintain. How do
we best support them? How do we find more hands to make lighter work for our
stalwart contributors?

Meanwhile, I want to think about the road leading to Emacs 26, and to work
with Eli, and the community as a whole, on what makes the most sense in terms
of what we have now, and what we want Emacs to become. For example, we have
compute-intensive applications -- such as Gnus -- that cannot take advantage
of the additional power offered by multi-core CPUs. How do we add concurrency
without increasing our maintenance burden due to impossible-to-reproduce bugs,
race conditions, and terrible error messages (a Backtrace, but from which
thread?). It will require significant collaboration to decide exactly what we
want, and what Emacs needs, from such features.

Another area we're falling behind in is the type of IDE features that are
taken for granted in special-purpose editing environments, such as effortless
code browsing, refactoring, and more interactive debugging. The things you can
do when editing Java and Javascript are downright impressive, and I see no
reason Emacs can't compete better here.

It would be hard to care about these issues in sufficient depth if the job of
project maintenance also required keeping an eye on every issue and technical
discussion. I think having co-maintainers (maybe several) who are good at
detail is absolutely essential to getting this job done.

John

Re: New maintainer

[Eli Zaretskii]

> From: John Wiegley <johnw@newartisans.com>
> Date: Mon, 05 Oct 2015 12:02:50 -0700
> 
> we have compute-intensive applications -- such as Gnus -- that
> cannot take advantage of the additional power offered by multi-core
> CPUs. How do we add concurrency without increasing our maintenance
> burden due to impossible-to-reproduce bugs, race conditions, and
> terrible error messages (a Backtrace, but from which thread?).

There's the concurrency branch in the repository.  It needs to be
revived (didn't see any activity for the last 2 years), and was not
really ready for prime time last time I looked at it (see
http://lists.gnu.org/archive/html/emacs-devel/2013-08/msg00704.html
and its continuation into Sep 2013).  But it's not far from its
proclaimed goal.

Re: New maintainer

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1176 bytes --]

On 10/05/2015 12:32 PM, Eli Zaretskii wrote:
>> From: John Wiegley <johnw@newartisans.com>
>> Date: Mon, 05 Oct 2015 12:02:50 -0700
>>
>> we have compute-intensive applications -- such as Gnus -- that
>> cannot take advantage of the additional power offered by multi-core
>> CPUs. How do we add concurrency without increasing our maintenance
>> burden due to impossible-to-reproduce bugs, race conditions, and
>> terrible error messages (a Backtrace, but from which thread?).
> 
> There's the concurrency branch in the repository.  It needs to be
> revived (didn't see any activity for the last 2 years), and was not
> really ready for prime time last time I looked at it (see
> http://lists.gnu.org/archive/html/emacs-devel/2013-08/msg00704.html
> and its continuation into Sep 2013).  But it's not far from its
> proclaimed goal.

It isn't, but I'm still not sure that goal is the right one. I much
prefer a world where we don't have shared mutable state: the web workers
model from the web is a good one, and like any concurrency solution for
Emacs, it addresses the problem of integrating multiprocessing into a
legacy single-threaded environment.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: New maintainer

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Daniel Colascione <dancol@dancol.org>
> Date: Mon, 5 Oct 2015 12:38:18 -0700
> 
> > There's the concurrency branch in the repository.  It needs to be
> > revived (didn't see any activity for the last 2 years), and was not
> > really ready for prime time last time I looked at it (see
> > http://lists.gnu.org/archive/html/emacs-devel/2013-08/msg00704.html
> > and its continuation into Sep 2013).  But it's not far from its
> > proclaimed goal.
> 
> It isn't, but I'm still not sure that goal is the right one. I much
> prefer a world where we don't have shared mutable state: the web workers
> model from the web is a good one, and like any concurrency solution for
> Emacs, it addresses the problem of integrating multiprocessing into a
> legacy single-threaded environment.

Maybe you are right, but IMO it would be silly for us not to try that
branch first, through which we will gain firsthand experience and
knowledge of what kind of concurrency we need.

Otherwise, it will be another idea that ended in just talking.

Re: New maintainer

[Eli Zaretskii]

> From: John Wiegley <johnw@newartisans.com>
> Date: Mon, 05 Oct 2015 12:02:50 -0700
> 
> I also think that detail-level maintenance is far more likely to induce
> burnout, since the flood at that level can be intense. Seeing the number of
> replies by Eli and Stefan in the past few weeks has been impressive to say the
> least, and requires a special kind of interest and energy to
> maintain.

Your observations regarding my responses are most probably biased due
to a public holiday we had here for the last 3 weeks.  That ends
tonight.

In general, I have much less time to work on Emacs than Stefan and
many others here.

> How do we best support them? How do we find more hands to make
> lighter work for our stalwart contributors?

Excellent questions.

Re: New maintainer

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> Your observations regarding my responses are most probably biased due to a
> public holiday we had here for the last 3 weeks. That ends tonight.

In that case, thank you for spending so much of your holiday attending to
Emacs! It is much appreciated. :)

John

Re: New maintainer

[Dmitry Gutov]

On 10/05/2015 10:02 PM, John Wiegley wrote:

> This is a great question, and one I've been pondering myself, since the most
> pressing variable for me in all of this is time.

I fear that might be a problem.

> Where I think I can contribute best is the bigger picture, or "meta issues":
> weighing in on technical discussions, making higher-level decisions about
> technical direction, keeping an eye on user experience within the community

In the end, you might encounter a lack of clearly defined points when 
someone asks you to make a decision.

More often, the regular contributors already have an idea what they want 
to do in the limited time they can spend working on Emacs, and often 
it's not easy to make such a person change their mind.

Not every change is announced or discussed either, so I think a 
maintainer should be subscribed to emacs-diffs.

Likewise, even if you make a decision that a certain aspect of Emacs 
needs work, there's no guarantee that someone else will readily begin 
working on it.

> and the quality of Emacs resources, coordinating volunteers, ensuring proper
> legal forms are maintained, liaising with the FSF, and assisting other
> maintainers so they don't burnout and receive the help they need.

We really don't have enough volunteers. So an ideal maintainer, IMHO, 
would find ways to energize more people to volunteer, maybe by making 
the contribution process easier somehow (one could mention a better bug 
tracker, code review process, CI, documentation, etc; in short, a lot of 
things could be better, and all of them require work, in the end, rather 
than simply discussions and decisions), making the development process 
more transparent to the community, or, you know, handling a lot of the 
grunt work themselves. Maybe all of the options together.

> Another area we're falling behind in is the type of IDE features that are
> taken for granted in special-purpose editing environments, such as effortless
> code browsing, refactoring, and more interactive debugging. The things you can
> do when editing Java and Javascript are downright impressive, and I see no
> reason Emacs can't compete better here.

That area is closer to my interests, and I'd happily see one more person 
(or several) participate in these discussions, but preferably in 
lower-level terms (like the details of the xref interface, or the 
project API). So far, they've ended more in disagreement than anything 
else, and it's pretty discouraging.

Re: New maintainer

[Artur Malabarba]

2015-10-05 22:20 GMT+01:00 Dmitry Gutov <dgutov@yandex.ru>:
>> and the quality of Emacs resources, coordinating volunteers, ensuring
>> proper
>> legal forms are maintained, liaising with the FSF, and assisting other
>> maintainers so they don't burnout and receive the help they need.
>
>
> We really don't have enough volunteers. So an ideal maintainer, IMHO, would
> find ways to energize more people to volunteer, maybe by making the
> contribution process easier somehow (one could mention a better bug tracker,
> code review process, CI, documentation, etc; in short, a lot of things could
> be better, and all of them require work, in the end, rather than simply
> discussions and decisions), making the development process more transparent
> to the community, or, you know, handling a lot of the grunt work themselves.
> Maybe all of the options together.

Also just invite more devs with Melpa packages to contribute their
packages to GNU Elpa.
This gives them a good motivation to take care of all the paperwork
and get sorted with a savannah account, at which point they have a
decent chance of starting to contribute more packages or helping out
with emacs.git directly.

I suspect there are many package authors out there who are potential
regular/semi-regular contributors, but they're unlikely to just come
and ask for permission for a number of reasons.
That was my entry-drug (thanks Stefan) and I'm sure it would work on a
few others.

Re: New maintainer

[John Wiegley]

>>>>> Artur Malabarba <bruce.connor.am@gmail.com> writes:

> I suspect there are many package authors out there who are potential
> regular/semi-regular contributors, but they're unlikely to just come and ask
> for permission for a number of reasons. That was my entry-drug (thanks
> Stefan) and I'm sure it would work on a few others.

I think you're exactly right, Artur. I bet if asked, a lot of potential
contributors would say, "Who, me?" They simply haven't considered themselves
in that role yet.

John

Re: New maintainer

[Artur Malabarba]

I'll try to provide some help with GNU Elpa (reading diffs, helping
new devs, inviting people). Sadly, my time is (and will continue to
be) very sporadic so I can't offer to be The Maintainer for elpa.git.


2015-10-05 23:24 GMT+01:00 John Wiegley <johnw@newartisans.com>:
>>>>>> Artur Malabarba <bruce.connor.am@gmail.com> writes:
>
>> I suspect there are many package authors out there who are potential
>> regular/semi-regular contributors, but they're unlikely to just come and ask
>> for permission for a number of reasons. That was my entry-drug (thanks
>> Stefan) and I'm sure it would work on a few others.
>
> I think you're exactly right, Artur. I bet if asked, a lot of potential
> contributors would say, "Who, me?" They simply haven't considered themselves
> in that role yet.
>
> John
>

Re: New maintainer

[John Wiegley]

>>>>> Artur Malabarba <bruce.connor.am@gmail.com> writes:

> I'll try to provide some help with GNU Elpa (reading diffs, helping new
> devs, inviting people). Sadly, my time is (and will continue to be) very
> sporadic so I can't offer to be The Maintainer for elpa.git.

Whatever help you could offer would be most appreciated, Artur. At the very
least you'll have trodden the ground, so you can help other potential
contributors get up to speed faster.

John

Re: New maintainer

[Andreas Röhler]

Am 05.10.2015 um 23:20 schrieb Dmitry Gutov:
> [ ... ]
> We really don't have enough volunteers. 

[ ... ]

Might the FSF copyright assigment policy being a major cause here?

Re: New maintainer

[Ricardo Wurmus]

Andreas Röhler <andreas.roehler@online.de> writes:

> Am 05.10.2015 um 23:20 schrieb Dmitry Gutov:
>> [ ... ]
>> We really don't have enough volunteers. 
>
> [ ... ]
>
> Might the FSF copyright assigment policy being a major cause here?

For me the copyright assigment policy hasn’t been an obstacle to
contribution in another GNU project.  What keeps me from contributing to
Emacs is just this peculiar mix of not knowing enough details and time
spent on other projects.

I found a list of simple tasks that new contributors could tackle in
another project (the Hurd) very encouraging, leaving only time
management as a reason not to contribute.

Copyright assignment is a one-time thing and it doesn’t take much time.

~~ Ricardo

Re: New maintainer

[David Kastrup]

Andreas Röhler <andreas.roehler@online.de> writes:

> Am 05.10.2015 um 23:20 schrieb Dmitry Gutov:
>> [ ... ]
>> We really don't have enough volunteers. 
>
> [ ... ]
>
> Might the FSF copyright assigment policy being a major cause here?

XEmacs doesn't have it and does not really fare better in that regard.
Elisp is more likely to be a contributing factor.  Either are not going
to go away, but there have been some improvements regarding how
burdensome either turn out in practice.

-- 
David Kastrup

Re: New maintainer

[Andreas Röhler]

Am 05.10.2015 um 23:20 schrieb Dmitry Gutov:
> On 10/05/2015 10:02 PM, John Wiegley wrote:
>
[ ... ]
>> Another area we're falling behind in is the type of IDE features that 
>> are
>> taken for granted in special-purpose editing environments, such as 
>> effortless
>> code browsing, refactoring, and more interactive debugging. The 
>> things you can
>> do when editing Java and Javascript are downright impressive, and I 
>> see no
>> reason Emacs can't compete better here.
>
> That area is closer to my interests, and I'd happily see one more 
> person (or several) participate in these discussions, but preferably 
> in lower-level terms (like the details of the xref interface, or the 
> project API). So far, they've ended more in disagreement than anything 
> else, and it's pretty discouraging.
>
>

IMO rather third party libraries should be encouraged.

For example WRT Python we have already several:

the python-mode.el/pymacs/rope eco-system
emacs-for-python,
and elpy, relying on shipped python.el  and jedi/rope

AFAICS Emacs provides all to built IDEs upon. OTOH doing it requires 
special knowledge in the languages, which can't be expected from 
core-developers.

Exists related efforts at other languages.

Emacs core-developers could have a stronger role by advising them.
Strengthening communication in community seems the way to go WRT IDEs.

Re: New maintainer

[Rasmus]

Andreas Röhler <andreas.roehler@online.de> writes:

> IMO rather third party libraries should be encouraged.

"Batteries included", especially for those of us who have do some work on
company issued PCs.  Though ELPA is making this less of a concern.

Rasmus

-- 
It was you, Jezebel, it was you

IDE

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Emacs with GUD is an IDE.  It has a big advantage compared with other IDEs:
when you see a source file, you're editing it with Emacs.

If it falls short compared with other IDEs, I think this would be a
great area for improvement of Emacs.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: IDE

[Tom]

> Emacs with GUD is an IDE.  It has a big advantage compared with
> other IDEs: when you see a source file, you're editing it with Emacs.

This is an advantage for Emacs users who want to use Emacs. Other
IDE users do not care about this that much.

> If it falls short compared with other IDEs, I think this would be a
> great area for improvement of Emacs.

The number one requirement by IDE users today is perfect code completion
and powerful refactoring support for the language they develop in
(Java, C++, etc.).

Any IDE which wants to compete with the popular IDEs must have these,
because users find these features much more helpful in development,
than keyboard macro support and such.

So if Emacs wants to compete with these tools then it has to have seamless,
context aware code completion and refactoring support, and GNU tools
has to provide Emacs the necessary information to implement these
features.

Re: IDE

[Eli Zaretskii]

> From: Tom <adatgyujto@gmail.com>
> Date: Sat, 10 Oct 2015 04:33:59 +0000 (UTC)
> 
> > If it falls short compared with other IDEs, I think this would be a
> > great area for improvement of Emacs.
> 
> The number one requirement by IDE users today is perfect code completion
> and powerful refactoring support for the language they develop in
> (Java, C++, etc.).
> 
> Any IDE which wants to compete with the popular IDEs must have these,
> because users find these features much more helpful in development,
> than keyboard macro support and such.
> 
> So if Emacs wants to compete with these tools then it has to have seamless,
> context aware code completion and refactoring support, and GNU tools
> has to provide Emacs the necessary information to implement these
> features.

I agree.  But to have that, the only way is to have motivated
volunteers step forward and work on these features.  Otherwise we will
never have them.

Right now, no one is working on that, though everyone is talking.  the
same as with weather.

Re: IDE

[Dmitry Gutov]

On 10/10/2015 10:56 AM, Eli Zaretskii wrote:

> Right now, no one is working on that, though everyone is talking.  the
> same as with weather.

No one?

There are quite a few packages that interface with external programs or 
daemons to provide code completion already. Such as Tern for JavaScript, 
Racer for Rust, Compliment for Clojure, gocode for Go, Eclim or Malabar 
for Java, ENSIME for Scala (there has been some movement lately in 
adding Java support, too), Distel for Erlang, Jedi for Python, ocp-index 
for OCaml, ESS for R, maybe some others.

For C/C++, the community has Irony and Rtags, both based on libclang. If 
libclang is unacceptable for you, you probably know a more appropriate 
mailing list to bring that up at.

Would you expect the programs mentioned above to become a part of Emacs? 
For most of them, it's technically unfeasible (not to mention 
organizationally), e.g. because they target several different editors 
(or aim to, in the future).

Re: IDE

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 10 Oct 2015 11:15:38 +0300
> 
> On 10/10/2015 10:56 AM, Eli Zaretskii wrote:
> 
> > Right now, no one is working on that, though everyone is talking.  the
> > same as with weather.
> 
> No one?

No one.

> There are quite a few packages that interface with external programs or 
> daemons to provide code completion already. Such as Tern for JavaScript, 
> Racer for Rust, Compliment for Clojure, gocode for Go, Eclim or Malabar 
> for Java, ENSIME for Scala (there has been some movement lately in 
> adding Java support, too), Distel for Erlang, Jedi for Python, ocp-index 
> for OCaml, ESS for R, maybe some others.

I was talking about working on IDE, not on completion.  And for the
most popular languages in the industry, not just for some a few niche
languages.

> For C/C++, the community has Irony and Rtags, both based on libclang. If 
> libclang is unacceptable for you, you probably know a more appropriate 
> mailing list to bring that up at.

Let's not reiterate past discussions: you forget CEDET.

And if anyone _really_ cares about supporting C/C++, they should be
working with and on GCC's libcc1, which is available for quite some
time already.

Instead, all we have is heated discussions and hurt feelings.  That
will never get us anywhere.

> Would you expect the programs mentioned above to become a part of Emacs? 

I expect to see a coherent, orchestrated effort towards having an IDE
mode in Emacs.  I don't see it, certainly not in discussions on this
list.  I also don't see any commits that would provide evidence of
such an effort.

If such activities happen somewhere else, I would suggest their
participants to come here and work with and within the core.  For
starters, I don't imagine they would succeed without some significant
changes and additions in the core infrastructure.  The place to
discuss that is here.

> For most of them, it's technically unfeasible (not to mention 
> organizationally), e.g. because they target several different editors 
> (or aim to, in the future).

Then what exactly is the nature of your objections to my observations?
It seems we agree on the bottom line: no one works on this.  The
reasons are immaterial.

Re: IDE

[Dmitry Gutov]

On 10/10/2015 11:30 AM, Eli Zaretskii wrote:

> I was talking about working on IDE, not on completion.  And for the
> most popular languages in the industry, not just for some a few niche
> languages.

You quoted the message that said "accurate code completion and powerful 
refactoring support". I can agree that the latter is barely touched (*), 
but it looked like you ignored the former.

> Let's not reiterate past discussions: you forget CEDET.

I was enumerating external programs. But sure, CEDET is a yet another 
option for completion. Though not too "accurate" one, if we're talking 
anything but C.

> And if anyone _really_ cares about supporting C/C++, they should be
> working with and on GCC's libcc1, which is available for quite some
> time already.

I wasn't aware of it. Is its API stable? Is it a good-enough replacement 
for libclang for the purposes of completion?

If you like, I can pass along the request to use it as an alternative to 
the Irony and Rtags issue trackers. But some more details wouldn't hurt, 
it's hard for me to advocate libcc1 myself.

> Instead, all we have is heated discussions and hurt feelings.  That
> will never get us anywhere.

My feelings aren't hurt, I just meant to add more information to the 
discussion.

> I expect to see a coherent, orchestrated effort towards having an IDE
> mode in Emacs.  I don't see it, certainly not in discussions on this
> list.  I also don't see any commits that would provide evidence of
> such an effort.

We definitely could have more in this department, yes. But what would 
you even call an "IDE mode"? A fixed multi-window setup a la ECB?

I prefer to approach this problem from the bottom - like adding new 
commands that perform certain advanced functions.

> If such activities happen somewhere else, I would suggest their
> participants to come here and work with and within the core.

That's... unlikely. At least, for most of them. My guess is that the 
mailing list interface is off-putting, but maybe we just need a better 
"community outreach" program, or something like that.

That would be something for the new maintainer(s) to consider.

> For
> starters, I don't imagine they would succeed without some significant
> changes and additions in the core infrastructure.  The place to
> discuss that is here.

For refactoring, yes. But just "accurate code completion", without 
extras like expanding the arguments or displaying the method source, can 
be (and is, in certain packages) implemented though the 
completion-at-point-function interface, present in Emacs since 24.1.

And you can have the extras using Company, which should be bundled with 
Emacs in the upcoming version. Or if the ELPA bundling isn't ready by 
then, in the version after that.

> Then what exactly is the nature of your objections to my observations?
> It seems we agree on the bottom line: no one works on this.  The
> reasons are immaterial.

If anything, my view of the situation is a lot brighter than yours.

I also should have more time at the end of this month to put into 
improving xref, which is a step, as you know, in adding a common 
framework for some of the IDE-ish features.

(*) There are some third-party packages providing refactoring commands 
(for dynamic languages, mostly), but they would definitely benefit from 
a nice common UI.

Re: IDE

[Eli Zaretskii]

> Cc: adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 10 Oct 2015 11:59:34 +0300
> 
> On 10/10/2015 11:30 AM, Eli Zaretskii wrote:
> 
> > I was talking about working on IDE, not on completion.  And for the
> > most popular languages in the industry, not just for some a few niche
> > languages.
> 
> You quoted the message that said "accurate code completion and powerful 
> refactoring support".

No, I responded to a response.  The origin was this:

> If it falls short compared with other IDEs, I think this would be a
> great area for improvement of Emacs.

IOW, the issue is making Emacs a good IDE in general, including
features for browsing code, refactoring, debugging, and all the other
features users expect from a modern IDE.

Come to think of that, even coming up with a list of such features
would be real progress, as I don't think we have that, or ever had.

> I can agree that the latter is barely touched (*), 
> but it looked like you ignored the former.

I didn't ignore that.  I just don't see an effort to make Emacs a
modern IDE.  Working on separate parts of that in separate
uncoordinated activities is not the way we should pursue this, IMO.
It's inefficient at best, and at worst will end up with those
uncoordinated parts falling into oblivion when their driving forces
move on.

> > Let's not reiterate past discussions: you forget CEDET.
> 
> I was enumerating external programs. But sure, CEDET is a yet another 
> option for completion. Though not too "accurate" one, if we're talking 
> anything but C.

It needs to be actively developed.  Much more actively than it is now.

> > And if anyone _really_ cares about supporting C/C++, they should be
> > working with and on GCC's libcc1, which is available for quite some
> > time already.
> 
> I wasn't aware of it. Is its API stable?

I don't know.  It's for someone who will work on this to find out.  I
know that a motivated individual in the GDB development team already
based a useful set of commands on it -- you can compile and inject
code into your program while debugging it.

> Is it a good-enough replacement for libclang for the purposes of
> completion?

I don't know.  If it isn't, then the team who will work on the Emacs
IDE will have to take care of extending libcc1 as well, or find
someone with the GCC team to do that.  Exactly like that GDB developer
did when he needed features that were missing: he implemented them
himself, with guidance from GCC developers.

> > I expect to see a coherent, orchestrated effort towards having an IDE
> > mode in Emacs.  I don't see it, certainly not in discussions on this
> > list.  I also don't see any commits that would provide evidence of
> > such an effort.
> 
> We definitely could have more in this department, yes. But what would 
> you even call an "IDE mode"? A fixed multi-window setup a la ECB?

I don't know, and neither do we as a project.  A useful step would be
to produce a detailed answer to that question.  That answer could both
serve as base for useful discussions, and might provide some anchor
for all those external packages you mentioned to target some coherent
vision.

> I prefer to approach this problem from the bottom - like adding new 
> commands that perform certain advanced functions.

I don't believe comprehensive features such as IDE can be developed
exclusively bottom up.  There should be some basic set of assumptions
and design rules/decisions that everyone should target and abide by.
There should also be some unified leadership.

> > If such activities happen somewhere else, I would suggest their
> > participants to come here and work with and within the core.
> 
> That's... unlikely. At least, for most of them. My guess is that the 
> mailing list interface is off-putting, but maybe we just need a better 
> "community outreach" program, or something like that.

When the work begins in earnest, discussions are rarely needed, except
for discussing some very important design decisions.  Most of the time
you just develop your corner.

Lots of discussions about some feature is IME the first sign that no
one is actually working on it in any serious way.

I remember the beginning of the bidi development: it started by a few
years of discussions (on a separate mailing list) that led nowhere,
and most of them didn't even contribute any useful ideas for what
eventually became the implementation we now have in Emacs.  Once I
started to actually work on this, there was a small number of threads
(maybe 5) here, when I felt I needed to share my main design decisions
and get some feedback.

> That would be something for the new maintainer(s) to consider.

Indeed.

> > For
> > starters, I don't imagine they would succeed without some significant
> > changes and additions in the core infrastructure.  The place to
> > discuss that is here.
> 
> For refactoring, yes. But just "accurate code completion", without 
> extras like expanding the arguments or displaying the method source, can 
> be (and is, in certain packages) implemented though the 
> completion-at-point-function interface, present in Emacs since 24.1.

We didn't even decide that we want that as our mechanism.  Did anyone
consider how this compares with what the other modern IDEs offer?
What if we build our completion on a UI that today's developers will
dislike?  Unlike with many traditional Emacs features, which were
developed when there was no prior art, the IDE features have lots of
prior art.  No need to invent the wheel, just implement similar look
and feel.

> > Then what exactly is the nature of your objections to my observations?
> > It seems we agree on the bottom line: no one works on this.  The
> > reasons are immaterial.
> 
> If anything, my view of the situation is a lot brighter than yours.

I'll be happy to stand corrected.  Unfortunately, I don't yet see any
significant changes in the right direction, so my pessimism is for now
at least as justified as your optimism.

Re: IDE

[Dmitry Gutov]

On 10/10/2015 12:40 PM, Eli Zaretskii wrote:

> I didn't ignore that.  I just don't see an effort to make Emacs a
> modern IDE.  Working on separate parts of that in separate
> uncoordinated activities is not the way we should pursue this, IMO.

At least we "have volunteers" for that.

> It's inefficient at best, and at worst will end up with those
> uncoordinated parts falling into oblivion when their driving forces
> move on.

That would be accurate to say about projects at early stages of 
development, but the respective third-party packages already work now. 
If anything, we could keep them working, even Emacs undergoes large 
internal changes.

> It needs to be actively developed.  Much more actively than it is now.

I suppose.

> I don't know.  It's for someone who will work on this to find out.  I
> know that a motivated individual in the GDB development team already
> based a useful set of commands on it -- you can compile and inject
> code into your program while debugging it.

That seems orthogonal to code completion capabilities, for where I'm 
standing. But I'm not a good person to judge.

This does make a good material for a feature request for Irony, 
unfortunately. Someone more knowledgeable should do that instead.

>> We definitely could have more in this department, yes. But what would
>> you even call an "IDE mode"? A fixed multi-window setup a la ECB?
>
> I don't know, and neither do we as a project.  A useful step would be
> to produce a detailed answer to that question.  That answer could both
> serve as base for useful discussions, and might provide some anchor
> for all those external packages you mentioned to target some coherent
> vision.

"We need a common interface for refactoring tools" sounds like a good 
problem statement. I hope that capability can grow from the xref 
interface, but that needs more work and thought.

> I don't believe comprehensive features such as IDE can be developed
> exclusively bottom up.  There should be some basic set of assumptions
> and design rules/decisions that everyone should target and abide by.
> There should also be some unified leadership.

A comprehensive set of IDE features might be too lofty a goal for us, in 
the foreseeable future.

> We didn't even decide that we want that as our mechanism.  Did anyone
> consider how this compares with what the other modern IDEs offer?

completion-at-point-functions is the API for backends to implement. We 
have a few frontends for it already. Company can use it, for one.

> What if we build our completion on a UI that today's developers will
> dislike?  Unlike with many traditional Emacs features, which were
> developed when there was no prior art, the IDE features have lots of
> prior art.  No need to invent the wheel, just implement similar look
> and feel.

Hence we're bundling Company.

Re: IDE

[Eli Zaretskii]

> Cc: adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 10 Oct 2015 13:14:53 +0300
> 
> On 10/10/2015 12:40 PM, Eli Zaretskii wrote:
> 
> > I don't know.  It's for someone who will work on this to find out.  I
> > know that a motivated individual in the GDB development team already
> > based a useful set of commands on it -- you can compile and inject
> > code into your program while debugging it.
> 
> That seems orthogonal to code completion capabilities, for where I'm 
> standing.

Of course, it is.  I didn't mean to say that injecting code and
completion/refactoring need the same capabilities.  But libcc1 doesn't
provide only what GDB uses, and it can be extended.

> >> We definitely could have more in this department, yes. But what would
> >> you even call an "IDE mode"? A fixed multi-window setup a la ECB?
> >
> > I don't know, and neither do we as a project.  A useful step would be
> > to produce a detailed answer to that question.  That answer could both
> > serve as base for useful discussions, and might provide some anchor
> > for all those external packages you mentioned to target some coherent
> > vision.
> 
> "We need a common interface for refactoring tools" sounds like a good 
> problem statement.

Is IDE just about refactoring?  I thought it meant much more.

> > I don't believe comprehensive features such as IDE can be developed
> > exclusively bottom up.  There should be some basic set of assumptions
> > and design rules/decisions that everyone should target and abide by.
> > There should also be some unified leadership.
> 
> A comprehensive set of IDE features might be too lofty a goal for us, in 
> the foreseeable future.

Depends on how many people will work on it.  In any case, having some
high-level design that is targeted by all the components will ensure
more or less seamless integration when each component becomes
available.

> > What if we build our completion on a UI that today's developers will
> > dislike?  Unlike with many traditional Emacs features, which were
> > developed when there was no prior art, the IDE features have lots of
> > prior art.  No need to invent the wheel, just implement similar look
> > and feel.
> 
> Hence we're bundling Company.

Last time I looked the IDEs I sometimes look at (Visual Studio and
Eclipse) present a much more pleasant UI for completion.  Why can't we
present something similar?

Re: IDE

[Dmitry Gutov]

On 10/10/2015 01:34 PM, Eli Zaretskii wrote:

>> "We need a common interface for refactoring tools" sounds like a good
>> problem statement.
>
> Is IDE just about refactoring?  I thought it meant much more.

The above more focused and, as such, more useful. "Comprehensive IDE 
features" is not as useful.

>> A comprehensive set of IDE features might be too lofty a goal for us, in
>> the foreseeable future.
>
> Depends on how many people will work on it.

How many people would you expect to work on it in the near future, 
realistically?

> In any case, having some
> high-level design that is targeted by all the components will ensure
> more or less seamless integration when each component becomes
> available.

 From where I'm standing, this sentence is not useful. Not only you're 
asking for a big design, you don't present a justification for it, e.g. 
how it would be reflected in all components.

> Last time I looked the IDEs I sometimes look at (Visual Studio and
> Eclipse) present a much more pleasant UI for completion.  Why can't we
> present something similar?

Well, that hurts (a bit). If Company's tooltip is not pleasant, what 
would be pleasant for you?

Re: IDE

[Eli Zaretskii]

> Cc: adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 10 Oct 2015 13:50:59 +0300
> 
> On 10/10/2015 01:34 PM, Eli Zaretskii wrote:
> 
> >> "We need a common interface for refactoring tools" sounds like a good
> >> problem statement.
> >
> > Is IDE just about refactoring?  I thought it meant much more.
> 
> The above more focused and, as such, more useful. "Comprehensive IDE 
> features" is not as useful.

But it narrows the field too much, IMO.

> >> A comprehensive set of IDE features might be too lofty a goal for us, in
> >> the foreseeable future.
> >
> > Depends on how many people will work on it.
> 
> How many people would you expect to work on it in the near future, 
> realistically?

I don't know.

If we cannot find enough, then it simply means it will take more time
to implement the features sequentially rather than in parallel.

> > In any case, having some
> > high-level design that is targeted by all the components will ensure
> > more or less seamless integration when each component becomes
> > available.
> 
>  From where I'm standing, this sentence is not useful. Not only you're 
> asking for a big design, you don't present a justification for it, e.g. 
> how it would be reflected in all components.

Are you saying that high-level design is generally not useful, and
should be avoided, unless "justified"?  That goes against the
engineering principles that the whole industry abides by.

> > Last time I looked the IDEs I sometimes look at (Visual Studio and
> > Eclipse) present a much more pleasant UI for completion.  Why can't we
> > present something similar?
> 
> Well, that hurts (a bit).

Sorry.

> If Company's tooltip is not pleasant, what would be pleasant for
> you?

I just gave you 2 examples.

And it's not really about me, it's about the expectations of users out
there.  Don't they expect something they see elsewhere?

Re: IDE

[Dmitry Gutov]

On 10/10/2015 02:03 PM, Eli Zaretskii wrote:

> Are you saying that high-level design is generally not useful, and
> should be avoided, unless "justified"?

A justification would further the discussion. Simply asking for a big 
design, doesn't.

>> If Company's tooltip is not pleasant, what would be pleasant for
>> you?
>
> I just gave you 2 examples.

You didn't explain how they are better.

> And it's not really about me, it's about the expectations of users out
> there.  Don't they expect something they see elsewhere?

Sometimes. A common question is for a popup that doesn't conflict with 
other packages that use overlay rendering for other things.

Did you also mean this aspect?

Re: IDE

[Eli Zaretskii]

> Cc: adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 10 Oct 2015 17:15:07 +0300
> 
> On 10/10/2015 02:03 PM, Eli Zaretskii wrote:
> 
> > Are you saying that high-level design is generally not useful, and
> > should be avoided, unless "justified"?
> 
> A justification would further the discussion. Simply asking for a big 
> design, doesn't.

It's standard software engineering practice, why should you ask for
its justification?

> >> If Company's tooltip is not pleasant, what would be pleasant for
> >> you?
> >
> > I just gave you 2 examples.
> 
> You didn't explain how they are better.

They look nicer.  I don't know how to explain better.

> > And it's not really about me, it's about the expectations of users out
> > there.  Don't they expect something they see elsewhere?
> 
> Sometimes. A common question is for a popup that doesn't conflict with 
> other packages that use overlay rendering for other things.

The other IDEs use something similar to a tooltip, or a drop-down menu
with different fonts and colors.

Re: IDE

[martin rudalics]

 > The other IDEs use something similar to a tooltip, or a drop-down menu
 > with different fonts and colors.

Cedet does have ‘semantic-displayor-tooltip-mode’.  And I'm using
tooltips all day for eldoc.  The only drawback is that Emacs tooltips
are too voracious.

martin

Re: IDE

[Dmitry Gutov]

On 10/10/2015 08:52 PM, martin rudalics wrote:

> Cedet does have ‘semantic-displayor-tooltip-mode’.

Could you explain how to try it out?

Re: IDE

[martin rudalics]

> Could you explain how to try it out?

No. I just noticed it when I started to use tooltips for ElDoc.

martin

Re: IDE

[Dmitry Gutov]

On 10/11/2015 01:25 PM, martin rudalics wrote:
>> Could you explain how to try it out?
>
> No. I just noticed it when I started to use tooltips for ElDoc.

Too bad.

I tried (setq semantic-complete-inline-analyzer-displayor-class 
'semantic-displayor-tooltip), with no apparent result.

Re: IDE

[David Engster]

Dmitry Gutov writes:
> On 10/11/2015 01:25 PM, martin rudalics wrote:
>>> Could you explain how to try it out?
>>
>> No. I just noticed it when I started to use tooltips for ElDoc.
>
> Too bad.
>
> I tried (setq semantic-complete-inline-analyzer-displayor-class
> 'semantic-displayor-tooltip), with no apparent result.

Make sure you call semantic-complete-analyze-inline to display
completions.

Personally, I don't use the tooltip displayor. I don't know if this is a
general problem or one with the implementation in CEDET, but I find them
too slow, and they don't play well with keyboard navigation.

-David

Re: IDE

[Dmitry Gutov]

On 10/11/2015 03:16 PM, David Engster wrote:

>> I tried (setq semantic-complete-inline-analyzer-displayor-class
>> 'semantic-displayor-tooltip), with no apparent result.
>
> Make sure you call semantic-complete-analyze-inline to display
> completions.

Tried that. No dice.

> Personally, I don't use the tooltip displayor. I don't know if this is a
> general problem or one with the implementation in CEDET, but I find them
> too slow, and they don't play well with keyboard navigation.

I thought tooltips don't affect the keyboard input?

Re: IDE

[David Engster]

Dmitry Gutov writes:
> On 10/11/2015 03:16 PM, David Engster wrote:
>
>>> I tried (setq semantic-complete-inline-analyzer-displayor-class
>>> 'semantic-displayor-tooltip), with no apparent result.
>>
>> Make sure you call semantic-complete-analyze-inline to display
>> completions.
>
> Tried that. No dice.

Works for me from 'emacs -Q' (emacs 24.5, lucid toolkit):

- M-x semantic-mode
- eval '(setq semantic-complete-inline-analyzer-displayor-class 'semantic-displayor-tooltip)'
- Load empty file 'test.c' and insert

void foo();
void foo2();

int main()
{
  fo
}

- Put cursor behind 'fo' and do M-x semantic-complete-analyze-inline.

>> Personally, I don't use the tooltip displayor. I don't know if this
>> is a
>> general problem or one with the implementation in CEDET, but I find
>> them
>> too slow, and they don't play well with keyboard navigation.
>
> I thought tooltips don't affect the keyboard input?

I mean I cannot choose completions with the keyboard with up/down. But
this is a problem with how it is done in CEDET, which was never my cup
of tea, but I also didn't work on it because as I said, I found it to be
too slow to display in the first place (that was years ago though, and I
guess it also depends on the toolkit, plus I often times worked over a
remote X connection).

-David

Re: IDE

[Dmitry Gutov]

On 10/11/2015 03:37 PM, David Engster wrote:

> Works for me from 'emacs -Q' (emacs 24.5, lucid toolkit):

Thanks, this sequence works for me too, even on master, but only with 
'-Q'. So something in my config is at fault.

>> I thought tooltips don't affect the keyboard input?
>
> I mean I cannot choose completions with the keyboard with up/down. But
> this is a problem with how it is done in CEDET, which was never my cup

Right, CEDET would have to handle keyboard input somehow, for it to work 
better.

Re: IDE

[Eric Ludlam]

On 10/11/2015 08:56 AM, Dmitry Gutov wrote:
> On 10/11/2015 03:37 PM, David Engster wrote:
>
>> Works for me from 'emacs -Q' (emacs 24.5, lucid toolkit):
>
> Thanks, this sequence works for me too, even on master, but only with
> '-Q'. So something in my config is at fault.
>
>>> I thought tooltips don't affect the keyboard input?
>>
>> I mean I cannot choose completions with the keyboard with up/down. But
>> this is a problem with how it is done in CEDET, which was never my cup
>
> Right, CEDET would have to handle keyboard input somehow, for it to work
> better.

Back when I added that, there was no way for me to get keyboard 
navigation to work with the tooltip on my linux.  There were menus 
available that did handle keyboard nav depending on toolkit, but those 
blocked typing, so I couldn't use them for the inline display that might 
popup automatically while you type.

I have found that some of the other completion UIs have worked much 
better than the ones I put together, so I had since focused on just 
getting the completions as correct as I can.

Eric

Re: IDE

[Dmitry Gutov]

On 10/12/2015 02:45 PM, Eric Ludlam wrote:

> There were menus
> available that did handle keyboard nav depending on toolkit, but those
> blocked typing, so I couldn't use them for the inline display that might
> popup automatically while you type.

Indeed. That's still the case, so using a menu for that is not an option.

Re: IDE

[Eli Zaretskii]

> Cc: martin rudalics <rudalics@gmx.at>, Eli Zaretskii <eliz@gnu.org>,
>  adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 12 Oct 2015 14:47:53 +0300
> 
>     There were menus
>     available that did handle keyboard nav depending on toolkit, but those
>     blocked typing, so I couldn't use them for the inline display that might
>     popup automatically while you type.
> 
> Indeed. That's still the case, so using a menu for that is not an option.

I don't see why you would conclude this.  Imagine that each keystroke
pops down the menu and then immediately pops it up with the updated
display -- what is the problem with that in this scenario?

Re: IDE

[Dmitry Gutov]

On 10/12/2015 06:55 PM, Eli Zaretskii wrote:

> I don't see why you would conclude this.  Imagine that each keystroke
> pops down the menu and then immediately pops it up with the updated
> display -- what is the problem with that in this scenario?

If that can work quickly enough, that minus ones problem.

But our menus are rather limited in terms of decoration. You asked for a 
popup with pictures and different fonts, I don't think we'd support it 
that way.

Re: IDE

[Eli Zaretskii]

> Cc: eric@siege-engine.com, deng@randomsample.de, rudalics@gmx.at,
>  adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 12 Oct 2015 19:21:43 +0300
> 
> But our menus are rather limited in terms of decoration. You asked for a 
> popup with pictures and different fonts, I don't think we'd support it 
> that way.

OK, then I guess having small frames is the way to go.  But then we'd
need to come up with some alternative for text-mode terminals.

Re: IDE

[Dmitry Gutov]

On 10/12/2015 07:58 PM, Eli Zaretskii wrote:

> OK, then I guess having small frames is the way to go.  But then we'd
> need to come up with some alternative for text-mode terminals.

In terminal, we won't be able to use the decorations anyway (right?). So 
if the Lisp interface for graphical-mode menus is the same as for 
terminal menus, we can create a menu-based popup implementation anyway.

In that case, since it'll work everywhere, it would be a good first 
step, while small frames can be left for later.

Re: IDE

[Eli Zaretskii]

> Cc: eric@siege-engine.com, deng@randomsample.de, rudalics@gmx.at,
>  adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Mon, 12 Oct 2015 20:26:44 +0300
> 
> On 10/12/2015 07:58 PM, Eli Zaretskii wrote:
> 
> > OK, then I guess having small frames is the way to go.  But then we'd
> > need to come up with some alternative for text-mode terminals.
> 
> In terminal, we won't be able to use the decorations anyway
> (right?).

On a TTY, there are no decorations, of course.  Worse,a frame you want
to display completely obscures all the other frames, which is
unacceptable for this kind of features.

> So if the Lisp interface for graphical-mode menus is the same as for
> terminal menus, we can create a menu-based popup implementation
> anyway.

Yes, the interface is the same: x-popup-menu.

> In that case, since it'll work everywhere, it would be a good first 
> step, while small frames can be left for later.

Right.

Re: IDE

[Dmitry Gutov]

On 10/10/2015 05:25 PM, Eli Zaretskii wrote:

> It's standard software engineering practice, why should you ask for
> its justification?

I'm asking for details. Again, to further the discussion.

"Let's unify features X, Y and Z" is not necessarily the standard 
practice. It all depends on the exact features and how they are used.

I've also given a few general reasons why a "big design" might be 
suboptimal: more engineering effort needed, the result is likely to turn 
out to be less flexible, and a significant number of users might prefer 
to use only some of the parts.

For instance, because we don't provide support for some feature in their 
environment (e.g. refactoring for Ruby), or because they already have a 
satisfactory solution for the feature in question. And as you know, some 
users dislike changing their habits.

> They look nicer.  I don't know how to explain better.

I would expect something akin to a normal bug report (enumerating things 
we're missing). But never mind.

> The other IDEs use something similar to a tooltip, or a drop-down menu
> with different fonts and colors.

You can already customize colors and fonts user for the Company popup. 
But if you end up using fonts with different dimensions, of course, that 
would result in jagged display.

I'd be happy to use a "native" tooltip in Emacs, but didn't have time to 
fully investigate it yet. However, there's a third-party package called 
pos-tip which tries to provide a "show tooltip" interface based on 
x-show-tip. From trying it out, I have the following complaints about 
x-show-tip capabilities:

- It's background rendering is inconsistent. As an example, the first 
time I evaluate (tooltip-show "abc") in an Emacs session, the background 
is yellow-ish. The next time, and after that, the background is black.
- Is there a way to show several tooltips at once? To display different 
elements of the completion UI side by side.
- If a tooltip is displayed, and I Alt-Tab to another program's window, 
the tooltip remains on top. This is by far the most annoying one.

Re: IDE

[martin rudalics]

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]

 > - It's background rendering is inconsistent. As an example, the first
 > time I evaluate (tooltip-show "abc") in an Emacs session, the
 > background is yellow-ish. The next time, and after that, the
 > background is black.

This is the reason I'm always using ‘x-show-tip’.

 > - Is there a way to show several tooltips at once? To display different elements of the completion UI side by side.

AFAICT no.  You'd have to put them into the same tooltip.

 > - If a tooltip is displayed, and I Alt-Tab to another program's window, the tooltip remains on top. This is by far the most annoying one.

Very annoying, indeed.  See my solution for this in
‘eldoc-tooltip-mode’.

martin

[-- Attachment #2: eldoc-tooltip.el --]
[-- Type: application/emacs-lisp, Size: 21418 bytes --]

Re: IDE

[Dmitry Gutov]

Thanks! I've tried your package out, and it's pretty nice. Especially 
the part where you carefully try to position the popup so it doesn't 
hide the code.

On 10/11/2015 01:17 PM, martin rudalics wrote:
>  > - It's background rendering is inconsistent. As an example, the first
>  > time I evaluate (tooltip-show "abc") in an Emacs session, the
>  > background is yellow-ish. The next time, and after that, the
>  > background is black.
>
> This is the reason I'm always using ‘x-show-tip’.

That doesn't fix the problem. Even with your package, it looked fine 
with greenish background for a while, then I switched away from Emacs, 
did some reading in the web browser. Then switched back to Emacs, and 
the background is black now.

>  > - Is there a way to show several tooltips at once? To display
> different elements of the completion UI side by side.
>
> AFAICT no.  You'd have to put them into the same tooltip.

I think I'll hold off on trying on integrate it in Company until this 
feature is implemented.

For feature parity with Intellij IDEA and MS VS, we should be able to 
display the list of completions and documentation for the currently 
selected completion in two separate popups:

https://i-msdn.sec.s-msft.com/dynimg/IC797655.jpeg
https://www.jetbrains.com/img/webhelp/idea/constructors_docs_in_completion.png

In Company (as well as the IDEA interface), "show documentation" is a 
separate action which can take some perceptible time, so we can't 
include the documentation in the same popup as completions.

>  > - If a tooltip is displayed, and I Alt-Tab to another program's
> window, the tooltip remains on top. This is by far the most annoying one.
>
> Very annoying, indeed.  See my solution for this in
> ‘eldoc-tooltip-mode’.

focus-out-hook? That'll be good enough, thanks.

Re: IDE

[martin rudalics]

 > That doesn't fix the problem. Even with your package, it looked fine
 > with greenish background for a while, then I switched away from Emacs,
 > did some reading in the web browser. Then switched back to Emacs, and
 > the background is black now.

Are you by any chance using GTK tooltips?  They are a pain.

 >>  > - Is there a way to show several tooltips at once? To display
 >> different elements of the completion UI side by side.
 >>
 >> AFAICT no.  You'd have to put them into the same tooltip.
 >
 > I think I'll hold off on trying on integrate it in Company until this
 > feature is implemented.

Maybe we should just use simple frames with all decorations removed.
The timeout to unshow the frame is not useful anyway and to remove the
overhead for frame creation it would suffice to make the frame invisible
as long as it's not needed.

martin

Re: IDE

[Dmitry Gutov]

On 10/11/2015 02:38 PM, martin rudalics wrote:

> Are you by any chance using GTK tooltips?  They are a pain.

I've built Emacs with GTK 3, if that's what you mean. The configure 
script does that by default here.

> Maybe we should just use simple frames with all decorations removed.

How do I do that?

> The timeout to unshow the frame is not useful anyway and to remove the
> overhead for frame creation it would suffice to make the frame invisible
> as long as it's not needed.

The timeout is not useful indeed.

Re: IDE

[martin rudalics]

 >> Are you by any chance using GTK tooltips?  They are a pain.
 >
 > I've built Emacs with GTK 3, if that's what you mean. The configure script does that by default here.

Try setting ‘x-gtk-use-system-tooltips’ to nil.

 >> Maybe we should just use simple frames with all decorations removed.
 >
 > How do I do that?

By making a frame without minibuffer, title, scroll, tool, menu bars and
borders.  Maybe it can't be done in Lisp alone.

martin

Re: IDE

[David Engster]

martin rudalics writes:
>>> Are you by any chance using GTK tooltips?  They are a pain.
>>
>> I've built Emacs with GTK 3, if that's what you mean. The configure
>> script does that by default here.
>
> Try setting ‘x-gtk-use-system-tooltips’ to nil.
>
>>> Maybe we should just use simple frames with all decorations
>>> removed.
>>
>> How do I do that?
>
> By making a frame without minibuffer, title, scroll, tool, menu bars
> and
> borders.  Maybe it can't be done in Lisp alone.

Here's what I use in my doc-present package[1] for displaying slides:

(with-selected-frame
    (make-frame '((minibuffer . nil)
		  (left-fringe . 0)
		  (right-fringe . 0)
		  (menu-bar-lines . 0)
		  (internal-border-width . 0)
		  (vertical-scroll-bars . nil)
		  (unsplittable . t)
		  (cursor-type . nil)
		  (tool-bar-lines . 0)))
  (pop-to-buffer (get-buffer-create "*test*"))
  (setq mode-line-format nil))

The only thing remaining are the decorations from the window manager. I
don't think you can get rid of those from within Emacs?

-David

Re: IDE

[martin rudalics]

 > The only thing remaining are the decorations from the window manager. I
 > don't think you can get rid of those from within Emacs?

 From Lisp I suppose only via the tooltip interface.  Also one has to
make sure that the frame always stays on top of the Z-order.

martin

Re: IDE

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,  Eli Zaretskii <eliz@gnu.org>,  adatgyujto@gmail.com,  emacs-devel@gnu.org
> Date: Sun, 11 Oct 2015 14:27:02 +0200
> 
> Here's what I use in my doc-present package[1] for displaying slides:
> 
> (with-selected-frame
>     (make-frame '((minibuffer . nil)
> 		  (left-fringe . 0)
> 		  (right-fringe . 0)
> 		  (menu-bar-lines . 0)
> 		  (internal-border-width . 0)
> 		  (vertical-scroll-bars . nil)
> 		  (unsplittable . t)
> 		  (cursor-type . nil)
> 		  (tool-bar-lines . 0)))
>   (pop-to-buffer (get-buffer-create "*test*"))
>   (setq mode-line-format nil))
> 
> The only thing remaining are the decorations from the window manager. I
> don't think you can get rid of those from within Emacs?

We could easily extend make-frame to allow that, by defining new
attributes.

Re: IDE

[Eli Zaretskii]

> Date: Sun, 11 Oct 2015 13:38:35 +0200
> From: martin rudalics <rudalics@gmx.at>
> CC: adatgyujto@gmail.com, emacs-devel@gnu.org
> 
> Maybe we should just use simple frames with all decorations removed.

How is that different from tooltips?  They are such frames, AFAIK.

Re: IDE

[martin rudalics]

 > How is that different from tooltips?  They are such frames, AFAIK.

Yes.  Just that we do not expose them to Lisp.

martin

Re: IDE

[Eli Zaretskii]

> Cc: adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 11 Oct 2015 14:02:34 +0300
> 
> For feature parity with Intellij IDEA and MS VS, we should be able to 
> display the list of completions and documentation for the currently 
> selected completion in two separate popups:
> 
> https://i-msdn.sec.s-msft.com/dynimg/IC797655.jpeg
> https://www.jetbrains.com/img/webhelp/idea/constructors_docs_in_completion.png

The first one looks like a popup menu, the second like a small frame,
is that right?

Re: IDE

[Dmitry Gutov]

On 10/11/2015 06:25 PM, Eli Zaretskii wrote:

>> https://i-msdn.sec.s-msft.com/dynimg/IC797655.jpeg
>> https://www.jetbrains.com/img/webhelp/idea/constructors_docs_in_completion.png
>
> The first one looks like a popup menu, the second like a small frame,
> is that right?

Both pictures show similar configurations: a menu with completions and a 
small frame with documentation.

In the second one the frame just overlaps the menu, for some reason (I 
couldn't find a better screenshot on a short notice).

Re: IDE

[Oleh Krehel]

Dmitry Gutov <dgutov@yandex.ru> writes:

> For feature parity with Intellij IDEA and MS VS, we should be able to
> display the list of completions and documentation for the currently
> selected completion in two separate popups:
>
> https://i-msdn.sec.s-msft.com/dynimg/IC797655.jpeg
> https://www.jetbrains.com/img/webhelp/idea/constructors_docs_in_completion.png

I like the first style a lot more. The second looks a lot like the ugly
mess of Eclipse.

Here's what's currently possible in Emacs for C++:

- show function arguments and docstring in an overlay:
    http://oremacs.com/download/fa-do-comments.png
- complete class member at point:
    http://oremacs.com/download/function-args-qt.png
- jump to tag in directory:
    http://oremacs.com/download/function-args-boost.png

The latter two can be done with powerful regexp-based completion, which
MS VS likely still doesn't include.

What's missing, compared to the MS VS picture:

- Candidate completion is in the minibuffer instead of at-point.
- The docstring (only the comment part) is shown separately.

The first part is just Emacs' style of doing things: we usually enter
stuff in the minibuffer, so it makes sense for completion to display
there.  The second part is arguably unnecessary: I usually just jump to
definition of symbol rather than look at the docstring inline.

What's missing, in my opinion, is only faster and more precise parser
(CEDET, GCC etc). For example, currently `semantic-fetch-tags' parses
public/private/protected labels as tags, instead of applying these
properties to actual tags. If that were so, it would be very easy to add
a public/private/protected icon to each tag, just like MS VS does it.

Another example is the QT code: it's a popular LGPL C++ framework that's
currently hard to setup for CEDET.
For instance, `#include <QtGui/QPushButton>` is a plain file without an 
extension with only this code inside:

    #include "qpushbutton.h"

Since the extension isn't recognized, it's not parsed by CEDET.  And I
have to write `#include "qpushbutton.h"` in my application instead of
the more preferred `#include <QtGui/QPushButton>`, because that way I
get tag completion.

These small things are what the competing IDE have been incrementally
improving over the last 20 years. I think we have a serviceable
foundation for C++ completion, it only needs to be polished *a lot*
more.

I also think that the way to do it right would be to integrate with GCC
for code parsing, for reasons of speed and precision.  As I mentioned
before, CEDET is usable, but it can't be as fast and as precise as
GCC. Add to that that the language standard is updated every 5 years or
so with new syntax. GCC has the people to update the parser
accordingly. Doing so for CEDET would be a duplication of effort, and we
don't have the people to do it anyway.

Could someone explain to me if making GCC the dependency of Emacs would
be a good idea, from technical and freedom point of view?  Personally, I
wouldn't care if Emacs executable would get inflated a bit more, if that
meant access to true IDE features, which are only possible with a
precise and fast parser.

    Oleh

Re: IDE

[Dmitry Gutov]

On 10/12/2015 02:05 PM, Oleh Krehel wrote:

> I like the first style a lot more. The second looks a lot like the ugly
> mess of Eclipse.

I agree, the Microsoft IDE looks slicker here, but the examples are 
basically the same, in that they use separate frames for the completion 
list and documentation, not the same one.

Eclipse uses a combined one, last I checked.

> The first part is just Emacs' style of doing things: we usually enter
> stuff in the minibuffer, so it makes sense for completion to display

A lot of users are fine with that, but I think we should do better.

Ivy also could use a popup to display completions, in the future. 
Ideally, I'd prefer something like Sublime/Atom popup at the top of the 
screen that you get after pressing C-p.

> there.  The second part is arguably unnecessary: I usually just jump to
> definition of symbol rather than look at the docstring inline.

You'd think so, but displaying the docstring automatically, like 
Auto-Complete does (as well as certain IDEs), has been a common request 
for a while. And now, https://github.com/expez/company-quickhelp is 
pretty popular.

> CEDET is usable, but it can't be as fast and as precise as
> GCC. Add to that that the language standard is updated every 5 years or
> so with new syntax. GCC has the people to update the parser
> accordingly. Doing so for CEDET would be a duplication of effort, and we
> don't have the people to do it anyway.

Agree.

> Could someone explain to me if making GCC the dependency of Emacs would
> be a good idea, from technical and freedom point of view?  Personally, I
> wouldn't care if Emacs executable would get inflated a bit more, if that
> meant access to true IDE features, which are only possible with a
> precise and fast parser.

Having the whole GCC as a dependency might be problematic, but that's 
not the #1 problem. AFAIK, GCC currently has no "code completion" 
feature anywhere.

Re: IDE

[Oleh Krehel]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Ivy also could use a popup to display completions, in the
> future. Ideally, I'd prefer something like Sublime/Atom popup at the
> top of the screen that you get after pressing C-p.

Eventually, yes. But it's hard for me to figure out how to make the text
and the overlays work together while the text is being updated.  Perhaps
I should try Sublime/Atom at least once, just to see how it's done
there.

>> Could someone explain to me if making GCC the dependency of Emacs would
>> be a good idea, from technical and freedom point of view?  Personally, I
>> wouldn't care if Emacs executable would get inflated a bit more, if that
>> meant access to true IDE features, which are only possible with a
>> precise and fast parser.
>
> Having the whole GCC as a dependency might be problematic, but that's
> not the #1 problem. AFAIK, GCC currently has no "code completion"
> feature anywhere.

There's no need for a specific "code completion" feature. Take this code
for example:

    #include "qapplication.h"
    #include "qfont.h"
    #include "qpushbutton.h"
    
    int main(int argc, char *argv[]) {
        QApplication app(argc, argv);
        QPushButton quit("Quit");
        quit.
            
        return 0;
    }

Here's what (semantic-fetch-tags) returns:

    (("qapplication.h"
      include
      nil
      nil
      #<overlay from 1 to 26 in main.cc>)
     ("qfont.h"
      include
      nil
      nil
      #<overlay from 27 to 45 in main.cc>)
     ("qpushbutton.h"
      include
      nil
      nil
      #<overlay from 46 to 70 in main.cc>)
     ("main"
      function
      (:arguments (("argc"
                    variable
                    (:type "int")
                    (reparse-symbol arg-sub-list)
                    #<overlay from 81 to 90 in main.cc>)
                   ("argv"
                    variable
                    (:pointer 1
                              :dereference 1
                              :type "char")
                    (reparse-symbol arg-sub-list)
                    #<overlay from 91 to 104 in main.cc>))
                  :type "int")
      nil
      #<overlay from 72 to 205 in main.cc>))

A similar data structure *has* to be somewhere in the GCC innards: it's
a first step for compilation. In addition, this information is used to
point out compilation errors/warnings.
This data structure would know all defined functions/variables/types.
For instance, it would know that the class QPushButton is defined in
qpushbutton.h at line 57, and it inherits from QAbstractButton.

It only remains to write a small wrapper that dumps all the AST included
into `file_name` up to the point `offset`:

    void gcc_ast_for_file (char* file_name, int offset, void* out)

I hope that I'm not being naive here and it's as simple to do as that.

    Oleh

Re: IDE

[Óscar Fuentes]

Oleh Krehel <ohwoeowho@gmail.com> writes:

[snip]

> It only remains to write a small wrapper that dumps all the AST included
> into `file_name` up to the point `offset`:
>
>     void gcc_ast_for_file (char* file_name, int offset, void* out)
>
> I hope that I'm not being naive here and it's as simple to do as that.

As mentioned on other message of mine, this is what a GCC frontend
developer has to say (search for the comments of mlopezibanez):

https://lwn.net/Articles/629259/

Basically what he says is: "No, it is not simple at all to get that
information from GCC, you'll better use Clang."

Re: IDE

[Oleh Krehel]

Óscar Fuentes <ofv@wanadoo.es> writes:

> As mentioned on other message of mine, this is what a GCC frontend
> developer has to say (search for the comments of mlopezibanez):
>
> https://lwn.net/Articles/629259/
>
> Basically what he says is: "No, it is not simple at all to get that
> information from GCC, you'll better use Clang."

The words of a single frontend developer made unofficially on someone's
blog post aren't very reassuring.  I'd rather hear the official response
of the current GCC head maintainer.

    Oleh

Re: IDE

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Mon, 12 Oct 2015 15:08:46 +0200
> 
> As mentioned on other message of mine, this is what a GCC frontend
> developer has to say (search for the comments of mlopezibanez):
> 
> https://lwn.net/Articles/629259/
> 
> Basically what he says is: "No, it is not simple at all to get that
> information from GCC, you'll better use Clang."

I'd urge interested and motivated individuals not to take one man's
opinions for more (or less) than it is.

I could fill several hours telling how many many opinions about what
could and what couldn't be done for bidi I heard before I decided to
sit down and do it.

My advice: read what others say, investigate the issues, then make up
your own mind.

Re: IDE

[Dmitry Gutov]

On 10/12/2015 03:37 PM, Oleh Krehel wrote:

> Eventually, yes. But it's hard for me to figure out how to make the text
> and the overlays work together while the text is being updated.  Perhaps
> I should try Sublime/Atom at least once, just to see how it's done
> there.

Not sure how looking at Atom would help, but you should absolutely try 
it. It's Free Software, after all.

> Here's what (semantic-fetch-tags) returns:
>
> ...
> A similar data structure *has* to be somewhere in the GCC innards: it's
> a first step for compilation. In addition, this information is used to
> point out compilation errors/warnings.
> This data structure would know all defined functions/variables/types.
> For instance, it would know that the class QPushButton is defined in
> qpushbutton.h at line 57, and it inherits from QAbstractButton.

You have to find out what the type of the call target is, first.

In your example, it's rather easy: "QPushButton quit" is right on the 
previous line.

But what if we're at the end of a chain of calls, like

app->window->resolve_handler(bar).|

?

And resolve_handler is overloaded, and its return type is dependent on 
the type of its argument?

What if `bar' is declared as `auto' (see C++11)? Or `app' itself?

I'm fairly sure an experienced C++ tooling developer could add other 
complicated examples.

Re: IDE

[Oleh Krehel]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Not sure how looking at Atom would help, but you should absolutely try
> it. It's Free Software, after all.

Thanks for mentioning that. I wasn't sure that was the case.

>> Here's what (semantic-fetch-tags) returns:
>>
>> ...
>> A similar data structure *has* to be somewhere in the GCC innards: it's
>> a first step for compilation. In addition, this information is used to
>> point out compilation errors/warnings.
>> This data structure would know all defined functions/variables/types.
>> For instance, it would know that the class QPushButton is defined in
>> qpushbutton.h at line 57, and it inherits from QAbstractButton.
>
> You have to find out what the type of the call target is, first.
>
> In your example, it's rather easy: "QPushButton quit" is right on the
> previous line.
>
> But what if we're at the end of a chain of calls, like
>
> app->window->resolve_handler(bar).|
>
> ?
>
> And resolve_handler is overloaded, and its return type is dependent on
> the type of its argument?

CEDET already resolves these complicated chains pretty well, as long as
it's got the correct AST.

> What if `bar' is declared as `auto' (see C++11)? Or `app' itself?

This would be harder, but still very doable, even within CEDET.

Now, there are several issues standing in the way of getting the correct
AST:

1. Finding where the relevant headers are located on the file
system. This means that the AST parser should hook into the currently
used build system, and correctly see which #ifdef switches apply where.
Only GCC or its likes have access to this info always. To reiterate, if
the program compiles with GCC, the exact same switches must be passed to
the AST parser, so that it parses exactly the same headers in exactly
the same way.

2. Parsing the newly introduced language features. Here CEDET and
cc-mode are behind because of the shear complexity of the full C++
syntax. However, GCC does it very well.

I've been working on function-args.el to extend CEDET C++ support for my
uses for around 2 years now. The most common error that I have to work
around is "Type definition not found", which happens either because the
header path wasn't resolved, or the wrong #ifdef path was taken.

And, of course, there's the issue of Elisp speed. While jumping to a C
tag in emacs/src, I collect the tags from 190 files in that
directory. They are all pre-parsed by CEDET, totaling to 19460 tags. It
takes more than a full second to just merge these 190 lists retrieved
for each file into a single list. It makes me think that maybe that list
should be built in an external process.
On the other hand, having the full info in Elisp data structures would
ease the application development significantly.

    Oleh

Re: IDE

[Dmitry Gutov]

On 10/12/2015 05:08 PM, Oleh Krehel wrote:

> Thanks for mentioning that. I wasn't sure that was the case.

It's basically MIT license, see 
https://directory.fsf.org/wiki/Atom#tab=Details.

> CEDET already resolves these complicated chains pretty well, as long as
> it's got the correct AST.

Semantic has some known issues in this general area, exactly because 
it's hard. Clang, on the other hand, resolves them well.

>> What if `bar' is declared as `auto' (see C++11)? Or `app' itself?
>
> This would be harder, but still very doable, even within CEDET.

You mean something would have to implement that. Good luck.

Another possible complication is using function templates in that method 
call chain.

> 1. Finding where the relevant headers are located on the file
> system. This means that the AST parser should hook into the currently
> used build system, and correctly see which #ifdef switches apply where.
> Only GCC or its likes have access to this info always. To reiterate, if
> the program compiles with GCC, the exact same switches must be passed to
> the AST parser, so that it parses exactly the same headers in exactly
> the same way.

This is non-trivial, because the switches are usually constructed by the 
build tool at run time. But see:

https://github.com/Sarcasm/irony-mode#compilation-database

> 2. Parsing the newly introduced language features. Here CEDET and
> cc-mode are behind because of the shear complexity of the full C++
> syntax. However, GCC does it very well.

Note that any CEDET feature that handles target type resolution is also 
subject to having to deal with new C++ language features.

> I've been working on function-args.el to extend CEDET C++ support for my
> uses for around 2 years now. The most common error that I have to work
> around is "Type definition not found", which happens either because the
> header path wasn't resolved, or the wrong #ifdef path was taken.

It could simply mean that until things work well on this level, you're 
not going to pay attention to the more subtle problems.

Re: IDE

[Eli Zaretskii]

> From: Oleh Krehel <ohwoeowho@gmail.com>
> Cc: martin rudalics <rudalics@gmx.at>,  Eli Zaretskii <eliz@gnu.org>,  adatgyujto@gmail.com,  emacs-devel@gnu.org
> Date: Mon, 12 Oct 2015 14:37:26 +0200
> 
> > Having the whole GCC as a dependency might be problematic, but that's
> > not the #1 problem. AFAIK, GCC currently has no "code completion"
> > feature anywhere.
> 
> There's no need for a specific "code completion" feature. Take this code
> for example:
> 
>     #include "qapplication.h"
>     #include "qfont.h"
>     #include "qpushbutton.h"
>     
>     int main(int argc, char *argv[]) {
>         QApplication app(argc, argv);
>         QPushButton quit("Quit");
>         quit.
>             
>         return 0;
>     }
> 
> Here's what (semantic-fetch-tags) returns:
> 
>     (("qapplication.h"
>       include
>       nil
>       nil
>       #<overlay from 1 to 26 in main.cc>)
>      ("qfont.h"
>       include
>       nil
>       nil
>       #<overlay from 27 to 45 in main.cc>)
>      ("qpushbutton.h"
>       include
>       nil
>       nil
>       #<overlay from 46 to 70 in main.cc>)
>      ("main"
>       function
>       (:arguments (("argc"
>                     variable
>                     (:type "int")
>                     (reparse-symbol arg-sub-list)
>                     #<overlay from 81 to 90 in main.cc>)
>                    ("argv"
>                     variable
>                     (:pointer 1
>                               :dereference 1
>                               :type "char")
>                     (reparse-symbol arg-sub-list)
>                     #<overlay from 91 to 104 in main.cc>))
>                   :type "int")
>       nil
>       #<overlay from 72 to 205 in main.cc>))
> 
> A similar data structure *has* to be somewhere in the GCC innards: it's
> a first step for compilation. In addition, this information is used to
> point out compilation errors/warnings.

See the documentation of the various -fdump-rtl-* switches to GCC.  Or
maybe you want the -dx switch.

RE: IDE

[Drew Adams]

> I agree, the Microsoft IDE looks slicker here, but the examples
> are basically the same, in that they use separate frames for
> the completion list and documentation, not the same one.

Whether separate frames are used or not is not so important.
It is important, though, to be _able_ to show the completions
without necessarily showing the help for the current one.
IOW, separate display can be useful, whether or not separate
frames are used for that.

The Icicles approach lets you show the complete help for the
current candidate (systematically or on demand), but it does
not mandate that.  It happens to use separate windows (or
frames), but that is not important here, IMO - the same effect
could be provided with a single frame.

Icicles also automatically (by default - option turns on/off)
shows short help in the mode line (of `*Completions*'),
systematically.  Maybe that's all you had in mind, a one-liner?

If so, then users would be missing the ability to show the
complete help for the current candidate (again, systematically
or on demand).  Being able to do that is a big help, IMO.

Summary:

1. Users should be able to show candidates without also the help.
2. Whether the same or separate frames or windows are used is
   not important.  (It could be important to a given user, and
   it could be made a user choice.)
3. For simple, one-liner reminder help, the mode-line suffices.
4. Users should be able to show the complete help for the current
   candidate.  One-liner help is no substitute for this.

> > The first part is just Emacs' style of doing things: we usually
> > enter stuff in the minibuffer, so it makes sense for completion
> to display
> 
> A lot of users are fine with that, but I think we should do better.

What is better, and why?  Please don't gloss over this.

I would not argue that minibuffer input is always better than other
methods (e.g. at-point cycling).  But it has different pros & cons.

In particular, (1) it allows actual (and complete) editing, and
(2) it provides its own keymap, for completion features or cycling
features or candidate-action features.  (And users can easily
customize that keymap.)

Those are pros.  A con is that the minibuffer and `*Completions*'
are not necessarily displayed close to point.  That's a display
question that could be addressed in various ways.

> You'd think so, but displaying the docstring automatically, like
> Auto-Complete does (as well as certain IDEs), has been a common
> request for a while. And now, https://github.com/expez/company-
> quickhelp is pretty popular.

Does it display the complete doc string?  If not, I'd say users
are missing out.  If yes, and this is done systematically, I'd
say that users are being force-fed.  They should have a choice.

For systematic display (but it should still be "offable"),
one-line help is fine.  But it's not a replacement for being
able to see the whole doc string.

Re: IDE

[Dmitry Gutov]

On 10/12/2015 05:39 PM, Drew Adams wrote:

> Whether separate frames are used or not is not so important.
> It is important, though, to be _able_ to show the completions
> without necessarily showing the help for the current one.
> IOW, separate display can be useful, whether or not separate
> frames are used for that.

If you're forced to use the same frame, you're forced to make the 
documentation pane and the completions menu the same height (or width), 
or just have an empty rectangle somewhere. That's not ideal.

Having separate display would also need work in that case.

> What is better, and why?  Please don't gloss over this.

Not having to always look at the minibuffer when entering stuff.

> Those are pros.  A con is that the minibuffer and `*Completions*'
> are not necessarily displayed close to point.  That's a display
> question that could be addressed in various ways.

Right.

> Does it display the complete doc string?  If not, I'd say users
> are missing out.  If yes, and this is done systematically, I'd
> say that users are being force-fed.  They should have a choice.

The user can disable the minor mode.

RE: IDE

[Drew Adams]

> > Whether separate frames are used or not is not so important.
> > It is important, though, to be _able_ to show the completions
> > without necessarily showing the help for the current one.
> > IOW, separate display can be useful, whether or not separate
> > frames are used for that.
> 
> If you're forced to use the same frame, you're forced to make the
> documentation pane and the completions menu the same height (or
> width), or just have an empty rectangle somewhere. That's not ideal.

I agree.  I personally use separate frames for them, and the
frames are fit to the content they display.

My point was that what is important is the ability to separate
their display - not couple them in a hardcoded way.  Whether
separate frames are used for that is a separate question, and
less important.  (But yes, I too prefer separate frames.)

Separate frames allow not only elimination of wasted space, for
the reason you gave.  They can also be overlapped, which can be
useful for additional space savings.  IOW, sometimes you don't
need to see all of the contents of each frame, and you want to
additionally see other stuff on your display at the same time.
Frames allow great flexibility for screen real estate.

> > What is better, and why?  Please don't gloss over this.
> 
> Not having to always look at the minibuffer when entering stuff.

If what you are entering is simple then you don't have to look
at it.  And if not then you still need to look at the place where
you are inputting/editing the pattern to complete.

I agree that it can be handy for that to be point in the main
buffer.  But a con in that scenario is the attendant "busyness"
of that area: input pattern editing + output completions & help
display.  And if you allow not just completion and help, but
also actions on the current candidate (a la Helm or Icicles)
then the display can become even busier in that area.

And for a complex input pattern, which might be multiline,
using a separate editing buffer (the minibuffer) is a plus, IMO.

> > Does it display the complete doc string?  If not, I'd say users
> > are missing out.  If yes, and this is done systematically, I'd
> > say that users are being force-fed.  They should have a choice.
> 
> The user can disable the minor mode.

What does that mean?  Does it (can it) display the complete doc
string?

Re: IDE

[Dmitry Gutov]

On 10/12/2015 06:02 PM, Drew Adams wrote:

> My point was that what is important is the ability to separate
> their display - not couple them in a hardcoded way.  Whether
> separate frames are used for that is a separate question, and
> less important.  (But yes, I too prefer separate frames.)

You should read the source code of Company. One can plug in a different 
frontend, and have different visualization as a result.

> Separate frames allow not only elimination of wasted space, for
> the reason you gave.  They can also be overlapped, which can be
> useful for additional space savings.  IOW, sometimes you don't
> need to see all of the contents of each frame, and you want to
> additionally see other stuff on your display at the same time.
> Frames allow great flexibility for screen real estate.

My frames are always fullscreen.

> And for a complex input pattern, which might be multiline,
> using a separate editing buffer (the minibuffer) is a plus, IMO.

Sure.

>> The user can disable the minor mode.
>
> What does that mean?  Does it (can it) display the complete doc
> string?

Everything is fine there, and the user is in control. Like I said, we 
have a separate command to display the doc in a new buffer.

Re: IDE

[Eli Zaretskii]

> From: Oleh Krehel <ohwoeowho@gmail.com>
> Date: Mon, 12 Oct 2015 13:05:02 +0200
> Cc: martin rudalics <rudalics@gmx.at>, Eli Zaretskii <eliz@gnu.org>,
> 	adatgyujto@gmail.com, emacs-devel@gnu.org
> 
> > For feature parity with Intellij IDEA and MS VS, we should be able to
> > display the list of completions and documentation for the currently
> > selected completion in two separate popups:
> >
> > https://i-msdn.sec.s-msft.com/dynimg/IC797655.jpeg
> > https://www.jetbrains.com/img/webhelp/idea/constructors_docs_in_completion.png
> 
> I like the first style a lot more. The second looks a lot like the ugly
> mess of Eclipse.
> 
> Here's what's currently possible in Emacs for C++:
> 
> - show function arguments and docstring in an overlay:
>     http://oremacs.com/download/fa-do-comments.png
> - complete class member at point:
>     http://oremacs.com/download/function-args-qt.png
> - jump to tag in directory:
>     http://oremacs.com/download/function-args-boost.png

Talking just about the visual appearance of these, IMO using overlays
limits us in our choice of faces, and fonts.  So I think it's better
to use tooltips or menus or maybe specialized small frames (which are
just tooltips with some limitations lifted and with automatic hide
action removed).

> The latter two can be done with powerful regexp-based completion, which
> MS VS likely still doesn't include.

This aspect is orthogonal to the visual appearance, I think.

> What's missing, compared to the MS VS picture:
> 
> - Candidate completion is in the minibuffer instead of at-point.
> - The docstring (only the comment part) is shown separately.
> 
> The first part is just Emacs' style of doing things: we usually enter
> stuff in the minibuffer, so it makes sense for completion to display
> there.  The second part is arguably unnecessary: I usually just jump to
> definition of symbol rather than look at the docstring inline.

IMO, we should support both the "traditional" Emacs style, and the
style users expect because other IDEs provide them.

> Another example is the QT code: it's a popular LGPL C++ framework that's
> currently hard to setup for CEDET.
> For instance, `#include <QtGui/QPushButton>` is a plain file without an 
> extension with only this code inside:
> 
>     #include "qpushbutton.h"
> 
> Since the extension isn't recognized, it's not parsed by CEDET.

Good C++ support indeed requires to be able to DTRT with
extension-less header files.  It would be good to add such a feature,
e.g. via magic-mode-alist and some creative regexp or match function.

> Could someone explain to me if making GCC the dependency of Emacs would
> be a good idea, from technical and freedom point of view?

You mean, invoke the compiler as part of some command?  No problem at
all (we actually do that already in a couple of commands).

Re: IDE

[Steinar Bang]

>>>>> Eli Zaretskii <eliz@gnu.org>:

> IMO, we should support both the "traditional" Emacs style, and the
> style users expect because other IDEs provide them.

As a long-time emacs user that uses IDEs these days, I don't need emacs
to look like an IDE to go back to using it for all programming.

I need the functionality I'm missing when I'm back in emacs:
 - run tests interactively and get easy-to-navigate feedback from
   failing tests
 - completion (I plan to check out company mode that I learned about
   from this thread)
 - navigation, most used:
  - go to definition of a symbol
  - go to usages of a symbol
  - go to derived symbols (in languages with inheritance)
  - go base symbols (ditto)
 - for refactoring, renaming support would cover >90% of what I use,
   other refactoring support I use, are:
    - copy/cut/paste of methods and other class members from a outline
      (maybe that's already supported these days?)
    - move members up and down in class hierarchies (I don't use this
      all that much, but it's nice to have when I need it)

That's all I can think about, offhand.

Re: IDE

[Eric Ludlam]

On 10/12/2015 07:05 AM, Oleh Krehel wrote:
> What's missing, in my opinion, is only faster and more precise parser
> (CEDET, GCC etc). For example, currently `semantic-fetch-tags' parses
> public/private/protected labels as tags, instead of applying these
> properties to actual tags. If that were so, it would be very easy to add
> a public/private/protected icon to each tag, just like MS VS does it.

The parser saves the buffer as close as it can - that allows it to be 
regenerated later.  If you use the 'semantic-format-*' functions, such 
as the uml version, it will identify the protection and use the right 
symbology.  If you are writing the code that calls the formatter, you 
need to specify the parent tag.

> Another example is the QT code: it's a popular LGPL C++ framework that's
> currently hard to setup for CEDET.
> For instance, `#include <QtGui/QPushButton>` is a plain file without an
> extension with only this code inside:
>
>      #include "qpushbutton.h"
>
> Since the extension isn't recognized, it's not parsed by CEDET.  And I
> have to write `#include "qpushbutton.h"` in my application instead of
> the more preferred `#include <QtGui/QPushButton>`, because that way I
> get tag completion.

You can solve this by adding the qt include directory to 
auto-mode-alist.  There is a workaround posted in emacswiki roughly like 
this:

(setq qt4-base-dir "/usr/include/qt4")
(setq qt4-gui-dir (concat qt4-base-dir "/QtGui"))
(semantic-add-system-include qt4-base-dir 'c++-mode)
(semantic-add-system-include qt4-gui-dir 'c++-mode)
(add-to-list 'auto-mode-alist (cons qt4-base-dir 'c++-mode))

There are a few extra steps for Qt preprocessor symbols and more, but 
the above lets you avoid the no extension problem.

> Could someone explain to me if making GCC the dependency of Emacs would
> be a good idea, from technical and freedom point of view?  Personally, I
> wouldn't care if Emacs executable would get inflated a bit more, if that
> meant access to true IDE features, which are only possible with a
> precise and fast parser.

There are folks using CEDET without gcc on their system, or at least, 
they've needed configuration help with alternate compilers.

Eric

RE: IDE

[Drew Adams]

[-- Attachment #1: Type: text/plain, Size: 1554 bytes --]

> For feature parity with Intellij IDEA and MS VS, we should be able
> to display the list of completions and documentation for the
> currently selected completion in two separate popups:
> https://i-msdn.sec.s-msft.com/dynimg/IC797655.jpeg
> https://www.jetbrains.com/img/webhelp/idea/constructors_docs_in_completion.png

FWIW, Icicles has long provided this, just by using *Completions*
for the completions and *Help* for the current-completion doc.

You can of course use separate frames for these.  There is also an
option that moves the *Completions* frame to the display edge when
you request candidate help or you act otherwise on a candidate.

Users control when/if candidate help is displayed (by key).
It is not just thrown in their face systematically.

Here is a screenshot of one possible layout:
http://www.emacswiki.org/emacs/Icicles_-_A_Propos_d'Apropos#CompFrameMovedLeft

FWIW, I also consider it a plus, not a minus, that the window-mgr
"decorations" are present.  Users can move the frames, iconify
them, shrink/enlarge them - do whatever they want with them,
including in some cases on the fly, by key.

They are ordinary Emacs frames, not just hard-coded popups
that exhibit only some developer's idea of the best behavior.

And you'll notice that the *Completions* mode line is made use
of; it provides brief help on the current candidate even when
*Help* is not shown.

I do agree that things like a menu-bar are best removed, by
default (as shown).  But users can always add them, if they
prefer.

[-- Attachment #2: drew-emacs-icicle-Comp-moved-left.png --]
[-- Type: image/png, Size: 38480 bytes --]

Re: IDE

[Dmitry Gutov]

On 10/11/2015 09:10 PM, Drew Adams wrote:

> FWIW, Icicles has long provided this, just by using *Completions*
> for the completions and *Help* for the current-completion doc.

Indeed, by default Company just pops a window for documentation. But 
many users have taken to using a third-party frontend that displays the 
documentation using pos-tip.

I don't want them to have to choose between that and displaying 
completion in a tooltip. They should be able to have both.

> Users control when/if candidate help is displayed (by key).
> It is not just thrown in their face systematically.

Many users like it to be displayed after a delay automatically.

> FWIW, I also consider it a plus, not a minus, that the window-mgr
> "decorations" are present.  Users can move the frames, iconify
> them, shrink/enlarge them - do whatever they want with them,
> including in some cases on the fly, by key.

Well, I don't consider it a plus.

Re: IDE

[Eli Zaretskii]

> Cc: adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 11 Oct 2015 13:10:10 +0300
> 
> On 10/10/2015 05:25 PM, Eli Zaretskii wrote:
> 
> > It's standard software engineering practice, why should you ask for
> > its justification?
> 
> I'm asking for details. Again, to further the discussion.

I don't understand what details you expect me to produce.

> "Let's unify features X, Y and Z" is not necessarily the standard 
> practice.

I didn't say anything even close.  I did suggest to _decide_ which
features we'd like to be in an Emacs IDE.

> I've also given a few general reasons why a "big design" might be 
> suboptimal

High-level design is not "big design".  Quite the contrary: it
generally leaves out all of the details, and documents only the main
design decisions.  Like which features will be included and which
definitely excluded, whether "fixed-window" appearance should be part
of it or not, etc.  You can call this "guidelines" if "design" sounds
too much.  Developers will benefit from such guidelines because they
will know what to expect and how to design their components.

This is all standard SE practice.

> the result is likely to turn out to be less flexible

A good tool strikes a fine balance between "flexible enough" and "too
flexible".  The latter more often than not means the tool is complex
and hard to set up.

> a significant number of users might prefer to use only some of the
> parts.

That's OK, and I see no problem with that.

> > The other IDEs use something similar to a tooltip, or a drop-down menu
> > with different fonts and colors.
> 
> You can already customize colors and fonts user for the Company popup. 
> But if you end up using fonts with different dimensions, of course, that 
> would result in jagged display.

Right.  Which is why tooltips or pop-up menus are better: they don't
suffer from these problems.

> From trying it out, I have the following complaints about x-show-tip
> capabilities:
> 
> - It's background rendering is inconsistent. As an example, the first 
> time I evaluate (tooltip-show "abc") in an Emacs session, the background 
> is yellow-ish. The next time, and after that, the background is black.

This could be the result of the recent changes by Ken, to make
tooltips less "voracious", to use Martin's term.  Trying in an older
Emacs will tell.

FWIW, I don't see anything like that here (on MS-Windows).

> - Is there a way to show several tooltips at once? To display different 
> elements of the completion UI side by side.

No, you need to lump them all together, or use menus.

> - If a tooltip is displayed, and I Alt-Tab to another program's window, 
> the tooltip remains on top. This is by far the most annoying one.

Martin told you how to solve this, I think.

Re: IDE

[Dmitry Gutov]

On 10/11/2015 06:23 PM, Eli Zaretskii wrote:

> High-level design is not "big design".  Quite the contrary: it
> generally leaves out all of the details, and documents only the main
> design decisions.  Like which features will be included and which
> definitely excluded, whether "fixed-window" appearance should be part
> of it or not, etc.  You can call this "guidelines" if "design" sounds
> too much.

Sounds good to me. Yes, "guidelines" is a more familiar term for that.

>> a significant number of users might prefer to use only some of the
>> parts.
>
> That's OK, and I see no problem with that.

So we need to make sure that they can.

>> - It's background rendering is inconsistent. As an example, the first
>> time I evaluate (tooltip-show "abc") in an Emacs session, the background
>> is yellow-ish. The next time, and after that, the background is black.
>
> This could be the result of the recent changes by Ken, to make
> tooltips less "voracious", to use Martin's term.  Trying in an older
> Emacs will tell.

No, that's a fairly old problem. And specific to GTK, IIUC.

>> - Is there a way to show several tooltips at once? To display different
>> elements of the completion UI side by side.
>
> No, you need to lump them all together, or use menus.

Neither option is good enough.

>> - If a tooltip is displayed, and I Alt-Tab to another program's window,
>> the tooltip remains on top. This is by far the most annoying one.
>
> Martin told you how to solve this, I think.

Yup.

Re: IDE

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > "Let's unify features X, Y and Z" is not necessarily the standard 
  > practice. It all depends on the exact features and how they are used.

I agree with this point.

Lots of subtle details of how Emacs commands behave in various
situations are convenient for some users.  Users have chosen their
usage based on those details.  A simple and clean scheme to
make things uniform can be very elegant, and yet make many users
unhappy.


-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: IDE

[John Wiegley]

>>>>> Richard Stallman <rms@gnu.org> writes:

> Lots of subtle details of how Emacs commands behave in various situations
> are convenient for some users. Users have chosen their usage based on those
> details. A simple and clean scheme to make things uniform can be very
> elegant, and yet make many users unhappy.

I do agree with this. Any uniform core must allow for the idiosyncratic
behaviors people have come to rely upon. If we find this cannot be done,
I'd give up on the attempt.

What I want to explore is whether things can be better. We have great
unification in some areas, and I've a sense there are more opportunities
if we can build the right layers of abstraction.

John

Re: IDE

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > What I want to explore is whether things can be better. We have great
  > unification in some areas, and I've a sense there are more opportunities
  > if we can build the right layers of abstraction.

I agree it is worth trying.  Sometimes it does manage to fly, and in those
cases it is a substantial improvement.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: IDE

[Dmitry Gutov]

On 10/10/2015 02:03 PM, Eli Zaretskii wrote:

>> The above more focused and, as such, more useful. "Comprehensive IDE
>> features" is not as useful.
>
> But it narrows the field too much, IMO.

I wonder.

 From what I've seen, Emacs facilities that try to do too much, end up 
over-specializing. That limits the number of users and, consequently, 
volunteers that would want to support it further. In my view, CEDET is 
an example of that.

Another example is ECB that controls how all windows are displayed, and 
trying to do that in the fashion that many IDE users are accustomed to.

My story (and many other users' too, I'm sure) is that I've tried using 
it for a while, and in the end decided that mimicking the IDE workflow 
in that respect is not very valuable. It hasn't seen any new development 
for years.

Re: IDE

[Eli Zaretskii]

> Cc: adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 10 Oct 2015 17:20:46 +0300
> 
> On 10/10/2015 02:03 PM, Eli Zaretskii wrote:
> 
> >> The above more focused and, as such, more useful. "Comprehensive IDE
> >> features" is not as useful.
> >
> > But it narrows the field too much, IMO.
> 
> I wonder.
> 
>  From what I've seen, Emacs facilities that try to do too much, end up 
> over-specializing. That limits the number of users and, consequently, 
> volunteers that would want to support it further. In my view, CEDET is 
> an example of that.

I didn't suggest to use CEDET as the starting point for this purpose.
I suggested to look at the popular IDEs out there, and use their
features as such a starting point.

Once again, we have prior art at our fingertips.  I believe the
features provided by the existing IDEs are a good approximation for
what people will generally expect from an IDE.  I think making a list
of the features we would like to see in the Emacs IDE, based on the
existing prior art, would be a good step forward.

But I repeat myself.  If you still don't agree, let's agree to
disagree on this.

> Another example is ECB that controls how all windows are displayed, and 
> trying to do that in the fashion that many IDE users are accustomed to.

Don't we lack features to support that?  Emacs generally doesn't let
you "dedicate" windows quite like IDEs do, even though we try.

Re: IDE

[David Engster]

Eli Zaretskii writes:
>> Cc: adatgyujto@gmail.com, emacs-devel@gnu.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Sat, 10 Oct 2015 17:20:46 +0300
>> Another example is ECB that controls how all windows are displayed, and 
>> trying to do that in the fashion that many IDE users are accustomed to.
>
> Don't we lack features to support that?

Yes. ECB uses lots of advices to achieve what it does. I think somebody
(Martin?) worked on a 'window group' feature which would make this
easier.

-David

Re: IDE

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,  adatgyujto@gmail.com,  emacs-devel@gnu.org
> Date: Sat, 10 Oct 2015 17:02:39 +0200
> 
> Eli Zaretskii writes:
> >> Cc: adatgyujto@gmail.com, emacs-devel@gnu.org
> >> From: Dmitry Gutov <dgutov@yandex.ru>
> >> Date: Sat, 10 Oct 2015 17:20:46 +0300
> >> Another example is ECB that controls how all windows are displayed, and 
> >> trying to do that in the fashion that many IDE users are accustomed to.
> >
> > Don't we lack features to support that?
> 
> Yes. ECB uses lots of advices to achieve what it does. I think somebody
> (Martin?) worked on a 'window group' feature which would make this
> easier.

That's what I thought.

So I guess we should ask Martin to please make that happen, as part of
work on Emacs IDE.

Re: IDE

[martin rudalics]

 >> Yes. ECB uses lots of advices to achieve what it does. I think somebody
 >> (Martin?) worked on a 'window group' feature which would make this
 >> easier.
 >
 > That's what I thought.
 >
 > So I guess we should ask Martin to please make that happen, as part of
 > work on Emacs IDE.

Several years ago I've been discussing this issue with Klaus Berndl.
The basic problem at that time was to get rid of the advices.  In the
sequel I installed the side windows code in window.el and later adapted
it to the then new buffer display functions.  There are two functions
‘display-buffer-in-side-window’, ‘display-buffer-in-major-side-window’
and a number of window parameters (‘window-side’, ‘window-slot’,
‘delete-window’ and ‘other-window’) which should suffice to emulate
practically everything I found in ECB.

I never use side windows so I can't tell whether they still work.  I
have written a frame-tabs.el package based on side windows but I don't
use that either.  At the time I installed the side windows functions I
also added a texinfo section but Stefan later asked me to remove it.
That info does not reflect later changes to the code so it might be
outdated.  You have to live with the doc-strings which should be fairly
accurate.

martin

Re: IDE

[John Wiegley]

>>>>> martin rudalics <rudalics@gmx.at> writes:

> I never use side windows so I can't tell whether they still work. I have
> written a frame-tabs.el package based on side windows but I don't use that
> either. At the time I installed the side windows functions I also added a
> texinfo section but Stefan later asked me to remove it. That info does not
> reflect later changes to the code so it might be outdated. You have to live
> with the doc-strings which should be fairly accurate.

We should define what we want from a "more IDE" Emacs. For example, I do not
want window-layout functionality. That's a presentation aspect of IDEs that's
entirely separate from what I want from them.

Right now we have a pretty nice infrastructure for things like indenting code.
That is, there are standard keybindings (TAB, C-M-\), a standard per-buffer
override variable (indent-line-function), a standard command
(indent-according-to-mode), and ways for packages like Yasnippet to override
the meaning of TAB without ruining functionality.

I think that what an "IDE is" has little to do with what it looks like. Emacs
being a better IDE, to me, means making IDE-like functionality a first-class
citizen, as we do with indenting. This would provide a core for fancy display
modules to build on top of.

I also don't think core Emacs should *provide* this functionality, since
that's impossible, given how many different languages and use cases there are.
It's more about giving developers a common API to base their work on, so that
we maximize collaboration between them.

Here is a list of functionality I think should be first-class, structurally
speaking (that is, an API will exist, but it may not do anything until a
contributor implements the functionality for their language). The ones with
a * mean areas where we're already succeeding to some degree:

  * indentation (see above)
    reformatting
  * syntax highlighting (font-lock)
    help at point
    documentation lookup (sadly, fewer projects use Info these days)
  ? completion
    refactoring
    semantic editing (for example, paredit)
  * compilation (M-x compile)
    live compilation (flymake/flycheck)
  * REPL (comint)
    running tests
  * debugging (GUD)
  * version control (VC)
    profiling
    code coverage
    app interaction

The reason I don't have a * next to completion, despite having so many things
capable of doing it (company, auto-complete, ido, hippie-expand, helm, ivy,
etc., etc.), is that there are too MANY ways to do it. This is where I think
proper IDE support could help.

If we have a single paradigm for "determining interesting details about the
buffer, and near point", with the ability to refine the query based on what is
need, optionally cache results, etc., then the competing libraries we have
today could share functionality. The present day `all-completions` function is
too spare to fit this bill, so it's less of an IDE feature in my book and more
just a Lisp library function. For example, I've written nearly the same
backend code for at least 4 different completion/lookup packages, and each
time I wonder how we could do better here. The differences are so minimal.

To reiterate: I think Emacs becomes more of an IDE when it provides the
backbone of an IDE, and not when it looks like one. We have some pieces of
that backbone already, which have avoided fragmentation in those areas, but we
need more. A standardized way to do tooltips, popups, visual completion
selection (or at least the structure), REPLs that interact with buffer
contents, etc., would give us a foundation to move to the next step.

John

Re: IDE

[David Kastrup]

"John Wiegley" <johnw@newartisans.com> writes:

> Right now we have a pretty nice infrastructure for things like
> indenting code.  That is, there are standard keybindings (TAB, C-M-\),
> a standard per-buffer override variable (indent-line-function), a
> standard command (indent-according-to-mode), and ways for packages
> like Yasnippet to override the meaning of TAB without ruining
> functionality.

Emacs sucks at multiple-language buffers out of the box.  They are
inherent for any form of literate programming, they are rather prevalent
for embedding languages in HTML, they are useful for things like Texinfo
with embedded examples, I'd need them for LilyPond (which integrates
Scheme code).  Many IDEs are single-language to start with, and if they
aren't, they are either generic or single-language per file.

There is mmm-mode but it's not part of ELPA (and probably not
copyright-assignable, at least that's my guess of why it might be out).
And I strongly suspect that some stronger core features inside of Emacs
would be desirable, like keeping indentation and font-locking engines
and syntax constrained to regions identified with overlays or text
properties.

-- 
David Kastrup

Re: IDE

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Date: Sat, 10 Oct 2015 20:46:13 +0200
> 
> Emacs sucks at multiple-language buffers out of the box.  They are
> inherent for any form of literate programming, they are rather prevalent
> for embedding languages in HTML, they are useful for things like Texinfo
> with embedded examples, I'd need them for LilyPond (which integrates
> Scheme code).  Many IDEs are single-language to start with, and if they
> aren't, they are either generic or single-language per file.
> 
> There is mmm-mode but it's not part of ELPA (and probably not
> copyright-assignable, at least that's my guess of why it might be out).
> And I strongly suspect that some stronger core features inside of Emacs
> would be desirable, like keeping indentation and font-locking engines
> and syntax constrained to regions identified with overlays or text
> properties.

I indeed think that we should have infrastructure to turn on a major
mode in a region of a buffer.

I'm not sure we should use text properties or overlays for that,
though.  The region could be part of the command that turns on the
mode with region limits stored in markers.

Re: IDE

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Date: Sat, 10 Oct 2015 20:46:13 +0200
>> 
>> Emacs sucks at multiple-language buffers out of the box.  They are
>> inherent for any form of literate programming, they are rather prevalent
>> for embedding languages in HTML, they are useful for things like Texinfo
>> with embedded examples, I'd need them for LilyPond (which integrates
>> Scheme code).  Many IDEs are single-language to start with, and if they
>> aren't, they are either generic or single-language per file.
>> 
>> There is mmm-mode but it's not part of ELPA (and probably not
>> copyright-assignable, at least that's my guess of why it might be out).
>> And I strongly suspect that some stronger core features inside of Emacs
>> would be desirable, like keeping indentation and font-locking engines
>> and syntax constrained to regions identified with overlays or text
>> properties.
>
> I indeed think that we should have infrastructure to turn on a major
> mode in a region of a buffer.
>
> I'm not sure we should use text properties or overlays for that,
> though.  The region could be part of the command that turns on the
> mode with region limits stored in markers.

A typical LilyPond score file switches into Scheme thousands of times
(most of the time just for a single scalar Scheme constant where an
actual mode switch would not necessarily be required, but also for all
user-defined functions and any non-trivial non-music expression).

-- 
David Kastrup

Re: IDE

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 10 Oct 2015 21:11:57 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I indeed think that we should have infrastructure to turn on a major
> > mode in a region of a buffer.
> >
> > I'm not sure we should use text properties or overlays for that,
> > though.  The region could be part of the command that turns on the
> > mode with region limits stored in markers.
> 
> A typical LilyPond score file switches into Scheme thousands of times
> (most of the time just for a single scalar Scheme constant where an
> actual mode switch would not necessarily be required, but also for all
> user-defined functions and any non-trivial non-music expression).

Not sure what you are saying.  Is it that a region with two ends is
not enough, and we need a list of regions?  If so, I'm okay with that,
and I don't see any serious obstacles to implement that.

Re: IDE

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Cc: emacs-devel@gnu.org
>> Date: Sat, 10 Oct 2015 21:11:57 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > I indeed think that we should have infrastructure to turn on a major
>> > mode in a region of a buffer.
>> >
>> > I'm not sure we should use text properties or overlays for that,
>> > though.  The region could be part of the command that turns on the
>> > mode with region limits stored in markers.
>> 
>> A typical LilyPond score file switches into Scheme thousands of times
>> (most of the time just for a single scalar Scheme constant where an
>> actual mode switch would not necessarily be required, but also for all
>> user-defined functions and any non-trivial non-music expression).
>
> Not sure what you are saying.  Is it that a region with two ends is
> not enough, and we need a list of regions?  If so, I'm okay with that,
> and I don't see any serious obstacles to implement that.

It's rather a host of regions determined by syntax.

This can look like:

compressMMRests =
#(define-music-function (music) (ly:music?)
  (_i "Remove the empty bars created by multi-measure rests,
leaving just the first bar containing the MM rest itself.")
   (music-map
    (lambda (m)
      (if (eq? 'MultiMeasureRestMusic (ly:music-property m 'name))
          #{ \once \set Score.skipBars = ##t #m #}
          #{ #m #} ))
    music))

The first # switches into Scheme mode until the end, but then every #{
escape backs into LilyPond mode, and every # inside until #} goes back
to Scheme in order to evaluate expressions #t, m, and m in Scheme again.

Basically, inside of LilyPond, # escapes into Scheme for a single sexp,
and inside of Scheme, #{ escapes into LilyPond until the matching #} is
encountered, of course with #sexp again escaping into Scheme.  Either
can easily form multi-line expressions (or not) but definitely wants to
change the mode of font-locking.

-- 
David Kastrup

RE: IDE

[Drew Adams]

> Not sure what you are saying.  Is it that a region with two ends is
> not enough, and we need a list of regions?  If so, I'm okay with
> that, and I don't see any serious obstacles to implement that.

FWIW, library `zones.el' provides that, along with various
commands for defining and using such multiple-zone "regions".

http://www.emacswiki.org/emacs/Zones

You can sort, intersect, or unite zones; narrow the buffer to
a zone or select one as the region, (un)highlight zones, make
a list of zones persistent (re-created with a bookmark), or
use incremental search across it (overlapping zones in the list
are united for this).  You can name lists of zones and switch
among them for different multi-zone operations.  You can
associate arbitrary information with a zone, which is otherwise
just a pair of buffer positions (e.g. markers).

This does _not_ provide the feature of having zones that are
in different major modes.  I agree that that particular
feature is sorely missing.  There have been various attempts
at providing it, but I'm not aware of any that has been really
satisfactory in a general way.

Re: IDE

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> 
>> Emacs sucks at multiple-language buffers out of the box.

> I indeed think that we should have infrastructure to turn on a major
> mode in a region of a buffer.

I strongly agree as well.  TextMate does an awesome job at this.

> I'm not sure we should use text properties or overlays for that,
> though.  The region could be part of the command that turns on the
> mode with region limits stored in markers.

I'm not sure what the right technical answer is either, but I'll add this to
the list.

John

Re: IDE

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I indeed think that we should have infrastructure to turn on a major
  > mode in a region of a buffer.

I have tried to encourage progress in this direction for 15 years.

  > I'm not sure we should use text properties or overlays for that,
  > though.  The region could be part of the command that turns on the
  > mode with region limits stored in markers.

The two are not necessarily alternatives.  They might be two
different levels of functionality.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: IDE

[Eli Zaretskii]

> From: "John Wiegley" <johnw@newartisans.com>
> Date: Sat, 10 Oct 2015 11:31:13 -0700
> 
> To reiterate: I think Emacs becomes more of an IDE when it provides the
> backbone of an IDE, and not when it looks like one.

I don't think this will fly.  In effect, you are telling Emacs users
they will have to code their own IDE by using the infrastructure
(a.k.a. "backbone") that we provide.  If I were that user, I'd
immediately turn away and look elsewhere.

An analogy to that would be if, instead of Gnus, we'd provide an
infrastructure for a news- and mail-reader.

To me, an IDE is not a set of functionalities.  It's a coherent
application that provides an IDE-like look-and-feel, and all the
related functions already integrated and ready for me to be used.
That includes window-layout, btw, because configuring Emacs windows
for IDE-like behavior is an exceedingly complex task, one that's
impossible without good command of ELisp.  Not something I'd offer a
user whose only wish is to build a project in some language we
support.

We cannot provide a toolbox and tell users make your own IDE from
these tools.  Integrating those tools together is a non-trivial task a
user should not be expected to perform.

Re: IDE

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "John Wiegley" <johnw@newartisans.com>
>> Date: Sat, 10 Oct 2015 11:31:13 -0700
>> 
>> To reiterate: I think Emacs becomes more of an IDE when it provides
>> the backbone of an IDE, and not when it looks like one.
>
> I don't think this will fly.  In effect, you are telling Emacs users
> they will have to code their own IDE by using the infrastructure
> (a.k.a. "backbone") that we provide.

I don't see that.  He is telling Emacs developers they'll be able to
rely on common infrastructure for creating an IDE catered to some
particular language/system.

If we are talking about smart completion and debugging support for
languages compiled with GCC, this could mean a lot of things working out
of the box (source-level debugging with expression
evaluation/inspection, syntactic indentation, jumping by statements or
other logical units, semantic information and completion etc etc) before
you even know the language we are talking about.

> To me, an IDE is not a set of functionalities.  It's a coherent
> application that provides an IDE-like look-and-feel, and all the
> related functions already integrated and ready for me to be used.

Without a uniform framework to create such IDEs, no uniformly usable IDE
for different languages will fall out.

> We cannot provide a toolbox and tell users make your own IDE from
> these tools.

It wouldn't be the task of the users to create those.

> Integrating those tools together is a non-trivial task a user should
> not be expected to perform.

Forcing every developer to redo it from scratch will lead to very spotty
and inconsistent support of languages.

-- 
David Kastrup

Re: IDE

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Cc: John Wiegley <johnw@newartisans.com>,  emacs-devel@gnu.org
> Date: Sat, 10 Oct 2015 21:07:50 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: "John Wiegley" <johnw@newartisans.com>
> >> Date: Sat, 10 Oct 2015 11:31:13 -0700
> >> 
> >> To reiterate: I think Emacs becomes more of an IDE when it provides
> >> the backbone of an IDE, and not when it looks like one.
> >
> > I don't think this will fly.  In effect, you are telling Emacs users
> > they will have to code their own IDE by using the infrastructure
> > (a.k.a. "backbone") that we provide.
> 
> I don't see that.  He is telling Emacs developers they'll be able to
> rely on common infrastructure for creating an IDE catered to some
> particular language/system.

OK, but IMO we should aim for more than that.  What John wrote sounded
like providing the infrastructure _is_ the goal.  Apologies if I
misunderstood.

Re: IDE

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

>> >> To reiterate: I think Emacs becomes more of an IDE when it provides
>> >> the backbone of an IDE, and not when it looks like one.
>> >
>> > I don't think this will fly.  In effect, you are telling Emacs users
>> > they will have to code their own IDE by using the infrastructure
>> > (a.k.a. "backbone") that we provide.
>> 
>> I don't see that.  He is telling Emacs developers they'll be able to
>> rely on common infrastructure for creating an IDE catered to some
>> particular language/system.

> OK, but IMO we should aim for more than that.  What John wrote sounded
> like providing the infrastructure _is_ the goal.  Apologies if I
> misunderstood.

Ah, I should have been clearer, Eli. What I wrote was addressed to Emacs
developers. I think we have succeeded *towards* having an IDE when we have the
right backbone in place.

Next comes the package contributors, who will build out functionality on top
of this backbone. If we do it right, it will mostly mean porting the backends
of existing libraries that already do a superb job, like company and helm.

Finally comes knitting it all together into a good user experience. This takes
some art, but we have a few examples where this has been done right already.

At the end of that road, we'd have an excellent IDE environment that a
non-Emacs user could intuitively appreciate, the way so many do with Org or
Spacemacs (each of which has won new Emacs users on its own, before those
users understood what Emacs was really about).

John

Re: IDE

[Dmitry Gutov]

On 10/10/2015 09:58 PM, Eli Zaretskii wrote:

> To me, an IDE is not a set of functionalities.  It's a coherent
> application that provides an IDE-like look-and-feel, and all the
> related functions already integrated and ready for me to be used.
> That includes window-layout, btw, because configuring Emacs windows
> for IDE-like behavior is an exceedingly complex task, one that's
> impossible without good command of ELisp.  Not something I'd offer a
> user whose only wish is to build a project in some language we
> support.

While I agree that working with windows in Emacs is often more trouble 
than it should be, I don't think that offering a fixed layout like ECB 
is the answer: it doesn't anticipate the needs of commands like vc-dir, 
and it doesn't solve all problems anyway.

Rather than that, we should provide more consistent guidelines for 
window behavior, like whether a command should use a new window, reuse 
an existing one, etc, and try harder not to destroy a layout the user 
created. Maybe include a more accessible alternative to winner-mode 
(which is a lifesaver, but is more of a kludge than a user-friendly 
solution).

Re: IDE

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1648 bytes --]

On 10/10/2015 02:23 PM, Dmitry Gutov wrote:
> On 10/10/2015 09:58 PM, Eli Zaretskii wrote:
> 
>> To me, an IDE is not a set of functionalities.  It's a coherent
>> application that provides an IDE-like look-and-feel, and all the
>> related functions already integrated and ready for me to be used.
>> That includes window-layout, btw, because configuring Emacs windows
>> for IDE-like behavior is an exceedingly complex task, one that's
>> impossible without good command of ELisp.  Not something I'd offer a
>> user whose only wish is to build a project in some language we
>> support.
> 
> While I agree that working with windows in Emacs is often more trouble
> than it should be, I don't think that offering a fixed layout like ECB
> is the answer: it doesn't anticipate the needs of commands like vc-dir,
> and it doesn't solve all problems anyway.
> 
> Rather than that, we should provide more consistent guidelines for
> window behavior, like whether a command should use a new window, reuse
> an existing one, etc, and try harder not to destroy a layout the user
> created. Maybe include a more accessible alternative to winner-mode
> (which is a lifesaver, but is more of a kludge than a user-friendly
> solution).

A pet peeve of mine is pop-to-buffer. I've been tempted on a few
occasions to make pop-to-buffer equivalent to display-buffer and use
buffer-display customization for all the differences.

I'm also a heavy winner user, but I'd appreciate the ability to "dock"
certain windows and buffers within them. (I haven't found the dedicated
window functionality useful, but _somebody_ must, right?)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: IDE

[Andreas Röhler]

Am 10.10.2015 um 23:23 schrieb Dmitry Gutov:
> On 10/10/2015 09:58 PM, Eli Zaretskii wrote:
>
>> To me, an IDE is not a set of functionalities.  It's a coherent
>> application that provides an IDE-like look-and-feel, and all the
>> related functions already integrated and ready for me to be used.
>> That includes window-layout, btw, because configuring Emacs windows
>> for IDE-like behavior is an exceedingly complex task, one that's
>> impossible without good command of ELisp.  Not something I'd offer a
>> user whose only wish is to build a project in some language we
>> support.
>
> While I agree that working with windows in Emacs is often more trouble 
> than it should be, I don't think that offering a fixed layout like ECB 
> is the answer: it doesn't anticipate the needs of commands like 
> vc-dir, and it doesn't solve all problems anyway.
>
> Rather than that, we should provide more consistent guidelines for 
> window behavior, like whether a command should use a new window, reuse 
> an existing one, etc, and try harder not to destroy a layout the user 
> created. Maybe include a more accessible alternative to winner-mode 
> (which is a lifesaver, but is more of a kludge than a user-friendly 
> solution).
>
>

AFAIU Emacs in past time changed too fast, introduced new features or 
new function without an appropriate discussion, broke backward code too 
fast.

Got the impression developers spend a lot of time with fruits of their 
colleages, which just weren't in time to sell it yet.

That's rather an impression than established knowledge.

In case its true, a policy focused on branches while feature-freeze 
being the normal state might help.

Re: IDE

[Steinar Bang]

>>>>> Eli Zaretskii <eliz@gnu.org>:

> To me, an IDE is not a set of functionalities.  It's a coherent
> application that provides an IDE-like look-and-feel, and all the
> related functions already integrated and ready for me to be used.
> That includes window-layout, btw, because configuring Emacs windows
> for IDE-like behavior is an exceedingly complex task, one that's
> impossible without good command of ELisp.

Ah, there we differ.  I don't care about the layout and would feel
constrained in that layout.

But I do want the functionalities.

Re: IDE

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Mon, 12 Oct 2015 20:12:22 +0200
> 
> >>>>> Eli Zaretskii <eliz@gnu.org>:
> 
> > To me, an IDE is not a set of functionalities.  It's a coherent
> > application that provides an IDE-like look-and-feel, and all the
> > related functions already integrated and ready for me to be used.
> > That includes window-layout, btw, because configuring Emacs windows
> > for IDE-like behavior is an exceedingly complex task, one that's
> > impossible without good command of ELisp.
> 
> Ah, there we differ.  I don't care about the layout and would feel
> constrained in that layout.
> 
> But I do want the functionalities.

The interesting question is what the youngsters want (or expect) these
days.  You and me are sold long ago.

Re: IDE

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Steinar Bang <sb@dod.no>
>> Date: Mon, 12 Oct 2015 20:12:22 +0200
>> 
>> >>>>> Eli Zaretskii <eliz@gnu.org>:
>> 
>> > To me, an IDE is not a set of functionalities.  It's a coherent
>> > application that provides an IDE-like look-and-feel, and all the
>> > related functions already integrated and ready for me to be used.
>> > That includes window-layout, btw, because configuring Emacs windows
>> > for IDE-like behavior is an exceedingly complex task, one that's
>> > impossible without good command of ELisp.
>> 
>> Ah, there we differ.  I don't care about the layout and would feel
>> constrained in that layout.
>> 
>> But I do want the functionalities.
>
> The interesting question is what the youngsters want (or expect) these
> days.  You and me are sold long ago.

So what?  I'm more interested in things that make Emacs better than
things that made Visual C++ better.  It's one thing to put lipstick on a
pig, but an octopus might not even have a good place for it.

-- 
David Kastrup

Re: IDE

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > So what?  I'm more interested in things that make Emacs better than
  > things that made Visual C++ better.  It's one thing to put lipstick on a
  > pig, but an octopus might not even have a good place for it.

This is true as a general statement, but in general I hope we can
find ways to integrate into Emacs the useful or appealing features
of other IDEs.  Let's at least try to find a way to make them fit,
before we dismiss the idea.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: IDE

[Steinar Bang]

>>>>> Richard Stallman <rms@gnu.org>:

> This is true as a general statement, but in general I hope we can find
> ways to integrate into Emacs the useful or appealing features of other
> IDEs.  Let's at least try to find a way to make them fit, before we
> dismiss the idea.

The IDE features I miss most in emacs, is:
 - Auto complete
 - Navigation (definition of symbol, usages of symbol)
 - Renaming support
 - Outline cut/copy/paste

All of these features can be implemented in emacs without needing to
support (or enforce) an IDE-like layout.

What I don't like in other IDEs is the need to use a mouse to switch between
buffers and change focus between IDE windows, and this is a place where
emacs shines: I never _need_ to use the mouse

My languages are currently Java and Python (the latter I use emacs for
already.  The former I use emacs for formatting cleanup, and commit, and
large scale text subistitution 

Re: IDE

[Eric Ludlam]

On 10/10/2015 02:31 PM, John Wiegley wrote:
> If we have a single paradigm for "determining interesting details about the
> buffer, and near point", with the ability to refine the query based on what is
> need, optionally cache results, etc., then the competing libraries we have
> today could share functionality. The present day `all-completions` function is
> too spare to fit this bill, so it's less of an IDE feature in my book and more
> just a Lisp library function.

In CEDET, the function / command `semantic-analyze-current-context' 
provides an output that has lots of details about the buffer near point. 
  Not just what the cursor is on, but if it is a chain of symbols such 
as dereferencing struct pointers, and in many cases, it figures out the 
data type of the symbol the cursor is on.  It also handles in-buffer 
caching, etc and plenty of performance tweaking is available.

This is independent of the functions that perform completions.

Eric

Re: IDE

[John Wiegley]

>>>>> Eric Ludlam <eric@siege-engine.com> writes:

> In CEDET, the function / command `semantic-analyze-current-context' provides
> an output that has lots of details about the buffer near point. Not just
> what the cursor is on, but if it is a chain of symbols such as dereferencing
> struct pointers, and in many cases, it figures out the data type of the
> symbol the cursor is on. It also handles in-buffer caching, etc and plenty
> of performance tweaking is available.

My preference is for each core feature to have an extremely simple and light
interface (taking indentation as an ideal), so that it can also be used for
non-IDE tasks we haven't imagined yet. From what I know about CEDET to date,
it is much more complex than this objective.

When I squint, I see ctags, imenu, pcomplete, helm, company, hippie-expand,
flycheck, and more, all starting to look somewhat alike: They each act upon
information relevant to the buffer in some way. Where they differ is in how
they derive this information, and how the user interacts with it. Can we
provide a common, low-level interface for this style of functionality? A
common API against which we can implement both data-gathering backends, and
display front-ends? And with an emphasis on efficiency and low-latency!

We need to harness the power of multiplication, so that every new backend
makes every frontend automatically more powerful, and vice versa. This will
also help us better leverage our existing base of contributors.

John

Re: IDE

[Eric Ludlam]

On 10/10/2015 07:20 PM, John Wiegley wrote:
>>>>>> Eric Ludlam <eric@siege-engine.com> writes:
>
>> In CEDET, the function / command `semantic-analyze-current-context' provides
>> an output that has lots of details about the buffer near point. Not just
>> what the cursor is on, but if it is a chain of symbols such as dereferencing
>> struct pointers, and in many cases, it figures out the data type of the
>> symbol the cursor is on. It also handles in-buffer caching, etc and plenty
>> of performance tweaking is available.
>
> My preference is for each core feature to have an extremely simple and light
> interface (taking indentation as an ideal), so that it can also be used for
> non-IDE tasks we haven't imagined yet. From what I know about CEDET to date,
> it is much more complex than this objective.

That is because the task is complex.

> When I squint, I see ctags, imenu, pcomplete, helm, company, hippie-expand,
> flycheck, and more, all starting to look somewhat alike: They each act upon
> information relevant to the buffer in some way. Where they differ is in how
> they derive this information, and how the user interacts with it. Can we
> provide a common, low-level interface for this style of functionality? A
> common API against which we can implement both data-gathering backends, and
> display front-ends? And with an emphasis on efficiency and low-latency!

The primary difference between your list and CEDET is that those other 
tools focus on picking up a current symbol, or perhaps a substring, and 
it is up to the next layer to figure out more about it.  I agree that 
that could be simplified across the variosu tools, but AFAIK the 
'thing-at-pt' stuff is that common interface.  IDEs know more about the 
buffer than just some symbol and the major-mode of the current buffer.

Things like dabbrev work pretty well finding similar strings that often 
have the feel of being 'smart', but that only works if you've typed it 
in before.

If you want a stronger set of smart behaviours at point, you will need 
to raise your standard to include more derived data.

> We need to harness the power of multiplication, so that every new backend
> makes every frontend automatically more powerful, and vice versa. This will
> also help us better leverage our existing base of contributors.

This I agree with.

Eric

Re: IDE

[John Wiegley]

>>>>> Eric Ludlam <eric@siege-engine.com> writes:

>> From what I know about CEDET to date, it is much more complex than this
>> objective.

> That is because the task is complex.

The task is, but the solution needn't be, if structured well.

I'm not willing right now to say "CEDET is the answer", mostly because I've
had bad experiences trying to use it over the years. I want to return to
basics, define what is wanted, and then ask what layers are needed to get
there.

> The primary difference between your list and CEDET is that those other tools
> focus on picking up a current symbol, or perhaps a substring, and it is up
> to the next layer to figure out more about it. I agree that that could be
> simplified across the variosu tools, but AFAIK the 'thing-at-pt' stuff is
> that common interface. IDEs know more about the buffer than just some symbol
> and the major-mode of the current buffer.

thing-at-pt is likely a piece of the puzzle, but a small piece.

> If you want a stronger set of smart behaviours at point, you will need to
> raise your standard to include more derived data.

Correct.

I'll compile a more in-depth proposal for this idea, within the context of a
larger plan for IDE functionality -- if it falls to me to do so.

John

Re: IDE

[raman]

"John Wiegley" <johnw@newartisans.com> writes:

Well said. Let's focus on functionality -- and the rest -- including a
multiplicity of visual look-and-feel setups will follow along with
everything else.>>>>>> martin rudalics <rudalics@gmx.at> writes:
>
>> I never use side windows so I can't tell whether they still work. I have
>> written a frame-tabs.el package based on side windows but I don't use that
>> either. At the time I installed the side windows functions I also added a
>> texinfo section but Stefan later asked me to remove it. That info does not
>> reflect later changes to the code so it might be outdated. You have to live
>> with the doc-strings which should be fairly accurate.
>
> We should define what we want from a "more IDE" Emacs. For example, I do not
> want window-layout functionality. That's a presentation aspect of IDEs that's
> entirely separate from what I want from them.
>
> Right now we have a pretty nice infrastructure for things like indenting code.
> That is, there are standard keybindings (TAB, C-M-\), a standard per-buffer
> override variable (indent-line-function), a standard command
> (indent-according-to-mode), and ways for packages like Yasnippet to override
> the meaning of TAB without ruining functionality.
>
> I think that what an "IDE is" has little to do with what it looks like. Emacs
> being a better IDE, to me, means making IDE-like functionality a first-class
> citizen, as we do with indenting. This would provide a core for fancy display
> modules to build on top of.
>
> I also don't think core Emacs should *provide* this functionality, since
> that's impossible, given how many different languages and use cases there are.
> It's more about giving developers a common API to base their work on, so that
> we maximize collaboration between them.
>
> Here is a list of functionality I think should be first-class, structurally
> speaking (that is, an API will exist, but it may not do anything until a
> contributor implements the functionality for their language). The ones with
> a * mean areas where we're already succeeding to some degree:
>
>   * indentation (see above)
>     reformatting
>   * syntax highlighting (font-lock)
>     help at point
>     documentation lookup (sadly, fewer projects use Info these days)
>   ? completion
>     refactoring
>     semantic editing (for example, paredit)
>   * compilation (M-x compile)
>     live compilation (flymake/flycheck)
>   * REPL (comint)
>     running tests
>   * debugging (GUD)
>   * version control (VC)
>     profiling
>     code coverage
>     app interaction
>
> The reason I don't have a * next to completion, despite having so many things
> capable of doing it (company, auto-complete, ido, hippie-expand, helm, ivy,
> etc., etc.), is that there are too MANY ways to do it. This is where I think
> proper IDE support could help.
>
> If we have a single paradigm for "determining interesting details about the
> buffer, and near point", with the ability to refine the query based on what is
> need, optionally cache results, etc., then the competing libraries we have
> today could share functionality. The present day `all-completions` function is
> too spare to fit this bill, so it's less of an IDE feature in my book and more
> just a Lisp library function. For example, I've written nearly the same
> backend code for at least 4 different completion/lookup packages, and each
> time I wonder how we could do better here. The differences are so minimal.
>
> To reiterate: I think Emacs becomes more of an IDE when it provides the
> backbone of an IDE, and not when it looks like one. We have some pieces of
> that backbone already, which have avoided fragmentation in those areas, but we
> need more. A standardized way to do tooltips, popups, visual completion
> selection (or at least the structure), REPLs that interact with buffer
> contents, etc., would give us a foundation to move to the next step.
>
> John
>

-- 

Re: IDE

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Yes. ECB uses lots of advices to achieve what it does. I think somebody
  > (Martin?) worked on a 'window group' feature which would make this
  > easier.

I used to have a rule of not installing anything in Emacs that put
advice on some other part of Emacs.  When module A wants module B to
do something special, rather than having A advise B, it's better for
mainenance to make B run a hook and have A set up that hook.

The hook is cleaner because, when you look at the code of B, you see
it runs the hook, and you you know to check what's in the hook.  If
advice is used, there is nothing in B to warn you that the function
does something you can't see.

Thus, whenever I installed a package which used advice, I added hooks
and changed it to use them instead.

Advice is for users to use, not for Emacs to use.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: IDE

[Przemysław Wojnowski]

W dniu 10.10.2015 o 11:40, Eli Zaretskii pisze:
> Come to think of that, even coming up with a list of such features
> would be real progress, as I don't think we have that, or ever had.

I can "step forward and work" conceptually here. :-)

One can start with a question:
What is purpose of an IDE (except for making many for a company behind it)?
The simple answer is: to help programmers do their work productively.

The answer tells two things:
1. Who is the target: programmers not average computer user
It could be possibly constrained to e.g. programmers working on projects, not 
single file programs.
2. How to optimize product backlog: maximization of productivity.

Both these things help to select features.
For example project-oriented programmers _have to have_ project support, whereas 
"single file" programmers may not care about it.
Features/bugs that impact their productivity have higher priority.

Let's say that we would target "project programmers", then we could ask:
1. What features would make such programmers productive?
2. What other environments offer that attract them?

IMHO the answer is: it depends on target language(s).

I cannot say what makes a Ruby programmer productive, but I can list a few 
things important for Java programmers. Others, having experience with other 
languages, may list _must have_ features for their environments.

For (project-oriented/enterprise) Java the features are:
1. Project support
IDE has to know the boundaries of a project - where are sources, tests, libs, 
resources/assets (additional files used in an app in runtime), docs - and what 
are relations between them. Also it has to know how to work with project's build 
tool (be it Maven, Gradle, etc.).
A programmer joining a project (99% of cases) should be able to open/import it 
and start work. Every Java IDE have that.

2. Code completion
 From whole project, used libraries, and resources

3. Jumping around the project code and resources.
Jumping to around the project code and used libraries. Another thing is jumping 
to/from resources (for example aspects can be defined in an XML file and IDE 
could allow to jump to matching classes).

4. Finding usages of fields, methods, types.
Helps to wrap head around project.

5. Automatic compilation and showing compilation erros/warnings.
Tightens development feedback loop.

6. Running the tests (current, selected, all)
Tighten development feedback loop.

7. Debugging
A must, especially for legacy code, so, basically 99% of projects. :-)

8. Running the app

9. Basic refactoring.
I do refactor _a lot_ and in my experience the most important refactoring is 
Extract Method. Others, while helpful, are less often used, compared to the EM.
One variation of Extract Method is "EM with finding duplicates", which works 
like this:
- ask user for a method name,
- find all occurrences of selected code in the buffer
- ask user if she wants to replace all occurrences with the call to the new method.
This is fantastic feature that Intellij has and helps to remove a lot of 
duplicated code.

10. Showing documentation (tooltip, etc.)
Maybe EWW could be used to show javadoc in a small window?


Of course, part of these things (e.g. build tool support) are language dependent 
and should not be in Emacs core.

Also some of the features are already scattered in different elpa libraries, but 
lack integration with others.

IMHO people having experience in other languages could list features that make 
them productive and maybe we will be able to find a set of core, absolutely must 
have, IDE features. :-)

IMHO the list of core IDE features will be fairly small and doable.

Sorry for long email.

Cheers,
Przemysław

Re: IDE

[Jean-Christophe Helary]

> On Oct 11, 2015, at 22:18, Przemysław Wojnowski <esperanto@cumego.com> wrote:
> 
> W dniu 10.10.2015 o 11:40, Eli Zaretskii pisze:
>> Come to think of that, even coming up with a list of such features
>> would be real progress, as I don't think we have that, or ever had.
> 
> I can "step forward and work" conceptually here. :-)
> 
> One can start with a question:
> What is purpose of an IDE (except for making many for a company behind it)?
> The simple answer is: to help programmers do their work productively.

Productivity for code *writers* could be defined by how many correct code strings they output in a given amount of time.

Anything that

1) increase the volume
2) increase the correctness

of the strings increases productivity.

Jean-Christophe Helary 

Re: IDE

[Przemysław Wojnowski]

W dniu 11.10.2015 o 15:26, Jean-Christophe Helary pisze:
> Productivity for code *writers* could be defined by how many correct code strings they output in a given amount of time.
>
> Anything that
>
> 1) increase the volume
> 2) increase the correctness
>
> of the strings increases productivity.

I like jokes, but adding your "list of features" would be great too. ;-)

Re: IDE

[Jean-Christophe Helary]

> On Oct 11, 2015, at 22:34, Przemysław Wojnowski <esperanto@cumego.com> wrote:
> 
> W dniu 11.10.2015 o 15:26, Jean-Christophe Helary pisze:
>> Productivity for code *writers* could be defined by how many correct code strings they output in a given amount of time.
>> 
>> Anything that
>> 
>> 1) increase the volume
>> 2) increase the correctness
>> 
>> of the strings increases productivity.
> 
> I like jokes, but adding your "list of features" would be great too. ;-)

It was not intended as a joke :) But if we start by defining what we mean by "productivity" in the context of code writing, then it's easier to see what can be on the feature list and what does not need to be.

Also, the above definition applies equally to any (non literary) writer. So instead of thinking IDE, you could think IWE and get perspectives that you would not have it you just focused on Emacs vs other IDEs.

Jean-Christophe 

Re: IDE

[Przemysław Wojnowski]

W dniu 11.10.2015 o 15:41, Jean-Christophe Helary pisze:
[...]
> It was not intended as a joke :) But if we start by defining what we mean by "productivity" in the context of code writing, then it's easier to see what can be on the feature list and what does not need to be.
Note one thing: I have written "to help programmers do their work productively".
It doesn't have to be "writing code". It can be analyzing code, learning it, 
refactoring, deleting code, running app/tests/debugging, etc.

Actually in many legacy systems writing code is minor activity.

In different projects productivity may be defined differently, but, where money 
counts, usually it's defined as "deliver software on time". Nobody care how many 
lines is that - the less, the better. If its oneliner and does the job then do 
it. :-)

Re: IDE

[Jean-Christophe Helary]

> On Oct 11, 2015, at 23:05, Przemysław Wojnowski <esperanto@cumego.com> wrote:
> 
> W dniu 11.10.2015 o 15:41, Jean-Christophe Helary pisze:
> [...]
>> It was not intended as a joke :) But if we start by defining what we mean by "productivity" in the context of code writing, then it's easier to see what can be on the feature list and what does not need to be.
> Note one thing: I have written "to help programmers do their work productively".
> It doesn't have to be "writing code". It can be analyzing code, learning it, refactoring, deleting code, running app/tests/debugging, etc.

That's what I'd put under "correctness".

> Actually in many legacy systems writing code is minor activity.

In which case your focus is on "correctness".

> In different projects productivity may be defined differently, but, where money counts, usually it's defined as "deliver software on time". Nobody care how many lines is that - the less, the better. If its oneliner and does the job then do it. :-)

Volume itself is a function of the expressiveness of the language. But even in a very expressive language, writing the most and most correct code in the smallest amount of time can be used as a definition for productivity and to identify basic features that you'd want to achieve that.

But the definition of productivity itself does not matter as long as you have one that allows you to specify what features are required and what features are not.

Jean-Christophe

Re: IDE

[Óscar Fuentes]

Jean-Christophe Helary <jean.christophe.helary@gmail.com> writes:

> Productivity for code *writers* could be defined by how many correct
> code strings they output in a given amount of time.
>
> Anything that
>
> 1) increase the volume
> 2) increase the correctness
>
> of the strings increases productivity.

I reckon you are a Java code writer.

<ducks>

Re: IDE

[Jean-Christophe Helary]

> On Oct 11, 2015, at 22:59, Óscar Fuentes <ofv@wanadoo.es> wrote:
> 
> Jean-Christophe Helary <jean.christophe.helary@gmail.com> writes:
> 
>> Productivity for code *writers* could be defined by how many correct
>> code strings they output in a given amount of time.
>> 
>> Anything that
>> 
>> 1) increase the volume
>> 2) increase the correctness
>> 
>> of the strings increases productivity.
> 
> I reckon you are a Java code writer.

No, I'm a translator :)

Jean-Christophe 

Re: IDE

[Przemysław Wojnowski]

I forgot to mention one thing about Java projects: very often they are 
multi-language. Main code in Java, tests in e.g. Groovy, resources mix of HTML, 
JavaScript, XML, JSON. So, IDE needs to have configurations for different 
languages in the same project.

Re: IDE

[Eli Zaretskii]

> Cc: adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Przemysław Wojnowski <esperanto@cumego.com>
> Date: Sun, 11 Oct 2015 15:18:17 +0200
> 
> For (project-oriented/enterprise) Java the features are:
> 1. Project support
> IDE has to know the boundaries of a project - where are sources, tests, libs, 
> resources/assets (additional files used in an app in runtime), docs - and what 
> are relations between them. Also it has to know how to work with project's build 
> tool (be it Maven, Gradle, etc.).
> A programmer joining a project (99% of cases) should be able to open/import it 
> and start work. Every Java IDE have that.
> 
> 2. Code completion
>  From whole project, used libraries, and resources
> 
> 3. Jumping around the project code and resources.
> Jumping to around the project code and used libraries. Another thing is jumping 
> to/from resources (for example aspects can be defined in an XML file and IDE 
> could allow to jump to matching classes).
> 
> 4. Finding usages of fields, methods, types.
> Helps to wrap head around project.
> 
> 5. Automatic compilation and showing compilation erros/warnings.
> Tightens development feedback loop.
> 
> 6. Running the tests (current, selected, all)
> Tighten development feedback loop.
> 
> 7. Debugging
> A must, especially for legacy code, so, basically 99% of projects. :-)
> 
> 8. Running the app
> 
> 9. Basic refactoring.
> I do refactor _a lot_ and in my experience the most important refactoring is 
> Extract Method. Others, while helpful, are less often used, compared to the EM.
> One variation of Extract Method is "EM with finding duplicates", which works 
> like this:
> - ask user for a method name,
> - find all occurrences of selected code in the buffer
> - ask user if she wants to replace all occurrences with the call to the new method.
> This is fantastic feature that Intellij has and helps to remove a lot of 
> duplicated code.
> 
> 10. Showing documentation (tooltip, etc.)

I think you forgot profiling.

Re: IDE

[Przemysław Wojnowski]

W dniu 11.10.2015 o 18:04, Eli Zaretskii pisze:
[...]
>
> I think you forgot profiling.

I don't know you and, most probably, we're in different cultures, but if that
was a sarcasm, then you are waisting your time and mine. It doesn't hurt me
(Bible: 1-Cor,13,11), but rather the project by not focusing on getting the job
done. I'm not complaining that no one does the job, but offer a help (even small 
one) and try to improve the project.

Cheers,
Przemysław

Re: IDE

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Przemysław Wojnowski <esperanto@cumego.com>
> Date: Sun, 11 Oct 2015 19:05:38 +0200
> 
> W dniu 11.10.2015 o 18:04, Eli Zaretskii pisze:
> [...]
> >
> > I think you forgot profiling.
> 
> I don't know you and, most probably, we're in different cultures, but if that
> was a sarcasm

It wasn't.  AFAIR, a profiler is indeed an important part of at least
one popular Jave IDE.

Re: IDE

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: emacs-devel@gnu.org
>> From: Przemysław Wojnowski <esperanto@cumego.com>
>> Date: Sun, 11 Oct 2015 19:05:38 +0200
>> 
>> W dniu 11.10.2015 o 18:04, Eli Zaretskii pisze:
>> [...]
>> >
>> > I think you forgot profiling.
>> 
>> I don't know you and, most probably, we're in different cultures, but if that
>> was a sarcasm
>
> It wasn't.  AFAIR, a profiler is indeed an important part of at least
> one popular Jave IDE.

Associating the output of gprof and gcov (and perf) with the
corresponding sources (possibly also selecting traced areas) and
allowing editing based on the results would certainly fit what
developers have to do.  Leafing through a profiled call graph while
displaying the corresponding source files in a parallel window: clear
IDE functionality.

Not sure why that would be sarcastic.

-- 
David Kastrup

Re: IDE

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

It seems that Przemysław took offense at

  > I think you forgot profiling.

because the words "you forgot" suggested criticism of him.

It would be better (though less colorful) to say the same point with

  > I think we need profiling too.

because that is less likely to be misunderstood.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: IDE

[Przemysław Wojnowski]

W dniu 11.10.2015 o 19:15, Eli Zaretskii pisze:
>> Cc: emacs-devel@gnu.org
>> From: Przemysław Wojnowski <esperanto@cumego.com>
>> Date: Sun, 11 Oct 2015 19:05:38 +0200
>>
>> W dniu 11.10.2015 o 18:04, Eli Zaretskii pisze:
>> [...]
>>>
>>> I think you forgot profiling.
>>
>> I don't know you and, most probably, we're in different cultures, but if that
>> was a sarcasm
>
> It wasn't.  AFAIR, a profiler is indeed an important part of at least
> one popular Jave IDE.

In such case I'm sorry for going too far. :-)

Anyway, I haven't forgotten about a profiler. In enterprise Java it doesn't
have to be a part of an IDE and in most cases an external product is used -
AFAIK it's true for other JVM languages too.

That's why people having experience in other environments should write what
makes them productive, in their environments. Maybe for C/C++ programmer a
profiler is a must. I don't know.

To be clear that we're on the same page. We are in the brainstorming phase now,
collecting ideas that will be evaluated later.

Re: IDE

[John Wiegley]

>>>>> Przemysław Wojnowski <esperanto@cumego.com> writes:

> I cannot say what makes a Ruby programmer productive, but I can list a few
> things important for Java programmers. Others, having experience with other
> languages, may list _must have_ features for their environments.

I presented a list of candidate IDE features a few days ago. If you could
present your list in terms of additions or subtractions from that one, it
would make things much easier. I believe I already covered everything that you
proposed, but I'd like your confirmation of that.

John

Re: IDE

[Dmitry Gutov]

On 10/11/2015 04:18 PM, Przemysław Wojnowski wrote:

> For (project-oriented/enterprise) Java the features are:
> 1. Project support
> IDE has to know the boundaries of a project - where are sources, tests,
> libs, resources/assets (additional files used in an app in runtime),
> docs - and what are relations between them. Also it has to know how to
> work with project's build tool (be it Maven, Gradle, etc.).
> A programmer joining a project (99% of cases) should be able to
> open/import it and start work. Every Java IDE have that.

Emacs master now has project.el, which will be a step in this direction.

We can extend that API to contain information about where certain 
resources are, but for each type of projects there will be a different 
set of resources. While we could have a jump-to-test command (tests 
should be common), we can't easily have a commands for each kind of 
"jump to the web template for this controller" action.

So I'm not sure how to solve this. Have a "jump-to-related-file" 
command, which will prompt for the type of the resource when invoked?

Support for build tools seems more straightforward, someone should just 
collect an overview of how users interact with different ones, like 
Make, Maven, Gradle, Rake, etc, to firmly establish the common ground.

> 2. Code completion
>  From whole project, used libraries, and resources

Unless CEDET delivers on that front, we're unlikely to have completion 
for Java in the near future in the core. But there are third-party 
packages that provide it.

> 3. Jumping around the project code and resources.
> Jumping to around the project code and used libraries. Another thing is
> jumping to/from resources (for example aspects can be defined in an XML
> file and IDE could allow to jump to matching classes).

Do you mean "jump to the thing at point"? That sounds complicated, and 
support for different "things" will have to be implemented separately.

> 4. Finding usages of fields, methods, types.
> Helps to wrap head around project.

That's within the purview of xref, and up to CEDET or external tools to 
implement. But you can try xref-find-references now, for a 
quick-and-dirty Grep-based solution.

> 5. Automatic compilation and showing compilation erros/warnings.
> Tightens development feedback loop.

Flycheck. Sadly, it's not in Emacs.

> 6. Running the tests (current, selected, all)
> Tighten development feedback loop.

Not sure which facility would be most fitting. A project *could* include 
metadata about how to run its tests better, but then the resulting 
buffer would need new compilation-error-regexp-alist entries, and/or an 
ability to interact with a debugger/REPL if it's triggered during the 
test run.

> One variation of Extract Method is "EM with finding duplicates", which
> works like this:
> - ask user for a method name,
> - find all occurrences of selected code in the buffer
> - ask user if she wants to replace all occurrences with the call to the
> new method.
> This is fantastic feature that Intellij has and helps to remove a lot of
> duplicated code.

That sounds useful indeed.

RE: IDE

[Drew Adams]

> > 3. Jumping around the project code and resources.
> > Jumping to around the project code and used libraries. Another
> > thing is jumping to/from resources (for example aspects can be
> > defined in an XML file and IDE could allow to jump to matching
> > classes).
> 
> Do you mean "jump to the thing at point"? That sounds complicated,
> and support for different "things" will have to be implemented
> separately.

Sounds like good ol' Emacs TAGS, to me (or across-project Imenu).
Of course, the limiting factor is TAGS files that support such
"things".  But the infrastructure is there for it.  People have
been using Emacs TAGS files to "jump to the [definition of the]
thing at point" for 40 years.

> > 4. Finding usages of fields, methods, types.
> > Helps to wrap head around project.
> 
> That's within the purview of xref, and up to CEDET or external
> tools to implement. But you can try xref-find-references now,
> for a quick-and-dirty Grep-based solution.

TAGS files are typically for definitions, but they can be for
anything, including "usages of fields, methods, types".  You
could have different TAGS files for each of these "usages",
and use (search) them selectively or together.

What has not been done is write the code to create such TAGS
files, AFAIK.  (Maybe Semantic helps here?)

Re: IDE

[Tom]

Drew Adams <drew.adams <at> oracle.com> writes:
> 
> TAGS files are typically for definitions, but they can be for
> anything, including "usages of fields, methods, types".  You
> could have different TAGS files for each of these "usages",
> and use (search) them selectively or together.

What if different objects have fields or methods of the same name?

E.g. there is field called 'name' in lots of classes and I want to
find all usage of a name field, but only with certain object types
and there is code like:

   obj->name

?


TAGS files have considerable limitiations compared to techniques 
which actually understand the code. 

RE: IDE

[Drew Adams]

> > TAGS files are typically for definitions, but they can be for
> > anything, including "usages of fields, methods, types".  You
> > could have different TAGS files for each of these "usages",
> > and use (search) them selectively or together.
> 
> What if different objects have fields or methods of the same name?
> 
> E.g. there is field called 'name' in lots of classes and I want to
> find all usage of a name field, but only with certain object types
> and there is code like:
> 
>    obj->name

That's why I mentioned creation of TAGS files that make such
distinctions.  Separate files for separate such usages, for
example.  (The files can be mixed-and-matched when used.)

A TAGS file is just an index.  It can index anything you like -
not just function and variable definitions.  Of course, as I said,
code to write such sophisticated TAGS files would need to be written.

> TAGS files have considerable limitiations compared to techniques
> which actually understand the code.

TAGS files are written by programs that "actually understand
the code".

The understanding of the existing tags-file-creation programs
is not up to the task.  Granted.  But a program that does
understand the code in a deeper way could write better TAGS
files.  That was my point.

IOW, the answer is not there yet, but TAGS files were designed
for precisely this use, and they have been used for it for a
long time.  What's needed is code that writes the TAGS files
needed today, for the languages and contexts used today.

Re: IDE

[Nix]

[Catching up on this thread...]

On 12 Oct 2015, Drew Adams verbalised:

>> > 3. Jumping around the project code and resources.
>> > Jumping to around the project code and used libraries. Another
>> > thing is jumping to/from resources (for example aspects can be
>> > defined in an XML file and IDE could allow to jump to matching
>> > classes).
>> 
>> Do you mean "jump to the thing at point"? That sounds complicated,
>> and support for different "things" will have to be implemented
>> separately.
>
> Sounds like good ol' Emacs TAGS, to me (or across-project Imenu).
> Of course, the limiting factor is TAGS files that support such
> "things".  But the infrastructure is there for it.  People have
> been using Emacs TAGS files to "jump to the [definition of the]
> thing at point" for 40 years.

btw, recent GNU GLOBAL has now shifted to using a SQLite database for
its tags files, which makes it hugely more extensible, in theory, and
also makes it possible that Emacs could (once the modules code lands so
we could write a glue layer to SQLite) directly extract info from it.

Raw TAGS files are more or less unsuitable for anything but C and Lisp,
and are pretty poor even for that (e.g. you can only jump from uses to
definitions, the definition can only be in one place...)

-- 
NULL && (void)

Re: IDE

[Eli Zaretskii]

> From: Nix <nix@esperi.org.uk>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,
>         Przemysław Wojnowski
>  <esperanto@cumego.com>,
>         Eli Zaretskii <eliz@gnu.org>, adatgyujto@gmail.com,
>         emacs-devel@gnu.org
> Emacs: The Awakening
> Date: Sat, 24 Oct 2015 15:17:10 +0100
> 
> Raw TAGS files are more or less unsuitable for anything but C and Lisp,
> and are pretty poor even for that (e.g. you can only jump from uses to
> definitions, the definition can only be in one place...)

Raw TAGS files are not supposed to be used for anything but
definitions.

For references, you are supposed to use ID-Utils or something similar,
which use a different format of their DB.

Re: IDE

[Nix]

On 24 Oct 2015, Eli Zaretskii outgrape:

>> From: Nix <nix@esperi.org.uk>
>> Cc: Dmitry Gutov <dgutov@yandex.ru>,
>>         Przemysław Wojnowski
>>  <esperanto@cumego.com>,
>>         Eli Zaretskii <eliz@gnu.org>, adatgyujto@gmail.com,
>>         emacs-devel@gnu.org
>> Emacs: The Awakening
>> Date: Sat, 24 Oct 2015 15:17:10 +0100
>> 
>> Raw TAGS files are more or less unsuitable for anything but C and Lisp,
>> and are pretty poor even for that (e.g. you can only jump from uses to
>> definitions, the definition can only be in one place...)
>
> Raw TAGS files are not supposed to be used for anything but
> definitions.
>
> For references, you are supposed to use ID-Utils or something similar,
> which use a different format of their DB.

Two different tools, for more or less identical jobs except that one is
one->many and the other is many->one? (In particular, the hard part
isn't the data structure, but the parsing.)

That strikes me as really, really ugly.

-- 
NULL && (void)

Re: IDE

[Eli Zaretskii]

> From: Nix <nix@esperi.org.uk>
> Cc: drew.adams@oracle.com, dgutov@yandex.ru, esperanto@cumego.com,
>         adatgyujto@gmail.com, emacs-devel@gnu.org
> Emacs: a Lisp interpreter masquerading as ... a Lisp interpreter!
> Date: Sat, 24 Oct 2015 17:29:31 +0100
> 
> On 24 Oct 2015, Eli Zaretskii outgrape:
> 
> >> From: Nix <nix@esperi.org.uk>
> >> Cc: Dmitry Gutov <dgutov@yandex.ru>,
> >>         Przemysław Wojnowski
> >>  <esperanto@cumego.com>,
> >>         Eli Zaretskii <eliz@gnu.org>, adatgyujto@gmail.com,
> >>         emacs-devel@gnu.org
> >> Emacs: The Awakening
> >> Date: Sat, 24 Oct 2015 15:17:10 +0100
> >> 
> >> Raw TAGS files are more or less unsuitable for anything but C and Lisp,
> >> and are pretty poor even for that (e.g. you can only jump from uses to
> >> definitions, the definition can only be in one place...)
> >
> > Raw TAGS files are not supposed to be used for anything but
> > definitions.
> >
> > For references, you are supposed to use ID-Utils or something similar,
> > which use a different format of their DB.
> 
> Two different tools, for more or less identical jobs except that one is
> one->many and the other is many->one? (In particular, the hard part
> isn't the data structure, but the parsing.)
> 
> That strikes me as really, really ugly.

Are we still talking about an Emacs IDE?  If so, there's only one
tool: Emacs.  What happens behind the scenes is of interest to us
developers, but the user doesn't need to know, or even suspect.

Right?

RE: IDE

[Drew Adams]

> > For references, you are supposed to use ID-Utils or something similar,
> > which use a different format of their DB.
> 
> Two different tools, for more or less identical jobs except that one is
> one->many and the other is many->one? (In particular, the hard part
> isn't the data structure, but the parsing.)

What prevents someone from creating a TAGS file that includes
"references" as (additions forms of) "definitions"?  How is
adding references different in principle from adding, say,
handling of defstructs to a program that previously only
handled defuns and defvars?

You can index pretty much anything.  You could presumably
even create a full-text index and write it out as a TAGS
file, if you were up to it.

(But I agree: the hard part is the parsing.  The TAGS file
data structure is not the problem.)

Re: IDE

[Dmitry Gutov]

On 10/24/2015 08:00 PM, Drew Adams wrote:

> (But I agree: the hard part is the parsing.  The TAGS file
> data structure is not the problem.)

It *is* a problem as well. You basically have to re-generate the file 
each time you reparse the project (--append has its drawbacks), and GNU 
Global optimizes that (someone familiar with SQL databases should easily 
imagine a way to do that).

Further, now we're forced to parse the TAGS file and perform filtering 
in Elisp, which can be performed faster in an external program. Global 
can help there as well (we don't really need FFI support, unless Global 
command line interface is found too limiting).

RE: IDE

[Drew Adams]

> > (But I agree: the hard part is the parsing.  The TAGS file
> > data structure is not the problem.)
> 
> It *is* a problem as well. You basically have to re-generate the file
> each time you reparse the project (--append has its drawbacks), and GNU
> Global optimizes that (someone familiar with SQL databases should easily
> imagine a way to do that).
>
> Further, now we're forced to parse the TAGS file and perform filtering
> in Elisp, which can be performed faster in an external program. Global
> can help there as well (we don't really need FFI support, unless Global
> command line interface is found too limiting).

I'm not contrasting TAGS with Gnu Global.  You are free to do
that.  I am not arguing in favor of TAGS over other indexing
and querying mechanisms.

The TAGS file feature defines an index format and an index
query mechanism.  That's all.  How and when the content of a
given index gets generated or updated is a different question.

And that generation/updating involves parsing, which has
been acknowledged to be the hard part.

Everything you say in support of claiming that the data
structure "*is* a problem" is, in fact statements about
the problem of parsing, not the data structure format.

If you want to argue that any use of *files* to hold the
index structure is problematic then do so explicitly.

Even then, that does not invalidate the TAGS index
structure.  It need not be stored on disk, in principle.

Re: IDE

[Dmitry Gutov]

On 10/24/2015 08:42 PM, Drew Adams wrote:

> I'm not contrasting TAGS with Gnu Global.  You are free to do
> that.  I am not arguing in favor of TAGS over other indexing
> and querying mechanisms.

Emacs doesn't have any real abstraction over TAGS files. etags.el 
basically operates on its contents.

> The TAGS file feature defines an index format and an index
> query mechanism.  That's all.  How and when the content of a
> given index gets generated or updated is a different question.

The index doesn't even say what kind of hit it is. Is it a definition? 
Is it a reference? Is it both? Like, a method override.

> And that generation/updating involves parsing, which has
> been acknowledged to be the hard part.

There are several parts, of varying difficulties.

> Everything you say in support of claiming that the data
> structure "*is* a problem" is, in fact statements about
> the problem of parsing, not the data structure format.

Nothing I've said about the file format yet is concerned with parsing.

> If you want to argue that any use of *files* to hold the
> index structure is problematic then do so explicitly.

Flat files, yes. Not any kind of files, obviously.

RE: IDE

[Drew Adams]

> > I'm not contrasting TAGS with Gnu Global.  You are free to do
> > that.  I am not arguing in favor of TAGS over other indexing
> > and querying mechanisms.
> 
> Emacs doesn't have any real abstraction over TAGS files. etags.el
> basically operates on its contents.

Yes, and?

> > The TAGS file feature defines an index format and an index
> > query mechanism.  That's all.  How and when the content of a
> > given index gets generated or updated is a different question.
> 
> The index doesn't even say what kind of hit it is. Is it a definition?
> Is it a reference? Is it both? Like, a method override.

It lets you index a location in a source file, associating it
with a name (symbol).  That's all it does.  No one is saying
that TAGS is the most general indexing system, or that it is
adequate for all indexing needs.

It is one tool among many possible index-and-query tools.

> > And that generation/updating involves parsing, which has
> > been acknowledged to be the hard part.
> 
> There are several parts, of varying difficulties.
> 
> > Everything you say in support of claiming that the data
> > structure "*is* a problem" is, in fact statements about
> > the problem of parsing, not the data structure format.
> 
> Nothing I've said about the file format yet is concerned
> with parsing.

Please read what you wrote, which you've elided.  The problems
you mentioned were about (1) having to "re-generate the file
each time you reparse the project" and (2) being "forced to
parse the TAGS file and perform filtering in Elisp".

I guess you could argue that #2 as a problem derives from
the TAGS data structure, but nothing specific was said about
what that problem is.  #1 seems clearly to be about parsing
the source files.

> > If you want to argue that any use of *files* to hold the
> > index structure is problematic then do so explicitly.
> 
> Flat files, yes. Not any kind of files, obviously.

You might want to elaborate, if there is something important
there.

But again.  No one is arguing that TAGS files are the only
or the "best" indexing feature.  It would be silly to do
so.  They, like Imenu, remain a useful feature for Emacs.

And it is unfair, I think, to point to current deficiencies
in support for a language as proof, by itself, that the Emacs
TAGS feature is problematic for that language.

There can be other ways in which it is not ideal, but current
lack of someone having written support for this or that language
is not, in itself, a reason.  The same holds for Imenu.

There are no doubt languages for which TAGS or Imenu is not
sufficient.  But just because a given language currently has
no support for creating a TAGS file or an Imenu menu is not a
sufficient reason for concluding that TAGS or Imenu is
inadequate for that language.  Some other, specific reasons
need to be given (your mention of methods, for example).

Re: IDE

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

How about making a separate thread for this discussion of tags files
and the like.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

RE: IDE

[Drew Adams]

> > Raw TAGS files are more or less unsuitable for anything but C and Lisp,
> > and are pretty poor even for that (e.g. you can only jump from uses to
> > definitions, the definition can only be in one place...)
> 
> Raw TAGS files are not supposed to be used for anything but
> definitions.

Where "definition" can be whatever you want, AFAIK.  So unless I'm
mistaken about this, I don't think your statement is very meaningful.

A TAGS file is just an index.  You can index whatever you like,
AFAIK.

Re: IDE

[Eli Zaretskii]

> Date: Sat, 24 Oct 2015 10:00:25 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: dgutov@yandex.ru, esperanto@cumego.com, adatgyujto@gmail.com,
>         emacs-devel@gnu.org
> 
> > > Raw TAGS files are more or less unsuitable for anything but C and Lisp,
> > > and are pretty poor even for that (e.g. you can only jump from uses to
> > > definitions, the definition can only be in one place...)
> > 
> > Raw TAGS files are not supposed to be used for anything but
> > definitions.
> 
> Where "definition" can be whatever you want, AFAIK.

"Definition" in this context means the implementation.  There's only
one implementation, but there might be many references
(a.k.a. "calls").

> A TAGS file is just an index.  You can index whatever you like,
> AFAIK.

An index where the key is the symbol itself can only hold one instance
of every symbol.

Re: IDE

[Steinar Bang]

>>>>> Eli Zaretskii <eliz@gnu.org>:

> An index where the key is the symbol itself can only hold one instance
> of every symbol.

This will come as a shock to all the search engines out there.

(unless you were talking about TAGS files specifically...?)

Re: IDE

[Eli Zaretskii]

> From: Steinar Bang <sb@dod.no>
> Date: Mon, 26 Oct 2015 18:32:36 +0100
> 
> >>>>> Eli Zaretskii <eliz@gnu.org>:
> 
> > An index where the key is the symbol itself can only hold one instance
> > of every symbol.
> 
> This will come as a shock to all the search engines out there.
> 
> (unless you were talking about TAGS files specifically...?)

Of course, I was.  That's what this thread is about.

Re: IDE

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> >>>>> Eli Zaretskii <eliz@gnu.org>:
>> 
>> > An index where the key is the symbol itself can only hold one instance
>> > of every symbol.
>> 
>> This will come as a shock to all the search engines out there.
>> 
>> (unless you were talking about TAGS files specifically...?)
>
> Of course, I was.  That's what this thread is about.

But TAGS files can handle multiple values for the same key just well.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: IDE

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Date: Mon, 26 Oct 2015 21:04:15 +0100
> Cc: Steinar Bang <sb@dod.no>, emacs-devel@gnu.org
> 
> TAGS files can handle multiple values for the same key just well.

If they do, they will not be able to distinguish between the
definition and the references.  All the values are equal.

Re: IDE

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Date: Mon, 26 Oct 2015 21:04:15 +0100
>> Cc: Steinar Bang <sb@dod.no>, emacs-devel@gnu.org
>> 
>> TAGS files can handle multiple values for the same key just well.
>
> If they do, they will not be able to distinguish between the
> definition and the references.  All the values are equal.

I use etags-select for displaying multiple entries for the same key. The
line corresponding to each value is shown, so you can locate what you
want reasonably fast. An example:

Finding tag: push-back

In: /home/oscar/dev/idb/lp0/lib/prelude.lp0
1 [push-back] (defmacro push-back (v a b &rest)
2 [push-back] (defun push-back (v e)

In: /home/oscar/dev/idb/lp0/lib/list.lp0
3 [push-back] (defun push-back (list e)
4 [push-back] (defun push-back (list)

Re: IDE

[Dmitry Gutov]

On 10/26/2015 10:27 PM, Óscar Fuentes wrote:

> I use etags-select for displaying multiple entries for the same key. The
> line corresponding to each value is shown, so you can locate what you
> want reasonably fast.

That still means that the tool you are using doesn't know the kinds of 
these references.

And the subject of this thread is "IDE". Modern IDEs do know stuff.

Re: IDE

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 10/26/2015 10:27 PM, Óscar Fuentes wrote:
>
>> I use etags-select for displaying multiple entries for the same key. The
>> line corresponding to each value is shown, so you can locate what you
>> want reasonably fast.
>
> That still means that the tool you are using doesn't know the kinds of
> these references.
>
> And the subject of this thread is "IDE". Modern IDEs do know stuff.

I was addressing the specific sub-topic about TAGS.

For knowing stuff you need tools which are as sophisticated as the
language they are working with, and there is no medium term solution for
that requirement as far as Emacs core is concerned on the C++ realm,
probably Java as well. There are external Emacs packages that are on
track for solving this problem, and an increasing number of features are
being implemented around those external packages. That makes this
discussion about IDEs on Emacs core to look like idle chatting.

Re: IDE

[Dmitry Gutov]

On 10/27/2015 12:09 AM, Óscar Fuentes wrote:

>> And the subject of this thread is "IDE". Modern IDEs do know stuff.
>
> I was addressing the specific sub-topic about TAGS.

That subtopic started exactly with complaint that TAGS only allow for 
"jump to definition". Then someone argued that no, you can store 
anything what you like in them, we discussed that a bit, and here you've 
come to repeat the beginning: "For references, TAGS is of little help".

> For knowing stuff you need tools which are as sophisticated as the
> language they are working with, and there is no medium term solution for
> that requirement as far as Emacs core is concerned on the C++ realm,
> probably Java as well.

There can be some middle ground. From what I've seen of GNU Global 
(admittedly not too much), it's better for both "find definitions" and 
"find references" than TAGS.

Even in C++ and Java it's not too hard to implement a reasonably 
accurate parser that will index definitions. Recognizing references 
accurately is more of a problem (for now we indeed use Grep or similar 
tools).

> There are external Emacs packages that are on
> track for solving this problem, and an increasing number of features are
> being implemented around those external packages.

What Emacs can do is provide a common interface those external packages 
to hook into. Like progmodes/xref.el, for example.

> That makes this
> discussion about IDEs on Emacs core to look like idle chatting.

It is idle chatting, for the most part. No discussions of the code in 
Emacs master, no patches, etc.

Re: IDE

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 10/27/2015 12:09 AM, Óscar Fuentes wrote:
>
>>> And the subject of this thread is "IDE". Modern IDEs do know stuff.
>>
>> I was addressing the specific sub-topic about TAGS.
>
> That subtopic started exactly with complaint that TAGS only allow for
> "jump to definition". Then someone argued that no, you can store
> anything what you like in them, we discussed that a bit, and here
> you've come to repeat the beginning: "For references, TAGS is of
> little help".

Sorry, I jumped into the middle of the thread.

>> For knowing stuff you need tools which are as sophisticated as the
>> language they are working with, and there is no medium term solution for
>> that requirement as far as Emacs core is concerned on the C++ realm,
>> probably Java as well.
>
> There can be some middle ground.

If by "middle ground" you refer to something that gives the right result
90% of the time when there are external packages that gives the right
result 100% of the time, that middle ground is a waste of time by the
Emacs core hackers.

> From what I've seen of GNU Global
> (admittedly not too much), it's better for both "find definitions" and
> "find references" than TAGS.
>
> Even in C++ and Java it's not too hard to implement a reasonably
> accurate parser that will index definitions. Recognizing references
> accurately is more of a problem (for now we indeed use Grep or similar
> tools).

Definitions can be tricky in C++, if you wish to distinguish the case
where multiple namespaces or classes defines the same keyword and you
expect from Emacs to jump to the correct definition, deduces from the
context. In that case you need a parser that is good enough to act as a
compiler front-end.

>> There are external Emacs packages that are on
>> track for solving this problem, and an increasing number of features are
>> being implemented around those external packages.
>
> What Emacs can do is provide a common interface those external
> packages to hook into. Like progmodes/xref.el, for example.

Trying to find a common ground on current use cases is difficult enough.
Anticipating future requirements is almost impossible. Good luck with
that.

[snip]

Re: IDE

[Dmitry Gutov]

On 10/27/2015 01:06 AM, Óscar Fuentes wrote:

> If by "middle ground" you refer to something that gives the right result
> 90% of the time when there are external packages that gives the right
> result 100% of the time, that middle ground is a waste of time by the
> Emacs core hackers.

A Grep-based implementation is fairly easy to do, so not too much time 
is wasted. On the other hand, it can exercise the new APIs and provide 
the necessary feedback.

> Definitions can be tricky in C++, if you wish to distinguish the case
> where multiple namespaces or classes defines the same keyword and you
> expect from Emacs to jump to the correct definition, deduces from the
> context. In that case you need a parser that is good enough to act as a
> compiler front-end.

"jump to definition" is about accurately recognizing references. But if 
you're jumping to foo.bar(), and Emacs doesn't know the type of 'foo', 
at least it can show you the definitions of all methods named 'bar', and 
you'll be able to choose for yourself. Using etags-select, for instance.

> Trying to find a common ground on current use cases is difficult enough.
> Anticipating future requirements is almost impossible. Good luck with
> that.

That's defeatism. Doesn't it bother you that every third-party package 
uses a (sometimes subtly) different set of key bindings, and a different 
way to present the same kinds of information (definitions, references, 
documentation, etc)?

Re: IDE

[Óscar Fuentes]

Dmitry Gutov <dgutov@yandex.ru> writes:

[snip]

>> Definitions can be tricky in C++, if you wish to distinguish the case
>> where multiple namespaces or classes defines the same keyword and you
>> expect from Emacs to jump to the correct definition, deduces from the
>> context. In that case you need a parser that is good enough to act as a
>> compiler front-end.
>
> "jump to definition" is about accurately recognizing references. But
> if you're jumping to foo.bar(), and Emacs doesn't know the type of
> 'foo', at least it can show you the definitions of all methods named
> 'bar', and you'll be able to choose for yourself. Using etags-select,
> for instance.

For not-so-big C++ projects, that can be as useless as displaying all
the uses of "bar" on the code base.

>> Trying to find a common ground on current use cases is difficult enough.
>> Anticipating future requirements is almost impossible. Good luck with
>> that.
>
> That's defeatism. Doesn't it bother you that every third-party package
> uses a (sometimes subtly) different set of key bindings, and a
> different way to present the same kinds of information (definitions,
> references, documentation, etc)?

If there is a common pattern, by all means, implement it (on a way that
provides for those subtle differences when they make sense.) But those
packages still need to maintain compatibility with older Emacsen.
Distributing the library on GNU ELPA instead of the Emacs core would
make things better for the external packages, though. But then, what's
the difference from developing the library outside of Emacs core and
working directly with the authors of those external packages? If the
library or API is developed on emacs-devel, is it acceptable to design
it by the requirements of rtags, for instance?

Re: IDE

[Dmitry Gutov]

On 10/27/2015 01:40 AM, Óscar Fuentes wrote:

> For not-so-big C++ projects, that can be as useless as displaying all
> the uses of "bar" on the code base.

Technically, it must be at least an order-of-magnitude better.

And there are other languages that we want to support, too. It's not 
just C++ out there.

> If there is a common pattern, by all means, implement it (on a way that
> provides for those subtle differences when they make sense.)

This kind of feature requests will have to be more detailed.

> But those
> packages still need to maintain compatibility with older Emacsen.

Not necessarily: they can have some duplication in the code, and work 
both ways.

> Distributing the library on GNU ELPA instead of the Emacs core would
> make things better for the external packages, though.

That's one solution, yes.

> But then, what's
> the difference from developing the library outside of Emacs core and
> working directly with the authors of those external packages?

GNU ELPA is still a part of Emacs core, organizationally. Anyway, I 
don't understand the question.

> If the
> library or API is developed on emacs-devel, is it acceptable to design
> it by the requirements of rtags, for instance?

Why not? As long as it's not *just* the requirements of rtags that are 
considered.

Re: IDE

[Eric Ludlam]

On 10/26/2015 07:06 PM, Óscar Fuentes wrote:
>>> There are external Emacs packages that are on
>>> >>track for solving this problem, and an increasing number of features are
>>> >>being implemented around those external packages.
>> >
>> >What Emacs can do is provide a common interface those external
>> >packages to hook into. Like progmodes/xref.el, for example.
 >
> Trying to find a common ground on current use cases is difficult enough.
> Anticipating future requirements is almost impossible. Good luck with
> that.

CEDET/Semantic already does this.  It can use itself, Global, idutils, 
or cscope and convert the output into a common semantic tag 
infrastructure.  It has a common searching mechanism so you just write 
one bit of code to find the symbol you want (via semanticdb) or 
references you want (via semantic-symref) and it will work fine no 
matter how the user may have set it up.

While there has certainly been debate here about if people should be 
writing parsers in elisp and how smart its smart completion is, but the 
above interfaces are indeed generic, rich, and provides the raw answers 
from those external tools with a common output format.

Eric

Re: IDE

[Nikolaus Rath]

On Oct 26 2015, Eric Ludlam <eric@siege-engine.com> wrote:
> On 10/26/2015 07:06 PM, Óscar Fuentes wrote:
>>>> There are external Emacs packages that are on
>>>> >>track for solving this problem, and an increasing number of features are
>>>> >>being implemented around those external packages.
>>> >
>>> >What Emacs can do is provide a common interface those external
>>> >packages to hook into. Like progmodes/xref.el, for example.
>>
>> Trying to find a common ground on current use cases is difficult enough.
>> Anticipating future requirements is almost impossible. Good luck with
>> that.
>
> CEDET/Semantic already does this.  It can use itself, Global, idutils,
> or cscope and convert the output into a common semantic tag
> infrastructure.  It has a common searching mechanism so you just write
> one bit of code to find the symbol you want (via semanticdb) or
> references you want (via semantic-symref) and it will work fine no
> matter how the user may have set it up.

Unfortunately, at least for me the "one bit of code" was not at all
obvious after reading the CEDET documentation. So while I believe that
any of Global/idutils/cscope would be good enough for the majority of my
use-cases, I wasn't able to make CEDET use any of them (or maybe CEDET
uses them, but I'm not actually invoking cedet with M-.?).

My impression that CEDET/Semantic doesn't lack functionality, but
end-user documentation.


Best,
-Nikolauss
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: IDE

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Date: Mon, 26 Oct 2015 20:01:06 -0700
> 
> My impression that CEDET/Semantic doesn't lack functionality, but
> end-user documentation.

Detailed bug reports about missing or incomplete CEDET documentation
are welcome.  Patches to improve that documentation are even more
welcome.

Re: IDE

[Nikolaus Rath]

On Oct 27 2015, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Nikolaus Rath <Nikolaus@rath.org>
>> Date: Mon, 26 Oct 2015 20:01:06 -0700
>> 
>> My impression that CEDET/Semantic doesn't lack functionality, but
>> end-user documentation.
>
> Detailed bug reports about missing or incomplete CEDET documentation
> are welcome.

It's hard to provide details about undocumented functionality, if one
doesn't know which functionality the code provides (since it's not
documented).

The best I can do is this: judging from this thread, Semantic offers a
lot of things that are not enabled when one just follows the "Using
Semantic" section of the info manual. There should be documentation
about how to enable and use those features.

> Patches to improve that documentation are even more
> welcome.

Naturally.


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: IDE

[Eli Zaretskii]

> From: Nikolaus Rath <Nikolaus@rath.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 26 Oct 2015 21:02:43 -0700
> 
> On Oct 27 2015, Eli Zaretskii <eliz@gnu.org> wrote:
> >> From: Nikolaus Rath <Nikolaus@rath.org>
> >> Date: Mon, 26 Oct 2015 20:01:06 -0700
> >> 
> >> My impression that CEDET/Semantic doesn't lack functionality, but
> >> end-user documentation.
> >
> > Detailed bug reports about missing or incomplete CEDET documentation
> > are welcome.
> 
> It's hard to provide details about undocumented functionality, if one
> doesn't know which functionality the code provides (since it's not
> documented).

I guess I've misunderstood you, sorry.  You said:

> Unfortunately, at least for me the "one bit of code" was not at all
> obvious after reading the CEDET documentation. So while I believe that
> any of Global/idutils/cscope would be good enough for the majority of my
> use-cases, I wasn't able to make CEDET use any of them (or maybe CEDET
> uses them, but I'm not actually invoking cedet with M-.?).

My interpretation of that, perhaps incorrect, was that you tried to
use some functionality and tried to find documentation about it, but
either the documentation was inadequate or completely missing.  Based
on that interpretation, I suggested that you report what you tried to
figure out from the documentation, but couldn't.

> The best I can do is this: judging from this thread, Semantic offers a
> lot of things that are not enabled when one just follows the "Using
> Semantic" section of the info manual. There should be documentation
> about how to enable and use those features.

A list if such issues that lack proper documentation would also be
useful, if you cannot provide a more detailed information.  TIA.

Re: IDE

[Nikolaus Rath]

On Oct 27 2015, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Nikolaus Rath <Nikolaus@rath.org>
>> Cc: emacs-devel@gnu.org
>> Date: Mon, 26 Oct 2015 21:02:43 -0700
>> 
>> On Oct 27 2015, Eli Zaretskii <eliz@gnu.org> wrote:
>> >> From: Nikolaus Rath <Nikolaus@rath.org>
>> >> Date: Mon, 26 Oct 2015 20:01:06 -0700
>> >> 
>> >> My impression that CEDET/Semantic doesn't lack functionality, but
>> >> end-user documentation.
>> >
>> > Detailed bug reports about missing or incomplete CEDET documentation
>> > are welcome.
>> 
>> It's hard to provide details about undocumented functionality, if one
>> doesn't know which functionality the code provides (since it's not
>> documented).
>
> I guess I've misunderstood you, sorry.  You said:
>
>> Unfortunately, at least for me the "one bit of code" was not at all
>> obvious after reading the CEDET documentation. So while I believe that
>> any of Global/idutils/cscope would be good enough for the majority of my
>> use-cases, I wasn't able to make CEDET use any of them (or maybe CEDET
>> uses them, but I'm not actually invoking cedet with M-.?).
>
> My interpretation of that, perhaps incorrect, was that you tried to
> use some functionality and tried to find documentation about it, but
> either the documentation was inadequate or completely missing.

Well, I followed the instructions in the manual but (as far as I can
tell) didn't get support for Global/idutils/cscope. At the time I didn't
suspect anything was wrong - until I read here that there ought to be
such support. Thus my conclusion that the "obvious bit of code" isn't
all that obvious (in both necessity and content).

> Based on that interpretation, I suggested that you report what you
> tried to figure out from the documentation, but couldn't.

I would have expected the documentation to tell me that I need to do
something in order to get Global et al support (and ideally also *what*
I need to do).


Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

Re: IDE

[Nix]

On 27 Oct 2015, Nikolaus Rath said:

> On Oct 27 2015, Eli Zaretskii <eliz@gnu.org> wrote:
>> My interpretation of that, perhaps incorrect, was that you tried to
>> use some functionality and tried to find documentation about it, but
>> either the documentation was inadequate or completely missing.
>
> Well, I followed the instructions in the manual but (as far as I can
> tell) didn't get support for Global/idutils/cscope. At the time I didn't
> suspect anything was wrong - until I read here that there ought to be
> such support. Thus my conclusion that the "obvious bit of code" isn't
> all that obvious (in both necessity and content).
>
>> Based on that interpretation, I suggested that you report what you
>> tried to figure out from the documentation, but couldn't.
>
> I would have expected the documentation to tell me that I need to do
> something in order to get Global et al support (and ideally also *what*
> I need to do).

The manual says

,----[ 7. Miscellaneous Commands ]
|    EDE can use external tools to help with file finding.  To do this,
| customize `ede-locate-setup-options'.
| 
|  -- Variable: ede-locate-setup-options
|      List of locate objects to try out by default.  Listed in order of
|      preference.  If the first item cannot be used in a particular
|      project, then the next one is tried.  It is always assumed that
|      "ede-locate-base" is at end of the list.
`----

It is true that 'Miscellaneous Commands' is not a very good place for
this, and it doesn't note that this is not only used for C-c . f (which
I have never used, and would never use, since it doesn't have any
completions at all so feels like something from a previous era), but for
all file searches done by EDE, Semantic, etc. None of the available
options are documented at all: you have to dig through the source code.

FWIW, I got it to work via

(setq ede-locate-setup-options '(ede-locate-global ede-locate-locate ede-locate-base))
;;...
(semantic-mode 1)
(semanticdb-enable-gnu-global-databases 'c-mode)
(semanticdb-enable-gnu-global-databases 'c++-mode)

but whether all of these are actually necessary or not (in particular,
whether the latter two options are redundant with part of the first) I
have no real idea.

-- 
NULL && (void)

Re: IDE

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Mon, 26 Oct 2015 21:27:56 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Andreas Schwab <schwab@linux-m68k.org>
> >> Date: Mon, 26 Oct 2015 21:04:15 +0100
> >> Cc: Steinar Bang <sb@dod.no>, emacs-devel@gnu.org
> >> 
> >> TAGS files can handle multiple values for the same key just well.
> >
> > If they do, they will not be able to distinguish between the
> > definition and the references.  All the values are equal.
> 
> I use etags-select for displaying multiple entries for the same key. The
> line corresponding to each value is shown, so you can locate what you
> want reasonably fast.

How fast it is depends on the number of references.  It is not
reasonable to ask the user to examine all the references one by one in
order to find the single definition.  "Go to definition" should be
instantaneous, if we want to satisfy users.

References is another matter: there users expect a list from which to
choose.

> An example:
> 
> Finding tag: push-back
> 
> In: /home/oscar/dev/idb/lp0/lib/prelude.lp0
> 1 [push-back] (defmacro push-back (v a b &rest)
> 2 [push-back] (defun push-back (v e)
> 
> In: /home/oscar/dev/idb/lp0/lib/list.lp0
> 3 [push-back] (defun push-back (list e)
> 4 [push-back] (defun push-back (list)

Your example includes only definitions, and a small number at that.
It doesn't scale.

Re: IDE

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> >> TAGS files can handle multiple values for the same key just well.
>> >
>> > If they do, they will not be able to distinguish between the
>> > definition and the references.  All the values are equal.
>> 
>> I use etags-select for displaying multiple entries for the same key. The
>> line corresponding to each value is shown, so you can locate what you
>> want reasonably fast.
>
> How fast it is depends on the number of references.  It is not
> reasonable to ask the user to examine all the references one by one in
> order to find the single definition.  "Go to definition" should be
> instantaneous, if we want to satisfy users.

My example shows multiple definitions of the same key taken from a TAGS
file, which proves that you can store there multiple values for the same
key and solve a real problem.

> References is another matter: there users expect a list from which to
> choose.

For references, TAGS is of little help. Grep is probably better.
Obviously, pointing to a data member or function and asking for the
references to *that* data member or function is something that is beyond
what Emacs core will provide on a realistic timeframe. People interested
on adding that feature to Emacs core would be more productive directing
their effort to GCC than to Emacs.

Re: IDE

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Mon, 26 Oct 2015 23:16:38 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> >> TAGS files can handle multiple values for the same key just well.
> >> >
> >> > If they do, they will not be able to distinguish between the
> >> > definition and the references.  All the values are equal.
> >> 
> >> I use etags-select for displaying multiple entries for the same key. The
> >> line corresponding to each value is shown, so you can locate what you
> >> want reasonably fast.
> >
> > How fast it is depends on the number of references.  It is not
> > reasonable to ask the user to examine all the references one by one in
> > order to find the single definition.  "Go to definition" should be
> > instantaneous, if we want to satisfy users.
> 
> My example shows multiple definitions of the same key taken from a TAGS
> file, which proves that you can store there multiple values for the same
> key and solve a real problem.

You are taking this sub-thread out of context.  Its context is that
TAGS cannot support both definitions and references because there's no
indication which one is which.

> > References is another matter: there users expect a list from which to
> > choose.
> 
> For references, TAGS is of little help. Grep is probably better.

ID-Utils is even better.  Etc., etc.  And that's exactly what this
sub-thread is about.

> Obviously, pointing to a data member or function and asking for the
> references to *that* data member or function is something that is beyond
> what Emacs core will provide on a realistic timeframe. People interested
> on adding that feature to Emacs core would be more productive directing
> their effort to GCC than to Emacs.

We are not talking about the Emacs core here (which does know how to
do what you say).  We are talking about TAGS and the functionality
that can be built only based on TAGS, without any other databases.

Re: IDE

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

[snip]

>> Obviously, pointing to a data member or function and asking for the
>> references to *that* data member or function is something that is beyond
>> what Emacs core will provide on a realistic timeframe. People interested
>> on adding that feature to Emacs core would be more productive directing
>> their effort to GCC than to Emacs.
>
> We are not talking about the Emacs core here (which does know how to
> do what you say).

Does it? AFAIK it has a framework that does that as long as something
else provides the required information. This does not qualify as
*knowing* how to do the necessary inferences.

I mention this just in case some user might get the impression that
Emacs has that feature.

[snip]

Re: IDE

[Eli Zaretskii]

> From: Óscar Fuentes <ofv@wanadoo.es>
> Date: Tue, 27 Oct 2015 13:24:37 +0100
> 
> >> Obviously, pointing to a data member or function and asking for the
> >> references to *that* data member or function is something that is beyond
> >> what Emacs core will provide on a realistic timeframe. People interested
> >> on adding that feature to Emacs core would be more productive directing
> >> their effort to GCC than to Emacs.
> >
> > We are not talking about the Emacs core here (which does know how to
> > do what you say).
> 
> Does it? AFAIK it has a framework that does that as long as something
> else provides the required information. This does not qualify as
> *knowing* how to do the necessary inferences.

The framework has a couple of back-ends that enable this.

> I mention this just in case some user might get the impression that
> Emacs has that feature.

It does.

Re: IDE

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Óscar Fuentes <ofv@wanadoo.es>
>> Date: Tue, 27 Oct 2015 13:24:37 +0100
>> 
>> >> Obviously, pointing to a data member or function and asking for the
>> >> references to *that* data member or function is something that is beyond
>> >> what Emacs core will provide on a realistic timeframe. People interested
>> >> on adding that feature to Emacs core would be more productive directing
>> >> their effort to GCC than to Emacs.
>> >
>> > We are not talking about the Emacs core here (which does know how to
>> > do what you say).
>> 
>> Does it? AFAIK it has a framework that does that as long as something
>> else provides the required information. This does not qualify as
>> *knowing* how to do the necessary inferences.
>
> The framework has a couple of back-ends that enable this.

Which ones?

>> I mention this just in case some user might get the impression that
>> Emacs has that feature.
>
> It does.

Maybe if you work on C and the expression at hand is not very
complicated. For C++, definitively not.

Re: IDE

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

> If they do, they will not be able to distinguish between the
> definition and the references.  All the values are equal.

A TAGS file does not know anything.  It is just an index, whose
interpretation is up to the application.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: IDE

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: sb@dod.no,  emacs-devel@gnu.org
> Date: Mon, 26 Oct 2015 22:14:41 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > If they do, they will not be able to distinguish between the
> > definition and the references.  All the values are equal.
> 
> A TAGS file does not know anything.  It is just an index, whose
> interpretation is up to the application.

The application just interprets what's in the file.  If the file
doesn't provide any indication that one of the entries is a
definition, the application will not know which one is.

Re: IDE

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Cc: sb@dod.no,  emacs-devel@gnu.org
>> Date: Mon, 26 Oct 2015 22:14:41 +0100
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > If they do, they will not be able to distinguish between the
>> > definition and the references.  All the values are equal.
>> 
>> A TAGS file does not know anything.  It is just an index, whose
>> interpretation is up to the application.
>
> The application just interprets what's in the file.  If the file
> doesn't provide any indication that one of the entries is a
> definition, the application will not know which one is.

There are producers and consumers.  They just have to agree on the
format.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: IDE

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Cc: sb@dod.no,  emacs-devel@gnu.org
> Date: Tue, 27 Oct 2015 18:39:17 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Andreas Schwab <schwab@linux-m68k.org>
> >> Cc: sb@dod.no,  emacs-devel@gnu.org
> >> Date: Mon, 26 Oct 2015 22:14:41 +0100
> >> 
> >> Eli Zaretskii <eliz@gnu.org> writes:
> >> 
> >> > If they do, they will not be able to distinguish between the
> >> > definition and the references.  All the values are equal.
> >> 
> >> A TAGS file does not know anything.  It is just an index, whose
> >> interpretation is up to the application.
> >
> > The application just interprets what's in the file.  If the file
> > doesn't provide any indication that one of the entries is a
> > definition, the application will not know which one is.
> 
> There are producers and consumers.  They just have to agree on the
> format.

I was talking about the _existing_ format.  It goes without saying
that it can be (almost trivially) extended to support additional
functionality.  But then it won't be TAGS format, strictly speaking.

Re: IDE

[Marcus Harnisch]

Eli Zaretskii <eliz@gnu.org> writes:

> "Definition" in this context means the implementation.  There's only
> one implementation, [...]

FWIW, in my day job I am spending most of my time in an aspect
oriented language. While there is an identifiable “master” definition,
having all aspects show up as definitions is far more useful. In many
cases (e.g vendor libraries) the “master” definitions are invisible,
yet I want to be able to refer to my own extensions as definition
points.

Cheers
Marcus

Re: IDE

[Dmitry Gutov]

On 10/24/2015 05:25 PM, Eli Zaretskii wrote:

> Raw TAGS files are not supposed to be used for anything but
> definitions.
>
> For references, you are supposed to use ID-Utils or something similar,
> which use a different format of their DB.

Why id-utils, really? AFAIK, Global supports different kinds of searches 
(definitions, references and others), *and* it can plug into ctags, 
which extends its list of supported languages (not sure if that limits 
the files scanned that way to "find definitions" search only).

Seems like a natural solution to try to use in Emacs. ggtags, in GNU 
ELPA, already has an Emacs Lisp interface for it.

Re: IDE

[Eli Zaretskii]

> Cc: drew.adams@oracle.com, esperanto@cumego.com, adatgyujto@gmail.com,
>  emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 24 Oct 2015 20:02:56 +0300
> 
> On 10/24/2015 05:25 PM, Eli Zaretskii wrote:
> 
> > Raw TAGS files are not supposed to be used for anything but
> > definitions.
> >
> > For references, you are supposed to use ID-Utils or something similar,
> > which use a different format of their DB.
> 
> Why id-utils, really? AFAIK, Global supports different kinds of searches 
> (definitions, references and others), *and* it can plug into ctags, 
> which extends its list of supported languages (not sure if that limits 
> the files scanned that way to "find definitions" search only).

We already support both, don't we?  So there's no contradiction.

My point was that the format of the database is not really important,
certainly not to the user.

Re: IDE

[Dmitry Gutov]

On 10/24/2015 08:11 PM, Eli Zaretskii wrote:

> We already support both, don't we?

For find-references, let's say yes. But not for find-definition, unless 
you include the GNU ELPA package.

> My point was that the format of the database is not really important,
> certainly not to the user.

A user might like fast project re-scanning, I imagine.

Re: IDE

[Eli Zaretskii]

> Cc: nix@esperi.org.uk, drew.adams@oracle.com, esperanto@cumego.com,
>  adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 24 Oct 2015 20:15:11 +0300
> 
> On 10/24/2015 08:11 PM, Eli Zaretskii wrote:
> 
> > We already support both, don't we?
> 
> For find-references, let's say yes. But not for find-definition, unless 
> you include the GNU ELPA package.

Why cannot we bundle it?

> > My point was that the format of the database is not really important,
> > certainly not to the user.
> 
> A user might like fast project re-scanning, I imagine.

If etags is not fast enough, then the user can switch the back-end.

Once again, the issue is not the format of the database.  That's
immaterial.

Re: IDE

[Dmitry Gutov]

On 10/24/2015 08:20 PM, Eli Zaretskii wrote:

> Why cannot we bundle it?

We "can" do anything. That's not an interesting question.

I believe Nix's suggestion was to use Global as the primary indexing 
solution (that role still belongs to etags.el, at least in part).

I think that might be a worthy change.

> Once again, the issue is not the format of the database.  That's
> immaterial.

Database format can have a real performance impact.

Re: IDE

[Eli Zaretskii]

> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sat, 24 Oct 2015 21:15:18 +0300
> Cc: nix@esperi.org.uk, esperanto@cumego.com, adatgyujto@gmail.com,
> 	drew.adams@oracle.com, emacs-devel@gnu.org
> 
> > Once again, the issue is not the format of the database.  That's
> > immaterial.
> 
> Database format can have a real performance impact.

Yes, but the issue is performance, not the database format.

Re: IDE

[Dmitry Gutov]

On 10/24/2015 09:59 PM, Eli Zaretskii wrote:

>> Database format can have a real performance impact.
>
> Yes, but the issue is performance, not the database format.

In a situation like that, someone else might have called lower 
performance a "symptom", and not "the issue".

Re: IDE

[Oleh Krehel]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Sat, 24 Oct 2015 21:15:18 +0300
>> Cc: nix@esperi.org.uk, esperanto@cumego.com, adatgyujto@gmail.com,
>> 	drew.adams@oracle.com, emacs-devel@gnu.org
>> 
>> > Once again, the issue is not the format of the database.  That's
>> > immaterial.
>> 
>> Database format can have a real performance impact.
>
> Yes, but the issue is performance, not the database format.

If I understood correctly what you mean by the database format, it
matters to me. The TAGS files are too simplistic, they don't understand
the language, either C and especially C++.  On the other hand, have a
look that these Semantic tags entries for e.g. etags.c:

("regex.h" include (:system-flag t) nil [4626 4644])
("CTAGS" variable (:constant-flag t) nil [4934 4939])
("streq" function
         (:typemodifiers ("static")
                         :arguments 
                         ( ("s" variable
                                (:pointer 1
                                          :constant-flag t
                                          :type "char")
                                (reparse-symbol arg-sub-list) [4973 4987])
                           ("t" variable
                                (:pointer 1
                                          :constant-flag t
                                          :type "char")
                                (reparse-symbol arg-sub-list) [4988 5002]))                  
                         :type "bool")
         nil [4954 5035])

This is a lot of useful information in a readable and usable format.
The only problem is that it's a little slow to parse: 190 files in
emacs/src take around 30 seconds for a full reparse. But then, all this
info is kept and is re-parsed only on timestamp changes.

I did a caching optimization for `moo-jump-local' from function-args
package. When called without update it takes <0.1s to bring up all 19356
semantic tags. The update (through a call with "C-u") takes ~3 seconds +
reparse time for any out-of-date file.

My point is that because `moo-jump-local' uses semantic, it's a lot more
precise than e.g. `ggtags-find-definition' that gives only the names of
9956 tags, with no semantic information.

Compare:

    MAX_PARAGRAPH_SEARCH
    x_parse_color

to:

#include <dispextern.h>                                                                           xterm.h
#include <termhooks.h>                                                                            xterm.h
#define BLACK_PIX_DEFAULT                                                                         xterm.h
#define WHITE_PIX_DEFAULT                                                                         xterm.h
#define STANDARD_EVENT_SET                                                                        xterm.h
x_bitmap_record                                                                                   xterm.h
  Pixmap pixmap                                                                                   xterm.h
  bool have_mask                                                                                  xterm.h
  Pixmap mask                                                                                     xterm.h
  char* file                                                                                      xterm.h
  int refcount                                                                                    xterm.h
  int height                                                                                      xterm.h
  int width                                                                                       xterm.h
  int depth                                                                                       xterm.h
color_name_cache_entry                                                                            xterm.h
  color_name_cache_entry* next                                                                    xterm.h
  XColor rgb                                                                                      xterm.h
  char* name                                                                                      xterm.h
Status x_parse_color (frame *f, const char *color_name, XColor *color);                           xterm.h

In my opinion, the tags format of semantic is very good, much better
than plain TAGS. Maybe some work needs to be done to make them generate
faster/more precise, e.g. make GCC output these tags files.
    

Re: IDE

[Eli Zaretskii]

> From: Oleh Krehel <ohwoeowho@gmail.com>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,  nix@esperi.org.uk,  esperanto@cumego.com,  adatgyujto@gmail.com,  drew.adams@oracle.com,  emacs-devel@gnu.org
> Date: Tue, 27 Oct 2015 09:21:40 +0100
> 
> >> > Once again, the issue is not the format of the database.  That's
> >> > immaterial.
> >> 
> >> Database format can have a real performance impact.
> >
> > Yes, but the issue is performance, not the database format.
> 
> If I understood correctly what you mean by the database format, it
> matters to me. The TAGS files are too simplistic, they don't understand
> the language, either C and especially C++.  On the other hand, have a
> look that these Semantic tags entries for e.g. etags.c:
> 
> ("regex.h" include (:system-flag t) nil [4626 4644])
> ("CTAGS" variable (:constant-flag t) nil [4934 4939])
> ("streq" function
>          (:typemodifiers ("static")
>                          :arguments 
>                          ( ("s" variable
>                                 (:pointer 1
>                                           :constant-flag t
>                                           :type "char")
>                                 (reparse-symbol arg-sub-list) [4973 4987])
>                            ("t" variable
>                                 (:pointer 1
>                                           :constant-flag t
>                                           :type "char")
>                                 (reparse-symbol arg-sub-list) [4988 5002]))                  
>                          :type "bool")
>          nil [4954 5035])
> 
> This is a lot of useful information in a readable and usable format.
> The only problem is that it's a little slow to parse: 190 files in
> emacs/src take around 30 seconds for a full reparse. But then, all this
> info is kept and is re-parsed only on timestamp changes.
> 
> I did a caching optimization for `moo-jump-local' from function-args
> package. When called without update it takes <0.1s to bring up all 19356
> semantic tags. The update (through a call with "C-u") takes ~3 seconds +
> reparse time for any out-of-date file.
> 
> My point is that because `moo-jump-local' uses semantic, it's a lot more
> precise than e.g. `ggtags-find-definition' that gives only the names of
> 9956 tags, with no semantic information.

The point I was trying to make is that for users all this is
unimportant.  All they want is functionality: performance, accuracy,
minimum false positives, etc.  How this affects the database format is
something they expect us the developers to figure out.

And even if you take the POV of a developer, first there should be
requirements: performance, accuracy, supported languages, etc.  Only
after that we get to design and implementation, where we select the
database which can enable the functionality and support the
requirements.  Note as you explain above why TAGS might not be
appropriate _because_ it cannot support certain important
functionality:

> In my opinion, the tags format of semantic is very good, much better
> than plain TAGS. Maybe some work needs to be done to make them generate
> faster/more precise, e.g. make GCC output these tags files.

And that's my point exactly: first there's functionality we want, and
only after that comes the selection of the database.

RE: IDE

[Drew Adams]

> > Sounds like good ol' Emacs TAGS, to me (or across-project Imenu).
> > Of course, the limiting factor is TAGS files that support such
> > "things".  But the infrastructure is there for it.  People have
> > been using Emacs TAGS files to "jump to the [definition of the]
> > thing at point" for 40 years.
> 
> btw, recent GNU GLOBAL has now shifted to using a SQLite database for
> its tags files, which makes it hugely more extensible, in theory, and
> also makes it possible that Emacs could (once the modules code lands so
> we could write a glue layer to SQLite) directly extract info from it.
> 
> Raw TAGS files are more or less unsuitable for anything but C and Lisp,
> and are pretty poor even for that (e.g. you can only jump from uses to
> definitions, the definition can only be in one place...)

I don't think this is something inherent in TAGS files.

You can write a TAGS file for any kind of "definitions".
And I put that in quotes because such a "definition" can
really mean anything at all.  A TAGS file is just an index
into a document or a set of documents.

The fact that a program might not exist yet for creating
useful TAGS files for some language does not change this.

And the same thing you say about TAGS could be said about,
say, Imenu:  Until/unless someone writes the code needed
to use Imenu in a particular mode, it does not support that
mode.  That's not a problem with or limitation of Imenu.
It's just a lack of interest in writing the requisite
support for it for that mode/language.

It's also not clear to me what you mean by "the definition
can only be in one place".  AFAIK, you can have multiple
definitions of ("defining" locations for) the same thingy
in a single TAGS file.

And certainly you can have multiple such in a _set_ of
TAGS files.  And part of the use of TAGS files is
searching across multiple TAGS files.  TAGS files are
composable: they can be used together.

Please correct me if I'm mistaken.  I'm no expert on
TAGS files.

(Also, it is welcome that SQLite and other indexing and
querying means also be made available to Emacs.  Emacs
is not limited to TAGS or Imenu or ...  Tomorrow you
might use a SQL database with SQL/JSON indexing and
querying.  Who knows?)

Re: IDE

[Przemysław Wojnowski]

W dniu 2015-10-12 13:46, Dmitry Gutov napisał(a):
>> For (project-oriented/enterprise) Java the features are:
>> 1. Project support
[...]
> Emacs master now has project.el, which will be a step in this
> direction.
I have seen that thread, but unfortunately haven't read it, yet.
It's waiting for the nearest holiday. :-|

> We can extend that API to contain information about where certain
> resources are, but for each type of projects there will be a different
> set of resources. While we could have a jump-to-test command (tests
> should be common), we can't easily have a commands for each kind of
> "jump to the web template for this controller" action.
IMHO defining relations between project elements should be delegated
to each type of project. For example Java Project knows where are
sources/tests/resources and can setup that using Project API.

Moreover in one project (lets call it Meta Project) there
should be a way to configure a set of language specific
subprojects, each one having its own backend(s) setup
(for code completion, docs, etc.).
A backend would be chosen by a mode in the current buffer (region?).

For example in a common "Java" webapp, the Meta Project setup could be:
{ languages:
   { java: {backend: classpath-indexer, build-tool: gradle, 
other-options: ...}
     javascript: {backend: tags-backend, build-tool: npm, ...}
     groovy: ...} }

> Support for build tools seems more straightforward, someone should
> just collect an overview of how users interact with different ones,
> like Make, Maven, Gradle, Rake, etc, to firmly establish the common
> ground.
IMHO better approach would be to provide an API that could be
used by build tool specific plugins to add build tasks
(in a Command Pattern manner). Such registered commands could be
presented to a user in some uniform form. For example:
In Maven plugin:
(build-api-add-command
   {name: "compile", command: function-ref})

When user selects a command the unified build tools runner does:
(defun build-api-run (command)
   (apply command.function-ref))

Of course, there can be more than one build tool in a project,
so, if windows/menus were presented, the user would see for each of them 
or maybe depending on current buffer's mode.
But the point is to provide an API not an implementation.

>> 2. Code completion
>>  From whole project, used libraries, and resources
>
> Unless CEDET delivers on that front, we're unlikely to have completion
> for Java in the near future in the core. But there are third-party
> packages that provide it.

IMHO, as many other things, this should be delegated to external tools 
that specializes in that. And CEDET should allow to use a set of 
external backends (at the same time).

One thing I'm not sure is whether CEDET should be used. IMHO yes, but 
AFAIR I've seen different voices on the list. So, not sure about that.

>> 3. Jumping around the project code and resources.
>> Jumping to around the project code and used libraries. Another thing
>> is
>> jumping to/from resources (for example aspects can be defined in an
>> XML file and IDE could allow to jump to matching classes).
>
> Do you mean "jump to the thing at point"? That sounds complicated, and
> support for different "things" will have to be implemented separately.
Yes, it would need to be implemented separately by each project type, 
because only project types know how to add semantic to different resources.
For example a Java project seeing Spring in dependencies may assume (or 
ask user) that applicationContext.xml file in actually Spring 
Application context, hence would know which elements are class names and 
then could use Java backend from the same project to find class' source.

>> 4. Finding usages of fields, methods, types.
>> Helps to wrap head around project.
>
> That's within the purview of xref, and up to CEDET or external tools
> to implement. But you can try xref-find-references now, for a
> quick-and-dirty Grep-based solution.
How to teach a backend (xref, CEDET, etc.) what my project is?
IMHO even good TAGS backend would be a good start if I, as an Emacs 
user, wouldn't need to configure it for a week and fight with each 
package to use it. This is where IDE steps in - it integrates.

>> 6. Running the tests (current, selected, all)
>> Tighten development feedback loop.
>
> Not sure which facility would be most fitting. A project *could*
> include metadata about how to run its tests better, but then the
> resulting buffer would need new compilation-error-regexp-alist
> entries,
Depending on language environment there may be different options. In 
Java world most build tools have plugins to run tests (single, group, 
all, etc.). So, Emacs Unified Test Runner could delegate that task to 
the tool and display the result (clearly the result can be displayed in 
many ways - other window being the simplest one, but the problem has 
been solved by MVX patterns decades ago).
Again, the test runner could provide an API for project types to 
register language specific runner Commands - this way a Java file may 
have, for example, two runners BuildToolRunner (that calls build tool 
command) and DirectJavaRunner (that instantiates and runs the test 
directly).

In the places where an API would be exposed I would see an EIEIO 
interface to make it explicit.

 > [...] and/or an ability to interact with a debugger/REPL if it's
 > triggered during the test run.
I have to think about that.


------------
To clarify, I've just listed features that I know are expected from a 
Java IDE. Some of them have implementations in Emacs/packages or 
external tools used by them. The point is that an IDE should _integrate_ 
that tools/packages, not end users.

At this moment whenever Emacs user tries to use a language she has to 
find a blog post that describes how to configure Emacs to be able to do 
something.

Re: IDE

[Dmitry Gutov]

On 10/13/2015 12:54 AM, Przemysław Wojnowski wrote:

>> Emacs master now has project.el, which will be a step in this
>> direction.
> I have seen that thread, but unfortunately haven't read it, yet.
> It's waiting for the nearest holiday. :-|

There have been several threads about it.

> IMHO defining relations between project elements should be delegated
> to each type of project. For example Java Project knows where are
> sources/tests/resources and can setup that using Project API.

Yes. So, what would do we do about the set of available commands?

> Moreover in one project (lets call it Meta Project) there
> should be a way to configure a set of language specific
> subprojects, each one having its own backend(s) setup
> (for code completion, docs, etc.).
> A backend would be chosen by a mode in the current buffer (region?).

I'm not sure about all that, but code completion and docs are outside of 
project.el's jurisdiction anyway.

> IMHO better approach would be to provide an API that could be
> used by build tool specific plugins to add build tasks
> (in a Command Pattern manner). Such registered commands could be
> presented to a user in some uniform form. For example:
> In Maven plugin:
> (build-api-add-command
>    {name: "compile", command: function-ref})

Mmm, yes. Allowing some code outside of the project definition to define 
the tasks and the way to run them might be better.

On the other hand, often the project definition file and the build file 
would be the same. So the project backend would have to read it anyway.

> Of course, there can be more than one build tool in a project,
> so, if windows/menus were presented, the user would see for each of them
> or maybe depending on current buffer's mode.

A build file (or several) is usually related to the project as a whole, 
so picking based on the current file might be suboptimal. On the other 
hand, we could present tasks from all build files combined.

> But the point is to provide an API not an implementation.

Sure. Please feel free to have a stab at defining it.

> IMHO, as many other things, this should be delegated to external tools
> that specializes in that. And CEDET should allow to use a set of
> external backends (at the same time).

Proposals which external tools exactly the Emacs core can use, will be 
welcome.

> How to teach a backend (xref, CEDET, etc.) what my project is?

An xref backend can call (project-current). But something (probably a 
minor mode) would have to set the appropriate xref backend.

> IMHO even good TAGS backend would be a good start if I, as an Emacs
> user, wouldn't need to configure it for a week and fight with each
> package to use it. This is where IDE steps in - it integrates.

The etags backend is currently the default one.

> Depending on language environment there may be different options. In
> Java world most build tools have plugins to run tests (single, group,
> all, etc.). So, Emacs Unified Test Runner could delegate that task to
> the tool and display the result (clearly the result can be displayed in
> many ways - other window being the simplest one, but the problem has
> been solved by MVX patterns decades ago).

Of course it would have to call the tool. That doesn't solve the 
questions of error highlighting and debugger integration.

> Again, the test runner could provide an API for project types to
> register language specific runner Commands - this way a Java file may
> have, for example, two runners BuildToolRunner (that calls build tool
> command) and DirectJavaRunner (that instantiates and runs the test
> directly).

I'd like to look at specific proposals for how this would look inside 
project.el, or some other, new file.

> To clarify, I've just listed features that I know are expected from a
> Java IDE. Some of them have implementations in Emacs/packages or
> external tools used by them. The point is that an IDE should _integrate_
> that tools/packages, not end users.

Sure, that would be better.

> At this moment whenever Emacs user tries to use a language she has to
> find a blog post that describes how to configure Emacs to be able to do
> something.

That might still be the case afterwards, to some extent, but at least 
after she's configured Emacs, the experience will be more consistent, 
and similar between different languages.

Re: IDE

[Eric Ludlam]

On 10/12/2015 05:54 PM, Przemysław Wojnowski wrote:
> IMHO defining relations between project elements should be delegated
> to each type of project. For example Java Project knows where are
> sources/tests/resources and can setup that using Project API.
>
> Moreover in one project (lets call it Meta Project) there
> should be a way to configure a set of language specific
> subprojects, each one having its own backend(s) setup
> (for code completion, docs, etc.).
> A backend would be chosen by a mode in the current buffer (region?).
>
> For example in a common "Java" webapp, the Meta Project setup could be:
> { languages:
>    { java: {backend: classpath-indexer, build-tool: gradle,
> other-options: ...}
>      javascript: {backend: tags-backend, build-tool: npm, ...}
>      groovy: ...} }
>
>> Support for build tools seems more straightforward, someone should
>> just collect an overview of how users interact with different ones,
>> like Make, Maven, Gradle, Rake, etc, to firmly establish the common
>> ground.
> IMHO better approach would be to provide an API that could be
> used by build tool specific plugins to add build tasks
> (in a Command Pattern manner). Such registered commands could be
> presented to a user in some uniform form. For example:
> In Maven plugin:
> (build-api-add-command
>    {name: "compile", command: function-ref})
>
> When user selects a command the unified build tools runner does:
> (defun build-api-run (command)
>    (apply command.function-ref))
>
> Of course, there can be more than one build tool in a project,
> so, if windows/menus were presented, the user would see for each of them
> or maybe depending on current buffer's mode.
> But the point is to provide an API not an implementation.

This is how EDE (a part of CEDET) is setup and works.  There are 
"projects", and in projects there are "targets".  There are project 
build commands, and target build commands.  Each project or target can 
have language specific features for setting up CEDET's parsers.

There is a set of different base classes for projects, and many 
specializations for various flavors of java projects such as maven and 
ant, C++ projects, lisp projects, and more.

Many folks besides myself have built support for different kinds of 
projects, so extending to new types isn't too hard.

Eric

Re: IDE

[Dmitry Gutov]

On 10/14/2015 05:45 AM, Eric Ludlam wrote:

> This is how EDE (a part of CEDET) is setup and works.  There are
> "projects", and in projects there are "targets".  There are project
> build commands, and target build commands.  Each project or target can
> have language specific features for setting up CEDET's parsers.

Is there a particular reason to have the notion of "target" in the 
project API? If the need is to simply disambiguate commands with the 
same name, the commands could be prefixed with the target name, e.g. 
"release:compile", "release:test".

> There is a set of different base classes for projects, and many
> specializations for various flavors of java projects such as maven and
> ant, C++ projects, lisp projects, and more.

What do you do if several different project types use the same build 
system (and so the logic to parse the build targets is the same)?

What would you do if a certain project type can be used with different 
build systems? Create an inheriting sub-type for each of them?

That approach looks worrying if we get several varying pieces of 
behavior like that: for example, different build tools and different 
test frameworks.

Re: IDE

[Alexis]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 10/14/2015 05:45 AM, Eric Ludlam wrote:
>
>> This is how EDE (a part of CEDET) is setup and works.  There 
>> are "projects", and in projects there are "targets".  There are 
>> project build commands, and target build commands.  Each 
>> project or target can have language specific features for 
>> setting up CEDET's parsers.
>
> Is there a particular reason to have the notion of "target" in 
> the  project API?

i've been wondering this too; in a post to this list in August:

    https://lists.gnu.org/archive/html/emacs-devel/2015-08/msg00278.html

i wrote:
 
> Eric Ludlam <address@hidden> writes: 
> 
>> I'm also interested in how to resolve some of the 
>> misconceptions about EDE. Perhaps looking at what prevents it 
>> from being 'on by default', which I'm sure is a long list, is a 
>> good way to move it in the right direction, even if it never is 
>> on by default. 
> 
> Well, say i'm developing a small Perl5 project - one main 
> module, supported by two helper modules, all of which are pure 
> Perl. When i look at the overview of EDE in the Emacs manual: 
> 
>    https://www.gnu.org/software/emacs/manual/html_node/emacs/EDE.html 
> 
> and read:  
>    A project may contain one or more targets. A target can be an 
>    object file, executable program, or some other type of file, 
>    which is “built” from one or more of the files in the 
>    project. 
> 
> i think to myself: "Okay, so EDE might be able to help me build 
> (say) a .tar.gz when i feel the project is ready for release and 
> distribution, but it's not relevant prior to that stage, since 
> the source .pm files /are/ the executable files. There's nothing 
> to 'build' during development and testing, so it seems i have no 
> use for EDE during those phases." 
> 
> i would also guess that, similarly, EDE might not be that useful 
> when developing in languages such as ELisp, Python or Ruby. 
> 
> Am i wrong? What would using EDE get me in such contexts?


Alexis.

Re: IDE

[Dmitry Gutov]

On 10/14/2015 03:14 PM, Alexis wrote:

>> i would also guess that, similarly, EDE might not be that useful when
>> developing in languages such as ELisp, Python or Ruby.

That has been my impression as well.

Re: IDE

[Eric Ludlam]

On 10/14/2015 09:53 AM, Dmitry Gutov wrote:
> On 10/14/2015 03:14 PM, Alexis wrote:
>
>>> i would also guess that, similarly, EDE might not be that useful when
>>> developing in languages such as ELisp, Python or Ruby.
>
> That has been my impression as well.

I use EDE with my Elisp projects and find it useful for 2 things:

* Simplify compiling elisp (when it matters)
* scoping of symref calls

Other lispy things supported with CEDET don't depend on the EDE part.

For languages that need ot fire up an external interpreter, it could 
provide a simple way to know where to start the interpreter so you can 
ask it questions.  That is similar to the other project.el project.

Eric

Re: IDE

[Eric Ludlam]

On 10/14/2015 07:42 AM, Dmitry Gutov wrote:
> On 10/14/2015 05:45 AM, Eric Ludlam wrote:
>
>> This is how EDE (a part of CEDET) is setup and works.  There are
>> "projects", and in projects there are "targets".  There are project
>> build commands, and target build commands.  Each project or target can
>> have language specific features for setting up CEDET's parsers.
>
> Is there a particular reason to have the notion of "target" in the 
> project API? If the need is to simply disambiguate commands with the 
> same name, the commands could be prefixed with the target name, e.g. 
> "release:compile", "release:test".

Historically, the 'targets' matched the makefile targets, and was used 
to generate a Makefile from a configuration.

For the other projects, the targets simply group source code together so 
if you use the 'compile' key sequence, you get something appropriate for 
that group of source files.  Sometime it is very simple so that .texi or 
.cpp execute different compile steps. Sometimes there is no difference.

>> There is a set of different base classes for projects, and many
>> specializations for various flavors of java projects such as maven and
>> ant, C++ projects, lisp projects, and more.
>
> What do you do if several different project types use the same build 
> system (and so the logic to parse the build targets is the same)?
>
> What would you do if a certain project type can be used with different 
> build systems? Create an inheriting sub-type for each of them?
>
> That approach looks worrying if we get several varying pieces of 
> behavior like that: for example, different build tools and different 
> test frameworks.
> .
>

It's fine for several project systems to do the same thing.  They could 
share some implementation or not, depending the way any set of lisp 
programs might.  Many shared behaviours are pushed up in the class 
hierarchy.  For example, handling include paths, java classpath, etc.  
For most things like 'compile', the similar code is about appending the 
string "make " with some target, or whatever it might be, so it isn't 
too deep.

There are some very simple projects in EDE, such as "emacs" which just 
knows how to find emacs.c and mark that directory tree as a project, and 
it knows how to assemble some compile and debug commands.  It also knows 
how to setup C preprocessor symbols and include paths.  Other project 
types are very complex, such as those that let you configure a project 
using 'customize' and then generate Makefiles for you.   That was a bit 
of a stretch.

Some are in between, such as the 'android' project type that finds the 
AndroidManifest.xml file, tags that tree, and queries your android SDK 
for build tools and paths to set everything up for you.

In one of these threads someone noted that it should be easy to setup 
and transparent.  When someone's project matches a supported type, all 
you need to do is turn on EDE and it then sets up the Semantic details 
for you.  Most C/C++ projects are not as magical, and need a more hands 
on approach, and that's where things get tricky.

Eric

Re: IDE

[Dmitry Gutov]

On 10/15/2015 03:14 AM, Eric Ludlam wrote:

> Historically, the 'targets' matched the makefile targets, and was used
> to generate a Makefile from a configuration.
>
> For the other projects, the targets simply group source code together so
> if you use the 'compile' key sequence, you get something appropriate for
> that group of source files.  Sometime it is very simple so that .texi or
> .cpp execute different compile steps. Sometimes there is no difference.

That doesn't sound portable: with flexibly scriptable build tools (e.g. 
Rake, and Ant, I imagine), an external tool reading the build file won't 
be able to determine, in general, which files or groups of files a task 
is related to (unless it executes the task and watches the disk for 
changes, which is not tenable).

> It's fine for several project systems to do the same thing.  They could
> share some implementation or not, depending the way any set of lisp
> programs might.  Many shared behaviours are pushed up in the class
> hierarchy.  For example, handling include paths, java classpath, etc.
> For most things like 'compile', the similar code is about appending the
> string "make " with some target, or whatever it might be, so it isn't
> too deep.

It's nice when things can fit into the single-inheritance hierarchy.

But consider if there are several axes of customization. For example, if 
we have Java, Scala and Groovy based projects, and each sort can use 
Ant, Maven or, say, SBT.

Will that be 9 project definitions that someone will have to type out?

If we decouple the "build system" from "project", however, that would be 
just 6 definitions, and one could add to either set without having to 
create multiple new definitions.

So, for project.el it seems appropriate to add either a variable 
(project-build-system) or a hook similar to project-find-functions.

Does a lot of code in EDE *really* need to know what build system a 
project uses?

> Some are in between, such as the 'android' project type that finds the
> AndroidManifest.xml file, tags that tree, and queries your android SDK
> for build tools and paths to set everything up for you.

That's one of the easier cases, conceptually, because there are no 
alternative choices for build tools.

Re: IDE

[Elias Mårtenson]

[-- Attachment #1: Type: text/plain, Size: 685 bytes --]

On 15 October 2015 at 12:21, Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 10/15/2015 03:14 AM, Eric Ludlam wrote:
>
> Some are in between, such as the 'android' project type that finds the
>>
> AndroidManifest.xml file, tags that tree, and queries your android SDK
>> for build tools and paths to set everything up for you.
>>
>
> That's one of the easier cases, conceptually, because there are no
> alternative choices for build tools.
>

For Android, there is at least three that I can think of: Old-style pure
Ant builds, IntelliJ IDEA builds and Gradle builds.

All these have different behaviours. In particular, they all have different
ways of generating R.java.

Regards,
Elias

[-- Attachment #2: Type: text/html, Size: 1352 bytes --]

Re: IDE

[Dmitry Gutov]

On 10/15/2015 08:07 AM, Elias Mårtenson wrote:

> For Android, there is at least three that I can think of: Old-style pure
> Ant builds, IntelliJ IDEA builds and Gradle builds.

Very well, I stand corrected. 
https://developer.android.com/sdk/installing/studio-build.html gave the 
impression that Gradle is the only one, or at least the current blessed 
solution.

I wonder if CEDET supports more than one of them.

Re: IDE

[Eric Ludlam]

On 10/15/2015 01:16 AM, Dmitry Gutov wrote:
> On 10/15/2015 08:07 AM, Elias Mårtenson wrote:
>
>> For Android, there is at least three that I can think of: Old-style pure
>> Ant builds, IntelliJ IDEA builds and Gradle builds.
>
> Very well, I stand corrected. 
> https://developer.android.com/sdk/installing/studio-build.html gave 
> the impression that Gradle is the only one, or at least the current 
> blessed solution.
>
> I wonder if CEDET supports more than one of them.
>
I added support for the original command line SDK that used ant for Linux.

I haven't upgraded to android studio and haven't hacked my phone in a 
while so can't say much more.  I don't know if anyone else really uses 
Emacs for
android as I've gotten no feedback here.

Eric

Re: IDE

[Eric Ludlam]

On 10/15/2015 12:21 AM, Dmitry Gutov wrote:
> It's fine for several project systems to do the same thing.  They could
>> share some implementation or not, depending the way any set of lisp
>> programs might.  Many shared behaviours are pushed up in the class
>> hierarchy.  For example, handling include paths, java classpath, etc.
>> For most things like 'compile', the similar code is about appending the
>> string "make " with some target, or whatever it might be, so it isn't
>> too deep.
>
> It's nice when things can fit into the single-inheritance hierarchy.
>
> But consider if there are several axes of customization. For example, 
> if we have Java, Scala and Groovy based projects, and each sort can 
> use Ant, Maven or, say, SBT.
>
> Will that be 9 project definitions that someone will have to type out?
>
> If we decouple the "build system" from "project", however, that would 
> be just 6 definitions, and one could add to either set without having 
> to create multiple new definitions.
>
> So, for project.el it seems appropriate to add either a variable 
> (project-build-system) or a hook similar to project-find-functions.
>
> Does a lot of code in EDE *really* need to know what build system a 
> project uses?
>

EDE, the high level framework, doesn't care about the build system.

Any particular project type may or may not care about the build system.  
Some do because they are the build system.  Some do because they look in 
the config files to try to extract some handy nuggets of information.  
Some do because the build system leaves a file behind that can be 
detected as the root of the project.

Several just let you configure a text string that says how to fork off 
the 'compile' command and leave it at that.

For your example above someone who is familiar with those tools would 
pick the best way to detect a project (maybe by build system like ant, 
or by some other means) and also pick the best way to extract whatever 
data is needed, such as a command to pass to 'compile', and hopefully a 
classpath.  It might be 6 independent types that share a lot of code or 
baseclasses, or maybe one hybrid. I don't think it really matters.

In the end, EDE the framework provides a common set of:
* commands a user can use to interact with the project such as compiling.
* A place to put logic that helps other tools that have project
   dependencies such as include paths, classpath, or project root detection.

The individual EDE projects then layer on bonus features, such as 
creating a specialized project (such as Android or Automake) from 
scratch, handling configurations, and a few other random things.

Eric

Re: IDE

[Przemysław Wojnowski]

W dniu 15.10.2015 o 15:18, Eric Ludlam pisze:
> EDE, the high level framework, doesn't care about the build system.
>
> Any particular project type may or may not care about the build system. Some do
> because they are the build system.  Some do because they look in the config
> files to try to extract some handy nuggets of information. Some do because the
> build system leaves a file behind that can be detected as the root of the project.
>
[...]

> In the end, EDE the framework provides a common set of:
> * commands a user can use to interact with the project such as compiling.
> * A place to put logic that helps other tools that have project
>    dependencies such as include paths, classpath, or project root detection.

IMHO this is very good start for project support, even though EDE is not
perfect. It is much better to refine it than to throw away and reinvent the
wheel, which may not have a happy end.

Anyway, I'll try it.

Thanks for explanations,
Przemysław

Re: IDE

[Dmitry Gutov]

On 10/15/2015 04:18 PM, Eric Ludlam wrote:

> Any particular project type may or may not care about the build system.
> Some do because they are the build system.  Some do because they look in
> the config files to try to extract some handy nuggets of information.
> Some do because the build system leaves a file behind that can be
> detected as the root of the project.

Then we might as well treat the build-file-as-project-root-marker and 
build-file-as-source-of-build-tasks as two unrelated things, in the API. 
And in certain cases both implementations can delegate to the same code.

> For your example above someone who is familiar with those tools would
> pick the best way to detect a project (maybe by build system like ant,
> or by some other means) and also pick the best way to extract whatever
> data is needed, such as a command to pass to 'compile', and hopefully a
> classpath.  It might be 6 independent types that share a lot of code or
> baseclasses, or maybe one hybrid. I don't think it really matters.

It matters if to *really* add support for a new build tool, the author 
has to add X new project definitions.

Re: IDE

[Przemysław Wojnowski]

W dniu 15.10.2015 o 22:31, Dmitry Gutov pisze:
> On 10/15/2015 04:18 PM, Eric Ludlam wrote:
>
>> Any particular project type may or may not care about the build system.
>> Some do because they are the build system.  Some do because they look in
>> the config files to try to extract some handy nuggets of information.
>> Some do because the build system leaves a file behind that can be
>> detected as the root of the project.
>
> Then we might as well treat the build-file-as-project-root-marker and
> build-file-as-source-of-build-tasks as two unrelated things, in the API. And in
> certain cases both implementations can delegate to the same code.
>
>> For your example above someone who is familiar with those tools would
>> pick the best way to detect a project (maybe by build system like ant,
>> or by some other means) and also pick the best way to extract whatever
>> data is needed, such as a command to pass to 'compile', and hopefully a
>> classpath.  It might be 6 independent types that share a lot of code or
>> baseclasses, or maybe one hybrid. I don't think it really matters.
>
> It matters if to *really* add support for a new build tool, the author has to
> add X new project definitions.
>
IIUC someone developing an EDE support (a plugin?) for a build tool can provide 
as many as s/he wants, right? For example for a build tool a developer may 
provide only two definitions: clean and build.

I think that two more features would be helpful:
- composition of tasks (e.g. run "clean" before "build")
- allow user to add own tasks (could be customizations of default tasks, like 
"compile --with-debug-symbols").

Correct me if I'm wrong, but EDE (xref and project.el) are open for 
modifications, right? It not "take it as it is or leave it".

BTW Why EDE (and CEDET) are in two different repositories (and projects?).
One is in Emacs sources, another here http://sourceforge.net/projects/cedet/ ?

Re: IDE

[Dmitry Gutov]

On 10/16/2015 10:39 AM, Przemysław Wojnowski wrote:

> IIUC someone developing an EDE support (a plugin?) for a build tool can
> provide as many as s/he wants, right? For example for a build tool a
> developer may provide only two definitions: clean and build.

No, they'll have to extend every relevant project type (thus creating 
new types), and in each of them, define clean and build. Or something 
along these lines.

Any user-defined types will miss out, because the author of the build 
tool support is not aware of them.

> I think that two more features would be helpful:
> - composition of tasks (e.g. run "clean" before "build")

Don't build tools do that?

> - allow user to add own tasks (could be customizations of default tasks,
> like "compile --with-debug-symbols").

The command line could be made editable when the "call build tool" 
command is invoked with a prefix.

> Correct me if I'm wrong, but EDE (xref and project.el) are open for
> modifications, right? It not "take it as it is or leave it".

Probably. Not sure what you're really asking. For modifications by whom 
and how?

> BTW Why EDE (and CEDET) are in two different repositories (and projects?).
> One is in Emacs sources, another here
> http://sourceforge.net/projects/cedet/ ?

CEDET is in Emacs sources as well (although is has some omissions). See 
lisp/cedet.

Re: IDE

[Przemysław Wojnowski]

W dniu 16.10.2015 o 12:27, Dmitry Gutov pisze:
> On 10/16/2015 10:39 AM, Przemysław Wojnowski wrote:
>
>> IIUC someone developing an EDE support (a plugin?) for a build tool can
>> provide as many as s/he wants, right? For example for a build tool a
>> developer may provide only two definitions: clean and build.
>
> No, they'll have to extend every relevant project type (thus creating new
> types), and in each of them, define clean and build. Or something along these
> lines.
>
> Any user-defined types will miss out, because the author of the build tool
> support is not aware of them.
IIUC you suggest to use the Bridge Pattern to separate Project Type and Build 
Tool abstractions and allow to exchange them independently, right?
IMHO this is good design and maybe EDE is already built is such way (I'll have
to use and read EDE first to get better understanding of it.)

(One thing Design Patterns give, except solutions to common design problems, is 
a common vocabulary to facilitate communication. So instead of writing 
elaborates on design, one can simply say "We could apply The Bridge between X 
and Y abstractions", etc.)

>> I think that two more features would be helpful:
>> - composition of tasks (e.g. run "clean" before "build")
>
> Don't build tools do that?
Some do. But even then a user may be willing to run them in a different way.
For example Maven doesn't run "clean" phase by default, when one runs other
phases (compile/package/install), so if an user wants to run "clean install"
she has to execute two separate commands. But that can be easily defined as one
compound command (clean-compile).

BTW in the Command Pattern, commands implement a common interface, so an
executor doesn't even know what is executed, except that it is a command, which
can be a Composite (pattern) Command that implements the same interface.

>> - allow user to add own tasks (could be customizations of default tasks,
>> like "compile --with-debug-symbols").
>
> The command line could be made editable when the "call build tool" command is
> invoked with a prefix.
Yes, that would be a good addition too. But sooner or later a use has a
workflow, a set of commands that she runs every time and it would be good to
allow to define in a separate, easily accessible command.

>> Correct me if I'm wrong, but EDE (xref and project.el) are open for
>> modifications, right? It not "take it as it is or leave it".
>
> Probably. Not sure what you're really asking. For modifications by whom and how?
By contributors. Sometime projects have very restrictive attitude towards
new contributions and just reject most of proposals if they are not perfect up
front and doesn't solve all their problems at once.

>> BTW Why EDE (and CEDET) are in two different repositories (and projects?).
>> One is in Emacs sources, another here
>> http://sourceforge.net/projects/cedet/ ?
>
> CEDET is in Emacs sources as well (although is has some omissions). See lisp/cedet.
Yes, so when one would like change EDE to, lets say, apply the Bridge pattern,
which one should be used?

Re: IDE

[Dmitry Gutov]

On 10/16/2015 02:17 PM, Przemysław Wojnowski wrote:

> IIUC you suggest to use the Bridge Pattern to separate Project Type and
> Build Tool abstractions and allow to exchange them independently, right?

Possibly. But I'm not sure yet how it'll end up looking in Elisp, and in 
the API. For all I know, the Project Type doesn't need to reference the 
Build Tool at all, in the general case. Certain commands would just 
reference both.

So we might be served better with separate notions of "current project" 
and "current build tool".

> (One thing Design Patterns give, except solutions to common design
> problems, is a common vocabulary to facilitate communication. So instead
> of writing elaborates on design, one can simply say "We could apply The
> Bridge between X and Y abstractions", etc.)

Yes and no. Details matter.

> For example Maven doesn't run "clean" phase by default, when one runs other
> phases (compile/package/install), so if an user wants to run "clean
> install"
> she has to execute two separate commands. But that can be easily defined
> as one
> compound command (clean-compile).

It should be easy to write as a user-defined command anyway.

> BTW in the Command Pattern, commands implement a common interface, so an
> executor doesn't even know what is executed, except that it is a
> command, which
> can be a Composite (pattern) Command that implements the same interface.

I know all these words as well. :) If you'd like a stab at the 
implementation (or just defining the API), be my guest.

If you'd like to use composite commands, my first question would be how 
a user would form them (that requires a UI). I think at this stage we 
should just be satisfied that this is doable, and can be delayed almost 
indefinitely (that's the beauty of the Composite pattern).

> Yes, that would be a good addition too. But sooner or later a use has a
> workflow, a set of commands that she runs every time and it would be
> good to
> allow to define in a separate, easily accessible command.

a) We can have command history, b) The user could define new commands. 
Certain tools could provide some pre-defined ones as well.

> By contributors. Sometime projects have very restrictive attitude towards
> new contributions and just reject most of proposals if they are not
> perfect up
> front and doesn't solve all their problems at once.

We try to "hone" the proposals to our general liking. On the flip side, 
that can result in long discussions and, at times, contributor 
dissatisfaction.

>> CEDET is in Emacs sources as well (although is has some omissions).
>> See lisp/cedet.
> Yes, so when one would like change EDE to, lets say, apply the Bridge
> pattern,
> which one should be used?

I'll let Eric answer this one.

Re: IDE

[Lluís]

Przemysław Wojnowski writes:

> W dniu 16.10.2015 o 12:27, Dmitry Gutov pisze:
>> On 10/16/2015 10:39 AM, Przemysław Wojnowski wrote:
>> 
>>> IIUC someone developing an EDE support (a plugin?) for a build tool can
>>> provide as many as s/he wants, right? For example for a build tool a
>>> developer may provide only two definitions: clean and build.
>> 
>> No, they'll have to extend every relevant project type (thus creating new
>> types), and in each of them, define clean and build. Or something along these
>> lines.
>> 
>> Any user-defined types will miss out, because the author of the build tool
>> support is not aware of them.
> IIUC you suggest to use the Bridge Pattern to separate Project Type and Build
> Tool abstractions and allow to exchange them independently, right?
> IMHO this is good design and maybe EDE is already built is such way (I'll have
> to use and read EDE first to get better understanding of it.)

Hmmm, I think I should read about that pattern more closely, but AFAIK this is
not what EDE uses. EDE is built around a class hierarchy of ede-project that
provides all the information and actions you might need for each type of project
and all possible operations and information sources on it.

But this bridge pattern does seem to match what I had in mind, though (I made up
the names just now):

* service-type: A type of service (one instance per type), like build, find root
  directory, include search paths, compilation flags, etc.

* service: A specific implementation of a service type, like using vc-dir for
  finding the root directory of a project.

* service-collection: A set of service implementations of the same service
  type. This allows aggregating results from each service into a single answer.

* project-type: A type of project defines the service collections it provides
  for each service type it knows about. An example could be an Emacs project
  (knows how to auto-detect it, things like the build system it uses, etc.), and
  Android project, a Linux kernel project, an autotools project, etc.

* project: A specific project built from a combination of project types. For
  example, an autotools project with some customized project type that overrides
  some of it. This would be what other tools use to interact with each of the
  services supported by the project.

This scheme is something I've had on the back of my mind for quite some time in
order to make EDE easier to develop for and for users to add persistent
customizations on top of existing project knowledge.


Thanks,
  Lluis

-- 
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth

Re: IDE

[Dmitry Gutov]

On 10/16/2015 07:47 PM, Lluís wrote:

> * service-type: A type of service (one instance per type), like build, find root
>    directory, include search paths, compilation flags, etc.
>
> * service: A specific implementation of a service type, like using vc-dir for
>    finding the root directory of a project.

This sounds good. The implementations could be different (e.g. Java-ish 
Service Locator [0]), but in the absence of other requirements the 
closest Emacs idiomatic structure would be a hook (list of functions) 
for each service type. Like project-find-functions, for example.

Note that the objects the latter currently returns provide both "find 
root" and "search path" services. Thus far it seemed to make sense. Can 
one, and would one want to create different search-path services working 
with the same find-root service? That's an interesting question, and I'd 
love to see a practical example.

More generally, if a package P registers services A, B and C, if B needs 
some information from A, would we expect them to go through P-internal 
channels, or the public interface? For instance, most services might 
like to know the project root (let's call that service type R).

We could standardize the dependencies like that. In the simplest case, 
all services depend on project-root (except for itself), and we'll pass 
the current project-root instance to each functions in their hooks. Then 
an element of service-b-functions will only return non-nil if that 
implementation of B can work with the given kind of project-root (or 
simply "project"; we might include some more info into that instance as 
well).

On the flip side, if there are no B implementations that can work with 
the currently detected R service r1, but there's a certain b2 in 
service-b-functions that would work with r2 (currently shadowed by r1), 
we may be missing out.

> * service-collection: A set of service implementations of the same service
>    type. This allows aggregating results from each service into a single answer.

I suppose you can get this from the same service-b-functions hook, if 
you treat it in a different way.

> * project-type: A type of project defines the service collections it provides
>    for each service type it knows about. An example could be an Emacs project
>    (knows how to auto-detect it, things like the build system it uses, etc.), and
>    Android project, a Linux kernel project, an autotools project, etc.

Having to declare each combination of service-types that a project can 
provide still seems unnecessarily limiting (and tedious). But that's 
what project-types would be for, right?

How would a client even use those declarations? If we were to go that 
way, it'd be better to simply have a project instance have a method that 
returns a list of all services it supports. Or, even simpler, return a 
nil or raise a not-implemented error in each of the generic methods it 
doesn't implement.

Next, suppose a project instance can tell whether it provides a "build 
tool" service. What does a "call build task" command does? Looks for the 
current project and gives up if it doesn't provide the required service, 
or specifically asks the locator for a project implementation that does 
provide that service?

In the latter case, we potentially buy more functionality, at the cost 
of having different commands disagree about what the current project 
actually is.

> Thanks,
>    Lluis
>

[0] http://martinfowler.com/articles/injection.html#ADynamicServiceLocator

Re: IDE

[Lluís]

Dmitry Gutov writes:

> On 10/16/2015 07:47 PM, Lluís wrote:
>> * service-type: A type of service (one instance per type), like build, find root
>> directory, include search paths, compilation flags, etc.
>> 
>> * service: A specific implementation of a service type, like using vc-dir for
>> finding the root directory of a project.

> This sounds good. The implementations could be different (e.g. Java-ish Service
> Locator [0]), but in the absence of other requirements the closest Emacs
> idiomatic structure would be a hook (list of functions) for each service
> type. Like project-find-functions, for example.

> Note that the objects the latter currently returns provide both "find root" and
> "search path" services. Thus far it seemed to make sense. Can one, and would one
> want to create different search-path services working with the same find-root
> service? That's an interesting question, and I'd love to see a practical
> example.

I didn't think of the "find path" functionality, but it does seem to make sense
to fold it into the "root path" service type.


> More generally, if a package P registers services A, B and C, if B needs some
> information from A, would we expect them to go through P-internal channels, or
> the public interface? For instance, most services might like to know the project
> root (let's call that service type R).

> We could standardize the dependencies like that. In the simplest case, all
> services depend on project-root (except for itself), and we'll pass the current
> project-root instance to each functions in their hooks. Then an element of
> service-b-functions will only return non-nil if that implementation of B can
> work with the given kind of project-root (or simply "project"; we might include
> some more info into that instance as well).

> On the flip side, if there are no B implementations that can work with the
> currently detected R service r1, but there's a certain b2 in service-b-functions
> that would work with r2 (currently shadowed by r1), we may be missing out.

Exactly. I started prototyping a few interfaces where all requests go through
the public interfaces. A project-type acts similarly to your "service locator"
link, and projects are simply a service locator that manage multiple
project-type instances grouped by the project.

More specifically, I was thinking about the following for deciding how to manage
results when information comes from different service implementations given a
service-type: for each public method of a service type, have a prioritized list
of method implementations, plus a property that tells us how to merge their
results. Possible properties would be returning the first non-error result from
the possible method instances or returning a concatenation of all non-error
results from all method instances.


>> * service-collection: A set of service implementations of the same service
>> type. This allows aggregating results from each service into a single answer.

> I suppose you can get this from the same service-b-functions hook, if you treat
> it in a different way.

While prototyping it, I actually decided it makes sense to implement this
functionality on the service-type. It should know how to merge results from
multiple services, and every service must declare what service-type it provides.


>> * project-type: A type of project defines the service collections it provides
>> for each service type it knows about. An example could be an Emacs project
>> (knows how to auto-detect it, things like the build system it uses, etc.), and
>> Android project, a Linux kernel project, an autotools project, etc.

> Having to declare each combination of service-types that a project can provide
> still seems unnecessarily limiting (and tedious). But that's what project-types
> would be for, right?

Exactly. My idea was that common project types would be written once declaring
all the service types they provide and through what service
implementations. In order to make projects easily customizable, there should
also be a project-type that can be dynamically constructed from some user
settings (some call in init.el or values on directory variables). Then you
simply overlay the custom project-type on top of some other project-type that is
auto-detected or specified by the user.


> How would a client even use those declarations? If we were to go that way, it'd
> be better to simply have a project instance have a method that returns a list of
> all services it supports. Or, even simpler, return a nil or raise a
> not-implemented error in each of the generic methods it doesn't implement.

I'm not sure I follow, but maybe I responded you on my previous paragraphs.


> Next, suppose a project instance can tell whether it provides a "build tool"
> service. What does a "call build task" command does? Looks for the current
> project and gives up if it doesn't provide the required service, or specifically
> asks the locator for a project implementation that does provide that service?

I'd say both. My idea is that all project-types provide an auto-detection
function; given a path, tell me if it conforms to this project-type. Then, if we
do not know of any project instance for that path, build a new one that contains
instances of all project-types that match with it.


> In the latter case, we potentially buy more functionality, at the cost of having
> different commands disagree about what the current project actually is.

> [0] http://martinfowler.com/articles/injection.html#ADynamicServiceLocator

Building a project from all matching project-types provides a clean solution to
this. But to be honest, I'm not sure if it makes much sense beyond taking a
single auto-detected project type plus an optional overlaid project-type that
contains user customizations.

For example, we could have a fallback "project-type-generic", a makefile-based
"project-type-make" and a very specific "project-type-linux". Should we overlay
all of them or just take the most "specific" one? (aka linux).

In the overlaying case, there should be a way to override decisions from other
project-types. This can get hairy, since in the beginning I said that each
per-service method implementation of a service-type method follows some property
(e.g., first match or concatenate), so now we want to override this. Should the
override be per method? Per service? At the entire project-type level?

In the second case, we can simply assign some priority number and take the
highest-priority project-type, and overlay the user-customizable project-type as
a special case.

I really am not sure about the project-type and project part.


Cheers,
  Lluis

-- 
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth

Re: IDE

[Dmitry Gutov]

On 10/17/2015 08:18 PM, Lluís wrote:

> I didn't think of the "find path" functionality, but it does seem to make sense
> to fold it into the "root path" service type.

You mean "find search path", right? Another name if it is "include 
path", and that's different from the executable path.

I think I've formed an idea how separating it from the find-root(s) 
service would be useful. Currently, we have this awkward 
project-search-path-function, which can be defined in a major mode (we 
want this kind of functionality to be major mode-providable).

But if it were a separate service, that would be tidier. Introduce 
project-search-path-functions, which would take the current project as 
an argument and return an object that contains the search path (and, as 
a bonus, maybe knows how to resolve a relative import string).

Then ruby-mode won't have to define a separate project type. It'll just 
add to project-search-path-functions that checks that there is a Gemfile 
somewhere above the current directory and below the project root, and 
use the information from it. The fact that returned object can also 
resolve imports is nice, because I wasn't sure where to put that behavior.

In general, though, this might be problematic when a language L will 
simply read its search path from an environment variable, and doesn't 
require a presence of any "bundle" file. Does it provide its search-path 
to absolutely any project? We might need a "list of project languages" 
accessor on the "root" service instances. User-customizable, of course.

>> On the flip side, if there are no B implementations that can work with the
>> currently detected R service r1, but there's a certain b2 in service-b-functions
>> that would work with r2 (currently shadowed by r1), we may be missing out.
>
> Exactly. I started prototyping a few interfaces where all requests go through
> the public interfaces. A project-type acts similarly to your "service locator"
> link, and projects are simply a service locator that manage multiple
> project-type instances grouped by the project.

You mean different project-types are like entirely different collections 
of services? Why call them project-types at all? I'm lost.

Isn't a project a collection of services (of different service-types)?

> More specifically, I was thinking about the following for deciding how to manage
> results when information comes from different service implementations given a
> service-type: for each public method of a service type, have a prioritized list
> of method implementations, plus a property that tells us how to merge their
> results.

We're unlikely to have too many service-types. No need for a property, 
merging could be done by a respective utility function (like 
`project-current', but for other services). Priority comes from the 
order of the elements in a hook.

> Possible properties would be returning the first non-error result from
> the possible method instances or returning a concatenation of all non-error
> results from all method instances.

Those are the two main options. Not sure if we'll ever really need 
anything else.

> While prototyping it, I actually decided it makes sense to implement this
> functionality on the service-type. It should know how to merge results from
> multiple services, and every service must declare what service-type it provides.

How about we reframe the description in terms of hooks? service-type is 
a hook variable, services are elements in it. You can call the elements 
of a hook in a sequence until one returns non-nil, or you can call all 
of them and combine.

>> Having to declare each combination of service-types that a project can provide
>> still seems unnecessarily limiting (and tedious). But that's what project-types
>> would be for, right?
>
> Exactly. My idea was that common project types would be written once declaring
> all the service types they provide and through what service
> implementations.

So, why?

Take a look at xref-find-regexp. It looks up the current project, takes 
its roots (service 1) and its search-path (service 2). If either of the 
services can be undefined in the current project, what would 
xref-find-regexp be supposed to do?

And what is the use of several common project types? Could you give an 
example?

> I'm not sure I follow, but maybe I responded you on my previous paragraphs.

How would a random command that needs certain info from services x, y 
and z, use this API?

>> Next, suppose a project instance can tell whether it provides a "build tool"
>> service. What does a "call build task" command does? Looks for the current
>> project and gives up if it doesn't provide the required service, or specifically
>> asks the locator for a project implementation that does provide that service?
>
> I'd say both. My idea is that all project-types provide an auto-detection
> function; given a path, tell me if it conforms to this project-type.

In Emacs parlance, a "path" is a list of directories (see exec-path or 
compilation-search-path).

The call-build-type needs the build-tool service. What project-type does 
it use, and why?

> Then, if we
> do not know of any project instance for that path, build a new one that contains
> instances of all project-types that match with it.

What if we already have a project instance for that directory, but it 
doesn't provide that service?

>> [0] http://martinfowler.com/articles/injection.html#ADynamicServiceLocator
>
> Building a project from all matching project-types provides a clean solution to
> this. But to be honest, I'm not sure if it makes much sense beyond taking a
> single auto-detected project type plus an optional overlaid project-type that
> contains user customizations.

I don't think user customizations should be a project-type. Rather, they 
can be applied via .dir-locals.el, inside each service-type utility 
function.

> For example, we could have a fallback "project-type-generic", a makefile-based
> "project-type-make" and a very specific "project-type-linux". Should we overlay
> all of them or just take the most "specific" one? (aka linux).

The one that comes first in the hook. Hook entries from minor modes 
usually come before the hook entries from major modes, and it's user's 
responsibility not to enable several minor modes at once that share the 
same responsibility.

Re: IDE

[Lluís]

Dmitry Gutov writes:

> On 10/17/2015 08:18 PM, Lluís wrote:
>> I didn't think of the "find path" functionality, but it does seem to make sense
>> to fold it into the "root path" service type.

> You mean "find search path", right? Another name if it is "include path", and
> that's different from the executable path.

No, I mean something as simple as concatenating the project's root with a
non-absolute path given as an argument. Search or include paths are specific to
the organization of the project at hand (e.g., on a C project not every
directory should be searched for an include), but could be implemented on top of
the "find path" functionality.


> I think I've formed an idea how separating it from the find-root(s) service
> would be useful. Currently, we have this awkward project-search-path-function,
> which can be defined in a major mode (we want this kind of functionality to be
> major mode-providable).

> But if it were a separate service, that would be tidier. Introduce
> project-search-path-functions, which would take the current project as an
> argument and return an object that contains the search path (and, as a bonus,
> maybe knows how to resolve a relative import string).

> Then ruby-mode won't have to define a separate project type. It'll just add to
> project-search-path-functions that checks that there is a Gemfile somewhere
> above the current directory and below the project root, and use the information
> from it. The fact that returned object can also resolve imports is nice, because
> I wasn't sure where to put that behavior.

> In general, though, this might be problematic when a language L will simply read
> its search path from an environment variable, and doesn't require a presence of
> any "bundle" file. Does it provide its search-path to absolutely any project? We
> might need a "list of project languages" accessor on the "root" service
> instances. User-customizable, of course.

That's one of my concerns with the project concept. As you pointed out,
behaviour can change, for example, based on the mode of the file you're working
on. This can be the case for projects that contain code in multiple languages,
so that each can be treated slightly differently.

The question is thus, for example, how do we fit the major-mode variable into
this scheme? The problem becomes multi-dimensional, and I'm not sure how to
specify that. Should some service-types be dependant on the major-mode, such
that a project can have mutiple services for the same type but for different
major-modes?

If that sounds reasonable, then why not also account for the path of the current
file? The same project might have different file search paths for different
files...


>>> On the flip side, if there are no B implementations that can work with the
>>> currently detected R service r1, but there's a certain b2 in service-b-functions
>>> that would work with r2 (currently shadowed by r1), we may be missing out.
>> 
>> Exactly. I started prototyping a few interfaces where all requests go through
>> the public interfaces. A project-type acts similarly to your "service locator"
>> link, and projects are simply a service locator that manage multiple
>> project-type instances grouped by the project.

> You mean different project-types are like entirely different collections of
> services? Why call them project-types at all? I'm lost.

> Isn't a project a collection of services (of different service-types)?

Both are collections of services. I was thinking of the project as a collection
of project-types too, so that you can combine multiple of them.


[...]
>> While prototyping it, I actually decided it makes sense to implement this
>> functionality on the service-type. It should know how to merge results from
>> multiple services, and every service must declare what service-type it provides.

> How about we reframe the description in terms of hooks? service-type is a hook
> variable, services are elements in it. You can call the elements of a hook in a
> sequence until one returns non-nil, or you can call all of them and combine.

I was thinking of service-types as providing multiple methods (hook variables),
one per function it exposes.


>>> Having to declare each combination of service-types that a project can provide
>>> still seems unnecessarily limiting (and tedious). But that's what project-types
>>> would be for, right?
>> 
>> Exactly. My idea was that common project types would be written once declaring
>> all the service types they provide and through what service
>> implementations.

> So, why?

> Take a look at xref-find-regexp. It looks up the current project, takes its
> roots (service 1) and its search-path (service 2). If either of the services can
> be undefined in the current project, what would xref-find-regexp be supposed to
> do?

> And what is the use of several common project types? Could you give an example?

Suppose you have project-type T1 that provides service 1 (S1) but not S2. As you
said, xref-find-regexp will fail.

Now, there can be some best-effort project-type T0 that provides a brain-dead S2
that looks at all sub-directories. If you create your project overlaying T0 and
T1, you can always get some best-effort functionality in case you're not using
any well-known project type.

Note that some other project-type might provide a more intelligent S2 that, for
example, takes information from a compilation database.


>> I'm not sure I follow, but maybe I responded you on my previous paragraphs.

> How would a random command that needs certain info from services x, y and z, use
> this API?

Does the previous paragraph answer it now?


>>> Next, suppose a project instance can tell whether it provides a "build tool"
>>> service. What does a "call build task" command does? Looks for the current
>>> project and gives up if it doesn't provide the required service, or specifically
>>> asks the locator for a project implementation that does provide that service?
>> 
>> I'd say both. My idea is that all project-types provide an auto-detection
>> function; given a path, tell me if it conforms to this project-type.

> In Emacs parlance, a "path" is a list of directories (see exec-path or
> compilation-search-path).

> The call-build-type needs the build-tool service. What project-type does it use,
> and why?

Suppose these three project-types:

* project-type-scons
* project-type-make
* project-type-linux

Each provides only the build-tool service, and you open a file from the linux
kernel. The type project-type-scons does not apply, but the other two do. If we
can assign some priority to them, we will end up with project-type-linux.


>> Then, if we
>> do not know of any project instance for that path, build a new one that contains
>> instances of all project-types that match with it.

> What if we already have a project instance for that directory, but it doesn't
> provide that service?

I'd say there's no way out of this one. What troubles me the most is nesting,
where each sub-directory might need to behave differently (e.g., have files for
different programming languages). Would then each file or directory have a
potentially different project instance? Or should the project abstraction
account for different services on different paths?


>>> [0] http://martinfowler.com/articles/injection.html#ADynamicServiceLocator
>> 
>> Building a project from all matching project-types provides a clean solution to
>> this. But to be honest, I'm not sure if it makes much sense beyond taking a
>> single auto-detected project type plus an optional overlaid project-type that
>> contains user customizations.

> I don't think user customizations should be a project-type. Rather, they can be
> applied via .dir-locals.el, inside each service-type utility function.

I'm still not sure about this one.


>> For example, we could have a fallback "project-type-generic", a makefile-based
>> "project-type-make" and a very specific "project-type-linux". Should we overlay
>> all of them or just take the most "specific" one? (aka linux).

> The one that comes first in the hook. Hook entries from minor modes usually come
> before the hook entries from major modes, and it's user's responsibility not to
> enable several minor modes at once that share the same responsibility.

I'd prefer the system to be able to auto-detect the proper type whenever
possible, instead of relying on the user establishing order for each possible
directory.


Thanks,
  Lluis

-- 
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth

Re: IDE

[Dmitry Gutov]

On 10/18/2015 09:21 PM, Lluís wrote:

> No, I mean something as simple as concatenating the project's root with a
> non-absolute path given as an argument. Search or include paths are specific to
> the organization of the project at hand (e.g., on a C project not every
> directory should be searched for an include), but could be implemented on top of
> the "find path" functionality.

Then, UUIC, maybe that would be called "find relative path". Similar to 
https://atom.io/docs/api/v1.0.19/Project#instance-relativizePath

> That's one of my concerns with the project concept. As you pointed out,
> behaviour can change, for example, based on the mode of the file you're working
> on. This can be the case for projects that contain code in multiple languages,
> so that each can be treated slightly differently.

We can't rely on the current file being the deciding factor, however. At 
least some users would expect that if we're in a given project (say, in 
an open Dired buffer, or in a YAML config, or in a Compilation buffer), 
and the project contains Ruby files, the search for references to `each' 
would still use the Ruby search path.

> The question is thus, for example, how do we fit the major-mode variable into
> this scheme? The problem becomes multi-dimensional, and I'm not sure how to
> specify that. Should some service-types be dependant on the major-mode, such
> that a project can have mutiple services for the same type but for different
> major-modes?

The language can be passed as a second argument to 
project-search-path-functions elements. But then the project would need 
to know the set of languages to pass in.

As long as we prohibit search-path from depending on the current buffer, 
any parameters need to passed in explicitly (except one, see below).

> If that sounds reasonable, then why not also account for the path of the current
> file? The same project might have different file search paths for different
> files...

Yes. Add the "directory" parameter (or pass it implicitly by binding 
`default-directory')? The list is getting longer...

> I was thinking of service-types as providing multiple methods (hook variables),
> one per function it exposes.

Nothing can contain multiple hook variables. Hook variables are global. 
The values returned from them, however, could define several methods.

> Now, there can be some best-effort project-type T0 that provides a brain-dead S2
> that looks at all sub-directories. If you create your project overlaying T0 and
> T1, you can always get some best-effort functionality in case you're not using
> any well-known project type.

At which point does a project become "overlaying" T0 and T1? Would that 
be a tricky configuration a user would have to perform in a file inside 
the root directory of each of their projects?

> Note that some other project-type might provide a more intelligent S2 that, for
> example, takes information from a compilation database.

Right.

>> How would a random command that needs certain info from services x, y and z, use
>> this API?
>
> Does the previous paragraph answer it now?

Not really. I "create" a project? Could you rephrase that in terms of 
the project.el API?

Here's an example: the command calls `project-current', which returns 
the current detected value of project. Then it passes it to 
`project-current-search-path', which consults 
project-search-path-functions and returns an object describing the 
search-path, which the command uses with another method.

Something will "create" a project (the user, somehow, or a minor mode), 
but that's of no interest to the API.

> What troubles me the most is nesting,
> where each sub-directory might need to behave differently (e.g., have files for
> different programming languages).

I worry more about directories with files for different programming 
languages all inside them. At some point, some code will either have to 
prompt the user, guess somehow, or ask for search-path for all the 
languages, and then combine the results.

> Would then each file or directory have a
> potentially different project instance? Or should the project abstraction
> account for different services on different paths?

Maybe we should decide that search-path must be the same for all 
directories inside a project. Otherwise, we'd get different results for 
commands called from different directories, sometimes with no apparent 
reason.

Still unsure about it.

>> The one that comes first in the hook. Hook entries from minor modes usually come
>> before the hook entries from major modes, and it's user's responsibility not to
>> enable several minor modes at once that share the same responsibility.
>
> I'd prefer the system to be able to auto-detect the proper type whenever
> possible, instead of relying on the user establishing order for each possible
> directory.

How would having numeric priorities help? They'd still be static and 
apply to all directories.

Also note that for project types supplied for Emacs, we'd be able to put 
them into hooks in the proper order. With third-party code, however, the 
order might be wrong sometimes, but we also have no guarantee that their 
authors would assign correct priorities.

Re: IDE

[Lluís]

Dmitry Gutov writes:

> On 10/18/2015 09:21 PM, Lluís wrote:
[...]
>> If that sounds reasonable, then why not also account for the path of the current
>> file? The same project might have different file search paths for different
>> files...

> Yes. Add the "directory" parameter (or pass it implicitly by binding
> `default-directory')? The list is getting longer...

That's the point I wanted to raise. How flexible should the underlying
infrastructure be? And based on what should it be flexible?


>> I was thinking of service-types as providing multiple methods (hook variables),
>> one per function it exposes.

> Nothing can contain multiple hook variables. Hook variables are global. The
> values returned from them, however, could define several methods.

I think I did not explain myself well. I meant having one hook variable for each
public method of each service type.


>> Now, there can be some best-effort project-type T0 that provides a brain-dead S2
>> that looks at all sub-directories. If you create your project overlaying T0 and
>> T1, you can always get some best-effort functionality in case you're not using
>> any well-known project type.

> At which point does a project become "overlaying" T0 and T1? Would that be a
> tricky configuration a user would have to perform in a file inside the root
> directory of each of their projects?

If every project-type T* has a probing function that tells you if it's
applicable to your current project, then you simply overlay all project types
that match (or support) your project.

IMHO, how to expose this to the user is a question that needs answering later


[...]
>>> How would a random command that needs certain info from services x, y and z, use
>>> this API?
>> 
>> Does the previous paragraph answer it now?

> Not really. I "create" a project? Could you rephrase that in terms of the
> project.el API?

> Here's an example: the command calls `project-current', which returns the
> current detected value of project. Then it passes it to
> `project-current-search-path', which consults project-search-path-functions and
> returns an object describing the search-path, which the command uses with
> another method.

> Something will "create" a project (the user, somehow, or a minor mode), but
> that's of no interest to the API.

Ok, so the first time you call `project-current', the system can check all
project-types it knows about and see which ones apply to your file. From that a
project object object is created overlaying the matched project-types. Next time
you request `project-current' a cached value can be returned, or maybe looking
at the value of some parent directory.


>> What troubles me the most is nesting,
>> where each sub-directory might need to behave differently (e.g., have files for
>> different programming languages).

> I worry more about directories with files for different programming languages
> all inside them. At some point, some code will either have to prompt the user,
> guess somehow, or ask for search-path for all the languages, and then combine
> the results.

>> Would then each file or directory have a
>> potentially different project instance? Or should the project abstraction
>> account for different services on different paths?

> Maybe we should decide that search-path must be the same for all directories
> inside a project. Otherwise, we'd get different results for commands called from
> different directories, sometimes with no apparent reason.

> Still unsure about it.

Ok, so it seems that we have two kinds of operations:

* global: an operation that behaves the same regardless of where you're
  performing the request from

* local: an operation that can behave differently depending on the path you're
  performing the request from, depending on the major-mode, depending on some
  environment variable, or depending on some other arbitrary condition

Would it ever make sense for a service method to provide both kinds of
operations? (i.e., discriminate through an argument)


>>> The one that comes first in the hook. Hook entries from minor modes usually come
>>> before the hook entries from major modes, and it's user's responsibility not to
>>> enable several minor modes at once that share the same responsibility.
>> 
>> I'd prefer the system to be able to auto-detect the proper type whenever
>> possible, instead of relying on the user establishing order for each possible
>> directory.

> How would having numeric priorities help? They'd still be static and apply to
> all directories.

> Also note that for project types supplied for Emacs, we'd be able to put them
> into hooks in the proper order. With third-party code, however, the order might
> be wrong sometimes, but we also have no guarantee that their authors would
> assign correct priorities.

That's true, the user always needs some "backdoor" to customize behavior, since
there's no global behavior that can fit all cases.


Cheers,
  Lluis

-- 
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth

Re: IDE

[Dmitry Gutov]

On 10/19/2015 04:04 PM, Lluís wrote:

> That's the point I wanted to raise. How flexible should the underlying
> infrastructure be? And based on what should it be flexible?

Well, yeah. These are the practical questions, decisions on which will 
shape the result.

Having limitations isn't bad, as long as the user will still be able to 
enjoy useful behavior, one way or another.

Option one: we allow search-path to depend on default-directory (in 
addition to the current project). Then the user can employ VC as a 
backend for projects, and yet have several different applications inside 
that repository. I'm describing this on the basis of one of the projects 
we have at work.

Option two: we disallow search-path depending on default-directory. 
Then, for the same project, VC can't be the backend, and the "project" 
will have to be split into several (VC can't be used as a backend 
heere). Which will make at least as much sense. It will require a 
special project type for Ruby, based on the presence of Gemfile.

> I think I did not explain myself well. I meant having one hook variable for each
> public method of each service type.

Why aggregate them into service types, then? Just call each "public 
method" (or hook) a separate service.

> If every project-type T* has a probing function that tells you if it's
> applicable to your current project, then you simply overlay all project types
> that match (or support) your project.
>
> IMHO, how to expose this to the user is a question that needs answering later

The "overlaying" process would also need some explanation. And 
consideration.

Suppose there are several implementations of the same service-type, and 
the results they return (like search-path elements) overlap. How would 
you produce the resulting list? (delete-dups (apply #'append ...))?

That would assume that there can't be "bad" (less precise) elements. For 
the "find references" service (if we had one), you would want to only 
keep the precise results, if available, a not mash them together with 
the output of Grep.

> Ok, so the first time you call `project-current', the system can check all
> project-types it knows about and see which ones apply to your file. From that a
> project object object is created overlaying the matched project-types. Next time
> you request `project-current' a cached value can be returned, or maybe looking
> at the value of some parent directory.

Caching is incidental. And I still don't know what "overlaying" is. In 
solid details, I mean.

> Ok, so it seems that we have two kinds of operations:
>
> * global: an operation that behaves the same regardless of where you're
>    performing the request from
>
> * local: an operation that can behave differently depending on the path you're
>    performing the request from, depending on the major-mode, depending on some
>    environment variable, or depending on some other arbitrary condition
>
> Would it ever make sense for a service method to provide both kinds of
> operations? (i.e., discriminate through an argument)

It might. The "find included file" operation would probably want to 
consider the type of the current file.

The environment variable can be looked up from the service itself. 
Arbitrary conditions are fine, as long they're not dependent on the 
current buffer too much.

> That's true, the user always needs some "backdoor" to customize behavior, since
> there's no global behavior that can fit all cases.

The user would always be able to rearrange the elements in a hook, or 
add their own function (service implementation), maybe composing 
existing ones.

Re: IDE

[Lluís]

Dmitry Gutov writes:

> On 10/19/2015 04:04 PM, Lluís wrote:
[...]
>> I think I did not explain myself well. I meant having one hook variable for each
>> public method of each service type.

> Why aggregate them into service types, then? Just call each "public method" (or
> hook) a separate service.

Just a matter of keeping tightly-related functionality together.


[...]
>> Ok, so the first time you call `project-current', the system can check all
>> project-types it knows about and see which ones apply to your file. From that a
>> project object object is created overlaying the matched project-types. Next time
>> you request `project-current' a cached value can be returned, or maybe looking
>> at the value of some parent directory.

> Caching is incidental. And I still don't know what "overlaying" is. In solid
> details, I mean.

Ok, a more specific example. Assume the service-type "service-type-symbols" that
provides a method to jump to symbols, and we're editing sources of the linux
kernel. Now, we have the following service implementations for that
service-type:

* service-symbols-cscope: Use external database.
* service-symbols-wisent: Parse source files, requires an implementation of
  "service-type-search-paths".

And we also have the following project-types:

* project-type-generic:
  * match: Return t if we can detect a project root using VC
  * service-types:
    * service-type-root: service-root-vc

* project-type-generic-c:
  * match: It's unclear when to activate it
  * service-types:
    * service-type-symbols: service-symbols-cscope, service-symbols-wisent

* project-type-linux:
  * match: We see a root directory for a Linux project
  * service-types:
    * service-type-root: service-root-linux
    * service-type-search-paths: service-search-paths-linux

If we assume all project types match, the resulting "overlaid" project for Linux
would have the following service-types:

* service-type-root: service-root-linux
* service-type-search-paths: service-search-paths-linux
* service-type-symbols: service-symbols-cscope, service-symbols-wisent

If we were not editing some Linux directory:

* service-type-root: service-root-vc
* service-type-symbols: service-symbols-cscope


Cheers,
  Lluis

-- 
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth

Re: IDE

[Dmitry Gutov]

On 10/20/2015 03:37 PM, Lluís wrote:

> And we also have the following project-types:
>
> * project-type-generic:
>    * match: Return t if we can detect a project root using VC
>    * service-types:
>      * service-type-root: service-root-vc

I see. This looks more like a classical structure, with one "project" 
objects, two attributes in there, and the second one containing a map 
with list values.

We could indeed interpret those values as hooks, but that question is 
less important.

The main benefit of this structure is familiarity to anyone who's done a 
little OOP.

What I was imagining, since we're talking about services:

project-types:
   - project-type-generic:
     * match
     * root
   - project-type-generic-c:
     * match
     * root

search-paths-types:
   - search-paths-linux (project)
     * search-paths
     * resolve-include-path <-- maybe
   - search-paths-c++ (project)
     * likewise
     * likewise

symbols-list-types:
   - symbols-list-cscope (project, search-paths)
   - symbols-list-wisent (project, search-paths)

The upside is that I can write a new package called magic-8-ball, which 
would add an element at the beginning of symbols-list-types, and that 
element will have a chance to be used in every kind of project. Now just 
the projects that were explicitly written with magic-8-ball in mind. 
That adds flexibility.

The downside is that indirection adds complexity as well, and could turn 
out to be mostly unused.

Re: IDE

[Lluís]

Dmitry Gutov writes:

> On 10/20/2015 03:37 PM, Lluís wrote:
>> And we also have the following project-types:
>> 
>> * project-type-generic:
>> * match: Return t if we can detect a project root using VC
>> * service-types:
>> * service-type-root: service-root-vc

> I see. This looks more like a classical structure, with one "project" objects,
> two attributes in there, and the second one containing a map with list values.

> We could indeed interpret those values as hooks, but that question is less
> important.

> The main benefit of this structure is familiarity to anyone who's done a little
> OOP.

> What I was imagining, since we're talking about services:

> project-types:
>   - project-type-generic:
>     * match
>     * root
>   - project-type-generic-c:
>     * match
>     * root

> search-paths-types:
>   - search-paths-linux (project)
>     * search-paths
>     * resolve-include-path <-- maybe
>   - search-paths-c++ (project)
>     * likewise
>     * likewise

> symbols-list-types:
>   - symbols-list-cscope (project, search-paths)
>   - symbols-list-wisent (project, search-paths)

> The upside is that I can write a new package called magic-8-ball, which would
> add an element at the beginning of symbols-list-types, and that element will
> have a chance to be used in every kind of project. Now just the projects that
> were explicitly written with magic-8-ball in mind. That adds flexibility.

> The downside is that indirection adds complexity as well, and could turn out to
> be mostly unused.

Aha, I see we were imagining different ways of structuring the concepts. What I
don't understand is what you mean with the parenthesis you add to some of the
elements of your "service-type hooks". Are these dependencies between services?

Now, how do you auto-detect what services to use on your currently open project?
I.e., how do you auto-detect what service implementations must be used?

Maybe I wasn't explicit enough in my case, but project-types are the only ones
that provide project detection, and therefore dictate the service
implementations to use on your project.


Cheers,
  Lluis

-- 
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth

Re: IDE

[Dmitry Gutov]

On 10/21/2015 05:40 PM, Lluís wrote:

> Aha, I see we were imagining different ways of structuring the concepts. What I
> don't understand is what you mean with the parenthesis you add to some of the
> elements of your "service-type hooks". Are these dependencies between services?

Yes. And they're also function arguments. So the search-paths services 
would be implemented something like this:

(defun search-paths (project)
   (run-hook-with-args-until-success
    'search-path-functions
    project))

And let the others look like this:

(defun current-project (dir)
   (run-hook-with-args-until-success
    'project-find-functions
    dir))

(defun symbols-list (project search-paths)
   (run-hook-with-args-until-success
    'symbols-list-functions
    project
    search-paths))

Then a command which wants to use the symbols list will have to call all 
three of these functions.

> Now, how do you auto-detect what services to use on your currently open project?
> I.e., how do you auto-detect what service implementations must be used?

Each service implementation decides whether it's eligible, based on the 
PROJECT argument. It can ask the project for its root, or it can even 
check its struct/eieio/etc type, if it only want to work with certain 
kind of projects (for instance, when some project-find-functions and 
symbols-list-functions come from the same package, such as EDE).

> Maybe I wasn't explicit enough in my case, but project-types are the only ones
> that provide project detection, and therefore dictate the service
> implementations to use on your project.

Right. In this case, if I want to provide an search-paths implementation 
(maybe inside ruby-mode package), I also have to create a new project 
implementation, or look for an existing one to hook into somehow (that 
will most likely require some code modification).

Whereas, really, I can provide a rather small search-paths 
implementation that would fit the vast majority of Ruby projects, and 
would only look for Gemfile.lock in a parent directory, and depend on 
its contents (and maybe a few other file that signify the version of 
Ruby that's being used). That would nicely compose with Projectile as 
the project implementation, for instance.

Re: IDE

[Steinar Bang]

>>>>> Dmitry Gutov <dgutov@yandex.ru>:

> Option one: we allow search-path to depend on default-directory (in
> addition to the current project). Then the user can employ VC as a
> backend for projects, and yet have several different applications
> inside that repository. I'm describing this on the basis of one of the
> projects we have at work.

> Option two: we disallow search-path depending on
> default-directory. Then, for the same project, VC can't be the
> backend, and the "project" will have to be split into several (VC
> can't be used as a backend heere). Which will make at least as much
> sense. It will require a special project type for Ruby, based on the
> presence of Gemfile.

With maven, I might do something like this:
 project/
   pom.xml
   .git/
   module1/
     pom.xml
     .git/
   module2/
     pom.xml
     .git/

The same projects in an eclipse workspace, might look something like
this:
 workspace/
   module1/
     pom.xml
     .git/
   module2/
     pom.xml
     .git/

Or perhaps something like this (if the parent also has common settings I
would like easily editable).
 workspace/
   project/
     pom.xml
     .git/
   module1/
     pom.xml
     .git/
   module2/
     pom.xml
     .git/

If the project was to be only edited with emacs, I would go for the top
layout, however if I was to edit the same projects in both emacs and
eclipse, it should handle the latter two layouts as well (though the
bottom one doesn't work too well with command line maven).

Re: IDE

[Dmitry Gutov]

Hi Steinar,

On 10/20/2015 06:23 PM, Steinar Bang wrote:
> With maven, I might do something like this: (A)
>   project/
>     pom.xml
>     .git/
>     module1/
>       pom.xml
>       .git/
>     module2/
>       pom.xml
>       .git/
>
> The same projects in an eclipse workspace, might look something like
> this: (B)
>   workspace/
>     module1/
>       pom.xml
>       .git/
>     module2/
>       pom.xml
>       .git/
>
> Or perhaps something like this (if the parent also has common settings I
> would like easily editable): (C)
>   workspace/
>     project/
>       pom.xml
>       .git/
>     module1/
>       pom.xml
>       .git/
>     module2/
>       pom.xml
>       .git/
>
> If the project was to be only edited with emacs, I would go for the top
> layout, however if I was to edit the same projects in both emacs and
> eclipse, it should handle the latter two layouts as well (though the
> bottom one doesn't work too well with command line maven).

That doesn't help, by itself. Surely we want to support all of these 
directory structures. The question is what each of them would translate 
to in the project API.

Consider A. It could be considered one project, but then it would have 
certain attributes dependent on the current directory. Take classpath, 
for example. Could we consider it to be constant value across the whole 
project, or would we have, for certain operations (like "find the class 
named foo.bar.Bar"), different values of classpath in module1 and 
module2? The build-tool behavior would certainly depend on the current 
directory, but could we say that all other important project attributes 
are kind of the same for project, module1 and module2?

Another option for A is to promote module1 and module2 to whole 
projects. But then, do we track project dependencies now? If I call a 
command "search for occurrences of 'xyz' in project", does it also 
search in module1 and module2? Or similarly for "list all files in 
project", does that include files in module1? That would be useful with 
Helm, for "jump to a project file". Does "list all files in project", 
when called in module1, include files from the parent project, and from 
module2?

B doesn't look too different, except we apparently don't have a 
top-level pom-file.

I don't understand C. Is module1 still inside project? Is it still a 
dependency? Do we treat it differently WRT to questions I've asked for 
the option A?

Re: IDE

[Lluís]

Dmitry Gutov writes:

> Hi Steinar,
> On 10/20/2015 06:23 PM, Steinar Bang wrote:
>> With maven, I might do something like this:
>>  project/
>>    pom.xml
>>    .git/
>>    module1/
>>      pom.xml
>>      .git/
>>    module2/
>>      pom.xml
>>      .git/
>>
>> The same projects in an eclipse workspace, might look something like
>> this:
>>  workspace/
>>    module1/
>>      pom.xml
>>      .git/
>>    module2/
>>      pom.xml
>>      .git/
>>
>> Or perhaps something like this (if the parent also has common settings I
>> would like easily editable).
>>  workspace/
>>    project/
>>      pom.xml
>>      .git/
>>    module1/
>>      pom.xml
>>      .git/
>>    module2/
>>      pom.xml
>>      .git/
>>
>> If the project was to be only edited with emacs, I would go for the top
>> layout, however if I was to edit the same projects in both emacs and
>> eclipse, it should handle the latter two layouts as well (though the
>> bottom one doesn't work too well with command line maven).

> That doesn't help, by itself. Surely we want to support all of these directory
> structures. The question is what each of them would translate to in the project
> API.

> Consider A. It could be considered one project, but then it would have certain
> attributes dependent on the current directory. Take classpath, for
> example. Could we consider it to be constant value across the whole project, or
> would we have, for certain operations (like "find the class named foo.bar.Bar"),
> different values of classpath in module1 and module2? The build-tool behavior
> would certainly depend on the current directory, but could we say that all other
> important project attributes are kind of the same for project, module1 and
> module2?

> Another option for A is to promote module1 and module2 to whole projects. But
> then, do we track project dependencies now? If I call a command "search for
> occurrences of 'xyz' in project", does it also search in module1 and module2? Or
> similarly for "list all files in project", does that include files in module1?
> That would be useful with Helm, for "jump to a project file". Does "list all
> files in project", when called in module1, include files from the parent
> project, and from module2?

> B doesn't look too different, except we apparently don't have a top-level
> pom-file.

> I don't understand C. Is module1 still inside project? Is it still a dependency?
> Do we treat it differently WRT to questions I've asked for the option A?

Ok, so what if we let project-types define project nesting?

Let's suppose example A has a "flat" configuration, while example B has a
nested one:

* service-search-paths-X-Y
  A "search path" service where X tells us whether paths are searched
  recursively, and where searches start at path Y.

* project-type-A:
  * children: nil
  * service-types
    * service-type-search-paths: service-search-paths-t-project/

* project-type-B:
  * children: project-type-B-m1 project-type-B-m2
  * service-types
    * service-type-search-paths: service-search-paths-nil-project/

* project-type-B-m1:
  * children: nil
  * service-types
    * service-type-search-paths: service-search-paths-nil-project/module1/

* project-type-B-m2:
  * children: nil
  * service-types
    * service-type-search-paths: service-search-paths-nil-project/module2/

Note that the same can be applied to things like whether cross-module searches
must work. For example, this would activate them for path searches:

    * service-type-search-paths: service-search-paths-nil-project/module1/ service-search-paths-nil-project/module2/

In most cases, it probably makes more sense to construct this nesting
programmatically by adding some logic during project auto-detection (e.g., read
some configuration file that is part of the project).


Thanks,
  Lluis

-- 
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth

Re: IDE

[Dmitry Gutov]

On 10/21/2015 05:52 PM, Lluís wrote:

>> I don't understand C. Is module1 still inside project? Is it still a dependency?
>> Do we treat it differently WRT to questions I've asked for the option A?
>
> Ok, so what if we let project-types define project nesting?

Every new thing, like projects being allowed to have children (or 
modules; are modules different from projects?), or paths being possibly 
non-recursive, raises complexity of the API, and makes it less 
straightforward to use it.

That's why I asking questions: which commands people would want to see 
implemented, that would consume information about project structure, and 
how they would expect the said commands to behave WRT to nesting, 
submodules, etc.

For example, if when we're working on a submodule we don't *really* need 
to know that we're inside a bigger project (or at least don't need to 
impart that information to most project-related commands), we can avoid 
the notion of nesting in the API, and just ask any project 
implementation to return the "module" we're currently in as the current 
project.

And a lot of languages don't have the same kind of modules that 
Maven-based Java projects use. Would the notion 'children' be only 
useful for Java projects?

> In most cases, it probably makes more sense to construct this nesting
> programmatically by adding some logic during project auto-detection (e.g., read
> some configuration file that is part of the project).

Yes, of course. A project implementation like that would probably read 
pom.xml (or equivalent), and construct the nesting based on that.

Re: IDE

[Lluís]

Dmitry Gutov writes:

> On 10/21/2015 05:52 PM, Lluís wrote:
>>> I don't understand C. Is module1 still inside project? Is it still a dependency?
>>> Do we treat it differently WRT to questions I've asked for the option A?
>> 
>> Ok, so what if we let project-types define project nesting?

> Every new thing, like projects being allowed to have children (or modules; are
> modules different from projects?), or paths being possibly non-recursive, raises
> complexity of the API, and makes it less straightforward to use it.

> That's why I asking questions: which commands people would want to see
> implemented, that would consume information about project structure, and how
> they would expect the said commands to behave WRT to nesting, submodules, etc.

> For example, if when we're working on a submodule we don't *really* need to know
> that we're inside a bigger project (or at least don't need to impart that
> information to most project-related commands), we can avoid the notion of
> nesting in the API, and just ask any project implementation to return the
> "module" we're currently in as the current project.

> And a lot of languages don't have the same kind of modules that Maven-based Java
> projects use. Would the notion 'children' be only useful for Java projects?

That's right. I see "modules" and "projects" as the same thing in terms of
services. The only point where it *might* make sense to expose nesting in the
interface is to define a project that uses the services of some other project in
a parent directory.

Internally, it would probably make sense to be aware of nesting, but I agree
that exposing it on the interface adds complexity that is better avoided.

Cheers,
  Lluis


-- 
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth

Re: IDE

[Steinar Bang]

>>>>> Dmitry Gutov <dgutov@yandex.ru>:

> Consider A. It could be considered one project, but then it would have
> certain attributes dependent on the current directory. Take classpath,
> for example.

(Classpath in maven is actually independent of the directory layout of
the checked out projects, but I won't get into that now)

> Could we consider it to be constant value across the whole project, or
> would we have, for certain operations (like "find the class named
> foo.bar.Bar"), different values of classpath in module1 and module2?

The classpath will be per project/module, and for each project it can
differ for compilation, runtime, test execution.

A *brief* explanation of how the class path is built (for the initiated:
by "identity" below, I mean the "maven coordinates" consisting of
groupId/artifactId/version):
 - Each pom.xml file, have:
  - an identity
  - the identity of a parent pom.xml file
  - a list of dependencies (including the scope of the dependency,
    eg. "test")
 - The combined dependency list of a pom.xml is:
  - The dependencies in the dependency list
  - The transitive dependencies of the dependencies in the dependency
    list
  - The combined dependencies of the parent pom.xml (which includes that
    pom.xml file's dependencies and transitive dependencies and those of
    its parent in turn)
 - Maven will download all dependencies and install them in the maven
   local cache (by default the maven local cache resides in
   $HOME/.m2/repository/ )
 - Maven will create the appropriate classpath for compile, test, run,
   etc. consisting of jar files in the maven local cache

The contents of the maven local cache can be both downloaded
dependencies and artifacts (e.g. jar files) built by other local maven
projects.

(That means that as long as the projects are built in dependency order,
they can in *theory* be be checked out separately and built separately,
parent, and dependencies to other projects can be resolved against the
local cache.  In practice, however, it is simpler to organize the maven
project in the recommended hierarchy and let maven handle built order
and stuff)

> The build-tool behavior would certainly depend on the current
> directory, but could we say that all other important project
> attributes are kind of the same for project, module1 and module2?

Depends on what you mean with important project attributes...?

Dependencies are resolved individually for module1 and module2, but
there can be a common set of dependencies in project (provided "project"
is the parent pom.xml of "module1" and "module2").

If "project" is the parent pom.xml of the two modules, you can set
properties in "project" that are shared by module1 and module2.  You can
also share plugin configurations in this way.

> Another option for A is to promote module1 and module2 to whole
> projects. But then, do we track project dependencies now?

The same way eclipse maven support ("m2e") does, perhaps...?
Even projects that should have been in a hierarchy are checked out
separately, and if "module1" is a dependency of "module2",

Ie. if we have only
 workspace/
   module1/
     pom.xml
     .git/
   module2/
     pom.xml
     .git/

eclipse m2e will still recognize that "module1" contains the dependency
of "module2" and build them in order (they will show up as project
dependencies in the project classpath in eclipse).

This is because each pom.xml file has a (hopefully) unique identity
consisting of groupId/artifactId/version.

> If I call a command "search for occurrences of 'xyz' in project", does
> it also search in module1 and module2? Or similarly for "list all
> files in project", does that include files in module1?

"project" is defined by the existence of a pom.xml file, for both
eclipse and maven, so "project" in eclipse would mean either "module1"
and "module2".

What it would mean if "project" was included into the workspace is a
little more unclear (there is an "enclosing projects" option in eclipse
search, but I don't know what this means").

The default search scope in eclipse is "workspace", which is the
projects seen in eclipse's package explorer.

> That would be useful with Helm, for "jump to a project file". Does
> "list all files in project", when called in module1, include files
> from the parent project, and from module2?

> B doesn't look too different, except we apparently don't have a
> top-level pom-file.

In this case, if the top level pom file isn't a parent of either module
it doesn't have to be there (it could be a build-only file).  If the top
level pom _is_ a dependency of the modules, it could be resolved against
the maven local cache (if not found there, the build would break).

> I don't understand C. Is module1 still inside project? Is it still a
> dependency? Do we treat it differently WRT to questions I've asked for
> the option A?

In eclipse, C would be treated identical to A (and look the same in a
workspace), but A would build from the command line, and C would not
(actually: would be cumbersome to build).

So C is just a theoretical possibility, and could be disregarded (though
I have done similar things during early m2e experimentations).

Re: IDE

[Filipp Gunbin]

On 27/10/2015 18:28 +0100, Steinar Bang wrote:

>>>>>> Dmitry Gutov <dgutov@yandex.ru>:
>
>> Could we consider it to be constant value across the whole project, or
>> would we have, for certain operations (like "find the class named
>> foo.bar.Bar"), different values of classpath in module1 and module2?
>
> The classpath will be per project/module, and for each project it can
> differ for compilation, runtime, test execution.
>
> A *brief* explanation of how the class path is built (for the initiated:
> by "identity" below, I mean the "maven coordinates" consisting of
> groupId/artifactId/version):
>  - Each pom.xml file, have:
>   - an identity
>   - the identity of a parent pom.xml file
>   - a list of dependencies (including the scope of the dependency,
>     eg. "test")
>  - The combined dependency list of a pom.xml is:
>   - The dependencies in the dependency list
>   - The transitive dependencies of the dependencies in the dependency
>     list
>   - The combined dependencies of the parent pom.xml (which includes that
>     pom.xml file's dependencies and transitive dependencies and those of
>     its parent in turn)
>  - Maven will download all dependencies and install them in the maven
>    local cache (by default the maven local cache resides in
>    $HOME/.m2/repository/ )
>  - Maven will create the appropriate classpath for compile, test, run,
>    etc. consisting of jar files in the maven local cache
>

I've been following this long thread with interest and there's one
thing that keeps bothering me - should we duplicate the logic of build
tool in Emacs IDE-like features?

In my `javaimp' package (available in GNU Elpa) Emacs calls Maven to get
classpath, then scans all jars in it (it takes them from local Maven
cache and reads what classes are inside).  This is done only once given
that pom.xml doesn't change (and if it changes this is repeated).

This requires some time to wait for Maven to finish and output the
classpath.  But jar scanning is inevitable, I guess, and takes more time
(both are cached then).

I'm hoping to implement something like this for Gradle, but I didn't get
to it yet and I don't know whether it can output classpath like Maven
can.

Btw, there are subtleties between different Maven versions in dealing
with dependency merging when what is called an `effective pom' is built
(that is, when pom hierarchy is merged to produce effective settings for
the current module).  I remember that initially Maven 3 was outputting a
slightly wrong tree with `mvn dependency:tree' rather than it was
actually using during build.

This looks a lot like the GCC AST problem.

Filipp

Re: IDE

[Steinar Bang]

>>>>> Filipp Gunbin <fgunbin@fastmail.fm>:

> I've been following this long thread with interest and there's one
> thing that keeps bothering me - should we duplicate the logic of build
> tool in Emacs IDE-like features?

If you mean: "should we re-implement things like maven in lisp?" then my
answer is: "no, not necessarily"

But it's useful to know how the interesting parts of the build config is
created and what it means (in this case how the classpath is set up when
multiple pom.xml files are involved).

> In my `javaimp' package (available in GNU Elpa) Emacs calls Maven to get
> classpath, then scans all jars in it (it takes them from local Maven
> cache and reads what classes are inside).

Sounds interesting, I will check it out.

> This is done only once given that pom.xml doesn't change (and if it
> changes this is repeated).

Have you given any thoughts to triggering this when the pom.xml is saved
and/or changed on disk?

> This requires some time to wait for Maven to finish and output the
> classpath.  But jar scanning is inevitable, I guess, and takes more time
> (both are cached then).

Do you cache this only in memory or on disk? (sounds like might be a
good idea to cache this information in an S-expression file, eg. pom.el,
toghether with each pom.xml and maintain the pom.el in a make-like
fashion...?) 

Re: IDE

[Filipp Gunbin]

On 28/10/2015 16:57 +0100, Steinar Bang wrote:

>>>>>> Filipp Gunbin <fgunbin@fastmail.fm>:
>> This is done only once given that pom.xml doesn't change (and if it
>> changes this is repeated).
>
> Have you given any thoughts to triggering this when the pom.xml is saved
> and/or changed on disk?

When a set of completions is prepared for an interactive command (like
`javaimp-add-import'), pom files' timestamps are checked and if they
changed then the information stored by the package is updated.  This
works only for current and one parent pom up the hierarchy for now, for
others it's better to do `javaimp-forget-all-visited-modules' manually.

The same is done for jar files: when a jar file classes are added to a
set of completion alternatives, its timestamp is checked - useful
because often we work with SNAPSHOT versions of the current module's
siblings and wish to read the latest information from them.

Current module's classes are added by scanning current source directory,
this is rather dumb scanning - no code parsing, just file names.

javaimp does not support importing of inner classes, but usually for me
it's not a problem because it's enough to import the top class in a
file.


>> This requires some time to wait for Maven to finish and output the
>> classpath.  But jar scanning is inevitable, I guess, and takes more time
>> (both are cached then).
>
> Do you cache this only in memory or on disk? (sounds like might be a
> good idea to cache this information in an S-expression file, eg. pom.el,
> toghether with each pom.xml and maintain the pom.el in a make-like
> fashion...?)

Currently only in memory, but I'm thinking of doing a "flush" to a
file.

My workflow is like that: when I "open" a project, I call
`javaimp-maven-visit-root' on a top-level project and it loads
information on it and its child projects (mvn help:effective-pom).  When
I call `javaimp-add-import' from one of the projects' files, it
recognizes which module needs to be scanned and invokes Maven on that
module (mvn dependency:build-classpath AFAIR), then scans jars which
require scanning and suggests completion alternatives.  So it's all on
demand (have to wait a bit during the first time, though).  This is to
eliminate constant indexing / scanning / updating which I hated in
Idea/Eclipse.

Filipp

Re: IDE

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If you mean: "should we re-implement things like maven in lisp?" then my
  > answer is: "no, not necessarily"

Quoth the maven, "implement me nevermore."
;-)

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: IDE

[Dmitry Gutov]

On 10/28/2015 02:34 PM, Filipp Gunbin wrote:

> I've been following this long thread with interest and there's one
> thing that keeps bothering me - should we duplicate the logic of build
> tool in Emacs IDE-like features?

Probably not. We could conceivably want to provide the access to some 
information, if someone wanted to implement an Emacs-y editor for the 
project files. That's ways off to the future, though.

Re: IDE

[Steinar Bang]

>>>>> Dmitry Gutov <dgutov@yandex.ru>:

> Probably not. We could conceivably want to provide the access to some
> information, if someone wanted to implement an Emacs-y editor for the
> project files. That's ways off to the future, though.

(If we're talking maven then nxml works file.  I believe I have an RNG
schema for maven lying around somewhere...)

Re: IDE

[Dmitry Gutov]

On 11/01/2015 10:10 PM, Steinar Bang wrote:

> (If we're talking maven then nxml works file.

Cool, I'm all for lightweight approaches. It was just the best example 
of using dependency information that I could think of right away.

> I believe I have an RNG
> schema for maven lying around somewhere...)

If we want to get fancier, Intellij IDEA also provides completion for 
e.g. artifact ids. But that's neither here nor there.

Re: IDE

[Dmitry Gutov]

Hi Steinar,

Sorry for the late response.

On 10/27/2015 07:28 PM, Steinar Bang wrote:
> (Classpath in maven is actually independent of the directory layout of
> the checked out projects, but I won't get into that now)

I meant that either we have several values: project-project, 
project-module1, project-module2, and each of those reports a different 
value of classpath, or we only have one value, project, and the accessor 
function project-classpath takes a additional argument, DIR, and can 
return different values, depending on which module, internally, DIR 
belongs to.

Note that classpath itself is a pretty JVM-centric attribute. It points 
to jars, as well as directories (the latter less often), and reading the 
contents of jars, if we want to, will have to be handled specially. 
Overall, classpath will be more relevant for features that particularly 
target the JVM.

More language agnostic attributes, which I'm more worried about, are 
sets of directories related to the project. We can search those 
directories, and we can list files in them (and implement, based on 
that, the feature "jump to file in the project", with completion).

We have currently selected two attributes like that: "project roots", 
directories which contain the code of the current project, and "search 
path" (which would probably be more appropriately named as "library 
roots"). Those directories contain the code that the current project 
refers to.

For C or C++ the closest analog would be the include path. For Ruby or 
Python - the directories in the current language installation where the 
libraries are installed.

For JVM languages, the classpath seems to be the closest thing, but it 
contains compiled code. The same jars can also contain sources (I 
think?), but searching through them, as well as listing contained files, 
or visiting them, is tricker. We should probably consider that for a 
later stage.

> (That means that as long as the projects are built in dependency order,
> they can in *theory* be be checked out separately and built separately,
> parent, and dependencies to other projects can be resolved against the
> local cache.  In practice, however, it is simpler to organize the maven
> project in the recommended hierarchy and let maven handle built order
> and stuff)

This is all great, but as long as Maven knows how to build the project, 
is there a reason for the project API to know these details?

> Depends on what you mean with important project attributes...?

The full set will probably emerge incrementally, over time, as 
developers who want to use the project API complain that it covers too 
little.

> If "project" is the parent pom.xml of the two modules, you can set
> properties in "project" that are shared by module1 and module2.  You can
> also share plugin configurations in this way.

All right. So either the project API will handle of property 
inheritance, or the implementations will need to do that.

The latter imposes some difficulties on how the project values are 
constructed, but since the multi-module project model seem to be 
prevalent in the Maven world only, I think I'd rather have a simpler API.

> eclipse m2e will still recognize that "module1" contains the dependency
> of "module2" and build them in order (they will show up as project
> dependencies in the project classpath in eclipse).

So I take it the dependencies are mostly important for the "build 
everything" command.

> The default search scope in eclipse is "workspace", which is the
> projects seen in eclipse's package explorer.

That's very interesting. Maybe we'll have a notion of a workspace (the 
set of the currently opened projects), as well as manual 
"open/visit/close project" commands. Like, in the second version.

Re: IDE

[Xue Fuqiao]

On Fri, Oct 16, 2015 at 3:39 PM, Przemysław Wojnowski
<esperanto@cumego.com> wrote:
> BTW Why EDE (and CEDET) are in two different repositories (and projects?).
> One is in Emacs sources, another here http://sourceforge.net/projects/cedet/
> ?

The same reason(s) for CC mode, Gnus, Org, Tramp, etc.

Some reasons I imagine are:

* It can leave bleeding edge features on its own repo (not necessary in
  the next Emacs release)
* It can have its own bug tracker (if its authors don't like Debbugs)
* It can include code not distributed with Emacs for some reasons (for
  example, a new author doesn't want to sign or haven't signed the
  copyright assignment yet)
* It can use a VCS Emacs is not using (for example, Gnus and Org was
  using Git before the Emacs Git transition)
* It can include compatibility helpers for XEmacs and other emacsen

On Fri, Oct 16, 2015 at 7:17 PM, Przemysław Wojnowski
<esperanto@cumego.com> wrote:
> Yes, so when one would like change EDE to, lets say, apply the Bridge
> pattern,
> which one should be used?

Both are OK, I think.  IIRC the SourceForge repo merges changes from
Emacs master from time to time.

Re: IDE

[Eric Ludlam]

On 10/16/2015 07:17 AM, Przemysław Wojnowski wrote:
>>> BTW Why EDE (and CEDET) are in two different repositories (and 
>>> projects?).
>>> One is in Emacs sources, another here
>>> http://sourceforge.net/projects/cedet/ ?
>>
>> CEDET is in Emacs sources as well (although is has some omissions). 
>> See lisp/cedet.
> Yes, so when one would like change EDE to, lets say, apply the Bridge 
> pattern,
> which one should be used?
> . 

Good question.  The recent EIEIO changes in Emacs made the two 
repositories really hard to merge, and EDE is steeped in EIEIO.

David Engster has been doing the merges and might have a good idea. I'm 
just not that good with git. ;(

Eric

Re: IDE

[David Engster]

Eric Ludlam writes:
> On 10/16/2015 07:17 AM, Przemysław Wojnowski wrote:
>>>> BTW Why EDE (and CEDET) are in two different repositories (and
>>>> projects?).
>>>> One is in Emacs sources, another here
>
>>>> http://sourceforge.net/projects/cedet/ ?
>>>
>>> CEDET is in Emacs sources as well (although is has some
>>> omissions). See lisp/cedet.
>> Yes, so when one would like change EDE to, lets say, apply the
>> Bridge pattern,
>> which one should be used?
>> . 
>
> Good question.  The recent EIEIO changes in Emacs made the two
> repositories really hard to merge, and EDE is steeped in EIEIO.
>
> David Engster has been doing the merges and might have a good
> idea. I'm just not that good with git. ;(

At the moment it is best to work in the Emacs repository.

-David

Re: IDE

[Eric Ludlam]

On 10/16/2015 03:39 AM, Przemysław Wojnowski wrote:
>> It matters if to *really* add support for a new build tool, the 
>> author has to
>> add X new project definitions.
>>
> IIUC someone developing an EDE support (a plugin?) for a build tool 
> can provide as many as s/he wants, right? For example for a build tool 
> a developer may provide only two definitions: clean and build.

EDE's framework starts with no assumptions about anything other than a 
basic "compile", and letting the project implementation populate it.

Some projects, such as the android and arduino ones, add new commands 
such as 'upload to device' which make no sense in other projects.  Those 
projects end up self consistent as far as keybindings.  Until there are 
lots of projects with different ideas, there isn't much in the way of 
guidelines.

> I think that two more features would be helpful:
> - composition of tasks (e.g. run "clean" before "build")
> - allow user to add own tasks (could be customizations of default 
> tasks, like "compile --with-debug-symbols").
>

Project implementations can have as many configurations or custom build 
commands as they like.

> Correct me if I'm wrong, but EDE (xref and project.el) are open for 
> modifications, right? It not "take it as it is or leave it".

Indeed.

> BTW Why EDE (and CEDET) are in two different repositories (and 
> projects?).
> One is in Emacs sources, another here 
> http://sourceforge.net/projects/cedet/ ?
> . 

I maintain CEDET, but I cannot get a permanent release from my company, 
so I have to keep getting it updated.  By having a separate repository I 
can keep going without an active release without getting Emacs legally 
dirty.

Eric

Is EDE only intended to be used with languages which /require/ a 'compile' step? [was: Re: IDE]

[Alexis]

Eric Ludlam <ericludlam@gmail.com> writes:

> EDE's framework starts with no assumptions about anything other 
> than a basic "compile", and letting the project implementation 
> populate it.

It seems to me that this assumption only holds for programming 
languages which require a distinct compile step in order to 
produce an executable program. If EDE isn't intended to be used 
with languages such as Python, JavaScript, Ruby, Lua, Perl etc., 
fair enough - but this should be made clear in this discussion and 
in the EDE documentation, so that developers/users wanting an 
Emacs-based software project management system for such languages 
can focus attention/efforts elsewhere (e.g. Projectile, 
project.el).


Alexis.

Re: Is EDE only intended to be used with languages which /require/ a 'compile' step? [was: Re: IDE]

[Eric Ludlam]

On 10/16/2015 11:24 PM, Alexis wrote:
>
> Eric Ludlam <ericludlam@gmail.com> writes:
>
>> EDE's framework starts with no assumptions about anything other than a
>> basic "compile", and letting the project implementation populate it.
>
> It seems to me that this assumption only holds for programming languages
> which require a distinct compile step in order to produce an executable
> program. If EDE isn't intended to be used with languages such as Python,
> JavaScript, Ruby, Lua, Perl etc., fair enough - but this should be made
> clear in this discussion and in the EDE documentation, so that
> developers/users wanting an Emacs-based software project management
> system for such languages can focus attention/efforts elsewhere (e.g.
> Projectile, project.el).

EDE's original intent was for handling compilation of compiled 
languages.  Since then, it also forms a base for anything that wants to 
organize code into a 'project' so that support code can say "what is the 
current project" and then "does that project have a language specific 
detail I can use".  It doesn't really matter if it compiles or not.

If your specific language has no need of a centralized project info, 
then sure, it won't need EDE.  If python, JS or whatever may need 
different information such as include paths, browser to connect to from 
Emacs, etc, then that could be wrapped up in EDE.

The advantage would be a consistent way to think about each step in 
using the project.  Your C++ project may have entirely different steps 
in the workflow than javascript, but users would know to look into the 
Development menu to see what that project can do.

In addition, EDE, plus other parts such as Semantic or SRecode provide 
project and language independent ways to get at basic questions, so 
tools like ECB or Speedbar can display information independent of 
language.  The more language specific the request, the harder that is of 
course, but having one place to get at the data makes writing those 
tools easier.

Eric

Re: Is EDE only intended to be used with languages which /require/ a 'compile' step? [was: Re: IDE]

[Alexis]

Eric Ludlam <eric@siege-engine.com> writes:

> EDE's original intent was for handling compilation of compiled 
> languages.  Since then, it also forms a base for anything that 
> wants to  organize code into a 'project' so that support code 
> can say "what is the  current project" and then "does that 
> project have a language specific  detail I can use".  It doesn't 
> really matter if it compiles or not.

Thank you for clarifying! i would like to suggest that the EDE 
documentation be modified to reflect this.

For example, the opening paragraph of the EDE Info manual says:

    "EDE is the Emacs Development Environment: an Emacs extension 
    that simplifies building and debugging programs in Emacs.  It 
    attempts to emulate a typical IDE (Integrated Development 
    Environment).  EDE can manage or create your makefiles and 
    other building environment duties"

The third sentence could instead say something along the lines of:

    "EDE can help you manage your software projects and any build 
    environments they might have (such as, for example, Makefiles) 
    ..."

Additionally, the "Quick Start" section of the manual could have 
an extra example added, about creating a project in a language 
that doesn't require a distinct compile step, nor a Makefile (for 
example, a Python project).

Modifications such as the preceding might help make it immediately 
clear to potential EDE users that EDE is not solely for people 
programming in C, C++, Objective-C, or compilation-requiring 
languages such as Java, Ada or Haskell.

Ranking the extent of usage of various programming languages is, 
of course, fraught with methodological issues, but not only the 
TIOBE ranking:

http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html

but also the January 2015 Redmonk ranking:

http://redmonk.com/sogrady/2015/01/14/language-rankings-1-15/

and a 2014 IEEE ranking:

http://spectrum.ieee.org/static/interactive-the-top-programming-languages

puts JavaScript, Python, PHP and Ruby amongst the current top 10 
programming languages. i feel that if Emacs is to work towards 
becoming a more general-purpose IDE "out of the box", we should 
avoid giving programmers of such languages the impression that the 
tools provided by Emacs are Not Relevant.


Alexis.

Re: Is EDE only intended to be used with languages which /require/ a 'compile' step? [was: Re: IDE]

[Aaron Ecay]

2015ko urriak 22an, Alexis-ek idatzi zuen:
> 
> Eric Ludlam <eric@siege-engine.com> writes:
> 
>> EDE's original intent was for handling compilation of compiled 
>> languages.  Since then, it also forms a base for anything that 
>> wants to  organize code into a 'project' so that support code 
>> can say "what is the  current project" and then "does that 
>> project have a language specific  detail I can use".  It doesn't 
>> really matter if it compiles or not.
> 
> Thank you for clarifying! i would like to suggest that the EDE 
> documentation be modified to reflect this.

+1.  I read the EDE info documentation several years ago, and had the
impression that EDE (and CEDET more broadly) would not work well for me
because I never use compiled languages, and (almost) never Makefiles.  I
got the same impression from the “Projects” section on the CEDET homepage,
which says it can recognize Makefile-based projects and other projects
“such as the Emacs sources, the Linux kernel, or any project built using
Automake.”  Nothing about JS, Python, or other dynamic languages (or
even Java).

There have been several threads about IDE-like features in the past
where CEDET has been mentioned, but this is the first one that has
actually allowed me to understand how it might actually be useful for
dynamic languages (even if that requires more extensions that don’t
exist yet).

In a related vein, it would be nice to see something about “why CEDET”
from the perspective of advanced users/developers.  There are lots of
projects that create development environments for Emacs in dynamic
languages: elpy for python, ESS for R, cider for clojure, etc.  These
combine “foolproof” setup of various emacs features (which-func, imenu,
repl interaction, completion, ...) with a tool that analyzes the source
and feeds information back to emacs.  I often find these tools a little
annoying, because they all work slightly differently, and sometimes
don’t obey emacs conventions for customization (e.g. using hard-coded
values instead of hooks and keymaps).

It would be good if someone(s) familiar with CEDET could articulate the
benefits of implementing these types of systems with CEDET.  It’s a
worthy goal to eventually transfer many of the features of these systems
to CEDET, ideally relying on the work that these projects have already
done on the external tooling.  This document could serve as a roadmap
for that work, as well as helping convince new projects to use CEDET.

Thanks,

-- 
Aaron Ecay

Re: IDE

[Eric Ludlam]

On 10/10/2015 04:59 AM, Dmitry Gutov wrote:
> On 10/10/2015 11:30 AM, Eli Zaretskii wrote:
>
>> I was talking about working on IDE, not on completion.  And for the
>> most popular languages in the industry, not just for some a few niche
>> languages.
>
> You quoted the message that said "accurate code completion and powerful
> refactoring support". I can agree that the latter is barely touched (*),
> but it looked like you ignored the former.
>
>> Let's not reiterate past discussions: you forget CEDET.
>
> I was enumerating external programs. But sure, CEDET is a yet another
> option for completion. Though not too "accurate" one, if we're talking
> anything but C.

I had always intended CEDET to be a baseline for IDE like features. 
Looking just at tagging files, those familiar with it's internals 
recognize that it can use external tools as weak as ctags or GLOBAL, and 
can use a more powerful external tool for parsing as well, such as using 
JAVAC for decompiling .jar files into tags.  As a backup, it also has 
in-buffer parsing for when you've only half-written your code, or when 
no external tool is available.

The same philosophy is throughout CEDET.  Unfortunately, those who are 
interested in tooling who brush casually past CEDET only see that people 
complain that it doesn't always complete right, or that the C++ part is 
hard to setup, and don't see that the cool piece they want integrated 
into Emacs could use CEDET as a framework.

A side effect is that every completion engine out there has a custom way 
of integrating random tools in.  In theory, completion engines could 
call into CEDET, and let CEDET dispatch to the other tools.  In that 
world, bootstrapping new tools is simpler with only one backend to worry 
about, and there would be more language independent creativity for IDE 
like features.

If we look at the two starting key features listed for IDEs, 
"completion" and "refactoring", the basics are all there.  CEDET has a 
completion engine, but it is only as good as the input data.  If better 
external tools (like GCC) or easier to configure tools could feed it, it 
would be more accurate.  CEDET also has 'srecode' which is about 
generating code.  This bit is trickier, as the intention is you could 
write one highlevel tool that might extract tag data from your file, 
permute the language-independent tags, and then write them back out with 
srecode.  There are tests showing this working, but they are simple and 
proof of concept and not stuff someone would depend on day-to-day.

What it needs is folks who care about cross language tooling, and not 
just the language specific part.  While I continue to maintain CEDET, I 
don't code for a living anymore, so do not run into CEDET's rough 
corners enough to know to fix them.

Eric

Re: IDE

[Dmitry Gutov]

Hi Eric,

On 10/10/2015 07:48 PM, Eric Ludlam wrote:

> I had always intended CEDET to be a baseline for IDE like features.
> Looking just at tagging files, those familiar with it's internals
> recognize that it can use external tools as weak as ctags or GLOBAL, and
> can use a more powerful external tool for parsing as well, such as using
> JAVAC for decompiling .jar files into tags.

It sounds good if all I have is a parser/tagger.

But if I already have an external tool that can give me a set of 
completions at a given position in a given buffer, and also provides a 
go-to-definition feature, what's the advantage of going through 
Semantic? SRecode support?

Not to mention that we have tools like Jedi that only (mostly) do these 
two things, but don't provide dumps of the syntax tree.

> As a backup, it also has
> in-buffer parsing for when you've only half-written your code, or when
> no external tool is available.

That assumes we also have a Semantic grammar for the said language. And 
then, we'll need to duplicate whatever logic is used by the external 
tool to disambiguate between same-named methods in different classes.

If that's even possible.

> The same philosophy is throughout CEDET.  Unfortunately, those who are
> interested in tooling who brush casually past CEDET only see that people
> complain that it doesn't always complete right, or that the C++ part is
> hard to setup

The example we've been given in one of the previous discussions of C++ 
support is that Semantic can't tell the difference between different 
methods with the same name? If Z#foo returns type A, and T#foo returns 
type B, it's been shown that Semantic doesn't know which is the type 
'x.foo'.

Given this, how would we expect it to properly support languages with 
type inference like Scala or Go? Not to mention dynamic languages which 
normally have no type annotations on methods, and often use more 
complicated algorithms for accurate code completion.

> If we look at the two starting key features listed for IDEs,
> "completion" and "refactoring", the basics are all there.  CEDET has a
> completion engine, but it is only as good as the input data.

It's limited by the input data, sure, but it also has to know what to do 
with it.

If only implementing accurate code completion was as easy as writing a 
language parser.

> CEDET also has 'srecode' which is about
> generating code.  This bit is trickier, as the intention is you could
> write one highlevel tool that might extract tag data from your file,
> permute the language-independent tags, and then write them back out with
> srecode.  There are tests showing this working, but they are simple and
> proof of concept and not stuff someone would depend on day-to-day.

SRecode should be in a better position, but I imagine it would also fail 
often enough in dynamic languages.

And in any case, one has to teach Semantic about scoping rules for each 
given language, for SRecode to only rename, say, a variable 'foo' but 
not function 'foo', or only the variable 'bar', but not the variable 
'bar' up the scope that's being shadowed.

Re: IDE

[Eli Zaretskii]

> Cc: adatgyujto@gmail.com, emacs-devel@gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Sun, 11 Oct 2015 07:38:31 +0300
> 
> On 10/10/2015 07:48 PM, Eric Ludlam wrote:
> 
> > I had always intended CEDET to be a baseline for IDE like features.
> > Looking just at tagging files, those familiar with it's internals
> > recognize that it can use external tools as weak as ctags or GLOBAL, and
> > can use a more powerful external tool for parsing as well, such as using
> > JAVAC for decompiling .jar files into tags.
> 
> It sounds good if all I have is a parser/tagger.
> 
> But if I already have an external tool that can give me a set of 
> completions at a given position in a given buffer, and also provides a 
> go-to-definition feature, what's the advantage of going through 
> Semantic?

One advantage that comes to mind is that you don't depend on an
external tool whose development is beyond our control.

Re: IDE

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: adatgyujto@gmail.com, emacs-devel@gnu.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Sun, 11 Oct 2015 07:38:31 +0300
>> 
>> On 10/10/2015 07:48 PM, Eric Ludlam wrote:
>> 
>> > I had always intended CEDET to be a baseline for IDE like features.
>> > Looking just at tagging files, those familiar with it's internals
>> > recognize that it can use external tools as weak as ctags or GLOBAL, and
>> > can use a more powerful external tool for parsing as well, such as using
>> > JAVAC for decompiling .jar files into tags.
>> 
>> It sounds good if all I have is a parser/tagger.
>> 
>> But if I already have an external tool that can give me a set of 
>> completions at a given position in a given buffer, and also provides a 
>> go-to-definition feature, what's the advantage of going through 
>> Semantic?
>
> One advantage that comes to mind is that you don't depend on an
> external tool whose development is beyond our control.

Well, we'd certainly like to depend on GCC.

-- 
David Kastrup

Re: IDE

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,  emacs-devel@gnu.org,  adatgyujto@gmail.com,  eric@siege-engine.com
> Date: Sun, 11 Oct 2015 17:23:10 +0200
> 
> Well, we'd certainly like to depend on GCC.

Or on GDB, or on any other flagship part of the GNU Project.

But I was talking about the rest of them.

Re: IDE

[Dmitry Gutov]

On 10/11/2015 06:08 PM, Eli Zaretskii wrote:

> One advantage that comes to mind is that you don't depend on an
> external tool whose development is beyond our control.

You still depend on an external tool for other things.

Even if that wasn't true, having to re-implement everything yourself, 
for each language, is not an advantage.

Certainly not for a relatively small project like Emacs.

Re: IDE

[Eric Ludlam]

On 10/11/2015 12:38 AM, Dmitry Gutov wrote:
> Hi Eric,
>
> On 10/10/2015 07:48 PM, Eric Ludlam wrote:
>
>> I had always intended CEDET to be a baseline for IDE like features.
>> Looking just at tagging files, those familiar with it's internals
>> recognize that it can use external tools as weak as ctags or GLOBAL, and
>> can use a more powerful external tool for parsing as well, such as using
>> JAVAC for decompiling .jar files into tags.
>
> It sounds good if all I have is a parser/tagger.
>
> But if I already have an external tool that can give me a set of 
> completions at a given position in a given buffer, and also provides a 
> go-to-definition feature, what's the advantage of going through 
> Semantic? SRecode support?

CEDET's tools are set up in layers.  As an language support author you 
can start with a series of overrides for writing a lexer or parser 
yourself.  If you have a tool that parses already (such as exuberent 
CTags), you can instead just start at the high level tagging parser and 
skip all the lower level stuff.

If you have an existing completion engine for cases where you happen to 
have an interpreter with completion, or something else, you can just 
override the completion engine directly.  The srecode tool does this, 
and there is an experimental clang version in CEDET's repository as well.

Having overrides at all 3 levels is the best, since different tools ask 
for different features, but it is not necessary as there is backup 
implementation for the higher level features.

> Not to mention that we have tools like Jedi that only (mostly) do 
> these two things, but don't provide dumps of the syntax tree.
>
>> As a backup, it also has
>> in-buffer parsing for when you've only half-written your code, or when
>> no external tool is available.
>
> That assumes we also have a Semantic grammar for the said language. 
> And then, we'll need to duplicate whatever logic is used by the 
> external tool to disambiguate between same-named methods in different 
> classes.
>
> If that's even possible.
>

Of course.  For strongly typed languages the problems are deterministic 
so it's possible.  For dynamic languages, they usually provide something 
you can ask.  It's just a matter of integration.

I enjoy working on puzzles like this, but for me it is a matter of free 
time.  Kids, job, etc get in the way.

>> The same philosophy is throughout CEDET. Unfortunately, those who are
>> interested in tooling who brush casually past CEDET only see that people
>> complain that it doesn't always complete right, or that the C++ part is
>> hard to setup
>
> The example we've been given in one of the previous discussions of C++ 
> support is that Semantic can't tell the difference between different 
> methods with the same name? If Z#foo returns type A, and T#foo returns 
> type B, it's been shown that Semantic doesn't know which is the type 
> 'x.foo'.
>
> Given this, how would we expect it to properly support languages with 
> type inference like Scala or Go? Not to mention dynamic languages 
> which normally have no type annotations on methods, and often use more 
> complicated algorithms for accurate code completion.

Semantic has the information to differentiate between two methods of 
different types.  There are still a few short-cuts in the completion 
engine  where it could do a little more to disambiguate.  There is a 
function to '-select-best-tag' in the analyzer, but at the end, it just 
grabs the first element from the list of possibilities.  If you have a 
particular use case in mind, it would good to have a simple example that 
shows what it is and maybe it will be easy to update.

>> If we look at the two starting key features listed for IDEs,
>> "completion" and "refactoring", the basics are all there.  CEDET has a
>> completion engine, but it is only as good as the input data.
>
> It's limited by the input data, sure, but it also has to know what to 
> do with it.
>
> If only implementing accurate code completion was as easy as writing a 
> language parser.

This is a matter of time and desire.  For the code I've been writing 
lately (arduino hobby stuff), it has been more than sufficient. David 
was interested in handling templates and was able to add support for 
that.  If someone wants it to handle something new, the data is there to 
do it.

>> CEDET also has 'srecode' which is about
>> generating code.  This bit is trickier, as the intention is you could
>> write one highlevel tool that might extract tag data from your file,
>> permute the language-independent tags, and then write them back out with
>> srecode.  There are tests showing this working, but they are simple and
>> proof of concept and not stuff someone would depend on day-to-day.
>
> SRecode should be in a better position, but I imagine it would also 
> fail often enough in dynamic languages.

I'm not quite sure what the intention is here.  Srecode is just template 
expansion, with mechanisms to enable multiple layers of templates, and 
support for "application" templates and user override templates for 
customization.

What's missing is someone taking the time to start building the logic to 
use semantic to pull buffers apart, and srecode to re-write the code.

> And in any case, one has to teach Semantic about scoping rules for 
> each given language, for SRecode to only rename, say, a variable 'foo' 
> but not function 'foo', or only the variable 'bar', but not the 
> variable 'bar' up the scope that's being shadowed.
>

There is a fair amount of logic for supporting language scopes. There is 
almost nothing about running filters for "all the calls to symbol FOO 
that has some particular feature".  The symref tool has a comment in it 
that says "do the filtering here", but no one has gotten around to it.

My assumption is that it would be easier to start there than start from 
scratch, unless you just hijack some external tool.

Eric

Re: IDE

[Dmitry Gutov]

On 10/11/2015 08:37 PM, Eric Ludlam wrote:
> If you have a tool that parses already (such as exuberent
> CTags), you can instead just start at the high level tagging parser and
> skip all the lower level stuff.

Right.

> If you have an existing completion engine for cases where you happen to
> have an interpreter with completion, or something else, you can just
> override the completion engine directly.

You haven't answered the question about the advantage of doing it this 
way. If I override the completion engine directly, what main benefits of 
using CEDET are left? I mean, are they worth working on defining a 
grammar for the language, and keeping it up-to-date. A grammar can take 
a lot of effort by itself.

> The srecode tool does this,
> and there is an experimental clang version in CEDET's repository as well.

srecode overrides the completion engine? Why?

> Having overrides at all 3 levels is the best, since different tools ask
> for different features, but it is not necessary as there is backup
> implementation for the higher level features.

I agree that, in general, having overridable implementations is pretty 
great.

> Of course.  For strongly typed languages the problems are deterministic
> so it's possible.  For dynamic languages, they usually provide something
> you can ask.  It's just a matter of integration.
>
> I enjoy working on puzzles like this, but for me it is a matter of free
> time.  Kids, job, etc get in the way.

My impression is that nobody has solved this puzzle for any of the 
dynamic languages that CEDET aims to support. Which is not a definitive 
evidence either way, but it's pretty discouraging for a person 
interested in implementing better support for a dynamic language they're 
interested in.

> Semantic has the information to differentiate between two methods of
> different types.  There are still a few short-cuts in the completion
> engine  where it could do a little more to disambiguate.  There is a
> function to '-select-best-tag' in the analyzer, but at the end, it just
> grabs the first element from the list of possibilities.  If you have a
> particular use case in mind, it would good to have a simple example that
> shows what it is and maybe it will be easy to update.

Try this: https://gist.github.com/dgutov/19c45ef43d1c90b96483

I should get "a" and "b" as completions, but instead I get "c" and "d".

> This is a matter of time and desire.  For the code I've been writing
> lately (arduino hobby stuff), it has been more than sufficient. David
> was interested in handling templates and was able to add support for
> that.  If someone wants it to handle something new, the data is there to
> do it.

Yes, templates are pretty important. And complex. I don't know if 
re-implementing the code to understand them in Elisp is the way to go.

> What's missing is someone taking the time to start building the logic to
> use semantic to pull buffers apart, and srecode to re-write the code.

All right. So we could try to use Srecode interface for refactoring.

Re: IDE

[Eric Ludlam]

On 10/12/2015 11:18 AM, Dmitry Gutov wrote:
> On 10/11/2015 08:37 PM, Eric Ludlam wrote:
>> If you have a tool that parses already (such as exuberent
>> CTags), you can instead just start at the high level tagging parser and
>> skip all the lower level stuff.
>
> Right.
>
>> If you have an existing completion engine for cases where you happen to
>> have an interpreter with completion, or something else, you can just
>> override the completion engine directly.
>
> You haven't answered the question about the advantage of doing it this 
> way. If I override the completion engine directly, what main benefits 
> of using CEDET are left? I mean, are they worth working on defining a 
> grammar for the language, and keeping it up-to-date. A grammar can 
> take a lot of effort by itself.

The primary reason is that having tag information in a buffer so you can 
access it quickly is helpful.

The reason you'd want a tag-level parser at all is to provide:
1) a database of tags in the buffer, plus positional information
2) a database of tags across a project to search through
3) a standard way of knowing where you are in relation to other tags

Simple things like showing the function you are editing, highlighting 
tags with various features in different ways,or knowing what class the 
method you are in are handy and quick little features that can be built 
generically on top of CEDET, but which require piles of code to do 
individually without that type of support. imenu, etags, ctags, global, 
ident, etc all exist because it is useful, but none of those tools get 
bound into a buffer, so their level of usefulness is limited to "jump to 
a location" instead of handy inline features.

>> The srecode tool does this,
>> and there is an experimental clang version in CEDET's repository as 
>> well.
>
> srecode overrides the completion engine? Why?
>

Because it is a multi-mode buffer, so sometimes you want to complete 
srecode symbols, and sometimes you want to complete from a different 
language.

Eric

Re: IDE

[Dmitry Gutov]

On 10/14/2015 01:29 AM, Eric Ludlam wrote:

> Simple things like showing the function you are editing, highlighting
> tags with various features in different ways,or knowing what class the
> method you are in are handy and quick little features that can be built
> generically on top of CEDET, but which require piles of code to do
> individually without that type of support. imenu, etags, ctags, global,
> ident, etc all exist because it is useful, but none of those tools get
> bound into a buffer, so their level of usefulness is limited to "jump to
> a location" instead of handy inline features.

Not sure what you mean by "bound into a buffer", but IMenu, in general, 
only requires a few regexps, and once you have those in place, 
which-func-mode can show up in which "tag" you are.

And ctags can be used for "a database of tags across a project". You 
require it either way, since only open buffers are parsed by Semantic.

 > 3) a standard way of knowing where you are in relation to other tags

How does that help?

> Because it is a multi-mode buffer, so sometimes you want to complete
> srecode symbols, and sometimes you want to complete from a different
> language.

Makes sense, thanks.

Re: IDE

[Eric Ludlam]

On 10/14/2015 11:16 PM, Dmitry Gutov wrote:
> On 10/14/2015 01:29 AM, Eric Ludlam wrote:
>
>> Simple things like showing the function you are editing, highlighting
>> tags with various features in different ways,or knowing what class the
>> method you are in are handy and quick little features that can be built
>> generically on top of CEDET, but which require piles of code to do
>> individually without that type of support. imenu, etags, ctags, global,
>> ident, etc all exist because it is useful, but none of those tools get
>> bound into a buffer, so their level of usefulness is limited to "jump to
>> a location" instead of handy inline features.
>
> Not sure what you mean by "bound into a buffer", but IMenu, in 
> general, only requires a few regexps, and once you have those in 
> place, which-func-mode can show up in which "tag" you are.

CEDET will store tags into a set of overlays in the buffer.  That means 
figuring out what tag the cursor is in is as fast as asking for what 
overlay the cursor is in.

Imenu stores it's tags in a list, so you need to scan the list to figure 
it out. Imenu's tags are also weak, so the elisp knows very little about 
the tag, only where it is, and enough to queue the reader.

> And ctags can be used for "a database of tags across a project". You 
> require it either way, since only open buffers are parsed by Semantic.
>

Yes.  There are other tools that do different pieces of what CEDET does.

> > 3) a standard way of knowing where you are in relation to other tags
>
> How does that help?
>

* It lets you 'copy' a tag, and 'yank' it somewhere else.
* It provides an accurate 'beginning of defun', 'end of defun',
   'narrow to defun'
* srecode can programmatically insert new tags between other tags
   using a hueristic.
* It can figure out where to place or find a header comment.
* You can decorate the tags accurately
* Provides a starting symbol for some commands, such as symref.
* Adding folding of tags to a buffer is pretty easy (though that
   contribution didn't get a release. :( )
* The stuff imenu / which-func does but with more options such as
   breadcrumb type format.
* You can parse the local context more quickly determining nesting
   context (ie - method in a class) for decoding symbols (like "this")
* There's a mode that tracks what tag you are in as you edit so you can
   jump through your history by name.
* From a self-dependency point of view, it enables incremental
   reparsing of a buffer.

-Eric

Re: IDE

[Przemysław Wojnowski]

W dniu 15.10.2015 o 14:57, Eric Ludlam pisze:
> * It lets you 'copy' a tag, and 'yank' it somewhere else.
> * It provides an accurate 'beginning of defun', 'end of defun',
>    'narrow to defun'
> * srecode can programmatically insert new tags between other tags
>    using a hueristic.
> * It can figure out where to place or find a header comment.
> * You can decorate the tags accurately
> * Provides a starting symbol for some commands, such as symref.
> * Adding folding of tags to a buffer is pretty easy (though that
>    contribution didn't get a release. :( )
> * The stuff imenu / which-func does but with more options such as
>    breadcrumb type format.
> * You can parse the local context more quickly determining nesting
>    context (ie - method in a class) for decoding symbols (like "this")
> * There's a mode that tracks what tag you are in as you edit so you can
>    jump through your history by name.
> * From a self-dependency point of view, it enables incremental
>    reparsing of a buffer.
>
> -Eric
>

IMHO Semantic + SRecode combo, even with information from only one buffer, is a 
great fit for implementation of many local refactorings: Extract Method, Extract 
Var/Const, Inline temp, etc. (see here: http://www.refactoring.com/catalog/)

Re: IDE

[Dmitry Gutov]

On 10/16/2015 01:00 PM, Przemysław Wojnowski wrote:

> IMHO Semantic + SRecode combo, even with information from only one
> buffer, is a great fit for implementation of many local refactorings:
> Extract Method, Extract Var/Const, Inline temp, etc. (see here:
> http://www.refactoring.com/catalog/)

In theory, it could be. But from what I've read from various mailing 
list postings, Semantic grammars often skip over the method contents. 
And that where most of the code lives (which you want to refactor).

Re: IDE

[Dmitry Gutov]

On 10/15/2015 03:57 PM, Eric Ludlam wrote:

> CEDET will store tags into a set of overlays in the buffer.  That means
> figuring out what tag the cursor is in is as fast as asking for what
> overlay the cursor is in.

I see. But you have to keep that info up-to-date as well, and that 
becomes less fast, because you implement it in Elisp. If we're comparing 
to an external program.

> Imenu stores it's tags in a list, so you need to scan the list to figure
> it out. Imenu's tags are also weak, so the elisp knows very little about
> the tag, only where it is, and enough to queue the reader.

All true. But we have other facilities as well. For instance, the modes 
which use SMIE for indentation can implement extraction of similar 
information, in a more accurate way.

If we were able to easily substitute in SMIE-based "current tag" 
implementation instead of using Wisent, that would be a plus.

> Yes.  There are other tools that do different pieces of what CEDET does.

I mean that you can't *really* use Semantic for "jump to a tag in the 
project", because one doesn't usually like to open all project files at 
once. But if the "daemon" proposal sees any development, maybe...

> * It lets you 'copy' a tag, and 'yank' it somewhere else.
> * It provides an accurate 'beginning of defun', 'end of defun',
>    'narrow to defun'

Emacs usually provides fairly accurate definitions of these as well.

> * srecode can programmatically insert new tags between other tags
>    using a hueristic.

I suppose it could use beginning-of-defun-function as well.

> * Provides a starting symbol for some commands, such as symref.

I wonder if it's ideal: in IntellijIDEA, say, you can click on any of 
the method's uses and to list the other references. With your scheme, 
however, one has to jump to its definition first.

For someone user to the former, it's counter-intuitive: you move point 
to 'bar' in foo.bar(), and instead semantic-symref suggests asking for 
uses of whatever function you're currently inside. If you don't actually 
read the prompt fully and just press y (or yes), the result will be 
puzzling.

> * The stuff imenu / which-func does but with more options such as
>    breadcrumb type format.

A new generic API could use a more detailed format as well.

> * You can parse the local context more quickly determining nesting
>    context (ie - method in a class) for decoding symbols (like "this")

Yes, you can usually resolve what 'this' is (and, consequently, 
this.foo()). But not what foo.bar() is, in the general case. Not in a 
duck-typed language anyway.

So, that's a problem. If we could, using semantic-symref could be made 
more natural.

> * There's a mode that tracks what tag you are in as you edit so you can
>    jump through your history by name.

That's pretty cool.

Re: IDE

[Eric Ludlam]

On 10/16/2015 09:05 AM, Dmitry Gutov wrote:
> On 10/15/2015 03:57 PM, Eric Ludlam wrote:
>
>> CEDET will store tags into a set of overlays in the buffer.  That means
>> figuring out what tag the cursor is in is as fast as asking for what
>> overlay the cursor is in.
>
> I see. But you have to keep that info up-to-date as well, and that 
> becomes less fast, because you implement it in Elisp. If we're 
> comparing to an external program.
>

The magic of overlays is that the overlays keep the bounds of the tag up 
to date for you.  Once you stop editing, and incremental parser fixes up 
any weird things you might have done, such as hacking a tag in half, 
adding new tags, etc.  Because it is incremental, is typically 
instantaneous.

>> Imenu stores it's tags in a list, so you need to scan the list to figure
>> it out. Imenu's tags are also weak, so the elisp knows very little about
>> the tag, only where it is, and enough to queue the reader.
>
> All true. But we have other facilities as well. For instance, the 
> modes which use SMIE for indentation can implement extraction of 
> similar information, in a more accurate way.
>
I am not familiar with SMIE as an acronym, but a google search makes it 
look like someone made a parser generator, in which case there is no 
real difference except that SMIE is tuned for indentation, and wisent is 
tuned for parsing tags.

> If we were able to easily substitute in SMIE-based "current tag" 
> implementation instead of using Wisent, that would be a plus.

The problem is the same (ie - go write a parser to find tags), except 
the SMIE tagger isn't implemented yet.

>> Yes.  There are other tools that do different pieces of what CEDET does.
>
> I mean that you can't *really* use Semantic for "jump to a tag in the 
> project", because one doesn't usually like to open all project files 
> at once. But if the "daemon" proposal sees any development, maybe...
>

If by "open all project files" you mean pull file contents into a buffer 
and leave it open, then that is not what semantic does.  The data is 
indeed resident in memory, but files stay closed unless the user asks to 
jump there.  The first time you make a request that needs searching, it 
will open files to parse them, but then it closes the file.  Later, it 
refers only the the cached data, not to the files.

>> * It lets you 'copy' a tag, and 'yank' it somewhere else.
>> * It provides an accurate 'beginning of defun', 'end of defun',
>>    'narrow to defun'
>
> Emacs usually provides fairly accurate definitions of these as well.

The difference is switching from "fairly" to "very", and mode authors 
would not need to go and write all those beginning-of-defun type overrides.

>
>> * Provides a starting symbol for some commands, such as symref.
>
> I wonder if it's ideal: in IntellijIDEA, say, you can click on any of 
> the method's uses and to list the other references. With your scheme, 
> however, one has to jump to its definition first.
>

There is always a desire to go two ways "where is the symbol under 
cursor", and "who uses the function I'm in".  I was describing the 
later.  The former is important, and not solved by my suggestion above.

>> * You can parse the local context more quickly determining nesting
>>    context (ie - method in a class) for decoding symbols (like "this")
>
> Yes, you can usually resolve what 'this' is (and, consequently, 
> this.foo()). But not what foo.bar() is, in the general case. Not in a 
> duck-typed language anyway.
>

CEDET/Semantic usually gets foo.bar() syntax correct, unless you are 
saying something else.  Can you elaborate?

> So, that's a problem. If we could, using semantic-symref could be made 
> more natural.

Yes.  There is a spot in symref waiting with a comment for that nugget 
of code to be written.  My primary concern is that of performance since 
that data is derived, not cached.

Eric

Re: IDE

[Dmitry Gutov]

On 10/17/2015 05:39 AM, Eric Ludlam wrote:

> The magic of overlays is that the overlays keep the bounds of the tag up
> to date for you.  Once you stop editing, and incremental parser fixes up
> any weird things you might have done, such as hacking a tag in half,
> adding new tags, etc.  Because it is incremental, is typically
> instantaneous.

Is that still true if you're editing at the beginning of the file?

> I am not familiar with SMIE as an acronym, but a google search makes it
> look like someone made a parser generator, in which case there is no
> real difference except that SMIE is tuned for indentation, and wisent is
> tuned for parsing tags.

My point is, Semantic grammars don't help with writing indentation code 
(could they?), and we already have SMIE grammars for several languages. 
It would make sense to be able to make do only with one or the other, 
not have them both for the same language.

>> If we were able to easily substitute in SMIE-based "current tag"
>> implementation instead of using Wisent, that would be a plus.
>
> The problem is the same (ie - go write a parser to find tags), except
> the SMIE tagger isn't implemented yet.

That's less work than writing a grammar, and it could be reusable (at 
least partially) across languages.

> If by "open all project files" you mean pull file contents into a buffer
> and leave it open, then that is not what semantic does.  The data is
> indeed resident in memory, but files stay closed unless the user asks to
> jump there.  The first time you make a request that needs searching, it
> will open files to parse them, but then it closes the file.  Later, it
> refers only the the cached data, not to the files.

I mean that you have to use etags at some point anyway. Or gtags, or 
id-utils, etc.

> The difference is switching from "fairly" to "very", and mode authors
> would not need to go and write all those beginning-of-defun type overrides.

True enough.

>> I wonder if it's ideal: in IntellijIDEA, say, you can click on any of
>> the method's uses and to list the other references. With your scheme,
>> however, one has to jump to its definition first.
>
> There is always a desire to go two ways "where is the symbol under
> cursor", and "who uses the function I'm in".  I was describing the
> later.  The former is important, and not solved by my suggestion above.

I was also describing the latter. Although, I suppose, my way could be 
implemented on top of what Semantic does now, as long as it can "jump to 
the definition" accurately.

> CEDET/Semantic usually gets foo.bar() syntax correct, unless you are
> saying something else.  Can you elaborate?

In the few languages it properly supports now? Maybe it does, most of 
the time (although not in the problem example I gave in this discussion: 
https://gist.github.com/dgutov/19c45ef43d1c90b96483; no matter the 
argument tee is called with, `C-c , J' jumps to the first function with 
that name).

But what about duck typed languages? If a method foo calls bar on its 
argument tee, we don't know the type of tee, all we know that it has a 
method bar.

>> So, that's a problem. If we could, using semantic-symref could be made
>> more natural.
>
> Yes.  There is a spot in symref waiting with a comment for that nugget
> of code to be written.  My primary concern is that of performance since
> that data is derived, not cached.

Not sure what you mean here (which part you believe to have to be 
written yet, or which data we're talking about).

But in any case, symref can afford to be a little slow when prompting 
for a symbol name, just as long the slowness is not proportional to the 
size of the repository (or the multiplier is small, at least).

Re: IDE

[Eric Ludlam]

On 10/16/2015 11:06 PM, Dmitry Gutov wrote:
> On 10/17/2015 05:39 AM, Eric Ludlam wrote:
>
>> The magic of overlays is that the overlays keep the bounds of the tag up
>> to date for you.  Once you stop editing, and incremental parser fixes up
>> any weird things you might have done, such as hacking a tag in half,
>> adding new tags, etc.  Because it is incremental, is typically
>> instantaneous.
>
> Is that still true if you're editing at the beginning of the file?
>

Yes.

>> I am not familiar with SMIE as an acronym, but a google search makes it
>> look like someone made a parser generator, in which case there is no
>> real difference except that SMIE is tuned for indentation, and wisent is
>> tuned for parsing tags.
>
> My point is, Semantic grammars don't help with writing indentation 
> code (could they?), and we already have SMIE grammars for several 
> languages. It would make sense to be able to make do only with one or 
> the other, not have them both for the same language.
>

Ah, I think I see what you are poking at.

There is nothing about semantic the framework that prevents using 
something like SMIE as a tagging engine.  It seems likely that if some 
language not supported by CEDET today has a good SMIE grammar, and if 
SMIE could be adapted to produce compatible tags (which isn't really 
that hard to do), and if SMIE has an entry point that allows parsing a 
tag set out of a region, then it would integrate into the semantic 
framework at the best level.

The difference is that the parser generators in Semantic today have 
optimizations for skipping parts of the buffer while tagging.  This lets 
it parse whole files more quickly, and more robustly.  Since it uses 
syntax tables to do that, I'll guess SMIE can do it, but I'm not that 
familiar with it.

Also, wisent type grammars could easily dump out data you need for 
indentation since actions can do anything you can write in lisp.  No one 
has tried to do it though.

>
>> If by "open all project files" you mean pull file contents into a buffer
>> and leave it open, then that is not what semantic does.  The data is
>> indeed resident in memory, but files stay closed unless the user asks to
>> jump there.  The first time you make a request that needs searching, it
>> will open files to parse them, but then it closes the file. Later, it
>> refers only the the cached data, not to the files.
>
> I mean that you have to use etags at some point anyway. Or gtags, or 
> id-utils, etc.
>

Have to, no.  Want to, probably. Those tools provide nice short-cuts 
when doing simple name lookup.

The mechanism in use is that there are detailed taggers, like those 
currently written for Semantic using wisent type parsers, and weak 
taggers, like GNU Global.

Once a buffer is open, the detailed tagger runs and is truth.  When a 
search occurs, a process of pulling all relevant databases together is 
started.  This includes tags from the detailed taggers already cached, 
and high level 'omniscient' taggers that are usually weak, but have 
scanned the whole project.  The output of the search includes a database 
table, and tags found in that table.  The tags are additionally refined 
based on whatever the layered criteria is, and when the code decides to 
work with a detailed tag from the search, it forces it to be real, and 
pulls it into a buffer if necessary, and any problems with the weak 
tagger is resolved.

The key here is that the detailed tagger is needed to do complex tasks, 
but weak taggers are great to integrate in due to the speed advantage.  
Having a mix gives you the best possible results.

>
>> CEDET/Semantic usually gets foo.bar() syntax correct, unless you are
>> saying something else.  Can you elaborate?
>
> In the few languages it properly supports now? Maybe it does, most of 
> the time (although not in the problem example I gave in this 
> discussion: https://gist.github.com/dgutov/19c45ef43d1c90b96483; no 
> matter the argument tee is called with, `C-c , J' jumps to the first 
> function with that name).
>

I missed that before.  The answer here is that:

C-c , J RET

has two results.  Since that is a prompt based query and only the string 
is available once it is up.  If instead you put the cursor on "tee" and do

M-x semantic-ia-fast-jump RET

it will go to the right spot.

For the keybinding you were using, the workflow is:

C-c , J tee TAB

shows you where it would jump, then

TAB

refines to the next possible target.  Hit RET when it shows you the 
place you do what to jump to.

It needs more typing since the default isn't on the prompt.  Perhaps 
that is a possible improvement.

Can all this be improved - sure.  I agree it would be nicer to mix the 
two workflows better.  Most of my time goes into the parsers etc.

> But what about duck typed languages? If a method foo calls bar on its 
> argument tee, we don't know the type of tee, all we know that it has a 
> method bar.
>

In the workflow above, you can just keep pressing TAB to pick the one 
you want.

Eric

Re: IDE

[Stephen Leake]

Eric Ludlam <ericludlam@gmail.com> writes:

> Also, wisent type grammars could easily dump out data you need for
> indentation since actions can do anything you can write in lisp.  No
> one has tried to do it though.

Not quite true; I tried for Ada mode. But I ended up rewriting the
parser to be generalized LALR; see
http://stephe-leake.org/emacs/ada-mode/emacs-ada-mode.html, and the
ada-mode package in Gnu ELPA.

That parser is missing the incremental features of wisent, mostly
because I ran out of steam.

I had also tried SMIE before that; it was not powerful enough for Ada.

-- 
-- Stephe

Re: IDE

[Dmitry Gutov]

On 10/17/2015 03:45 PM, Eric Ludlam wrote:

> There is nothing about semantic the framework that prevents using
> something like SMIE as a tagging engine.  It seems likely that if some
> language not supported by CEDET today has a good SMIE grammar, and if
> SMIE could be adapted to produce compatible tags (which isn't really
> that hard to do), and if SMIE has an entry point that allows parsing a
> tag set out of a region, then it would integrate into the semantic
> framework at the best level.

With SMIE, though, it might make sense to optimize the process and only 
ask it for the current tag (and maybe the nearby ones), because it would 
otherwise be slower (no incremental parsing, or anything like that). So 
a straightforward conversion might not be optimal.

But from the main examples you gave of tag usage, knowing the current 
tag is by far the most important.

> The difference is that the parser generators in Semantic today have
> optimizations for skipping parts of the buffer while tagging.  This lets
> it parse whole files more quickly, and more robustly.  Since it uses
> syntax tables to do that, I'll guess SMIE can do it, but I'm not that
> familiar with it.

Skipping strings and comments - probably. Method bodies - unlikely 
(except in C-like languages: skipping over a pair of braces is easy).

> Also, wisent type grammars could easily dump out data you need for
> indentation since actions can do anything you can write in lisp.  No one
> has tried to do it though.

Someone should try and tell us about it.

> The mechanism in use is that there are detailed taggers, like those
> currently written for Semantic using wisent type parsers, and weak
> taggers, like GNU Global.
>
> Once a buffer is open, the detailed tagger runs and is truth.  When a
> search occurs, a process of pulling all relevant databases together is
> started.  This includes tags from the detailed taggers already cached,
> and high level 'omniscient' taggers that are usually weak, but have
> scanned the whole project.

Well, that's my point: if you don't use etags, to have all project files 
indexed you have to open each of them at some point.

Some languages have import systems where you have to declare all file's 
dependencies at its top (and so you can know which files to reindex if 
you're only interested in this file; although even that could be a lot). 
But in Ruby and pre-ES6 JavaScript, you don't usually have that.

> The output of the search includes a database
> table, and tags found in that table.  The tags are additionally refined
> based on whatever the layered criteria is, and when the code decides to
> work with a detailed tag from the search, it forces it to be real, and
> pulls it into a buffer if necessary, and any problems with the weak
> tagger is resolved.

Yup.

> The key here is that the detailed tagger is needed to do complex tasks,
> but weak taggers are great to integrate in due to the speed advantage.
> Having a mix gives you the best possible results.

If the program generating TAGS is smart enough, the result can be pretty 
accurate already (and contain qualified method name). Eli has recently 
made some improvements to that effect to etags.

> If instead you put the cursor on "tee" and do
>
> M-x semantic-ia-fast-jump RET
>
> it will go to the right spot.

Indeed it does, thanks. But it goes to the second definition 
irrespective of which argument is passed to tee. Which brings us back to 
the completion problem I reported with that snippet.

> For the keybinding you were using, the workflow is:

Right, why doesn't semantic-ia-fast-jump have a default keybinding? I 
was only looking for the appropriate command in the active keymaps.

> C-c , J tee TAB
>
> shows you where it would jump, then
>
> TAB

Oh, so it uses something more advanced than completing-read. That's 
unexpected.

>> But what about duck typed languages? If a method foo calls bar on its
>> argument tee, we don't know the type of tee, all we know that it has a
>> method bar.
>
> In the workflow above, you can just keep pressing TAB to pick the one
> you want.

I would want semantic-ia-fast-jump to work with them, of course. As well 
as semantic-symref.

Re: IDE

[David Engster]

Dmitry Gutov writes:
> (except in C-like languages: skipping over a pair of braces is easy).

After the pre-processor has run over it, yes...

#ifdef WHATEVER
struct something : public foo {
#else
struct something : public bar {
#
 // body
}

This stuff is more common than one would think...

-David

Re: IDE

[Dmitry Gutov]

On 10/17/2015 05:41 PM, David Engster wrote:
> Dmitry Gutov writes:
>> (except in C-like languages: skipping over a pair of braces is easy).
>
> After the pre-processor has run over it, yes...

Fair point. But whatever your solution for that is, it's unlikely to 
help to quickly jump over do ... end blocks in Ruby.

Re: IDE

[Eric Ludlam]

On 10/17/2015 10:25 AM, Dmitry Gutov wrote:
> On 10/17/2015 03:45 PM, Eric Ludlam wrote:
>
>> There is nothing about semantic the framework that prevents using
>> something like SMIE as a tagging engine.  It seems likely that if some
>> language not supported by CEDET today has a good SMIE grammar, and if
>> SMIE could be adapted to produce compatible tags (which isn't really
>> that hard to do), and if SMIE has an entry point that allows parsing a
>> tag set out of a region, then it would integrate into the semantic
>> framework at the best level.
>
> With SMIE, though, it might make sense to optimize the process and 
> only ask it for the current tag (and maybe the nearby ones), because 
> it would otherwise be slower (no incremental parsing, or anything like 
> that). So a straightforward conversion might not be optimal.
>
> But from the main examples you gave of tag usage, knowing the current 
> tag is by far the most important.

There are many reasons why having all the tags in the current buffer is 
useful.  The simplest of which are things like imenu - the menu part, 
speedbar, or ECB listings of all tags so you can jump to them.  There is 
a command for jumping to a tag in a buffer by name with completion.  
There's an isearch flavor that searched only tag names in the current 
buffer.

Of course, the most important is that if you have all the detailed tags 
from a file that is closed, you don't need to open the file again if you 
need to reference it.

>> The mechanism in use is that there are detailed taggers, like those
>> currently written for Semantic using wisent type parsers, and weak
>> taggers, like GNU Global.
>>
>> Once a buffer is open, the detailed tagger runs and is truth. When a
>> search occurs, a process of pulling all relevant databases together is
>> started.  This includes tags from the detailed taggers already cached,
>> and high level 'omniscient' taggers that are usually weak, but have
>> scanned the whole project.
>
> Well, that's my point: if you don't use etags, to have all project 
> files indexed you have to open each of them at some point.
>

Sure - something does have to open them.  If not emacs, then something will.

> Some languages have import systems where you have to declare all 
> file's dependencies at its top (and so you can know which files to 
> reindex if you're only interested in this file; although even that 
> could be a lot). But in Ruby and pre-ES6 JavaScript, you don't usually 
> have that.
>

Using imports and includes, the semantic framework tracks what the 
dependencies are, and will reparse files that change outside of Emacs.  
It also will only open files listed as dependencies.

It will also, in idle time, pull in things it suspects you might want to 
open later, but that is to make visiting those things faster.  It is 
easy to turn off, and we could turn it off by default if it is causing 
problems.

>
>> The key here is that the detailed tagger is needed to do complex tasks,
>> but weak taggers are great to integrate in due to the speed advantage.
>> Having a mix gives you the best possible results.
>
> If the program generating TAGS is smart enough, the result can be 
> pretty accurate already (and contain qualified method name). Eli has 
> recently made some improvements to that effect to etags.
>

If it can produce the needed data, then it would be simple enough to 
swap it in as an external parser.  Since Emacs has full control over 
etags, it could be adapted to handle regions, and thus fit in just fine.

>> If instead you put the cursor on "tee" and do
>>
>> M-x semantic-ia-fast-jump RET
>>
>> it will go to the right spot.
>
> Indeed it does, thanks. But it goes to the second definition 
> irrespective of which argument is passed to tee. Which brings us back 
> to the completion problem I reported with that snippet.
>

You are right, I missed that.  I've started a new test suite around this 
missing feature and will look into it.

>> For the keybinding you were using, the workflow is:
>
> Right, why doesn't semantic-ia-fast-jump have a default keybinding? I 
> was only looking for the appropriate command in the active keymaps.
>

I've run out of steam trying to design the perfect set of keybindings.  
Everyone seems to like something different.  ;)

>> C-c , J tee TAB
>>
>> shows you where it would jump, then
>>
>> TAB
>
> Oh, so it uses something more advanced than completing-read. That's 
> unexpected.
>

Yes.  I wrote my own so that the completion engine could weed out the 
maximum number of possible completions and speed up the process.  Some 
of the tricks others have done that do take all possible completions in 
their UIs I think have obsoleted that advantage.

But also, I was looking for ways to allow selection of different methods 
of the same name that was easy and quick, and this is what I came up with.


Eric

Re: IDE

[Dmitry Gutov]

On 10/19/2015 02:51 PM, Eric Ludlam wrote:

> There are many reasons why having all the tags in the current buffer is
> useful.  The simplest of which are things like imenu - the menu part,
> speedbar, or ECB listings of all tags so you can jump to them.  There is
> a command for jumping to a tag in a buffer by name with completion.
> There's an isearch flavor that searched only tag names in the current
> buffer.

All right. But I would say that IMenu was conceived as a poor cousin to 
tagging. After all, if we have up-to-date information about tags in the 
current project, we can jump anywhere, not just in the current file.

My point is, we could have a more limited "protocol" to be used when we 
don't have a list of tags for the current file (and "jump to xxx" 
command is implemented via overrides, using an external tool). And a 
"full" protocol for when the tags are available.

> Using imports and includes, the semantic framework tracks what the
> dependencies are, and will reparse files that change outside of Emacs.
> It also will only open files listed as dependencies.

Right. It's quite nifty. I was just pointing out that it won't work for 
some languages.

> If it can produce the needed data, then it would be simple enough to
> swap it in as an external parser.  Since Emacs has full control over
> etags, it could be adapted to handle regions, and thus fit in just fine.

I suppose so. ctags (not in the Emacs repo) supports more file formats, 
though.

> You are right, I missed that.  I've started a new test suite around this
> missing feature and will look into it.

Thank you.

> I've run out of steam trying to design the perfect set of keybindings.
> Everyone seems to like something different.  ;)

I understand the difficulty, but surely you'd want to advertise 
semantic-ia-fast-jump? It's more precise, and thus, more powerful.

> But also, I was looking for ways to allow selection of different methods
> of the same name that was easy and quick, and this is what I came up with.

We could use some mix of the current xref behavior, with this one, for 
xref-find-definitions. xref is static, but shows everything. This one is 
dynamic, but shows only one option at a time.

Re: IDE

[Eric Ludlam]

On 10/19/2015 08:56 PM, Dmitry Gutov wrote:
> On 10/19/2015 02:51 PM, Eric Ludlam wrote:
>
>> There are many reasons why having all the tags in the current buffer is
>> useful.  The simplest of which are things like imenu - the menu part,
>> speedbar, or ECB listings of all tags so you can jump to them. There is
>> a command for jumping to a tag in a buffer by name with completion.
>> There's an isearch flavor that searched only tag names in the current
>> buffer.
>
> All right. But I would say that IMenu was conceived as a poor cousin 
> to tagging. After all, if we have up-to-date information about tags in 
> the current project, we can jump anywhere, not just in the current file.
>
It was an example, and there are plenty of cases where you want to 
restrict your destination to be local simply because completing on a 
zillion options is slow, regardless of the tool used to create said list.

> My point is, we could have a more limited "protocol" to be used when 
> we don't have a list of tags for the current file (and "jump to xxx" 
> command is implemented via overrides, using an external tool). And a 
> "full" protocol for when the tags are available.
>

That might work in a simple case, but for current tag to truly be 
accurate, you need to parse the whole buffer from start to point so you 
don't get confused by text in strings, text in comments, nested function 
syntax, and #ifdef type syntax.  Once you do that, you might as well 
just parse the whole buffer and cache it.

>
>> I've run out of steam trying to design the perfect set of keybindings.
>> Everyone seems to like something different.  ;)
>
> I understand the difficulty, but surely you'd want to advertise 
> semantic-ia-fast-jump? It's more precise, and thus, more powerful.
>

Yes, there are some guidelines on what functions there are but I haven't 
tied much together into minor modes or keybindings.  We used to have 
'senator' mode which we updated with new stuff, but the Emacs merge has 
gotten that a bit confused.

As an added bonus, all the semantic-ia-* functions were originally meant 
to be examples on how to use the infrastructure for someone who wanted 
build interface functions, so they lack a bit of polish and haven't been 
promoted into keymaps.

It would be great if someone interested in the user interface side could 
help out getting the right interface together on top of some of the 
semantic concepts.

Eric

Re: IDE

[David Engster]

Eli Zaretskii writes:
> And if anyone _really_ cares about supporting C/C++, they should be
> working with and on GCC's libcc1,

Getting the AST is not a technical problem; libcc1 does not change
anything.

-David

Re: IDE

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,  adatgyujto@gmail.com,  emacs-devel@gnu.org
> Date: Sat, 10 Oct 2015 11:51:14 +0200
> 
> Eli Zaretskii writes:
> > And if anyone _really_ cares about supporting C/C++, they should be
> > working with and on GCC's libcc1,
> 
> Getting the AST is not a technical problem; libcc1 does not change
> anything.

We don't even know whether libcc1 provides any features useful for an
IDE, including information that could be used fro completion.

We also don't know what is the final verdict on having GCC emit tree
information.

I'd expect a person who is motivated to work on these features, if we
have such a person among us, to be on top of those two issues, as well
as many others.

But I agree: it's not a technical problem.  It's a problem of being
motivated enough to overcome such obstacles.

Re: IDE

[David Engster]

Eli Zaretskii writes:
>> From: David Engster <deng@randomsample.de>
>> Cc: Dmitry Gutov <dgutov@yandex.ru>,  adatgyujto@gmail.com,  emacs-devel@gnu.org
>> Date: Sat, 10 Oct 2015 11:51:14 +0200
>> 
>> Eli Zaretskii writes:
>> > And if anyone _really_ cares about supporting C/C++, they should be
>> > working with and on GCC's libcc1,
>> 
>> Getting the AST is not a technical problem; libcc1 does not change
>> anything.
>
> We don't even know whether libcc1 provides any features useful for an
> IDE, including information that could be used fro completion.

AFAICS, libcc1 does not provide anything new. It seems to be some kind
of library wrapper for a plugin to access gcc's frontend. It's not even
mentioned in gcc's release notes.

> We also don't know what is the final verdict on having GCC emit tree
> information.

Any year now.

> I'd expect a person who is motivated to work on these features, if we
> have such a person among us, to be on top of those two issues, as well
> as many others.
>
> But I agree: it's not a technical problem.  It's a problem of being
> motivated enough to overcome such obstacles.

Yes, I should've stuck with clang in the first place.

-David

Re: IDE

[Lluís]

Eli Zaretskii writes:
[...]
>> For C/C++, the community has Irony and Rtags, both based on libclang. If 
>> libclang is unacceptable for you, you probably know a more appropriate 
>> mailing list to bring that up at.

> Let's not reiterate past discussions: you forget CEDET.

Just thinking out loud: it seems to me that many people forget that CEDET is,
from my understanding, a framework for writing tools first, and a set of such
example tools later.


> And if anyone _really_ cares about supporting C/C++, they should be
> working with and on GCC's libcc1, which is available for quite some
> time already.

If this is the libgcc1 you mean [1], I'm not sure it's suitable for code
completion. Instead, GCC should be modified to hook into the frontend parser or
the generic AST and then parsing that, which is no small feat. Fortunately,
hooking is already possible using GCC plugins [2].

[1] http://gcc.gnu.org/onlinedocs/gccint/Libgcc.html
[2] http://www.codesynthesis.com/~boris/blog/2010/05/03/parsing-cxx-with-gcc-plugin-part-1/

With this, it's a relatively easy (but time-consuming) task to build an external
tool that parses files on-demand. The ideal would be some kind of persistent
daemon+database, as was discussed in the CEDET list quite some time ago, but
that's an entirely different story.


[...]
>> Would you expect the programs mentioned above to become a part of Emacs? 

> I expect to see a coherent, orchestrated effort towards having an IDE
> mode in Emacs.  I don't see it, certainly not in discussions on this
> list.  I also don't see any commits that would provide evidence of
> such an effort.

> If such activities happen somewhere else, I would suggest their
> participants to come here and work with and within the core.  For
> starters, I don't imagine they would succeed without some significant
> changes and additions in the core infrastructure.  The place to
> discuss that is here.

I think that things are happening outside (completion, automatic project
detection, etc) because there is no common goal on what features should be
available and through what interface.

This, and that giving an opinion on these topics is way much less work than
actually implementing them (and I include myself on the first group of
non-implementors).


Cheers,
  Lluis

-- 
"And it's much the same thing with knowledge, for whenever you learn
something new, the whole world becomes that much richer."
-- The Princess of Pure Reason, as told by Norton Juster in The Phantom
Tollbooth

Re: IDE

[John Wiegley]

>>>>> Lluís  <xscript@gmx.net> writes:

> Eli Zaretskii writes:
> [...]
>>> For C/C++, the community has Irony and Rtags, both based on libclang. If
>>> libclang is unacceptable for you, you probably know a more appropriate
>>> mailing list to bring that up at.

>> Let's not reiterate past discussions: you forget CEDET.

CEDET first came out in 2003. If it were the answer to our present questions,
we would not be asking them.

I'm willing to hear how CEDET can offer solutions to issues we've brought up,
but I won't curtail the discussion "because CEDET".

I'm also OK with reiteration, because I wasn't present during those past
discussions, and I want to know what is most relevant today.

John

Re: IDE

[David Kastrup]

"John Wiegley" <johnw@newartisans.com> writes:

>>>>>> Lluís  <xscript@gmx.net> writes:
>
>> Eli Zaretskii writes:
>> [...]
>>>> For C/C++, the community has Irony and Rtags, both based on libclang. If
>>>> libclang is unacceptable for you, you probably know a more appropriate
>>>> mailing list to bring that up at.
>
>>> Let's not reiterate past discussions: you forget CEDET.
>
> CEDET first came out in 2003. If it were the answer to our present
> questions, we would not be asking them.

But since it did come out in 2003, we really should be asking _why_ it
isn't the answer to our present questions, in order to avoid the effort
of creating CEDET2 and CEDET3.

> I'm willing to hear how CEDET can offer solutions to issues we've
> brought up, but I won't curtail the discussion "because CEDET".

I don't think the idea is to curtail it but rather to _shape_ it.  If we
decide we need $x and CEDET provides $x, then either we haven't fully
figured out the details of the $x we need or CEDET does something wrong
when providing it.  Figuring out either will hopefully save us time in
arriving at something actually doing what we want.

-- 
David Kastrup

Re: IDE

[John Wiegley]

>>>>> David Kastrup <dak@gnu.org> writes:

> But since it did come out in 2003, we really should be asking _why_ it isn't
> the answer to our present questions, in order to avoid the effort of
> creating CEDET2 and CEDET3.

I certainly do want to avoid CEDET2.

> I don't think the idea is to curtail it but rather to _shape_ it. If we
> decide we need $x and CEDET provides $x, then either we haven't fully
> figured out the details of the $x we need or CEDET does something wrong when
> providing it. Figuring out either will hopefully save us time in arriving at
> something actually doing what we want.

I will not approach this by asking how CEDET can be improved to meet the needs
of an Emacs IDE. That is the most likely path leading to CEDET2.

Emacs' needs as an IDE should be considered on their own, as I've said before.
Any or all existing methodologies can be taken into account, but none deserve
preference until an architecture begins to take shape.

John

Re: IDE

[Eric Ludlam]

On 10/13/2015 12:28 PM, David Kastrup wrote:
> "John Wiegley" <johnw@newartisans.com> writes:
>
>>>>>>> Lluís  <xscript@gmx.net> writes:
>>
>>> Eli Zaretskii writes:
>>> [...]
>>>>> For C/C++, the community has Irony and Rtags, both based on libclang. If
>>>>> libclang is unacceptable for you, you probably know a more appropriate
>>>>> mailing list to bring that up at.
>>
>>>> Let's not reiterate past discussions: you forget CEDET.
>>
>> CEDET first came out in 2003. If it were the answer to our present
>> questions, we would not be asking them.
>
> But since it did come out in 2003, we really should be asking _why_ it
> isn't the answer to our present questions, in order to avoid the effort
> of creating CEDET2 and CEDET3.

Based on the many emails I've seen on the topic, I suspect the answer is:

* It is hard to configure (ie - setting up project files,
   include paths, or whatever.)
* Specific implementations are incomplete (ie - c++ || other parser is
   imperfect, the project system doesn't implement some feature, etc)
* It is compared against better staffed tools

>> I'm willing to hear how CEDET can offer solutions to issues we've
>> brought up, but I won't curtail the discussion "because CEDET".
>
> I don't think the idea is to curtail it but rather to _shape_ it.  If we
> decide we need $x and CEDET provides $x, then either we haven't fully
> figured out the details of the $x we need or CEDET does something wrong
> when providing it.  Figuring out either will hopefully save us time in
> arriving at something actually doing what we want.

My main concern is about folks claiming CEDET is complicated (which it 
is) then oversimplifying the problem space to kick off some new thing 
which will likely end up just as complicated.

I know I thought the problem space seemed simple when I started.  I 
might not have started if I'd known how big it is.

Eric

Re: IDE

[John Wiegley]

>>>>> Eric Ludlam <eric@siege-engine.com> writes:

> My main concern is about folks claiming CEDET is complicated (which it is)
> then oversimplifying the problem space to kick off some new thing which will
> likely end up just as complicated.

> I know I thought the problem space seemed simple when I started. I might not
> have started if I'd known how big it is.

I appreciate your experience and point of view in this, Eric, and definitely
want you to be a part of this conversation. This same phenomenon happens with
build tools: there's a new one almost every year, because everyone thinks
they're trivial, yet no one has ever fully solved the problem.

John

Re: IDE

[David Kastrup]

Eric Ludlam <eric@siege-engine.com> writes:

> On 10/13/2015 12:28 PM, David Kastrup wrote:
>> "John Wiegley" <johnw@newartisans.com> writes:
>>
>>>>>>>> Lluís  <xscript@gmx.net> writes:
>>>
>>>> Eli Zaretskii writes:
>>>> [...]
>>>>>> For C/C++, the community has Irony and Rtags, both based on libclang. If
>>>>>> libclang is unacceptable for you, you probably know a more appropriate
>>>>>> mailing list to bring that up at.
>>>
>>>>> Let's not reiterate past discussions: you forget CEDET.
>>>
>>> CEDET first came out in 2003. If it were the answer to our present
>>> questions, we would not be asking them.
>>
>> But since it did come out in 2003, we really should be asking _why_ it
>> isn't the answer to our present questions, in order to avoid the effort
>> of creating CEDET2 and CEDET3.
>
> Based on the many emails I've seen on the topic, I suspect the answer is:
>
> * It is hard to configure (ie - setting up project files,
>   include paths, or whatever.)
> * Specific implementations are incomplete (ie - c++ || other parser is
>   imperfect, the project system doesn't implement some feature, etc)
> * It is compared against better staffed tools

I got rid of it because it tended to eat all my CPU repeatedly digging
through buffers and files in the background.  I don't want some tool to
go treasure-hunting for hours in my directories without concrete cause,
then restart for inscrutable reasons.

It had its own idea of projects not matching the projects I was working
with, and it's an absolute no-go for Emacs to meddle with project
organization: I want to be able to jump in with Emacs into any project
without any pre- or post-configuration.

Maybe that's a decisive difference between what people got to expect
from an IDE and I expect from Emacs: if someone develops stuff in Visual
C++, everybody in the project is expected to use the project
organization tools of the Visual C++ IDE.  But I don't want my choice of
Emacs as an editor bleed all over a project.

Now you'll say that EDE (or Semantic, or whatever other component) is
entirely optional but it's hard to figure out just what the relations of
the various parts of CEDET are.  If you want to just work with the code
you have and not get stuff messed up, at some point of time it's easier
to just forego the whole inscrutable package and simplify one's life.

Again, that's a main difference to what a normal IDE is doing: it tends
to focus on a small set of languages and does them well when I buy into
the IDE, and I can use IDE features as needed.

But my buy-in was to Emacs.  I don't want to buy into a competing
framework CEDET.  If I want completion, I enable an option or package
for it, and I don't want it to come with a host of things I have no idea
how to keep from messing with my work environment.  It's nice when there
is some framework with which major mode writers can easily provide a lot
of functionality commonly expected of IDEs.  But CEDET appears to be
mainly a user choice, and it leaves the user with the job of maintaining
Emacs/CEDET integrity for his workflows.

And it did not particularly help that seminal parts of CEDET like its
parser generators were kept out of Emacs for very, very long: you needed
to install a third-party CEDET in order to even be able to maintain some
Emacs-internal modes.

-- 
David Kastrup

Re: IDE

[Eric Ludlam]

On 10/14/2015 04:09 AM, David Kastrup wrote:
> Eric Ludlam <eric@siege-engine.com> writes:
>
>> On 10/13/2015 12:28 PM, David Kastrup wrote:
>> Based on the many emails I've seen on the topic, I suspect the answer 
>> is: * It is hard to configure (ie - setting up project files, include 
>> paths, or whatever.) * Specific implementations are incomplete (ie - 
>> c++ || other parser is imperfect, the project system doesn't 
>> implement some feature, etc) * It is compared against better staffed 
>> tools 
> I got rid of it because it tended to eat all my CPU repeatedly digging
> through buffers and files in the background.  I don't want some tool to
> go treasure-hunting for hours in my directories without concrete cause,
> then restart for inscrutable reasons.
>
> It had its own idea of projects not matching the projects I was working
> with, and it's an absolute no-go for Emacs to meddle with project
> organization: I want to be able to jump in with Emacs into any project
> without any pre- or post-configuration.
Thanks for the diverse feedback.

The need to move your files into some new structure is something I've 
always avoided.  There is a "do it for you" project structure if you 
don't care, and several other types that just uses what you have, and 
can detect a bunch of variants without leaving its own files behind.

>
> Maybe that's a decisive difference between what people got to expect
> from an IDE and I expect from Emacs: if someone develops stuff in Visual
> C++, everybody in the project is expected to use the project
> organization tools of the Visual C++ IDE.  But I don't want my choice of
> Emacs as an editor bleed all over a project.
>
> Now you'll say that EDE (or Semantic, or whatever other component) is
> entirely optional but it's hard to figure out just what the relations of
> the various parts of CEDET are.  If you want to just work with the code
> you have and not get stuff messed up, at some point of time it's easier
> to just forego the whole inscrutable package and simplify one's life

The puzzle for me here is that while the different pieces are 
technically independent, the more complex tasks, such as completion, 
depend on the other tools doing their job.  Good smart completion 
depends on a knowledge of a project's structure to find headers (C/C++), 
and it also depends on rummaging around in your files to find the needed 
symbols.

AFAIK, every smart completion engine out there has the same dependency.  
There are plenty of others that don't, like Global which just finds 
what's there and makes the most of it, but it isn't smart completion.

I suspect what you'd really like is to say "yeah, I'd like some smart 
completion with a side of API doc", and have an auto-configure thingy do 
the rest.  Sounds great!  To make that happen though, we need Emacs to 
be taught how to detect your files and rummage through them to make it 
happen.  If you work on code of a style I or other contributors never 
worked on, it probably isn't in CEDET.

Pulling in external tools like gcc, clang, whatever to do the work is a 
great way to make that happen as it pushes the CPU work off of Emacs' 
thread, and in some cases brings knowledge of the project along with 
it.  Doing that type of integration can be done with CEDET's framework, 
or independent of it.  I am not advocating to not do that type of 
integration, but to consider doing it in CEDET's framework because:

a) it will be easier than starting from scratch
b) doesn't preclude other types of integration later

On 10/14/2015 06:47 AM, Dmitry Gutov wrote:
> My already-stated impression is that it's over-specialized and tightly 
> coupled.
>

There are definitely dependencies.  I don't think it is 
over-specialized, but perhaps overly-generalized.  Every layer was set 
up so new languages, modes, projects, whatever can be slotted into the 
system.  The tendency is that many are not complete which lends itself 
to disappointment.  This is not uncommon in Emacs. There are lots of 
modes floating around with no indentation, poor syntax tables and 
incomplete highlighting.

> Not saying that the problem domain is easy, but being able to use 
> different pieces of the solution separately would go a long way 
> towards alleviating the complaint that certain other parts are 
> incomplete.
>

I agree, this would be nice.

> Especially if it were easier to swap in different solutions for some 
> of those parts (and do entirely without some others), and do that in 
> not too many lines, all as part of the user's configuration.

It is possible to swap in different solutions (under the CEDET 
framework) but in many cases, there is currently only one solution.

In these conversations it is hard to distinguish if the (usually valid) 
criticism are about CEDET the framework, or about various 
implementations under CEDET.

It is also hard since I don't really have time to address the issues raised.

Eric

Re: IDE

[Dmitry Gutov]

On 10/14/2015 03:05 PM, Eric Ludlam wrote:

> The puzzle for me here is that while the different pieces are
> technically independent, the more complex tasks, such as completion,
> depend on the other tools doing their job.  Good smart completion
> depends on a knowledge of a project's structure to find headers (C/C++),
> and it also depends on rummaging around in your files to find the needed
> symbols.

It seems what we need is a set of "juncture points", which define how 
separate systems/tools should communicate. That's what xref and 
project.el are about, for finding locations and project information 
respectively; everything else in there is mostly for convenience.

> There are definitely dependencies.  I don't think it is
> over-specialized, but perhaps overly-generalized.

Could be both. But by over-specialized, I mean that a lot of stuff in 
e.g. ede-project doesn't make sense for many projects. targets, for 
example. And mailinglist/web-site-url/ftp-site?..

I'm also under impression that EDE projects are always one directory 
deep (and for each subdirectory, you need subprojects). Which reflects 
the common Makefile usage, but not how projects are structured under 
many other build systems.

> It is possible to swap in different solutions (under the CEDET
> framework) but in many cases, there is currently only one solution.

Yet still, to even become pluggable, a piece of functionality has to buy 
into the CEDET fundamentals, like using EIEIO, mode-locals, and CEDET's 
base classes.

Re: IDE

[Eric Ludlam]

On 10/14/2015 11:40 PM, Dmitry Gutov wrote:
> On 10/14/2015 03:05 PM, Eric Ludlam wrote:
>
>> The puzzle for me here is that while the different pieces are
>> technically independent, the more complex tasks, such as completion,
>> depend on the other tools doing their job.  Good smart completion
>> depends on a knowledge of a project's structure to find headers (C/C++),
>> and it also depends on rummaging around in your files to find the needed
>> symbols.
>
> It seems what we need is a set of "juncture points", which define how 
> separate systems/tools should communicate. That's what xref and 
> project.el are about, for finding locations and project information 
> respectively; everything else in there is mostly for convenience.

Indeed.  That is a similarity.

>> There are definitely dependencies.  I don't think it is
>> over-specialized, but perhaps overly-generalized.
>
> Could be both. But by over-specialized, I mean that a lot of stuff in 
> e.g. ede-project doesn't make sense for many projects. targets, for 
> example. And mailinglist/web-site-url/ftp-site?..
>

There is indeed a bunch of extra not so useful things for all classes in 
the baseclasses.  I would not claim it could not be improved.

Some, such as 'version' and 'name' seem trivial, but become more 
convenient when you start browsing through your open projects.

> I'm also under impression that EDE projects are always one directory 
> deep (and for each subdirectory, you need subprojects). Which reflects 
> the common Makefile usage, but not how projects are structured under 
> many other build systems.
>

Different projects handle this differently.  The original set regarding 
both Automake and Makefile based projects are as you describe.

Last year I simplified a bunch of systems regarding project detection 
and directory hierarchy.  At this point, the 'simplest' project as far 
as setup is the 'generic' project type, one of which finds a project 
based on your VC root (git, cvs, and a few others). It is one project 
covering all the subdirectories.  You can 'customize' the project and 
just fill in the blanks for your build system, include-path and others, 
or just ignore it all.

>> It is possible to swap in different solutions (under the CEDET
>> framework) but in many cases, there is currently only one solution.
>
> Yet still, to even become pluggable, a piece of functionality has to 
> buy into the CEDET fundamentals, like using EIEIO, mode-locals, and 
> CEDET's base classes.
>

Yes.  The structure I started building used traditional emacsy tools for 
handling structure and buffer local behaviours.  The broader the tool 
became, the more of a PITA it was managing all the random lists of 
behaviour differences, and random list references, so I put in missing 
infrastructure.

You will note other tools that wrangle data now use defstruct (now part 
of emacs) or seem to create their own data structure with their own 
accessors, so building such tools is not unusual.  I just tried to make 
mine more generally useful.

Eric

Re: IDE

[Dmitry Gutov]

On 10/15/2015 04:08 PM, Eric Ludlam wrote:

> Indeed.  That is a similarity.

Not an accidental one.

> Some, such as 'version' and 'name' seem trivial, but become more
> convenient when you start browsing through your open projects.

I'm not saying there aren't useful, just not essential.

> Different projects handle this differently.  The original set regarding
> both Automake and Makefile based projects are as you describe.

I see. And the distinction is managed by the 
ede-find-subproject-for-directory implementation.

> You can 'customize' the project and
> just fill in the blanks for your build system, include-path and others,
> or just ignore it all.

How does one customize an instance of the generic project?

> You will note other tools that wrangle data now use defstruct (now part
> of emacs) or seem to create their own data structure with their own
> accessors, so building such tools is not unusual.  I just tried to make
> mine more generally useful.

Sure.

Re: IDE

[Eric Ludlam]

On 10/15/2015 05:03 PM, Dmitry Gutov wrote:
> On 10/15/2015 04:08 PM, Eric Ludlam wrote:
>
>> You can 'customize' the project and
>> just fill in the blanks for your build system, include-path and others,
>> or just ignore it all.
>
> How does one customize an instance of the generic project?
If you open a project that keeps a save file, which would be either the 
Makefile or Automakefile project, or any of the 'generic' project types, 
then you can do:

M-x customize-project


If you want to try it out, you would need:

(global-ede-mode 1)
(ede-enable-generic-projects)

then visit something in a version-control system (not emacs, that has 
it's own project type)

Eric

Re: IDE

[Dmitry Gutov]

On 10/16/2015 05:40 AM, Eric Ludlam wrote:

> If you open a project that keeps a save file, which would be either the
> Makefile or Automakefile project, or any of the 'generic' project types,
> then you can do:
>
> M-x customize-project (...)

That's very nice, thank you. We might reuse that code for in project.el 
sometime.

Re: IDE

[Stephen Leake]

David Kastrup <dak@gnu.org> writes:

> Eric Ludlam <eric@siege-engine.com> writes:
>
>> On 10/13/2015 12:28 PM, David Kastrup wrote:
>>> "John Wiegley" <johnw@newartisans.com> writes:
>>>
>>>>>>>>> Lluís  <xscript@gmx.net> writes:
>>>>
>>>>> Eli Zaretskii writes:
>>>>> [...]
>>>>>>> For C/C++, the community has Irony and Rtags, both based on libclang. If
>>>>>>> libclang is unacceptable for you, you probably know a more appropriate
>>>>>>> mailing list to bring that up at.
>>>>
>>>>>> Let's not reiterate past discussions: you forget CEDET.
>>>>
>>>> CEDET first came out in 2003. If it were the answer to our present
>>>> questions, we would not be asking them.
>>>
>>> But since it did come out in 2003, we really should be asking _why_ it
>>> isn't the answer to our present questions, in order to avoid the effort
>>> of creating CEDET2 and CEDET3.
>>
>> Based on the many emails I've seen on the topic, I suspect the answer is:
>>
>> * It is hard to configure (ie - setting up project files,
>>   include paths, or whatever.)
>> * Specific implementations are incomplete (ie - c++ || other parser is
>>   imperfect, the project system doesn't implement some feature, etc)
>> * It is compared against better staffed tools
>
> I got rid of it because it tended to eat all my CPU repeatedly digging
> through buffers and files in the background.  I don't want some tool to
> go treasure-hunting for hours in my directories without concrete cause,
> then restart for inscrutable reasons.
>
> It had its own idea of projects not matching the projects I was working
> with, and it's an absolute no-go for Emacs to meddle with project
> organization: I want to be able to jump in with Emacs into any project
> without any pre- or post-configuration.
>
> Maybe that's a decisive difference between what people got to expect
> from an IDE and I expect from Emacs: if someone develops stuff in Visual
> C++, everybody in the project is expected to use the project
> organization tools of the Visual C++ IDE.  But I don't want my choice of
> Emacs as an editor bleed all over a project.

That means CEDET needs to recognize your Visual C++ project, just like
the Visual C++ IDE does. CEDET does not currently support this.

> Now you'll say that EDE (or Semantic, or whatever other component) is
> entirely optional but it's hard to figure out just what the relations of
> the various parts of CEDET are.  If you want to just work with the code
> you have and not get stuff messed up, at some point of time it's easier
> to just forego the whole inscrutable package and simplify one's life.

You seem to be implying that something in CEDET was changing things on
the disk without your permission; is that what you are actually saying?

> Again, that's a main difference to what a normal IDE is doing: it tends
> to focus on a small set of languages and does them well when I buy into
> the IDE, and I can use IDE features as needed.

It's more than just the language; it's also the build tools and cross
reference tools, and the associated configuration files.

-- 
-- Stephe

Re: IDE

[David Kastrup]

Stephen Leake <stephen_leake@stephe-leake.org> writes:

> David Kastrup <dak@gnu.org> writes:
>>
>> I got rid of it because it tended to eat all my CPU repeatedly digging
>> through buffers and files in the background.  I don't want some tool to
>> go treasure-hunting for hours in my directories without concrete cause,
>> then restart for inscrutable reasons.
>>
>> It had its own idea of projects not matching the projects I was working
>> with, and it's an absolute no-go for Emacs to meddle with project
>> organization: I want to be able to jump in with Emacs into any project
>> without any pre- or post-configuration.
>>
>> Maybe that's a decisive difference between what people got to expect
>> from an IDE and I expect from Emacs: if someone develops stuff in Visual
>> C++, everybody in the project is expected to use the project
>> organization tools of the Visual C++ IDE.  But I don't want my choice of
>> Emacs as an editor bleed all over a project.
>
> That means CEDET needs to recognize your Visual C++ project, just like
> the Visual C++ IDE does. CEDET does not currently support this.

Uh, no?  I don't think I ever used Visual C++.  Projects I work on use
Makefiles if anything.  Or some other build infrastructure.

So if CEDET needs project information, its first idea of getting it
should be to look at some toplevel Makefiles and, if it finds them, ask
GNU Make for dependencies and stuff and probably look at a few standard
Make targets.  That's what to expect in GNU projects, the most important
clientele.

But I don't think it should ever get foraging for stuff on its own.

>> Now you'll say that EDE (or Semantic, or whatever other component) is
>> entirely optional but it's hard to figure out just what the relations
>> of the various parts of CEDET are.  If you want to just work with the
>> code you have and not get stuff messed up, at some point of time it's
>> easier to just forego the whole inscrutable package and simplify
>> one's life.
>
> You seem to be implying that something in CEDET was changing things on
> the disk without your permission; is that what you are actually
> saying?

No.  I was saying

>> I got rid of it because it tended to eat all my CPU repeatedly
>> digging through buffers and files in the background.  I don't want
>> some tool to go treasure-hunting for hours in my directories without
>> concrete cause, then restart for inscrutable reasons.

Is there any reason to assume I mean something different when I write
stuff like that?

>> Again, that's a main difference to what a normal IDE is doing: it
>> tends to focus on a small set of languages and does them well when I
>> buy into the IDE, and I can use IDE features as needed.
>
> It's more than just the language; it's also the build tools and cross
> reference tools, and the associated configuration files.

Whatever.  I wrote why I got rid of CEDET, you answer that the world is
difficult for an IDE.  That's nice but irrelevant.  I'm fine with only
requiring those services of an IDE which it can provide without being
painful to me.  If I don't get any obvious way to choose, if it's "take
it all or leave it", then the probability is that I'll settle on the
"leave it" option.

-- 
David Kastrup

Re: IDE

[Dmitry Gutov]

On 10/14/2015 06:16 AM, Eric Ludlam wrote:

> My main concern is about folks claiming CEDET is complicated (which it
> is) then oversimplifying the problem space to kick off some new thing
> which will likely end up just as complicated.

My already-stated impression is that it's over-specialized and tightly 
coupled.

Not saying that the problem domain is easy, but being able to use 
different pieces of the solution separately would go a long way towards 
alleviating the complaint that certain other parts are incomplete.

Especially if it were easier to swap in different solutions for some of 
those parts (and do entirely without some others), and do that in not 
too many lines, all as part of the user's configuration.

Re: IDE

[John Wiegley]

>>>>> Dmitry Gutov <dgutov@yandex.ru> writes:

> My already-stated impression is that it's over-specialized and tightly
> coupled.
>
> Not saying that the problem domain is easy, but being able to use different
> pieces of the solution separately would go a long way towards alleviating
> the complaint that certain other parts are incomplete.
>
> Especially if it were easier to swap in different solutions for some of
> those parts (and do entirely without some others), and do that in not too
> many lines, all as part of the user's configuration.

You've taken the reply right out of my mouth, Dmitry. David's response was
also very much in line with my thinking. As I said before, if CEDET were the
answer to our questions, we wouldn't still be asking them.

If, later in our design discussions, CEDET has experiences, code, and
perspectives to offer, this could be of tremendous value. I hope Eric will be
avail of us his time then, and share those nuggets with us.

John

Re: IDE

[Eli Zaretskii]

> From: "John Wiegley" <johnw@newartisans.com>
> Date: Fri, 16 Oct 2015 15:58:57 -0700
> 
> >>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
> > My already-stated impression is that it's over-specialized and tightly
> > coupled.
> >
> > Not saying that the problem domain is easy, but being able to use different
> > pieces of the solution separately would go a long way towards alleviating
> > the complaint that certain other parts are incomplete.
> >
> > Especially if it were easier to swap in different solutions for some of
> > those parts (and do entirely without some others), and do that in not too
> > many lines, all as part of the user's configuration.
> 
> You've taken the reply right out of my mouth, Dmitry. David's response was
> also very much in line with my thinking. As I said before, if CEDET were the
> answer to our questions, we wouldn't still be asking them.

Could it be that we don't understand the answer?

I'd suggest to be very careful with such conclusions.  They can only
be valid when based on a detailed analysis of what is and isn't in
CEDET, and on good knowledge and understanding of its design and
implementation.  My impression so far is that neither is particularly
true, and my evidence is the number of times Eric and David Engster
described some CEDET features that came as a surprise to us.

I'm quite sure CEDET has collected and expressed in code a lot of
experience and solutions to many problems that arise in the context of
building an IDE.  It's OK to discard that, if we sure that's the
proverbial 1st variant everyone throws away, but we need first to be
sure we know what we are discarding.

Re: IDE

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: "John Wiegley" <johnw@newartisans.com>
>> Date: Fri, 16 Oct 2015 15:58:57 -0700
>> 
>> >>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
>> 
>> > My already-stated impression is that it's over-specialized and tightly
>> > coupled.
>> >
>> > Not saying that the problem domain is easy, but being able to use different
>> > pieces of the solution separately would go a long way towards alleviating
>> > the complaint that certain other parts are incomplete.
>> >
>> > Especially if it were easier to swap in different solutions for some of
>> > those parts (and do entirely without some others), and do that in not too
>> > many lines, all as part of the user's configuration.
>> 
>> You've taken the reply right out of my mouth, Dmitry. David's response was
>> also very much in line with my thinking. As I said before, if CEDET were the
>> answer to our questions, we wouldn't still be asking them.
>
> Could it be that we don't understand the answer?
>
> I'd suggest to be very careful with such conclusions.  They can only
> be valid when based on a detailed analysis of what is and isn't in
> CEDET, and on good knowledge and understanding of its design and
> implementation.

If we wanted that, we would be using vi.  The amount of functionality
available with Emacs is so diversified that it needs to fall apart into
pieces that can be comprehended on their own with reasonable effort.
Actually, I am being cheeky here: a morning with vi documentation will
get you set.  A week with CEDET documentation won't.

An IDE is principally a tool that is supposed to make things easier for
the programmer and let him figure out fewer things on his own rather
than more.  CEDET didn't do that for me.

I'm old and experienced enough that I have the arrogance to claim that
something that is too hard for me to understand and/or use effectively
after putting in a reasonable amount of effort is not a generally useful
tool.

> My impression so far is that neither is particularly true, and my
> evidence is the number of times Eric and David Engster described some
> CEDET features that came as a surprise to us.

Is that supposed to be a good thing?

> I'm quite sure CEDET has collected and expressed in code a lot of
> experience and solutions to many problems that arise in the context of
> building an IDE.  It's OK to discard that, if we sure that's the
> proverbial 1st variant everyone throws away, but we need first to be
> sure we know what we are discarding.

I am not qualified to have much of an opinion about the substance, the
raw functionality implemented in CEDET.  That's probably 90% of the
work, and redoing that will not likely make a decisive difference with
regard to the remaining 10% that are for putting that functionality at
the fingertips of the programmer.

My refactoring work tends to be done using stuff like

git grep '^MAKE_[_A-Z]*SCHEME_CALLBACK[_A-Z]* ' lily|sed -n 's/^\([^:]*\):MAKE_[_A-Z]*SCHEME_CALLBACK[_A-Z]* (\([^,]*\), \([^,]*\), \([^)]*\)).*$/\1 \2 \3 \4/p' |
    while read file class name args
    do
	argname=$(sed -n "s/^$class::$name (SCM \\?\\([^,)]*\\)[,)].*\$/\\1/p" $file)
	sed -i "/^\\(class\\|struct\\) $class\\( {\\)\\?\$/,/^}/s/^\\( *DECLARE_\\)SCHEME\\(_CALLBACK ($name, (\\)SCM[^,)]*\\(, \\)\\?/\\1MEMBER\\2/" $(git grep -l "^\\(class\\|struct\\) $class\\( \\|\$\\)") &&
	    case "$argname" in
		[a-zA-Z]*)
		    thisexpr=$(echo "unsmob<[_a-zA-Z]*> *($argname)"
			       sed -n "/^$class::$name (/,/^}/s/^ *\\([_a-zA-Z]\\+\\) *\\* *\\([_a-zA-Z]\\+\\) *= *unsmob *<\\1> ($argname);\$/\\2/p" $file |
				   while read alias
				   do
				       sed -i "/^$class::$name (/,/^}/{/^ *\\([_a-zA-Z]\\+\\) *\\* *$alias *= *unsmob *<\\1> ($argname);\$/d;}" $file
				       echo $alias
				   done)
		    thisexpr=$(echo "$thisexpr"|sed -n '1h;1!H;${x;s/\n/\\|/g;p}')
		    sed -i "/^$class::$name (/,/^}/{s/\\b\\($thisexpr\\)\\b/this/g;s/\\bthis *-> *//g;s/\\*this\\.//g;}" $file
		    sed -i "s/^\\(MAKE_[_A-Z]*\\)SCHEME\\(_CALLBACK[_A-Z]* ($class, $name,\\)/\\1MEMBER\\2/
s/^\\($class::$name (\\)SCM \\?[^,)]*\\(, \\)\\?\\(.*\\)\$/\\1\\3/" $file
		    ;;
		*)
		    sed -i "s/^\\(MAKE_[_A-Z]*\\)SCHEME\\(_CALLBACK[_A-Z]* ($class, $name,\\)/\\1MEMBER\\2/
s/^\\($class::$name (\\)SCM \\?[^,)]*\\(, \\)\\?\\(.*\\)\$/\\1\\3/" $file
	esac
#	sed "/^MAKE_SCHEME_CALLBACK ($class, $name, $args)/d;

#	echo $file $class $name $args $argname $argtype $param
    done

Yes, this is for real, bulk rewriting certain types of static member
functions into proper member functions, changing certain constructs
systematically while doing that.  Doing that kind of stuff requires
knowing about 5 pages of sed programming and shell programming that
mostly worked already 20 years ago (quoting constructs have become more
robust).

I don't relish doing stuff in that manner.  It's not the kind of job an
automated tool could do easily, but an automated tool could at least
find the code that needs changing and possibly even offer some way to do
replacements that are a bit more syntactically conscious than sed.

I've had bulk changes touching several thousands of lines and I prefer
investing two days into creating a one-shot automated tool than 6 hours
doing everything manually.

I'm pretty sure that somewhere between those two-days sed scripting and
6 hours of completely manual work there must be a sweet spot where an
IDE/refactoring/thing-aware tool tied into Emacs should save both time
and nerves.

M-x grep RET works a lot smoother than M-! grep and the kind of
difference between the two are where Emacs shines, providing the editor
connection to technology.

-- 
David Kastrup

Re: IDE

[Przemysław Wojnowski]

W dniu 17.10.2015 o 10:39, David Kastrup pisze:
[...]
> My refactoring work tends to be done using stuff like
 >[...]
With all due respect, please spend a morning with a modern IDE (like
Intellij, Eclipse) to learn how they make programmers productive.
Especially look at their refactoring features. It's a different league.

Re: IDE

[David Kastrup]

Przemysław Wojnowski <esperanto@cumego.com> writes:

> W dniu 17.10.2015 o 10:39, David Kastrup pisze:
> [...]
>> My refactoring work tends to be done using stuff like
>>[...]
> With all due respect, please spend a morning with a modern IDE (like
> Intellij, Eclipse) to learn how they make programmers productive.

Not really interested.  This refactoring was necessitated by stuff being
implemented with a contorted C macro system.  Any refactoring system
able to do this out of the box would be too complicated to learn.

> Especially look at their refactoring features. It's a different
> league.

To sed?  Uh, that's not a particularly interesting contest since sed has
no refactoring features at all.  Multi-line replacements are noisome
enough.  At any rate, I am not interested in using other tools here.
Scripting means I can work incrementally and put the respective
responsible script into the commit message.

Which means that redoing that particular refactoring when branches are
rebased or my commit takes a week for reviewing while other changes pour
in is then trivial.

-- 
David Kastrup

Re: IDE

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Cc: John Wiegley <johnw@newartisans.com>,  emacs-devel@gnu.org
> Date: Sat, 17 Oct 2015 10:39:51 +0200
> 
> > Could it be that we don't understand the answer?
> >
> > I'd suggest to be very careful with such conclusions.  They can only
> > be valid when based on a detailed analysis of what is and isn't in
> > CEDET, and on good knowledge and understanding of its design and
> > implementation.
> 
> If we wanted that, we would be using vi.

Since vi isn't part of Emacs, this is irrelevant.

> I'm old and experienced enough that I have the arrogance to claim that
> something that is too hard for me to understand and/or use effectively
> after putting in a reasonable amount of effort is not a generally useful
> tool.

I'm older, but until now there wasn't a single thing I wanted to
understand and/or use effectively that I couldn't.  I guess our idea
of "reasonable amount" differs.

> > My impression so far is that neither is particularly true, and my
> > evidence is the number of times Eric and David Engster described some
> > CEDET features that came as a surprise to us.
> 
> Is that supposed to be a good thing?

Not if we pretend to claim we know what CEDET is.

> M-x grep RET works a lot smoother than M-! grep and the kind of
> difference between the two are where Emacs shines, providing the editor
> connection to technology.

That's fine.  IDE isn't for everyone.

Re: IDE

[Steinar Bang]

>>>>> David Kastrup <dak@gnu.org>:

> M-x grep RET works a lot smoother than M-! grep and the kind of
> difference between the two are where Emacs shines, providing the editor
> connection to technology.

M-x rgrep is even nicer, particularily in maven Java projects (with
deeply nested directory hierarchies).

My own mass edits tends to be done in a combination of emacs keyboard
macros run over rgrep hits.

And the mass edits tend to be done in emacs, rather than the IDE I'm
officially using at that time...

(Side note: If I had a task that would require 6 hour manual editing I
would probably script it in perl (I am trying to replace my use of perl
with python, but I keep having relapses...))

Re: IDE

[David Engster]

Eli Zaretskii writes:
>> From: "John Wiegley" <johnw@newartisans.com>
>> Date: Fri, 16 Oct 2015 15:58:57 -0700
>> 
>
>> >>>>> Dmitry Gutov <dgutov@yandex.ru> writes:
>> 
>> > My already-stated impression is that it's over-specialized and tightly
>> > coupled.
>> >
>> > Not saying that the problem domain is easy, but being able to use different
>> > pieces of the solution separately would go a long way towards alleviating
>> > the complaint that certain other parts are incomplete.
>> >
>> > Especially if it were easier to swap in different solutions for some of
>> > those parts (and do entirely without some others), and do that in not too
>> > many lines, all as part of the user's configuration.
>> 
>> You've taken the reply right out of my mouth, Dmitry. David's response was
>> also very much in line with my thinking. As I said before, if CEDET were the
>> answer to our questions, we wouldn't still be asking them.
>
> Could it be that we don't understand the answer?
>
> I'd suggest to be very careful with such conclusions.  They can only
> be valid when based on a detailed analysis of what is and isn't in
> CEDET, and on good knowledge and understanding of its design and
> implementation.  My impression so far is that neither is particularly
> true, and my evidence is the number of times Eric and David Engster
> described some CEDET features that came as a surprise to us.
>
> I'm quite sure CEDET has collected and expressed in code a lot of
> experience and solutions to many problems that arise in the context of
> building an IDE.  It's OK to discard that, if we sure that's the
> proverbial 1st variant everyone throws away, but we need first to be
> sure we know what we are discarding.

Actually, Eric rewrote Semantic once already...

From the discussion so far, I think the main issue at least w.r.t. to
Semantic is: do you actually want Semantic's tag-based system, or more
general: do you want quick access to AST information in your buffer?

If I understand Dmitry correctly, he is not really interested in that,
as for dynamic languages, the AST information is usually missing
important information (unless you bother to implement a complete
frontend). He'd rather call external binaries for complicated stuff like
completion, and use simpler tools (like pure regexp-based parsing) for
stuff like font-locking, navigation, folding, etc. Of course, you can
hook external binaries into Semantic pretty easily, but I can understand
Dmitry that if he does not need the rest of Semantic, why should he
bother?

Now, I think having AST information in your buffer is great, and I don't
like depending on external binaries if I don't have to, because I want
as much as possible in Emacs Lisp. For me, that's what Emacs is about
and why I still use it in the first place.

-David

Re: IDE

[Dmitry Gutov]

On 10/17/2015 03:00 PM, David Engster wrote:
> Eli Zaretskii writes:
>> I'm quite sure CEDET has collected and expressed in code a lot of
>> experience and solutions to many problems that arise in the context of
>> building an IDE.  It's OK to discard that, if we sure that's the
>> proverbial 1st variant everyone throws away, but we need first to be
>> sure we know what we are discarding.

I wouldn't want to discard Semantic, as long as it works better than the 
new solution, in the domain that it's actually targeting. And we should 
learn from its implementation either way.

>  From the discussion so far, I think the main issue at least w.r.t. to
> Semantic is: do you actually want Semantic's tag-based system, or more
> general: do you want quick access to AST information in your buffer?

Not really the main issue. Whether the AST is saved in overlays (and 
thus not refreshed as long as buffer is not modified), for me is an open 
question.

But not many of the existing code assistance tools that are currently 
available, provide that info, at least for dynamic languages. So it 
would be more productive to focus on the things they do provide, and 
grow the internal IDE APIs from that.

Whenever we have a tool that does provide AST dumps at will, or Semantic 
supports the language in question well, it would of course be great to 
use it.

> If I understand Dmitry correctly, he is not really interested in that,
> as for dynamic languages, the AST information is usually missing
> important information (unless you bother to implement a complete
> frontend).

Not sure what you mean exactly by "complete frontend", but it'll often 
be missing anyway. If the current method takes 'foo' as an argument, and 
the method is public, and the method contains 'foo.bar()', often the 
best thing we can tell about 'foo' is that its class contains a method 
called 'bar'.

So at least the Semantic database would have to be extended to work 
meaningfully with tags like that.

> He'd rather call external binaries for complicated stuff like
> completion, and use simpler tools (like pure regexp-based parsing) for
> stuff like font-locking, navigation, folding, etc.

I mentioned SMIE as well. It's a bit more advanced than that.

> Of course, you can
> hook external binaries into Semantic pretty easily, but I can understand
> Dmitry that if he does not need the rest of Semantic, why should he
> bother?

Being able to mix and match would be best, of course.

> Now, I think having AST information in your buffer is great, and I don't
> like depending on external binaries if I don't have to, because I want
> as much as possible in Emacs Lisp. For me, that's what Emacs is about
> and why I still use it in the first place.

My biggest issue with Semantic is that it tries to implement type 
analysis in Elisp, and still hasn't gotten it right even for simple C++ 
code (no classes or templates). Allow me to repeat myself with this: 
https://gist.github.com/dgutov/19c45ef43d1c90b96483

Suggested completions should depend on the argument tee receives ("a" 
and "b" for i, "c" and "d" for l), but they don't.

And I think re-implementing type analysis for each language in Elisp is 
untenable. Correct me if I'm wrong, but you cannot generalize that 
logic: different languages have different type systems, and for each one 
you'd have to write the analysis from scratch.

Type analysis is often at least as complex as parsing (if not more). I'm 
not in any hurry to rewrite e.g. Tern in Elisp, and then keep it up-to-date.

And speaking of more personal reasons, I already have written a code 
assistance tool for Ruby, in Ruby (which is not a Lisp, but still a 
pretty okay language), and smooth integration with whatever APIs we 
choose would be nice.

Re: IDE

[David Engster]

Dmitry Gutov writes:
> On 10/17/2015 03:00 PM, David Engster wrote:
>> Eli Zaretskii writes:
>>> I'm quite sure CEDET has collected and expressed in code a lot of
>>> experience and solutions to many problems that arise in the context of
>>> building an IDE.  It's OK to discard that, if we sure that's the
>>> proverbial 1st variant everyone throws away, but we need first to be
>>> sure we know what we are discarding.
>
> I wouldn't want to discard Semantic, as long as it works better than
> the new solution, in the domain that it's actually targeting. And we
> should learn from its implementation either way.
>
>>  From the discussion so far, I think the main issue at least w.r.t. to
>> Semantic is: do you actually want Semantic's tag-based system, or more
>> general: do you want quick access to AST information in your buffer?
>
> Not really the main issue. Whether the AST is saved in overlays (and
> thus not refreshed as long as buffer is not modified), for me is an
> open question.

Well, all I can say is it works well.

>> If I understand Dmitry correctly, he is not really interested in that,
>> as for dynamic languages, the AST information is usually missing
>> important information (unless you bother to implement a complete
>> frontend).
>
> Not sure what you mean exactly by "complete frontend", but it'll often
> be missing anyway. If the current method takes 'foo' as an argument,
> and the method is public, and the method contains 'foo.bar()', often
> the best thing we can tell about 'foo' is that its class contains a
> method called 'bar'.
>
> So at least the Semantic database would have to be extended to work
> meaningfully with tags like that.

The database wouldn't be much of a problem, I think. It's just
incredibly hard, if not impossible, to parse such a dynamic language
statically. For Javascript, I wrote a small semanticdb backend to query
a running Firefox process through mozrepl. That worked pretty well for
getting completions on stuff like jQuery.

>> Of course, you can
>> hook external binaries into Semantic pretty easily, but I can understand
>> Dmitry that if he does not need the rest of Semantic, why should he
>> bother?
>
> Being able to mix and match would be best, of course.

I fully agree. The above mentioned mozrepl-binding worked together with
our small Javascript parser.

>> Now, I think having AST information in your buffer is great, and I don't
>> like depending on external binaries if I don't have to, because I want
>> as much as possible in Emacs Lisp. For me, that's what Emacs is about
>> and why I still use it in the first place.
>
> My biggest issue with Semantic is that it tries to implement type
> analysis in Elisp, and still hasn't gotten it right even for simple
> C++ code (no classes or templates). Allow me to repeat myself with
> this: https://gist.github.com/dgutov/19c45ef43d1c90b96483

It's a matter of someone implementing overloads. And such a
implementation could be used from any other languages that has
overloading on function arguments, which is pretty common.

> And I think re-implementing type analysis for each language in Elisp
> is untenable. Correct me if I'm wrong, but you cannot generalize that
> logic: different languages have different type systems, and for each
> one you'd have to write the analysis from scratch.

There are a lot of similarities in C-like languages. Also, any
OOP-language will have something like classes, parents, methods,
attributes. But yes, type inference and dynamic languages make things
more complicated. Querying an external REPL or some tool that analyzes
the code would often be necessary.

> Type analysis is often at least as complex as parsing (if not
> more).

For C++11, it has become more complex, which is why I will indeed ask an
external tool for type information. Since such a tool will have to build
the AST anyway, it will provide that as well.

> And speaking of more personal reasons, I already have written a code
> assistance tool for Ruby, in Ruby (which is not a Lisp, but still a
> pretty okay language), and smooth integration with whatever APIs we
> choose would be nice.

Yes, of course.

To be honest, I mostly lost track about what is actually discussed
here. I just took offense at John's "if CEDET was the answer we wouldn't
still be asking questions" and avoiding a "CEDET2", like the CEDET we
have in core is a failure.

CEDET tries to walk a narrow path, trying to provide IDE-like features
without Emacs actually becoming a "typical IDE". The IDEs out there have
it easier, as they usually force you into organizing your projects in a
certain way, and they usually target only one language (or language
family).

-David

Re: IDE

[Dmitry Gutov]

On 10/17/2015 06:26 PM, David Engster wrote:

> The database wouldn't be much of a problem, I think. It's just
> incredibly hard, if not impossible, to parse such a dynamic language
> statically.

Maybe not the database, but the "tag" value is used in many places in 
Semantic. I'd want to be able to use them for, say, JavaScript as well. 
For instance, in semantic-symref, not just for completions.

> For Javascript, I wrote a small semanticdb backend to query
> a running Firefox process through mozrepl. That worked pretty well for
> getting completions on stuff like jQuery.

I'll have to check it out later.

> It's a matter of someone implementing overloads. And such a
> implementation could be used from any other languages that has
> overloading on function arguments, which is pretty common.

I'm sure it can be implemented in Elisp, just surprised it hasn't been 
taken care of since a similar example came came up 6 (or 12?) months 
ago, in a Clang-related discussion.

> There are a lot of similarities in C-like languages. Also, any
> OOP-language will have something like classes, parents, methods,
> attributes. But yes, type inference and dynamic languages make things
> more complicated. Querying an external REPL or some tool that analyzes
> the code would often be necessary.

C-like languages - probably, but every one also has tiny peculiarities 
which have to be handled in a different way.

But Emacs users are a diverse bunch, and for many the draw is in being 
able to use many different languages, including very young ones. So a 
general IDE solution should anticipate having to handle different type 
systems.

>> Type analysis is often at least as complex as parsing (if not
>> more).
>
> For C++11, it has become more complex, which is why I will indeed ask an
> external tool for type information. Since such a tool will have to build
> the AST anyway, it will provide that as well.

I'm glad we agree on this. But Java has also become more complex (since 
1.5, maybe?), and Java 8 added lambdas, which allow omitting argument 
types. Not sure about things about Objective-C land.

But otherwise, if it'll be common to use an external tool for type 
resolution, and even parsing the buffer contents, maybe at some point 
you'll deprecate Wisent grammars. Or use, say, a very simplified format 
for them, only what's strictly necessary to find the names and 
boundaries of tags.

> To be honest, I mostly lost track about what is actually discussed
> here. I just took offense at John's "if CEDET was the answer we wouldn't
> still be asking questions" and avoiding a "CEDET2", like the CEDET we
> have in core is a failure.

Personally, I'm happy discussing and figuring out the current state of 
affairs, so that our common understanding goes further than "CEDET 
didn't live up to its promises" or "don't throw out CEDET, we've got 
nothing better".

The EDE subthread also brought up some ideas for project.el.

> CEDET tries to walk a narrow path, trying to provide IDE-like features
> without Emacs actually becoming a "typical IDE". The IDEs out there have
> it easier, as they usually force you into organizing your projects in a
> certain way, and they usually target only one language (or language
> family).

That's not really true. IntelliJ IDEA supports a big swath of languages, 
and my colleagues use it successfully for our "non-standard" Ruby 
projects (no Rails), and also with JS and different markup languages. 
But of course you have different challenges with C++, and the JetBrains 
team has more manpower anyway.

But I'd be ecstatic to even have a consistent UI for features that VS 
Code (smaller cousin of Visual Studio) touts here: 
https://code.visualstudio.com/docs/editor/editingevolved

Re: IDE

[Przemysław Wojnowski]

W dniu 17.10.2015 o 22:19, Dmitry Gutov pisze:
[...]
>> There are a lot of similarities in C-like languages. Also, any
>> OOP-language will have something like classes, parents, methods,
>> attributes. But yes, type inference and dynamic languages make things
>> more complicated. Querying an external REPL or some tool that analyzes
>> the code would often be necessary.
>
> C-like languages - probably, but every one also has tiny peculiarities which
> have to be handled in a different way.
>
> But Emacs users are a diverse bunch, and for many the draw is in being able to
> use many different languages, including very young ones. So a general IDE
> solution should anticipate having to handle different type systems.
I think it is not a problem. An IDE could switch (or enable) language backend
depending on current language. For C-like (or maybe statically typed in
general) languages (covering most of programming world) could use Semantic, for
other languages maybe something other (like tern for JS).

[...]
> The EDE subthread also brought up some ideas for project.el.
Does that mean that you don't want to reuse EDE, but reimplement everything
from scratch? Don't you think it would be better to reuse what already is and
just change parts of it to be more flexible?

>> CEDET tries to walk a narrow path, trying to provide IDE-like features
>> without Emacs actually becoming a "typical IDE". The IDEs out there have
>> it easier, as they usually force you into organizing your projects in a
>> certain way, and they usually target only one language (or language
>> family).
>
> That's not really true. IntelliJ IDEA supports a big swath of languages, and my
> colleagues use it successfully for our "non-standard" Ruby projects (no Rails),
> and also with JS and different markup languages. But of course you have
> different challenges with C++, and the JetBrains team has more manpower anyway.
Intellij's support for JavaScript is not so great. Compared to Java it is poor.

Re: IDE

[Dmitry Gutov]

On 10/18/2015 01:03 AM, Przemysław Wojnowski wrote:

> I think it is not a problem. An IDE could switch (or enable) language
> backend
> depending on current language. For C-like (or maybe statically typed in
> general) languages (covering most of programming world) could use
> Semantic, for
> other languages maybe something other (like tern for JS).

It would be better if we could support a set of common operations for 
both static and dynamic languages. If those were implemented by Semantic 
by static languages, more power to it.

>> The EDE subthread also brought up some ideas for project.el.
> Does that mean that you don't want to reuse EDE, but reimplement everything
> from scratch? Don't you think it would be better to reuse what already
> is and
> just change parts of it to be more flexible?

Have you looked at lisp/progmodes/project.el yet?

> Intellij's support for JavaScript is not so great. Compared to Java it
> is poor.

Compared to most other JS editors, it's pretty great. Or so I'm told.

Re: IDE

[Przemysław Wojnowski]

W dniu 18.10.2015 o 00:07, Dmitry Gutov pisze:
> On 10/18/2015 01:03 AM, Przemysław Wojnowski wrote:
>
>> I think it is not a problem. An IDE could switch (or enable) language
>> backend
>> depending on current language. For C-like (or maybe statically typed in
>> general) languages (covering most of programming world) could use
>> Semantic, for
>> other languages maybe something other (like tern for JS).
>
> It would be better if we could support a set of common operations for both
> static and dynamic languages. If those were implemented by Semantic by static
> languages, more power to it.
I don't think it is possible, because languages are very different and their
surrounding tooling is very different.

>>> The EDE subthread also brought up some ideas for project.el.
>> Does that mean that you don't want to reuse EDE, but reimplement everything
>> from scratch? Don't you think it would be better to reuse what already
>> is and
>> just change parts of it to be more flexible?
>
> Have you looked at lisp/progmodes/project.el yet?
Yes. And I've looked at EDE at SF 
(http://sourceforge.net/p/cedet/git/ci/master/tree/lisp/cedet/ede/). There's 
support for many types of projects, build tools, etc. There are even some tests.

What's the point of reimplementing that from scratch?

Re: IDE

[Dmitry Gutov]

On 10/18/2015 01:28 AM, Przemysław Wojnowski wrote:

> I don't think it is possible, because languages are very different and
> their
> surrounding tooling is very different.

The meanings of "go to definition", "find references" and "complete text 
at point" are very much the same across languages. Some refactorings, too.

> Yes. And I've looked at EDE at SF
> (http://sourceforge.net/p/cedet/git/ci/master/tree/lisp/cedet/ede/).
> There's support for many types of projects, build tools, etc. There are
> even some tests.

A more idiomatic, Emacs-y API that isn't tied to CEDET, that's friendly 
to third-party packages, and doesn't ask its caller to know the 
intricacies of EDE's object hierarchies.

You're welcome to work on making EDE more flexible, though. We can 
compare progress later.

> What's the point of reimplementing that from scratch?

We can reuse code from EDE, if it fits.

Re: IDE

[Przemysław Wojnowski]

W dniu 18.10.2015 o 00:37, Dmitry Gutov pisze:
> On 10/18/2015 01:28 AM, Przemysław Wojnowski wrote:
>
>> I don't think it is possible, because languages are very different and
>> their
>> surrounding tooling is very different.
>
> The meanings of "go to definition", "find references" and "complete text at
> point" are very much the same across languages.
Yes, and EDE could be changed to provide such API.

> Some refactorings, too.
And some are not and will never be, because languages are too different. So
IDE's support for them will be different or crippled to the lowest denominator
as you seem to suggest.

>> Yes. And I've looked at EDE at SF
>> (http://sourceforge.net/p/cedet/git/ci/master/tree/lisp/cedet/ede/).
>> There's support for many types of projects, build tools, etc. There are
>> even some tests.
>
> A more idiomatic, Emacs-y
What does that mean? Is it described somewhere what is idiomatic in Emacs Lisp?

> API that isn't tied to CEDET, that's friendly to
> third-party packages, and doesn't ask its caller to know the intricacies of
> EDE's object hierarchies.
Yes. This part would need to be changed in EDE.

> You're welcome to work on making EDE more flexible, though. We can compare
> progress later.
I don't waste resources on duplicated efforts just to show my ego. Have many 
other things to work on.

>> What's the point of reimplementing that from scratch?
>
> We can reuse code from EDE, if it fits.
Depending on what you means by "it fits", but if that's possible, it's ok for
me.

My only concern is that EDE is something that already works, but needs 
improvement. Creating, more or less, the same from scratch is IMHO reinventing 
the wheel, which may not even end being as round as EDE is.

Also reusing EDE maybe would "reuse" developers that were/are working on it and
hence Emacs would have another contributors for free. Not doing so may mean
that some devs will work on EDE and some on project.el, duplicating many things
and loosing steam by lone work. Future? Two incomplete packages that do mostly
the same, but unusable for users - the most common scenario in open source
world (see JDEE, malabar-mode, eclim).

Re: IDE

[Dmitry Gutov]

On 10/18/2015 12:08 PM, Przemysław Wojnowski wrote:

>> The meanings of "go to definition", "find references" and "complete
>> text at
>> point" are very much the same across languages.
> Yes, and EDE could be changed to provide such API.

You probably mean Semantic.

>> Some refactorings, too.
> And some are not and will never be, because languages are too different. So
> IDE's support for them will be different or crippled to the lowest
> denominator as you seem to suggest.

You're just arguing for the sake of arguing here. We could have a common 
UI for implementing refactoring commands, and some of those commands 
could be different for different languages. Or something along these lines.

>> A more idiomatic, Emacs-y
> What does that mean? Is it described somewhere what is idiomatic in
> Emacs Lisp?

In the manual, maybe. I wouldn't know.

Point is, we can have a better, friendlier and more general API. And 
when we know that it makes sense, we can also change the existing code 
to fit it.

> I don't waste resources on duplicated efforts just to show my ego. Have
> many other things to work on.

Working on EDE can be more than an ego trip: maybe you'll stumble on a 
good, practical flexible design (which isn't out of the question, given 
a lot of code you'd have to keep working). And if EDE is more flexible, 
most likely we could reuse pieces of it more easily as well.

Did you actually intend to do some work on project support, or was that 
just backseat driving?

>> We can reuse code from EDE, if it fits.
> Depending on what you means by "it fits", but if that's possible, it's
> ok for me.

Build tool support code should fit the bill when project.el supports 
build tools (or maybe we'll separate that into build-tool.el).

> My only concern is that EDE is something that already works, but needs
> improvement. Creating, more or less, the same from scratch is IMHO
> reinventing the wheel, which may not even end being as round as EDE is.

Projectile already works, too. And eproject. And ENSIME, and Eclim. The 
two last examples also use their own code to manage projects, and I 
suspect it would be easier to ask them to implement a bridge to 
project.el, rather than migrate to EDE.

> Also reusing EDE maybe would "reuse" developers that were/are working on
> it and

Nope. David and Eric are pretty busy, and won't magically become active 
Emacs contributors just because of that.

> hence Emacs would have another contributors for free. Not doing so may mean
> that some devs will work on EDE and some on project.el, duplicating many
> things
> and loosing steam by lone work.

project.el is foremost an API, so it needs better design proposals, 
rather than more teamwork, right now. Improving EDE design would also 
help there.

> the same, but unusable for users - the most common scenario in open source
> world (see JDEE, malabar-mode, eclim).

I wish wish we could say that each of those projects has got *something* 
working really well, so it would make sense to talk about combining them.

Re: IDE

[Eric Ludlam]

On 10/18/2015 05:42 AM, Dmitry Gutov wrote:
>>> Some refactorings, too.
>> And some are not and will never be, because languages are too
>> different. So
>> IDE's support for them will be different or crippled to the lowest
>> denominator as you seem to suggest.
>
> You're just arguing for the sake of arguing here. We could have a common
> UI for implementing refactoring commands, and some of those commands
> could be different for different languages. Or something along these lines.

Here is an example of a refactoring toolset that Tu Do started using 
CEDET/Semantic as a starting point.

https://github.com/tuhdo/semantic-refactor

Eric

Re: IDE

[Dmitry Gutov]

On 10/20/2015 04:07 AM, Eric Ludlam wrote:

> Here is an example of a refactoring toolset that Tu Do started using
> CEDET/Semantic as a starting point.
>
> https://github.com/tuhdo/semantic-refactor

It's promising, but if we're talking about a UI, I would expect 
something like a preview of the operation. That would be most useful for 
things like rename.

That's not there yet, but it does support "rename local variable" and 
"extract method", so anyone sorely missing refactoring in Emacs (in C or 
C++), should check it out. AFAICS, it's definitely not getting enough 
feedback.

Re: IDE

[David Engster]

Dmitry Gutov writes:
> On 10/17/2015 06:26 PM, David Engster wrote:
>
>> The database wouldn't be much of a problem, I think. It's just
>> incredibly hard, if not impossible, to parse such a dynamic language
>> statically.
>
> Maybe not the database, but the "tag" value is used in many places in
> Semantic. I'd want to be able to use them for, say, JavaScript as
> well. For instance, in semantic-symref, not just for completions.

In dynamic languages, you would mostly have tags with no specific type.

>> For Javascript, I wrote a small semanticdb backend to query
>> a running Firefox process through mozrepl. That worked pretty well for
>> getting completions on stuff like jQuery.
>
> I'll have to check it out later.

It is only upstream, and I wouldn't be surprised if it is broken by now
due to changes in mozrepl (this is why I don't like depending on
external binaries). I don't code Javascript anymore.

>> It's a matter of someone implementing overloads. And such a
>> implementation could be used from any other languages that has
>> overloading on function arguments, which is pretty common.
>
> I'm sure it can be implemented in Elisp, just surprised it hasn't been
> taken care of since a similar example came came up 6 (or 12?) months
> ago, in a Clang-related discussion.

Well, it's not like a bunch of people are hacking on the C
parser. Several things happened:

- Emacs switched to git, so CEDET had to switch as well, which broke my
  merge setup. I still need to repair it, which is extremely boring
  work.

- CEDET does not work with Emacs25 due to large changes in
  EIEIO. Someone has to fix that as well. Again: extremely boring work.

- This gcc-AST discussion.

- New kid.

- In general, hacking Emacs isn't as much fun as it used to be.

>> There are a lot of similarities in C-like languages. Also, any
>> OOP-language will have something like classes, parents, methods,
>> attributes. But yes, type inference and dynamic languages make things
>> more complicated. Querying an external REPL or some tool that analyzes
>> the code would often be necessary.
>
> C-like languages - probably, but every one also has tiny peculiarities
> which have to be handled in a different way.

Which is why Semantic allows to tweak pretty much any part through
mode-local overrides.

> But Emacs users are a diverse bunch, and for many the draw is in being
> able to use many different languages, including very young ones. So a
> general IDE solution should anticipate having to handle different type
> systems.

Of course. That's the goal of CEDET. If it had just focused on C/C++, it
would be simpler.

>>> Type analysis is often at least as complex as parsing (if not
>>> more).
>>
>> For C++11, it has become more complex, which is why I will indeed ask an
>> external tool for type information. Since such a tool will have to build
>> the AST anyway, it will provide that as well.
>
> I'm glad we agree on this. But Java has also become more complex
> (since 1.5, maybe?), and Java 8 added lambdas, which allow omitting
> argument types.

Yes. If people want to hook in Eclim or whatever, that's fine be me. But
it's always the same thing: Those people are *only* iterested in Java,
and they don't want to bother with the bigger picture. This is exactly
why we have a plethora of solutions for different languages at the
moment.

> Not sure about things about Objective-C land.

It's deserted and people move to Swift land.

> But otherwise, if it'll be common to use an external tool for type
> resolution, and even parsing the buffer contents, maybe at some point
> you'll deprecate Wisent grammars. Or use, say, a very simplified
> format for them, only what's strictly necessary to find the names and
> boundaries of tags.

Maybe.

>> CEDET tries to walk a narrow path, trying to provide IDE-like features
>> without Emacs actually becoming a "typical IDE". The IDEs out there have
>> it easier, as they usually force you into organizing your projects in a
>> certain way, and they usually target only one language (or language
>> family).
>
> That's not really true. IntelliJ IDEA supports a big swath of
> languages, and my colleagues use it successfully for our
> "non-standard" Ruby projects (no Rails), and also with JS and
> different markup languages. But of course you have different
> challenges with C++, and the JetBrains team has more manpower anyway.

Yes, and do you know how the Jetbrains guys achieve this? They have an
extensive framework for writing grammars, lexers, etc. Those guys are
weird!

> But I'd be ecstatic to even have a consistent UI for features that VS
> Code (smaller cousin of Visual Studio) touts here:
> https://code.visualstudio.com/docs/editor/editingevolved

Ever tried to load some random make-based C++ project into Visual C++?

-David

Re: IDE

[David Kastrup]

David Engster <deng@randomsample.de> writes:

> Well, it's not like a bunch of people are hacking on the C
> parser. Several things happened:
>
> - Emacs switched to git, so CEDET had to switch as well, which broke my
>   merge setup. I still need to repair it, which is extremely boring
>   work.
>
> - CEDET does not work with Emacs25 due to large changes in
>   EIEIO. Someone has to fix that as well. Again: extremely boring work.
>
> - This gcc-AST discussion.
>
> - New kid.
>
> - In general, hacking Emacs isn't as much fun as it used to be.

To me it seems like changing the last item would likely have the most
long-term impact while likely being the most elusive target.

-- 
David Kastrup

Re: IDE

[Dmitry Gutov]

On 10/18/2015 02:56 PM, David Engster wrote:

> In dynamic languages, you would mostly have tags with no specific type.

I'd say 50/50. Calls with implicit target can roughly be expected to 
have the current class (which can be determined lexically) as the 
target. Similarly for calls on 'this' or 'super'.

Add to this calls on variables that have been assigned a value 
instantiated in the current lexical scope (or, say, method), or the 
results of standard library calls that we've analyzed in advance, where 
we can determine the type of output. This kind of analysis would be 
better left to an external tool, though.

> It is only upstream, and I wouldn't be surprised if it is broken by now
> due to changes in mozrepl (this is why I don't like depending on
> external binaries). I don't code Javascript anymore.

Yes, it didn't seem to work last time I checked (a while ago). The 
implementation should be interesting anyway.

> Well, it's not like a bunch of people are hacking on the C
> parser. Several things happened:

It's not like I'm blaming anyone, really. But it leaves an impression of 
CEDET being more of a research project.

> - In general, hacking Emacs isn't as much fun as it used to be.

Many of us can sympathize, I'm sure.

> Yes, and do you know how the Jetbrains guys achieve this? They have an
> extensive framework for writing grammars, lexers, etc. Those guys are
> weird!

I'm sure that's not all they have. They have completion, and they have 
(at least some kind of) refactorings using the same interface across 
products. That hints at flexible coupling between components.

Or maybe not. EDE-like structure might work for them as well.

>> But I'd be ecstatic to even have a consistent UI for features that VS
>> Code (smaller cousin of Visual Studio) touts here:
>> https://code.visualstudio.com/docs/editor/editingevolved
>
> Ever tried to load some random make-based C++ project into Visual C++?

It probably won't work. But so what? It's great that you have a solution 
for this in CEDET, but it shouldn't impose particular constraints on 
what a project API should look like. At least I don't see why or how it 
should.

My point was about user-facing features anyway. If we only provide them 
initially for languages with simpler/standardized dependency management, 
that would be a big step forward already.

Re: IDE

[David Engster]

Dmitry Gutov writes:
>> Well, it's not like a bunch of people are hacking on the C
>> parser. Several things happened:
>
> It's not like I'm blaming anyone, really. But it leaves an impression
> of CEDET being more of a research project.

It leaves the impression of an understaffed project.

> I'm sure that's not all they have. They have completion, and they have
> (at least some kind of) refactorings using the same interface across
> products. That hints at flexible coupling between components.

It's not a secret:

http://www.jetbrains.org/intellij/sdk/docs/reference_guide/custom_language_support.html

I haven't looked at it in detail, but it looks familiar to me.

>> Ever tried to load some random make-based C++ project into Visual C++?
>
> It probably won't work. But so what? It's great that you have a
> solution for this in CEDET,

We don't. Makefiles are way too complex. The best solution IMHO is using
compilation databases, and EDE upstream has support for them. Of course,
it's another LLVM thingy, so I'm not sure if I'm allowed to merge it.

> but it shouldn't impose particular constraints on what a project API
> should look like. At least I don't see why or how it should.

My point is that people say they want an Emacs to be an "IDE", but at
the same time they don't want to be restricted in their Emacs usage, and
rightfully so. A typical IDE like Visual C++ forces you into using their
project system, otherwise nothing will work. So people want
IDE-features, but at the same time they want all the freedom Emacs
allows. And that's what CEDET tries to support.

Eric started with EDE as a project system similar to other IDEs, that
means you can start a project from scratch with EDE and it will manage
the build for you by generating a Makefile. While technically
impressive, I never liked this kind of project system, and honestly, I
think we should probably drop that part from EDE. But you are not forced
to use it, because Eric also added a custom project detection that is
very flexible. But if you manage the build yourself, you have to specify
everything manually (include paths, used compiler, etc.).

It doesn't help that Emacs is a very conservative piece of software. A
good example was already given: C++ includes without an extension. By
default, Emacs will open such files in fundamental mode. The GNU libc
actually has local variables comments in their includes just for that
reason! But of course, Qt does not have those, so Emacs cannot parse
those files by default, and people complain about Semantic not parsing
anything. Any other C++ IDE will make that decision for you, but guess
what would happen if we simply modified people's auto-mode-alist?

-David

Re: IDE

[David Kastrup]

David Engster <deng@randomsample.de> writes:

> Dmitry Gutov writes:
>>> Well, it's not like a bunch of people are hacking on the C
>>> parser. Several things happened:
>>
>> It's not like I'm blaming anyone, really. But it leaves an impression
>> of CEDET being more of a research project.
>
> It leaves the impression of an understaffed project.

Which is the rule rather than the exception for Free Software projects.
Even when done under corporate veil, the expectation for Free Software
(partly due to the claims of the Open Source movement) is very much that
many people may pitch in.

Most projects are neither prepared to deal with significantly fewer
people pitching in than expected (the more likely case) as they are for
significantly more people pitching in.

The key provision for either possibility is a modular architecture where
people can work on multiple fronts without either depending too much on
each other for their individual progress, nor getting in each other's
hair.

I think that if CEDET fell apart into more independently accessible,
usable, and changeable parts, it might gather more buy-in on its
independent components.  Of course, having a separate repository that
has diverged from the versions in upstream Emacs does not exactly help
with having contributors become active on just their favorite parts of
it.

-- 
David Kastrup

Re: IDE

[Eric Ludlam]

On 10/18/2015 01:28 PM, David Kastrup wrote:
> I think that if CEDET fell apart into more independently accessible,
> usable, and changeable parts, it might gather more buy-in on its
> independent components.  Of course, having a separate repository that
> has diverged from the versions in upstream Emacs does not exactly help
> with having contributors become active on just their favorite parts of
> it.

There are many examples where different developers with none to minimal 
help have successfully:

* Added new language parsers
* created new EDE project types and detection schemes
* Added support for new code generation steps.
* built whole new language independent tools using APIs from semantic

The core parts of CEDET is all about allowing extension by having well 
defined interfaces, and clear places where your custom thing can diverge 
and grow.

Many of the contributed pieces are in the upstream "contrib" area when 
they couldn't not provide assignment, and some are part of CEDET when we 
do have assignments.

I think it is instead more fun and easier to go off and whip up your own 
thing wrapping up some external tool than it is to become part of 
something else.

Also, I'm not much of a verbose advocate, which is probably what is 
needed here.

Eric

Re: IDE

[Achim Gratz]

David Engster writes:
> It doesn't help that Emacs is a very conservative piece of software. A
> good example was already given: C++ includes without an extension. By
> default, Emacs will open such files in fundamental mode.

If I use /usr/bin/file on such an include, it happily tells me it's been
looking at "C++ source, ASCII text".  So instead of insisting on a known
extension to determine the major mode, Emacs could check what the file
mode is supposed to be in its absence before falling back to fundamental
mode.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Re: IDE

[David Kastrup]

Achim Gratz <Stromeko@nexgo.de> writes:

> David Engster writes:
>> It doesn't help that Emacs is a very conservative piece of software. A
>> good example was already given: C++ includes without an extension. By
>> default, Emacs will open such files in fundamental mode.
>
> If I use /usr/bin/file on such an include, it happily tells me it's been
> looking at "C++ source, ASCII text".  So instead of insisting on a known
> extension to determine the major mode, Emacs could check what the file
> mode is supposed to be in its absence before falling back to fundamental
> mode.

Emacs can do that.

magic-mode-alist is a variable defined in ‘files.el’.
Its value is nil

  This variable may be risky if used as a file-local variable.

Documentation:
Alist of buffer beginnings vs. corresponding major mode functions.
Each element looks like (REGEXP . FUNCTION) or (MATCH-FUNCTION . FUNCTION).
After visiting a file, if REGEXP matches the text at the beginning of the
buffer, or calling MATCH-FUNCTION returns non-nil, ‘normal-mode’ will
call FUNCTION rather than allowing ‘auto-mode-alist’ to decide the buffer’s
major mode.

If FUNCTION is nil, then it is not called.  (That is a way of saying
"allow ‘auto-mode-alist’ to decide for these files.")

[back]




-- 
David Kastrup

Re: IDE

[Achim Gratz]

David Kastrup writes:
> Achim Gratz <Stromeko@nexgo.de> writes:
>
>> David Engster writes:
>>> It doesn't help that Emacs is a very conservative piece of software. A
>>> good example was already given: C++ includes without an extension. By
>>> default, Emacs will open such files in fundamental mode.
>>
>> If I use /usr/bin/file on such an include, it happily tells me it's been
>> looking at "C++ source, ASCII text".  So instead of insisting on a known
>> extension to determine the major mode, Emacs could check what the file
>> mode is supposed to be in its absence before falling back to fundamental
>> mode.
>
> Emacs can do that.
>
> magic-mode-alist is a variable defined in ‘files.el’.
> Its value is nil
>
>   This variable may be risky if used as a file-local variable.
>
> Documentation:
> Alist of buffer beginnings vs. corresponding major mode functions.
> Each element looks like (REGEXP . FUNCTION) or (MATCH-FUNCTION . FUNCTION).
> After visiting a file, if REGEXP matches the text at the beginning of the
> buffer, or calling MATCH-FUNCTION returns non-nil, ‘normal-mode’ will
> call FUNCTION rather than allowing ‘auto-mode-alist’ to decide the buffer’s
> major mode.
>
> If FUNCTION is nil, then it is not called.  (That is a way of saying
> "allow ‘auto-mode-alist’ to decide for these files.")
>
> [back]

Well, that does the opposite of what I described: it doesn't check
auto-mode-alist at all when it matches.  I want auto-mode-alist to take
precedence and only if it doesn't know any better than
"fundamental-mode" should it consult some other mechanism.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables

Re: IDE

[David Kastrup]

Achim Gratz <Stromeko@nexgo.de> writes:

> David Kastrup writes:
>> Achim Gratz <Stromeko@nexgo.de> writes:
>>
>>> David Engster writes:
>>>> It doesn't help that Emacs is a very conservative piece of software. A
>>>> good example was already given: C++ includes without an extension. By
>>>> default, Emacs will open such files in fundamental mode.
>>>
>>> If I use /usr/bin/file on such an include, it happily tells me it's been
>>> looking at "C++ source, ASCII text".  So instead of insisting on a known
>>> extension to determine the major mode, Emacs could check what the file
>>> mode is supposed to be in its absence before falling back to fundamental
>>> mode.
>>
>> Emacs can do that.
>>
>> magic-mode-alist is a variable defined in ‘files.el’.

[...]

> Well, that does the opposite of what I described: it doesn't check
> auto-mode-alist at all when it matches.  I want auto-mode-alist to take
> precedence and only if it doesn't know any better than
> "fundamental-mode" should it consult some other mechanism.

Ok, how about a different approach using auto-mode-alist?
auto-mode-alist contains several patterns including directories, so one
could match on /include/[a-zA-Z-]+\' or similar.  That's somewhat crude
(and non C++ programmers might protest the results) but at least for a
C++ programmer it seems like a reasonable default setting.  Another
possibility for particular projects would be to use directory variables.

-- 
David Kastrup

Re: IDE

[Dmitry Gutov]

On 10/18/2015 08:12 PM, David Engster wrote:

> http://www.jetbrains.org/intellij/sdk/docs/reference_guide/custom_language_support.html

Thanks a lot, that should be very helpful. I have their Community 
Edition installed and try it out from time to time, but I haven't looked 
into their documentation, and this looks pretty good.

> We don't. Makefiles are way too complex. The best solution IMHO is using
> compilation databases, and EDE upstream has support for them. Of course,
> it's another LLVM thingy, so I'm not sure if I'm allowed to merge it.

I really think we should move CEDET to ELPA. It can allow you to keep 
the users up-to-date more easily, and would make cedet-devel-load.el 
unnecessary. The standard of what we're "allowed" to distribute, is also 
a bit more lax there, in practice.

But if it turns out to be a problem, as long as the compilation database 
is pluggable, you could extract it into a new package and let users 
install it from MELPA, until the policy here gets with the times.

> My point is that people say they want an Emacs to be an "IDE", but at
> the same time they don't want to be restricted in their Emacs usage, and
> rightfully so. A typical IDE like Visual C++ forces you into using their
> project system, otherwise nothing will work. So people want
> IDE-features, but at the same time they want all the freedom Emacs
> allows. And that's what CEDET tries to support.

Yes. The "project system" I'd expect from Emacs is pretty different. A 
lot of users have lived with a "lightweight" one (Projectile) for years, 
so I try to approach the subject in project.el also from that direction.

> Eric started with EDE as a project system similar to other IDEs, that
> means you can start a project from scratch with EDE and it will manage
> the build for you by generating a Makefile. While technically
> impressive, I never liked this kind of project system, and honestly, I
> think we should probably drop that part from EDE.

I'd say drop it, but you might have users depending on it.

> But you are not forced
> to use it, because Eric also added a custom project detection that is
> very flexible. But if you manage the build yourself, you have to specify
> everything manually (include paths, used compiler, etc.).

But you can also create a specialized project implementation that knows 
how to read e.g. include paths from a project file, right?

> It doesn't help that Emacs is a very conservative piece of software. A
> good example was already given: C++ includes without an extension. By
> default, Emacs will open such files in fundamental mode. The GNU libc
> actually has local variables comments in their includes just for that
> reason! But of course, Qt does not have those, so Emacs cannot parse
> those files by default, and people complain about Semantic not parsing
> anything. Any other C++ IDE will make that decision for you, but guess
> what would happen if we simply modified people's auto-mode-alist?

I always thought that magic-mode-alist takes action only after no 
matches in auto-mode-alist have been found. Maybe that's a worthwhile 
change to make.

Or add a new, third variable, meta-magic-mode-alist, that would take 
action after auto-mode-alist, when the fundamental mode is picked.

Re: IDE

[Eric Ludlam]

On 10/17/2015 09:21 AM, Dmitry Gutov wrote:
> And speaking of more personal reasons, I already have written a code
> assistance tool for Ruby, in Ruby (which is not a Lisp, but still a
> pretty okay language), and smooth integration with whatever APIs we
> choose would be nice.

We should poke at this.  Can you share a little about what your tool 
does?  Then perhaps we hypothesize about the efficacy of integrating 
into CEDET as an example of integrating external tools.

Eric

Re: IDE

[Dmitry Gutov]

On 10/20/2015 04:03 AM, Eric Ludlam wrote:

> We should poke at this.  Can you share a little about what your tool
> does?  Then perhaps we hypothesize about the efficacy of integrating
> into CEDET as an example of integrating external tools.

It probably wouldn't gain too much from CEDET integration.

It has:

- Server, serving JSON over HTTP, with a RPC-like protocol.

- Emacs client in the shape of a minor mode. It defines a 
completion-at-point-functions element and a keymap with essentially 
three commands: "jump to the definition", "jump back", "show 
documentation for the method at point".

- To determine the current context (which would be similar to "current 
tag" in Semantic), it calls ruby-add-log-current-method and parses the 
returned string. I've improved that function in ruby-mode a year or two 
ago, and it's pretty accurate.

Robe is also pretty hacky compared to most other similar tools:

- The server loads itself into and runs from an REPL buffer. We either 
expect the user to already have such a REPL running (with the project 
loaded), or offer to launch one automatically (which fails in certain 
configurations).

- It doesn't support multiple projects in the same Emacs session.

- It misses some trivial opportunities to infer the type of a local 
variable. That would be my first priority to work on... when I deal with 
all that project and xref stuff in the core, I guess.

Lives here: https://github.com/dgutov/robe/

Re: IDE

[Eric Ludlam]

On 10/20/2015 07:28 AM, Dmitry Gutov wrote:
> On 10/20/2015 04:03 AM, Eric Ludlam wrote:
>
>> We should poke at this.  Can you share a little about what your tool
>> does?  Then perhaps we hypothesize about the efficacy of integrating
>> into CEDET as an example of integrating external tools.
>
> It probably wouldn't gain too much from CEDET integration.

Ruby support tooling itself would not benefit from CEDET integration, 
but tools built on CEDET would gain Ruby support, and that improves the 
diversity of Ruby related tools available.

> It has:
>
> - Server, serving JSON over HTTP, with a RPC-like protocol.
>
> - Emacs client in the shape of a minor mode. It defines a 
> completion-at-point-functions element and a keymap with essentially 
> three commands: "jump to the definition", "jump back", "show 
> documentation for the method at point".
>

Inside this feature you must have a way to query for the location of a 
particular symbol, and convert a symbol into a doc reference.

In CEDET speak, the tooling that converts a symbol into a location would 
be a special kind of Semantic Database.  This could be tied to a project 
or tied in as a system database depending on what it is querying.  Or 
one of each.  The job of such databases is to provide a search for tag 
by name, completion substring, regexp, and for languages that care, type.

The output is a list of matches as faux tags.  If an application wanted 
to know more about the symbol, it would pull in the reference file, and 
extract real tag data using whatever parser is available.

This would enable Semantic's jump to tag system to be as accurate as yours.

I am not sure what your doc piece might be like.  There is some limited 
support for finding doc strings, but usually it just looks for comments 
preceding a tag.

> - To determine the current context (which would be similar to "current 
> tag" in Semantic), it calls ruby-add-log-current-method and parses the 
> returned string. I've improved that function in ruby-mode a year or 
> two ago, and it's pretty accurate.

We used to have a way to tweak add-log but I think the mechanism is for 
an older version of Emacs where we needed to use advice.  It would make 
sense to update this if there is a better way.

> Robe is also pretty hacky compared to most other similar tools:
>
> - The server loads itself into and runs from an REPL buffer. We either 
> expect the user to already have such a REPL running (with the project 
> loaded), or offer to launch one automatically (which fails in certain 
> configurations).
>
> - It doesn't support multiple projects in the same Emacs session.
>

An EDE type project for ruby (whatever that looks like) would provide a 
place to hang project specific REPL buffers as needed.

> - It misses some trivial opportunities to infer the type of a local 
> variable. That would be my first priority to work on... when I deal 
> with all that project and xref stuff in the core, I guess.

I'm not sure which code bit you are referencing here.  If you do your 
tag parsing with a semantic grammar, then you can optionally use that 
same grammar to parse function bodies, and thus make detecting local 
variable types a bit easier.   I'm speculating however as I am not 
familiar with Ruby.

There is a wisent based grammar for Ruby in the 'contrib' area that 
seems straightforward.  It would probably not be much of a stretch to 
build one with the right releases to include in Emacs, solve this one 
problem, and then get support from other CEDET tools.

Eric

Re: IDE

[Dmitry Gutov]

On 10/21/2015 06:13 AM, Eric Ludlam wrote:

> Ruby support tooling itself would not benefit from CEDET integration,
> but tools built on CEDET would gain Ruby support, and that improves the
> diversity of Ruby related tools available.

If CEDET support won't improve Robe, and if the said support will amount 
to writing a Wisent grammar, it just becomes a separate task.

Which I don't object to doing, in principle, but it'll go to the bottom 
of the pile, in terms of priority.

> Inside this feature you must have a way to query for the location of a
> particular symbol, and convert a symbol into a doc reference.

In both cases, I often have to prompt the user (with completing-read) to 
disambiguate between classes that define the method with the given name.

> The output is a list of matches as faux tags.  If an application wanted
> to know more about the symbol, it would pull in the reference file, and
> extract real tag data using whatever parser is available.

So as faux tags, I could return all methods with the same name from 
different classes?

> This would enable Semantic's jump to tag system to be as accurate as yours.

Would that be semantic-complete-jump or semantic-ia-fast-jump?

> I am not sure what your doc piece might be like.  There is some limited
> support for finding doc strings, but usually it just looks for comments
> preceding a tag.

It's a struct, containing both the arguments list, and a doc string.

I display it specially, though: do a little lightweight parsing, and 
highlight the Ruby examples parts with ruby-mode font-lock rules, and C 
sources with c-mode. Plus highlighted function signature at the top and 
a button to go to the source.

> We used to have a way to tweak add-log but I think the mechanism is for
> an older version of Emacs where we needed to use advice.  It would make
> sense to update this if there is a better way.

ruby-add-log-current-method is the value of 
add-log-current-defun-function, but I don't know if you'd be able to use 
it for different languages: the format doesn't seem to be particularly 
standardized.

> An EDE type project for ruby (whatever that looks like) would provide a
> place to hang project specific REPL buffers as needed.

How? Using which major mode? I current use inf-ruby for that (not 
available in Emacs, for copyright assignment reasons). So it seems I'd 
have to add multi-REPL support for it first.

The conceptually hard question is what do you do with external files 
that can be referenced from different projects? Suppose you have two 
open projects, each with its own REPL, you made a jump to an external 
file from project1. The you want to "jump to definition" on some method 
call in there. How do you pick the right REPL to ask for info?

Suppose, after the first jump, you saved the reference to the right 
project in a buffer-local variable, so you can refer to it for the 
second jump. What if I want to do the next jump not from the same exact 
file, but from its neighbor? As a user, I can be confident that both 
files must be referenced by the project, but there will be no 
buffer-local value to use.

Finding an idiomatic approach to that would be great.

>> - It misses some trivial opportunities to infer the type of a local
>> variable. That would be my first priority to work on... when I deal
>> with all that project and xref stuff in the core, I guess.
>
> I'm not sure which code bit you are referencing here.  If you do your
> tag parsing with a semantic grammar, then you can optionally use that
> same grammar to parse function bodies, and thus make detecting local
> variable types a bit easier.   I'm speculating however as I am not
> familiar with Ruby.

I don't know how much work would that be. Ruby doesn't have anything 
close to official, up-to-date BNF grammar. And it's pretty complex.

At the moment, I'm doing context parsing in Elisp, so fix for the 
"trivial opportunities" might also be in Elisp, at first, with a few 
simple regexp searches.

However, I'm seriously considering moving that part (and more) to Ruby, 
so that authors of integration with other editors won't have to redo 
that work:

- There's a feature request for Vim support.
- Someone actually implemented an Atom plugin already (not at all 
popular so far, but that doesn't matter).

I'd also be able to reuse an existing parser. There is one that's been 
gaining in popularity over the last years.

> There is a wisent based grammar for Ruby in the 'contrib' area that
> seems straightforward.  It would probably not be much of a stretch to
> build one with the right releases to include in Emacs, solve this one
> problem, and then get support from other CEDET tools.

Since that grammar has been outside of Emacs for so long, I always 
assumed that the author vanished, and obtaining the release is impossible.

Re: IDE

[Eric Ludlam]

On 10/21/2015 06:54 AM, Dmitry Gutov wrote:
> On 10/21/2015 06:13 AM, Eric Ludlam wrote:
>
>> Ruby support tooling itself would not benefit from CEDET integration,
>> but tools built on CEDET would gain Ruby support, and that improves the
>> diversity of Ruby related tools available.
>
> If CEDET support won't improve Robe, and if the said support will 
> amount to writing a Wisent grammar, it just becomes a separate task.
>
> Which I don't object to doing, in principle, but it'll go to the 
> bottom of the pile, in terms of priority.
I don't think it is about if CEDET improves ROBE.  I looked at ROBE on 
github, and think there is some core piece of it regarding talking to 
ruby that can be integrated as an extension of CEDET. You could then use 
the company-mode, auto-complete, plus many other tools that already use 
CEDET APIs, and could delete that code from robe.

In other words, your ruby experience would have more options without 
having to add them to robe yourself.

It is speculative to suggest that writing a ruby wisent parser would 
take less time than to write all the same types of extensions already on 
top of CEDET in Robe since I'm not sure which subset of features is the 
useful subset for something like ruby.

>> Inside this feature you must have a way to query for the location of a
>> particular symbol, and convert a symbol into a doc reference.
>
> In both cases, I often have to prompt the user (with completing-read) 
> to disambiguate between classes that define the method with the given 
> name.
>

In this case, things like semantic-complete-jump you could use the TAB 
technique to disambiguate, and semantic-ia-complete-jump might be able 
to distinguish depending on what sort of type information is available.

There is at least one contribution on top of cedet that do similar 
disambiguation using fancy uis.  I think it copied a textmate feature.

>> The output is a list of matches as faux tags.  If an application wanted
>> to know more about the symbol, it would pull in the reference file, and
>> extract real tag data using whatever parser is available.
>
> So as faux tags, I could return all methods with the same name from 
> different classes?

Yes.  You would provide what you do know (full class names, fields, 
methods, different file locations, etc) which could be initially 
reasoned on.  Sometimes no more is needed.  Other times those files 
might be opened to get more data using a better wisent parser.

The semantic javap extension pulls java tags from Jar files, and does 
not need to claim they are temporary tags because the data provided is 
as-good as the java parser.  If your ruby system is similar, then the 
system would not need to extract additional data from the source file.

>> This would enable Semantic's jump to tag system to be as accurate as 
>> yours.
>
> Would that be semantic-complete-jump or semantic-ia-fast-jump?
>

Both, plus other similar functions different people have written.

>> An EDE type project for ruby (whatever that looks like) would provide a
>> place to hang project specific REPL buffers as needed.
>
> How? Using which major mode? I current use inf-ruby for that (not 
> available in Emacs, for copyright assignment reasons). So it seems I'd 
> have to add multi-REPL support for it first.
>
> The conceptually hard question is what do you do with external files 
> that can be referenced from different projects? Suppose you have two 
> open projects, each with its own REPL, you made a jump to an external 
> file from project1. The you want to "jump to definition" on some 
> method call in there. How do you pick the right REPL to ask for info?
>

In C++ and Java, the common files would be 'system' includes.  The 
naming is a bit C centric, but the basic idea is that there are system 
includes projects refer to, and those in turn aren't in a project, but 
refer to system paths to continue navigating between themselves.

If you instead refer to a library of files unrelated to the system, then 
ideally those would have their own project structure around them to 
enable correct navigation.

Of course, the above examples for C don't have external REPL buffers to 
maintain, but might use independent GNU Global databases which are less 
heavy weight than an external process-per-project.

> Suppose, after the first jump, you saved the reference to the right 
> project in a buffer-local variable, so you can refer to it for the 
> second jump. What if I want to do the next jump not from the same 
> exact file, but from its neighbor? As a user, I can be confident that 
> both files must be referenced by the project, but there will be no 
> buffer-local value to use.
>

I'm not sure I followed this.  The shared code refers back to project1 
in some way?  This sounds like messy code you wouldn't want to write so 
I'm not sure I understand what you are trying to do.

Standard emacs mark-ring stuff is usually sufficient for traveling back.

> Finding an idiomatic approach to that would be great.
>
>>> - It misses some trivial opportunities to infer the type of a local
>>> variable. That would be my first priority to work on... when I deal
>>> with all that project and xref stuff in the core, I guess.
>>
>> I'm not sure which code bit you are referencing here.  If you do your
>> tag parsing with a semantic grammar, then you can optionally use that
>> same grammar to parse function bodies, and thus make detecting local
>> variable types a bit easier.   I'm speculating however as I am not
>> familiar with Ruby.
>
> I don't know how much work would that be. Ruby doesn't have anything 
> close to official, up-to-date BNF grammar. And it's pretty complex.
>
> At the moment, I'm doing context parsing in Elisp, so fix for the 
> "trivial opportunities" might also be in Elisp, at first, with a few 
> simple regexp searches.
>
> However, I'm seriously considering moving that part (and more) to 
> Ruby, so that authors of integration with other editors won't have to 
> redo that work:
>
> - There's a feature request for Vim support.
> - Someone actually implemented an Atom plugin already (not at all 
> popular so far, but that doesn't matter).
>
> I'd also be able to reuse an existing parser. There is one that's been 
> gaining in popularity over the last years.
>

Semantic doesn't demand it's parsers be in wisent, or even in Emacs at 
all.  If you have a nice Ruby grammar in Ruby, and you can convert it's 
internal data into lisp-like syntax, pulling it into the Semantic system 
is pretty easy.  What you would loose is dynamic reparsing without 
having to save, thus it may be a bit quirky.

>> There is a wisent based grammar for Ruby in the 'contrib' area that
>> seems straightforward.  It would probably not be much of a stretch to
>> build one with the right releases to include in Emacs, solve this one
>> problem, and then get support from other CEDET tools.
>
> Since that grammar has been outside of Emacs for so long, I always 
> assumed that the author vanished, and obtaining the release is 
> impossible.
>

Indeed, I suggested it to simply show someone made a useful grammar that 
looks relatively simple.

Eric

Re: IDE

[Dmitry Gutov]

On 10/22/2015 01:52 AM, Eric Ludlam wrote:

> I don't think it is about if CEDET improves ROBE.  I looked at ROBE on
> github, and think there is some core piece of it regarding talking to
> ruby that can be integrated as an extension of CEDET. You could then use
> the company-mode, auto-complete,

Well, that part doesn't make sense. I couldn't care less about 
auto-complete, and company-mode already supports 
completion-at-point-functions. Semantic does, too. 
completion-at-point-functions is a thinner dependency, and its API is at 
least as rich as what Semantic provides.

> plus many other tools that already use
> CEDET APIs, and could delete that code from robe.

Being able to delete any considerable amount of code is doubtful. 
Speaking of "many other tools", a compelling example would help. IMenu 
already works adequately thanks to ruby-mode, and which-func-mode works 
as well.

> In other words, your ruby experience would have more options without
> having to add them to robe yourself.

I get that idea, of course. An extra file with a bridge to Semantic 
wouldn't be the worst thing to have.

> It is speculative to suggest that writing a ruby wisent parser would
> take less time than to write all the same types of extensions already on
> top of CEDET in Robe since I'm not sure which subset of features is the
> useful subset for something like ruby.

Therein lies the problem: I've been trying out CEDET for a while, and 
I'm still not sure which extensions I'd want to reimplement. TAB 
completion in semantic-complete-jump is kinda nice, but it has a 
downside compared to the xref interface as well.

> In this case, things like semantic-complete-jump you could use the TAB
> technique to disambiguate, and semantic-ia-complete-jump might be able
> to distinguish depending on what sort of type information is available.

At first I was going to say it might be a solid (if small) improvement, 
but... there might be a mismatch:

- Simply searching for methods with the given name, like 
semantic-complete-jump, iterating through all matches, is wasteful IMO. 
It's much better to use the context if you can (or allow the user to 
input the fully qualified method name, as an extra feature).

- Even when the context is used, Robe can still return multiple matches. 
So if semantic-ia-complete-jump can't deal with that (and display all 
matches, for instance), it's not a good fit either.

> There is at least one contribution on top of cedet that do similar
> disambiguation using fancy uis.  I think it copied a textmate feature.

I'd love to take a look.

> The semantic javap extension pulls java tags from Jar files, and does
> not need to claim they are temporary tags because the data provided is
> as-good as the java parser.  If your ruby system is similar, then the
> system would not need to extract additional data from the source file.

Likewise, Robe would return a list of more-or-less exact tags.

Ruby runtimes keep the location of every defined method.

> In C++ and Java, the common files would be 'system' includes.  The
> naming is a bit C centric, but the basic idea is that there are system
> includes projects refer to, and those in turn aren't in a project, but
> refer to system paths to continue navigating between themselves.

That can be a system include, or it can be a library in a Git checkout 
that, say, two projects are using (it's included in the build by both of 
them).

> If you instead refer to a library of files unrelated to the system, then
> ideally those would have their own project structure around them to
> enable correct navigation.

a) There might not be a sufficient project structure (as with the system 
includes).
b) The project structure of that external dependency might not be 
enough. Here's what should be a clearer analogy:

Let's say that my projects P1 and P2 include the library lib_foo. It 
contains something like this:

class IFoo {
   public:
     virtual void bar();
};

class Tee {
   public:
     int qux(IFoo& foo) {
       foo.bar();
     }
}

Now, somewhere in P1 there's a snippet of code like:

   tee = new Tee();
   tee.qux(new P1Foo());

To follow the control flow, I move cursor to the qux call, jump to its 
definition (thus moving into lib_foo), and then I need to find the 
definition of bar that qux is calling (or rather, all of its overrides).

But only looking in lib_foo is insufficient for that: P1Foo is defined 
inside P1, and lib_foo, by itself, doesn't know anything about P1. It's 
P1 that depends on it.

So a seemingly obvious solution would be, when jumping to lib_foo, to 
store the fact that we came from P1 this time around. But there's no 
obvious place to keep it (directory-local storage?), and there might be 
other difficulties I haven't accounted for.

> Of course, the above examples for C don't have external REPL buffers to
> maintain, but might use independent GNU Global databases which are less
> heavy weight than an external process-per-project.

Yes, I believe you're going to have the same problem with etags, gtags 
or etc, if you try to use it in the described scenario.

> I'm not sure I followed this.  The shared code refers back to project1
> in some way?  This sounds like messy code you wouldn't want to write so
> I'm not sure I understand what you are trying to do.

I want to keep track of which project I'm working in right now. For code 
navigation forward, not back, like described above.

> Standard emacs mark-ring stuff is usually sufficient for traveling back.

Mark rings are highly inadequate, but that's beside the point.

> Semantic doesn't demand it's parsers be in wisent, or even in Emacs at
> all.  If you have a nice Ruby grammar in Ruby, and you can convert it's
> internal data into lisp-like syntax, pulling it into the Semantic system
> is pretty easy.  What you would loose is dynamic reparsing without
> having to save, thus it may be a bit quirky.

Why would I have to sacrifice reparsing without saving? I could sent the 
current buffer contents to the completion server, e.g. over HTTP. That's 
actually common practice now.

> Indeed, I suggested it to simply show someone made a useful grammar that
> looks relatively simple.

Simply redoing it to clear the copyright status won't be enough. Last I 
checked (some 3 years ago), the grammar didn't work. I've also seen 
complaints to that effect somewhere on the web.

Re: IDE

[Eric Ludlam]

On 10/21/2015 07:57 PM, Dmitry Gutov wrote:
> On 10/22/2015 01:52 AM, Eric Ludlam wrote:
>
>> I don't think it is about if CEDET improves ROBE.  I looked at ROBE on
>> github, and think there is some core piece of it regarding talking to
>> ruby that can be integrated as an extension of CEDET. You could then use
>> the company-mode, auto-complete,
>
> Well, that part doesn't make sense. I couldn't care less about 
> auto-complete, and company-mode already supports 
> completion-at-point-functions. Semantic does, too. 
> completion-at-point-functions is a thinner dependency, and its API is 
> at least as rich as what Semantic provides.
>

I assumed auto-complete and company were important because the ROBE 
documentation talked about how it supported those tools.

I'm not sure what is relevant here as far as listing features. There are 
12 minor modes that come with semantic doing various diverse things.  
Things like idle-summary-mode will show you an api summary of the symbol 
under point.  Breadcrumbs shows you a more oo version of where the 
cursor is.  Outside of those modes, the core analyzer and completion 
engines both use database backends to figure out what is going on near 
cursor and do completion.  There is COGRE which can pull data from the 
buffer, reference some databases, and pop up a uml diagram of your code 
which is a bit over-the-top. There are random utilities like semantic 
refactor that would need to be extended to new langauges but would gain 
base support from cedet.  There are browsers like speedbar and ECB that 
pull in data and let you navigate the structure of the code.

Of course, if none of those things are important, then you would have no 
reason to want to integrate w/ CEDET.

>
>> There is at least one contribution on top of cedet that do similar
>> disambiguation using fancy uis.  I think it copied a textmate feature.
>
> I'd love to take a look.
>
It is called eassist.  Looks like I was remembering it from 2006, so not 
sure how it will work with recent Emacs changes.

>
>> If you instead refer to a library of files unrelated to the system, then
>> ideally those would have their own project structure around them to
>> enable correct navigation.
>
> a) There might not be a sufficient project structure (as with the 
> system includes).
> b) The project structure of that external dependency might not be 
> enough. Here's what should be a clearer analogy:
>
> Let's say that my projects P1 and P2 include the library lib_foo. It 
> contains something like this:
>
> class IFoo {
>   public:
>     virtual void bar();
> };
>
> class Tee {
>   public:
>     int qux(IFoo& foo) {
>       foo.bar();
>     }
> }
>
> Now, somewhere in P1 there's a snippet of code like:
>
>   tee = new Tee();
>   tee.qux(new P1Foo());
>
> To follow the control flow, I move cursor to the qux call, jump to its 
> definition (thus moving into lib_foo), and then I need to find the 
> definition of bar that qux is calling (or rather, all of its overrides).
>
> But only looking in lib_foo is insufficient for that: P1Foo is defined 
> inside P1, and lib_foo, by itself, doesn't know anything about P1. 
> It's P1 that depends on it.
>
> So a seemingly obvious solution would be, when jumping to lib_foo, to 
> store the fact that we came from P1 this time around. But there's no 
> obvious place to keep it (directory-local storage?), and there might 
> be other difficulties I haven't accounted for.
>

Ah, I see.  I hadn't considered a workflow like this.  I can see how 
that would be useful.  I haven't thought about how to solve that 
problem.  I suspect the tooling would be custom where it knows more 
about the language, and can infer intent based on things like 'virtual' 
or the prospect of there being overrides.


>
>> Semantic doesn't demand it's parsers be in wisent, or even in Emacs at
>> all.  If you have a nice Ruby grammar in Ruby, and you can convert it's
>> internal data into lisp-like syntax, pulling it into the Semantic system
>> is pretty easy.  What you would loose is dynamic reparsing without
>> having to save, thus it may be a bit quirky.
>
> Why would I have to sacrifice reparsing without saving? I could sent 
> the current buffer contents to the completion server, e.g. over HTTP. 
> That's actually common practice now.
>
That sounds good.


Eric

Re: IDE

[Dmitry Gutov]

On 10/22/2015 04:35 AM, Eric Ludlam wrote:

> I assumed auto-complete and company were important because the ROBE
> documentation talked about how it supported those tools.

Oh, right. Sorry.

> I'm not sure what is relevant here as far as listing features. There are
> 12 minor modes that come with semantic doing various diverse things.
> Things like idle-summary-mode will show you an api summary of the symbol
> under point.

Robe already supports eldoc-mode. If idle-summary-mode has some UI 
advantages, I believe they should be folded into eldoc-mode as well.

> There is COGRE which can pull data from the
> buffer, reference some databases, and pop up a uml diagram of your code
> which is a bit over-the-top.

COGRE sounds good, but I imagine it'll need more support than just 
dumping the AST.

And I can't get it to do anything (here's the documentation for 
automatic generation: 
http://www.randomsample.de/cedetdocs/cogre/Semantic-Support.html#Semantic-Support, 
and trying to interact with the editor manually leads to "eieio-oref: 
eieio-oref called on a class: cogre-class", etc).

It's also not in Emacs, for some reason.

> It is called eassist.  Looks like I was remembering it from 2006, so not
> sure how it will work with recent Emacs changes.

I've taken a look, and it works okay as an alternative to IMenu, but 
jumping to method at point doesn't work (shows an error).

> Ah, I see.  I hadn't considered a workflow like this.  I can see how
> that would be useful.  I haven't thought about how to solve that
> problem.  I suspect the tooling would be custom where it knows more
> about the language, and can infer intent based on things like 'virtual'
> or the prospect of there being overrides.

The tooling needs to be smart enough, of course.

Java IDEs support workflows like that, and I think they're more 
important than speedbar or showing breadcrumbs (*).

Just to let you know where my priorities are. Here's the current plan 
for that feature, by the way: 
https://github.com/nonsequitur/inf-ruby/issues/76#issuecomment-150182991

(*) Other things that the users ask for is fuzzy completion, showing 
completions right away after dot (Robe isn't fast enough for that) and 
working over TRAMP inside a Vagrant environment. It doesn't seem like 
CEDET integration will help with any of those.

Re: IDE

[Eric Ludlam]

On 10/22/2015 07:17 AM, Dmitry Gutov wrote:
> On 10/22/2015 04:35 AM, Eric Ludlam wrote:
>
>> I'm not sure what is relevant here as far as listing features. There are
>> 12 minor modes that come with semantic doing various diverse things.
>> Things like idle-summary-mode will show you an api summary of the symbol
>> under point.
>
> Robe already supports eldoc-mode. If idle-summary-mode has some UI 
> advantages, I believe they should be folded into eldoc-mode as well.
>

Semantic's idle process handling makes sure databases are synchronized 
once, then enables tools to run after, so it is more about the 
scheduling of different tools that use semantic.

In addition, by going through the semantic version, there are a range of 
different formatting options to use that the user can select from.  That 
doesn't require idle-summary-mode, but is a side effect of using 
semantic to extract contextual information.

Also, eldoc only supported Emacs Lisp and no extensions when I wrote the 
semantic idle handler.  Other features of idle-summary-mode would port 
fine between either.

>> There is COGRE which can pull data from the
>> buffer, reference some databases, and pop up a uml diagram of your code
>> which is a bit over-the-top.
>
> COGRE sounds good, but I imagine it'll need more support than just 
> dumping the AST.
>
> And I can't get it to do anything (here's the documentation for 
> automatic generation: 
> http://www.randomsample.de/cedetdocs/cogre/Semantic-Support.html#Semantic-Support, 
> and trying to interact with the editor manually leads to "eieio-oref: 
> eieio-oref called on a class: cogre-class", etc).
>
> It's also not in Emacs, for some reason.

It was deemed optional when Yidong merged CEDET into Emacs.  You will 
probably need to use Emacs24 to make it work.  To try it out do this:

1) Install graphviz (it uses that for the layout engine)
2) Start emacs 24
3) Use CEDET from it's git repository
4) M-x find-library RET cogre RET
5) find cogre-element-peer in the code
6) M-x cogre-uml-quick-class RET

should get you something to play with.

> (*) Other things that the users ask for is fuzzy completion, showing 
> completions right away after dot (Robe isn't fast enough for that) and 
> working over TRAMP inside a Vagrant environment. It doesn't seem like 
> CEDET integration will help with any of those.
>

CEDET is a framework that provides an abstraction for connecting 
different tools that need to talk about hard problems together.  The 
problems it solves are related to project information, abstracting 'tag' 
information down to something Lisp programs can reason on, and 
abstracting code generation into a scheme that can allow lisp programs 
to support multiple languages.  CEDET doesn't have 'fuzzy completion' 
but it can feed a fuzzy completion engine.  CEDET doesn't do anything 
special with TRAMP, but someone could use CEDET to bind that workflow 
into the common workflow.  When thinking about CEDET, it isn't about a 
bullet list of user facing features but about how it can enable someone 
working on said feature to have their work leveraged to a wider audience.

Eric

Re: IDE

[Louis Höfler]

Hello everyone,
I followed that topic with much interest.

I developed a starting point of a IDE implementation in emacs.
http://scm.mathematek.de/index.cgi/emacside/index
It is not nearly finished, but I use it for my daily work.
Most ide's I started to use have tons of options but don't do
the core things well.
For me it is important to have a ide which:
1. Does not break
2. Does not change

I even do not need colored editor output, I just want a working IDE.
An IDE is in my point of view, a toolset which allows me to
compile things fast, and do repeating processes without much time loss.
I dont like IDE environments where you need 200 Dialogs to reach
functions.
My package currently can do compilations, and I started to implement
a emacsserver API, so that you can send a compilation configuration to
a remote Computer, which does the compilation.
If you want to have more functions arround that, you can just hack some
elisp code arround each executed command with hooks, so you can get full
debugger functionalities.
Running gdb after a compilation for example, throws me directly into the
debugger environment, where I can set breakpoints etc.

This is a out-of-the-need 1-man project. So you can't expect it to be as 
complete
and functional as other packages with many contributors.

For me personally, the simple things are the ones I continue to use 
nowadays.
The complex things are those who break from time to time.
Zero administration is one of the biggest concerns if I start to use 
software.


Greetings, Louis

Am 22.10.2015 um 14:58 schrieb Eric Ludlam:
> On 10/22/2015 07:17 AM, Dmitry Gutov wrote:
>> On 10/22/2015 04:35 AM, Eric Ludlam wrote:
>>
>>> I'm not sure what is relevant here as far as listing features. There 
>>> are
>>> 12 minor modes that come with semantic doing various diverse things.
>>> Things like idle-summary-mode will show you an api summary of the 
>>> symbol
>>> under point.
>>
>> Robe already supports eldoc-mode. If idle-summary-mode has some UI 
>> advantages, I believe they should be folded into eldoc-mode as well.
>>
>
> Semantic's idle process handling makes sure databases are synchronized 
> once, then enables tools to run after, so it is more about the 
> scheduling of different tools that use semantic.
>
> In addition, by going through the semantic version, there are a range 
> of different formatting options to use that the user can select from.  
> That doesn't require idle-summary-mode, but is a side effect of using 
> semantic to extract contextual information.
>
> Also, eldoc only supported Emacs Lisp and no extensions when I wrote 
> the semantic idle handler.  Other features of idle-summary-mode would 
> port fine between either.
>
>>> There is COGRE which can pull data from the
>>> buffer, reference some databases, and pop up a uml diagram of your code
>>> which is a bit over-the-top.
>>
>> COGRE sounds good, but I imagine it'll need more support than just 
>> dumping the AST.
>>
>> And I can't get it to do anything (here's the documentation for 
>> automatic generation: 
>> http://www.randomsample.de/cedetdocs/cogre/Semantic-Support.html#Semantic-Support, 
>> and trying to interact with the editor manually leads to "eieio-oref: 
>> eieio-oref called on a class: cogre-class", etc).
>>
>> It's also not in Emacs, for some reason.
>
> It was deemed optional when Yidong merged CEDET into Emacs.  You will 
> probably need to use Emacs24 to make it work.  To try it out do this:
>
> 1) Install graphviz (it uses that for the layout engine)
> 2) Start emacs 24
> 3) Use CEDET from it's git repository
> 4) M-x find-library RET cogre RET
> 5) find cogre-element-peer in the code
> 6) M-x cogre-uml-quick-class RET
>
> should get you something to play with.
>
>> (*) Other things that the users ask for is fuzzy completion, showing 
>> completions right away after dot (Robe isn't fast enough for that) 
>> and working over TRAMP inside a Vagrant environment. It doesn't seem 
>> like CEDET integration will help with any of those.
>>
>
> CEDET is a framework that provides an abstraction for connecting 
> different tools that need to talk about hard problems together. The 
> problems it solves are related to project information, abstracting 
> 'tag' information down to something Lisp programs can reason on, and 
> abstracting code generation into a scheme that can allow lisp programs 
> to support multiple languages.  CEDET doesn't have 'fuzzy completion' 
> but it can feed a fuzzy completion engine.  CEDET doesn't do anything 
> special with TRAMP, but someone could use CEDET to bind that workflow 
> into the common workflow. When thinking about CEDET, it isn't about a 
> bullet list of user facing features but about how it can enable 
> someone working on said feature to have their work leveraged to a 
> wider audience.
>
> Eric
>

Re: IDE

[Dmitry Gutov]

On 10/22/2015 03:58 PM, Eric Ludlam wrote:

> Semantic's idle process handling makes sure databases are synchronized
> once, then enables tools to run after, so it is more about the
> scheduling of different tools that use semantic.

Cool. But I guess we could just say I was thinking of 
'semantic-idle-summary-idle-function, and not semantic-idle-summary-mode 
itself.

> In addition, by going through the semantic version, there are a range of
> different formatting options to use that the user can select from.  That
> doesn't require idle-summary-mode, but is a side effect of using
> semantic to extract contextual information.

The formatting options depend on using some rich structure like Semantic 
tags. So it would make sense to port that only after we standardize on 
some tag format universally across Emacs (are Semantic tags optimal? I 
don't know).

> It was deemed optional when Yidong merged CEDET into Emacs.  You will
> probably need to use Emacs24 to make it work.  To try it out do this:

Step 6 is broken for me.

> 1) Install graphviz (it uses that for the layout engine)
> 2) Start emacs 24
> 3) Use CEDET from it's git repository
> 4) M-x find-library RET cogre RET
> 5) find cogre-element-peer in the code
> 6) M-x cogre-uml-quick-class RET

I only get a "Class:" prompt, without a default value or completions. 
Typing "cogre-element-peer" gives me "Could not find class ...", even 
though cogre is obviously loaded. That's in Emacs 24.5.

> When thinking about CEDET, it isn't about a
> bullet list of user facing features but about how it can enable someone
> working on said feature to have their work leveraged to a wider audience.

People working on said features should be encouraged, of course. 
Unfortunately, the two more interesting projects that I've seen utilize 
CEDET are language-specific:

- SRefactor only does anything useful for C/C++.
- Oleh Krehel's function-args even mentions C/C++ in its summary.

Perhaps, if there were more broadly applicable examples, it would lead 
to broader adoption. Maybe we should wonder why prefer making tools for 
CEDET that only target C and C++.

Re: IDE

[Eric Ludlam]

On 10/27/2015 10:16 PM, Dmitry Gutov wrote:
> On 10/22/2015 03:58 PM, Eric Ludlam wrote:
>
>> In addition, by going through the semantic version, there are a range of
>> different formatting options to use that the user can select from.  That
>> doesn't require idle-summary-mode, but is a side effect of using
>> semantic to extract contextual information.
>
> The formatting options depend on using some rich structure like 
> Semantic tags. So it would make sense to port that only after we 
> standardize on some tag format universally across Emacs (are Semantic 
> tags optimal? I don't know).
>

What is optimal?  For me, it was about pulling all the datatype 
information out of the syntax, and differentiating between different tag 
classes, and having something that was Emacs friendly, and extensible to 
any language, no matter how strange.  For me, Semantic's tag structure 
is pretty good.  It has 5 slots:

* The name, as 'car' so you can use assoc and other friendly things.
* The class of the tag is next, as all languages have different kinds of 
tags.
* plist of attributes, extensible to use anything, but with some common 
attribute names defined for use via API.
* plist of properties, or bits of data needed by the tooling, which is 
arbitrary and extensible for whatever a tool might need.
* Buffer binding information - either the [start end] of the tag, or an 
overlay.

That makes the tag extensible for almost any situation, and keeps TAG 
data and OPERATIONAL data separate, saveable, and searchable.

>> It was deemed optional when Yidong merged CEDET into Emacs.  You will
>> probably need to use Emacs24 to make it work.  To try it out do this:
>
> Step 6 is broken for me.
>
>> 1) Install graphviz (it uses that for the layout engine)
>> 2) Start emacs 24
>> 3) Use CEDET from it's git repository
>> 4) M-x find-library RET cogre RET
>> 5) find cogre-element-peer in the code
>> 6) M-x cogre-uml-quick-class RET
>
> I only get a "Class:" prompt, without a default value or completions. 
> Typing "cogre-element-peer" gives me "Could not find class ...", even 
> though cogre is obviously loaded. That's in Emacs 24.5.

Did you enable semantic-mode?  The buffer would need to be parsed for 
the data to be available, and for the prompt to have completion.

>> When thinking about CEDET, it isn't about a
>> bullet list of user facing features but about how it can enable someone
>> working on said feature to have their work leveraged to a wider 
>> audience.
>
> People working on said features should be encouraged, of course. 
> Unfortunately, the two more interesting projects that I've seen 
> utilize CEDET are language-specific:
>
> - SRefactor only does anything useful for C/C++.
> - Oleh Krehel's function-args even mentions C/C++ in its summary.
>
> Perhaps, if there were more broadly applicable examples, it would lead 
> to broader adoption. Maybe we should wonder why prefer making tools 
> for CEDET that only target C and C++.
> .

This is a good point.  I always have thought about language independence 
and extensibility, but most folks are just trying to solve a problem and 
may not be interested in extending to anything else.

I've built many little tools over the years, but perhaps they are 
observed as parts of CEDET core, and not as examples.  The semantic-ia 
functions were supposed to be examples for how to use the 
infrastructure, but users have ended up using them as their interface 
and they became more complex.

Eric

Universal tag structure, was: Re: IDE

[Dmitry Gutov]

On 10/28/2015 01:39 PM, Eric Ludlam wrote:

> What is optimal?

Can't really say.

> For me, it was about pulling all the datatype
> information out of the syntax, and differentiating between different tag
> classes, and having something that was Emacs friendly, and extensible to
> any language, no matter how strange.

Sounds like a good direction.

> For me, Semantic's tag structure
> is pretty good.  It has 5 slots:
>
> * The name, as 'car' so you can use assoc and other friendly things.
> * The class of the tag is next, as all languages have different kinds of
> tags.
> * plist of attributes, extensible to use anything, but with some common
> attribute names defined for use via API.
> * plist of properties, or bits of data needed by the tooling, which is
> arbitrary and extensible for whatever a tool might need.
> * Buffer binding information - either the [start end] of the tag, or an
> overlay.

Sections 3 and 4 are a little vague, for a format description. If they 
contain important data, it should be standardized somehow.

Regarding "buffer binding information", I think that's too 
Semantic-specific. Instead, maybe a tag should be able to "find" itself 
(probably by implementing some generic function). Whether it's via 
keeping a reference to an overlay, consulting some index, or external 
process, or simply running Grep again.

> That makes the tag extensible for almost any situation, and keeps TAG
> data and OPERATIONAL data separate, saveable, and searchable.

Overlays are a part of the "operational" data, IMO.

I don't intend working on this in the near future, though, so I'd like 
to bow out of the detailed discussion of the format, for now.

Re: IDE

[Dmitry Gutov]

On 10/28/2015 01:39 PM, Eric Ludlam wrote:

> Did you enable semantic-mode?  The buffer would need to be parsed for
> the data to be available, and for the prompt to have completion.

Yes, that was the problem, thanks. It works, and looks kinda nice.

> This is a good point.  I always have thought about language independence
> and extensibility, but most folks are just trying to solve a problem and
> may not be interested in extending to anything else.

One way to encourage wider applicability is to offer smaller, maybe 
less-functional, but more generic APIs.

> I've built many little tools over the years, but perhaps they are
> observed as parts of CEDET core, and not as examples.  The semantic-ia
> functions were supposed to be examples for how to use the
> infrastructure, but users have ended up using them as their interface
> and they became more complex.

Maybe it would be better if they were more polished and 
fully-functional, instead.

Re: IDE

[joakim]

Eric Ludlam <ericludlam@gmail.com> writes:

> On 10/27/2015 10:16 PM, Dmitry Gutov wrote:
>> On 10/22/2015 03:58 PM, Eric Ludlam wrote:
>>
>>> In addition, by going through the semantic version, there are a range of
>>> different formatting options to use that the user can select from.  That
>>> doesn't require idle-summary-mode, but is a side effect of using
>>> semantic to extract contextual information.
>>
>> The formatting options depend on using some rich structure like
>> Semantic tags. So it would make sense to port that only after we
>> standardize on some tag format universally across Emacs (are
>> Semantic tags optimal? I don't know).
>>
>
> What is optimal?  For me, it was about pulling all the datatype
> information out of the syntax, and differentiating between different
> tag classes, and having something that was Emacs friendly, and
> extensible to any language, no matter how strange.  For me, Semantic's
> tag structure is pretty good.  It has 5 slots:
>
> * The name, as 'car' so you can use assoc and other friendly things.
> * The class of the tag is next, as all languages have different kinds
> of tags.
> * plist of attributes, extensible to use anything, but with some
> common attribute names defined for use via API.
> * plist of properties, or bits of data needed by the tooling, which is
> arbitrary and extensible for whatever a tool might need.
> * Buffer binding information - either the [start end] of the tag, or
> an overlay.
>
> That makes the tag extensible for almost any situation, and keeps TAG
> data and OPERATIONAL data separate, saveable, and searchable.
>
>>> It was deemed optional when Yidong merged CEDET into Emacs.  You will
>>> probably need to use Emacs24 to make it work.  To try it out do this:
>>
>> Step 6 is broken for me.
>>
>>> 1) Install graphviz (it uses that for the layout engine)
>>> 2) Start emacs 24
>>> 3) Use CEDET from it's git repository
>>> 4) M-x find-library RET cogre RET
>>> 5) find cogre-element-peer in the code
>>> 6) M-x cogre-uml-quick-class RET
>>
>> I only get a "Class:" prompt, without a default value or
>> completions. Typing "cogre-element-peer" gives me "Could not find
>> class ...", even though cogre is obviously loaded. That's in Emacs
>> 24.5.
>
> Did you enable semantic-mode?  The buffer would need to be parsed for
> the data to be available, and for the prompt to have completion.
>
>>> When thinking about CEDET, it isn't about a
>>> bullet list of user facing features but about how it can enable someone
>>> working on said feature to have their work leveraged to a wider
>>> audience.
>>
>> People working on said features should be encouraged, of
>> course. Unfortunately, the two more interesting projects that I've
>> seen utilize CEDET are language-specific:
>>
>> - SRefactor only does anything useful for C/C++.
>> - Oleh Krehel's function-args even mentions C/C++ in its summary.

I'm not sure this is related, but I've done many little project specific
tools using parts of Cedet. For instance SRecode, and Cogre, and
Semantic. Since the things I made were tools for one-off tasks, I didn't
bother to publish them. I think a lot of use-cases are like this, and a
lot of people use various parts of Cedet like this.

Now you will say that this is a thread about making an IDE, but I still
think this use-case deserves mentioning. 

>>
>> Perhaps, if there were more broadly applicable examples, it would
>> lead to broader adoption. Maybe we should wonder why prefer making
>> tools for CEDET that only target C and C++.
>> .
>
> This is a good point.  I always have thought about language
> independence and extensibility, but most folks are just trying to
> solve a problem and may not be interested in extending to anything
> else.
>
> I've built many little tools over the years, but perhaps they are
> observed as parts of CEDET core, and not as examples.  The semantic-ia
> functions were supposed to be examples for how to use the
> infrastructure, but users have ended up using them as their interface
> and they became more complex.
>
> Eric
>
>

-- 
Joakim Verona

Re: IDE

[Oleh Krehel]

Dmitry Gutov <dgutov@yandex.ru> writes:

> - Oleh Krehel's function-args even mentions C/C++ in its summary.
>
> Perhaps, if there were more broadly applicable examples, it would lead
> to broader adoption. Maybe we should wonder why prefer making tools
> for CEDET that only target C and C++.

I targeted C++ because that's a language that I use a lot and I needed
support for.

Dynamic languages like JavaScript/Ruby/Python/Elisp/CL/Clojure/Scheme
don't need to rely on Semantic for tags info, they can just get it from
the REPL. It's still a choice, and both things can work and cooperate.

However, for static languages like C++ Semantic is the only choice for
getting the tag metadata. Which other popular language is in the static
camp? Only Java, the rest I label as hipster, no offense.  I guess some
good progress could be made by extending Semantic for Java, however it's
hard to find Emacs people who are enthusiastic about that language. And
the amount of work to be done would have to be enormous to trump the
popular Java IDE.

    Oleh

Re: IDE

[Dmitry Gutov]

On 10/29/2015 01:12 PM, Oleh Krehel wrote:

> I targeted C++ because that's a language that I use a lot and I needed
> support for.

Makes sense. Why make it C++-specific, though?

> Dynamic languages like JavaScript/Ruby/Python/Elisp/CL/Clojure/Scheme
> don't need to rely on Semantic for tags info, they can just get it from
> the REPL. It's still a choice, and both things can work and cooperate.

C++ can get it from rtags or irony-mode. And "just get it from the REPL" 
is a big simplification.

> However, for static languages like C++ Semantic is the only choice for
> getting the tag metadata. Which other popular language is in the static
> camp? Only Java, the rest I label as hipster, no offense.

a) Why is the dynamic/static distinction important for function-args? 
Ruby and Python, for instance, have function signatures that don't look 
too different from C++/Java ones.

b) There's Scala, and the fairly popular ENSIME. They are working on 
Java support, by the way: https://github.com/ensime/ensime-server/issues/345

Re: IDE

[Oleh Krehel]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 10/29/2015 01:12 PM, Oleh Krehel wrote:
>
>> I targeted C++ because that's a language that I use a lot and I needed
>> support for.
>
> Makes sense. Why make it C++-specific, though?

It works for C as well. I was using a lot of C++ templates at that time.
And Semantic didn't display them nicely.

>> Dynamic languages like JavaScript/Ruby/Python/Elisp/CL/Clojure/Scheme
>> don't need to rely on Semantic for tags info, they can just get it from
>> the REPL. It's still a choice, and both things can work and cooperate.
>
> C++ can get it from rtags or irony-mode. And "just get it from the
> REPL" is a big simplification.

Well, I know that the mentioned lisps can really get this info from the
REPL. Of course, the restriction is that the code needs to be loaded.
The same applies to Jedi and mozrepl/tern, I guess. Not sure if there's
any good similar tooling for Ruby.

>> However, for static languages like C++ Semantic is the only choice for
>> getting the tag metadata. Which other popular language is in the static
>> camp? Only Java, the rest I label as hipster, no offense.
>
> a) Why is the dynamic/static distinction important for function-args?
> Ruby and Python, for instance, have function signatures that don't
> look too different from C++/Java ones.

The thing is that function-args adds some functionality that would be
missing otherwise for C++.  Ruby and Python already have some of this
functionality by extracting it from the REPL. Of course, it would be
nice to make it work everywhere, but it's not urgent if the gap is
already filled by something else, e.g. Jedi.

> b) There's Scala, and the fairly popular ENSIME. They are working on
> Java support, by the way:
> https://github.com/ensime/ensime-server/issues/345

That's nice. But somehow I don't see why anyone would not just use
Clojure if you need a JVM hosted language.

Re: IDE

[Dmitry Gutov]

On 10/29/2015 01:37 PM, Oleh Krehel wrote:

> The thing is that function-args adds some functionality that would be
> missing otherwise for C++.

I'm not exactly sure what you're referring to, but shouldn't that go 
straight into CEDET?

> That's nice. But somehow I don't see why anyone would not just use
> Clojure if you need a JVM hosted language.

If only everyone worked on greenfield projects, liked Lisp and had 
freedom to choose it at the workplace.

Re: IDE

[Oleh Krehel]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 10/29/2015 01:37 PM, Oleh Krehel wrote:
>
>> The thing is that function-args adds some functionality that would be
>> missing otherwise for C++.
>
> I'm not exactly sure what you're referring to, but shouldn't that go
> straight into CEDET?

I'm referring to the display of complex C++ types, some inheritance of
members from one namespace into another, some heuristics for
symbol-at-point completion, the ability to jump-to-tag in a whole
directory instead of a single file etc.  These are mostly minor
improvements here and there. Since I make these small improvements often
and don't have write access to CEDET, I keep them in a separate package.

>> That's nice. But somehow I don't see why anyone would not just use
>> Clojure if you need a JVM hosted language.
>
> If only everyone worked on greenfield projects, liked Lisp and had
> freedom to choose it at the workplace.

Oh well, can't have it all I guess:)

Re: IDE

[Dmitry Gutov]

On 10/29/2015 02:56 PM, Oleh Krehel wrote:

> I'm referring to the display of complex C++ types, some inheritance of
> members from one namespace into another, some heuristics for
> symbol-at-point completion, the ability to jump-to-tag in a whole
> directory instead of a single file etc.  These are mostly minor
> improvements here and there. Since I make these small improvements often
> and don't have write access to CEDET

There never was a prohibition to committing changes to CEDET in the 
Emacs repository, and now according to the recent discussions on the 
CEDET mailing list, all CEDET development is going to move to Emacs.

So, commit away?

Re: IDE

[Eric Ludlam]

On 10/29/2015 09:13 AM, Dmitry Gutov wrote:
> On 10/29/2015 02:56 PM, Oleh Krehel wrote:
>
>> I'm referring to the display of complex C++ types, some inheritance of
>> members from one namespace into another, some heuristics for
>> symbol-at-point completion, the ability to jump-to-tag in a whole
>> directory instead of a single file etc.  These are mostly minor
>> improvements here and there. Since I make these small improvements often
>> and don't have write access to CEDET
>
> There never was a prohibition to committing changes to CEDET in the 
> Emacs repository, and now according to the recent discussions on the 
> CEDET mailing list, all CEDET development is going to move to Emacs.
>
> So, commit away?
>
I read through the function-args readme briefly today, and it looks like 
some nice stuff.

I can see why some of the pieces would be c++ specific.

The usability improvements in this tool are really nice.  If you have an 
assignment w/ the FSF, I would have been happy to provide write access 
and have help getting patches into CEDET, or add features.   Your 
function args package notes that it has improved versions of CEDET 
functions.  I rarely had time to focus on usability of keybindings and 
such, as I usually have my hands full keeping parsers and other 
infrastructure up to date.   If someone has interests in interfaces and 
what kind of data is useful while editing, I'll always be glad to pull 
in those changes, and help keep things language neutral where applicable.

Eric

Re: IDE

[Oleh Krehel]

Hi Eric,

Eric Ludlam <ericludlam@gmail.com> writes:

> I read through the function-args readme briefly today, and it looks
> like some nice stuff.
>
> I can see why some of the pieces would be c++ specific.
>
> The usability improvements in this tool are really nice.  If you have
> an assignment w/ the FSF, I would have been happy to provide write
> access and have help getting patches into CEDET, or add features.
> Your function args package notes that it has improved versions of
> CEDET functions.  I rarely had time to focus on usability of
> keybindings and such, as I usually have my hands full keeping parsers
> and other infrastructure up to date.   If someone has interests in
> interfaces and what kind of data is useful while editing, I'll always
> be glad to pull in those changes, and help keep things language
> neutral where applicable.

I have an assignment and commit access to Emacs.  Should I wait until
CEDET gets a git branch in Emacs until committing?
Or just commit to the CEDET that's already in Emacs core and hope that
it will magically be pulled to the base repository?
I could also commit directly to CEDET if I had write access, but I think
CEDET doesn't use git, and that's the only thing I (know how to) use.

    Oleh

Re: IDE

[Xue Fuqiao]

On Fri, Oct 30, 2015 at 5:04 PM, Oleh Krehel <ohwoeowho@gmail.com> wrote:
> but I think CEDET doesn't use git, and that's the only thing I (know
> how to) use.

CEDET uses Git now.  You can browse the repository here:
http://sourceforge.net/p/cedet/git/ci/master/tree/

Related information:

* http://sourceforge.net/p/cedet/mailman/message/33106122/
* http://sourceforge.net/p/cedet/mailman/message/33188785/
* http://sourceforge.net/p/cedet/mailman/message/33209931/

Re: IDE

[Oleh Krehel]

Xue Fuqiao <xfq.free@gmail.com> writes:

> On Fri, Oct 30, 2015 at 5:04 PM, Oleh Krehel <ohwoeowho@gmail.com> wrote:
>> but I think CEDET doesn't use git, and that's the only thing I (know
>> how to) use.
>
> CEDET uses Git now.  You can browse the repository here:
> http://sourceforge.net/p/cedet/git/ci/master/tree/

Thanks, Xue. I was using http://git.code.sf.net/p/cedet/git for a long
time now. I thought it was only a Git mirror though.

Eric, could I possibly get write access there? I promise not commit
anything too strange without asking first.

Re: IDE

[Eric Ludlam]

On 10/31/2015 07:40 AM, Oleh Krehel wrote:
> Xue Fuqiao <xfq.free@gmail.com> writes:
>
>> On Fri, Oct 30, 2015 at 5:04 PM, Oleh Krehel <ohwoeowho@gmail.com> wrote:
>>> but I think CEDET doesn't use git, and that's the only thing I (know
>>> how to) use.
>> CEDET uses Git now.  You can browse the repository here:
>> http://sourceforge.net/p/cedet/git/ci/master/tree/
> Thanks, Xue. I was using http://git.code.sf.net/p/cedet/git for a long
> time now. I thought it was only a Git mirror though.
>
> Eric, could I possibly get write access there? I promise not commit
> anything too strange without asking first.
> .
>
Source Forge wouldn't let me edit user permissions this morning.  :(

If you email me your SF id, I can add you when SF next allows me to.

I think you might be better off checking directly into Emacs however.  
We need to do a final merge from the CEDET repository into Emacs proper, 
after which we will probably abandon the separate repository.  Keeping 
them in sync has been a problem, so we will probably come up with an 
alternate branching strategy.

For function-args proper, lets consider how to bring it into the CEDET 
space.  For example, there is this line in the commentary:

;; * `moo-complete' -- a c++-specific version of 
`semantic-ia-complete-symbol'. Perhaps we can figure out how to either 
merge the two, or add the right way to extend -ia-complete-jump in that 
way for any language.

Thanks!
Eric

Re: IDE

[Oleh Krehel]

Eric Ludlam <ericludlam@gmail.com> writes:

> On 10/31/2015 07:40 AM, Oleh Krehel wrote:
>> Eric, could I possibly get write access there? I promise not commit
>> anything too strange without asking first.

> If you email me your SF id, I can add you when SF next allows me to.

I actually don't have a Source Forge account. Does Source Forge allow to
simply upload id_rsa.pub to allow access, like Github or Savannah?

> I think you might be better off checking directly into Emacs however.
> We need to do a final merge from the CEDET repository into Emacs
> proper, after which we will probably abandon the separate repository.
> Keeping them in sync has been a problem, so we will probably come up
> with an alternate branching strategy.

This is also fine with me. Only depends on how long until the final
merge is done.

> For function-args proper, lets consider how to bring it into the CEDET
> space.  For example, there is this line in the commentary:
>
> ;; * `moo-complete' -- a c++-specific version of
> `semantic-ia-complete-symbol'. Perhaps we can figure out how to either
> merge the two, or add the right way to extend -ia-complete-jump in
> that way for any language.

It's very C++ specific (namespaces, smart pointers, templates etc) to
generalize. I think a dispatch can be made on `major-mode' that forwards
`semantic-ia-complete-symbol' to `moo-complete' for c++-mode (and maybe
c-mode).

It may be possible to simplify `moo-complete' a lot depending on how
much better we can make the parser in terms of understanding C++.  A lot
of the code in `moo-complete' are "dumb" heuristics to come up with at
least something when `semantic-ia-complete-symbol' fails.

However, e.g. semantic-directory.el could be generalized to any
language. It may overlap a bit with the built-in CEDET functionality,
but I just couldn't find a good function that returns a joined list of
tags for a list of files, with all tags up-to-date, even if one of the
files was modified very recently.

Also the function arguments overlays could be generalized, making a good
alternative to Eldoc for all languages supported by semantic.

Re: IDE

[Evgeniy Dushistov]

On Wed, Oct 21, 2015 at 06:52:55PM -0400, Eric Ludlam wrote:
> 
> Semantic doesn't demand it's parsers be in wisent, or even in Emacs at all.
> If you have a nice Ruby grammar in Ruby, and you can convert it's internal
> data into lisp-like syntax, pulling it into the Semantic system is pretty
> easy.  What you would loose is dynamic reparsing without having to save,
> thus it may be a bit quirky.
> 

Is any documentation about this:
- format of data that Semantic understand
- which hooks should be used to give this information to semantic,
so it can use it to colorization, completition, tags navigation etc
?


For example, if I write such code
#include <string>
int main() { std::string str; }

and call semantic-ia-fast-jump on "std::string" ede/cedet/semantic
tell me "Could not find `string'. Jump to std?"
It works in more simple case for std::vector,
but fails with std::string.

But if I ask emacs via rtags-find-symbol, all works fine and
I see
  typedef basic_string<char>    string;   

So I think it would be good integrate
rtags(https://github.com/Andersbakken/rtags) (or some another existing
daemon) for such complex language as c++ and cedet.

But I can not find any clear description for external parsers usage?

-- 
/Evgeniy

Re: IDE

[David Engster]

Evgeniy Dushistov writes:
> On Wed, Oct 21, 2015 at 06:52:55PM -0400, Eric Ludlam wrote:
>> 
>> Semantic doesn't demand it's parsers be in wisent, or even in Emacs at all.
>> If you have a nice Ruby grammar in Ruby, and you can convert it's internal
>> data into lisp-like syntax, pulling it into the Semantic system is pretty
>> easy.  What you would loose is dynamic reparsing without having to save,
>> thus it may be a bit quirky.
>> 
>
> Is any documentation about this:
> - format of data that Semantic understand

See chapter 1 in the Semantic AppDev manual:

  http://www.randomsample.de/cedetdocs/semantic-appdev/

Running M-x bovinate in a C file will already give you a good impression.

> - which hooks should be used to give this information to semantic, so
> it can use it to colorization, completition, tags navigation etc ?

Hooks are not well suited for this task, since you want to extend or
override functionality depending on the current major-mode. For this,
the mode-local package is used, so one should get familiar with it
first. The most important functions to override are listed in the
Semantic AppDev manual (see for instance chapters 7 and 12). It is also
good to look into a Semantic backend for a language which you know
reasonably well to see how it is done. Last not least, you can ask on
cedet-devel.

> So I think it would be good integrate
> rtags(https://github.com/Andersbakken/rtags) (or some another existing
> daemon) for such complex language as c++ and cedet.

AFAIK, rtags solely uses libclang, which does not give you direct access
to the AST, so I wouldn't know how rtags could be used to export the
semantic tags of the current buffer. You could hook rtags into Semantic
purely for the end-user features, like completion/references.

This is also why I will use libtooling for C++, which lets you walk the
AST.

-David

Re: IDE

[Evgeniy Dushistov]

On Fri, Oct 23, 2015 at 04:55:06PM +0200, David Engster wrote:
> Evgeniy Dushistov writes:
> > On Wed, Oct 21, 2015 at 06:52:55PM -0400, Eric Ludlam wrote:
> >> 
> >> Semantic doesn't demand it's parsers be in wisent, or even in Emacs at all.
> >> If you have a nice Ruby grammar in Ruby, and you can convert it's internal
> >> data into lisp-like syntax, pulling it into the Semantic system is pretty
> >> easy.  What you would loose is dynamic reparsing without having to save,
> >> thus it may be a bit quirky.
> >> 
> >
> > Is any documentation about this:
> > - format of data that Semantic understand
> 
> See chapter 1 in the Semantic AppDev manual:
> 
>   http://www.randomsample.de/cedetdocs/semantic-appdev/
> 

Thanks, I used 'info semantic' and have no idea that there is also
semantic-appdev


> > So I think it would be good integrate
> > rtags(https://github.com/Andersbakken/rtags) (or some another existing
> > daemon) for such complex language as c++ and cedet.
> 
> AFAIK, rtags solely uses libclang, which does not give you direct access
> to the AST, so I wouldn't know how rtags could be used to export the
> semantic tags of the current buffer. You could hook rtags into Semantic
> purely for the end-user features, like completion/references.
> 
> This is also why I will use libtooling for C++, which lets you walk the
> AST.

Why do you think that libclang not give access to AST?
I used it from python like this:

def callexpr_visitor(node, parent, parser):
# your code here
    for c in node.get_children():
        callexpr_visitor(c, node, parser)
    return 2 # means continue visiting recursively

tu = TranslationUnit.from_source(fname, args)
callexpr_visitor(tu.cursor, None, parser)

and callexpr_visitor  visit each node in AST.
Because of clang's python API just part of C/C++ API,
then it should be possible in rtags to have access to AST.


-- 
/Evgeniy

Re: IDE

[David Engster]

Evgeniy Dushistov writes:
> Because of clang's python API just part of C/C++ API,
> then it should be possible in rtags to have access to AST.

You are right, the stuff that is available through libclang could be
exported as Semantic tags. But last I checked lots of stuff was not
accessible through libclang as it was only exported as so called
"unexposed statements".

-David

------------------------------------------------------------------------------

Re: IDE

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> I'm quite sure CEDET has collected and expressed in code a lot of experience
> and solutions to many problems that arise in the context of building an IDE.
> It's OK to discard that, if we sure that's the proverbial 1st variant
> everyone throws away, but we need first to be sure we know what we are
> discarding.

I'm not suggesting we discard experiences. What I'm saying is: it doesn't make
sense to proceed by looking at CEDET, and then asking what should be changed.

CEDET is like a hammer. When it was made, the problem looked like nail.

Today, the problem might be a screw (is it? do we know?). We're not going to
arrive at the best answer by asking ourselves how a hammer can be changed to
meet the needs of a screw. It deserves looking at the problem anew.

It doesn't mean we throw out the hammer. Maybe we do have a nail, maybe we
don't. The point is: If we make technical assumptions before learning what we
want to end up with, we're going to arrive at something shaped more by those
assumptions than by our needs.

So unless there are other features I should bear in mind, I'm going to turn my
attention away from CEDET now and back to the IDE vision I'd like everyone's
help with, once there is more to say.

John

Re: IDE

[Eli Zaretskii]

> From: John Wiegley <johnw@newartisans.com>
> Date: Sat, 17 Oct 2015 22:23:30 -0700
> 
> >>>>> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I'm quite sure CEDET has collected and expressed in code a lot of experience
> > and solutions to many problems that arise in the context of building an IDE.
> > It's OK to discard that, if we sure that's the proverbial 1st variant
> > everyone throws away, but we need first to be sure we know what we are
> > discarding.
> 
> I'm not suggesting we discard experiences. What I'm saying is: it doesn't make
> sense to proceed by looking at CEDET, and then asking what should be changed.
> 
> CEDET is like a hammer. When it was made, the problem looked like nail.
> 
> Today, the problem might be a screw (is it? do we know?). We're not going to
> arrive at the best answer by asking ourselves how a hammer can be changed to
> meet the needs of a screw. It deserves looking at the problem anew.
> 
> It doesn't mean we throw out the hammer. Maybe we do have a nail, maybe we
> don't. The point is: If we make technical assumptions before learning what we
> want to end up with, we're going to arrive at something shaped more by those
> assumptions than by our needs.

I wasn't aware that the IDE landscape might have changed in such a
significant way recently.  This discussion seems to focus on details,
which seems to indicate no such radical changes happened.  But I'm not
an expert, so maybe you are right.

I did suggest up-thread to come up with a list of main features we
think an Emacs IDE should have.  If that is what you have in mind, I
obviously agree.

In any case, CEDET is not an EDE, AFAIK.  It is an infrastructure and
a set of tools for building an IDE.  IOW, it's neither a hammer nor a
screwdriver, but something that allows us to make one or the other (or
something else entirely).  So it could very well be a good basis for
an Emacs IDE.

Re: IDE

[John Wiegley]

>>>>> Eli Zaretskii <eliz@gnu.org> writes:

> I wasn't aware that the IDE landscape might have changed in such a
> significant way recently. This discussion seems to focus on details, which
> seems to indicate no such radical changes happened. But I'm not an expert,
> so maybe you are right.
> 
> I did suggest up-thread to come up with a list of main features we think an
> Emacs IDE should have. If that is what you have in mind, I obviously agree.

That's great, Eli. I don't know if the landscape has really changed or not,
only that I'd love to take a step back together and survey the field: if for
no other reason than to help us feel we've reached our conclusions on the same
footing. Who knows, we may end up where we are now; except that given the
current level of disgruntlement, I'd be surprised by that outcome as well.

My list of main features from a previous message (and this is just a draft,
subject to change) is:

  indentation (see above)
  reformatting
  syntax highlighting (font-lock)
  help at point
  documentation lookup (sadly, fewer projects use Info these days)
  completion
  refactoring
  semantic editing (for example, paredit)
  compilation (M-x compile)
  live compilation (flymake/flycheck)
  REPL (comint)
  running tests
  debugging (GUD)
  version control (CV)
  profiling
  code coverage
  app interaction

I'll refine this shortly and come back with a better list, and then we can
start new threads for each sub-area, and discover what expertise we already
have in those areas within the community.

BTW- I used to work at Borland on their C++Builder IDE, and I've worked full
time on Java J2EE projects using IntelliJ, so that is the basis for some of my
opinions about IDEs.

> In any case, CEDET is not an EDE, AFAIK. It is an infrastructure and a set
> of tools for building an IDE. IOW, it's neither a hammer nor a screwdriver,
> but something that allows us to make one or the other (or something else
> entirely). So it could very well be a good basis for an Emacs IDE.

It could be! I'm pretty sure it will come up in discussions again shortly,
with some valuable experiences to add, and perhaps even code.

John

Re: IDE

[Andreas Röhler]

  On 18.10.2015 07:23, John Wiegley wrote:
>>>>>> Eli Zaretskii<eliz@gnu.org>  writes:
>> I'm quite sure CEDET has collected and expressed in code a lot of experience
>> and solutions to many problems that arise in the context of building an IDE.
>> It's OK to discard that, if we sure that's the proverbial 1st variant
>> everyone throws away, but we need first to be sure we know what we are
>> discarding.
> I'm not suggesting we discard experiences. What I'm saying is: it doesn't make
> sense to proceed by looking at CEDET, and then asking what should be changed.
>
> CEDET is like a hammer. When it was made, the problem looked like nail.
>
> Today, the problem might be a screw (is it? do we know?). We're not going to
> arrive at the best answer by asking ourselves how a hammer can be changed to
> meet the needs of a screw. It deserves looking at the problem anew.
>
> It doesn't mean we throw out the hammer. Maybe we do have a nail, maybe we
> don't. The point is: If we make technical assumptions before learning what we
> want to end up with, we're going to arrive at something shaped more by those
> assumptions than by our needs.
>
> So unless there are other features I should bear in mind, I'm going to turn my
> attention away from CEDET now and back to the IDE vision I'd like everyone's
> help with, once there is more to say.
>
> John
>

"hammer" seems a suitable abstraction for the moment :)

IMO the difficulty is twofold:

- CEDET tries to achieve things, which require detailed knowledge of the 
goal-languages - hardly to expect WRT several hundreds in use meanwhile.

- it's not written in plain Emacs Lisp, thus extending CEDET requires 
another special knowledge, which makes shrink the circle of would-be 
developers again.

Re: IDE

[Eric Ludlam]

On 10/13/2015 09:02 AM, Lluís wrote:
>> Let's not reiterate past discussions: you forget CEDET.
> Just thinking out loud: it seems to me that many people forget that CEDET is,
> from my understanding, a framework for writing tools first, and a set of such
> example tools later.

Thanks for the reminder.  I forgot to bring that part up in this 
iteration of the conversation.

>> >And if anyone_really_  cares about supporting C/C++, they should be
>> >working with and on GCC's libcc1, which is available for quite some
>> >time already.
> If this is the libgcc1 you mean [1], I'm not sure it's suitable for code
> completion. Instead, GCC should be modified to hook into the frontend parser or
> the generic AST and then parsing that, which is no small feat. Fortunately,
> hooking is already possible using GCC plugins [2].
>
> [1]http://gcc.gnu.org/onlinedocs/gccint/Libgcc.html
> [2]http://www.codesynthesis.com/~boris/blog/2010/05/03/parsing-cxx-with-gcc-plugin-part-1/
>
> With this, it's a relatively easy (but time-consuming) task to build an external
> tool that parses files on-demand. The ideal would be some kind of persistent
> daemon+database, as was discussed in the CEDET list quite some time ago, but
> that's an entirely different story.

Yes, I remember those discussions. I was able to put in examples of an 
external parser by integrating exuberent ctags which is a bit weak for 
CEDET needs, but was never able to get any help figuring out how to make 
gcc or any other compiler provide the data I wanted.  I tried some stuff 
with using debug symbols and gdb, but that was a flop.  The ectags 
example showed marked performance improvement by delegating parsing 
outside the Emacs process.

A second area was an external database for looking up symbols.  I pulled 
in GNU Global and a couple others as proof of concept examples, but 
never got around to anything custom, as that was a bit bigger task.  The 
lack of an external parser to use with it made the project more daunting.

Eric

RE: IDE

[Drew Adams]

> > Where "definition" can be whatever you want, AFAIK.
> 
> "Definition" in this context means the implementation.  There's only
> one implementation, but there might be many references
> (a.k.a. "calls").

That just assumes that you index only the "implementation".

In principle, a TAGS file could be created (including on the
fly) from not only "the implementation" files but also the
"calls" files.

> > A TAGS file is just an index.  You can index whatever you like,
> > AFAIK.
> 
> An index where the key is the symbol itself can only hold one instance
> of every symbol.

Is a TAGS file limited to symbols?  I didn't think so.

And I definitely have TAGS files that have multiple entries
for the same symbol definition.  The definitions are from
different source files, but they are in the same TAGS file
(in different sections, separated by form-feed chars).

For example:

^L
frame-cmds-OLD.el,1980
(defun iconify-everything ()\x7ficonify-everything\x01298,11152
...
^L
frame-cmds.el,1890
(defun iconify-everything ()\x7ficonify-everything\x01141,5218

Re: IDE

[Eli Zaretskii]

> Date: Sat, 24 Oct 2015 10:37:47 -0700 (PDT)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: nix@esperi.org.uk, dgutov@yandex.ru, esperanto@cumego.com,
>         adatgyujto@gmail.com, emacs-devel@gnu.org
> 
> > > Where "definition" can be whatever you want, AFAIK.
> > 
> > "Definition" in this context means the implementation.  There's only
> > one implementation, but there might be many references
> > (a.k.a. "calls").
> 
> That just assumes that you index only the "implementation".

No, it's a definition of "definition" in this context.

> Is a TAGS file limited to symbols?  I didn't think so.

The "tags" are symbols, yes.

> And I definitely have TAGS files that have multiple entries
> for the same symbol definition.  The definitions are from
> different source files, but they are in the same TAGS file
> (in different sections, separated by form-feed chars).
> 
> For example:
> 
> ^L
> frame-cmds-OLD.el,1980
> (defun iconify-everything ()\x7ficonify-everything\x01298,11152
> ...
> ^L
> frame-cmds.el,1890
> (defun iconify-everything ()\x7ficonify-everything\x01141,5218

These are two different symbols, because the file name is (implicitly)
part of it.  There can be at most one definition per file, but many
references.

Anyway, what's this thread about, exactly?

Re: IDE

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Tom <adatgyujto@gmail.com>
>> Date: Sat, 10 Oct 2015 04:33:59 +0000 (UTC)
>> 
>> > If it falls short compared with other IDEs, I think this would be a
>> > great area for improvement of Emacs.
>> 
>> The number one requirement by IDE users today is perfect code completion
>> and powerful refactoring support for the language they develop in
>> (Java, C++, etc.).
>> 
>> Any IDE which wants to compete with the popular IDEs must have these,
>> because users find these features much more helpful in development,
>> than keyboard macro support and such.
>> 
>> So if Emacs wants to compete with these tools then it has to have seamless,
>> context aware code completion and refactoring support, and GNU tools
>> has to provide Emacs the necessary information to implement these
>> features.
>
> I agree.  But to have that, the only way is to have motivated
> volunteers step forward and work on these features.  Otherwise we will
> never have them.
>
> Right now, no one is working on that, though everyone is talking.  the
> same as with weather.

Not quite.  IIRC, company-mode integrated (integrates?) the parsing
facilities of Clang with Emacs.  This contribution was rejected (though
I don't have an overview over the actual execution of the rejection)
because of promoting non-GCC compilers.  However, attempts of letting
GCC similarly output AST data were rejected as making it too easy to
support non-free applications (obviously, if Emacs can use GCC for
purposes of syntax analysis from the command line, so can anybody else).
Using the recently admitted GCC plugin interface (previously, plugins
were rejected because they would make supporting non-free applications
too easy) for outputting a full AST was rejected since it would...

You get the drift.  A number of would-be contributors, partly with
solutions or the impetus and skills for creating them, finally went
elsewhere in disgust rather than trying to figure out the maze of which
interoperations between GNU applications were acceptable and which not.

So "no one is working on that, though everyone is talking" is sort of an
unfair characterization because it implies that no one was willing to
put his money and time where his mouth is.

The main problem here is that the default behavior of GNU code is to
refrain from serious interoperability since the GPL (as well as pretty
much anything relying on copyright alone) does not reach across
interoperable interfaces, until such a time that a concrete,
strategically important application requires such interoperability, and
then ways are searched to restrict the interoperability to the
particular combination/application.

So people's hands are bound until a complete plan has been worked out
and/or blessed by Richard and shouting "no one is working on that" is
disingenuous.

I don't think this kind of micromanagement of interoperation can scale.
Lots of worthwhile things come into being by plugging together existing
components in surprising ways and that is an essential component of
academic work and also of software freedom.  Copping out whenever it
gets serious is not going to help free software.

At the same time, the FSF does not turn on a dime.  Which is one of its
strengths.  The future of Emacs is one of the things which has an
impact, and so is the future of GCC.  The GPLv3 is a large "poison pill"
that actually uses copyright in order to disarm the use of patents as a
weapon.  In my opinion, we should rely on its leverage for keeping the
worst of the proprietary misuses in check and give mostly free rein on
interoperability with regard to technical measures.  It's been good
enough for making Apple go away and leave GCC alone for building its
future developer prisons.

For that reason I am glad that John has volunteered to take a look Emacs
maintainership and that he brings the motivation and skills to work on
Emacs being a good IDE and hopefully the motivation to work out with GCC
and RMS about what would be required and how to get there.

Because we've made use of the "it's all talk" excuse far too long for
postponing tackling the increasing issue of interoperability as a design
criterion for Free Software under the GNU umbrella of the FSF.

It's not "all talk" with John on board and I'm glad that he is willing
to talk his application through with Richard.  I severely doubt that it
will be the only time he'll need to talk over things with Richard.

And they are coming at Emacs from different sides and views that will
never be the same and don't need to be.  But the current incompatibility
seems gratuitously bad to me and I think progress on that is quite
necessary.

-- 
David Kastrup

Re: IDE

[Dmitry Gutov]

On 10/10/2015 12:00 PM, David Kastrup wrote:

> Not quite.  IIRC, company-mode integrated (integrates?) the parsing
> facilities of Clang with Emacs.  This contribution was rejected (though
> I don't have an overview over the actual execution of the rejection)
> because of promoting non-GCC compilers.

company-clang is still in GNU ELPA (but shh). It could be considered a 
"toy" completion backend, though, because it only uses the clang 
executable, not libclang, so it doesn't, for instance, do any caching of 
the parsing results, which is necessary for speedy completion in real 
C++ projects.

If there's a program in GCC suite with similar features to 'clang 
-code-completion-at', I'd be happy to integrate it likewise.

Re: IDE

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Cc: Tom <adatgyujto@gmail.com>,  emacs-devel@gnu.org
> Date: Sat, 10 Oct 2015 11:00:46 +0200
> 
> attempts of letting GCC similarly output AST data were rejected as
> making it too easy to support non-free applications (obviously, if
> Emacs can use GCC for purposes of syntax analysis from the command
> line, so can anybody else).

That's not how that discussion ended.  It ended by Richard saying he
wanted to study the issue in more depth, before making his decision.

In any case, libcc1 is a fait accompli, for several months at least,
and still no one (AFAIK) investigated whether it can serve as basis
for any of the IDE-related features.

> You get the drift.  A number of would-be contributors, partly with
> solutions or the impetus and skills for creating them, finally went
> elsewhere in disgust rather than trying to figure out the maze of which
> interoperations between GNU applications were acceptable and which not.

Yes, that's the "hurt feelings" part of my previous message.

I agree that you need to have some serious will power, perseverance,
and sometimes just stubbornness to get stuff like that done.  I still
hope motivated individuals with enough of that will emerge.

> So "no one is working on that, though everyone is talking" is sort of an
> unfair characterization because it implies that no one was willing to
> put his money and time where his mouth is.

Well, they are not willing badly enough.

Sorry for being blunt, but that's my opinion, being the one who did
something similar, for 10 years, with a task that, given its size and
complexity, was clearly beyond my humble talents.  To this day, I
still don't understand how I succeeded.

> So people's hands are bound until a complete plan has been worked out
> and/or blessed by Richard

No one's hands are bound.  Whoever is motivated enough will find a way
to bypass the restrictions and limitations.  It certainly means more
work, but that's life.

> and shouting "no one is working on that" is disingenuous.

It's a simple fact.  There's nothing disingenuous in reiterating
facts.  If people are unhappy about such a blunt representation of the
facts, then that's fine by me: I actually want people to become
unhappy, because that just might cause someone to stop complaining and
start working.

Re: IDE

[David Engster]

Eli Zaretskii writes:
> I agree that you need to have some serious will power, perseverance,
> and sometimes just stubbornness to get stuff like that done.

Yes, like that guy who ported gccxml from version to version for
*years*. It's now called CastXML.

>  people are unhappy about such a blunt representation of the facts,
> then that's fine by me: I actually want people to become unhappy,
> because that just might cause someone to stop complaining and start
> working.

Oh, I do.

-David

Re: IDE

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Date: Sat, 10 Oct 2015 12:02:11 +0200
> Cc: David Kastrup <dak@gnu.org>, adatgyujto@gmail.com, emacs-devel@gnu.org
> 
> Eli Zaretskii writes:
> > I agree that you need to have some serious will power, perseverance,
> > and sometimes just stubbornness to get stuff like that done.
> 
> Yes, like that guy who ported gccxml from version to version for
> *years*. It's now called CastXML.

There were 2 bidi implementations for Emacs when I started to work on
what we have now.  Both of them were not good enough, according to the
then Emacs head maintainer.  Imagine what would we have now if I were
less determined to get it right, even though I didn't yet understand
then why those existing implementations couldn't be used.  (I do now,
10 years and many more gray hair later.)

Yes, the situation with the IDE is different, but the morale still
stands: given enough will power, nothing can stand in one's way that
cannot be solved/overcome/worked around.

Re: IDE

[David Engster]

Eli Zaretskii writes:
> Yes, the situation with the IDE is different, but the morale still
> stands: given enough will power, nothing can stand in one's way that
> cannot be solved/overcome/worked around.

You still seem to think I abandoned my work. I did not. I merely
switched libraries.

-David

Re: IDE

[Eli Zaretskii]

> From: David Engster <deng@randomsample.de>
> Cc: adatgyujto@gmail.com,  emacs-devel@gnu.org
> Date: Sat, 10 Oct 2015 12:29:52 +0200
> 
> Eli Zaretskii writes:
> > Yes, the situation with the IDE is different, but the morale still
> > stands: given enough will power, nothing can stand in one's way that
> > cannot be solved/overcome/worked around.
> 
> You still seem to think I abandoned my work. I did not. I merely
> switched libraries.

If that means your work will some day be accepted into Emacs, I'm glad
to hear that.

TIA

Re: IDE

[David Engster]

Eli Zaretskii writes:
>> From: David Engster <deng@randomsample.de>
>> Cc: adatgyujto@gmail.com,  emacs-devel@gnu.org
>> Date: Sat, 10 Oct 2015 12:29:52 +0200
>
>> 
>> Eli Zaretskii writes:
>> > Yes, the situation with the IDE is different, but the morale still
>> > stands: given enough will power, nothing can stand in one's way that
>> > cannot be solved/overcome/worked around.
>> 
>> You still seem to think I abandoned my work. I did not. I merely
>> switched libraries.
>
> If that means your work will some day be accepted into Emacs, I'm glad
> to hear that.

Not for me to decide.

-David

Re: IDE

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Cc: Tom <adatgyujto@gmail.com>,  emacs-devel@gnu.org
>> Date: Sat, 10 Oct 2015 11:00:46 +0200
>> 
>> attempts of letting GCC similarly output AST data were rejected as
>> making it too easy to support non-free applications (obviously, if
>> Emacs can use GCC for purposes of syntax analysis from the command
>> line, so can anybody else).
>
> That's not how that discussion ended.  It ended by Richard saying he
> wanted to study the issue in more depth, before making his decision.

It's been more than half a year.  The time frames in which people
involve themselves in personal projects are smaller than that.  So the
issue becomes a theoretic one and does no longer warrant taking risks or
making difficult decisions.

Rinse and repeat.

That's why I think we need a general course change regarding
interoperability without an immediate tangible purpose.  Because
otherwise we kill incubation.  The most important tool of the programmer
is the garbage bin.  If we take that away, if we demand a proof of
success before opening any possibility, we are taking one of the most
important assets of freedom: the possibility that someone may use that
freedom for purposes beyond our imagination.  Yes, that may cut both
ways.  But we do have the GPLv3 and we should trust it to keep the
damage in check because the alternative is limiting what we can do with
free software for advancing free software.

>> So people's hands are bound until a complete plan has been worked out
>> and/or blessed by Richard
>
> No one's hands are bound.  Whoever is motivated enough will find a way
> to bypass the restrictions and limitations.

That's not particularly inspiring.

-- 
David Kastrup

Re: IDE

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Cc: adatgyujto@gmail.com,  emacs-devel@gnu.org
> Date: Sat, 10 Oct 2015 12:23:11 +0200
> 
> >> So people's hands are bound until a complete plan has been worked out
> >> and/or blessed by Richard
> >
> > No one's hands are bound.  Whoever is motivated enough will find a way
> > to bypass the restrictions and limitations.
> 
> That's not particularly inspiring.

I hope it is.  All I have is my personal example; if that's no
inspiring, then I don't know what will or could be.

Re: IDE

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Cc: adatgyujto@gmail.com,  emacs-devel@gnu.org
>> Date: Sat, 10 Oct 2015 12:23:11 +0200
>> 
>> >> So people's hands are bound until a complete plan has been worked out
>> >> and/or blessed by Richard
>> >
>> > No one's hands are bound.  Whoever is motivated enough will find a
>> > way to bypass the restrictions and limitations.
>> 
>> That's not particularly inspiring.
>
> I hope it is.  All I have is my personal example;

Your personal example was not about a policy and decision outside of
your influence ruling your work unwelcome.

Your example was about overcoming technical difficulties and surpassing
a threshold of code/performance quality.

-- 
David Kastrup

Re: IDE

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Cc: adatgyujto@gmail.com,  emacs-devel@gnu.org
> Date: Sat, 10 Oct 2015 12:47:44 +0200
> 
> > I hope it is.  All I have is my personal example;
> 
> Your personal example was not about a policy and decision outside of
> your influence

You don't know all the details.

I also don't see how the exact cause for rejection of existing work
can matter here.

> ruling your work unwelcome.

Ah, so we are again talking about hurt feelings?

> Your example was about overcoming technical difficulties and surpassing
> a threshold of code/performance quality.

Policy decisions can be easily translated into "technical
difficulties", by writing more code.

Re: IDE

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: David Kastrup <dak@gnu.org>
>> Cc: adatgyujto@gmail.com,  emacs-devel@gnu.org
>> Date: Sat, 10 Oct 2015 12:47:44 +0200
>> 
>> > I hope it is.  All I have is my personal example;
>> 
>> Your personal example was not about a policy and decision outside of
>> your influence
>
> You don't know all the details.
>
> I also don't see how the exact cause for rejection of existing work
> can matter here.
>
>> ruling your work unwelcome.
>
> Ah, so we are again talking about hurt feelings?

I had no work invested in that area, so trying to turn this into an ad
hominem attack is a bit pointless.  Anyway, as long as we rely on most
software being written by humans, "hurt feelings" is not a category we
can consider entirely irrelevant.  Even though in this case it is
entirely a straw man you are trying to create from my choice of words.

>> Your example was about overcoming technical difficulties and surpassing
>> a threshold of code/performance quality.
>
> Policy decisions can be easily translated into "technical
> difficulties", by writing more code.

Shrug.  One can rewrite GCC in Elisp in order to avoid the interdict
against generic interfaces and data exchange.  Sure.  But that's the
kind of obstacle one expects proprietary software to pose, not free
software entirely under copyright and control of the FSF.

Rewriting something from scratch for the sake of escaping the shackles
of proprietary software is a motivation we can expect some contributors
to Emacs and GCC to have.  Rewriting something from scratch for the sake
of escaping the shackles of applied FSF policies isn't.  That's just not
the typical target clientele of GCC and Emacs and it shouldn't need to
be.

-- 
David Kastrup

Re: IDE

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Cc: adatgyujto@gmail.com,  emacs-devel@gnu.org
> Date: Sat, 10 Oct 2015 13:20:48 +0200
> 
> >> ruling your work unwelcome.
> >
> > Ah, so we are again talking about hurt feelings?
> 
> I had no work invested in that area, so trying to turn this into an ad
> hominem attack is a bit pointless.

No attack here, David, certainly not on you.  I just wanted to know
whether we are talking about technical difficulties or psychological
ones.

The latter is exactly where will power should come into play.

> > Policy decisions can be easily translated into "technical
> > difficulties", by writing more code.
> 
> Shrug.  One can rewrite GCC in Elisp in order to avoid the interdict
> against generic interfaces and data exchange.

I don't think one needs to rewrite GCC in any language to get the
information we need.  Some coding is indeed in order.

> Rewriting something from scratch for the sake of escaping the shackles
> of proprietary software is a motivation we can expect some contributors
> to Emacs and GCC to have.  Rewriting something from scratch for the sake
> of escaping the shackles of applied FSF policies isn't.  That's just not
> the typical target clientele of GCC and Emacs and it shouldn't need to
> be.

Now you are creating a straw man.

Re: IDE

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

> In any case, libcc1 is a fait accompli, for several months at least,
> and still no one (AFAIK) investigated whether it can serve as basis
> for any of the IDE-related features.

Here's the answer, from the horse's mouth:

https://lwn.net/Articles/629259/

Read the comments written by mlopezibanez.

Re: IDE

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 1293 bytes --]

On Tue, Oct 6, 2015 at 8:18 PM, Richard Stallman <rms@gnu.org> wrote:

> Emacs with GUD is an IDE.


Wow Richard!  I get that you probably have never used an IDE.  Have you
ever had one demo-ed for you?  Please assure me that your grasp of modern
IDE functionality extends beyond what that statement suggests.  I can never
recall a time when anyone working in the tools and development environment
areas would have concurred that Emacs+GUD constitutes an IDE.


> It has a big advantage compared with other IDEs:

when you see a source file, you're editing it with Emacs.
>

To what are you contrasting Emacs+GUD?  GDB with only a command-line
interface?  GDB with TUI?  For decades source display integrated with
visual indications of the locations of breakpoints, and highlighting the
current execution line has been the absolute minimum ante for debugging
support within an IDE.

If it falls short compared with other IDEs, I think this would be a
> great area for improvement of Emacs.
>

I use GUD + gdb-many-windows.  I put up with its quirks, shortcoming and
idiosyncrasies because I am an old geezer devoted to emacs.  And I would
love to see an improved debugging environment.  But no amount of
improvement along those lines will give Emacs any stronger claim to being
an IDE.

/john

[-- Attachment #2: Type: text/html, Size: 2311 bytes --]

Re: IDE

[Eli Zaretskii]

> Date: Sun, 11 Oct 2015 18:23:55 -0400
> From: John Yates <john@yates-sheets.org>
> Cc: Emacs developers <emacs-devel@gnu.org>, Dmitry Gutov <dgutov@yandex.ru>
> 
> I use GUD + gdb-many-windows. I put up with its quirks, shortcoming and
> idiosyncrasies because I am an old geezer devoted to emacs. And I would love to
> see an improved debugging environment. But no amount of improvement along those
> lines will give Emacs any stronger claim to being an IDE.

I don't understand what made you worked out.  All Richard was saying
that a debugger front-end is an important part of an IDE.  I'm sure
you agree.  Yes, much more is needed to make it a more complete IDE.

Re: IDE

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 1101 bytes --]

On Sun, Oct 11, 2015 at 10:45 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> All Richard was saying that a debugger front-end is an important part of
> an IDE.


Eli,

With all due respect, while you may interpret Richard's words in that
manner, that is not what he wrote.  I quoted at the top of my posting
Richard's very first sentence that started this extended thread.  Here it
is again in its entirety.:

   > Emacs with GUD is an IDE.

Given the amount of influence Richard will exert over efforts to fashion
Emacs into something approximating a modern IDE I believe it is reasonable
to wonder how familiar he is (or is willing to become) with such
technologies.  When we had the long thread some while back about supporting
completion and refactoring I got the sense that Richard was unfamiliar with
the functionality and user experience of modern IDEs.  IIRC he committed to
seeking some outside guidance which might have included becoming more
familiar with the current state of typical IDEs.  If that has yet to happen
I wonder how John's anticipated f2f discussion with Richard will go.

/john

[-- Attachment #2: Type: text/html, Size: 1827 bytes --]

Re: IDE

[Daniel Colascione]

[-- Attachment #1: Type: text/plain, Size: 1935 bytes --]

On 10/12/2015 02:45 AM, John Yates wrote:
> On Sun, Oct 11, 2015 at 10:45 PM, Eli Zaretskii <eliz@gnu.org
> <mailto:eliz@gnu.org>> wrote:
> 
>     All Richard was saying that a debugger front-end is an important
>     part of an IDE.
> 
> 
> Eli,
> 
> With all due respect, while you may interpret Richard's words in that
> manner, that is not what he wrote.  I quoted at the top of my posting
> Richard's very first sentence that started this extended thread.  Here
> it is again in its entirety.:
> 
>    > Emacs with GUD is an IDE.
> 
> Given the amount of influence Richard will exert over efforts to fashion
> Emacs into something approximating a modern IDE I believe it is
> reasonable to wonder how familiar he is (or is willing to become) with
> such technologies.  When we had the long thread some while back about
> supporting completion and refactoring I got the sense that Richard was
> unfamiliar with the functionality and user experience of modern IDEs. 
> IIRC he committed to seeking some outside guidance which might have
> included becoming more familiar with the current state of typical IDEs. 
> If that has yet to happen I wonder how John's anticipated f2f discussion
> with Richard will go.

Well, Emacs with GUD is integrated with GDB, is for development, and is
an environment. That this combination still lacks some features of other
IDEs doesn't matter, since some heavyweight IDEs lack the features of
some others. (Is an IDE without autocompletion an IDE? Is an IDE without
automated refactoring tools an IDE? If not, which tools does it need?)

"IDE" isn't really a distinct category of program. There's a continuum
all the way from ed the standard editor to IntelliJ. We want to move
Emacs toward the latter, but there's no distinct point at which Emacs
will stop being a text editor and start being an IDE; according to some
people, it's already over that line.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: IDE

[Eli Zaretskii]

> Date: Mon, 12 Oct 2015 05:45:43 -0400
> From: John Yates <john@yates-sheets.org>
> Cc: Richard Stallman <rms@gnu.org>, Emacs developers <emacs-devel@gnu.org>, Brief Busters <dgutov@yandex.ru>
> 
>     All Richard was saying that a debugger front-end is an important part of an
>     IDE.
> 
> Eli,
> 
> With all due respect, while you may interpret Richard's words in that manner,
> that is not what he wrote. I quoted at the top of my posting Richard's very
> first sentence that started this extended thread. Here it is again in its
> entirety.:
> 
> > Emacs with GUD is an IDE.

I'll let it to Richard to tell which interpretation, yours or mine, is
closer to what he meant.

> Given the amount of influence Richard will exert over efforts to fashion Emacs
> into something approximating a modern IDE I believe it is reasonable to wonder
> how familiar he is (or is willing to become) with such technologies.

We don't usually ask people here to present their credentials before
they are allowed to speak up on some issue.  Nor do I think we should.

FWIW, I never saw Richard speaking about something without knowing the
issues, or asking some experts.

> When we had the long thread some while back about supporting
> completion and refactoring I got the sense that Richard was
> unfamiliar with the functionality and user experience of modern
> IDEs. IIRC he committed to seeking some outside guidance which might
> have included becoming more familiar with the current state of
> typical IDEs. If that has yet to happen I wonder how John's
> anticipated f2f discussion with Richard will go.

I suggest we leave that to the 2 participants ;-)

Re: New maintainer

[Paul Nathan]

On Mon, Oct 5, 2015 at 12:02 PM, John Wiegley <johnw@newartisans.com> wrote:
>

[snip]

> Another area we're falling behind in is the type of IDE features that are
> taken for granted in special-purpose editing environments, such as effortless
> code browsing, refactoring, and more interactive debugging. The things you can
> do when editing Java and Javascript are downright impressive, and I see no
> reason Emacs can't compete better here.
>

[snip]

>
> John
>


Hi,

As a daily Emacs user, I would like to submit that the above snippet
is keenly important.  I would urge maintainers to run Visual Studio
(C#) and IntelliJ(Java) and experiment with their capabilities....
then beat them in Emacs.

Using SLIME and the Rust completion user interfaces is still a subpar
UI experience compared to IntelliJ or VS.


I would hope that the maintainer would accept the task of beating
proprietary software capabilities head to head (not a simple task!),
instead of simply being excellent. :-)

Re: New maintainer

[Dmitry Gutov]

On 10/06/2015 04:15 AM, Paul Nathan wrote:

> Using SLIME and the Rust completion user interfaces is still a subpar
> UI experience compared to IntelliJ or VS.

You might want to clarify that statement in more detail. Creating a bug 
report would be best.

> I would hope that the maintainer would accept the task of beating
> proprietary software capabilities head to head (not a simple task!),
> instead of simply being excellent. :-)

Personally, I wouldn't want to have exactly the same interface, but 
foremost add features that give the most benefit over cost of 
implementation.

So, what are your primary pet peeves about using Racer [0]?

[0] https://github.com/racer-rust/emacs-racer

Re: New maintainer

[Andreas Röhler]

Am 06.10.2015 um 06:30 schrieb Dmitry Gutov:
> On 10/06/2015 04:15 AM, Paul Nathan wrote:
>
>> Using SLIME and the Rust completion user interfaces is still a subpar
>> UI experience compared to IntelliJ or VS.
>
> You might want to clarify that statement in more detail. Creating a 
> bug report would be best.

Decisions where to head into these area train considerable investments.
Complex questions can't be rolled into a simple bug-report. Also it 
surpasses a feature request.

If Emacs wants to keep path in nowadays evolving world, it needs to 
adapt its discussion-format.

Re: New maintainer

[Rasmus]

Andreas Röhler <andreas.roehler@online.de> writes:

> Am 06.10.2015 um 06:30 schrieb Dmitry Gutov:
>> On 10/06/2015 04:15 AM, Paul Nathan wrote:
>>
>>> Using SLIME and the Rust completion user interfaces is still a subpar
>>> UI experience compared to IntelliJ or VS.
>>
>> You might want to clarify that statement in more detail. Creating a
>> bug report would be best.
>
> Decisions where to head into these area train considerable investments.
> Complex questions can't be rolled into a simple bug-report. Also it
> surpasses a feature request.

Andreas, with all respect, this statement is untrue.  We are talking
software here, not rocked science.  The reason it is better can be broken
down into a set of independent functionalities, that makes it betterᵀᴹ.
For each feature in this set you can make a bug request.  In order to make
any progress, you've got to break down the problem into its parts.

Rasmus

-- 
Dung makes an excellent fertilizer

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I am very backlogged on email this week.

  > As a daily Emacs user, I would like to submit that the above snippet
  > is keenly important.  I would urge maintainers to run Visual Studio
  > (C#) and IntelliJ(Java) and experiment with their capabilities....
  > then beat them in Emacs.

I want the GNU system to have this functionality.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I think it's up to the people in the new maintainer team
to decide how to work together.  There is no rule that says they
have to formalize it.  Whatever works for them, is good.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

You're right that three people can't deadlock -- but I hope that the
people in a group of maintainers will reach consensus through
persuasion, rather than deciding by voting.  Stefan and Yidong were
able to reach decisions together, and did not need to do it by voting.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: New maintainer

[Bastien]

Hello,

"John Wiegley" <johnw@newartisans.com> writes:

>>>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> So, now that I stepped down, we need to find a new maintainer (or a new
>> maintainer-team).
>
> I'd like to self-nominate for that role, Stefan.

I strongly support John as a possible new maintainer for Emacs,
possibly in a team with Eli, which I strongly support too.

I maintain Org-mode since 2011 (with a lower activity in 2015 but
more time in 2016).  Along those years, interacting with John has
always been both helpful and enjoyable.

I completely trust John for moving Emacs forward into the right
direction, and for supporting GNU's goal as a whole.

Having discussions on what are the best ways to achieve freedom
through softwares is both natural and important: varying points
of view keep us vigilant on what freedom is in general, and for
software users in particular.  This is something I value very
much, as long as it does not affect our collectively assumed
goals, which are, right now, the goals of the GNU project.

-- 
 Bastien

Re: New maintainer

[Travis Jeffery]

[-- Attachment #1: Type: text/plain, Size: 1597 bytes --]

On Tue, Sep 29, 2015 at 5:46 PM, John Wiegley <johnw@newartisans.com> wrote:

> >>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
> > So, now that I stepped down, we need to find a new maintainer (or a new
> > maintainer-team).
>
> I'd like to self-nominate for that role, Stefan.
>

John would be an excellent maintainer (or co/head maintainer), in my
opinion.

- Travis

On Tue, Sep 29, 2015 at 5:46 PM, John Wiegley <johnw@newartisans.com> wrote:

> >>>>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
> > So, now that I stepped down, we need to find a new maintainer (or a new
> > maintainer-team).
>
> I'd like to self-nominate for that role, Stefan. I've been contributing to
> Emacs since 1994, and have loved it all the while. Emacs Lisp remains a
> very
> enjoyable language to write certain types of code in.
>
> Some things I'd like to see happen to Emacs are more efficiency, closing
> bugs,
> and wider adoption of some of our newest features, like lexical scoping.
> That
> said, I'm also excited by new prospects, and wonder what can be done in the
> area of concurrency (in some form), a new language under the hood (Guile?),
> etc.
>
> Emacs is my favorite application, by far, and the one I spend the most time
> in, both professionally and personally. It's my programming environment,
> E-mail reader, IRC client, task manager, note taker, and occasional shell.
> I'm
> hoping it will still be the best choice for these things after twenty
> _more_
> years of use, and perhaps as head maintainer I could help keep things
> moving
> in that direction.
>
> John
>
>

[-- Attachment #2: Type: text/html, Size: 2734 bytes --]

Re: New maintainer

[Xue Fuqiao]

On Tue, Sep 29, 2015 at 2:28 PM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:

> But we still need a head maintainer, whose task is mostly to keep an eye
> on the general direction, can make a final decision when we can't reach
> an agreement (of course, we could also delegate that task to
> /dev/random), and to be the official contact point with the FSF.

Agreed.  In old-timers, I'd recommend Glenn Morris, Eli Zaretskii, Paul
Eggert, Lars Magne Ingebrigtsen, and Martin Rudalics.  In the "new
blood", I'd recommend Dmitry Gutov and Artur Malabarba.  Just my 2 cents.

(Although I'm also interested in maintaining Emacs, and I shall have
some spare time to devote to this, I'm not sure if I'm qualified for the
work.)

New maintainer

[Bastien]

Dear all,

I'm stepping down as the Org maintainer.

Carsten accepted to step up, if the community agrees.
Please raise your thumbs up or your concerns, if any.

I'm glad I had this opportunity to work as "Robin" and
I'm even more glad "Batman" may strike back!

:)

-- 
 Bastien

Re: New maintainer

[Jambunathan K]

> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.

To the new maintainer, 

My changes to ox-html.el and ox-odt.el are not assigned to FSF.  So
please correct the headers of these file to reflect the reality.  

Also be respectful of my wish and refrain from merging my changes to
Emacs.  It is a polite request.  Write to me if you have questions.

Jambunathan K.

Re: New maintainer

[John Hendy]

On Thu, Apr 18, 2013 at 12:10 PM, Jambunathan K <kjambunathan@gmail.com> wrote:
>
>> Carsten accepted to step up, if the community agrees.
>> Please raise your thumbs up or your concerns, if any.
>
> To the new maintainer,
>
> My changes to ox-html.el and ox-odt.el are not assigned to FSF.  So
> please correct the headers of these file to reflect the reality.
>
> Also be respectful of my wish and refrain from merging my changes to
> Emacs.  It is a polite request.  Write to me if you have questions.

I'd like some definitive resolution to this issue or list-wide
understanding of where things are at. I've been trying to follow your
saga, and my understanding is:

- You signed over contribution rights to FSF at one point. Is this correct?

- You have contacted FSF (or Emacs?) and requested that these rights
be revoked/reversed?

- Has the FSF (or Emacs?) provided any statement/ruling on whether or
not they will honor your request?

I hear you, and you definitely make it known that you would like to
undo what you once did... but my perception is that your emails speak
as though there is no agreement in place at all... but you *did* sign
the contract at one point, right? I've not contributed to code, so
I've never seen the FSF papers and don't really know what they
entail... but if they are signing away rights to future contributions,
it would seem that your work is under that umbrella until *they*
provide confirmation that the umbrella no longer exists (*and* no
longer exists retroactively).

Just trying to reconcile my understanding of your situation with how
you write about it.


Thanks,
John

>
> Jambunathan K.
>

Re: New maintainer

[Bastien]

Please let's not reopen this issue.

Jambunathan signed the FSF copyright assignment for his past and
future changes to Emacs code.  He claims that he can retract this
assignment for changes he made against some Emacs files.  FSF says
this is not possible.

And it does not take too big a brain to understand why: if people
were allowed to retract their assignment when they want for changes
that have been published, the copyright assignment process would
undermine the whole purpose of the GPL license, which is to make
it possible to let *others* contribute to free code.

If there were a problem, it would be a problem for Emacs, not for
upstream Org-mode, which can include any GPL code.  But there is
no problem.  Just someone upset, in bad need for a ban.

-- 
 Bastien

Re: New maintainer

[Jambunathan K]

Bastien <bzg@gnu.org> writes:

> And it does not take too big a brain to understand why: if people
> were allowed to retract their assignment when they want for changes
> that have been published, the copyright assignment process would
> undermine the whole purpose of the GPL license, which is to make
> it possible to let *others* contribute to free code.

As a maintainer of GNU project, I expect that you should have a basic
understanding of the purpose of the copyright assignment and GPL
license.  From what I read above, I am not convinced that you have the
right understanding.  Your articulation is clearly confusing and falling
short.

See 

        http://www.gnu.org/licenses/gpl-faq.html#AssignCopyright

You assign copyright to FSF so that you don't have to enforce GPL.  By
assigning, one outsources the legal work of actual enforcing to FSF.
Single holder of rights just makes the legal procedures lot more easy.

A contract that cannot be enforced is worthless.  A license that you
cannot enforce is equally so.  

FSF says, assign me the rights, I will go after all the violators and
force them to comply with GPL.

Jambunathan K.

Re: New maintainer

[Alan L Tyree]

Jambunathan K writes:

> Bastien <bzg@gnu.org> writes:
>
>> And it does not take too big a brain to understand why: if people
>> were allowed to retract their assignment when they want for changes
>> that have been published, the copyright assignment process would
>> undermine the whole purpose of the GPL license, which is to make
>> it possible to let *others* contribute to free code.
>
> As a maintainer of GNU project, I expect that you should have a basic
> understanding of the purpose of the copyright assignment and GPL
> license.  From what I read above, I am not convinced that you have the
> right understanding.  Your articulation is clearly confusing and falling
> short.

As a former teacher of copyright law (University of Sydney), I think
that Bastien displays a very clear understanding of the effects of
copyright assignment. Your understanding is less than clear.  Bastien
gets a "Distinction" in my class. You do not.

Of course, I know that you will think that I am confused.

Bastien, thanks for your patience and help during your time in the
hotseat. You've done a marvelous job.

>
> See 
>
>         http://www.gnu.org/licenses/gpl-faq.html#AssignCopyright
>
> You assign copyright to FSF so that you don't have to enforce GPL.  By
> assigning, one outsources the legal work of actual enforcing to FSF.
> Single holder of rights just makes the legal procedures lot more easy.
>
> A contract that cannot be enforced is worthless.  A license that you
> cannot enforce is equally so.  
>
> FSF says, assign me the rights, I will go after all the violators and
> force them to comply with GPL.
>
> Jambunathan K.


-- 
Alan L Tyree           http://www2.austlii.edu.au/~alan
Tel:  04 2748 6206     sip:172385@iptel.org

Re: New maintainer

[Jambunathan K]

Alan L Tyree <alantyree@gmail.com> writes:

> Of course, I know that you will think that I am confused.

You are not only confused. You are in hurry and in grave error.

I am quoting an extract of Bastien's words,

        the copyright assignment process would undermine the whole
        purpose of the GPL license

It is wrong to say copyright assigment will undermine the purpose of GPL
license.  Copyright assignment is there to bolster the enforcement of
GPL.  I provided a reference.

----------------------------------------------------------------

My claim is that there is no assignment.  Because out of my own
initiative I informed FSF that "this work is not covered by contract"
and also cancelled the assignment.

How do you interpret the following block extracted from my assignment

    ,----
    | 2. Developer will report occasionally, on Developer’s initiative
    | and whenever requested by FSF, the changes and/ or enhancements
    | which are covered by this contract, and (to the extent known to
    | Developer) any outstanding rights, or claims of rights, of any
    | person, that might be adverse to the rights of Developer or FSF
    | or to the purpose of this contract.
    `----

FSF clearly side-steps the important question - when is a work actually
assigned.  Assignment is not a process but an event tied to specific
time and date.

Will you disagree if I claim - "The intent to act is not the act
itself".  Replacement act with <whatever>.

Jambunathan K.

Re: New maintainer

[Alan L Tyree]

Jambunathan K writes:

> Alan L Tyree <alantyree@gmail.com> writes:
>
>> Of course, I know that you will think that I am confused.
>
> You are not only confused. You are in hurry and in grave error.

I thought so. Thanks so much for clearing this up for me.

>
> I am quoting an extract of Bastien's words,
>
>         the copyright assignment process would undermine the whole
>         purpose of the GPL license
>
> It is wrong to say copyright assigment will undermine the purpose of GPL
> license.  Copyright assignment is there to bolster the enforcement of
> GPL.  I provided a reference.
>
> ----------------------------------------------------------------
>
> My claim is that there is no assignment.  Because out of my own
> initiative I informed FSF that "this work is not covered by contract"
> and also cancelled the assignment.
>
> How do you interpret the following block extracted from my assignment
>
>     ,----
>     | 2. Developer will report occasionally, on Developer’s initiative
>     | and whenever requested by FSF, the changes and/ or enhancements
>     | which are covered by this contract, and (to the extent known to
>     | Developer) any outstanding rights, or claims of rights, of any
>     | person, that might be adverse to the rights of Developer or FSF
>     | or to the purpose of this contract.
>     `----
>
> FSF clearly side-steps the important question - when is a work actually
> assigned.  Assignment is not a process but an event tied to specific
> time and date.
>
> Will you disagree if I claim - "The intent to act is not the act
> itself".  Replacement act with <whatever>.
>
> Jambunathan K.


-- 
Alan L Tyree           http://www2.austlii.edu.au/~alan
Tel:  04 2748 6206     sip:172385@iptel.org

Re: New maintainer

[Tassilo Horn]

Jambunathan K <kjambunathan@gmail.com> writes:

> How do you interpret the following block extracted from my assignment
>
>     ,----
>     | 2. Developer will report occasionally, on Developer’s initiative
>     | and whenever requested by FSF, the changes and/ or enhancements
>     | which are covered by this contract, and (to the extent known to
>     | Developer) any outstanding rights, or claims of rights, of any
>     | person, that might be adverse to the rights of Developer or FSF
>     | or to the purpose of this contract.
>     `----

Well, the FSF's intention here is to make sure that contributors report
back when they change employers, and the new employer doesn't want that
his employees contribute to some GNU project (maybe because that project
is in the same business as the company).  So I think of that more of a
safety measure in order not to run into long-running, painful lawsuits.

> FSF clearly side-steps the important question - when is a work
> actually assigned.  Assignment is not a process but an event tied to
> specific time and date.

As far as I understand it (just after reading one of my FSF CAs), the
changes you apply to a program where you've assigned past & future
changes are assigned as soon as they are written.  They don't need to be
published, distributed, placed in a special repository location, etc.

Bye,
Tassilo

Re: New maintainer

[Jambunathan K]

Tassilo Horn <tsdh@gnu.org> writes:

> Jambunathan K <kjambunathan@gmail.com> writes:
>
>> How do you interpret the following block extracted from my assignment
>>
>>     ,----
>>     | 2. Developer will report occasionally, on Developer’s initiative
>>     | and whenever requested by FSF, the changes and/ or enhancements
>>     | which are covered by this contract, and (to the extent known to
                                             ^^^
                                             ^^^
                                             ^^^
>>     | Developer) any outstanding rights, or claims of rights, of any
>>     | person, that might be adverse to the rights of Developer or FSF
>>     | or to the purpose of this contract.
>>     `----
>
> Well, the FSF's intention here is to make sure that contributors report
> back when they change employers, and the new employer doesn't want that
> his employees contribute to some GNU project (maybe because that project
> is in the same business as the company).  So I think of that more of a
> safety measure in order not to run into long-running, painful
> lawsuits.

Your interpretation is too narrow.  The phrase "and" (marked above)
there makes all the difference.

My reading of the above para and Richard's response down below are
consistent.

  ,----  http://lists.gnu.org/archive/html/emacs-devel/2013-04/msg00066.html
  | If a contributor wants to specify precisely which changes are
  | assigned, he, she or it can talk with the FSF about it.  We can
  | work something out.  However, we'd prefer to be able to use all of
  | someone's changes without specific arrangements.
  `----

>> FSF clearly side-steps the important question - when is a work
>> actually assigned.  Assignment is not a process but an event tied to
>> specific time and date.
>
> As far as I understand it (just after reading one of my FSF CAs), the
> changes you apply to a program where you've assigned past & future
> changes are assigned as soon as they are written.  They don't need to be
> published, distributed, placed in a special repository location, etc.

Assignment of rights is for the purpose of defending GPL.  It is *not*
and *cannot* be an annexation policy.  What you state above amounts to
annexation policy.

Assignment of rights is my prerogative.  I can do it selectively or
cancel it.  FSF cannot interfere with what is an individual decision.

I will only quote an authoritative and responsible source. Focus on last
sentence in the below quote.

    ,----  http://lwn.net/Articles/543436/
    |
    | Anyway, it's unfortunate the Corbet's article above doesn't
    | reiterate the advantages of assigning to FSF to
    | developers. Specifically, the FSF takes on the obligation of being
    | the publisher of the code (which can sometimes be a dangerous act
    | in today's world), and also, FSF handles enforcement of the GPL
    | for the codebase. Finally, FSF gives a liberal license back to the
    | developer (i.e., Jambunathan could have always made proprietary
    | software out of his own assigned works after doing the
    | assignment), and FSF further promises never to publish a
    | proprietary version of the software itself.
    |
    `----

I interpret "proprietary" above to mean "any work (available to public,
it is GPL right?) that is not part of Emacs distribution".

Theoretically speaking, it is OK to have future assignment in place
*and* have works that are assigned, as well as non-assigned
*simultaneously*.  If a work is not part of Emacs, then it is "not
assigned" to FSF.  Simple and practical definition.

----------------------------------------------------------------

It is also important to note that the above paragraph from a FSF board
member is in some conflict with RMS's own claim.

    ,---- http://lists.gnu.org/archive/html/emacs-devel/2013-03/msg00409.html
    |
    | A diff for Emacs is always a change to Emacs.  I will think about
    | the questions raised by a separate Lisp file.
    |
    `----

In my opinion, the most appropriate thing to state would be 

   "A diff to Emacs submitted to the official channels of the suite -
    either the maintainer, mailing list or bug subsystem, *unless stated
    otherwise* is potentially part of Emacs, in a non-revocable way."

It will be inappropriate for anyone to claim, my local changes to
doc-view.el that I share with a co-worker is a diff to Emacs and hence
part of Emacs.  A change is either part of Emacs or not.  When I say
Emacs, I mean the official GNU Emacs distributed from gnu.org.

----------------------------------------------------------------

Now the primary part of current dispute is that it falls in a grey area

    1. My files are new.

    2. "Org in Emacs Bzr trunk" is not the same as "Org outside of Emacs
        trunk".  One is "part of Emacs" while the other is "not part of
        Emacs".  There is a clear dichotomy here.  It is easy to answer
        "Is this file part of Emacs?" with a "Yes" or "No".  Saying that
        a file is intended to be part of Emacs is equivalent to saying
        "No."

        Thought experiment: You cannot claim ownership of a house merely
        because the other party "intended" to sell it.

----------------------------------------------------------------

I will advise all vigorous enterprising young men *never* to assign his
rights to any organization other than his own.

----------------------------------------------------------------

"Intent to act is not the act itself"
Jambunathan K,
        
> Bye,
> Tassilo

Re: New maintainer

[Jambunathan K]

Jambunathan K <kjambunathan@gmail.com> writes:

>> Well, the FSF's intention here is to make sure that contributors report
>> back when they change employers, and the new employer doesn't want that
>> his employees contribute to some GNU project (maybe because that project
>> is in the same business as the company).  So I think of that more of a
>> safety measure in order not to run into long-running, painful
>> lawsuits.

You are missing out an important aspect - that of "enforcement".  An
organization will most likely "choose to enforce" but an RJH (like me)
won't.

That is, the employer can (presumably) send his lawyer to a the court
with the employment contract and say

    "Employee can assign rights (and FSF can very well accept it).  But
     the assignation has no legal validity because it is not within
     employee's right to do so.  Employee himself agreed that he will
     abide by <whatever> while on our pay.  We are asserting and
     enforcing our position now."

For an assignment to have legal validity, multiple parties - FSF,
contributor and contributor's employer - should *converge*.

When there is no convergence of *all* parties , the "assignment" stands
on weaker grounds.

Standing on weaker ground is precisely what FSF wants to avoid at all
costs.

Jambunathan K.

Re: New maintainer

[Carsten Dominik]

On 21.4.2013, at 10:06, Jambunathan K <kjambunathan@gmail.com> wrote:

> Jambunathan K <kjambunathan@gmail.com> writes:
> 
>>> Well, the FSF's intention here is to make sure that contributors report
>>> back when they change employers, and the new employer doesn't want that
>>> his employees contribute to some GNU project (maybe because that project
>>> is in the same business as the company).  So I think of that more of a
>>> safety measure in order not to run into long-running, painful
>>> lawsuits.
> 
> You are missing out an important aspect - that of "enforcement".  An
> organization will most likely "choose to enforce" but an RJH (like me)
> won't.
> 
> That is, the employer can (presumably) send his lawyer to a the court
> with the employment contract and say
> 
>    "Employee can assign rights (and FSF can very well accept it).  But
>     the assignation has no legal validity because it is not within
>     employee's right to do so.  Employee himself agreed that he will
>     abide by <whatever> while on our pay.  We are asserting and
>     enforcing our position now."
> 
> For an assignment to have legal validity, multiple parties - FSF,
> contributor and contributor's employer - should *converge*.
> 
> When there is no convergence of *all* parties , the "assignment" stands
> on weaker grounds.
> 
> Standing on weaker ground is precisely what FSF wants to avoid at all
> costs.
> 
> Jambunathan K.

This discussion is now considered off-topic for this list.
Please take it elsewhere.

- Carsten

Re: New maintainer

[Jambunathan K]

> Just trying to reconcile my understanding of your situation with how
> you write about it.

This post (not by me) summarizes my position well.

        http://lists.gnu.org/archive/html/emacs-devel/2013-04/msg00096.html

I have also criticized FSF's arbitrary handling of my request here.

        http://lwn.net/Articles/547737/

If you move over to emacs-devel, you will see some of the commenters
there are very guarded in their response.  It will only go on to show
that I raise valid questions.

People who have flamed me cannot differentiate between oranges and
lemons.  It is not their work that is at stake.  So they can just call
me a troll and dismiss the broader questions I am raising.

Jambunathan K.

Re: New maintainer

[Glyn Millington]

Bastien <bzg@gnu.org> writes:

> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.  Please raise
> your thumbs up or your concerns, if any.
>
> I'm glad I had this opportunity to work as "Robin" and I'm even more
> glad "Batman" may strike back!

A huge thumbs up from this extremely grateful user!

And an equally huge thank you, Bastien, for shouldering the burden of
maintainership, for the diligence with which you have handled the task
(and the hundreds/thousands of emails which have kept the project moving)
and maybe above all for your patience and steadfastness under totally
unmerited fire. 

with all good wishes





Glyn

Re: New maintainer

[Aaron Ecay]

Hi Bastien,

2013ko apirilak 18an, Bastien-ek idatzi zuen:

[...]

> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!

Just as Batman and Robin are often seen together, I hope you’ll continue
to be a presence in the org community.  I always appreciate the
helpfulness and patience that you embody.

Thanks,

-- 
Aaron Ecay

Re: New maintainer

[Rasmus]

Bastien <bzg@gnu.org> writes:

> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!

Thanks for your hard work, Bastien.  Great with release 8.0!

I expect you will still be around the mailing list, no?

–Rasmus

-- 
This is the kind of tedious nonsense up with which I will not put

Re: New maintainer

[Bastien]

Rasmus <rasmus@gmx.us> writes:

> I expect you will still be around the mailing list, no?

I'll focus on something else, but I'll stick around for sure,
at least to ask questions and report bugs!

-- 
 Bastien

Re: New maintainer

[Detlef Steuer]

The king is dead, long live the king!
:-)

Thank you, Bastien, for all the work!

And thank you, Carsten, for all the work to come! 

Detlef


On Thu, 18 Apr 2013 18:53:32 +0200
Bastien <bzg@gnu.org> wrote:

> Dear all,
> 
> I'm stepping down as the Org maintainer.
> 
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
> 
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
> 
> :)
> 
> -- 
>  Bastien
> 
> 
> 

Re: New maintainer

[François Pinard]

Bastien <bzg@gnu.org> writes:

> Please raise your thumbs up or your concerns, if any.

Bastien, your maintainership has been just outstanding, so far that I
could judge.  You're among the great maintainers I happened to meet, and
I tremendously enjoyed your way of driving the project.  Let me thank
you for it all.

I wish Carsten will get, from all of us — OK! given a proper email kill
file :-) — the same level of good will, enthusiasm and collaboration we
have seen on this user group all along for the recent year.

François

Re: New maintainer

[Alan Schmitt]

Bastien writes:

> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.

Thumbs up! And thank you so much for what you did.

Alan

Re: New maintainer

[Thomas S. Dye]

Aloha Bastien,

Bastien <bzg@gnu.org> writes:

> Dear all,
>
> I'm stepping down as the Org maintainer.

Thank you for your excellent work as maintainer.  I'm pleased to have
had the opportunity to work with you.

>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>

Yes, thumbs up.

> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!

You should be proud that you're giving back a much improved piece of
software.  From my point of view, the new exporter framework is a huge
step forward.  Congratulations!

All the best,
Tom

-- 
Thomas S. Dye
http://www.tsdye.com

Re: New maintainer

[Bastien]

Hi Thomas,

tsd@tsdye.com (Thomas S. Dye) writes:

> Thank you for your excellent work as maintainer.  I'm pleased to have
> had the opportunity to work with you.

Having you improve the documentation while Nicolas and I were hard
working on it was a great relief, thanks again for that.

> You should be proud that you're giving back a much improved piece of
> software.  From my point of view, the new exporter framework is a huge
> step forward.  Congratulations!

Yes -- I second Carsten's surprise that someone was crazy enough to
undertake this enormous work.  And its potential still waits to be
completely unfold! 

-- 
 Bastien

Re: New maintainer

[Jonathan Leech-Pepin]

[-- Attachment #1: Type: text/plain, Size: 459 bytes --]

Hello Bastien,


On 18 April 2013 12:53, Bastien <bzg@gnu.org> wrote:

> Dear all,
>
> I'm stepping down as the Org maintainer.
>

Thank you for all the work you've done.

>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>
> Thumbs up from me as well.


> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
>
> :)
>
> --
>  Bastien
>
> --
Jon

[-- Attachment #2: Type: text/html, Size: 1222 bytes --]

Re: New maintainer

[Tom Davey]

Hi everybody,

I'm just an Org user, one of the many anonymous persons who have
benefited from this fantastic piece of software. Over the past two
years I have come to use Org every day, all day long, more than any
other application with the possible exception of a Web browser. It's
hard to overestimate the value I receive from Org, the sheer personal
effectiveness I've gained.

Bastien, a thousand thanks for your work as maintainer. Thanks as well
to all the other skillful and creative programmers on this list who
make org continually more powerful and astonishing. Especial thanks to
Carsten, both for the past and now in advance as the new maintainer.

With grateful regards to all,
Tom Davey

--
Tom Davey
tom@tomdavey.com
New York NY USA

Re: New maintainer

[Ian Barton]

On 18/04/13 22:52, Tom Davey wrote:
> Hi everybody,
>
> I'm just an Org user, one of the many anonymous persons who have
> benefited from this fantastic piece of software. Over the past two
> years I have come to use Org every day, all day long, more than any
> other application with the possible exception of a Web browser. It's
> hard to overestimate the value I receive from Org, the sheer personal
> effectiveness I've gained.
>
> Bastien, a thousand thanks for your work as maintainer. Thanks as well
> to all the other skillful and creative programmers on this list who
> make org continually more powerful and astonishing. Especial thanks to
> Carsten, both for the past and now in advance as the new maintainer.
>
> With grateful regards to all,
> Tom Davey

Me too:) Many thanks to Bastien for all his hard work and for patiently 
answering questions and fixing bugs. Also welcome back to Carsten!

Best wishes,

Ian.

Re: New maintainer

[Charles Berry]

Bastien <bzg <at> gnu.org> writes:

> 
> Dear all,
> 
> I'm stepping down as the Org maintainer.
> 

You have done an awesome job. 

This is still the "friendliest little town" on the internet thanks to 
your patience and good judgment in no small part.

And 8.0 is a fine product.


> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
> 

+2 thumbs

Gmane requires me to type the capcha 'uplift' to send this message.
How appropriate!

All the Best,

Re: New maintainer

[Suvayu Ali]

On Fri, Apr 19, 2013 at 12:24:46AM +0000, Charles Berry wrote:
> Bastien <bzg <at> gnu.org> writes:
> 
> > Dear all,
> > 
> > I'm stepping down as the Org maintainer.
> 
> You have done an awesome job. 
> 
> This is still the "friendliest little town" on the internet thanks to 
> your patience and good judgment in no small part.

Indeed!  Thanks a lot :).  And welcome back Carsten.

:)

-- 
Suvayu

Open source is the future. It sets us free.

Re: New maintainer

[Bernt Hansen]

Bastien <bzg@gnu.org> writes:

> Dear all,
>
> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
>
> :)


Dear Bastien and Carsten,

Thanks, Bastien, for doing an awesome job as the org-mode maintainer
these past few years.  You had a huge job filling Carsten's shoes and
you did an excellent job.  I really appreciate all of the work you have
done as our org-mode maintainer and hope you will continue to contribute
in the future.

Welcome back Carsten!  Please let us know how we can help make your job
easier.  None of us have infinite time to devote to this excellent
project but I'm sure we can help lessen the burden that maintaining this
beast must be.  I use this awesome tool everyday and really appreciate
the thought and effort put into making org-mode the fabulous tool that
it is.

/me can't draw two thumbs up in Ascii without making it look obscene ;)

Warmest regards,
Bernt

Re: New maintainer

[Yagnesh Raghava Yakkala]

Hello Bastien,

Thanks a lot for the great work so far. You are the best.

On Apr 19 2013, Bastien <bzg@gnu.org> wrote:

> Dear all,
>
> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.

👍

>
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
>
> :)


Thanks.,
--
ఎందరో మహానుభావులు అందరికి వందనములు.
YYR

Re: New maintainer

[Noorul Islam Kamal Malmiyoda]

[-- Attachment #1: Type: text/plain, Size: 599 bytes --]

Bastien,

I use org-mode every day and it is really great to see that this project is
always growing. You did an excellent job taking this to 8.0.

Carsten,

Welcome back and looking forward to 8.x

Thanks and Regards
Noorul


On Thu, Apr 18, 2013 at 10:23 PM, Bastien <bzg@gnu.org> wrote:

> Dear all,
>
> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
>
> :)
>
> --
>  Bastien
>
>
>

[-- Attachment #2: Type: text/html, Size: 1190 bytes --]

Re: New maintainer

[Robert Klein]

Hi Bastien,

thank you very very much for maintaining org-mode
for the last two years.

You did a great job maintaining org-mode and keeping
its community together.

I agree with everything Carsten says in his mail
"Changing the maintainer".

While I am happy it is Carsten of all people who
takes up the maintainership, at the same time I'm
very sorry you are stepping down.


Best regards
Robert



On 04/18/2013 06:53 PM, Bastien wrote:
> Dear all,
> 
> I'm stepping down as the Org maintainer.
> 
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
> 
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
> 
> :)
> 


-- 
Robert Klein - Max Planck-Institut für Polymerforschung
Ackermannweg 10
55128 Mainz

Re: New maintainer

[Christian Moe]

Bastien,

What the others said... Thanks for all your work! It's been a pleasure to
follow Org-mode.

Yours,
Christian

Bastien writes:

> Dear all,
>
> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
>
> :)

Re: New maintainer

[Thorsten Jolitz]

Bastien <bzg@gnu.org> writes:

> I'm stepping down as the Org maintainer.

Thats bad news, managing Org-mode appears like such a huge task, and I
always wondered how you were able to deal with all those mails and
patches and bugs with such efficiency, and detailled knowledge about all
those unnumerable features of Org-mode. 

> Carsten accepted to step up

Luckily the good news are attached to the bad news. 

-- 
cheers,
Thorsten

Re: New maintainer

[Sean O'Halpin]

Hi Bastien,

I'd like to thank you for the fabulous job you've done as maintainer.

Best wishes,
Sean

On Thu, Apr 18, 2013 at 5:53 PM, Bastien <bzg@gnu.org> wrote:
> Dear all,
>
> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
>
> :)
>
> --
>  Bastien
>
>

Re: New maintainer

[Charles Philip Chan]

Bastien <bzg@gnu.org> wrote:

Hi Bastien,

>I'm stepping down as the Org maintainer.

Thank you for all the hard work you have done in maintaining Org-mode.

>Carsten accepted to step up, if the community agrees.
>Please raise your thumbs up or your concerns, if any.

+1

Charles


--
Sent from Kaiten Mail. Please excuse my brevity.

Re: New maintainer

[Adolfo Benedetti]

[-- Attachment #1: Type: text/plain, Size: 626 bytes --]

Hi Bastien,

Thank you for your hard work and efforts as org-mode-maintainer,

>Carsten accepted to step up, if the community agrees.

nice, thank you Carsten!


cheers

--
Adolfo Benedetti
M +31 614 706 176


2013/4/19 Charles Philip Chan <cpchan@bell.net>

> Bastien <bzg@gnu.org> wrote:
>
> Hi Bastien,
>
> >I'm stepping down as the Org maintainer.
>
> Thank you for all the hard work you have done in maintaining Org-mode.
>
> >Carsten accepted to step up, if the community agrees.
> >Please raise your thumbs up or your concerns, if any.
>
> +1
>
> Charles
>
>
> --
> Sent from Kaiten Mail. Please excuse my brevity.
>
>

[-- Attachment #2: Type: text/html, Size: 1617 bytes --]

Re: New maintainer

[Rainer Stengele]

Am 18.04.2013 18:53, schrieb Bastien:
> Dear all,
> 
> I'm stepping down as the Org maintainer.
> 
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
> 
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
> 
> :)
> 
Bastien,

let me also thank you so much for your great work and patience answering so many questions!
A lot of questions and ideas I had you simply answered by coding the solution - awesome!
I think you did a very good job and I hope you will stay in the newsgroup in the future.

Carsten, welcome back!

Rainer

Re: New maintainer

[Russell Adams]

On Thu, Apr 18, 2013 at 06:53:32PM +0200, Bastien wrote:
> Dear all,
>
> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!

Bastien,

You've done an excellent job. Your patience has been incredible, and
Org has prospered. Take a break, you deserve it!

8.0 looks amazing, and I'm excited to see what the community has
created. Org certainly has the benefit of many talented coders.

Thanks!

------------------------------------------------------------------
Russell Adams                            RLAdams@AdamsInfoServ.com

PGP Key ID:     0x1160DCB3           http://www.adamsinfoserv.com/

Fingerprint:    1723 D8CA 4280 1EC9 557F  66E8 1154 E018 1160 DCB3

Re: New maintainer

[Christopher Allan Webber]

You've done great work Bastien!

And I look forward to Batman Returns!

Bastien writes:

> Dear all,
>
> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
>
> :)

Re: New maintainer

[T.F. Torrey]

Thank you for your hard work, Bastien. You've done a fantastic job
under unusually adversarial conditions.

On Thu, Apr 18, 2013 at 9:53 AM, Bastien <bzg@gnu.org> wrote:
> Dear all,
>
> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
>
> :)
>
> --
>  Bastien
>
>

Re: New maintainer

[Eric Abrahamsen]

Bastien <bzg@gnu.org> writes:

> Dear all,
>
> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
>
> :)

Hats off, Bastien, it was deftly done.

It's also been a pleasure to witness the surprisingly successful
marriage of two different coding styles: Bastien's damn-the-torpedoes
patch-the-SOB-and-get-it-out-the-door approach, matched with Nicolas'
return to first principles: structure and cleanliness. I'm quite
convinced that the two approaches have been equally essential to Org
mode's current success (and advance apologies for any perceived
mischaracterizations!).

I'm one of the post-Carsten young'uns, but I can't imagine we'll have
any complaints with the man who started it all...

Eric

Re: New maintainer

[Bastien]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> It's also been a pleasure to witness the surprisingly successful
> marriage of two different coding styles: Bastien's damn-the-torpedoes
> patch-the-SOB-and-get-it-out-the-door approach, matched with Nicolas'
> return to first principles: structure and cleanliness. I'm quite
> convinced that the two approaches have been equally essential to Org
> mode's current success (and advance apologies for any perceived
> mischaracterizations!).

Yeah.

Let me quote Jamie Zawinski's interview from "Coders at work":

  Zawinski: [...] It's great to rewrite your code and make it cleaner
  and by the third time it'll actually be pretty.  But that's not the
  point---you're not here to write code, you're here to ship products.
  
  Seibel: Folks engaged in overengineering usually say, "Well, once
  I've got this framework in place everything will be easy after that.
  So I'll actually save time by doing this.
  
  Zawinski: That is always the theory.

  Seibel: And there are times when that theory is true, when someone
  has good sense and the framework isn't too elaborate, and it does
  save time.

I actually agree with both points of view, especially with the last
sentence.  And it's easy to play jwz when you can trust someone for
playing the other role :)

-- 
 Bastien

Re: New maintainer

[Eric Abrahamsen]

Bastien <bzg@gnu.org> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> It's also been a pleasure to witness the surprisingly successful
>> marriage of two different coding styles: Bastien's damn-the-torpedoes
>> patch-the-SOB-and-get-it-out-the-door approach, matched with Nicolas'
>> return to first principles: structure and cleanliness. I'm quite
>> convinced that the two approaches have been equally essential to Org
>> mode's current success (and advance apologies for any perceived
>> mischaracterizations!).
>
> Yeah.
>
> Let me quote Jamie Zawinski's interview from "Coders at work":
>
>   Zawinski: [...] It's great to rewrite your code and make it cleaner
>   and by the third time it'll actually be pretty.  But that's not the
>   point---you're not here to write code, you're here to ship products.
>   
>   Seibel: Folks engaged in overengineering usually say, "Well, once
>   I've got this framework in place everything will be easy after that.
>   So I'll actually save time by doing this.
>   
>   Zawinski: That is always the theory.
>
>   Seibel: And there are times when that theory is true, when someone
>   has good sense and the framework isn't too elaborate, and it does
>   save time.
>
> I actually agree with both points of view, especially with the last
> sentence.  And it's easy to play jwz when you can trust someone for
> playing the other role :)

And, without re-opening any tedious discussions that we've already put
behind us, it's generally the person playing the jwz role who ends up as
"maintainer" -- and that's probably as it should be.

Re: New maintainer

[Andreas Röhler]

Am 18.04.2013 18:53, schrieb Bastien:
> Dear all,
>
> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
>
> :)
>

Hi Bastien,

have been afk for some days, please let me close up to all thanks seen here.
What's have been quite impressive already was the pure amount of mailings, you delt with day by day.

Having seen you working that hard also raises some concern for now: that daily galeere must be costly: whoever undertakes it, will pay a price. It's your honor having payed 
it for all of us. So far can't consider this stepping down anything else as a loss for org-mode.

While ignoring circumstances of your resign, it's no secret, some unpleasant events happended last weeks, made your task more burdensome as necessary.
I'm not speaking of possible errors - everyone who works will make errors. Who works outstanding might make outstanding errors.
Purely abstractly spoken(!)

OTHO the very best mind a team has --i.e. Carsten--, should not take the most burdensome tasks. IMHO Carsten should be spared for strategic decisions, define and decide the 
path of further development.

In case you didn't lose your interest and just that recent unpleasant experiences caused your resign, what about staying maintainer backed by all this confidence revealed 
beside of some new experience?

Also maintainer must not mean being strictly a single person, even if languages grammar doesn't foresee otherwise.
Regarding recent difficulties, probably it's wise, if Carsten has a closer look, decides in cases from time to time.
IIUC Emacs itself was driven in a similar way last years more or less outspoken - consider this part of it's success story.

Best regards,

Andreas

Re: New maintainer

[Bastien]

Hi Andreas,

thanks for the kind words.

The decision to step down after 8.0 was taken a long time ago,
before the recent problems on the list.  I had to find someone
willing to step in before I could announce this.

I agree "maintainer" is not necessary a single person: my main
purpose was to build a team, so that future maintainer(s) would
feel okay to act as you suggest, for strategic decisions rather
than everyday ground-level stuff.

This position is easy to describe but difficult to hold, because
it depends so much on the community.

This *is* the real challenge I see now: that each of us endorses
some kind of responsability, some co-maintainership feeling, and
act as constructively as possible---be it for org-mode, worg, the
website or whatever.

I already can feel some go in this direction and that's great :)

-- 
 Bastien

Re: New maintainer

[Andreas Röhler]

Hi Bastien,

Am 22.04.2013 00:39, schrieb Bastien:
> Hi Andreas,
>
> thanks for the kind words.
>
> The decision to step down after 8.0 was taken a long time ago,
> before the recent problems on the list.  I had to find someone
> willing to step in before I could announce this.
>

Okay, a good news in circumstances.

> I agree "maintainer" is not necessary a single person: my main
> purpose was to build a team, so that future maintainer(s) would
> feel okay to act as you suggest, for strategic decisions rather
> than everyday ground-level stuff.
>
> This position is easy to describe but difficult to hold,

Precisely.

> because it depends so much on the community.
>
> This *is* the real challenge I see now: that each of us endorses
> some kind of responsability, some co-maintainership feeling, and
> act as constructively as possible---be it for org-mode, worg, the
> website or whatever.
>
> I already can feel some go in this direction and that's great :)
>

Indeed.

Nonetheless, WRT the amount of traffic IMHO having someone to range things a little bit before Carsten must tell will be a great advantage.

Thanks all proving that great stuff,

Andreas

Re: New maintainer

[AG]

Bastien <bzg <at> gnu.org> writes:

> 
> Dear all,
> 
> I'm stepping down as the Org maintainer.
> 
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
> 
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
> 
> :)
> 


Dear Bastien,
 I too would like to express my thanks for all your hard work in keeping
org running smoothly over the last couple of years, and unfailing
(preternatural) graciousness even in the face of personal and unprofessional
attacks. I hope you are leaving of your own volition. Illegitimi non
carborundum.


I am also grateful to Carsten for his great contributions of Org, reftex and
cdlatex, and happy to hear he will be taking over.

AG

Re: New maintainer

[Julian M. Burgos]

I echo all the thanks that other people already gave.  My digital life
orbits around org-mode, so thanks to everyone who contributed to this
project.  Keep it up!!

Julian  

-- 
Julian Mariano Burgos, PhD
Hafrannsóknastofnunin/Marine Research Institute
Skúlagata 4, 121 Reykjavík, Iceland
Sími/Telephone : +354-5752037
Bréfsími/Telefax:  +354-5752001
Netfang/Email: julian@hafro.is


Bastien writes:

> Dear all,
>
> I'm stepping down as the Org maintainer.
>
> Carsten accepted to step up, if the community agrees.
> Please raise your thumbs up or your concerns, if any.
>
> I'm glad I had this opportunity to work as "Robin" and
> I'm even more glad "Batman" may strike back!
>
> :)

Re: New maintainer

[Jay Kerns]

On Mon, Apr 22, 2013 at 6:27 AM, Julian M. Burgos <julian@hafro.is> wrote:
> I echo all the thanks that other people already gave.  My digital life
> orbits around org-mode, so thanks to everyone who contributed to this
> project.  Keep it up!!
>
> Julian
>

I have been watching these multiple messages go by trying to find a
space to get a word in edgewise, but instead of waiting longer let me
just say now:  I am sorry to see you step down, Bastien, but also, I
am happy about your bright future ahead.  Congratulations! on a job
very well done!

Regards,

-- 
Jay

Re: New maintainer

[Matt Price]

Just echoing what everyone else has said:  Bastien, your tenure at the
helm has just been fabulous.  8.0 is just an amazing release and org
already just amazingly great has become even better.  Carsten, it's so
generous for you to come back to this project to which you have
already devoted so much energy.  No other tool I use has had such a
great pair of lead developers or such an open and helpful community.
thank you both!

Matt

Re: New maintainer

[Jason Dunsmore]

[-- Attachment #1: Type: text/plain, Size: 241 bytes --]

Thanks for all of your hard work on org-mode Bastien!  Watching you in
action has taught me a great deal about good project/community leadership.
 If there's anything I can do to help with the transition, please let me
know.

Regards,
Jason

[-- Attachment #2: Type: text/html, Size: 315 bytes --]