Emacs User Survey 2020 Results

Emacs User Survey 2020 Results

[Adrien Brochard]

Hi everyone,

After a week of reading every submission, cleaning up the data, and
leaning matplotlib, I finally have enough confidence to publish the
results of the Emacs User Survey 2020.

https://emacssurvey.org/2020/

I want to thank everyone who responded, commented, and shared it!
There's over 7300 responses and it's really thanks to this amazing
community.

There is still a lot to do, the data could always be analyzed
differently, the website could be nicer, etc, but the responses have
been so overwhelmingly positive that I just have to publish without more
delay. If you have feedback or feel like contributing, all source and
data are public.

Thank you again!

Adrien Brochard

Re: Emacs User Survey 2020 Results

[Jean Louis]

Hello Adrien,

I find efforts nice and friendly. Don't think it is badmind that I may
be picky on some specifics on your presentation.

* Adrien Brochard <abrochard@gmx.com> [2020-12-09 19:43]:
> Hi everyone,
> 
> After a week of reading every submission, cleaning up the data, and
> leaning matplotlib, I finally have enough confidence to publish the
> results of the Emacs User Survey 2020.

That is definitely hard work as you allowed non-strict data to enter
the survey as both answering by email and by web is not easy to
categorize.

> https://emacssurvey.org/2020/
> 
> I want to thank everyone who responded, commented, and shared it!
> There's over 7300 responses and it's really thanks to this amazing
> community.

That is surprising result, maybe you remember I was expecting much
less people to respond. To me that speaks that there may be 7.3
millions Emacs users minimum, as I consider 1 survey submitted for
1000 people who did not submit. This is a vague fact known in media
such as newspapers. It may not be.

> There is still a lot to do, the data could always be analyzed
> differently, the website could be nicer, etc, but the responses have
> been so overwhelmingly positive that I just have to publish without
> more delay. If you have feedback or feel like contributing, all
> source and data are public.

Thank you.

Now here are my comments:

Number one critic to you is that you drive people who use free
software to non-free software. You maybe have to research why Emacs
came to be Emacs and why is it named GNU Emacs. There are reasons for
that. I cannot be soft-hearted on that. When you call it "Emacs
Survey" why not make it in the spirit of Emacs as free software and
stop promoting non-free proprietary program such as Matlab. GNU Emacs

GNU Octave, as Scientific Programming Language, is pPowerful
mathematics-oriented syntax with built-in 2D/3D plotting and
visualization tools Free software, runs on GNU/Linux, macOS, BSD, and
Microsoft Windows Drop-in compatible with many Matlab scripts.

See: https://www.gnu.org/software/octave/

Same critics remain that you have driven people to non-free
proprietary Javascript. In the second attempt to make the survey is
that going to happen again? I hope not.

Now comes the technical part of my critics which is meant that you
maybe, hopefull, kindly, try to improve next time.

- personally I do not find submission medium important, email or
  website, alright and fine, but it does not contribute back to Emacs
  any useful information, so it is with submissions over time

- your graphs are confusing and not common to me. It is not conclusive
  what you wish to present with the graphs such as "How do you use
  Emacs" where you are showing about 6000+ people using it for work
  and 2000+ people using it for studies. Your visual comparison is
  conflicting itself in my opinion as it does not make it conclusive
  if 2000 people among 6000 people use it for studies and for the work
  or only 2000 among 7300 use it for studies. As it is not definitely
  conclusive what you wanted to present I cannot be sure.

- it is good for me to know that people use it for work. But compared
  to the statistics if they use it for programming it looks like
  majority of programmers use Emacs for work. And that is not quite
  clear what "work" means. If it means programming or means something
  else.

- writing? Does it mean writing books, articles, reports, research
  writing is not writing? Programming is not writing? Hard to say what
  is meant with it and what is subset or intersection or union of
  what. I do not mind, as it looks like hobby project. But you may
  maybe draw conclusiong from here.

- the OS type of a question tells me there is much more work to do to
  replace those proprietary OSes with free software, that is useful
  information, thank you

- it is informative how people turn off tool bars and similar.

- question about editors is unfair somehow as it gives impression that
  people would use exclusively one editor. I am using mostly Emacs,
  and will use vi such as nvi version often to edit some files, but
  not to dwell in there, and I am frequently using ed when editing
  configuration files especially when I edit some configuration files
  on remote servers by using eshell or M-x shell trough Emacs. And
  when Emacs is getting crazy, I am using zille and will not hesitate
  using any text editor for notes or even editing Emacs configuration
  file by any editor. Maybe majority uses one true editor and never
  switch, I don't mind. For me is Emacs good as programming
  environment and I could use its features as programming language
  also from outside or within other editor by invoking a command that
  runs Emacs in background. For now I program in Emacs Lisp as many
  things get easier that way. Maybe I switch to something later who
  knows.

- that ivy is mostly used completion package is interesting which
  should make incentive to include it in the Emacs core. But it is not
  as polished.

- now the statistics "Can you list some of your favorite packages"
  where you have placed "other" as the longest item becomes less
  meaningfull because "Other" could be represented in words, such as
  that majority answered "Other" and then the rest you could display
  visually. That way the rest gets it visual meaning. This way, the
  longest item is so long that those named packages are visually not
  easily comparable to each other.

- same comment is valid for themes

- flycheck is not specifically error checking it is spell checking.

- html/css is not a programming language, SQL is a structured query
  language with some functions but it is not a programming
  language. If you wish to mix programming languages and other stuff,
  then why not include all other markup languages as well.

- your Jypiter notebook can most probably be done also in Org
  mode. All the graphs could be also generated in Emacs as well and
  without proprietary external software. Graphviz and dot systems
  could be efficient.

- from all the graphs that deserve to be the pie graph you have placed
  only one "how have you heard about survey" on the end.

Enjoy,
Jean

Re: Emacs User Survey 2020 Results

[Jack Kamm]

> Number one critic to you is that you drive people who use free
> software to non-free software. You maybe have to research why Emacs
> came to be Emacs and why is it named GNU Emacs. There are reasons for
> that. I cannot be soft-hearted on that. When you call it "Emacs
> Survey" why not make it in the spirit of Emacs as free software and
> stop promoting non-free proprietary program such as Matlab. GNU Emacs

matplotlib is not matlab. It is Python.

Re: Emacs User Survey 2020 Results

[Jean Louis]

* Jack Kamm <jackkamm@gmail.com> [2020-12-09 21:25]:
> > Number one critic to you is that you drive people who use free
> > software to non-free software. You maybe have to research why Emacs
> > came to be Emacs and why is it named GNU Emacs. There are reasons for
> > that. I cannot be soft-hearted on that. When you call it "Emacs
> > Survey" why not make it in the spirit of Emacs as free software and
> > stop promoting non-free proprietary program such as Matlab. GNU Emacs
> 
> matplotlib is not matlab. It is Python.

Whatever it is, it was just a time-local variable that now does not
exist on the page. Good move!

Re: Emacs User Survey 2020 Results

[Yuri Khan]

On Thu, 10 Dec 2020 at 00:34, Jean Louis <bugs@gnu.support> wrote:

> - your Jypiter notebook can most probably be done also in Org
>   mode. All the graphs could be also generated in Emacs as well and
>   without proprietary external software. Graphviz and dot systems
>   could be efficient.

You should probably not assume all software other than Emacs is
proprietary. Jupyter Notebook is Free software under BSD license.

Re: Emacs User Survey 2020 Results

[Jean Louis]

[-- Attachment #1: Type: text/plain, Size: 2711 bytes --]

* Yuri Khan <yuri.v.khan@gmail.com> [2020-12-09 21:30]:
> On Thu, 10 Dec 2020 at 00:34, Jean Louis <bugs@gnu.support> wrote:
> 
> > - your Jypiter notebook can most probably be done also in Org
> >   mode. All the graphs could be also generated in Emacs as well and
> >   without proprietary external software. Graphviz and dot systems
> >   could be efficient.
> 
> You should probably not assume all software other than Emacs is
> proprietary. Jupyter Notebook is Free software under BSD license.

Definitely I did not assume and did not say so. It was not related to
proprietary issue. Emacs is specific computing environment that has
Org Babel built-in that may as well do what Jupyter does. But I have
not checked Jupyter well. That is not related to proprietary issue.

We speak of Emacs. Sure we are free to use any free software. But it
is specific context. I would not even be doing charts by nothing else
but Emacs be it SVG image created by Emacs with or the HTML-ized
export from chart-bar-quickie function, or simple text finally, in
similar way how gnuplout outputs it.

The HTML I would create with Emacs of course, and there would be no
need for Javascript, did that before, doing it now as well. It is
quick. The CGI program would be, guess what, in Emacs, and data would
be stored in LISP structure.

Is it extremism?

When doing survey about let us say Python, why would I be creating
images and stuff, calculations and analysis with let us say Perl. Of
course I can do, but somehow does not align well.

I am daily using chart-bar-quickie (nice), to generate graphs from
database. This simple function produces the graph in attachment. Even
if exported as screenshot I would find it fancier and better for
Emacs. I hope there will be more entertainment next time.

(defun hyperscope-statistics-types (&optional limit)
  "Show statistics for most 5+ used hyperdocument types"
  (interactive)
  (let* ((limit (or limit 5))
	 (sql (format "SELECT hlinktypes_name, count(hlinktypes_id) count FROM hlinks, hlinktypes WHERE hlinktypes_id = hlinks_hlinktypes GROUP BY hlinks_hlinktypes, hlinktypes_name ORDER BY count DESC LIMIT %s" limit))
	 (data (rcd-sql-list sql *cf*))
	 (title (concat "Hyperdocument Types Statistics " (rcd-timestamp-date)))
	 (namelst '())
	 (namelst (dolist (i data namelst) (push (car i) namelst)))
	 (nametitle "Hyperdocument Types")
	 (numlst '())
	 (numlst (dolist (i data numlst) (push (cadr i) numlst)))
	 (numtitle "Number"))
    (chart-bar-quickie 'vertical title namelst nametitle numlst numtitle)))

And I use gnuplot to generate statistic graphs such as the one in
attachment.

It can all be done from within Emacs for Emacs environemnt, it would
fancier.

Jean

[-- Attachment #2: 2020-12-20-19:30:51.png --]
[-- Type: image/png, Size: 87581 bytes --]

[-- Attachment #3: sent.png --]
[-- Type: image/png, Size: 11177 bytes --]

Re: Emacs User Survey 2020 Results

[Adrien Brochard]

That is some fancy stuff.

Re: Emacs User Survey 2020 Results

[Göktuğ Kayaalp]

On 2020-12-09 20:22 +03, Jean Louis <bugs@gnu.support> wrote:
> Number one critic to you is that you drive people who use free
> software to non-free software.

Please, please, let we be done with this.  People have the choice as to
what software they decide to use, so let’s not spam people’s inboxes
with this useless tirade, especially if it’ll be in this patronising and
ungrateful manner.

If this person chose to use Matlab, it’s totally their decision.
Software they used on their own device.  Nobody gets to say anything
about that.

Now, the choice of Google forms is not great, but nobody forced anybody
to fill in the survey, and nobody has the right or authority to decide
with which software people get to use the name of Emacs with.  That sort
of language policing is easily fascistic, whatever the yardstick looks
like.  Also, looking over your email again, I see no link to any online
survey tools that could replace the use of Google Forms here.

Also, just because it’s about Emacs, they need not be using Emacs
solutions for everything they do.  If they like Jupyter over Org mode,
that’s it, nobody gets a word to say about it.  Furthermore, there is a
package that lets you control Jupyter notebooks from inside Emacs.
Besides, the data is public.

Overall, this is terrible attitude.

> - html/css is not a programming language, SQL is a structured query
>   language with some functions but it is not a programming
>   language.

Both of those claims are debatable.  The latter _highly_ debatable.

--
İ. Göktuğ Kayaalp / @cadadr / <https://www.gkayaalp.com/>
pgp:   024C 30DD 597D 142B 49AC 40EB 465C D949 B101 2427

Re: Emacs User Survey 2020 Results

[Jean Louis]

* Göktuğ Kayaalp <self@gkayaalp.com> [2020-12-09 23:14]:
> 
> On 2020-12-09 20:22 +03, Jean Louis <bugs@gnu.support> wrote:
> > Number one critic to you is that you drive people who use free
> > software to non-free software.
> 
> Please, please, let we be done with this.  People have the choice as to
> what software they decide to use, so let’s not spam people’s inboxes
> with this useless tirade, especially if it’ll be in this patronising and
> ungrateful manner.

Your comment could be maybe proper in the context in some other
mailing list, maybe on Matlab mailing list, but not on GNU mailing
list where people use free software and promote free software. B

Issue is solved quickly on Adrien's website.

There are various communities and their unions and intersections and
subsets. The union of Emacs community is large and includes all kind
of people, I do not mind what is their relation to proprietary
software. Here we are in the subset community which is on GNU server
where we help and teach people to use free software. Not
proprietary. 

> If this person chose to use Matlab, it’s totally their decision.
> Software they used on their own device.  Nobody gets to say anything
> about that.

When Emacs Survey is promoted here, I will tell that there is Octave
as free software as that is what people should know. Using proprietary
software and promoting it to others is detrimental for safety of
people and in itself unjust. I will always say something about
that. 

> Now, the choice of Google forms is not great, but nobody forced anybody
> to fill in the survey, and nobody has the right or authority to decide
> with which software people get to use the name of Emacs with. That sort
> of language policing is easily fascistic, whatever the yardstick looks
> like.  Also, looking over your email again, I see no link to any online
> survey tools that could replace the use of Google Forms here.

I have already pointed out to Adrien before the survey how that can be
done. I would be happy to write the HTML with the form and I have my
own system to write it. The survey is very simple to implement in
simple HTML which can look nice with some stylesheet. And there are
few free survey options. Adrien could ask for that and would get an
answer on Emacs devel mailing list when he presented it first time.

Yet, there would be no need for complex software, as simple HTML and
simple one program in background on the server could collect
information without problem. Times change and people are new and do
not know that problem have been solved before decades since form.cgi

German quality LimeSurvey is free software:
https://demo.limesurvey.org/index.php?r=admin/authentication/sa/login

Change language here:
https://demo.limesurvey.org/index.php?r=admin/globalsettings#language

> Also, just because it’s about Emacs, they need not be using Emacs
> solutions for everything they do.  If they like Jupyter over Org
> mode, that’s it, nobody gets a word to say about it.  Furthermore,
> there is a package that lets you control Jupyter notebooks from
> inside Emacs.  Besides, the data is public.

Definitely, you are right. I have commented how it would be more
entertaining. There is no must to use this or that. Yet Emacs is
specific case and Emacs survey even more specific. Many using Emacs
are enthusiasts, some more or less. It is nice to see nifty fancy
stuff done with it.

> > - html/css is not a programming language, SQL is a structured query
> >   language with some functions but it is not a programming
> >   language.
> 
> Both of those claims are debatable.  The latter _highly_ debatable.

Could be. I think it was meant to say what all modes are you using, so
it is just small inconsistency. I would prefer for example that
markdown is included, I have not seen anything about that. LaTeX and
TeX are excluded completely. This is because of how design of the
survey have been made.

Jean

Re: Emacs User Survey 2020 Results

[Adrien Brochard]

>> If this person chose to use Matlab, it’s totally their decision.
>> Software they used on their own device.  Nobody gets to say anything
>> about that.
>
> When Emacs Survey is promoted here, I will tell that there is Octave
> as free software as that is what people should know. Using proprietary
> software and promoting it to others is detrimental for safety of
> people and in itself unjust. I will always say something about
> that.

Let's just chill about Matlab please. There was no Matlab involved here.
I'm 100% against Matlab.


>> Now, the choice of Google forms is not great, but nobody forced anybody
>> to fill in the survey, and nobody has the right or authority to decide
>> with which software people get to use the name of Emacs with. That sort
>> of language policing is easily fascistic, whatever the yardstick looks
>> like.  Also, looking over your email again, I see no link to any online
>> survey tools that could replace the use of Google Forms here.

Also no Google forms involved. I used Jotform which is not free software
indeed, but much more compliant (HIPAA for example). But also correct
that nobody was forced and email answers were an option specifically for
that purpose.

> I have already pointed out to Adrien before the survey how that can be
> done. I would be happy to write the HTML with the form and I have my
> own system to write it. The survey is very simple to implement in
> simple HTML which can look nice with some stylesheet. And there are
> few free survey options. Adrien could ask for that and would get an
> answer on Emacs devel mailing list when he presented it first time.
>
> Yet, there would be no need for complex software, as simple HTML and
> simple one program in background on the server could collect
> information without problem. Times change and people are new and do
> not know that problem have been solved before decades since form.cgi

https://emacssurvey.org/faq.html#js

> German quality LimeSurvey is free software:
> https://demo.limesurvey.org/index.php?r=admin/authentication/sa/login

I did look at them. Their hosted version is prohibitively expensive for
the volume we had. And I am not willing to self-host.

Best,
Adrien Brochard

Re: Emacs User Survey 2020 Results

[Jean Louis]

* Adrien Brochard <abrochard@gmx.com> [2020-12-10 00:39]:
> Let's just chill about Matlab please. There was no Matlab involved
> here.  I'm 100% against Matlab.

What I know is that there was link with Matlab, after my comment there
is no such link any more. Maybe it was what it was. Or was not there
at all? If it was not there, then why would I be mentioning it? And
why would you mentioning it back. I do not mind any more as it is
really not there.
  
> > > Now, the choice of Google forms is not great, but nobody forced anybody
> > > to fill in the survey, and nobody has the right or authority to decide
> > > with which software people get to use the name of Emacs with. That sort
> > > of language policing is easily fascistic, whatever the yardstick looks
> > > like.  Also, looking over your email again, I see no link to any online
> > > survey tools that could replace the use of Google Forms here.
> 
> Also no Google forms involved. I used Jotform which is not free software
> indeed, but much more compliant (HIPAA for example). But also correct
> that nobody was forced and email answers were an option specifically for
> that purpose.

So why not make your own HTML? It is very simple to make it especially
when editing in any text editor.

Do you have logical reason to use Javascript? I have not seen
any. Javascript on forms is used when user would dynamically add some
lines of text, let us say additional checkboxes or similar. I have not
seen anything complex in your form. Just ask, I would help you to make
the form right.

From your website:

> Why does the online form use Javascript?
> It is true that it is possible to do an online form with no Javascript
> at all. However, I think that Javascript contributes overall to a much
> better experience. For example, it offers better feedback regarding
> input validation and saves the user session in case of browser
> crash.

I did not see nothing that contribute to better experience. I have
seen simple, not complex form with repetitive check boxes, questions,
etc. All can be done with any editor within 20-30 minutes. One has to
know basic HTML. In general, I must say that I find it funny
entertaining project of you as it is related to Emacs that we like and
use.

Regarding better feedback regarding input validation, we worked with
input validation 15 and 20 years back. Those issues are solved in
majority programming languages handling server side inputs. And if
user turns off javascript, what input validation you get if you rely
on Javascript only? None.

Additionally this problem has been solved in HTML5 since quite some
time, so you may directly validate input with tags, please see here:
https://www.the-art-of-web.com/html/html5-form-validation/

Saving user session if browser crushes? I think one good reason that
browser could crush today is some wrong Javascript, but HTML
hardly. Anyway that browser crushing stuff does not really look as
good justification.

In general I would expect Emacs user survey to be usable with eww, and
probably it was, I did not try it. But that would be good design goal.

> My position is that filling out the survey should be as pleasant as
> possbile. People are choosing to spend their time to do it, we
> should not ask for more if asking for less is possible.

Not impressed on that. Keep your persistence, as I do not mean it
bad. It does not justify nothing. You could at least ask people and I
would give you or even make that for you. 

> Also, a no-Javascript solution means a homemade solution as all big
> online forms platform use Javascript.

What is wrong with homemade solutions? Emacs survey is simple website
and there is no need to compare your website to other websites. It is
successful and would be anyway successful without it.

> And to me that is just not an option at this point. I am web
> developer during the day and I know how much time it would cost to
> build a reliable online survey solution and monitor it during the
> six weeks the survey runs for. That is simply not a price I am
> willing to pay, and I find it a much better deal to turn to an
> online platform that will be safer and much more reliable.

Due to my experience I cannot see why it would cost much. Not more
than maybe few hours of work on which work you have spent probably
days. It is also matter of which programming languages you know or not
know. Maybe Javascript is not so good for server side solution. I have
no idea about Javascript and never used it for anything. But I do know
there is json format and just maybe, by guessing, I think Javascript
could convert the form element data into json structure and submit as
a simple POST. On the server side the POST data could be saved into
simple file. Finished there. Later you just run the conversion program
over all the files to get the csv or what other data for graphical
visualization.

I would not torture me with it. Since two decades I have been using
generic form that accepts any kind of form data and sends by
email. Later I have rewritten it to Common Lisp and save data into
encrypted files. If data is in Lisp or any other structure, it is easy
to iterate over such to form anything else like CSV or even to import
it into SQL database.

Emacs CGI package also exists, it can accept data from online form.

There is Perl CGI, form validators, you name it on CPAN Perl
archive. It supports sessions without end, multi pages forms and so
on, and is simple to setup.

That is why, technically your justifications do not hold. 

I am taking it so that you like and prefer Javascript. That would be
much better explanation. 

> > German quality LimeSurvey is free software:
> > https://demo.limesurvey.org/index.php?r=admin/authentication/sa/login
> 
> I did look at them. Their hosted version is prohibitively expensive for
> the volume we had. And I am not willing to self-host.

You have a website, so you are already self-hosting and you are
developer.

Jean

Re: Emacs User Survey 2020 Results

[chad]

[-- Attachment #1: Type: text/plain, Size: 868 bytes --]

On Thu, Dec 10, 2020 at 12:33 AM Jean Louis <bugs@gnu.support> wrote:

> * Adrien Brochard <abrochard@gmx.com> [2020-12-10 00:39]:
> > Let's just chill about Matlab please. There was no Matlab involved
> > here.  I'm 100% against Matlab.
>
> What I know is that there was link with Matlab, after my comment there
> is no such link any more. Maybe it was what it was. Or was not there
> at all? If it was not there, then why would I be mentioning it? And
> why would you mentioning it back. I do not mind any more as it is
> really not there.
>

There was no mention of Matlab (before the complaint). There was a mention
of matplotlib, which someone mistook as a reference to Matlab, rather than
the reference to Python package that it is.

Whatever the cause of the mistake (read error, parse error, etc.) I hope
the group can let this one go.

Hope that helps,
~Chad

[-- Attachment #2: Type: text/html, Size: 1330 bytes --]

Re: Emacs User Survey 2020 Results

[Göktuğ Kayaalp]

On 2020-12-10 00:39 +03, Adrien Brochard <abrochard@gmx.com> wrote:
> Let's just chill about Matlab please. There was no Matlab involved here.
> I'm 100% against Matlab.
[...]
> Also no Google forms involved. I used Jotform which is not free software
> indeed, but much more compliant (HIPAA for example). But also correct
> that nobody was forced and email answers were an option specifically for
> that purpose.

Yep, sorry for participating in off topic stuff when the center piece is
your great work.  Thanks for doing this, really enjoyed doing the survey
and going through the results.

One thing I really find interesting and maybe troubling is how dominant
r/emacs is.  Personally I generally enjoy r/emacs and r/orgmode, along
with a couple other islands of humanity in the reddit cesspool (and the
odd cool thread on other subs), but IMHO Emacs deserves a better place.

I wonder if you have any insight into whether the reason r/emacs is
represented so heavily is because most your respondents come from
r/emacs or is it because r/emacs has become the de-facto place for Emacs
community.

If the latter, maybe it’d be nice to try to set up something like a
Lemmy [1] instance for Emacs in order to replace that.

[1] https://join.lemmy.ml/


--
İ. Göktuğ Kayaalp / @cadadr / <https://www.gkayaalp.com/>
pgp:   024C 30DD 597D 142B 49AC 40EB 465C D949 B101 2427

Re: Emacs User Survey 2020 Results

[Sacha Chua]

[-- Attachment #1: Type: text/plain, Size: 588 bytes --]

Hello, all!

On Thu., Dec. 10, 2020, 00:40 Göktuğ Kayaalp, <self@gkayaalp.com> wrote:

r/emacs or is it because r/emacs has become the de-facto place for Emacs
> community.
> If the latter, maybe it’d be nice to try to set up something like a
> Lemmy [1] instance for Emacs in order to replace that.
> [1] https://join.lemmy.ml/


Sounds like a good plan. It would be nice to have an alternative way for
people to share, vote on, and discuss links. Please let me know if you get
it going so that I can figure out how to work it into my Emacs News
workflow. :)

Sacha

[-- Attachment #2: Type: text/html, Size: 1070 bytes --]

Re: Emacs User Survey 2020 Results

[Adrien Brochard]

> Yep, sorry for participating in off topic stuff when the center piece is
> your great work.  Thanks for doing this, really enjoyed doing the survey
> and going through the results.
>
> One thing I really find interesting and maybe troubling is how dominant
> r/emacs is.  Personally I generally enjoy r/emacs and r/orgmode, along
> with a couple other islands of humanity in the reddit cesspool (and the
> odd cool thread on other subs), but IMHO Emacs deserves a better place.
>
> I wonder if you have any insight into whether the reason r/emacs is
> represented so heavily is because most your respondents come from
> r/emacs or is it because r/emacs has become the de-facto place for Emacs
> community.

Thank you! I do have a theory formed from watching the data aggregate
over time: there's a big portion of r/emacs because the survey was
advertised there for a long time, forming a "long tail" of submissions.
The post of the survey was pinned at the top of r/emacs for 5 weeks. And
I remember that when the Hacker News post started trending, it brought
in more responses than any other sources. Just that the Hacker News post
disappeared after a day, when r/emacs stayed. We should plot aggregate
submissions over time grouped by source to confirm.

Re: Emacs User Survey 2020 Results

[Adrien Brochard]

> That is surprising result, maybe you remember I was expecting much
> less people to respond. To me that speaks that there may be 7.3
> millions Emacs users minimum, as I consider 1 survey submitted for
> 1000 people who did not submit. This is a vague fact known in media
> such as newspapers. It may not be.

That's interesting. Do you have any literature around that? My quick
thinking on the topic is that Stackoverflow survey consistently reported
4.5% of developers using Emacs over the past years. Latest numbers
indicate 4 million software engineers in the US, so that would be 180000
US software engineer Emacs users. Of course, that's rough given that
there are Emacs users who do not fit that label. Very loose numbers say
21 millions software engineers worldwide, so by the same logic, that's
945000 users. I don't want to go too much into this discussion because
really the data is missing, but it's interesting.


> Number one critic to you is that you drive people who use free
> software to non-free software. You maybe have to research why Emacs
> came to be Emacs and why is it named GNU Emacs. There are reasons for
> that. I cannot be soft-hearted on that. When you call it "Emacs
> Survey" why not make it in the spirit of Emacs as free software and
> stop promoting non-free proprietary program such as Matlab.

I think you misread. I used matplotlib which is a common Python library
to make graphs.

> Same critics remain that you have driven people to non-free
> proprietary Javascript. In the second attempt to make the survey is
> that going to happen again? I hope not.

It depends. I have already expressed my reasons on the topic before
(https://emacssurvey.org/faq.html#js). I am thinking about writing an
entire blog post about lessons learned here.

> - your graphs are confusing and not common to me. It is not conclusive
>    what you wish to present with the graphs such as "How do you use
>    Emacs" where you are showing about 6000+ people using it for work
>    and 2000+ people using it for studies. Your visual comparison is
>    conflicting itself in my opinion as it does not make it conclusive
>    if 2000 people among 6000 people use it for studies and for the work
>    or only 2000 among 7300 use it for studies. As it is not definitely
>    conclusive what you wanted to present I cannot be sure.


This question allowed multiple choice answers which makes graphing it
always a bit tricky. I absolutely encourage you to look at the data and
answer your question. As I stated at the top of the results page, this
is a simple per-question analysis. I could have spent months looking at
the data under every angle.


> - now the statistics "Can you list some of your favorite packages"
>    where you have placed "other" as the longest item becomes less
>    meaningfull because "Other" could be represented in words, such as
>    that majority answered "Other" and then the rest you could display
>    visually. That way the rest gets it visual meaning. This way, the
>    longest item is so long that those named packages are visually not
>    easily comparable to each other.
>
> - same comment is valid for themes

Yes free text analysis on "long-tail" data is particularly difficult. I
have mentioned it at the top of the results page.


> - flycheck is not specifically error checking it is spell checking.

It is. https://www.flycheck.org/en/latest/
You might confuse it for flyspell.


> - your Jypiter notebook can most probably be done also in Org
>    mode. All the graphs could be also generated in Emacs as well and
>    without proprietary external software. Graphviz and dot systems
>    could be efficient.

It probably can but data analyst are more comfortable with Jypiter.

> - from all the graphs that deserve to be the pie graph you have placed
>    only one "how have you heard about survey" on the end.

That's actually not true. Pie graphs are good when the question is
single choice answer. Most of the questions were multiple choice, which
means that a pie graph would be confusing and the trick viewers into
thinking that a user can only belong to one of the slices of the pie.
Bar charts are not perfect, but they seem to reduce that risk. Maybe I
could have also tried a bubble chart

Thank you for the feedback. If there is another iteration, I will use it
to improve.
Best,
Adrien Brochard

Re: Emacs User Survey 2020 Results

[Jean Louis]

* Adrien Brochard <abrochard@gmx.com> [2020-12-10 00:31]:
> 
> > That is surprising result, maybe you remember I was expecting much
> > less people to respond. To me that speaks that there may be 7.3
> > millions Emacs users minimum, as I consider 1 survey submitted for
> > 1000 people who did not submit. This is a vague fact known in media
> > such as newspapers. It may not be.
> 
> That's interesting. Do you have any literature around that?

It comes from "Letters to editor" as single reader of a newspaper
writing such a letter was counted as 1000 other people. Cannot find
references. You could ask some professional newspapers how they regard
letters to editor. 

> My quick thinking on the topic is that Stackoverflow survey
> consistently reported 4.5% of developers using Emacs over the past
> years. Latest numbers indicate 4 million software engineers in the
> US, so that would be 180000 US software engineer Emacs users. Of
> course, that's rough given that there are Emacs users who do not fit
> that label. Very loose numbers say 21 millions software engineers
> worldwide, so by the same logic, that's 945000 users. I don't want
> to go too much into this discussion because really the data is
> missing, but it's interesting.

If they would say how many users did not answer the survey then you
could get some ratio to estimate better final numbers.

> > - your graphs are confusing and not common to me. It is not conclusive
> >    what you wish to present with the graphs such as "How do you use
> >    Emacs" where you are showing about 6000+ people using it for work
> >    and 2000+ people using it for studies. Your visual comparison is
> >    conflicting itself in my opinion as it does not make it conclusive
> >    if 2000 people among 6000 people use it for studies and for the work
> >    or only 2000 among 7300 use it for studies. As it is not definitely
> >    conclusive what you wanted to present I cannot be sure.
> 
> This question allowed multiple choice answers which makes graphing it
> always a bit tricky. I absolutely encourage you to look at the data and
> answer your question. As I stated at the top of the results page, this
> is a simple per-question analysis. I could have spent months looking at
> the data under every angle.

Programming is your helper. When you design it well from ground up you
never think again about it. I have been doing forms since
decades. Then I re-wrote the basic program 1-2 times in different
programming language.

There are good libraries available from Perl world. This page should
give you good insights to think about:

https://www.perlmonks.org/bare/?node_id=383250

Quotes:

,----
| Without knowing all the details, I'd say your best bet would be to
| validate the user input via JavaScript. Since JavaScript is
| client-side, you won't need to send the user input to the server and
| then back again if it's incorrect.
`----

where he gets the answer:

,----
| I'd say your best bet would be to validate the user input via JavaScript
| 
| Bad advice. Client side validation is fine, provided you accept that
| it is simply to improve the end user experience. It offers absolutely
| no security or reliable validation
| 
| Form validation is a server side task. Period. Do what you like on the
| client side, you still need to validate everything (again if using JS
| client side) on the server side.
`----

But that was 2004, today you have HTML5, you can validate pretty much
without Javascript on client side. Attempting to validate on client
side only would lead to insecurities. Using proprietary software even
more. 

> > - now the statistics "Can you list some of your favorite packages"
> >    where you have placed "other" as the longest item becomes less
> >    meaningfull because "Other" could be represented in words, such as
> >    that majority answered "Other" and then the rest you could display
> >    visually. That way the rest gets it visual meaning. This way, the
> >    longest item is so long that those named packages are visually not
> >    easily comparable to each other.
> > 
> > - same comment is valid for themes
> 
> Yes free text analysis on "long-tail" data is particularly difficult. I
> have mentioned it at the top of the results page.

Myself I do not use polls to analyze it for pure reason of analyzing
it. Rather to find out what is needed and wanted by majority and to
improve it. It can be a bridge that has to be repaired or school or
medical problem in some rural village. What starts cominng would would
be relevant. Analying it by percentages would not be useful for me.

If you say free form, then this is not really for analysis. Probably
it was not analyzed.

For accepting large number of users inputs I would use SQL
database. Those free form text can also be inserted into SQL
database. Its rigid data definition can equally well take care that
data too long cannot be inserted. Then comes the best part as
PostgreSQL at least supports full text relevance search.

For me it means that one could make one SQL query and find or group
relevant columns of data together. That would give the tabularized
summary.

Even better would be to have a survey running all the time where users
could just click to get the new fresh results be it tabularized or
graphical. 

> > - flycheck is not specifically error checking it is spell checking.
> 
> It is. https://www.flycheck.org/en/latest/
> You might confuse it for flyspell.

Thank you, did not know that.

> > - your Jypiter notebook can most probably be done also in Org
> >    mode. All the graphs could be also generated in Emacs as well and
> >    without proprietary external software. Graphviz and dot systems
> >    could be efficient.
> 
> It probably can but data analyst are more comfortable with Jypiter.

Does that information come from another survey?

Look at this query:
https://duckduckgo.com/?q=jupyter+vs+org+mode&atb=v154-3am&ia=web

Look at this result:
https://www.slant.co/versus/4411/15716/~emacs-org-mode_vs_jupyter

Rant:
https://rgoswami.me/posts/jupyter-orgmode/

Anyway, it does not matter. Personal preferenes. I would find it more
entertaining if all things were done with Emacs.

What we do with Emacs in Uganda nearby Bwindi Impenetrable Forest, we
track people's complaint, work, make police reports, we track stolen
goats, Org tables are used to calculate salaries and people look at
them and actually understand it. I am using Calc to calculate mineral
values. Common people watch and find it understandable. All videos and
images are converted, resized, managed by using Emacs as high level
language. All the 4000 pages of various websites are managed through
Emacs editing the PostgreSQL database entries. I know Perl, but if I
promote Perl I would recommend tools and stuff around and within Perl
environment. If Emacs, then I would recommend that within Emacs
environment. 

> > - from all the graphs that deserve to be the pie graph you have placed
> >    only one "how have you heard about survey" on the end.
> 
> That's actually not true. Pie graphs are good when the question is
> single choice answer. Most of the questions were multiple choice, which
> means that a pie graph would be confusing and the trick viewers into
> thinking that a user can only belong to one of the slices of the pie.
> Bar charts are not perfect, but they seem to reduce that risk. Maybe I
> could have also tried a bubble chart

That was one of possible options to understand it. But it is not
conclusive as there is no description on what you mean with
visualization.

Example:
https://emacssurvey.org/2020/how-would-you-characterize-your-use-of-emacs.png

It is now even less conclusive after your explanation if this means
one of following that 2500 people said "I use it for work" and also "I
use it for studies". As those doing study do not work yet. So the
visualization is missing the point to clearly tell the website visitor
what is the real result.

In that case graph alone is doing nothing but confusion. You would
rather say

- 65% of users said to use it for work
- 75% of users said to use it for studies

as those would be intersections of the union of results.

Example of more confusion:
https://emacssurvey.org/2020/for-how-many-years-have-you-been-using-emacs.png

Frequency? Frequency of what? Do you mean number of people? It is
interesting that some report using it over 40 years.

Another example:
https://emacssurvey.org/2020/which-os-do-you-primarily-use-emacs-on.png

Does that mean that 500 users use Windows and Mac and GNU/Linux
together how you explained it?

About 100 of them use other OS which they do not know which one it is,
Windows, Mac, GNU/Linux and BSD together. The logic is not consistent
with the explanation you gave.

Another example:
https://emacssurvey.org/2020/prior-to-using-emacs-what-was-your-primary-editor.png

Prior to using Emacs what was your primary editor?

- almost 1500 users answered NONE

- more than that answered OTHER which is is not consistent with your
  explanation, and not consistent with other 1500 users who used NONE
  editor before

- and not consistent to those 1500 users of vim, as it cannot be NONE
  and VIM and OTHER all together

About Org mode, that is also not how you explain, as than 1500 people
have said "I do not use Org mode" but among those 1500 people there
are those 1500 who "use it daily" -- sorry makes no sense.

Now all of those results are interesting and entertaining in the same
time. There may be some use of it. Comparing all the results to some
few bug reports from last weeks that have been pinpointed and handled
by Emacs developers, I still find the reports and their solutions so
much more remarkable and efficient. The survey itself, while
interesting, being conclusive or not conclusive, is not practically
useful.

And in regards to being representative, I would say you have got more
than representative data. With 1000 people would be already enough
to have it representative. You could and should say that you have
representative data. Congratulation.

But the visualization is wrong. It is not conclusive and is confusing
obviously.

There are 2 major things I see there missing and find the survey
wasted time and effort for nothing but few insights that could be also
by obtained faster and simpler way.

Those two things are: the objective or purpose and usability as one
possible purpose.

The objective for any survey, in my opinion, should be to improve
conditions. In this case to improve Emacs. But people were not asked
what they wish to improve. It seem that objective was just to make a
survey. That is why I say it is entertaining but less useful then
several solved bugs from last days on the bugs mailing list.

Usability I talk about is well explained here:
https://www.nngroup.com/articles/usability-101-introduction-to-usability/

If you read more from Nielsen you may also find that one need not have
more than 5 users to observe their habits and to find out what is
wrong and what can be improved.

Yet among 7300+ participants no question was answered in regards on
what they wish to improve in Emacs, what is their major problem, or
where usability could be improved.

I do hope you will make a checklist of those items you find yourself
reasonable and that it gives better more usable results in future.

Jean

P.S. You said before the surve you would give the unchanged raw data
back to public. I have signed my information with PGP key and sent it
to you. Now I cannot find my PGP information "raw" and for that reason
myself I do not find it more than funny time, nothing serious. When
you say to publish it all, then why not really do it.

Re: Emacs User Survey 2020 Results

[Qiantan Hong]

[-- Attachment #1: Type: text/plain, Size: 1011 bytes --]

Hi Adrien,

Seems that in the programming language chart,
the data processing script somehow recognize “common lisp”
as two languages “common” and “lisp"

> On Dec 9, 2020, at 11:40 AM, Adrien Brochard <abrochard@gmx.com> wrote:
> 
> Hi everyone,
> 
> After a week of reading every submission, cleaning up the data, and
> leaning matplotlib, I finally have enough confidence to publish the
> results of the Emacs User Survey 2020.
> 
> https://emacssurvey.org/2020/
> 
> I want to thank everyone who responded, commented, and shared it!
> There's over 7300 responses and it's really thanks to this amazing
> community.
> 
> There is still a lot to do, the data could always be analyzed
> differently, the website could be nicer, etc, but the responses have
> been so overwhelmingly positive that I just have to publish without more
> delay. If you have feedback or feel like contributing, all source and
> data are public.
> 
> Thank you again!
> 
> Adrien Brochard
> 


[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: Emacs User Survey 2020 Results

[Lars Ingebrigtsen]

Adrien Brochard <abrochard@gmx.com> writes:

> After a week of reading every submission, cleaning up the data, and
> leaning matplotlib, I finally have enough confidence to publish the
> results of the Emacs User Survey 2020.
>
> https://emacssurvey.org/2020/

Thanks a lot for doing this -- there's a bunch of interesting results
here.

Things I'm surprised by:

* How many people prefer daemon + emacsclient

* That the GUI is only used by 3-4x as many people as the terminal

* I'm not surprised that virtually everybody disables the toolbar, but
I'm a bit surprised that so many people disable the scroll bar (I do,
too).

* That the default keybindings are just 2x as popular as the vim-a-like
key bindings.  This one is really surprising to me, because we seldom
get bug reports about things in that corner.  But perhaps they just work
perfectly?

* That ~two thirds of people who responded uses magit.  I knew it was
popular, but that's astounding.

Now, no surveys are representative, but it's interesting.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: Emacs User Survey 2020 Results

[Jean Louis]

* Lars Ingebrigtsen <larsi@gnus.org> [2020-12-09 23:59]:
> * That the default keybindings are just 2x as popular as the vim-a-like
> key bindings.  This one is really surprising to me, because we seldom
> get bug reports about things in that corner.  But perhaps they just work
> perfectly?

Bug report requires specific mind set and some experience, it also
requires English, courage, not being shy, etc. Non English speaking
people will rather skip it. Users may not know what is bug even if
they see it. Only a small subset of users are sending bug reports.

I would count for ever bug report other 1000-10000 people affected.

Re: Emacs User Survey 2020 Results

[Gregory Heytings via Emacs news and miscellaneous discussions outside the scope of other Emacs mailing lists]

>
> * That the GUI is only used by 3-4x as many people as the terminal
>

In fact the results show that almost everyone use the GUI (~7000 on ~7300) 
and among them some also use the TUI (~2000).

>
> * That the default keybindings are just 2x as popular as the vim-a-like 
> key bindings.  This one is really surprising to me, because we seldom 
> get bug reports about things in that corner.  But perhaps they just work 
> perfectly?
>

Amusingly the first comment on Reddit says almost the opposite: "I'm 
surprised [...] There are almost twice as many default keybinding users 
are Evil users.  I feel like I read about so many people using Evil 
keybindings that I imagined it would be a lot more." ;-)

>
> * That ~two thirds of people who responded uses magit.  I knew it was 
> popular, but that's astounding.
>

I'm not sure how to interpret this, Magit is a "favorite package" for 
~2500 respondents, but ~5000 respondents use it.  Does that mean that half 
of its users do not like it?

Re: Emacs User Survey 2020 Results

[Samuel Wales]

it's possible that few bug reports are done in a particular area
because they seem unlikely to be done.

On 12/9/20, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> key bindings.  This one is really surprising to me, because we seldom
> get bug reports about things in that corner.  But perhaps they just work
> perfectly?

-- 
The Kafka Pandemic

Please learn what misopathy is.
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

Re: Emacs User Survey 2020 Results

[Samuel Wales]

(as an analogy, if you go to a doctor who does not treat you, you will
not go back to that doctor.  but the doctor might think that you were
cured because you did not go back.  this is a serious problem in
medicine.)

On 12/10/20, Samuel Wales <samologist@gmail.com> wrote:
> it's possible that few bug reports are done in a particular area
> because they seem unlikely to be done.
>
> On 12/9/20, Lars Ingebrigtsen <larsi@gnus.org> wrote:
>> key bindings.  This one is really surprising to me, because we seldom
>> get bug reports about things in that corner.  But perhaps they just work
>> perfectly?
>
> --
> The Kafka Pandemic
>
> Please learn what misopathy is.
> https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html
>


-- 
The Kafka Pandemic

Please learn what misopathy is.
https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html

Re: Emacs User Survey 2020 Results

[Eli Zaretskii]

> From: Samuel Wales <samologist@gmail.com>
> Date: Thu, 10 Dec 2020 14:30:01 -0700
> Cc: emacs-tangents@gnu.org, Adrien Brochard <abrochard@gmx.com>
> 
> (as an analogy, if you go to a doctor who does not treat you, you will
> not go back to that doctor.  but the doctor might think that you were
> cured because you did not go back.  this is a serious problem in
> medicine.)

But since we are not talking about one person, but many different
persons, it is still surprising that we don't get such bug reports,
because surely some of these persons may not know or assume up front
that these problems are unlikely to be fixed.  It isn't like there's a
secret cabal of people who spread the idea that these problems are
unlikely to be fixed and therefore shouldn't be reported.  The law or
large numbers should have won here.  It is surprising that it didn't.

Re: Emacs User Survey 2020 Results

[Jean Louis]

* Eli Zaretskii <eliz@gnu.org> [2020-12-11 11:01]:
> > From: Samuel Wales <samologist@gmail.com>
> > Date: Thu, 10 Dec 2020 14:30:01 -0700
> > Cc: emacs-tangents@gnu.org, Adrien Brochard <abrochard@gmx.com>
> > 
> > (as an analogy, if you go to a doctor who does not treat you, you will
> > not go back to that doctor.  but the doctor might think that you were
> > cured because you did not go back.  this is a serious problem in
> > medicine.)
> 
> But since we are not talking about one person, but many different
> persons, it is still surprising that we don't get such bug reports,
> because surely some of these persons may not know or assume up front
> that these problems are unlikely to be fixed.  It isn't like there's a
> secret cabal of people who spread the idea that these problems are
> unlikely to be fixed and therefore shouldn't be reported.  The law or
> large numbers should have won here.  It is surprising that it didn't.

When I was using Emacs on Redhat Linux back in 1999, if I remember
well it was crashing in each session. And I have never reported those
many many bugs neither to Redhat neiter to Emacs development. There
were various versions of Emacs and I coped with it. Often I used other
editors.

Then when I started using Debian GNU/Linux I have not been reporting
bugs probably for few years. I have just assumed it is not working
well, but that I could do something about it was not clear. There will
be whole series of users not being aware of it.

Then later I have been reporting all Debian GNU/Linux related bugs
only to Debian, straight there, and nowhere else. I have not even know
that there is some central way to report bug. I remember that under
Help there was menu item to report Emacs bug, but I have not used that
interface, rather Debian interface. Maybe maintainers reported to
upstream.

Long time passed until I started reporting to Emacs and this is
probably because I switched to Hyperbola GNU/Linux-libre and somebody
directed me. Otherwise I was reporting to https://www.hyperbola.info
anything related to Hyperbola packages as I was used from Debian this
way.

Look:

https://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=unstable;package=emacs

https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=emacs-common

https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=emacs-el

https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=emacs-gtk

https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=emacs-lucid

https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=emacs-nox

https://bugs.debian.org/cgi-bin/pkgreport.cgi?package=emacs25

https://issues.hyperbola.info/index.php?string=emacs&project=0&do=index&type%5B%5D=&sev%5B%5D=&pri%5B%5D=&due%5B%5D=&reported%5B%5D=&cat%5B%5D=&status%5B%5D=open&percent%5B%5D=&opened=&dev=&closed=&duedatefrom=&duedateto=&changedfrom=&changedto=&openedfrom=&openedto=&closedfrom=&closedto=

https://bugs.archlinux.org/index.php?string=emacs&project=1&search_in_comments=1&search_in_details=1&search_for_all=1&type%5B%5D=&sev%5B%5D=&pri%5B%5D=&due%5B%5D=&reported%5B%5D=&cat%5B%5D=&status%5B%5D=open&percent%5B%5D=&opened=&dev=&closed=&duedatefrom=&duedateto=&changedfrom=&changedto=&openedfrom=&openedto=&closedfrom=&closedto=&do=index

https://bugs.launchpad.net/ubuntu?field.searchtext=emacs&search=Search&field.status%3Alist=NEW&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.assignee=&field.bug_reporter=&field.omit_dupes=on&field.has_patch=&field.has_no_package=