From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Jean Louis Newsgroups: gmane.emacs.devel Subject: Re: Emacs Survey: Toolbars Date: Tue, 22 Dec 2020 17:02:11 +0300 Message-ID: References: <87o8iv3ac3.fsf@gnus.org> <877dpjp30g.fsf@ucl.ac.uk> <87zh2fnmwq.fsf@gnus.org> <87o8ivumn5.fsf@telefonica.net> <87v9d3nkxk.fsf@gnus.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="7270"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Mutt/2.0 (3d08634) (2020-11-07) Cc: Christopher Dimech , emacs-devel@gnu.org To: Gregory Heytings Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Dec 22 15:07:43 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kriKE-0001m5-Pn for ged-emacs-devel@m.gmane-mx.org; Tue, 22 Dec 2020 15:07:42 +0100 Original-Received: from localhost ([::1]:60348 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kriKD-0006FO-6x for ged-emacs-devel@m.gmane-mx.org; Tue, 22 Dec 2020 09:07:41 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:46712) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kriIw-0004yO-D7 for emacs-devel@gnu.org; Tue, 22 Dec 2020 09:06:22 -0500 Original-Received: from stw1.rcdrun.com ([217.170.207.13]:41165) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kriIl-00008Z-7t for emacs-devel@gnu.org; Tue, 22 Dec 2020 09:06:18 -0500 Original-Received: from localhost ([::ffff:41.202.241.37]) (AUTH: PLAIN securesender, TLS: TLS1.2,256bits,ECDHE_RSA_AES_256_GCM_SHA384) by stw1.rcdrun.com with ESMTPSA id 0000000000012043.000000005FE1FD4E.0000609B; Tue, 22 Dec 2020 07:06:06 -0700 Content-Disposition: inline In-Reply-To: Received-SPF: pass client-ip=217.170.207.13; envelope-from=bugs@gnu.support; helo=stw1.rcdrun.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:261486 Archived-At: * Gregory Heytings via "Emacs development discussions. [2020-12-22 13:59]: > > Christopher Dimech: > > > > Shitty data obtained without care, attention, and skill is useless data. > > Forget the weighting! > > > > As the saying goes: "Criticism is easy, and art is difficult." Could you > please create and conduct a "good" survey, according to your criteria of > "good"? Or at least enlighten us, poor mortals, and explain what should > have been done, and how? Data may be obtained with care, attention and skill and still be misleading. That there are good intentions with the survey, I have no doubts. On how results were presented and how survey has been conducted I have no doubt that author lacks the skill on the specific subject of making an opinion poll. I have no doubts that author of the survey did conduct the survey, connected websites together and made some marketing for people to place their polls. And I have no doubt on his other skills, programming, using mathematics, and I did not review the way of counting, but I would not doubt on the manner of counting statistics at this moment. Emacs survey has a lot of misleading information that was already mentioned here. Major misleading information is the difference between general survey and specific survey that relates to a subset of people from specific groups of people. Then we come here to discuss the toolbar that only a subset of people from specific groups of people said something about that. I do not agree that "any data is better than no data". The information may be useful but due to its presentation it may be even more misleading than useful. When one says it is "Emacs Survey" I do not see it and do not understand it by such general term. As one may see on this picture of the survey in question this survey represents 54.1% of people comming exclusively from Reddit and Hacker News, then comes 7.8% Twitter. Majority of people 61.9% answering the survey are groups coming from Reddit, Hacker news and Twitter users. https://emacssurvey.org/2020/how-did-you-hear-about-this-survey.svg Better would be to say that it is Emacs Survey that represents almost 62% of Emacs users coming from Reddit, Hacker News and Twitter. One may see here that 1.4% of users answered the survey by email: https://emacssurvey.org/2020/submissions-medium.svg There are 7344 total responses. 1.4% of 7344 is 102+ users (more than 102 due to floating point). Maybe those users did not want to submit the online form as they did not want to use non-free Javascript? But maybe those users did not want to submit as they wanted to express themselves better by using the Org file sent by email? We do not know why, but we can assume that 1.4% did not want to submit the form online. We may compare data, we could take the Debian popularity contest: https://qa.debian.org/popcon.php?package=popularity-contest where recent data says that 200504 people installed the package `popularity-contest' that counts which other packages are installed on the system. Then we may look into the subset of those 200504 Debian users who installed Emacs: https://qa.debian.org/popcon.php?package=emacs To me it looks like about 14,000 users installed Emacs on their Debian GNU/Linux system. That amounts to some 6.98% of 200504 users who use the Debian popularity contest package. As Ubuntu comes from Debian GNU/Linux some vague information says there are 20 million Ubuntu users. If we would follow the trend of 6.98% let us truncated it to 6% of people using Emacs on Debian derivative distribution, that would amount already to (* 20000000 0.06) 1,200,000 people using Emacs in Ubuntu. https://ubuntu.com/blog/ubuntu-is-everywhere maybe this number is larger or bigger, but it does give some traces on how to conclude how many people are using Emacs. This page says there must be 100+ millions of GNU/Linux users on Desktop (not counting Android): https://www.quora.com/How-many-Linux-users-are-there-in-the-world/answer/Juha-Kaskiharju but I cannot know where and how information has been obtained: https://www.netmarketshare.com/linux-market-share If we do assume there are 100+ millions GNU/Linux users who use their browser as that is how companies detect such users and that 6.98% trend as established from Debian popularity contest uses Emacs than that may amount to 6-7 millions of Emacs users. If we now come back to those 1.4% that did not want to answer the survey by using non-free Javascript, maybe the real amount of those people, who did not see the survey, but would not answer the WWW survey for some of not unknown reasons, is already (* 6980000 0.014) 97720 users. Emacs survey of 62% of Emacs users coming from Reddit, Hacker News and Twitter has shown that that subset of users being about 4340 users would gladly answer the survey over WWW form, with or without considering if they use free or non-free Javascript. I am personally, not by mathematical estimation, thinking that 90,000 of Emacs users and more than that, would not answer the survey only because of the non-free Javascript or because how survey was conducted. Now let us say we discuss of toolbars, we do not discuss of toolbars for Emacs users. We discuss of the fact that 85.1% of users using Reddit, Stackexchange and Twitter who answered the Emacs survey that relates to 62% to those groups are disabling the toolbar. MacOS users from the Emacs survey are 26.6% plus Windos 7.4%, total being 34% among all Emacs survey users or 34% from 7000 amounting to some 2380 users. When considering then if toolbar shall be disabled, shall we also consider are we disabling it for majority of GNU/Linux users, or for majority of proprietary system users, or for those Emacs users who never answered the survey due to not unknown reasons which may amount to almost 100,000 but which user habits we do not know. In general the survey results would be much more useful would we have intersections of people by various characteristics. We do not have intersections, though it could be possible to make such intersections with the data at hand. Then survey did not give the result if users would like by default the toolbar to be disabled. As when user disables the toolbar it does not mean user would like it by default to be disabled upon first installation. The Emacs survey we speak about is not general Emacs survey. That is in my opinion major misconception that people will get as it all sounds so much official with nice and shiny charts. It is Emacs survey that represents for 62% those users coming from Reddit, Stackexchange, Twitter and we do not know many of intersection results as such are not presented on the website. If one only looks at one chart, like on chart among Emacs Survey representing 62% users from Reddit, Stackexchange, Twitter, then one may draw conclusions which are counter useful to the real number of Emacs users. What would be useful to know for development could be interesections of: - users using Windows, using Emacs with daemon mode - users using MacOS, using Emacs with daemon mode - users using GNU/Linux, using Emacs with daemon mode - users using Windows, using Emacs as GUI or terminal - users using MacOS, using Emacs as GUI or terminal - users using GNU/Linux, using Emacs as GUI or terminal - users using Windows, disabling toolbar - users using MacOS, disabling toolbar - users using GNU/Linux, disabling toolbar then again intersections for menu bar, splash screen, etc. Taking just one chart and looking how 85.1% users disabled toolbar is incorrect interpretation that there is something to be handled as that data comes from specific groups of people. One cannot even ask experienced user if the tool bar should be disabled as that is misleading. One could only ask a very new user who has never seen Emacs, by placing that user on the chair and showing him Emacs interface and then asking him if the toolbar is useful or not useful. Make that a first question to 10 people who never used Emacs and then see the result and compare it. I guess that 10 people would be enough, Nielssen says 5 people are enough, and I understand his methodology. It is thus in my opinion more productive and more useful to take those 5 people or make 3 groups or 5 groups of 5 people and place them in front of Emacs to tell us if they find toolbar useful. It is counter productive to make assumptions to disable toolbar because users who already used Emacs probably for years disabled the toolbar. They are not new users. While they may tell that disabled toolbar for them is good, they cannot tell that for new users for which toolbar would be disabled by default, they are not qualified for that as they are not new users any more. Jean