From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id hQtuOJ8cv1/sVQAA0tVLHw (envelope-from ) for ; Thu, 26 Nov 2020 03:10:23 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id uC+8M58cv1+FHgAAB5/wlQ (envelope-from ) for ; Thu, 26 Nov 2020 03:10:23 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 171879404FE for ; Thu, 26 Nov 2020 03:10:23 +0000 (UTC) Received: from localhost ([::1]:46034 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ki7fp-0005hj-T8 for larch@yhetil.org; Wed, 25 Nov 2020 22:10:21 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:48040) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ki7ew-0005fV-4H for emacs-orgmode@gnu.org; Wed, 25 Nov 2020 22:09:26 -0500 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]:34084) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ki7es-0007IG-OY for emacs-orgmode@gnu.org; Wed, 25 Nov 2020 22:09:25 -0500 Received: by mail-pf1-x430.google.com with SMTP id w6so369582pfu.1 for ; Wed, 25 Nov 2020 19:09:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version; bh=a0BuK0EDMaRldPWs5YqljRGXaVOE3ahfYJrZsWKCbI8=; b=DwfT4ftp4q6OmOLvnQERR9Yzqtw2wwmtyAzTcaRQHd1RiwPNOyf3f7XinGdwS/Iu0R FlE52f4Jnsxmwg1EOd9qyq6m4Byj4xCK4cf7jN5SCPW8c+/rG9QIc4wJK42PpD2wX71/ m3n2oamMlsfQ/JKLbd65hAkSvVnGItWVkNaNA6zD3CGg3FMlvBhaYKeJ1wO+OuNi2zLU yn+YUS2/1l2QGcmGlQodtD/Yc9Q4ZPdpzF7h4Iq4vGVfsCHF2CXBhHdQ3DPdb7lm+nwx +itIlEcOOXUbH1KXgXadHEFkicWL83dVdKRPWUPMqIo2kkuBaBPvA7VvNpovoY8fyGzy /umw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version; bh=a0BuK0EDMaRldPWs5YqljRGXaVOE3ahfYJrZsWKCbI8=; b=MUOUHnwmCbBUbQ1qunj6sHoRfGL/1FDA9ZGLEcW2stFUSK05gHCe6qIVKSIwcmpiWL PAqN8kkRDitS21O9Etqy2gxAZipKshNslDRO71D91Y4adoX7gML6PmL/5vIvTxC8rlrx GsL7PxPJj2YmyNupyUB7kAx0ZpBXNtNH5zsSOES1hNLydZTYIyJPgBUbH7zKW7SnpYDU mGurt6e4Y6j5DbB670czjKiYFMymlhwbZaMDpp7xQu/Vm+tEVLW5lMkc6LI9oOyybSvn v3KBQtAc0NAyDzRNZPZvdbV0dZAE/9TUkpc2Gy0YrnksZZKJOd0CdNRN0C6OIBuhe3bf P8sg== X-Gm-Message-State: AOAM530GXEDloAejHmq41fz3qPsz5j4zupnrx2xxRwM0o9/M8+aMLIuP 8XwiV+ZVFmUcWZZbyDhXUrg= X-Google-Smtp-Source: ABdhPJz7Hyz0VVCjlnbaA1+C+PTVRVBQsE/+d5wWpYSejSylBE7MsBrn28zWLZ5d/2N0i9fWSlIVJQ== X-Received: by 2002:a62:5b05:0:b029:197:fafb:50f3 with SMTP id p5-20020a625b050000b0290197fafb50f3mr905842pfb.76.1606360160700; Wed, 25 Nov 2020 19:09:20 -0800 (PST) Received: from localhost ([104.250.131.79]) by smtp.gmail.com with ESMTPSA id f7sm3008146pfe.30.2020.11.25.19.09.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 25 Nov 2020 19:09:19 -0800 (PST) From: Ihor Radchenko To: Jean Louis Subject: Re: Is Org really so simple? In-Reply-To: References: <87y2ive1i4.fsf@localhost> <878sauhhv1.fsf@web.de> <875z5ygwwr.fsf@web.de> <87r1olfvh4.fsf@web.de> <878sascg5j.fsf@localhost> Date: Thu, 26 Nov 2020 11:08:15 +0800 Message-ID: <877dq8bysg.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=2607:f8b0:4864:20::430; envelope-from=yantar92@gmail.com; helo=mail-pf1-x430.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Dr. Arne Babenhauserheide" , Texas Cyberthal , "emacs-orgmode@gnu.org" Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Migadu-Flow: inc X-Scanner: ns3122888.ip-94-23-21.eu Authentication-Results: aspmx1.migadu.com; dkim=fail (headers rsa verify failed) header.d=gmail.com header.s=20161025 header.b=DwfT4ftp; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Spam-Score: 3.09 X-TUID: 1xKTFSKtf24p > Only philosophy I know is that it is plain text. Is there any official > philosophy? I have no idea, at least manual does not give me > references. I cannot find "philosophy", send me references. You are right. There is no official "philosophy" in org. In my reply I tried to follow the topic starter's view: Texas Cyberthal : > By philosophy, I mean the dev consensus on the correct way to do > things, and coded configuration and usability biases. According to my experience with org-mode development (I am not talking about third-party packages here), it is discouraged to change org-mode towards hiding metadata in org files or store *unique* data (that cannot be derived from the contents of the org files) related to org-mode externally (not in org files). It is not official statement, but rather my impression so far. > Question is what is meant by database, it can be anything. One can > save LISP data. Recent files, desktop, eww bookmarks, init.el or > .emacs file are also all similar databases, there is the underused > EIEIO with persistent stuff that represent built-in database > functionality. Org-mode does not limit user customisations. It does not limit third-party packages either. Users are free to use any other tools, store their data anywhere other than org files (why would org force users to do otherwise?). What I referred to earlier is just the core org-mode development. > And people try to do exactly that, people are developing Org in the > manner to relate objects in Org file to anything else. And they do > hard work as they do it manually. Relational database speeds up and > does not tell user to manually hyperlink various relations. Could you provide some more examples? I do not see how relational database is different from creating hyperlinks in org. Either way, the user needs to file an object/headline somewhere into org file, which is inherently manual. > I see hard work by many people who try to enhance Org as hierarchical > knowledge of data because people want to have feature X, but group of > those enhancements in reality belong to relational databases and not > to text files. It is an interesting point. I would be happy if some existing tools could be reused instead of re-inventing the wheel for org. Do you have concrete examples where it can be useful? If you have, I encourage you to bring up a feature request to discuss this possibility with org-mode devs. Best, Ihor Jean Louis writes: > * Ihor Radchenko [2020-11-23 17:18]: > :PROPERTIES: > :CREATED: [2020-11-23 Mon 18:42] > :ID: edebb3e7-e755-4ecc-a5e8-a3353a3f5fd0 > :END: >> Dear Jean Louis, >> >> Your description of the database reminds me how org-roam handles the >> files - it also uses an external database for linking and allows quick >> incremental search that does not really depend on where the >> files/headings are stored. > > Sounds good, I can see there is graph database used. > >> However, what you are talking about is against org-mode philosophy, >> as I know it. > > Only philosophy I know is that it is plain text. Is there any official > philosophy? I have no idea, at least manual does not give me > references. I cannot find "philosophy", send me references. > > It says "to keep simple things simple". But Org is far far from being > simple any more. It offers good principles, paradigms and people built > many enhancements upon those. Speedily it becomes way much more than > simple. > > Headings do not look any more as headings, it looks like pieces of > code to a person that is new. Properties, tags, clocks, schedule, > deadline, all that tries to store itself under specific heading. There > is easily too much of it and things are not simple any more. > >> Currently, the dev's stance is that org files must be >> self-sufficient. > > There is no compact principle there practically. Anything is > possible. That Org files are not practically self-sufficient shows the > fact that there are 129 Emacs packages in one Org distribution. > >> Org-mode should not depend on external database to manage the org >> files and operations with them. Everything must be plain text! > > Question is what is meant by database, it can be anything. One can > save LISP data. Recent files, desktop, eww bookmarks, init.el or > .emacs file are also all similar databases, there is the underused > EIEIO with persistent stuff that represent built-in database > functionality. > > That Org files are not self-sufficient shows the demand that there is > almost no Org user who does not have add-ons, packages, modifications, > configurations. > > Would it be really self-sufficient there would be no development going > on, right? > > Babel executions clearly show that Org is not self sufficient and > depends on number of external software. > > I don't mind of philosophy, in fact I would like that philosophy is > really that what it wanted to be, but that time is over. > > I am just pointing out that it is by many means not self sufficient. > > Is by default LaTeX export enabled? I think it is. How big is the > LaTeX package? It is huge, and Org depends on it for export. > > Thus Org is far far from being self-sufficient. > > Almost every system has GDBM database, if Emacs would have bindings to > GDBM, there would be so much less of development in general for many > various packages and Emacs would be expanding faster and easier. > > In fact I think that author and initial developers could not predict > at the time where the Org goes and that speaking of self-sufficient > and "plain text" only is history. > > Before I found out about Org I was using back in time `hnb' in console > to track various planning tasks. It works nice and simple. That is > really self sufficient. Org definitely not. > > HNB - Hierarchical Notebook > http://hnb.sourceforge.net/ > > In the mean time I have created database where I can store TODO, > Notes, Calls, SMS, People, Invoices, Groups, Mailing Lists and so on, > and made my own shell and Perl interfaces to it. And I used to manage > it through: GeDaFe - PostgreSQL Generic Database Interface > http://gedafe.github.io/doc/gedafe-sql.en.html and this was and is > hierarchical or better graph knowledge management and relational > database. > > Creating simple table in the database automatically helped me to > manage that table. It is trivial to create NOTES table with schedule > date, clock in, TODO or other conditions and tags. The interface is > just there and automatic to whatever table I create the interface is > there to add/modify/delete/search/refine records. That is what I would > say "simple" and keeping things simple and indefinitely extensible > without modification of software. The fundamental GeDaFe principle I > would like to try to implement in Emacs. And same database I use for > web interface I am using also within Emacs. > > GeDaFe principle is following: > > - define your data (like handling notes, TODO, or executing scripts within notes) > > - work and forget about any underlying functions, > add/create/delete/modify/index or search, make reports with simple exports > > - expand with more definitions of data when necessary (like add > various properties, or other data tables, like contacts, invoices, > etc.) and repeat the process. > > Org also shows that it does not hold "Notes" only, it holds more than > that, I have written average book size technical documents with > it. Only just one part of the document is printed from one single node > that belongs to single project among many. People use such documents > on the ground. My use case is not for simple notes. > > A node in a tree can be anything, and Org enhancements develop by that > principle. For example there is org-contacts where nodes are meant to > be "People". Such development is so much hard coded. > > Simpler would be to define the type of nodes and work by their > types. One could need just one type table and one node table and that > is about all. > > The type table could say: > > - this node is heading > - or, this node is text under heading > - or, this node is paragraph among many others > - or, this node is hyperlink to WWW URI > - or, this node is hyperlink to file > - this node is movie to play > - this type is PDF to be opened on page 3 > - this one is really note > - this one is note but with action assigned like TODO, etc. > > Nodes could have its properties defined like for anything. Nodes can > reference its parent nodes. Node can be parent to any heading. > > Once such 2 tables are defined magic starts to take place, it becomes > its subtree with all kinds of node types including Babel execution and > similar. Meta data is meta, it can be updated but need not be > visible. Computer is handling meta data. > > Node can be a page in a subtree that becomes a website. > > Node can be object for person or company, or just anything. > > I am currently using my nodes to quickly research various subjects by > using that type of dynamic knowledge repository. > > Org file is dynamic knowledge repository. > > About Dynamic Knowledge Repositories (DKR) > https://www.dougengelbart.org/content/view/190/163/ > > Then around 2015 I have discovered Cherrytree and have made bunch of > notes with it, and that is self-sufficient one program that keeps > simple things simple as it is much more simpler than Org. It offers a > visual interface to all features related to the knowledge management > > Cherrytree - hierarchical note taking application with rich text and syntax highlighting > https://www.giuspen.com/cherrytree/ > > TAGS in Cherrytree are automatic if I remember well, TAG is simply > name of the node. If I call node TODO, then nodes under are simply > TODO items. > > Later I discovered there is something similar in Emacs so I started > with Org and use it mostly for instruction writing and project > management. And I find many options over kill for me. On the other > hand my usage of Org would be overkill for somebody, so it depends of > viewpoints. > > In general I was always using headings and subheadings, trees, lists, notes by using text. > > Somewhere 2004 I started using Markdown one among first as I found it > simpler than ASCIIDOC and M4, but not as perfect. > > 2020 and 2021 I like to raise the level of dynamic knowledge > journaling to the meta level where I think less about underlying > functions in software. > > That experience also tells that Org is definitely not simple. > > Among hnb, GeDaFe database approach, Cherrytree and Org mode, for > "keeping things simple" like note taking and TODO items, project > management, Cherrytree was the best for me. > > Among all those for keeping complex things simple the database > approach is the best. Of course that I use database within Emacs and > it is not visible to user what it is. Database allows simultaneous > operation by several people on distance and that is the groupware > feature as envisioned by Doug Engelbart. > > For document writing and nice formatting with LaTeX export, Org mode > is the best personally. > > For notes, database based notes are best as only so I have relations > between the note and other objects. With 200,000 contacts in the > database which I can quickly access and assign notes to them, how > would that work with Org? Hardly. Notes are related to people, to > projects, plans, opportunities, research subjects, companies and so > on. There is no simple way in Org mode to relate one note to multiple > other related subjects. > > And people try to do exactly that, people are developing Org in the > manner to relate objects in Org file to anything else. And they do > hard work as they do it manually. Relational database speeds up and > does not tell user to manually hyperlink various relations. > > I have sent thousands of tasks to people by using this function > below. And I had to define for each Org file who are "assignees" for > that Org file. And I have like 50 assignees, so I need to copy and > paste their nick names or identifiers into the Org file. There it > comes the attribute of being "self-sufficient", it becomes > self-sufficient because I had to define all assignees into that > specific file, but I find it tedious and not useful. > > (defun rcd/org-send-assigned-task () > "Sends assigned task to designated individual as Org" > (interactive) > (let* ((member-data (rcd-org-extract-assigned-member-email-data)) > (id (if member-data (first member-data) nil)) > (signature (if (equal (type-of (symbol-value (fifth member-data))) 'cons) > (third (symbol-value (fifth member-data))) "")) > (file (rcd-org-subtree-to-file signature)) > (subject (rcd/org-find-headline)) > (esubject (escape-% subject)) > (ask (unless id (y-or-n-p "No assignment found. Do you want to send it by email?"))) > (name (if id (third member-data))) > ;; (name (if ask (read-from-minibuffer "Name:") name)) > (voice (format "The task '%s' is being sent to '%s'" subject name)) > (email (if id (if (equal (type-of (fourth member-data)) 'cons) > (car (fourth member-data)) > (fourth member-data)))) > (email (if ask (cf-search-email (read-from-minibuffer "Search for email: ")) email)) > (really (y-or-n-p (format "Do you really want to send it to: %s?" (if ask email name))))) > (if (and really (or id ask)) > (if (string-match "@" email) > (progn > ;; (message (escape-% subject)) > (speak voice) > (mutt-send-file name email esubject file)) > (message "No email specified")) > (message "Aborted sending.")))) > >> Moreover, some devs are even reluctant to hide metadata (like unique >> ID implemented in org-id module) from users (which is possible and >> also partially implemented). > > I hope that I have demonstrated that one who develops could review > several various paradigms and learn more. I am fine with any decision > of design for Org mode as I use it as it is and I have for me other > ways of managing data. I am not sure how much those features have been > discussed to say that hiding meta data is good or not good. It is > better to discuss and find what is more useful. > > What I can see is that people complain for speed and they build > extensions that help with it. Extensions are external while built-in > Org does not keep up with the dynamic how people expect it to be. > > For example I would expect Org to be very simple, very very simple, I > would rewind it back to its roots. Somebody else would like > complexities. Currently I can see that Org is not made for > complexities. It is better to unwind the development and make Org in > core very basic and speedy and let people enhance it with external > packages. In general it is better to remain simple. Even that may not > be possible any more. > > I see hard work by many people who try to enhance Org as hierarchical > knowledge of data because people want to have feature X, but group of > those enhancements in reality belong to relational databases and not > to text files. Developers wish to accommodate various people and yet > by doing so they develop it into complex software. (129 .el packages > for one Emacs mode!?) > > Among those one shall read the Doug Engelbart's paradigms, then > especially if one is developer of system of data management that many > people use, one shall explore other paradigms, including various > approaches to data handling. And overall one shall keep it simple as > it is main fundament of Org to be simple, while practical fact remains > that it is not anymore simple, not at all. > > I remember back in time with BASIC programming language that it had > also something like a built-in database that one could put on the > bottom of the file. Then there is also Perl's __DATA__ that is placed > straight into the file and one could also place images and other > stuff. Maybe the meta data could be kept this way in just one heading > named Meta data, and this heading would not be exported, it could be > hidden from the user and it could contain the database necessary for > single Org file. By pressing key to show properties one could see > properties or simply make them hidden. By making a copy of subtree the > metadata would also copy as usual. By exporting, one could make Org > without meta data, and I like using this information as I like sending > Org headings without personal meta data to third parties.