From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id CG9XCQJCyV5LMQAA0tVLHw (envelope-from ) for ; Sat, 23 May 2020 15:32:18 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id +Av8BAJCyV6/cQAAbx9fmQ (envelope-from ) for ; Sat, 23 May 2020 15:32:18 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5102D9404CD for ; Sat, 23 May 2020 15:32:17 +0000 (UTC) Received: from localhost ([::1]:54616 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jcW8E-0002gN-S1 for larch@yhetil.org; Sat, 23 May 2020 11:32:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37972) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jcW7P-0002g3-PR for emacs-orgmode@gnu.org; Sat, 23 May 2020 11:31:23 -0400 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]:35540) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jcW7L-0002XK-TD for emacs-orgmode@gnu.org; Sat, 23 May 2020 11:31:23 -0400 Received: by mail-pj1-x102f.google.com with SMTP id 5so6339660pjd.0 for ; Sat, 23 May 2020 08:31:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-transfer-encoding; bh=WlGSggtxdaJ8Sd3j/w9w9UyPjSOAO/d9SHdIvszNTtY=; b=JaqHnk7kSi+Y7euVGIz7sNNekIfcKxVjB5MZbUwzvX7RNJYlDOSA2u3FOothYHtTqN 5MYuXcz2Z5s4UeIoR0AeLAFIatROWeSiGcbXwF5lUzyTWKL53+RTldF2YqlRIQP5rj7o FgqWpQUB0LvGuRdqQa4I+RXWSrn9sl0WhbhCMioPIUtF71fMevqhG+1P0dp5utksww3x zGN4MgjroSwKn2gTc/I8Sef6L5lMj0ChM3NyIOi2u59U3J3dAe6/gu9m03GWJQLM4JFm PkGhQ2yguLPsVJZ94sr6to7IJVotqcJ5TK6L+oa7b1Rs75ezz0kz9slYCVkBNHiOVDyi PsgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=WlGSggtxdaJ8Sd3j/w9w9UyPjSOAO/d9SHdIvszNTtY=; b=lpGVTFrbeOpU20UGT1a+7lW7Zmfaht1nDxm3RRkxrr9eL6ax2r33zg8t6H8WQDU8sT 5ScOqCndxm/tFaCF9TdWdqABHscZQ2Zlml/WUBNGfKHer2zs/5qPKPwhp0cni/eWhuCI OAgEAfFehAhWdPHMq/H2lmY+0VzPPsrsTVZL1ATLDGV8Lqoe3vPgIeWsbarrIq3OlatO JD0vedsG5ir0gCRbrTSdbk4H8wwbPsVr944IGcDEGRfbAB3h68ahbtws8HtvktgUD8Cr t81NgZQ9g7CqsNyPTJpJcHR1Sk3MpFfMQ8+NGMFF73kwNIz40PGdFb0Ijw+wt0RRPFI3 q8Og== X-Gm-Message-State: AOAM532y8ggePZ+uwqxZR2eAhXX3+JU0dIfJ5wJFfdfri6gJTkaTS++z 6sOKaEhbXO/i1vtzef5cOYjbAZfXTxyvlA== X-Google-Smtp-Source: ABdhPJwqN7bfn7mTJDNlg/gu33+/AxY1cE09+XaLCg5dnCGmOwYT7C5pF+igNrri27dVkyXBONTl3A== X-Received: by 2002:a17:90a:8c01:: with SMTP id a1mr11310243pjo.127.1590247876977; Sat, 23 May 2020 08:31:16 -0700 (PDT) Received: from localhost ([104.151.6.52]) by smtp.gmail.com with ESMTPSA id q134sm9580660pfc.143.2020.05.23.08.31.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 23 May 2020 08:31:16 -0700 (PDT) From: Ihor Radchenko To: Nicolas Goaziou Subject: Re: [patch suggestion] Mitigating the poor Emacs performance on huge org files: Do not use overlays for PROPERTY and LOGBOOK drawers In-Reply-To: <87pnauu595.fsf@localhost> References: <87h7x9e5jo.fsf@localhost> <875zdpia5i.fsf@nicolasgoaziou.fr> <87y2qi8c8w.fsf@localhost> <87r1vu5qmc.fsf@nicolasgoaziou.fr> <87imh5w1zt.fsf@localhost> <87blmxjckl.fsf@localhost> <87y2q13tgs.fsf@nicolasgoaziou.fr> <878si1j83x.fsf@localhost> <87d07bzvhd.fsf@nicolasgoaziou.fr> <87imh34usq.fsf@localhost> <87pnbby49m.fsf@nicolasgoaziou.fr> <87tv0efvyd.fsf@localhost> <874kse1seu.fsf@localhost> <87r1vhqpja.fsf@nicolasgoaziou.fr> <87tv0d2nk7.fsf@localhost> <87o8qkhy3g.fsf@nicolasgoaziou.fr> <87sgfqu5av.fsf@localhost> <87pnauu595.fsf@localhost> Date: Sat, 23 May 2020 23:26:38 +0800 Message-ID: <874ks6lljl.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=yantar92@gmail.com; helo=mail-pj1-x102f.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: emacs-orgmode@gnu.org Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: "Emacs-orgmode" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (body hash did not verify) header.d=gmail.com header.s=20161025 header.b=JaqHnk7k; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of emacs-orgmode-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=emacs-orgmode-bounces@gnu.org X-Spam-Score: 0.59 X-TUID: EsbGyBO/MlOC Github link to the patch: https://gist.github.com/yantar92/6447754415457927293acda43a7fcaef=20 Ihor Radchenko writes: > The patch is attached > > diff --git a/contrib/lisp/org-notify.el b/contrib/lisp/org-notify.el > index 9f8677871..ab470ea9b 100644 > --- a/contrib/lisp/org-notify.el > +++ b/contrib/lisp/org-notify.el > @@ -246,7 +246,7 @@ seconds. The default value for SECS is 20." > (switch-to-buffer (find-file-noselect file)) > (org-with-wide-buffer > (goto-char begin) > - (outline-show-entry)) > + (org-show-entry)) > (goto-char begin) > (search-forward "DEADLINE: <") > (search-forward ":") > diff --git a/contrib/lisp/org-velocity.el b/contrib/lisp/org-velocity.el > index bfc4d6c3e..2312b235c 100644 > --- a/contrib/lisp/org-velocity.el > +++ b/contrib/lisp/org-velocity.el > @@ -325,7 +325,7 @@ use it." > (save-excursion > (when narrow > (org-narrow-to-subtree)) > - (outline-show-all))) > + (org-show-all))) >=20=20 > (defun org-velocity-edit-entry/inline (heading) > "Edit entry at HEADING in the original buffer." > diff --git a/doc/org-manual.org b/doc/org-manual.org > index 96b160175..2ebe94538 100644 > --- a/doc/org-manual.org > +++ b/doc/org-manual.org > @@ -509,11 +509,11 @@ Org uses just two commands, bound to {{{kbd(TAB)}}}= and > Switch back to the startup visibility of the buffer (see [[*Initial > visibility]]). >=20=20 > -- {{{kbd(C-u C-u C-u TAB)}}} (~outline-show-all~) :: > +- {{{kbd(C-u C-u C-u TAB)}}} (~org-show-all~) :: >=20=20 > #+cindex: show all, command > #+kindex: C-u C-u C-u TAB > - #+findex: outline-show-all > + #+findex: org-show-all > Show all, including drawers. >=20=20 > - {{{kbd(C-c C-r)}}} (~org-reveal~) :: > @@ -529,18 +529,18 @@ Org uses just two commands, bound to {{{kbd(TAB)}}}= and > headings. With a double prefix argument, also show the entire > subtree of the parent. >=20=20 > -- {{{kbd(C-c C-k)}}} (~outline-show-branches~) :: > +- {{{kbd(C-c C-k)}}} (~org-show-branches~) :: >=20=20 > #+cindex: show branches, command > #+kindex: C-c C-k > - #+findex: outline-show-branches > + #+findex: org-show-branches > Expose all the headings of the subtree, but not their bodies. >=20=20 > -- {{{kbd(C-c TAB)}}} (~outline-show-children~) :: > +- {{{kbd(C-c TAB)}}} (~org-show-children~) :: >=20=20 > #+cindex: show children, command > #+kindex: C-c TAB > - #+findex: outline-show-children > + #+findex: org-show-children > Expose all direct children of the subtree. With a numeric prefix > argument {{{var(N)}}}, expose all children down to level > {{{var(N)}}}. > @@ -7294,7 +7294,7 @@ its location in the outline tree, but behaves in th= e following way: > command (see [[*Visibility Cycling]]). You can force cycling archived > subtrees with {{{kbd(C-TAB)}}}, or by setting the option > ~org-cycle-open-archived-trees~. Also normal outline commands, like > - ~outline-show-all~, open archived subtrees. > + ~org-show-all~, open archived subtrees. >=20=20 > - > #+vindex: org-sparse-tree-open-archived-trees > diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el > index ab13f926c..ad9244940 100644 > --- a/lisp/org-agenda.el > +++ b/lisp/org-agenda.el > @@ -6826,7 +6826,7 @@ and stored in the variable `org-prefix-format-compi= led'." > (t " %-12:c%?-12t% s"))) > (start 0) > varform vars var e c f opt) > - (while (string-match "%\\(\\?\\)?\\([-+]?[0-9.]*\\)\\([ .;,:!?=3D|/<= >]?\\)\\([cltseib]\\|(.+)\\)" > + (while (string-match "%\\(\\?\\)?\\([-+]?[0-9.]*\\)\\([ .;,:!?=3D|/<= >]?\\)\\([cltseib]\\|(.+?)\\)" > s start) > (setq var (or (cdr (assoc (match-string 4 s) > '(("c" . category) ("t" . time) ("l" . level) ("s" . extra) > @@ -9138,20 +9138,20 @@ if it was hidden in the outline." > ((and (called-interactively-p 'any) (=3D more 1)) > (message "Remote: show with default settings")) > ((=3D more 2) > - (outline-show-entry) > + (org-show-entry) > (org-show-children) > (save-excursion > (org-back-to-heading) > (run-hook-with-args 'org-cycle-hook 'children)) > (message "Remote: CHILDREN")) > ((=3D more 3) > - (outline-show-subtree) > + (org-show-subtree) > (save-excursion > (org-back-to-heading) > (run-hook-with-args 'org-cycle-hook 'subtree)) > (message "Remote: SUBTREE")) > ((> more 3) > - (outline-show-subtree) > + (org-show-subtree) > (message "Remote: SUBTREE AND ALL DRAWERS"))) > (select-window win))) >=20=20 > diff --git a/lisp/org-archive.el b/lisp/org-archive.el > index d3e12d17b..d864dad8a 100644 > --- a/lisp/org-archive.el > +++ b/lisp/org-archive.el > @@ -330,7 +330,7 @@ direct children of this heading." > (insert (if datetree-date "" "\n") heading "\n") > (end-of-line 0)) > ;; Make the subtree visible > - (outline-show-subtree) > + (org-show-subtree) > (if org-archive-reversed-order > (progn > (org-back-to-heading t) > diff --git a/lisp/org-colview.el b/lisp/org-colview.el > index e50a4d7c8..e656df555 100644 > --- a/lisp/org-colview.el > +++ b/lisp/org-colview.el > @@ -699,7 +699,7 @@ FUN is a function called with no argument." > (move-beginning-of-line 2) > (org-at-heading-p t))))) > (unwind-protect (funcall fun) > - (when hide-body (outline-hide-entry))))) > + (when hide-body (org-hide-entry))))) >=20=20 > (defun org-columns-previous-allowed-value () > "Switch to the previous allowed value for this column." > diff --git a/lisp/org-compat.el b/lisp/org-compat.el > index 635a38dcd..8fe271896 100644 > --- a/lisp/org-compat.el > +++ b/lisp/org-compat.el > @@ -139,12 +139,8 @@ This is a floating point number if the size is too l= arge for an integer." > ;;; Emacs < 25.1 compatibility >=20=20 > (when (< emacs-major-version 25) > - (defalias 'outline-hide-entry 'hide-entry) > - (defalias 'outline-hide-sublevels 'hide-sublevels) > - (defalias 'outline-hide-subtree 'hide-subtree) > (defalias 'outline-show-branches 'show-branches) > (defalias 'outline-show-children 'show-children) > - (defalias 'outline-show-entry 'show-entry) > (defalias 'outline-show-subtree 'show-subtree) > (defalias 'xref-find-definitions 'find-tag) > (defalias 'format-message 'format) > diff --git a/lisp/org-element.el b/lisp/org-element.el > index ac41b7650..2d5c8d771 100644 > --- a/lisp/org-element.el > +++ b/lisp/org-element.el > @@ -4320,7 +4320,7 @@ element or object. Meaningful values are `first-se= ction', > TYPE is the type of the current element or object. >=20=20 > If PARENT? is non-nil, assume the next element or object will be > -located inside the current one. " > +located inside the current one." > (if parent? > (pcase type > (`headline 'section) > diff --git a/lisp/org-keys.el b/lisp/org-keys.el > index c006e9c12..deb5d7b90 100644 > --- a/lisp/org-keys.el > +++ b/lisp/org-keys.el > @@ -437,7 +437,7 @@ COMMANDS is a list of alternating OLDDEF NEWDEF comma= nd names." > #'org-next-visible-heading) > (define-key org-mode-map [remap outline-previous-visible-heading] > #'org-previous-visible-heading) > -(define-key org-mode-map [remap show-children] #'org-show-children) > +(define-key org-mode-map [remap outline-show-children] #'org-show-childr= en) >=20=20 > ;;;; Make `C-c C-x' a prefix key > (org-defkey org-mode-map (kbd "C-c C-x") (make-sparse-keymap)) > diff --git a/lisp/org-macs.el b/lisp/org-macs.el > index a02f713ca..fa0a658f0 100644 > --- a/lisp/org-macs.el > +++ b/lisp/org-macs.el > @@ -682,7 +682,7 @@ When NEXT is non-nil, check the next line instead." >=20=20 >=20=20 > > -;;; Overlays > +;;; Overlays and text properties >=20=20 > (defun org-overlay-display (ovl text &optional face evap) > "Make overlay OVL display TEXT with face FACE." > @@ -705,18 +705,99 @@ If DELETE is non-nil, delete all those overlays." > (delete (delete-overlay ov)) > (t (push ov found)))))) >=20=20 > +(defun org-remove-text-properties (start end properties &optional object) > + "Remove text properties as in `remove-text-properties', but keep 'invi= sibility specs for folded regions. > +Do not remove invisible text properties specified by 'outline, > +'org-hide-block, and 'org-hide-drawer (but remove i.e. 'org-link) this > +is needed to keep outlines, drawers, and blocks hidden unless they are > +toggled by user. > +Note: The below may be too specific and create troubles if more > +invisibility specs are added to org in future" > + (when (plist-member properties 'invisible) > + (let ((pos start) > + next spec) > + (while (< pos end) > + (setq next (next-single-property-change pos 'invisible nil end) > + spec (get-text-property pos 'invisible)) > + (unless (memq spec (list 'org-hide-block > + 'org-hide-drawer > + 'outline)) > + (remove-text-properties pos next '(invisible nil) object)) > + (setq pos next)))) > + (when-let ((properties-stripped (org-plist-delete properties 'invisibl= e))) > + (remove-text-properties start end properties-stripped object))) > + > +(defun org--find-text-property-region (pos prop) > + "Find a region containing PROP text property around point POS." > + (let* ((beg (and (get-text-property pos prop) pos)) > + (end beg)) > + (when beg > + ;; when beg is the first point in the region, `previous-single-pro= perty-change' > + ;; will return nil. > + (setq beg (or (previous-single-property-change pos prop) > + beg)) > + ;; when end is the last point in the region, `next-single-property= -change' > + ;; will return nil. > + (setq end (or (next-single-property-change pos prop) > + end)) > + (unless (=3D beg end) ; this should not happen > + (cons beg end))))) > + > (defun org-flag-region (from to flag spec) > "Hide or show lines from FROM to TO, according to FLAG. > SPEC is the invisibility spec, as a symbol." > - (remove-overlays from to 'invisible spec) > - ;; Use `front-advance' since text right before to the beginning of > - ;; the overlay belongs to the visible line than to the contents. > - (when flag > - (let ((o (make-overlay from to nil 'front-advance))) > - (overlay-put o 'evaporate t) > - (overlay-put o 'invisible spec) > - (overlay-put o 'isearch-open-invisible #'delete-overlay)))) > - > + (pcase spec > + ('outline > + (remove-overlays from to 'invisible spec) > + ;; Use `front-advance' since text right before to the beginning of > + ;; the overlay belongs to the visible line than to the contents. > + (when flag > + (let ((o (make-overlay from to nil 'front-advance))) > + (overlay-put o 'evaporate t) > + (overlay-put o 'invisible spec) > + (overlay-put o 'isearch-open-invisible #'delete-overlay)))) > + (_ > + ;; Use text properties instead of overlays for speed. > + ;; Overlays are too slow (Emacs Bug#35453). > + (with-silent-modifications > + ;; keep a backup stack of old text properties > + (save-excursion > + (goto-char from) > + (while (< (point) to) > + (let ((old-spec (get-text-property (point) 'invisible)) > + (end (next-single-property-change (point) 'invisible nil to))) > + (when old-spec > + (alter-text-property (point) end 'org-property-stack-invisible > + (lambda (stack) > + (if (or (eq old-spec (car stack)) > + (eq spec old-spec) > + (eq old-spec 'outline)) > + stack > + (cons old-spec stack))))) > + (goto-char end)))) > + > + ;; cleanup everything > + (remove-text-properties from to '(invisible nil)) > + > + ;; Recover properties from the backup stack > + (unless flag > + (save-excursion > + (goto-char from) > + (while (< (point) to) > + (let ((stack (get-text-property (point) 'org-property-stack= -invisible)) > + (end (next-single-property-change (point) 'org-property-stack-invis= ible nil to))) > + (if (not stack) > + (remove-text-properties (point) end '(org-property-stack-invisible = nil)) > + (put-text-property (point) end 'invisible (car stack)) > + (alter-text-property (point) end 'org-property-stack-invisible > + (lambda (stack) > + (cdr stack)))) > + (goto-char end))))) > +=20=20=20=20=20=20=20 > + (when flag > + (put-text-property from to 'rear-non-sticky nil) > + (put-text-property from to 'front-sticky t) > + (put-text-property from to 'invisible spec)))))) >=20=20 > > ;;; Regexp matching > diff --git a/lisp/org-src.el b/lisp/org-src.el > index c9eef744e..e89a1c580 100644 > --- a/lisp/org-src.el > +++ b/lisp/org-src.el > @@ -523,8 +523,8 @@ Leave point in edit buffer." > (org-src-switch-to-buffer buffer 'edit) > ;; Insert contents. > (insert contents) > - (remove-text-properties (point-min) (point-max) > - '(display nil invisible nil intangible nil)) > + (org-remove-text-properties (point-min) (point-max) > + '(display nil invisible nil intangible nil)) > (unless preserve-ind (org-do-remove-indentation)) > (set-buffer-modified-p nil) > (setq buffer-file-name nil) > diff --git a/lisp/org-table.el b/lisp/org-table.el > index 6462b99c4..75801161b 100644 > --- a/lisp/org-table.el > +++ b/lisp/org-table.el > @@ -2001,7 +2001,7 @@ toggle `org-table-follow-field-mode'." > (arg > (let ((b (save-excursion (skip-chars-backward "^|") (point))) > (e (save-excursion (skip-chars-forward "^|\r\n") (point)))) > - (remove-text-properties b e '(invisible t intangible t)) > + (org-remove-text-properties b e '(invisible t intangible t)) > (if (and (boundp 'font-lock-mode) font-lock-mode) > (font-lock-fontify-block)))) > (t > @@ -2028,7 +2028,7 @@ toggle `org-table-follow-field-mode'." > (setq word-wrap t) > (goto-char (setq p (point-max))) > (insert (org-trim field)) > - (remove-text-properties p (point-max) '(invisible t intangible t)) > + (org-remove-text-properties p (point-max) '(invisible t intangible= t)) > (goto-char p) > (setq-local org-finish-function 'org-table-finish-edit-field) > (setq-local org-window-configuration cw) > diff --git a/lisp/org.el b/lisp/org.el > index e577dc661..360974135 100644 > --- a/lisp/org.el > +++ b/lisp/org.el > @@ -114,6 +114,7 @@ Stars are put in group 1 and the trimmed body in grou= p 2.") > (declare-function cdlatex-math-symbol "ext:cdlatex") > (declare-function Info-goto-node "info" (nodename &optional fork strict-= case)) > (declare-function isearch-no-upper-case-p "isearch" (string regexp-flag)) > +(declare-function isearch-filter-visible "isearch" (beg end)) > (declare-function org-add-archive-files "org-archive" (files)) > (declare-function org-agenda-entry-get-agenda-timestamp "org-agenda" (po= m)) > (declare-function org-agenda-list "org-agenda" (&optional arg start-day = span with-hour)) > @@ -192,6 +193,9 @@ Stars are put in group 1 and the trimmed body in grou= p 2.") >=20=20 > (defvar ffap-url-regexp) > (defvar org-element-paragraph-separate) > +(defvar org-element-all-objects) > +(defvar org-element-all-elements) > +(defvar org-element-greater-elements) > (defvar org-indent-indentation-per-level) > (defvar org-radio-target-regexp) > (defvar org-target-link-regexp) > @@ -4734,9 +4738,381 @@ This is for getting out of special buffers like c= apture.") >=20=20 > ;;;; Define the Org mode >=20=20 > +;;; Handling buffer modifications > + > (defun org-before-change-function (_beg _end) > "Every change indicates that a table might need an update." > (setq org-table-may-need-update t)) > + > +(defvar-local org--modified-elements nil > + "List of elements, marked as recently modified. > +There is no guarantee that the elements in this list are fully parsed. > +Only the element type, :begin and :end properties of the elements are > +guaranteed to be available. The :begin and :end element properties > +contain markers instead of positions.") > + > +(defvar org-track-element-modification-default-sensitive-commands '(self= -insert-command) > + "List of commands triggerring element modifications unconditionally.") > + > +(defvar org--element-beginning-re-alist `((center-block . "^[ \t]*#\\+be= gin_center[ \t]*$") > + (property-drawer . ,org-property-= start-re) > + (drawer . ,org-drawer-regexp) > + (quote-block . "^[ \t]*#\\+begin_= quote[ \t]*$") > + (special-block . "^[ \t]*#\\+begi= n_\\([^ ]+\\).*$")) > + "Alist of regexps matching beginning of elements. > +Group 1 in the regexps (if any) contains the element type.") > + > +(defvar org--element-end-re-alist `((center-block . "^[ \t]*#\\+end_cent= er[ \t]*$") > + (property-drawer . ,org-property-end-re) > + (drawer . ,org-property-end-re) > + (quote-block . "^[ \t]*#\\+end_quote[ \t]*$") > + (special-block . "^[ \t]*#\\+end_\\([^ ]+\\).*$")) > + "Alist of regexps matching end of elements. > +Group 1 in the regexps (if any) contains the element type or END.") > + > +(defvar org-track-element-modifications > + `((property-drawer . (:after-change-function > + org--drawer-or-block-unfold-maybe)) > + (drawer . (:after-change-function > + org--drawer-or-block-unfold-maybe)) > + (center-block . (:after-change-function > + org--drawer-or-block-unfold-maybe)) > + (quote-block . (:after-change-function > + org--drawer-or-block-unfold-maybe)) > + (special-block . (:after-change-function > + org--drawer-or-block-unfold-maybe))) > + "Alist of elements to be tracked for modifications. > +The modification is only triggered according to :sensitive-re-list and > +:sensitive-command-list (see below). > +Each element of the alist is a cons of an element symbol and plist > +defining how and when the modifications are handled. > +In case of recursive elements/duplicates, the first element from the > +list is considered. > +The plist can have the following properties: > +- :element-beginning-re :: regex matching beginning of the element > + (default) :: (alist-get element org--element-beginning-r= e-alist) > +- :element-end-re :: regex matching end of the element > + (default) :: (alist-get element org--element-end-re-alis= t) > +- :after-change-function :: function called after the modification > +The function must accept a single argument - element from > +`org--modified-elements'.") > + > +(defun org--get-element-region-at-point (types) > + "Return TYPES element at point or nil. > +If TYPES is a list, return first element at point from the list. The > +returned value is partially parsed element only containing :begin and > +:end properties. Only elements listed in > +org--element-beginning-re-alist and org--element-end-re-alist can be > +parsed here." > + (catch 'exit > + (dolist (type (if (listp types) types (list types))) > + (let ((begin-re (alist-get type org--element-beginning-re-alist)) > + (end-re (alist-get type org--element-end-re-alist)) > + (begin-limit (save-excursion (org-with-limited-levels > + (org-back-to-heading-or-point-min 'invisible-ok)) > + (point))) > + (end-limit (or (save-excursion (outline-next-heading)) > + (point-max))) > + (point (point)) > + begin end closest-begin) > + (when (and begin-re end-re) > + (save-excursion > + (end-of-line) > + (when (re-search-backward begin-re begin-limit 'noerror) (setq begi= n (point))) > + (when (re-search-forward end-re end-limit 'noerror) (setq end (poin= t))) > + (setq closest-begin begin) > + ;; slurp unmatched begin-re > + (when (and begin end) > + (goto-char begin) > + (while (and (re-search-backward begin-re begin-limit 'noer= ror) > + (=3D end (save-excursion (re-search-forward end-re end-limit 'noerr= or)))) > + (setq begin (point))) > + (when (and (>=3D point begin) (<=3D point end)) > + (throw 'exit > + (list type > + (list > + :begin begin > + :end end))))))))))) > + > +(defun org--get-next-element-region-at-point (types &optional limit prev= ious) > + "Return TYPES element after point or nil. > +If TYPES is a list, return first element after point from the list. > +If PREVIOUS is non-nil, return first TYPES element before point. > +Limit search by LIMIT or previous/next heading. > +The returned value is partially parsed element only containing :begin > +and :end properties. Only elements listed in > +org--element-beginning-re-alist and org--element-end-re-alist can be > +parsed here." > + (catch 'exit > + (dolist (type (if (listp types) types (list types))) > + (let* ((begin-re (alist-get type org--element-beginning-re-alist)) > + (end-re (alist-get type org--element-end-re-alist)) > + (limit (or limit (if previous > + (save-excursion > + (org-with-limited-levels > + (org-back-to-heading-or-point-min 'invisible-ok) > + (point))) > + (or (save-excursion (outline-next-heading)) > + (point-max))))) > + begin end) > + (when (and begin-re end-re) > + (save-excursion > + (if previous > + (when (re-search-backward begin-re limit 'noerror) > + (when-let ((el (org--get-element-region-at-point type))) > + (setq begin (org-element-property :begin el)) > + (setq end (org-element-property :end el)))) > + (when (re-search-forward begin-re limit 'noerror) > + (when-let ((el (org--get-element-region-at-point type))) > + (setq begin (org-element-property :begin el)) > + (setq end (org-element-property :end el)))))) > + (when (and begin end) > + (throw 'exit > + (list type > + (list > + :begin begin > + :end end))))))))) > + > +(defun org--find-elements-in-region (beg end elements &optional include-= partial include-neighbours) > + "Find all elements from ELEMENTS in region BEG . END. > +All the listed elements must be resolvable by > +`org--get-element-region-at-point'. > +Include elements if they are partially inside region when > +INCLUDE-PARTIAL is non-nil. > +Include preceding/subsequent neighbouring elements when no partial > +element is found at the beginning/end of the region and > +INCLUDE-NEIGHBOURS is non-nil." > + (when include-partial > + (org-with-point-at beg > + (let ((new-beg (org-element-property :begin (org--get-element-regi= on-at-point elements)))) > + (if new-beg > + (setq beg new-beg) > + (when (and include-neighbours > + (setq new-beg (org-element-property :begin > + (org--get-next-element-region-at-point elements > + (point-min) > + 'previous)))) > + (setq beg new-beg)))) > + (when (memq 'headline elements) > + (when-let ((new-beg (save-excursion > + (org-with-limited-levels (outline-previous-heading))))) > + (setq beg new-beg)))) > + (org-with-point-at end > + (let ((new-end (org-element-property :end (org--get-element-region= -at-point elements)))) > + (if new-end > + (setq end new-end) > + (when (and include-neighbours > + (setq new-end (org-element-property :end > + (org--get-next-element-region-at-point elements (point-max))= ))) > + (setq end new-end)))) > + (when (memq 'headline elements) > + (when-let ((new-end (org-with-limited-levels (outline-next-heading)))) > + (setq end (1- new-end)))))) > + (save-excursion > + (save-restriction > + (narrow-to-region beg end) > + (goto-char (point-min)) > + (let (result el) > + (while (setq el (org--get-next-element-region-at-point elements end)) > + (push el result) > + (goto-char (org-element-property :end el))) > + result)))) > + > +(defun org--drawer-or-block-unfold-maybe (el) > + "Update visibility of modified folded drawer/block EL. > +If text was added to hidden drawer/block, make sure that the text is > +also hidden, unless the change was done by a command listed in > +`org-track-element-modification-default-sensitive-commands'. If the > +modification destroyed the drawer/block, reveal the hidden text in > +former drawer/block. If the modification shrinked/expanded the > +drawer/block beyond the hidden text, reveal the affected > +drawers/blocks as well. > +Examples: > +---------------------------------------------- > +---------------------------------------------- > +Case #1 (the element content is hidden): > +---------------------------------------------- > +:PROPERTIES: > +:ID: 279e797c-f4a7-47bb-80f6-e72ac6f3ec55 > +:END: > +---------------------------------------------- > +is changed to > +---------------------------------------------- > +:ROPERTIES: > +:ID: 279e797c-f4a7-47bb-80f6-e72ac6f3ec55 > +:END: > +---------------------------------------------- > +Text is revealed, because we have drawer in place of property-drawer > +---------------------------------------------- > +---------------------------------------------- > +Case #2 (the element content is hidden): > +---------------------------------------------- > +:ROPERTIES: > +:ID: 279e797c-f4a7-47bb-80f6-e72ac6f3ec55 > +:END: > +---------------------------------------------- > +is changed to > +---------------------------------------------- > +:OPERTIES: > +:ID: 279e797c-f4a7-47bb-80f6-e72ac6f3ec55 > +:END: > +---------------------------------------------- > +The text remains hidden since it is still a drawer. > +---------------------------------------------- > +---------------------------------------------- > +Case #3: (the element content is hidden): > +---------------------------------------------- > +:FOO: > +bar > +tmp > +:END: > +---------------------------------------------- > +is changed to > +---------------------------------------------- > +:FOO: > +bar > +:END: > +tmp > +:END: > +---------------------------------------------- > +The text is revealed because the drawer contents shrank. > +---------------------------------------------- > +---------------------------------------------- > +Case #4: (the element content is hidden in both the drawers): > +---------------------------------------------- > +:FOO: > +bar > +tmp > +:END: > +:BAR: > +jjd > +:END: > +---------------------------------------------- > +is changed to > +---------------------------------------------- > +:FOO: > +bar > +tmp > +:BAR: > +jjd > +:END: > +---------------------------------------------- > +The text is revealed in both the drawers because the drawers are merged > +into a new drawer. > +---------------------------------------------- > +---------------------------------------------- > +Case #5: (the element content is hidden) > +---------------------------------------------- > +:test: > +Vivamus id enim. > +:end: > +---------------------------------------------- > +is changed to > +---------------------------------------------- > +:drawer: > +:test: > +Vivamus id enim. > +:end: > +---------------------------------------------- > +The text is revealed in the drawer because the drawer expended. > +---------------------------------------------- > +---------------------------------------------- > +Case #6: (the element content is hidden): > +---------------------------------------------- > +:test: > +Vivamus id enim. > +:end: > +---------------------------------------------- > +is changed to > +---------------------------------------------- > +:test: > +Vivamus id enim. > +:end: > +Nam a sapien. > +:end: > +---------------------------------------------- > +The text remains hidden because drawer contents is always before the > +first :end:." > + (save-match-data > + (save-excursion > + (save-restriction > + (goto-char (org-element-property :begin el)) > + (let* ((newel (org--get-element-region-at-point > + (mapcar (lambda (el) > + (when (string-match-p (regexp-opt '("block" "drawer")) > + (symbol-name (car el))) > + (car el))) > + org-track-element-modifications))) > + (spec (if (string-match-p "block" (symbol-name (org-element-type= el))) > + 'org-hide-block > + (if (string-match-p "drawer" (symbol-name (org-element-type el)= )) > + 'org-hide-drawer > + t))) > + (toggle-func (if (eq spec 'org-hide-drawer) > + #'org-hide-drawer-toggle > + (if (eq spec 'org-hide-block) > + #'org-hide-block-toggle > + #'ignore)))) ; this should not happen > + (if (and (equal (org-element-type el) (org-element-type newel)) > + (equal (marker-position (org-element-property :begin el)) > + (org-element-property :begin newel)) > + (equal (marker-position (org-element-property :end el)) > + (org-element-property :end newel))) > + (when (text-property-any (marker-position (org-element-property := begin el)) > + (marker-position (org-element-property :end el)) > + 'invisible spec) > + (goto-char (org-element-property :begin newel)) > + (if (memq this-command org-track-element-modification-default-sensitiv= e-commands) > + ;; reveal if change was made by typing > + (funcall toggle-func 'off) > + ;; re-hide the inserted text > + ;; FIXME: opening the drawer before hiding should not be needed here > + (funcall toggle-func 'off) ; this is needed to avoid showing double = ellipsis > + (funcall toggle-func 'hide))) > + ;; The element was destroyed. Reveal everything. > + (org-flag-region (marker-position (org-element-property :beg= in el)) > + (marker-position (org-element-property :end el)) > + nil spec) > + (when newel > + (org-flag-region (org-element-property :begin newel) > + (org-element-property :end newel) > + nil spec)))))))) > + > +(defun org--before-element-change-function (beg end) > + "Register upcoming element modifications in `org--modified-elements' f= or all elements interesting with BEG END." > + (save-match-data > + (save-excursion > + (save-restriction > + (widen) > + (dolist (el (org--find-elements-in-region beg > + end > + (mapcar #'car org-track-element-modifications) > + 'include-partial > + 'include-neighbours)) > + (let* ((beg-marker (copy-marker (org-element-property :begin el) 't)) > + (end-marker (copy-marker (org-element-property :end el) 't))) > + (when (and (marker-position beg-marker) (marker-position end-marker= )) > + (org-element-put-property el :begin beg-marker) > + (org-element-put-property el :end end-marker) > + (add-to-list 'org--modified-elements el)))))))) > + > +;; FIXME: this function may be called many times during routine modifica= tions > +;; The normal way to avoid this is `combine-after-change-calls' - not > +;; the case in most org primitives. > +(defun org--after-element-change-function (&rest _) > + "Handle changed elements from `org--modified-elements'." > + (dolist (el org--modified-elements) > + (save-match-data > + (save-excursion > + (save-restriction > + (widen) > + (when-let* ((type (org-element-type el)) > + (change-func (plist-get (alist-get type org-track-element-modifi= cations) > + :after-change-function))) > + (with-demoted-errors > + (funcall (symbol-function change-func) el))))))) > + (setq org--modified-elements nil)) > + > (defvar org-mode-map) > (defvar org-inhibit-startup-visibility-stuff nil) ; Dynamically-scoped p= aram. > (defvar org-agenda-keep-modes nil) ; Dynamically-scoped param. > @@ -4818,6 +5194,9 @@ The following commands are available: > ;; Activate before-change-function > (setq-local org-table-may-need-update t) > (add-hook 'before-change-functions 'org-before-change-function nil 'lo= cal) > + (add-hook 'before-change-functions 'org--before-element-change-functio= n nil 'local) > + ;; Activate after-change-function > + (add-hook 'after-change-functions 'org--after-element-change-function = nil 'local) > ;; Check for running clock before killing a buffer > (add-hook 'kill-buffer-hook 'org-check-running-clock nil 'local) > ;; Initialize macros templates. > @@ -4869,6 +5248,10 @@ The following commands are available: > (setq-local outline-isearch-open-invisible-function > (lambda (&rest _) (org-show-context 'isearch))) >=20=20 > + ;; Make isearch search in blocks hidden via text properties > + (setq-local isearch-filter-predicate #'org--isearch-filter-predicate) > + (add-hook 'isearch-mode-end-hook #'org--clear-isearch-overlays nil 'lo= cal) > + > ;; Setup the pcomplete hooks > (setq-local pcomplete-command-completion-function #'org-pcomplete-init= ial) > (setq-local pcomplete-command-name-function #'org-command-at-point) > @@ -5050,8 +5433,8 @@ stacked delimiters is N. Escaping delimiters is no= t possible." > (when verbatim? > (org-remove-flyspell-overlays-in > (match-beginning 0) (match-end 0)) > - (remove-text-properties (match-beginning 2) (match-end 2) > - '(display t invisible t intangible t))) > + (org-remove-text-properties (match-beginning 2) (match-end 2) > + '(display t invisible t intangible t))) > (add-text-properties (match-beginning 2) (match-end 2) > '(font-lock-multiline t org-emphasis t)) > (when (and org-hide-emphasis-markers > @@ -5166,7 +5549,7 @@ This includes angle, plain, and bracket links." > (if (not (eq 'bracket style)) > (add-text-properties start end properties) > ;; Handle invisible parts in bracket links. > - (remove-text-properties start end '(invisible nil)) > + (org-remove-text-properties start end '(invisible nil)) > (let ((hidden > (append `(invisible > ,(or (org-link-get-parameter type :display) > @@ -5186,8 +5569,8 @@ This includes angle, plain, and bracket links." > (defun org-activate-code (limit) > (when (re-search-forward "^[ \t]*\\(:\\(?: .*\\|$\\)\n?\\)" limit t) > (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0)) > - (remove-text-properties (match-beginning 0) (match-end 0) > - '(display t invisible t intangible t)) > + (org-remove-text-properties (match-beginning 0) (match-end 0) > + '(display t invisible t intangible t)) > t)) >=20=20 > (defcustom org-src-fontify-natively t > @@ -5258,8 +5641,8 @@ by a #." > (setq block-end (match-beginning 0)) ; includes the final newline. > (when quoting > (org-remove-flyspell-overlays-in bol-after-beginline nl-before-en= dline) > - (remove-text-properties beg end-of-endline > - '(display t invisible t intangible t))) > + (org-remove-text-properties beg end-of-endline > + '(display t invisible t intangible t))) > (add-text-properties > beg end-of-endline '(font-lock-fontified t font-lock-multiline t)) > (org-remove-flyspell-overlays-in beg bol-after-beginline) > @@ -5313,8 +5696,8 @@ by a #." > '(font-lock-fontified t face org-document-info)))) > ((string-prefix-p "+caption" dc1) > (org-remove-flyspell-overlays-in (match-end 2) (match-end 0)) > - (remove-text-properties (match-beginning 0) (match-end 0) > - '(display t invisible t intangible t)) > + (org-remove-text-properties (match-beginning 0) (match-end 0) > + '(display t invisible t intangible t)) > ;; Handle short captions. > (save-excursion > (beginning-of-line) > @@ -5336,8 +5719,8 @@ by a #." > '(font-lock-fontified t face font-lock-comment-face))) > (t ;; just any other in-buffer setting, but not indented > (org-remove-flyspell-overlays-in (match-beginning 0) (match-end 0)) > - (remove-text-properties (match-beginning 0) (match-end 0) > - '(display t invisible t intangible t)) > + (org-remove-text-properties (match-beginning 0) (match-end 0) > + '(display t invisible t intangible t)) > (add-text-properties beg (match-end 0) > '(font-lock-fontified t face org-meta-line)) > t)))))) > @@ -5859,10 +6242,11 @@ If TAG is a number, get the corresponding match g= roup." > (inhibit-modification-hooks t) > deactivate-mark buffer-file-name buffer-file-truename) > (decompose-region beg end) > - (remove-text-properties beg end > - '(mouse-face t keymap t org-linked-text t > - invisible t intangible t > - org-emphasis t)) > + (org-remove-text-properties beg end > + '(mouse-face t keymap t org-linked-text t > + invisible t > + intangible t > + org-emphasis t)) > (org-remove-font-lock-display-properties beg end))) >=20=20 > (defconst org-script-display '(((raise -0.3) (height 0.7)) > @@ -5970,6 +6354,29 @@ open and agenda-wise Org files." >=20=20 > ;;;; Headlines visibility >=20=20 > +(defun org-hide-entry () > + "Hide the body directly following this heading." > + (interactive) > + (save-excursion > + (outline-back-to-heading) > + (outline-end-of-heading) > + (org-flag-region (point) (progn (outline-next-preface) (point)) t 'o= utline))) > + > +(defun org-hide-subtree () > + "Hide everything after this heading at deeper levels." > + (interactive) > + (org-flag-subtree t)) > + > +(defun org-hide-sublevels (levels) > + "Hide everything but the top LEVELS levels of headers, in whole buffer. > +This also unhides the top heading-less body, if any. > + > +Interactively, the prefix argument supplies the value of LEVELS. > +When invoked without a prefix argument, LEVELS defaults to the level > +of the current heading, or to 1 if the current line is not a heading." > + (cl-letf (((symbol-function 'outline-flag-region) #'org-flag-region)) > + (org-hide-sublevels levels))) > + > (defun org-show-entry () > "Show the body directly following this heading. > Show the heading too, if it is currently invisible." > @@ -5988,6 +6395,16 @@ Show the heading too, if it is currently invisible= ." > 'outline) > (org-cycle-hide-property-drawers 'children)))) >=20=20 > +(defun org-show-heading () > + "Show the current heading and move to its end." > + (org-flag-region (- (point) > + (if (bobp) 0 > + (if (and outline-blank-line > + (eq (char-before (1- (point))) ?\n)) > + 2 1))) > + (progn (outline-end-of-heading) (point)) > + nil)) > + > (defun org-show-children (&optional level) > "Show all direct subheadings of this heading. > Prefix arg LEVEL is how many levels below the current level > @@ -6031,6 +6448,11 @@ heading to appear." > (org-flag-region > (point) (save-excursion (org-end-of-subtree t t)) nil 'outline)) >=20=20 > +(defun org-show-branches () > + "Show all subheadings of this heading, but not their bodies." > + (interactive) > + (org-show-children 1000)) > + > ;;;; Blocks and drawers visibility >=20=20 > (defun org--hide-wrapper-toggle (element category force no-error) > @@ -6064,8 +6486,8 @@ Return a non-nil value when toggling is successful." > (unless (let ((eol (line-end-position))) > (and (> eol start) (/=3D eol end))) > (let* ((spec (cond ((eq category 'block) 'org-hide-block) > - ((eq type 'property-drawer) 'outline) > - (t 'org-hide-drawer))) > + ((eq category 'drawer) 'org-hide-drawer) > + (t 'outline))) > (flag > (cond ((eq force 'off) nil) > (force t) > @@ -6158,10 +6580,7 @@ STATE should be one of the symbols listed in the d= ocstring of > (when (org-at-property-drawer-p) > (let* ((case-fold-search t) > (end (re-search-forward org-property-end-re))) > - ;; Property drawers use `outline' invisibility spec > - ;; so they can be swallowed once we hide the > - ;; outline. > - (org-flag-region start end t 'outline))))))))))) > + (org-flag-region start end t 'org-hide-drawer))))))))))) >=20=20 > ;;;; Visibility cycling >=20=20 > @@ -6536,7 +6955,7 @@ With a numeric prefix, show all headlines up to tha= t level." > (org-narrow-to-subtree) > (org-content)))) > ((or "all" "showall") > - (outline-show-subtree)) > + (org-show-subtree)) > (_ nil))) > (org-end-of-subtree))))))) >=20=20 > @@ -6609,7 +7028,7 @@ This function is the default value of the hook `org= -cycle-hook'." > (while (re-search-forward re nil t) > (when (and (not (org-invisible-p)) > (org-invisible-p (line-end-position))) > - (outline-hide-entry)))) > + (org-hide-entry)))) > (org-cycle-hide-property-drawers 'all) > (org-cycle-show-empty-lines 'overview))))) >=20=20 > @@ -6683,8 +7102,13 @@ information." > ;; expose it. > (dolist (o (overlays-at (point))) > (when (memq (overlay-get o 'invisible) > - '(org-hide-block org-hide-drawer outline)) > + '(outline)) > (delete-overlay o))) > + (when (memq (get-text-property (point) 'invisible) > + '(org-hide-block org-hide-drawer)) > + (let ((spec (get-text-property (point) 'invisible)) > + (region (org--find-text-property-region (point) 'invisible))) > + (org-flag-region (car region) (cdr region) nil spec))) > (unless (org-before-first-heading-p) > (org-with-limited-levels > (cl-case detail > @@ -7661,7 +8085,7 @@ When REMOVE is non-nil, remove the subtree from the= clipboard." > (skip-chars-forward " \t\n\r") > (setq beg (point)) > (when (and (org-invisible-p) visp) > - (save-excursion (outline-show-heading))) > + (save-excursion (org-show-heading))) > ;; Shift if necessary. > (unless (=3D shift 0) > (save-restriction > @@ -8103,7 +8527,7 @@ function is being called interactively." > (point)) > what "children") > (goto-char start) > - (outline-show-subtree) > + (org-show-subtree) > (outline-next-heading)) > (t > ;; we will sort the top-level entries in this file > @@ -13150,7 +13574,7 @@ drawer is immediately hidden." > (inhibit-read-only t)) > (unless (bobp) (insert "\n")) > (insert ":PROPERTIES:\n:END:") > - (org-flag-region (line-end-position 0) (point) t 'outline) > + (org-flag-region (line-end-position 0) (point) t 'org-hide-drawer) > (when (or (eobp) (=3D begin (point-min))) (insert "\n")) > (org-indent-region begin (point)))))) >=20=20 > @@ -17612,11 +18036,11 @@ Move point to the beginning of first heading or= end of buffer." > (defun org-show-branches-buffer () > "Show all branches in the buffer." > (org-flag-above-first-heading) > - (outline-hide-sublevels 1) > + (org-hide-sublevels 1) > (unless (eobp) > - (outline-show-branches) > + (org-show-branches) > (while (outline-get-next-sibling) > - (outline-show-branches))) > + (org-show-branches))) > (goto-char (point-min))) >=20=20 > (defun org-kill-note-or-show-branches () > @@ -17630,8 +18054,8 @@ Move point to the beginning of first heading or e= nd of buffer." > (t > (let ((beg (progn (org-back-to-heading) (point))) > (end (save-excursion (org-end-of-subtree t t) (point)))) > - (outline-hide-subtree) > - (outline-show-branches) > + (org-hide-subtree) > + (org-show-branches) > (org-hide-archived-subtrees beg end))))) >=20=20 > (defun org-delete-indentation (&optional arg) > @@ -17787,9 +18211,9 @@ Otherwise, call `org-show-children'. ARG is the = level to hide." > (if (org-before-first-heading-p) > (progn > (org-flag-above-first-heading) > - (outline-hide-sublevels (or arg 1)) > + (org-hide-sublevels (or arg 1)) > (goto-char (point-min))) > - (outline-hide-subtree) > + (org-hide-subtree) > (org-show-children arg)))) >=20=20 > (defun org-ctrl-c-star () > @@ -20933,6 +21357,80 @@ Started from `gnus-info-find-node'." > (t default-org-info-node)))))) >=20=20 > > + > +;;; Make isearch search in some text hidden via text propertoes > + > +(defvar org--isearch-overlays nil > + "List of overlays temporarily created during isearch. > +This is used to allow searching in regions hidden via text properties. > +As for [2020-05-09 Sat], Isearch only has special handling of hidden ove= rlays. > +Any text hidden via text properties is not revealed even if `search-invi= sible' > +is set to 't.") > + > +;; Not sure if it needs to be a user option > +;; One might want to reveal hidden text in, for example, hidden parts of= the links. > +;; Currently, hidden text in links is never revealed by isearch. > +(defvar org-isearch-specs '(org-hide-block > + org-hide-drawer) > + "List of text invisibility specs to be searched by isearch. > +By default ([2020-05-09 Sat]), isearch does not search in hidden text, > +which was made invisible using text properties. Isearch will be forced > +to search in hidden text with any of the listed 'invisible property valu= e.") > + > +(defun org--create-isearch-overlays (beg end) > + "Replace text property invisibility spec by overlays between BEG and E= ND. > +All the regions with invisibility text property spec from > +`org-isearch-specs' will be changed to use overlays instead > +of text properties. The created overlays will be stored in > +`org--isearch-overlays'." > + (let ((pos beg)) > + (while (< pos end) > + (when-let* ((spec (get-text-property pos 'invisible)) > + (spec (memq spec org-isearch-specs)) > + (region (org--find-text-property-region pos 'invisible))) > + (setq spec (get-text-property pos 'invisible)) > + ;; Changing text properties is considered buffer modification. > + ;; We do not want it here. > + (with-silent-modifications > + ;; The overlay is modelled after `org-flag-region' [2020-05-09= Sat] > + ;; overlay for 'outline blocks. > + (let ((o (make-overlay (car region) (cdr region) nil 'front-ad= vance))) > + (overlay-put o 'evaporate t) > + (overlay-put o 'invisible spec) > + ;; `delete-overlay' here means that spec information will be= lost > + ;; for the region. The region will remain visible. > + (overlay-put o 'isearch-open-invisible #'delete-overlay) > + (push o org--isearch-overlays)) > + (org-flag-region (car region) (cdr region) nil spec))) > + (setq pos (next-single-property-change pos 'invisible nil end))))) > + > +(defun org--isearch-filter-predicate (beg end) > + "Return non-nil if text between BEG and END is deemed visible by Isear= ch. > +This function is intended to be used as `isearch-filter-predicate'. > +Unlike `isearch-filter-visible', make text with 'invisible text property > +value listed in `org-isearch-specs' visible to Isearch." > + (org--create-isearch-overlays beg end) ;; trick isearch by creating ov= erlays in place of invisible text > + (isearch-filter-visible beg end)) > + > +(defun org--clear-isearch-overlay (ov) > + "Convert OV region back into using text properties." > + (when-let ((spec (overlay-get ov 'invisible))) ;; ignore deleted overl= ays > + ;; Changing text properties is considered buffer modification. > + ;; We do not want it here. > + (with-silent-modifications > + (org-flag-region (overlay-start ov) (overlay-end ov) t spec))) > + (when (member ov isearch-opened-overlays) > + (setq isearch-opened-overlays (delete ov isearch-opened-overlays))) > + (delete-overlay ov)) > + > +(defun org--clear-isearch-overlays () > + "Convert overlays from `org--isearch-overlays' back into using text pr= operties." > + (when org--isearch-overlays > + (mapc #'org--clear-isearch-overlay org--isearch-overlays) > + (setq org--isearch-overlays nil))) > + > + > + > ;;; Finish up >=20=20 > (add-hook 'org-mode-hook ;remove overlays when changing major mode > > > Ihor Radchenko writes: > >> Hello, >> >> [The patch itself will be provided in the following email] >> >> I have five updates from the previous version of the patch: >> >> 1. I implemented a simplified version of element parsing to detect >> changes in folded drawers or blocks. No computationally expensive calls >> of org-element-at-point or org-element-parse-buffer are needed now. >> >> 2. The patch is now compatible with master (commit 2e96dc639). I >> reverted the earlier change in folding drawers and blocks. Now, they are >> back to using 'org-hide-block and 'org-hide-drawer. Using 'outline would >> achieve nothing when we use text properties. >> >> 3. 'invisible text property can now be nested. This is important, for >> example, when text inside drawers contains fontified links (which also >> use 'invisible text property to hide parts of the link). Now, the old >> 'invisible spec is recovered after unfolding. >> >> 4. Some outline-* function calls in org referred to outline-flag-region >> implementation, which is not in sync with org-flag-region in this patch. >> I have implemented their org-* versions and replaced the calls >> throughout .el files. Actually, some org-* versions were already >> implemented in org, but not used for some reason (or not mentioned in >> the manual). I have updated the relevant sections of manual. These >> changes might be relevant to org independently of this feature branch. >> >> 5. I have managed to get a working version of outline folding via text >> properties. However, that approach has a big downside - folding state >> cannot be different in indirect buffer when we use text properties. I >> have seen packages relying on this feature of org and I do not see any >> obvious way to achieve different folding state in indirect buffer while >> using text properties for outline folding. >> >> ----------------------------------------------------------------------- >> ----------------------------------------------------------------------- >> >> More details on the new implementation for tracking changes: >> >>> Of course we can. It is only necessary to focus on changes that would >>> break the structure of the element. This does not entail a full parsing. >> >> I have limited parsing to matching beginning and end of a drawer/block. >> The basic functions are org--get-element-region-at-point, >> org--get-next-element-region-at-point, and org--find-elements-in-region. >> They are simplified versions of org-element-* parsers and do not require >> parsing everything from the beginning of the section. >> >> For now, I keep everything in org.el, but those simplified parsers >> probably belong to org-element.el. >> >>> If we can stick with `after-change-functions' (or local equivalent), >>> that's better. It is more predictable than `before-change-functions' and >>> alike. >> >> For now, I still used before/after-change-functions combination. >> I see the following problems with using only after-change-functions:=20 >> >> 1. They are not guaranteed to be called after every single change: >> >> From (elisp) Change Hooks: >> "... some complex primitives call =E2=80=98before-change-functions=E2=80= =99 once before >> making changes, and then call =E2=80=98after-change-functions=E2=80=99 z= ero or more >> times" >> >> The consequence of it is a possibility that region passed to the >> after-change-functions is quite big (including all the singular changes, >> even if they are distant). This region may contain changed drawers as >> well and unchanged drawers and needs to be parsed to determine which >> drawers need to be re-folded. >> >>> And, more importantly, they are not meant to be used together, i.e., you >>> cannot assume that a single call to `before-change-functions' always >>> happens before calling `after-change-functions'. This can be tricky if >>> you want to use the former to pass information to the latter. >> >> The fact that before-change-functions can be called multiple times >> before after-change-functions, is trivially solved by using buffer-local >> changes register (see org--modified-elements). The register is populated >> by before-change-functions and cleared by after-change-functions. >> >>> Well, `before-change-fuctions' and `after-change-functions' are not >>> clean at all: you modify an unrelated part of the buffer, but still call >>> those to check if a drawer needs to be unfolded somewhere. >> >> 2. As you pointed, instead of global before-change-functions, we can use >> modification-hooks text property on sensitive parts of the >> drawers/blocks. This would work, but I am concerned about one annoying >> special case: >> >> ------------------------------------------------------------------------- >> :BLAH: >> >> >> >> :DRAWER: >> Donec at pede. >> :END: >> ------------------------------------------------------------------------- >> In this example, the user would not be able to unfold the folder DRAWER >> because it will technically become a part of a new giant BLAH drawer. >> This may be especially annoying if is more than one screen >> long and there is no easy way to identify why unfolding does not work >> (with point at :DRAWER:). >> >> Because of this scenario, limiting before-change-functions to folded >> drawers is not sufficient. Any change in text may need to trigger >> unfolding. >> >> In the patch, I always register possible modifications in the >> blocks/drawers intersecting with the modified region + a drawer/block >> right next to the region. >> >> ----------------------------------------------------------------------- >> ----------------------------------------------------------------------- >> >> More details on the nested 'invisible text property implementation. >> >> The idea is to keep 'invisible property stack push and popping from it >> as we add/remove 'invisible text property. All the work is done in >> org-flag-region. >> >> This was originally intended for folding outlines via text properties. >> Since using text properties for folding outlines is not a good idea, >> nested text properties have much less use. As I mentioned, they do >> preserve link fontification, but I am not sure if it worth it for the >> overhead to org-flag-region. Keeping this here mostly in the case if >> someone has any ideas how it can be useful. >> >> ----------------------------------------------------------------------- >> ----------------------------------------------------------------------- >> >> More details on replaced outline-* -> org-* function calls. >> >> I have implemented org-* versions of the following functions: >> >> - outline-hide-entry >> - outline-hide-subtree >> - outline-hide-sublevels >> - outline-show-heading >> - outline-show-branches >> >> The org-* versions trivially use org-flag-region instead of >> outline-flag-region. >> >> Replaced outline-* calls where org- versions were already available: >> >> - outline-show-entry >> - outline-show-all >> - outline-show-subtree >> >> I reflected the new (including already available) functions in the >> manual and removed some defalias from org-compat.el where they are not >> needed.=20 >> >> ----------------------------------------------------------------------- >> ----------------------------------------------------------------------- >> >> Further work: >> >> 1. after-change-functions use org-hide-drawer/block-toggle to >> fold/unfold after modification. However, I just found that they call >> org-element-at-point, which slows down modifications in folded >> drawers/blocks. For example, realigning a long table inside folded >> drawer takes >1sec, while it is instant in the unfolded drawer. >> >> 2. org-toggle-custom-properties is terribly slow on large org documents, >> similarly to folded drawers on master. It should be relatively easy to >> use text properties there instead of overlays. >> >> 3. Multiple calls to before/after-change-functions is still a problem. I >> am looking into following ways to reduce this number: >> - reduce the number of elements registered as potentially modified >> + do not add duplicates to org--modified-elements >> + do not add unfolded elements to org--modified-elements >> + register after-change-function as post-command hook and remove it >> from global after-change-functions. This way, it will be called >> twice per command only. >> - determine common region containing org--modified-elements. if change >> is happening within that region, there is no need to parse >> drawers/blocks there again. >> >> P.S. >> >>>> It was mostly an annoyance, because they returned different results on >>>> the same element. Specifically, they returned different :post-blank and >>>> :end properties, which does not sound right. >>> >>> OK. If you have a reproducible recipe, I can look into it and see what >>> can be done. >> >> Recipe to have different (org-element-at-point) and >> (org-element-parse-buffer 'element) >> ------------------------------------------------------------------------- >> >> :PROPERTIES: >> :CREATED: [2020-05-23 Sat 02:32] >> :END: >> >> >> >> ------------------------------------------------------------------------- >> >> >> Best, >> Ihor >> >> Nicolas Goaziou writes: >> >>> Hello, >>> >>> Ihor Radchenko writes: >>> >>>>> As you noticed, using Org Element is a no-go, unfortunately. Parsing = an >>>>> element is a O(N) operation by the number of elements before it in >>>>> a section. In particular, it is not bounded, and not mitigated by >>>>> a cache. For large documents, it is going to be unbearably slow, too. >>>> >>>> Ouch. I thought it is faster. >>>> What do you mean by "not mitigated by a cache"? >>> >>> Parsing starts from the closest headline, every time. So, if Org parses >>> the Nth element in the entry two times, it really parses 2N elements. >>> >>> With a cache, assuming the buffer wasn't modified, Org would parse >>> N elements only. With a smarter cache, with fine grained cache >>> invalidation, it could also reduce the number of subsequent parsed >>> elements. >>> >>>> The reason I would like to utilise org-element parser to make tracking >>>> modifications more robust. Using details of the syntax would make the >>>> code fragile if any modifications are made to syntax in future. >>> >>> I don't think the code would be more fragile. Also, the syntax we're >>> talking about is not going to be modified anytime soon. Moreover, if >>> folding breaks, it is usually visible, so the bug will not be unnoticed. >>> >>> This code is going to be as low-level as it can be. >>> >>>> Debugging bugs in modification functions is not easy, according to my >>>> experience. >>> >>> No, it's not.=20 >>> >>> But this is not really related to whether you use Element or not. >>> >>>> One possible way to avoid performance issues during modification is >>>> running parser in advance. For example, folding an element may >>>> as well add information about the element to its text properties. >>>> This will not degrade performance of folding since we are already >>>> parsing the element during folding (at least, in >>>> org-hide-drawer-toggle). >>> >>> We can use this information stored at fold time. But I'm not even sure >>> we need it. >>> >>>> The problem with parsing an element during folding is that we cannot >>>> easily detect changes like below without re-parsing. >>> >>> Of course we can. It is only necessary to focus on changes that would >>> break the structure of the element. This does not entail a full parsing. >>> >>>> :PROPERTIES: >>>> :CREATED: [2020-05-18 Mon] >>>> :END: <- added line >>>> :ID: test >>>> :END: >>>> >>>> or even >>>> >>>> :PROPERTIES: >>>> :CREATED: [2020-05-18 Mon] >>>> :ID: test >>>> :END: <- delete this line >>>> >>>> :DRAWER: >>>> test >>>> :END: >>> >>> Please have a look at the "sensitive parts" I wrote about. This takes >>> care of this kind of breakage. >>> >>>> The re-parsing can be done via regexp, as you suggested, but I don't >>>> like this idea, because it will end up re-implementing >>>> org-element-*-parser. >>> >>> You may have misunderstood my suggestion. See below. >>> >>>> Would it be acceptable to run org-element-*-parser >>>> in after-change-functions? >>> >>> I'd rather not do that. This is unnecessary consing, and matching, etc. >>> >>>> If I understand correctly, it is not as easy. >>>> Consider the following example: >>>> >>>> :PROPERTIES: >>>> :CREATED: [2020-05-18 Mon] >>>> >>>> :ID: example >>>> :END: >>>> >>>> <... a lot of text, maybe containing other drawers ...> >>>> >>>> Nullam rutrum. >>>> Pellentesque dapibus suscipit ligula. >>>> >>>> Proin quam nisl, tincidunt et, mattis eget, convallis nec, purus. >>>> >>>> If the region gets deleted, the modification hooks from chars inside >>>> drawer will be called as (hook-function >>>> ). So, there is still a need to find the drawer somehow to >>>> mark it as about to be modified (modification hooks are ran before >>>> actual modification). >>> >>> If we can stick with `after-change-functions' (or local equivalent), >>> that's better. It is more predictable than `before-change-functions' and >>> alike. >>> >>> If it is a deletion, here is the kind of checks we could do, depending >>> on when they are performed. >>> >>> Before actual changes : >>> >>> 1. The deletion is happening within a folded drawer (unnecessary step >>> in local functions). >>> 2. The change deleted the sensitive line ":END:". >>> 3. Conclusion : unfold. >>> >>> Or, after actual changes : >>> >>> 1. The deletion involves a drawer. >>> 2. Text properties indicate that the beginning of the propertized part >>> of the buffer start with org-drawer-regexp, but doesn't end with >>> `org-property-end-re'. A "sensitive part" disappeared! >>> 3. Conclusion : unfold >>> >>> This is far away from parsing. IMO, a few checks cover all cases. Let me >>> know if you have questions about it. >>> >>> Also, note that the kind of change you describe will happen perhaps >>> 0.01% of the time. Most change are about one character, or a single >>> line, long. >>> >>>> The only difference between using modification hooks and >>>> before-change-functions is that modification hooks will trigger less >>>> frequently.=20 >>> >>> Exactly. Much less frequently. But extra care is required, as you noted >>> already. >>> >>>> Considering the performance of org-element-at-point, it is >>>> probably worth doing. Initially, I wanted to avoid it because setting a >>>> single before-change-functions hook sounded cleaner than setting >>>> modification-hooks, insert-behind-hooks, and insert-in-front-hooks. >>> >>> Well, `before-change-fuctions' and `after-change-functions' are not >>> clean at all: you modify an unrelated part of the buffer, but still call >>> those to check if a drawer needs to be unfolded somewhere. >>> >>> And, more importantly, they are not meant to be used together, i.e., you >>> cannot assume that a single call to `before-change-functions' always >>> happens before calling `after-change-functions'. This can be tricky if >>> you want to use the former to pass information to the latter. >>> >>> But I understand that they are easier to use than their local >>> counterparts. If you stick with (before|after)-change-functions, the >>> function being called needs to drop the ball very quickly if the >>> modification is not about folding changes. Also, I very much suggest to >>> stick to only `after-change-functions', if feasible (I think it is), per >>> above. >>> >>>> Moreover, these text properties would be copied by default if one uses= =20 >>>> buffer-substring. Then, the hooks will also trigger later in the yanked >>>> text, which may cause all kinds of bugs. >>> >>> Indeed, that would be something to handle specifically. I.e., >>> destructive modifications (i.e., those that unfold) could clear such >>> properties. >>> >>> It is more work. I don't know if it is worth the trouble if we can get >>> out quickly of `after-change-functions' for unrelated changes. >>> >>>> It was mostly an annoyance, because they returned different results on >>>> the same element. Specifically, they returned different :post-blank and >>>> :end properties, which does not sound right. >>> >>> OK. If you have a reproducible recipe, I can look into it and see what >>> can be done. >>> >>> Regards, >>> >>> --=20 >>> Nicolas Goaziou >> >> --=20 >> Ihor Radchenko, >> PhD, >> Center for Advancing Materials Performance from the Nanoscale (CAMP-nano) >> State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaoton= g University, Xi'an, China >> Email: yantar92@gmail.com, ihor_radchenko@alumni.sutd.edu.sg > > --=20 > Ihor Radchenko, > PhD, > Center for Advancing Materials Performance from the Nanoscale (CAMP-nano) > State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong= University, Xi'an, China > Email: yantar92@gmail.com, ihor_radchenko@alumni.sutd.edu.sg --=20 Ihor Radchenko, PhD, Center for Advancing Materials Performance from the Nanoscale (CAMP-nano) State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong U= niversity, Xi'an, China Email: yantar92@gmail.com, ihor_radchenko@alumni.sutd.edu.sg