Splitting large org file into smaller ones

Splitting large org file into smaller ones

[Marvin M. Doyley]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]

Hi there,

Is there an efficient way to split a large org file into smaller ones?

Lets say I have a master file with the following


* Notes
**  Candidate A
 - Strong technical background
 - High H-factor
 - good funding
- More established investigator
- Huge startup need $$

**  Candidate B
- lower startup need
- signal processing  
- young investigator


How do I export the subtree to separate org files, say

- Candidate-a.org <http://candidate-a.org/>
- Candidate-b.org <http://candidate-b.org/>


Thanks

M

[-- Attachment #2: Type: text/html, Size: 1487 bytes --]

Re: Splitting large org file into smaller ones

[Marcin Borkowski]

On 2020-05-30, at 11:51, Marvin M. Doyley <mdoyley@ur.rochester.edu> wrote:

> Hi there,
>
> Is there an efficient way to split a large org file into smaller ones?

I do not know about "efficient", but you might find my org-one-to-many
library useful: https://github.com/mbork/org-one-to-many
(Note that I don't wven know if it works with newer versions of Org; if
not, drop me a line, maybe I'll be able to fix it.)

Best,

-- 
Marcin Borkowski
http://mbork.pl

Re: Splitting large org file into smaller ones

[Tim Visher]

[-- Attachment #1: Type: text/plain, Size: 629 bytes --]

On Sat, May 30, 2020 at 6:09 AM Marvin M. Doyley <mdoyley@ur.rochester.edu>
wrote:

> Is there an efficient way to split a large org file into smaller ones?
>
> Lets say I have a master file with the following
>
> * Notes
> **  Candidate A
> …
> **  Candidate B
> …
>
> How do I export the subtree to separate org files, say
>
> - Candidate-a.org <http://candidate-a.org>
> - Candidate-b.org <http://candidate-b.org>
>

This seems like it would be fairly trivial with a keyboard macro. Certainly
not the most satisfying solution but if you're not doing this all the time
it might be the closest option.

[-- Attachment #2: Type: text/html, Size: 1140 bytes --]

Re: Splitting large org file into smaller ones

[William Denton]

On 30 May 2020, Marvin M. Doyley wrote:

> How do I export the subtree to separate org files, say
>
> - Candidate-a.org <http://candidate-a.org/>
> - Candidate-b.org <http://candidate-b.org/>

I'll throw out an uninformed suggestion of a possible direction:  archiving 
subtrees.  I guess you'd have to keep changing the name of the target archive 
file, or rename it after you'd moved a tree to it, but at least it's getting a 
whole tree out of the way at once.

Bill
--
William Denton :: Toronto, Canada   ---   Listening to Art: https://listeningtoart.org/
https://www.miskatonic.org/         ---   GHG.EARTH: https://ghg.earth/
Caveat lector.                      ---   STAPLR: https://staplr.org/

Re: Splitting large org file into smaller ones

[Julius Dittmar]

Am 30.05.20 um 17:36 schrieb William Denton:
> On 30 May 2020, Marvin M. Doyley wrote:
>
>> How do I export the subtree to separate org files, say
>>
>> - Candidate-a.org <http://candidate-a.org/>
>> - Candidate-b.org <http://candidate-b.org/>
>
> I'll throw out an uninformed suggestion of a possible direction: 
> archiving subtrees.  I guess you'd have to keep changing the name of
> the target archive file, or rename it after you'd moved a tree to it,
> but at least it's getting a whole tree out of the way at once.

If you'll end up naming the files yourself, then I suggest killing the
whole subtree by C-c C-x C-w (|org-cut-subtree|) with point in the
heading, opening the new file, then yanking the subtree by C-c C-x C-y
(org-paste-subtree).

You could also use the refile mechanism for that, though I never managed
to wrap my head around that one.

HTH,
Julius

Re: Splitting large org file into smaller ones

[Diego Zamboni]

Hi Marvin,

I replied to a similar question some time ago, here's my answer (with
updated links since the files have changed in the meantime):

I do something similar in my ox-leanpub-book module [1], which exports
each top-level heading to a different file. The general idea is to use
=org-map-entries= to loop over the entire buffer [2]. The function you
call can then check whether the current entry is a header at the level
you want [3] and then export it to the corresponding file. The title
can be used to deduct the filename [4].

I found that I had to mark the entire subtree before calling the
export function [5], otherwise the headline was not getting included
in the export.

I based my code originally on this blog post, which might be a simpler
starting point:
https://medium.com/@lakshminp/publishing-a-book-using-org-mode-9e817a56d144
- this code does not select the entire subtree before exporting, which
means only the contents of the section is exported, but not the
headline itself.

I wrote a simplified function which puts the above ideas together,
feel free to use it:
https://gist.github.com/zzamboni/2e6ac3c4f577249d98efb224d9d34488

Hope this helps!
--Diego

[1] https://github.com/zzamboni/ox-leanpub/tree/book-and-markua
[2] https://github.com/zzamboni/ox-leanpub/blob/f7ae0f86f517a321a5150e1c712554b12acf144b/ox-leanpub-book.el#L216-L217
[3] https://github.com/zzamboni/ox-leanpub/blob/f7ae0f86f517a321a5150e1c712554b12acf144b/ox-leanpub-book.el#L145
[4] https://github.com/zzamboni/ox-leanpub/blob/f7ae0f86f517a321a5150e1c712554b12acf144b/ox-leanpub-book.el#L154-L158
[5] https://github.com/zzamboni/ox-leanpub/blob/f7ae0f86f517a321a5150e1c712554b12acf144b/ox-leanpub-book.el#L192

On Sat, May 30, 2020 at 12:08 PM Marvin M. Doyley
<mdoyley@ur.rochester.edu> wrote:
>
> Hi there,
>
> Is there an efficient way to split a large org file into smaller ones?
>
> Lets say I have a master file with the following
>
>
> * Notes
> **  Candidate A
>  - Strong technical background
>  - High H-factor
>  - good funding
> - More established investigator
> - Huge startup need $$
>
> **  Candidate B
> - lower startup need
> - signal processing
> - young investigator
>
>
> How do I export the subtree to separate org files, say
>
> - Candidate-a.org
> - Candidate-b.org
>
>
> Thanks
>
> M

Re: Splitting large org file into smaller ones

[Marvin M. Doyley]

Hi Marcin,

I appreciate this, I will give it a go.

Thanks,
M

> On May 30, 2020, at 9:13 AM, Marcin Borkowski <mbork@mbork.pl> wrote:
> 
> 
> On 2020-05-30, at 11:51, Marvin M. Doyley <mdoyley@ur.rochester.edu> wrote:
> 
>> Hi there,
>> 
>> Is there an efficient way to split a large org file into smaller ones?
> 
> I do not know about "efficient", but you might find my org-one-to-many
> library useful: https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_mbork_org-2Done-2Dto-2Dmany&d=DwIBAg&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=iHWQZlKMtTtC96Iqx0ymA0VpA5m6A73UehcV_vuh1tM&s=6rsipfH8vh-XeWLb1NEWAcdpcTFAfyUOLHD5Pkmea2o&e= 
> (Note that I don't wven know if it works with newer versions of Org; if
> not, drop me a line, maybe I'll be able to fix it.)
> 
> Best,
> 
> -- 
> Marcin Borkowski
> https://urldefense.proofpoint.com/v2/url?u=http-3A__mbork.pl&d=DwIBAg&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=iHWQZlKMtTtC96Iqx0ymA0VpA5m6A73UehcV_vuh1tM&s=VLUzcQS7YObsdTrrwTldDMYEUZJlqE3mMIxxY8tN9JQ&e= 

Re: Splitting large org file into smaller ones

[Marvin M. Doyley]

Cool! I will try this too

Thanks

Cheers,
M

> On May 30, 2020, at 6:23 PM, Diego Zamboni <diego@zzamboni.org> wrote:
> 
> Hi Marvin,
> 
> I replied to a similar question some time ago, here's my answer (with
> updated links since the files have changed in the meantime):
> 
> I do something similar in my ox-leanpub-book module [1], which exports
> each top-level heading to a different file. The general idea is to use
> =org-map-entries= to loop over the entire buffer [2]. The function you
> call can then check whether the current entry is a header at the level
> you want [3] and then export it to the corresponding file. The title
> can be used to deduct the filename [4].
> 
> I found that I had to mark the entire subtree before calling the
> export function [5], otherwise the headline was not getting included
> in the export.
> 
> I based my code originally on this blog post, which might be a simpler
> starting point:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__medium.com_-40lakshminp_publishing-2Da-2Dbook-2Dusing-2Dorg-2Dmode-2D9e817a56d144&d=DwIBaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=249GZv4Zgx7zyEm64J3m2eEYuchXSdodRltw_gmrqlg&s=exfuDxb81D5anZ5KD-hCvUI4PRaQxsb812EaBW3b9p4&e= 
> - this code does not select the entire subtree before exporting, which
> means only the contents of the section is exported, but not the
> headline itself.
> 
> I wrote a simplified function which puts the above ideas together,
> feel free to use it:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__gist.github.com_zzamboni_2e6ac3c4f577249d98efb224d9d34488&d=DwIBaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=249GZv4Zgx7zyEm64J3m2eEYuchXSdodRltw_gmrqlg&s=ARDFenWJJT9XTsBKAReDC6bDx9VwbP3ADCw3o-_hQbY&e= 
> 
> Hope this helps!
> --Diego
> 
> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_zzamboni_ox-2Dleanpub_tree_book-2Dand-2Dmarkua&d=DwIBaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=249GZv4Zgx7zyEm64J3m2eEYuchXSdodRltw_gmrqlg&s=peXBLnzs1ABFnCABzVUsley3-VMFDR0_2oLF3TgiBPM&e= 
> [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_zzamboni_ox-2Dleanpub_blob_f7ae0f86f517a321a5150e1c712554b12acf144b_ox-2Dleanpub-2Dbook.el-23L216-2DL217&d=DwIBaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=249GZv4Zgx7zyEm64J3m2eEYuchXSdodRltw_gmrqlg&s=3K2q6g7FxxjjxKJZtqdsE5bpMFT6RTf3S4mmyldNdLw&e= 
> [3] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_zzamboni_ox-2Dleanpub_blob_f7ae0f86f517a321a5150e1c712554b12acf144b_ox-2Dleanpub-2Dbook.el-23L145&d=DwIBaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=249GZv4Zgx7zyEm64J3m2eEYuchXSdodRltw_gmrqlg&s=QOtk1D413Gju7Agm6zgbAMlIwww6e-UV1SZsjomzuZA&e= 
> [4] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_zzamboni_ox-2Dleanpub_blob_f7ae0f86f517a321a5150e1c712554b12acf144b_ox-2Dleanpub-2Dbook.el-23L154-2DL158&d=DwIBaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=249GZv4Zgx7zyEm64J3m2eEYuchXSdodRltw_gmrqlg&s=ynLKEu4CLjIJqR3y3zl2UMEkS_onMBSSb2TDwOaixt4&e= 
> [5] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_zzamboni_ox-2Dleanpub_blob_f7ae0f86f517a321a5150e1c712554b12acf144b_ox-2Dleanpub-2Dbook.el-23L192&d=DwIBaQ&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=249GZv4Zgx7zyEm64J3m2eEYuchXSdodRltw_gmrqlg&s=quomDHaK1zPArr8wu5m8uwkgug6NZ6eDChE5VMZwrG4&e= 
> 
> On Sat, May 30, 2020 at 12:08 PM Marvin M. Doyley
> <mdoyley@ur.rochester.edu> wrote:
>> 
>> Hi there,
>> 
>> Is there an efficient way to split a large org file into smaller ones?
>> 
>> Lets say I have a master file with the following
>> 
>> 
>> * Notes
>> **  Candidate A
>> - Strong technical background
>> - High H-factor
>> - good funding
>> - More established investigator
>> - Huge startup need $$
>> 
>> **  Candidate B
>> - lower startup need
>> - signal processing
>> - young investigator
>> 
>> 
>> How do I export the subtree to separate org files, say
>> 
>> - Candidate-a.org
>> - Candidate-b.org
>> 
>> 
>> Thanks
>> 
>> M

Re: Splitting large org file into smaller ones

[Marvin M. Doyley]

Thanks Bill

> On May 30, 2020, at 11:36 AM, William Denton <wtd@pobox.com> wrote:
> 
> On 30 May 2020, Marvin M. Doyley wrote:
> 
>> How do I export the subtree to separate org files, say
>> 
>> - Candidate-a.org <https://urldefense.proofpoint.com/v2/url?u=http-3A__candidate-2Da.org_&d=DwIBAg&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=bIZSMkqv64C1tMXjpulbll5ho0_g78uEvbSGB385-YQ&s=qfP2YDuIkAJM2S_aXOWV8p65RBNCtPMSSd8Y7h4nWPU&e= >
>> - Candidate-b.org <https://urldefense.proofpoint.com/v2/url?u=http-3A__candidate-2Db.org_&d=DwIBAg&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=bIZSMkqv64C1tMXjpulbll5ho0_g78uEvbSGB385-YQ&s=m7juMWMKRPTM8hGOlYOQJy31TXiDJ52t2-Uu7t6i9EQ&e= >
> 
> I'll throw out an uninformed suggestion of a possible direction:  archiving subtrees.  I guess you'd have to keep changing the name of the target archive file, or rename it after you'd moved a tree to it, but at least it's getting a whole tree out of the way at once.
> 
> Bill
> --
> William Denton :: Toronto, Canada   ---   Listening to Art: https://urldefense.proofpoint.com/v2/url?u=https-3A__listeningtoart.org_&d=DwIBAg&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=bIZSMkqv64C1tMXjpulbll5ho0_g78uEvbSGB385-YQ&s=3SYzbH-wXEZ1etjDC-J5keIPie3ll0ngiWihrTPputs&e= https://urldefense.proofpoint.com/v2/url?u=https-3A__www.miskatonic.org_&d=DwIBAg&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=bIZSMkqv64C1tMXjpulbll5ho0_g78uEvbSGB385-YQ&s=Eyy6EE7boaclNVgLZFy6YP4RM57VQQMr5pBFDOdY5Zo&e=          ---   GHG.EARTH: https://urldefense.proofpoint.com/v2/url?u=https-3A__ghg.earth_&d=DwIBAg&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=bIZSMkqv64C1tMXjpulbll5ho0_g78uEvbSGB385-YQ&s=WZpqujYA7GRmIP00WVRuyH4g-hRRFcrNhWV9O27p0F0&e= Caveat lector.                      ---   STAPLR: https://urldefense.proofpoint.com/v2/url?u=https-3A__staplr.org_&d=DwIBAg&c=kbmfwr1Yojg42sGEpaQh5ofMHBeTl9EI2eaqQZhHbOU&r=T41F_5QsIVBGYhPPUkgYHUp9iPHgs2rOCjs7rfKaTMU&m=bIZSMkqv64C1tMXjpulbll5ho0_g78uEvbSGB385-YQ&s=sKITlXUZ5Q6MQoPb4RFPynZVydyakhlNKabGQieVC7c&e= 

Re: Splitting large org file into smaller ones

[John Kitchin]

This seems like an important thing to get right, so I would hesitate to
do this very automatically.

I would write a function that does what you want, maybe like this:

#+BEGIN_SRC emacs-lisp
(defun subtree-to-file ()
  (interactive)
  (org-narrow-to-subtree)
  (goto-char (point-min))
  (let* ((heading (fifth (org-heading-components)))
	 (fname (concat (mapconcat 'downcase (split-string heading) "-") ".org")))
    (when
	(y-or-n-p (format "Write subtree to %s" fname))
      (org-cut-subtree)
      (with-temp-file fname
	(org-yank))))
  (widen)
  (when (not (org-at-heading-p))
    (org-next-visible-heading 1)))
#+END_SRC

This will generate filenames based on the heading name, and move the
content to that file if you agree with the filename. It does not check
if the fname exists, which may be a good idea. It does not do anything
fancy like promote the heading, it just moves it.

It is a good idea to do this on a copy of the original file, so that you
have an un-altered version in case it does not go as planned.


Marvin M. Doyley <mdoyley@ur.rochester.edu> writes:

> Hi there,
>
> Is there an efficient way to split a large org file into smaller ones?
>
> Lets say I have a master file with the following
>
>
> * Notes
> **  Candidate A
>  - Strong technical background
>  - High H-factor
>  - good funding
> - More established investigator
> - Huge startup need $$
>
> **  Candidate B
> - lower startup need
> - signal processing
> - young investigator
>
>
> How do I export the subtree to separate org files, say
>
> - Candidate-a.org <http://candidate-a.org/>
> - Candidate-b.org <http://candidate-b.org/>
>
>
> Thanks
>
> M


--
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu