Integrating eev and org-capture/org-capture-set-target-location

Integrating eev and org-capture/org-capture-set-target-location

[Eduardo Ochs]

Hi Jean Louis,

this is an answer to these messages that you sent:

  https://lists.gnu.org/archive/html/help-gnu-emacs/2021-06/msg00028.html
  https://lists.gnu.org/archive/html/help-gnu-emacs/2021-06/msg00055.html

I decided to create a new thread, and I'm working on org-capture
first. I'll leave org-protocol for later.

I watched Prot's video about org-capture -
https://www.youtube.com/watch?v=qCdScs4YO8k - and I am trying to
implement a function that works as a sexp hyperlink to a "target"
structure like the ones that appear in org-capture-templates. These
target structures are documented here:

  (find-evardescr 'org-capture-templates)
  (find-evardescr 'org-capture-templates "\ntarget")
  (find-evardescr 'org-capture-templates "\ntarget" "file+headline")
  (find-orgnode "Capture templates" "file+headline")
  (find-orgnode "Template elements" "\ntarget")

In Prot's video he uses targets like these,

  (file+headline "tasks.org" "Task list")
  (file+headline "tasks.org" "Links to buffers/files")

and I am trying to write a function `find-orgctarget' such that

  (find-orgctarget '(file+headline "tasks.org" "Task list"))

would open the file

  (concat org-directory "/tasks.org")

and then find the headline "* Task list" in it, and put the point in
the line following that headline. After a few very naive attempts I
got this definition for `find-orgctarget':

  (require 'org-capture)

  (defun find-orgctarget (target)
    (org-capture-set-target-location target)
    (org-capture-finalize t))

but when I run this

  (find-orgctarget '(file+headline "tasks.org" "Task list"))

I get this error:

  org-capture-finalize: This does not seem to be a capture buffer for Org mode

Do you - or anyone else - know how to fix that?

  Thanks in advance!
    Eduardo Ochs
    http://angg.twu.net/#eev
    http://angg.twu.net/eev-intros/find-eev-quick-intro.html#3
    (find-eev-quick-intro "3. Elisp hyperlinks")

Re: Integrating eev and org-capture/org-capture-set-target-location

[Jean Louis]

* Eduardo Ochs <eduardoochs@gmail.com> [2021-06-04 06:41]:
> I watched Prot's video about org-capture -
> https://www.youtube.com/watch?v=qCdScs4YO8k - and I am trying to
> implement a function that works as a sexp hyperlink to a "target"
> structure like the ones that appear in org-capture-templates. These
> target structures are documented here:
> 
>   (find-evardescr 'org-capture-templates)

My opinion is that org-capture-templates are user unfriendly
(see description of org-capture-templates). Since months I have
stopped using it without regret. 

If I would need a template I would or could just add template
column to the database type. It would be settings that is in the
database for specific user among possibly many users. There is
variety of templating options to be used.

One templating option could be the Emacs library `skeleton'

(setq name "John")

(define-skeleton my-skel
    "Test"
  nil
  "Hello " some "")

Other templating option could be my rcd-template.el package with
some variables to be expanded on the fly and without limit.

>   (find-evardescr 'org-capture-templates "\ntarget")
>   (find-evardescr 'org-capture-templates "\ntarget" "file+headline")

When capturing external text or information let us say from the
browser, maybe from Xpdf viewer what one as a user needs is what?
A simple click. Later user can sort or edit things. Keeping
things simple. Capturing with Org is not quite simple. I can
think of so many different ways to capture things in some more
reasonable manner than what is Org capturing.

Devising your own function is maybe better way to go.

>   (find-orgnode "Capture templates" "file+headline")
>   (find-orgnode "Template elements" "\ntarget")

Those must be new eev functions.

> In Prot's video he uses targets like these,

>   (file+headline "tasks.org" "Task list")
>   (file+headline "tasks.org" "Links to buffers/files")

I have thousands of sets, which means parent nodes. So when I am
capturing, I am capturing and have full text search access to
anything. Then I can mark many nodes at once with single key and put
nodes into a set or category. Imagine if I would now need to:

- define the headline like "Task list" in the file, then go to
  org-capture-templates and again define the headline where stuff
  needs to be captured, then if I change the headline in the buffer to
  remember to go back to org-capture-templates and to define stuff to
  be captured and so on; too much; as I said I have 1000+ sets or
  categories where things could be captured; but WWW hyperlinks are
  captured in just one single set from where they could be fast-sorted
  in other sets; 

  Capturing sessions are normally thematically related to a period of
  time on my side. If I am researching subject X, I may capture 20
  items related to that subject in specific period of time. Moving
  over items by time shows me similar captured items. Then by pressing
  mmmmm I can mark 5 items or mmm 3 more, and "c p" change parent to a
  different set. 

  I don't think about files, as I don't use them with any capture.

  I don't think about exact headings, and need not put any care to
  re-write such a heading. 

  Yet categorizing is fast.

Keeping links to buffers and files in one single file together
with task lists does not seem sound. It implies that user has to
make now even more efforts to find those links.

Each task is separate object with separate type on my side, each
link to buffer is separate object and type. They can be in one
set, but still remain separate. They are not in the same file --
why? Because they are different types. Unique in their own. This
is good concept, what makes it impractical with files is that
opening or handling files is not integrated well in the
system. Would it be better integrated people would store
different types in different files and that those are different
files it would not bother them, they would not even think of
files, they would just think of editing, not even saving things.

When working with a database, storing objects of different type
separately is integral to the system. And yet different objects
may appear "together" and can be used together. That is where I
say that Org wants to be a database, but is not. A lot of
programming has been done to make the Org more or less structured
database where it is not. 

Keywords like TODO/DONE make the heading a task or completed
task. Thus it is of one type. Accidental shift-left or shift-right and
the type is already changed, now it is not anymore a type of a task,
something to do, it is just a heading, maybe just a note. Type
disappeared, gone. Cannot be found as the type of "task" any more. I
find that too much low-level for users who need a higher level
friendlier interface. It should be either of Task type or Note type or
something else. 

Like is it a meeting? Then better have an exact meeting type, not
easily changeable, it is a meeting, it had related date, time, related
people to meet, an event in life. It should not be "Note" by a key
press. It is an event that had to take place, did happen or maybe not,
and as such should remain for future as it creates a background
knowledge of all people involved.

Same principle I would employ with capturing to files, I would keep
things separate by their types in different files. Reason why people
put everything in one file is rather to escape the effort of finding
things in multiple files, not the fact that keeping variety of things
in one file is comfortable in itself, it is not. Such files cannot be
shared to other people easily, that is one good example, private
research about explosives one does not want to print together with tax
letters. Hyperlinks to dating sites one does not want to keep together
with personal messages to the loved one which she/he could eventually
see on user's computer.

> and I am trying to write a function `find-orgctarget' such that
> 
>   (find-orgctarget '(file+headline "tasks.org" "Task list"))

> would open the file
> 
>   (concat org-directory "/tasks.org")
> 
> and then find the headline "* Task list" in it, and put the point in
> the line following that headline. After a few very naive attempts I
> got this definition for `find-orgctarget':
> 
>   (require 'org-capture)
> 
>   (defun find-orgctarget (target)
>     (org-capture-set-target-location target)
>     (org-capture-finalize t))
> 
> but when I run this
> 
>   (find-orgctarget '(file+headline "tasks.org" "Task list"))
> 
> I get this error:
> 
>   org-capture-finalize: This does not seem to be a capture buffer
>   for Org mode

It says in the function `org-capture-finalize':

  (unless (and org-capture-mode
	       (buffer-base-buffer (current-buffer)))
    (error "This does not seem to be a capture buffer for Org mode"))

Maybe it is not a real error that was just designated as "error". It
asks you to be in `org-capture-mode' as it maybe wants to `finalize'
something, I did not look into the obfuscation yet.

What you want is to find a headling:

(find-orgctarget '(file+headline "tasks.org" "Task list"))

There is function:

(org-find-exact-headline-in-buffer HEADING &optional BUFFER POS-ONLY)

(defun my-org-find-exact-headline (file heading) ;; headline/heading????
  (find-file file)  ;; if extension is .org it would 
  (let ((marker (org-find-exact-headline-in-buffer heading)))
    (goto-char marker)))

And this works well:

(my-org-find-exact-headline "~/tmp/new.org" "Something")

with file:
----------
* Something

Somet text

* Here is more

More text

--------------- 

Thus finding works. But maybe you wish to use capture facility
and that requires more thinking.

To put a point after the headline you can do by adding `forward-line'
function. Just that in Org files there are often properties, 

I would not recommend finding by exact heading as heading
change. Using function `org-find-entry-with-id' is better, as it
looks for property with exact value. 

I don't use Org files, but I can export information into Org
files, each item has its ID number:

** TODO Data
   :PROPERTIES:
   :TABLE: hyobjects
   :COLUMN: hyobjects_text
   :ID: 37704
   :CUSTOM_ID: 37704
   :END:

Maybe you wish to use other functions such as `org-forward-paragraph'
as that one would move after the :END: above. But better is
(org-end-of-meta-data t) as that would move you from heading to end of
the meta data if any. That is the place where you wish to place new
stuff.

That would however mean that you are prepending, your newest items
would appear just under the heading, not as last items.

* My heading

Newest
Newer
New
Old

Maybe is better to jump to headings by using ID number as headings
would be changing, and ID would never change (unless it is
changed). 

When working with the database then text is edited, some fundamental
attributes are not edited. With files, anything can be edited.

- My heading is text -- changeable.

- Heading is attribute or place where it is stored, in the
  column: hyobjects_name (before it was hlinks_name) -- this is
  not changeable, it is fixed place. There are other "places" or
  columns, such as text, report, author, etc.

- Heading has its ID number automatically assigned to specific
  entry. Same ID applies to other columns. 

- Because of those constraints it is trivial to capture for any
  change, the user who changed it, date modified, previous version
  of the heading, and so on.

Then I can inspect how many times did I change headings in the
`vc' version control table:

SELECT count(1) FROM vc WHERE vc_column = 'hyobjects_name' OR vc_column = 'hlinks_name';
 count 
-------
   649

Well 649 times? That is what I speak about. Would I change the
heading anywhere the `org-capture-templates' would need to get
changed as well, and that is manual work. So I don't agree with
the design of `org-capture-templates' as it gives work to users. 

User view point at computer as something to minimize the work,
not to increase the work. That is why we use editing modes, they
help in minimizing the work. Obviously programmers may be so
enthusiastic to forget about the basic principle and would like
to program for the sake of program, not sake of the user. That is
where troubled program parts such as `org-capture' and
`org-capture-templates' get born:

- M-x org-capture -- presents blocking interface to user, instead
  of allowing user to move from buffer to buffer. There is
  contradiction between the `org-capture-templates' that gives
  incentives to create so many templates and the blocking limited
  interface that disables user to verify buffers while using the
  menu. 

  Solution: Providing a simple read-only Org type interface with
  Org hyperlinks to org-capture functions is trivial and
  non-blocking, but author of org-capture-template likes the
  blocking one.

- Defining `org-capture-templates' is everything but user
  friendly. Many will consider Lisp not friendly, but this is not
  even Lisp. It is small langugage in itself. A surprising "Oh,
  not again" for users who anyway found it hard to understand
  Org.

  Instead, the templates should be definable in the Org file directly,
  not in the variable. If there is already plethora of `org-find...'
  functions then why not use Org to define itself in templates. Why
  not use any Org file and its heading as template. Then user could
  write Org file and use it as templates.

entry type? There is no difference to me in the type of "entry" as Org
node as a headline with template being:

"* %?

%a"

with the "type" of "item" with template being "- %?"

I don't see a clear "type" there, I just see different template.

A "checkitem" type is yet another template "- [ ] %?" and not a type
really.

Then "plain" type is not a type it is just empty template. Oh mamma
mia. 

Those "types" should be or are redundant as they are just a template.

(defun my-org-find-exact-headline (file heading)
  "Jumps to exact headline first things after meta data"
  (find-file file)
  (let ((marker (org-find-exact-headline-in-buffer heading)))
    (goto-char marker)
    (org-end-of-meta-data t)))

(my-org-find-exact-headline "~/tmp/new.org" "Something")

Now I think you may wish to append it, not prepend it...


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/

Re: Integrating eev and org-capture/org-capture-set-target-location

[Eduardo Ochs]

On Fri, 4 Jun 2021 at 03:02, Jean Louis <bugs@gnu.support> wrote:
>
> >   (find-orgnode "Capture templates" "file+headline")
> >   (find-orgnode "Template elements" "\ntarget")
>
> Those must be new eev functions.

 Hi Jean-Louis,

`find-orgnode' is almost 20 years old, actually...
It is defined with `code-c-d', that is from 1995 (!!!) =P.
The idea is explained here:

  (find-eev-quick-intro "9.1. `code-c-d'")
  (find-eev-quick-intro "9.2. Extra arguments to `code-c-d'")

and here:

  (find-eev2020video "8:20" "`code-c-d' defines several new functions")
  (find-eev2020video "8:34" "the result is the name of one of the functions")
  (find-eev2020video "8:45" "`find-orggitfile'")
  (find-eev2020video "9:09" "prepends this string")
  (find-eev2020video "9:40" "`find-orggitnode'")
  (find-eev2020video "17:20" "find-code-c-d shows the code")

Are the sexp hyperlinks to videos working well for you?
By the way, if you would like to experiment with putting your own
videos online you can create a variant of this function,

  (find-eev "eev-tlinks.el" "find-eevvideo-links")
  http://angg.twu.net/eev-current/eev-tlinks.el.html#find-eevvideo-links

called, say, find-jeanlouisvideo-links, by replacing the
"http://angg.twu.net/eev-videos/%s.mp4" in find-eevvideo-links by
something else.

I'll answer the rest in other e-mails!

  Cheers, =)
    Eduardo Ochs
    http://angg.twu.net/#eev
    http://angg.twu.net/2021-video-links.html

Re: Integrating eev and org-capture/org-capture-set-target-location

[Jean Louis]

* Eduardo Ochs <eduardoochs@gmail.com> [2021-06-04 19:09]:
> On Fri, 4 Jun 2021 at 03:02, Jean Louis <bugs@gnu.support> wrote:
> >
> > >   (find-orgnode "Capture templates" "file+headline")
> > >   (find-orgnode "Template elements" "\ntarget")
> >
> > Those must be new eev functions.

Not working here, eev mode is always on. Maybe I did not load some
library?

eval: Symbol’s function definition is void: find-orgnode

> Are the sexp hyperlinks to videos working well for you?

I am in developed country for a while, video downloading is anyway
slow and not accessible. Usually I will first download video and then
watch it. I can upload easier than download. Links work probably and I
don't use them as it is not feasible to wait too long, you know.

> By the way, if you would like to experiment with putting your own
> videos online you can create a variant of this function,
> 
>   (find-eev "eev-tlinks.el" "find-eevvideo-links")
>   http://angg.twu.net/eev-current/eev-tlinks.el.html#find-eevvideo-links

Everything ever that comes along is index on my side into the Dynamic
Knowledge Repository. And I can create the eev style links if I want,
but I don't, as other people will not use them.

I am working on link creation depending of the mode. Usually I split
window in two, in one I open Hyperscope, press "w" and link is copied
in other video, here is example, title on top, link down. Link without
title is under-described.

Hans-Peter Dürr "Weil es ums Ganze geht" // Vortrag lang // GLOBART 2011
https://www.youtube.com/watch?v=RKma6xCTIBE

Now my idea is that hyperlinking should be sensitive to the mode, so
if the other mode is mail-mode or message mode, the hyperlink would
look like that above.

But if the other mode is Markdown, the hyperlink has to be in Markdown
style, if it is Org mode, it has to be killed and yanked in Org
markup, or Asciidoc markup, if HTML then in HTML markup. Isn't that a
good idea? User could press prefix C-u and get a choice of link
format, then I could choose eev link. That is how it should be. 

(defun wrs-hyperlink-by-mode (name link)
  "Insert hyperlink formatted corresponding to major mode."
  (cond ((eq major-mode 'adoc-mode) (format "%s[%s]" link name))
	((eq major-mode 'org-mode) (format "[[%s][%s]]" link name))
	((eq major-mode 'markdown-mode) (format "[%s](%s \"%s\")" name link name))
	((eq major-mode 'html-mode) (format "<a href=\"%s\">%s<a>" link name))
	((eq major-mode 'emacs-lisp-mode) (format ";; %s\n;; %s\n" name link))
	(t (format "\n\n<a href=\"%s\">%s<a>\n\n" link name))))

eev is not yet there. You can see that link may be inserted even into
emacs-lisp-mode as commented one. Default is HTML link, but I think it
should be the textual one.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/

Re: Integrating eev and org-capture/org-capture-set-target-location

[Eduardo Ochs]

On Fri, 4 Jun 2021 at 15:17, Jean Louis <bugs@gnu.support> wrote:
>
> Not working here, eev mode is always on. Maybe I did not load some
> library?
>
> eval: Symbol’s function definition is void: find-orgnode
>
> > Are the sexp hyperlinks to videos working well for you?

Ouch! My fault, sorry!

I was sure that there was a line like this

  (code-c-d "org" (ee-locate-library "org") :info "org")

in the list of `code-c-d's that are run by default, that is here:

  (find-eev "eev-code.el" "code-c-d-s")
  http://angg.twu.net/eev-current/eev-code.el.html#code-c-d-s

But no, my (code-c-d "org" ...) is only in my ~/.emacs. I'll fix this
in the next eev version - but a temporary solution is to just execute
the code-c-d sexp above. Try this, and after that try again the
`find-orgnode' sexp hyperlinks from the other e-mail... let me copy
them here for convenience:

  (find-orgnode "Capture templates" "file+headline")
  (find-orgnode "Template elements" "\ntarget")



About the videos: nowadays I have a good connection and when I'm home
I can usually open a video on youtube in about 10 seconds, but this is
a new situation for me, and I have lots of reasons for wanting to make
everything in eev usable in situations in which the internet is very
slow or intermittent - like in the campus in which I work, or more
precisely in which I work when we're not in quarentine...

The docs of eev refer to some videos - the eight ones listed here:

  http://angg.twu.net/eev-intros/find-videos-intro.html#1
  (find-videos-intro "1. Some videos")

All of them can be downloaded with wget, and a sexp like this one

  (find-eevvlinksvideo "0:00")

creates a temporary buffer with several ways to access the video -
including a way to wget it from my site as an MP4 file, and to play
the local copy of the MP4. In this case the shell commands in the
temporary buffer are these ones,

  export S=$HOME/snarf
  mkdir -p $S/http/angg.twu.net/eev-videos/
  cd       $S/http/angg.twu.net/eev-videos/
  wget -nc 'http://angg.twu.net/eev-videos/2021-video-links.mp4'

and the video has 15 minutes and 29 megabytes. Downloading videos from
youtube requires an updated version of youtube-dl and involves a lot
of overhead; downloading an MP4 with wget is much simpler - and I hope
that this becomes a more popular way to share videos about Emacs.

So: if you have 29 megabytes of bandwidth to spare, try the
`find-eevvlinksvideo' sexp above! =)

  Cheers,
    Eduardo Ochs
    http://angg.twu.net/#eev
    http://angg.twu.net/2021-video-links.html

Re: Integrating eev and org-capture/org-capture-set-target-location

[Jean Louis]

Is this the way to make an universal link launchable by eev key bindings?

(browse-url "https://www.example.com")

For me that sounds 50% good, following would be better, more visual
and descriptive:

(WWW "https://www.example.com" "Example Website")

And this link, should be self-updateable:

(WWW "https://www.example.com")

If user presses prefix like C-u M-e then such link would update itself
into:

(WWW "https://www.example.com" "Example Website")

You could study these functions and find out how to fetch the URL, how
to extract title, functions are not perfect, it would be better to
decode some HTML.

(defun hyperscope-fetch-title (url)
  "Return title for URL or if there is no match url."
  (let* ((string (hyperscope-url-to-string url)) 
	 (match (string-match "<title>\\(.*\\)</title>" string)))
    (if match
	(replace-regexp-in-string "<title>\\|</title>" "" (match-string 0 string))
      url)))

(defun hyperscope-url-to-string (url)
  "Fetch URL and return as string."
  (url-retrieve-synchronously url)
  (let ((buffer (url-retrieve-synchronously url)))
    (with-current-buffer buffer
      (buffer-string))))

Some highlighting:

(defun rcd-highlight-list (list)
  "Uses LIST to highlight strings in buffer."
  (hi-lock-mode)
  (let* ((list (delete "" list))
	(highlights hi-lock-face-defaults))
    (while list
      (highlight-regexp (regexp-quote (pop list)) (pop highlights)))))

(rcd-highlight-list '("WWW" "Set"))

I suggest exploring highlighting options as that could make the eev
buffer more visual.

The eev is a hypertext system:
https://en.wikipedia.org/wiki/Hypertext

with many implementations:
https://en.wikipedia.org/wiki/Hypertext#Implementations

and in Hyperscope there are types, like: 

Action Asciidoc Asciidoctor "Asciidoctor with" PDF DISEASE DJVU DONE
Directory "Directory Action ➜" EPUB "Emacs Lisp" "Emacs Lisp
Hyperlink" Enriched FOLLOW-UP File HTML, etc.

So that gives me idea that I could be exporting the system into `eev'
files, with highlighting and generate automatically hyperlinks such as:

(Set 29 "People")

(WWW 38233 "Modular Buildings & Prefab Homes | Karmod Construction")         

As that way Hyperscope would activate the ID 38233 and open up the
website. 

I also believe that quoting could be excluded:

(WWW 38233 Modular Buildings & Prefab Homes | Karmod Construction)

And it works this way:

(defmacro WWW (id &rest args)
  (hyperscope-tabulated-action id) ;; open up action based on the type
  (espeak (mapconcat 'symbol-name args " "))) ;; speak symbols as string 

It would be nice to highlight the symbols "Modular
Buldings.. Construction" only but I don't know how to do it.

I can easier generate eev style links then Org buffers.


-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/