On Bookmarks+ package

On Bookmarks+ package

[Jean Louis]

I changed this to GNU Emacs Help, as not to spoil the subject of the
thread.

* Drew Adams <drew.adams@oracle.com> [2023-02-15 19:35]:
> Emacs bookmarks can "hold" anything you like.  For bookmarks that
> you want to persist, the "anything" needs to be persistable and
> readable.

I have 53111 elementary objects currently. While I call them
Hyperdocuments and elementary objects, you call it bookmarks. 

I was thinking bookmarks in Emacs are more narrowed types of
objects. And did not expand my personal use of them.

Do you think that bookmarks(+) with it's implementation could scale to
that number of objects without losing speed?

> But typically bookmarks don't "hold" much data.  Instead, they point
> to data that's available somewhere else (whether it's persisted or
> not at that "somewhere else").

These elementary objects my hold as much data, for example, LaTeX
processing generates various files (aux log pdf tex) which I really do
not like and do not want to think about it. By using LaTeX object in
the database program may decide internally where is the directory,
where is the file, it can generate it, and me as user do not think of
it. I only edit, and generate PDF. I can share the file with other
people without knowing where in file system it is.

If I would keep LaTeX object or LaTeX text in bookmark, I would not
know where it is stored really, it would be one file, but that file is
not protected from influences from file system, and then single
bookmark file would be bloated with hundreds or thousands of such
objects, and external access to such objects, beyond Emacs Lisp, would
not be possible. With the database it is easy, I can use external
scripts to insert or share any objects.

I find that you have perfected Bookmark+ and that I can re-use those
ideas including your way of doing things systematically, pedantically.

> There should be no inherent problem bookmarking a "non-file database
> entry".  You just have to decide what you need to put in the
> bookmark, to later get to it (e.g. restore it).

Exactly, I understand it, that is how it works here too. Though you
speak in terms of programming capability, not of something that is
already implemented.

> You can sort, edit, tag – whatever non-invokable bookmarks, i.e.,
> you can use bookmarks just to organize things.  (And you have
> bookmarks that are invokable only in certain contexts.)

Please let me know, do you dwell in Bookmarks?

That is what I do. My workspace is in Hyperscope and people. I have
dashboard, reminders, and often dwell, spend time in that
organization, finishing a document, sharing, editing, generated
mixed-object document, sharing it, like that. All time there.

Do you actually use your bookmarks?

How many times per day?

Here is how it works on my side, there are 215 elementary objects from
last week. In a way they are similar to bookmarks, though I did not
put much thinking into it. Is that how you spend time as well in your
bookmarks? 

┌────────────────────────────┬───────┐
│   Objects for last week    │ Count │
├────────────────────────────┼───────┤
│ Hyperscope, new objects    │   215 │
│ Hyperscope, objects sent   │    21 │
│ Hyperscope objects tagged  │   268 │
│ Hyperscope, people related │    25 │
│ Messages Sent              │   486 │
│ New People                 │    35 │
│ People, tagged             │   268 │
└────────────────────────────┴───────┘

> > I have it same, I can combine, or find interesection of elementary
> > objects, I can mark what I need or take all list of visible ones and
> > make new object contaning all those.
> 
> Bookmark tags are a good way to categorize things
> and access them by combinations of such categories.
> 
> https://www.emacswiki.org/emacs/BookmarkPlus#BookmarkTags

I understand tags, and I use them too, though in Hyperscope
system. Each elementary object may be tagged. There are types of tags
in my system, so there may be "subject tag" like "Computer tag", where
meanining need not be same to other tag of same name. By using
database, there is privilege that tags may be renamed, edited,
described, they are objects itself, not only a word.

,----
| * You can tag bookmarks, a la del.icio.us. In effect, bookmark tags
|  define bookmark sets. A bookmark can have any number of tags, and
|  multiple bookmarks can have the same tag. You can sort, show/hide, or
|  mark bookmarks based on their tags.
`----

That I understand. 

,----
| * Bookmark+ tags can be more than just names. They can be full-fledged
|   user-defined attributes, with EmacsLisp objects as their values.
`----

I understand that next level.

> Org mode, for example, lets you embed different kinds of things in
> an Org buffer/file.  You can tag them, link them, annotate them, and
> act on them in any number of ways.  A big difference here is that
> the info/data of the things, and the tags and links and
> annotations... are in the Org file, along with the other things and
> any "main" text.

Ok, that gives me idea that your bookmark+ bookmarks can also be
tasks, notes, right?

Then by using tags, or other associations, you could reconstruct whole
document, right?

> That has some advantages, no doubt.  But storing annotations, links,
> tags, actions, other metadata, ... separately (e.g. in a bookmark
> file) also has (different) advantages.

I find that later shall be on top level, on highest level, and former,
part of the later.

- instead of having Org as file, I find it more convenient, having Org
  as elementary object of a set 

- instead of having too many things in single Org object, I find it
  more convenient having separate objects and then combining them in a
  mixed-object document

- this way of thinking, and I am using it, also opens the opportunity
  of combining any markups together in a mixed-object document

More practical examples:

- task is written in Org format, 1
- but other task is written in Asciidoctor format, 2
- one is in txt2tags, 3
- there is list of equipment required for industry, 4
- there is HTML page, 5
- there is PDF to be shared with assigned people, 6

Then list of equipment (4) may be included in HTML page (5), but also
reused in the PDF (6) produced with primary markup being let us say
LaTeX, which contains tasks 1, 2 and 3, which will be all converted to
LaTeX.

Many systems like Org and LaTeX already have features to include other
files, just that such are not integrated or rather difficult to
implement.

For example, I cannot just list multiple Org files over different
directories in file system on single screen, mark intersection of them
and produce mixed-object document containing those files. I would need
to do it pretty manually and include files one by one in Org.

With bookmarks, or elementary objects, I could use tags, or various
intersections, mark some objects and with single key press produce the
mixed-object document.

> The implementation of bookmarks, as text files of Lisp code, is fine
> for many use cases.  It fits the use of any number of bookmark
> files, each of which is not gigantic.

Could they really work with 50000 objects? Did you ever test the speed
of accessing, using them?

> But yes, Someone (TM), sometime, will hopefully do things such as:
> 
> 1. Allow (also) for binary ("compiled") bookmark
>    formats (and thus files).

Is that for reasons of speed? Once below subject 2 is solved, speed is
also solved.

> 2. Allow storing and accessing bookmarks using
>    (multiple) databases.

By using PostgreSQL network database that is easy without
problems. Hyperscope can work with multiple databases. 

> > Even then, objects are saved in the database, and related to
> > planning, and represent themselves notes, tasks, not just
> > bookmarks pointing to notes and tasks, they are related to people,
> > can be shared and so I canot reuse your library, apart from
> > reusing ideas, food for thought.
> 
> Not clear to me.  Why would notes, etc., whether related to people
> or not, be something you can't leverage bookmarks for?  I don't
> doubt you're saying something true, but it's not clear to me what it
> is.

I asked you if that following is also possible with Bookmarks Plus:

- create bookmark, let us say "ABC"

- edit ABC as text, representing note, task

I understand that it can be done programmatically in future, but is it
already there maybe? I know annotations are there, so in the sense it
is very similar to my use cases.

> > > Bookmark+ adds over 70 types of bookmark. (You can see what they
> > > all are by invoking function ‘bmkp-types-alist’.)
> > 
> > But that does not seem to be the function, it is alist, but I do
> > not see 70 types.
> 
> No, it's a function, not a variable.  The function returns an alist
> of known types (their names) and the associated history list (for
> completion).  It's a function because which predefined types exist
> depends on your environment, e.g., on whether you have this or that
> other library, and on what Emacs release you have.

I cannot find it:

-*- mode: grep; default-directory: "~/.emacs.d/Drew Adams/" -*-
Grep started at Thu Feb 16 11:11:19

grep --color=auto -nH --null -e "defun bmkp-types-alist" *.el

Grep finished with no matches found at Thu Feb 16 11:11:20

When I load "bookmark+" I cannot find that function.

> What's important is that it's easy for you to define new types.  The
> predefined types are usable OOTB, for things like jumping to a
> bookmark of a given type, or choosing a navigation list of bookmarks
> of a given type.
> 
> FWIW, (bmkp-types-alist) shows this, for me:

(featurep 'bookmark+)
t

(bmkp-types-alist)
Debugger entered--Lisp error: (void-function bmkp-types-alist)
  (bmkp-types-alist)
  (progn (bmkp-types-alist))

I cannot find the function.

> > Can you save window configuration over session? I would like to do
> > that. That is not same as desktop which is slow and cannot narrow to
> > window configuration.
> 
> That's one of my wishes too.
> 
> Last time I checked, window configs are not Lisp
> readable.  I tried to get Emacs (Juanma, who did
> the desktop restore stuff) to raise window and
> frame configs to the level of Lisp-readable
> thingies (so persistable) - but to no avail.
> 
> See, e.g., bug #14964 (also #10348).
> 
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=14964#11

Pity that data structure is not exposed to Emacs Lisp.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

RE: [External] : On Bookmarks+ package

[Drew Adams]

> > Emacs bookmarks can "hold" anything you like.  For bookmarks that
> > you want to persist, the "anything" needs to be persistable and
> > readable.
> 
> I have 53111 elementary objects currently. While I call them
> Hyperdocuments and elementary objects, you call it bookmarks.
> 
> I was thinking bookmarks in Emacs are more narrowed types of
> objects. And did not expand my personal use of them.
> 
> Do you think that bookmarks(+) with it's implementation could scale to
> that number of objects without losing speed?

No idea.

As I said in the emacs-devel thread, typically bookmarks hold relatively little content.  They can refer to other content.  They can also hold however much content you like.  That doesn't mean it's a great idea to store all of the text from all of Wikipedia in a bookmark.  YMMV.

> > But typically bookmarks don't "hold" much data.  Instead, they point
> > to data that's available somewhere else (whether it's persisted or
> > not at that "somewhere else").
> 
> ...
> If I would keep LaTeX object or LaTeX text in bookmark, I would not
> know where it is stored really, it would be one file, but that file is
> not protected from influences from file system, and then single
> bookmark file would be bloated with hundreds or thousands of such
> objects, and external access to such objects, beyond Emacs Lisp, would
> not be possible. With the database it is easy, I can use external
> scripts to insert or share any objects.

Don't use bookmarks to hold tons of data.
Use a database for that, if you like, or
just files.  It's easy to store Lisp code
in a file, any file: see `write-file' and
`write-region', to start with.
 
> > There should be no inherent problem bookmarking a "non-file database
> > entry".  You just have to decide what you need to put in the
> > bookmark, to later get to it (e.g. restore it).
> 
> Exactly, I understand it, that is how it works here too. Though you
> speak in terms of programming capability, not of something that is
> already implemented.

You can bookmark any buffer, OOTB.

However, if just creating to or going to the
buffer isn't sufficient to restore what you
want there, then yes, of course, you need to
put into your bookmark data whatever you need
to get the buffer content/state you want.

Some predefined bookmark types for non-file
buffers do exactly that.  `man' bookmarks,
for example.  And Dired bookmarks.

If you're adding bookmarks for some kind of
buffer that doesn't have a predefined bookmark
type, then yes, just define what you need.

The rule is simple: whatever you store in a
bookmark is what's used to restore/"jump" to
its target.  Garbage in, or not enough info
in, means garbage out, or not enough info out.

> > You can sort, edit, tag – whatever non-invokable bookmarks, i.e.,
> > you can use bookmarks just to organize things.  (And you have
> > bookmarks that are invokable only in certain contexts.)
> 
> Please let me know, do you dwell in Bookmarks?

Nope.  I dwell on Earth.  Nor do bookmarks
dwell within me, AFAIK.

> That is what I do. My workspace is in Hyperscope and people. I have
> dashboard, reminders, and often dwell, spend time in that
> organization, finishing a document, sharing, editing, generated
> mixed-object document, sharing it, like that. All time there.
> 
> Do you actually use your bookmarks?

Yes.  Probably not as much as some others use theirs.

> How many times per day?

No idea.  Why?

> Here is how it works on my side, there are 215 elementary objects from
> last week. In a way they are similar to bookmarks, though I did not
> put much thinking into it. Is that how you spend time as well in your
> bookmarks?

I don't spend time in my bookmarks.

> > Bookmark tags are a good way to categorize things
> > and access them by combinations of such categories.
> 
> I understand tags, and I use them too, though in Hyperscope
> system. Each elementary object may be tagged. There are types of tags
> in my system, so there may be "subject tag" like "Computer tag", where
> meanining need not be same to other tag of same name. By using
> database, there is privilege that tags may be renamed, edited,
> described, they are objects itself, not only a word.

Bookmark+ tags are arbitrary strings, by default.
They're not jsut words.  And they can also be
conses: (STRING . ANYTHING), where ANYTHING is
any Lisp value.

If the tagged bookmark is to be persisted
(which is typical), then ANYTHING needs to be
persistable and Lisp-readable.

I've mentioned this (all of this, in fact) to
you before.  Maybe you're really wanting to
talk about your database setup and what you do
with it?  If so, I have nothing to say about
your setup; sorry.

I don't say you should use Bookmark+ or bookmarks,
or in particular that you should use them instead
of what you're using now.

I can tell that you're somewhat interested in
Bookmark+ and bookmarks.  Then I'd say try using
bookmarks for some things, and see if you like
that and it helps.  If not, don't bother with
bookmarks.

> > Org mode, for example, lets you embed different kinds of things in
> > an Org buffer/file.  You can tag them, link them, annotate them, and
> > act on them in any number of ways.  A big difference here is that
> > the info/data of the things, and the tags and links and
> > annotations... are in the Org file, along with the other things and
> > any "main" text.
> 
> Ok, that gives me idea that your bookmark+ bookmarks can also be
> tasks, notes, right?

Dunno what you have in mind.  Maybe.  A bookmark
is just a persisted bit of text (code) that you
can use to do something.  It can do anything you
like.  As I said, it's essentially a named,
persistent (doesn't have to be, but can be),
closure.  What you define it to be, it is.  What
you use it for is up to you.

> Then by using tags, or other associations, you could reconstruct whole
> document, right?

I suppose so.  I can't really help with your
imaginings, and I'm sure you can imagine what
you want/need.  If bookmarks help you get what
you imagine, great.  If not, open the tool
chest and try another tool.

> [Ideas about Org and LaTeX...]

Bookmarks and Bookmark+ are not a replacement
for Org mode or LaTeX etc.  They're a basic
tool.  If you want to build an alternative to
Org or whatever, maybe you can make use of
bookmarks somehow to help you do that.  Dunno.

> With bookmarks, or elementary objects, I could use tags, or various
> intersections, mark some objects and with single key press produce the
> mixed-object document.

Yes, I suppose you could.  You'll need to do
some coding, to get there, I expect.

> > The implementation of bookmarks, as text files of Lisp code, is fine
> > for many use cases.  It fits the use of any number of bookmark
> > files, each of which is not gigantic.
> 
> Could they really work with 50000 objects? Did you ever test the speed
> of accessing, using them?

No, I haven't.  What would be in your 50,000
Lisp codelets?  How would they be put to use?

You're asking whether 50,000 snippets of Lisp
code can be used.   W a y   t o o   abstract.

> > But yes, Someone (TM), sometime, will hopefully do things such as:
> >
> > 1. Allow (also) for binary ("compiled") bookmark
> >    formats (and thus files).
> 
> Is that for reasons of speed?

If you expect bookmarks to be large code
blocks or large data blocks, then obviously
text isn't the most performant or disk-space
conserving.

It's a bit like Elisp vs Common Lisp.  Most
uses of Elisp don't require hyperperformance.
Common Lisp is a much more general programming
language, and it's important (for some apps)
that it be as performant as Fortran or C (or
better).

Not so (so far), for most Elisp uses.  Same
thing for most bookmark uses.  If you want to
push the envelope then you might want binary
bookmark data.  Bookmark+ doesn't have/support
that now.  And I doubt that I, at least, will
ever get around to adding that.  But sure, it
would be a good thing to have - why not?

> I asked you if that following is also possible with Bookmarks Plus:
> - create bookmark, let us say "ABC"
> - edit ABC as text, representing note, task
> 
> I understand that it can be done programmatically in future, but is it
> already there maybe? I know annotations are there, so in the sense it
> is very similar to my use cases.

Didn't know that you asked me that.

You can edit the source code of any bookmark,
yes.  The Edit submenu of menu Bookmark+ has
these items for that:

 Edit Full Record (Lisp)            - bound to `e'
 Edit Full Records of Marked (Lisp) - bound to `E'       

(You can also do the same for just bookmark
annotations.)

> I cannot find [function `bmkp-types-alist' using `grep']
> When I load "bookmark+" I cannot find that function.
> Debugger entered--Lisp error: (void-function bmkp-types-alist)
> I cannot find the function.

It's been in Bookmark+ since 2021/03/04.  It's in
file bookmark+-1.el.  The latest published version
of the file is this - maybe you need to update?

;; Last-Updated: Wed Sep  7 15:50:09 2022 (-0700)
;;           By: dradams
;;     Update #: 9488

HTH.