Re: Programming Emacs visually with semantic triplets

[Jean Louis <bugs@gnu.support>]

* Eduardo Ochs <eduardoochs@gmail.com> [2023-01-21 13:40]:
> Hi Jean,
> do you have tools that show the Lisp code behind each button and that show
> pretty-printed versions of the main data structures? Can you send us
> screenshots of that? Some people - like me =/ - understand these things
> much more easily when they see how a prototype was implemented...

I create my tables semi-automagically, by using function:

(defun rcd-db-create-table (table pg)
  "Create TABLE with database handle PG."
  (let* ((table (or table (rcd-ask-get "New database table name: ")))
	 (sql (format "CREATE TABLE %s (
                      %s_id SERIAL NOT NULL PRIMARY KEY,
                      %s_uuid UUID NOT NULL DEFAULT gen_random_uuid() UNIQUE,
                      %s_datecreated TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP NOT NULL,
                      %s_datemodified TIMESTAMP WITH TIME ZONE,
                      %s_usercreated TEXT NOT NULL DEFAULT current_user,
                      %s_usermodified TEXT NOT NULL DEFAULT current_user,
                      %s_name TEXT NOT NULL,
                      %s_description TEXT)"
		      table table table table table table table table table)))
    (rcd-sql sql pg)))

Which takes care of nice structure, so each column name speaks about
table as "predicates_uuid" clearly belong to table "predicates". This
I have learnt from Gedafe:

GeDaFe - PostgreSQL Generic Database Interface:
http://gedafe.github.io/doc/gedafe-sql.en.html 

And I keep following Gedafe design, as all of my stuff worked and will
still work in Gedafe.

For the concept of triplets, there are just few tables, one is "predicates".

                                                    Table "public.predicates"
┌─────────────────────────┬──────────────────────────┬───────────┬──────────┬───────────────────────────────────────────────────┐
│         Column          │           Type           │ Collation │ Nullable │                      Default                      │
├─────────────────────────┼──────────────────────────┼───────────┼──────────┼───────────────────────────────────────────────────┤
│ predicates_id           │ integer                  │           │ not null │ nextval('predicates_predicates_id_seq'::regclass) │
│ predicates_uuid         │ uuid                     │           │ not null │ gen_random_uuid()                                 │
│ predicates_datecreated  │ timestamp with time zone │           │ not null │ CURRENT_TIMESTAMP                                 │
│ predicates_datemodified │ timestamp with time zone │           │          │                                                   │
│ predicates_usercreated  │ text                     │           │ not null │ CURRENT_USER                                      │
│ predicates_usermodified │ text                     │           │ not null │ CURRENT_USER                                      │
│ predicates_name         │ text                     │           │ not null │ '>>>UNKNOWN<<<'::text                             │
│ predicates_description  │ text                     │           │          │                                                   │
└─────────────────────────┴──────────────────────────┴───────────┴──────────┴───────────────────────────────────────────────────┘
Indexes:
    "predicates_pkey" PRIMARY KEY, btree (predicates_id)
    "predicates_predicates_name_idx" UNIQUE, btree (predicates_name)
    "predicates_predicates_uuid_key" UNIQUE CONSTRAINT, btree (predicates_uuid)
Triggers:
    insert_username_predicates BEFORE INSERT OR UPDATE ON predicates FOR EACH ROW EXECUTE FUNCTION insert_username('predicates_usermodified')
    predicates_moddatetime BEFORE UPDATE ON predicates FOR EACH ROW EXECUTE FUNCTION moddatetime('predicates_datemodified')


Another one is "sobjects", which mean "subjects or objects", as why
have it separate? 


                                                  Table "public.sobjects"
┌───────────────────────┬──────────────────────────┬───────────┬──────────┬───────────────────────────────────────────────┐
│        Column         │           Type           │ Collation │ Nullable │                    Default                    │
├───────────────────────┼──────────────────────────┼───────────┼──────────┼───────────────────────────────────────────────┤
│ sobjects_id           │ integer                  │           │ not null │ nextval('sobjects_sobjects_id_seq'::regclass) │
│ sobjects_uuid         │ uuid                     │           │ not null │ gen_random_uuid()                             │
│ sobjects_datecreated  │ timestamp with time zone │           │ not null │ CURRENT_TIMESTAMP                             │
│ sobjects_datemodified │ timestamp with time zone │           │          │                                               │
│ sobjects_usercreated  │ text                     │           │ not null │ CURRENT_USER                                  │
│ sobjects_usermodified │ text                     │           │ not null │ CURRENT_USER                                  │
│ sobjects_name         │ text                     │           │ not null │ '>>>UNKNOWN<<<'::text                         │
│ sobjects_description  │ text                     │           │          │                                               │
└───────────────────────┴──────────────────────────┴───────────┴──────────┴───────────────────────────────────────────────┘
Indexes:
    "sobjects_pkey" PRIMARY KEY, btree (sobjects_id)
    "sobjects_sobjects_uuid_key" UNIQUE CONSTRAINT, btree (sobjects_uuid)
Triggers:
    insert_username_sobjects BEFORE INSERT OR UPDATE ON sobjects FOR EACH ROW EXECUTE FUNCTION insert_username('sobjects_usermodified')
    sobjects_moddatetime BEFORE UPDATE ON sobjects FOR EACH ROW EXECUTE FUNCTION moddatetime('sobjects_datemodified')

The whole concept is based on UUID which is not a primary key, which
is not relational database conformant, but it gives that way some
flexibility.

Then there is this table:

                                                   Table "public.uuid2uuid"
┌────────────────────────┬──────────────────────────┬───────────┬──────────┬─────────────────────────────────────────────────┐
│         Column         │           Type           │ Collation │ Nullable │                     Default                     │
├────────────────────────┼──────────────────────────┼───────────┼──────────┼─────────────────────────────────────────────────┤
│ uuid2uuid_id           │ integer                  │           │ not null │ nextval('uuid2uuid_uuid2uuid_id_seq'::regclass) │
│ uuid2uuid_uuid         │ uuid                     │           │ not null │ gen_random_uuid()                               │
│ uuid2uuid_datecreated  │ timestamp with time zone │           │ not null │ CURRENT_TIMESTAMP                               │
│ uuid2uuid_datemodified │ timestamp with time zone │           │          │                                                 │
│ uuid2uuid_usercreated  │ text                     │           │ not null │ CURRENT_USER                                    │
│ uuid2uuid_usermodified │ text                     │           │ not null │ CURRENT_USER                                    │
│ uuid2uuid_description  │ text                     │           │          │ ''::text                                        │
│ uuid2uuid_subjects     │ uuid                     │           │ not null │                                                 │
│ uuid2uuid_predicates   │ uuid                     │           │ not null │                                                 │
│ uuid2uuid_objects      │ uuid                     │           │ not null │                                                 │
└────────────────────────┴──────────────────────────┴───────────┴──────────┴─────────────────────────────────────────────────┘
Indexes:
    "uuid2uuid_pkey" PRIMARY KEY, btree (uuid2uuid_id)
    "uuid2uuid_uuid2uuid_subjects_uuid2uuid_predicates_uuid2uuid_idx" UNIQUE, btree (uuid2uuid_subjects, uuid2uuid_predicates, uuid2uuid_objects)
    "uuid2uuid_uuid2uuid_uuid_key" UNIQUE CONSTRAINT, btree (uuid2uuid_uuid)

How I see the issue at hand, that becomes enough to reference anything
to anything.

If I have table "people" with Eduardo entry there, it has UUID:
EBD4E7DA-2321-4E1C-97CA-A7F659015AD6, and then I can either:

- make new entry in "sobjects" and create new UUID holding Eduardo's
  UUID: EBD4E7DA-2321-4E1C-97CA-A7F659015AD6

- then I can tell "Eduardo" with underlying UUID, "is" as predicate,
  "programmer" with "programmer" also having its own UUID

That is stored in "uuid2uuid" table:

Entry does not look human friendly:

-[ RECORD 49 ]---------+-------------------------------------
uuid2uuid_id           | 49
uuid2uuid_uuid         | 764114e5-2d7a-4b9a-b1b0-a5719e6f3468
uuid2uuid_datecreated  | 2023-01-21 19:26:17.890422+03
uuid2uuid_datemodified | 
uuid2uuid_usercreated  | maddox
uuid2uuid_usermodified | maddox
uuid2uuid_description  | 
uuid2uuid_subjects     | fe54313c-7472-4498-a993-1b4ac949ff5c
uuid2uuid_predicates   | 6effc368-b66c-4216-a230-6916e3bfdc70
uuid2uuid_objects      | df35a22c-725e-4202-b79e-9ab4a147bff7

Though that gives flexibility, as UUID does not need to reference any
table in underlying database, it can reference any UUID, inside or
outside of the database.

There is more to implement. Functions are here:

;;; Semantic Triplets

;; This below is more fundamental function, it just creates new entry
;; and returned it's ID for further update. And it demands that all
;; entries have their default values, like if there is no name, the
;; name will appear as '>>>UNKNOWN<<<' which demands updating it.

(defun rcd-db-table-insert-default-values (table db)
  "Insert default values into TABLE with DB handle."
  (rcd-sql-first (format "INSERT INTO %s DEFAULT VALUES RETURNING %s_id" table table) db))

;; This below is function to get the primary key of the entry by using
;; TABLE and UUID. I don't use UUID's as primary key as it is not
;; convenient, but many entries do have UUID to be searchable without
;; being referenced.

;; Below works because all tables have conformant format, like
;; "people" will have "people_uuid", and "people_id", and because of
;; the format, magic is possible just as explained in Gedafe
;; documents. It becomes possible to prepare the underlying structure
;; and to have EDIT, DUPLICATE, DELETE functions automatically in GUI
;; or Emacs interface, or WWW interface.

(defun rcd-db-uuid-by-id (table id db)
  "Return UUID for TABLE with ID and DB handle."
  (rcd-sql-first (format "SELECT %s_uuid FROM %s WHERE %s_id = %s" table table table id) db))

;; Of course, as not to interfer with other modes, it is better to
;; create new mode, which can hold kew bindings. The text mode or
;; buffer is used as menu system with buttons, similar like Hypertext.

(define-derived-mode rcd-triplets-view-mode org-mode 
  "RCD Notes Triplets View Mode" 
  "Semantic Triplets in RCD Notes")

;; This is new experiment, as I am tired of writing always different
;; function in key bindings, so I have decided to write always some
;; function, like in this case`rcd-triplets-add-new' which does
;; different job depending of the key binding.

(keymap-set rcd-triplets-view-mode-map "a s" #'rcd-triplets-add-new)
(keymap-set rcd-triplets-view-mode-map "a p" #'rcd-triplets-add-new)
(keymap-set rcd-triplets-view-mode-map "a o" #'rcd-triplets-add-new)
(keymap-set rcd-triplets-view-mode-map "q" #'quit-window)

;; This is only to add new "sobject" which is subject or object. But
;; the main triplet table named "uuid2uuid" above does not really
;; require table "sobjects" as it will accept any UUIDs, referencing
;; to other tables, or even to objects with UUID outside of database
;; space.

(defun rcd-triplets-sobject-new ()
  "Add new subject or object and return it's UUID."
  (interactive)
  (let ((id (rcd-db-table-insert-default-values "sobjects" cf-db))
	(name (rcd-ask-get "Add new subject or object: ")))
    (rcd-db-update-entry "sobjects" "sobjects_name" id name cf-db)
    (rcd-db-uuid-by-id "sobjects" id cf-db)))

;; In similar way, this function adds new predicate. There are
;; depending functions, like `rcd-db-update-entry'

(defun rcd-triplets-predicate-new ()
  "Add new predicate and return it's UUID."
  (interactive)
  (let ((id (rcd-db-table-insert-default-values "predicates" cf-db))
	(name (rcd-ask-get "Add new predicate: ")))
    (rcd-db-update-entry "predicates" "predicates_name" id name cf-db)
    (rcd-db-uuid-by-id "predicates" id cf-db)))

;; This is helper function to find last key pressed.

(defun rcd-last-key ()
  "Return last key."
    (elt (seq-reverse (recent-keys)) 0))

;; This function is for adding entries, like objects or subjects, predicates.

(defun rcd-triplets-add-new ()
  "Add new subject, predicate or object.

Function may be called interactively, or as part of key bindings.

For last key `s' add new subject.
For last key `p' add new predicate.
For last key `o' add new predicate."
  (interactive)
  ;; I use here the helper function to recognize what was last key
  ;; pressed, as not to mess to much with keymap settings. Rather I
  ;; keep decision making in this function.
  (let* ((key (rcd-last-key))
	 (new-id 
	  (cond ((and (numberp key) (= ?s key)) (rcd-triplets-sobject-new))
		((and (numberp key) (= ?p key)) (rcd-triplets-predicate-new))
		((and (numberp key) (= ?o key)) (rcd-triplets-sobject-new))
		;; but if no corresponding last key is found, then I
		;; offer to user choice what to add instead.
		(t (let ((choice (rcd-choose '("Subject or Object" "Predicate"))))
		     (cond ((string-match "Subject" choice) (rcd-triplets-subject-new))
			   ((string= "Predicate" choice) (rcd-triplets-predicate-new))
			   (t (user-error "Cannot handle choice"))))))))
    ;; And then the list of objects is re-displayed.
    (cond ((and (numberp key) (= ?s key)) (rcd-triplets-db-list "sobjects" new-id))
	  ((and (numberp key) (= ?p key)) (rcd-triplets-db-list "predicates" new-id))
	  ((and (numberp key) (= ?o key)) (rcd-triplets-db-list "sobjects" new-id)))))

;; This is initiating the buffer with proper mode.

(defun rcd-triplets-buffer (buffer)
  (unless (buffer-live-p (get-buffer buffer))
    (generate-new-buffer buffer))
  (set-buffer buffer)
  (erase-buffer)
  (display-buffer buffer)
  (rcd-triplets-view-mode))

;; Header function with buttons, you will recognize these functions
;; from previous discussion.

(defun rcd-triplets-header-insert ()
  (insert "\n      RCD Notes ⭐ Semantic Triplets\n\n  ")
  (rcd-button-insert "[ADD]" (lambda (_) (rcd-triplets-new)))
  (insert "  ")
  (rcd-button-insert "[TRIPLETS]" #'rcd-triplets-list)
  (insert "  ")
  (rcd-button-insert "[SUBJECTS or OBJECTS]" #'rcd-triplets-db-list-sobjects)
  (insert "  ")
  (rcd-button-insert "[PREDICATES]" #'rcd-triplets-db-list-predicates)
  (insert "\n\n"))

;; I have generalized this function to us both "subjects" and
;; "predicates", so it works, it lists the items in the buffer. And it
;; generates "[edit]" button to rename the object or predicate.

(defun rcd-triplets-db-list (table &optional new-id)
  (let* ((sql (format "SELECT %s_id, %s_uuid, %s_name
                                     FROM %s
                                 ORDER BY %s_id"
		      table table table table table))
	 (items (rcd-sql-list sql cf-db))
	 (width (format "%s" (rcd-db-column-width table (format "%s_id" table) cf-db)))
	(buffer (format "*RCD Notes: All %s for Semantic Triplets*" table)))
    (rcd-triplets-buffer buffer)
    (rcd-triplets-header-insert)
    (insert 
     (rcd-report-underlined 
      (upcase table)
      (with-temp-buffer
	(while items
	  (let* ((item (pop items))
		 (id (nth 0 item))
		 (uuid (nth 1 item))
		 (name (nth 2 item)))
	    (insert (format (concat "%0" width "d. ") id))
	    (rcd-button-insert 
	     "[edit]"
	     (lambda (_)
	       (rcd-db-edit-entry table (concat table "_name") id cf-db)
	       (rcd-triplets-db-list table id)))
	    (insert " " name)
	    (insert "\n")))
	(buffer-string))))
    (cond (new-id
	   (goto-char (point-min))
	   (search-forward-regexp (format "^%s. " new-id) nil t))
	  (t (goto-char (point-min))
	     (forward-line 3)))))

;; This is extensible function, for now it looks for the name in 2
;; tables like "sobjects" or "predicates", or if name is not
;; available, it constructs name of the full triplet. There is more
;; work to do, as if UUID belongs to people, then person's name shall
;; appear, if it belongs to document, document's name should appear,
;; and so on for many other tables in the database.

(defun rcd-triplets-name-by-uuid (uuid)
  (let* ((find-uuid (lambda (table uuid)
		      (rcd-db-get-entry-where 
		       table 
		       (format "%s_name" table) 
		       (format "%s_uuid = %s" table (sql-escape-string uuid))
		       cf-db)))
	 (names (mapcar (lambda (table) (funcall find-uuid table uuid))
			'("sobjects" "predicates")))
	 (names (delq nil names))
	 (name (car names)))
    (cond (name name)
	  (t (rcd-triplets-return-triplet uuid)))))

;; For example this function below, gives following:
;; (rcd-triplets-name-by-uuid "f26d9e92-ed61-4281-a27e-668b76daa5e2") ➜ "known universe is/are inside [2] of/from unknown universe [3]"

;; This is similar function as to list items, just specifically for
;; all of the triplets in the buffer.

;; It offers 2 buttons, like [subject] and [object] which gives to
;; user possibility to take whole triplet and make it object or
;; subject with some predicate.

;; That enables definitions like:

;; Emacs Lisp is/are programming language                
;; help to human                                         
;; Emacs Lisp is/are help to human [41]                  
;; programming language has/have function                
;; programming language is/are language                  
;; English is/are language                               
;; verb is/are predicate                                 
;; run is/are verb                                       
;; programming language has/have function [43] may be run
;; language has/have verb                                

;; Doing it gives me some ideas, as that way it is possible to quickly
;; define concepts. Triplet of basic understanding builds on other
;; triplets of basic understandings. It can build into "contact
;; entry", which by semantics "may require" other information like
;; "phone number", or "e-mail" address, and by designing
;; relationships, one can then say that "contact entry" is a function
;; (and that must be programmed), and that function shall ask for
;; other entries, and those can be parsed automatically. So I think
;; that by parsing semi-visually and interactively defined meanings it
;; is possible to program computer in somewhat easier manner than by
;; writing code. At least for some basic applications, let us say for
;; invoices, or notes, tasks.

(defun rcd-triplets-list (&optional _)
  (interactive)
  (let* ((triplets (rcd-sql-list 
		    "SELECT uuid2uuid_id, uuid2uuid_uuid, uuid2uuid_subjects, uuid2uuid_predicates, uuid2uuid_objects
                       FROM uuid2uuid"
		    cf-db))
	 (width (rcd-db-column-width "uuid2uuid" "uuid2uuid_id" cf-db))
	 (buffer "*RCD Notes: Semantic Triplets*"))
    (rcd-triplets-buffer buffer)
    (rcd-triplets-header-insert)
    (insert 
     (rcd-report-underlined 
      "Semantic Triplets"
      (with-temp-buffer
	(while triplets
	  (let* ((triplet (pop triplets))
		 (id (nth 0 triplet))
		 (uuid (nth 1 triplet))
		 (subject (nth 2 triplet))
		 (subject-name (rcd-triplets-name-by-uuid subject))
		 (predicate (nth 3 triplet))
		 (predicate-name (rcd-triplets-name-by-uuid predicate))
		 (object (nth 4 triplet))
		 (object-name (rcd-triplets-name-by-uuid object)))
	    (rcd-button-insert 
	     "[subject]" 
	     (lambda (_)
	       (rcd-triplets-new uuid)))
	    (insert " ")
	    (rcd-button-insert 
	     "[object]" 
	     (lambda (_)
	       (rcd-triplets-new nil nil uuid)))
	    (insert " " subject-name " " predicate-name " " object-name)
	    (insert "\n")))
	(buffer-string))))
    (goto-char (point-min))))

;; (rcd-triplets-name-by-uuid "f26d9e92-ed61-4281-a27e-668b76daa5e2") ➜ "known universe is/are inside [2] of/from unknown universe [3]"

;; (rcd-triplets-return-triplet "764114e5-2d7a-4b9a-b1b0-a5719e6f3468") ➜ "language has/have verb [49]"

;; This is function that writes representation of triplet. I think of
;; expanding representation by adding correct singular, and plural
;; forms, as that way I will avoid having "is/are", but will have
;; proper "is" or "are" as predicate

(defun rcd-triplets-return-triplet (uuid)
  (let* ((triplet (rcd-sql-list-first "SELECT uuid2uuid_subjects, uuid2uuid_predicates, uuid2uuid_objects
                                   FROM uuid2uuid
                                  WHERE uuid2uuid_uuid = $1"
				cf-db uuid))
	 (id (cadr (rcd-db-id-by-uuid uuid)))
	 (subject (nth 0 triplet))
	 (subject-name (rcd-triplets-name-by-uuid subject))
	 (predicate (nth 1 triplet))
	 (predicate-name (rcd-triplets-name-by-uuid predicate))
	 (object (nth 2 triplet))
	 (object-name (rcd-triplets-name-by-uuid object)))
    (format "%s %s %s [%s]" subject-name predicate-name object-name id)))

;; This one just list the "sobject" items in buffer. Must have "_"
;; optional argument to be used as button.

(defun rcd-triplets-db-list-sobjects (&optional _)
  (interactive)
  (rcd-triplets-db-list "sobjects"))

;; List predicates in buffer.

(defun rcd-triplets-db-list-predicates (&optional _)
  (interactive)
  (rcd-triplets-db-list "predicates"))

;; Select one of "sobjects" or "prdicates" by using completing read.

(defun rcd-triplets-sobject-select (&optional prompt)
  (rcd-db-combo-selection "sobjects" (or prompt "Select subject or object: ")))

(defun rcd-triplets-predicate-select (&optional prompt)
  (rcd-db-combo-selection "predicates" (or prompt "Select predicate: ")))

;; Record triplet, this one accepts UUID only and eventually
;; description. You never know why some relation shall be described,
;; it is note about the note. This is lower level function.

(defun rcd-triplets-record-triplet (subject-uuid predicate-uuid object-uuid &optional description)
  (let ((description (or description "")))
    (rcd-sql-first
     "INSERT INTO uuid2uuid (uuid2uuid_subjects, uuid2uuid_predicates, uuid2uuid_objects, uuid2uuid_description)
           VALUES ($1, $2, $3, $4) 
        RETURNING uuid2uuid_id"
     cf-db subject-uuid predicate-uuid object-uuid description)))

;; This is higher level function to record new triplet. UUID can be
;; anything, what if it is other triplet? Other database and table
;; UUID reference? It could be Org reference as well.

(defun rcd-triplets-new (&optional subject-uuid predicate-uuid object-uuid description)
  (interactive)
  (let* ((subject (unless subject-uuid (rcd-triplets-sobject-select "Select subject: ")))
	 (subject-uuid (or subject-uuid (rcd-db-uuid-by-id "sobjects" subject cf-db)))
	 (predicate (unless predicate-uuid (rcd-triplets-predicate-select)))
	 (predicate-uuid (or predicate-uuid (rcd-db-uuid-by-id "predicates" predicate cf-db)))
	 (object (unless object-uuid (rcd-triplets-sobject-select "Select object: ")))
	 (object-uuid (or object-uuid (rcd-db-uuid-by-id "sobjects" object cf-db)))
	 (description (rcd-ask "Description: ")))
    (rcd-triplets-record-triplet subject-uuid predicate-uuid object-uuid description)))

There is more to expand, I have to continue "teaching" computer as to
understand how to implement automated programming functions. It may
involve injecting basic functions in the database, which I already do
with "types" of objects, if object is of PDF type, then I let the "PDF
type" decide by itself how to open PDF, and not that I put it in the
code. That way user has option to change how to open PDF, without
(much) coding.

I am thinking that once it is defined well for example:

- that task has created time
- that task must have name
- that task may be related to people
- that people is database table
- that task may have text body

Then the final triplet may be defined to be actionable, something like
"task", and some triplets would become buttons.

User would then be able to create own useful screen or search through
notes, and then click on button like "add task" to add, or later
delete, duplicate, etc. Some basic objects I have to define as
actionable, like screens, menu, etc. for program to be parsable and
executable, so that human can enter some data, and represent that data
in some way, or search through it, share it.

I have only blurry idea that it may work that way.

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/