Add elisa to GNU ELPA

Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 262 bytes --]

Hi all,

Please add https://github.com/s-kostyaev/elisa to GNU ELPA. This package implements RAG (Retrieval Augmented Generation) for `ellama'. Today I have released 1.0.0 with info manuals, web search and local files support.

Best regards,
Sergey Kostyaev

[-- Attachment #2: Type: text/html, Size: 576 bytes --]

Re: Add elisa to GNU ELPA

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 777 bytes --]

Sergey Kostyaev <sskostyaev@gmail.com> writes:

> Hi all,
>
> Please add https://github.com/s-kostyaev/elisa to GNU ELPA. This
> package implements RAG (Retrieval Augmented Generation) for
> `ellama'. Today I have released 1.0.0 with info manuals, web search
> and local files support.
>
> Best regards,
> Sergey Kostyaev

I'm attaching my comments and suggestions to the end of this message.
Just one point at the beginning, so that you don't miss it: Despite
reading through the entire source code, I have no idea what you are
trying to do with this package.  It would be nice to have some more
context in the Commentary section and elaborate a number of user-facing
docstrings, unless this is intended to be expert software, for people
familiar with whatever the field is.


[-- Attachment #2: Type: text/plain, Size: 25207 bytes --]

diff --git a/elisa.el b/elisa.el
index 09cc28975d..0b79745bd5 100644
--- a/elisa.el
+++ b/elisa.el
@@ -28,7 +28,6 @@
 ;; ELISA (Emacs Lisp Information System Assistant) is a system designed
 ;; to provide informative answers to user queries by leveraging a
 ;; Retrieval Augmented Generation (RAG) approach.
-;;
 
 ;;; Code:
 (require 'ellama)
@@ -48,68 +47,61 @@
 					    (make-llm-ollama
 					     :embedding-model "nomic-embed-text"))
   "Embeddings provider to generate embeddings."
-  :group 'elisa
-  :type '(sexp :validate 'cl-struct-p))
+  :type '(sexp :validate 'cl-struct-p)) ;a more specific predicate here?
 
 (defcustom elisa-chat-provider (progn (require 'llm-ollama)
 				      (make-llm-ollama
 				       :chat-model "sskostyaev/openchat:8k-rag"
 				       :embedding-model "nomic-embed-text"))
   "Chat provider."
-  :group 'elisa
   :type '(sexp :validate 'cl-struct-p))
 
 (defcustom elisa-db-directory (file-truename
 			       (file-name-concat
 				user-emacs-directory "elisa"))
   "Directory for elisa database."
-  :group 'elisa
-  :type 'directory)
+  :type 'directory)                     ;is it necessary that it exists?
 
 (defcustom elisa-limit 5
   "Count quotes to pass into llm context for answer."
-  :group 'elisa
-  :type 'integer)
+  :type 'integer)                       ;or natnum?
 
-(defcustom elisa-find-executable "find"
+(defcustom elisa-find-executable find-program
   "Path to find executable."
-  :group 'elisa
   :type 'string)
 
 (defcustom elisa-tar-executable "tar"
   "Path to tar executable."
-  :group 'elisa
   :type 'string)
 
 (defcustom elisa-sqlite-vss-version "v0.1.2"
   "Sqlite VSS version."
-  :group 'elisa
   :type 'string)
 
 (defcustom elisa-sqlite-vss-path nil
   "Path to sqlite-vss extension."
-  :group 'elisa
   :type 'file)
 
 (defcustom elisa-sqlite-vector-path nil
   "Path to sqlite-vector extension."
-  :group 'elisa
   :type 'file)
 
-(defcustom elisa-semantic-split-function 'elisa-split-by-paragraph
+(defcustom elisa-semantic-split-function #'elisa-split-by-paragraph
   "Function for semantic text split."
-  :group 'elisa
   :type 'function)
 
 (defcustom elisa-prompt-rewriting-enabled t
   "Enable prompt rewriting for better retrieving."
-  :group 'elisa
   :type 'boolean)
 
-(defcustom elisa-chat-prompt-template "Answer user query based on context above. If you can answer it partially do it. Provide list of open questions if any. Say \"not enough data\" if you can't answer user query based on provided context. User query:
+(defcustom elisa-chat-prompt-template
+  "Answer user query based on context above. \
+If you can answer it partially do it. \
+Provide list of open questions if any. \
+Say \"not enough data\" if you can't answer user \
+query based on provided context. User query:
 %s"
-  "Chat prompt template."
-  :group 'elisa
+  "Chat prompt template."               ;some more explanation would be nice.
   :type 'string)
 
 (defcustom elisa-rewrite-prompt-template
@@ -130,81 +122,70 @@ How to buy a pony?
  User prompt:
 %s"
   "Prompt template for prompt rewriting."
-  :group 'elisa
   :type 'string)
 
 (defcustom elisa-searxng-url "http://localhost:8080/"
   "Searxng url for web search.  Json format should be enabled for this instance."
-  :group 'elisa
   :type 'string)
 
 (defcustom elisa-pandoc-executable "pandoc"
-  "Path to pandoc executable."
-  :group 'elisa
+  "Path to pandoc (https://pandoc.org/) executable."
   :type 'string)
 
-(defcustom elisa-webpage-extraction-function 'elisa-get-webpage-buffer
+(defcustom elisa-webpage-extraction-function #'elisa-get-webpage-buffer
   "Function to get buffer with webpage content."
-  :group 'elisa
   :type 'function)
 
-(defcustom elisa-web-search-function 'elisa-search-duckduckgo
+(defcustom elisa-web-search-function #'elisa-search-duckduckgo
   "Function to search the web.
 Function should get prompt and return list of urls."
-  :group 'elisa
   :type 'function)
 
 (defcustom elisa-web-pages-limit 10
   "Limit of web pages to parse during web search."
-  :group 'elisa
-  :type 'integer)
+  :type 'natnum)
 
 (defcustom elisa-breakpoint-threshold-amount 0.4
   "Breakpoint threshold amount.
 Increase it if you need decrease semantic split granularity."
-  :group 'elisa
-  :type 'float)
+  :type 'number)
 
 (defcustom elisa-reranker-enabled nil
   "Enable reranker to improve retrieving quality."
-  :group 'elisa
   :type 'boolean)
 
 (defcustom elisa-reranker-url "http://127.0.0.1:8787/"
-  "Reranker service url."
-  :group 'elisa
+  "Reranker service url."               ;here as well, this doesn't mean much to someone who don't already know what is going on.
   :type 'string)
 
 (defcustom elisa-reranker-similarity-threshold 0
   "Reranker similarity threshold.
 If set, all quotes with similarity less than threshold will be filtered out."
-  :group 'elisa
-  :type 'string)
+  :type 'string)                        ;wrong type?
 
 (defcustom elisa-reranker-limit 20
   "Number of quotes for send to reranker."
-  :group 'elisa
   :type 'integer)
 
 (defcustom elisa-ignore-patterns-files '(".gitignore" ".ignore" ".rgignore")
   "Files with patterns to ignore during file parsing."
-  :group 'elisa
-  :type '(list string))
+  :type '(repeat string))
 
 (defcustom elisa-ignore-invisible-files t
   "Ignore invisible files and directories during file parsing."
-  :group 'elisa
   :type 'boolean)
 
 (defcustom elisa-enabled-collections '("builtin manuals" "external manuals")
   "Enabled collections for elisa chat."
-  :group 'elisa
-  :type '(list string))
+  :type '(repeat string))
 
 (defun elisa-sqlite-vss-download-url ()
+  ;; It seems to be a general problem that your documentation strings
+  ;; are not giving any context, and just barley touching on what is
+  ;; going on...
   "Generate sqlite vss download url based on current system."
-  (cond  ((string-equal system-type "darwin")
-	  (if (string-prefix-p "aarch64" system-configuration)
+  (cond  ((eq system-type 'darwin)
+	  (if (string-prefix-p "aarch64" system-configuration) ;how robust is this?
 	      (format
 	       "https://github.com/asg017/sqlite-vss/releases/download/%s/sqlite-vss-%s-loadable-macos-aarch64.tar.gz"
 	       elisa-sqlite-vss-version
@@ -213,7 +194,7 @@ If set, all quotes with similarity less than threshold will be filtered out."
 	     "https://github.com/asg017/sqlite-vss/releases/download/%s/sqlite-vss-%s-loadable-macos-x86_64.tar.gz"
 	     elisa-sqlite-vss-version
 	     elisa-sqlite-vss-version)))
-	 ((string-equal system-type "gnu/linux")
+	 ((eq system-type 'gnu/linux)
 	  (format
 	   "https://github.com/asg017/sqlite-vss/releases/download/%s/sqlite-vss-%s-loadable-linux-x86_64.tar.gz"
 	   elisa-sqlite-vss-version
@@ -223,18 +204,14 @@ If set, all quotes with similarity less than threshold will be filtered out."
 (defun elisa--vss-path ()
   "Path to vss sqlite extension."
   (or elisa-sqlite-vss-path
-      (let* ((ext (if (string-equal system-type "darwin")
-		      "dylib"
-		    "so"))
+      (let* ((ext (if (eq system-type 'darwin) "dylib" "so"))
 	     (file (format "vss0.%s" ext)))
 	(file-name-concat elisa-db-directory file))))
 
 (defun elisa--vector-path ()
   "Path to vector sqlite extension."
   (or elisa-sqlite-vector-path
-      (let* ((ext (if (string-equal system-type "darwin")
-		      "dylib"
-		    "so"))
+      (let* ((ext (if (string-equal system-type 'darwin) "dylib" "so"))
 	     (file (format "vector0.%s" ext)))
 	(file-name-concat elisa-db-directory file))))
 
@@ -260,45 +237,45 @@ If set, all quotes with similarity less than threshold will be filtered out."
 
 (defun elisa-embeddings-create-table-sql ()
   "Generate sql for create embeddings table."
-  "drop table if exists elisa_embeddings;")
+  "DROP TABLE IF EXISTS elisa_embeddings;") ;just my personal taste, ignore if you disagree (i do it because there is no sql syntax highlighting in elisp strings)
 
 (defun elisa-data-embeddings-create-table-sql ()
   "Generate sql for create data embeddings table."
-  (format "create virtual table if not exists data_embeddings using vss0(embedding(%d));"
+  (format "CREATE VIRTUAL TABLE IF NOT EXISTS data_embeddings USING vss0(embedding(%d));"
 	  (elisa-get-embedding-size)))
 
 (defun elisa-data-fts-create-table-sql ()
   "Generate sql for create full text search table."
-  "create virtual table if not exists data_fts using fts5(data);")
+  "CREATE VIRTUAL TABLE IF NOT EXISTS data_fts USING FTS5(data);")
 
 (defun elisa-info-create-table-sql ()
   "Generate sql for create info table."
-  "drop table if exists info;")
+  "DROP TABLE IF EXISTS info;")
 
 (defun elisa-collections-create-table-sql ()
   "Generate sql for create collections table."
-  "create table if not exists collections (name text unique);")
+  "CREATE TABLE IF NOT EXISTS collections (name TEXT UNIQUE);")
 
 (defun elisa-kinds-create-table-sql ()
   "Generate sql for create kinds table."
-  "create table if not exists kinds (name text unique);")
+  "CREATE TABLE IF NOT EXISTS kinds (name TEXT UNIQUE);")
 
 (defun elisa-fill-kinds-sql ()
   "Generate sql for fill kinds table."
-  "insert into kinds (name) values ('web'), ('file'), ('info') on conflict do nothing;")
+  "INSERT INTO KINDS (name) VALUES ('web'), ('file'), ('info') ON CONFLICT DO NOTHING;")
 
 (defun elisa-files-create-table-sql ()
   "Generate sql for create files table."
-  "create table if not exists files (path text unique, hash text)")
+  "CREATE TABLE IF NOT EXISTS files (path TEXT UNIQUE, hash TEXT)")
 
 (defun elisa-data-create-table-sql ()
   "Generate sql for create data table."
-  "create table if not exists data (
+  "CREATE TABLE IF NOT EXISTS data (
 kind_id INTEGER,
 collection_id INTEGER,
-path text,
-hash text,
-data text,
+path TEXT,
+hash TEXT,
+data TEXT,
 FOREIGN KEY(kind_id) REFERENCES kinds(rowid),
 FOREIGN KEY(collection_id) REFERENCES collections(rowid)
 );")
@@ -307,13 +284,9 @@ FOREIGN KEY(collection_id) REFERENCES collections(rowid)
   "Initialize elisa DB."
   (if (not (file-exists-p (elisa--vss-path)))
       (warn "Please run M-x `elisa-download-sqlite-vss' to use this package")
-    (sqlite-pragma db "PRAGMA journal_mode=WAL;")
-    (sqlite-load-extension
-     db
-     (elisa--vector-path))
-    (sqlite-load-extension
-     db
-     (elisa--vss-path))
+    (sqlite-pragma db "PRAGMA journal_mode=WALL;")
+    (sqlite-load-extension db (elisa--vector-path))
+    (sqlite-load-extension db (elisa--vss-path))
     (sqlite-execute db (elisa-embeddings-create-table-sql))
     (sqlite-execute db (elisa-info-create-table-sql))
     (sqlite-execute db (elisa-collections-create-table-sql))
@@ -324,44 +297,46 @@ FOREIGN KEY(collection_id) REFERENCES collections(rowid)
     (sqlite-execute db (elisa-data-embeddings-create-table-sql))
     (sqlite-execute db (elisa-data-fts-create-table-sql))))
 
-(defvar elisa-db (progn
-		   (make-directory elisa-db-directory t)
-		   (let ((db (sqlite-open (file-name-concat elisa-db-directory "elisa.sqlite"))))
-		     (elisa--init-db db)
-		     db)))
+(defvar elisa-db
+  (let ((_ (make-directory elisa-db-directory t))
+        (db (sqlite-open (file-name-concat elisa-db-directory "elisa.sqlite"))))
+    (elisa--init-db db)
+    db))
 
 (defun elisa-vector-to-sqlite (data)
   "Convert DATA to sqlite vector representation."
-  (format "vector_from_json(json('%s'))"
-	  (json-encode data)))
-
-(defun elisa-sqlite-escape (s)
-  "Escape single quotes in S for sqlite."
-  (thread-last
-    s
-    (string-replace "'" "''")
-    (string-replace "\\" "\\\\")
-    (string-replace "\0" "\n")))
+  (format "vector_from_json(json('%s'))" (json-encode data)))
+
+(defun elisa-sqlite-escape (string)
+  "Escape single quotes in STRING for sqlite."
+  (let ((reps '(("'" . "''")
+                ("\\" . "\\\\")
+                ("\0" . "\n"))))
+    (replace-regexp-in-string         ;simultanious replacement
+     (regexp-opt (mapcar #'car reps)) ;is the last one really \0 or \\0?
+     (lambda (str) (alist-get str reps nil nil #'string=))
+     string nil t)))
 
 (defun elisa-sqlite-format-int-list (ids)
   "Convert list of integer IDS list to sqlite list representation."
   (format
    "(%s)"
-   (string-join (mapcar (lambda (id) (format "%d" id)) ids) ", ")))
+   (mapconcat (lambda (id) (format "%d" id)) ids ", ")))
 
 (defun elisa-sqlite-format-string-list (names)
   "Convert list of string NAMES list to sqlite list representation."
   (format
    "(%s)"
-   (string-join (mapcar (lambda (name)
-			  (format "'%s'"
-				  (elisa-sqlite-escape name))) names) ", ")))
+   (mapconcat (lambda (name)
+		(format "'%s'"
+			(elisa-sqlite-escape name)))
+              names ", ")))
 
-(defun elisa-avg (lst)
-  "Calculate arithmetic average value of LST."
-  (let ((len (length lst))
-	(sum (cl-reduce #'+ lst :initial-value 0.0)))
-    (/ sum len)))
+(defun elisa-avg (list)
+  "Calculate arithmetic average value of LIST."
+  (cl-loop for elem in list for count from 0
+           summing elem into sum
+           finally (return (/ sum (float count)))))
 
 (defun elisa-std-dev (lst)
   "Calculate standart deviation value of LST."
@@ -450,6 +425,11 @@ FOREIGN KEY(collection_id) REFERENCES collections(rowid)
 		     (setq continue nil))))
 	      (setq continue nil))))))))
 
+;; this is pretty verbose, do you think there might be a more readable
+;; way to express this?  It is pretty easy to create a small
+;; templating language in Elisp
+;; (e.g. https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/elpa-admin.el?h=elpa-admin#n1138),
+;; perhaps that could also be useful here.
 (defun elisa--find-similar (text collections)
   "Find similar to TEXT results in COLLECTIONS.
 Return sqlite query.  For asyncronous execution."
@@ -520,7 +500,7 @@ Evaluate ON-DONE with result."
 	(result nil))
     (save-excursion
       (goto-char (point-min))
-      (while (< (point) (point-max))
+      (while (not (eobp))
 	(funcall func)
 	(push (buffer-substring-no-properties pt (point)) result)
 	(setq pt (point)))
@@ -534,6 +514,7 @@ Evaluate ON-DONE with result."
   "Split buffer to list of paragraphs."
   (elisa--split-by #'forward-paragraph))
 
+;; a number of these functions seem like something that should be added to the core of Emacs or at least a common ELPA package...
 (defun elisa-dot-product (v1 v2)
   "Calculate the dot produce of vectors V1 and V2."
   (let ((result 0))
@@ -608,14 +589,12 @@ than T, it will be packed into single semantic chunk."
 	    (current (car chunks))
 	    (tail (cdr chunks)))
       (let* ((result nil))
-	(mapc
-	 (lambda (el)
-	   (if (<= el threshold)
+        (dolist (el distances)
+          (if (<= el threshold)
 	       (setq current (concat current (car tail)))
 	     (push current result)
 	     (setq current (car tail)))
 	   (setq tail (cdr tail)))
-	 distances)
 	(push current result)
 	(cl-remove-if
 	 #'string-empty-p
@@ -626,6 +605,7 @@ than T, it will be packed into single semantic chunk."
 		 (nreverse result))))
     (list (buffer-substring-no-properties (point-min) (point-max)))))
 
+;; why not use `wildcard-to-regexp'?
 (defun elisa--gitignore-to-elisp-regexp (pattern)
   "Convert a .gitignore PATTERN to an Emacs Lisp regexp."
   (let ((result "")
@@ -676,11 +656,11 @@ than T, it will be packed into single semantic chunk."
 
 (defun elisa--text-file-p (filename)
   "Check if FILENAME contain text."
-  (or (when (get-file-buffer filename) t) ;; if file opened assume it text
+  (or (and (get-file-buffer filename) t) ;; if file opened assume it text
       (with-current-buffer (find-file-noselect filename t t)
 	(prog1
 	    ;; if there is null byte in file, file is binary
-	    (not (re-search-forward "\0" nil t 1))
+	    (not (search-forward "\0" nil t 1))
 	  (kill-buffer)))))
 
 (defun elisa--file-list (directory)
@@ -727,41 +707,39 @@ When FORCE parse even if already parsed."
 	     (format "delete from files where path = '%s';"
 		     (elisa-sqlite-escape path))))
 	  ;; add new data
-	  (mapc
-	   (lambda (text)
-	     (let* ((hash (secure-hash 'sha256 text))
-		    (rowid
-		     (if-let ((rowid (caar (sqlite-select
-					    elisa-db
-					    (format "select rowid from data where kind_id = %s and collection_id = %s and path = '%s' and hash = '%s';"
-						    kind-id collection-id
-						    (elisa-sqlite-escape path) hash)))))
-			 (progn
-			   (push rowid row-ids)
-			   nil)
-		       (sqlite-execute
-			elisa-db
-			(format
-			 "insert into data(kind_id, collection_id, path, hash, data) values (%s, %s, '%s', '%s', '%s');"
-			 kind-id collection-id
-			 (elisa-sqlite-escape path) hash (elisa-sqlite-escape text)))
-		       (caar (sqlite-select
-			      elisa-db
-			      (format "select rowid from data where kind_id = %s and collection_id = %s and path = '%s' and hash = '%s';"
-				      kind-id collection-id
-				      (elisa-sqlite-escape path) hash))))))
-	       (when rowid
-		 (sqlite-execute
-		  elisa-db
-		  (format "insert into data_embeddings(rowid, embedding) values (%s, %s);"
-			  rowid (elisa-vector-to-sqlite
-				 (llm-embedding elisa-embeddings-provider text))))
-		 (sqlite-execute
-		  elisa-db
-		  (format "insert into data_fts(rowid, data) values (%s, '%s');"
-			  rowid (elisa-sqlite-escape text)))
-		 (push rowid row-ids))))
-	   chunks)
+          (dolist (text chunks)
+            (let* ((hash (secure-hash 'sha256 text))
+		   (rowid
+		    (if-let ((rowid (caar (sqlite-select
+					   elisa-db
+					   (format "select rowid from data where kind_id = %s and collection_id = %s and path = '%s' and hash = '%s';"
+						   kind-id collection-id
+						   (elisa-sqlite-escape path) hash)))))
+			(progn
+			  (push rowid row-ids)
+			  nil)
+		      (sqlite-execute
+		       elisa-db
+		       (format
+			"insert into data(kind_id, collection_id, path, hash, data) values (%s, %s, '%s', '%s', '%s');"
+			kind-id collection-id
+			(elisa-sqlite-escape path) hash (elisa-sqlite-escape text)))
+		      (caar (sqlite-select
+			     elisa-db
+			     (format "select rowid from data where kind_id = %s and collection_id = %s and path = '%s' and hash = '%s';"
+				     kind-id collection-id
+				     (elisa-sqlite-escape path) hash))))))
+	      (when rowid
+		(sqlite-execute
+		 elisa-db
+		 (format "insert into data_embeddings(rowid, embedding) values (%s, %s);"
+			 rowid (elisa-vector-to-sqlite
+				(llm-embedding elisa-embeddings-provider text))))
+		(sqlite-execute
+		 elisa-db
+		 (format "insert into data_fts(rowid, data) values (%s, '%s');"
+			 rowid (elisa-sqlite-escape text)))
+		(push rowid row-ids))))
 	  ;; remove old data
 	  (when row-ids
 	    (let ((delete-rows (cl-remove-if (lambda (id)
@@ -779,7 +757,7 @@ When FORCE parse even if already parsed."
 
 (defun elisa--delete-data (ids)
   "Delete data with IDS."
-  (sqlite-execute
+  (sqlite-execute                       ;perhaps it would be worth extracting the (sqlite-execute elisa-db ...) part into a separate function
    elisa-db
    (format "delete from data_fts where rowid in %s;"
 	   (elisa-sqlite-format-int-list ids)))
@@ -815,10 +793,9 @@ When FORCE parse even if already parsed."
 			collection-id
 			(elisa-sqlite-format-string-list files))))))
     (elisa--delete-data delete-ids)
-    (mapc (lambda (file)
-	    (message "parsing %s" file)
-	    (elisa-parse-file collection-id file))
-	  files)))
+    (dolist (file files)
+      (message "parsing %s" file)
+      (elisa-parse-file collection-id file))))
 
 ;;;###autoload
 (defun elisa-async-parse-directory (dir)
@@ -853,14 +830,14 @@ When FORCE parse even if already parsed."
 	  (libxml-parse-html-region
 	   (point) (point-max))
 	  'a))
-	:test 'string-equal)))))
+	:test #'string-equal)))))
 
 (defun elisa-search-searxng (prompt)
   "Search searxng for PROMPT and return list of urls.
 You can customize `elisa-searxng-url' to use non local instance."
   (let ((url (format "%s/search?format=json&q=%s" elisa-searxng-url (url-hexify-string prompt))))
     (thread-last
-      (plz 'get url :as 'json-read)
+      (plz 'get url :as 'json-read)     ;I am not familiar with the "plz" library, is the `json-read' a function?
       (alist-get 'results)
       (mapcar (lambda (el) (alist-get 'url el))))))
 
@@ -896,14 +873,13 @@ You can customize `elisa-searxng-url' to use non local instance."
     (with-current-buffer buffer-name
       (shell-command-on-region
        (point-min) (point-max)
-       (format "%s -f html --to plain"
-	       (executable-find elisa-pandoc-executable))
+       (format "%s --from html --to plain" elisa-pandoc-executable)
        buffer-name t)
       buffer-name)))
 
 (defun elisa-fts-query (prompt)
   "Return fts match query for PROMPT."
-  (thread-last
+  (thread-last                          ;i belive you can do all of this with a single regular expression...
     prompt
     (string-trim)
     (downcase)
@@ -938,7 +914,7 @@ You can customize `elisa-searxng-url' to use non local instance."
 		  :headers `(("Content-Type" . "application/json"))
 		  :body-type 'text
 		  :body (elisa--rerank-request prompt ids)
-		  :as #'json-read)))))
+		  :as #'json-read)))))  ;so it is a function!
 
 (defun elisa-rerank (prompt ids)
   "Rerank IDS according to PROMPT and return top `elisa-limit' IDS."
@@ -962,11 +938,10 @@ You can customize `elisa-searxng-url' to use non local instance."
 (defun elisa--parse-web-page (collection-id url)
   "Parse URL into collection with COLLECTION-ID."
   (let ((kind-id (caar (sqlite-select
-			elisa-db "select rowid from kinds where name = 'web';"))))
-    (message "collecting data from %s" url)
-    (mapc
-     (lambda (chunk)
-       (let* ((hash (secure-hash 'sha256 chunk))
+			elisa-db "SELECT rowid FROM kinds WHERE name = 'web';"))))
+    (message "collecting data from %S..." url)
+    (dolist (chunk (elisa-extact-webpage-chunks url))
+      (let* ((hash (secure-hash 'sha256 chunk))
 	      (embedding (llm-embedding elisa-embeddings-provider chunk))
 	      (rowid
 	       (if-let ((rowid (caar (sqlite-select
@@ -989,8 +964,7 @@ You can customize `elisa-searxng-url' to use non local instance."
 	   (sqlite-execute
 	    elisa-db
 	    (format "insert into data_fts(rowid, data) values (%s, '%s');"
-		    rowid (elisa-sqlite-escape chunk))))))
-     (elisa-extact-webpage-chunks url))))
+		    rowid (elisa-sqlite-escape chunk))))))))
 
 (defun elisa--web-search (prompt)
   "Search the web for PROMPT.
@@ -1007,11 +981,10 @@ Return sqlite query that extract data for adding to context."
 				(elisa-sqlite-escape prompt)))))
 	 (urls (funcall elisa-web-search-function prompt))
 	 (collected-pages 0))
-    (mapc (lambda (url)
-	    (when (<= collected-pages elisa-web-pages-limit)
-	      (elisa--parse-web-page collection-id url)
-	      (cl-incf collected-pages)))
-	  urls)))
+    (dolist (url urls)
+      (when (<= collected-pages elisa-web-pages-limit)
+	(elisa--parse-web-page collection-id url)
+	(cl-incf collected-pages)))))
 
 (defun elisa--rewrite-prompt (prompt action)
   "Rewrite PROMPT if `elisa-prompt-rewriting-enabled'.
@@ -1071,7 +1044,7 @@ WHERE d.rowid in %s;"
 		   (when-let ((kind (cl-first row))
 			      (path (cl-second row))
 			      (text (cl-third row)))
-		     (pcase kind
+		     (pcase kind        ;is this a `pcase-exhaustive'?
 		       ("web"
 			(ellama-context-add-webpage-quote-noninteractive path path text))
 		       ("file"
@@ -1096,15 +1069,16 @@ WHERE d.rowid in %s;"
   (mapcar
    #'file-name-base
    (cl-remove-if-not
-    (lambda (s) (or (string-suffix-p ".info" s)
-		    (string-suffix-p ".info.gz" s)))
+    (lambda (s)
+      (or (string-suffix-p ".info" s)
+	  (string-suffix-p ".info.gz" s)))
     (directory-files (with-temp-buffer
 		       (info "emacs" (current-buffer))
 		       (file-name-directory Info-current-file))))))
 
 (defun elisa-get-external-manuals ()
   "Get external manual names list."
-  (cl-remove-if
+  (cl-remove-if                         ;a `thread-last' might be nice here
    #'not
    (mapcar
     #'elisa--info-valid-p
@@ -1112,7 +1086,7 @@ WHERE d.rowid in %s;"
      (mapcar
       #'file-name-base
       (process-lines
-       (executable-find elisa-find-executable)
+       elisa-find-executable
        (file-truename
 	(file-name-concat user-emacs-directory "elpa"))
        "-name" "*.info"))))))
@@ -1207,7 +1181,7 @@ It does nothing if buffer file not inside one of existing collections."
   (when-let* ((collections (flatten-tree
 			    (sqlite-select
 			     elisa-db
-			     "select name from collections;")))
+			     "SELECT name FROM collections;")))
 	      (dirs (cl-remove-if-not #'file-directory-p collections))
 	      (file (buffer-file-name))
 	      (collection (cl-find-if (lambda (dir)
@@ -1283,7 +1257,7 @@ It does nothing if buffer file not inside one of existing collections."
   "Add webpage by URL to COLLECTION."
   (interactive
    (list
-    (if-let ((url (or (and (fboundp 'thing-at-point) (thing-at-point 'url))
+    (if-let ((url (or (and (fboundp 'thing-at-point) (thing-at-point 'url)) ;why not always use `thing-at-point'?
                       (shr-url-at-point nil))))
         url
       (read-string "Enter URL you want to summarize: "))

[-- Attachment #3: Type: text/plain, Size: 39 bytes --]




-- 
	Philip Kaludercic on peregrine

Re: Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 3008 bytes --]

Thank you for review.

> 16 июля 2024 г., в 19:54, Philip Kaludercic <philipk@posteo.net> написал(а):
> 
> Despite
> reading through the entire source code, I have no idea what you are
> trying to do with this package.  It would be nice to have some more
> context in the Commentary section and elaborate a number of user-facing
> docstrings, unless this is intended to be expert software, for people
> familiar with whatever the field is.

I need to improve it. With README.org <http://readme.org/> it should be cleaner, but It should be understandable without it.

Why should we remove elisa customization group? I think there are many customization options to move it to group.

> @@ -48,68 +47,61 @@
> 					    (make-llm-ollama
> 					     :embedding-model "nomic-embed-text"))
>   "Embeddings provider to generate embeddings."
> -  :group 'elisa
> -  :type '(sexp :validate 'cl-struct-p))
> +  :type '(sexp :validate 'cl-struct-p)) ;a more specific predicate here?

It should be `llm` provider. Unfortunately there are no more specific predicate for that.

> (defcustom elisa-db-directory (file-truename
> 			       (file-name-concat
> 				user-emacs-directory "elisa"))
>   "Directory for elisa database."
> -  :group 'elisa
> -  :type 'directory)
> +  :type 'directory)                     ;is it necessary that it exists?

If you mean option - then yes. If you mean directory, I can create it in `elisa' package.

> +     (regexp-opt (mapcar #'car reps)) ;is the last one really \0 or \\0?

Really. Probably impossible case, but it’s defensive programming.

> +;; a number of these functions seem like something that should be added to the core of Emacs or at least a common ELPA package...
> (defun elisa-dot-product (v1 v2)
>   "Calculate the dot produce of vectors V1 and V2."

Good idea. How can we implement it?

> -       (format "%s -f html --to plain"
> -	       (executable-find elisa-pandoc-executable))
> +       (format "%s --from html --to plain" elisa-pandoc-executable)
>        buffer-name t)
>       buffer-name)))

We should keep executable-find call here. It was feature request from users. It can be useful with per-directory PATH, like with direnv.

> 
> (defun elisa-fts-query (prompt)
>   "Return fts match query for PROMPT."
> -  (thread-last
> +  (thread-last                          ;i belive you can do all of this with a single regular expression...
>     prompt
>     (string-trim)
>     (downcase)

Maybe. But I prefer to keep it readable.
> @@ -1071,7 +1044,7 @@ WHERE d.rowid in %s;"
> 		   (when-let ((kind (cl-first row))
> 			      (path (cl-second row))
> 			      (text (cl-third row)))
> -		     (pcase kind
> +		     (pcase kind        ;is this a `pcase-exhaustive'?
> 		       ("web"
> 			(ellama-context-add-webpage-quote-noninteractive path path text))
> 		       ("file"

I think it should be extendable, but for now it is not.

Best regards,
Sergey Kostyaev


[-- Attachment #2: Type: text/html, Size: 7120 bytes --]

Re: Add elisa to GNU ELPA

[Philip Kaludercic]

Sergey Kostyaev <sskostyaev@gmail.com> writes:

> Thank you for review.
>
>> 16 июля 2024 г., в 19:54, Philip Kaludercic <philipk@posteo.net> написал(а):
>> 
>> Despite
>> reading through the entire source code, I have no idea what you are
>> trying to do with this package.  It would be nice to have some more
>> context in the Commentary section and elaborate a number of user-facing
>> docstrings, unless this is intended to be expert software, for people
>> familiar with whatever the field is.
>
> I need to improve it. With README.org <http://readme.org/> it should be cleaner, but It should be understandable without it.

I wouldn't want to rely on the README.  Or at least I think of a README
file less as an introduction to the package, and more of a introduction
to the source code repository.  Having a self-contained commentary
section that just explains what the package was made for and how to use
it is usually what I like seeing the most.

> Why should we remove elisa customization group? I think there are many customization options to move it to group.

Because unless I missed something, you had only defined a single group.
By default, a `defcustom' will be added to the last group defined in
file, making it redundant to specify it again and again.  That being
said, if you prefer this explicit redundancy, feel free to keep the
:group attributes.

>> @@ -48,68 +47,61 @@
>> 					    (make-llm-ollama
>> 					     :embedding-model "nomic-embed-text"))
>>   "Embeddings provider to generate embeddings."
>> -  :group 'elisa
>> -  :type '(sexp :validate 'cl-struct-p))
>> +  :type '(sexp :validate 'cl-struct-p)) ;a more specific predicate here?
>
> It should be `llm` provider. Unfortunately there are no more specific predicate for that.

So the `llm' struct doesn't havea `llm-p' predicate?

>> (defcustom elisa-db-directory (file-truename
>> 			       (file-name-concat
>> 				user-emacs-directory "elisa"))
>>   "Directory for elisa database."
>> -  :group 'elisa
>> -  :type 'directory)
>> +  :type 'directory)                     ;is it necessary that it exists?
>
> If you mean option - then yes. If you mean directory, I can create it in `elisa' package.

I meant if you should add a :must-match t tag.

>> +     (regexp-opt (mapcar #'car reps)) ;is the last one really \0 or \\0?
>
> Really. Probably impossible case, but it’s defensive programming.

Ok!

>> +;; a number of these functions seem like something that should be added to the core of Emacs or at least a common ELPA package...
>> (defun elisa-dot-product (v1 v2)
>>   "Calculate the dot produce of vectors V1 and V2."
>
> Good idea. How can we implement it?

To my knowledge, there is no maths package in the core or ELPA (someone
correct me if I am wrong).  There is calc, but IIRC it wasn't that nice
to use from an Elisp caller.  I could imagine that just pulling out the
definition you gave here, renaming them from "elisa-foo" to "foo" would
be enough for a first daft that could be added to the core, and then to
ELPA as a core package.

>> -       (format "%s -f html --to plain"
>> -	       (executable-find elisa-pandoc-executable))
>> +       (format "%s --from html --to plain" elisa-pandoc-executable)
>>        buffer-name t)
>>       buffer-name)))
>
> We should keep executable-find call here. It was feature request from users. It can be useful with per-directory PATH, like with direnv.

I might be missing something, but resolving a executable in PATH before
invoking it shouldn't have any effect, as call-process et. AL. will
resolve the executable name in the same context as an explicit
`executable-find' does anyway.

>> 
>> (defun elisa-fts-query (prompt)
>>   "Return fts match query for PROMPT."
>> -  (thread-last
>> +  (thread-last                          ;i belive you can do all of this with a single regular expression...
>>     prompt
>>     (string-trim)
>>     (downcase)
>
> Maybe. But I prefer to keep it readable.

I don't necessarily believe it would be less readable, and now you made
me want to try it out.  I'll report back at some point.

>> @@ -1071,7 +1044,7 @@ WHERE d.rowid in %s;"
>> 		   (when-let ((kind (cl-first row))
>> 			      (path (cl-second row))
>> 			      (text (cl-third row)))
>> -		     (pcase kind
>> +		     (pcase kind        ;is this a `pcase-exhaustive'?
>> 		       ("web"
>> 			(ellama-context-add-webpage-quote-noninteractive path path text))
>> 		       ("file"
>
> I think it should be extendable, but for now it is not.

Ok.

> Best regards,
> Sergey Kostyaev
>

-- 
	Philip Kaludercic on peregrine

Re: Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 238 bytes --]



> 16 июля 2024 г., в 23:04, Philip Kaludercic <philipk@posteo.net> написал(а):
> 
> So the `llm' struct doesn't havea `llm-p' predicate?

No. Maybe Andrew Hyatt can help with it?

Best regards,
Sergey Kostyaev

[-- Attachment #2: Type: text/html, Size: 1115 bytes --]

Re: Add elisa to GNU ELPA

[Andrew Hyatt]

Sergey Kostyaev <sskostyaev@gmail.com> writes:

>  16 июля 2024 г., в 23:04, Philip Kaludercic <philipk@posteo.net> написал(а):
>
>  So the `llm' struct doesn't havea `llm-p' predicate?
>
> No. Maybe Andrew Hyatt can help with it?

You could use `llm-standard-chat-provider-p' for now, which should get
everything except `llm-fake' and any providers people have been
developing on the side.

I can add something even more comprehensive in a next release if it
helps, though.

>
> Best regards,
> Sergey Kostyaev

Re: Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 574 bytes --]


> 16 июля 2024 г., в 23:04, Philip Kaludercic <philipk@posteo.net> написал(а):
> 
> To my knowledge, there is no maths package in the core or ELPA (someone
> correct me if I am wrong).  There is calc, but IIRC it wasn't that nice
> to use from an Elisp caller.  I could imagine that just pulling out the
> definition you gave here, renaming them from "elisa-foo" to "foo" would
> be enough for a first daft that could be added to the core, and then to
> ELPA as a core package.

What should be the first step?

Best regards,
Sergey Kostyaev


[-- Attachment #2: Type: text/html, Size: 4874 bytes --]

Re: Add elisa to GNU ELPA

[Philip Kaludercic]

Sergey Kostyaev <sskostyaev@gmail.com> writes:

>> 16 июля 2024 г., в 23:04, Philip Kaludercic <philipk@posteo.net> написал(а):
>> 
>> To my knowledge, there is no maths package in the core or ELPA (someone
>> correct me if I am wrong).  There is calc, but IIRC it wasn't that nice
>> to use from an Elisp caller.  I could imagine that just pulling out the
>> definition you gave here, renaming them from "elisa-foo" to "foo" would
>> be enough for a first daft that could be added to the core, and then to
>> ELPA as a core package.
>
> What should be the first step?

Someone™ should propose a concrete suggestion.

> Best regards,
> Sergey Kostyaev
>

-- 
	Philip Kaludercic on peregrine

Adding a generic mathematical library

[Philip Kaludercic]

Philip Kaludercic <philipk@posteo.net> writes:

> Sergey Kostyaev <sskostyaev@gmail.com> writes:
>
>>> 16 июля 2024 г., в 23:04, Philip Kaludercic <philipk@posteo.net> написал(а):
>>> 
>>> To my knowledge, there is no maths package in the core or ELPA (someone
>>> correct me if I am wrong).  There is calc, but IIRC it wasn't that nice
>>> to use from an Elisp caller.  I could imagine that just pulling out the
>>> definition you gave here, renaming them from "elisa-foo" to "foo" would
>>> be enough for a first daft that could be added to the core, and then to
>>> ELPA as a core package.
>>
>> What should be the first step?
>
> Someone™ should propose a concrete suggestion.

I've rename the thread to get some more attention, and also wanted to
mention, that if we do so, I think putting some thought into the design
wouldn't be bad.  Including an infix-notation macro and perhaps even
overloaded operator support ala APL would be a nice thing to have.  We'd
have to weigh the cost of this kind of flexibility with performance,
unless someone were to write a specialised byte-compiler.

>> Best regards,
>> Sergey Kostyaev
>>

-- 
	Philip Kaludercic on peregrine

Re: Adding a generic mathematical library

[Emanuel Berg]

Philip Kaludercic wrote:

>> Someone™ should propose a concrete suggestion.
>
> I've rename the thread to get some more attention, and also
> wanted to mention, that if we do so, I think putting some
> thought into the design wouldn't be bad. Including an
> infix-notation macro and perhaps even overloaded operator
> support ala APL would be a nice thing to have. We'd have to
> weigh the cost of this kind of flexibility with performance,
> unless someone were to write a specialised byte-compiler.

Yeah, I've pushed for a math library for a long time.

I have several files that are very close to that (for stats,
time, permutations and so on) but I also have a math.el which
I yank below if any of it can be used.

I'd love to stay in the loop and read more on this!

;;; -*- lexical-binding: t -*-
;;
;; this file:
;;   https://dataswamp.org/~incal/emacs-init/math.el

(require 'cl-lib)

(require 'dwim) ; https://dataswamp.org/~incal/emacs-init/dwim.el
(require 'perm) ; https://dataswamp.org/~incal/emacs-init/perm.el

(defun nand (&rest conditions)
  (when (member nil conditions)
    t) )

;; (nand)           ; nil
;; (nand 1 2 3)     ; nil
;; (nand nil)       ; t
;; (nand 1 2 nil 3) ; t

;; (not (and))           ; nil
;; (not (and 1 2 3))     ; nil
;; (not (and nil))       ; t
;; (not (and 1 2 nil 3)) ; t

(defalias '**   #'expt)
(defalias 'ceil #'ceiling)

(defun percent-change (from to)
  (if (= from to)
      0
    (/ (- to from) (abs from) 0.01)))

;; (percent-change  1  2) ;  100
;; (percent-change  1  1) ;    0
;; (percent-change  1  0) ; -100
;; (percent-change  1 -1) ; -200
;;
;; (percent-change  0  1) ;  1.0e+INF
;; (percent-change  0  0) ;  0
;; (percent-change  0 -1) ; -1.0e+INF
;;
;; (percent-change -1  1) ;  200
;; (percent-change -1  0) ;  100
;; (percent-change -1 -1) ;    0
;; (percent-change -1 -2) ; -100

(defun distance-point (min max)
  (+ min (/ (- max min) 2.0)) )

;; (distance-point  1   5) ; 3
;; (distance-point  1  10) ; 5.5
;; (distance-point 60 100) ; 80

(defun // (nom denom)
  (/ nom denom 1.0) )

;; (/  8 256) ; 0
;; (// 8 256) ; 0.03125

(defun arith-sum (n)
  (cl-loop
    with sum = 0
    for i from 1 to (1- n)
    do (cl-incf sum i)
    finally return sum) )

(defun digs (n)
  (* n (1- n)) )

(defun mean-value (vs)
  (let*((sum  (apply #'+ vs))
        (mean (/ sum (length vs) 1.0)) )
    mean) )

(defun mean-str (strs)
  (mean-value (cl-map 'list #'length strs)) )

(defun faster (old new)
  (format "%.1f%%" (* 100 (1- (/ new old 1.0)))) )

;; (faster 0.2  0.5)  ; 150.0%
;; (faster 1.35 0.84) ; -37.8%
;; (faster  200  400) ; 100.0%
;; (faster 1000 1000) ;   0.0%
;; (faster 1000  100) ; -90.0%

(defun percent (nom &optional denom str)
  (or denom (setq denom 1))
  (let ((perc (/ nom denom 0.01)))
    (if str
        (format "%.1f%%" perc)
      perc)))

;; (percent 0.25)           ; 25.0
;; (percent 50 75)          ; 66.66 ...
;; (percent (// 1 2) nil t) ; 50.0%
;; (percent 1019 22 t)      ; 4631.8%

(let ((min-def 0)
      (max-def 9)
      (inc-def 1) )
  (defun interval (&optional min max inc)
    (interactive `(,(read-number "min: " min-def)
                   ,(read-number "max: " max-def)
                   ,(read-number "inc: " inc-def)) )
    (or min (setq min min-def))
    (or max (setq max max-def))
    (or inc (setq inc inc-def))
    (unless (<= min max)
      (error "Bogus interval") )
    (unless (> inc 0)
      (error "Bogus increment") )
    (cl-loop
      for i from min to max by inc
      collect i) )
  (declare-function interval nil) )

;; (interval 10 5)        ; Bogus interval
;; (interval 1 3 -1)      ; Bogus increment
;; (interval 5 10)        ; (5 6 7 8 9 10)
;; (interval 1.8 2.0 0.1) ; (1.8 1.9 2.0)
;; (interval)             ; (0 1 2 3 4 5 6 7 8 9)
;; (interval 19 99)       ; (19 20 21 ... 97 98 99)

(defun digits-sum (&optional beg end)
  (interactive (use-region))
  (or beg (setq beg (point-min)))
  (or end (setq end (point-max)))
  (let ((sum 0))
    (goto-char beg)
    (while (re-search-forward " [[:digit:].]+ " end t)
      (cl-incf sum (string-to-number (match-string-no-properties 0))) )
    (prog1
        sum
      (message "sum: %.1f" sum) )))

(defalias 'di #'digits-sum)

(defun film-mean ()
  (interactive)
  (let ((scores)
        (beg (point-min))
        (end (point-max)) )
    (save-excursion
      (goto-char beg)
      (while (re-search-forward "[[:digit:]]" end t)
        (when (= (current-column) 73)
          (push (string-to-number (match-string-no-properties 0)) scores) )))
    (when scores
      (message (format "%.3f" (mean-value scores))) )))

(defun positive-integer-p (n)
  (and
    (integerp n)
    (< 0 n) ))

;; (positive-integer-p  1.5) ; nil
;; (positive-integer-p  1)   ; t
;; (positive-integer-p  0)   ; nil
;; (positive-integer-p -1)   ; nil

(defun string-to-number-number (str)
  (let ((s (string-trim str)))
    (when (string-match-p
           "\\`[+-]?\\([[:digit:]]+\\|[[:digit:]]*\\.[[:digit:]]+\\)\\'" s)
      (string-to-number s) )))

(defun string-to-number-number-test ()
  (cl-map 'list (lambda (e)
                  (pcase-let ((`(,a ,b) e))
                    (if (and a b)
                        (= a b)
                      (eq a b) )))
          (list
           (list (string-to-number-number " 10")                   10)
           (list (string-to-number-number "  1.5")                 1.5)
           (list (string-to-number-number " +0")                   0)
           (list (string-to-number-number "  0")                   0)
           (list (string-to-number-number " -0.0")                 -0.0)
           (list (string-to-number-number " -1.5")                -1.5)
           (list (string-to-number-number "-10")                  -10)
           (list (string-to-number-number "123this used to work")  nil)
           (list (string-to-number-number "NAN")                   nil) )))

;; (string-to-number-number-test)

(defun read-integer ()
  (let ((str)
        (str-number)
        (n) )
    (cl-loop until (progn
                     (setq str (read-string "integer: "))
                     (if (string= str "0")
                         (setq n 0)
                       (setq str-number (string-to-number str))
                       (unless (= str-number 0)
                         (setq n str-number) ))
                     (integerp n)) )
    n) )

;; (read-integer)

(defun compare (old new)
  (format "%.2f%%" (* 100 (/ (- old new)
                             (* old 1.0) ))))
;; (compare 185 80) ; 56.76%

(defun hypotenuse (c1 c2)
  (sqrt (+ (* c1 c1) (* c2 c2))) )

(defun speed (hour minute second km)
  (let*((s   (+ second (* 60 minute) (* 60 60 hour)))
        (m   (* 1000 km))
        (mps (/ m s)) )
    (* 3.6 mps) ))

;; (speed 04 44 10  42.195) ;  8.909208211143694
;; (speed 02 01 39  42.195) ; 20.811344019728732
;; (speed 19 04 00 168    ) ;  7.2

(defun binom (n k)
  (/ (cl-factorial n)
     (* (cl-factorial k)
        (cl-factorial (- n k)) )))

(defun product (l)
  (cl-loop with prod = 1
    for i in l do
    (setq prod (* prod i))
    finally return prod) )

;; (product '(1 2 3 4)) ; 24

(defun tau-ceti (speed)
  (let*((dist       (* 11.8 9.46 (expt 10 12)))
        (time       (/ dist speed) )
        (days       (/ time 24))
        (whole-days (round days)) )
    (format-seconds "%y years"
                    (- (float-time (encode-time 0 0 0 whole-days 0 0))
                       (float-time (encode-time 0 0 0 0 0 0)) ))))

;; (tau-ceti 5000) ; 2 548 584 years

(provide 'math)

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Wednesday, July 17, 2024 at 10:06 AM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Philip Kaludercic wrote:
> 
> >> Someone™ should propose a concrete suggestion.
> >
> > I've rename the thread to get some more attention, and also
> > wanted to mention, that if we do so, I think putting some
> > thought into the design wouldn't be bad. Including an
> > infix-notation macro and perhaps even overloaded operator
> > support ala APL would be a nice thing to have. We'd have to
> > weigh the cost of this kind of flexibility with performance,
> > unless someone were to write a specialised byte-compiler.
> 
> Yeah, I've pushed for a math library for a long time.
> 
> I have several files that are very close to that (for stats,
> time, permutations and so on) but I also have a math.el which
> I yank below if any of it can be used.

Tools for permutations would be of much benefit.
 
> I'd love to stay in the loop and read more on this!
> 
> ;;; -*- lexical-binding: t -*-
> ;;
> ;; this file:
> ;;   https://dataswamp.org/~incal/emacs-init/math.el
> 
> (require 'cl-lib)
> 
> (require 'dwim) ; https://dataswamp.org/~incal/emacs-init/dwim.el
> (require 'perm) ; https://dataswamp.org/~incal/emacs-init/perm.el
> 
> (defun nand (&rest conditions)
>   (when (member nil conditions)
>     t) )
> 
> ;; (nand)           ; nil
> ;; (nand 1 2 3)     ; nil
> ;; (nand nil)       ; t
> ;; (nand 1 2 nil 3) ; t
> 
> ;; (not (and))           ; nil
> ;; (not (and 1 2 3))     ; nil
> ;; (not (and nil))       ; t
> ;; (not (and 1 2 nil 3)) ; t
> 
> (defalias '**   #'expt)
> (defalias 'ceil #'ceiling)
> 
> (defun percent-change (from to)
>   (if (= from to)
>       0
>     (/ (- to from) (abs from) 0.01)))
> 
> ;; (percent-change  1  2) ;  100
> ;; (percent-change  1  1) ;    0
> ;; (percent-change  1  0) ; -100
> ;; (percent-change  1 -1) ; -200
> ;;
> ;; (percent-change  0  1) ;  1.0e+INF
> ;; (percent-change  0  0) ;  0
> ;; (percent-change  0 -1) ; -1.0e+INF
> ;;
> ;; (percent-change -1  1) ;  200
> ;; (percent-change -1  0) ;  100
> ;; (percent-change -1 -1) ;    0
> ;; (percent-change -1 -2) ; -100
> 
> (defun distance-point (min max)
>   (+ min (/ (- max min) 2.0)) )
> 
> ;; (distance-point  1   5) ; 3
> ;; (distance-point  1  10) ; 5.5
> ;; (distance-point 60 100) ; 80
> 
> (defun // (nom denom)
>   (/ nom denom 1.0) )
> 
> ;; (/  8 256) ; 0
> ;; (// 8 256) ; 0.03125
> 
> (defun arith-sum (n)
>   (cl-loop
>     with sum = 0
>     for i from 1 to (1- n)
>     do (cl-incf sum i)
>     finally return sum) )
> 
> (defun digs (n)
>   (* n (1- n)) )
> 
> (defun mean-value (vs)
>   (let*((sum  (apply #'+ vs))
>         (mean (/ sum (length vs) 1.0)) )
>     mean) )
> 
> (defun mean-str (strs)
>   (mean-value (cl-map 'list #'length strs)) )
> 
> (defun faster (old new)
>   (format "%.1f%%" (* 100 (1- (/ new old 1.0)))) )
> 
> ;; (faster 0.2  0.5)  ; 150.0%
> ;; (faster 1.35 0.84) ; -37.8%
> ;; (faster  200  400) ; 100.0%
> ;; (faster 1000 1000) ;   0.0%
> ;; (faster 1000  100) ; -90.0%
> 
> (defun percent (nom &optional denom str)
>   (or denom (setq denom 1))
>   (let ((perc (/ nom denom 0.01)))
>     (if str
>         (format "%.1f%%" perc)
>       perc)))
> 
> ;; (percent 0.25)           ; 25.0
> ;; (percent 50 75)          ; 66.66 ...
> ;; (percent (// 1 2) nil t) ; 50.0%
> ;; (percent 1019 22 t)      ; 4631.8%
> 
> (let ((min-def 0)
>       (max-def 9)
>       (inc-def 1) )
>   (defun interval (&optional min max inc)
>     (interactive `(,(read-number "min: " min-def)
>                    ,(read-number "max: " max-def)
>                    ,(read-number "inc: " inc-def)) )
>     (or min (setq min min-def))
>     (or max (setq max max-def))
>     (or inc (setq inc inc-def))
>     (unless (<= min max)
>       (error "Bogus interval") )
>     (unless (> inc 0)
>       (error "Bogus increment") )
>     (cl-loop
>       for i from min to max by inc
>       collect i) )
>   (declare-function interval nil) )
> 
> ;; (interval 10 5)        ; Bogus interval
> ;; (interval 1 3 -1)      ; Bogus increment
> ;; (interval 5 10)        ; (5 6 7 8 9 10)
> ;; (interval 1.8 2.0 0.1) ; (1.8 1.9 2.0)
> ;; (interval)             ; (0 1 2 3 4 5 6 7 8 9)
> ;; (interval 19 99)       ; (19 20 21 ... 97 98 99)
> 
> (defun digits-sum (&optional beg end)
>   (interactive (use-region))
>   (or beg (setq beg (point-min)))
>   (or end (setq end (point-max)))
>   (let ((sum 0))
>     (goto-char beg)
>     (while (re-search-forward " [[:digit:].]+ " end t)
>       (cl-incf sum (string-to-number (match-string-no-properties 0))) )
>     (prog1
>         sum
>       (message "sum: %.1f" sum) )))
> 
> (defalias 'di #'digits-sum)
> 
> (defun film-mean ()
>   (interactive)
>   (let ((scores)
>         (beg (point-min))
>         (end (point-max)) )
>     (save-excursion
>       (goto-char beg)
>       (while (re-search-forward "[[:digit:]]" end t)
>         (when (= (current-column) 73)
>           (push (string-to-number (match-string-no-properties 0)) scores) )))
>     (when scores
>       (message (format "%.3f" (mean-value scores))) )))
> 
> (defun positive-integer-p (n)
>   (and
>     (integerp n)
>     (< 0 n) ))
> 
> ;; (positive-integer-p  1.5) ; nil
> ;; (positive-integer-p  1)   ; t
> ;; (positive-integer-p  0)   ; nil
> ;; (positive-integer-p -1)   ; nil
> 
> (defun string-to-number-number (str)
>   (let ((s (string-trim str)))
>     (when (string-match-p
>            "\\`[+-]?\\([[:digit:]]+\\|[[:digit:]]*\\.[[:digit:]]+\\)\\'" s)
>       (string-to-number s) )))
> 
> (defun string-to-number-number-test ()
>   (cl-map 'list (lambda (e)
>                   (pcase-let ((`(,a ,b) e))
>                     (if (and a b)
>                         (= a b)
>                       (eq a b) )))
>           (list
>            (list (string-to-number-number " 10")                   10)
>            (list (string-to-number-number "  1.5")                 1.5)
>            (list (string-to-number-number " +0")                   0)
>            (list (string-to-number-number "  0")                   0)
>            (list (string-to-number-number " -0.0")                 -0.0)
>            (list (string-to-number-number " -1.5")                -1.5)
>            (list (string-to-number-number "-10")                  -10)
>            (list (string-to-number-number "123this used to work")  nil)
>            (list (string-to-number-number "NAN")                   nil) )))
> 
> ;; (string-to-number-number-test)
> 
> (defun read-integer ()
>   (let ((str)
>         (str-number)
>         (n) )
>     (cl-loop until (progn
>                      (setq str (read-string "integer: "))
>                      (if (string= str "0")
>                          (setq n 0)
>                        (setq str-number (string-to-number str))
>                        (unless (= str-number 0)
>                          (setq n str-number) ))
>                      (integerp n)) )
>     n) )
> 
> ;; (read-integer)
> 
> (defun compare (old new)
>   (format "%.2f%%" (* 100 (/ (- old new)
>                              (* old 1.0) ))))
> ;; (compare 185 80) ; 56.76%
> 
> (defun hypotenuse (c1 c2)
>   (sqrt (+ (* c1 c1) (* c2 c2))) )
> 
> (defun speed (hour minute second km)
>   (let*((s   (+ second (* 60 minute) (* 60 60 hour)))
>         (m   (* 1000 km))
>         (mps (/ m s)) )
>     (* 3.6 mps) ))
> 
> ;; (speed 04 44 10  42.195) ;  8.909208211143694
> ;; (speed 02 01 39  42.195) ; 20.811344019728732
> ;; (speed 19 04 00 168    ) ;  7.2
> 
> (defun binom (n k)
>   (/ (cl-factorial n)
>      (* (cl-factorial k)
>         (cl-factorial (- n k)) )))
> 
> (defun product (l)
>   (cl-loop with prod = 1
>     for i in l do
>     (setq prod (* prod i))
>     finally return prod) )
> 
> ;; (product '(1 2 3 4)) ; 24
> 
> (defun tau-ceti (speed)
>   (let*((dist       (* 11.8 9.46 (expt 10 12)))
>         (time       (/ dist speed) )
>         (days       (/ time 24))
>         (whole-days (round days)) )
>     (format-seconds "%y years"
>                     (- (float-time (encode-time 0 0 0 whole-days 0 0))
>                        (float-time (encode-time 0 0 0 0 0 0)) ))))
> 
> ;; (tau-ceti 5000) ; 2 548 584 years
> 
> (provide 'math)
> 
> -- 
> underground experts united
> https://dataswamp.org/~incal
> 
> 
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

>> Yeah, I've pushed for a math library for a long time.
>> 
>> I have several files that are very close to that (for stats,
>> time, permutations and so on) but I also have a math.el which
>> I yank below if any of it can be used.
>
> Tools for permutations would be of much benefit.

;;; -*- lexical-binding: t -*-
;;
;; this file:
;;   https://dataswamp.org/~incal/emacs-init/perm.el

(require 'cl-lib)
(require 'spell)

;; Christoph Conrad @ https://www.emacswiki.org/emacs/StringPermutations
(defun perms (l)
  (if l (cl-mapcan (lambda (a)
                     (cl-mapcan (lambda (p)
                                  (list (cons a p)))
                                (perms (cl-remove a l :count 1)) )) l)
    '(()) ))

(defun string-perms (str)
  (let*((chars      (string-to-list str))
        (char-perms (perms chars)) )
    (mapcar (lambda (a)
              (concat a) )
            char-perms) ))
;; (string-perms "hte") ; ("hte" "het" "the" "teh" "eht" "eth")

(defun string-perms-filter (str)
  (let ((strs (cl-remove-duplicates
                (cl-remove-if-not (lambda (w) (spell-word w)) (string-perms str))
                :test #'string=) ))
    (if (= 1 (length strs))
        (car strs)
      strs) ))

;; eht kudtce pigelsen tagni ogod seot erontuf si ni fo hte
;;
;; (string-perms-filter "ogod")     ; good
;; (string-perms-filter "erontuf")  ; fortune
;; (string-perms-filter "si")       ; is
;; (string-perms-filter "kudtce")   ; tucked
;; (string-perms-filter "ni")       ; in
;; (string-perms-filter "eth")      ; the
;; (string-perms-filter "seot")     ; toes
;; (string-perms-filter "fo")       ; of
;; (string-perms-filter "hte")      ; the
;; (string-perms-filter "pigelsen") ; ("peelings" "sleeping")
;; (string-perms-filter "tagni")    ; giant

;; (string-perms-filter "emacs")

(defun factorial (n)
  (if (> n 1)
      (* n (factorial (1- n)))
    1))
;; (factorial  5) ;       120
;; (factorial 10) ; 3 628 800

(defun cl-factorial (n)
  (cl-loop
    with prod = 1
    for i from 2 to n
    do (setq prod (* i prod))
    finally return prod) )
;; (cl-factorial  5) ;       120
;; (cl-factorial 10) ; 3 628 800

(defun product-string (str)
  (let*((str-list         (string-to-list str))
        (str-list-no-dups (cl-remove-duplicates str-list))
        (prod 1) )
    (dolist (e str-list-no-dups)
      (setq prod (* prod (cl-factorial (cl-count e str-list)))) )
    prod ))
;; (product-string "ogod") ; 2

(defun perms-string-num (str)
  (let ((n (cl-factorial (length str)))
        (r (product-string str)) )
    (/ n r) ))
;; (perms-string-num "ogod")   ;  12
;; (perms-string-num "kudtce") ; 720

(provide 'perm)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >  but I also have a math.el which
  > I yank below if any of it can be used.

Most of the fnctions in this library are very close to trivial.

Onsider, for instance, `hypotenuse'.  The benefit of having that
functoin would not quite be zero, but it would be small.  Is it enough
to justify the additional function name, and the complexity of
documenting it?

Perhaps there are some nontricial mathematical functions that would be
worth adding to Emacs, but that brings up the question of what
functions would be useful,  With two people expressing a desire
for a "math library", do they both wan the same thing?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Saturday, July 20, 2024 at 4:16 AM
> From: "Richard Stallman" <rms@gnu.org>
> To: "Emanuel Berg" <incal@dataswamp.org>
> Cc: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   >  but I also have a math.el which
>   > I yank below if any of it can be used.
>
> Most of the fnctions in this library are very close to trivial.

The demand for robust mathematical functions is valid and points to the need
for developing comprehensive mathematical libraries within Emacs that can support
more advanced projects.

> Onsider, for instance, `hypotenuse'.  The benefit of having that
> functoin would not quite be zero, but it would be small.  Is it enough
> to justify the additional function name, and the complexity of
> documenting it?
>
> Perhaps there are some nontricial mathematical functions that would be
> worth adding to Emacs, but that brings up the question of what
> functions would be useful,  With two people expressing a desire
> for a "math library", do they both wan the same thing?

To ensure the inclusion of genuinely useful functions, it is
essential to first identify their practical applications and
the reasons for their inclusion. Determining which functions
will be most beneficial requires a thorough examination of
specific use cases. This assessment must precede the development
of the mathematical library to achieve consensus and secure approval.

Currently, Emanuel is not going to be the one to do it, but he is
happy to offer assistance.

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> The demand for robust mathematical functions is valid and
> points to the need for developing comprehensive mathematical
> libraries within Emacs that can support more
> advanced projects.

There are a lot basic stuff we need first. The "trivial"
stuff, in your phrasing. But they will be used in advanced
projects as well by all means.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Max Nikulin]

On 19/07/2024 23:16, Richard Stallman wrote:
> Onsider, for instance, `hypotenuse'.  The benefit of having that
> functoin would not quite be zero, but it would be small.  Is it enough
> to justify the additional function name, and the complexity of
> documenting it?

As usual, when implementation is replaced with a better one, all callers 
may benefit with no efforts on their side.

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Sunday, July 21, 2024 at 12:45 AM
> From: "Max Nikulin" <manikulin@gmail.com>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> On 19/07/2024 23:16, Richard Stallman wrote:
> > Onsider, for instance, `hypotenuse'.  The benefit of having that
> > functoin would not quite be zero, but it would be small.  Is it enough
> > to justify the additional function name, and the complexity of
> > documenting it?
>
> As usual, when implementation is replaced with a better one, all callers
> may benefit with no efforts on their side.

That function can be used as a meaningful starting point to enable adoption
and feedback.   Providing the necessary infrastructure that can be easily
scaled.

Re: Adding a generic mathematical library

[Emanuel Berg]

Richard Stallman wrote:

>> but I also have a math.el which I yank below if any of it
>> can be used.
>
> Most of the fnctions in this library are very close
> to trivial.

Oh, my! Eli thinks they are so exotic only I use them, RMS
thinks they are close to trivial.

But I have many more files, I'll see what I have that can
maybe impress you and Eli more.

I have already showed you math.el and perm.el, here are two
more. But I'll look thru it all some other day and make
a directory with all of them.

;;; -*- lexical-binding: t -*-
;;
;; this file:
;;   https://dataswamp.org/~incal/emacs-init/stats.el

(require 'cl-lib)

(defun decade ()
  (interactive)
  (cl-loop
    with entries = ()
    for d from 190 to 202
    do (push (how-many (format "^.\\{14\\}%d[[:digit:]]" d)) entries)
    finally (message "%s" (nreverse entries)) ))

(defun random-distribution (n sum &optional min no-error)
  "Return a list of N random numbers.
\nThe sum of the numbers is SUM.
\nEvery number is at least MIN. (default: 0)
\nIf NO-ERROR, recompute unsolvable systems with MIN = 0. (default: nil)"
  (or min (setq min 0))
  (let ((min-share (* n min)))
    (if (< sum min-share)
        (if no-error
            (random-distribution n sum)
          (error "Bogus indata, unsolvable") )
      (let ((vals (make-list n min))
            (pool (- sum min-share)) )
        (cl-loop while (< 0 pool)
                 do (cl-incf (nth (random n) vals))
                    (cl-decf pool) )
        vals) )))

(defalias 'rd #'random-distribution)

;; (apply #'+ (rd 10 100 3)) ; sum is 100, OK
;; (rd 10 100)               ; omitted/default min = 0, OK
;; (rd 10 100 10)            ; border case, but OK
;; (rd 10 100 20)            ; error: Bogus indata, unsolvable
;; (rd 10 100 20 t)          ; unsolvable but no-error, recompute for min = 0
;;
;; 10 example distributions for
;; n = 10, sum = 100, and min = 3,
;; i.e. (rd 10 100 3):
;;
;;   (12  8  8 10  7  6 19  9  7 14)
;;   ( 8 12  7 12  9 10  9  9 12 12)
;;   (14  6 15 11  9  9  6  8 11 11)
;;   ( 8 12  7  9 10  9 10 13 10 12)
;;   (12 12 15  6 10  8  6 10 14  7)
;;   ( 9  7 10  8 11 10  9 10 13 13)
;;   ( 7 13 13 10 13 10  5 13  5 11)
;;   (13  8  6  9 13  8 11  7 10 15)
;;   ( 7 10  5  6 17 14  8 10 14  9)
;;   (11 10  8  9 11 10 14 11  8  8)

(provide 'stats)

;;; -*- lexical-binding: t -*-
;;
;; help from:
;;   https://codereview.stackexchange.com/q/186840
;;
;; this file:
;;   https://dataswamp.org/~incal/emacs-init/str-to-bits.el

(require 'esh-util)

(defun num-to-bits (num)
  (let ((s ""))
    (while (> num 0)
      (setq s (concat (number-to-string (logand num 1)) s))
      (setq num (ash num -1)) )
    (if (string= "" s)
        "0"
      s) ))

(defalias 'dec2bits  #'num-to-bits)
(defalias 'char2bits #'num-to-bits)

(defun str-to-bits (str &optional delim)
  (interactive "sString: ")
  (or delim (setq delim " "))
  (let ((bits))
    (dolist (c (string-to-list str) (mapconcat #'eshell-stringify bits delim))
      (push (num-to-bits c) bits) )))

;; (str-to-bits "What's up Emacs community?") ; 111111 1110000 1110101 [...]

(provide 'str-to-bits)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

>> Most of the functions in this library are very close to
>> trivial. [...]
>
> I have already showed you math.el and perm.el, here are two
> more. But I'll look thru it all some other day and make
> a directory with all of them.

No, but let's do this thoroughly then.

I will look at this situation now in another way, let's calm
down again, maybe we can do something useful still.

It will be really interesting to hear your comments - no irony
att all. I'm ready.

I'll do the directory later, here is just a list with URLs.
It is the first 12 files a found but there is more. At least
one is a proper package while other are very short files.

I'll post the rest some other day, God willing.

Okay, here goes!

Computation packages, files, and projects. PART 1, 12 files.

* random-urandom.c
  https://dataswamp.org/~incal/emacs-init/random-urandom/random-urandom.c

  Dynamic module (C language) to get to Linux urandom which is
  more random than `random', so you can use that from Elisp
  as well.

* all-substance.el
  https://dataswamp.org/~incal/emacs-init/all-substance.el

  Substance reduction (half-time) over time.

* audio.el
  https://dataswamp.org/~incal/emacs-init/audio.el

  Audio db multiplier.

* beer.el
  https://dataswamp.org/~incal/emacs-init/beer.el

  Convert between beer types, e.g. how many 5.3% beers are one
  six-pack 3.5% plus one cider?

* bike.el
  https://dataswamp.org/~incal/emacs-init/bike.el

  Compute the step and gear (roll out).

* color-incal.el
  https://dataswamp.org/~incal/emacs-init/color-incal.el

  Computations on colors, determines "is this color blue?"
  with some method for that.

* dice.el
  https://dataswamp.org/~incal/emacs-init/dice.el

  Can do 1D6, 2D3 etc.

* eggs.el
  https://dataswamp.org/~incal/emacs-init/eggs.el

  Solves an egg math problem.

* epwgen.el
  https://dataswamp.org/~incal/emacs-init/epwgen.el

  Compute password strength.

* frame-size.el
  https://dataswamp.org/~incal/emacs-init/frame-size.el

  More computations on the bicycle, this time the frame.

* hex.el
  https://dataswamp.org/~incal/emacs-init/hex.el

  Decodes a hex string.

* isbn-verify.el
  https://dataswamp.org/~incal/emacs-init/isbn-verify.el

  Verify ISBNs with the check digit and a known algorithm.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Here is the entire list. 28 files. Dig in.

I'll make a directory and unified list later, maybe. In the
meantime, well you see in what directory the files are, if
you'd like to browse them. That will include other files
for now.

MATH

Second part, 16 files.

Last: the first part, 12 files. Total 28 files.

* issn-verify.el
  https://dataswamp.org/~incal/emacs-init/issn-verify.el

Verify ISSN numbers.

* lights.el
  https://dataswamp.org/~incal/emacs-init/lights.el

Compute LED luminous efficacy (lm/W).

* list.el
  https://dataswamp.org/~incal/emacs-init/list.el

Stats on lists.

* math.el
  https://dataswamp.org/~incal/emacs-init/math.el

The infamous math file, critized by many for as many different
reasons. Fairly so? Maybe! It has no less than 23
`defun' anyway.

* negative-subtraction.el
  https://dataswamp.org/~incal/emacs-init/negative-subtraction.el

Subtraction digit by digit.

* perm.el
  https://dataswamp.org/~incal/emacs-init/perm.el

Permutations. Known from the math field combinatorics but also
stats and actually many science areas.

Bonus: this file solves a fun word problem from a TV show.

* psea.el
  https://dataswamp.org/~incal/emacs-init/psea.el
  https://dataswamp.org/~incal/emacs-init/b-a-original.el

Has to do with string distance and then stats and tables on
that. Yes, I tried that two times it seems! In b-a-original.el
you can output this kind of table (below) saying the first
line is 81% original - so scores a number 1 position at
407 originality points. Based on a weighted sum.

  1. (require 'pcase)      81% 407
  2. (require 'cl-lib)     80% 402
  3. (require 'psea)       80% 402
  4. (require 'subr-x)     80% 402
  5. (require 'thingatpt)  77% 387

* random-generic.el (random.el)
  https://dataswamp.org/~incal/emacs-init/random-generic.el
  https://dataswamp.org/~incal/emacs-init/random.el

Randomizer for arbitrary data types, so you can feed whatever
into it and mix it up transparently. Plus some misc random
stuff, looks like it is more practical in nature but
randomness is a part of math (and stats, and CS, and ...) so
I include it.

* scale.el
  https://dataswamp.org/~incal/emacs-init/scale.el

Draw a scale.

[ Sorting we don't consider math, right? But CS? Cool, not
  included. ]

* stats.el
  https://dataswamp.org/~incal/emacs-init/stats.el

I'm not sure what this does, it outputs some digit set with
some constrains? I'll look it up, looks interesting!

* str-to-bits.el
  https://dataswamp.org/~incal/emacs-init/str-to-bits.el

(str-to-bits "What's up Emacs community?")
 111111 1111001 1110100 1101001 1101110 1110101 1101101 1101101
1101111 1100011  100000 1110011 1100011 1100001 1101101 1000101
 100000 1110000 1110101  100000 1110011  100111 1110100 1100001
1101000 1010111

* survivor.el
  https://dataswamp.org/~incal/emacs-init/survivor.el

[ OT: read if you are from Australia or Sweden. Maybe the US ]

This is computation on the TV show Survivor (or
Expedition Robinson as the English original idea was called on
Swedish TV where it was first realized). Computation is based
on the idea that it gets more difficult all the time but not
linearly so but much worse. So, the difficulty at day 48 in
Australian Survivor is computed to 1176 (compare for example
US Survivor at only 27 days and scores a mere 378). Is
AUS Survivor the world's most difficult reality TV show? Well,
the numbers don't lie ;) Congrats to AUS people reading this,
you are a tough bunch - in a good way - and thanks for that
awsome show, the very best national instance ever of the show!

Here is the full list:

;; (cost-at-day 48) ; 1176 AUS
;; (cost-at-day 39) ;  780 USA old era
;; (cost-at-day 28) ;  406 SWE
;; (cost-at-day 27) ;  378 USA new era
;; (cost-at-day  1) ;    1
;; (cost-at-day  0) ;    0

* time-cmp.el
  https://dataswamp.org/~incal/emacs-init/time-cmp.el

Computation on time. Very useful.

* variance.el
  https://dataswamp.org/~incal/emacs-init/variance.el

This has to do with stats, one is to parse a file with data.
Very old but looks good anyway.

* wood.el
  https://dataswamp.org/~incal/emacs-init/wood.el

How to cut wood and build with wood. Several ideas are
implemented here and I have tried them in practice and they
work there as well (especially there). See photo linked
from file.

also:

* distance.el
  https://dataswamp.org/~incal/distance/distance.el

Calculate the distance between two Earth locations using the
Haversine formula (yes, the globe is very much so round enough
for this to work, alltho it is math and geometry method based
on a perfect sphere).

Probably the program is br0ken now since Wikipedia should have
changed their UI and/or API many times since that program
written also ages ago. But the math I hope is still OK.

MATH

First part, 12 files.

* random-urandom.c
  https://dataswamp.org/~incal/emacs-init/random-urandom/random-urandom.c

  Dynamic module to get to Linux urandom which is more random
  than `random', so you can use that from Elisp as well.

* all-substance.el
  https://dataswamp.org/~incal/emacs-init/all-substance.el

  Substance reduction (half-time).

* audio.el
  https://dataswamp.org/~incal/emacs-init/audio.el

  db multiplier.

* beer.el
  https://dataswamp.org/~incal/emacs-init/beer.el

  Convert between beer types, e.g. how many 5.3% beer are one
  six-pack 3.5%?

* bike.el
  https://dataswamp.org/~incal/emacs-init/bike.el

  Compute the step and gear (roll out).

* color-incal.el
  https://dataswamp.org/~incal/emacs-init/color-incal.el

  Computations on colors, determines "is a color blue"?

* dice.el
  https://dataswamp.org/~incal/emacs-init/dice.el

  Can do 1D6 but also 2D3.

* eggs.el
  https://dataswamp.org/~incal/emacs-init/eggs.el

  Solves the egg math problem.

* epwgen.el
  https://dataswamp.org/~incal/emacs-init/epwgen.el

  How strong is a password.

* frame-size.el
  https://dataswamp.org/~incal/emacs-init/frame-size.el

  More computations on the bicycle, this time the frame.

* hex.el
  https://dataswamp.org/~incal/emacs-init/hex.el

  Decodes a hex string.

* isbn-verify.el
  https://dataswamp.org/~incal/emacs-init/isbn-verify.el

  Verify ISBNs with the check digit and known algorithm.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

If we do this, I think we should do it basic first.

No new representation or new infix notation or anything like
that, nothing too fancy, just implement a bunch of stuff - but
in a systematic way, of course.

All will have to be discussed of course but as for me, I don't
see it as too advanced from the get-go, not in terms of math
and absolutely not in terms of Elisp.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

OK, here the complete file, 28 files, URL to where the text
file is stored, with all references to antagonism removed.

Okay, not a lot more to say. Keep it real and be rational on
the number line, time to study the integers for me
namely ZZZ.

-------------------------------------------------------------------------------
  https://dataswamp.org/~incal/emacs-init/docs/math.txt            Emacs Lisp
-------------------------------------------------------------------------------

MATH - 28 files.

* random-urandom.c
  https://dataswamp.org/~incal/emacs-init/random-urandom/random-urandom.c

  Dynamic module (C language) to get to Linux urandom which is
  more random than `random', so you can use that from Elisp
  as well.

* all-substance.el
  https://dataswamp.org/~incal/emacs-init/all-substance.el

  Substance reduction (half-time) over time.

* audio.el
  https://dataswamp.org/~incal/emacs-init/audio.el

  Audio db multiplier.

* beer.el
  https://dataswamp.org/~incal/emacs-init/beer.el

  Convert between beer types, e.g. how many 5.3% beers are one
  six-pack 3.5% plus one cider of this volume etc?

* bike.el
  https://dataswamp.org/~incal/emacs-init/bike.el

  Compute the step and gear (roll out).

* color-incal.el
  https://dataswamp.org/~incal/emacs-init/color-incal.el

  Computations on colors, determines "is this color blue?"
  with a method for that.

* dice.el
  https://dataswamp.org/~incal/emacs-init/dice.el

  Can do 1D6, 2D3 etc.

* eggs.el
  https://dataswamp.org/~incal/emacs-init/eggs.el

  Solves an egg math problem.

* epwgen.el
  https://dataswamp.org/~incal/emacs-init/epwgen.el

  Compute password strength.

* frame-size.el
  https://dataswamp.org/~incal/emacs-init/frame-size.el

  More computations on the bicycle, this time the frame.

* hex.el
  https://dataswamp.org/~incal/emacs-init/hex.el

  Decodes a hex string.

* isbn-verify.el
  https://dataswamp.org/~incal/emacs-init/isbn-verify.el

  Verify ISBNs with the check digit and a known algorithm.

* issn-verify.el
  https://dataswamp.org/~incal/emacs-init/issn-verify.el

Verify ISSN numbers.

* lights.el
  https://dataswamp.org/~incal/emacs-init/lights.el

Compute LED luminous efficacy (lm/W).

* list.el
  https://dataswamp.org/~incal/emacs-init/list.el

Stats on lists.

* math.el
  https://dataswamp.org/~incal/emacs-init/math.el

The infamous math file. 23 `defun'.

* negative-subtraction.el
  https://dataswamp.org/~incal/emacs-init/negative-subtraction.el

Subtraction digit by digit.

* perm.el
  https://dataswamp.org/~incal/emacs-init/perm.el

Permutations. Known from the math field combinatorics but also
stats and actually many science areas.
Bonus: this file solves a fun word problem from a TV show.

* psea.el
  https://dataswamp.org/~incal/emacs-init/psea.el
  https://dataswamp.org/~incal/emacs-init/b-a-original.el

Has to do with string distance and then stats and tables on
that. Yes, I tried that two times it seems! In b-a-original.el
you can output this kind of table (below) saying the first
line is 81% original - so scores a number 1 position at
407 originality points. Based on a weighted sum.

  1. (require 'pcase)      81% 407
  2. (require 'cl-lib)     80% 402
  3. (require 'psea)       80% 402
  4. (require 'subr-x)     80% 402
  5. (require 'thingatpt)  77% 387

* random-generic.el (random.el)
  https://dataswamp.org/~incal/emacs-init/random-generic.el
  https://dataswamp.org/~incal/emacs-init/random.el

Randomizer for arbitrary data types, so you can feed whatever
into it and mix it up transparently. Plus some misc random
stuff, looks like it is more practical in nature but
randomness is a part of math (and stats, and CS, and ...) so
I include it.

* scale.el
  https://dataswamp.org/~incal/emacs-init/scale.el

Draw a scale.

[ Sorting we don't consider math, right? But CS? OK, not included. ]

* stats.el
  https://dataswamp.org/~incal/emacs-init/stats.el

I'm not sure what this does, it outputs some digit set with
some constrains? I'll look it up, looks interesting! TODO

* str-to-bits.el
  https://dataswamp.org/~incal/emacs-init/str-to-bits.el

(str-to-bits "What's up Emacs community?")
 111111 1111001 1110100 1101001 1101110 1110101 1101101 1101101
1101111 1100011  100000 1110011 1100011 1100001 1101101 1000101
 100000 1110000 1110101  100000 1110011  100111 1110100 1100001
1101000 1010111

* survivor.el
  https://dataswamp.org/~incal/emacs-init/survivor.el

[ OT: read if you are from Australia or Sweden. Maybe the US ]

This is computation on the TV show Survivor (or
Expedition Robinson as the English original idea was called on
Swedish TV where it was first realized). Computation is based
on the idea that it gets more difficult all the time but not
linearly so but much worse. So, the difficulty at day 48 in
Australian Survivor is computed to 1176 (compare for example
US Survivor at only 27 days and scores a mere 378). Is
AUS Survivor the world's most difficult reality TV show? Well,
the numbers don't lie ;) Congrats to AUS people reading this,
you are a tough bunch - in a good way - and thanks for that
awsome show, the very best national instance ever of the show.

Here is the full list:

;; (cost-at-day 48) ; 1176 AUS
;; (cost-at-day 39) ;  780 USA old era
;; (cost-at-day 28) ;  406 SWE
;; (cost-at-day 27) ;  378 USA new era
;; (cost-at-day  1) ;    1
;; (cost-at-day  0) ;    0

* time-cmp.el
  https://dataswamp.org/~incal/emacs-init/time-cmp.el

Computation on time. Very useful.

* variance.el
  https://dataswamp.org/~incal/emacs-init/variance.el

This has to do with stats, one is to parse a file with data.
Very old but looks good anyway.

* wood.el
  https://dataswamp.org/~incal/emacs-init/wood.el

How to cut wood and build with wood. Several ideas are
implemented here and I have tried them in practice and they
work there as well (or especially there). See photo linked
from file.

also:

* distance.el
  https://dataswamp.org/~incal/distance/distance.el

Calculate the distance between two Earth locations using the
Haversine formula (yes, the globe is very much so round enough
for this to work, alltho it is math and geometry method based
on the perfect sphere abstract concept).

Probably the program is br0ken now since Wikipedia should have
changed their UI and/or API many times since that program was
written, also ages ago. But the math is still OK.

-------------------------------------------------------------------------------
  incal@dataswamp.org                                              2024-07-21
-------------------------------------------------------------------------------

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Sunday, July 21, 2024 at 5:19 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Richard Stallman wrote:
>
> >> but I also have a math.el which I yank below if any of it
> >> can be used.
> >
> > Most of the fnctions in this library are very close
> > to trivial.
>
> Oh, my! Eli thinks they are so exotic only I use them, RMS
> thinks they are close to trivial.
>
> But I have many more files, I'll see what I have that can
> maybe impress you and Eli more.

We should ask the community what tools they would like to use,
whether combinatorics, random numbers (Gaussian, Bernoulli, Binomial), ...

May I suggest "Units and Conversions".  "Symbolic Simplification"
could also be a candidate.

To promote a mathematical library in Emacs and ensure its broad adoption,
we can think of some things outside the realm of your libraries but which
others would appreciate.

> I have already showed you math.el and perm.el, here are two
> more. But I'll look thru it all some other day and make
> a directory with all of them.
>
> ;;; -*- lexical-binding: t -*-
> ;;
> ;; this file:
> ;;   https://dataswamp.org/~incal/emacs-init/stats.el
>
> (require 'cl-lib)
>
> (defun decade ()
>   (interactive)
>   (cl-loop
>     with entries = ()
>     for d from 190 to 202
>     do (push (how-many (format "^.\\{14\\}%d[[:digit:]]" d)) entries)
>     finally (message "%s" (nreverse entries)) ))
>
> (defun random-distribution (n sum &optional min no-error)
>   "Return a list of N random numbers.
> \nThe sum of the numbers is SUM.
> \nEvery number is at least MIN. (default: 0)
> \nIf NO-ERROR, recompute unsolvable systems with MIN = 0. (default: nil)"
>   (or min (setq min 0))
>   (let ((min-share (* n min)))
>     (if (< sum min-share)
>         (if no-error
>             (random-distribution n sum)
>           (error "Bogus indata, unsolvable") )
>       (let ((vals (make-list n min))
>             (pool (- sum min-share)) )
>         (cl-loop while (< 0 pool)
>                  do (cl-incf (nth (random n) vals))
>                     (cl-decf pool) )
>         vals) )))
>
> (defalias 'rd #'random-distribution)
>
> ;; (apply #'+ (rd 10 100 3)) ; sum is 100, OK
> ;; (rd 10 100)               ; omitted/default min = 0, OK
> ;; (rd 10 100 10)            ; border case, but OK
> ;; (rd 10 100 20)            ; error: Bogus indata, unsolvable
> ;; (rd 10 100 20 t)          ; unsolvable but no-error, recompute for min = 0
> ;;
> ;; 10 example distributions for
> ;; n = 10, sum = 100, and min = 3,
> ;; i.e. (rd 10 100 3):
> ;;
> ;;   (12  8  8 10  7  6 19  9  7 14)
> ;;   ( 8 12  7 12  9 10  9  9 12 12)
> ;;   (14  6 15 11  9  9  6  8 11 11)
> ;;   ( 8 12  7  9 10  9 10 13 10 12)
> ;;   (12 12 15  6 10  8  6 10 14  7)
> ;;   ( 9  7 10  8 11 10  9 10 13 13)
> ;;   ( 7 13 13 10 13 10  5 13  5 11)
> ;;   (13  8  6  9 13  8 11  7 10 15)
> ;;   ( 7 10  5  6 17 14  8 10 14  9)
> ;;   (11 10  8  9 11 10 14 11  8  8)
>
> (provide 'stats)
>
> ;;; -*- lexical-binding: t -*-
> ;;
> ;; help from:
> ;;   https://codereview.stackexchange.com/q/186840
> ;;
> ;; this file:
> ;;   https://dataswamp.org/~incal/emacs-init/str-to-bits.el
>
> (require 'esh-util)
>
> (defun num-to-bits (num)
>   (let ((s ""))
>     (while (> num 0)
>       (setq s (concat (number-to-string (logand num 1)) s))
>       (setq num (ash num -1)) )
>     (if (string= "" s)
>         "0"
>       s) ))
>
> (defalias 'dec2bits  #'num-to-bits)
> (defalias 'char2bits #'num-to-bits)
>
> (defun str-to-bits (str &optional delim)
>   (interactive "sString: ")
>   (or delim (setq delim " "))
>   (let ((bits))
>     (dolist (c (string-to-list str) (mapconcat #'eshell-stringify bits delim))
>       (push (num-to-bits c) bits) )))
>
> ;; (str-to-bits "What's up Emacs community?") ; 111111 1110000 1110101 [...]
>
> (provide 'str-to-bits)
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> We should ask the community what tools they would like to
> use, whether combinatorics, random numbers (Gaussian,
> Bernoulli, Binomial), ...
>
> May I suggest "Units and Conversions". "Symbolic
> Simplification" could also be a candidate.
>
> To promote a mathematical library in Emacs and ensure its
> broad adoption, we can think of some things outside the
> realm of your libraries but which others would appreciate.

Yes, I'll see what we can find and reuse. They are not really
complete libraries, even those who sound like they are - time,
random, permutations etc - they are just files where I put
everything that has to do with it.

It is not a bad method, but does not intend to create complete
libraries. None are and as for the whole math field,
that's huge, not covered by my files, haha (impossible).

I'd like to join this project formally now, after so many
times suggesting it and now doing this as well.

But for sure, I won't be able to do it alone and also I don't
have _that_ strong background in math. I did complete a bunch
of university courses, but yeah, how much did I understand,
truly? It feels like: to some degree that matters, sure, and
to a practical degree, fine, but not to that HUGE
a degree, really.

I know one person, from Poland, but he is probably busy.
And I know one person from Russia, maybe he is reading this?
Anyone else, join - if we get to be a bunch of guys it will
not be that difficult and big a project. Or rather, all it
will take is time.

Yes, you Mr Dimech, join as well. Why not?

I'll end this by sharing this file. Does the ASCII look good?
(If not, follow the URL.)

If you want to evaluate, first do (defalias '** #'expt)

-------------------------------------------------------------------------------
  BINARY UNITS                                            incal@dataswamp.org
-------------------------------------------------------------------------------




        name               bytes   unit   max value
-------------------------------------------------------------------------------
   char/byte                   1          (1- (** 2 (* 8     (** 2  0)   )))
        word                   2          (1- (** 2 (* 8     (** 2  1)   )))
  doubleword                   4          (1- (** 2 (* 8     (** 2  2)   )))
    quadword                   8          (1- (** 2 (* 8     (** 2  3)   )))
   paragraph                  16          (1- (** 2 (* 8     (** 2  4)   )))
    kilobyte               1 024   KiB    (1- (** 2 (* 8 (** (** 2 10) 1))))
    megabyte           1 048 576   MiB    (1- (** 2 (* 8 (** (** 2 10) 2))))
    gigabyte       1 073 741 824   GiB    (1- (** 2 (* 8 (** (** 2 10) 3))))
    terabyte   1 099 511 627 776   TiB    (1- (** 2 (* 8 (** (** 2 10) 4))))
-------------------------------------------------------------------------------
                                                                 ^^^^
                                                                 1024



-------------------------------------------------------------------------------
  https://dataswamp.org/~incal/data/BINARY-UNITS                   2022-12-01
-------------------------------------------------------------------------------

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Sunday, July 21, 2024 at 8:03 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech wrote:
>
> > We should ask the community what tools they would like to
> > use, whether combinatorics, random numbers (Gaussian,
> > Bernoulli, Binomial), ...
> >
> > May I suggest "Units and Conversions". "Symbolic
> > Simplification" could also be a candidate.
> >
> > To promote a mathematical library in Emacs and ensure its
> > broad adoption, we can think of some things outside the
> > realm of your libraries but which others would appreciate.
>
> Yes, I'll see what we can find and reuse. They are not really
> complete libraries, even those who sound like they are - time,
> random, permutations etc - they are just files where I put
> everything that has to do with it.
>
> It is not a bad method, but does not intend to create complete
> libraries. None are and as for the whole math field,
> that's huge, not covered by my files, haha (impossible).
>
> I'd like to join this project formally now, after so many
> times suggesting it and now doing this as well.
>
> But for sure, I won't be able to do it alone and also I don't
> have _that_ strong background in math. I did complete a bunch
> of university courses, but yeah, how much did I understand,
> truly? It feels like: to some degree that matters, sure, and
> to a practical degree, fine, but not to that HUGE
> a degree, really.

There are among us with extremely strong mathematical background.

Could you list for us a categorisation of the five main areas your
libraries focus on.  Please use the standard classification
from https://msc2020.org/MSC_2020.pdf

Example List

05 Combinatorics
51 Geometry
60 Probability
62 Statistics

You could focus upon devising the library framework.  Although your
mathematical background might not be strong, your elisp should be.

I will focus on what can be included for a first release.  But I need
approval from at least two others (e.g. Eli, Richard, Stefan, Michael).

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> There are among us with extremely strong
> mathematical background.

Yes, I know, they are welcome to join us :)

> Could you list for us a categorisation of the five main
> areas your libraries focus on. Please use the standard
> classification from
> https://msc2020.org/MSC_2020.pdf
>
> Example List
>
> 05 Combinatorics
> 51 Geometry
> 60 Probability
> 62 Statistics
>
> You could focus upon devising the library framework.
> Although your mathematical background might not be strong,
> your elisp should be.

There is no pulling rank, everyone is one the same level.
But I can show you my CV, sure. It is no secret and I have no
problem with it.

But yes, I already have a package at ELPA so I know some of
the process anyway!

This will be a consensus based project. Conflicts are not
allowed, public dismissiveness of people efforts or
backgrounds isn't either, the worst thing anyone is allowed to
say is "we need to work more on this".

Before we think about who does what, we have to agree what to
do first in pretty good details.

> I will focus on what can be included for a first release.
> But I need approval from at least two others (e.g. Eli,
> Richard, Stefan, Michael).

Stefan and Michael <3

Can't you guys join?

As long as there it is consensus based group and no conflicts
allowed, no dismissive or elitist attitude, no web cop, no
leadership group, none of that. That only attracts people who
are into power and status, I know this from many, many places.
So you just have to say we don't do that, and they don't come.
I know they won't, this method works.

Np: As soon as there is confusion, we stop and we figure
it out.

If this is a project for you, let's do it :)

But let's see if someone more would like to join.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

> This will be a consensus based project. Conflicts are not
> allowed, public dismissiveness of people efforts or
> backgrounds isn't either, the worst thing anyone is allowed
> to say is "we need to work more on this".

If people really, really insist on one guy in charge then
I'm not 100% against it but then ... find someone we are sure
of isn't in it for the power at all.

Some nerdy/intellectual guy who has been active with his stuff
and been talking on the MLs optimally so we know him.

Because if he does "meaningless things", that only he (or she)
is into, then we know, okay, he is into that, he didn't come
here for any other reason.

Because people who are into power - avoid, avoid, avoid.
And they are much more common than I would have ever thought!
And they are everywhere, literally, including places where you
would be like "who would ever, ever care one bit who is in
charge HERE". But not only do these people care, that was the
reason they came. Yes, it is scary.

Well, I'm sure you guys solve this in a good way.

Ah, man, I'm too tired, that is why I write so long about
everything. See you ... zzz

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Christopher Dimech]

> Sent: Sunday, July 21, 2024 at 11:20 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> > This will be a consensus based project. Conflicts are not
> > allowed, public dismissiveness of people efforts or
> > backgrounds isn't either, the worst thing anyone is allowed
> > to say is "we need to work more on this".
>
> If people really, really insist on one guy in charge then
> I'm not 100% against it but then ... find someone we are sure
> of isn't in it for the power at all.
>
> Some nerdy/intellectual guy who has been active with his stuff
> and been talking on the MLs optimally so we know him.

All depends on who wants to produce a first release that handles
two initial mathematical classifications, and which helps others
extend the library to other additional topic classification.

The choice would be of someone who would actually get an agreed
first release out.

> Because if he does "meaningless things", that only he (or she)
> is into, then we know, okay, he is into that, he didn't come
> here for any other reason.
>
> Because people who are into power - avoid, avoid, avoid.
> And they are much more common than I would have ever thought!
> And they are everywhere, literally, including places where you
> would be like "who would ever, ever care one bit who is in
> charge HERE". But not only do these people care, that was the
> reason they came. Yes, it is scary.
>
> Well, I'm sure you guys solve this in a good way.
>
> Ah, man, I'm too tired, that is why I write so long about
> everything. See you ... zzz
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> All depends on who wants to produce a first release that
> handles two initial mathematical classifications, and which
> helps others extend the library to other additional
> topic classification.

Yes, geometry and arithmetic, right?

Let's start with arithmetic then. We make a plan what we want,
what we already have, obviously we don't do, and we do
the rest.

> The choice would be of someone who would actually get an
> agreed first release out.

What parts of arithmetics would you like to have? You are the
math guy then, if that is the way you like it. You can tell me
what functions and operators are missing - maybe definitions,
actually whatever. Most should be possible to put into code
quite easily, I dare say.

Let's start coding :)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

> What parts of arithmetics would you like to have? You are
> the math guy then, if that is the way you like it. You can
> tell me what functions and operators are missing - maybe
> definitions, actually whatever.

And applications, like formulas and math functions.

This -

(defun distance-point (min max)
  (+ min (/ (- max min) 2.0)) )

- or special, fun things. This even has the name arithmetic in
it. Well, almost.

(defun arith-sum (n)
  (cl-loop
    with sum = 0
    for i from 1 to (1- n)
    do (cl-incf sum i)
    finally return sum) )

(Hm - there are many ways to solve that, and this is fine by
all means.)

You can also compare Emacs arithmetic capabilities with what
you find in a book. Do we have everything? Or do we want more?
That is where it begins.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Made it a little more easy to see.

Ah, I think the only straight thing is find a textbook from
the bookshelf or get one on the black market and see what
stuff they focus on. And then think if we do the same or think
differently case-by-case.

With these building blocks you can do a lot and we have a lot
already in core Emacs already and in ELPA for that matter.

Right, people, send me e-mails if this isn't a good place to
discuss this back and forth :)

Arithmetic

(+ &rest numbers)
(- &rest numbers)
(* &rest numbers)
(/ number &rest divisors)
(% x y)
(mod x y)
(1+ number)
(1- number)

Predicates

(= number &rest numbers)
(eql obj1 obj2)
(/= num1 num2)
(< number &rest numbers)
(<= number &rest numbers)
(> number &rest numbers)
(>= number &rest numbers)
(zerop number)
(natnump object)
(cl-plusp number)
(cl-minusp number)
(cl-oddp integer)
(cl-evenp integer)
(bignump object)
(fixnump object)
(floatp object)
(integerp object)
(numberp object)
(cl-digit-char-p char &optional radix)

Operations

(max number &rest numbers)
(min number &rest numbers)
(abs arg)
(float arg)
(truncate arg &optional divisor)
(floor arg &optional divisor)
(ceiling arg &optional divisor)
(round arg &optional divisor)
(random &optional limit)

Bit Operations

(ash value count)
(logand &rest ints-or-markers)
(logior &rest ints-or-markers)
(logxor &rest ints-or-markers)
(lognot number)
(logcount value)

Floating Point

(isnan x)
(frexp x)
(ldexp sgnfcand exponent)
(logb arg)
(ffloor arg)
(fceiling arg)
(ftruncate arg)
(fround arg)

Standard Math Functions

(sin arg)
(cos arg)
(tan arg)
(asin arg)
(acos arg)
(atan y &optional x)
(exp arg)
(log arg &optional base)
(expt arg1 arg2)
(sqrt arg)

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

Emanuel, looking at calc.el you will find

;;   Provide a better implementation for math-sin-cos-raw.
;;   Provide a better implementation for math-hypot.

You can focus on these two things so you can provide us with better
implementations of those two things.  Together with the necessary
framework for extension and for use by calc.

> Sent: Monday, July 22, 2024 at 12:27 AM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> > What parts of arithmetics would you like to have? You are
> > the math guy then, if that is the way you like it. You can
> > tell me what functions and operators are missing - maybe
> > definitions, actually whatever.
>
> And applications, like formulas and math functions.
>
> This -
>
> (defun distance-point (min max)
>   (+ min (/ (- max min) 2.0)) )
>
> - or special, fun things. This even has the name arithmetic in
> it. Well, almost.
>
> (defun arith-sum (n)
>   (cl-loop
>     with sum = 0
>     for i from 1 to (1- n)
>     do (cl-incf sum i)
>     finally return sum) )
>
> (Hm - there are many ways to solve that, and this is fine by
> all means.)
>
> You can also compare Emacs arithmetic capabilities with what
> you find in a book. Do we have everything? Or do we want more?
> That is where it begins.
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> Provide a better implementation for math-sin-cos-raw
> Provide a better implementation for math-hypot

I'll get right to it!

If just Eli and Richard could spend a couple of hours
explaining what improvements are called for?

So there won't be any suddenly divergent ideas, I mean?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Sun, 21 Jul 2024 15:17:01 +0200
> 
> Christopher Dimech wrote:
> 
> > Provide a better implementation for math-sin-cos-raw
> > Provide a better implementation for math-hypot
> 
> I'll get right to it!
> 
> If just Eli and Richard could spend a couple of hours
> explaining what improvements are called for?

I don't know what math-sin-cos-raw is about, but as for math-hypot,
there's no need to implement anything, since this function is standard
in any C libm, and we can easily expose it to Lisp if we need to.

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 2:33 AM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: "Emanuel Berg" <incal@dataswamp.org>
> Cc: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> > From: Emanuel Berg <incal@dataswamp.org>
> > Date: Sun, 21 Jul 2024 15:17:01 +0200
> >
> > Christopher Dimech wrote:
> >
> > > Provide a better implementation for math-sin-cos-raw
> > > Provide a better implementation for math-hypot
> >
> > I'll get right to it!
> >
> > If just Eli and Richard could spend a couple of hours
> > explaining what improvements are called for?
>
> I don't know what math-sin-cos-raw is about, but as for math-hypot,
> there's no need to implement anything, since this function is standard
> in any C libm, and we can easily expose it to Lisp if we need to.

Eli. perhaps there can be a math library that calc can use.  We will
then have a generic math library, and calc and other packages could use
some of its functionality.  Would that work for you ?  But if Emanuel
does not want to touch it, what is the point !

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Christopher Dimech <dimech@gmx.com>
> Cc: Emanuel Berg <incal@dataswamp.org>, emacs-devel@gnu.org
> Date: Sun, 21 Jul 2024 16:41:38 +0200
> Sensitivity: Normal
> 
> > > If just Eli and Richard could spend a couple of hours
> > > explaining what improvements are called for?
> >
> > I don't know what math-sin-cos-raw is about, but as for math-hypot,
> > there's no need to implement anything, since this function is standard
> > in any C libm, and we can easily expose it to Lisp if we need to.
> 
> Eli. perhaps there can be a math library that calc can use.  We will
> then have a generic math library, and calc and other packages could use
> some of its functionality.  Would that work for you ?  But if Emanuel
> does not want to touch it, what is the point !

This is too abstract a question.

The original issue was not about Calc, it was about a standalone math
library.  For that, we should first define the scope.  One possibility
is to expose to Lisp everything in a typical libm, but is that needed?
Another possibility is to provide a statistical library (average,
median, standard deviation, t-Student, F-test, chi-square, etc.) --
but do we need this?  Etc. etc. -- "math library" can be interpreted
in many different ways.

If we are now talking about improving Calc, someone who knows Calc
well should describe what is needed, and we can then discuss that in
detail.

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 2:49 AM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: "Christopher Dimech" <dimech@gmx.com>
> Cc: incal@dataswamp.org, emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> > From: Christopher Dimech <dimech@gmx.com>
> > Cc: Emanuel Berg <incal@dataswamp.org>, emacs-devel@gnu.org
> > Date: Sun, 21 Jul 2024 16:41:38 +0200
> > Sensitivity: Normal
> >
> > > > If just Eli and Richard could spend a couple of hours
> > > > explaining what improvements are called for?
> > >
> > > I don't know what math-sin-cos-raw is about, but as for math-hypot,
> > > there's no need to implement anything, since this function is standard
> > > in any C libm, and we can easily expose it to Lisp if we need to.
> >
> > Eli. perhaps there can be a math library that calc can use.  We will
> > then have a generic math library, and calc and other packages could use
> > some of its functionality.  Would that work for you ?  But if Emanuel
> > does not want to touch it, what is the point !
>
> This is too abstract a question.
>
> The original issue was not about Calc, it was about a standalone math
> library.  For that, we should first define the scope.  One possibility
> is to expose to Lisp everything in a typical libm, but is that needed?
> Another possibility is to provide a statistical library (average,
> median, standard deviation, t-Student, F-test, chi-square, etc.) --
> but do we need this?  Etc. etc. -- "math library" can be interpreted
> in many different ways.
>
> If we are now talking about improving Calc, someone who knows Calc
> well should describe what is needed, and we can then discuss that in
> detail.

Using Emacs Calc as a foundation for a new mathematical library
in Emacs is a practical and efficient approach.  Calc is a powerful tool
with extensive mathematical capabilities, so building on it allows
for leveraging its strengths while focusing on specific needs and
functionalities.

I rather have the people who work on calc do it as they have much more
experience and grasp of what a math library for emacs should do.

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Christopher Dimech <dimech@gmx.com>
> Cc: incal@dataswamp.org, emacs-devel@gnu.org
> Date: Sun, 21 Jul 2024 16:58:44 +0200
> 
> Using Emacs Calc as a foundation for a new mathematical library
> in Emacs is a practical and efficient approach.  Calc is a powerful tool
> with extensive mathematical capabilities, so building on it allows
> for leveraging its strengths while focusing on specific needs and
> functionalities.

I don't understand what "building on Calc" means in practice.  Any
Lisp program can load Calc and invoke its functions, so what else is
needed?  IOW, why what we already have in Calc is not enough?

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 3:02 AM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: "Christopher Dimech" <dimech@gmx.com>
> Cc: incal@dataswamp.org, emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> > From: Christopher Dimech <dimech@gmx.com>
> > Cc: incal@dataswamp.org, emacs-devel@gnu.org
> > Date: Sun, 21 Jul 2024 16:58:44 +0200
> >
> > Using Emacs Calc as a foundation for a new mathematical library
> > in Emacs is a practical and efficient approach.  Calc is a powerful tool
> > with extensive mathematical capabilities, so building on it allows
> > for leveraging its strengths while focusing on specific needs and
> > functionalities.
>
> I don't understand what "building on Calc" means in practice.  Any
> Lisp program can load Calc and invoke its functions, so what else is
> needed?  IOW, why what we already have in Calc is not enough?

Agreed.  Ask Emanuel, I don't know.  Do the calc people think we will be
better with an additional library that calc could also use ?  I would stick
to what they say.

Adding a generic mathematical library

[Christopher Dimech]

----- Kristof Đymek
Administrator General - Chilkat Design Build - Naiad Informatics - Gnu Project

Society has become too quick to pass judgement and declare someone
Persona Non-Grata, the most extreme form of censure a country can
bestow.

In a new era of destructive authoritarianism, I support Richard
Stallman.  Times of great crisis are also times of great
opportunity.  I call upon you to make this struggle yours as well !

https://www.gnu.org     https://www.fsf.org/


> Sent: Monday, July 22, 2024 at 1:03 AM
> From: "Christopher Dimech" <dimech@gmx.com>
> To: "Emanuel Berg" <incal@dataswamp.org>
> Cc: emacs-devel@gnu.org
> Subject: Adding a generic mathematical library
>
> Emanuel, looking at calc.el you will find
> 
> ;;   Provide a better implementation for math-sin-cos-raw.
> ;;   Provide a better implementation for math-hypot.
> 
> You can focus on these two things so you can provide us with better
> implementations of those two things.  Together with the necessary
> framework for extension and for use by calc.

math-hypot is found in file calc-math.el
 
> > Sent: Monday, July 22, 2024 at 12:27 AM
> > From: "Emanuel Berg" <incal@dataswamp.org>
> > To: emacs-devel@gnu.org
> > Subject: Re: Adding a generic mathematical library
> >
> > > What parts of arithmetics would you like to have? You are
> > > the math guy then, if that is the way you like it. You can
> > > tell me what functions and operators are missing - maybe
> > > definitions, actually whatever.
> >
> > And applications, like formulas and math functions.
> >
> > This -
> >
> > (defun distance-point (min max)
> >   (+ min (/ (- max min) 2.0)) )
> >
> > - or special, fun things. This even has the name arithmetic in
> > it. Well, almost.
> >
> > (defun arith-sum (n)
> >   (cl-loop
> >     with sum = 0
> >     for i from 1 to (1- n)
> >     do (cl-incf sum i)
> >     finally return sum) )
> >
> > (Hm - there are many ways to solve that, and this is fine by
> > all means.)
> >
> > You can also compare Emacs arithmetic capabilities with what
> > you find in a book. Do we have everything? Or do we want more?
> > That is where it begins.
> >
> > --
> > underground experts united
> > https://dataswamp.org/~incal
> >
> >
> >
> 
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> [...]

Yeah, as for you, feel free to do whatever you want.

As for me, I don't think I'm getting into calc, ever, sorry :)

Fiddling with libraries - why not - but if they are software
underbelly, to me that isn't a library, more like a poor man's
modular solution or a C or C++ header file associated with
a particular main file.

If you think of libraries like that then yes, my Elisp files,
all of them, are libraries. You want to use them? Sorry, I still
need my computer!

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 1:26 AM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech wrote:
>
> > [...]
>
> Yeah, as for you, feel free to do whatever you want.
>
> As for me, I don't think I'm getting into calc, ever, sorry :)
>
> Fiddling with libraries - why not - but if they are software
> underbelly, to me that isn't a library, more like a poor man's
> modular solution or a C or C++ header file associated with
> a particular main file.

Calc will then start using the library you suggest.  The task is to produce
an improvement of hypot.  If one suggests a mathematical library
it has to fit with the current tools.  Calc has the capability to
do the things you suggest and is extremely much more advanced.

To get what you want, you have to agree with something.  Otherwise
you get the resistance you have witnessed.  I am in agreement with Eli
on this thing.  We should first discuss its scope and design.

> If you think of libraries like that then yes, my Elisp files,
> all of them, are libraries. You want to use them? Sorry, I still
> need my computer!
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> To get what you want, you have to agree with something.
> Otherwise you get the resistance you have witnessed. I am in
> agreement with Eli on this thing. We should first discuss
> its scope and design.

Maybe there is some field of math that isn't taken by all
mighty Calc already?

That can happen. Scope would then be that. Design would be to
program a bunch of functions, operators, constants, combine
into popular formulas and so on.

I think the names are the only thing one has to agree on, what
to call everything. Is there and ISO document to tell you?

If not, one could agree to use some hefty Anglo-American
textbook with a good reputation and a modern edition and then
just consistently do whatever they do.

Most important thing is to have a plan, a good plan, and then
stick to it.

In general quite clearly: "Straight math" (Linear Algebra) is
easier to program than the derivative, limes functions,
probability distributions, and such stuff - also integers, as
in Discrete Math, are easier than floats (Analysis).

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> To get what you want, you have to agree with something.
> Otherwise you get the resistance you have witnessed. I am in
> agreement with Eli on this thing. We should first discuss
> its scope and design.

Relational Algebra - we should have a lot of that as they are
set based and we are list based, and a list is an ordered set
but don't have to be - don't know if we have everything.

Don't know what more math there is. Automata Theory if that
isn't considered computers actually.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> To get what you want, you have to agree with something.
> Otherwise you get the resistance you have witnessed. I am in
> agreement with Eli on this thing. We should first discuss
> its scope and design.

It matters where things are put, as long as they are shipped
and can be found from anywhere maybe that can be handled then.

It matters what things are called, if they have some long
prefix from some package it is like a joke asking math people
to use it - I'm not one of them but I know how they think and
would feel like an idiot handing them such software - if this
can be solved as well, so no prefix and ISO/conventional names
for everything according to a chosen method, then only one
thing remains and that is ...

The scope of the "library", will it also have a bunch of Calc
specific helpers, settings and so on?

Because if it doesn't it is just located in the Calc dir.
Why ever anyone would place it there I don't know but also
don't really care.

So yes, examine what Calc doesn't have then.

Individual functions ... hard to care about them. After you do
one, someone else will pop up, and you can spend a decade on
that and not a single new file or anything visibly
has improved. So no thanks to that.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> To get what you want, you have to agree with something.
> Otherwise you get the resistance you have witnessed. I am in
> agreement with Eli on this thing. We should first discuss
> its scope and design.

This is the reason the same code gets written over and over in
the Elisp world.

This is a more common thing than really odd inclusion of
software where one program that has absolutely nothing to do
with some other still has that (require 'software) - ???

People want to do something to their number or string, there
is such a string function, but it is in Gnus. Their program
has nothing to do with Gnus so to bring that in is insane.

But is it better to kill/yank the Gnus function and rename it
and have the same thing two, three ... times? No, both
alternatives are equally bad.

And the solution are libraries for everyone to use, not mere
files one is expected to find is some alien body of code and
then ... yes, what are you supposed to do in that situation,
if I may have a good answer to that question?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> To get what you want, you have to agree with something.
> Otherwise you get the resistance you have witnessed. I am in
> agreement with Eli on this thing. We should first discuss
> its scope and design.

To show you an example of what I mentioned in a post that
hasn't arrived, check out line number 8. What do you see?

Yes: ERC!

Elisp is the only language when you include an IRC client to
sort a bunch of wards. Yeah, crazy.

[ BTW don't understand why this is so complicated, should
  re-write that last part. Anyway, very useful! ]

PS. Now I've talked about this enough. You want the truth?
    You can _handle_ the truth. What I can come up with at
    home fiddling with a bunch of Elisp files, Emacs
    programmers cannot solve collectively since 1984?

    That is: 39y 6m 20d if we count from 1984-12-31; no idea
    what exact date GNU Emacs was first operational. Should be
    enough time to get libraries one would think but wrong
    policy, people that are used to leaders that they don't
    question ever ... then this is what happens. Anyway enough
    of this then but some of it was fun :)
DS.

(defun sort-line-words (beg end &optional set-delim)
  (interactive "r\nP")
  (or beg (setq beg (point-at-bol)))
  (or end (setq end (point-at-eol)))
  (let*((str       (buffer-substring-no-properties beg end))
        (delim-str (or set-delim (read-string "delimiter: ")))
        (str-list  (split-string str delim-str))
        (sorted    (erc-sort-strings str-list)) )
    (kill-region beg end)
    (if set-delim
        (progn
          (dolist (s (nreverse (cdr (nreverse sorted))))
            (insert (format "%s%s" s delim-str)))
          (insert (format "%s" (car (last sorted)))))
      (insert-string-list sorted) )))

;; sort me: a is just string test this
;; sorted:  a is just string test this
;; and me with a dash delim: this-is-just-a-test-string

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 12:10 AM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech wrote:
>
> > All depends on who wants to produce a first release that
> > handles two initial mathematical classifications, and which
> > helps others extend the library to other additional
> > topic classification.
>
> Yes, geometry and arithmetic, right?
>
> Let's start with arithmetic then. We make a plan what we want,
> what we already have, obviously we don't do, and we do
> the rest.
>
> > The choice would be of someone who would actually get an
> > agreed first release out.
>
> What parts of arithmetics would you like to have? You are the
> math guy then, if that is the way you like it. You can tell me
> what functions and operators are missing - maybe definitions,
> actually whatever. Most should be possible to put into code
> quite easily, I dare say.
>
> Let's start coding :)

Richard has suggested extracting a calc function that includes a substantial
amount of code to do a specific kind of calculation.

To address Eli's valid criticism and focus on something practical and
existing, let's enhance the existing emacs calc package for mathematical
computations.

Focusing on improving the calc package, which is already a powerful
tool for mathematical calculations in Emacs.  Specifically, make a
library that extends its capabilities, and which calc can use.

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> Richard has suggested extracting a calc function that
> includes a substantial amount of code to do a specific kind
> of calculation.
>
> To address Eli's valid criticism and focus on something
> practical and existing, let's enhance the existing emacs
> calc package for mathematical computations.

Ah, right, that is true, the way both reacted to this whole
thing and now this timely decision, there is absolutely
nothing fishy going on, they do it out of concern for calc -
I see.

Calc is as you say a package and a tool. Why it should
monopolize and assimilate generic math libraries I wonder?
Instead, it should focus on its interface and some settings
perhaps at the most and when in need of math, consult
a library for that - a generic library that anyone and
everyone could use just as well.

That said, calc has a lot of good stuff, just using that code
isn't natural in any way for whatever computation you would
like to do in Elisp or in your own application? Yeah,
it is completely backwards actually!

Instead of a tool using a library, we here attach the library
as an underbelly to the tool! Maybe some guy in the 80s came
up with that design but, as the saying goes, development has
gone forward.

Truth to the matter, people are not gonna be enthusiastic
about that, everyone knows that, and that tool isn't used
a lot as it is and this is likely to decrease even more.

If we had real libraries, people could do new tools, including
tools we cannot imagine right now. And some people could work
on the libraries, other people could work on the tools. But it
is what it is. Couple of too many mistakes and then it is not
going to happen.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> Richard has suggested extracting a calc function that
> includes a substantial amount of code to do a specific kind
> of calculation.

Guys, what about this idea - we move everything from
string.el, the string library, to gnus.el.

`gnus-string-or' and many other string functions are already
there! And when you think about it, what is Gnus if
not strings?

And then everyone else can use it from Gnus! Simple as that,
Gnus is shipped with vanilla Emacs so should be no
problem whatsoever.

Wait - we _do_ have a string.el library ... ?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> All depends on who wants to produce a first release that
> handles two initial mathematical classifications, and which
> helps others extend the library to other additional
> topic classification.

Here is arithmetic!

What more do we want? :)

Arithmetic:

(+ &rest numbers)
  Return sum of any number of arguments, which are numbers or markers.
  (+ 1 2)
    => 3
  (+ 1 2 3 4)
    => 10
-------------------------------------------------------------------------------

(- &rest numbers)
  Negate number or subtract numbers or markers and return the result.
  (- 3 2)
    => 1
  (- 6 3 2)
    => 1
-------------------------------------------------------------------------------

(* &rest numbers)
  Return product of any number of arguments, which are numbers or markers.
  (* 3 4 5)
    => 60
-------------------------------------------------------------------------------

(/ number &rest divisors)
  Divide number by divisors and return the result.
  (/ 10 5)
    => 2
  (/ 10 6)
    => 1
  (/ 10.0 6)
    => 1.6666666666666667
  (/ 10.0 3 3)
    => 1.1111111111111112
-------------------------------------------------------------------------------

(% x y)
  Return remainder of X divided by Y.
  (% 10 5)
    => 0
  (% 10 6)
    => 4
-------------------------------------------------------------------------------

(mod x y)
  Return X modulo Y.
  (mod 10 5)
    => 0
  (mod 10 6)
    => 4
  (mod 10.5 6)
    => 4.5
-------------------------------------------------------------------------------

(1+ number)
  Return NUMBER plus one.  NUMBER may be a number or a marker.
  (1+ 2)
    => 3
-------------------------------------------------------------------------------

(1- number)
  Return NUMBER minus one.  NUMBER may be a number or a marker.
  (1- 4)
    => 3

Predicates

(= number &rest numbers)
  Return t if args, all numbers or markers, are equal.
  (= 4 4)
    => t
  (= 4.0 4.0)
    => t
  (= 4 4.0)
    => t
  (= 4 4 4 4)
    => t
-------------------------------------------------------------------------------

(eql obj1 obj2)
  Return t if the two args are ‘eq’ or are indistinguishable numbers.
  (eql 4 4)
    => t
  (eql 4.0 4.0)
    => t
-------------------------------------------------------------------------------

(/= num1 num2)
  Return t if first arg is not equal to second arg.  Both must be numbers or markers.
  (/= 4 4)
    => nil
-------------------------------------------------------------------------------

(< number &rest numbers)
  Return t if each arg (a number or marker), is less than the next arg.
  (< 4 4)
    => nil
  (< 1 2 3)
    => t
-------------------------------------------------------------------------------

(<= number &rest numbers)
  Return t if each arg (a number or marker) is less than or equal to the next.
  (<= 4 4)
    => t
  (<= 1 2 2 3)
    => t
-------------------------------------------------------------------------------

(> number &rest numbers)
  Return t if each arg (a number or marker) is greater than the next arg.
  (> 4 4)
    => nil
  (> 3 2 1)
    => t
-------------------------------------------------------------------------------

(>= number &rest numbers)
  Return t if each arg (a number or marker) is greater than or equal to the next.
  (>= 4 4)
    => t
  (>= 3 2 2 1)
    => t
-------------------------------------------------------------------------------

(zerop number)
  Return t if NUMBER is zero.
  (zerop 0)
    => t
-------------------------------------------------------------------------------

(natnump object)
  Return t if OBJECT is a nonnegative integer.
  (natnump -1)
    => nil
  (natnump 0)
    => t
  (natnump 23)
    => t
-------------------------------------------------------------------------------

(cl-plusp number)
  Return t if NUMBER is positive.
  (cl-plusp 0)
    => nil
  (cl-plusp 1)
    => t
-------------------------------------------------------------------------------

(cl-minusp number)
  Return t if NUMBER is negative.
  (cl-minusp 0)
    => nil
  (cl-minusp -1)
    => t
-------------------------------------------------------------------------------

(cl-oddp integer)
  Return t if INTEGER is odd.
  (cl-oddp 3)
    => t
-------------------------------------------------------------------------------

(cl-evenp integer)
  Return t if INTEGER is even.
  (cl-evenp 6)
    => t
-------------------------------------------------------------------------------

(bignump object)
  Return t if OBJECT is a bignum.
  (bignump 4)
    => nil
  (bignump (expt 2 90))
    => t
-------------------------------------------------------------------------------

(fixnump object)
  Return t if OBJECT is a fixnum.
  (fixnump 4)
    => t
  (fixnump (expt 2 90))
    => nil
-------------------------------------------------------------------------------

(floatp object)
  Return t if OBJECT is a floating point number.
  (floatp 5.4)
    => t
-------------------------------------------------------------------------------

(integerp object)
  Return t if OBJECT is an integer.
  (integerp 5.4)
    => nil
-------------------------------------------------------------------------------

(numberp object)
  Return t if OBJECT is a number (floating point or integer).
  (numberp "5.4")
    => nil
-------------------------------------------------------------------------------

(cl-digit-char-p char &optional radix)
  Test if CHAR is a digit in the specified RADIX (default 10).
  (cl-digit-char-p 53 10)
    => 5
  (cl-digit-char-p 102 16)
    => 15

Operations

(max number &rest numbers)
  Return largest of all the arguments (which must be numbers or markers).
  (max 7 9 3)
    => 9
-------------------------------------------------------------------------------

(min number &rest numbers)
  Return smallest of all the arguments (which must be numbers or markers).
  (min 7 9 3)
    => 3
-------------------------------------------------------------------------------

(abs arg)
  Return the absolute value of ARG.
  (abs -4)
    => 4
-------------------------------------------------------------------------------

(float arg)
  Return the floating point number equal to ARG.
  (float 2)
    => 2.0
-------------------------------------------------------------------------------

(truncate arg &optional divisor)
  Truncate a floating point number to an int.
  (truncate 1.2)
    => 1
  (truncate -1.2)
    => -1
  (truncate 5.4 2)
    => 2
-------------------------------------------------------------------------------

(floor arg &optional divisor)
  Return the largest integer no greater than ARG.
  (floor 1.2)
    => 1
  (floor -1.2)
    => -2
  (floor 5.4 2)
    => 2
-------------------------------------------------------------------------------

(ceiling arg &optional divisor)
  Return the smallest integer no less than ARG.
  (ceiling 1.2)
    => 2
  (ceiling -1.2)
    => -1
  (ceiling 5.4 2)
    => 3
-------------------------------------------------------------------------------

(round arg &optional divisor)
  Return the nearest integer to ARG.
  (round 1.2)
    => 1
  (round -1.2)
    => -1
  (round 5.4 2)
    => 3
-------------------------------------------------------------------------------

(random &optional limit)
  Return a pseudo-random integer.
  (random 6)
    => 5

Bit Operations

(ash value count)
  Return integer VALUE with its bits shifted left by COUNT bit positions.
  (ash 1 4)
    => 16
  (ash 16 -1)
    => 8
-------------------------------------------------------------------------------

(logand &rest ints-or-markers)
  Return bitwise-and of all the arguments.
  (logand #b10 #b111)
    => #b10
-------------------------------------------------------------------------------

(logior &rest ints-or-markers)
  Return bitwise-or of all the arguments.
  (logior 4 16)
    => 20
-------------------------------------------------------------------------------

(logxor &rest ints-or-markers)
  Return bitwise-exclusive-or of all the arguments.
  (logxor 4 16)
    => 20
-------------------------------------------------------------------------------

(lognot number)
  Return the bitwise complement of NUMBER.  NUMBER must be an integer.
  (lognot 5)
    => -6
-------------------------------------------------------------------------------

(logcount value)
  Return population count of VALUE.
  (logcount 5)
    => 2

Floating Point

(isnan x)
  Return non-nil if argument X is a NaN.
  (isnan 5.0)
    => nil
-------------------------------------------------------------------------------

(frexp x)
  Get significand and exponent of a floating point number.
  (frexp 5.7)
    => (0.7125 . 3)
-------------------------------------------------------------------------------

(ldexp sgnfcand exponent)
  Return SGNFCAND * 2**EXPONENT, as a floating point number.
  (ldexp 0.7125 3)
    => 5.7
-------------------------------------------------------------------------------

(logb arg)
  Returns largest integer <= the base 2 log of the magnitude of ARG.
  (logb 10.5)
    => 3
-------------------------------------------------------------------------------

(ffloor arg)
  Return the largest integer no greater than ARG, as a float.
  (ffloor 1.2)
    => 1.0
-------------------------------------------------------------------------------

(fceiling arg)
  Return the smallest integer no less than ARG, as a float.
  (fceiling 1.2)
    => 2.0
-------------------------------------------------------------------------------

(ftruncate arg)
  Truncate a floating point number to an integral float value.
  (ftruncate 1.2)
    => 1.0
-------------------------------------------------------------------------------

(fround arg)
  Return the nearest integer to ARG, as a float.
  (fround 1.2)
    => 1.0

Standard Math Functions

(sin arg)
  Return the sine of ARG.
  (sin float-pi)
    => 1.2246467991473532e-16
-------------------------------------------------------------------------------

(cos arg)
  Return the cosine of ARG.
  (cos float-pi)
    => -1.0
-------------------------------------------------------------------------------

(tan arg)
  Return the tangent of ARG.
  (tan float-pi)
    => -1.2246467991473532e-16
-------------------------------------------------------------------------------

(asin arg)
  Return the inverse sine of ARG.
  (asin float-pi)
    => 0.0e+NaN
-------------------------------------------------------------------------------

(acos arg)
  Return the inverse cosine of ARG.
  (acos float-pi)
    => 0.0e+NaN
-------------------------------------------------------------------------------

(atan y &optional x)
  Return the inverse tangent of the arguments.
  (atan float-pi)
    => 1.2626272556789118
-------------------------------------------------------------------------------

(exp arg)
  Return the exponential base e of ARG.
  (exp 4)
    => 54.598150033144236
-------------------------------------------------------------------------------

(log arg &optional base)
  Return the natural logarithm of ARG.
  (log 54.59)
    => 3.9998507157936665
-------------------------------------------------------------------------------

(expt arg1 arg2)
  Return the exponential ARG1 ** ARG2.
  (expt 2 16)
    => 65536
-------------------------------------------------------------------------------

(sqrt arg)
  Return the square root of ARG.
  (sqrt -1)
    => -0.0e+NaN

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Where to actually start? After all this meta, maybe some words
on that the way I see it.

They way I would do it is to identify the simplest level or
place of math. Some people say logic is math, other say math
is based on logic, but to me, here, it doesn't matter since
I think arithmetic is much simpler than logic anyway.

So if we start there (this is again how I see it) one would
look around, what do we already have - this will be a lot, of
course. But are there things we don't have that are essential
and basic, or really interesting for that matter - then we
implement them, because at such a low level, we should cover
all or close to all things that are important.

And one shouldn't think, "well, we don't have it but one can
solve that with Elisp". Because one can solve everything with
Elisp! Here we want specific functions and we want them to be
fast, easy-to-use and readily available. (One will still have
to think what to add or not, as always.)

So when arithmetic is done - good work, pad pad - and we can
move up someplace else, maybe more challenging as well.
And the higher one will come, the more people will just do
what they are into, because it will be impossible to cover
everything of everything when it becomes that specialized.
But that is OK, free software is like that, people do what
they are into. And I dare say that people _will_ want to do
Combinatorics, Probability, Linear Algebra, and all crazy
stuff there is.

A challenge will be to find people who already have that kind
of stuff, maybe half-done. That will be a huge help to see,
what did they do, how far did they come, why didn't they
complete it, etc.

That is: at the basic level one must have pretty much
everything so why not start there?

Know that if we plan this for ELPA people will have made an
active choise to use it, that means we shouldn't be too afraid
to include stuff. They use it because they want it, so we
should make sure there is something there for them to get :)

-- 
underground experts united
https://dataswamp.org/~incal

hypotenuse (was: Re: Adding a generic mathematical library)

[Emanuel Berg]

Richard Stallman wrote:

> Onsider, for instance, `hypotenuse'. The benefit of having
> that functoin would not quite be zero, but it would be
> small. Is it enough to justify the additional function name,
> and the complexity of documenting it?

RMS has really performed excellently in this discussion and my
favorite part of his critique must be this.

What a sharp eye, to immediately identify the problem of
a proposed solution!

Indeed: Is it enough to justify the "complexity of documenting
it"?

That is an excellent question, and I leave it free for all
to contemplate. Can we justify the complexity?

Or does the below two lines of Elisp belong to the dustbin
of computing?

(defun hypotenuse (c1 c2)
  (sqrt (+ (* c1 c1) (* c2 c2))) )

-- 
underground experts united
https://dataswamp.org/~incal

Re: hypotenuse (was: Re: Adding a generic mathematical library)

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Sun, 21 Jul 2024 16:30:26 +0200
> 
> Richard Stallman wrote:
> 
> > Onsider, for instance, `hypotenuse'. The benefit of having
> > that functoin would not quite be zero, but it would be
> > small. Is it enough to justify the additional function name,
> > and the complexity of documenting it?
> 
> RMS has really performed excellently in this discussion and my
> favorite part of his critique must be this.
> 
> What a sharp eye, to immediately identify the problem of
> a proposed solution!
> 
> Indeed: Is it enough to justify the "complexity of documenting
> it"?
> 
> That is an excellent question, and I leave it free for all
> to contemplate. Can we justify the complexity?

You are mocking questions which happen to be serious and non-trivial
to answer.

> Or does the below two lines of Elisp belong to the dustbin
> of computing?
> 
> (defun hypotenuse (c1 c2)
>   (sqrt (+ (* c1 c1) (* c2 c2))) )

This is not the best way of computing hypot: squaring a number could
overflow or underflow, and you get bad result.  Try

    (hypotenuse 1.e-600 1.e-600)
or
    (hypotenuse 1.e600 1.e600)

and you will see it.

Like I said: a high-quality math library needs to use solid algorithms
that produce valid results whenever the problem has a solution.
Stable and accurate algorithms for hypot are known, and the libm
implementation usually follows it (and if it doesn't, we can roll our
own).

Re: hypotenuse (was: Re: Adding a generic mathematical library)

[Eli Zaretskii]

> Date: Sun, 21 Jul 2024 17:44:47 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > (defun hypotenuse (c1 c2)
> >   (sqrt (+ (* c1 c1) (* c2 c2))) )
> 
> This is not the best way of computing hypot: squaring a number could
> overflow or underflow, and you get bad result.  Try
> 
>     (hypotenuse 1.e-600 1.e-600)
> or
>     (hypotenuse 1.e600 1.e600)
> 
> and you will see it.

Sorry, that should have been 1.e-300 and 1.e300 instead.

hypotenuse (was: Re: Adding a generic mathematical library)

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 3:00 AM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: incal@dataswamp.org
> Cc: emacs-devel@gnu.org
> Subject: Re: hypotenuse (was: Re: Adding a generic mathematical library)
>
> > Date: Sun, 21 Jul 2024 17:44:47 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: emacs-devel@gnu.org
> >
> > > (defun hypotenuse (c1 c2)
> > >   (sqrt (+ (* c1 c1) (* c2 c2))) )
> >
> > This is not the best way of computing hypot: squaring a number could
> > overflow or underflow, and you get bad result.  Try
> >
> >     (hypotenuse 1.e-600 1.e-600)
> > or
> >     (hypotenuse 1.e600 1.e600)
> >
> > and you will see it.
>
> Sorry, that should have been 1.e-300 and 1.e300 instead.

I have a solution to the problem in fortran, but not in elisp.  I could
include it.  Its implementation required serious concentration.

Re: hypotenuse (was: Re: Adding a generic mathematical library)

[Eli Zaretskii]

> From: Christopher Dimech <dimech@gmx.com>
> Cc: incal@dataswamp.org, emacs-devel@gnu.org
> Date: Sun, 21 Jul 2024 17:12:10 +0200
> Sensitivity: Normal
> 
> > Sent: Monday, July 22, 2024 at 3:00 AM
> > From: "Eli Zaretskii" <eliz@gnu.org>
> > To: incal@dataswamp.org
> > Cc: emacs-devel@gnu.org
> > Subject: Re: hypotenuse (was: Re: Adding a generic mathematical library)
> >
> > > Date: Sun, 21 Jul 2024 17:44:47 +0300
> > > From: Eli Zaretskii <eliz@gnu.org>
> > > Cc: emacs-devel@gnu.org
> > >
> > > > (defun hypotenuse (c1 c2)
> > > >   (sqrt (+ (* c1 c1) (* c2 c2))) )
> > >
> > > This is not the best way of computing hypot: squaring a number could
> > > overflow or underflow, and you get bad result.  Try
> > >
> > >     (hypotenuse 1.e-600 1.e-600)
> > > or
> > >     (hypotenuse 1.e600 1.e600)
> > >
> > > and you will see it.
> >
> > Sorry, that should have been 1.e-300 and 1.e300 instead.
> 
> I have a solution to the problem in fortran, but not in elisp.  I could
> include it.  Its implementation required serious concentration.

AFAIK, every libm already includes a similar implementation, so we
should just expose it to Lisp (again, assuming this function is indeed
required to be part of Emacs).

hypotenuse (was: Re: Adding a generic mathematical library)

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 3:42 AM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: "Christopher Dimech" <dimech@gmx.com>
> Cc: incal@dataswamp.org, emacs-devel@gnu.org
> Subject: Re: hypotenuse (was: Re: Adding a generic mathematical library)
>
> > From: Christopher Dimech <dimech@gmx.com>
> > Cc: incal@dataswamp.org, emacs-devel@gnu.org
> > Date: Sun, 21 Jul 2024 17:12:10 +0200
> > Sensitivity: Normal
> >
> > > Sent: Monday, July 22, 2024 at 3:00 AM
> > > From: "Eli Zaretskii" <eliz@gnu.org>
> > > To: incal@dataswamp.org
> > > Cc: emacs-devel@gnu.org
> > > Subject: Re: hypotenuse (was: Re: Adding a generic mathematical library)
> > >
> > > > Date: Sun, 21 Jul 2024 17:44:47 +0300
> > > > From: Eli Zaretskii <eliz@gnu.org>
> > > > Cc: emacs-devel@gnu.org
> > > >
> > > > > (defun hypotenuse (c1 c2)
> > > > >   (sqrt (+ (* c1 c1) (* c2 c2))) )
> > > >
> > > > This is not the best way of computing hypot: squaring a number could
> > > > overflow or underflow, and you get bad result.  Try
> > > >
> > > >     (hypotenuse 1.e-600 1.e-600)
> > > > or
> > > >     (hypotenuse 1.e600 1.e600)
> > > >
> > > > and you will see it.
> > >
> > > Sorry, that should have been 1.e-300 and 1.e300 instead.
> >
> > I have a solution to the problem in fortran, but not in elisp.  I could
> > include it.  Its implementation required serious concentration.
>
> AFAIK, every libm already includes a similar implementation, so we
> should just expose it to Lisp (again, assuming this function is indeed
> required to be part of Emacs).

The people involved in calc do think it is required.

Re: hypotenuse

[Max Nikulin]

On 21/07/2024 22:12, Christopher Dimech wrote:
>>>> (defun hypotenuse (c1 c2)
>>>>    (sqrt (+ (* c1 c1) (* c2 c2))) )
> 
> I have a solution to the problem in fortran, but not in elisp.  I could
> include it.  Its implementation required serious concentration.

Underflow/overflow issues may be solved with a few lines of code and it 
does matter if it is fortran, elisp, or another language (besides 
pathological ones). The problem is to avoid error above 1 ULP.

In 2021 hypot implementation in glibc was replaced by a more precise one:
https://sourceware.org/git/?p=glibc.git;a=commit;h=6c848d70383e1dbe932ef41723ac0abfdeec7ca8

Re: hypotenuse

[Eli Zaretskii]

> From: Max Nikulin <manikulin@gmail.com>
> Date: Sun, 21 Jul 2024 22:54:53 +0700
> 
> On 21/07/2024 22:12, Christopher Dimech wrote:
> >>>> (defun hypotenuse (c1 c2)
> >>>>    (sqrt (+ (* c1 c1) (* c2 c2))) )
> > 
> > I have a solution to the problem in fortran, but not in elisp.  I could
> > include it.  Its implementation required serious concentration.
> 
> Underflow/overflow issues may be solved with a few lines of code and it 
> does matter if it is fortran, elisp, or another language (besides 
> pathological ones). The problem is to avoid error above 1 ULP.

Right, overflow and underflow are just the tip of a very large
iceberg.

> In 2021 hypot implementation in glibc was replaced by a more precise one:
> https://sourceware.org/git/?p=glibc.git;a=commit;h=6c848d70383e1dbe932ef41723ac0abfdeec7ca8

Which is exactly why I said we should use the existing implementation
in the standard library.

hypotenuse

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 4:12 AM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: "Max Nikulin" <manikulin@gmail.com>
> Cc: emacs-devel@gnu.org
> Subject: Re: hypotenuse
>
> > From: Max Nikulin <manikulin@gmail.com>
> > Date: Sun, 21 Jul 2024 22:54:53 +0700
> >
> > On 21/07/2024 22:12, Christopher Dimech wrote:
> > >>>> (defun hypotenuse (c1 c2)
> > >>>>    (sqrt (+ (* c1 c1) (* c2 c2))) )
> > >
> > > I have a solution to the problem in fortran, but not in elisp.  I could
> > > include it.  Its implementation required serious concentration.
> >
> > Underflow/overflow issues may be solved with a few lines of code and it
> > does matter if it is fortran, elisp, or another language (besides
> > pathological ones). The problem is to avoid error above 1 ULP.
>
> Right, overflow and underflow are just the tip of a very large
> iceberg.
>
> > In 2021 hypot implementation in glibc was replaced by a more precise one:
> > https://sourceware.org/git/?p=glibc.git;a=commit;h=6c848d70383e1dbe932ef41723ac0abfdeec7ca8
>
> Which is exactly why I said we should use the existing implementation
> in the standard library.

So we should.  I am in agreement.

Re: hypotenuse (was: Re: Adding a generic mathematical library)

[Emanuel Berg]

Eli Zaretskii wrote:

> Like I said: a high-quality math library needs to use solid
> algorithms that produce valid results whenever the problem
> has a solution.

Why do programming languages have libraries?

I don't understand why not they just ship with a calculator,
an e-mail client and ... yeah, you think ERC is enough
for networking? If we have two of them, possibly.

Anyway, what about this then:

  https://dataswamp.org/~incal/hypo.el

-- 
underground experts united
https://dataswamp.org/~incal

Re: hypotenuse (was: Re: Adding a generic mathematical library)

[Emanuel Berg]

Eli Zaretskii wrote:

> This is not the best way of computing hypot: squaring
> a number could overflow or underflow, and you get
> bad result.

Merge with Common Lisp and get their libraries or have Stefan
and Andrea compensated to write a new Lisp that is better than
CL but still compatible so their user base, libraries and
tools become available to us.

CL is also faster, not that Elisp is slow anymore thanks to
native compilation. People in the CL world has a bigger
collective experience from programming so these beginner's
mistakes hopefully happen less often, or get stuck from the
80s/90s without anyone doing anything because "<something> is
a tool" which is especially hilarious and an embarrassment to
the sport because that is actually the reason _not_ to couple
libraries onto them, whoever came up with that heat
struck idea.

-- 
underground experts united
https://dataswamp.org/~incal

Re: hypotenuse (was: Re: Adding a generic mathematical library)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 510 bytes --]

On Sun, Jul 21, 2024 at 04:30:26PM +0200, Emanuel Berg wrote:

[..]

> (defun hypotenuse (c1 c2)
>   (sqrt (+ (* c1 c1) (* c2 c2))) )

Conceptually, yes. Practically... just no.

The shallow (handling /just/ overflow):

  https://www.johndcook.com/blog/2010/06/02/whats-so-hard-about-finding-a-hypotenuse/

The deeper (but with /very/ nice pics):

  https://www.cs.umd.edu/~ntoronto/papers/toronto-2014cise-floating-point.pdf

(I guess Calc is already beyond your formula).

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Adding a generic mathematical library

[Michael Heerdegen via Emacs development discussions.]

Philip Kaludercic <philipk@posteo.net> writes:

> > Someone™ should propose a concrete suggestion.

Have we considered to write a bit of glue code that would make it easier
and more convenient to use the things defined in Calc?  Why invest all of
the time again?


Michael.

Re: Adding a generic mathematical library

[Philip Kaludercic]

Michael Heerdegen via "Emacs development discussions."
<emacs-devel@gnu.org> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> > Someone™ should propose a concrete suggestion.
>
> Have we considered to write a bit of glue code that would make it easier
> and more convenient to use the things defined in Calc?  Why invest all of
> the time again?

No concrete suggestions have been made yet, so perhaps this could be the
easiest solution.  Again, it might all depend on what the priorities of
such a library would be (Flexibility, Extensibility, Performance, etc.).

> Michael.
>
>
>

-- 
	Philip Kaludercic on peregrine

Re: Adding a generic mathematical library

[Michael Heerdegen via Emacs development discussions.]

Michael Heerdegen via "Emacs development discussions."
<emacs-devel@gnu.org> writes:

> Have we considered to write a bit of glue code that would make it easier
> and more convenient to use the things defined in Calc?  Why invest all of
> the time again?

Here is what we already have in Emacs:

Calc uses a dedicated format to represent math expressions.  This line
reads in such an expression (load Calc before eval'ing):

#+begin_src emacs-lisp
  (car (calc-do-alg-entry "" "Calc expr: " t 'calc-quick-calc-history))
#+end_src

If you enter for example "x + (10!+3)^2" this returns

#+begin_src emacs-lisp
  (+ (var x var-x) (^ (+ (calcFunc-fact 10) 3) 2))
#+end_src

Eval with `math-evaluate-expr':

#+begin_src emacs-lisp
(math-evaluate-expr
 '(+ (var x var-x) (^ (+ (calcFunc-fact 10) 3) 2)))
==> (+ (var x var-x) 13168211212809)
#+end_src

Again the result is using same representation of mathematical
expressions.  Use `math-format-value' to print the result:

#+begin_src emacs-lisp
(math-format-value '(+ (var x var-x) 13168211212809))
 ==> "x + 13168211212809"
#+end_src

A new library will probably also need to invent a new dedicated type of
expressions that is able to represent everything needed.  I don't
understand why it would be a good idea to start from scratch.


Michael.

Re: Adding a generic mathematical library

[Emanuel Berg]

Michael Heerdegen via "Emacs development discussions." wrote:

> I don't understand why it would be a good idea to start
> from scratch.

Maybe we should have a math library in ELPA, then it will be
less hesitation to just add stuff that is desired as we go
along, not having to argue what relevance it has to Emacs
per se as a piece of text editing software.

In general libraries for everything is a good way of not
having people write code that solves the same problems over
and over. Optimally what would happen is you would know where
to look for things and if it wasn't there, you'd write it _and
then add it to that same place_, so what happened won't repeat
itself for the next dude in the same situation.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Christopher Dimech]

> Sent: Thursday, July 18, 2024 at 6:07 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Michael Heerdegen via "Emacs development discussions." wrote:
>
> > I don't understand why it would be a good idea to start
> > from scratch.
>
> Maybe we should have a math library in ELPA, then it will be
> less hesitation to just add stuff that is desired as we go
> along, not having to argue what relevance it has to Emacs
> per se as a piece of text editing software.

The problem that usually occurs is that once things are in place
things would be much more difficult to change in design.  Because
we tend to be quite strict about not breaking their previous use.

> In general libraries for everything is a good way of not
> having people write code that solves the same problems over
> and over. Optimally what would happen is you would know where
> to look for things and if it wasn't there, you'd write it _and
> then add it to that same place_, so what happened won't repeat
> itself for the next dude in the same situation.
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

>> Maybe we should have a math library in ELPA, then it will
>> be less hesitation to just add stuff that is desired as we
>> go along, not having to argue what relevance it has to
>> Emacs per se as a piece of text editing software.
>
> The problem that usually occurs is that once things are in
> place things would be much more difficult to change in
> design. Because we tend to be quite strict about not
> breaking their previous use.

The software doesn't exist, for this reason it hasn't been put
anywhere. Are you worrying about difficulties moving it
from there?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Christopher Dimech]

> Sent: Thursday, July 18, 2024 at 7:12 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech wrote:
>
> >> Maybe we should have a math library in ELPA, then it will
> >> be less hesitation to just add stuff that is desired as we
> >> go along, not having to argue what relevance it has to
> >> Emacs per se as a piece of text editing software.
> >
> > The problem that usually occurs is that once things are in
> > place things would be much more difficult to change in
> > design. Because we tend to be quite strict about not
> > breaking their previous use.
>
> The software doesn't exist, for this reason it hasn't been put
> anywhere. Are you worrying about difficulties moving it
> from there?
>
> --
> underground experts united
> https://dataswamp.org/~incal

I am aligned with Eli's comment about how the library has to move beyond
those intended for personal use.  A well thought out robust design would
be appreciated if things are to move forward.  Because decisions done in
the initial stages will impact upon the capabilities others can add to it.

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> I am aligned with Eli's comment about how the library has to
> move beyond those intended for personal use.

If you examine the packages in ELPA you will see they conform
very well to a decided form. Everyone who has done work on
such or even used them and examined the source would see that
file isn't prepared for that.

And och course nothing intended for personal use with no
usefulness to the general Emacs public is ever added to core
Emacs _or_ ELPA.

Did you and Eli just discovered this?

Okay, whatever, we are on the same page then that a math
library must be prepared and written very well, but to the
form and what it offers.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Thu, 18 Jul 2024 08:07:04 +0200
> 
> Michael Heerdegen via "Emacs development discussions." wrote:
> 
> > I don't understand why it would be a good idea to start
> > from scratch.
> 
> Maybe we should have a math library in ELPA, then it will be
> less hesitation to just add stuff that is desired as we go
> along, not having to argue what relevance it has to Emacs
> per se as a piece of text editing software.

To have a math library on ELPA, the library should be
reasonably-comprehensive, use solid algorithms, and solve problems
that happen in real life frequently enough.  By contrast, my
impression from looking at your library is that it is an ad-hoc
collection of functions that solve problems you personally find
useful, but their general usefulness is IMO questionable, and at least
in some cases the algorithms are not the best ones known for the job.
For example, your mean-value function implements a naïve algorithm
that has known accuracy and round-off issues.

It is perfectly okay to have such libraries for personal use, of
course, but that is not the issue at hand here.

If what you offer is to have a completely different library, then we
need first to talk what will be in it and which algorithms it will
implement.  "Math functions" is a very broad term, so defining the
scope for such a library is not a trivial job.

> In general libraries for everything is a good way of not
> having people write code that solves the same problems over
> and over.

Yes, but since the functions you propose are not in Emacs, and neither
(AFAICT) is any code which resembles them, I wonder whether the need
is a real one and general enough to justify such a package.

Re: Adding a generic mathematical library

[Emanuel Berg]

Eli Zaretskii wrote:

> To have a math library on ELPA, the library should be
> reasonably-comprehensive, use solid algorithms, and solve
> problems that happen in real life frequently enough.
> By contrast, my impression from looking at your library
> [...]

Unbelievable, no one has suggested adding that file to ELPA.

> is that it is an ad-hoc collection of functions that solve
> problems you personally find useful, but their general
> usefulness is IMO questionable

Incorrect, most math functions in it are really basic so one
would expect them to be present in one form of another.

> and at least in some cases the algorithms are not the best
> ones known for the job. For example, your mean-value
> function implements a naive algorithm that has known
> accuracy and round-off issues.

Unbelievable comment. This is indeed the problem that happens,
as has been said in this very thread: without libraries and
common projects individual people solve their problems locally
with no incremental gain from one person to another. So it
won't be as good. And now you see an example of that, bringing
it up as a reason we can't have a math library in ELPA?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Thu, 18 Jul 2024 09:57:40 +0200
> 
> Eli Zaretskii wrote:
> 
> > To have a math library on ELPA, the library should be
> > reasonably-comprehensive, use solid algorithms, and solve
> > problems that happen in real life frequently enough.
> > By contrast, my impression from looking at your library
> > [...]
> 
> Unbelievable, no one has suggested adding that file to ELPA.

You said, and I quote:

> Maybe we should have a math library in ELPA, then it will be
> less hesitation to just add stuff that is desired as we go
> along, not having to argue what relevance it has to Emacs
> per se as a piece of text editing software.

I was not sure whether you were talking about the library you posted,
so I qualified myself, quote again:

> If what you offer is to have a completely different library, then we
> need first to talk what will be in it and which algorithms it will
> implement.  "Math functions" is a very broad term, so defining the
> scope for such a library is not a trivial job.

Okay?

> > is that it is an ad-hoc collection of functions that solve
> > problems you personally find useful, but their general
> > usefulness is IMO questionable
> 
> Incorrect, most math functions in it are really basic so one
> would expect them to be present in one form of another.

I disagree.  My evidence is that the need for functions presented in
your library, if it exists, was not strong enough, otherwise people
would be asking for some of these functions long ago.

> > and at least in some cases the algorithms are not the best
> > ones known for the job. For example, your mean-value
> > function implements a naive algorithm that has known
> > accuracy and round-off issues.
> 
> Unbelievable comment. This is indeed the problem that happens,
> as has been said in this very thread: without libraries and
> common projects individual people solve their problems locally
> with no incremental gain from one person to another. So it
> won't be as good. And now you see an example of that, bringing
> it up as a reason we can't have a math library in ELPA?

OK, so if you want to talk about adding a math library, I point you
back to my comment above: we should first discuss its scope and
design.

Re: Adding a generic mathematical library

[Emanuel Berg]

Eli Zaretskii wrote:

>>> To have a math library on ELPA, the library should be
>>> reasonably-comprehensive, use solid algorithms, and solve
>>> problems that happen in real life frequently enough.
>>> By contrast, my impression from looking at your library [...]
>> 
>> Unbelievable, no one has suggested adding that file
>> to ELPA.
>
> You said, and I quote:
>
>> Maybe we should have a math library in ELPA, then it will be
>> less hesitation to just add stuff that is desired as we go
>> along, not having to argue what relevance it has to Emacs
>> per se as a piece of text editing software.
>
> I was not sure whether you were talking about the library
> you posted, so I qualified myself, quote again:

That's not a library or file prepared for ELPA as you well
know, it is just a bunch of functions. I have several such
files that I have written myself since Emacs don't have it.
Time, permutations, negative subtraction, some stats,
yada yada.

>> If what you offer is to have a completely different
>> library, then we need first to talk what will be in it and
>> which algorithms it will implement. "Math functions" is
>> a very broad term, so defining the scope for such a library
>> is not a trivial job.
>
> Okay?

(I didn't write that.)

>>> is that it is an ad-hoc collection of functions that solve
>>> problems you personally find useful, but their general
>>> usefulness is IMO questionable
>> 
>> Incorrect, most math functions in it are really basic so
>> one would expect them to be present in one form of another.
>
> I disagree. My evidence is that the need for functions
> presented in your library, if it exists, was not strong
> enough, otherwise people would be asking for some of these
> functions long ago.

Math is not popular enough?

We have Tetris in core Emacs and stuff in ELPA no one would
ever know existed.

But math - naah, people don't need it in the world's
programmer's editor number one. No one told you they want it
so they cannot have ever wanted it. Simple!

Well, I have said it several times and now other people say it
as well. Math is huge in tech and science and a lot of other
fields as well were we want to be. It is also something a lot
of programmers are passionate about. _And_ a lot of
non-programmers do math!

Old school engineers used to know two things: how to do math
and how to draw drawings and schematics. That was the basic.

And there are many more examples.

>> Unbelievable comment. This is indeed the problem that
>> happens, as has been said in this very thread: without
>> libraries and common projects individual people solve their
>> problems locally with no incremental gain from one person
>> to another. So it won't be as good. And now you see an
>> example of that, bringing it up as a reason we can't have
>> a math library in ELPA?
>
> OK, so if you want to talk about adding a math library,
> I point you back to my comment above: we should first
> discuss its scope and design.

Well, the people doing it, at least.

The scope to begin with: everything easy and basic you expect
to find in it.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Eli Zaretskii wrote:

>> In general libraries for everything is a good way of not
>> having people write code that solves the same problems over
>> and over.
>
> Yes, but since the functions you propose are not in Emacs,
> and neither (AFAICT) is any code which resembles them,
> I wonder whether the need is a real one and general enough
> to justify such a package.

There is no need for a math library for the Emacs editor,
which is extensively used by programmers and tech people all
over the world. Because math is not a general need.

That people write their own Elisp because they don't have
official software to do even trivial things doesn't make the
need real.

Yeah, what can I say? Just unbelievable.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Thu, 18 Jul 2024 10:15:18 +0200
> 
> Eli Zaretskii wrote:
> 
> >> In general libraries for everything is a good way of not
> >> having people write code that solves the same problems over
> >> and over.
> >
> > Yes, but since the functions you propose are not in Emacs,
> > and neither (AFAICT) is any code which resembles them,
> > I wonder whether the need is a real one and general enough
> > to justify such a package.
> 
> There is no need for a math library for the Emacs editor,
> which is extensively used by programmers and tech people all
> over the world. Because math is not a general need.

That's not what I said, not even close.

Re: Adding a generic mathematical library

[Christopher Dimech]

> Sent: Thursday, July 18, 2024 at 8:15 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Eli Zaretskii wrote:
>
> >> In general libraries for everything is a good way of not
> >> having people write code that solves the same problems over
> >> and over.
> >
> > Yes, but since the functions you propose are not in Emacs,
> > and neither (AFAICT) is any code which resembles them,
> > I wonder whether the need is a real one and general enough
> > to justify such a package.
>
> There is no need for a math library for the Emacs editor,
> which is extensively used by programmers and tech people all
> over the world. Because math is not a general need.
>
> That people write their own Elisp because they don't have
> official software to do even trivial things doesn't make the
> need real.
>
> Yeah, what can I say? Just unbelievable.
>
> --
> underground experts united
> https://dataswamp.org/~incal

He simply needs to be convinced of actual tasks where mathematical operations would
be fundamental.  Even simple calculations within Emacs Lisp scripts can significantly
benefit from a dedicated math library.  But there has to be a practical application
that would be flexible and extensible to other things people would want to do.

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> He simply needs to be convinced of actual tasks where
> mathematical operations would be fundamental. Even simple
> calculations within Emacs Lisp scripts can significantly
> benefit from a dedicated math library. But there has to be
> a practical application that would be flexible and
> extensible to other things people would want to do.

There could be but don't have to be any application included.

It could just be a library offering first all basic functions
so people didn't have to write them themselves.

After that: more advanced stuff.

Applications to math are everywhere so no worries people will
find them.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

I'm not gonna sabotage the possibility of this happening by
continue to discuss this, the way it has started it is better
to just leave it be.

I'm not gonna be the one to do it anyway, but I'd be happy to
offer any assistance I can if someone gets some
momentum going.

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

I have been thinking of an Automated Schedule Generator for conferences, using
permutations to explore possible schedules that meet constraints.

> Sent: Saturday, July 20, 2024 at 1:22 AM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> I'm not gonna sabotage the possibility of this happening by
> continue to discuss this, the way it has started it is better
> to just leave it be.
>
> I'm not gonna be the one to do it anyway, but I'd be happy to
> offer any assistance I can if someone gets some
> momentum going.
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Stefan Kangas]

Christopher Dimech <dimech@gmx.com> writes:

> I have been thinking of an Automated Schedule Generator for conferences, using
> permutations to explore possible schedules that meet constraints.

Thanks, but what does this have to do with Emacs development?

May I suggest that you discuss this issue on another mailing list?

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Saturday, July 20, 2024 at 4:15 AM
> From: "Stefan Kangas" <stefankangas@gmail.com>
> To: "Christopher Dimech" <dimech@gmx.com>, "Emanuel Berg" <incal@dataswamp.org>
> Cc: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech <dimech@gmx.com> writes:
>
> > I have been thinking of an Automated Schedule Generator for conferences, using
> > permutations to explore possible schedules that meet constraints.
>
> Thanks, but what does this have to do with Emacs development?

Possible tool for Org-Mode for instance that would require a mathematical library
for emacs.  This is on topic to the suggestion here, which I support.

Re: Adding a generic mathematical library

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Have we considered to write a bit of glue code that would make it easier
  > and more convenient to use the things defined in Calc?  Why invest all of
  > the time again?

If a calc function includes a substanial amount of code
to do a specific kind of calculation, we could extract that part
as a separate function and put it in a suitable place.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Saturday, July 20, 2024 at 4:16 AM
> From: "Richard Stallman" <rms@gnu.org>
> To: "Michael Heerdegen" <michael_heerdegen@web.de>
> Cc: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Have we considered to write a bit of glue code that would make it easier
>   > and more convenient to use the things defined in Calc?  Why invest all of
>   > the time again?
>
> If a calc function includes a substanial amount of code
> to do a specific kind of calculation, we could extract that part
> as a separate function and put it in a suitable place.

That would be a good starting point.  At least we would know that the functions
are indeed currently useful and being used.  We can find any mathematical
implementations in the emacs code, and extract some good ones to an initial
emacs mathematical library emacs can start to access.

Re: Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 133 bytes --]

I have fixed review issues and improve package documentation. Philip, please see one more time.

Best regards,
Sergey Kostyaev


[-- Attachment #2: Type: text/html, Size: 418 bytes --]

Re: Add elisa to GNU ELPA

[Philip Kaludercic]

Sergey Kostyaev <sskostyaev@gmail.com> writes:

> I have fixed review issues and improve package documentation. Philip, please see one more time.

I admit I was lazy; there are a few SQL strings that could still
upcased.

A general question I have is why you depend on Emacs 29.2?  Package-lint
indicates that you don't need anything before 28.1, with the exception
of the SQLite integration, which would mean you could reduce the minimal
version to 29.1.

Otherwise I guess it looks fine, I'll add it to elpa.git.

> Best regards,
> Sergey Kostyaev
>

-- 
	Philip Kaludercic on peregrine

Re: Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 810 bytes --]

Allowlist of sqlite extensions. vector0 and vss0 was added to allowlist in 29.2 release.


17 июля 2024 г. 22:12:44 UTC, Philip Kaludercic <philipk@posteo.net> пишет:
>Sergey Kostyaev <sskostyaev@gmail.com> writes:
>
>> I have fixed review issues and improve package documentation. Philip, please see one more time.
>
>I admit I was lazy; there are a few SQL strings that could still
>upcased.
>
>A general question I have is why you depend on Emacs 29.2?  Package-lint
>indicates that you don't need anything before 28.1, with the exception
>of the SQLite integration, which would mean you could reduce the minimal
>version to 29.1.
>
>Otherwise I guess it looks fine, I'll add it to elpa.git.
>
>> Best regards,
>> Sergey Kostyaev
>>
>
>-- 
>	Philip Kaludercic on peregrine

[-- Attachment #2: Type: text/html, Size: 1455 bytes --]

Re: Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 810 bytes --]

All sql strings upcased.

> 18 июля 2024 г., в 05:12, Philip Kaludercic <philipk@posteo.net> написал(а):
> 
> Sergey Kostyaev <sskostyaev@gmail.com> writes:
> 
>> I have fixed review issues and improve package documentation. Philip, please see one more time.
> 
> I admit I was lazy; there are a few SQL strings that could still
> upcased.
> 
> A general question I have is why you depend on Emacs 29.2?  Package-lint
> indicates that you don't need anything before 28.1, with the exception
> of the SQLite integration, which would mean you could reduce the minimal
> version to 29.1.
> 
> Otherwise I guess it looks fine, I'll add it to elpa.git.
> 
>> Best regards,
>> Sergey Kostyaev
>> 
> 
> -- 
> 	Philip Kaludercic on peregrine

Best regards,
Sergey Kostyaev


[-- Attachment #2: Type: text/html, Size: 1346 bytes --]

Adding a generic mathematical library

[Kepa]

>"We should ask the community what tools they would like to use,
>whether combinatorics, random numbers (Gaussian, Bernoulli, Binomial), ...

>May I suggest "Units and Conversions".  "Symbolic Simplification"
>could also be a candidate."

Units are a differentiating element. I have found very few languages that use units and I consider myself lucky to have calc. Calc seems to have bugs when working with units though:
https://lists.gnu.org/archive/html/help-gnu-emacs/2024-02/msg00144.html

The embedded mode is also amazing: being able to write text, mixed with mathematical operations, variables and units.

Best regards

Re: Adding a generic mathematical library

[Emanuel Berg]

Kepa wrote:

>> "We should ask the community what tools they would like to
>> use, whether combinatorics, random numbers (Gaussian,
>> Bernoulli, Binomial), ...
>>
>> May I suggest "Units and Conversions". "Symbolic
>> Simplification" could also be a candidate."
>
> Units are a differentiating element. I have found very few
> languages that use units and I consider myself lucky to have
> calc. Calc seems to have bugs when working with units
> though:
> https://lists.gnu.org/archive/html/help-gnu-emacs/2024-02/msg00144.html
>
> The embedded mode is also amazing: being able to write text,
> mixed with mathematical operations, variables and units.

Calc was brought into the discussion as a way of opposing the
idea of a library. To say, we don't need it, because we
already have it.

Calc is doing it like it is done everywhere else in Emacs IINM.

This way of doing things isn't good, to me it is pretty much
_the opposite_ to having libraries. But it isn't calc's fault
that they have adapted to this policy/model/reality, I mean
what else to do?

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Thursday, July 25, 2024 at 4:19 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Kepa wrote:
>
> >> "We should ask the community what tools they would like to
> >> use, whether combinatorics, random numbers (Gaussian,
> >> Bernoulli, Binomial), ...
> >>
> >> May I suggest "Units and Conversions". "Symbolic
> >> Simplification" could also be a candidate."
> >
> > Units are a differentiating element. I have found very few
> > languages that use units and I consider myself lucky to have
> > calc. Calc seems to have bugs when working with units
> > though:
> > https://lists.gnu.org/archive/html/help-gnu-emacs/2024-02/msg00144.html
> >
> > The embedded mode is also amazing: being able to write text,
> > mixed with mathematical operations, variables and units.
>
> Calc was brought into the discussion as a way of opposing the
> idea of a library. To say, we don't need it, because we
> already have it.

Incorrect, we generally do not accept new packages that substantially overlap
with existing GNU packages.  The procedure is for GNU to have a given package
to do a given job, and people in that area to contribute to and improve that
package, working together, instead of having many packages that each do different
parts of a job, each developed on its own.

Similarly, a small program often fits better as part of an existing package than
being a new package of its own.

> Calc is doing it like it is done everywhere else in Emacs IINM.
>
> This way of doing things isn't good, to me it is pretty much
> _the opposite_ to having libraries. But it isn't calc's fault
> that they have adapted to this policy/model/reality, I mean
> what else to do?

You can get good results if you discuss the possibilities and capabilities
of a library with the developers of calc.  And do some work with them.
You will be doing a generic mathematical library anyway.  Is there a good
reason why an association with them is so terrible ?  Have you worked with
them before ?

Re: Adding a generic mathematical library

[Stefan Kangas]

Christopher Dimech <dimech@gmx.com> writes:

> You can get good results if you discuss the possibilities and capabilities
> of a library with the developers of calc.  And do some work with them.
> You will be doing a generic mathematical library anyway.  Is there a good
> reason why an association with them is so terrible ?  Have you worked with
> them before ?

FYI, we basically don't have any active "calc developers" to talk to.
There are some people that occasionally contribute here and there
though, see "git log lisp/calc".

Re: Adding a generic mathematical library

[T.V Raman]

That said, we might do well to build up some understanding of the calc
code base, it's been around a long time and it's rock solid.

Another approach to building up Emacs' math skills might be to interface
more tightly with Sagemath -- the current state of Emacs interaction
with Sage is already quite good.

-- 

Thanks,

--Raman(I Search, I Find, I Misplace, I Research)
♉ Id: kg:/m/0285kf1  🦮
LOC: https://id.loc.gov/authorities/names/n97059241.html

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Friday, July 26, 2024 at 2:20 AM
> From: "Stefan Kangas" <stefankangas@gmail.com>
> To: "Christopher Dimech" <dimech@gmx.com>, "Emanuel Berg" <incal@dataswamp.org>
> Cc: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech <dimech@gmx.com> writes:
>
> > You can get good results if you discuss the possibilities and capabilities
> > of a library with the developers of calc.  And do some work with them.
> > You will be doing a generic mathematical library anyway.  Is there a good
> > reason why an association with them is so terrible ?  Have you worked with
> > them before ?
>
> FYI, we basically don't have any active "calc developers" to talk to.
> There are some people that occasionally contribute here and there
> though, see "git log lisp/calc".

The suggestion to have Emanuel handle the Calc package in Emacs raises
several considerations about the responsibilities and experiences needed
to maintain such a tool, as well as the broader context of contributing
to and maintaining GNU packages.

Maintaining a GNU package like Calc requires a deep understanding of both
the package itself and the broader GNU/Emacs ecosystem.  This includes
familiarity with the expectations of the Emacs user community.

There might be concerns about readiness because potential maintainers
often gain experience by working closely with current maintainers.
This apprenticeship-like model helps them understand the responsibilities,
workflows, and expectations involved in maintaining a GNU package.

The role of a maintainer includes making difficult decisions about the
future direction of the package.  A trial period could be considered.

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

>> FYI, we basically don't have any active "calc developers"
>> to talk to. There are some people that occasionally
>> contribute here and there though, see "git log lisp/calc".
>
> The suggestion to have Emanuel handle the Calc package in
> Emacs raises several considerations about the
> responsibilities and experiences needed to maintain such
> a tool, as well as the broader context of contributing to
> and maintaining GNU packages.

??? What? :P

I'm not maintaining anything and especially not Calc which
I don't even use.

I was only offering my services for a math library, if that
was indicative of a realization and shared view that we should
have that and other such libraries instead of the model we
have now. Well, I have just described it.

Because then it could be an alternative, it could be the
right-way-from-now-on never minding the past, it could even in
time be a complete replacement.

But people are opposed to this, which is okay. Unfortunately,
people didn't say that but instead started to talk about _me_.
Now I'm ready for that (see above, I didn't even get angry).
But all in all, now I'm really not up to do anything about
this since there is no agreement why it should happen in the
first place.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Stefan Kangas wrote:

>> You can get good results if you discuss the possibilities
>> and capabilities of a library with the developers of calc.
>> And do some work with them. You will be doing a generic
>> mathematical library anyway. Is there a good reason why an
>> association with them is so terrible ? Have you worked with
>> them before ?
>
> FYI, we basically don't have any active "calc developers" to
> talk to. There are some people that occasionally contribute
> here and there though, see "git log lisp/calc".

Okay, that bad.

Well, this is another example why that model isn't good.
When people stop working on a program, no one is working on
the libraries anymore either.

Solution: Decouple.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

>> Calc was brought into the discussion as a way of opposing
>> the idea of a library. To say, we don't need it, because we
>> already have it.
>
> Incorrect, we generally do not accept new packages that
> substantially overlap with existing GNU packages.

That means the more you do this "program as a library", the
less you can have a common library for many programs.

So the solution to the problem cannot be done, because of what
is the result of not having it arranged soundly in the first
place. When it would have been super-easy. "Here, here is
math.el and string.el, use it and put new stuff in it!"
Instead, that stuff has ended up all over Emacs.

But it is what it is, anyone thinking that model is good, go
ahead and think so.

"We don't do generic libraries. We think it is normal that
a program that has nothing to do with ERC, Gnus and Calc to do
generic operations on strings and numbers require files that
are part of all these programs."

Instead of blaming why the problem cannot be fixed on others
who have nothing to do with that idea.

> The procedure is for GNU to have a given package to do
> a given job, and people in that area to contribute to and
> improve that package, working together, instead of having
> many packages that each do different parts of a job, each
> developed on its own.

In practice "program as a library" means the opposite.
Every program does a little of everything. Instead of adding
to the common library, what is missing is added to the
program. When something is needed by another program, instead
of bringing it from library, it brings in the first program.
Until there is something additionally needed, then that is
added, again not to the library but to the second program.
And so on.

> Is there a good reason why an association with them is
> so terrible?

Oh, absolutely not! Like I said, it is like this all over
the place.

Here, check out the string library! Only those files are all
over Emacs.

(shortdoc-display-group "string")

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Friday, July 26, 2024 at 2:24 AM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech wrote:
>
> >> Calc was brought into the discussion as a way of opposing
> >> the idea of a library. To say, we don't need it, because we
> >> already have it.
> >
> > Incorrect, we generally do not accept new packages that
> > substantially overlap with existing GNU packages.
>
> That means the more you do this "program as a library", the
> less you can have a common library for many programs.
>
> So the solution to the problem cannot be done, because of what
> is the result of not having it arranged soundly in the first
> place. When it would have been super-easy. "Here, here is
> math.el and string.el, use it and put new stuff in it!"
> Instead, that stuff has ended up all over Emacs.
>
> But it is what it is, anyone thinking that model is good, go
> ahead and think so.
>
> "We don't do generic libraries. We think it is normal that
> a program that has nothing to do with ERC, Gnus and Calc to do
> generic operations on strings and numbers require files that
> are part of all these programs."
>
> Instead of blaming why the problem cannot be fixed on others
> who have nothing to do with that idea.
>
> > The procedure is for GNU to have a given package to do
> > a given job, and people in that area to contribute to and
> > improve that package, working together, instead of having
> > many packages that each do different parts of a job, each
> > developed on its own.
>
> In practice "program as a library" means the opposite.
> Every program does a little of everything. Instead of adding
> to the common library, what is missing is added to the
> program. When something is needed by another program, instead
> of bringing it from library, it brings in the first program.
> Until there is something additionally needed, then that is
> added, again not to the library but to the second program.
> And so on.

What you are suggesting is to have an improved design where the functionality
of calc can be used elsewhere.  We can still have the tool and the foundational
code to be calc.  We do not need specifically a different name for a library.
Many people know about calc but would not be aware of some new library that
could be popping up.

The GNU Project, and by extension, the Emacs development community, has a
unique culture and set of practices that distinguish it from other software
development projects.  Understanding these nuances is crucial for anyone
involved, especially those considering taking on a maintainer role.

> > Is there a good reason why an association with them is
> > so terrible?
>
> Oh, absolutely not! Like I said, it is like this all over
> the place.
>
> Here, check out the string library! Only those files are all
> over Emacs.
>
> (shortdoc-display-group "string")
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> What you are suggesting is to have an improved design where
> the functionality of calc can be used elsewhere. We can
> still have the tool and the foundational code to be calc.
> We do not need specifically a different name for a library.
> Many people know about calc but would not be aware of some
> new library that could be popping up.

Again, do it right from the beginning all is good.

The more you do it some other way, the more difficult it will
be to get right.

Sure. Alltho this wouldn't be _that_ difficult to do, at least
to get started. There are ways to do it that will make the
transition smoothly.

But the policy would have to change, yes.

> The GNU Project, and by extension, the Emacs development
> community, has a unique culture and set of practices that
> distinguish it from other software development projects.
> Understanding these nuances is crucial for anyone involved,
> especially those considering taking on a maintainer role.

You can talk how much you want about people, I don't care.

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Shouran Ma]

Glad to see that there is an active discussion on Emacs Math library.

0. Readiness of becoming one of the maintainers

> Sent: Thursday, July 25, 2024 at 17:34 +0200
> From: "Christopher Dimech" <dimech@gmx.com>
> There might be concerns about readiness because potential maintainers
> often gain experience by working closely with current maintainers.
> This apprenticeship-like model helps them understand the responsibilities,
> workflows, and expectations involved in maintaining a GNU package.

I would like to be one of a maintainers.



1. Introduction

I looked inside the Calc subsystem in the past days. In this mail I will
summarize several wishing features and potential problems during
developing.

Calc provides several types, c.f. calc-prog.el the "calcFunc-typeof"
function. I will talk about the members in this function.

About my experiments involve Common-Lisp (CL), my settings is, connect to
sbcl and open lisp-mode for a buffer, then put the cursor after the
expression and hit "C-x C-e" to view the result at *Messages* buffer.



2. Wish-list: Extend the basic number types: frac & cplx

Although basic number types are even not a part of a math library, but
that's crucial that do not set a barriers for the users especially at
such fundamental things. For example, we can include the rational type
(frac) and complex type (cplx) as the basic Elisp number types.
Especially, CL has implemented this feature.

Tasks would involve how to INTERPRET the slash "/" and small groups of
numbers, , e.g., in CL,
- (type-of 5/2)          ; => TATIO
- (type-of 5 / 2)        ; => invalid number of arguments: 3
- (type-of #C(1.0 1.0))  ; => (COMPLEX (SINGLE-FLOAT 1.0 1.0))
- (type-of #C(5/2 1/5))  ; => (COMPLEX (RATIONAL 1/5 5/2))

We should figure out a unified way to represent the rational and complex.

After that, we should implement eGCD and overload the "+ - * % < >" to
include the rational number arithmetics. This would be a huge workload
to expand the number group.

For eGCD we can look into NTL library for more efficient impl.



3. Wish-list: Pretty output format for integers and rationals

In Emacs, hit "M-:" to "Eval: " 100, we get
100 (#o144, #x64, ?d)
In CL, put the cursor after 100 and "C-x C-e" we get
=> 100 (7 bits, #x64, #o144, #b1100100)
Similarly for the number "5/2", we get
=> 5/2 (2.5)
But for the number "5 / 2" (separated by space) we get
=> 2 (2 bits, #x2, #o2, #b10)

It's obviously that CL's style is more friendly to the user. We can
improve the representation of the Elisp result to:
=> 100 (7 bits, #x64, #o144, #b1100100, ?d)



4. Problem of representing the vectors/lists and the corresponding funcs

In Calc, the vector types are prefixed with a "vec" symbol in this way:
(vec 1 2 3)
so that in Calc, things in vectors are indexed from 1, not from 0 (like
many other languages). This is a bad way to index a vector and would
cause a nightmare for the potential developers who will use Emacs-Math.

Besides (vec 1 2 3), there are many other ways to represent an array-like
object, c.f.
https://www.gnu.org/software/emacs/manual/html_node/elisp/Sequences-Arrays-Vectors.html
- cons/list type: (1 2 3 4 5)
- vector type: [1 2 3 4 5]

However, WE DON'T EVEN HAVE A UNIFIED WAY TO REPRESENTING A VECTOR.

As for my knowledge, vector type in Emacs support random access while
list/cons type is accessed by "link-list" search way. However, there are
many builtin functions to handle the list type, but too few to handle
the vector type. If you try to rewrite "calcFunc-diag" (in calc-vec.el)
you would know the feeling.

So if we want to use the builtin vector type to represent vectors and
handle vectors, we must first of all complete the corresponding
vector-handling functions, for example, "slice".



5. Functions in GNU C library, and power (expt)

> Sent: Sunday, July 21, 2024 at 17:33 +0200
> From: "Eli Zaretskii" <eliz@gnu.org>
>
> I don't know what math-sin-cos-raw is about, but as for math-hypot,
> there's no need to implement anything, since this function is standard
> in any C libm, and we can easily expose it to Lisp if we need to.

> Send: Sunday, July 21, 2024 at 14:46 +0200
> From: "Emanuel Berg" <incal@dataswamp.org>

Here is the full list of the functions from GNU C library:
https://www.gnu.org/software/emacs/manual/html_node/elisp/Numbers.html

If new data types (especially the complex) are introduced, functions in
this list should be carefully overloaded to the complex number case.

Besides, where is the origin of the "expt" for the power? (CL use this
name, though).

Personally, I would suggest to add "powm" or "exptm" or such name, to
implement the power-mod.



6. Problem of inclusion the statistical functions

> Sent: Sunday, July 21, 2024 at 09:27 +0200
> From: "Christopher Dimech" <dimech@gmx.com>
>
> We should ask the community what tools they would like to use,
> whether combinatorics, random numbers (Gaussian, Bernoulli, Binomial),
> ...

> Sent: Sunday, July 21, 2024 at 17:49 +0200
> From: "Eli Zaretskii" <eliz@gnu.org>
>
> The original issue was not about Calc, it was about a standalone math
> library.  For that, we should first define the scope.  One possibility
> is to expose to Lisp everything in a typical libm, but is that needed?
> Another possibility is to provide a statistical library (average,
> median, standard deviation, t-Student, F-test, chi-square, etc.) --
> but do we need this?  Etc. etc. -- "math library" can be interpreted
> in many different ways.

Mean and variance are the most frequently used statistical functions,
but before this, we should solve bullet 4, after that, we can then
considering to generate an VECTOR that subjected to Gaussian, Bernoulli,
Binomial distribution.



7. Symbolic functions and suggestion of inclusion math library as a
minor mode

> Sent: Sunday, July 21, 2024 at 09:27 +0200
> From: "Christopher Dimech" <dimech@gmx.com>
>
> May I suggest "Units and Conversions".  "Symbolic Simplification"
> could also be a candidate.

From a user aspect, If I want to use symbolic calculus, I would do so:
- "M-x" to open "math-symbolic-mode"
- Input "(sin x)" will just return the expression "(sin x)", not void-variable
- Input "(math-read-expr "(x+1)*(x+2)")" gives "(* (+ x 1) (+ x 2))"
instead of "(* (+ (var x var-x) 1) (+ (var x var-x) 2))"

That means, in such "math-symbolic-mode" (the minor mode), symbol "x" is
just symbol "x", not "void-variable", so that things would look prettier
in the returned expression instead of "(var x var-x)".



8. Summary

The first four bullets are not even involve the affairs of implementing
a math library or transferring the codes from Calc. But that would be a
great start for the further work if we implementing the mentioned
features.


-- 
Best Regards,
Shouran MA

Re: Adding a generic mathematical library

[Emanuel Berg]

Shouran Ma wrote:

> I would like to be one of a maintainers.

Excellent! \o/

See if you can attract 1-2 partners as well :)

> However, WE DON'T EVEN HAVE A UNIFIED WAY TO REPRESENTING
> A VECTOR.

Yeah, what do you say about that? Just crazy. However it used
to be even worse, when we didn't even have clocks! I remember
some guys were using unit circles instead.

GLHF

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Shouran Ma]

> Sent: Saturday, July 27, 2024 at 16:57 +0200
> From: "Shouran Ma" <shouran.ma@gmail.com>
>
> However, WE DON'T EVEN HAVE A UNIFIED WAY TO REPRESENTING A VECTOR.

I need to elaborate this statement, otherwise it cause confusion to
others.

Elisp provides "List" and "Vector" as an array of numbers:
https://www.gnu.org/software/emacs/manual/html_node/elisp/Sequences-Arrays-Vectors.html

To express an array of numbers, a developer would either use two of them
- (setq x '(1 2 3 4))  ; (type-of x) => cons
- (setq x [1 2 3 4])   ; (type-of x) => vector

"cons" (or "list") is the first thing that everybody would choose, for
example, in Sergey's elisa:
https://github.com/s-kostyaev/elisa/blob/main/elisa.el
In the function "elisa--distances", Sergey puts "head" and "(car tail)"
as arguments to "elisa-cosine-distance", this means Sergey uses
cons/list to organize array of numbers.

However, Elisp also has "array" types:
https://www.gnu.org/software/emacs/manual/html_node/elisp/Arrays.html
> An array object has slots that hold a number of other Lisp objects,
> called the elements of the array. Any element of an array MAY BE
> ACCESSED IN CONSTANT TIME. In contrast, the time to access an element
> of a list is proportional to the position of that element in the list.

cons/list is something like "linked list" which is not accessed in const
time, but vector support this, e.g. it takes more time to access the
last element if we use cons/list, but less time if use vector.

However, Elisp provides too few functions on operating the vector type,
for example, to slice a vector like the way in python: array[2:4].

So my saying "don't even have a unified way to representing a vector", I
mean, to organize/represent a MATHEMATICAL vector
- I would prefer to choose the object that support const time access,
i.e. the BUILTIN vector type, over the cons/list.
- However, the BUILTIN vector type supports quite a few functions, which
forces me to change my mind to organize/represent the MATHEMATICAL
vectors by cons/list.

This is the dilemma during developing my own math libraries.

Besides, in the Calc subroutine, the MATHEMATICAL vectors are
represented by cons/list in this way
(vec 3 5 2 1 0)  ; not (3 5 2 1 0) or [3 5 2 1 0]
i.e. a symbol "vec" is placed at the beginning of the list, in order to
simplify the predication. But this leads to the indexing problem, that
elements are indexed since 1, not 0. Such an indexing scheme would cause
a nightmare to the developer who want to use our math subroutine to do
further development.

In summary, if the builtin vector type is preferred way to represent the
MATHEMATICAL vector, we should first of all complete the fundamental
vector operations, like vector slicing etc. And of course, we should
take a look at how sbcl
https://git.code.sf.net/p/sbcl/sbcl
organize their vector and their vector functions.

From my side, I don't want to touch the vector things so far, but instead
the rational number (the bullet 2 & 3 in my previous mail).

Finally,
> Sent: Saturday, July 27, 2024 at 17:49 +0200
> From: "Emanuel Berg" <incal@dataswamp.org>
>
> However it used to be even worse, when we didn't even have clocks!

If the "clock" you said is the clock to measure how long a function
runs, we have, it is "benchmark-run".


-- 
Best Regards,
Shouran Ma

Re: Adding a generic mathematical library

[Christopher Dimech]

> Sent: Sunday, July 28, 2024 at 7:07 AM
> From: "Shouran Ma" <shouran.ma@gmail.com>
> To: "Emanuel Berg" <incal@dataswamp.org>
> Cc: emacs-devel@gnu.org, "Sergey Kostyaev" <sskostyaev@gmail.com>, "Philip Kaludercic" <philipk@posteo.net>
> Subject: Re: Adding a generic mathematical library
>
>
> > Sent: Saturday, July 27, 2024 at 16:57 +0200
> > From: "Shouran Ma" <shouran.ma@gmail.com>
> >
> > However, WE DON'T EVEN HAVE A UNIFIED WAY TO REPRESENTING A VECTOR.
>
> I need to elaborate this statement, otherwise it cause confusion to
> others.
>
> Elisp provides "List" and "Vector" as an array of numbers:
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Sequences-Arrays-Vectors.html
>
> To express an array of numbers, a developer would either use two of them
> - (setq x '(1 2 3 4))  ; (type-of x) => cons
> - (setq x [1 2 3 4])   ; (type-of x) => vector
>
> "cons" (or "list") is the first thing that everybody would choose, for
> example, in Sergey's elisa:
> https://github.com/s-kostyaev/elisa/blob/main/elisa.el
> In the function "elisa--distances", Sergey puts "head" and "(car tail)"
> as arguments to "elisa-cosine-distance", this means Sergey uses
> cons/list to organize array of numbers.
>
> However, Elisp also has "array" types:
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Arrays.html
> > An array object has slots that hold a number of other Lisp objects,
> > called the elements of the array. Any element of an array MAY BE
> > ACCESSED IN CONSTANT TIME. In contrast, the time to access an element
> > of a list is proportional to the position of that element in the list.
>
> cons/list is something like "linked list" which is not accessed in const
> time, but vector support this, e.g. it takes more time to access the
> last element if we use cons/list, but less time if use vector.
>
> However, Elisp provides too few functions on operating the vector type,
> for example, to slice a vector like the way in python: array[2:4].
>
> So my saying "don't even have a unified way to representing a vector", I
> mean, to organize/represent a MATHEMATICAL vector
> - I would prefer to choose the object that support const time access,
> i.e. the BUILTIN vector type, over the cons/list.
> - However, the BUILTIN vector type supports quite a few functions, which
> forces me to change my mind to organize/represent the MATHEMATICAL
> vectors by cons/list.

You do not need a unified way to representing a vector.  You only need
the tool to work conveniently with vectors. Having two or three ways to
do it, is not a bad thing, unless the number of ways are excessive.

> This is the dilemma during developing my own math libraries.

If you support both, there will be no dilemma.


> Besides, in the Calc subroutine, the MATHEMATICAL vectors are
> represented by cons/list in this way
> (vec 3 5 2 1 0)  ; not (3 5 2 1 0) or [3 5 2 1 0]
> i.e. a symbol "vec" is placed at the beginning of the list, in order to
> simplify the predication. But this leads to the indexing problem, that
> elements are indexed since 1, not 0. Such an indexing scheme would cause
> a nightmare to the developer who want to use our math subroutine to do
> further development.
>
> In summary, if the builtin vector type is preferred way to represent the
> MATHEMATICAL vector, we should first of all complete the fundamental
> vector operations, like vector slicing etc. And of course, we should
> take a look at how sbcl
> https://git.code.sf.net/p/sbcl/sbcl
> organize their vector and their vector functions.

The focus for the builtin vector type should first be on the ways it is
currently used.  Support those first, then look at some other things
that can be added.

Making a complete thing could waste precious time for minimal return.
Meaning, it is more important to support the evolution of the code
through its design, than anything else.  We have many implementers
but very few focused designers.

> From my side, I don't want to touch the vector things so far, but instead
> the rational number (the bullet 2 & 3 in my previous mail).
>
> Finally,
> > Sent: Saturday, July 27, 2024 at 17:49 +0200
> > From: "Emanuel Berg" <incal@dataswamp.org>
> >
> > However it used to be even worse, when we didn't even have clocks!
>
> If the "clock" you said is the clock to measure how long a function
> runs, we have, it is "benchmark-run".
>
>
> --
> Best Regards,
> Shouran Ma
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> You do not need a unified way to representing a vector.
> You only need the tool to work conveniently with vectors.
> Having two or three ways to do it, is not a bad thing,
> unless the number of ways are excessive.

For the purpose of the library normalizing that can be
a good idea.

It is not a bad thing to have several ways to do things, but
in Lisp they also tend to do several things each depending on
context. So it is a m*n curve. A lot of Lisp project gets out
of hand with complexity, this is perhaps not the main reason
but it is one of them.

So if one can stick to a subset methods and a consistent style
at least for one file it is good :)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Christopher Dimech]

> Sent: Sunday, July 28, 2024 at 12:29 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech wrote:
>
> > You do not need a unified way to representing a vector.
> > You only need the tool to work conveniently with vectors.
> > Having two or three ways to do it, is not a bad thing,
> > unless the number of ways are excessive.
>
> For the purpose of the library normalizing that can be
> a good idea.
>
> It is not a bad thing to have several ways to do things, but
> in Lisp they also tend to do several things each depending on
> context. So it is a m*n curve. A lot of Lisp project gets out
> of hand with complexity, this is perhaps not the main reason
> but it is one of them.

If things get out of hand, it is a problem.

> So if one can stick to a subset methods and a consistent style
> at least for one file it is good :)

Depends on what parts need work.  Do not plan to solve all that in one go.

> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> If things get out of hand, it is a problem.

It is the z-dimension or hyperpattern aspect of Lisp.

Observe Python:

if threads.is_set():
    self.send('QUIT')
    self.socket.close()
    return

And Lisp:

(if (< fst otr)
    (push (or (and read (apply read nil))
              (buffer-substring-no-properties fst otr))
          strs)
  (setq mov nil))

>> So if one can stick to a subset methods and a consistent
>> style at least for one file it is good :)
>
> Depends on what parts need work. Do not plan to solve all
> that in one go.

Well, I don't look like a bug fixer to anyone.

BTW it sounds like you didn't have a look at the Calc code :)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Christopher Dimech]

> Sent: Sunday, July 28, 2024 at 11:39 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech wrote:
>
> > If things get out of hand, it is a problem.
>
> It is the z-dimension or hyperpattern aspect of Lisp.

I suspect it is unusual to have vec as in (vec 1 2 3)
for other list types.

> Observe Python:
>
> if threads.is_set():
>     self.send('QUIT')
>     self.socket.close()
>     return
>
> And Lisp:
>
> (if (< fst otr)
>     (push (or (and read (apply read nil))
>               (buffer-substring-no-properties fst otr))
>           strs)
>   (setq mov nil))
>
> >> So if one can stick to a subset methods and a consistent
> >> style at least for one file it is good :)
> >
> > Depends on what parts need work. Do not plan to solve all
> > that in one go.
>
> Well, I don't look like a bug fixer to anyone.
>
> BTW it sounds like you didn't have a look at the Calc code :)

I did actually, but likely not as deep as to what you want to do.
Nevertheless, I can understand your points, and am getting convinced.

Does functionality from calc happen often in other parts of emacs ?

I suggest getting some changes to calc, then try to get some appropriate
package and employ solely calc on it, to see where things go.

> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> I did actually, but likely not as deep as to what you want
> to do.

Ah, I'm not doing anything about this. Lost the appetite. And,
wrote just enough Elisp on another project

> Does functionality from calc happen often in other parts of
> emacs ?

$ grep --color=never -i '\bcalc\b' lisp/**/*.el | grep -v '^lisp/calc/' | wc -l
516

> I suggest getting some changes to calc, then try to get some
> appropriate package and employ solely calc on it, to see
> where things go.

I don't think it has to be changed. Rather, what is missing?
People who are into this have a good way forward there.

Asking of them to add it to a local Calc regular Elisp file -
Calc that they don't use probably and isn't even maintained -
yeah, don't do that.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

It seems to me that writing a generic mathematical library is not very hard,
but writing a good one requires a lot of experience and understanding
of how suc a library could be used.  The obvious solutions are clunky,
but what makes for a good kibrary is a solution that is elegant.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Adding a generic mathematical library

[Emanuel Berg]

Richard Stallman wrote:

> It seems to me that writing a generic mathematical library
> is not very hard, but writing a good one requires a lot of
> experience and understanding of how suc a library could be
> used. The obvious solutions are clunky, but what makes for
> a good kibrary is a solution that is elegant.

On the contrary, I think our library would have been great.

We could have taken a look at other programming languages and
see what they have, for example.

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Tuesday, July 30, 2024 at 2:53 PM
> From: "Richard Stallman" <rms@gnu.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> It seems to me that writing a generic mathematical library is not very hard,
> but writing a good one requires a lot of experience and understanding
> of how suc a library could be used.  The obvious solutions are clunky,
> but what makes for a good kibrary is a solution that is elegant.

The distinction between a good library (easy to understand and maintain)
and a merely functional one.  How things are named, how parameters are
structured, and how different components interact, is what determines its
utility and adoption.  We need to see an actual application with it.

> --
> Dr Richard Stallman (https://stallman.org)
> Chief GNUisance of the GNU Project (https://gnu.org)
> Founder, Free Software Foundation (https://fsf.org)
> Internet Hall-of-Famer (https://internethalloffame.org)
>
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

>> It seems to me that writing a generic mathematical library
>> is not very hard, but writing a good one requires a lot of
>> experience and understanding of how suc a library could be
>> used. The obvious solutions are clunky, but what makes for
>> a good kibrary is a solution that is elegant.
>
> The distinction between a good library (easy to understand
> and maintain) and a merely functional one. How things are
> named, how parameters are structured, and how different
> components interact, is what determines its utility and
> adoption. We need to see an actual application with it.

The distinction is between discussing things and pretending to
do so.

Remember, this is not obvious to many, maybe most people.

When they later realize they have taken part in a mock
discussion that didn't have a real purpose - they are not
going to respond well to that.

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Tuesday, July 30, 2024 at 7:20 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech wrote:
>
> >> It seems to me that writing a generic mathematical library
> >> is not very hard, but writing a good one requires a lot of
> >> experience and understanding of how suc a library could be
> >> used. The obvious solutions are clunky, but what makes for
> >> a good kibrary is a solution that is elegant.
> >
> > The distinction between a good library (easy to understand
> > and maintain) and a merely functional one. How things are
> > named, how parameters are structured, and how different
> > components interact, is what determines its utility and
> > adoption. We need to see an actual application with it.
>
> The distinction is between discussing things and pretending to
> do so.
>
> Remember, this is not obvious to many, maybe most people.
>
> When they later realize they have taken part in a mock
> discussion that didn't have a real purpose - they are not
> going to respond well to that.

How many mathematical libraries have you done for use by mathematicians
and scientists ?  If it was so easy, calc would have been great for you.
And perhaps later, others will deride your implementation.  Take what you
can from comments.

> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> How many mathematical libraries have you done for use by
> mathematicians and scientists?

First they decide not to do it.

Then everyone is expected to say and imply it is because _I_
am this and that, I did or didn't do this or that, blah blah
blah Emanuel Berg.

I don't like that since it is a waste of time and enthusiasm
for anyone involved thinking the discussion is genuine, it is
also unpleasant and prone to conflicts.

It is much better to just say "we have decided not to do it"
in post one.

But for the record I'm 100% confident I could do it and have
a great result.

> If it was so easy, calc would have been great for you.
> And perhaps later, others will deride your implementation.
> Take what you can from comments.

What about you Christopher, do you also have skills in
various fields?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Michael Heerdegen via Emacs development discussions.]

Emanuel Berg <incal@dataswamp.org> writes:

> It is much better to just say "we have decided not to do it"
> in post one.

Nobody decided anything.  Eli and others have to keep an eye on that
what people contribute fits into Emacs and doesn't cause problems in the
future.

That's why questions have been raised.  Personally I still would like to
know if you had a look at Calc, why you think that it doesn't suffice,
which purpose your library aims at, etc etc.

You started a discussion, valid questions and concerns have been raised
in response, and you just don't answer them, you ignore valid questions.
What do you expect?  That emacs-dev completely ignores any valid
question and concerns?  Or only when the author is Emanuel Berg?

Why don't you answer those few simple questions if you want to have an
explicit "Go!" from here?  Instead you are flooding this thread with
sarcasm and insinuations.  Please stop that, and please don't take
everything directly personally.


> But for the record I'm 100% confident I could do it and have
> a great result.

Which is very good but obviously, not enough for anyone to say blindly,
"yes, in any case we will integrate the result in Emacs".  How could
that be?

Apart from that, if you work on anything and it fits into Emacs or Elpa
I am 100% sure that your contribution, any contribution from you, will
be accepted.  I know the people around here for a while and nobody here
who decides tends to personal feuds.  I don't know why you think that
this is the case just because there were valid doubts and questions
raised, like would be in any other case as well.  Which - not
unimportant - also helps contributors not to waste their time.


Michael.

Re: Adding a generic mathematical library

[Emanuel Berg]

Michael Heerdegen via "Emacs development discussions." wrote:

>> It is much better to just say "we have decided not to do
>> it" in post one.
>
> Nobody decided anything. Eli and others have to keep an eye
> on that what people contribute fits into Emacs and doesn't
> cause problems in the future.
>
> You started a discussion, valid questions and concerns have
> been raised in response, and you just don't answer them, you
> ignore valid questions.

Yes, I ignore, or optimally that would be the case anyway,
anything and everything that has to do with _me_ whenever
there is, or should be, a discussion about something else.

If we, collectively, want generic libraries instead of having
everything everywhere (and math in Calc as well as local
code), then let's do it.

If we, collectively, don't want to do it for whatever reason,
that's fine - but then just don't say it is because something
that has to do with _me_.

I have nothing to do with other people not wanting to do
things, people can take their insults somewhere else.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Wed, 31 Jul 2024 23:15:21 +0200
> 
> Michael Heerdegen via "Emacs development discussions." wrote:
> 
> >> It is much better to just say "we have decided not to do
> >> it" in post one.
> >
> > Nobody decided anything. Eli and others have to keep an eye
> > on that what people contribute fits into Emacs and doesn't
> > cause problems in the future.
> >
> > You started a discussion, valid questions and concerns have
> > been raised in response, and you just don't answer them, you
> > ignore valid questions.
> 
> Yes, I ignore, or optimally that would be the case anyway,
> anything and everything that has to do with _me_ whenever
> there is, or should be, a discussion about something else.

It is not about _you_, it's about the ideas you express and the code
you post.  Ignoring questions and concerns about those because you
interpret them as personal attacks is not wise, because it doesn't
advance the discussion in any constructive direction.

> If we, collectively, want generic libraries instead of having
> everything everywhere (and math in Calc as well as local
> code), then let's do it.
> 
> If we, collectively, don't want to do it for whatever reason,
> that's fine - but then just don't say it is because something
> that has to do with _me_.

Nothing in Emacs is done "collectively", ever.  Every progress in
Emacs is always a work of a single motivated individual, sometimes
with help from a small group of others who suggest changes and debug
and fix problems in the initial code.

So for any of this to happen, someone, a single person, should show
code for such a library, and the code must exhibit a good design and
in this case also use high-quality algorithms that provide accurate
and numerically-stable results.  I've not yet seen any such designs or
implementations suggested in this discussion, with the single possible
exception of a library that provides wrappers around Calc (which I
think you didn't like).

Re: Adding a generic mathematical library

[Emanuel Berg]

Eli Zaretskii wrote:

>> Yes, I ignore, or optimally that would be the case anyway,
>> anything and everything that has to do with _me_ whenever
>> there is, or should be, a discussion about something else.
>
> It is not about _you_, it's about the ideas you express and
> the code you post.

Oh my god, it is so terrible. Emanuel Berg and his code - the
ultimate terror!

Look, by now I don't understand what you guys are talking
about so long after all happened? I don't care about this
anymore because it won't happen. Why discuss it?

The next time you don't want to do something, just say we
don't want to do it. That will be enough, I promise. I like my
ideas but am not fanatical about some math library issue, I'm
happy to drop the discussion anytime!

If it really is about me, and my supposed lack of skills, then
I look forward to meet the person who you guys think have just
the right competence to do it. You will actively look for this
person, I take it? Then let me ask, where will you start?
The local sewer?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Michael Heerdegen via "Emacs development discussions." wrote:

> Apart from that, if you work on anything and it fits into
> Emacs or Elpa I am 100% sure that your contribution [...]
> will be accepted.

Really, why didn't you say so?

Problem solved :)

I have a lot of stuff I'd like to add but one package at
a time then. This

  https://elpa.gnu.org/packages/wrap-search.html

was added in 2023 and the one I submitted a while back maybe
will appear as well then.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Christopher Dimech]

> Sent: Sunday, July 28, 2024 at 7:07 AM
> From: "Shouran Ma" <shouran.ma@gmail.com>
> To: "Emanuel Berg" <incal@dataswamp.org>
> Cc: emacs-devel@gnu.org, "Sergey Kostyaev" <sskostyaev@gmail.com>, "Philip Kaludercic" <philipk@posteo.net>
> Subject: Re: Adding a generic mathematical library
>
>
> > Sent: Saturday, July 27, 2024 at 16:57 +0200
> > From: "Shouran Ma" <shouran.ma@gmail.com>
> >
> > However, WE DON'T EVEN HAVE A UNIFIED WAY TO REPRESENTING A VECTOR.
>
> I need to elaborate this statement, otherwise it cause confusion to
> others.
>
> Elisp provides "List" and "Vector" as an array of numbers:
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Sequences-Arrays-Vectors.html
>
> To express an array of numbers, a developer would either use two of them
> - (setq x '(1 2 3 4))  ; (type-of x) => cons
> - (setq x [1 2 3 4])   ; (type-of x) => vector
>
> "cons" (or "list") is the first thing that everybody would choose, for
> example, in Sergey's elisa:
> https://github.com/s-kostyaev/elisa/blob/main/elisa.el
> In the function "elisa--distances", Sergey puts "head" and "(car tail)"
> as arguments to "elisa-cosine-distance", this means Sergey uses
> cons/list to organize array of numbers.
>
> However, Elisp also has "array" types:
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Arrays.html
> > An array object has slots that hold a number of other Lisp objects,
> > called the elements of the array. Any element of an array MAY BE
> > ACCESSED IN CONSTANT TIME. In contrast, the time to access an element
> > of a list is proportional to the position of that element in the list.
>
> cons/list is something like "linked list" which is not accessed in const
> time, but vector support this, e.g. it takes more time to access the
> last element if we use cons/list, but less time if use vector.
>
> However, Elisp provides too few functions on operating the vector type,
> for example, to slice a vector like the way in python: array[2:4].
>
> So my saying "don't even have a unified way to representing a vector", I
> mean, to organize/represent a MATHEMATICAL vector
> - I would prefer to choose the object that support const time access,
> i.e. the BUILTIN vector type, over the cons/list.
> - However, the BUILTIN vector type supports quite a few functions, which
> forces me to change my mind to organize/represent the MATHEMATICAL
> vectors by cons/list.
>
> This is the dilemma during developing my own math libraries.
>
> Besides, in the Calc subroutine, the MATHEMATICAL vectors are
> represented by cons/list in this way
> (vec 3 5 2 1 0)  ; not (3 5 2 1 0) or [3 5 2 1 0]
> i.e. a symbol "vec" is placed at the beginning of the list, in order to
> simplify the predication. But this leads to the indexing problem, that
> elements are indexed since 1, not 0. Such an indexing scheme would cause
> a nightmare to the developer who want to use our math subroutine to do
> further development.

An indexing task should never be a problem to the seasoned developer.
If they find it a nightmare, there is something missing with the developer.

Indexing in an everyday task in scientific programming.  One just has to focus
and think about what one is doing a little bit.

> In summary, if the builtin vector type is preferred way to represent the
> MATHEMATICAL vector, we should first of all complete the fundamental
> vector operations, like vector slicing etc. And of course, we should
> take a look at how sbcl
> https://git.code.sf.net/p/sbcl/sbcl
> organize their vector and their vector functions.
>
> From my side, I don't want to touch the vector things so far, but instead
> the rational number (the bullet 2 & 3 in my previous mail).
>
> Finally,
> > Sent: Saturday, July 27, 2024 at 17:49 +0200
> > From: "Emanuel Berg" <incal@dataswamp.org>
> >
> > However it used to be even worse, when we didn't even have clocks!
>
> If the "clock" you said is the clock to measure how long a function
> runs, we have, it is "benchmark-run".
>
>
> --
> Best Regards,
> Shouran Ma
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> An indexing task should never be a problem to the seasoned
> developer. If they find it a nightmare, there is something
> missing with the developer.
>
> Indexing in an everyday task in scientific programming.
> One just has to focus and think about what one is doing
> a little bit.

Without getting into the details, part of the idea with
a library is you don't have to be a seasoned, scientific or
focused developer - you just have to know the function name
and how to use it.

But actually this is as beneficial to anyone. _Everyone_ wants
libraries when programming, it doesn't matter what skill
level, experience etc you have.

"Sure, we don't have it, but one can solve it in just a few
lines of Elisp" - this is badness 010.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Immanuel Litzroth]

[-- Attachment #1: Type: text/plain, Size: 4584 bytes --]

> Indexing in an everyday task ...
Like writing?
Immanuel

On Sat, Jul 27, 2024 at 9:40 PM Christopher Dimech <dimech@gmx.com> wrote:

>
> > Sent: Sunday, July 28, 2024 at 7:07 AM
> > From: "Shouran Ma" <shouran.ma@gmail.com>
> > To: "Emanuel Berg" <incal@dataswamp.org>
> > Cc: emacs-devel@gnu.org, "Sergey Kostyaev" <sskostyaev@gmail.com>,
> "Philip Kaludercic" <philipk@posteo.net>
> > Subject: Re: Adding a generic mathematical library
> >
> >
> > > Sent: Saturday, July 27, 2024 at 16:57 +0200
> > > From: "Shouran Ma" <shouran.ma@gmail.com>
> > >
> > > However, WE DON'T EVEN HAVE A UNIFIED WAY TO REPRESENTING A VECTOR.
> >
> > I need to elaborate this statement, otherwise it cause confusion to
> > others.
> >
> > Elisp provides "List" and "Vector" as an array of numbers:
> >
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Sequences-Arrays-Vectors.html
> >
> > To express an array of numbers, a developer would either use two of them
> > - (setq x '(1 2 3 4))  ; (type-of x) => cons
> > - (setq x [1 2 3 4])   ; (type-of x) => vector
> >
> > "cons" (or "list") is the first thing that everybody would choose, for
> > example, in Sergey's elisa:
> > https://github.com/s-kostyaev/elisa/blob/main/elisa.el
> > In the function "elisa--distances", Sergey puts "head" and "(car tail)"
> > as arguments to "elisa-cosine-distance", this means Sergey uses
> > cons/list to organize array of numbers.
> >
> > However, Elisp also has "array" types:
> > https://www.gnu.org/software/emacs/manual/html_node/elisp/Arrays.html
> > > An array object has slots that hold a number of other Lisp objects,
> > > called the elements of the array. Any element of an array MAY BE
> > > ACCESSED IN CONSTANT TIME. In contrast, the time to access an element
> > > of a list is proportional to the position of that element in the list.
> >
> > cons/list is something like "linked list" which is not accessed in const
> > time, but vector support this, e.g. it takes more time to access the
> > last element if we use cons/list, but less time if use vector.
> >
> > However, Elisp provides too few functions on operating the vector type,
> > for example, to slice a vector like the way in python: array[2:4].
> >
> > So my saying "don't even have a unified way to representing a vector", I
> > mean, to organize/represent a MATHEMATICAL vector
> > - I would prefer to choose the object that support const time access,
> > i.e. the BUILTIN vector type, over the cons/list.
> > - However, the BUILTIN vector type supports quite a few functions, which
> > forces me to change my mind to organize/represent the MATHEMATICAL
> > vectors by cons/list.
> >
> > This is the dilemma during developing my own math libraries.
> >
> > Besides, in the Calc subroutine, the MATHEMATICAL vectors are
> > represented by cons/list in this way
> > (vec 3 5 2 1 0)  ; not (3 5 2 1 0) or [3 5 2 1 0]
> > i.e. a symbol "vec" is placed at the beginning of the list, in order to
> > simplify the predication. But this leads to the indexing problem, that
> > elements are indexed since 1, not 0. Such an indexing scheme would cause
> > a nightmare to the developer who want to use our math subroutine to do
> > further development.
>
> An indexing task should never be a problem to the seasoned developer.
> If they find it a nightmare, there is something missing with the developer.
>
> Indexing in an everyday task in scientific programming.  One just has to
> focus
> and think about what one is doing a little bit.
>
> > In summary, if the builtin vector type is preferred way to represent the
> > MATHEMATICAL vector, we should first of all complete the fundamental
> > vector operations, like vector slicing etc. And of course, we should
> > take a look at how sbcl
> > https://git.code.sf.net/p/sbcl/sbcl
> > organize their vector and their vector functions.
> >
> > From my side, I don't want to touch the vector things so far, but instead
> > the rational number (the bullet 2 & 3 in my previous mail).
> >
> > Finally,
> > > Sent: Saturday, July 27, 2024 at 17:49 +0200
> > > From: "Emanuel Berg" <incal@dataswamp.org>
> > >
> > > However it used to be even worse, when we didn't even have clocks!
> >
> > If the "clock" you said is the clock to measure how long a function
> > runs, we have, it is "benchmark-run".
> >
> >
> > --
> > Best Regards,
> > Shouran Ma
> >
> >
>
>

-- 
-- A man must either resolve to point out nothing new or to become a slave
to defend it. -- Sir Isaac Newton

[-- Attachment #2: Type: text/html, Size: 6607 bytes --]

Re: Adding a generic mathematical library

[Christopher Dimech]

Sent: Monday, July 29, 2024 at 1:34 AM
From: "Immanuel Litzroth" <immanuel.litzroth@gmail.com>
To: "Christopher Dimech" <dimech@gmx.com>
Cc: "Shouran Ma" <shouran.ma@gmail.com>, "Emanuel Berg" <incal@dataswamp.org>, emacs-devel@gnu.org, "Sergey Kostyaev" <sskostyaev@gmail.com>, "Philip Kaludercic" <philipk@posteo.net>
Subject: Re: Adding a generic mathematical library

> Indexing in an everyday task ...
Like writing?
Immanuel 

Yes, like writing.  Referring to elements within a collection (like arrays, 
lists, or strings) using numerical indices, is a fundamental concept in 
programming.  Despite its simplicity, proper understanding and handling of 
indexing are crucial for writing correct and efficient code. Indexing errors, 
like off-by-one mistakes, are among the most common bugs in software development.
It shows that developers do require a solid understanding as much as architects
and engineers.  Handling indices correctly, including understanding when to 
increment or decrement by one, is a critical skill.  These seemingly trivial 
issues can lead to significant bugs, from incorrect data processing to security 
vulnerabilities.

The idea that anyone can code to a specific standard without foundational 
knowledge is wrong.  While many people can learn to write code at a basic 
level, developing high-quality software requires a deeper understanding of 
algorithms, data structures, software design, and debugging techniques.

In fields like architecture and engineering, professionals undergo rigorous 
training to understand the principles and practices necessary to ensure safety, 
functionality, and aesthetics in their work.  This foundation is not present 
in computing today.

But, Emanuel has mentioned a different problem, the unusual symbol vec that is placed.
And that there is often complex mix of different ways to do things, all over emacs,
making development very slow paced.  It is the latter consideration that should be improved.

On Sat, Jul 27, 2024 at 9:40 PM Christopher Dimech <dimech@gmx.com[mailto:dimech@gmx.com]> wrote:
> Sent: Sunday, July 28, 2024 at 7:07 AM
> From: "Shouran Ma" <shouran.ma@gmail.com[mailto:shouran.ma@gmail.com]>
> To: "Emanuel Berg" <incal@dataswamp.org[mailto:incal@dataswamp.org]>
> Cc: emacs-devel@gnu.org[mailto:emacs-devel@gnu.org], "Sergey Kostyaev" <sskostyaev@gmail.com[mailto:sskostyaev@gmail.com]>, "Philip Kaludercic" <philipk@posteo.net[mailto:philipk@posteo.net]>
> Subject: Re: Adding a generic mathematical library
>
>
> > Sent: Saturday, July 27, 2024 at 16:57 +0200
> > From: "Shouran Ma" <shouran.ma@gmail.com[mailto:shouran.ma@gmail.com]>
> >
> > However, WE DON'T EVEN HAVE A UNIFIED WAY TO REPRESENTING A VECTOR.
>
> I need to elaborate this statement, otherwise it cause confusion to
> others.
>
> Elisp provides "List" and "Vector" as an array of numbers:
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Sequences-Arrays-Vectors.html[https://www.gnu.org/software/emacs/manual/html_node/elisp/Sequences-Arrays-Vectors.html]
>
> To express an array of numbers, a developer would either use two of them
> - (setq x '(1 2 3 4))  ; (type-of x) => cons
> - (setq x [1 2 3 4])   ; (type-of x) => vector
>
> "cons" (or "list") is the first thing that everybody would choose, for
> example, in Sergey's elisa:
> https://github.com/s-kostyaev/elisa/blob/main/elisa.el[https://github.com/s-kostyaev/elisa/blob/main/elisa.el]
> In the function "elisa--distances", Sergey puts "head" and "(car tail)"
> as arguments to "elisa-cosine-distance", this means Sergey uses
> cons/list to organize array of numbers.
>
> However, Elisp also has "array" types:
> https://www.gnu.org/software/emacs/manual/html_node/elisp/Arrays.html[https://www.gnu.org/software/emacs/manual/html_node/elisp/Arrays.html]
> > An array object has slots that hold a number of other Lisp objects,
> > called the elements of the array. Any element of an array MAY BE
> > ACCESSED IN CONSTANT TIME. In contrast, the time to access an element
> > of a list is proportional to the position of that element in the list.
>
> cons/list is something like "linked list" which is not accessed in const
> time, but vector support this, e.g. it takes more time to access the
> last element if we use cons/list, but less time if use vector.
>
> However, Elisp provides too few functions on operating the vector type,
> for example, to slice a vector like the way in python: array[2:4].
>
> So my saying "don't even have a unified way to representing a vector", I
> mean, to organize/represent a MATHEMATICAL vector
> - I would prefer to choose the object that support const time access,
> i.e. the BUILTIN vector type, over the cons/list.
> - However, the BUILTIN vector type supports quite a few functions, which
> forces me to change my mind to organize/represent the MATHEMATICAL
> vectors by cons/list.
>
> This is the dilemma during developing my own math libraries.
>
> Besides, in the Calc subroutine, the MATHEMATICAL vectors are
> represented by cons/list in this way
> (vec 3 5 2 1 0)  ; not (3 5 2 1 0) or [3 5 2 1 0]
> i.e. a symbol "vec" is placed at the beginning of the list, in order to
> simplify the predication. But this leads to the indexing problem, that
> elements are indexed since 1, not 0. Such an indexing scheme would cause
> a nightmare to the developer who want to use our math subroutine to do
> further development.

An indexing task should never be a problem to the seasoned developer.
If they find it a nightmare, there is something missing with the developer.

Indexing in an everyday task in scientific programming.  One just has to focus
and think about what one is doing a little bit.

> In summary, if the builtin vector type is preferred way to represent the
> MATHEMATICAL vector, we should first of all complete the fundamental
> vector operations, like vector slicing etc. And of course, we should
> take a look at how sbcl
> https://git.code.sf.net/p/sbcl/sbcl[https://git.code.sf.net/p/sbcl/sbcl]
> organize their vector and their vector functions.
>
> From my side, I don't want to touch the vector things so far, but instead
> the rational number (the bullet 2 & 3 in my previous mail).
>
> Finally,
> > Sent: Saturday, July 27, 2024 at 17:49 +0200
> > From: "Emanuel Berg" <incal@dataswamp.org[mailto:incal@dataswamp.org]>
> >
> > However it used to be even worse, when we didn't even have clocks!
>
> If the "clock" you said is the clock to measure how long a function
> runs, we have, it is "benchmark-run".
>
>
> --
> Best Regards,
> Shouran Ma
>
>
  
 --

-- A man must either resolve to point out nothing new or to become a slave to defend it. -- Sir Isaac Newton

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

>>> Indexing in an everyday task ...
>>
>> Like writing?
>
> Yes, like writing. Referring to elements within a collection
> (like arrays, lists, or strings) using numerical indices, is
> a fundamental concept in programming. Despite its
> simplicity, proper understanding and handling of indexing
> are crucial for writing correct and efficient code.
> Indexing errors, like off-by-one mistakes, are among the
> most common bugs in software development. It shows that
> developers do require a solid understanding as much as
> architects and engineers. Handling indices correctly,
> including understanding when to increment or decrement by
> one, is a critical skill. These seemingly trivial issues can
> lead to significant bugs, from incorrect data processing to
> security vulnerabilities.
>
> The idea that anyone can code to a specific standard without
> foundational knowledge is wrong. While many people can learn
> to write code at a basic level, developing high-quality
> software requires a deeper understanding of algorithms, data
> structures, software design, and debugging techniques.
>
> In fields like architecture and engineering, professionals
> undergo rigorous training to understand the principles and
> practices necessary to ensure safety, functionality, and
> aesthetics in their work. This foundation is not present in
> computing today.

... ? :O

> But, Emanuel has mentioned a different problem, the unusual
> symbol vec that is placed.

Someone else said that, I think?

> And that there is often complex mix of different ways to do
> things, all over emacs, making development very slow paced.

It is one of the reasons, yes - part of the "too much freedom"
reason :)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Shouran Ma wrote:

> I need to elaborate this statement, otherwise it cause
> confusion to others.

See if you can get going with what you would like to do most
right now.

Moving and mucking around with all that Calc stuff will be
a hard pitch for everyone but also boring work for you, and as
soon as you make a mistake, people will whine "it used to work
just fine" and be unappreciative of your efforts.

See if you instead can use the Calc stuff from and with your
new and expanded global Emacs math library, whenever it has
what you want.

If you gain momentum going forward one could merge Calc and
your stuff potentially sometime. Either by moving and renaming
physically or providing some interface to translate and
bridge incompatibilities.

So don't worry about that big picture in time and space.
Start today expanding complex numbers or vectors if you feel
strongly about them, or what do you think?

-- 
underground experts united
https://dataswamp.org/~incal