Add elisa to GNU ELPA

Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 262 bytes --]

Hi all,

Please add https://github.com/s-kostyaev/elisa to GNU ELPA. This package implements RAG (Retrieval Augmented Generation) for `ellama'. Today I have released 1.0.0 with info manuals, web search and local files support.

Best regards,
Sergey Kostyaev

[-- Attachment #2: Type: text/html, Size: 576 bytes --]

Re: Add elisa to GNU ELPA

[Philip Kaludercic]

[-- Attachment #1: Type: text/plain, Size: 777 bytes --]

Sergey Kostyaev <sskostyaev@gmail.com> writes:

> Hi all,
>
> Please add https://github.com/s-kostyaev/elisa to GNU ELPA. This
> package implements RAG (Retrieval Augmented Generation) for
> `ellama'. Today I have released 1.0.0 with info manuals, web search
> and local files support.
>
> Best regards,
> Sergey Kostyaev

I'm attaching my comments and suggestions to the end of this message.
Just one point at the beginning, so that you don't miss it: Despite
reading through the entire source code, I have no idea what you are
trying to do with this package.  It would be nice to have some more
context in the Commentary section and elaborate a number of user-facing
docstrings, unless this is intended to be expert software, for people
familiar with whatever the field is.


[-- Attachment #2: Type: text/plain, Size: 25207 bytes --]

diff --git a/elisa.el b/elisa.el
index 09cc28975d..0b79745bd5 100644
--- a/elisa.el
+++ b/elisa.el
@@ -28,7 +28,6 @@
 ;; ELISA (Emacs Lisp Information System Assistant) is a system designed
 ;; to provide informative answers to user queries by leveraging a
 ;; Retrieval Augmented Generation (RAG) approach.
-;;
 
 ;;; Code:
 (require 'ellama)
@@ -48,68 +47,61 @@
 					    (make-llm-ollama
 					     :embedding-model "nomic-embed-text"))
   "Embeddings provider to generate embeddings."
-  :group 'elisa
-  :type '(sexp :validate 'cl-struct-p))
+  :type '(sexp :validate 'cl-struct-p)) ;a more specific predicate here?
 
 (defcustom elisa-chat-provider (progn (require 'llm-ollama)
 				      (make-llm-ollama
 				       :chat-model "sskostyaev/openchat:8k-rag"
 				       :embedding-model "nomic-embed-text"))
   "Chat provider."
-  :group 'elisa
   :type '(sexp :validate 'cl-struct-p))
 
 (defcustom elisa-db-directory (file-truename
 			       (file-name-concat
 				user-emacs-directory "elisa"))
   "Directory for elisa database."
-  :group 'elisa
-  :type 'directory)
+  :type 'directory)                     ;is it necessary that it exists?
 
 (defcustom elisa-limit 5
   "Count quotes to pass into llm context for answer."
-  :group 'elisa
-  :type 'integer)
+  :type 'integer)                       ;or natnum?
 
-(defcustom elisa-find-executable "find"
+(defcustom elisa-find-executable find-program
   "Path to find executable."
-  :group 'elisa
   :type 'string)
 
 (defcustom elisa-tar-executable "tar"
   "Path to tar executable."
-  :group 'elisa
   :type 'string)
 
 (defcustom elisa-sqlite-vss-version "v0.1.2"
   "Sqlite VSS version."
-  :group 'elisa
   :type 'string)
 
 (defcustom elisa-sqlite-vss-path nil
   "Path to sqlite-vss extension."
-  :group 'elisa
   :type 'file)
 
 (defcustom elisa-sqlite-vector-path nil
   "Path to sqlite-vector extension."
-  :group 'elisa
   :type 'file)
 
-(defcustom elisa-semantic-split-function 'elisa-split-by-paragraph
+(defcustom elisa-semantic-split-function #'elisa-split-by-paragraph
   "Function for semantic text split."
-  :group 'elisa
   :type 'function)
 
 (defcustom elisa-prompt-rewriting-enabled t
   "Enable prompt rewriting for better retrieving."
-  :group 'elisa
   :type 'boolean)
 
-(defcustom elisa-chat-prompt-template "Answer user query based on context above. If you can answer it partially do it. Provide list of open questions if any. Say \"not enough data\" if you can't answer user query based on provided context. User query:
+(defcustom elisa-chat-prompt-template
+  "Answer user query based on context above. \
+If you can answer it partially do it. \
+Provide list of open questions if any. \
+Say \"not enough data\" if you can't answer user \
+query based on provided context. User query:
 %s"
-  "Chat prompt template."
-  :group 'elisa
+  "Chat prompt template."               ;some more explanation would be nice.
   :type 'string)
 
 (defcustom elisa-rewrite-prompt-template
@@ -130,81 +122,70 @@ How to buy a pony?
  User prompt:
 %s"
   "Prompt template for prompt rewriting."
-  :group 'elisa
   :type 'string)
 
 (defcustom elisa-searxng-url "http://localhost:8080/"
   "Searxng url for web search.  Json format should be enabled for this instance."
-  :group 'elisa
   :type 'string)
 
 (defcustom elisa-pandoc-executable "pandoc"
-  "Path to pandoc executable."
-  :group 'elisa
+  "Path to pandoc (https://pandoc.org/) executable."
   :type 'string)
 
-(defcustom elisa-webpage-extraction-function 'elisa-get-webpage-buffer
+(defcustom elisa-webpage-extraction-function #'elisa-get-webpage-buffer
   "Function to get buffer with webpage content."
-  :group 'elisa
   :type 'function)
 
-(defcustom elisa-web-search-function 'elisa-search-duckduckgo
+(defcustom elisa-web-search-function #'elisa-search-duckduckgo
   "Function to search the web.
 Function should get prompt and return list of urls."
-  :group 'elisa
   :type 'function)
 
 (defcustom elisa-web-pages-limit 10
   "Limit of web pages to parse during web search."
-  :group 'elisa
-  :type 'integer)
+  :type 'natnum)
 
 (defcustom elisa-breakpoint-threshold-amount 0.4
   "Breakpoint threshold amount.
 Increase it if you need decrease semantic split granularity."
-  :group 'elisa
-  :type 'float)
+  :type 'number)
 
 (defcustom elisa-reranker-enabled nil
   "Enable reranker to improve retrieving quality."
-  :group 'elisa
   :type 'boolean)
 
 (defcustom elisa-reranker-url "http://127.0.0.1:8787/"
-  "Reranker service url."
-  :group 'elisa
+  "Reranker service url."               ;here as well, this doesn't mean much to someone who don't already know what is going on.
   :type 'string)
 
 (defcustom elisa-reranker-similarity-threshold 0
   "Reranker similarity threshold.
 If set, all quotes with similarity less than threshold will be filtered out."
-  :group 'elisa
-  :type 'string)
+  :type 'string)                        ;wrong type?
 
 (defcustom elisa-reranker-limit 20
   "Number of quotes for send to reranker."
-  :group 'elisa
   :type 'integer)
 
 (defcustom elisa-ignore-patterns-files '(".gitignore" ".ignore" ".rgignore")
   "Files with patterns to ignore during file parsing."
-  :group 'elisa
-  :type '(list string))
+  :type '(repeat string))
 
 (defcustom elisa-ignore-invisible-files t
   "Ignore invisible files and directories during file parsing."
-  :group 'elisa
   :type 'boolean)
 
 (defcustom elisa-enabled-collections '("builtin manuals" "external manuals")
   "Enabled collections for elisa chat."
-  :group 'elisa
-  :type '(list string))
+  :type '(repeat string))
 
 (defun elisa-sqlite-vss-download-url ()
+  ;; It seems to be a general problem that your documentation strings
+  ;; are not giving any context, and just barley touching on what is
+  ;; going on...
   "Generate sqlite vss download url based on current system."
-  (cond  ((string-equal system-type "darwin")
-	  (if (string-prefix-p "aarch64" system-configuration)
+  (cond  ((eq system-type 'darwin)
+	  (if (string-prefix-p "aarch64" system-configuration) ;how robust is this?
 	      (format
 	       "https://github.com/asg017/sqlite-vss/releases/download/%s/sqlite-vss-%s-loadable-macos-aarch64.tar.gz"
 	       elisa-sqlite-vss-version
@@ -213,7 +194,7 @@ If set, all quotes with similarity less than threshold will be filtered out."
 	     "https://github.com/asg017/sqlite-vss/releases/download/%s/sqlite-vss-%s-loadable-macos-x86_64.tar.gz"
 	     elisa-sqlite-vss-version
 	     elisa-sqlite-vss-version)))
-	 ((string-equal system-type "gnu/linux")
+	 ((eq system-type 'gnu/linux)
 	  (format
 	   "https://github.com/asg017/sqlite-vss/releases/download/%s/sqlite-vss-%s-loadable-linux-x86_64.tar.gz"
 	   elisa-sqlite-vss-version
@@ -223,18 +204,14 @@ If set, all quotes with similarity less than threshold will be filtered out."
 (defun elisa--vss-path ()
   "Path to vss sqlite extension."
   (or elisa-sqlite-vss-path
-      (let* ((ext (if (string-equal system-type "darwin")
-		      "dylib"
-		    "so"))
+      (let* ((ext (if (eq system-type 'darwin) "dylib" "so"))
 	     (file (format "vss0.%s" ext)))
 	(file-name-concat elisa-db-directory file))))
 
 (defun elisa--vector-path ()
   "Path to vector sqlite extension."
   (or elisa-sqlite-vector-path
-      (let* ((ext (if (string-equal system-type "darwin")
-		      "dylib"
-		    "so"))
+      (let* ((ext (if (string-equal system-type 'darwin) "dylib" "so"))
 	     (file (format "vector0.%s" ext)))
 	(file-name-concat elisa-db-directory file))))
 
@@ -260,45 +237,45 @@ If set, all quotes with similarity less than threshold will be filtered out."
 
 (defun elisa-embeddings-create-table-sql ()
   "Generate sql for create embeddings table."
-  "drop table if exists elisa_embeddings;")
+  "DROP TABLE IF EXISTS elisa_embeddings;") ;just my personal taste, ignore if you disagree (i do it because there is no sql syntax highlighting in elisp strings)
 
 (defun elisa-data-embeddings-create-table-sql ()
   "Generate sql for create data embeddings table."
-  (format "create virtual table if not exists data_embeddings using vss0(embedding(%d));"
+  (format "CREATE VIRTUAL TABLE IF NOT EXISTS data_embeddings USING vss0(embedding(%d));"
 	  (elisa-get-embedding-size)))
 
 (defun elisa-data-fts-create-table-sql ()
   "Generate sql for create full text search table."
-  "create virtual table if not exists data_fts using fts5(data);")
+  "CREATE VIRTUAL TABLE IF NOT EXISTS data_fts USING FTS5(data);")
 
 (defun elisa-info-create-table-sql ()
   "Generate sql for create info table."
-  "drop table if exists info;")
+  "DROP TABLE IF EXISTS info;")
 
 (defun elisa-collections-create-table-sql ()
   "Generate sql for create collections table."
-  "create table if not exists collections (name text unique);")
+  "CREATE TABLE IF NOT EXISTS collections (name TEXT UNIQUE);")
 
 (defun elisa-kinds-create-table-sql ()
   "Generate sql for create kinds table."
-  "create table if not exists kinds (name text unique);")
+  "CREATE TABLE IF NOT EXISTS kinds (name TEXT UNIQUE);")
 
 (defun elisa-fill-kinds-sql ()
   "Generate sql for fill kinds table."
-  "insert into kinds (name) values ('web'), ('file'), ('info') on conflict do nothing;")
+  "INSERT INTO KINDS (name) VALUES ('web'), ('file'), ('info') ON CONFLICT DO NOTHING;")
 
 (defun elisa-files-create-table-sql ()
   "Generate sql for create files table."
-  "create table if not exists files (path text unique, hash text)")
+  "CREATE TABLE IF NOT EXISTS files (path TEXT UNIQUE, hash TEXT)")
 
 (defun elisa-data-create-table-sql ()
   "Generate sql for create data table."
-  "create table if not exists data (
+  "CREATE TABLE IF NOT EXISTS data (
 kind_id INTEGER,
 collection_id INTEGER,
-path text,
-hash text,
-data text,
+path TEXT,
+hash TEXT,
+data TEXT,
 FOREIGN KEY(kind_id) REFERENCES kinds(rowid),
 FOREIGN KEY(collection_id) REFERENCES collections(rowid)
 );")
@@ -307,13 +284,9 @@ FOREIGN KEY(collection_id) REFERENCES collections(rowid)
   "Initialize elisa DB."
   (if (not (file-exists-p (elisa--vss-path)))
       (warn "Please run M-x `elisa-download-sqlite-vss' to use this package")
-    (sqlite-pragma db "PRAGMA journal_mode=WAL;")
-    (sqlite-load-extension
-     db
-     (elisa--vector-path))
-    (sqlite-load-extension
-     db
-     (elisa--vss-path))
+    (sqlite-pragma db "PRAGMA journal_mode=WALL;")
+    (sqlite-load-extension db (elisa--vector-path))
+    (sqlite-load-extension db (elisa--vss-path))
     (sqlite-execute db (elisa-embeddings-create-table-sql))
     (sqlite-execute db (elisa-info-create-table-sql))
     (sqlite-execute db (elisa-collections-create-table-sql))
@@ -324,44 +297,46 @@ FOREIGN KEY(collection_id) REFERENCES collections(rowid)
     (sqlite-execute db (elisa-data-embeddings-create-table-sql))
     (sqlite-execute db (elisa-data-fts-create-table-sql))))
 
-(defvar elisa-db (progn
-		   (make-directory elisa-db-directory t)
-		   (let ((db (sqlite-open (file-name-concat elisa-db-directory "elisa.sqlite"))))
-		     (elisa--init-db db)
-		     db)))
+(defvar elisa-db
+  (let ((_ (make-directory elisa-db-directory t))
+        (db (sqlite-open (file-name-concat elisa-db-directory "elisa.sqlite"))))
+    (elisa--init-db db)
+    db))
 
 (defun elisa-vector-to-sqlite (data)
   "Convert DATA to sqlite vector representation."
-  (format "vector_from_json(json('%s'))"
-	  (json-encode data)))
-
-(defun elisa-sqlite-escape (s)
-  "Escape single quotes in S for sqlite."
-  (thread-last
-    s
-    (string-replace "'" "''")
-    (string-replace "\\" "\\\\")
-    (string-replace "\0" "\n")))
+  (format "vector_from_json(json('%s'))" (json-encode data)))
+
+(defun elisa-sqlite-escape (string)
+  "Escape single quotes in STRING for sqlite."
+  (let ((reps '(("'" . "''")
+                ("\\" . "\\\\")
+                ("\0" . "\n"))))
+    (replace-regexp-in-string         ;simultanious replacement
+     (regexp-opt (mapcar #'car reps)) ;is the last one really \0 or \\0?
+     (lambda (str) (alist-get str reps nil nil #'string=))
+     string nil t)))
 
 (defun elisa-sqlite-format-int-list (ids)
   "Convert list of integer IDS list to sqlite list representation."
   (format
    "(%s)"
-   (string-join (mapcar (lambda (id) (format "%d" id)) ids) ", ")))
+   (mapconcat (lambda (id) (format "%d" id)) ids ", ")))
 
 (defun elisa-sqlite-format-string-list (names)
   "Convert list of string NAMES list to sqlite list representation."
   (format
    "(%s)"
-   (string-join (mapcar (lambda (name)
-			  (format "'%s'"
-				  (elisa-sqlite-escape name))) names) ", ")))
+   (mapconcat (lambda (name)
+		(format "'%s'"
+			(elisa-sqlite-escape name)))
+              names ", ")))
 
-(defun elisa-avg (lst)
-  "Calculate arithmetic average value of LST."
-  (let ((len (length lst))
-	(sum (cl-reduce #'+ lst :initial-value 0.0)))
-    (/ sum len)))
+(defun elisa-avg (list)
+  "Calculate arithmetic average value of LIST."
+  (cl-loop for elem in list for count from 0
+           summing elem into sum
+           finally (return (/ sum (float count)))))
 
 (defun elisa-std-dev (lst)
   "Calculate standart deviation value of LST."
@@ -450,6 +425,11 @@ FOREIGN KEY(collection_id) REFERENCES collections(rowid)
 		     (setq continue nil))))
 	      (setq continue nil))))))))
 
+;; this is pretty verbose, do you think there might be a more readable
+;; way to express this?  It is pretty easy to create a small
+;; templating language in Elisp
+;; (e.g. https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/elpa-admin.el?h=elpa-admin#n1138),
+;; perhaps that could also be useful here.
 (defun elisa--find-similar (text collections)
   "Find similar to TEXT results in COLLECTIONS.
 Return sqlite query.  For asyncronous execution."
@@ -520,7 +500,7 @@ Evaluate ON-DONE with result."
 	(result nil))
     (save-excursion
       (goto-char (point-min))
-      (while (< (point) (point-max))
+      (while (not (eobp))
 	(funcall func)
 	(push (buffer-substring-no-properties pt (point)) result)
 	(setq pt (point)))
@@ -534,6 +514,7 @@ Evaluate ON-DONE with result."
   "Split buffer to list of paragraphs."
   (elisa--split-by #'forward-paragraph))
 
+;; a number of these functions seem like something that should be added to the core of Emacs or at least a common ELPA package...
 (defun elisa-dot-product (v1 v2)
   "Calculate the dot produce of vectors V1 and V2."
   (let ((result 0))
@@ -608,14 +589,12 @@ than T, it will be packed into single semantic chunk."
 	    (current (car chunks))
 	    (tail (cdr chunks)))
       (let* ((result nil))
-	(mapc
-	 (lambda (el)
-	   (if (<= el threshold)
+        (dolist (el distances)
+          (if (<= el threshold)
 	       (setq current (concat current (car tail)))
 	     (push current result)
 	     (setq current (car tail)))
 	   (setq tail (cdr tail)))
-	 distances)
 	(push current result)
 	(cl-remove-if
 	 #'string-empty-p
@@ -626,6 +605,7 @@ than T, it will be packed into single semantic chunk."
 		 (nreverse result))))
     (list (buffer-substring-no-properties (point-min) (point-max)))))
 
+;; why not use `wildcard-to-regexp'?
 (defun elisa--gitignore-to-elisp-regexp (pattern)
   "Convert a .gitignore PATTERN to an Emacs Lisp regexp."
   (let ((result "")
@@ -676,11 +656,11 @@ than T, it will be packed into single semantic chunk."
 
 (defun elisa--text-file-p (filename)
   "Check if FILENAME contain text."
-  (or (when (get-file-buffer filename) t) ;; if file opened assume it text
+  (or (and (get-file-buffer filename) t) ;; if file opened assume it text
       (with-current-buffer (find-file-noselect filename t t)
 	(prog1
 	    ;; if there is null byte in file, file is binary
-	    (not (re-search-forward "\0" nil t 1))
+	    (not (search-forward "\0" nil t 1))
 	  (kill-buffer)))))
 
 (defun elisa--file-list (directory)
@@ -727,41 +707,39 @@ When FORCE parse even if already parsed."
 	     (format "delete from files where path = '%s';"
 		     (elisa-sqlite-escape path))))
 	  ;; add new data
-	  (mapc
-	   (lambda (text)
-	     (let* ((hash (secure-hash 'sha256 text))
-		    (rowid
-		     (if-let ((rowid (caar (sqlite-select
-					    elisa-db
-					    (format "select rowid from data where kind_id = %s and collection_id = %s and path = '%s' and hash = '%s';"
-						    kind-id collection-id
-						    (elisa-sqlite-escape path) hash)))))
-			 (progn
-			   (push rowid row-ids)
-			   nil)
-		       (sqlite-execute
-			elisa-db
-			(format
-			 "insert into data(kind_id, collection_id, path, hash, data) values (%s, %s, '%s', '%s', '%s');"
-			 kind-id collection-id
-			 (elisa-sqlite-escape path) hash (elisa-sqlite-escape text)))
-		       (caar (sqlite-select
-			      elisa-db
-			      (format "select rowid from data where kind_id = %s and collection_id = %s and path = '%s' and hash = '%s';"
-				      kind-id collection-id
-				      (elisa-sqlite-escape path) hash))))))
-	       (when rowid
-		 (sqlite-execute
-		  elisa-db
-		  (format "insert into data_embeddings(rowid, embedding) values (%s, %s);"
-			  rowid (elisa-vector-to-sqlite
-				 (llm-embedding elisa-embeddings-provider text))))
-		 (sqlite-execute
-		  elisa-db
-		  (format "insert into data_fts(rowid, data) values (%s, '%s');"
-			  rowid (elisa-sqlite-escape text)))
-		 (push rowid row-ids))))
-	   chunks)
+          (dolist (text chunks)
+            (let* ((hash (secure-hash 'sha256 text))
+		   (rowid
+		    (if-let ((rowid (caar (sqlite-select
+					   elisa-db
+					   (format "select rowid from data where kind_id = %s and collection_id = %s and path = '%s' and hash = '%s';"
+						   kind-id collection-id
+						   (elisa-sqlite-escape path) hash)))))
+			(progn
+			  (push rowid row-ids)
+			  nil)
+		      (sqlite-execute
+		       elisa-db
+		       (format
+			"insert into data(kind_id, collection_id, path, hash, data) values (%s, %s, '%s', '%s', '%s');"
+			kind-id collection-id
+			(elisa-sqlite-escape path) hash (elisa-sqlite-escape text)))
+		      (caar (sqlite-select
+			     elisa-db
+			     (format "select rowid from data where kind_id = %s and collection_id = %s and path = '%s' and hash = '%s';"
+				     kind-id collection-id
+				     (elisa-sqlite-escape path) hash))))))
+	      (when rowid
+		(sqlite-execute
+		 elisa-db
+		 (format "insert into data_embeddings(rowid, embedding) values (%s, %s);"
+			 rowid (elisa-vector-to-sqlite
+				(llm-embedding elisa-embeddings-provider text))))
+		(sqlite-execute
+		 elisa-db
+		 (format "insert into data_fts(rowid, data) values (%s, '%s');"
+			 rowid (elisa-sqlite-escape text)))
+		(push rowid row-ids))))
 	  ;; remove old data
 	  (when row-ids
 	    (let ((delete-rows (cl-remove-if (lambda (id)
@@ -779,7 +757,7 @@ When FORCE parse even if already parsed."
 
 (defun elisa--delete-data (ids)
   "Delete data with IDS."
-  (sqlite-execute
+  (sqlite-execute                       ;perhaps it would be worth extracting the (sqlite-execute elisa-db ...) part into a separate function
    elisa-db
    (format "delete from data_fts where rowid in %s;"
 	   (elisa-sqlite-format-int-list ids)))
@@ -815,10 +793,9 @@ When FORCE parse even if already parsed."
 			collection-id
 			(elisa-sqlite-format-string-list files))))))
     (elisa--delete-data delete-ids)
-    (mapc (lambda (file)
-	    (message "parsing %s" file)
-	    (elisa-parse-file collection-id file))
-	  files)))
+    (dolist (file files)
+      (message "parsing %s" file)
+      (elisa-parse-file collection-id file))))
 
 ;;;###autoload
 (defun elisa-async-parse-directory (dir)
@@ -853,14 +830,14 @@ When FORCE parse even if already parsed."
 	  (libxml-parse-html-region
 	   (point) (point-max))
 	  'a))
-	:test 'string-equal)))))
+	:test #'string-equal)))))
 
 (defun elisa-search-searxng (prompt)
   "Search searxng for PROMPT and return list of urls.
 You can customize `elisa-searxng-url' to use non local instance."
   (let ((url (format "%s/search?format=json&q=%s" elisa-searxng-url (url-hexify-string prompt))))
     (thread-last
-      (plz 'get url :as 'json-read)
+      (plz 'get url :as 'json-read)     ;I am not familiar with the "plz" library, is the `json-read' a function?
       (alist-get 'results)
       (mapcar (lambda (el) (alist-get 'url el))))))
 
@@ -896,14 +873,13 @@ You can customize `elisa-searxng-url' to use non local instance."
     (with-current-buffer buffer-name
       (shell-command-on-region
        (point-min) (point-max)
-       (format "%s -f html --to plain"
-	       (executable-find elisa-pandoc-executable))
+       (format "%s --from html --to plain" elisa-pandoc-executable)
        buffer-name t)
       buffer-name)))
 
 (defun elisa-fts-query (prompt)
   "Return fts match query for PROMPT."
-  (thread-last
+  (thread-last                          ;i belive you can do all of this with a single regular expression...
     prompt
     (string-trim)
     (downcase)
@@ -938,7 +914,7 @@ You can customize `elisa-searxng-url' to use non local instance."
 		  :headers `(("Content-Type" . "application/json"))
 		  :body-type 'text
 		  :body (elisa--rerank-request prompt ids)
-		  :as #'json-read)))))
+		  :as #'json-read)))))  ;so it is a function!
 
 (defun elisa-rerank (prompt ids)
   "Rerank IDS according to PROMPT and return top `elisa-limit' IDS."
@@ -962,11 +938,10 @@ You can customize `elisa-searxng-url' to use non local instance."
 (defun elisa--parse-web-page (collection-id url)
   "Parse URL into collection with COLLECTION-ID."
   (let ((kind-id (caar (sqlite-select
-			elisa-db "select rowid from kinds where name = 'web';"))))
-    (message "collecting data from %s" url)
-    (mapc
-     (lambda (chunk)
-       (let* ((hash (secure-hash 'sha256 chunk))
+			elisa-db "SELECT rowid FROM kinds WHERE name = 'web';"))))
+    (message "collecting data from %S..." url)
+    (dolist (chunk (elisa-extact-webpage-chunks url))
+      (let* ((hash (secure-hash 'sha256 chunk))
 	      (embedding (llm-embedding elisa-embeddings-provider chunk))
 	      (rowid
 	       (if-let ((rowid (caar (sqlite-select
@@ -989,8 +964,7 @@ You can customize `elisa-searxng-url' to use non local instance."
 	   (sqlite-execute
 	    elisa-db
 	    (format "insert into data_fts(rowid, data) values (%s, '%s');"
-		    rowid (elisa-sqlite-escape chunk))))))
-     (elisa-extact-webpage-chunks url))))
+		    rowid (elisa-sqlite-escape chunk))))))))
 
 (defun elisa--web-search (prompt)
   "Search the web for PROMPT.
@@ -1007,11 +981,10 @@ Return sqlite query that extract data for adding to context."
 				(elisa-sqlite-escape prompt)))))
 	 (urls (funcall elisa-web-search-function prompt))
 	 (collected-pages 0))
-    (mapc (lambda (url)
-	    (when (<= collected-pages elisa-web-pages-limit)
-	      (elisa--parse-web-page collection-id url)
-	      (cl-incf collected-pages)))
-	  urls)))
+    (dolist (url urls)
+      (when (<= collected-pages elisa-web-pages-limit)
+	(elisa--parse-web-page collection-id url)
+	(cl-incf collected-pages)))))
 
 (defun elisa--rewrite-prompt (prompt action)
   "Rewrite PROMPT if `elisa-prompt-rewriting-enabled'.
@@ -1071,7 +1044,7 @@ WHERE d.rowid in %s;"
 		   (when-let ((kind (cl-first row))
 			      (path (cl-second row))
 			      (text (cl-third row)))
-		     (pcase kind
+		     (pcase kind        ;is this a `pcase-exhaustive'?
 		       ("web"
 			(ellama-context-add-webpage-quote-noninteractive path path text))
 		       ("file"
@@ -1096,15 +1069,16 @@ WHERE d.rowid in %s;"
   (mapcar
    #'file-name-base
    (cl-remove-if-not
-    (lambda (s) (or (string-suffix-p ".info" s)
-		    (string-suffix-p ".info.gz" s)))
+    (lambda (s)
+      (or (string-suffix-p ".info" s)
+	  (string-suffix-p ".info.gz" s)))
     (directory-files (with-temp-buffer
 		       (info "emacs" (current-buffer))
 		       (file-name-directory Info-current-file))))))
 
 (defun elisa-get-external-manuals ()
   "Get external manual names list."
-  (cl-remove-if
+  (cl-remove-if                         ;a `thread-last' might be nice here
    #'not
    (mapcar
     #'elisa--info-valid-p
@@ -1112,7 +1086,7 @@ WHERE d.rowid in %s;"
      (mapcar
       #'file-name-base
       (process-lines
-       (executable-find elisa-find-executable)
+       elisa-find-executable
        (file-truename
 	(file-name-concat user-emacs-directory "elpa"))
        "-name" "*.info"))))))
@@ -1207,7 +1181,7 @@ It does nothing if buffer file not inside one of existing collections."
   (when-let* ((collections (flatten-tree
 			    (sqlite-select
 			     elisa-db
-			     "select name from collections;")))
+			     "SELECT name FROM collections;")))
 	      (dirs (cl-remove-if-not #'file-directory-p collections))
 	      (file (buffer-file-name))
 	      (collection (cl-find-if (lambda (dir)
@@ -1283,7 +1257,7 @@ It does nothing if buffer file not inside one of existing collections."
   "Add webpage by URL to COLLECTION."
   (interactive
    (list
-    (if-let ((url (or (and (fboundp 'thing-at-point) (thing-at-point 'url))
+    (if-let ((url (or (and (fboundp 'thing-at-point) (thing-at-point 'url)) ;why not always use `thing-at-point'?
                       (shr-url-at-point nil))))
         url
       (read-string "Enter URL you want to summarize: "))

[-- Attachment #3: Type: text/plain, Size: 39 bytes --]




-- 
	Philip Kaludercic on peregrine

Re: Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 3008 bytes --]

Thank you for review.

> 16 июля 2024 г., в 19:54, Philip Kaludercic <philipk@posteo.net> написал(а):
> 
> Despite
> reading through the entire source code, I have no idea what you are
> trying to do with this package.  It would be nice to have some more
> context in the Commentary section and elaborate a number of user-facing
> docstrings, unless this is intended to be expert software, for people
> familiar with whatever the field is.

I need to improve it. With README.org <http://readme.org/> it should be cleaner, but It should be understandable without it.

Why should we remove elisa customization group? I think there are many customization options to move it to group.

> @@ -48,68 +47,61 @@
> 					    (make-llm-ollama
> 					     :embedding-model "nomic-embed-text"))
>   "Embeddings provider to generate embeddings."
> -  :group 'elisa
> -  :type '(sexp :validate 'cl-struct-p))
> +  :type '(sexp :validate 'cl-struct-p)) ;a more specific predicate here?

It should be `llm` provider. Unfortunately there are no more specific predicate for that.

> (defcustom elisa-db-directory (file-truename
> 			       (file-name-concat
> 				user-emacs-directory "elisa"))
>   "Directory for elisa database."
> -  :group 'elisa
> -  :type 'directory)
> +  :type 'directory)                     ;is it necessary that it exists?

If you mean option - then yes. If you mean directory, I can create it in `elisa' package.

> +     (regexp-opt (mapcar #'car reps)) ;is the last one really \0 or \\0?

Really. Probably impossible case, but it’s defensive programming.

> +;; a number of these functions seem like something that should be added to the core of Emacs or at least a common ELPA package...
> (defun elisa-dot-product (v1 v2)
>   "Calculate the dot produce of vectors V1 and V2."

Good idea. How can we implement it?

> -       (format "%s -f html --to plain"
> -	       (executable-find elisa-pandoc-executable))
> +       (format "%s --from html --to plain" elisa-pandoc-executable)
>        buffer-name t)
>       buffer-name)))

We should keep executable-find call here. It was feature request from users. It can be useful with per-directory PATH, like with direnv.

> 
> (defun elisa-fts-query (prompt)
>   "Return fts match query for PROMPT."
> -  (thread-last
> +  (thread-last                          ;i belive you can do all of this with a single regular expression...
>     prompt
>     (string-trim)
>     (downcase)

Maybe. But I prefer to keep it readable.
> @@ -1071,7 +1044,7 @@ WHERE d.rowid in %s;"
> 		   (when-let ((kind (cl-first row))
> 			      (path (cl-second row))
> 			      (text (cl-third row)))
> -		     (pcase kind
> +		     (pcase kind        ;is this a `pcase-exhaustive'?
> 		       ("web"
> 			(ellama-context-add-webpage-quote-noninteractive path path text))
> 		       ("file"

I think it should be extendable, but for now it is not.

Best regards,
Sergey Kostyaev


[-- Attachment #2: Type: text/html, Size: 7120 bytes --]

Re: Add elisa to GNU ELPA

[Philip Kaludercic]

Sergey Kostyaev <sskostyaev@gmail.com> writes:

> Thank you for review.
>
>> 16 июля 2024 г., в 19:54, Philip Kaludercic <philipk@posteo.net> написал(а):
>> 
>> Despite
>> reading through the entire source code, I have no idea what you are
>> trying to do with this package.  It would be nice to have some more
>> context in the Commentary section and elaborate a number of user-facing
>> docstrings, unless this is intended to be expert software, for people
>> familiar with whatever the field is.
>
> I need to improve it. With README.org <http://readme.org/> it should be cleaner, but It should be understandable without it.

I wouldn't want to rely on the README.  Or at least I think of a README
file less as an introduction to the package, and more of a introduction
to the source code repository.  Having a self-contained commentary
section that just explains what the package was made for and how to use
it is usually what I like seeing the most.

> Why should we remove elisa customization group? I think there are many customization options to move it to group.

Because unless I missed something, you had only defined a single group.
By default, a `defcustom' will be added to the last group defined in
file, making it redundant to specify it again and again.  That being
said, if you prefer this explicit redundancy, feel free to keep the
:group attributes.

>> @@ -48,68 +47,61 @@
>> 					    (make-llm-ollama
>> 					     :embedding-model "nomic-embed-text"))
>>   "Embeddings provider to generate embeddings."
>> -  :group 'elisa
>> -  :type '(sexp :validate 'cl-struct-p))
>> +  :type '(sexp :validate 'cl-struct-p)) ;a more specific predicate here?
>
> It should be `llm` provider. Unfortunately there are no more specific predicate for that.

So the `llm' struct doesn't havea `llm-p' predicate?

>> (defcustom elisa-db-directory (file-truename
>> 			       (file-name-concat
>> 				user-emacs-directory "elisa"))
>>   "Directory for elisa database."
>> -  :group 'elisa
>> -  :type 'directory)
>> +  :type 'directory)                     ;is it necessary that it exists?
>
> If you mean option - then yes. If you mean directory, I can create it in `elisa' package.

I meant if you should add a :must-match t tag.

>> +     (regexp-opt (mapcar #'car reps)) ;is the last one really \0 or \\0?
>
> Really. Probably impossible case, but it’s defensive programming.

Ok!

>> +;; a number of these functions seem like something that should be added to the core of Emacs or at least a common ELPA package...
>> (defun elisa-dot-product (v1 v2)
>>   "Calculate the dot produce of vectors V1 and V2."
>
> Good idea. How can we implement it?

To my knowledge, there is no maths package in the core or ELPA (someone
correct me if I am wrong).  There is calc, but IIRC it wasn't that nice
to use from an Elisp caller.  I could imagine that just pulling out the
definition you gave here, renaming them from "elisa-foo" to "foo" would
be enough for a first daft that could be added to the core, and then to
ELPA as a core package.

>> -       (format "%s -f html --to plain"
>> -	       (executable-find elisa-pandoc-executable))
>> +       (format "%s --from html --to plain" elisa-pandoc-executable)
>>        buffer-name t)
>>       buffer-name)))
>
> We should keep executable-find call here. It was feature request from users. It can be useful with per-directory PATH, like with direnv.

I might be missing something, but resolving a executable in PATH before
invoking it shouldn't have any effect, as call-process et. AL. will
resolve the executable name in the same context as an explicit
`executable-find' does anyway.

>> 
>> (defun elisa-fts-query (prompt)
>>   "Return fts match query for PROMPT."
>> -  (thread-last
>> +  (thread-last                          ;i belive you can do all of this with a single regular expression...
>>     prompt
>>     (string-trim)
>>     (downcase)
>
> Maybe. But I prefer to keep it readable.

I don't necessarily believe it would be less readable, and now you made
me want to try it out.  I'll report back at some point.

>> @@ -1071,7 +1044,7 @@ WHERE d.rowid in %s;"
>> 		   (when-let ((kind (cl-first row))
>> 			      (path (cl-second row))
>> 			      (text (cl-third row)))
>> -		     (pcase kind
>> +		     (pcase kind        ;is this a `pcase-exhaustive'?
>> 		       ("web"
>> 			(ellama-context-add-webpage-quote-noninteractive path path text))
>> 		       ("file"
>
> I think it should be extendable, but for now it is not.

Ok.

> Best regards,
> Sergey Kostyaev
>

-- 
	Philip Kaludercic on peregrine

Re: Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 238 bytes --]



> 16 июля 2024 г., в 23:04, Philip Kaludercic <philipk@posteo.net> написал(а):
> 
> So the `llm' struct doesn't havea `llm-p' predicate?

No. Maybe Andrew Hyatt can help with it?

Best regards,
Sergey Kostyaev

[-- Attachment #2: Type: text/html, Size: 1115 bytes --]

Re: Add elisa to GNU ELPA

[Andrew Hyatt]

Sergey Kostyaev <sskostyaev@gmail.com> writes:

>  16 июля 2024 г., в 23:04, Philip Kaludercic <philipk@posteo.net> написал(а):
>
>  So the `llm' struct doesn't havea `llm-p' predicate?
>
> No. Maybe Andrew Hyatt can help with it?

You could use `llm-standard-chat-provider-p' for now, which should get
everything except `llm-fake' and any providers people have been
developing on the side.

I can add something even more comprehensive in a next release if it
helps, though.

>
> Best regards,
> Sergey Kostyaev

Re: Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 574 bytes --]


> 16 июля 2024 г., в 23:04, Philip Kaludercic <philipk@posteo.net> написал(а):
> 
> To my knowledge, there is no maths package in the core or ELPA (someone
> correct me if I am wrong).  There is calc, but IIRC it wasn't that nice
> to use from an Elisp caller.  I could imagine that just pulling out the
> definition you gave here, renaming them from "elisa-foo" to "foo" would
> be enough for a first daft that could be added to the core, and then to
> ELPA as a core package.

What should be the first step?

Best regards,
Sergey Kostyaev


[-- Attachment #2: Type: text/html, Size: 4874 bytes --]

Re: Add elisa to GNU ELPA

[Philip Kaludercic]

Sergey Kostyaev <sskostyaev@gmail.com> writes:

>> 16 июля 2024 г., в 23:04, Philip Kaludercic <philipk@posteo.net> написал(а):
>> 
>> To my knowledge, there is no maths package in the core or ELPA (someone
>> correct me if I am wrong).  There is calc, but IIRC it wasn't that nice
>> to use from an Elisp caller.  I could imagine that just pulling out the
>> definition you gave here, renaming them from "elisa-foo" to "foo" would
>> be enough for a first daft that could be added to the core, and then to
>> ELPA as a core package.
>
> What should be the first step?

Someone™ should propose a concrete suggestion.

> Best regards,
> Sergey Kostyaev
>

-- 
	Philip Kaludercic on peregrine

Adding a generic mathematical library

[Philip Kaludercic]

Philip Kaludercic <philipk@posteo.net> writes:

> Sergey Kostyaev <sskostyaev@gmail.com> writes:
>
>>> 16 июля 2024 г., в 23:04, Philip Kaludercic <philipk@posteo.net> написал(а):
>>> 
>>> To my knowledge, there is no maths package in the core or ELPA (someone
>>> correct me if I am wrong).  There is calc, but IIRC it wasn't that nice
>>> to use from an Elisp caller.  I could imagine that just pulling out the
>>> definition you gave here, renaming them from "elisa-foo" to "foo" would
>>> be enough for a first daft that could be added to the core, and then to
>>> ELPA as a core package.
>>
>> What should be the first step?
>
> Someone™ should propose a concrete suggestion.

I've rename the thread to get some more attention, and also wanted to
mention, that if we do so, I think putting some thought into the design
wouldn't be bad.  Including an infix-notation macro and perhaps even
overloaded operator support ala APL would be a nice thing to have.  We'd
have to weigh the cost of this kind of flexibility with performance,
unless someone were to write a specialised byte-compiler.

>> Best regards,
>> Sergey Kostyaev
>>

-- 
	Philip Kaludercic on peregrine

Re: Adding a generic mathematical library

[Emanuel Berg]

Philip Kaludercic wrote:

>> Someone™ should propose a concrete suggestion.
>
> I've rename the thread to get some more attention, and also
> wanted to mention, that if we do so, I think putting some
> thought into the design wouldn't be bad. Including an
> infix-notation macro and perhaps even overloaded operator
> support ala APL would be a nice thing to have. We'd have to
> weigh the cost of this kind of flexibility with performance,
> unless someone were to write a specialised byte-compiler.

Yeah, I've pushed for a math library for a long time.

I have several files that are very close to that (for stats,
time, permutations and so on) but I also have a math.el which
I yank below if any of it can be used.

I'd love to stay in the loop and read more on this!

;;; -*- lexical-binding: t -*-
;;
;; this file:
;;   https://dataswamp.org/~incal/emacs-init/math.el

(require 'cl-lib)

(require 'dwim) ; https://dataswamp.org/~incal/emacs-init/dwim.el
(require 'perm) ; https://dataswamp.org/~incal/emacs-init/perm.el

(defun nand (&rest conditions)
  (when (member nil conditions)
    t) )

;; (nand)           ; nil
;; (nand 1 2 3)     ; nil
;; (nand nil)       ; t
;; (nand 1 2 nil 3) ; t

;; (not (and))           ; nil
;; (not (and 1 2 3))     ; nil
;; (not (and nil))       ; t
;; (not (and 1 2 nil 3)) ; t

(defalias '**   #'expt)
(defalias 'ceil #'ceiling)

(defun percent-change (from to)
  (if (= from to)
      0
    (/ (- to from) (abs from) 0.01)))

;; (percent-change  1  2) ;  100
;; (percent-change  1  1) ;    0
;; (percent-change  1  0) ; -100
;; (percent-change  1 -1) ; -200
;;
;; (percent-change  0  1) ;  1.0e+INF
;; (percent-change  0  0) ;  0
;; (percent-change  0 -1) ; -1.0e+INF
;;
;; (percent-change -1  1) ;  200
;; (percent-change -1  0) ;  100
;; (percent-change -1 -1) ;    0
;; (percent-change -1 -2) ; -100

(defun distance-point (min max)
  (+ min (/ (- max min) 2.0)) )

;; (distance-point  1   5) ; 3
;; (distance-point  1  10) ; 5.5
;; (distance-point 60 100) ; 80

(defun // (nom denom)
  (/ nom denom 1.0) )

;; (/  8 256) ; 0
;; (// 8 256) ; 0.03125

(defun arith-sum (n)
  (cl-loop
    with sum = 0
    for i from 1 to (1- n)
    do (cl-incf sum i)
    finally return sum) )

(defun digs (n)
  (* n (1- n)) )

(defun mean-value (vs)
  (let*((sum  (apply #'+ vs))
        (mean (/ sum (length vs) 1.0)) )
    mean) )

(defun mean-str (strs)
  (mean-value (cl-map 'list #'length strs)) )

(defun faster (old new)
  (format "%.1f%%" (* 100 (1- (/ new old 1.0)))) )

;; (faster 0.2  0.5)  ; 150.0%
;; (faster 1.35 0.84) ; -37.8%
;; (faster  200  400) ; 100.0%
;; (faster 1000 1000) ;   0.0%
;; (faster 1000  100) ; -90.0%

(defun percent (nom &optional denom str)
  (or denom (setq denom 1))
  (let ((perc (/ nom denom 0.01)))
    (if str
        (format "%.1f%%" perc)
      perc)))

;; (percent 0.25)           ; 25.0
;; (percent 50 75)          ; 66.66 ...
;; (percent (// 1 2) nil t) ; 50.0%
;; (percent 1019 22 t)      ; 4631.8%

(let ((min-def 0)
      (max-def 9)
      (inc-def 1) )
  (defun interval (&optional min max inc)
    (interactive `(,(read-number "min: " min-def)
                   ,(read-number "max: " max-def)
                   ,(read-number "inc: " inc-def)) )
    (or min (setq min min-def))
    (or max (setq max max-def))
    (or inc (setq inc inc-def))
    (unless (<= min max)
      (error "Bogus interval") )
    (unless (> inc 0)
      (error "Bogus increment") )
    (cl-loop
      for i from min to max by inc
      collect i) )
  (declare-function interval nil) )

;; (interval 10 5)        ; Bogus interval
;; (interval 1 3 -1)      ; Bogus increment
;; (interval 5 10)        ; (5 6 7 8 9 10)
;; (interval 1.8 2.0 0.1) ; (1.8 1.9 2.0)
;; (interval)             ; (0 1 2 3 4 5 6 7 8 9)
;; (interval 19 99)       ; (19 20 21 ... 97 98 99)

(defun digits-sum (&optional beg end)
  (interactive (use-region))
  (or beg (setq beg (point-min)))
  (or end (setq end (point-max)))
  (let ((sum 0))
    (goto-char beg)
    (while (re-search-forward " [[:digit:].]+ " end t)
      (cl-incf sum (string-to-number (match-string-no-properties 0))) )
    (prog1
        sum
      (message "sum: %.1f" sum) )))

(defalias 'di #'digits-sum)

(defun film-mean ()
  (interactive)
  (let ((scores)
        (beg (point-min))
        (end (point-max)) )
    (save-excursion
      (goto-char beg)
      (while (re-search-forward "[[:digit:]]" end t)
        (when (= (current-column) 73)
          (push (string-to-number (match-string-no-properties 0)) scores) )))
    (when scores
      (message (format "%.3f" (mean-value scores))) )))

(defun positive-integer-p (n)
  (and
    (integerp n)
    (< 0 n) ))

;; (positive-integer-p  1.5) ; nil
;; (positive-integer-p  1)   ; t
;; (positive-integer-p  0)   ; nil
;; (positive-integer-p -1)   ; nil

(defun string-to-number-number (str)
  (let ((s (string-trim str)))
    (when (string-match-p
           "\\`[+-]?\\([[:digit:]]+\\|[[:digit:]]*\\.[[:digit:]]+\\)\\'" s)
      (string-to-number s) )))

(defun string-to-number-number-test ()
  (cl-map 'list (lambda (e)
                  (pcase-let ((`(,a ,b) e))
                    (if (and a b)
                        (= a b)
                      (eq a b) )))
          (list
           (list (string-to-number-number " 10")                   10)
           (list (string-to-number-number "  1.5")                 1.5)
           (list (string-to-number-number " +0")                   0)
           (list (string-to-number-number "  0")                   0)
           (list (string-to-number-number " -0.0")                 -0.0)
           (list (string-to-number-number " -1.5")                -1.5)
           (list (string-to-number-number "-10")                  -10)
           (list (string-to-number-number "123this used to work")  nil)
           (list (string-to-number-number "NAN")                   nil) )))

;; (string-to-number-number-test)

(defun read-integer ()
  (let ((str)
        (str-number)
        (n) )
    (cl-loop until (progn
                     (setq str (read-string "integer: "))
                     (if (string= str "0")
                         (setq n 0)
                       (setq str-number (string-to-number str))
                       (unless (= str-number 0)
                         (setq n str-number) ))
                     (integerp n)) )
    n) )

;; (read-integer)

(defun compare (old new)
  (format "%.2f%%" (* 100 (/ (- old new)
                             (* old 1.0) ))))
;; (compare 185 80) ; 56.76%

(defun hypotenuse (c1 c2)
  (sqrt (+ (* c1 c1) (* c2 c2))) )

(defun speed (hour minute second km)
  (let*((s   (+ second (* 60 minute) (* 60 60 hour)))
        (m   (* 1000 km))
        (mps (/ m s)) )
    (* 3.6 mps) ))

;; (speed 04 44 10  42.195) ;  8.909208211143694
;; (speed 02 01 39  42.195) ; 20.811344019728732
;; (speed 19 04 00 168    ) ;  7.2

(defun binom (n k)
  (/ (cl-factorial n)
     (* (cl-factorial k)
        (cl-factorial (- n k)) )))

(defun product (l)
  (cl-loop with prod = 1
    for i in l do
    (setq prod (* prod i))
    finally return prod) )

;; (product '(1 2 3 4)) ; 24

(defun tau-ceti (speed)
  (let*((dist       (* 11.8 9.46 (expt 10 12)))
        (time       (/ dist speed) )
        (days       (/ time 24))
        (whole-days (round days)) )
    (format-seconds "%y years"
                    (- (float-time (encode-time 0 0 0 whole-days 0 0))
                       (float-time (encode-time 0 0 0 0 0 0)) ))))

;; (tau-ceti 5000) ; 2 548 584 years

(provide 'math)

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Wednesday, July 17, 2024 at 10:06 AM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Philip Kaludercic wrote:
> 
> >> Someone™ should propose a concrete suggestion.
> >
> > I've rename the thread to get some more attention, and also
> > wanted to mention, that if we do so, I think putting some
> > thought into the design wouldn't be bad. Including an
> > infix-notation macro and perhaps even overloaded operator
> > support ala APL would be a nice thing to have. We'd have to
> > weigh the cost of this kind of flexibility with performance,
> > unless someone were to write a specialised byte-compiler.
> 
> Yeah, I've pushed for a math library for a long time.
> 
> I have several files that are very close to that (for stats,
> time, permutations and so on) but I also have a math.el which
> I yank below if any of it can be used.

Tools for permutations would be of much benefit.
 
> I'd love to stay in the loop and read more on this!
> 
> ;;; -*- lexical-binding: t -*-
> ;;
> ;; this file:
> ;;   https://dataswamp.org/~incal/emacs-init/math.el
> 
> (require 'cl-lib)
> 
> (require 'dwim) ; https://dataswamp.org/~incal/emacs-init/dwim.el
> (require 'perm) ; https://dataswamp.org/~incal/emacs-init/perm.el
> 
> (defun nand (&rest conditions)
>   (when (member nil conditions)
>     t) )
> 
> ;; (nand)           ; nil
> ;; (nand 1 2 3)     ; nil
> ;; (nand nil)       ; t
> ;; (nand 1 2 nil 3) ; t
> 
> ;; (not (and))           ; nil
> ;; (not (and 1 2 3))     ; nil
> ;; (not (and nil))       ; t
> ;; (not (and 1 2 nil 3)) ; t
> 
> (defalias '**   #'expt)
> (defalias 'ceil #'ceiling)
> 
> (defun percent-change (from to)
>   (if (= from to)
>       0
>     (/ (- to from) (abs from) 0.01)))
> 
> ;; (percent-change  1  2) ;  100
> ;; (percent-change  1  1) ;    0
> ;; (percent-change  1  0) ; -100
> ;; (percent-change  1 -1) ; -200
> ;;
> ;; (percent-change  0  1) ;  1.0e+INF
> ;; (percent-change  0  0) ;  0
> ;; (percent-change  0 -1) ; -1.0e+INF
> ;;
> ;; (percent-change -1  1) ;  200
> ;; (percent-change -1  0) ;  100
> ;; (percent-change -1 -1) ;    0
> ;; (percent-change -1 -2) ; -100
> 
> (defun distance-point (min max)
>   (+ min (/ (- max min) 2.0)) )
> 
> ;; (distance-point  1   5) ; 3
> ;; (distance-point  1  10) ; 5.5
> ;; (distance-point 60 100) ; 80
> 
> (defun // (nom denom)
>   (/ nom denom 1.0) )
> 
> ;; (/  8 256) ; 0
> ;; (// 8 256) ; 0.03125
> 
> (defun arith-sum (n)
>   (cl-loop
>     with sum = 0
>     for i from 1 to (1- n)
>     do (cl-incf sum i)
>     finally return sum) )
> 
> (defun digs (n)
>   (* n (1- n)) )
> 
> (defun mean-value (vs)
>   (let*((sum  (apply #'+ vs))
>         (mean (/ sum (length vs) 1.0)) )
>     mean) )
> 
> (defun mean-str (strs)
>   (mean-value (cl-map 'list #'length strs)) )
> 
> (defun faster (old new)
>   (format "%.1f%%" (* 100 (1- (/ new old 1.0)))) )
> 
> ;; (faster 0.2  0.5)  ; 150.0%
> ;; (faster 1.35 0.84) ; -37.8%
> ;; (faster  200  400) ; 100.0%
> ;; (faster 1000 1000) ;   0.0%
> ;; (faster 1000  100) ; -90.0%
> 
> (defun percent (nom &optional denom str)
>   (or denom (setq denom 1))
>   (let ((perc (/ nom denom 0.01)))
>     (if str
>         (format "%.1f%%" perc)
>       perc)))
> 
> ;; (percent 0.25)           ; 25.0
> ;; (percent 50 75)          ; 66.66 ...
> ;; (percent (// 1 2) nil t) ; 50.0%
> ;; (percent 1019 22 t)      ; 4631.8%
> 
> (let ((min-def 0)
>       (max-def 9)
>       (inc-def 1) )
>   (defun interval (&optional min max inc)
>     (interactive `(,(read-number "min: " min-def)
>                    ,(read-number "max: " max-def)
>                    ,(read-number "inc: " inc-def)) )
>     (or min (setq min min-def))
>     (or max (setq max max-def))
>     (or inc (setq inc inc-def))
>     (unless (<= min max)
>       (error "Bogus interval") )
>     (unless (> inc 0)
>       (error "Bogus increment") )
>     (cl-loop
>       for i from min to max by inc
>       collect i) )
>   (declare-function interval nil) )
> 
> ;; (interval 10 5)        ; Bogus interval
> ;; (interval 1 3 -1)      ; Bogus increment
> ;; (interval 5 10)        ; (5 6 7 8 9 10)
> ;; (interval 1.8 2.0 0.1) ; (1.8 1.9 2.0)
> ;; (interval)             ; (0 1 2 3 4 5 6 7 8 9)
> ;; (interval 19 99)       ; (19 20 21 ... 97 98 99)
> 
> (defun digits-sum (&optional beg end)
>   (interactive (use-region))
>   (or beg (setq beg (point-min)))
>   (or end (setq end (point-max)))
>   (let ((sum 0))
>     (goto-char beg)
>     (while (re-search-forward " [[:digit:].]+ " end t)
>       (cl-incf sum (string-to-number (match-string-no-properties 0))) )
>     (prog1
>         sum
>       (message "sum: %.1f" sum) )))
> 
> (defalias 'di #'digits-sum)
> 
> (defun film-mean ()
>   (interactive)
>   (let ((scores)
>         (beg (point-min))
>         (end (point-max)) )
>     (save-excursion
>       (goto-char beg)
>       (while (re-search-forward "[[:digit:]]" end t)
>         (when (= (current-column) 73)
>           (push (string-to-number (match-string-no-properties 0)) scores) )))
>     (when scores
>       (message (format "%.3f" (mean-value scores))) )))
> 
> (defun positive-integer-p (n)
>   (and
>     (integerp n)
>     (< 0 n) ))
> 
> ;; (positive-integer-p  1.5) ; nil
> ;; (positive-integer-p  1)   ; t
> ;; (positive-integer-p  0)   ; nil
> ;; (positive-integer-p -1)   ; nil
> 
> (defun string-to-number-number (str)
>   (let ((s (string-trim str)))
>     (when (string-match-p
>            "\\`[+-]?\\([[:digit:]]+\\|[[:digit:]]*\\.[[:digit:]]+\\)\\'" s)
>       (string-to-number s) )))
> 
> (defun string-to-number-number-test ()
>   (cl-map 'list (lambda (e)
>                   (pcase-let ((`(,a ,b) e))
>                     (if (and a b)
>                         (= a b)
>                       (eq a b) )))
>           (list
>            (list (string-to-number-number " 10")                   10)
>            (list (string-to-number-number "  1.5")                 1.5)
>            (list (string-to-number-number " +0")                   0)
>            (list (string-to-number-number "  0")                   0)
>            (list (string-to-number-number " -0.0")                 -0.0)
>            (list (string-to-number-number " -1.5")                -1.5)
>            (list (string-to-number-number "-10")                  -10)
>            (list (string-to-number-number "123this used to work")  nil)
>            (list (string-to-number-number "NAN")                   nil) )))
> 
> ;; (string-to-number-number-test)
> 
> (defun read-integer ()
>   (let ((str)
>         (str-number)
>         (n) )
>     (cl-loop until (progn
>                      (setq str (read-string "integer: "))
>                      (if (string= str "0")
>                          (setq n 0)
>                        (setq str-number (string-to-number str))
>                        (unless (= str-number 0)
>                          (setq n str-number) ))
>                      (integerp n)) )
>     n) )
> 
> ;; (read-integer)
> 
> (defun compare (old new)
>   (format "%.2f%%" (* 100 (/ (- old new)
>                              (* old 1.0) ))))
> ;; (compare 185 80) ; 56.76%
> 
> (defun hypotenuse (c1 c2)
>   (sqrt (+ (* c1 c1) (* c2 c2))) )
> 
> (defun speed (hour minute second km)
>   (let*((s   (+ second (* 60 minute) (* 60 60 hour)))
>         (m   (* 1000 km))
>         (mps (/ m s)) )
>     (* 3.6 mps) ))
> 
> ;; (speed 04 44 10  42.195) ;  8.909208211143694
> ;; (speed 02 01 39  42.195) ; 20.811344019728732
> ;; (speed 19 04 00 168    ) ;  7.2
> 
> (defun binom (n k)
>   (/ (cl-factorial n)
>      (* (cl-factorial k)
>         (cl-factorial (- n k)) )))
> 
> (defun product (l)
>   (cl-loop with prod = 1
>     for i in l do
>     (setq prod (* prod i))
>     finally return prod) )
> 
> ;; (product '(1 2 3 4)) ; 24
> 
> (defun tau-ceti (speed)
>   (let*((dist       (* 11.8 9.46 (expt 10 12)))
>         (time       (/ dist speed) )
>         (days       (/ time 24))
>         (whole-days (round days)) )
>     (format-seconds "%y years"
>                     (- (float-time (encode-time 0 0 0 whole-days 0 0))
>                        (float-time (encode-time 0 0 0 0 0 0)) ))))
> 
> ;; (tau-ceti 5000) ; 2 548 584 years
> 
> (provide 'math)
> 
> -- 
> underground experts united
> https://dataswamp.org/~incal
> 
> 
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

>> Yeah, I've pushed for a math library for a long time.
>> 
>> I have several files that are very close to that (for stats,
>> time, permutations and so on) but I also have a math.el which
>> I yank below if any of it can be used.
>
> Tools for permutations would be of much benefit.

;;; -*- lexical-binding: t -*-
;;
;; this file:
;;   https://dataswamp.org/~incal/emacs-init/perm.el

(require 'cl-lib)
(require 'spell)

;; Christoph Conrad @ https://www.emacswiki.org/emacs/StringPermutations
(defun perms (l)
  (if l (cl-mapcan (lambda (a)
                     (cl-mapcan (lambda (p)
                                  (list (cons a p)))
                                (perms (cl-remove a l :count 1)) )) l)
    '(()) ))

(defun string-perms (str)
  (let*((chars      (string-to-list str))
        (char-perms (perms chars)) )
    (mapcar (lambda (a)
              (concat a) )
            char-perms) ))
;; (string-perms "hte") ; ("hte" "het" "the" "teh" "eht" "eth")

(defun string-perms-filter (str)
  (let ((strs (cl-remove-duplicates
                (cl-remove-if-not (lambda (w) (spell-word w)) (string-perms str))
                :test #'string=) ))
    (if (= 1 (length strs))
        (car strs)
      strs) ))

;; eht kudtce pigelsen tagni ogod seot erontuf si ni fo hte
;;
;; (string-perms-filter "ogod")     ; good
;; (string-perms-filter "erontuf")  ; fortune
;; (string-perms-filter "si")       ; is
;; (string-perms-filter "kudtce")   ; tucked
;; (string-perms-filter "ni")       ; in
;; (string-perms-filter "eth")      ; the
;; (string-perms-filter "seot")     ; toes
;; (string-perms-filter "fo")       ; of
;; (string-perms-filter "hte")      ; the
;; (string-perms-filter "pigelsen") ; ("peelings" "sleeping")
;; (string-perms-filter "tagni")    ; giant

;; (string-perms-filter "emacs")

(defun factorial (n)
  (if (> n 1)
      (* n (factorial (1- n)))
    1))
;; (factorial  5) ;       120
;; (factorial 10) ; 3 628 800

(defun cl-factorial (n)
  (cl-loop
    with prod = 1
    for i from 2 to n
    do (setq prod (* i prod))
    finally return prod) )
;; (cl-factorial  5) ;       120
;; (cl-factorial 10) ; 3 628 800

(defun product-string (str)
  (let*((str-list         (string-to-list str))
        (str-list-no-dups (cl-remove-duplicates str-list))
        (prod 1) )
    (dolist (e str-list-no-dups)
      (setq prod (* prod (cl-factorial (cl-count e str-list)))) )
    prod ))
;; (product-string "ogod") ; 2

(defun perms-string-num (str)
  (let ((n (cl-factorial (length str)))
        (r (product-string str)) )
    (/ n r) ))
;; (perms-string-num "ogod")   ;  12
;; (perms-string-num "kudtce") ; 720

(provide 'perm)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >  but I also have a math.el which
  > I yank below if any of it can be used.

Most of the fnctions in this library are very close to trivial.

Onsider, for instance, `hypotenuse'.  The benefit of having that
functoin would not quite be zero, but it would be small.  Is it enough
to justify the additional function name, and the complexity of
documenting it?

Perhaps there are some nontricial mathematical functions that would be
worth adding to Emacs, but that brings up the question of what
functions would be useful,  With two people expressing a desire
for a "math library", do they both wan the same thing?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Saturday, July 20, 2024 at 4:16 AM
> From: "Richard Stallman" <rms@gnu.org>
> To: "Emanuel Berg" <incal@dataswamp.org>
> Cc: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   >  but I also have a math.el which
>   > I yank below if any of it can be used.
>
> Most of the fnctions in this library are very close to trivial.

The demand for robust mathematical functions is valid and points to the need
for developing comprehensive mathematical libraries within Emacs that can support
more advanced projects.

> Onsider, for instance, `hypotenuse'.  The benefit of having that
> functoin would not quite be zero, but it would be small.  Is it enough
> to justify the additional function name, and the complexity of
> documenting it?
>
> Perhaps there are some nontricial mathematical functions that would be
> worth adding to Emacs, but that brings up the question of what
> functions would be useful,  With two people expressing a desire
> for a "math library", do they both wan the same thing?

To ensure the inclusion of genuinely useful functions, it is
essential to first identify their practical applications and
the reasons for their inclusion. Determining which functions
will be most beneficial requires a thorough examination of
specific use cases. This assessment must precede the development
of the mathematical library to achieve consensus and secure approval.

Currently, Emanuel is not going to be the one to do it, but he is
happy to offer assistance.

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> The demand for robust mathematical functions is valid and
> points to the need for developing comprehensive mathematical
> libraries within Emacs that can support more
> advanced projects.

There are a lot basic stuff we need first. The "trivial"
stuff, in your phrasing. But they will be used in advanced
projects as well by all means.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Max Nikulin]

On 19/07/2024 23:16, Richard Stallman wrote:
> Onsider, for instance, `hypotenuse'.  The benefit of having that
> functoin would not quite be zero, but it would be small.  Is it enough
> to justify the additional function name, and the complexity of
> documenting it?

As usual, when implementation is replaced with a better one, all callers 
may benefit with no efforts on their side.

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Sunday, July 21, 2024 at 12:45 AM
> From: "Max Nikulin" <manikulin@gmail.com>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> On 19/07/2024 23:16, Richard Stallman wrote:
> > Onsider, for instance, `hypotenuse'.  The benefit of having that
> > functoin would not quite be zero, but it would be small.  Is it enough
> > to justify the additional function name, and the complexity of
> > documenting it?
>
> As usual, when implementation is replaced with a better one, all callers
> may benefit with no efforts on their side.

That function can be used as a meaningful starting point to enable adoption
and feedback.   Providing the necessary infrastructure that can be easily
scaled.

Re: Adding a generic mathematical library

[Emanuel Berg]

Richard Stallman wrote:

>> but I also have a math.el which I yank below if any of it
>> can be used.
>
> Most of the fnctions in this library are very close
> to trivial.

Oh, my! Eli thinks they are so exotic only I use them, RMS
thinks they are close to trivial.

But I have many more files, I'll see what I have that can
maybe impress you and Eli more.

I have already showed you math.el and perm.el, here are two
more. But I'll look thru it all some other day and make
a directory with all of them.

;;; -*- lexical-binding: t -*-
;;
;; this file:
;;   https://dataswamp.org/~incal/emacs-init/stats.el

(require 'cl-lib)

(defun decade ()
  (interactive)
  (cl-loop
    with entries = ()
    for d from 190 to 202
    do (push (how-many (format "^.\\{14\\}%d[[:digit:]]" d)) entries)
    finally (message "%s" (nreverse entries)) ))

(defun random-distribution (n sum &optional min no-error)
  "Return a list of N random numbers.
\nThe sum of the numbers is SUM.
\nEvery number is at least MIN. (default: 0)
\nIf NO-ERROR, recompute unsolvable systems with MIN = 0. (default: nil)"
  (or min (setq min 0))
  (let ((min-share (* n min)))
    (if (< sum min-share)
        (if no-error
            (random-distribution n sum)
          (error "Bogus indata, unsolvable") )
      (let ((vals (make-list n min))
            (pool (- sum min-share)) )
        (cl-loop while (< 0 pool)
                 do (cl-incf (nth (random n) vals))
                    (cl-decf pool) )
        vals) )))

(defalias 'rd #'random-distribution)

;; (apply #'+ (rd 10 100 3)) ; sum is 100, OK
;; (rd 10 100)               ; omitted/default min = 0, OK
;; (rd 10 100 10)            ; border case, but OK
;; (rd 10 100 20)            ; error: Bogus indata, unsolvable
;; (rd 10 100 20 t)          ; unsolvable but no-error, recompute for min = 0
;;
;; 10 example distributions for
;; n = 10, sum = 100, and min = 3,
;; i.e. (rd 10 100 3):
;;
;;   (12  8  8 10  7  6 19  9  7 14)
;;   ( 8 12  7 12  9 10  9  9 12 12)
;;   (14  6 15 11  9  9  6  8 11 11)
;;   ( 8 12  7  9 10  9 10 13 10 12)
;;   (12 12 15  6 10  8  6 10 14  7)
;;   ( 9  7 10  8 11 10  9 10 13 13)
;;   ( 7 13 13 10 13 10  5 13  5 11)
;;   (13  8  6  9 13  8 11  7 10 15)
;;   ( 7 10  5  6 17 14  8 10 14  9)
;;   (11 10  8  9 11 10 14 11  8  8)

(provide 'stats)

;;; -*- lexical-binding: t -*-
;;
;; help from:
;;   https://codereview.stackexchange.com/q/186840
;;
;; this file:
;;   https://dataswamp.org/~incal/emacs-init/str-to-bits.el

(require 'esh-util)

(defun num-to-bits (num)
  (let ((s ""))
    (while (> num 0)
      (setq s (concat (number-to-string (logand num 1)) s))
      (setq num (ash num -1)) )
    (if (string= "" s)
        "0"
      s) ))

(defalias 'dec2bits  #'num-to-bits)
(defalias 'char2bits #'num-to-bits)

(defun str-to-bits (str &optional delim)
  (interactive "sString: ")
  (or delim (setq delim " "))
  (let ((bits))
    (dolist (c (string-to-list str) (mapconcat #'eshell-stringify bits delim))
      (push (num-to-bits c) bits) )))

;; (str-to-bits "What's up Emacs community?") ; 111111 1110000 1110101 [...]

(provide 'str-to-bits)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

>> Most of the functions in this library are very close to
>> trivial. [...]
>
> I have already showed you math.el and perm.el, here are two
> more. But I'll look thru it all some other day and make
> a directory with all of them.

No, but let's do this thoroughly then.

I will look at this situation now in another way, let's calm
down again, maybe we can do something useful still.

It will be really interesting to hear your comments - no irony
att all. I'm ready.

I'll do the directory later, here is just a list with URLs.
It is the first 12 files a found but there is more. At least
one is a proper package while other are very short files.

I'll post the rest some other day, God willing.

Okay, here goes!

Computation packages, files, and projects. PART 1, 12 files.

* random-urandom.c
  https://dataswamp.org/~incal/emacs-init/random-urandom/random-urandom.c

  Dynamic module (C language) to get to Linux urandom which is
  more random than `random', so you can use that from Elisp
  as well.

* all-substance.el
  https://dataswamp.org/~incal/emacs-init/all-substance.el

  Substance reduction (half-time) over time.

* audio.el
  https://dataswamp.org/~incal/emacs-init/audio.el

  Audio db multiplier.

* beer.el
  https://dataswamp.org/~incal/emacs-init/beer.el

  Convert between beer types, e.g. how many 5.3% beers are one
  six-pack 3.5% plus one cider?

* bike.el
  https://dataswamp.org/~incal/emacs-init/bike.el

  Compute the step and gear (roll out).

* color-incal.el
  https://dataswamp.org/~incal/emacs-init/color-incal.el

  Computations on colors, determines "is this color blue?"
  with some method for that.

* dice.el
  https://dataswamp.org/~incal/emacs-init/dice.el

  Can do 1D6, 2D3 etc.

* eggs.el
  https://dataswamp.org/~incal/emacs-init/eggs.el

  Solves an egg math problem.

* epwgen.el
  https://dataswamp.org/~incal/emacs-init/epwgen.el

  Compute password strength.

* frame-size.el
  https://dataswamp.org/~incal/emacs-init/frame-size.el

  More computations on the bicycle, this time the frame.

* hex.el
  https://dataswamp.org/~incal/emacs-init/hex.el

  Decodes a hex string.

* isbn-verify.el
  https://dataswamp.org/~incal/emacs-init/isbn-verify.el

  Verify ISBNs with the check digit and a known algorithm.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Here is the entire list. 28 files. Dig in.

I'll make a directory and unified list later, maybe. In the
meantime, well you see in what directory the files are, if
you'd like to browse them. That will include other files
for now.

MATH

Second part, 16 files.

Last: the first part, 12 files. Total 28 files.

* issn-verify.el
  https://dataswamp.org/~incal/emacs-init/issn-verify.el

Verify ISSN numbers.

* lights.el
  https://dataswamp.org/~incal/emacs-init/lights.el

Compute LED luminous efficacy (lm/W).

* list.el
  https://dataswamp.org/~incal/emacs-init/list.el

Stats on lists.

* math.el
  https://dataswamp.org/~incal/emacs-init/math.el

The infamous math file, critized by many for as many different
reasons. Fairly so? Maybe! It has no less than 23
`defun' anyway.

* negative-subtraction.el
  https://dataswamp.org/~incal/emacs-init/negative-subtraction.el

Subtraction digit by digit.

* perm.el
  https://dataswamp.org/~incal/emacs-init/perm.el

Permutations. Known from the math field combinatorics but also
stats and actually many science areas.

Bonus: this file solves a fun word problem from a TV show.

* psea.el
  https://dataswamp.org/~incal/emacs-init/psea.el
  https://dataswamp.org/~incal/emacs-init/b-a-original.el

Has to do with string distance and then stats and tables on
that. Yes, I tried that two times it seems! In b-a-original.el
you can output this kind of table (below) saying the first
line is 81% original - so scores a number 1 position at
407 originality points. Based on a weighted sum.

  1. (require 'pcase)      81% 407
  2. (require 'cl-lib)     80% 402
  3. (require 'psea)       80% 402
  4. (require 'subr-x)     80% 402
  5. (require 'thingatpt)  77% 387

* random-generic.el (random.el)
  https://dataswamp.org/~incal/emacs-init/random-generic.el
  https://dataswamp.org/~incal/emacs-init/random.el

Randomizer for arbitrary data types, so you can feed whatever
into it and mix it up transparently. Plus some misc random
stuff, looks like it is more practical in nature but
randomness is a part of math (and stats, and CS, and ...) so
I include it.

* scale.el
  https://dataswamp.org/~incal/emacs-init/scale.el

Draw a scale.

[ Sorting we don't consider math, right? But CS? Cool, not
  included. ]

* stats.el
  https://dataswamp.org/~incal/emacs-init/stats.el

I'm not sure what this does, it outputs some digit set with
some constrains? I'll look it up, looks interesting!

* str-to-bits.el
  https://dataswamp.org/~incal/emacs-init/str-to-bits.el

(str-to-bits "What's up Emacs community?")
 111111 1111001 1110100 1101001 1101110 1110101 1101101 1101101
1101111 1100011  100000 1110011 1100011 1100001 1101101 1000101
 100000 1110000 1110101  100000 1110011  100111 1110100 1100001
1101000 1010111

* survivor.el
  https://dataswamp.org/~incal/emacs-init/survivor.el

[ OT: read if you are from Australia or Sweden. Maybe the US ]

This is computation on the TV show Survivor (or
Expedition Robinson as the English original idea was called on
Swedish TV where it was first realized). Computation is based
on the idea that it gets more difficult all the time but not
linearly so but much worse. So, the difficulty at day 48 in
Australian Survivor is computed to 1176 (compare for example
US Survivor at only 27 days and scores a mere 378). Is
AUS Survivor the world's most difficult reality TV show? Well,
the numbers don't lie ;) Congrats to AUS people reading this,
you are a tough bunch - in a good way - and thanks for that
awsome show, the very best national instance ever of the show!

Here is the full list:

;; (cost-at-day 48) ; 1176 AUS
;; (cost-at-day 39) ;  780 USA old era
;; (cost-at-day 28) ;  406 SWE
;; (cost-at-day 27) ;  378 USA new era
;; (cost-at-day  1) ;    1
;; (cost-at-day  0) ;    0

* time-cmp.el
  https://dataswamp.org/~incal/emacs-init/time-cmp.el

Computation on time. Very useful.

* variance.el
  https://dataswamp.org/~incal/emacs-init/variance.el

This has to do with stats, one is to parse a file with data.
Very old but looks good anyway.

* wood.el
  https://dataswamp.org/~incal/emacs-init/wood.el

How to cut wood and build with wood. Several ideas are
implemented here and I have tried them in practice and they
work there as well (especially there). See photo linked
from file.

also:

* distance.el
  https://dataswamp.org/~incal/distance/distance.el

Calculate the distance between two Earth locations using the
Haversine formula (yes, the globe is very much so round enough
for this to work, alltho it is math and geometry method based
on a perfect sphere).

Probably the program is br0ken now since Wikipedia should have
changed their UI and/or API many times since that program
written also ages ago. But the math I hope is still OK.

MATH

First part, 12 files.

* random-urandom.c
  https://dataswamp.org/~incal/emacs-init/random-urandom/random-urandom.c

  Dynamic module to get to Linux urandom which is more random
  than `random', so you can use that from Elisp as well.

* all-substance.el
  https://dataswamp.org/~incal/emacs-init/all-substance.el

  Substance reduction (half-time).

* audio.el
  https://dataswamp.org/~incal/emacs-init/audio.el

  db multiplier.

* beer.el
  https://dataswamp.org/~incal/emacs-init/beer.el

  Convert between beer types, e.g. how many 5.3% beer are one
  six-pack 3.5%?

* bike.el
  https://dataswamp.org/~incal/emacs-init/bike.el

  Compute the step and gear (roll out).

* color-incal.el
  https://dataswamp.org/~incal/emacs-init/color-incal.el

  Computations on colors, determines "is a color blue"?

* dice.el
  https://dataswamp.org/~incal/emacs-init/dice.el

  Can do 1D6 but also 2D3.

* eggs.el
  https://dataswamp.org/~incal/emacs-init/eggs.el

  Solves the egg math problem.

* epwgen.el
  https://dataswamp.org/~incal/emacs-init/epwgen.el

  How strong is a password.

* frame-size.el
  https://dataswamp.org/~incal/emacs-init/frame-size.el

  More computations on the bicycle, this time the frame.

* hex.el
  https://dataswamp.org/~incal/emacs-init/hex.el

  Decodes a hex string.

* isbn-verify.el
  https://dataswamp.org/~incal/emacs-init/isbn-verify.el

  Verify ISBNs with the check digit and known algorithm.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

If we do this, I think we should do it basic first.

No new representation or new infix notation or anything like
that, nothing too fancy, just implement a bunch of stuff - but
in a systematic way, of course.

All will have to be discussed of course but as for me, I don't
see it as too advanced from the get-go, not in terms of math
and absolutely not in terms of Elisp.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

OK, here the complete file, 28 files, URL to where the text
file is stored, with all references to antagonism removed.

Okay, not a lot more to say. Keep it real and be rational on
the number line, time to study the integers for me
namely ZZZ.

-------------------------------------------------------------------------------
  https://dataswamp.org/~incal/emacs-init/docs/math.txt            Emacs Lisp
-------------------------------------------------------------------------------

MATH - 28 files.

* random-urandom.c
  https://dataswamp.org/~incal/emacs-init/random-urandom/random-urandom.c

  Dynamic module (C language) to get to Linux urandom which is
  more random than `random', so you can use that from Elisp
  as well.

* all-substance.el
  https://dataswamp.org/~incal/emacs-init/all-substance.el

  Substance reduction (half-time) over time.

* audio.el
  https://dataswamp.org/~incal/emacs-init/audio.el

  Audio db multiplier.

* beer.el
  https://dataswamp.org/~incal/emacs-init/beer.el

  Convert between beer types, e.g. how many 5.3% beers are one
  six-pack 3.5% plus one cider of this volume etc?

* bike.el
  https://dataswamp.org/~incal/emacs-init/bike.el

  Compute the step and gear (roll out).

* color-incal.el
  https://dataswamp.org/~incal/emacs-init/color-incal.el

  Computations on colors, determines "is this color blue?"
  with a method for that.

* dice.el
  https://dataswamp.org/~incal/emacs-init/dice.el

  Can do 1D6, 2D3 etc.

* eggs.el
  https://dataswamp.org/~incal/emacs-init/eggs.el

  Solves an egg math problem.

* epwgen.el
  https://dataswamp.org/~incal/emacs-init/epwgen.el

  Compute password strength.

* frame-size.el
  https://dataswamp.org/~incal/emacs-init/frame-size.el

  More computations on the bicycle, this time the frame.

* hex.el
  https://dataswamp.org/~incal/emacs-init/hex.el

  Decodes a hex string.

* isbn-verify.el
  https://dataswamp.org/~incal/emacs-init/isbn-verify.el

  Verify ISBNs with the check digit and a known algorithm.

* issn-verify.el
  https://dataswamp.org/~incal/emacs-init/issn-verify.el

Verify ISSN numbers.

* lights.el
  https://dataswamp.org/~incal/emacs-init/lights.el

Compute LED luminous efficacy (lm/W).

* list.el
  https://dataswamp.org/~incal/emacs-init/list.el

Stats on lists.

* math.el
  https://dataswamp.org/~incal/emacs-init/math.el

The infamous math file. 23 `defun'.

* negative-subtraction.el
  https://dataswamp.org/~incal/emacs-init/negative-subtraction.el

Subtraction digit by digit.

* perm.el
  https://dataswamp.org/~incal/emacs-init/perm.el

Permutations. Known from the math field combinatorics but also
stats and actually many science areas.
Bonus: this file solves a fun word problem from a TV show.

* psea.el
  https://dataswamp.org/~incal/emacs-init/psea.el
  https://dataswamp.org/~incal/emacs-init/b-a-original.el

Has to do with string distance and then stats and tables on
that. Yes, I tried that two times it seems! In b-a-original.el
you can output this kind of table (below) saying the first
line is 81% original - so scores a number 1 position at
407 originality points. Based on a weighted sum.

  1. (require 'pcase)      81% 407
  2. (require 'cl-lib)     80% 402
  3. (require 'psea)       80% 402
  4. (require 'subr-x)     80% 402
  5. (require 'thingatpt)  77% 387

* random-generic.el (random.el)
  https://dataswamp.org/~incal/emacs-init/random-generic.el
  https://dataswamp.org/~incal/emacs-init/random.el

Randomizer for arbitrary data types, so you can feed whatever
into it and mix it up transparently. Plus some misc random
stuff, looks like it is more practical in nature but
randomness is a part of math (and stats, and CS, and ...) so
I include it.

* scale.el
  https://dataswamp.org/~incal/emacs-init/scale.el

Draw a scale.

[ Sorting we don't consider math, right? But CS? OK, not included. ]

* stats.el
  https://dataswamp.org/~incal/emacs-init/stats.el

I'm not sure what this does, it outputs some digit set with
some constrains? I'll look it up, looks interesting! TODO

* str-to-bits.el
  https://dataswamp.org/~incal/emacs-init/str-to-bits.el

(str-to-bits "What's up Emacs community?")
 111111 1111001 1110100 1101001 1101110 1110101 1101101 1101101
1101111 1100011  100000 1110011 1100011 1100001 1101101 1000101
 100000 1110000 1110101  100000 1110011  100111 1110100 1100001
1101000 1010111

* survivor.el
  https://dataswamp.org/~incal/emacs-init/survivor.el

[ OT: read if you are from Australia or Sweden. Maybe the US ]

This is computation on the TV show Survivor (or
Expedition Robinson as the English original idea was called on
Swedish TV where it was first realized). Computation is based
on the idea that it gets more difficult all the time but not
linearly so but much worse. So, the difficulty at day 48 in
Australian Survivor is computed to 1176 (compare for example
US Survivor at only 27 days and scores a mere 378). Is
AUS Survivor the world's most difficult reality TV show? Well,
the numbers don't lie ;) Congrats to AUS people reading this,
you are a tough bunch - in a good way - and thanks for that
awsome show, the very best national instance ever of the show.

Here is the full list:

;; (cost-at-day 48) ; 1176 AUS
;; (cost-at-day 39) ;  780 USA old era
;; (cost-at-day 28) ;  406 SWE
;; (cost-at-day 27) ;  378 USA new era
;; (cost-at-day  1) ;    1
;; (cost-at-day  0) ;    0

* time-cmp.el
  https://dataswamp.org/~incal/emacs-init/time-cmp.el

Computation on time. Very useful.

* variance.el
  https://dataswamp.org/~incal/emacs-init/variance.el

This has to do with stats, one is to parse a file with data.
Very old but looks good anyway.

* wood.el
  https://dataswamp.org/~incal/emacs-init/wood.el

How to cut wood and build with wood. Several ideas are
implemented here and I have tried them in practice and they
work there as well (or especially there). See photo linked
from file.

also:

* distance.el
  https://dataswamp.org/~incal/distance/distance.el

Calculate the distance between two Earth locations using the
Haversine formula (yes, the globe is very much so round enough
for this to work, alltho it is math and geometry method based
on the perfect sphere abstract concept).

Probably the program is br0ken now since Wikipedia should have
changed their UI and/or API many times since that program was
written, also ages ago. But the math is still OK.

-------------------------------------------------------------------------------
  incal@dataswamp.org                                              2024-07-21
-------------------------------------------------------------------------------

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Sunday, July 21, 2024 at 5:19 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Richard Stallman wrote:
>
> >> but I also have a math.el which I yank below if any of it
> >> can be used.
> >
> > Most of the fnctions in this library are very close
> > to trivial.
>
> Oh, my! Eli thinks they are so exotic only I use them, RMS
> thinks they are close to trivial.
>
> But I have many more files, I'll see what I have that can
> maybe impress you and Eli more.

We should ask the community what tools they would like to use,
whether combinatorics, random numbers (Gaussian, Bernoulli, Binomial), ...

May I suggest "Units and Conversions".  "Symbolic Simplification"
could also be a candidate.

To promote a mathematical library in Emacs and ensure its broad adoption,
we can think of some things outside the realm of your libraries but which
others would appreciate.

> I have already showed you math.el and perm.el, here are two
> more. But I'll look thru it all some other day and make
> a directory with all of them.
>
> ;;; -*- lexical-binding: t -*-
> ;;
> ;; this file:
> ;;   https://dataswamp.org/~incal/emacs-init/stats.el
>
> (require 'cl-lib)
>
> (defun decade ()
>   (interactive)
>   (cl-loop
>     with entries = ()
>     for d from 190 to 202
>     do (push (how-many (format "^.\\{14\\}%d[[:digit:]]" d)) entries)
>     finally (message "%s" (nreverse entries)) ))
>
> (defun random-distribution (n sum &optional min no-error)
>   "Return a list of N random numbers.
> \nThe sum of the numbers is SUM.
> \nEvery number is at least MIN. (default: 0)
> \nIf NO-ERROR, recompute unsolvable systems with MIN = 0. (default: nil)"
>   (or min (setq min 0))
>   (let ((min-share (* n min)))
>     (if (< sum min-share)
>         (if no-error
>             (random-distribution n sum)
>           (error "Bogus indata, unsolvable") )
>       (let ((vals (make-list n min))
>             (pool (- sum min-share)) )
>         (cl-loop while (< 0 pool)
>                  do (cl-incf (nth (random n) vals))
>                     (cl-decf pool) )
>         vals) )))
>
> (defalias 'rd #'random-distribution)
>
> ;; (apply #'+ (rd 10 100 3)) ; sum is 100, OK
> ;; (rd 10 100)               ; omitted/default min = 0, OK
> ;; (rd 10 100 10)            ; border case, but OK
> ;; (rd 10 100 20)            ; error: Bogus indata, unsolvable
> ;; (rd 10 100 20 t)          ; unsolvable but no-error, recompute for min = 0
> ;;
> ;; 10 example distributions for
> ;; n = 10, sum = 100, and min = 3,
> ;; i.e. (rd 10 100 3):
> ;;
> ;;   (12  8  8 10  7  6 19  9  7 14)
> ;;   ( 8 12  7 12  9 10  9  9 12 12)
> ;;   (14  6 15 11  9  9  6  8 11 11)
> ;;   ( 8 12  7  9 10  9 10 13 10 12)
> ;;   (12 12 15  6 10  8  6 10 14  7)
> ;;   ( 9  7 10  8 11 10  9 10 13 13)
> ;;   ( 7 13 13 10 13 10  5 13  5 11)
> ;;   (13  8  6  9 13  8 11  7 10 15)
> ;;   ( 7 10  5  6 17 14  8 10 14  9)
> ;;   (11 10  8  9 11 10 14 11  8  8)
>
> (provide 'stats)
>
> ;;; -*- lexical-binding: t -*-
> ;;
> ;; help from:
> ;;   https://codereview.stackexchange.com/q/186840
> ;;
> ;; this file:
> ;;   https://dataswamp.org/~incal/emacs-init/str-to-bits.el
>
> (require 'esh-util)
>
> (defun num-to-bits (num)
>   (let ((s ""))
>     (while (> num 0)
>       (setq s (concat (number-to-string (logand num 1)) s))
>       (setq num (ash num -1)) )
>     (if (string= "" s)
>         "0"
>       s) ))
>
> (defalias 'dec2bits  #'num-to-bits)
> (defalias 'char2bits #'num-to-bits)
>
> (defun str-to-bits (str &optional delim)
>   (interactive "sString: ")
>   (or delim (setq delim " "))
>   (let ((bits))
>     (dolist (c (string-to-list str) (mapconcat #'eshell-stringify bits delim))
>       (push (num-to-bits c) bits) )))
>
> ;; (str-to-bits "What's up Emacs community?") ; 111111 1110000 1110101 [...]
>
> (provide 'str-to-bits)
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> We should ask the community what tools they would like to
> use, whether combinatorics, random numbers (Gaussian,
> Bernoulli, Binomial), ...
>
> May I suggest "Units and Conversions". "Symbolic
> Simplification" could also be a candidate.
>
> To promote a mathematical library in Emacs and ensure its
> broad adoption, we can think of some things outside the
> realm of your libraries but which others would appreciate.

Yes, I'll see what we can find and reuse. They are not really
complete libraries, even those who sound like they are - time,
random, permutations etc - they are just files where I put
everything that has to do with it.

It is not a bad method, but does not intend to create complete
libraries. None are and as for the whole math field,
that's huge, not covered by my files, haha (impossible).

I'd like to join this project formally now, after so many
times suggesting it and now doing this as well.

But for sure, I won't be able to do it alone and also I don't
have _that_ strong background in math. I did complete a bunch
of university courses, but yeah, how much did I understand,
truly? It feels like: to some degree that matters, sure, and
to a practical degree, fine, but not to that HUGE
a degree, really.

I know one person, from Poland, but he is probably busy.
And I know one person from Russia, maybe he is reading this?
Anyone else, join - if we get to be a bunch of guys it will
not be that difficult and big a project. Or rather, all it
will take is time.

Yes, you Mr Dimech, join as well. Why not?

I'll end this by sharing this file. Does the ASCII look good?
(If not, follow the URL.)

If you want to evaluate, first do (defalias '** #'expt)

-------------------------------------------------------------------------------
  BINARY UNITS                                            incal@dataswamp.org
-------------------------------------------------------------------------------




        name               bytes   unit   max value
-------------------------------------------------------------------------------
   char/byte                   1          (1- (** 2 (* 8     (** 2  0)   )))
        word                   2          (1- (** 2 (* 8     (** 2  1)   )))
  doubleword                   4          (1- (** 2 (* 8     (** 2  2)   )))
    quadword                   8          (1- (** 2 (* 8     (** 2  3)   )))
   paragraph                  16          (1- (** 2 (* 8     (** 2  4)   )))
    kilobyte               1 024   KiB    (1- (** 2 (* 8 (** (** 2 10) 1))))
    megabyte           1 048 576   MiB    (1- (** 2 (* 8 (** (** 2 10) 2))))
    gigabyte       1 073 741 824   GiB    (1- (** 2 (* 8 (** (** 2 10) 3))))
    terabyte   1 099 511 627 776   TiB    (1- (** 2 (* 8 (** (** 2 10) 4))))
-------------------------------------------------------------------------------
                                                                 ^^^^
                                                                 1024



-------------------------------------------------------------------------------
  https://dataswamp.org/~incal/data/BINARY-UNITS                   2022-12-01
-------------------------------------------------------------------------------

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Sunday, July 21, 2024 at 8:03 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech wrote:
>
> > We should ask the community what tools they would like to
> > use, whether combinatorics, random numbers (Gaussian,
> > Bernoulli, Binomial), ...
> >
> > May I suggest "Units and Conversions". "Symbolic
> > Simplification" could also be a candidate.
> >
> > To promote a mathematical library in Emacs and ensure its
> > broad adoption, we can think of some things outside the
> > realm of your libraries but which others would appreciate.
>
> Yes, I'll see what we can find and reuse. They are not really
> complete libraries, even those who sound like they are - time,
> random, permutations etc - they are just files where I put
> everything that has to do with it.
>
> It is not a bad method, but does not intend to create complete
> libraries. None are and as for the whole math field,
> that's huge, not covered by my files, haha (impossible).
>
> I'd like to join this project formally now, after so many
> times suggesting it and now doing this as well.
>
> But for sure, I won't be able to do it alone and also I don't
> have _that_ strong background in math. I did complete a bunch
> of university courses, but yeah, how much did I understand,
> truly? It feels like: to some degree that matters, sure, and
> to a practical degree, fine, but not to that HUGE
> a degree, really.

There are among us with extremely strong mathematical background.

Could you list for us a categorisation of the five main areas your
libraries focus on.  Please use the standard classification
from https://msc2020.org/MSC_2020.pdf

Example List

05 Combinatorics
51 Geometry
60 Probability
62 Statistics

You could focus upon devising the library framework.  Although your
mathematical background might not be strong, your elisp should be.

I will focus on what can be included for a first release.  But I need
approval from at least two others (e.g. Eli, Richard, Stefan, Michael).

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> There are among us with extremely strong
> mathematical background.

Yes, I know, they are welcome to join us :)

> Could you list for us a categorisation of the five main
> areas your libraries focus on. Please use the standard
> classification from
> https://msc2020.org/MSC_2020.pdf
>
> Example List
>
> 05 Combinatorics
> 51 Geometry
> 60 Probability
> 62 Statistics
>
> You could focus upon devising the library framework.
> Although your mathematical background might not be strong,
> your elisp should be.

There is no pulling rank, everyone is one the same level.
But I can show you my CV, sure. It is no secret and I have no
problem with it.

But yes, I already have a package at ELPA so I know some of
the process anyway!

This will be a consensus based project. Conflicts are not
allowed, public dismissiveness of people efforts or
backgrounds isn't either, the worst thing anyone is allowed to
say is "we need to work more on this".

Before we think about who does what, we have to agree what to
do first in pretty good details.

> I will focus on what can be included for a first release.
> But I need approval from at least two others (e.g. Eli,
> Richard, Stefan, Michael).

Stefan and Michael <3

Can't you guys join?

As long as there it is consensus based group and no conflicts
allowed, no dismissive or elitist attitude, no web cop, no
leadership group, none of that. That only attracts people who
are into power and status, I know this from many, many places.
So you just have to say we don't do that, and they don't come.
I know they won't, this method works.

Np: As soon as there is confusion, we stop and we figure
it out.

If this is a project for you, let's do it :)

But let's see if someone more would like to join.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

> This will be a consensus based project. Conflicts are not
> allowed, public dismissiveness of people efforts or
> backgrounds isn't either, the worst thing anyone is allowed
> to say is "we need to work more on this".

If people really, really insist on one guy in charge then
I'm not 100% against it but then ... find someone we are sure
of isn't in it for the power at all.

Some nerdy/intellectual guy who has been active with his stuff
and been talking on the MLs optimally so we know him.

Because if he does "meaningless things", that only he (or she)
is into, then we know, okay, he is into that, he didn't come
here for any other reason.

Because people who are into power - avoid, avoid, avoid.
And they are much more common than I would have ever thought!
And they are everywhere, literally, including places where you
would be like "who would ever, ever care one bit who is in
charge HERE". But not only do these people care, that was the
reason they came. Yes, it is scary.

Well, I'm sure you guys solve this in a good way.

Ah, man, I'm too tired, that is why I write so long about
everything. See you ... zzz

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Christopher Dimech]

> Sent: Sunday, July 21, 2024 at 11:20 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> > This will be a consensus based project. Conflicts are not
> > allowed, public dismissiveness of people efforts or
> > backgrounds isn't either, the worst thing anyone is allowed
> > to say is "we need to work more on this".
>
> If people really, really insist on one guy in charge then
> I'm not 100% against it but then ... find someone we are sure
> of isn't in it for the power at all.
>
> Some nerdy/intellectual guy who has been active with his stuff
> and been talking on the MLs optimally so we know him.

All depends on who wants to produce a first release that handles
two initial mathematical classifications, and which helps others
extend the library to other additional topic classification.

The choice would be of someone who would actually get an agreed
first release out.

> Because if he does "meaningless things", that only he (or she)
> is into, then we know, okay, he is into that, he didn't come
> here for any other reason.
>
> Because people who are into power - avoid, avoid, avoid.
> And they are much more common than I would have ever thought!
> And they are everywhere, literally, including places where you
> would be like "who would ever, ever care one bit who is in
> charge HERE". But not only do these people care, that was the
> reason they came. Yes, it is scary.
>
> Well, I'm sure you guys solve this in a good way.
>
> Ah, man, I'm too tired, that is why I write so long about
> everything. See you ... zzz
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> All depends on who wants to produce a first release that
> handles two initial mathematical classifications, and which
> helps others extend the library to other additional
> topic classification.

Yes, geometry and arithmetic, right?

Let's start with arithmetic then. We make a plan what we want,
what we already have, obviously we don't do, and we do
the rest.

> The choice would be of someone who would actually get an
> agreed first release out.

What parts of arithmetics would you like to have? You are the
math guy then, if that is the way you like it. You can tell me
what functions and operators are missing - maybe definitions,
actually whatever. Most should be possible to put into code
quite easily, I dare say.

Let's start coding :)

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

> What parts of arithmetics would you like to have? You are
> the math guy then, if that is the way you like it. You can
> tell me what functions and operators are missing - maybe
> definitions, actually whatever.

And applications, like formulas and math functions.

This -

(defun distance-point (min max)
  (+ min (/ (- max min) 2.0)) )

- or special, fun things. This even has the name arithmetic in
it. Well, almost.

(defun arith-sum (n)
  (cl-loop
    with sum = 0
    for i from 1 to (1- n)
    do (cl-incf sum i)
    finally return sum) )

(Hm - there are many ways to solve that, and this is fine by
all means.)

You can also compare Emacs arithmetic capabilities with what
you find in a book. Do we have everything? Or do we want more?
That is where it begins.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Made it a little more easy to see.

Ah, I think the only straight thing is find a textbook from
the bookshelf or get one on the black market and see what
stuff they focus on. And then think if we do the same or think
differently case-by-case.

With these building blocks you can do a lot and we have a lot
already in core Emacs already and in ELPA for that matter.

Right, people, send me e-mails if this isn't a good place to
discuss this back and forth :)

Arithmetic

(+ &rest numbers)
(- &rest numbers)
(* &rest numbers)
(/ number &rest divisors)
(% x y)
(mod x y)
(1+ number)
(1- number)

Predicates

(= number &rest numbers)
(eql obj1 obj2)
(/= num1 num2)
(< number &rest numbers)
(<= number &rest numbers)
(> number &rest numbers)
(>= number &rest numbers)
(zerop number)
(natnump object)
(cl-plusp number)
(cl-minusp number)
(cl-oddp integer)
(cl-evenp integer)
(bignump object)
(fixnump object)
(floatp object)
(integerp object)
(numberp object)
(cl-digit-char-p char &optional radix)

Operations

(max number &rest numbers)
(min number &rest numbers)
(abs arg)
(float arg)
(truncate arg &optional divisor)
(floor arg &optional divisor)
(ceiling arg &optional divisor)
(round arg &optional divisor)
(random &optional limit)

Bit Operations

(ash value count)
(logand &rest ints-or-markers)
(logior &rest ints-or-markers)
(logxor &rest ints-or-markers)
(lognot number)
(logcount value)

Floating Point

(isnan x)
(frexp x)
(ldexp sgnfcand exponent)
(logb arg)
(ffloor arg)
(fceiling arg)
(ftruncate arg)
(fround arg)

Standard Math Functions

(sin arg)
(cos arg)
(tan arg)
(asin arg)
(acos arg)
(atan y &optional x)
(exp arg)
(log arg &optional base)
(expt arg1 arg2)
(sqrt arg)

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

Emanuel, looking at calc.el you will find

;;   Provide a better implementation for math-sin-cos-raw.
;;   Provide a better implementation for math-hypot.

You can focus on these two things so you can provide us with better
implementations of those two things.  Together with the necessary
framework for extension and for use by calc.

> Sent: Monday, July 22, 2024 at 12:27 AM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> > What parts of arithmetics would you like to have? You are
> > the math guy then, if that is the way you like it. You can
> > tell me what functions and operators are missing - maybe
> > definitions, actually whatever.
>
> And applications, like formulas and math functions.
>
> This -
>
> (defun distance-point (min max)
>   (+ min (/ (- max min) 2.0)) )
>
> - or special, fun things. This even has the name arithmetic in
> it. Well, almost.
>
> (defun arith-sum (n)
>   (cl-loop
>     with sum = 0
>     for i from 1 to (1- n)
>     do (cl-incf sum i)
>     finally return sum) )
>
> (Hm - there are many ways to solve that, and this is fine by
> all means.)
>
> You can also compare Emacs arithmetic capabilities with what
> you find in a book. Do we have everything? Or do we want more?
> That is where it begins.
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> Provide a better implementation for math-sin-cos-raw
> Provide a better implementation for math-hypot

I'll get right to it!

If just Eli and Richard could spend a couple of hours
explaining what improvements are called for?

So there won't be any suddenly divergent ideas, I mean?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Sun, 21 Jul 2024 15:17:01 +0200
> 
> Christopher Dimech wrote:
> 
> > Provide a better implementation for math-sin-cos-raw
> > Provide a better implementation for math-hypot
> 
> I'll get right to it!
> 
> If just Eli and Richard could spend a couple of hours
> explaining what improvements are called for?

I don't know what math-sin-cos-raw is about, but as for math-hypot,
there's no need to implement anything, since this function is standard
in any C libm, and we can easily expose it to Lisp if we need to.

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 2:33 AM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: "Emanuel Berg" <incal@dataswamp.org>
> Cc: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> > From: Emanuel Berg <incal@dataswamp.org>
> > Date: Sun, 21 Jul 2024 15:17:01 +0200
> >
> > Christopher Dimech wrote:
> >
> > > Provide a better implementation for math-sin-cos-raw
> > > Provide a better implementation for math-hypot
> >
> > I'll get right to it!
> >
> > If just Eli and Richard could spend a couple of hours
> > explaining what improvements are called for?
>
> I don't know what math-sin-cos-raw is about, but as for math-hypot,
> there's no need to implement anything, since this function is standard
> in any C libm, and we can easily expose it to Lisp if we need to.

Eli. perhaps there can be a math library that calc can use.  We will
then have a generic math library, and calc and other packages could use
some of its functionality.  Would that work for you ?  But if Emanuel
does not want to touch it, what is the point !

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Christopher Dimech <dimech@gmx.com>
> Cc: Emanuel Berg <incal@dataswamp.org>, emacs-devel@gnu.org
> Date: Sun, 21 Jul 2024 16:41:38 +0200
> Sensitivity: Normal
> 
> > > If just Eli and Richard could spend a couple of hours
> > > explaining what improvements are called for?
> >
> > I don't know what math-sin-cos-raw is about, but as for math-hypot,
> > there's no need to implement anything, since this function is standard
> > in any C libm, and we can easily expose it to Lisp if we need to.
> 
> Eli. perhaps there can be a math library that calc can use.  We will
> then have a generic math library, and calc and other packages could use
> some of its functionality.  Would that work for you ?  But if Emanuel
> does not want to touch it, what is the point !

This is too abstract a question.

The original issue was not about Calc, it was about a standalone math
library.  For that, we should first define the scope.  One possibility
is to expose to Lisp everything in a typical libm, but is that needed?
Another possibility is to provide a statistical library (average,
median, standard deviation, t-Student, F-test, chi-square, etc.) --
but do we need this?  Etc. etc. -- "math library" can be interpreted
in many different ways.

If we are now talking about improving Calc, someone who knows Calc
well should describe what is needed, and we can then discuss that in
detail.

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 2:49 AM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: "Christopher Dimech" <dimech@gmx.com>
> Cc: incal@dataswamp.org, emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> > From: Christopher Dimech <dimech@gmx.com>
> > Cc: Emanuel Berg <incal@dataswamp.org>, emacs-devel@gnu.org
> > Date: Sun, 21 Jul 2024 16:41:38 +0200
> > Sensitivity: Normal
> >
> > > > If just Eli and Richard could spend a couple of hours
> > > > explaining what improvements are called for?
> > >
> > > I don't know what math-sin-cos-raw is about, but as for math-hypot,
> > > there's no need to implement anything, since this function is standard
> > > in any C libm, and we can easily expose it to Lisp if we need to.
> >
> > Eli. perhaps there can be a math library that calc can use.  We will
> > then have a generic math library, and calc and other packages could use
> > some of its functionality.  Would that work for you ?  But if Emanuel
> > does not want to touch it, what is the point !
>
> This is too abstract a question.
>
> The original issue was not about Calc, it was about a standalone math
> library.  For that, we should first define the scope.  One possibility
> is to expose to Lisp everything in a typical libm, but is that needed?
> Another possibility is to provide a statistical library (average,
> median, standard deviation, t-Student, F-test, chi-square, etc.) --
> but do we need this?  Etc. etc. -- "math library" can be interpreted
> in many different ways.
>
> If we are now talking about improving Calc, someone who knows Calc
> well should describe what is needed, and we can then discuss that in
> detail.

Using Emacs Calc as a foundation for a new mathematical library
in Emacs is a practical and efficient approach.  Calc is a powerful tool
with extensive mathematical capabilities, so building on it allows
for leveraging its strengths while focusing on specific needs and
functionalities.

I rather have the people who work on calc do it as they have much more
experience and grasp of what a math library for emacs should do.

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Christopher Dimech <dimech@gmx.com>
> Cc: incal@dataswamp.org, emacs-devel@gnu.org
> Date: Sun, 21 Jul 2024 16:58:44 +0200
> 
> Using Emacs Calc as a foundation for a new mathematical library
> in Emacs is a practical and efficient approach.  Calc is a powerful tool
> with extensive mathematical capabilities, so building on it allows
> for leveraging its strengths while focusing on specific needs and
> functionalities.

I don't understand what "building on Calc" means in practice.  Any
Lisp program can load Calc and invoke its functions, so what else is
needed?  IOW, why what we already have in Calc is not enough?

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 3:02 AM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: "Christopher Dimech" <dimech@gmx.com>
> Cc: incal@dataswamp.org, emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> > From: Christopher Dimech <dimech@gmx.com>
> > Cc: incal@dataswamp.org, emacs-devel@gnu.org
> > Date: Sun, 21 Jul 2024 16:58:44 +0200
> >
> > Using Emacs Calc as a foundation for a new mathematical library
> > in Emacs is a practical and efficient approach.  Calc is a powerful tool
> > with extensive mathematical capabilities, so building on it allows
> > for leveraging its strengths while focusing on specific needs and
> > functionalities.
>
> I don't understand what "building on Calc" means in practice.  Any
> Lisp program can load Calc and invoke its functions, so what else is
> needed?  IOW, why what we already have in Calc is not enough?

Agreed.  Ask Emanuel, I don't know.  Do the calc people think we will be
better with an additional library that calc could also use ?  I would stick
to what they say.

Adding a generic mathematical library

[Christopher Dimech]

----- Kristof Đymek
Administrator General - Chilkat Design Build - Naiad Informatics - Gnu Project

Society has become too quick to pass judgement and declare someone
Persona Non-Grata, the most extreme form of censure a country can
bestow.

In a new era of destructive authoritarianism, I support Richard
Stallman.  Times of great crisis are also times of great
opportunity.  I call upon you to make this struggle yours as well !

https://www.gnu.org     https://www.fsf.org/


> Sent: Monday, July 22, 2024 at 1:03 AM
> From: "Christopher Dimech" <dimech@gmx.com>
> To: "Emanuel Berg" <incal@dataswamp.org>
> Cc: emacs-devel@gnu.org
> Subject: Adding a generic mathematical library
>
> Emanuel, looking at calc.el you will find
> 
> ;;   Provide a better implementation for math-sin-cos-raw.
> ;;   Provide a better implementation for math-hypot.
> 
> You can focus on these two things so you can provide us with better
> implementations of those two things.  Together with the necessary
> framework for extension and for use by calc.

math-hypot is found in file calc-math.el
 
> > Sent: Monday, July 22, 2024 at 12:27 AM
> > From: "Emanuel Berg" <incal@dataswamp.org>
> > To: emacs-devel@gnu.org
> > Subject: Re: Adding a generic mathematical library
> >
> > > What parts of arithmetics would you like to have? You are
> > > the math guy then, if that is the way you like it. You can
> > > tell me what functions and operators are missing - maybe
> > > definitions, actually whatever.
> >
> > And applications, like formulas and math functions.
> >
> > This -
> >
> > (defun distance-point (min max)
> >   (+ min (/ (- max min) 2.0)) )
> >
> > - or special, fun things. This even has the name arithmetic in
> > it. Well, almost.
> >
> > (defun arith-sum (n)
> >   (cl-loop
> >     with sum = 0
> >     for i from 1 to (1- n)
> >     do (cl-incf sum i)
> >     finally return sum) )
> >
> > (Hm - there are many ways to solve that, and this is fine by
> > all means.)
> >
> > You can also compare Emacs arithmetic capabilities with what
> > you find in a book. Do we have everything? Or do we want more?
> > That is where it begins.
> >
> > --
> > underground experts united
> > https://dataswamp.org/~incal
> >
> >
> >
> 
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> [...]

Yeah, as for you, feel free to do whatever you want.

As for me, I don't think I'm getting into calc, ever, sorry :)

Fiddling with libraries - why not - but if they are software
underbelly, to me that isn't a library, more like a poor man's
modular solution or a C or C++ header file associated with
a particular main file.

If you think of libraries like that then yes, my Elisp files,
all of them, are libraries. You want to use them? Sorry, I still
need my computer!

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 1:26 AM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech wrote:
>
> > [...]
>
> Yeah, as for you, feel free to do whatever you want.
>
> As for me, I don't think I'm getting into calc, ever, sorry :)
>
> Fiddling with libraries - why not - but if they are software
> underbelly, to me that isn't a library, more like a poor man's
> modular solution or a C or C++ header file associated with
> a particular main file.

Calc will then start using the library you suggest.  The task is to produce
an improvement of hypot.  If one suggests a mathematical library
it has to fit with the current tools.  Calc has the capability to
do the things you suggest and is extremely much more advanced.

To get what you want, you have to agree with something.  Otherwise
you get the resistance you have witnessed.  I am in agreement with Eli
on this thing.  We should first discuss its scope and design.

> If you think of libraries like that then yes, my Elisp files,
> all of them, are libraries. You want to use them? Sorry, I still
> need my computer!
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> To get what you want, you have to agree with something.
> Otherwise you get the resistance you have witnessed. I am in
> agreement with Eli on this thing. We should first discuss
> its scope and design.

Maybe there is some field of math that isn't taken by all
mighty Calc already?

That can happen. Scope would then be that. Design would be to
program a bunch of functions, operators, constants, combine
into popular formulas and so on.

I think the names are the only thing one has to agree on, what
to call everything. Is there and ISO document to tell you?

If not, one could agree to use some hefty Anglo-American
textbook with a good reputation and a modern edition and then
just consistently do whatever they do.

Most important thing is to have a plan, a good plan, and then
stick to it.

In general quite clearly: "Straight math" (Linear Algebra) is
easier to program than the derivative, limes functions,
probability distributions, and such stuff - also integers, as
in Discrete Math, are easier than floats (Analysis).

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> To get what you want, you have to agree with something.
> Otherwise you get the resistance you have witnessed. I am in
> agreement with Eli on this thing. We should first discuss
> its scope and design.

Relational Algebra - we should have a lot of that as they are
set based and we are list based, and a list is an ordered set
but don't have to be - don't know if we have everything.

Don't know what more math there is. Automata Theory if that
isn't considered computers actually.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> To get what you want, you have to agree with something.
> Otherwise you get the resistance you have witnessed. I am in
> agreement with Eli on this thing. We should first discuss
> its scope and design.

It matters where things are put, as long as they are shipped
and can be found from anywhere maybe that can be handled then.

It matters what things are called, if they have some long
prefix from some package it is like a joke asking math people
to use it - I'm not one of them but I know how they think and
would feel like an idiot handing them such software - if this
can be solved as well, so no prefix and ISO/conventional names
for everything according to a chosen method, then only one
thing remains and that is ...

The scope of the "library", will it also have a bunch of Calc
specific helpers, settings and so on?

Because if it doesn't it is just located in the Calc dir.
Why ever anyone would place it there I don't know but also
don't really care.

So yes, examine what Calc doesn't have then.

Individual functions ... hard to care about them. After you do
one, someone else will pop up, and you can spend a decade on
that and not a single new file or anything visibly
has improved. So no thanks to that.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> To get what you want, you have to agree with something.
> Otherwise you get the resistance you have witnessed. I am in
> agreement with Eli on this thing. We should first discuss
> its scope and design.

This is the reason the same code gets written over and over in
the Elisp world.

This is a more common thing than really odd inclusion of
software where one program that has absolutely nothing to do
with some other still has that (require 'software) - ???

People want to do something to their number or string, there
is such a string function, but it is in Gnus. Their program
has nothing to do with Gnus so to bring that in is insane.

But is it better to kill/yank the Gnus function and rename it
and have the same thing two, three ... times? No, both
alternatives are equally bad.

And the solution are libraries for everyone to use, not mere
files one is expected to find is some alien body of code and
then ... yes, what are you supposed to do in that situation,
if I may have a good answer to that question?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> To get what you want, you have to agree with something.
> Otherwise you get the resistance you have witnessed. I am in
> agreement with Eli on this thing. We should first discuss
> its scope and design.

To show you an example of what I mentioned in a post that
hasn't arrived, check out line number 8. What do you see?

Yes: ERC!

Elisp is the only language when you include an IRC client to
sort a bunch of wards. Yeah, crazy.

[ BTW don't understand why this is so complicated, should
  re-write that last part. Anyway, very useful! ]

PS. Now I've talked about this enough. You want the truth?
    You can _handle_ the truth. What I can come up with at
    home fiddling with a bunch of Elisp files, Emacs
    programmers cannot solve collectively since 1984?

    That is: 39y 6m 20d if we count from 1984-12-31; no idea
    what exact date GNU Emacs was first operational. Should be
    enough time to get libraries one would think but wrong
    policy, people that are used to leaders that they don't
    question ever ... then this is what happens. Anyway enough
    of this then but some of it was fun :)
DS.

(defun sort-line-words (beg end &optional set-delim)
  (interactive "r\nP")
  (or beg (setq beg (point-at-bol)))
  (or end (setq end (point-at-eol)))
  (let*((str       (buffer-substring-no-properties beg end))
        (delim-str (or set-delim (read-string "delimiter: ")))
        (str-list  (split-string str delim-str))
        (sorted    (erc-sort-strings str-list)) )
    (kill-region beg end)
    (if set-delim
        (progn
          (dolist (s (nreverse (cdr (nreverse sorted))))
            (insert (format "%s%s" s delim-str)))
          (insert (format "%s" (car (last sorted)))))
      (insert-string-list sorted) )))

;; sort me: a is just string test this
;; sorted:  a is just string test this
;; and me with a dash delim: this-is-just-a-test-string

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 12:10 AM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech wrote:
>
> > All depends on who wants to produce a first release that
> > handles two initial mathematical classifications, and which
> > helps others extend the library to other additional
> > topic classification.
>
> Yes, geometry and arithmetic, right?
>
> Let's start with arithmetic then. We make a plan what we want,
> what we already have, obviously we don't do, and we do
> the rest.
>
> > The choice would be of someone who would actually get an
> > agreed first release out.
>
> What parts of arithmetics would you like to have? You are the
> math guy then, if that is the way you like it. You can tell me
> what functions and operators are missing - maybe definitions,
> actually whatever. Most should be possible to put into code
> quite easily, I dare say.
>
> Let's start coding :)

Richard has suggested extracting a calc function that includes a substantial
amount of code to do a specific kind of calculation.

To address Eli's valid criticism and focus on something practical and
existing, let's enhance the existing emacs calc package for mathematical
computations.

Focusing on improving the calc package, which is already a powerful
tool for mathematical calculations in Emacs.  Specifically, make a
library that extends its capabilities, and which calc can use.

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> Richard has suggested extracting a calc function that
> includes a substantial amount of code to do a specific kind
> of calculation.
>
> To address Eli's valid criticism and focus on something
> practical and existing, let's enhance the existing emacs
> calc package for mathematical computations.

Ah, right, that is true, the way both reacted to this whole
thing and now this timely decision, there is absolutely
nothing fishy going on, they do it out of concern for calc -
I see.

Calc is as you say a package and a tool. Why it should
monopolize and assimilate generic math libraries I wonder?
Instead, it should focus on its interface and some settings
perhaps at the most and when in need of math, consult
a library for that - a generic library that anyone and
everyone could use just as well.

That said, calc has a lot of good stuff, just using that code
isn't natural in any way for whatever computation you would
like to do in Elisp or in your own application? Yeah,
it is completely backwards actually!

Instead of a tool using a library, we here attach the library
as an underbelly to the tool! Maybe some guy in the 80s came
up with that design but, as the saying goes, development has
gone forward.

Truth to the matter, people are not gonna be enthusiastic
about that, everyone knows that, and that tool isn't used
a lot as it is and this is likely to decrease even more.

If we had real libraries, people could do new tools, including
tools we cannot imagine right now. And some people could work
on the libraries, other people could work on the tools. But it
is what it is. Couple of too many mistakes and then it is not
going to happen.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> Richard has suggested extracting a calc function that
> includes a substantial amount of code to do a specific kind
> of calculation.

Guys, what about this idea - we move everything from
string.el, the string library, to gnus.el.

`gnus-string-or' and many other string functions are already
there! And when you think about it, what is Gnus if
not strings?

And then everyone else can use it from Gnus! Simple as that,
Gnus is shipped with vanilla Emacs so should be no
problem whatsoever.

Wait - we _do_ have a string.el library ... ?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> All depends on who wants to produce a first release that
> handles two initial mathematical classifications, and which
> helps others extend the library to other additional
> topic classification.

Here is arithmetic!

What more do we want? :)

Arithmetic:

(+ &rest numbers)
  Return sum of any number of arguments, which are numbers or markers.
  (+ 1 2)
    => 3
  (+ 1 2 3 4)
    => 10
-------------------------------------------------------------------------------

(- &rest numbers)
  Negate number or subtract numbers or markers and return the result.
  (- 3 2)
    => 1
  (- 6 3 2)
    => 1
-------------------------------------------------------------------------------

(* &rest numbers)
  Return product of any number of arguments, which are numbers or markers.
  (* 3 4 5)
    => 60
-------------------------------------------------------------------------------

(/ number &rest divisors)
  Divide number by divisors and return the result.
  (/ 10 5)
    => 2
  (/ 10 6)
    => 1
  (/ 10.0 6)
    => 1.6666666666666667
  (/ 10.0 3 3)
    => 1.1111111111111112
-------------------------------------------------------------------------------

(% x y)
  Return remainder of X divided by Y.
  (% 10 5)
    => 0
  (% 10 6)
    => 4
-------------------------------------------------------------------------------

(mod x y)
  Return X modulo Y.
  (mod 10 5)
    => 0
  (mod 10 6)
    => 4
  (mod 10.5 6)
    => 4.5
-------------------------------------------------------------------------------

(1+ number)
  Return NUMBER plus one.  NUMBER may be a number or a marker.
  (1+ 2)
    => 3
-------------------------------------------------------------------------------

(1- number)
  Return NUMBER minus one.  NUMBER may be a number or a marker.
  (1- 4)
    => 3

Predicates

(= number &rest numbers)
  Return t if args, all numbers or markers, are equal.
  (= 4 4)
    => t
  (= 4.0 4.0)
    => t
  (= 4 4.0)
    => t
  (= 4 4 4 4)
    => t
-------------------------------------------------------------------------------

(eql obj1 obj2)
  Return t if the two args are ‘eq’ or are indistinguishable numbers.
  (eql 4 4)
    => t
  (eql 4.0 4.0)
    => t
-------------------------------------------------------------------------------

(/= num1 num2)
  Return t if first arg is not equal to second arg.  Both must be numbers or markers.
  (/= 4 4)
    => nil
-------------------------------------------------------------------------------

(< number &rest numbers)
  Return t if each arg (a number or marker), is less than the next arg.
  (< 4 4)
    => nil
  (< 1 2 3)
    => t
-------------------------------------------------------------------------------

(<= number &rest numbers)
  Return t if each arg (a number or marker) is less than or equal to the next.
  (<= 4 4)
    => t
  (<= 1 2 2 3)
    => t
-------------------------------------------------------------------------------

(> number &rest numbers)
  Return t if each arg (a number or marker) is greater than the next arg.
  (> 4 4)
    => nil
  (> 3 2 1)
    => t
-------------------------------------------------------------------------------

(>= number &rest numbers)
  Return t if each arg (a number or marker) is greater than or equal to the next.
  (>= 4 4)
    => t
  (>= 3 2 2 1)
    => t
-------------------------------------------------------------------------------

(zerop number)
  Return t if NUMBER is zero.
  (zerop 0)
    => t
-------------------------------------------------------------------------------

(natnump object)
  Return t if OBJECT is a nonnegative integer.
  (natnump -1)
    => nil
  (natnump 0)
    => t
  (natnump 23)
    => t
-------------------------------------------------------------------------------

(cl-plusp number)
  Return t if NUMBER is positive.
  (cl-plusp 0)
    => nil
  (cl-plusp 1)
    => t
-------------------------------------------------------------------------------

(cl-minusp number)
  Return t if NUMBER is negative.
  (cl-minusp 0)
    => nil
  (cl-minusp -1)
    => t
-------------------------------------------------------------------------------

(cl-oddp integer)
  Return t if INTEGER is odd.
  (cl-oddp 3)
    => t
-------------------------------------------------------------------------------

(cl-evenp integer)
  Return t if INTEGER is even.
  (cl-evenp 6)
    => t
-------------------------------------------------------------------------------

(bignump object)
  Return t if OBJECT is a bignum.
  (bignump 4)
    => nil
  (bignump (expt 2 90))
    => t
-------------------------------------------------------------------------------

(fixnump object)
  Return t if OBJECT is a fixnum.
  (fixnump 4)
    => t
  (fixnump (expt 2 90))
    => nil
-------------------------------------------------------------------------------

(floatp object)
  Return t if OBJECT is a floating point number.
  (floatp 5.4)
    => t
-------------------------------------------------------------------------------

(integerp object)
  Return t if OBJECT is an integer.
  (integerp 5.4)
    => nil
-------------------------------------------------------------------------------

(numberp object)
  Return t if OBJECT is a number (floating point or integer).
  (numberp "5.4")
    => nil
-------------------------------------------------------------------------------

(cl-digit-char-p char &optional radix)
  Test if CHAR is a digit in the specified RADIX (default 10).
  (cl-digit-char-p 53 10)
    => 5
  (cl-digit-char-p 102 16)
    => 15

Operations

(max number &rest numbers)
  Return largest of all the arguments (which must be numbers or markers).
  (max 7 9 3)
    => 9
-------------------------------------------------------------------------------

(min number &rest numbers)
  Return smallest of all the arguments (which must be numbers or markers).
  (min 7 9 3)
    => 3
-------------------------------------------------------------------------------

(abs arg)
  Return the absolute value of ARG.
  (abs -4)
    => 4
-------------------------------------------------------------------------------

(float arg)
  Return the floating point number equal to ARG.
  (float 2)
    => 2.0
-------------------------------------------------------------------------------

(truncate arg &optional divisor)
  Truncate a floating point number to an int.
  (truncate 1.2)
    => 1
  (truncate -1.2)
    => -1
  (truncate 5.4 2)
    => 2
-------------------------------------------------------------------------------

(floor arg &optional divisor)
  Return the largest integer no greater than ARG.
  (floor 1.2)
    => 1
  (floor -1.2)
    => -2
  (floor 5.4 2)
    => 2
-------------------------------------------------------------------------------

(ceiling arg &optional divisor)
  Return the smallest integer no less than ARG.
  (ceiling 1.2)
    => 2
  (ceiling -1.2)
    => -1
  (ceiling 5.4 2)
    => 3
-------------------------------------------------------------------------------

(round arg &optional divisor)
  Return the nearest integer to ARG.
  (round 1.2)
    => 1
  (round -1.2)
    => -1
  (round 5.4 2)
    => 3
-------------------------------------------------------------------------------

(random &optional limit)
  Return a pseudo-random integer.
  (random 6)
    => 5

Bit Operations

(ash value count)
  Return integer VALUE with its bits shifted left by COUNT bit positions.
  (ash 1 4)
    => 16
  (ash 16 -1)
    => 8
-------------------------------------------------------------------------------

(logand &rest ints-or-markers)
  Return bitwise-and of all the arguments.
  (logand #b10 #b111)
    => #b10
-------------------------------------------------------------------------------

(logior &rest ints-or-markers)
  Return bitwise-or of all the arguments.
  (logior 4 16)
    => 20
-------------------------------------------------------------------------------

(logxor &rest ints-or-markers)
  Return bitwise-exclusive-or of all the arguments.
  (logxor 4 16)
    => 20
-------------------------------------------------------------------------------

(lognot number)
  Return the bitwise complement of NUMBER.  NUMBER must be an integer.
  (lognot 5)
    => -6
-------------------------------------------------------------------------------

(logcount value)
  Return population count of VALUE.
  (logcount 5)
    => 2

Floating Point

(isnan x)
  Return non-nil if argument X is a NaN.
  (isnan 5.0)
    => nil
-------------------------------------------------------------------------------

(frexp x)
  Get significand and exponent of a floating point number.
  (frexp 5.7)
    => (0.7125 . 3)
-------------------------------------------------------------------------------

(ldexp sgnfcand exponent)
  Return SGNFCAND * 2**EXPONENT, as a floating point number.
  (ldexp 0.7125 3)
    => 5.7
-------------------------------------------------------------------------------

(logb arg)
  Returns largest integer <= the base 2 log of the magnitude of ARG.
  (logb 10.5)
    => 3
-------------------------------------------------------------------------------

(ffloor arg)
  Return the largest integer no greater than ARG, as a float.
  (ffloor 1.2)
    => 1.0
-------------------------------------------------------------------------------

(fceiling arg)
  Return the smallest integer no less than ARG, as a float.
  (fceiling 1.2)
    => 2.0
-------------------------------------------------------------------------------

(ftruncate arg)
  Truncate a floating point number to an integral float value.
  (ftruncate 1.2)
    => 1.0
-------------------------------------------------------------------------------

(fround arg)
  Return the nearest integer to ARG, as a float.
  (fround 1.2)
    => 1.0

Standard Math Functions

(sin arg)
  Return the sine of ARG.
  (sin float-pi)
    => 1.2246467991473532e-16
-------------------------------------------------------------------------------

(cos arg)
  Return the cosine of ARG.
  (cos float-pi)
    => -1.0
-------------------------------------------------------------------------------

(tan arg)
  Return the tangent of ARG.
  (tan float-pi)
    => -1.2246467991473532e-16
-------------------------------------------------------------------------------

(asin arg)
  Return the inverse sine of ARG.
  (asin float-pi)
    => 0.0e+NaN
-------------------------------------------------------------------------------

(acos arg)
  Return the inverse cosine of ARG.
  (acos float-pi)
    => 0.0e+NaN
-------------------------------------------------------------------------------

(atan y &optional x)
  Return the inverse tangent of the arguments.
  (atan float-pi)
    => 1.2626272556789118
-------------------------------------------------------------------------------

(exp arg)
  Return the exponential base e of ARG.
  (exp 4)
    => 54.598150033144236
-------------------------------------------------------------------------------

(log arg &optional base)
  Return the natural logarithm of ARG.
  (log 54.59)
    => 3.9998507157936665
-------------------------------------------------------------------------------

(expt arg1 arg2)
  Return the exponential ARG1 ** ARG2.
  (expt 2 16)
    => 65536
-------------------------------------------------------------------------------

(sqrt arg)
  Return the square root of ARG.
  (sqrt -1)
    => -0.0e+NaN

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Where to actually start? After all this meta, maybe some words
on that the way I see it.

They way I would do it is to identify the simplest level or
place of math. Some people say logic is math, other say math
is based on logic, but to me, here, it doesn't matter since
I think arithmetic is much simpler than logic anyway.

So if we start there (this is again how I see it) one would
look around, what do we already have - this will be a lot, of
course. But are there things we don't have that are essential
and basic, or really interesting for that matter - then we
implement them, because at such a low level, we should cover
all or close to all things that are important.

And one shouldn't think, "well, we don't have it but one can
solve that with Elisp". Because one can solve everything with
Elisp! Here we want specific functions and we want them to be
fast, easy-to-use and readily available. (One will still have
to think what to add or not, as always.)

So when arithmetic is done - good work, pad pad - and we can
move up someplace else, maybe more challenging as well.
And the higher one will come, the more people will just do
what they are into, because it will be impossible to cover
everything of everything when it becomes that specialized.
But that is OK, free software is like that, people do what
they are into. And I dare say that people _will_ want to do
Combinatorics, Probability, Linear Algebra, and all crazy
stuff there is.

A challenge will be to find people who already have that kind
of stuff, maybe half-done. That will be a huge help to see,
what did they do, how far did they come, why didn't they
complete it, etc.

That is: at the basic level one must have pretty much
everything so why not start there?

Know that if we plan this for ELPA people will have made an
active choise to use it, that means we shouldn't be too afraid
to include stuff. They use it because they want it, so we
should make sure there is something there for them to get :)

-- 
underground experts united
https://dataswamp.org/~incal

hypotenuse (was: Re: Adding a generic mathematical library)

[Emanuel Berg]

Richard Stallman wrote:

> Onsider, for instance, `hypotenuse'. The benefit of having
> that functoin would not quite be zero, but it would be
> small. Is it enough to justify the additional function name,
> and the complexity of documenting it?

RMS has really performed excellently in this discussion and my
favorite part of his critique must be this.

What a sharp eye, to immediately identify the problem of
a proposed solution!

Indeed: Is it enough to justify the "complexity of documenting
it"?

That is an excellent question, and I leave it free for all
to contemplate. Can we justify the complexity?

Or does the below two lines of Elisp belong to the dustbin
of computing?

(defun hypotenuse (c1 c2)
  (sqrt (+ (* c1 c1) (* c2 c2))) )

-- 
underground experts united
https://dataswamp.org/~incal

Re: hypotenuse (was: Re: Adding a generic mathematical library)

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Sun, 21 Jul 2024 16:30:26 +0200
> 
> Richard Stallman wrote:
> 
> > Onsider, for instance, `hypotenuse'. The benefit of having
> > that functoin would not quite be zero, but it would be
> > small. Is it enough to justify the additional function name,
> > and the complexity of documenting it?
> 
> RMS has really performed excellently in this discussion and my
> favorite part of his critique must be this.
> 
> What a sharp eye, to immediately identify the problem of
> a proposed solution!
> 
> Indeed: Is it enough to justify the "complexity of documenting
> it"?
> 
> That is an excellent question, and I leave it free for all
> to contemplate. Can we justify the complexity?

You are mocking questions which happen to be serious and non-trivial
to answer.

> Or does the below two lines of Elisp belong to the dustbin
> of computing?
> 
> (defun hypotenuse (c1 c2)
>   (sqrt (+ (* c1 c1) (* c2 c2))) )

This is not the best way of computing hypot: squaring a number could
overflow or underflow, and you get bad result.  Try

    (hypotenuse 1.e-600 1.e-600)
or
    (hypotenuse 1.e600 1.e600)

and you will see it.

Like I said: a high-quality math library needs to use solid algorithms
that produce valid results whenever the problem has a solution.
Stable and accurate algorithms for hypot are known, and the libm
implementation usually follows it (and if it doesn't, we can roll our
own).

Re: hypotenuse (was: Re: Adding a generic mathematical library)

[Eli Zaretskii]

> Date: Sun, 21 Jul 2024 17:44:47 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > (defun hypotenuse (c1 c2)
> >   (sqrt (+ (* c1 c1) (* c2 c2))) )
> 
> This is not the best way of computing hypot: squaring a number could
> overflow or underflow, and you get bad result.  Try
> 
>     (hypotenuse 1.e-600 1.e-600)
> or
>     (hypotenuse 1.e600 1.e600)
> 
> and you will see it.

Sorry, that should have been 1.e-300 and 1.e300 instead.

hypotenuse (was: Re: Adding a generic mathematical library)

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 3:00 AM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: incal@dataswamp.org
> Cc: emacs-devel@gnu.org
> Subject: Re: hypotenuse (was: Re: Adding a generic mathematical library)
>
> > Date: Sun, 21 Jul 2024 17:44:47 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: emacs-devel@gnu.org
> >
> > > (defun hypotenuse (c1 c2)
> > >   (sqrt (+ (* c1 c1) (* c2 c2))) )
> >
> > This is not the best way of computing hypot: squaring a number could
> > overflow or underflow, and you get bad result.  Try
> >
> >     (hypotenuse 1.e-600 1.e-600)
> > or
> >     (hypotenuse 1.e600 1.e600)
> >
> > and you will see it.
>
> Sorry, that should have been 1.e-300 and 1.e300 instead.

I have a solution to the problem in fortran, but not in elisp.  I could
include it.  Its implementation required serious concentration.

Re: hypotenuse (was: Re: Adding a generic mathematical library)

[Eli Zaretskii]

> From: Christopher Dimech <dimech@gmx.com>
> Cc: incal@dataswamp.org, emacs-devel@gnu.org
> Date: Sun, 21 Jul 2024 17:12:10 +0200
> Sensitivity: Normal
> 
> > Sent: Monday, July 22, 2024 at 3:00 AM
> > From: "Eli Zaretskii" <eliz@gnu.org>
> > To: incal@dataswamp.org
> > Cc: emacs-devel@gnu.org
> > Subject: Re: hypotenuse (was: Re: Adding a generic mathematical library)
> >
> > > Date: Sun, 21 Jul 2024 17:44:47 +0300
> > > From: Eli Zaretskii <eliz@gnu.org>
> > > Cc: emacs-devel@gnu.org
> > >
> > > > (defun hypotenuse (c1 c2)
> > > >   (sqrt (+ (* c1 c1) (* c2 c2))) )
> > >
> > > This is not the best way of computing hypot: squaring a number could
> > > overflow or underflow, and you get bad result.  Try
> > >
> > >     (hypotenuse 1.e-600 1.e-600)
> > > or
> > >     (hypotenuse 1.e600 1.e600)
> > >
> > > and you will see it.
> >
> > Sorry, that should have been 1.e-300 and 1.e300 instead.
> 
> I have a solution to the problem in fortran, but not in elisp.  I could
> include it.  Its implementation required serious concentration.

AFAIK, every libm already includes a similar implementation, so we
should just expose it to Lisp (again, assuming this function is indeed
required to be part of Emacs).

hypotenuse (was: Re: Adding a generic mathematical library)

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 3:42 AM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: "Christopher Dimech" <dimech@gmx.com>
> Cc: incal@dataswamp.org, emacs-devel@gnu.org
> Subject: Re: hypotenuse (was: Re: Adding a generic mathematical library)
>
> > From: Christopher Dimech <dimech@gmx.com>
> > Cc: incal@dataswamp.org, emacs-devel@gnu.org
> > Date: Sun, 21 Jul 2024 17:12:10 +0200
> > Sensitivity: Normal
> >
> > > Sent: Monday, July 22, 2024 at 3:00 AM
> > > From: "Eli Zaretskii" <eliz@gnu.org>
> > > To: incal@dataswamp.org
> > > Cc: emacs-devel@gnu.org
> > > Subject: Re: hypotenuse (was: Re: Adding a generic mathematical library)
> > >
> > > > Date: Sun, 21 Jul 2024 17:44:47 +0300
> > > > From: Eli Zaretskii <eliz@gnu.org>
> > > > Cc: emacs-devel@gnu.org
> > > >
> > > > > (defun hypotenuse (c1 c2)
> > > > >   (sqrt (+ (* c1 c1) (* c2 c2))) )
> > > >
> > > > This is not the best way of computing hypot: squaring a number could
> > > > overflow or underflow, and you get bad result.  Try
> > > >
> > > >     (hypotenuse 1.e-600 1.e-600)
> > > > or
> > > >     (hypotenuse 1.e600 1.e600)
> > > >
> > > > and you will see it.
> > >
> > > Sorry, that should have been 1.e-300 and 1.e300 instead.
> >
> > I have a solution to the problem in fortran, but not in elisp.  I could
> > include it.  Its implementation required serious concentration.
>
> AFAIK, every libm already includes a similar implementation, so we
> should just expose it to Lisp (again, assuming this function is indeed
> required to be part of Emacs).

The people involved in calc do think it is required.

Re: hypotenuse

[Max Nikulin]

On 21/07/2024 22:12, Christopher Dimech wrote:
>>>> (defun hypotenuse (c1 c2)
>>>>    (sqrt (+ (* c1 c1) (* c2 c2))) )
> 
> I have a solution to the problem in fortran, but not in elisp.  I could
> include it.  Its implementation required serious concentration.

Underflow/overflow issues may be solved with a few lines of code and it 
does matter if it is fortran, elisp, or another language (besides 
pathological ones). The problem is to avoid error above 1 ULP.

In 2021 hypot implementation in glibc was replaced by a more precise one:
https://sourceware.org/git/?p=glibc.git;a=commit;h=6c848d70383e1dbe932ef41723ac0abfdeec7ca8

Re: hypotenuse

[Eli Zaretskii]

> From: Max Nikulin <manikulin@gmail.com>
> Date: Sun, 21 Jul 2024 22:54:53 +0700
> 
> On 21/07/2024 22:12, Christopher Dimech wrote:
> >>>> (defun hypotenuse (c1 c2)
> >>>>    (sqrt (+ (* c1 c1) (* c2 c2))) )
> > 
> > I have a solution to the problem in fortran, but not in elisp.  I could
> > include it.  Its implementation required serious concentration.
> 
> Underflow/overflow issues may be solved with a few lines of code and it 
> does matter if it is fortran, elisp, or another language (besides 
> pathological ones). The problem is to avoid error above 1 ULP.

Right, overflow and underflow are just the tip of a very large
iceberg.

> In 2021 hypot implementation in glibc was replaced by a more precise one:
> https://sourceware.org/git/?p=glibc.git;a=commit;h=6c848d70383e1dbe932ef41723ac0abfdeec7ca8

Which is exactly why I said we should use the existing implementation
in the standard library.

hypotenuse

[Christopher Dimech]

> Sent: Monday, July 22, 2024 at 4:12 AM
> From: "Eli Zaretskii" <eliz@gnu.org>
> To: "Max Nikulin" <manikulin@gmail.com>
> Cc: emacs-devel@gnu.org
> Subject: Re: hypotenuse
>
> > From: Max Nikulin <manikulin@gmail.com>
> > Date: Sun, 21 Jul 2024 22:54:53 +0700
> >
> > On 21/07/2024 22:12, Christopher Dimech wrote:
> > >>>> (defun hypotenuse (c1 c2)
> > >>>>    (sqrt (+ (* c1 c1) (* c2 c2))) )
> > >
> > > I have a solution to the problem in fortran, but not in elisp.  I could
> > > include it.  Its implementation required serious concentration.
> >
> > Underflow/overflow issues may be solved with a few lines of code and it
> > does matter if it is fortran, elisp, or another language (besides
> > pathological ones). The problem is to avoid error above 1 ULP.
>
> Right, overflow and underflow are just the tip of a very large
> iceberg.
>
> > In 2021 hypot implementation in glibc was replaced by a more precise one:
> > https://sourceware.org/git/?p=glibc.git;a=commit;h=6c848d70383e1dbe932ef41723ac0abfdeec7ca8
>
> Which is exactly why I said we should use the existing implementation
> in the standard library.

So we should.  I am in agreement.

Re: hypotenuse (was: Re: Adding a generic mathematical library)

[Emanuel Berg]

Eli Zaretskii wrote:

> Like I said: a high-quality math library needs to use solid
> algorithms that produce valid results whenever the problem
> has a solution.

Why do programming languages have libraries?

I don't understand why not they just ship with a calculator,
an e-mail client and ... yeah, you think ERC is enough
for networking? If we have two of them, possibly.

Anyway, what about this then:

  https://dataswamp.org/~incal/hypo.el

-- 
underground experts united
https://dataswamp.org/~incal

Re: hypotenuse (was: Re: Adding a generic mathematical library)

[Emanuel Berg]

Eli Zaretskii wrote:

> This is not the best way of computing hypot: squaring
> a number could overflow or underflow, and you get
> bad result.

Merge with Common Lisp and get their libraries or have Stefan
and Andrea compensated to write a new Lisp that is better than
CL but still compatible so their user base, libraries and
tools become available to us.

CL is also faster, not that Elisp is slow anymore thanks to
native compilation. People in the CL world has a bigger
collective experience from programming so these beginner's
mistakes hopefully happen less often, or get stuck from the
80s/90s without anyone doing anything because "<something> is
a tool" which is especially hilarious and an embarrassment to
the sport because that is actually the reason _not_ to couple
libraries onto them, whoever came up with that heat
struck idea.

-- 
underground experts united
https://dataswamp.org/~incal

Re: hypotenuse (was: Re: Adding a generic mathematical library)

[tomas]

[-- Attachment #1: Type: text/plain, Size: 510 bytes --]

On Sun, Jul 21, 2024 at 04:30:26PM +0200, Emanuel Berg wrote:

[..]

> (defun hypotenuse (c1 c2)
>   (sqrt (+ (* c1 c1) (* c2 c2))) )

Conceptually, yes. Practically... just no.

The shallow (handling /just/ overflow):

  https://www.johndcook.com/blog/2010/06/02/whats-so-hard-about-finding-a-hypotenuse/

The deeper (but with /very/ nice pics):

  https://www.cs.umd.edu/~ntoronto/papers/toronto-2014cise-floating-point.pdf

(I guess Calc is already beyond your formula).

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Adding a generic mathematical library

[Michael Heerdegen via Emacs development discussions.]

Philip Kaludercic <philipk@posteo.net> writes:

> > Someone™ should propose a concrete suggestion.

Have we considered to write a bit of glue code that would make it easier
and more convenient to use the things defined in Calc?  Why invest all of
the time again?


Michael.

Re: Adding a generic mathematical library

[Philip Kaludercic]

Michael Heerdegen via "Emacs development discussions."
<emacs-devel@gnu.org> writes:

> Philip Kaludercic <philipk@posteo.net> writes:
>
>> > Someone™ should propose a concrete suggestion.
>
> Have we considered to write a bit of glue code that would make it easier
> and more convenient to use the things defined in Calc?  Why invest all of
> the time again?

No concrete suggestions have been made yet, so perhaps this could be the
easiest solution.  Again, it might all depend on what the priorities of
such a library would be (Flexibility, Extensibility, Performance, etc.).

> Michael.
>
>
>

-- 
	Philip Kaludercic on peregrine

Re: Adding a generic mathematical library

[Michael Heerdegen via Emacs development discussions.]

Michael Heerdegen via "Emacs development discussions."
<emacs-devel@gnu.org> writes:

> Have we considered to write a bit of glue code that would make it easier
> and more convenient to use the things defined in Calc?  Why invest all of
> the time again?

Here is what we already have in Emacs:

Calc uses a dedicated format to represent math expressions.  This line
reads in such an expression (load Calc before eval'ing):

#+begin_src emacs-lisp
  (car (calc-do-alg-entry "" "Calc expr: " t 'calc-quick-calc-history))
#+end_src

If you enter for example "x + (10!+3)^2" this returns

#+begin_src emacs-lisp
  (+ (var x var-x) (^ (+ (calcFunc-fact 10) 3) 2))
#+end_src

Eval with `math-evaluate-expr':

#+begin_src emacs-lisp
(math-evaluate-expr
 '(+ (var x var-x) (^ (+ (calcFunc-fact 10) 3) 2)))
==> (+ (var x var-x) 13168211212809)
#+end_src

Again the result is using same representation of mathematical
expressions.  Use `math-format-value' to print the result:

#+begin_src emacs-lisp
(math-format-value '(+ (var x var-x) 13168211212809))
 ==> "x + 13168211212809"
#+end_src

A new library will probably also need to invent a new dedicated type of
expressions that is able to represent everything needed.  I don't
understand why it would be a good idea to start from scratch.


Michael.

Re: Adding a generic mathematical library

[Emanuel Berg]

Michael Heerdegen via "Emacs development discussions." wrote:

> I don't understand why it would be a good idea to start
> from scratch.

Maybe we should have a math library in ELPA, then it will be
less hesitation to just add stuff that is desired as we go
along, not having to argue what relevance it has to Emacs
per se as a piece of text editing software.

In general libraries for everything is a good way of not
having people write code that solves the same problems over
and over. Optimally what would happen is you would know where
to look for things and if it wasn't there, you'd write it _and
then add it to that same place_, so what happened won't repeat
itself for the next dude in the same situation.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Christopher Dimech]

> Sent: Thursday, July 18, 2024 at 6:07 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Michael Heerdegen via "Emacs development discussions." wrote:
>
> > I don't understand why it would be a good idea to start
> > from scratch.
>
> Maybe we should have a math library in ELPA, then it will be
> less hesitation to just add stuff that is desired as we go
> along, not having to argue what relevance it has to Emacs
> per se as a piece of text editing software.

The problem that usually occurs is that once things are in place
things would be much more difficult to change in design.  Because
we tend to be quite strict about not breaking their previous use.

> In general libraries for everything is a good way of not
> having people write code that solves the same problems over
> and over. Optimally what would happen is you would know where
> to look for things and if it wasn't there, you'd write it _and
> then add it to that same place_, so what happened won't repeat
> itself for the next dude in the same situation.
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

>> Maybe we should have a math library in ELPA, then it will
>> be less hesitation to just add stuff that is desired as we
>> go along, not having to argue what relevance it has to
>> Emacs per se as a piece of text editing software.
>
> The problem that usually occurs is that once things are in
> place things would be much more difficult to change in
> design. Because we tend to be quite strict about not
> breaking their previous use.

The software doesn't exist, for this reason it hasn't been put
anywhere. Are you worrying about difficulties moving it
from there?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Christopher Dimech]

> Sent: Thursday, July 18, 2024 at 7:12 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech wrote:
>
> >> Maybe we should have a math library in ELPA, then it will
> >> be less hesitation to just add stuff that is desired as we
> >> go along, not having to argue what relevance it has to
> >> Emacs per se as a piece of text editing software.
> >
> > The problem that usually occurs is that once things are in
> > place things would be much more difficult to change in
> > design. Because we tend to be quite strict about not
> > breaking their previous use.
>
> The software doesn't exist, for this reason it hasn't been put
> anywhere. Are you worrying about difficulties moving it
> from there?
>
> --
> underground experts united
> https://dataswamp.org/~incal

I am aligned with Eli's comment about how the library has to move beyond
those intended for personal use.  A well thought out robust design would
be appreciated if things are to move forward.  Because decisions done in
the initial stages will impact upon the capabilities others can add to it.

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> I am aligned with Eli's comment about how the library has to
> move beyond those intended for personal use.

If you examine the packages in ELPA you will see they conform
very well to a decided form. Everyone who has done work on
such or even used them and examined the source would see that
file isn't prepared for that.

And och course nothing intended for personal use with no
usefulness to the general Emacs public is ever added to core
Emacs _or_ ELPA.

Did you and Eli just discovered this?

Okay, whatever, we are on the same page then that a math
library must be prepared and written very well, but to the
form and what it offers.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Thu, 18 Jul 2024 08:07:04 +0200
> 
> Michael Heerdegen via "Emacs development discussions." wrote:
> 
> > I don't understand why it would be a good idea to start
> > from scratch.
> 
> Maybe we should have a math library in ELPA, then it will be
> less hesitation to just add stuff that is desired as we go
> along, not having to argue what relevance it has to Emacs
> per se as a piece of text editing software.

To have a math library on ELPA, the library should be
reasonably-comprehensive, use solid algorithms, and solve problems
that happen in real life frequently enough.  By contrast, my
impression from looking at your library is that it is an ad-hoc
collection of functions that solve problems you personally find
useful, but their general usefulness is IMO questionable, and at least
in some cases the algorithms are not the best ones known for the job.
For example, your mean-value function implements a naïve algorithm
that has known accuracy and round-off issues.

It is perfectly okay to have such libraries for personal use, of
course, but that is not the issue at hand here.

If what you offer is to have a completely different library, then we
need first to talk what will be in it and which algorithms it will
implement.  "Math functions" is a very broad term, so defining the
scope for such a library is not a trivial job.

> In general libraries for everything is a good way of not
> having people write code that solves the same problems over
> and over.

Yes, but since the functions you propose are not in Emacs, and neither
(AFAICT) is any code which resembles them, I wonder whether the need
is a real one and general enough to justify such a package.

Re: Adding a generic mathematical library

[Emanuel Berg]

Eli Zaretskii wrote:

> To have a math library on ELPA, the library should be
> reasonably-comprehensive, use solid algorithms, and solve
> problems that happen in real life frequently enough.
> By contrast, my impression from looking at your library
> [...]

Unbelievable, no one has suggested adding that file to ELPA.

> is that it is an ad-hoc collection of functions that solve
> problems you personally find useful, but their general
> usefulness is IMO questionable

Incorrect, most math functions in it are really basic so one
would expect them to be present in one form of another.

> and at least in some cases the algorithms are not the best
> ones known for the job. For example, your mean-value
> function implements a naive algorithm that has known
> accuracy and round-off issues.

Unbelievable comment. This is indeed the problem that happens,
as has been said in this very thread: without libraries and
common projects individual people solve their problems locally
with no incremental gain from one person to another. So it
won't be as good. And now you see an example of that, bringing
it up as a reason we can't have a math library in ELPA?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Thu, 18 Jul 2024 09:57:40 +0200
> 
> Eli Zaretskii wrote:
> 
> > To have a math library on ELPA, the library should be
> > reasonably-comprehensive, use solid algorithms, and solve
> > problems that happen in real life frequently enough.
> > By contrast, my impression from looking at your library
> > [...]
> 
> Unbelievable, no one has suggested adding that file to ELPA.

You said, and I quote:

> Maybe we should have a math library in ELPA, then it will be
> less hesitation to just add stuff that is desired as we go
> along, not having to argue what relevance it has to Emacs
> per se as a piece of text editing software.

I was not sure whether you were talking about the library you posted,
so I qualified myself, quote again:

> If what you offer is to have a completely different library, then we
> need first to talk what will be in it and which algorithms it will
> implement.  "Math functions" is a very broad term, so defining the
> scope for such a library is not a trivial job.

Okay?

> > is that it is an ad-hoc collection of functions that solve
> > problems you personally find useful, but their general
> > usefulness is IMO questionable
> 
> Incorrect, most math functions in it are really basic so one
> would expect them to be present in one form of another.

I disagree.  My evidence is that the need for functions presented in
your library, if it exists, was not strong enough, otherwise people
would be asking for some of these functions long ago.

> > and at least in some cases the algorithms are not the best
> > ones known for the job. For example, your mean-value
> > function implements a naive algorithm that has known
> > accuracy and round-off issues.
> 
> Unbelievable comment. This is indeed the problem that happens,
> as has been said in this very thread: without libraries and
> common projects individual people solve their problems locally
> with no incremental gain from one person to another. So it
> won't be as good. And now you see an example of that, bringing
> it up as a reason we can't have a math library in ELPA?

OK, so if you want to talk about adding a math library, I point you
back to my comment above: we should first discuss its scope and
design.

Re: Adding a generic mathematical library

[Emanuel Berg]

Eli Zaretskii wrote:

>>> To have a math library on ELPA, the library should be
>>> reasonably-comprehensive, use solid algorithms, and solve
>>> problems that happen in real life frequently enough.
>>> By contrast, my impression from looking at your library [...]
>> 
>> Unbelievable, no one has suggested adding that file
>> to ELPA.
>
> You said, and I quote:
>
>> Maybe we should have a math library in ELPA, then it will be
>> less hesitation to just add stuff that is desired as we go
>> along, not having to argue what relevance it has to Emacs
>> per se as a piece of text editing software.
>
> I was not sure whether you were talking about the library
> you posted, so I qualified myself, quote again:

That's not a library or file prepared for ELPA as you well
know, it is just a bunch of functions. I have several such
files that I have written myself since Emacs don't have it.
Time, permutations, negative subtraction, some stats,
yada yada.

>> If what you offer is to have a completely different
>> library, then we need first to talk what will be in it and
>> which algorithms it will implement. "Math functions" is
>> a very broad term, so defining the scope for such a library
>> is not a trivial job.
>
> Okay?

(I didn't write that.)

>>> is that it is an ad-hoc collection of functions that solve
>>> problems you personally find useful, but their general
>>> usefulness is IMO questionable
>> 
>> Incorrect, most math functions in it are really basic so
>> one would expect them to be present in one form of another.
>
> I disagree. My evidence is that the need for functions
> presented in your library, if it exists, was not strong
> enough, otherwise people would be asking for some of these
> functions long ago.

Math is not popular enough?

We have Tetris in core Emacs and stuff in ELPA no one would
ever know existed.

But math - naah, people don't need it in the world's
programmer's editor number one. No one told you they want it
so they cannot have ever wanted it. Simple!

Well, I have said it several times and now other people say it
as well. Math is huge in tech and science and a lot of other
fields as well were we want to be. It is also something a lot
of programmers are passionate about. _And_ a lot of
non-programmers do math!

Old school engineers used to know two things: how to do math
and how to draw drawings and schematics. That was the basic.

And there are many more examples.

>> Unbelievable comment. This is indeed the problem that
>> happens, as has been said in this very thread: without
>> libraries and common projects individual people solve their
>> problems locally with no incremental gain from one person
>> to another. So it won't be as good. And now you see an
>> example of that, bringing it up as a reason we can't have
>> a math library in ELPA?
>
> OK, so if you want to talk about adding a math library,
> I point you back to my comment above: we should first
> discuss its scope and design.

Well, the people doing it, at least.

The scope to begin with: everything easy and basic you expect
to find in it.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

Eli Zaretskii wrote:

>> In general libraries for everything is a good way of not
>> having people write code that solves the same problems over
>> and over.
>
> Yes, but since the functions you propose are not in Emacs,
> and neither (AFAICT) is any code which resembles them,
> I wonder whether the need is a real one and general enough
> to justify such a package.

There is no need for a math library for the Emacs editor,
which is extensively used by programmers and tech people all
over the world. Because math is not a general need.

That people write their own Elisp because they don't have
official software to do even trivial things doesn't make the
need real.

Yeah, what can I say? Just unbelievable.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Thu, 18 Jul 2024 10:15:18 +0200
> 
> Eli Zaretskii wrote:
> 
> >> In general libraries for everything is a good way of not
> >> having people write code that solves the same problems over
> >> and over.
> >
> > Yes, but since the functions you propose are not in Emacs,
> > and neither (AFAICT) is any code which resembles them,
> > I wonder whether the need is a real one and general enough
> > to justify such a package.
> 
> There is no need for a math library for the Emacs editor,
> which is extensively used by programmers and tech people all
> over the world. Because math is not a general need.

That's not what I said, not even close.

Re: Adding a generic mathematical library

[Christopher Dimech]

> Sent: Thursday, July 18, 2024 at 8:15 PM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Eli Zaretskii wrote:
>
> >> In general libraries for everything is a good way of not
> >> having people write code that solves the same problems over
> >> and over.
> >
> > Yes, but since the functions you propose are not in Emacs,
> > and neither (AFAICT) is any code which resembles them,
> > I wonder whether the need is a real one and general enough
> > to justify such a package.
>
> There is no need for a math library for the Emacs editor,
> which is extensively used by programmers and tech people all
> over the world. Because math is not a general need.
>
> That people write their own Elisp because they don't have
> official software to do even trivial things doesn't make the
> need real.
>
> Yeah, what can I say? Just unbelievable.
>
> --
> underground experts united
> https://dataswamp.org/~incal

He simply needs to be convinced of actual tasks where mathematical operations would
be fundamental.  Even simple calculations within Emacs Lisp scripts can significantly
benefit from a dedicated math library.  But there has to be a practical application
that would be flexible and extensible to other things people would want to do.

Re: Adding a generic mathematical library

[Emanuel Berg]

Christopher Dimech wrote:

> He simply needs to be convinced of actual tasks where
> mathematical operations would be fundamental. Even simple
> calculations within Emacs Lisp scripts can significantly
> benefit from a dedicated math library. But there has to be
> a practical application that would be flexible and
> extensible to other things people would want to do.

There could be but don't have to be any application included.

It could just be a library offering first all basic functions
so people didn't have to write them themselves.

After that: more advanced stuff.

Applications to math are everywhere so no worries people will
find them.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Adding a generic mathematical library

[Emanuel Berg]

I'm not gonna sabotage the possibility of this happening by
continue to discuss this, the way it has started it is better
to just leave it be.

I'm not gonna be the one to do it anyway, but I'd be happy to
offer any assistance I can if someone gets some
momentum going.

-- 
underground experts united
https://dataswamp.org/~incal

Adding a generic mathematical library

[Christopher Dimech]

I have been thinking of an Automated Schedule Generator for conferences, using
permutations to explore possible schedules that meet constraints.

> Sent: Saturday, July 20, 2024 at 1:22 AM
> From: "Emanuel Berg" <incal@dataswamp.org>
> To: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> I'm not gonna sabotage the possibility of this happening by
> continue to discuss this, the way it has started it is better
> to just leave it be.
>
> I'm not gonna be the one to do it anyway, but I'd be happy to
> offer any assistance I can if someone gets some
> momentum going.
>
> --
> underground experts united
> https://dataswamp.org/~incal
>
>
>

Re: Adding a generic mathematical library

[Stefan Kangas]

Christopher Dimech <dimech@gmx.com> writes:

> I have been thinking of an Automated Schedule Generator for conferences, using
> permutations to explore possible schedules that meet constraints.

Thanks, but what does this have to do with Emacs development?

May I suggest that you discuss this issue on another mailing list?

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Saturday, July 20, 2024 at 4:15 AM
> From: "Stefan Kangas" <stefankangas@gmail.com>
> To: "Christopher Dimech" <dimech@gmx.com>, "Emanuel Berg" <incal@dataswamp.org>
> Cc: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> Christopher Dimech <dimech@gmx.com> writes:
>
> > I have been thinking of an Automated Schedule Generator for conferences, using
> > permutations to explore possible schedules that meet constraints.
>
> Thanks, but what does this have to do with Emacs development?

Possible tool for Org-Mode for instance that would require a mathematical library
for emacs.  This is on topic to the suggestion here, which I support.

Re: Adding a generic mathematical library

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Have we considered to write a bit of glue code that would make it easier
  > and more convenient to use the things defined in Calc?  Why invest all of
  > the time again?

If a calc function includes a substanial amount of code
to do a specific kind of calculation, we could extract that part
as a separate function and put it in a suitable place.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Adding a generic mathematical library

[Christopher Dimech]

> Sent: Saturday, July 20, 2024 at 4:16 AM
> From: "Richard Stallman" <rms@gnu.org>
> To: "Michael Heerdegen" <michael_heerdegen@web.de>
> Cc: emacs-devel@gnu.org
> Subject: Re: Adding a generic mathematical library
>
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > Have we considered to write a bit of glue code that would make it easier
>   > and more convenient to use the things defined in Calc?  Why invest all of
>   > the time again?
>
> If a calc function includes a substanial amount of code
> to do a specific kind of calculation, we could extract that part
> as a separate function and put it in a suitable place.

That would be a good starting point.  At least we would know that the functions
are indeed currently useful and being used.  We can find any mathematical
implementations in the emacs code, and extract some good ones to an initial
emacs mathematical library emacs can start to access.

Re: Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 133 bytes --]

I have fixed review issues and improve package documentation. Philip, please see one more time.

Best regards,
Sergey Kostyaev


[-- Attachment #2: Type: text/html, Size: 418 bytes --]

Re: Add elisa to GNU ELPA

[Philip Kaludercic]

Sergey Kostyaev <sskostyaev@gmail.com> writes:

> I have fixed review issues and improve package documentation. Philip, please see one more time.

I admit I was lazy; there are a few SQL strings that could still
upcased.

A general question I have is why you depend on Emacs 29.2?  Package-lint
indicates that you don't need anything before 28.1, with the exception
of the SQLite integration, which would mean you could reduce the minimal
version to 29.1.

Otherwise I guess it looks fine, I'll add it to elpa.git.

> Best regards,
> Sergey Kostyaev
>

-- 
	Philip Kaludercic on peregrine

Re: Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 810 bytes --]

Allowlist of sqlite extensions. vector0 and vss0 was added to allowlist in 29.2 release.


17 июля 2024 г. 22:12:44 UTC, Philip Kaludercic <philipk@posteo.net> пишет:
>Sergey Kostyaev <sskostyaev@gmail.com> writes:
>
>> I have fixed review issues and improve package documentation. Philip, please see one more time.
>
>I admit I was lazy; there are a few SQL strings that could still
>upcased.
>
>A general question I have is why you depend on Emacs 29.2?  Package-lint
>indicates that you don't need anything before 28.1, with the exception
>of the SQLite integration, which would mean you could reduce the minimal
>version to 29.1.
>
>Otherwise I guess it looks fine, I'll add it to elpa.git.
>
>> Best regards,
>> Sergey Kostyaev
>>
>
>-- 
>	Philip Kaludercic on peregrine

[-- Attachment #2: Type: text/html, Size: 1455 bytes --]

Re: Add elisa to GNU ELPA

[Sergey Kostyaev]

[-- Attachment #1: Type: text/plain, Size: 810 bytes --]

All sql strings upcased.

> 18 июля 2024 г., в 05:12, Philip Kaludercic <philipk@posteo.net> написал(а):
> 
> Sergey Kostyaev <sskostyaev@gmail.com> writes:
> 
>> I have fixed review issues and improve package documentation. Philip, please see one more time.
> 
> I admit I was lazy; there are a few SQL strings that could still
> upcased.
> 
> A general question I have is why you depend on Emacs 29.2?  Package-lint
> indicates that you don't need anything before 28.1, with the exception
> of the SQLite integration, which would mean you could reduce the minimal
> version to 29.1.
> 
> Otherwise I guess it looks fine, I'll add it to elpa.git.
> 
>> Best regards,
>> Sergey Kostyaev
>> 
> 
> -- 
> 	Philip Kaludercic on peregrine

Best regards,
Sergey Kostyaev


[-- Attachment #2: Type: text/html, Size: 1346 bytes --]