Patch to implement sorting Org tables by IP address

Patch to implement sorting Org tables by IP address

[Jon Snader]

[-- Attachment #1.1: Type: text/plain, Size: 501 bytes --]

There is currently no easy way to sort an Org table by IP address. The only method I could find involves selecting the IP addresses in a rectangle and piping them to sort with a complicated sort recipe. Even though I am not a system administrator, I sometimes need to maintain tables that I’d like to keep sorted by IP address.

The attached patch implements sorting Org tables natively.

I’ve just completed the FSF Copyright assignment process—let me know if you need details.

jcs



[-- Attachment #1.2: 0001-org.el-Implement-sorting-Org-tables-by-IP-address.patch --]
[-- Type: application/octet-stream, Size: 6155 bytes --]

From d52b9ef1fce4d4db8c948489f009805da6078817 Mon Sep 17 00:00:00 2001
From: Jon Snader <jcs@manfredII.local>
Date: Tue, 9 Dec 2014 13:31:25 -0500
Subject: [PATCH] org.el: Implement sorting Org tables by IP address

* lisp/org.el: Require cl-lib.
  (org-ip-lessp): Implement an IP address comparison function.
  (org-do-sort): Add dispatch to an IP sort. Fix the user prompt
  asking for type of sort.

* lisp/org-table.el (org-table-sort-lines): Added `i' and `I' to the
  description of options in the DOC string.

* doc/org.texi (org-table-sort-lines): Added the capability to sort by
  IP address to the documentation of `org-table-sort-lines'.

These patches implement the capability to sort an Org table by IP address.
---
 doc/org.texi      | 16 ++++++++--------
 lisp/org-table.el | 10 +++++-----
 lisp/org.el       | 25 ++++++++++++++++++++++++-
 3 files changed, 37 insertions(+), 14 deletions(-)

diff --git a/doc/org.texi b/doc/org.texi
index 7b767a3..3b3c085 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -2199,14 +2199,14 @@ below that line.
 @c
 @orgcmd{C-c ^,org-table-sort-lines}
 Sort the table lines in the region.  The position of point indicates the
-column to be used for sorting, and the range of lines is the range
-between the nearest horizontal separator lines, or the entire table.  If
-point is before the first column, you will be prompted for the sorting
-column.  If there is an active region, the mark specifies the first line
-and the sorting column, while point should be in the last line to be
-included into the sorting.  The command prompts for the sorting type
-(alphabetically, numerically, or by time).  When called with a prefix
-argument, alphabetic sorting will be case-sensitive.
+column to be used for sorting, and the range of lines is the range between
+the nearest horizontal separator lines, or the entire table.  If point is
+before the first column, you will be prompted for the sorting column.  If
+there is an active region, the mark specifies the first line and the sorting
+column, while point should be in the last line to be included into the
+sorting.  The command prompts for the sorting type (alphabetically,
+numerically, by time, or by IP address).  When called with a prefix argument,
+alphabetic sorting will be case-sensitive.
 
 @tsubheading{Regions}
 @orgcmd{C-c C-x M-w,org-table-copy-region}
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 8f36d22..7e7fad4 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -1656,16 +1656,16 @@ specifies the first line and the sorting column, while point
 should be in the last line to be included into the sorting.
 
 The command then prompts for the sorting type which can be
-alphabetically, numerically, or by time (as given in a time stamp
-in the field, or as a HH:MM value).  Sorting in reverse order is
-also possible.
+alphabetically, numerically, by time (as given in a time stamp
+in the field, or as a HH:MM value), or by IP address.  Sorting
+in reverse order is also possible.
 
 With prefix argument WITH-CASE, alphabetic sorting will be case-sensitive.
 
 If SORTING-TYPE is specified when this function is called from a Lisp
 program, no prompting will take place.  SORTING-TYPE must be a character,
-any of (?a ?A ?n ?N ?t ?T) where the capital letter indicate that sorting
-should be done in reverse order."
+any of (?a ?A ?n ?N ?t ?T ?i ?I) where the capital letter indicate that
+sorting should be done in reverse order."
   (interactive "P")
   (let* ((thisline (org-current-line))
 	 (thiscol (org-table-current-column))
diff --git a/lisp/org.el b/lisp/org.el
index bed5cb9..814a229 100755
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -76,6 +76,7 @@
 (require 'calendar)
 (require 'find-func)
 (require 'format-spec)
+(require 'cl-lib)
 
 (or (equal this-command 'eval-buffer)
     (condition-case nil
@@ -9051,6 +9052,23 @@ When sorting is done, call `org-after-sorting-entries-or-items-hook'."
 	(move-marker org-clock-marker (point))))
     (message "Sorting entries...done")))
 
+(defun org-ip-lessp (ip1 ip2 &optional op)
+  "Compare two IP addresses.
+Unless the optional argument OP is provided, this function will return T
+if IP1 is less than IP2 or NIL otherwise. The optional argument OP is
+intended to be #'> to support reverse sorting."
+  (setq cmp (or op #'<))
+  (cl-labels ((compare (l1 l2)
+                       (if (or (null l1) (null l2))
+                           nil
+                         (let ((n1 (string-to-number (car l1)))
+                               (n2 (string-to-number (car l2))))
+                           (cond
+                            ((funcall cmp n1 n2) t)
+                            ((= n1 n2) (compare (cdr l1) (cdr l2)))
+                            (t nil))))))
+    (compare (split-string ip1 "\\.") (split-string ip2 "\\."))))
+
 (defun org-do-sort (table what &optional with-case sorting-type)
   "Sort TABLE of WHAT according to SORTING-TYPE.
 The user will be prompted for the SORTING-TYPE if the call to this
@@ -9061,7 +9079,7 @@ the table.
 If WITH-CASE is non-nil, the sorting will be case-sensitive."
   (unless sorting-type
     (message
-     "Sort %s: [a]lphabetic, [n]umeric, [t]ime.  A/N/T means reversed:"
+     "Sort %s: [a]lphabetic, [n]umeric, [t]ime, [i]p address.  A/N/T/I means reversed:"
      what)
     (setq sorting-type (read-char-exclusive)))
   (let ((dcst (downcase sorting-type))
@@ -9089,6 +9107,11 @@ If WITH-CASE is non-nil, the sorting will be case-sensitive."
 		     (org-hh:mm-string-to-minutes x))
 		    (t 0)))
 	    comparefun (if (= dcst sorting-type) '< '>)))
+     ((= dcst ?i)
+      (setq extractfun (lambda (x) (org-sort-remove-invisible x)))
+      (setq comparefun (if (= dcst sorting-type)
+                           #'org-ip-lessp
+                         (lambda (i1 i2) (org-ip-lessp i1 i2 #'>)))))
      (t (error "Invalid sorting type `%c'" sorting-type)))
 
     (sort (mapcar (lambda (x) (cons (funcall extractfun (car x)) (cdr x)))
-- 
1.9.3 (Apple Git-50)


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: Patch to implement sorting Org tables by IP address

[Nicolas Goaziou]

Hello,

Jon Snader <jsnader@mac.com> writes:

> There is currently no easy way to sort an Org table by IP address. The
> only method I could find involves selecting the IP addresses in
> a rectangle and piping them to sort with a complicated sort recipe.
> Even though I am not a system administrator, I sometimes need to
> maintain tables that I’d like to keep sorted by IP address.
>
> The attached patch implements sorting Org tables natively.

Thank you for your patch.

However I think the feature implemented is too specific. Instead,
`org-do-sort' could provide a way to pass an arbitrary predicate,
like ?f and ?F in `org-sort-list'.

WDYT?


Regards,

-- 
Nicolas Goaziou

Re: Patch to implement sorting Org tables by IP address

[Jon Snader]

[-- Attachment #1: Type: text/plain, Size: 1034 bytes --]


> On Dec 12, 2014, at 5:58 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

> However I think the feature implemented is too specific. Instead,
> `org-do-sort' could provide a way to pass an arbitrary predicate,
> like ?f and ?F in `org-sort-list'.
> 
> WDYT?

Actually, when I originally looked at org-do-sort, my first thought was that it should be table driven. But this is my first patch and I’m the new kid so I didn’t want to come in and start tearing down walls.

My idea for making it table driven is to add an alist whose elements look something like

(prompt-char prompt extraction-function compare-function reverse-compare-function with-case-extraction-function)

The alist would be initialized with the built-in options (including the IP address sort) but a user could call add-to-list to add a custom sort to his or her environment.

If the list agrees, I will resubmit the patch implementing what I’ve described. As a procedural matter, should I submit it to this thread or add a new one?


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: Patch to implement sorting Org tables by IP address

[Nicolas Goaziou]

Jon Snader <jsnader@mac.com> writes:

> My idea for making it table driven is to add an alist whose elements look something like
>
> (prompt-char prompt extraction-function compare-function reverse-compare-function with-case-extraction-function)
>
> The alist would be initialized with the built-in options (including
> the IP address sort) but a user could call add-to-list to add a custom
> sort to his or her environment.

I think my proposal is simpler: add ?f and ?F to built-in options, in
which case user is prompted for a custom sorting function (in your case,
`org-ip-lessp'). It is also more consistent with `org-sort-list'.

Also, IMO, IP address sort should not be built-in, and needs not with
the proposal above.

Regards,

Re: Patch to implement sorting Org tables by IP address

[Jon Snader]

[-- Attachment #1: Type: text/plain, Size: 1947 bytes --]


> On Dec 13, 2014, at 9:29 AM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

> I think my proposal is simpler: add ?f and ?F to built-in options, in
> which case user is prompted for a custom sorting function (in your case,
> `org-ip-lessp'). It is also more consistent with `org-sort-list’.

The problem with that solution is that the user has to answer the additional prompts every time he calls org-table-sort-lines with a custom sort. Imagine, for example, a networking researcher who often builds tables that he or she wants to sort by IP address (or any other special format). Why should the researcher have to answer additional prompts every time a table needs to be sorted? What if a user wants to call org-do-sort from Elisp? We’d have to add additional parameters for the custom sort functions.

The solution I’m suggesting will simplify org-do-sort, make it easily expandable, and allow programmatic calls to it. It is, it seems to me, consistent with how Emacs does things: add to little Elisp to expand a functionality to cover an additional case. As things stand now, it’s not possible for me to add an IP sort locally. Your solution does provide that, after a fashion, but at the cost of having to enter additional information each time I call the sort. It doesn’t really make it easier for the user because the extraction and compare functions will still have to be written (at least in most cases I can think of).

As for org-sort-list, it too would benefit from being table driven for the same reason that org-do-sort would: it enables users to define custom sorts. If we agree on this approach, I’ll submit another patch for org-sort-list.

> Also, IMO, IP address sort should not be built-in, and needs not with
> the proposal above.

I don’t care overmuch if the IP address sort is built in (although others might appreciate the functionality) only that I have a way of adding it locally.


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: Patch to implement sorting Org tables by IP address

[Nicolas Goaziou]

Jon Snader <jsnader@mac.com> writes:

> The problem with that solution is that the user has to answer the
> additional prompts every time he calls org-table-sort-lines with
> a custom sort. Imagine, for example, a networking researcher who often
> builds tables that he or she wants to sort by IP address (or any other
> special format). Why should the researcher have to answer additional
> prompts every time a table needs to be sorted?

He doesn't need to. He could write his own sorting function, and bind it
to a key.

> What if a user wants to call org-do-sort from Elisp? We’d have to add
> additional parameters for the custom sort functions.

`org-do-sort' is an internal function, which isn't meant to be used
publicly. It really should be named `org-table--do-sort' and be moved
within "org-table.el". OTOH, we can extend `org-table-sort-lines' to
allow custom sorting function as a parameter.

> The solution I’m suggesting will simplify org-do-sort, make it easily
> expandable, and allow programmatic calls to it.

I disagree. It makes `org-do-sort'/`org-table-sort-line' more powerful,
but certainly not simpler. I'm pretty sure that anyone able to provide
an alist like you suggest is able to write his own sorting function
anyway. IOW, no one really gains any power in the process.

> It doesn’t really make it easier for the user because the extraction
> and compare functions will still have to be written (at least in most
> cases I can think of).

Extraction function is standard, AFAICT. Your solution doesn't solve
second point either.

> As for org-sort-list, it too would benefit from being table driven for
> the same reason that org-do-sort would: it enables users to define
> custom sorts. If we agree on this approach, I’ll submit another patch
> for org-sort-list.

As pointed out earlier, my concern is about simplicity.

What about simplifying your proposal so that associations are

  (prompt-char . compare-function)

Extraction function would be always the same: the contents of the cell,
as a string, and prompt would be hard-coded for default compare
functions?


Regards,

Re: Patch to implement sorting Org tables by IP address

[Jon Snader]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]


> On Dec 13, 2014, at 11:01 AM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

> `org-do-sort' is an internal function, which isn't meant to be used
> publicly. It really should be named `org-table--do-sort' and be moved
> within "org-table.el". OTOH, we can extend `org-table-sort-lines' to
> allow custom sorting function as a parameter.

I like the idea of providing optional parameters to org-table-sort-lines.

The extraction function varies with the type of sort so it has to be specified one way or the other. In org-sort-list, that’s what the getkey-func parameter supplies. Thus if we wanted an alist we’d need to provide it too. If you want simplicity, how about this: Add ?f ?F and two optional parameters for getkey-func and compare-func to org-table-sort-lines. Then add optional parameters to org-do-sort for the getkey-func and compare-func functions. If the sort type is ?f or ?F org-do-sort would use getkey-func and compare-func instead of the built-in functions.

This would make org-table-sort-lines behave the same way that org-sort-list does. Then I could solve my original problem by writing a local function that calls org-table-sort-lines with the ?f or ?F sort type and appropriate extraction and compare functions.

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: Patch to implement sorting Org tables by IP address

[Nicolas Goaziou]

Jon Snader <jsnader@mac.com> writes:

> The extraction function varies with the type of sort so it has to be
> specified one way or the other.

I don't think it needs to vary. As I suggested already, we can use cells
contents (in the appropriate column) as trimmed strings. So, it could be
built-in.

Do you see any downside to it?


Regards,

Re: Patch to implement sorting Org tables by IP address

[Jon Snader]

[-- Attachment #1: Type: text/plain, Size: 1279 bytes --]


> On Dec 13, 2014, at 5:07 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

> I don't think it needs to vary. As I suggested already, we can use cells
> contents (in the appropriate column) as trimmed strings. So, it could be
> built-in.
> 
> Do you see any downside to it?

Well, I’m just going by what happens now. In org-do-sort, each of the sort options sets a different extraction function. For example, if you want a numeric sort, the extraction function calls string-to-number, while if you want an alphabetic sort it calls org-sort-remove-invisible. Really, this doesn’t matter because I was merely commenting on why (prompt . comparison) isn’t enough. Of course, you could roll any special extraction functionality into the comparison but I don’t really like that.

Anyway, what I was suggesting in my last post was that we duplicate the functionality of org-sort-list. There, if you’re calling it programmatically you specify getkey-func and compare-func. If you call it interactively, it asks you for the extraction function (which must return a string or number) and it tests it to see which comparison function to use. I like this approach because it makes org-sort-list and org-table-sort-lines work the same way. What’s not to like?


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: Patch to implement sorting Org tables by IP address

[Nicolas Goaziou]

Jon Snader <jsnader@mac.com> writes:

> Well, I’m just going by what happens now. In org-do-sort, each of the
> sort options sets a different extraction function. For example, if you
> want a numeric sort, the extraction function calls string-to-number,
> while if you want an alphabetic sort it calls
> org-sort-remove-invisible. Really, this doesn’t matter because I was
> merely commenting on why (prompt . comparison) isn’t enough. Of
> course, you could roll any special extraction functionality into the
> comparison but I don’t really like that.

Then 

  (prompt comparison extraction)

while still allowing 

  (prompt comparison)

which would be a special case for

  (prompt comparison #'org-sort-remove-invisible)

> Anyway, what I was suggesting in my last post was that we duplicate
> the functionality of org-sort-list.

This what I initially suggested. However, I'm trying to see if a table
approach would ultimately be better.

> There, if you’re calling it programmatically you specify getkey-func
> and compare-func. If you call it interactively, it asks you for the
> extraction function (which must return a string or number) and it
> tests it to see which comparison function to use. I like this approach
> because it makes org-sort-list and org-table-sort-lines work the same
> way. What’s not to like?

The networking researcher will have to provide its sorting function each
time, which was one of your arguments.


Regards,

Re: Patch to implement sorting Org tables by IP address

[Jon Snader]

[-- Attachment #1: Type: text/plain, Size: 2951 bytes --]


> On Dec 14, 2014, at 6:25 AM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
> 
> Jon Snader <jsnader@mac.com> writes:

>> Really, this doesn’t matter because I was

>> merely commenting on why (prompt . comparison) isn’t enough. Of
>> course, you could roll any special extraction functionality into the
>> comparison but I don’t really like that.
> 
> Then
> 
>  (prompt comparison extraction)
> 
> while still allowing
> 
>  (prompt comparison)
> 
> which would be a special case for
> 
>  (prompt comparison #'org-sort-remove-invisible)

That would make sense. Also, I think the prompt should be something like “(t)ime” so that the prompt message can be built on-the-fly. The `t’  (or whatever) can be extracted from the prompt to build a list of legal answers.

> 
>> Anyway, what I was suggesting in my last post was that we duplicate
>> the functionality of org-sort-list.
> 
> This what I initially suggested. However, I'm trying to see if a table
> approach would ultimately be better.

You’ve convinced me that your original suggestion is the right one. It allows users to add additional sorting options if they need one. Even a user without Elisp skills could use it (by calling it interactively) if the extraction function was already available either in Emacs/Org itself or locally.

The important thing to me is that the user have the capability to add another sort when (admittedly rarely) needed. The ?f ?F mechanism provides the necessary escape hatch and is what current users are used to.

> 
>> There, if you’re calling it programmatically you specify getkey-func
>> and compare-func. If you call it interactively, it asks you for the
>> extraction function (which must return a string or number) and it
>> tests it to see which comparison function to use. I like this approach
>> because it makes org-sort-list and org-table-sort-lines work the same
>> way. What’s not to like?
> 
> The networking researcher will have to provide its sorting function each
> time, which was one of your arguments.

Sorry. That was because I didn’t completely understand your suggestion. The point is that the ?f ?F mechanism enables both interactive AND programmatic use. The researcher could write a bit of Elisp that calls org-table-sort-lines with his own extraction and comparison functions and then call that Elisp interactively.

As I see it, we’ve reached the conclusion that we should either have some sort of minimal table-driven mechanism or duplicate the functionality in org-sort-list. I like duplicating the org-sort-list functionality best. As you pointed out before, org-do-sort is an internal function and we probably shouldn’t let users mess with it. We could, I suppose, move all the work out to org-table-sort-lines but then org-do-sort is little more than a call to sort.

As I said above, you’ve convinced me that ?f ?F is the right solution.


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: Patch to implement sorting Org tables by IP address

[Nicolas Goaziou]

Jon Snader <jsnader@mac.com> writes:

> As I see it, we’ve reached the conclusion that we should either have
> some sort of minimal table-driven mechanism or duplicate the
> functionality in org-sort-list. I like duplicating the org-sort-list
> functionality best. As you pointed out before, org-do-sort is an
> internal function and we probably shouldn’t let users mess with it. We
> could, I suppose, move all the work out to org-table-sort-lines but
> then org-do-sort is little more than a call to sort.

You can move `org-do-sort' into "org-table.el" with an "org-table--"
prefix, if you want to.

> As I said above, you’ve convinced me that ?f ?F is the right solution.

Fair enough. Let's settle on that, then.

Thanks.


Regards,

Re: Patch to implement sorting Org tables by IP address

[Jon Snader]

[-- Attachment #1.1: Type: text/plain, Size: 774 bytes --]


> On Dec 14, 2014, at 12:18 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

>> As I said above, you’ve convinced me that ?f ?F is the right solution.
> 
> Fair enough. Let's settle on that, then.

Here is the new patch. It extends org-table-sort-lines to allow a user to specify custom extraction and comparison functions using the ?f ?F sorting type as in org-sort-list.

The user can call org-table-sort-lines programmatically specifying a SORTING-TYPE of ?f or ?F and provide custom extraction and comparison functions. If the user calls org-table-sort lines interactively and specifies ?f or ?F, org-do-sort will prompt for a custom extraction function. The comparison will be either string or numeric depending on the type of the first extracted key.

[-- Attachment #1.2: 0001-org.el-Implement-user-specified-sorting-functions.patch --]
[-- Type: application/octet-stream, Size: 6932 bytes --]

From 53f1929b7a8c1b98230d60e5313e7550ab3d1b13 Mon Sep 17 00:00:00 2001
From: Jon Snader <jcs@manfredII.local>
Date: Wed, 17 Dec 2014 11:52:23 -0500
Subject: [PATCH] org.el: Implement user specified sorting functions

* lisp/org.el (org-do-sort): Implemented the ability for the user
to specify custom extraction and comparison functions.	Updated
the DOC string.

* lisp/org-table.el (org-table-sort-lines): Added GETKEY-FUNC and
COMPARE-FUNC as optional arguments and passed them to
`org-do-sort'.	Updated DOC string to reflect the new
functionality.

* doc/org.texi (Structure editing): Document the ability for users
to specify their own extraction and comparison functions.
---
 doc/org.texi      |  6 ++++--
 lisp/org-table.el | 15 +++++++++++----
 lisp/org.el       | 36 ++++++++++++++++++++++++++++++------
 3 files changed, 45 insertions(+), 12 deletions(-)

diff --git a/doc/org.texi b/doc/org.texi
index d617259..f56e824 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -2205,8 +2205,10 @@ point is before the first column, you will be prompted for the sorting
 column.  If there is an active region, the mark specifies the first line
 and the sorting column, while point should be in the last line to be
 included into the sorting.  The command prompts for the sorting type
-(alphabetically, numerically, or by time).  When called with a prefix
-argument, alphabetic sorting will be case-sensitive.
+(alphabetically, numerically, or by time).  You can sort in normal or reverse
+order.  You can also supply your own key extraction and comparison
+functions.  When called with a prefix argument, alphabetic sorting will
+be case-sensitive.
 
 @tsubheading{Regions}
 @orgcmd{C-c C-x M-w,org-table-copy-region}
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 06a8ab7..9048ac9 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -1657,7 +1657,7 @@ In particular, this does handle wide and invisible characters."
 			      dline -1 dline))))
 
 ;;;###autoload
-(defun org-table-sort-lines (with-case &optional sorting-type)
+(defun org-table-sort-lines (with-case &optional sorting-type getkey-func compare-func)
   "Sort table lines according to the column at point.
 
 The position of point indicates the column to be used for
@@ -1677,8 +1677,15 @@ With prefix argument WITH-CASE, alphabetic sorting will be case-sensitive.
 
 If SORTING-TYPE is specified when this function is called from a Lisp
 program, no prompting will take place.  SORTING-TYPE must be a character,
-any of (?a ?A ?n ?N ?t ?T) where the capital letter indicate that sorting
-should be done in reverse order."
+any of (?a ?A ?n ?N ?t ?T ?f ?F) where the capital letter indicate that sorting
+should be done in reverse order.
+
+If the SORTING-TYPE is ?f or ?F, then GETKEY-FUNC specifies
+a function to be called to extract the key.  It must return either
+a string or a number that should serve as the sorting key for that
+row.  It will then use COMPARE-FUNC to compare entries.  If GETKEY-FUNC
+is specified interactively, the comparison will be either a string or
+numeric compare based on the type of the first key in the table."
   (interactive "P")
   (let* ((thisline (org-current-line))
 	 (thiscol (org-table-current-column))
@@ -1730,7 +1737,7 @@ should be done in reverse order."
 					(org-split-string x "[ \t]*|[ \t]*")))
 				  x))
 		      (org-split-string (buffer-substring beg end) "\n")))
-    (setq lns (org-do-sort lns "Table" with-case sorting-type))
+    (setq lns (org-do-sort lns "Table" with-case sorting-type getkey-func compare-func))
     (when org-table-overlay-coordinates
       (org-table-toggle-coordinate-overlays))
     (delete-region beg end)
diff --git a/lisp/org.el b/lisp/org.el
index bed5cb9..23f5b07 100755
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -9051,21 +9051,27 @@ When sorting is done, call `org-after-sorting-entries-or-items-hook'."
 	(move-marker org-clock-marker (point))))
     (message "Sorting entries...done")))
 
-(defun org-do-sort (table what &optional with-case sorting-type)
+(defun org-do-sort (table what &optional with-case sorting-type getkey-func compare-func)
   "Sort TABLE of WHAT according to SORTING-TYPE.
 The user will be prompted for the SORTING-TYPE if the call to this
 function does not specify it.
 WHAT is only for the prompt, to indicate what is being sorted.
 The sorting key will be extracted from the car of the elements of
-the table.
-If WITH-CASE is non-nil, the sorting will be case-sensitive."
+the table. If WITH-CASE is non-nil, the sorting will be case-sensitive.
+
+If the SORTING-TYPE is ?f or ?F, then GETKEY-FUNC specifies
+a function to be called to extract the key.  It must return either
+a string or a number that should serve as the sorting key for that
+row.  It will then use COMPARE-FUNC to compare entries.  If GETKEY-FUNC
+is specified interactively, the comparison will be either a string or
+numeric compare based on the type of the first key in the table."
   (unless sorting-type
     (message
-     "Sort %s: [a]lphabetic, [n]umeric, [t]ime.  A/N/T means reversed:"
+     "Sort %s: [a]lphabetic, [n]umeric, [t]ime, [f]unc.  A/N/T/F means reversed:"
      what)
     (setq sorting-type (read-char-exclusive)))
   (let ((dcst (downcase sorting-type))
-	extractfun comparefun)
+	extractfun comparefun tempfun extract-string-p)
     ;; Define the appropriate functions
     (cond
      ((= dcst ?n)
@@ -9075,7 +9081,7 @@ If WITH-CASE is non-nil, the sorting will be case-sensitive."
       (setq extractfun (if with-case (lambda(x) (org-sort-remove-invisible x))
 			 (lambda(x) (downcase (org-sort-remove-invisible x))))
 	    comparefun (if (= dcst sorting-type)
-			   'string<
+			   #'string<
 			 (lambda (a b) (and (not (string< a b))
 					    (not (string= a b)))))))
      ((= dcst ?t)
@@ -9089,6 +9095,24 @@ If WITH-CASE is non-nil, the sorting will be case-sensitive."
 		     (org-hh:mm-string-to-minutes x))
 		    (t 0)))
 	    comparefun (if (= dcst sorting-type) '< '>)))
+     ((= dcst ?f)
+      (setq tempfun (or getkey-func
+			(intern (org-icompleting-read
+				 "Sort using function: "
+				 obarray 'fboundp t nil nil))))
+      (setq extract-string-p (stringp (funcall tempfun (caar table))))
+      (setq extractfun (if (and extract-string-p (not with-case))
+			   (lambda (x) (downcase (funcall tempfun x)))
+			 tempfun))
+      (setq comparefun (or compare-func
+			   (if extract-string-p 
+			       (if (= sorting-type ?f)
+				   #'string<
+				 (lambda (a b) (and (not (string< a b))
+						    (not (string= a b)))))
+			     (if (= sorting-type ?f)
+				 #'<
+			       #'>)))))
      (t (error "Invalid sorting type `%c'" sorting-type)))
 
     (sort (mapcar (lambda (x) (cons (funcall extractfun (car x)) (cdr x)))
-- 
1.9.3 (Apple Git-50)


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: Patch to implement sorting Org tables by IP address

[Nicolas Goaziou]

Jon Snader <jsnader@mac.com> writes:

>> On Dec 14, 2014, at 12:18 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
>
>>> As I said above, you’ve convinced me that ?f ?F is the right solution.
>> 
>> Fair enough. Let's settle on that, then.
>
> Here is the new patch. It extends org-table-sort-lines to allow a user
> to specify custom extraction and comparison functions using the ?f ?F
> sorting type as in org-sort-list.
>
> The user can call org-table-sort-lines programmatically specifying
> a SORTING-TYPE of ?f or ?F and provide custom extraction and
> comparison functions. If the user calls org-table-sort lines
> interactively and specifies ?f or ?F, org-do-sort will prompt for
> a custom extraction function. The comparison will be either string or
> numeric depending on the type of the first extracted key.

Thank you. Some comments follow.

> -any of (?a ?A ?n ?N ?t ?T) where the capital letter indicate that sorting
> -should be done in reverse order."
> +any of (?a ?A ?n ?N ?t ?T ?f ?F) where the capital letter indicate
> that sorting

"indicates" or "capital letters"

> +	extractfun comparefun tempfun extract-string-p)

EXTRACT-STRING-P, and possibly TEMPFUN, are bound too early. See below.

>      ;; Define the appropriate functions
>      (cond
>       ((= dcst ?n)
> @@ -9075,7 +9081,7 @@ If WITH-CASE is non-nil, the sorting will be case-sensitive."
>        (setq extractfun (if with-case (lambda(x) (org-sort-remove-invisible x))
>  			 (lambda(x) (downcase (org-sort-remove-invisible x))))
>  	    comparefun (if (= dcst sorting-type)
> -			   'string<
> +			   #'string<

OK, but then there are other occurrences in the function to modify.
I suggest to remove this.

> +     ((= dcst ?f)
> +      (setq tempfun (or getkey-func
> +			(intern (org-icompleting-read
> +				 "Sort using function: "
> +				 obarray 'fboundp t nil nil))))
> +      (setq extract-string-p (stringp (funcall tempfun (caar table))))
> +      (setq extractfun (if (and extract-string-p (not with-case))
> +			   (lambda (x) (downcase (funcall tempfun x)))
> +			 tempfun))
> +      (setq comparefun (or compare-func
> +			   (if extract-string-p 
> +			       (if (= sorting-type ?f)
> +				   #'string<
> +				 (lambda (a b) (and (not (string< a b))
> +						    (not (string= a b)))))
> +			     (if (= sorting-type ?f)
> +				 #'<
> +			       #'>)))))

I suggest something like

  (let* ((tempfun (or getkey-func
                      (intern ...)))
         (extract-string-p (stringp (funcall tempfun (caar table)))))
    (setq extractfun (if (and extract-string-p (not with-case))
                         `(lambda (x) (downcase (funcall ',tempfun x)))
                       tempfun))
    (setq comparefun
         (cond (compare-func)
               (extract-string-p (if (= sorting-type ?f) #'string< #'org-string>))
               ((= sorting-type ?f) #'<)
               (t #'>))))


Regards,

Re: Patch to implement sorting Org tables by IP address

[Jon Snader]

[-- Attachment #1.1: Type: text/plain, Size: 1104 bytes --]


> On Dec 20, 2014, at 6:57 AM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

>> +	extractfun comparefun tempfun extract-string-p)
> 
> EXTRACT-STRING-P, and possibly TEMPFUN, are bound too early. See below.

I moved EXTRACT-STRING-P to a lower let but TEMPFUN has to be available to the call to sort so I left it in the outer let.

> I suggest something like
> 
>  (let* ((tempfun (or getkey-func
>                      (intern ...)))
>         (extract-string-p (stringp (funcall tempfun (caar table)))))
>    (setq extractfun (if (and extract-string-p (not with-case))
>                         `(lambda (x) (downcase (funcall ',tempfun x)))
>                       tempfun))
>    (setq comparefun
>         (cond (compare-func)
>               (extract-string-p (if (= sorting-type ?f) #'string< #'org-string>))
>               ((= sorting-type ?f) #'<)
>               (t #'>))))

I followed this suggestion modulo leaving TEMPFUN in the outer let.

Thanks for your suggestions.

The attached patch was against the latest master branch at the time I generated it.



[-- Attachment #1.2.1: Type: text/html, Size: 12374 bytes --]

[-- Attachment #1.2.2: 0001-org.el-Implement-user-defined-table-sorting.patch --]
[-- Type: application/octet-stream, Size: 6857 bytes --]

From 9d526679cd958b46605c60d092062cf6db441cb3 Mon Sep 17 00:00:00 2001
From: Jon Snader <jcs@manfredII.local>
Date: Sat, 20 Dec 2014 13:13:17 -0500
Subject: [PATCH] org.el: Implement user-defined table sorting

* lisp/org.el (org-do-sort): Implement the ?f and ?F sorting options
  to allow user-defined table sorting.  Update the DOC string.

* lisp/org-table (org-table-sort-lines): Add the GETKEY-FUNC and
  COMPARE-FUNC optional parameters and pass them to the call to
  `org-do-sort'.  Update the DOC string.

* doc/org.texi (org-table-sort-lines): Update documentation to reflect
  the addition of the ?f and ?F options.

This patch implements user-defined extraction and comparison functions
for table sorting.  Thanks to Nicolas Goaziou for helpful suggestions.

This patch was discussed on the Org Mode mailing list:
http://article.gmane.org/gmane.emacs.orgmode/93334
---
 doc/org.texi      |  6 ++++--
 lisp/org-table.el | 15 +++++++++++----
 lisp/org.el       | 35 +++++++++++++++++++++++++++++------
 3 files changed, 44 insertions(+), 12 deletions(-)

diff --git a/doc/org.texi b/doc/org.texi
index 7c464ca..33a6a0d 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -2205,8 +2205,10 @@ point is before the first column, you will be prompted for the sorting
 column.  If there is an active region, the mark specifies the first line
 and the sorting column, while point should be in the last line to be
 included into the sorting.  The command prompts for the sorting type
-(alphabetically, numerically, or by time).  When called with a prefix
-argument, alphabetic sorting will be case-sensitive.
+(alphabetically, numerically, or by time).  You can sort in normal or
+reverse order.  You can also supply your own key extraction and comparison
+functions.  When called with a prefix argument, alphabetic sorting will be
+case-sensitive.
 
 @tsubheading{Regions}
 @orgcmd{C-c C-x M-w,org-table-copy-region}
diff --git a/lisp/org-table.el b/lisp/org-table.el
index fa59113..9941160 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -1657,7 +1657,7 @@ In particular, this does handle wide and invisible characters."
 			      dline -1 dline))))
 
 ;;;###autoload
-(defun org-table-sort-lines (with-case &optional sorting-type)
+(defun org-table-sort-lines (with-case &optional sorting-type getkey-func compare-func)
   "Sort table lines according to the column at point.
 
 The position of point indicates the column to be used for
@@ -1677,8 +1677,15 @@ With prefix argument WITH-CASE, alphabetic sorting will be case-sensitive.
 
 If SORTING-TYPE is specified when this function is called from a Lisp
 program, no prompting will take place.  SORTING-TYPE must be a character,
-any of (?a ?A ?n ?N ?t ?T) where the capital letter indicate that sorting
-should be done in reverse order."
+any of (?a ?A ?n ?N ?t ?T ?f ?F) where the capital letters indicate that
+sorting should be done in reverse order.
+
+If the SORTING-TYPE is ?f or ?F, then GETKEY-FUNC specifies
+a function to be called to extract the key.  It must return either
+a string or a number that should serve as the sorting key for that
+row.  It will then use COMPARE-FUNC to compare entries.  If GETKEY-FUNC
+is specified interactively, the comparison will be either a string or
+numeric compare based on the type of the first key in the table."
   (interactive "P")
   (let* ((thisline (org-current-line))
 	 (thiscol (org-table-current-column))
@@ -1730,7 +1737,7 @@ should be done in reverse order."
 					(org-split-string x "[ \t]*|[ \t]*")))
 				  x))
 		      (org-split-string (buffer-substring beg end) "\n")))
-    (setq lns (org-do-sort lns "Table" with-case sorting-type))
+    (setq lns (org-do-sort lns "Table" with-case sorting-type getkey-func compare-func))
     (when org-table-overlay-coordinates
       (org-table-toggle-coordinate-overlays))
     (delete-region beg end)
diff --git a/lisp/org.el b/lisp/org.el
index 1383d76..29439d2 100755
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -9057,21 +9057,27 @@ When sorting is done, call `org-after-sorting-entries-or-items-hook'."
 	(move-marker org-clock-marker (point))))
     (message "Sorting entries...done")))
 
-(defun org-do-sort (table what &optional with-case sorting-type)
+(defun org-do-sort (table what &optional with-case sorting-type getkey-func compare-func)
   "Sort TABLE of WHAT according to SORTING-TYPE.
 The user will be prompted for the SORTING-TYPE if the call to this
 function does not specify it.
 WHAT is only for the prompt, to indicate what is being sorted.
 The sorting key will be extracted from the car of the elements of
-the table.
-If WITH-CASE is non-nil, the sorting will be case-sensitive."
+the table. If WITH-CASE is non-nil, the sorting will be case-sensitive.
+
+If the SORTING-TYPE is ?f or ?F, then GETKEY-FUNC specifies
+a function to be called to extract the key.  It must return either
+a string or a number that should serve as the sorting key for that
+row.  It will then use COMPARE-FUNC to compare entries.  If GETKEY-FUNC
+is specified interactively, the comparison will be either a string or
+numeric compare based on the type of the first key in the table."
   (unless sorting-type
     (message
-     "Sort %s: [a]lphabetic, [n]umeric, [t]ime.  A/N/T means reversed:"
+     "Sort %s: [a]lphabetic, [n]umeric, [t]ime, [f]unc.  A/N/T/F means reversed:"
      what)
     (setq sorting-type (read-char-exclusive)))
   (let ((dcst (downcase sorting-type))
-	extractfun comparefun)
+	extractfun comparefun tempfun)
     ;; Define the appropriate functions
     (cond
      ((= dcst ?n)
@@ -9095,13 +9101,30 @@ If WITH-CASE is non-nil, the sorting will be case-sensitive."
 		     (org-hh:mm-string-to-minutes x))
 		    (t 0)))
 	    comparefun (if (= dcst sorting-type) '< '>)))
+     ((= dcst ?f)
+      (setq tempfun (or getkey-func
+			(intern (org-icompleting-read
+				 "Sort using function: "
+				 obarray #'fboundp t nil nil))))
+      (let* ((extract-string-p (stringp (funcall tempfun (caar table)))))
+	(setq extractfun (if (and extract-string-p (not with-case))
+			     (lambda (x) (downcase (funcall tempfun x)))
+			   tempfun))
+	(setq comparefun (cond (compare-func)
+			       (extract-string-p
+				(if (= sorting-type ?f)
+				    #'string<
+				  #'org-string>))
+			       (t	;numeric
+				(if (= sorting-type ?f)
+				    #'<
+				  #'>))))))
      (t (error "Invalid sorting type `%c'" sorting-type)))
 
     (sort (mapcar (lambda (x) (cons (funcall extractfun (car x)) (cdr x)))
 		  table)
 	  (lambda (a b) (funcall comparefun (car a) (car b))))))
 
-
 ;;; The orgstruct minor mode
 
 ;; Define a minor mode which can be used in other modes in order to
-- 
1.9.3 (Apple Git-50)


[-- Attachment #1.2.3: Type: text/html, Size: 237 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 841 bytes --]

Re: Patch to implement sorting Org tables by IP address

[Nicolas Goaziou]

Hello,

Jon Snader <jsnader@mac.com> writes:

> I moved EXTRACT-STRING-P to a lower let but TEMPFUN has to be
> available to the call to sort so I left it in the outer let.

Not in the snippet I suggested, but it doesn't matter much.

> The attached patch was against the latest master branch at the time I
> generated it.

Thank you. Applied.


Regards,