[PATCH] Add yank-media handler for LibreOffice Calc tables

[PATCH] Add yank-media handler for LibreOffice Calc tables

[Visuwesh]

[-- Attachment #1: Type: text/plain, Size: 199 bytes --]

Attached patch adds support for yanking LibreOffice Calc tables as Org
tables.  Tables from docx files (or LO's equivalent) cannot be yanked
though since the mimetype for them are richtext or HTML.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-yank-media-handler-for-LibreOffice-Calc-tables.patch --]
[-- Type: text/x-diff, Size: 3740 bytes --]

From 0cd4b1c5402256ed64c9e1711db11ad215c6c1a3 Mon Sep 17 00:00:00 2001
From: Visuwesh <visuweshm@gmail.com>
Date: Mon, 14 Oct 2024 22:04:23 +0530
Subject: [PATCH] Add `yank-media' handler for LibreOffice Calc tables

* lisp/org.el (org--libreoffice-table-handler): Add new `yank-media'
handler for tables copied from LibreOffice Calc documents.
(org-setup-yank-dnd-handlers): Register it.
* doc/org-manual.org: (Drag and Drop & ~yank-media~): Update the
manual to mention the new feature.
* etc/ORG-NEWS: Announce the new feature.
---
 doc/org-manual.org |  6 +++++-
 etc/ORG-NEWS       |  5 +++++
 lisp/org.el        | 14 +++++++++++++-
 3 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 451fd72cd..e8da07356 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -21640,7 +21640,8 @@ value as ~org-attach-method~ (~cp~ by default).]
 
 #+cindex: pasting files, images from clipboard
 Starting from Emacs 29, Org mode supports ~yank-media~ command to yank
-images from the clipboard and files from a file manager.
+images from the clipboard, files from a file manager and tables copied
+from LibreOffice Calc documents.
 
 #+vindex: org-yank-image-save-method
 When yanking images from clipboard, Org saves the image on disk and
@@ -21658,6 +21659,9 @@ of ~org-yank-dnd-method~.  Image files pasted this way also respect
 the value of ~org-yank-image-save-method~ when the action to perform
 is =attach=.
 
+When yanking tables copied from LibreOffice Calc documents, they are
+inserted as Org tables when yanking them.
+
 ** Repeating commands
 :PROPERTIES:
 :DESCRIPTION: Repeating navigation commands
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 5d421172f..37ee9a52f 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -114,6 +114,11 @@ The keybindings in the repeat-maps can be changed by customizing
 
 See the new [[info:org#Repeating commands]["Repeating commands"]] section in Org mode manual.
 
+*** Tables copied from LibreOffice Calc documents can be pasted as Org tables
+
+Tables copied into the clipboard from LibreOffice Calc documents can
+now be pasted as an Org table using ~yank-media~.
+
 ** New and changed options
 
 # Chanes deadling with changing default values of customizations,
diff --git a/lisp/org.el b/lisp/org.el
index 6177856c3..82ad40527 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -20860,7 +20860,9 @@ (defun org-setup-yank-dnd-handlers ()
     ;; Looks like different DEs go for different handler names,
     ;; https://larsee.com/blog/2019/05/clipboard-files/.
     (yank-media-handler "x/special-\\(?:gnome\\|KDE\\|mate\\)-files"
-                        #'org--copied-files-yank-media-handler))
+                        #'org--copied-files-yank-media-handler)
+    (yank-media-handler "application/x-libreoffice-tsvc"
+                        #'org--libreoffice-table-handler))
   (when (boundp 'x-dnd-direct-save-function)
     (setq-local x-dnd-direct-save-function #'org--dnd-xds-function)))
 
@@ -20953,6 +20955,16 @@ (defun org--copied-files-yank-media-handler (_mimetype data)
           (org--dnd-local-file-handler f action sep)
         (message "File `%s' is not readable, skipping" f)))))
 
+(defun org--libreoffice-table-handler (_mimetype data)
+  "Insert LibreOffice Calc table DATA as an Org table.
+DATA is in the TSV format."
+  (let ((beg (point)))
+    (insert data)
+    ;; LibreOffice source code defines
+    ;; 'application/x-libreoffice-tsvc' as Text TSV-Calc so it should
+    ;; be safe to handle this as TSV always.
+    (org-table-convert-region beg (point) '(16))))
+
 (defcustom org-yank-dnd-method 'ask
   "Action to perform on the dropped and the pasted files.
 When the value is the symbol,
-- 
2.45.2

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Ihor Radchenko]

Visuwesh <visuweshm@gmail.com> writes:

> Attached patch adds support for yanking LibreOffice Calc tables as Org
> tables.  Tables from docx files (or LO's equivalent) cannot be yanked
> though since the mimetype for them are richtext or HTML.

Thanks!

I just tried your patch (make repro) with a simple table

foo	bar	
1	2	3

I got

* This is test
| f\0o\0o\0 | \0b\0a\0r\0 | \0    |
| \01\0     | \02\0       | \03\0 |
| \0        |             |       |

Something is off with encoding.

> +(defun org--libreoffice-table-handler (_mimetype data)
> +  "Insert LibreOffice Calc table DATA as an Org table.
> +DATA is in the TSV format."
> +  (let ((beg (point)))
> +    (insert data)
> +    ;; LibreOffice source code defines
> +    ;; 'application/x-libreoffice-tsvc' as Text TSV-Calc so it should
> +    ;; be safe to handle this as TSV always.
> +    (org-table-convert-region beg (point) '(16))))

What if the point is in the middle of something else?
For example, in the middle of another table.
Then, conversion may yield wild outcomes.
Maybe it is safer to convert region in a temporary buffer and
insert the final result into the Org document being edited.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Visuwesh]

[திங்கள் அக்டோபர் 14, 2024] Ihor Radchenko wrote:

> Visuwesh <visuweshm@gmail.com> writes:
>
>> Attached patch adds support for yanking LibreOffice Calc tables as Org
>> tables.  Tables from docx files (or LO's equivalent) cannot be yanked
>> though since the mimetype for them are richtext or HTML.
>
> Thanks!
>
> I just tried your patch (make repro) with a simple table
>
> foo	bar	
> 1	2	3
>
> I got
>
> * This is test
> | f\0o\0o\0 | \0b\0a\0r\0 | \0    |
> | \01\0     | \02\0       | \03\0 |
> | \0        |             |       |
>
> Something is off with encoding.

I cannot reproduce it on my end.  I tried with different documents and
document types (Excel and LO format).  Would it be possible to send the
faulty file?

>> +(defun org--libreoffice-table-handler (_mimetype data)
>> +  "Insert LibreOffice Calc table DATA as an Org table.
>> +DATA is in the TSV format."
>> +  (let ((beg (point)))
>> +    (insert data)
>> +    ;; LibreOffice source code defines
>> +    ;; 'application/x-libreoffice-tsvc' as Text TSV-Calc so it should
>> +    ;; be safe to handle this as TSV always.
>> +    (org-table-convert-region beg (point) '(16))))
>
> What if the point is in the middle of something else?
> For example, in the middle of another table.
> Then, conversion may yield wild outcomes.
> Maybe it is safer to convert region in a temporary buffer and
> insert the final result into the Org document being edited.

Even if we do the processing in a temporary buffer, we would still end
up inserting the table after point.  I say we leave it to the user to be
prudent and not call yank-media when in the middle of a table or
somesuch.  We do not do any special handling in other handlers, so why
do it here?

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 1534 bytes --]

Visuwesh <visuweshm@gmail.com> writes:

>> Something is off with encoding.
>
> I cannot reproduce it on my end.  I tried with different documents and
> document types (Excel and LO format).  Would it be possible to send the
> faulty file?

Here is the recipe
1. make repro (I tried with the latest Emacs master and with Emacs 29)
2. Open test.odt (attached)
3. M-x org-mode
4. Copy the non-empty cell area
5. M-x yank-media <RET> app<TAB> <RET>
6. Observe encoding problems

>>> +    (insert data)
>>> +    ;; LibreOffice source code defines
>>> +    ;; 'application/x-libreoffice-tsvc' as Text TSV-Calc so it should
>>> +    ;; be safe to handle this as TSV always.
>>> +    (org-table-convert-region beg (point) '(16))))
>>
>> What if the point is in the middle of something else?
>> For example, in the middle of another table.
>> Then, conversion may yield wild outcomes.
>> Maybe it is safer to convert region in a temporary buffer and
>> insert the final result into the Org document being edited.
>
> Even if we do the processing in a temporary buffer, we would still end
> up inserting the table after point.  I say we leave it to the user to be
> prudent and not call yank-media when in the middle of a table or
> somesuch.  We do not do any special handling in other handlers, so why
> do it here?

Consider the following example:

* Heading <point>

M-x yank-media ... will slurp the heading

| * Heading a | b | c |

In contrast, the existing handlers will only insert some text at point,
never touching the text around.


[-- Attachment #2: test.ods --]
[-- Type: application/vnd.oasis.opendocument.spreadsheet, Size: 8912 bytes --]

[-- Attachment #3: Type: text/plain, Size: 224 bytes --]


-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Visuwesh]

[செவ்வாய் அக்டோபர் 15, 2024] Ihor Radchenko wrote:

> Visuwesh <visuweshm@gmail.com> writes:
>
>>> Something is off with encoding.
>>
>> I cannot reproduce it on my end.  I tried with different documents and
>> document types (Excel and LO format).  Would it be possible to send the
>> faulty file?
>
> Here is the recipe
> 1. make repro (I tried with the latest Emacs master and with Emacs 29)
> 2. Open test.odt (attached)
> 3. M-x org-mode
> 4. Copy the non-empty cell area
> 5. M-x yank-media <RET> app<TAB> <RET>
> 6. Observe encoding problems

Thanks for the reproducer.  I think this might be locale-dependent.  I
fail to reproduce on my end with the following locale settings:

    LANG=en_GB.UTF-8
    LC_COLLAGE=ta_IN.UTF-8
    LC_MONETARY=ta_IN.UTF-8
    LC_NUMERIC=ta_IN.UTF-8

    locale-coding-system = 'utf-8-unix

I have no idea how to deal with locale issues like this unfortunately.

>>>> +    (insert data)
>>>> +    ;; LibreOffice source code defines
>>>> +    ;; 'application/x-libreoffice-tsvc' as Text TSV-Calc so it should
>>>> +    ;; be safe to handle this as TSV always.
>>>> +    (org-table-convert-region beg (point) '(16))))
>>>
>>> What if the point is in the middle of something else?
>>> For example, in the middle of another table.
>>> Then, conversion may yield wild outcomes.
>>> Maybe it is safer to convert region in a temporary buffer and
>>> insert the final result into the Org document being edited.
>>
>> Even if we do the processing in a temporary buffer, we would still end
>> up inserting the table after point.  I say we leave it to the user to be
>> prudent and not call yank-media when in the middle of a table or
>> somesuch.  We do not do any special handling in other handlers, so why
>> do it here?
>
> Consider the following example:
>
> * Heading <point>
>
> M-x yank-media ... will slurp the heading
>
> | * Heading a | b | c |
>
> In contrast, the existing handlers will only insert some text at point,
> never touching the text around.

Would it be sufficient to unconditionally insert a newline after point?
Maybe even after going to end of current element (if that can be done).

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Ihor Radchenko]

Visuwesh <visuweshm@gmail.com> writes:

>> Consider the following example:
>>
>> * Heading <point>
>>
>> M-x yank-media ... will slurp the heading
>>
>> | * Heading a | b | c |
>>
>> In contrast, the existing handlers will only insert some text at point,
>> never touching the text around.
>
> Would it be sufficient to unconditionally insert a newline after point?
> Maybe even after going to end of current element (if that can be done).

It will be more fragile compared to using temporary buffer.
As an alternative, you can apply narrowing around the call to
`org-table-convert-region'.

Also, do note that any text exceeding
`org-table-convert-region-max-lines' will lead to
`org-table-convert-region' throwing an error.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Visuwesh]

[வியாழன் அக்டோபர் 17, 2024] Ihor Radchenko wrote:

> Visuwesh <visuweshm@gmail.com> writes:
>
>>> Consider the following example:
>>>
>>> * Heading <point>
>>>
>>> M-x yank-media ... will slurp the heading
>>>
>>> | * Heading a | b | c |
>>>
>>> In contrast, the existing handlers will only insert some text at point,
>>> never touching the text around.
>>
>> Would it be sufficient to unconditionally insert a newline after point?
>> Maybe even after going to end of current element (if that can be done).
>
> It will be more fragile compared to using temporary buffer.
> As an alternative, you can apply narrowing around the call to
> `org-table-convert-region'.

Thanks for the explanation.  I understand your initial suggestion.  I
will use a temp buffer in an updated patch.

> Also, do note that any text exceeding
> `org-table-convert-region-max-lines' will lead to
> `org-table-convert-region' throwing an error.

Catching that and messaging the user should be sufficient?

I will post an updated patch if someone comments on how to resolve the
encoding issue.

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Ihor Radchenko]

Visuwesh <visuweshm@gmail.com> writes:

>> Also, do note that any text exceeding
>> `org-table-convert-region-max-lines' will lead to
>> `org-table-convert-region' throwing an error.
>
> Catching that and messaging the user should be sufficient?

Maybe. Or we may catch and ask user to lift the restriction temporarily.
Something like:

Inserting large table with more than `org-table-convert-region-max-lines' <number>. Continue? (yes/no)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Visuwesh]

[சனி அக்டோபர் 19, 2024] Ihor Radchenko wrote:

> Visuwesh <visuweshm@gmail.com> writes:
>
>>> Also, do note that any text exceeding
>>> `org-table-convert-region-max-lines' will lead to
>>> `org-table-convert-region' throwing an error.
>>
>> Catching that and messaging the user should be sufficient?
>
> Maybe. Or we may catch and ask user to lift the restriction temporarily.
> Something like:
>
> Inserting large table with more than `org-table-convert-region-max-lines' <number>. Continue? (yes/no)

Ah, something like what Calc does.  Yes, I will go with this approach.
Thanks for the suggestion.

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Ihor Radchenko]

Visuwesh <visuweshm@gmail.com> writes:

>> Here is the recipe
>> 1. make repro (I tried with the latest Emacs master and with Emacs 29)
>> 2. Open test.odt (attached)
>> 3. M-x org-mode
>> 4. Copy the non-empty cell area
>> 5. M-x yank-media <RET> app<TAB> <RET>
>> 6. Observe encoding problems
>
> Thanks for the reproducer.  I think this might be locale-dependent.  I
> fail to reproduce on my end with the following locale settings:
>
>     LANG=en_GB.UTF-8
>     LC_COLLAGE=ta_IN.UTF-8
>     LC_MONETARY=ta_IN.UTF-8
>     LC_NUMERIC=ta_IN.UTF-8
>
>     locale-coding-system = 'utf-8-unix
>
> I have no idea how to deal with locale issues like this unfortunately.

I tested a bit further, and it might not be locale. Maybe it is some
LibreOffice version issue or build issue.

Below is the DATA passed to yank handler:

Debugger entered--entering a function:
* org--libreoffice-table-handler(application/x-libreoffice-tsvc #("1\0\11\0002\0\11\0003\0\n\0004\0\11\0005\0\11\0005\0\n\0" 0 24 (foreign-selection application/x-libreoffice-tsvc)))
  yank-media()

As you can see, the data contains \0 characters.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Visuwesh]

[சனி அக்டோபர் 19, 2024] Ihor Radchenko wrote:

> Visuwesh <visuweshm@gmail.com> writes:
>
>>> Here is the recipe
>>> 1. make repro (I tried with the latest Emacs master and with Emacs 29)
>>> 2. Open test.odt (attached)
>>> 3. M-x org-mode
>>> 4. Copy the non-empty cell area
>>> 5. M-x yank-media <RET> app<TAB> <RET>
>>> 6. Observe encoding problems
>>
>> Thanks for the reproducer.  I think this might be locale-dependent.  I
>> fail to reproduce on my end with the following locale settings:
>>
>>     LANG=en_GB.UTF-8
>>     LC_COLLAGE=ta_IN.UTF-8
>>     LC_MONETARY=ta_IN.UTF-8
>>     LC_NUMERIC=ta_IN.UTF-8
>>
>>     locale-coding-system = 'utf-8-unix
>>
>> I have no idea how to deal with locale issues like this unfortunately.
>
> I tested a bit further, and it might not be locale. Maybe it is some
> LibreOffice version issue or build issue.

My LibreOffice version is

    % libreoffice --version
    LibreOffice 24.2.5.2 420(Build:2)

> Below is the DATA passed to yank handler:
>
> Debugger entered--entering a function:
> * org--libreoffice-table-handler(application/x-libreoffice-tsvc #("1\0\11\0002\0\11\0003\0\n\0004\0\11\0005\0\11\0005\0\n\0" 0 24 (foreign-selection application/x-libreoffice-tsvc)))
>   yank-media()
>
> As you can see, the data contains \0 characters.

Just to be sure that yank-media is not in the middle of encoding
problem, can you say what

    (gui-get-selection 'PRIMARY 'application/x-libreoffice-tsvc)

returns?  If it contains NULL bytes too, then I would be completely
clueless.

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Ihor Radchenko]

Visuwesh <visuweshm@gmail.com> writes:

> My LibreOffice version is
>
>     % libreoffice --version
>     LibreOffice 24.2.5.2 420(Build:2)

LibreOffice 7.6.7.2 60(Build:2)

>> As you can see, the data contains \0 characters.
>
> Just to be sure that yank-media is not in the middle of encoding
> problem, can you say what
>
>     (gui-get-selection 'PRIMARY 'application/x-libreoffice-tsvc)
>
> returns?  If it contains NULL bytes too, then I would be completely
> clueless.

Debugger entered--returning value: #("1\0\11\0002\0\11\0003\0\n\0004\0\11\0005\0\11\0006\0\n\0" 0 24 (foreign-selection application/x-libreoffice-tsvc))
  (gui-get-selection PRIMARY application/x-libreoffice-tsvc)
  (progn (debug) (gui-get-selection 'PRIMARY 'application/x-libreoffice-tsvc))

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Visuwesh]

[சனி அக்டோபர் 19, 2024] Ihor Radchenko wrote:

> Visuwesh <visuweshm@gmail.com> writes:
>
>> My LibreOffice version is
>>
>>     % libreoffice --version
>>     LibreOffice 24.2.5.2 420(Build:2)
>
> LibreOffice 7.6.7.2 60(Build:2)

It might not be the version after all...  I downloaded the AppImage from
https://appimages.libreitalia.org/ (listed in
https://www.libreoffice.org/download/appimage/) and it does not have
NULL bytes at all:

    Debugger entered: (#("1\0112\0113\n4\0115\0116\n" 0 12 (foreign-selection application/x-libreoffice-tsvc)))
      (progn (debug nil (gui-get-selection 'PRIMARY 'application/x-libreoffice-tsvc)))


\011 is the TAB character.

    % ./LibreOffice-7.4.6.2.basic-x86_64.AppImage --version
    LibreOffice 7.4.6.2 5b1f5509c2decdade7fda905e3e1429a67acd63d

>>> As you can see, the data contains \0 characters.
>>
>> Just to be sure that yank-media is not in the middle of encoding
>> problem, can you say what
>>
>>     (gui-get-selection 'PRIMARY 'application/x-libreoffice-tsvc)
>>
>> returns?  If it contains NULL bytes too, then I would be completely
>> clueless.
>
> Debugger entered--returning value: #("1\0\11\0002\0\11\0003\0\n\0004\0\11\0005\0\11\0006\0\n\0" 0 24 (foreign-selection application/x-libreoffice-tsvc))
>   (gui-get-selection PRIMARY application/x-libreoffice-tsvc)
>   (progn (debug) (gui-get-selection 'PRIMARY 'application/x-libreoffice-tsvc))

Thanks so it is confirmed that yank-media does not do any conversion.
What if you instead do

    (x-get-selection-internal 'PRIMARY 'application/x-libreoffice-tsvc)

?  I am not sure if it will reveal anything.  Also, what is the value of
x-select-request-type?

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Ihor Radchenko]

Visuwesh <visuweshm@gmail.com> writes:

>> Debugger entered--returning value: #("1\0\11\0002\0\11\0003\0\n\0004\0\11\0005\0\11\0006\0\n\0" 0 24 (foreign-selection application/x-libreoffice-tsvc))
>>   (gui-get-selection PRIMARY application/x-libreoffice-tsvc)
>>   (progn (debug) (gui-get-selection 'PRIMARY 'application/x-libreoffice-tsvc))
>
> Thanks so it is confirmed that yank-media does not do any conversion.
> What if you instead do
>
>     (x-get-selection-internal 'PRIMARY 'application/x-libreoffice-tsvc)

Debugger entered--returning value: #("1\0\11\0002\0\11\0003\0\n\0004\0\11\0005\0\11\0006\0\n\0" 0 24 (foreign-selection STRING))
  (x-get-selection-internal PRIMARY application/x-libreoffice-tsvc)
  (progn (debug) (x-get-selection-internal 'PRIMARY 'application/x-libreoffice-tsvc))

> ?  I am not sure if it will reveal anything.  Also, what is the value of
> x-select-request-type?

nil

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Visuwesh]

[சனி அக்டோபர் 19, 2024] Ihor Radchenko wrote:

> Visuwesh <visuweshm@gmail.com> writes:
>
>>> Debugger entered--returning value: #("1\0\11\0002\0\11\0003\0\n\0004\0\11\0005\0\11\0006\0\n\0" 0 24 (foreign-selection application/x-libreoffice-tsvc))
>>>   (gui-get-selection PRIMARY application/x-libreoffice-tsvc)
>>>   (progn (debug) (gui-get-selection 'PRIMARY 'application/x-libreoffice-tsvc))
>>
>> Thanks so it is confirmed that yank-media does not do any conversion.
>> What if you instead do
>>
>>     (x-get-selection-internal 'PRIMARY 'application/x-libreoffice-tsvc)
>
> Debugger entered--returning value: #("1\0\11\0002\0\11\0003\0\n\0004\0\11\0005\0\11\0006\0\n\0" 0 24 (foreign-selection STRING))
>   (x-get-selection-internal PRIMARY application/x-libreoffice-tsvc)
>   (progn (debug) (x-get-selection-internal 'PRIMARY 'application/x-libreoffice-tsvc))
>
>> ?  I am not sure if it will reveal anything.  Also, what is the value of
>> x-select-request-type?
>
> nil

Sorry, I am seriously out of clues.  We could outright remove the NULL
bytes if they do not change the underlying text but this feels like a
hack...  I hope someone else can reproduce this issue on their end.  I
do not have another Linux system in which I could freely install Emacs
and test this out.

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Rens Oliemans]

Visuwesh <visuweshm@gmail.com> writes:

> I hope someone else can reproduce this issue on their end. I do not have
> another Linux system in which I could freely install Emacs and test this
> out.

I'm not familiar with yank-media in any way, but here are the results of the
commands you sent in this thread. They're all tested on the test.ods sent by
Ihor.

    $ libreoffice --version
    LibreOffice 24.2.6.2 420(Build:2)

    (version)
    "GNU Emacs 29.4 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.43, cairo version 1.18.2)"

>>> Just to be sure that yank-media is not in the middle of encoding
>>> problem, can you say what
>>>
>>>     (gui-get-selection 'PRIMARY 'application/x-libreoffice-tsvc)
>>>
>>> returns?

    (gui-get-selection 'PRIMARY 'application/x-libreoffice-tsvc)

nil. However,

    (gui-get-selection 'CLIPBOARD 'application/x-libreoffice-tsvc)

#("foo	bar	
1	2	3
" 0 15 (foreign-selection application/x-libreoffice-tsvc))

> What if you instead do
>
>   (x-get-selection-internal 'PRIMARY 'application/x-libreoffice-tsvc)  
>
> ?

    (x-get-selection-internal 'PRIMARY 'application/x-libreoffice-tsvc)

nil. However,

    (x-get-selection-internal 'CLIPBOARD 'application/x-libreoffice-tsvc)

#("foo	bar	
1	2	3
" 0 15 (foreign-selection STRING))

Also:

    Debugger entered--entering a function:
    * org--libreoffice-table-handler(application/x-libreoffice-tsvc #("foo\11bar\11\n1\0112\0113\n" 0 15 (foreign-selection STRING)))
      yank-media()

> Also, what is the value of x-select-request-type?

nil.

`current-locale-environment' is "en_US.UTF-8"

I hope that this is helpful.

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Visuwesh]

[சனி அக்டோபர் 19, 2024] Rens Oliemans wrote:

> Visuwesh <visuweshm@gmail.com> writes:
>
>> I hope someone else can reproduce this issue on their end. I do not have
>> another Linux system in which I could freely install Emacs and test this
>> out.
>
> I'm not familiar with yank-media in any way, but here are the results of the
> commands you sent in this thread. They're all tested on the test.ods sent by
> Ihor.
>
>     $ libreoffice --version
>     LibreOffice 24.2.6.2 420(Build:2)
>
>     (version)
>     "GNU Emacs 29.4 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.43, cairo version 1.18.2)"
>
>>>> Just to be sure that yank-media is not in the middle of encoding
>>>> problem, can you say what
>>>>
>>>>     (gui-get-selection 'PRIMARY 'application/x-libreoffice-tsvc)
>>>>
>>>> returns?
>
>     (gui-get-selection 'PRIMARY 'application/x-libreoffice-tsvc)
>
> nil. However,
>
>     (gui-get-selection 'CLIPBOARD 'application/x-libreoffice-tsvc)
>
> #("foo	bar	
> 1	2	3
> " 0 15 (foreign-selection application/x-libreoffice-tsvc))
>
>> What if you instead do
>>
>>   (x-get-selection-internal 'PRIMARY 'application/x-libreoffice-tsvc)  
>>
>> ?
>
>     (x-get-selection-internal 'PRIMARY 'application/x-libreoffice-tsvc)
>
> nil. However,
>
>     (x-get-selection-internal 'CLIPBOARD 'application/x-libreoffice-tsvc)
>
> #("foo	bar	
> 1	2	3
> " 0 15 (foreign-selection STRING))
>
> Also:
>
>     Debugger entered--entering a function:
>     * org--libreoffice-table-handler(application/x-libreoffice-tsvc #("foo\11bar\11\n1\0112\0113\n" 0 15 (foreign-selection STRING)))
>       yank-media()
>
>> Also, what is the value of x-select-request-type?
>
> nil.
>
> `current-locale-environment' is "en_US.UTF-8"
>
> I hope that this is helpful.

Thank you, this helps a lot!  The fact that you had to change PRIMARY to
CLIPBOARD should not be a problem in practise: (1) LO should be putting
the data in the CLIPBOARD anyway, and (2) yank-media abstracts this away
for us.

If possible, could you please try LibreOffice 7.6.4? The version Ihor
has installed in his system.  There is an AppImage available for you to
try.

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Rens Oliemans]

Visuwesh <visuweshm@gmail.com> writes:

> If possible, could you please try LibreOffice 7.6.4? The version Ihor
> has installed in his system.  There is an AppImage available for you to
> try.

Yep, on 7.6.4 the problem occurs.

    $ ./LibreOffice-7.6.4.1.basic-x86_64.AppImage --version
    LibreOffice 7.6.4.1 e19e193f88cd6c0525a17fb7a176ed8e6a3e2aa1

    Debugger entered--entering a function:
    * org--libreoffice-table-handler(application/x-libreoffice-tsvc #("f\0o\0o\0\11\0b\0a\0r\0\11\0\n\0001\0\11\0002\0\11\0003\0\n\0" 0 30 (foreign-selection STRING)))
      yank-media()

, resulting in

    | f\0o\0o\0 | \0b\0a\0r\0 | \0    |
    | \01\0     | \02\0       | \03\0 |
    | \0        |             |       |

To narrow it down a bit more: the problem exists in 7.6.5.2, and does not exist in the
following release, 7.6.6.3.

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Visuwesh]

[திங்கள் அக்டோபர் 21, 2024] Rens Oliemans wrote:

> Visuwesh <visuweshm@gmail.com> writes:
>
>> If possible, could you please try LibreOffice 7.6.4? The version Ihor
>> has installed in his system.  There is an AppImage available for you to
>> try.
>
> Yep, on 7.6.4 the problem occurs.
>
>     $ ./LibreOffice-7.6.4.1.basic-x86_64.AppImage --version
>     LibreOffice 7.6.4.1 e19e193f88cd6c0525a17fb7a176ed8e6a3e2aa1
>
>     Debugger entered--entering a function:
>     * org--libreoffice-table-handler(application/x-libreoffice-tsvc #("f\0o\0o\0\11\0b\0a\0r\0\11\0\n\0001\0\11\0002\0\11\0003\0\n\0" 0 30 (foreign-selection STRING)))
>       yank-media()
>
> , resulting in
>
>     | f\0o\0o\0 | \0b\0a\0r\0 | \0    |
>     | \01\0     | \02\0       | \03\0 |
>     | \0        |             |       |
>
> To narrow it down a bit more: the problem exists in 7.6.5.2, and does not exist in the
> following release, 7.6.6.3.

Thank you!  So it is most likely not locale related---a major headache
saved.  I will look into this soon (cannot promise anything this week
unfortunately).

Re: [PATCH] Add yank-media handler for LibreOffice Calc tables

[Visuwesh]

[-- Attachment #1: Type: text/plain, Size: 2713 bytes --]

Sorry for the late reply.  I was busy with uni.

[திங்கள் அக்டோபர் 21, 2024] Visuwesh wrote:

> [திங்கள் அக்டோபர் 21, 2024] Rens Oliemans wrote:
>
>> Visuwesh <visuweshm@gmail.com> writes:
>>
>>> If possible, could you please try LibreOffice 7.6.4? The version Ihor
>>> has installed in his system.  There is an AppImage available for you to
>>> try.
>>
>> Yep, on 7.6.4 the problem occurs.
>>
>>     $ ./LibreOffice-7.6.4.1.basic-x86_64.AppImage --version
>>     LibreOffice 7.6.4.1 e19e193f88cd6c0525a17fb7a176ed8e6a3e2aa1
>>
>>     Debugger entered--entering a function:
>>     * org--libreoffice-table-handler(application/x-libreoffice-tsvc #("f\0o\0o\0\11\0b\0a\0r\0\11\0\n\0001\0\11\0002\0\11\0003\0\n\0" 0 30 (foreign-selection STRING)))
>>       yank-media()
>>
>> , resulting in
>>
>>     | f\0o\0o\0 | \0b\0a\0r\0 | \0    |
>>     | \01\0     | \02\0       | \03\0 |
>>     | \0        |             |       |
>>
>> To narrow it down a bit more: the problem exists in 7.6.5.2, and does not exist in the
>> following release, 7.6.6.3.
>
> Thank you!  So it is most likely not locale related---a major headache
> saved.  I will look into this soon (cannot promise anything this week
> unfortunately).

For reasons that escape me completely, I did not see null bytes in the
selection in

    % ./LibreOffice-7.6.4.1.standard.help-x86_64.AppImage --version
    LibreOffice 7.6.4.1 e19e193f88cd6c0525a17fb7a176ed8e6a3e2aa1

But considering that the chances of having null bytes in documents are
very low, I decided to bruteforce strip them.

[சனி அக்டோபர் 19, 2024] Visuwesh wrote:

> [சனி அக்டோபர் 19, 2024] Ihor Radchenko wrote:
>
>> Visuwesh <visuweshm@gmail.com> writes:
>>
>>>> Also, do note that any text exceeding
>>>> `org-table-convert-region-max-lines' will lead to
>>>> `org-table-convert-region' throwing an error.
>>>
>>> Catching that and messaging the user should be sufficient?
>>
>> Maybe. Or we may catch and ask user to lift the restriction temporarily.
>> Something like:
>>
>> Inserting large table with more than `org-table-convert-region-max-lines' <number>. Continue? (yes/no)
>
> Ah, something like what Calc does.  Yes, I will go with this approach.
> Thanks for the suggestion.

I have now done this in the attached patch.  I use a temporary buffer to
convert the table, and also decode it as per buffer-file-coding-system
of the Org buffer as I could not tell whether LibreOffice always uses
UTF-8 or not.  I believe buffer-file-coding-system should be utf-8 on
most systems so it should be a safe choice.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-yank-media-handler-for-LibreOffice-Calc-tables.patch --]
[-- Type: text/x-diff, Size: 4524 bytes --]

From f392dc1f3eb550c3d833683113c2d958f7f9b722 Mon Sep 17 00:00:00 2001
From: Visuwesh <visuweshm@gmail.com>
Date: Mon, 14 Oct 2024 22:04:23 +0530
Subject: [PATCH] Add `yank-media' handler for LibreOffice Calc tables

* lisp/org.el (org--libreoffice-table-handler): Add new `yank-media'
handler for tables copied from LibreOffice Calc documents.
(org-setup-yank-dnd-handlers): Register it.
* doc/org-manual.org: (Drag and Drop & ~yank-media~): Update the
manual to mention the new feature.
* etc/ORG-NEWS: Announce the new feature.
---
 doc/org-manual.org |  6 +++++-
 etc/ORG-NEWS       |  5 +++++
 lisp/org.el        | 27 ++++++++++++++++++++++++++-
 3 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 95846b475..e23e388a0 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -21684,7 +21684,8 @@ value as ~org-attach-method~ (~cp~ by default).]
 
 #+cindex: pasting files, images from clipboard
 Starting from Emacs 29, Org mode supports ~yank-media~ command to yank
-images from the clipboard and files from a file manager.
+images from the clipboard, files from a file manager and tables copied
+from LibreOffice Calc documents.
 
 #+vindex: org-yank-image-save-method
 When yanking images from clipboard, Org saves the image on disk and
@@ -21702,6 +21703,9 @@ of ~org-yank-dnd-method~.  Image files pasted this way also respect
 the value of ~org-yank-image-save-method~ when the action to perform
 is =attach=.
 
+Yanking tables copied from LibreOffice Calc documents are inserted as
+Org tables.
+
 ** Repeating commands
 :PROPERTIES:
 :DESCRIPTION: Repeating navigation commands
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 08857962b..9ad2da5d4 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -136,6 +136,11 @@ The keybindings in the repeat-maps can be changed by customizing
 
 See the new [[info:org#Repeating commands]["Repeating commands"]] section in Org mode manual.
 
+*** Tables copied from LibreOffice Calc documents can be pasted as Org tables
+
+Tables copied into the clipboard from LibreOffice Calc documents can
+now be pasted as an Org table using ~yank-media~.
+
 ** New and changed options
 
 # Chanes deadling with changing default values of customizations,
diff --git a/lisp/org.el b/lisp/org.el
index def094a9a..f9c275ca7 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -20354,7 +20354,9 @@ (defun org-setup-yank-dnd-handlers ()
     ;; Looks like different DEs go for different handler names,
     ;; https://larsee.com/blog/2019/05/clipboard-files/.
     (yank-media-handler "x/special-\\(?:gnome\\|KDE\\|mate\\)-files"
-                        #'org--copied-files-yank-media-handler))
+                        #'org--copied-files-yank-media-handler)
+    (yank-media-handler "application/x-libreoffice-tsvc"
+                        #'org--libreoffice-table-handler))
   (when (boundp 'x-dnd-direct-save-function)
     (setq-local x-dnd-direct-save-function #'org--dnd-xds-function)))
 
@@ -20447,6 +20449,29 @@ (defun org--copied-files-yank-media-handler (_mimetype data)
           (org--dnd-local-file-handler f action sep)
         (message "File `%s' is not readable, skipping" f)))))
 
+(defun org--libreoffice-table-handler (_mimetype data)
+  "Insert LibreOffice Calc table DATA as an Org table.
+DATA is in the TSV format."
+  ;; Some LibreOffice versions have the null byte in the selection.
+  ;; It should be safe to remove it.
+  (when (string-search "\0" data)
+    (setq data (string-replace "\0" "" data)))
+  (let ((orig-buf (current-buffer)))
+    (with-temp-buffer
+      (decode-coding-string data buffer-file-coding-system nil (current-buffer))
+      (let ((tmp (current-buffer))
+            (nlines (count-lines (point-min) (point-max))))
+        (when (> nlines org-table-convert-region-max-lines)
+          (unless (yes-or-no-p
+                   (format "Inserting large table with %d lines, more than `org-table-convert-region-max-lines'.  Continue? "
+                           nlines))
+            (user-error "Table is larger than limit `org-table-convert-region-max-lines'")))
+        ;; User has chosen to ignore the limit.
+        (let ((org-table-convert-region-max-lines most-positive-fixnum))
+          (org-table-convert-region (point-min) (point-max)))
+        (with-current-buffer orig-buf
+          (insert-buffer-substring tmp))))))
+
 (defcustom org-yank-dnd-method 'ask
   "Action to perform on the dropped and the pasted files.
 When the value is the symbol,
-- 
2.45.2