org table with datestamp convert to csv and then xlsx or ods: problem

org table with datestamp convert to csv and then xlsx or ods: problem

[Uwe Brauer]

Hi

consider please

| Entry | Date       |
|   100 | <2018-01-07 Sun> |

I first export this to csv


Entry,Date
100,<2018-01-07 Sun>



and then via gnumeric ssconvert or LO unoconv
to xlsx or ods, however the datestamp is not correctly converted to a
datestamp understood my xlsx or unoconv.

Is this a lacking feature or a bug?

Uwe Brauer 

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Nicolas Goaziou]

Hello,

Uwe Brauer <oub@mat.ucm.es> writes:

> Hi
>
> consider please
>
> | Entry | Date       |
> |   100 | <2018-01-07 Sun> |
>
> I first export this to csv
>
>
> Entry,Date
> 100,<2018-01-07 Sun>
>
>
>
> and then via gnumeric ssconvert or LO unoconv
> to xlsx or ods, however the datestamp is not correctly converted to a
> datestamp understood my xlsx or unoconv.
>
> Is this a lacking feature or a bug?

None. The export to CSV looks correct and complete. Conversion to CSV
does not pretend converting anything else, in particular timestamps, to
another format.

Regards,

-- 
Nicolas Goaziou

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Uwe Brauer]

   > Hello,
   > Uwe Brauer <oub@mat.ucm.es> writes:


   > None. The export to CSV looks correct and complete. Conversion to CSV
   > does not pretend converting anything else, in particular timestamps, to
   > another format.

Ok,

Now how could that be achieved? I seem not the only one missing that a
feature.
https://lists.gnu.org/archive/html/emacs-orgmode/2016-11/msg00398.html

So I googled and 
I tried

https://stackoverflow.com/questions/23297422/org-mode-timestamp-format-when-exported

Like
(defun org-export-filter-timestamp-remove-brackets (timestamp backend info)
  "removes relevant brackets from a timestamp"
  (cond
   ((org-export-derived-backend-p backend 'latex)
    (replace-regexp-in-string "[<>]\\|[][]" "" timestamp))
   ((org-export-derived-backend-p backend 'orgtbl-to-csv)
    (replace-regexp-in-string "[<>]\\|[][]" "" timestamp))
   ((org-export-derived-backend-p backend 'ascii)
    (replace-regexp-in-string "[<>]\\|[][]" "" timestamp))
   ((org-export-derived-backend-p backend 'html)
    (replace-regexp-in-string "&[lg]t;\\|[][]" "" timestamp))))

(eval-after-load 'ox '(add-to-list
                       'org-export-filter-timestamp-functions
                       'org-export-filter-timestamp-remove-brackets))


(defun org-ascii-timestamp (timestamp _contents info)
  (format-time-string
   "%d.%m.%y"
   (org-read-date nil t (org-timestamp-translate timestamp))))


(add-to-list 'org-export-filter-timestamp-functions
             #'endless/filter-timestamp)

(defun endless/filter-timestamp (trans back _comm)
  "Remove <> around time-stamps."
  (pcase back
    ((or `jekyll `html)
     (replace-regexp-in-string "&[lg]t;" "" trans))
    (`latex
     (replace-regexp-in-string "[<>]" "" trans))
    (`ascii
     (replace-regexp-in-string "[<>]" "" trans))
    (`csv
     (replace-regexp-in-string "[<>]" "" trans))))

But nothing helped.

Thanks

Uwe Brauer 

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Nicolas Goaziou]

Uwe Brauer <oub@mat.ucm.es> writes:

>    > Hello,
>    > Uwe Brauer <oub@mat.ucm.es> writes:
>
>
>    > None. The export to CSV looks correct and complete. Conversion to CSV
>    > does not pretend converting anything else, in particular timestamps, to
>    > another format.
>
> Ok,
>
> Now how could that be achieved? I seem not the only one missing that a
> feature.
> https://lists.gnu.org/archive/html/emacs-orgmode/2016-11/msg00398.html

For example:

  (defun my-format-timestamps (cell)
    (org-quote-csv-field
     (replace-regexp-in-string org-ts-regexp-both "\\1" cell)))

  (defun my-tbl-to-csv (table params)
    (orgtbl-to-csv table
                   (org-combine-plists '(:fmt my-format-timestamps) params)))

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Uwe Brauer]

   > Uwe Brauer <oub@mat.ucm.es> writes:

   > For example:

   >   (defun my-format-timestamps (cell)
   >     (org-quote-csv-field
   >      (replace-regexp-in-string org-ts-regexp-both "\\1" cell)))

   >   (defun my-tbl-to-csv (table params)
   >     (orgtbl-to-csv table
   >                    (org-combine-plists '(:fmt my-format-timestamps) params)))

Thanks, but forgive me my ignorance, how are these functions suppose to
work. They are not interactive, so shall I put them in some hook or
other function? Sorry for this elementary question.

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Nicolas Goaziou]

Uwe Brauer <oub@mat.ucm.es> writes:

>    > Uwe Brauer <oub@mat.ucm.es> writes:
>
>    > For example:
>
>    >   (defun my-format-timestamps (cell)
>    >     (org-quote-csv-field
>    >      (replace-regexp-in-string org-ts-regexp-both "\\1" cell)))
>
>    >   (defun my-tbl-to-csv (table params)
>    >     (orgtbl-to-csv table
>    >                    (org-combine-plists '(:fmt my-format-timestamps) params)))
>
> Thanks, but forgive me my ignorance, how are these functions suppose to
> work. They are not interactive, so shall I put them in some hook or
> other function? Sorry for this elementary question.

Use `my-tbl-to-csv' as a replacement for `orgtbl-to-csv'. How did you
"export table to CSV" in the first place?

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Uwe Brauer]

   > Uwe Brauer <oub@mat.ucm.es> writes:

   > Use `my-tbl-to-csv' as a replacement for `orgtbl-to-csv'. How did you
that is what I thought.

So I called org-table-export
but then the prompt did not allow my to specify my-tbl-to-csv
so I customized (customize-option (quote org-table-export-default-format))
to your new function. But that did not work, again a prompt appeared and
I could not select your new filter

   > "export table to CSV" in the first place?

The same way I just described but selecting orgtbl-to-csv

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Nicolas Goaziou]

Uwe Brauer <oub@mat.ucm.es> writes:

>    > Uwe Brauer <oub@mat.ucm.es> writes:
>
>    > Use `my-tbl-to-csv' as a replacement for `orgtbl-to-csv'. How did you
> that is what I thought.
>
> So I called org-table-export
> but then the prompt did not allow my to specify my-tbl-to-csv
> so I customized (customize-option (quote org-table-export-default-format))
> to your new function. But that did not work, again a prompt appeared and
> I could not select your new filter

I have no trouble calling M-x org-table-export RET then choosing a file
name and forcing my-tbl-to-csv during prompt.

Otherwise, just evaluate

  (org-table-export "/file/name" "my-tbl-to-csv")

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Uwe Brauer]

    > Uwe Brauer <oub@mat.ucm.es> writes:

    > I have no trouble calling M-x org-table-export RET then choosing a file
    > name and forcing my-tbl-to-csv during prompt.

Hm I had to copy org-export-table into my addons file and
	(let* ((formats '("my-tbl-to-csv" "orgtbl-to-csv" "orgtbl-to-tsv"  "orgtbl-to-latex"
			  "orgtbl-to-html" "orgtbl-to-generic"
			  "orgtbl-to-texinfo" "orgtbl-to-orgtbl"
			  "orgtbl-to-unicode"))

It seems that the variable org-table-export-default-format is ignored, a
bug?

    > Otherwise, just evaluate

    >   (org-table-export "/file/name" "my-tbl-to-csv")

Right, thanks. A last question though.

I have set

org-time-stamp-custom-formats to
 (" %d.%m.%y " . " %d.%m.%y %a %H:%M "))

'(org-time-stamp-custom-formats (quote (" %d.%m.%y " . " %d.%m.%y %a %H:%M ")))

And indeed in my org files the timestamp are displayed 
for example as <19.12.17>.

However when I use
org-toggle-time-stamp-overlays
they are displayed as <2017-12-19 Tue>
the point is your conversion function will lead to
2017-12-19 Tue


which is much better for my purpose since the < > are deleted.



I am still wondering whether the format I chose via org-time-stamp-custom-formats
could be somehow used, so  that the result could be
19.12.17

But maybe this is impossible

Thanks

Uwe Brauer 

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Nicolas Goaziou]

Uwe Brauer <oub@mat.ucm.es> writes:

>     > Uwe Brauer <oub@mat.ucm.es> writes:
>
>     > I have no trouble calling M-x org-table-export RET then choosing a file
>     > name and forcing my-tbl-to-csv during prompt.
>
> Hm I had to copy org-export-table into my addons file and
> 	(let* ((formats '("my-tbl-to-csv" "orgtbl-to-csv" "orgtbl-to-tsv"  "orgtbl-to-latex"
> 			  "orgtbl-to-html" "orgtbl-to-generic"
> 			  "orgtbl-to-texinfo" "orgtbl-to-orgtbl"
> 			  "orgtbl-to-unicode"))

You don't need to. These are only suggestions, the final format needs
not matching any item in this list.

> It seems that the variable org-table-export-default-format is ignored, a
> bug?

Not really. See `org-table-export' docstring, last paragraph.

> Right, thanks. A last question though.
>
> I have set
>
> org-time-stamp-custom-formats to
>  (" %d.%m.%y " . " %d.%m.%y %a %H:%M "))
>
> '(org-time-stamp-custom-formats (quote (" %d.%m.%y " . " %d.%m.%y %a %H:%M ")))
>
> And indeed in my org files the timestamp are displayed 
> for example as <19.12.17>.
>
> However when I use
> org-toggle-time-stamp-overlays
> they are displayed as <2017-12-19 Tue>
> the point is your conversion function will lead to
> 2017-12-19 Tue
>
>
> which is much better for my purpose since the < > are deleted.
>
>
>
> I am still wondering whether the format I chose via org-time-stamp-custom-formats
> could be somehow used, so  that the result could be
> 19.12.17

You could try (untested):

    (defun my-format-timestamps (cell)
      (org-quote-csv-field
       (replace-regexp-in-string
        org-ts-regexp-both
        (lambda (m)
          (if (not org-display-custom-times) (substring m 1 -1)
            (let ((hours? (string-match-p "[0-9]+:[0-9]+" m)))
              (format-time-string (funcall (if hours? #'cdr #'car)
                                           org-time-stamp-custom-formats)
                                  (org-parse-time-string m)))))
        cell)))

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Uwe Brauer]

    > Uwe Brauer <oub@mat.ucm.es> writes:

    > You don't need to. These are only suggestions, the final format needs
    > not matching any item in this list.


    > Not really. See `org-table-export' docstring, last paragraph.


    > You could try (untested):

    >     (defun my-format-timestamps (cell)
    >       (org-quote-csv-field
    >        (replace-regexp-in-string
    >         org-ts-regexp-both
    >         (lambda (m)
    >           (if (not org-display-custom-times) (substring m 1 -1)
    >             (let ((hours? (string-match-p "[0-9]+:[0-9]+" m)))
    >               (format-time-string (funcall (if hours? #'cdr #'car)
    >                                            org-time-stamp-custom-formats)
    >                                   (org-parse-time-string m)))))
    >         cell)))


Thanks, but the outcome of that function for the table

  | <2017-12-19 Tue> | 189.09 € |
  | <2017-12-21 Wed> | 27.86  € |


< 01.01.70 >,189.09 €
< 01.01.70 >,27.86  €


For all timestamp could expanded into the same string and the < > were
left in place.

Uwe 

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Uwe Brauer]

    > Uwe Brauer <oub@mat.ucm.es> writes:

    > You don't need to. These are only suggestions, the final format needs
    > not matching any item in this list.


    > Not really. See `org-table-export' docstring, last paragraph.


    > You could try (untested):

    >     (defun my-format-timestamps (cell)
    >       (org-quote-csv-field
    >        (replace-regexp-in-string
    >         org-ts-regexp-both
    >         (lambda (m)
    >           (if (not org-display-custom-times) (substring m 1 -1)
    >             (let ((hours? (string-match-p "[0-9]+:[0-9]+" m)))
    >               (format-time-string (funcall (if hours? #'cdr #'car)
    >                                            org-time-stamp-custom-formats)
    >                                   (org-parse-time-string m)))))
    >         cell)))


Thanks very much, but the outcome of that function for the table

  | <2017-12-19 Tue> | 189.09 € |
  | <2017-12-21 Wed> | 27.86  € |


< 01.01.70 >,189.09 €
< 01.01.70 >,27.86  €


For all timestamp could expanded into the same string and the < > were
left in place.

Uwe 

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Nicolas Goaziou]

Uwe Brauer <oub@mat.ucm.es> writes:

>     > Uwe Brauer <oub@mat.ucm.es> writes:
>
>     > You don't need to. These are only suggestions, the final format needs
>     > not matching any item in this list.
>
>
>     > Not really. See `org-table-export' docstring, last paragraph.
>
>
>     > You could try (untested):
>
>     >     (defun my-format-timestamps (cell)
>     >       (org-quote-csv-field
>     >        (replace-regexp-in-string
>     >         org-ts-regexp-both
>     >         (lambda (m)
>     >           (if (not org-display-custom-times) (substring m 1 -1)
>     >             (let ((hours? (string-match-p "[0-9]+:[0-9]+" m)))
>     >               (format-time-string (funcall (if hours? #'cdr #'car)
>     >                                            org-time-stamp-custom-formats)
>     >                                   (org-parse-time-string m)))))
>     >         cell)))
>
>
> Thanks very much, but the outcome of that function for the table
>
>   | <2017-12-19 Tue> | 189.09 € |
>   | <2017-12-21 Wed> | 27.86  € |
>
>
> < 01.01.70 >,189.09 €
> < 01.01.70 >,27.86  €
>
>
> For all timestamp could expanded into the same string and the < > were
> left in place.

Ah well, this was a basis to get you started... This one should work

  (defun my-format-timestamps (cell)
    (org-quote-csv-field
     (replace-regexp-in-string
      org-ts-regexp-both
      (lambda (m)
        (format-time-string
         (let ((hours? (string-match-p "[0-9]+:[0-9]+" m)))
           (funcall (if hours? #'cdr #'car) org-time-stamp-custom-formats))
         (apply #'encod-time (save-match-data (org-parse-time-string m)))))
      cell)))

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Uwe Brauer]

    > Uwe Brauer <oub@mat.ucm.es> writes:

    > Ah well, this was a basis to get you started... This one should work

    >   (defun my-format-timestamps (cell)
    >     (org-quote-csv-field
    >      (replace-regexp-in-string
    >       org-ts-regexp-both
    >       (lambda (m)
    >         (format-time-string
    >          (let ((hours? (string-match-p "[0-9]+:[0-9]+" m)))
    >            (funcall (if hours? #'cdr #'car) org-time-stamp-custom-formats))
    >          (apply #'encod-time (save-match-data (org-parse-time-string m)))))
    >       cell)))

It does, works like charm, thank you so much. I suggest to include that
or a similar function into the org-table-export function!!

Re: org table with datestamp convert to csv and then xlsx or ods: problem

[Uwe Brauer]

 
   > Uwe Brauer <oub@mat.ucm.es> writes: 
 
   > Ah well, this was a basis to get you started... This one 
   > should work 
 
   >   (defun my-format-timestamps (cell) 
   >     (org-quote-csv-field 
   >      (replace-regexp-in-string 
   >       org-ts-regexp-both (lambda (m) 
   >         (format-time-string 
   >          (let ((hours? (string-match-p "[0-9]+:[0-9]+" m))) 
   >            (funcall (if hours? #'cdr #'car) 
   >            org-time-stamp-custom-formats)) 
   >          (apply #'encod-time (save-match-data 
   >          (org-parse-time-string m))))) 
   >       cell))) 

There is a small misspelling. Should be encode-time not encod-time.