[RFC] Shrink columns dynamically

[RFC] Shrink columns dynamically

[Nicolas Goaziou]

Hello,

Currently, the only way to shrink an Org table column is to use
a so-called width cookie (e.g., "<c10>" or "<10>") so that aligning
table forces the column to fit within the specified number of
characters. When the width is less than the number of characters in the
field, contents is hidden with "=>" string.

Besides, width cookies are interpreted by some export back-ends.

I think this situation is not good for two major reasons:
  - width cookies have two orthogonal uses,
  - hiding a column implies modifying the table.

I'd like to replace the visual effect of width cookies (not
interpretation by export back-ends) with a more dynamic one. To that
effect, I pushed a "hide-table-column" branch in the repository. 

There, the single user-facing function is
`org-table-toggle-column-visibility'. Its docstring is

      Shrink or expand current column in an Org table.

    When optional argument ARG is a string, use it as white space
    separated list of column ranges.  A column range can be one of
    the following patterns:

      N    column N only
      N-M  every column between N and M (both inclusive)
      N-   every column between N (inclusive) and the last column
      -M   every column between the first one and M (inclusive)
      -    every column

    When called with `\\[universal-argument]' prefix, ask for the \
    range specification.

    When called with `\\[universal-argument] \\[universal-argument]' \
    prefix, expand all columns.

It allows to quickly shrink or expand one of more columns in a table,
e.g., "1 3-4 6-". A shrunk column is only 1 character wide, not counting
vertical bars:

  | 123 |    =>   |…|

The character used for display (here "…"), is customizable with
`org-table-shrunk-column-display' variable.

A shrunk column expands whenever an interactive change happens in some
of its fields. Thus, re-calculating a table does not count.

Unlike to width cookies, no column is shrunk upon opening the buffer.
Due to its dynamic state, shrunk columns are not restored across
sessions either. I thought about "fixing" the first point with a new
STARTUP value that would shrink any column with a width cookie, but that
conflicts with the first motivation given earlier.

Feedback welcome. In particular, please test it and report if this is
a viable replacement for width cookies. Also, the
`org-table-toggle-column-visibility' function current misses a binding.
Suggestions welcome.

Regards,

-- 
Nicolas Goaziou                                                0x80A93738

Re: [RFC] Shrink columns dynamically

[Uwe Brauer]

>>> "Nicolas" == Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

   > Hello,
   > I'd like to replace the visual effect of width cookies (not
   > interpretation by export back-ends) with a more dynamic one. To that
   > effect, I pushed a "hide-table-column" branch in the repository. 

This is the patch I applied and tested? If so, I think it is a very good
thing and should be in master.

Uwe Brauer 

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Hello,

Uwe Brauer <oub@mat.ucm.es> writes:

> This is the patch I applied and tested? If so, I think it is a very good
> thing and should be in master.

This is the same patch, along with the complete removal of old column
narrowing using width cookies.

Regards,

-- 
Nicolas Goaziou

Re: [RFC] Shrink columns dynamically

[Uwe Brauer]

>>> "Nicolas" == Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

    > Hello,
    > Uwe Brauer <oub@mat.ucm.es> writes:

    >> This is the patch I applied and tested? If so, I think it is a very good
    >> thing and should be in master.

    > This is the same patch, along with the complete removal of old column
    > narrowing using width cookies.

Removing, oh oops


I just was faced with a table which just contained 3 columns but very
many rows. In second column however was very wide, it contained titled
of journals. So 'cutting' them to half was sufficient to identify them
but allowed me to see the third column. So I say narrowing and hiding
are two useful features. I would not say one can substitute the other.
Hiding is useful if I have very many small columns but narrowing for
very wide columns.


So please do not remove narrowing.

Uwe Brauer 
    > Regards,

Re: [RFC] Shrink columns dynamically

[Nick Dokos]

Uwe Brauer <oub@mat.ucm.es> writes:

>>>> "Nicolas" == Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:
>
>     > Hello,
>     > Uwe Brauer <oub@mat.ucm.es> writes:
>
>     >> This is the patch I applied and tested? If so, I think it is a very good
>     >> thing and should be in master.
>
>     > This is the same patch, along with the complete removal of old column
>     > narrowing using width cookies.
>
> Removing, oh oops
>
>
> I just was faced with a table which just contained 3 columns but very
> many rows. In second column however was very wide, it contained titled
> of journals. So 'cutting' them to half was sufficient to identify them
> but allowed me to see the third column. So I say narrowing and hiding
> are two useful features. I would not say one can substitute the other.
> Hiding is useful if I have very many small columns but narrowing for
> very wide columns.
>
>
> So please do not remove narrowing.
>
> Uwe Brauer 
>

I would agree with that: I haven't tried the patch and column hiding,
but I have used narrowing in the past, in exactly the context that Uwe
describes.

-- 
Nick

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Hello,

Uwe Brauer <oub@mat.ucm.es> writes:

> I just was faced with a table which just contained 3 columns but very
> many rows. In second column however was very wide, it contained titled
> of journals. So 'cutting' them to half was sufficient to identify them
> but allowed me to see the third column. So I say narrowing and hiding
> are two useful features. I would not say one can substitute the other.
> Hiding is useful if I have very many small columns but narrowing for
> very wide columns.
>
>
> So please do not remove narrowing.

There is much overlapping between the two features. I'd rather implement
so sort of narrowing using shrunk columns than have the two features in
the code base.

Regards,

-- 
Nicolas Goaziou

Re: [RFC] Shrink columns dynamically

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 963 bytes --]

On Mon, Jul 10, 2017 at 10:44 AM Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> Hello,
>
> Uwe Brauer <oub@mat.ucm.es> writes:
>
> > This is the patch I applied and tested? If so, I think it is a very good
> > thing and should be in master.
>
> This is the same patch, along with the complete removal of old column
> narrowing using width cookies.
>

The feature works nicely as explained, but please do not remove the width
cookie!

The new feature supports only completely hiding a column. But what we lose
by removal of width cookie is:

- Ability to truncate text in columns to a particular width (like 10) so
that I can at least know what the content of that cell is about. And then
to edit the cell, I would do C-c ` (org-table-edit-field). This is very
useful for cells with verbose texts. That way, I can fit all the columns in
a single screen width.
- Also heavily missed will be retaining the narrowed/hidden state of the
columns.
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 1486 bytes --]

Re: [RFC] Shrink columns dynamically

[Michael Brand]

Hi Nicolas

On Tue, Jul 11, 2017 at 12:11 AM, Kaushal Modi <kaushal.modi@gmail.com> wrote:

> The feature works nicely as explained, but please do not remove the width
> cookie!
>
> The new feature supports only completely hiding a column. But what we lose
> by removal of width cookie is:
>
> - Ability to truncate text in columns to a particular width (like 10) so
> that I can at least know what the content of that cell is about. And then to
> edit the cell, I would do C-c ` (org-table-edit-field). This is very useful
> for cells with verbose texts. That way, I can fit all the columns in a
> single screen width.
> - Also heavily missed will be retaining the narrowed/hidden state of the
> columns.

+1

Also I hope that you can build in complete removal of columns for
export similar to as discussed around here:
http://lists.gnu.org/archive/html/emacs-orgmode/2016-04/msg00672.html

Michael

Re: [RFC] Shrink columns dynamically

[Uwe Brauer]

    > Hello,
    > Uwe Brauer <oub@mat.ucm.es> writes:


    > There is much overlapping between the two features. I'd rather implement
    > so sort of narrowing using shrunk columns than have the two features in
    > the code base.

Well all I am saying is there is the need for two features:

Shrinking (hiding) of many columns
shrinking of one particular wide column by n-char.

If you want to implement the second feature differently, because of
maintain reasons that sound reasonable, but please don't simple remove
the second feature.

    > Regards,

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Hello,

Uwe Brauer <oub@mat.ucm.es> writes:

> If you want to implement the second feature differently, because of
> maintain reasons that sound reasonable, but please don't simple remove
> the second feature.

Then I'll just drop this branch. I'm against having the same (sub-set of
a) feature implemented in two different ways.

I still think width cookies in their current state are wrong since they
really do two different things.

Thank you for the feedback.

Regards,

-- 
Nicolas Goaziou

Re: [RFC] Shrink columns dynamically

[Uwe Brauer]

>>> "Nicolas" == Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

    > Hello,
    > Uwe Brauer <oub@mat.ucm.es> writes:

    >> If you want to implement the second feature differently, because of
    >> maintain reasons that sound reasonable, but please don't simple remove
    >> the second feature.

    > Then I'll just drop this branch. I'm against having the same (sub-set of
    > a) feature implemented in two different ways.

That is a real pity! So in master we have the feature of narrowing one
column to a width we want. That is useful in scenario 1.

Now in the branch there is another *very nice* feature, which allows you
to hide many columns on the fly. That is useful in scenario 2.

So you are saying we cannot have both.

I still have your patch and use it often, I really don't want to miss
it, since it enhance the features of the table very much. It is a pity
that you won't conserve that. I hope that patch can survive in the
future.

    > I still think width cookies in their current state are wrong since
    > they really do two different things.

But couldn't the feature of shrink one particular wide column to a
certain width  be implemented using your new implementation?

Thank you very much for you work and I still hope this can somehow be
saved and merged into master.

Regards

Uwe 

Re: [RFC] Shrink columns dynamically

[Uwe Brauer]

>>> "Nicolas" == Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

   > Hello,
   > Uwe Brauer <oub@mat.ucm.es> writes:

   >> If you want to implement the second feature differently, because of
   >> maintain reasons that sound reasonable, but please don't simple remove
   >> the second feature.

   > Then I'll just drop this branch. I'm against having the same (sub-set of
   > a) feature implemented in two different ways.

Sub-set? May be I missed the point and I have to apologize. But I tried
to play around with org-table-toggle-column-visibility and I don't see
how can I shrink a column say to the width of 10 chars as I can with
narrowing via:

|      | <10>                         |
| this | very long long long lon long |

Deleting org table columns during export (Was: [RFC] Shrink columns dynamically)

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 549 bytes --]

Hello,

On Tue, Jul 11, 2017, 2:19 AM Michael Brand <michael.ch.brand@gmail.com>
wrote:

>
> Also I hope that you can build in complete removal of columns for
> export similar to as discussed around here:
> http://lists.gnu.org/archive/html/emacs-orgmode/2016-04/msg00672.html


I am also interested in this feature.

- I liked Nicolas' proposal to have "<#>" as markers for column deletions
instead of "/".
- If I understand correctly, Michael, you need to send a git formatted
patch of your proposal with documentation included?
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 1149 bytes --]

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Uwe Brauer <oub@mat.ucm.es> writes:

> So you are saying we cannot have both.

I'm not. I'm opposed to have overlapping features with two different
implementations, that's all.

> But couldn't the feature of shrink one particular wide column to a
> certain width  be implemented using your new implementation?

I already made that offer.

However, the narrowing (as opposed to shrinking) feature is yet to be
defined. In particular, I think width cookies should not be used, i.e.,
you shouldn't have to modify _data_ to alter the _view_.

It is possible, however, to, e.g., reduce by half the current column, or
to a pre-defined width.

Regards,

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Hello,

Michael Brand <michael.ch.brand@gmail.com> writes:

> Also I hope that you can build in complete removal of columns for
> export similar to as discussed around here:
> http://lists.gnu.org/archive/html/emacs-orgmode/2016-04/msg00672.html

This is orthogonal to the issue at hand.

Besides, was I supposed to implement it? I cannot access the page right
now, but IIRC, you suggested the idea, with a proof of concept. Then, we
eventually ended up on some specifications. I thought you would resume
the work.

Regards,

-- 
Nicolas Goaziou

Re: [RFC] Shrink columns dynamically

[Uwe Brauer]

>>> "Nicolas" == Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

   > Uwe Brauer <oub@mat.ucm.es> writes:
   >> So you are saying we cannot have both.

   > I'm not. I'm opposed to have overlapping features with two different
   > implementations, that's all.

   >> But couldn't the feature of shrink one particular wide column to a
   >> certain width  be implemented using your new implementation?

   > I already made that offer.

Then I really misunderstood you, and apologize. I find the new
implementation very useful and useable and I am grateful that you
implemented it.


   > However, the narrowing (as opposed to shrinking) feature is yet to be
   > defined. In particular, I think width cookies should not be used, i.e.,
   > you shouldn't have to modify _data_ to alter the _view_.

I am really not a fan of that current implementation, but it is the only
one we have.

   > It is possible, however, to, e.g., reduce by half the current column, or
   > to a pre-defined width.

That would be great indeed. So please don't drop the branch but instead,
if you have time, try that and I would be more than happy to test it and
then hopefully narrowing based on cookies could be dropped.

Regards

Uwe 

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Uwe Brauer <oub@mat.ucm.es> writes:

> That would be great indeed. So please don't drop the branch but instead,
> if you have time, try that and I would be more than happy to test it and
> then hopefully narrowing based on cookies could be dropped.

I will as soon as I have an idea about the "that" I have to try.

I don't use column narrowing; I don't know what other users expect from
it either.

Are there some rules to decide what would be an acceptable narrowing
(e.g., narrow columns larger than 10 characters by half or to 20
characters, whichever is the smaller) or do users really need to decide
piece-wise the number of characters needed? If the latter, would
a numeric argument for the narrowing command be sufficient? Note that in
this case, it may not be possible to narrow multiple columns at a time.

Also, do we need to commands for that, or would cycling between
expanded/narrowed by some factor (see above)/shrunk states be
sufficient? What about the command line to control multiple columns?

IOW, let's discuss about specifications.

Regards,

Re: [RFC] Shrink columns dynamically

[Michael Brand]

Hi Nicolas

On Tue, Jul 11, 2017 at 1:47 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

> Michael Brand <michael.ch.brand@gmail.com> writes:

>> Also I hope that you can build in complete removal of columns for
>> export similar to as discussed around here:
>> http://lists.gnu.org/archive/html/emacs-orgmode/2016-04/msg00672.html

> Besides, was I supposed to implement it?

No.

> I cannot access the page right
> now, but IIRC, you suggested the idea, with a proof of concept. Then, we
> eventually ended up on some specifications.

Yes. To summarize the old thread, this is what I have since then in my
config file to use with | <#> |:

  (add-hook 'org-export-before-processing-hook
            #'f-ox-filter-table-column-del)
  (defun f-ox-filter-table-column-del (back-end)
    (while (re-search-forward "^[ \t]*|\\(.*|\\)? *\\(<#>\\) *|" nil t)
      (goto-char (match-beginning 2))
      (org-table-delete-column)
      (beginning-of-line)))

> I thought you would resume the work.

On my list it is still in someday/maybe, not likely to change. The
"problem" is that the above code is already enough.

Quoting from the old thread:

On Tue, Apr 26, 2016 at 10:56 PM, Michael Brand
<michael.ch.brand@gmail.com> wrote:

> On Tue, Apr 26, 2016 at 9:14 PM, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:

>> Does anyone feel like doing that?

> Let me see. If anyone wants to beat me please go ahead.

Michael

Re: Deleting org table columns during export (Was: [RFC] Shrink columns dynamically)

[Michael Brand]

Hi Kaushal

On Tue, Jul 11, 2017 at 1:18 PM, Kaushal Modi <kaushal.modi@gmail.com> wrote:

> On Tue, Jul 11, 2017, 2:19 AM Michael Brand <michael.ch.brand@gmail.com>
> wrote:

>> Also I hope that you can build in complete removal of columns for
>> export similar to as discussed around here:
>> http://lists.gnu.org/archive/html/emacs-orgmode/2016-04/msg00672.html

> - If I understand correctly, Michael, you need to send a git
>   formatted patch of your proposal with documentation included?

No, I just hacked (summarized from the old thread)

  (add-hook 'org-export-before-processing-hook
            #'f-ox-filter-table-column-del)
  (defun f-ox-filter-table-column-del (back-end)
    (while (re-search-forward "^[ \t]*|\\(.*|\\)? *\\(<#>\\) *|" nil t)
      (goto-char (match-beginning 2))
      (org-table-delete-column)
      (beginning-of-line)))

in my config file to use with | <#> |. For all the tasks discussed in
the old thread I did not more than discuss. From my side there exists
nothing that can be made ready to be committed.

Michael

Re: [RFC] Shrink columns dynamically

[Kaushal Modi]

[-- Attachment #1.1: Type: text/plain, Size: 2238 bytes --]

On Tue, Jul 11, 2017 at 8:26 AM Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

>
> I don't use column narrowing; I don't know what other users expect from
> it either.
>

I use column narrowing when I need to fit all columns of an org table in a
screen width (roughly 100 chars wide).

If I have a column called "Description" that can have 100's of characters
in a cell, I'd like to not push the other columns to the right outside the
screen.

- Having line truncation on hides the columns that go beyond the screen
width
- Disabling line truncation makes org tables difficult to read
- So the column width cookie is the best thing we have at the moment

[image: image.png]
The latter table without truncation enabled:
[image: image.png]

Are there some rules to decide what would be an acceptable narrowing
> (e.g., narrow columns larger than 10 characters by half or to 20
> characters, whichever is the smaller) or do users really need to decide
> piece-wise the number of characters needed?


In my use case above, I would do the narrowing just enough to have the
tables fit in my usual window widths.


> If the latter, would
> a numeric argument for the narrowing command be sufficient?


If the column widths do not get saved, it would get tedious to repeat those
narrowing steps each time the same Org file is opened.

Note that in
> this case, it may not be possible to narrow multiple columns at a time.
>

Narrowing multiple columns would be a necessity. That, plus doing so
automatically when an Org file is opened. See above screenshots to see my
typical use of column width cookies.


> Also, do we need to commands for that, or would cycling between
> expanded/narrowed by some factor (see above)/shrunk states be
> sufficient?


I typically just set it and forget it. If I need to remove the widening, I
comment out the "#+STARTUP: align" line and do revert-buffer.


> IOW, let's discuss about specifications.
>

1. Need to save the column narrowed state somehow individually for each
column, specific to a table in a document.
2. Alternative: Look at the window width and calculate the factor by which
all columns should be narrowed so that the whole table fits the window
(Sounds very complicated).
-- 

Kaushal Modi

[-- Attachment #1.2: Type: text/html, Size: 3726 bytes --]

[-- Attachment #2: image.png --]
[-- Type: image/png, Size: 41611 bytes --]

[-- Attachment #3: image.png --]
[-- Type: image/png, Size: 29641 bytes --]

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Hello,

Kaushal Modi <kaushal.modi@gmail.com> writes:

> 1. Need to save the column narrowed state somehow individually for each
> column, specific to a table in a document.

This is not possible, and is exactly what annoys me in the current
implementation.

Moreover, I'm pretty sure you don't review every table in your document
at the second it is opened. So there is no need to store the columns
narrowing, as long as it is easy enough to narrow a table at point when
you need to look at it.

> 2. Alternative: Look at the window width and calculate the factor by which
> all columns should be narrowed so that the whole table fits the window
> (Sounds very complicated).

There are multiple solutions to this problem. None of the is optimal,
because "optimal" depends on each user.

I cannot implement any of you suggestions.

Regards,

-- 
Nicolas Goaziou

Re: [RFC] Shrink columns dynamically

[Kaushal Modi]

[-- Attachment #1: Type: text/plain, Size: 1649 bytes --]

On Tue, Jul 11, 2017 at 3:09 PM Nicolas Goaziou <mail@nicolasgoaziou.fr>
wrote:

> Hello,
>
> Kaushal Modi <kaushal.modi@gmail.com> writes:
>
> > 1. Need to save the column narrowed state somehow individually for each
> > column, specific to a table in a document.
>
> This is not possible, and is exactly what annoys me in the current
> implementation.
>

I didn't understand the root reason for this change. If one doesn't like
using the column width cookies, then they can choose to not use those.


> Moreover, I'm pretty sure you don't review every table in your document
> at the second it is opened.


Even if I don't review all the tables in the document, presentation is also
what matters. I'd like to see neatly aligned tables that fit within the
screen than having those clipped or badly wrapped. It's the same reason why
one would choose to auto-fill in Org documents vs not; because functionally
that doesn't matter too.


> So there is no need to store the columns
> narrowing, as long as it is easy enough to narrow a table at point when
> you need to look at it.
>

I can only request to please retain this feature; for the sake of people
who like it.

> 2. Alternative: Look at the window width and calculate the factor by which
> > all columns should be narrowed so that the whole table fits the window
> > (Sounds very complicated).
>
> There are multiple solutions to this problem. None of the is optimal,
> because "optimal" depends on each user.
>

That's understood.

I cannot implement any of you suggestions.


That's understood too. I am just voicing an opinion to retain the column
width cookie feature.
-- 

Kaushal Modi

[-- Attachment #2: Type: text/html, Size: 2811 bytes --]

Re: [RFC] Shrink columns dynamically

[Uwe Brauer]

    > Hello,
    > Kaushal Modi <kaushal.modi@gmail.com> writes:


    > This is not possible, and is exactly what annoys me in the current
    > implementation.

    > Moreover, I'm pretty sure you don't review every table in your
    > document at the second it is opened. So there is no need to store
    > the columns narrowing, as long as it is easy enough to narrow a
    > table at point when you need to look at it.

I tend to agree saving narrowing, I find rather annoying. However narrow
a column to a certain width dynamically I find useful.

What's about the following idea: can't we have the current configuration
option of narrowing

|      | <20>           |
| this | very very long |

and then executing org-table-toggle-column-visibility (or maybe a new
function) would display


|      | <20>       |
| this | very...... |


But not save it in the buffer?

Re: [RFC] Shrink columns dynamically

[Colin Baxter]

Hello,

I have come to this discussion late, but I totally agree with Kaushal,
whose use of tables almost mirrors my own. 

    Kaushal> I didn't understand the root reason for this change. If one
    Kaushal> doesn't like using the column width cookies, then they can
    Kaushal> choose to not use those.

Agreed!

      >> Moreover, I'm pretty sure you don't review every table in
      >> your document at the second it is opened.

I do review every table immediately the org file is opened because this
is my working document. The file only contains a single table of 10
columns and hundreds of rows.

    Kaushal> I can only request to please retain this feature; for the
    Kaushal> sake of people who like it.

Agreed!

There may be other solutions, but I wouldn't get paid searching for them.

Column cookies work - and they work well for me - so why on earth remove
something that is actually *useful*?

Sincerely,

Colin.

-- 
--
Colin Baxter
m43cap@yandex.com
GnuPG fingerprint: 68A8 799C 0230 16E7 BF68  2A27 BBFA 2492 91F5 41C8

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Hello,

Colin Baxter <m43cap@yandex.com> writes:

> Column cookies work - and they work well for me - so why on earth remove
> something that is actually *useful*?

Really, I explained already the motivation, right from my initial post.
Anyway, let me re-iterate and expound it a bit.

The first thing to know is I'm *no longer* wanting to remove the
feature, as I initially suggested. As explained before, I understood
narrowing was an important feature, so I'm offering to rewrite it. Why?
Because I'm trying to introduce a new feature which is close to this
one, and I would like both features to converge under the same
implementation.

Besides, columns cookies may work for you, but, as pointed out, they are
limited:

- Setting a width cookie also changes how the table is exported (e.g.,
  in ASCII export). However I may want to narrow view of the table and,
  yet, export it to its full extent.

- Setting a width cookie hard-codes how the column is displayed. I may
  want to completely hide the column temporarily, or expand it without
  affecting other narrowed columns.

- Setting a width cookie segregates other columns. I can only narrow
  columns with a width cookie. I may want to temporarily hide another
  column without modifying my table.

The conclusion is that columns narrowing should be independent from
width cookies. More specifically, there is nothing wrong in narrowing
obeying to a width cookie, but it should also be able to ignore it. So
here is why "on earth" I'm suggesting to think about it and, maybe,
implement something more general, possibly more useful, too.

However, rewriting it implies some changes in the current behaviour.
This is why I'm trying to discuss with actual users of width cookies
and, with their help, find a solution that would satisfy everyone. 

I may be trying to square the circle. I don't know yet. Note that
however, so far, most answers are somewhat like "width cookies are
exactly what we want, don't remove them", which is missing the point.
Also, please don't focus on how tables should look like upon opening an
Org document. This is really putting us off-track. It can be postponed.

The real question for now is: how can we alter columns display when at
a table? E.g.,

- Do we need two commands, one for narrowing (to a given number of
  characters) and one for shrinking (to one character only)? Or would
  a command toggling between the three states be sufficient?

- Is there some rule of thumb to narrow a column when no width cookie is
  supplied or should we consider this kind of columns has only two
  states, shrunk and expanded?

- Supposing we focus on a single, cycling, command, how should it behave
  when called on multiple columns at a time? Since some columns may have
  two states and other ones three, it may end up being confusing for the
  user.

Food for thought.

Regards,

-- 
Nicolas Goaziou

Re: [RFC] Shrink columns dynamically

[Colin Baxter]

Dear Nicolas,

I apologise for not understanding fully what you propose. I use org-mode
extensively, so any possibility of "change" tends to make me anxious. 

    Nicolas> Besides, columns cookies may work for you, but, as pointed
    Nicolas> out, they are limited:

    Nicolas> - Setting a width cookie also changes how the table is
    Nicolas> exported (e.g., in ASCII export). However I may want to
    Nicolas> narrow view of the table and, yet, export it to its full
    Nicolas> extent.

    Nicolas> - Setting a width cookie hard-codes how the column is
    Nicolas> displayed. I may want to completely hide the column
    Nicolas> temporarily, or expand it without affecting other narrowed
    Nicolas> columns.

    Nicolas> - Setting a width cookie segregates other columns. I can
    Nicolas> only narrow columns with a width cookie. I may want to
    Nicolas> temporarily hide another column without modifying my table.

I appreciate these points and indeed I regularly face export
problems. However, I just manually remove the cookies if they become an
issue. I accept other users may wish to work differently.

    Nicolas> The real question for now is: how can we alter columns
    Nicolas> display when at a table? E.g.,

    Nicolas> - Do we need two commands, one for narrowing (to a given
    Nicolas> number of characters) and one for shrinking (to one
    Nicolas> character only)? Or would a command toggling between the
    Nicolas> three states be sufficient?

    Nicolas> - Is there some rule of thumb to narrow a column when no
    Nicolas> width cookie is supplied or should we consider this kind of
    Nicolas> columns has only two states, shrunk and expanded?

In my own case, the values of the column widths are not static but vary 
from file-to-file. I don't have any rule of thumb, except, from my own
experience, I have never found the need to shrink any column to a single
character.

    Nicolas> - Supposing we focus on a single, cycling, command, how
    Nicolas> should it behave when called on multiple columns at a time?
    Nicolas> Since some columns may have two states and other ones
    Nicolas> three, it may end up being confusing for the user.

To me, that's the beauty of my present arrangement: it's simple,
intuitive and controllable. I am sure that my requirements are more
modest than those of other users.

    Nicolas> Food for thought.

Indeed.


Best wishes,

Colin.

-- 
--
Colin Baxter
m43cap@yandex.com
GnuPG fingerprint: 68A8 799C 0230 16E7 BF68  2A27 BBFA 2492 91F5 41C8

Re: [RFC] Shrink columns dynamically

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 1467 bytes --]

Hi Nicolas,

I agree completely with your stated objective, specifically separating
viewing from content and/or export.  However, I am not sure this is
achievable without losing some quite appealing current functionality.

I have two use cases which currently are managed with the width cookies
and I can see that they probably should be managed differently.  The
cases are:

1. a table with wide columns that is used to collect information
   (publications with authors, title, journal, etc. is an example) and
   which I use mostly by looking at it.  That is, I open the document
   and scan down the table or page down or whatever.  I want this table
   displayed initially with columns (e.g. title) narrowed to a specified
   width.

2. a table with many columns, too many of the screen width even if each
   column may only be a few characters wide, but one which I intend to
   be manipulating extensively.  In this case, I may want to
   shrink/narrow/hide columns to make it easier to edit and do
   calculations.

At present, I use width cookies for both.  They work well for use case
1; much less well for use case 2.  For me, these use cases are
orthogonal.  I would be happy with two different approaches or the same
approach so long as the latter would retain the ability to save the view
(for use case 1).

As always, I look forward to whatever you decide to implement!

thanks,
eric

-- 
: Eric S Fraga via Emacs 26.0.50, Org release_9.0.9-573-g09e612

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: [RFC] Shrink columns dynamically

[Rick Frankel]

So I haven't posted in a while, but i use table a lot, so here's my $0.02.

Personally, my use of width cookies has been mostly for visual display of
columns are very long and will wrap in an html or latex table export, but
would force the display too wide. My ideal default would be to have a setting
which would shrink tables a reasonable (fill-column?, window-width?) visual
display, with perhaps a min-column-width option and trying to balance the
width of each column base on its un-narrowed total width, perhaps trying to
keep narrow columns as-is in the process. For me this would probably align
cognitively with how much info is needed to grok the cell contents. As to your
behavior questions:

On Wed, Jul 12, 2017 at 12:17:26PM +0200, Nicolas Goaziou wrote:


> - Do we need two commands, one for narrowing (to a given number of
>   characters) and one for shrinking (to one character only)? Or would
>   a command toggling between the three states be sufficient?

I like a single cycling command.

> - Is there some rule of thumb to narrow a column when no width cookie is
>   supplied or should we consider this kind of columns has only two
>   states, shrunk and expanded?

So i think you are saying that a column w/ a width cookie would have three
states and one without only two? I think this in some ways negates the appeal
of separating narrowing from the cookies. Also, my suggested default narrowing
approach above would probably remove the need for cookies in many if not most
cases.

> - Supposing we focus on a single, cycling, command, how should it behave
>   when called on multiple columns at a time? Since some columns may have
>   two states and other ones three, it may end up being confusing for the
>   user.

Wouldn't a min-column-width setting to use when there is no cookie solve the
the bi- vs tri-state problem (every column has three states)

rick

P.S. and (somewhat) off topic: The doc string for
`org-ascii-table-use-ascii-art' is out of date. `ascii-art-to-unicode.el' is
now in elpa and the url no longer works.

Re: [RFC] Shrink columns dynamically

[B.V. Raghav]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Hi Nicolas,
>
> I agree completely with your stated objective, specifically separating
> viewing from content and/or export.  However, I am not sure this is
> achievable without losing some quite appealing current functionality.
>
If I understant correctly, `<cNN>' defines a persistent view and export
state, but is written as data, into the table. And simply put, it does
not fit well with the aforementioned design objective?

Can I request/suggest a new feature `#+TABLE_PROPERTIES: ' analogous to
`#+TBLFM: ' that can save the state for those of us who want to, and the
others may enjoy the volatileness[?]

Pardon my intrusion!
r
-- 
(B.V. Raghav)

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Hello,

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> I have two use cases which currently are managed with the width cookies
> and I can see that they probably should be managed differently.  The
> cases are:
>
> 1. a table with wide columns that is used to collect information
>    (publications with authors, title, journal, etc. is an example) and
>    which I use mostly by looking at it.  That is, I open the document
>    and scan down the table or page down or whatever.  I want this table
>    displayed initially with columns (e.g. title) narrowed to a specified
>    width.
>
> 2. a table with many columns, too many of the screen width even if each
>    column may only be a few characters wide, but one which I intend to
>    be manipulating extensively.  In this case, I may want to
>    shrink/narrow/hide columns to make it easier to edit and do
>    calculations.
>
> At present, I use width cookies for both.  They work well for use case
> 1; much less well for use case 2.  For me, these use cases are
> orthogonal.  I would be happy with two different approaches or the same
> approach so long as the latter would retain the ability to save the view
> (for use case 1).

I pushed another take on the problem in "hide-table-column" branch.
Rebasing is required.

There, two commands are defined

  - `org-table-toggle-column-width'

     Shrink or expand column at point. More specifically, if the column
     contains a width cookie, the column is shrunk according to it.
     Otherwise, it is shrunk to 1 character. Here is its docstring:

        Shrink or expand current column in an Org table.

        If a width cookie specifies a width W for the column, the first
        W visible characters are displayed.  Otherwise, the column is
        shrunk to a single character.

        When optional argument ARG is a string, use it as white space
        separated list of column ranges.  A column range can be one of
        the following patterns:

          N    column N only
          N-M  every column between N and M (both inclusive)
          N-   every column between N (inclusive) and the last column
          -M   every column between the first one and M (inclusive)
          -    every column

        When called with `C-u]' prefix, ask for the range specification.

        When called with `C-u C-u' prefix, expand all columns.

  - `org-table-shrink'

     Shrink all columns with a width cookie. Other columns are expanded.
     This is morally equivalent to the current behaviour, although it is
     now decoupled from aligning.

Moreover, I added two new STARTUP keywords: "shrink" and "noshrink",
which allow to apply aforementioned `org-table-shrink' command on all
tables upon opening a document. Not that "align" no longer toggle column
width.

Feedback welcome.

Regards,

-- 
Nicolas Goaziou

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Hello,

bvraghav@iitk.ac.in (B.V. Raghav) writes:

> Can I request/suggest a new feature `#+TABLE_PROPERTIES: ' analogous to
> `#+TBLFM: ' that can save the state for those of us who want to, and the
> others may enjoy the volatileness[?]

Thank you for your suggestion. 

However, I'm looking for something simpler, that doesn't involve
introducing new keywords. See my new implementation explained in the
thread.

Regards,

-- 
Nicolas Goaziou

Re: [RFC] Shrink columns dynamically

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 200 bytes --]

Thanks Nicolas.  Sounds very nice, matching at least my use cases quite
well.  I'll give this a try soon and will get back to you.

-- 
: Eric S Fraga via Emacs 26.0.50, Org release_9.0.9-573-g09e612

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 194 bytes --]

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Hello,

Rick Frankel <rick@rickster.com> writes:

> Personally, my use of width cookies has been mostly for visual display of
> columns are very long and will wrap in an html or latex table export, but
> would force the display too wide. My ideal default would be to have a setting
> which would shrink tables a reasonable (fill-column?, window-width?) visual
> display, with perhaps a min-column-width option and trying to balance the
> width of each column base on its un-narrowed total width, perhaps trying to
> keep narrow columns as-is in the process. For me this would probably align
> cognitively with how much info is needed to grok the cell contents.

It would be a fun optimization problem to solve, but I doubt it would be
much useful (e.g., pathological cases getting in the way...).

> I like a single cycling command.

[...]

> Wouldn't a min-column-width setting to use when there is no cookie solve the
> the bi- vs tri-state problem (every column has three states)

For the sake of simplicity, I opted for two states cycling in the latest
proof of concept. It means that columns with a width cookie cannot be
narrowed to 1 character, but I assume it is an acceptable restriction
since the width cookie is probably there for a reason.

> P.S. and (somewhat) off topic: The doc string for
> `org-ascii-table-use-ascii-art' is out of date. `ascii-art-to-unicode.el' is
> now in elpa and the url no longer works.

Fixed. Thank you.

Regards,

-- 
Nicolas Goaziou

Re: [RFC] Shrink columns dynamically

[Adam Porter]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Moreover, I added two new STARTUP keywords: "shrink" and "noshrink",
> which allow to apply aforementioned `org-table-shrink' command on all
> tables upon opening a document. Not that "align" no longer toggle column
> width.

Hi Nicolas,

I'm late to this conversation, but thank you for keeping the width
cookies.  I have a large Org file with some large tables with wide
columns, and being able to restrict the width of one of the columns is
essential to keeping others on the screen.

My only feedback here is that maybe the startup keywords would be more
descriptive if they were something like "table-shrink" or
"column-shrink".  I could imagine some new Org users conflating "folded"
and "shrink" until they have used a more advanced feature like table
column width cookies.  :)

Thanks,
Adam

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Hello,

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Thanks Nicolas.  Sounds very nice, matching at least my use cases quite
> well.  I'll give this a try soon and will get back to you.

For the record, I updated the "hide-table-column" branch again.

`org-table-toggle-column-width' is now bound to "C-c TAB". I removed
"noshrink" STARTUP keyword, which wasn't useful. I also introduced an
honest test coverage.

Still no documentation, though.

Feedback welcome.

Regards,

-- 
Nicolas Goaziou

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Hello,

Adam Porter <adam@alphapapa.net> writes:

> My only feedback here is that maybe the startup keywords would be more
> descriptive if they were something like "table-shrink" or
> "column-shrink".  I could imagine some new Org users conflating "folded"
> and "shrink" until they have used a more advanced feature like table
> column width cookies.  :)

I understand, but tables already use "align" and "noalign". I just
followed the trend.


Regards,

-- 
Nicolas Goaziou

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Completing myself,

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> `org-table-toggle-column-width' is now bound to "C-c TAB". I removed
> "noshrink" STARTUP keyword, which wasn't useful. I also introduced an
> honest test coverage.
>
> Still no documentation, though.

I made some final changes to "hide-table-column" branch:

  - Now C-u C-c TAB shrink columns with a width cookie and expands the
    others;

  - When point is before the first column or after the last one, C-c TAB
    asks for columns ranges;

  - I added documentation.

If there is no objection or major bug (including usability ones), I will
merge this branch into master in a couple of days.

Regards,

Re: [RFC] Shrink columns dynamically

[Nicolas Goaziou]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> I made some final changes to "hide-table-column" branch:
>
>   - Now C-u C-c TAB shrink columns with a width cookie and expands the
>     others;
>
>   - When point is before the first column or after the last one, C-c TAB
>     asks for columns ranges;
>
>   - I added documentation.
>
> If there is no objection or major bug (including usability ones), I will
> merge this branch into master in a couple of days.

Merged.

Regards,