format/fill of text in a cell in tables

format/fill of text in a cell in tables

[George Michaelson]

Thank you for the continuing support for Org mode. I really appreciate
the work done.

I would love to understand why there is no built-in 'fill/wrap'
function for text in a cell, inside an org mode table.

Given how this is a layout mechanism, and heads to M-x ox-<exportform>
outcomes, it would be useful to recognize the output is often table
edit states which do respect fill/wrap/center

(If this is documented or I missed how it can be done, apologies. My
reading strongly suggests its in extensions, and not really supported
in the model)

-George

Re: format/fill of text in a cell in tables

[Ihor Radchenko]

George Michaelson <ggm@pobox.com> writes:

> Thank you for the continuing support for Org mode. I really appreciate
> the work done.
>
> I would love to understand why there is no built-in 'fill/wrap'
> function for text in a cell, inside an org mode table.
>
> Given how this is a layout mechanism, and heads to M-x ox-<exportform>
> outcomes, it would be useful to recognize the output is often table
> edit states which do respect fill/wrap/center

AFAIK, mostly technical difficulty. Emacs does not natively distinguish
individual table cells, so we would need to emulate cell editing if it
spans over multiple lines.

Patches are welcome!

Best,
Ihor

Re: format/fill of text in a cell in tables

[Tim Cross]

Ihor Radchenko <yantar92@gmail.com> writes:

> George Michaelson <ggm@pobox.com> writes:
>
>> Thank you for the continuing support for Org mode. I really appreciate
>> the work done.
>>
>> I would love to understand why there is no built-in 'fill/wrap'
>> function for text in a cell, inside an org mode table.
>>
>> Given how this is a layout mechanism, and heads to M-x ox-<exportform>
>> outcomes, it would be useful to recognize the output is often table
>> edit states which do respect fill/wrap/center
>
> AFAIK, mostly technical difficulty. Emacs does not natively distinguish
> individual table cells, so we would need to emulate cell editing if it
> spans over multiple lines.
>
> Patches are welcome!
>

I agree. This is actually a much harder problem to solve than it may
appear on the surface. There have been a number of proposals regarding
how to do this over the years, but unfortunately, they have all had
significant drawbacks or limitations. One of which is that formatting
tables with multiple lines in LaTeX is actually a bit tricky (probably
the only exporter where it is relatively straight-forward is HTML). This
means you actually have two broad problems - handling cell wrapping in
the org buffers and handling it cleanly in the different exporters. 

In addition to the problem of defining some mechanism to treat each
table cell as if it was it's own 'minibuffer' (in TeX, you sometimes see
the 'minipage' environment used in this context), you also have to
define some mechanism to describe the relationship between the cells in
order to handle things like centring and justification. screen/page
width etc. To further complicate matters, you also have to consider what
this would mean for org's spreadsheet like capabilities when applied to
tables with cells wrapped over multiple lines. 

If you have lots of cells needing wrapping, the table is perhaps not the
right layout mechanism to use. While it may seem like a convenient way
to present content, often it isn't a great way to consume it. Donald
Knuth wrote a bit about why tables with multiple cell lines were a poor
choice. After years of dealing with project managers who too often use
Excel to record, present and share data, I tend to agree with his views.
I'm also old enough to remember when the table was the 'goto' solution
for managing layout in HTML files and what a mess that became.

While tables are great when you want to show 2-dimensional
relationships, for other situations, alternatives like definition lists,
nested lists and breaking the content up into subsections are often a
good alternative. It could be argued that not supporting table wrapping
is a positive aspect as it makes the author consider alternative layout
approaches which may actually improve readability of the content.

Finally, there is also an accessibility issue with multiple line tables.
For users who rely on assistive technology to consume content,
presenting information in a meaningful manner which is easy to navigate
and can represent larger 'chunks' of information with appropriate
indicators for the relationships between the cells is challenging.

As an aside, I sometimes find it useful when thinking about how to
layout information, how a typical user will consume it. We tend prefer
layouts with infinite length (pages), but with set width. Scrolling
up/down is convenient. Scrolling left/right is not. While larger screens
means we have more width to work with when reading on the screen, this
does not map well to printed pages as that 'width' has not changed -
best we can do is switch to 'landscape' rather than portrait mode. The
other problem with width is in variability - many people have wide
screens on desktops, but narrow screens on mobile devices. We can see
how quickly this becomes complex when we consider all the challenges we
have had with respect to being able to render web content on different
devices with varying screen sizes. Much of the complexity of CSS is
related to column layout and screen sizes. 

Re: format/fill of text in a cell in tables

[tomas]

[-- Attachment #1: Type: text/plain, Size: 3934 bytes --]

On Fri, Dec 17, 2021 at 08:51:59AM +1100, Tim Cross wrote:

[on flowing text whithin table cells]

> I agree. This is actually a much harder problem to solve than it may
> appear on the surface [...]

If you want to get completely dizzy, watch the recurrent threads about
proportional fonts in emacs-user or emacs devel :-)

That's when things start getting interesting.

> If you have lots of cells needing wrapping, the table is perhaps not the
> right layout mechanism to use. While it may seem like a convenient way
> to present content, often it isn't a great way to consume it. Donald
> Knuth wrote a bit about why tables with multiple cell lines were a poor
> choice. After years of dealing with project managers who too often use
> Excel to record, present and share data, I tend to agree with his views.
> I'm also old enough to remember when the table was the 'goto' solution
> for managing layout in HTML files and what a mess that became.

Very interesting points you make. I'd like to add a couple or two ;-)

* static vs dynamic, user vs master

An admirer of Knuth myself, I tend to relativise his position: he's a
book writer in the classical sense. Lots of things happen at "compile
time", you (the reader) get to watch (in awe) the process's results.

Tables have an advantage if your approach is an explorative one, i.e. if
the process is part of the result. I don't think they are as successful
as they are for no reason (SQL, or R's data frames are about tables, after
all, so it's not only Excel). If you want your reader to take part in
the exploration process, a table might just be right.

The cell lines is again in the same pattern: once the layout is fixed,
you can tweak your appearance so you can drop the lines. The result is
astounding, but only if you know your trade damn well. Knuth does. If
things are still in flow, or if you aren't a Grandmaster, perhaps lines
do help [1]. Now sometimes, this "in flow" is part of what you want to
convey, so...

* acculturation & perception

Very interesting point you make about "project managers". This reminds
me that there's not that One Perception Ruleset. People tend to justify
nearly anything with anything (remember those arguing with grey levels
and contrast to prove that serif fonts are more readable? Or was it the
other way around?). To the project managers, tables are probably the
most readable, because they read them all the time. Especially if they
are made with Microsoft (that's the basis of the power of those corps,
after all). Human perception seems so adaptable that it's nearly scary.
So it is all part of a giant feedback loop. Difficult to spot some
bedrock in this mess. Perception is culture is perception.

The point you make about assistive technologies is hugely important. I
haven't much experience with blind people myself, but I'm convinced that
their perception of dimensionality (2D, 2D vs 3D) could be quite
different from that of sighted people. Is a table an advantage or a
disadvantage then? Does it depend on the strategic path they have
chosen? Do some feel better at 3D? 5D? [2]

* WYS ain't WYG

Lastly, Org ain't WYSIWYG (well, duh). But such things as flowed cells
are measuring it up to one, up to a point (although, at some point, I
admit to having yearned for some). A strength is a weakness is a
strength. I think it is the nature of Org to live with such conflicts.
It's an interesting place, where it lives :-)

Cheers

[1] You better go with your camera's defaults to take an everyday photo.
   If you're planning that astounding grainy B&W portrait, you're in for
   some training.
[2] I had once a prof in functional analysis: the way he drew his things
   on the blackboard gave us the impression that he really was /seeing/
   those infinite-dimensional vector spaces he constantly talked about.
   Scary :)

-- 
tomás

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: format/fill of text in a cell in tables

[Tim Cross]

<tomas@tuxteam.de> writes:

> [[PGP Signed Part:Undecided]]
> On Fri, Dec 17, 2021 at 08:51:59AM +1100, Tim Cross wrote:
>
> [on flowing text whithin table cells]
>
>> I agree. This is actually a much harder problem to solve than it may
>> appear on the surface [...]
>
> Tables have an advantage if your approach is an explorative one, i.e. if
> the process is part of the result. I don't think they are as successful
> as they are for no reason (SQL, or R's data frames are about tables, after
> all, so it's not only Excel). If you want your reader to take part in
> the exploration process, a table might just be right.
>

Yes, sometimes tables are extremely useful - especially wrt 2-d
relationships I'm not against the use of tables, but do find their use
as a formatting/layout tool limited. Unfortunately, this is often how I
see them used. Much of my work has been with databases, so conceptually,
I often think in terms of tables, rows and relationships. More often
than not, while tables are good for simple entities, they are poor for
modelling because things often break down into hierarchies and related
sets etc.  

Many years ago, I worked on a system which used an interesting interface
which used 'cubes' to represent database information. At the time, all
of us who worked on the system dreamed of the day when you could have a
hologram display, true 3-d manipulation and linking of data cubes with
the ability to interact and walk around it to cut (slice) the data in
different ways. It was a lot of fun, but at the time computing power was
not quite up to the task. We even used an interesting logic based on
work done by a mathematician called Charles Peirce, who defined a
deductive logic based on graphs where your basic logic operations were
done through the union, intersection and projection of graphs etc. An
interesting approach which seems well suited to modern computer
interfaces (despite the fact he lived from 1839-1914). 

> The point you make about assistive technologies is hugely important. I
> haven't much experience with blind people myself, but I'm convinced that
> their perception of dimensionality (2D, 2D vs 3D) could be quite
> different from that of sighted people. Is a table an advantage or a
> disadvantage then? Does it depend on the strategic path they have
> chosen? Do some feel better at 3D? 5D? [2]
>

I have been legally blind my whole life and for 17 years, I had nothing
but light/dark perception (I could tell when it was day and when it was
night, or if a light was on, but that was about it). I was lucky to
regain some sight about 11 years ago and can now see colour, shapes and
even sometimes recognise people (though much of that is about other
cues, like hair length/style, clothing, deportment etc. I have a large
screen and use a large font, but rely heavily on text-to-speech. I'm
sort of between worlds - enough sight to prevent me really developing
'sightless'skills, but not enough to rely on sight for reading etc. My
braille is comprehension is terrible.

I know a few totally blind mathematicians and their skills are
impressive. Quite a few of them have ended up working in fields relating
to topology. I suspect that not having sight actually helps them in
their mental model of n-dimensions when n > 3. Sighted people seem to
find such dimensional thinking challenging and I suspect it is because
they are more accustomed to a 3-d world. For me, I have only 1 eye, so
no depth perception and a somewhat 'flat' view of the world :-).  

> * WYS ain't WYG
>
> Lastly, Org ain't WYSIWYG (well, duh). But such things as flowed cells
> are measuring it up to one, up to a point (although, at some point, I
> admit to having yearned for some). A strength is a weakness is a
> strength. I think it is the nature of Org to live with such conflicts.
> It's an interesting place, where it lives :-)
>

Yes, I can see why people like WYSWYG when editing. However, my
experience with such systems as more often than not been extremely
frustrating as I seem to end up constantly fighting with the system to
get it looking right rather than focusing on the content. I remember the
joy I had when I discovered Latex and how it allowed me to focus on
content and leave presentation to others who undoubtedly had far better
skills then me. Often the problem with WYSWYG is that it isn't always
true - more like WYSCWYG (what you see is close to what you get).  

Re: format/fill of text in a cell in tables

[tomas]

[-- Attachment #1: Type: text/plain, Size: 2595 bytes --]

On Fri, Dec 17, 2021 at 11:11:47PM +1100, Tim Cross wrote:

[...]

> Yes, sometimes tables are extremely useful - especially wrt 2-d
> relationships I'm not against the use of tables, but do find their use
> as a formatting/layout tool limited [...]

Yes, but at the end, layout is but a thinking device as all the others
are :-)

This reminds me of people advocating "semantic backup" (e.g. use
"emphasis" instead of "italics", until one realises that you just
managed to peel off one layer of the sematic onion. The onion just got
smaller (some literature perhaps might want to play with the ambiguity
of italics?), and if you continue, you end up with no onion at all.

> Many years ago, I worked on a system which used an interesting interface
> which used 'cubes' to represent database information [...]

Sounds a bit awkward, at the same time. I guess the gains to be had from
going from 1D to 2D are significantly higher than those beyond (OTOH,
beyond 5 or 6, things get strange again, with most of the volume of
things getting stuck in thin shells).

> I have been legally blind my whole life and for 17 years [...]
> [...] I'm sort of between worlds - enough sight to prevent me really
> developing 'sightless'skills

Thanks for sharing your experience. That's the only way others can
learn.

> I know a few totally blind mathematicians and their skills are
> impressive. Quite a few of them have ended up working in fields relating
> to topology [...]

This is interesting. I always was wondering how different person's
strategies in this situation have anything in common and how they
vary wildly.

> > * WYS ain't WYG

[...]

> Yes, I can see why people like WYSWYG when editing. However, my
> experience with such systems as more often than not been extremely
> frustrating as I seem to end up constantly fighting with the system to
> get it looking right rather than focusing on the content [...]

There is a lot of potential in being "between the worlds", and Org
tries to exploit exactly that. But that's, I think, also where its
conflicts are.

You can see similar patterns in other fields. Serialization formats
which deliver abstract and concrete syntax in "one package"
(S-expressions, XML (and those others), JSON) tend to be wildly successful,
but tend to run into the same troubles time and again; trying to do
the right thing and separating abstract and concrete syntax (think
ASN.1) tend to end up in niches (OK, ASN.1 is actually successful,
but only because people get paid to do that :-)

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: format/fill of text in a cell in tables

[Juan Manuel Macías]

tomas@tuxteam.de writes:

> This reminds me of people advocating "semantic backup" (e.g. use
> "emphasis" instead of "italics", until one realises that you just
> managed to peel off one layer of the sematic onion. The onion just got
> smaller (some literature perhaps might want to play with the ambiguity
> of italics?), and if you continue, you end up with no onion at all.

There is a lot of confusion between the terms 'emphasis' and
'italics/cursive'. The second term is strictly typographic-calligraphic.
There are entire codices that are wrote in Byzantine cursive. And you
have the Porson typeface, from the Oxford editions of Ancient Greek
texts, which is a cursive, but which is used as a normal font. In an
ancient text there is no notion for 'emphasis': how do we know when
Homer or Sappho wanted to emphasize a phrase? Typography has
historically used italics as a resource for emphasis (not in all
languages, some use the separation of letters to emphasize; there are
also writing systems where the concept of 'italics' or 'cursive' does
not make sense), but it uses the italic also for more varied purposes:
depends on the era, fashion and trends. Consider also the avant-garde
poetry of the early last century, which had a great typographical
dependency, as a sort of game. In addition, there is the maremagnum of
graphic design, which is not strictly typography (although both terms
are also confused), but use the typography for expressive purposes:
advertising, etc.

I remember that a long time ago I use to wrote in a typewriter, and
there was a common convention in typed texts, which consisted of marking
the emphases like this: _emphasis_. WYSIWYG word processors imposed a
form quite unnatural to write, by confusing format and content. And they
force authors to have typographical concerns at the most inopportune
moment, which is the creation of content (as if Hemingway used a
monotype instead of a typewriter). The proof is that hardly any of the
Word users use Word styles or apply them consistently. The normal thing,
with rare exceptions, is to degrade the documents using direct format,
which is the great plague.

I believe that a text whose main purpose is not to produce a visual
impact (advertising, ritual, magic, etc.) but to transmit a thought and
a content, it must have a structure. And then there will be time for
that structure can be translated to other supports. Typography, in its
most basic and utilitarian sense (not visual) is nothing more than a
language to translate that structure, offering the maximum possible
readability, through a series of techniques. And every type of
content, for example a 'table' (in Org terms not typographic terms), can
have many possible typographic translations, even translations that
don't consist of a 'table', in typographical terms. But typography is not
the only possible language to translate a content: think of texts
written to be heard, or texts written for absolutely personal use.
That's why I believe the least unhealthy way to put content in writing
is within a environment as agnostic as possible of the format. In that
environment is where the term 'emphasis' makes sense.

Best regards,

Juan Manuel 

Re: format/fill of text in a cell in tables

[Uwe Brauer]

[-- Attachment #1: Type: text/plain, Size: 1383 bytes --]


> Ihor Radchenko <yantar92@gmail.com> writes:


> I agree. This is actually a much harder problem to solve than it may
> appear on the surface. There have been a number of proposals regarding
> how to do this over the years, but unfortunately, they have all had
> significant drawbacks or limitations. One of which is that formatting
> tables with multiple lines in LaTeX is actually a bit tricky (probably
> the only exporter where it is relatively straight-forward is HTML). This
> means you actually have two broad problems - handling cell wrapping in
> the org buffers and handling it cleanly in the different exporters. 

Actually there is a, non official, odt exporter which allows you, use
lists that are then converted to odt tables, in which the cells a nicely
filled and formatted. It also allows more complex, nested tables.


On the org-mode only side there is list-cruncher that also converts lists
to org tables, so you can, somehow «avoid» the filling issue when you
type, but I have also to add that the resulting cells in the tables are
not really filled.

I think, one attempt could be to use org-edit-special, to edit a cell,
type the text in a temporary buffer, fill it and then return to the table. 

That seems so obvious that I think there might be technical problems,
because otherwise it would have been implemented already.


[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5673 bytes --]

Re: format/fill of text in a cell in tables

[Juan Manuel Macías]

Uwe Brauer writes:

> I think, one attempt could be to use org-edit-special, to edit a cell,
> type the text in a temporary buffer, fill it and then return to the table. 
>
> That seems so obvious that I think there might be technical problems,
> because otherwise it would have been implemented already.

I proposed that here the last time a similar discussion came up, but now
I don't think it's a good idea. I believe the root problem is that the
visual representation of the tables in Org is line oriented. Cells with
a lot of content (paragraphs, for example) should be shrunken, because
the content is a single line; when editing each cell in its special
buffer, that content would have to be converted into paragraphs, and
when saving the edition buffer, convert it back to a single line again.
I tried to write something similar for my personal use, but it is very
tricky. Maybe such an implementation would work better for AUCTeX, since
tables in LaTeX are not line oriented.

About a possible solution for Org regarding the topic of 'complex'
tables, I have ended up giving up. I think it is better to create the
complex tables in raw LaTeX. To export them to HTML you could perhaps
think of a LaTeX block that would return the results compiling with
tex4ht (https://ctan.org/pkg/tex4ht?lang=en) instead of LaTeX. But I
don't know if it would work. The point is that anything other than a
simple and clean table (visually speaking) in Org, is to enter the land
of the tricks.

Best regards,

Juan Manuel