* bug#45926: Tabulated-list-mode wasteful properties
@ 2021-01-17 7:32 Boruch Baum
2021-01-17 13:40 ` Basil L. Contovounesios
0 siblings, 1 reply; 13+ messages in thread
From: Boruch Baum @ 2021-01-17 7:32 UTC (permalink / raw)
To: 45926
Tabulated lists can grow quite large. In fact, the most common use for
the mode is the *Packages* buffer listing, which for me today clocks in
at 5211 lines * five columns. In the case of the *Packages* buffer, Performing
(buffer-substring (point) (line-end-position))
on a sampling of lines yields 'PP eval' buffers of length ~500 - ~700
characters.
Most of this looks to me to be unnecessary duplicate data. In using the
mode for my own purpose, I see that the mode is putting all data of all
columns in an entry (row) in each and every single column, ie the size
grows exponentially by number of columns (in my personal case, ten
columns).
This isn't breaking anything, but it looks like bad design unnecessarily
hogging memory and filling buffers with cruft.
--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#45926: Tabulated-list-mode wasteful properties
2021-01-17 7:32 bug#45926: Tabulated-list-mode wasteful properties Boruch Baum
@ 2021-01-17 13:40 ` Basil L. Contovounesios
2021-01-17 15:11 ` Boruch Baum
0 siblings, 1 reply; 13+ messages in thread
From: Basil L. Contovounesios @ 2021-01-17 13:40 UTC (permalink / raw)
To: Boruch Baum; +Cc: 45926
[-- Attachment #1: Type: text/plain, Size: 1351 bytes --]
Boruch Baum <boruch_baum@gmx.com> writes:
> Tabulated lists can grow quite large. In fact, the most common use for
> the mode is the *Packages* buffer listing, which for me today clocks in
> at 5211 lines * five columns. In the case of the *Packages* buffer, Performing
>
> (buffer-substring (point) (line-end-position))
>
> on a sampling of lines yields 'PP eval' buffers of length ~500 - ~700
> characters.
>
> Most of this looks to me to be unnecessary duplicate data. In using the
> mode for my own purpose, I see that the mode is putting all data of all
> columns in an entry (row) in each and every single column, ie the size
> grows exponentially by number of columns (in my personal case, ten
> columns).
>
> This isn't breaking anything, but it looks like bad design unnecessarily
> hogging memory and filling buffers with cruft.
Are you sure the data is actually duplicated and doesn't share
structure? I evaluated that expression over the xref package, and got
the attached string. Nothing looks obviously duplicated, except for the
shared #N= and #N# structures. Also, some of the repetition of shared
structures is probably just due to how string intervals are
represented/printed.
Not that any of this is necessarily optimal, of course, but it's not
immediately alarming to me, either, unless I'm missing something.
--
Basil
[-- Attachment #2: pp-eval.el --]
[-- Type: application/emacs-lisp, Size: 3635 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#45926: Tabulated-list-mode wasteful properties
2021-01-17 13:40 ` Basil L. Contovounesios
@ 2021-01-17 15:11 ` Boruch Baum
2021-01-19 5:24 ` Lars Ingebrigtsen
0 siblings, 1 reply; 13+ messages in thread
From: Boruch Baum @ 2021-01-17 15:11 UTC (permalink / raw)
To: Basil L. Contovounesios; +Cc: 45926
On 2021-01-17 13:40, Basil L. Contovounesios wrote:
> Boruch Baum <boruch_baum@gmx.com> writes:
>
> Are you sure the data is actually duplicated and doesn't share
> structure?
Until you wrote, I definitely was. I think I still am, but I'm willing
to look again. It's easier for me since I just recently wrote a
ten-column tabulated list mode application. In my case I do see it
clearly because the text of each row in my tabulated list isn't
disturbed/confused by any text properties, and I do see an exact
duplication of each entire row entry in the property for each column of
that property.
I expect to publish the software within a day or two, and plan to
announce it on the emacs-devel and possibly other lists, at which time
you'll be able to see the internals for yourself. The package is planned
to be named 'crossword.el' and it brings to emacs the ability to
download and play crossword puzzles.
--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#45926: Tabulated-list-mode wasteful properties
2021-01-17 15:11 ` Boruch Baum
@ 2021-01-19 5:24 ` Lars Ingebrigtsen
2021-03-12 0:52 ` Stefan Kangas
0 siblings, 1 reply; 13+ messages in thread
From: Lars Ingebrigtsen @ 2021-01-19 5:24 UTC (permalink / raw)
To: Boruch Baum; +Cc: Basil L. Contovounesios, 45926
Boruch Baum <boruch_baum@gmx.com> writes:
> Until you wrote, I definitely was. I think I still am, but I'm willing
> to look again.
Like Basil, I looked at a tabulated list line, and I didn't see any
excessive duplication -- tabulated list mode puts the data on the entire
line, but it's the same data structure, so it's all shared data, I
think?
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#45926: Tabulated-list-mode wasteful properties
2021-01-19 5:24 ` Lars Ingebrigtsen
@ 2021-03-12 0:52 ` Stefan Kangas
2021-03-12 2:57 ` Boruch Baum
0 siblings, 1 reply; 13+ messages in thread
From: Stefan Kangas @ 2021-03-12 0:52 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: Basil L. Contovounesios, Boruch Baum, 45926
Lars Ingebrigtsen <larsi@gnus.org> writes:
> Boruch Baum <boruch_baum@gmx.com> writes:
>
>> Until you wrote, I definitely was. I think I still am, but I'm willing
>> to look again.
Borouch, did you have the time to look into if this data is indeed
shared or not? Thanks.
> Like Basil, I looked at a tabulated list line, and I didn't see any
> excessive duplication -- tabulated list mode puts the data on the entire
> line, but it's the same data structure, so it's all shared data, I
> think?
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#45926: Tabulated-list-mode wasteful properties
2021-03-12 0:52 ` Stefan Kangas
@ 2021-03-12 2:57 ` Boruch Baum
2021-03-12 3:39 ` Stefan Kangas
` (2 more replies)
0 siblings, 3 replies; 13+ messages in thread
From: Boruch Baum @ 2021-03-12 2:57 UTC (permalink / raw)
To: Stefan Kangas; +Cc: Basil L. Contovounesios, Lars Ingebrigtsen, 45926
On 2021-03-11 18:52, Stefan Kangas wrote:
> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
> > Boruch Baum <boruch_baum@gmx.com> writes:
> >
> >> Until you wrote, I definitely was. I think I still am, but I'm willing
> >> to look again.
>
> Borouch, did you have the time to look into if this data is indeed
> shared or not? Thanks.
Oh. I thought I had done that right away ...
Doing it again now. Yes. It's very simple to check for yourself. Open a
buffer that uses the mode and evaluate from any table line ...
(buffer-substring (line-beginning-position) (line-end-position))
You should be able to clearly see all the duplication. It's easiest when
you have pretty-print set up to display the output in a dedicated
buffer, but even without that you can isearch through the duplications.
--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#45926: Tabulated-list-mode wasteful properties
2021-03-12 2:57 ` Boruch Baum
@ 2021-03-12 3:39 ` Stefan Kangas
2021-03-12 5:46 ` Boruch Baum
2021-04-21 3:35 ` Stefan Kangas
2021-03-12 10:51 ` Basil L. Contovounesios
2021-04-21 4:31 ` Jean Louis
2 siblings, 2 replies; 13+ messages in thread
From: Stefan Kangas @ 2021-03-12 3:39 UTC (permalink / raw)
To: Boruch Baum; +Cc: Basil L. Contovounesios, Lars Ingebrigtsen, 45926
Boruch Baum <boruch_baum@gmx.com> writes:
>> Borouch, did you have the time to look into if this data is indeed
>> shared or not? Thanks.
>
> Oh. I thought I had done that right away ...
>
> Doing it again now. Yes. It's very simple to check for yourself. Open a
> buffer that uses the mode and evaluate from any table line ...
>
> (buffer-substring (line-beginning-position) (line-end-position))
>
> You should be able to clearly see all the duplication. It's easiest when
> you have pretty-print set up to display the output in a dedicated
> buffer, but even without that you can isearch through the duplications.
I think the interesting question is if that data is duplicated in memory
or if these are just pointers to the same shared data.
I would try modifying this data with something like `setcdr' to see if
it updates everywhere -- if the answer yes, then this data is shared and
there is no problem here.
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#45926: Tabulated-list-mode wasteful properties
2021-03-12 3:39 ` Stefan Kangas
@ 2021-03-12 5:46 ` Boruch Baum
2021-03-12 11:01 ` Basil L. Contovounesios
2021-04-21 3:35 ` Stefan Kangas
1 sibling, 1 reply; 13+ messages in thread
From: Boruch Baum @ 2021-03-12 5:46 UTC (permalink / raw)
To: Stefan Kangas; +Cc: Basil L. Contovounesios, Lars Ingebrigtsen, 45926
On 2021-03-11 21:39, Stefan Kangas wrote:
> I would try modifying this data with something like `setcdr' to see if
> it updates everywhere -- if the answer yes, then this data is shared and
> there is no problem here.
Why are you telling me this when it's just as easy for you to do it
yourself?
--
hkp://keys.gnupg.net
CA45 09B5 5351 7C11 A9D1 7286 0036 9E45 1595 8BC0
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#45926: Tabulated-list-mode wasteful properties
2021-03-12 2:57 ` Boruch Baum
2021-03-12 3:39 ` Stefan Kangas
@ 2021-03-12 10:51 ` Basil L. Contovounesios
2021-04-21 4:36 ` Jean Louis
2021-04-21 4:31 ` Jean Louis
2 siblings, 1 reply; 13+ messages in thread
From: Basil L. Contovounesios @ 2021-03-12 10:51 UTC (permalink / raw)
To: Boruch Baum; +Cc: Lars Ingebrigtsen, Stefan Kangas, 45926
Boruch Baum <boruch_baum@gmx.com> writes:
> Doing it again now. Yes. It's very simple to check for yourself. Open a
> buffer that uses the mode and evaluate from any table line ...
>
> (buffer-substring (line-beginning-position) (line-end-position))
>
> You should be able to clearly see all the duplication. It's easiest when
> you have pretty-print set up to display the output in a dedicated
> buffer, but even without that you can isearch through the duplications.
Have you tried setting print-circle non-nil first? This will help you
see what is actually duplicated and what is shared.
--
Basil
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#45926: Tabulated-list-mode wasteful properties
2021-03-12 5:46 ` Boruch Baum
@ 2021-03-12 11:01 ` Basil L. Contovounesios
0 siblings, 0 replies; 13+ messages in thread
From: Basil L. Contovounesios @ 2021-03-12 11:01 UTC (permalink / raw)
To: Boruch Baum; +Cc: Lars Ingebrigtsen, Stefan Kangas, 45926
Boruch Baum <boruch_baum@gmx.com> writes:
> On 2021-03-11 21:39, Stefan Kangas wrote:
>> I would try modifying this data with something like `setcdr' to see if
>> it updates everywhere -- if the answer yes, then this data is shared and
>> there is no problem here.
>
> Why are you telling me this when it's just as easy for you to do it
> yourself?
The opinions voiced so far are that there does not seem to be excessive
duplication of data, since most of it is shared.
You can help the discussion by providing either evidence to the
contrary, or a patch that demonstrably lowers the memory usage of
tabulated-list-mode and the package list.
Stefan is being helpful by nudging a bug discussion that fell silent; it
is unfair to expect him to do all the work (and where's the fun in that
anyway ;).
--
Basil
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#45926: Tabulated-list-mode wasteful properties
2021-03-12 3:39 ` Stefan Kangas
2021-03-12 5:46 ` Boruch Baum
@ 2021-04-21 3:35 ` Stefan Kangas
1 sibling, 0 replies; 13+ messages in thread
From: Stefan Kangas @ 2021-04-21 3:35 UTC (permalink / raw)
To: Boruch Baum; +Cc: Basil L. Contovounesios, Lars Ingebrigtsen, 45926-done
Stefan Kangas <stefan@marxist.se> writes:
> Boruch Baum <boruch_baum@gmx.com> writes:
>
>>> Borouch, did you have the time to look into if this data is indeed
>>> shared or not? Thanks.
>>
>> Oh. I thought I had done that right away ...
>>
>> Doing it again now. Yes. It's very simple to check for yourself. Open a
>> buffer that uses the mode and evaluate from any table line ...
>>
>> (buffer-substring (line-beginning-position) (line-end-position))
>>
>> You should be able to clearly see all the duplication. It's easiest when
>> you have pretty-print set up to display the output in a dedicated
>> buffer, but even without that you can isearch through the duplications.
>
> I think the interesting question is if that data is duplicated in memory
> or if these are just pointers to the same shared data.
>
> I would try modifying this data with something like `setcdr' to see if
> it updates everywhere -- if the answer yes, then this data is shared and
> there is no problem here.
More information was requested, but none was given within 5 weeks, so
I'm closing this bug.
If anyone can provide further information on this issue to show that
there is a bug here, please reply to this email (use "Reply to all" in
your email client) and we can reopen the bug report.
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#45926: Tabulated-list-mode wasteful properties
2021-03-12 2:57 ` Boruch Baum
2021-03-12 3:39 ` Stefan Kangas
2021-03-12 10:51 ` Basil L. Contovounesios
@ 2021-04-21 4:31 ` Jean Louis
2 siblings, 0 replies; 13+ messages in thread
From: Jean Louis @ 2021-04-21 4:31 UTC (permalink / raw)
To: Boruch Baum
Cc: Basil L. Contovounesios, Lars Ingebrigtsen, Stefan Kangas, 45926
* Boruch Baum <boruch_baum@gmx.com> [2021-03-12 05:58]:
> On 2021-03-11 18:52, Stefan Kangas wrote:
> > Lars Ingebrigtsen <larsi@gnus.org> writes:
> >
> > > Boruch Baum <boruch_baum@gmx.com> writes:
> > >
> > >> Until you wrote, I definitely was. I think I still am, but I'm willing
> > >> to look again.
> >
> > Borouch, did you have the time to look into if this data is indeed
> > shared or not? Thanks.
>
> Oh. I thought I had done that right away ...
>
> Doing it again now. Yes. It's very simple to check for yourself. Open a
> buffer that uses the mode and evaluate from any table line ...
>
> (buffer-substring (line-beginning-position) (line-end-position))
>
> You should be able to clearly see all the duplication. It's easiest when
> you have pretty-print set up to display the output in a dedicated
> buffer, but even without that you can isearch through the
> duplications.
I have done this excercise, as I have people management in the
database.
If tabulated-list-entries is ⇒ ((320295 ["320295" "Boruch Baum" "Emacs Users"]))
Then this below is what I get when invoking: (buffer-substring (line-beginning-position) (line-end-position)):
#(" 320295 Boruch Baum Emacs Users" 0 1 (tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 1 7 (help-echo "ID: 320295" tabulated-list-column-name "ID" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 7 8 (tabulated-list-column-name "ID" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 8 10 (display (space :align-to 10) tabulated-list-column-name "ID" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 10 22 (help-echo "Name: Boruch Baum" tabulated-list-column-name "Name" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 22 23 (tabulated-list-column-name "Name" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 23 51 (display (space :align-to 51) tabulated-list-column-name "Name" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]) 51 62 (help-echo "Account: Emacs Users" tabulated-list-column-name "Account" tabulated-list-id 320295 tabulated-list-entry ["320295" "Boruch Baum" "Emacs Users"]))
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#45926: Tabulated-list-mode wasteful properties
2021-03-12 10:51 ` Basil L. Contovounesios
@ 2021-04-21 4:36 ` Jean Louis
0 siblings, 0 replies; 13+ messages in thread
From: Jean Louis @ 2021-04-21 4:36 UTC (permalink / raw)
To: Basil L. Contovounesios
Cc: Lars Ingebrigtsen, Stefan Kangas, Boruch Baum, 45926
* Basil L. Contovounesios <contovob@tcd.ie> [2021-03-12 13:52]:
> Boruch Baum <boruch_baum@gmx.com> writes:
>
> > Doing it again now. Yes. It's very simple to check for yourself. Open a
> > buffer that uses the mode and evaluate from any table line ...
> >
> > (buffer-substring (line-beginning-position) (line-end-position))
> >
> > You should be able to clearly see all the duplication. It's easiest when
> > you have pretty-print set up to display the output in a dedicated
> > buffer, but even without that you can isearch through the duplications.
>
> Have you tried setting print-circle non-nil first? This will help you
> see what is actually duplicated and what is shared.
I have done that exercise where my tabulated-list-entries is only
this:
⇒ ((320295 ["320295" "Boruch Baum" "Emacs Users"]))
and
(buffer-substring (line-beginning-position) (line-end-position))
gives me this:
a ⇒ #(" 320295 Boruch Baum Emacs Users" 0 1 (tabulated-list-id 320295 tabulated-list-entry #1=["320295" "Boruch Baum" "Emacs Users"]) 1 7 (help-echo "ID: 320295" tabulated-list-column-name #3="ID" tabulated-list-id 320295 tabulated-list-entry #1#) 7 8 (tabulated-list-column-name #3# tabulated-list-id 320295 tabulated-list-entry #1#) 8 10 (display (space :align-to 10) tabulated-list-column-name #3# tabulated-list-id 320295 tabulated-list-entry #1#) 10 22 (help-echo "Name: Boruch Baum" tabulated-list-column-name #2="Name" tabulated-list-id 320295 tabulated-list-entry #1#) 22 23 (tabulated-list-column-name #2# tabulated-list-id 320295 tabulated-list-entry #1#) 23 51 (display (space :align-to 51) tabulated-list-column-name #2# tabulated-list-id 320295 tabulat
ed-list-entry #1#) 51 62 (help-echo "Account: Emacs Users" tabulated-list-column-name "Account" tabulated-list-id 320295 tabulated-list-entry #1#))
If non-nil, shared substructures anywhere in the structure are printed
with ‘#N=’ before the first occurrence (in the order of the print
representation) and ‘#N#’ in place of each subsequent occurrence,
where N is a positive decimal integer.
Thus I can see that those are shared structures mostly.
Jean
Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns
Sign an open letter in support of Richard M. Stallman
https://stallmansupport.org/
https://rms-support-letter.github.io/
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2021-04-21 4:36 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-17 7:32 bug#45926: Tabulated-list-mode wasteful properties Boruch Baum
2021-01-17 13:40 ` Basil L. Contovounesios
2021-01-17 15:11 ` Boruch Baum
2021-01-19 5:24 ` Lars Ingebrigtsen
2021-03-12 0:52 ` Stefan Kangas
2021-03-12 2:57 ` Boruch Baum
2021-03-12 3:39 ` Stefan Kangas
2021-03-12 5:46 ` Boruch Baum
2021-03-12 11:01 ` Basil L. Contovounesios
2021-04-21 3:35 ` Stefan Kangas
2021-03-12 10:51 ` Basil L. Contovounesios
2021-04-21 4:36 ` Jean Louis
2021-04-21 4:31 ` Jean Louis
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).