unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
* [PATCH 00/13] smsg: remove tricky {mime} field
@ 2020-06-01 10:06 Eric Wong
  2020-06-01 10:06 ` [PATCH 01/13] inbox: introduce smsg_eml method Eric Wong
                   ` (13 more replies)
  0 siblings, 14 replies; 15+ messages in thread
From: Eric Wong @ 2020-06-01 10:06 UTC (permalink / raw)
  To: meta

Storing a large PublicInbox::Eml (or in the past, Email::MIME)
object inside a small PublicInbox::Smsg object has historically
been bloat-prone[1] since there may be many small smsgs in
memory at once

Hundreds or thousands of $smsg objects can linger in memory due
to search results and message threading operations.  So keep
$eml and $smsg objects independent of each other, for now.
Instead, we'll introduce a $smsg->populate($eml) API to handle
filling in the keys for the importer, indexer, and
non-SQLite-using WWW users.

Furthermore, $smsg->$field dispatch has always been measurably
faster than $smsg->{$field} access in NNTP.  Since $smsg->$field
became read-only with the removal of $smsg->{mime}, we can
abandon the $smsg->$field invocations and favor of direct hash
access.

[1] the prime example being what commit 7d02b9e64455831d fixed
    ("view: stop storing all MIME objects on large threads")

Eric Wong (13):
  inbox: introduce smsg_eml method
  wwwatomstream: convert callers to use smsg_eml
  v2writable: fix non-sensical interpolation in BUG message
  import: modernize to use Perl 5.10 features
  smsg: introduce ->populate method
  smsg: get rid of ->wrap initializer, too
  inbox: msg_by_*: remove $(size)ref args
  www: remove smsg_mime API and adjust callers
  nntp: smsg_range_i: favor ->{$field} lookups when possible
  smsg: get rid of remaining {mime} users
  smsg: remove ->bytes and ->lines methods
  smsg: remove remaining accessor methods
  wwwatomstream: drop smsg->{mid} fallback for non-SQLite

 Documentation/mknews.perl        |   7 +-
 lib/PublicInbox/ExtMsg.pm        |   2 +-
 lib/PublicInbox/Feed.pm          |   8 +-
 lib/PublicInbox/Import.pm        |  69 ++++++++---------
 lib/PublicInbox/Inbox.pm         |  32 ++++----
 lib/PublicInbox/Mbox.pm          |   2 +-
 lib/PublicInbox/NNTP.pm          |  14 +++-
 lib/PublicInbox/OverIdx.pm       |   3 +-
 lib/PublicInbox/SearchIdx.pm     |  33 ++++-----
 lib/PublicInbox/SearchView.pm    |   6 +-
 lib/PublicInbox/Smsg.pm          | 123 +++++++++++--------------------
 lib/PublicInbox/SolverGit.pm     |   4 +-
 lib/PublicInbox/V2Writable.pm    |  11 +--
 lib/PublicInbox/View.pm          |  63 ++++++++--------
 lib/PublicInbox/WwwAtomStream.pm |   8 +-
 t/altid.t                        |   3 +-
 t/altid_v2.t                     |   3 +-
 t/import.t                       |   3 +-
 t/search.t                       |  46 +++++++-----
 t/v2mda.t                        |   4 +-
 t/v2writable.t                   |   5 +-
 21 files changed, 207 insertions(+), 242 deletions(-)


^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2020-06-01 16:45 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-01 10:06 [PATCH 00/13] smsg: remove tricky {mime} field Eric Wong
2020-06-01 10:06 ` [PATCH 01/13] inbox: introduce smsg_eml method Eric Wong
2020-06-01 10:06 ` [PATCH 02/13] wwwatomstream: convert callers to use smsg_eml Eric Wong
2020-06-01 10:06 ` [PATCH 03/13] v2writable: fix non-sensical interpolation in BUG message Eric Wong
2020-06-01 10:06 ` [PATCH 04/13] import: modernize to use Perl 5.10 features Eric Wong
2020-06-01 10:06 ` [PATCH 05/13] smsg: introduce ->populate method Eric Wong
2020-06-01 10:06 ` [PATCH 06/13] smsg: get rid of ->wrap initializer, too Eric Wong
2020-06-01 10:06 ` [PATCH 07/13] inbox: msg_by_*: remove $(size)ref args Eric Wong
2020-06-01 10:06 ` [PATCH 08/13] www: remove smsg_mime API and adjust callers Eric Wong
2020-06-01 10:06 ` [PATCH 09/13] nntp: smsg_range_i: favor ->{$field} lookups when possible Eric Wong
2020-06-01 10:06 ` [PATCH 10/13] smsg: get rid of remaining {mime} users Eric Wong
2020-06-01 10:06 ` [PATCH 11/13] smsg: remove ->bytes and ->lines methods Eric Wong
2020-06-01 10:06 ` [PATCH 12/13] smsg: remove remaining accessor methods Eric Wong
2020-06-01 10:06 ` [PATCH 13/13] wwwatomstream: drop smsg->{mid} fallback for non-SQLite Eric Wong
2020-06-01 16:45 ` [PATCH 00/13] smsg: remove tricky {mime} field Eric Wong

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).