unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
* [PATCH 0/5] v2: drop Xapian skeleton for SQLite overview DB
@ 2018-04-02  0:04 Eric Wong (Contractor, The Linux Foundation)
  2018-04-02  0:04 ` [PATCH 1/5] replace Xapian skeleton with " Eric Wong (Contractor, The Linux Foundation)
                   ` (4 more replies)
  0 siblings, 5 replies; 7+ messages in thread
From: Eric Wong (Contractor, The Linux Foundation) @ 2018-04-02  0:04 UTC (permalink / raw)
  To: meta

400ms to load the top-level /$INBOX/ for /git/ is pretty bad on
my systems as-is.  800ms for /lkml/ was not unacceptable
(though probably 400ms on decent servers).  It's now around 60ms
across the board with the new overview DB.

This also affects OVER/XOVER/NEWNEWS NNTP commands, as well
as Atom feeds.

Eric Wong (Contractor, The Linux Foundation) (5):
  replace Xapian skeleton with SQLite overview DB
  v2writable: simplify barrier vs checkpoints
  t/over: test empty Subject: line matching
  www: rework query responses to avoid COUNT in SQLite
  over: speedup get_thread by avoiding JOIN

 MANIFEST                                           |   5 +-
 lib/PublicInbox/Feed.pm                            |  11 +-
 lib/PublicInbox/Inbox.pm                           |  15 +-
 lib/PublicInbox/Mbox.pm                            |  12 +-
 lib/PublicInbox/Msgmap.pm                          |   1 +
 lib/PublicInbox/NNTP.pm                            |  29 +-
 lib/PublicInbox/Over.pm                            | 134 ++++++++
 lib/PublicInbox/OverIdx.pm                         | 370 +++++++++++++++++++++
 .../{SearchIdxSkeleton.pm => OverIdxFork.pm}       | 131 +++++---
 lib/PublicInbox/Search.pm                          | 124 ++-----
 lib/PublicInbox/SearchIdx.pm                       | 214 +++---------
 lib/PublicInbox/SearchIdxPart.pm                   |  16 +-
 lib/PublicInbox/SearchMsg.pm                       |  26 +-
 lib/PublicInbox/V2Writable.pm                      | 114 +++----
 lib/PublicInbox/View.pm                            |  17 +-
 script/public-inbox-compact                        |  32 +-
 t/altid.t                                          |   9 +-
 t/convert-compact.t                                |   6 +-
 t/over.t                                           |  63 ++++
 t/psgi_search.t                                    |   6 +-
 t/search-thr-index.t                               |   9 +-
 t/search.t                                         |  92 ++---
 t/v2-add-remove-add.t                              |   6 +-
 t/v2mda.t                                          |   4 +-
 t/v2writable.t                                     |   2 +-
 t/watch_maildir_v2.t                               |  26 +-
 26 files changed, 907 insertions(+), 567 deletions(-)
 create mode 100644 lib/PublicInbox/Over.pm
 create mode 100644 lib/PublicInbox/OverIdx.pm
 rename lib/PublicInbox/{SearchIdxSkeleton.pm => OverIdxFork.pm} (54%)
 create mode 100644 t/over.t

-- 
EW

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2018-04-05  8:59 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-04-02  0:04 [PATCH 0/5] v2: drop Xapian skeleton for SQLite overview DB Eric Wong (Contractor, The Linux Foundation)
2018-04-02  0:04 ` [PATCH 1/5] replace Xapian skeleton with " Eric Wong (Contractor, The Linux Foundation)
2018-04-05  8:59   ` Eric Wong
2018-04-02  0:04 ` [PATCH 2/5] v2writable: simplify barrier vs checkpoints Eric Wong (Contractor, The Linux Foundation)
2018-04-02  0:04 ` [PATCH 3/5] t/over: test empty Subject: line matching Eric Wong (Contractor, The Linux Foundation)
2018-04-02  0:04 ` [PATCH 4/5] www: rework query responses to avoid COUNT in SQLite Eric Wong (Contractor, The Linux Foundation)
2018-04-02  0:04 ` [PATCH 5/5] over: speedup get_thread by avoiding JOIN Eric Wong (Contractor, The Linux Foundation)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).