unofficial mirror of meta@public-inbox.org
 help / color / mirror / Atom feed
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: [PATCH 06/11] search: avoid needless decode
Date: Thu, 20 Aug 2015 02:57:18 +0000	[thread overview]
Message-ID: <1440039443-27052-6-git-send-email-e@80x24.org> (raw)
In-Reply-To: <1440039443-27052-1-git-send-email-e@80x24.org>

Email::MIME should handle everything for us and make things
work nicely with Xapian (assuming I understand how encoding
works in Perl).

While we're at it, reduce temporary strings and arrays by
using destructive operations and clobbering parts as we
iterate through them.
---
 lib/PublicInbox/Search.pm | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/lib/PublicInbox/Search.pm b/lib/PublicInbox/Search.pm
index c28401b..aa29ae5 100644
--- a/lib/PublicInbox/Search.pm
+++ b/lib/PublicInbox/Search.pm
@@ -6,7 +6,6 @@ use strict;
 use warnings;
 use PublicInbox::SearchMsg;
 use Search::Xapian qw/:standard/;
-require PublicInbox::View;
 use Email::MIME;
 use PublicInbox::MID qw/mid_clean mid_compressed/;
 
@@ -88,7 +87,6 @@ sub add_message {
 	my $mid = mid_compressed($mid_orig);
 	my $was_ghost = 0;
 	my $ct_msg = $mime->header('Content-Type') || 'text/plain';
-	my $enc_msg = PublicInbox::View::enc_for($ct_msg);
 
 	eval {
 		my $smsg = $self->lookup_message($mid);
@@ -151,9 +149,11 @@ sub add_message {
 			# account for filter bugs...
 			$ct =~ m!\btext/plain\b!i or return;
 
-			my $enc = PublicInbox::View::enc_for($ct, $enc_msg);
 			my (@orig, @quot);
-			foreach my $l (split(/\n/, $enc->decode($part->body))) {
+			my $body = $part->body;
+			$part->body_set('');
+			my @lines = split(/\n/, $body);
+			while (defined(my $l = shift @lines)) {
 				if ($l =~ /^\s*>/) {
 					push @quot, $l;
 				} else {
@@ -162,10 +162,12 @@ sub add_message {
 			}
 			if (@quot) {
 				$tg->index_text(join("\n", @quot), 0);
+				@quot = ();
 				$tg->increase_termpos;
 			}
 			if (@orig) {
 				$tg->index_text(join("\n", @orig));
+				@orig = ();
 				$tg->increase_termpos;
 			}
 		});
-- 
EW


  parent reply	other threads:[~2015-08-20  2:57 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-08-20  2:57 [PATCH 01/11] feed: remove threading from index Eric Wong
2015-08-20  2:57 ` [PATCH 02/11] feed: move timestamp parsing to view Eric Wong
2015-08-20  2:57 ` [PATCH 03/11] use tables for rendering comment nesting Eric Wong
2015-08-20  2:57 ` [PATCH 04/11] view: avoid nesting <a> tags from auto-linkification Eric Wong
2015-08-20  2:57 ` [PATCH 05/11] index: simplify main landing page if search-enabled Eric Wong
2015-08-20  2:57 ` Eric Wong [this message]
2015-08-20  2:57 ` [PATCH 07/11] search: reject ghosts in all cases Eric Wong
2015-08-20  2:57 ` [PATCH 08/11] view: reduce memory usage when displaying large threads Eric Wong
2015-08-20  2:57 ` [PATCH 09/11] search: bump schema version to 5 for subject_path Eric Wong
2015-08-20  2:57 ` [PATCH 10/11] index: layout fix + title and Atom feed links at top Eric Wong
2015-08-20  2:57 ` [PATCH 11/11] view: do not fold top-level messages in thread Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://public-inbox.org/README

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1440039443-27052-6-git-send-email-e@80x24.org \
    --to=e@80x24.org \
    --cc=meta@public-inbox.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).