[DRAFT] New features in Vim interface

[DRAFT] New features in Vim interface

[Bartosz]

Hi,

I really like to use notmuch for organising my emails. However, I noticed that vim interface is slightly lagging behind emacs. Some of the most important features still missing are:

- adding attachments (PATCH 4)
- saving sent emails to "Sent" box in maildir (PATCH 3)
- forwarding messages (not implemented yet)
- folding read messages in thread view (PATCH 1 and 2)

This patch series attempts to implement some of the features. It is not a finished product, but rather a prototype of the interface. If you consider it worthwhile I will try to polish the patches.

[PATCH 1/4] VIM: implemented message folding in thread view

This patch automatically folds all read messages in thread view and jumps to the first unread message (if present). It uses vim folding mechanisms (based on syntax definition), so folds can be easily opened/closed.

[PATCH 2/4] VIM: move backward trough messages in thread mode

Currently, vim frontend allows to jump forward through messages in a thread using Tab key. This patch introduces Shift-Tab shortcut to move backwards. In addition, jumping to a new message automatically unfolds it.

[PATCH 3/4] VIM: save sent message to maildir

This patch uses "notmuch insert" for adding a message to a sent folder in mail archives after it is sent. The path to "sent" folder is configurable through .notmuch-config, by defining a section called [vim]. So far there is a single parameter in the section "sent_dirs" that should define a mapping between email addresses and folders. For example:

  [vim]
  sent_dirs=telenczb@hu-berlin.de=>HU/Sent;telenczuk@unic.cnrs-gif.fr=>CNRS/Sent

Email address is parsed from the "From" header of the sent email.

[PATCH 4/4] VIM: adding attachments

Paths to files to be attached should be listed as the final lines of the composed message and prefixed by "Attachment:" (one line per file). For example:
   
   <HEADER>

   <MESSAGE BODY>

   Attachment: /home/bartosz/test.pdf
   Attachment: /home/bartosz/test.txt

I will be glad to hear your opinion about the design and implementation.

Yours,

Bartosz

[PATCH 1/4] VIM: implemented message folding in thread view

[Bartosz]

---
 vim/notmuch.vim | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/vim/notmuch.vim b/vim/notmuch.vim
index cad9517..34d4f92 100644
--- a/vim/notmuch.vim
+++ b/vim/notmuch.vim
@@ -345,8 +345,13 @@ ruby << EOF
 		VIM::command("syntax region nmShowMsg#{i}Desc start='\\%%%il' end='\\%%%il' contains=@nmShowMsgDesc" % [msg.start, msg.start + 1])
 		VIM::command("syntax region nmShowMsg#{i}Head start='\\%%%il' end='\\%%%il' contains=@nmShowMsgHead" % [msg.start + 1, msg.body_start])
 		VIM::command("syntax region nmShowMsg#{i}Body start='\\%%%il' end='\\%%%dl' contains=@nmShowMsgBody" % [msg.body_start, msg.end])
+		VIM::command("syntax region nmShowMsg#{i}Fold start='\\%%%il' end='\\%%%dl' contains=nmShowMsg#{i}Body,nmShowMsg#{i}Head,nmShowMsg#{i}Desc fold" % [msg.start, msg.end])
 	end
+	VIM::command("g/^.*(.*unread.*)$/normal zo")
+	VIM::command("nohl")
+	VIM::command("normal gg/unread/1") 
 EOF
+    set foldmethod=syntax
 	setlocal nomodifiable
 	call s:set_map(g:notmuch_show_maps)
 endfunction
@@ -957,5 +962,6 @@ EOF
 endfunction
 
 command -nargs=* NotMuch call s:NotMuch(<f-args>)
+set foldtext=v:folddashes.substitute(getline(v:foldstart),'{{{','','g')
 
 " vim: set noexpandtab:
-- 
1.9.3 (Apple Git-50)

[PATCH 2/4] VIM: move backward trough messages in thread mode

[Bartosz]

---
 vim/notmuch.vim | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/vim/notmuch.vim b/vim/notmuch.vim
index 34d4f92..4f90d79 100644
--- a/vim/notmuch.vim
+++ b/vim/notmuch.vim
@@ -39,7 +39,8 @@ let g:notmuch_show_maps = {
 	\ 'p':		'show_save_patches()',
 	\ 'r':		'show_reply()',
 	\ '?':		'show_info()',
-	\ '<Tab>':	'show_next_msg()',
+	\ '<Tab>':	'show_next_msg(1)',
+	\ '<S-Tab>':	'show_next_msg(-1)',
 	\ 'c':		'compose()',
 	\ }
 
@@ -114,15 +115,18 @@ EOF
 	call s:kill_this_buffer()
 endfunction
 
-function! s:show_next_msg()
+function! s:show_next_msg(inc)
 ruby << EOF
+	inc = VIM::evaluate('a:inc')
 	r, c = $curwin.cursor
 	n = $curbuf.line_number
 	i = $messages.index { |m| n >= m.start && n <= m.end }
-	m = $messages[i + 1]
-	if m
+	m = $messages[i + inc]
+	if m and !((i + inc)<0) 
 		r = m.body_start + 1
+		VIM::command("normal zM")
 		VIM::command("normal #{m.start}zt")
+		VIM::command("normal zo")
 		$curwin.cursor = r, c
 	end
 EOF
-- 
1.9.3 (Apple Git-50)

[PATCH 3/4] VIM: save sent message to maildir

[Bartosz]

maildir folder is slected based on the address in
from header and it is configurable via notmuch
config.
---
 vim/notmuch.vim | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/vim/notmuch.vim b/vim/notmuch.vim
index 4f90d79..e95db4d 100644
--- a/vim/notmuch.vim
+++ b/vim/notmuch.vim
@@ -110,6 +110,9 @@ EOF
 		echohl None
 		return
 	endif
+
+	call s:save_to_sent(fname)
+
 	call delete(fname)
 	echo 'Mail sent successfully.'
 	call s:kill_this_buffer()
@@ -257,6 +260,20 @@ function! s:folders_refresh()
 	setlocal nomodifiable
 endfunction
 
+function! s:save_to_sent(fname)
+ruby << EOF
+    if $sent_dirs
+		fname = VIM::evaluate('a:fname')
+		m = Mail.read(fname)
+		from_address = m.from[0]
+		sent_box = $sent_dirs[from_address]
+		if sent_box
+			system "notmuch insert --folder:#{sent_box} +sent < #{fname}"
+		end
+	end
+EOF
+endfunction
+
 "" basic
 
 function! s:show_cursor_moved()
@@ -505,6 +522,8 @@ ruby << EOF
 		$email_address = get_config_item('user.primary_email')
 		$email_name = get_config_item('user.name')
 		$email = "%s <%s>" % [$email_name, $email_address]
+		sent_dirs_config = get_config_item('vim.sent_dirs')
+		$sent_dirs = Hash[sent_dirs_config.split("\n").collect{|x| x.strip.split("=>")}]
 	end
 
 	def vim_puts(s)
@@ -698,6 +717,7 @@ ruby << EOF
 		end
 	end
 
+
 	module DbHelper
 		def init(name)
 			@name = name
-- 
1.9.3 (Apple Git-50)

[PATCH 4/4] VIM: adding attachments

[Bartosz]

files to be attached are listed at the end of composed message as
paths prefixed by "Attachment:"
---
 vim/notmuch.vim | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/vim/notmuch.vim b/vim/notmuch.vim
index e95db4d..d999aef 100644
--- a/vim/notmuch.vim
+++ b/vim/notmuch.vim
@@ -92,9 +92,16 @@ function! s:compose_send()
 
 ruby << EOF
 	# Generate proper mail to send
-	text = VIM::evaluate('lines').join("\n")
-	fname = VIM::evaluate('fname')
+	text = VIM::evaluate('lines')
+	attached_filenames = text.reverse.take_while { |line| line.start_with?("Attachment:") }
+	text = text.reverse.drop_while { |line| line.start_with?("Attachment:") } . reverse
+	text = text.join("\n")
 	transport = Mail.new(text)
+	attached_filenames.each do |afname|
+	    transport.add_file(afname[11..-1].strip)
+	end
+	    
+	fname = VIM::evaluate('fname')
 	transport.message_id = generate_message_id
 	transport.charset = 'utf-8'
 	File.write(fname, transport.to_s)
-- 
1.9.3 (Apple Git-50)

Re: [DRAFT] New features in Vim interface

[David Bremner]

Bartosz <telenczuk@unic.cnrs-gif.fr> writes:

> Hi,
>
> I really like to use notmuch for organising my emails. However, I noticed that vim interface is slightly lagging behind emacs. Some of the most important features still missing are:
>
> - adding attachments (PATCH 4)
> - saving sent emails to "Sent" box in maildir (PATCH 3)
> - forwarding messages (not implemented yet)
> - folding read messages in thread view (PATCH 1 and 2)
>
> This patch series attempts to implement some of the features. It is
> not a finished product, but rather a prototype of the interface. If
> you consider it worthwhile I will try to polish the patches.

Before Christmas we had a flurry of patches from Ian Main (in copy). I
think there is some functionality overlap with what you proposed. As far
as I know Ian has mainly been working on his private fork of the vim
client, but maybe if the two (or three, including Franz) of you can work
together to review each other's patches and polish things we can get
some of these things upstream.

You have look at some of the existing vim patches under review at

        http://nmbug.tethera.net/status/#Review

Re: [DRAFT] New features in Vim interface

[Bartosz Telenczuk]

Dear David, 
(CC Ian and Franz)

> Before Christmas we had a flurry of patches from Ian Main (in copy). I
> think there is some functionality overlap with what you proposed. As far
> as I know Ian has mainly been working on his private fork of the vim
> client, but maybe if the two (or three, including Franz) of you can work
> together to review each other's patches and polish things we can get
> some of these things upstream.

Thanks a lot for pointing me to Ian's patches. Indeed, some of the functionality overlaps; this only means that these features are on everyone wishlist. I will be happy to help Ian and Franz get these patches upstream.

I am not familiar with the review process at NM. If someone could help me to get started, I will be much obliged.

Yours,

Bartoz

Re: [DRAFT] New features in Vim interface

[David Bremner]

Bartosz Telenczuk <telenczuk@unic.cnrs-gif.fr> writes:

>
> I am not familiar with the review process at NM. If someone could help me to get started, I will be much obliged.
>

Sure.  We have tried to document the process at

       http://notmuchmail.org/contributing/

In particular  steps 14, 15

An example (hopefully not too daunting) of a recently review process is
at

        id:1415147159-19946-1-git-send-email-sojkam1@fel.cvut.cz

Tagging things in nmbug is really not the main issue; the main issue is
that we see on the list that a kind of "pair programming" happens and
people work through the patches together.

d

Re: [DRAFT] New features in Vim interface

[Ian Main]

Bartosz Telenczuk wrote:
> Dear David, 
> (CC Ian and Franz)
> 
> > Before Christmas we had a flurry of patches from Ian Main (in copy). I
> > think there is some functionality overlap with what you proposed. As far
> > as I know Ian has mainly been working on his private fork of the vim
> > client, but maybe if the two (or three, including Franz) of you can work
> > together to review each other's patches and polish things we can get
> > some of these things upstream.
> 
> Thanks a lot for pointing me to Ian's patches. Indeed, some of the functionality overlaps; this only means that these features are on everyone wishlist. I will be happy to help Ian and Franz get these patches upstream.
> 
> I am not familiar with the review process at NM. If someone could help me to get started, I will be much obliged.
> 
> Yours,
> 
> Bartoz

Yeah that would be ideal.  Franz and I have been working in a github repo here:

https://github.com/imain/notmuch-vim

There are a huge number of changes from what is in the notmuch tree though.
The idea of splitting out the patches and getting them in here is a bit
daunting but we could certainly try.  We also really wanted to refactor the
code but the pace has slowed now; probably because the client is very usable as
it is :).

Let me know what you think of the code base and if you are willing to help out
with merging the changes (lots of reviews I think mostly).

    Ian

Re: [DRAFT] New features in Vim interface

[David Bremner]

Ian Main <imain@stemwinder.org> writes:

> Let me know what you think of the code base and if you are willing to help out
> with merging the changes (lots of reviews I think mostly).

In nmbug commit cb9e7acbb6b I marked the outstanding vim patches from
Ian and Franz as "moreinfo"; we can revisit their status when you guys
decide what you want to do.

Cheers,

d

Re: [DRAFT] New features in Vim interface

[Bartosz Telenczuk]

> Yeah that would be ideal.  Franz and I have been working in a github repo here:
> 
> https://github.com/imain/notmuch-vim

Indeed, that is a lot of commits. It is almost an independent fork. Even more so we should work hard to include the changes back into upstream for "the benefit of the community".

> Let me know what you think of the code base and if you are willing to help out
> with merging the changes (lots of reviews I think mostly).

I would suggest to start with the patches you already sent to notmuch mailing list. I already reviewed three:

VIM: Automatically refresh folder screen, id:54bd89e062df1_a663fec9d49e200a3@bts-MacBook-Pro.local.notmuch
Make an option to save sent mail locally, id:1412617904-27252-1-git-send-email-imain@stemwinder.org
Improve moving between messages in a thread, id:1412579537-7921-1-git-send-email-imain@stemwinder.org

The last one with only a minor comment. Let me know what you think.

Bartosz

RE: [PATCH 1/4] VIM: implemented message folding in thread view

[Franz Fellner]

Hi Bartosz,

We already had folding via syntax in Ians fork. It turned out to be not that great.
https://github.com/imain/notmuch-vim/issues/3
In short:
Those syntax fold marks are for one specific notmuch-show-buffer. If you open a second show-buffer those
new fold marks will overwrite the ones from the previous show-buffer. Switching between those two show-buffers
will spin up CPU for quite some time and completely lock vim.
The fix I worked out was to switch to manual folding. Manual folds get chached for each buffer.

I could come up with a patch, but I think there are more important things, like attachments or storing sent mails.
It also would help a lot if we could reoranize the sources as in Ians fork. Would simplify testing, coding (proper filetype) and backporting patches.

Franz

Bartosz wrote:
> ---
>  vim/notmuch.vim | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/vim/notmuch.vim b/vim/notmuch.vim
> index cad9517..34d4f92 100644
> --- a/vim/notmuch.vim
> +++ b/vim/notmuch.vim
> @@ -345,8 +345,13 @@ ruby << EOF
>  		VIM::command("syntax region nmShowMsg#{i}Desc start='\\%%%il' end='\\%%%il' contains=@nmShowMsgDesc" % [msg.start, msg.start + 1])
>  		VIM::command("syntax region nmShowMsg#{i}Head start='\\%%%il' end='\\%%%il' contains=@nmShowMsgHead" % [msg.start + 1, msg.body_start])
>  		VIM::command("syntax region nmShowMsg#{i}Body start='\\%%%il' end='\\%%%dl' contains=@nmShowMsgBody" % [msg.body_start, msg.end])
> +		VIM::command("syntax region nmShowMsg#{i}Fold start='\\%%%il' end='\\%%%dl' contains=nmShowMsg#{i}Body,nmShowMsg#{i}Head,nmShowMsg#{i}Desc fold" % [msg.start, msg.end])
>  	end
> +	VIM::command("g/^.*(.*unread.*)$/normal zo")
> +	VIM::command("nohl")
> +	VIM::command("normal gg/unread/1") 
>  EOF
> +    set foldmethod=syntax
>  	setlocal nomodifiable
>  	call s:set_map(g:notmuch_show_maps)
>  endfunction
> @@ -957,5 +962,6 @@ EOF
>  endfunction
>  
>  command -nargs=* NotMuch call s:NotMuch(<f-args>)
> +set foldtext=v:folddashes.substitute(getline(v:foldstart),'{{{','','g')
>  
>  " vim: set noexpandtab:
> -- 
> 1.9.3 (Apple Git-50)
> 
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

RE: [PATCH 1/4] VIM: implemented message folding in thread view

[Bartosz Telenczuk]

Hi Franz,

> We already had folding via syntax in Ians fork. It turned out to be not that great.
> https://github.com/imain/notmuch-vim/issues/3

Ah all right, I just thought syntax fold would be more elegant.

> In short:
> Those syntax fold marks are for one specific notmuch-show-buffer. If you open a second show-buffer those
> new fold marks will overwrite the ones from the previous show-buffer. Switching between those two show-buffers
> will spin up CPU for quite some time and completely lock vim.

True, I can reproduce this bug in my implementation. 

> It also would help a lot if we could reoranize the sources as in Ians fork. Would simplify testing, coding (proper filetype) and backporting patches.

Agree. We should come up with a plan how to integrate the changes back into notmuch. Do you have any ideas?

Cheers,

Bartosz