[RFC] vim plugin rewrite II

[RFC] vim plugin rewrite II

[anton]

Hi,
this is a followup to my mail from spring where i presented a partial
rewrite of the vim plugin using python. There weren't many comments back
then, so I hope there will be more now.

I've changed the filenames so my version now coexists with the current
vim plugin. You can find it in

git://git.khirnov.net/git/notmuch

branch vim. Simply copy vim/plugin/{nm_vim.py,notmuch-vimpy.vim} to the
vim plugins dir and vim/syntax/{nm_vimpy*} to the vim syntax dir and run
:NMVimpy() in vim. You'll need vim with python support and
python-notmuch bindings.

The advantages over current vim client are still the following:
* sending and displaying/saving attachments
* much better unicode support
* tag name and search commands completion
* proper representation of the thread structure
* easier to extend thanks to python's massive standard library

Please comment.

-- 
Anton Khirnov

Re: [RFC] vim plugin rewrite II

[David Bremner]

On Sat, 14 Jan 2012 08:54:43 +0100, anton@khirnov.net wrote:
> 
> The advantages over current vim client are still the following:
> * sending and displaying/saving attachments
> * much better unicode support
> * tag name and search commands completion
> * proper representation of the thread structure
> * easier to extend thanks to python's massive standard library
> 
> Please comment.

Hi Anton;

I'm not a vim user, so I probably can't say anything very helpful, but
it does sound like some nice improvements.  Hopefully one of our vim
front-end users can comment a bit more.  One thing I wondered about is
if your version is still completely synchronous, with the resulting
problems dealing with large (say, more than 10000 messages) search
results?

All the best,

David

Re: [RFC] vim plugin rewrite II

[Anton Khirnov]

On Sat, 14 Jan 2012 22:15:07 -0400, David Bremner <david@tethera.net> wrote:
> On Sat, 14 Jan 2012 08:54:43 +0100, anton@khirnov.net wrote:
> > 
> > The advantages over current vim client are still the following:
> > * sending and displaying/saving attachments
> > * much better unicode support
> > * tag name and search commands completion
> > * proper representation of the thread structure
> > * easier to extend thanks to python's massive standard library
> > 
> > Please comment.
> 
> Hi Anton;
> 
> I'm not a vim user, so I probably can't say anything very helpful, but
> it does sound like some nice improvements.  Hopefully one of our vim
> front-end users can comment a bit more.  One thing I wondered about is
> if your version is still completely synchronous, with the resulting
> problems dealing with large (say, more than 10000 messages) search
> results?
> 

Yes, it's still synchronous. I suppose making it asynchronous shouldn't
be all that hard, i just never found enough time and motivation for that
(it was never a huge problem here).

-- 
Anton Khirnov

Re: [RFC] vim plugin rewrite II

[Anton Khirnov]

On Mon, 16 Jan 2012 09:34:59 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Sun, 15 Jan 2012 16:34:08 +0100, Anton Khirnov <anton@khirnov.net> wrote:
> > Yes, it's still synchronous. I suppose making it asynchronous shouldn't
> > be all that hard, i just never found enough time and motivation for that
> > (it was never a huge problem here).
> 
> My understanding was that vim was not actually capable of loading files
> asynchronously.  That would be very interesting if it was.
> 

It should be possible to run the query in a separate python thread which
would write the results into the vim buffer.

-- 
Anton Khirnov

Re: [RFC] vim plugin rewrite II

[Felipe Contreras]

On Sat, Jan 14, 2012 at 9:54 AM,  <anton@khirnov.net> wrote:
> branch vim. Simply copy vim/plugin/{nm_vim.py,notmuch-vimpy.vim} to the
> vim plugins dir and vim/syntax/{nm_vimpy*} to the vim syntax dir and run
> :NMVimpy() in vim. You'll need vim with python support and
> python-notmuch bindings.

I gave this a try, copying those files makes vim crash for me.

I probably need to install notmuch's python bindings, but either way
it shouldn't crash.

Cheers.

-- 
Felipe Contreras

Re: [RFC] vim plugin rewrite II

[Felipe Contreras]

On Wed, Apr 18, 2012 at 5:21 PM, Felipe Contreras
<felipe.contreras@gmail.com> wrote:
> On Sat, Jan 14, 2012 at 9:54 AM,  <anton@khirnov.net> wrote:
>> branch vim. Simply copy vim/plugin/{nm_vim.py,notmuch-vimpy.vim} to the
>> vim plugins dir and vim/syntax/{nm_vimpy*} to the vim syntax dir and run
>> :NMVimpy() in vim. You'll need vim with python support and
>> python-notmuch bindings.
>
> I gave this a try, copying those files makes vim crash for me.
>
> I probably need to install notmuch's python bindings, but either way
> it shouldn't crash.

All right, with the bindings it works, but if it cannot find the
database, it crashes too.

And this slows by 5 times the startup time of vim for me:

vim -c 'quit'  0.47s user 0.02s system 99% cpu 0.501 total
vim -c 'quit'  0.08s user 0.01s system 96% cpu 0.092 total

It is interesting, but I personally I would not use if it's going to
slow vim for everything else, there must be a way to solve that. Also,
would be nice if you rebased your branch on top of the latest release.

Cheers.

-- 
Felipe Contreras

Re: [RFC] vim plugin rewrite II

[Felipe Contreras]

On Wed, Apr 18, 2012 at 5:42 PM, Felipe Contreras
<felipe.contreras@gmail.com> wrote:
> On Wed, Apr 18, 2012 at 5:21 PM, Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
>> On Sat, Jan 14, 2012 at 9:54 AM,  <anton@khirnov.net> wrote:
>>> branch vim. Simply copy vim/plugin/{nm_vim.py,notmuch-vimpy.vim} to the
>>> vim plugins dir and vim/syntax/{nm_vimpy*} to the vim syntax dir and run
>>> :NMVimpy() in vim. You'll need vim with python support and
>>> python-notmuch bindings.
>>
>> I gave this a try, copying those files makes vim crash for me.
>>
>> I probably need to install notmuch's python bindings, but either way
>> it shouldn't crash.
>
> All right, with the bindings it works, but if it cannot find the
> database, it crashes too.
>
> And this slows by 5 times the startup time of vim for me:
>
> vim -c 'quit'  0.47s user 0.02s system 99% cpu 0.501 total
> vim -c 'quit'  0.08s user 0.01s system 96% cpu 0.092 total
>
> It is interesting, but I personally I would not use if it's going to
> slow vim for everything else, there must be a way to solve that. Also,
> would be nice if you rebased your branch on top of the latest release.

I fixed the issue this way:

--- notmuch-vimpy.vim	2012-04-18 22:38:16.193358898 +0300
+++ notmuch-vimpy-mod.vim	2012-04-19 17:07:19.390693437 +0300
@@ -29,11 +29,7 @@
     finish
 endif

-" init the python layer
-let s:python_path = expand('<sfile>:p:h')
-python import sys
-exec "python sys.path += [r'" . s:python_path . "']"
-python import vim, nm_vim
+let s:notmuch_loaded = 1

 command! NMVimpy call NMVimpy()

@@ -815,7 +811,11 @@
 " --- command handler {{{1

 function! NMVimpy()
-    call <SID>NM_cmd_folders(g:nm_vimpy_folders)
+	let s:python_path = expand('<sfile>:p:h')
+	python import sys
+	exec "python sys.path += [r'" . s:python_path . "']"
+	python import vim, nm_vim
+	call <SID>NM_cmd_folders(g:nm_vimpy_folders)
 endfunction

 "Custom foldtext() for show buffers, which indents folds to
@@ -859,5 +859,3 @@
     python nm_vim.vim_get_tags()
     return prefix . substitute(taglist, "\n", "\n" . prefix, "g")
 endfunction
-
-let s:notmuch_loaded = 1

I was seriously considering to concentrate on this plugin instead of
the current one, but I'm afraid every little error causes a crash,
even when a subprocess fails (e.g. msmtp), so it's not really usable
for me. Not to mention that it's really hard to debug, because every
bug causes a crash, and sometimes I get random crashes with no
information about what caused it at all.

I am starting to work on a version that uses ruby, and it doesn't seem
to have these issues, but lets see. I'm still not sure if we should
depend on ruby/python bindings, maybe there's a way to make them
optional.

Anyway, if you find a way to improve the crash issues, let me know, so
far it's the only real issue I see with this plug-in.

Cheers.

-- 
Felipe Contreras

Re: [RFC] vim plugin rewrite II

[Anton Khirnov]

Hi,
sorry for the late reply, I see you already reached the same point as
me, except with ruby ;) Yay for competition.

On Thu, 19 Apr 2012 19:36:56 +0300, Felipe Contreras <felipe.contreras@gmail.com> wrote:
> On Wed, Apr 18, 2012 at 5:42 PM, Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> > On Wed, Apr 18, 2012 at 5:21 PM, Felipe Contreras
> > <felipe.contreras@gmail.com> wrote:
> >> On Sat, Jan 14, 2012 at 9:54 AM,  <anton@khirnov.net> wrote:
> >>> branch vim. Simply copy vim/plugin/{nm_vim.py,notmuch-vimpy.vim} to the
> >>> vim plugins dir and vim/syntax/{nm_vimpy*} to the vim syntax dir and run
> >>> :NMVimpy() in vim. You'll need vim with python support and
> >>> python-notmuch bindings.
> >>
> >> I gave this a try, copying those files makes vim crash for me.
> >>
> >> I probably need to install notmuch's python bindings, but either way
> >> it shouldn't crash.
> >
> > All right, with the bindings it works, but if it cannot find the
> > database, it crashes too.
> >
> > And this slows by 5 times the startup time of vim for me:
> >
> > vim -c 'quit'  0.47s user 0.02s system 99% cpu 0.501 total
> > vim -c 'quit'  0.08s user 0.01s system 96% cpu 0.092 total
> >
> > It is interesting, but I personally I would not use if it's going to
> > slow vim for everything else, there must be a way to solve that. Also,
> > would be nice if you rebased your branch on top of the latest release.
> 
> I fixed the issue this way:
> 
> --- notmuch-vimpy.vim	2012-04-18 22:38:16.193358898 +0300
> +++ notmuch-vimpy-mod.vim	2012-04-19 17:07:19.390693437 +0300
> @@ -29,11 +29,7 @@
>      finish
>  endif
> 
> -" init the python layer
> -let s:python_path = expand('<sfile>:p:h')
> -python import sys
> -exec "python sys.path += [r'" . s:python_path . "']"
> -python import vim, nm_vim
> +let s:notmuch_loaded = 1
> 
>  command! NMVimpy call NMVimpy()
> 
> @@ -815,7 +811,11 @@
>  " --- command handler {{{1
> 
>  function! NMVimpy()
> -    call <SID>NM_cmd_folders(g:nm_vimpy_folders)
> +	let s:python_path = expand('<sfile>:p:h')
> +	python import sys
> +	exec "python sys.path += [r'" . s:python_path . "']"
> +	python import vim, nm_vim
> +	call <SID>NM_cmd_folders(g:nm_vimpy_folders)
>  endfunction
> 
>  "Custom foldtext() for show buffers, which indents folds to
> @@ -859,5 +859,3 @@
>      python nm_vim.vim_get_tags()
>      return prefix . substitute(taglist, "\n", "\n" . prefix, "g")
>  endfunction
> -
> -let s:notmuch_loaded = 1
> 

Thanks, looks good.

> I was seriously considering to concentrate on this plugin instead of
> the current one, but I'm afraid every little error causes a crash,
> even when a subprocess fails (e.g. msmtp), so it's not really usable
> for me. Not to mention that it's really hard to debug, because every
> bug causes a crash, and sometimes I get random crashes with no
> information about what caused it at all.
> 
> I am starting to work on a version that uses ruby, and it doesn't seem
> to have these issues, but lets see. I'm still not sure if we should
> depend on ruby/python bindings, maybe there's a way to make them
> optional.
> 
> Anyway, if you find a way to improve the crash issues, let me know, so
> far it's the only real issue I see with this plug-in.

That is weird, I'm not getting any crashes here. On any exception in the
python code it prints the backtrace and continues normally. I don't
think I've ever seen it actually crash (not counting my ultimately
unsuccessfull attempts at threading). I wonder what could cause this.

-- 
Anton Khirnov

Re: [RFC] vim plugin rewrite II

[Felipe Contreras]

On Wed, May 2, 2012 at 10:23 PM, Anton Khirnov <anton@khirnov.net> wrote:

> sorry for the late reply, I see you already reached the same point as
> me, except with ruby ;) Yay for competition.

I think my plugin is a bit further. And yeah, competition is good, but
also collaboration; hopefully the 3 plugins can share as much as
possible.

>> I was seriously considering to concentrate on this plugin instead of
>> the current one, but I'm afraid every little error causes a crash,
>> even when a subprocess fails (e.g. msmtp), so it's not really usable
>> for me. Not to mention that it's really hard to debug, because every
>> bug causes a crash, and sometimes I get random crashes with no
>> information about what caused it at all.
>>
>> I am starting to work on a version that uses ruby, and it doesn't seem
>> to have these issues, but lets see. I'm still not sure if we should
>> depend on ruby/python bindings, maybe there's a way to make them
>> optional.
>>
>> Anyway, if you find a way to improve the crash issues, let me know, so
>> far it's the only real issue I see with this plug-in.
>
> That is weird, I'm not getting any crashes here. On any exception in the
> python code it prints the backtrace and continues normally. I don't
> think I've ever seen it actually crash (not counting my ultimately
> unsuccessfull attempts at threading). I wonder what could cause this.

Interesting. I would need to check that, but I don't have time right now =/

Cheers.

-- 
Felipe Contreras