I just did another test with a different vc backend: the Mercurial backend. 

I created a local Mercurial repo and submitted a file that is not completely rendered by vc-annotate with the SVN backend.

I modified the Mercurial hosted file, committing my changes in the Mercurial repo.  Then I used vc-annotate on that Mercurial hosted file and every line was annotated as it should be.

So it does seem to be specific to the SVN backend.