In fact, I now notice that setting visual-line-fringe-indicators is
irrelevant for the bug.

With ...

$ emacs-23-1 -geometry 80x10 -Q 80columns -f visual-line-mode

... I get: