doc-view: Document quality and find-file

doc-view: Document quality and find-file

[Amy Templeton]

Hi all,

I'm a student, and I've been making lots of use of the new doc-view PDF
viewer since the year started. (I'd been hoping for an Emacs PDF reader for
a while, because trying to look at xpdf and Emacs at the same time is kind
of annoying, especially when trying to, well, do anything (xpdf is fine, of
course--just another application to deal with, and which pops up
windows...)) The searching-within-a-PDF function is particularly useful, but
the whole thing makes things more convenient.

Anyway, I have a couple of questions.

(1)  Is there any way to increase the image quality? The letters of the PDF
     files seem to end up somewhat butchered, making some hard to read. (I am
     reading in English, if that information comes in handy.)

(2)  When using "M-x doc-view" to find a file, Emacs tries very hard to
     ignore PDF files and will actually just complete around them when doing
     this (i.e., if a directory contains a PDF file and another directory
     and I press <TAB>, it'll just complete the directory without popping up
     a completions buffer, per the standard behavior of find-file). Though
     this behavior is normally beneficial, has anyone come up with a hack to
     circumvent it when using doc-view in particular?

Thanks to Tassilo Horn for writing this package and to the folks whom I've
seen contributing on this list!

Thanks,
Amy

-- 
By golly, I'm beginning to think Linux really *is* the best thing since
sliced bread.
                -- Vance Petree, Virginia Power

Re: doc-view: Document quality and find-file

[Peter Dyballa]

Am 17.09.2007 um 17:17 schrieb Amy Templeton:

> (1)  Is there any way to increase the image quality? The letters of  
> the PDF
>      files seem to end up somewhat butchered, making some hard to  
> read.

The xpdf suite also contains the programmes pdfinfo and pdffonts. The  
latter can be useful to determine whether the PDF file has real fonts  
(vector based fonts in PostScript or TrueType format [OpenType is  
some wrapper around PostScript or TrueType]) or just bitmaps from  
TeX. The "real fonts" are clear names while the TeX fonts seem to  
come from some cryptographic engine ...


One option could be to correct the default resolution with which  
Ghostscript is converting the vector fonts in a PDF document to  
bitmaps ("-r100"). Maybe your screen has coarser or finer resolution,  
so you might update the value.

	(defcustom doc-view-ghostscript-options
	  '("-dNOPAUSE" "-sDEVICE=png16m" "-dTextAlphaBits=4"
	    "-dBATCH" "-dGraphicsAlphaBits=4" "-dQUIET"
	    "-r100")
	  "A list of options to give to ghostview."
	  :type '(sexp)
	  :group 'doc-view)

Another cause could be that -dGraphicsAlphaBits=4" can make lines  
that should join end before the joint. Setting the numerical value of  
4 to 1 can correct this behaviour.

Another option might be to substitute the default output format  
"png16m" (best suited for 24 bit RGB displays) with any of "pnggray"  
for greyscale, "png256" for 8-bit colour, "png16" for 4-bit colour,  
or "pngmono" for black-and-white. They all won't show the same amount  
of anti-aliasing power!

--
Greetings

   Pete

"One person with a belief is a social power equal to ninety-nine who  
have only interests."
                                                  - John Stuart Mill

Re: doc-view: Document quality and find-file

[Amy Templeton]

Peter Dyballa <Peter_Dyballa@Web.DE> wrote:
> Am 17.09.2007 um 17:17 schrieb Amy Templeton:
> > (1) Is there any way to increase the image quality? The letters of the
> > PDF files seem to end up somewhat butchered, making some hard to read.

> One option could be to correct the default resolution with which
> Ghostscript is converting the vector fonts in a PDF document to bitmaps
> ("-r100"). Maybe your screen has coarser or finer resolution, so you might
> update the value.

Hm. Well, I tried setting it to 150, and it certainly makes it more readable
(also, larger!) However, I seem to be running into the problem there that
sometimes an article will be scanned "landscape"-style, in which case I
can't scroll over, and 125 seems to have *both* problems with a lot of the
particular articles in question. Oh well. I'll mess with that some more and
see if I can find one that works.

> Another cause could be that -dGraphicsAlphaBits=4" can make lines that
> should join end before the joint. Setting the numerical value of 4 to 1
> can correct this behaviour.

I'm not sure whether this helped or not, but I'll keep it at one. Thanks!

> Another option might be to substitute the default output format "png16m"
> (best suited for 24 bit RGB displays) with any of "pnggray" for greyscale,
> "png256" for 8-bit colour, "png16" for 4-bit colour, or "pngmono" for
> black-and-white. They all won't show the same amount of anti-aliasing
> power!

Interesting. I think I'll stick with the default here, because that's the
kind of display I'm using, but I may try the others later on.

Thanks a lot!

Amy

-- 
Tehee quod she, and clapte the wyndow to.
                -- Geoffrey Chaucer

Re: doc-view: Document quality and find-file

[Peter Dyballa]

Am 17.09.2007 um 20:59 schrieb Amy Templeton:

>  article will be scanned

So your PDFs are not generated by some text system, but scanned pages  
from books or magazines! This of course explains that the characters  
will look ragged: printing is not perfect and scanning is even  
farther away from perfection. I presume the corners of the characters  
might look better when you use a resolution that is one half, one  
quarter, or one eighth etc. of the original scan's resolution. This  
way small distortions will get averaged. And this effect allows to  
reduce the amount of anti-aliasing Ghostscript uses to create the PNG  
picture. And depending on the actual bit depth of the scan you might  
adjust the PNG device for which gs converts the PDF file. It's  
useless to expand an 8-bit image to 24 bits.

You know that you can't search a text in such a scan? OTOH, an OCR  
software could improve the scans ...

--
Greetings

   Pete

To be is to do.
                        -- I. Kant
To do is to be.
                        -- A. Sartre
Yabba-Dabba-Doo!
                        -- F. Flintstone

Re: doc-view: Document quality and find-file

[Amy Templeton]

Peter Dyballa <Peter_Dyballa@Web.DE> wrote:
> Am 17.09.2007 um 20:59 schrieb Amy Templeton:

> >  article will be scanned

> So your PDFs are not generated by some text system, but scanned pages from
> books or magazines! This of course explains that the characters will look
> ragged: printing is not perfect and scanning is even farther away from
> perfection. I presume the corners of the characters might look better when
> you use a resolution that is one half, one quarter, or one eighth etc. of
> the original scan's resolution. This way small distortions will get
> averaged. And this effect allows to reduce the amount of anti-aliasing
> Ghostscript uses to create the PNG picture. 

Ah, of course. I guess that should've occurred to me. However, reduced sizes
just seem to make it worse, oddly, whereas larger ones make it easier to
read. I don't presume to have an explanation.

> And depending on the actual bit depth of the scan you might adjust the PNG
> device for which gs converts the PDF file. It's useless to expand an 8-bit
> image to 24 bits.

Fair enough. I'll look into it.

> You know that you can't search a text in such a scan?

Oh, you're right. I can do it in a lot of the articles I read, though,
because they're from academic journals' sites, which I guess do things
right.

> OTOH, an OCR software could improve the scans ...

OCR? I'm not familiar with that term. Also, I doubt I'd be able to convince
my school to switch to anything decent...I mean, the campus computer
repair/maintenance/IT people *brag* on their website that all the software
they use and that they provide for students is proprietary. This is extra
weird because in most other realms, my school is fairly activistic as these
things go and so actually does a surprising number of things *right* (for
example, working to provide handicapped access even to older buildings). But
all that's another story altogether. Thanks for the tip, though.

Amy

-- 
If you are honest because honesty is the best policy, your honesty is
corrupt.

Re: doc-view: Document quality and find-file

[Peter Dyballa]

Am 18.09.2007 um 00:15 schrieb Amy Templeton:

> However, reduced sizes
> just seem to make it worse, oddly, whereas larger ones make it  
> easier to
> read. I don't presume to have an explanation.

Can you send me such a representative scan?

>>
> OCR? I'm not familiar with that term.


Optical Character Recognition. A software finds characters from the  
black spots and white gaps and creates words out of them. Sometimes  
it can be funny ... A good OCR software creates a very small text  
document from a large scan of some MB (compare it with pdftotext,  
though it is much simpler) and even can preserve elements of the  
original layout (columns, paragraphs, headers, footnotes).

--
Greetings

   Pete

To most people solutions mean finding the answers. But to chemists  
solutions
are things that are still all mixed up.

Re: doc-view: Document quality and find-file

[Amy Templeton]

Peter Dyballa <Peter_Dyballa@Web.DE> wrote:
> Am 18.09.2007 um 00:15 schrieb Amy Templeton:

> > However, reduced sizes just seem to make it worse, oddly, whereas larger
> > ones make it easier to read. I don't presume to have an explanation.

> Can you send me such a representative scan?

I thought I'd replied to this, but it looks like I forgot to. Upon further
examination, it turns out you were right--the characters do become nicer at
smaller sizes, but I personally do still find larger ones easier to read
because in these particular cases the text is small to begin with.

Thanks again,
Amy

-- 
November, n.:
        The eleventh twelfth of a weariness.
                -- Ambrose Bierce, "The Devil's Dictionary"

Re: doc-view: Document quality and find-file

[Peter Dyballa]

Am 17.09.2007 um 17:17 schrieb Amy Templeton:

> Anyway, I have a couple of questions.

And I have forgotten to mention two things:

  • you don't need to customise the variable for this session for  
each of the tests, you can do the conversion from the command line in  
*shell* buffer and then simply view the resulting file from dired,  
and if you see kind of garbage then just type C-c C-c and you'll be  
able to make a picture from this garbage;

  • if you make the tests with doc-view, then you'll need to clear  
the cache (M-x doc-view-clear-cache RET) before every test or you'll  
see the picture from the previous conversion, because no new  
conversion was performed.

--
Greetings

   Pete

There are three types of people in this world: those who can count,  
and those who cannot.

Re: doc-view: Document quality and find-file

[Peter Dyballa]

Am 17.09.2007 um 17:17 schrieb Amy Templeton:

> (2)  When using "M-x doc-view" to find a file, Emacs tries very  
> hard to
>      ignore PDF files

Would it make sense to add some file types to file-name-handler-alist  
and make doc-view their handler? Then, I think, a simple v in dired  
buffer would invoke some doc-view method for exactly this file and  
file name completion would not be needed ...

--
Mit friedvollen Grüßen

   Pete

Hupende Autos beißen nicht.

Re: doc-view: Document quality and find-file

[Amy Templeton]

> Peter Dyballa <Peter_Dyballa@Web.DE> wrote:
> Am 17.09.2007 um 17:17 schrieb Amy Templeton:
> > (2) When using "M-x doc-view" to find a file, Emacs tries very hard to
> > ignore PDF files

> Would it make sense to add some file types to file-name-handler-alist and
> make doc-view their handler? Then, I think, a simple v in dired buffer
> would invoke some doc-view method for exactly this file and file name
> completion would not be needed ...

Well, when I'm using dired I just use the doc-view-dired command (bound to
C-c d), but thanks for the suggestion. It's just that sometimes I don't
bother with dired if I'm just opening one file. So not a big deal, just a
minor annoyance.

Thanks again,
Amy

-- 
The  White Rabbit put on his spectacles.
        "Where shall  I  begin, please your Majesty ?" he asked.
        "Begin at the beginning,", the King said, very gravely, "and go on
till you come to the end: then stop."
                -- Lewis Carroll

Re: doc-view: Document quality and find-file

[Tassilo Horn]

Peter Dyballa <Peter_Dyballa@Web.DE> writes:

Hi Peter, hi Amy!

Sorry for the late answer, but I was in holidays and had a lot of other
stuff to do.

>> (2)  When using "M-x doc-view" to find a file, Emacs tries very hard to
>>      ignore PDF files

Indeed, that's very bad.  I've fixed it.

> Would it make sense to add some file types to file-name-handler-alist
> and make doc-view their handler?

No, I think those handlers are for other use-cases and much to complex
here.

> Then, I think, a simple v in dired buffer would invoke some doc-view
> method for exactly this file and file name completion would not be
> needed ...

I advised `view-file'.  If you hit `v' on a pdf, ps or dvi file now,
you'll be asked if you want to use doc-view instead of view-file.

The new version is in my git-repository (see on www.tsdh.de) and I've
sent a patch to emacs-devel.

Bye,
Tassilo
-- 
      "OS's and GUI's come and go, only Emacs has lasting power."
          Per Abrahamsen in <rjbsysc7n1.fsf@zuse.dina.kvl.dk>

Re: doc-view: Document quality and find-file

[Xavier Maillard]

   The new version is in my git-repository (see on www.tsdh.de) and I've
   sent a patch to emacs-devel.

Has it been installed into the CVS ?

	Xavier
-- 
http://www.gnu.org
http://www.april.org
http://www.lolica.org

Re: doc-view: Document quality and find-file

[Peter Dyballa]

Am 04.10.2007 um 03:00 schrieb Xavier Maillard:

> Has it been installed into the CVS ?

Which or whose CVS?

--
Greetings

   Pete

To most people solutions mean finding the answers. But to chemists  
solutions
are things that are still all mixed up.

Re: doc-view: Document quality and find-file

[Tassilo Horn]

Peter Dyballa <Peter_Dyballa@Web.DE> writes:

Hi Peter,

>> Has it been installed into the CVS ?
>
> Which or whose CVS?

GNU Emacs' official CVS repository at Savannah.

Bye,
Tassilo
-- 
Chuck Norris CAN believe it's not butter. 

Re: doc-view: Document quality and find-file

[Tassilo Horn]

Xavier Maillard <xma@gnu.org> writes:

Hi Xavier,

>    The new version is in my git-repository (see on www.tsdh.de) and
>    I've sent a patch to emacs-devel.
>
> Has it been installed into the CVS ?

The changes to doc-view.el (don't hide pdf/dvi files in filename
completion are in CVS now.  The change in dired-view-file which asks to
open ps/pdf/dvi files with doc-view instead of view-file is still
pending.

Bye,
Tassilo
-- 
People sometimes  ask me if it  is a sin in  the Church of  Emacs to use
vi. Using a free  version of vi is not a sin; it  is a penance. So happy
hacking. (Richard M. Stallman)

Re: doc-view: Document quality and find-file

[Amy Templeton]

Tassilo Horn <tassilo@member.fsf.org> wrote:
> Sorry for the late answer, but I was in holidays and had a lot of other
> stuff to do.

Fair enough.

> > Amy Templeton <amy.g.templeton@gmail.com> wrote:
> >> (2)  When using "M-x doc-view" to find a file, Emacs tries very hard to
> >>      ignore PDF files

> Indeed, that's very bad.  I've fixed it.

Great!

> Peter Dyballa <Peter_Dyballa@Web.DE> writes:
> > Would it make sense to add some file types to file-name-handler-alist
> > and make doc-view their handler?

> No, I think those handlers are for other use-cases and much to complex
> here.

> > Then, I think, a simple v in dired buffer would invoke some doc-view
> > method for exactly this file and file name completion would not be
> > needed ...

> I advised `view-file'.  If you hit `v' on a pdf, ps or dvi file now,
> you'll be asked if you want to use doc-view instead of view-file.

Ooh, that seems like a good feature, and actually makes me wonder whether
something like AUCTeX integration might be possible at some point (modifying
the command to view a file so that it could optionally be with doc-view. Of
course, I haven't actually looked at that code yet but it could be cool.

Amy

-- 
There is nothing which cannot be answered by means of my doctrine," said a
monk, coming into a teahouse where Nasrudin sat.
        "And yet just a short time ago, I was challenged by a scholar with
an unanswerable question," said Nasrudin.
        "I could have answered it if I had been there."
        "Very well. He asked, 'Why are you breaking into my house in the
middle of the night?'"

Re: doc-view: Document quality and find-file

[Tassilo Horn]

Amy Templeton <amy.g.templeton@gmail.com> writes:

Hi Amy,

>> I advised `view-file'.  If you hit `v' on a pdf, ps or dvi file now,
>> you'll be asked if you want to use doc-view instead of view-file.
>
> Ooh, that seems like a good feature,

BTW, I've changed dired-view-file instead of this advice.  Advices
aren't very popular with a lot of emacs devs.  Unfortunately the change
is still pending till RMS gives his ok.

> and actually makes me wonder whether something like AUCTeX integration
> might be possible at some point (modifying the command to view a file
> so that it could optionally be with doc-view. Of course, I haven't
> actually looked at that code yet but it could be cool.

That should be possible, but I'm not sure if doc-view is a good viewer
for documents you're currently editing because each time it would need
to convert the whole document again.  For small docs it might be ok,
anyway.

Bye,
Tassilo