Magic MIME

Magic MIME

[Juri Linkov]

Visiting an Arc Lisp file in Emacs fails with the error
"Buffer format not recognized".

Even though Emacs doesn't have an auto-mode-alist entry
associated with Arc, it should visit it in fundamental-mode,
not to fail trying to open it as an archive.  Archive files
have a well-known signature, and using content-based file
type recognition rather than filename-based is more reliable.
This will also improve the recognition of image file formats.

Are there any obstacles to using magic-mime to determine the file
type in Emacs?  If linking libmagic is not possible, then maybe
we could parse the magic.mime file, and even run the `file'
command directly on a visited file?

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Magic MIME

[Stefan Monnier]

> not to fail trying to open it as an archive.  Archive files
> have a well-known signature, and using content-based file
> type recognition rather than filename-based is more reliable.

C'mon Juri, you know better than that.  We've been through it with the
image-recognition thingy, so it *should* be clear to everyone on this
list now that content based recognition is no better than filename-based
recognition (and vice versa).  We need to use both.


        Stefan

Re: Magic MIME

[Eli Zaretskii]

> From: Juri Linkov <juri@jurta.org>
> Date: Mon, 11 Feb 2008 01:18:41 +0200
> 
> Are there any obstacles to using magic-mime to determine the file
> type in Emacs?  If linking libmagic is not possible, then maybe
> we could parse the magic.mime file, and even run the `file'
> command directly on a visited file?

Whatever the decision is, please avoid relying in external commands
that are not available on all supported platforms.  `file' isn't.

Emacs should be powerful enough to be able to visit files without
requiring an external tool; if it needs to consult some database for
that, it should be easy enough to include such a data base in the
Emacs distribution, perhaps even in a form that is convenient for
reading in Lisp.

Re: Magic MIME

[Juri Linkov]

>> not to fail trying to open it as an archive.  Archive files
>> have a well-known signature, and using content-based file
>> type recognition rather than filename-based is more reliable.
>
> C'mon Juri, you know better than that.  We've been through it with the
> image-recognition thingy, so it *should* be clear to everyone on this
> list now that content based recognition is no better than filename-based
> recognition (and vice versa).  We need to use both.

Yes, I remember what a mess was the recognition of image files.  But I think
that for other binary formats this could by simplified.  If, say, a file
doesn't start with the "0x8080ffff 0x0000081a" signature there is no sense
to try treating it as an arc archive even if it has the .arc extension.

IIRC, the decision for the image file extensions taking precedence over
the image file content was based solely on the security holes that some
image libraries reportedly have.  So that the user downloaded a .c file
somewhere from the Internet actually with an image inside should not take
the risk of opening it in image-mode in Emacs.

I don't know if such precautions are relevant for other binary formats
like archives.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Magic MIME

[Juri Linkov]

>> Are there any obstacles to using magic-mime to determine the file
>> type in Emacs?  If linking libmagic is not possible, then maybe
>> we could parse the magic.mime file, and even run the `file'
>> command directly on a visited file?
>
> Whatever the decision is, please avoid relying in external commands
> that are not available on all supported platforms.  `file' isn't.

Emacs could follow the principle of graceful degradation here: if there
is no necessary external command or mime database on the system, then
it will provide less reliable result that will be no worse than the
current without using the magic.mime based file type recognition.

> Emacs should be powerful enough to be able to visit files without
> requiring an external tool; if it needs to consult some database for
> that, it should be easy enough to include such a data base in the
> Emacs distribution, perhaps even in a form that is convenient for
> reading in Lisp.

This is one option  However, I'm not sure if licence conditions of the
mime-magic database allow including it to Emacs.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: Magic MIME

[Eli Zaretskii]

> From: Juri Linkov <juri@jurta.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 11 Feb 2008 22:59:38 +0200
> 
> > Emacs should be powerful enough to be able to visit files without
> > requiring an external tool; if it needs to consult some database for
> > that, it should be easy enough to include such a data base in the
> > Emacs distribution, perhaps even in a form that is convenient for
> > reading in Lisp.
> 
> This is one option  However, I'm not sure if licence conditions of the
> mime-magic database allow including it to Emacs.

Even if the license precludes that, it can only prevent us from
including the data base verbatim.  If we create our own data base,
with equivalent contents, but in utterly different form, no license
can prevent that, I think, especially if the person who writes that
data base for Emacs never looks at the original data base, but instead
works from a description written by someone else (that someone could
look at mime-magic to produce the description).

Re: Magic MIME

[Stefan Monnier]

> Yes, I remember what a mess was the recognition of image files.  But I think
> that for other binary formats this could by simplified.  If, say, a file
> doesn't start with the "0x8080ffff 0x0000081a" signature there is no sense
> to try treating it as an arc archive even if it has the .arc extension.

Agreed.  But just because a file starts with "0x8080ffff 0x0000081a"
shouldn't be enough of a reason to treat it as an arc archive: we need to
look at the filename as well.

> IIRC, the decision for the image file extensions taking precedence over
> the image file content was based solely on the security holes that some
> image libraries reportedly have.

I don't think so.  I do remember that it was decided not to introduce
the two-sided (file-name and file-contents) method so close to
the release.  So now would be a better time.

We could associate "\\.arc\\'" with '(lisp-arc-mode archive-mode)
and associate "0x8080ffff 0x0000081a" to archive-mode (and some other
pseudo-magic thingy for Arc-the-language).


        Stefan

Re: Magic MIME

[Richard Stallman]

    Emacs could follow the principle of graceful degradation here: if there
    is no necessary external command or mime database on the system, then
    it will provide less reliable result that will be no worse than the
    current without using the magic.mime based file type recognition.

Unless that gets us big benefits, it is better if we don't use it.
There is an advantage in keeping Emacs self-contained.