* Re: [Emacs-diffs] master c4782ea: Improve and extend filepos-to-bufferpos
[not found] ` <E1Z5Yc4-0005ti-Ic@vcs.savannah.gnu.org>
@ 2015-06-19 3:17 ` Stefan Monnier
2015-06-19 6:59 ` Eli Zaretskii
0 siblings, 1 reply; 3+ messages in thread
From: Stefan Monnier @ 2015-06-19 3:17 UTC (permalink / raw)
To: emacs-devel; +Cc: Eli Zaretskii
> + (if (<= byte eol-offset)
> + (setq pos (point-min))
> + (setq pos (point-max))))
Aka (setq pos (if (<= byte eol-offset) (point-min) (point-max)))
> (let ((eol (coding-system-eol-type coding-system))
> (type (coding-system-type coding-system))
> + (base (coding-system-base coding-system))
> (pm (save-restriction (widen) (point-min))))
> + (and (eq type 'utf-8-emacs)
> + (setq type 'utf-8))
(coding-system-type 'utf-8-emacs) returns `utf-8', so how/when can
`type' be `utf-8-emacs'?
> + (and (eq type 'utf-8)
> + ;; Any post-read/pre-write conversions mean it's not really UTF-8.
> + (not (null (coding-system-get coding-system :pos-read-conversion)))
> + (setq type 'not-utf-8))
I guess this also applies for latin-N and utf-16, IOW for any value of
`type', right?
> + (and (not (eq type 'utf-8))
> + (eq quality 'exact)
> + (setq type 'use-exact))
IIUC this makes us use the slow exact code for latin-N. Why is it needed?
> + (`utf-16
> + ;; Account for BOM, which is always 2 bytes in UTF-16.
> + (setq byte (- byte 2))
Should that only be done for utf1-16B-with-signature?
> + ;; In approximate mode, assume all characters are within the
> + ;; BMP, i.e. take up 2 bytes.
> + (setq byte (/ byte 2))
> + (if (= eol 1)
> + (filepos-to-bufferpos--dos (+ pm byte) #'byte-to-position)
> + (byte-to-position (+ pm byte))))
Shouldn't this use `identity' rather than `byte-to-position'?
Stefan
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [Emacs-diffs] master c4782ea: Improve and extend filepos-to-bufferpos
2015-06-19 3:17 ` [Emacs-diffs] master c4782ea: Improve and extend filepos-to-bufferpos Stefan Monnier
@ 2015-06-19 6:59 ` Eli Zaretskii
2015-06-19 8:20 ` Eli Zaretskii
0 siblings, 1 reply; 3+ messages in thread
From: Eli Zaretskii @ 2015-06-19 6:59 UTC (permalink / raw)
To: Stefan Monnier; +Cc: emacs-devel
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>
> Date: Thu, 18 Jun 2015 23:17:28 -0400
>
> > + (if (<= byte eol-offset)
> > + (setq pos (point-min))
> > + (setq pos (point-max))))
>
> Aka (setq pos (if (<= byte eol-offset) (point-min) (point-max)))
Yes, but my code is clearer, IMO.
> > (let ((eol (coding-system-eol-type coding-system))
> > (type (coding-system-type coding-system))
> > + (base (coding-system-base coding-system))
> > (pm (save-restriction (widen) (point-min))))
> > + (and (eq type 'utf-8-emacs)
> > + (setq type 'utf-8))
>
> (coding-system-type 'utf-8-emacs) returns `utf-8', so how/when can
> `type' be `utf-8-emacs'?
Never. I guess I got confused with coding-system-base.
>
> > + (and (eq type 'utf-8)
> > + ;; Any post-read/pre-write conversions mean it's not really UTF-8.
> > + (not (null (coding-system-get coding-system :pos-read-conversion)))
> > + (setq type 'not-utf-8))
>
> I guess this also applies for latin-N and utf-16, IOW for any value of
> `type', right?
Not really, no. UTF-8 is special here, in that we believe we know how
to compute the byte position exactly, which is not true when there are
conversions. Some profoundly non-UTF-8 encodings have that type, but
then apply conversions that make them something very different.
> > + (and (not (eq type 'utf-8))
> > + (eq quality 'exact)
> > + (setq type 'use-exact))
>
> IIUC this makes us use the slow exact code for latin-N.
Only if they ask for 'exact'.
> Why is it needed?
They asked for it, didn't they?
A more important problem is that we handle type before accuracy, so
when exact is requested, the type checks should be bypassed, except
with UTF-8.
> > + (`utf-16
> > + ;; Account for BOM, which is always 2 bytes in UTF-16.
> > + (setq byte (- byte 2))
>
> Should that only be done for utf1-16B-with-signature?
Do we have a UTF-16 encoding without a signature?
> > + ;; In approximate mode, assume all characters are within the
> > + ;; BMP, i.e. take up 2 bytes.
> > + (setq byte (/ byte 2))
> > + (if (= eol 1)
> > + (filepos-to-bufferpos--dos (+ pm byte) #'byte-to-position)
> > + (byte-to-position (+ pm byte))))
>
> Shouldn't this use `identity' rather than `byte-to-position'?
This code tested OK for me, feel free to change if you have a test
that fails.
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [Emacs-diffs] master c4782ea: Improve and extend filepos-to-bufferpos
2015-06-19 6:59 ` Eli Zaretskii
@ 2015-06-19 8:20 ` Eli Zaretskii
0 siblings, 0 replies; 3+ messages in thread
From: Eli Zaretskii @ 2015-06-19 8:20 UTC (permalink / raw)
To: monnier; +Cc: emacs-devel
I've fixed in commit a2bb6c7 a few of the issues you pointed out:
> Date: Fri, 19 Jun 2015 09:59:38 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
>
> > > (let ((eol (coding-system-eol-type coding-system))
> > > (type (coding-system-type coding-system))
> > > + (base (coding-system-base coding-system))
> > > (pm (save-restriction (widen) (point-min))))
> > > + (and (eq type 'utf-8-emacs)
> > > + (setq type 'utf-8))
> >
> > (coding-system-type 'utf-8-emacs) returns `utf-8', so how/when can
> > `type' be `utf-8-emacs'?
>
> Never. I guess I got confused with coding-system-base.
This one.
> > > + (and (not (eq type 'utf-8))
> > > + (eq quality 'exact)
> > > + (setq type 'use-exact))
> >
> > IIUC this makes us use the slow exact code for latin-N.
>
> Only if they ask for 'exact'.
>
> > Why is it needed?
And this one.
> > > + (`utf-16
> > > + ;; Account for BOM, which is always 2 bytes in UTF-16.
> > > + (setq byte (- byte 2))
> >
> > Should that only be done for utf1-16B-with-signature?
>
> Do we have a UTF-16 encoding without a signature?
And this one. (Yes, we do have such systems, I just missed them when
I reviewed all the definitions.)
> > > + ;; In approximate mode, assume all characters are within the
> > > + ;; BMP, i.e. take up 2 bytes.
> > > + (setq byte (/ byte 2))
> > > + (if (= eol 1)
> > > + (filepos-to-bufferpos--dos (+ pm byte) #'byte-to-position)
> > > + (byte-to-position (+ pm byte))))
> >
> > Shouldn't this use `identity' rather than `byte-to-position'?
>
> This code tested OK for me, feel free to change if you have a test
> that fails.
And this one; I believe you are right here, and my testing was
probably limited to ASCII-only files.
Thanks.
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2015-06-19 8:20 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <20150618120808.22624.13860@vcs.savannah.gnu.org>
[not found] ` <E1Z5Yc4-0005ti-Ic@vcs.savannah.gnu.org>
2015-06-19 3:17 ` [Emacs-diffs] master c4782ea: Improve and extend filepos-to-bufferpos Stefan Monnier
2015-06-19 6:59 ` Eli Zaretskii
2015-06-19 8:20 ` Eli Zaretskii
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.