Time not representable

Time not representable

[Carsten Dominik]

Hi everyone,

in the Org-mode mailing list, an old issue just came up again:
Some dates are not representable in Emacs when using functions
like encode-time.

The problem has to do with integer representations.

From the past I know that only times between 1970 and 2038 were
representable (that may have been Emacs 20 or something like this).

Recently I have noticed that my system now *can* represent dates
past 2038

   (encode-time 0 0 0 1 1 2040) => (33706 28784)

attesting to the fact that time is represented now with more than
32 bits.

Another data point is that on a different system, Bastien has found
that he can get times before 1970 represented, but not after 2038.
So apparently his system is using something else to represent time.

I can look up all the details about the systems if needed, but the
question I have is more general:

Why is it that these things are different on different systems?  Is
this under the control of Emacs, or does this depend on system
libraries which are being used?

Are there plans to use a time representation which will consistently
allow dates past 2038 and, if possible also before 1970?

Any pointers to educate me about these issues would be appreciated.

Thanks!

- Carsten

Re: Time not representable

[Andreas Schwab]

Carsten Dominik <carsten.dominik@gmail.com> writes:

> Another data point is that on a different system, Bastien has found
> that he can get times before 1970 represented, but not after 2038.
> So apparently his system is using something else to represent time.

If your system's time_t is a signed 32-bit integer then your system is
able to represent times between 1901-12-13 20:45:53 UTC and 2038-01-19
03:14:07 UTC.  If your system's time_t is an unsigned 32-bit integer
your system can represent times between 1970-01-01 00:00:00 UTC and
2106-02-07 06:28:15 UTC.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Time not representable

[Paul Eggert]

On 03/11/2011 05:01 AM, Carsten Dominik wrote:
> Why is it that these things are different on different systems?  Is
> this under the control of Emacs, or does this depend on system
> libraries which are being used?

Both.

Time stamps on most modern systems have nanosecond resolution,
but Emacs's internal time stamps only use microsecond resolution.
This means (for example) that Emacs cannot determine that
one file is newer than another, even when it is newer.
This bug really should get fixed at some point.

In addition to resolution problems, there are also the
range problems that you alluded to.  time_t is usually
a signed 32-bit or 64-bit integer; a few systems use unsigned
integers, and several use signed integers but do not allow
negative values.  (Emacs in theory could use 32-bit EMACS_INT
internally on a system with 64-bit time_t, but I expect that
combination is rare.)

You have to be careful about inferring ranges from behavior,
though, because when Emacs converts times, it sometimes doesn't check
for overflow.  This means you can get undefined behavior if you use
time stamps that cannot be represented internally.
For example, (encode-time 0 0 0 1 1 1152921504606846976)
returns the obviously-bogus value (-948597 62170)
on my RHEL 5.5 x86-64 host, whereas it correctly reports
a "Specified time is not representable" error on
my Ubuntu 10.10 x86 host.  This is a bug that should get
fixed at some point too, though I expect it's less important
in practice than the nanosecond-resolution bug.

Re: Time not representable

[Carsten Dominik]

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> 
> On 03/11/2011 05:01 AM, Carsten Dominik wrote:
> > Why is it that these things are different on different systems?  Is
> > this under the control of Emacs, or does this depend on system
> > libraries which are being used?
> 
> Both.
> 
> Time stamps on most modern systems have nanosecond resolution,
> but Emacs's internal time stamps only use microsecond resolution.
> This means (for example) that Emacs cannot determine that
> one file is newer than another, even when it is newer.
> This bug really should get fixed at some point.
> 
> In addition to resolution problems, there are also the
> range problems that you alluded to.  time_t is usually
> a signed 32-bit or 64-bit integer; a few systems use unsigned
> integers, and several use signed integers but do not allow
> negative values.  (Emacs in theory could use 32-bit EMACS_INT
> internally on a system with 64-bit time_t, but I expect that
> combination is rare.)
> 
> You have to be careful about inferring ranges from behavior,
> though, because when Emacs converts times, it sometimes doesn't check
> for overflow.  This means you can get undefined behavior if you use
> time stamps that cannot be represented internally.
> For example, (encode-time 0 0 0 1 1 1152921504606846976)
> returns the obviously-bogus value (-948597 62170)
> on my RHEL 5.5 x86-64 host, whereas it correctly reports
> a "Specified time is not representable" error on
> my Ubuntu 10.10 x86 host.  This is a bug that should get
> fixed at some point too, though I expect it's less important
> in practice than the nanosecond-resolution bug.


Hi Paul (and Andreas)

thanks for the information.  So if a system uses a signed
64 bit integer, then the representation problems will be
fixed automatically, without further changes in Emacs?
This does sound hopeful!  Is there anyone here who has
a 64 bit system and con confirm that encode-time can
handle times both before 1970 and after 2040 without
any problems?

- Carsten

Re: Time not representable

[Paul Eggert]

On 03/12/2011 02:13 PM, Carsten Dominik wrote:
> So if a system uses a signed 64 bit integer, then the representation
> problems will be fixed automatically, without further changes in
> Emacs?

Sorry, no.  Perhaps my explanation wasn't clear?

> Is there anyone here who has a 64 bit system and con confirm that
> encode-time can handle times both before 1970 and after 2040 without 
> any problems?

I have one (RHEL 5.5, x86-64), and can verify that it
can handle times well outside the 1970-2038 range;
but there are problems, as mentioned in my previous message.
For example, (decode-time (encode-time 0 0 0 1 1 2147485548))
returns (0 0 0 1 1 -2147481748 4 nil -28378) on my host, and
this is obviously bogus.

Re: Time not representable

[Paul Eggert]

Following up on this, I just now committed a change
(bzr 103643) that should fix the problems noted in this
thread for time stamp overflow.  (This may require
minor changes to Windows makefiles to capture new
dependencies, but it shouldn't break Windows builds.)

I'd also like to fix the time stamp resolution problems
by adding more information to the output of current-time
and similar functions.  For example, if the time is
2011-03-13 06:47:51.191380874 UTC, (current-time) currently
yields (19836 26775 191380), losing the bottom 3 digits.
I'd like to change it to yield (19836 26775 191380 874000)
so that no information is lost.  That is, I'd like to
extend the time stamp format to have a picoseconds count
(in addition to the current microseconds count) on hosts
that have higher-resolution time stamps.  No host currently
keeps time to picosecond resolution, but many do track
nanoseconds so the extended format will be a win.

This change would affect all functions that use these
internal time stamps, namely float-time, format-time-string,
decode-time, current-time-string, current-time-zone,
set-file-times, current-time, file-attributes,
encode-time, and visited-file-modtime.  Current-format
internal time stamps would continue to work.

Re: Time not representable

[Eli Zaretskii]

> Date: Sat, 12 Mar 2011 23:05:37 -0800
> From: Paul Eggert <eggert@cs.ucla.edu>
> Cc: emacs-devel@gnu.org
> 
> This may require minor changes to Windows makefiles to capture new
> dependencies

Done.

Re: Time not representable

[carsten Dominik]

Paul Eggert <eggert <at> cs.ucla.edu> writes:

> 
> Following up on this, I just now committed a change
> (bzr 103643) that should fix the problems noted in this
> thread for time stamp overflow.  (This may require
> minor changes to Windows makefiles to capture new
> dependencies, but it shouldn't break Windows builds.)
> 


Hi,

thank you for your work!

- Carsten

> 
>