dates before 1970

dates before 1970

[Eric S Fraga]

This is a sort of bug report but possibly more a curiosity...

I imagine this has something to do with time 0 in Unix but I cannot seem
to be able to enter any date earlier than 1 Jan 1970 using C-c! (say).
However, once I have entered a date (later than that), I can use
S-<down> on the year to get to the date I want.  This seems rather
inconsistent?

To be precise, I get the wrong date recorded if I try:

  C-c ! 1968-12-10 RET

(where C-c ! is =org-time-stamp-inactive=).
The result is =[2011-12-10 Sat]=

The bug is not so much that I cannot input dates I want but that the
inactive timestamp generated is *incorrect* and yet there is no error
message.

Thanks,
eric

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.25.gaaf0b.dirty)

Re: dates before 1970

[Nick Dokos]

Eric S Fraga <e.fraga@ucl.ac.uk> wrote:

> This is a sort of bug report but possibly more a curiosity...
> 
> I imagine this has something to do with time 0 in Unix but I cannot seem
> to be able to enter any date earlier than 1 Jan 1970 using C-c! (say).
> However, once I have entered a date (later than that), I can use
> S-<down> on the year to get to the date I want.  This seems rather
> inconsistent?
> 
> To be precise, I get the wrong date recorded if I try:
> 
>   C-c ! 1968-12-10 RET
> 
> (where C-c ! is =org-time-stamp-inactive=).
> The result is =[2011-12-10 Sat]=
> 
> The bug is not so much that I cannot input dates I want but that the
> inactive timestamp generated is *incorrect* and yet there is no error
> message.
> 

Good one! The culprit is org-read-date-analyze which near the end contains
this snippet of code:

,----
|     ...
|     (if (< year 100) (setq year (+ 2000 year)))
|     (if (< year 1970) (setq year (nth 5 defdecode))) ; not representable
|     (setq org-read-date-analyze-futurep futurep)
|     (list second minute hour day month year)))
`----

The trouble is that the caller (org-read-date) takes the result and
does a round-trip through the emacs time encode/decode functions to make
sure the result is sane. Dates before 1970 would break that (I get (0 9
10 26 11 2033 6 nil -18000)) so it seems it wraps around to 2033 or so).

In addition, most callers of org-read-date call it with a non-nil
to-time argument: that makes it return an emacs-encoded time (which is
then manipulated as such and which I believe has to satisfy the >=1970
requirement).

So I'd guess raising an exception might be the simplest way to deal with
this. Here's a patch to try out:

--8<---------------cut here---------------start------------->8---
diff --git a/lisp/org.el b/lisp/org.el
index 92f2406..b9acf11 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -14718,7 +14718,8 @@ user."
 	     (nth 2 tl))
 	(setq org-time-was-given t))
     (if (< year 100) (setq year (+ 2000 year)))
-    (if (< year 1970) (setq year (nth 5 defdecode))) ; not representable
+;    (if (< year 1970) (setq year (nth 5 defdecode))) ; not representable
+    (if (< year 1970) (error "Year must be >= 1970"))
     (setq org-read-date-analyze-futurep futurep)
     (list second minute hour day month year)))
--8<---------------cut here---------------end--------------->8---

I think it does not break anything but I'm not sure I like it much.

Patchwork note: this should not be applied without a lot more thought
and experimentation.

Nick

Re: dates before 1970

[Bastien]

Hi,

Nick Dokos <nicholas.dokos@hp.com> writes:

> So I'd guess raising an exception might be the simplest way to deal with
> this. Here's a patch to try out:

This patch has side-effects that Carsten have been recently exploring a
bit.  Those side-effects seem to depend on how Emacs has been compiled.

For now it's best to stick to this restriction.

Best,

-- 
 Bastien

Re: dates before 1970

[Eric S Fraga]

Nick Dokos <nicholas.dokos@hp.com> writes:

> Eric S Fraga <e.fraga@ucl.ac.uk> wrote:
>
>> This is a sort of bug report but possibly more a curiosity...
>> 
>> I imagine this has something to do with time 0 in Unix but I cannot seem
>> to be able to enter any date earlier than 1 Jan 1970 using C-c! (say).
>> However, once I have entered a date (later than that), I can use
>> S-<down> on the year to get to the date I want.  This seems rather
>> inconsistent?
>> 
>> To be precise, I get the wrong date recorded if I try:
>> 
>>   C-c ! 1968-12-10 RET
>> 
>> (where C-c ! is =org-time-stamp-inactive=).
>> The result is =[2011-12-10 Sat]=
>> 
>> The bug is not so much that I cannot input dates I want but that the
>> inactive timestamp generated is *incorrect* and yet there is no error
>> message.
>> 
>
> Good one! The culprit is org-read-date-analyze which near the end contains
> this snippet of code:
>
> ,----
> |     ...
> |     (if (< year 100) (setq year (+ 2000 year)))
> |     (if (< year 1970) (setq year (nth 5 defdecode))) ; not representable
> |     (setq org-read-date-analyze-futurep futurep)
> |     (list second minute hour day month year)))
> `----
>
> The trouble is that the caller (org-read-date) takes the result and
> does a round-trip through the emacs time encode/decode functions to make
> sure the result is sane. Dates before 1970 would break that (I get (0 9
> 10 26 11 2033 6 nil -18000)) so it seems it wraps around to 2033 or
> so).

Yes, that makes sense.

> In addition, most callers of org-read-date call it with a non-nil
> to-time argument: that makes it return an emacs-encoded time (which is
> then manipulated as such and which I believe has to satisfy the >=1970
> requirement).
>
> So I'd guess raising an exception might be the simplest way to deal with
> this. Here's a patch to try out:

This seems to work fine.  Thanks.

I am glad, however, that I can enter any date and then use the S-<down>
etc. keys to get the date I want.  Of course, I am not sure if anything
else in org breaks as a result...  org-sparse-tree with very old
scheduled dates seems to work.  Haven't tried much else and I would
guess few would notice?

Thanks again,
eric

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.27.gefa56.dirty)

Re: dates before 1970

[Carsten Dominik]

On Mar 11, 2011, at 9:31 AM, Bastien wrote:

> Hi,
> 
> Nick Dokos <nicholas.dokos@hp.com> writes:
> 
>> So I'd guess raising an exception might be the simplest way to deal with
>> this. Here's a patch to try out:
> 
> This patch has side-effects that Carsten have been recently exploring a
> bit.  Those side-effects seem to depend on how Emacs has been compiled.
> 
> For now it's best to stick to this restriction.

But beeping or so to alert the user that a date is being changed
behind his back might be a good idea.

I think we should ask on emacs-devel what the official position
of Emacs development is regarding non-representable times.

- Carsten

Re: dates before 1970

[Carsten Dominik]

On Mar 11, 2011, at 9:47 AM, Eric S Fraga wrote:

> Nick Dokos <nicholas.dokos@hp.com> writes:
> 
>> Eric S Fraga <e.fraga@ucl.ac.uk> wrote:
>> 
>>> This is a sort of bug report but possibly more a curiosity...
>>> 
>>> I imagine this has something to do with time 0 in Unix but I cannot seem
>>> to be able to enter any date earlier than 1 Jan 1970 using C-c! (say).
>>> However, once I have entered a date (later than that), I can use
>>> S-<down> on the year to get to the date I want.  This seems rather
>>> inconsistent?
>>> 
>>> To be precise, I get the wrong date recorded if I try:
>>> 
>>>  C-c ! 1968-12-10 RET
>>> 
>>> (where C-c ! is =org-time-stamp-inactive=).
>>> The result is =[2011-12-10 Sat]=
>>> 
>>> The bug is not so much that I cannot input dates I want but that the
>>> inactive timestamp generated is *incorrect* and yet there is no error
>>> message.
>>> 
>> 
>> Good one! The culprit is org-read-date-analyze which near the end contains
>> this snippet of code:
>> 
>> ,----
>> |     ...
>> |     (if (< year 100) (setq year (+ 2000 year)))
>> |     (if (< year 1970) (setq year (nth 5 defdecode))) ; not representable
>> |     (setq org-read-date-analyze-futurep futurep)
>> |     (list second minute hour day month year)))
>> `----
>> 
>> The trouble is that the caller (org-read-date) takes the result and
>> does a round-trip through the emacs time encode/decode functions to make
>> sure the result is sane. Dates before 1970 would break that (I get (0 9
>> 10 26 11 2033 6 nil -18000)) so it seems it wraps around to 2033 or
>> so).
> 
> Yes, that makes sense.
> 
>> In addition, most callers of org-read-date call it with a non-nil
>> to-time argument: that makes it return an emacs-encoded time (which is
>> then manipulated as such and which I believe has to satisfy the >=1970
>> requirement).
>> 
>> So I'd guess raising an exception might be the simplest way to deal with
>> this. Here's a patch to try out:
> 
> This seems to work fine.  Thanks.
> 
> I am glad, however, that I can enter any date and then use the S-<down>
> etc. keys to get the date I want.  Of course, I am not sure if anything
> else in org breaks as a result...  org-sparse-tree with very old
> scheduled dates seems to work.  Haven't tried much else and I would
> guess few would notice?

THis is exactly the point, that it depends on how Emacs was compiled, and what kind of integer is used in the date representation.  Signed or unsigend, 32 or 64 bits (I think).

For example, Bastien can represent dates before 1970. I cannot.
I can represent dates after 2038, Bastien cannot.

The work-around is to use diary sexps for dates before 1970, that seems to be safe.
And then hope that by 2038, all computers will use 64 bit integers....

- Carsten

Re: dates before 1970

[Eric S Fraga]

Carsten Dominik <carsten.dominik@gmail.com> writes:

[...]

> THis is exactly the point, that it depends on how Emacs was compiled,
> and what kind of integer is used in the date representation.  Signed
> or unsigend, 32 or 64 bits (I think).

Yes, that makes sense.

> For example, Bastien can represent dates before 1970. I cannot.
> I can represent dates after 2038, Bastien cannot.

I can do before 1970 but not after 2038.  Interesting.
-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.27.gefa56)

Re: dates before 1970

[Carsten Dominik]

Hi,

I asked in emacs-develop and got:

> by Andreas Schwabon 2011-03-11T13:19:43+00:00.
> 
> If your system's time-t is a signed 32-bit integer then your system is
> able to represent times between 1901-12-13 20:45:53 UTC and 2038-01-19
> 03:14:07 UTC. If your system's time-t is an unsigned 32-bit integer
> your system can represent times between 1970-01-01 00:00:00 UTC and
> 2106-02-07 06:28:15 UTC.
> Andreas.

So I am not sure what 64 bit systems do now or in the future, but
it seems that we need to live with a restriction for now.
Maybe this should be documented somewhere.

- Carsten


On Mar 11, 2011, at 1:00 PM, Eric S Fraga wrote:

> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
> [...]
> 
>> THis is exactly the point, that it depends on how Emacs was compiled,
>> and what kind of integer is used in the date representation.  Signed
>> or unsigend, 32 or 64 bits (I think).
> 
> Yes, that makes sense.
> 
>> For example, Bastien can represent dates before 1970. I cannot.
>> I can represent dates after 2038, Bastien cannot.
> 
> I can do before 1970 but not after 2038.  Interesting.
> -- 
> : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
> : using Org-mode version 7.5 (release_7.5.27.gefa56)

- Carsten

Re: dates before 1970

[Nick Dokos]

Eric S Fraga <e.fraga@ucl.ac.uk> wrote:

> > So I'd guess raising an exception might be the simplest way to deal with
> > this. Here's a patch to try out:
> 
> This seems to work fine.  Thanks.
> 

Maybe not - see Bastien's mail.

> I am glad, however, that I can enter any date and then use the S-<down>
> etc. keys to get the date I want.  Of course, I am not sure if anything
> else in org breaks as a result...  org-sparse-tree with very old
> scheduled dates seems to work.  Haven't tried much else and I would
> guess few would notice?
> 

That's the problem: one does not know whether the way from one
date to another passes through the quicksand of internal emacs time.
And as you say, any effects might escape notice.

Nick

Re: dates before 1970

[Nick Dokos]

Carsten Dominik <carsten.dominik@gmail.com> wrote:


> THis is exactly the point, that it depends on how Emacs was compiled, and what kind of integer is used in the date representation.  Signed or unsigend, 32 or 64 bits (I think).
> 
> For example, Bastien can represent dates before 1970. I cannot.
> I can represent dates after 2038, Bastien cannot.
> 
> The work-around is to use diary sexps for dates before 1970, that seems to be safe.
> And then hope that by 2038, all computers will use 64 bit integers....
> 

But it's even more than that, no? Emacs's time implementation
(current-time, encode/decode etc) would have to change. In fact, this
might be the most significant limitation right now: the values they pass
around are (hi16 lo16 ms) so they assume that time values are 32 bits,
no matter what the underlying implementation says. I use 64-bit Linux on
an x86-64 laptop and my time_t is 64 bits (but I don't know if it's
signed or unsigned). Time for some experimentation I guess...

Nick

Re: dates before 1970

[Gregor Zattler]

Hi Carsten, orgers,
* Carsten Dominik <carsten.dominik@gmail.com> [11. Mar. 2011]:
> I asked in emacs-develop and got:
> 
>> by Andreas Schwabon 2011-03-11T13:19:43+00:00.
>> 
>> If your system's time-t is a signed 32-bit integer then your system is
>> able to represent times between 1901-12-13 20:45:53 UTC and 2038-01-19
>> 03:14:07 UTC. If your system's time-t is an unsigned 32-bit integer
>> your system can represent times between 1970-01-01 00:00:00 UTC and
>> 2106-02-07 06:28:15 UTC.

Therefore it should be possible to either use a pre 1970 date or
a post 2038 date, but:

If I type (as Eric suggested in the starting message of this
thread) 
C-c ! 1968-12-10 RET 
I get 
[2011-12-10 Sa] 
(same as Eric).

But if I type
C-c ! 2040
I get an error message "specified time not representable".

So for me it's neither--nor but in different ways.  The second
way of failing is much better because I get feedback.



Ciao, Gregor
-- 
 -... --- .-. . -.. ..--.. ...-.-

Re: dates before 1970

[Robert Horn]

> So I am not sure what 64 bit systems do now or in the future, but
> it seems that we need to live with a restriction for now.
> Maybe this should be documented somewhere.
> 
> - Carsten

Most 64-bit systems use a 64-bit int.  All of the 64-bit Linux systems
that I've used use a signed 64-bit int.  Some systems use a 64-bit
unsigned int. Some use a double.  The only way to know for sure is to
look at their definition of time_t in time.h, as provided by the system.

http://en.wikipedia.org/wiki/Time_t is as good a starting point as any.

The precise words from the Open Group Base standard are:
	time_t and clock_t shall be integer or real-floating types.
The usage of time_t in various functions is specified, but range and
type is not defined.

R Horn

Re: dates before 1970

[Carsten Dominik]

[-- Attachment #1: Type: text/plain, Size: 2703 bytes --]

Hi everyone,

On 11.3.2011, at 09:31, Bastien wrote:

> Hi,
> 
> Nick Dokos <nicholas.dokos@hp.com> writes:
> 
>> So I'd guess raising an exception might be the simplest way to deal with
>> this. Here's a patch to try out:
> 
> This patch has side-effects that Carsten have been recently exploring a
> bit.  Those side-effects seem to depend on how Emacs has been compiled.
> 
> For now it's best to stick to this restriction.

I am attaching a patch which tries to implement some kind of a
solution for this problem.  The patch introduces a new variable
which will allow you to use dates outside the safe range 1970-2037
once you have convinced yourself that all Emacs implementation you
are likely to use now and in the future will support the extended
range.  To be clear, I do not recommend to make use of this, the danger
that you will find yourself on a system where this issue is not
resolved is not negligible.  That is also why I would recommend
the default `t' for this variable, and why I will continue to use diary-sexp
dates for the few dates where I need a greater range.

The patch also introduces a warning with a beep when Org has forced the year,
which, I think, was really the main concern in this thread.

Here is the docstring of the new variable - it explains what is going on.

(defcustom org-read-date-force-compatible-dates t
  "Should date/time prompt force dates that are guaranteed to work in Emacs?

Depending on the system Emacs is running on, certain dates cannot
be represented with the type used internally to represent time.
Dates between 1970-1-1 and 2038-1-1 can always be represented
correctly.  Some systems allow for earlier dates, some for later,
some for both.  One way to find out it to insert any date into an
Org buffer, putting the cursor on the year and hitting S-up and
S-down to test the range.

When this variable is set to t, the date/time prompt will not let
you specify dates outside the 1970-2037 range, so it is certain that
these dates will work in whatever version of Emacs you are
running, and also that you can move a file from one Emacs implementation
to another.  WHenever Org is forcing the year for you, it will display
a message and beep.

When this variable is nil, Org will check if the date is
representable in the specific Emacs implementation you are using.
If not, it will force a year, usually the current year, and beep
to remind you.  Currently this setting is not recommended because
the likelihood that you will open your Org files in an Emacs that
has limited date range is not negligible.

A workaround for this problem is to use diary sexp dates for time
stamps outside of this range."
  :group 'org-time
  :type 'boolean)




[-- Attachment #2: 0001-Improve-handling-of-years-with-limitations-on-repres.patch --]
[-- Type: application/octet-stream, Size: 5783 bytes --]

From 138615942fe12b40f8fbb1de5ead5ae5960032f3 Mon Sep 17 00:00:00 2001
From: Carsten Dominik <carsten.dominik@gmail.com>
Date: Sun, 13 Mar 2011 08:06:22 +0100
Subject: [PATCH] Improve handling of years with limitations on representable dates

* lisp/org.el (org-read-date-force-compatible-dates): New option.
(org-read-date, org-read-date-analyze): Check representable date range.
* doc/org.texi (The date/time prompt): Document date range protection.

New variable `org-read-date-force-compatible-dates' to control
handling of dates.
---
 doc/org.texi |   12 ++++++++++--
 lisp/org.el  |   54 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
 2 files changed, 62 insertions(+), 4 deletions(-)

diff --git a/doc/org.texi b/doc/org.texi
index e13af4b..dd6d945 100644
--- a/doc/org.texi
+++ b/doc/org.texi
@@ -5394,9 +5394,17 @@ The function understands English month and weekday abbreviations.  If
 you want to use unabbreviated names and/or other languages, configure
 the variables @code{parse-time-months} and @code{parse-time-weekdays}.
 
+@vindex org-read-date-force-compatible-dates
+Not all dates can be represented in a given Emacs implementation.  By default
+Org mode forces dates into the compatibility range 1970--2037 which works on
+all Emacs implementations.  If you want to use dates outside of this range,
+read the docstring of the variable
+@code{org-read-date-force-compatible-dates}.
+
 You can specify a time range by giving start and end times or by giving a
-start time and a duration (in HH:MM format). Use `-' or `-@{@}-' as the separator
-in the former case and use '+' as the separator in the latter case. E.g.@:
+start time and a duration (in HH:MM format). Use `-' or `-@{@}-' as the
+separator in the former case and use '+' as the separator in the latter
+case. E.g.@:
 
 @example
 11am-1:15pm    @result{} 11:00-13:15
diff --git a/lisp/org.el b/lisp/org.el
index 96ad8ff..a145de6 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -2651,6 +2651,36 @@ This may t or nil, or the symbol `org-read-date-prefer-future'."
 	  (const :tag "Never" nil)
 	  (const :tag "Always" t)))
 
+(defcustom org-read-date-force-compatible-dates t
+  "Should date/time prompt force dates that are guaranteed to work in Emacs?
+
+Depending on the system Emacs is running on, certain dates cannot
+be represented with the type used internally to represent time.
+Dates between 1970-1-1 and 2038-1-1 can always be represented
+correctly.  Some systems allow for earlier dates, some for later,
+some for both.  One way to find out it to insert any date into an
+Org buffer, putting the cursor on the year and hitting S-up and
+S-down to test the range.
+
+When this variable is set to t, the date/time prompt will not let
+you specify dates outside the 1970-2037 range, so it is certain that
+these dates will work in whatever version of Emacs you are
+running, and also that you can move a file from one Emacs implementation
+to another.  WHenever Org is forcing the year for you, it will display
+a message and beep.
+
+When this variable is nil, Org will check if the date is
+representable in the specific Emacs implementation you are using.
+If not, it will force a year, usually the current year, and beep
+to remind you.  Currently this setting is not recommended because
+the likelihood that you will open your Org files in an Emacs that
+has limited date range is not negligible.
+
+A workaround for this problem is to use diary sexp dates for time
+stamps outside of this range."
+  :group 'org-time
+  :type 'boolean)
+
 (defcustom org-read-date-display-live t
   "Non-nil means display current interpretation of date prompt live.
 This display will be in an overlay, in the minibuffer."
@@ -14503,6 +14533,13 @@ user."
 
     (setq final (org-read-date-analyze ans def defdecode))
 
+    (when org-read-date-analyze-forced-year
+      (message "Year was forced into %s"
+	       (if org-read-date-force-compatible-dates
+		   "compatible range (1970-2037)"
+		 "range representable on this machine"))
+      (ding))
+
     ;; One round trip to get rid of 34th of August and stuff like that....
     (setq final (decode-time (apply 'encode-time final)))
 
@@ -14520,6 +14557,7 @@ user."
 (defvar defdecode)
 (defvar with-time)
 (defvar org-read-date-analyze-futurep nil)
+(defvar org-read-date-analyze-forced-year nil)
 (defun org-read-date-display ()
   "Display the current date prompt interpretation in the minibuffer."
   (when org-read-date-display-live
@@ -14562,7 +14600,8 @@ user."
 	delta deltan deltaw deltadef year month day
 	hour minute second wday pm h2 m2 tl wday1
 	iso-year iso-weekday iso-week iso-year iso-date futurep kill-year)
-    (setq org-read-date-analyze-futurep nil)
+    (setq org-read-date-analyze-futurep nil
+	  org-read-date-analyze-forced-year nil)
     (when (string-match "\\`[ \t]*\\.[ \t]*\\'" ans)
       (setq ans "+0"))
 
@@ -14720,7 +14759,18 @@ user."
 	     (nth 2 tl))
 	(setq org-time-was-given t))
     (if (< year 100) (setq year (+ 2000 year)))
-    (if (< year 1970) (setq year (nth 5 defdecode))) ; not representable
+    ;; Check of the date is representable
+    (if org-read-date-force-compatible-dates
+	(progn
+	  (if (< year 1970)
+	      (setq year 1970 org-read-date-analyze-forced-year t))
+	  (if (> year 2037)
+	      (setq year 2037  org-read-date-analyze-forced-year t)))
+      (condition-case nil
+	  (encode-time second minute hour day month year)
+	(error
+	 (setq year (nth 5 defdecode))
+	 (setq org-read-date-analyze-forced-year t))))
     (setq org-read-date-analyze-futurep futurep)
     (list second minute hour day month year)))
 
-- 
1.7.1.575.gf526


[-- Attachment #3: Type: text/plain, Size: 2 bytes --]

Re: dates before 1970

[Eric S Fraga]

Carsten Dominik <carsten.dominik@gmail.com> writes:

[...]

> I am attaching a patch which tries to implement some kind of a
> solution for this problem.  The patch introduces a new variable
> which will allow you to use dates outside the safe range 1970-2037

Thanks Carsten.

[...]

> The patch also introduces a warning with a beep when Org has forced the year,
> which, I think, was really the main concern in this thread.

Yes, at least for me this is key.  I can use diary sexps, as you say,
for those dates that lie outside the range supported by emacs directly
but I need to know when something strange happens!

Thanks again,
eric

-- 
: Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
: using Org-mode version 7.5 (release_7.5.32.gdf26.dirty)

Re: dates before 1970

[Carsten Dominik]

Hi Bastien,

please let me know if you want this patch installed.

- Carsten

On 13.3.2011, at 21:08, Eric S Fraga wrote:

> Carsten Dominik <carsten.dominik@gmail.com> writes:
> 
> [...]
> 
>> I am attaching a patch which tries to implement some kind of a
>> solution for this problem.  The patch introduces a new variable
>> which will allow you to use dates outside the safe range 1970-2037
> 
> Thanks Carsten.
> 
> [...]
> 
>> The patch also introduces a warning with a beep when Org has forced the year,
>> which, I think, was really the main concern in this thread.
> 
> Yes, at least for me this is key.  I can use diary sexps, as you say,
> for those dates that lie outside the range supported by emacs directly
> but I need to know when something strange happens!
> 
> Thanks again,
> eric
> 
> -- 
> : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 24.0.50.1
> : using Org-mode version 7.5 (release_7.5.32.gdf26.dirty)

Re: dates before 1970

[Bastien]

Hi Carsten,

Carsten Dominik <carsten.dominik@gmail.com> writes:

> please let me know if you want this patch installed.

I won't have time to have a close look at this before tomorrow 
morning, as I'm still out on a week-end.

But please go ahead if you think it's okay.  

Best,

-- 
 Bastien

Re: dates before 1970

[Carsten Dominik]

On Mar 11, 2011, at 5:30 PM, Nick Dokos wrote:

> Carsten Dominik <carsten.dominik@gmail.com> wrote:
> 
> 
>> THis is exactly the point, that it depends on how Emacs was compiled, and what kind of integer is used in the date representation.  Signed or unsigend, 32 or 64 bits (I think).
>> 
>> For example, Bastien can represent dates before 1970. I cannot.
>> I can represent dates after 2038, Bastien cannot.
>> 
>> The work-around is to use diary sexps for dates before 1970, that seems to be safe.
>> And then hope that by 2038, all computers will use 64 bit integers....
>> 
> 
> But it's even more than that, no? Emacs's time implementation
> (current-time, encode/decode etc) would have to change. In fact, this
> might be the most significant limitation right now: the values they pass
> around are (hi16 lo16 ms) so they assume that time values are 32 bits,
> no matter what the underlying implementation says. I use 64-bit Linux on
> an x86-64 laptop and my time_t is 64 bits (but I don't know if it's
> signed or unsigned). Time for some experimentation I guess...

I believe that on your system you might get

(hi48 lo16 ms)

You can test if it is signed by trying a date before 1970

   (encode-time 0 0 0 1 1 1960)

- Carsten

Re: dates before 1970

[Nick Dokos]

Carsten Dominik <carsten.dominik@gmail.com> wrote:

> 
> On Mar 11, 2011, at 5:30 PM, Nick Dokos wrote:
> 
> > Carsten Dominik <carsten.dominik@gmail.com> wrote:
> > 
> > 
> >> THis is exactly the point, that it depends on how Emacs was compiled, and what kind of integer is used in the date representation.  Signed or unsigend, 32 or 64 bits (I think).
> >> 
> >> For example, Bastien can represent dates before 1970. I cannot.
> >> I can represent dates after 2038, Bastien cannot.
> >> 
> >> The work-around is to use diary sexps for dates before 1970, that seems to be safe.
> >> And then hope that by 2038, all computers will use 64 bit integers....
> >> 
> > 
> > But it's even more than that, no? Emacs's time implementation
> > (current-time, encode/decode etc) would have to change. In fact, this
> > might be the most significant limitation right now: the values they pass
> > around are (hi16 lo16 ms) so they assume that time values are 32 bits,
> > no matter what the underlying implementation says. I use 64-bit Linux on
> > an x86-64 laptop and my time_t is 64 bits (but I don't know if it's
> > signed or unsigned). Time for some experimentation I guess...
> 
> I believe that on your system you might get
> 
> (hi48 lo16 ms)
> 
> You can test if it is signed by trying a date before 1970
> 
>    (encode-time 0 0 0 1 1 1960)
> 

Ah, OK - thanks! I looked in current-time, saw the 0xffff mask and I
thought that the extra bits are truncated, but apparently not: I need to
go back and look at the C rules again.

I get

(encode-time 0 0 0 1 1 1960)
(-4816 20176)

(decode-time '(-4816 20176))
(0 0 0 1 1 1960 5 nil -18000)

so it is indeed signed.

I just pulled latest emacs and apparently Paul Eggert has been active in
making this code more robust, presumably after your conversations on the
emacs list.

Thanks,
Nick

Re: dates before 1970

[Carsten Dominik]

On Mar 14, 2011, at 4:11 PM, Nick Dokos wrote:

> Carsten Dominik <carsten.dominik@gmail.com> wrote:
> 
>> 
>> On Mar 11, 2011, at 5:30 PM, Nick Dokos wrote:
>> 
>>> Carsten Dominik <carsten.dominik@gmail.com> wrote:
>>> 
>>> 
>>>> THis is exactly the point, that it depends on how Emacs was compiled, and what kind of integer is used in the date representation.  Signed or unsigend, 32 or 64 bits (I think).
>>>> 
>>>> For example, Bastien can represent dates before 1970. I cannot.
>>>> I can represent dates after 2038, Bastien cannot.
>>>> 
>>>> The work-around is to use diary sexps for dates before 1970, that seems to be safe.
>>>> And then hope that by 2038, all computers will use 64 bit integers....
>>>> 
>>> 
>>> But it's even more than that, no? Emacs's time implementation
>>> (current-time, encode/decode etc) would have to change. In fact, this
>>> might be the most significant limitation right now: the values they pass
>>> around are (hi16 lo16 ms) so they assume that time values are 32 bits,
>>> no matter what the underlying implementation says. I use 64-bit Linux on
>>> an x86-64 laptop and my time_t is 64 bits (but I don't know if it's
>>> signed or unsigned). Time for some experimentation I guess...
>> 
>> I believe that on your system you might get
>> 
>> (hi48 lo16 ms)
>> 
>> You can test if it is signed by trying a date before 1970
>> 
>>   (encode-time 0 0 0 1 1 1960)
>> 
> 
> Ah, OK - thanks! I looked in current-time, saw the 0xffff mask and I
> thought that the extra bits are truncated, but apparently not: I need to
> go back and look at the C rules again.

I have no idea what you are talking about :)

> 
> I get
> 
> (encode-time 0 0 0 1 1 1960)
> (-4816 20176)
> 
> (decode-time '(-4816 20176))
> (0 0 0 1 1 1960 5 nil -18000)
> 
> so it is indeed signed.
> 
> I just pulled latest emacs and apparently Paul Eggert has been active in
> making this code more robust, presumably after your conversations on the
> emacs list.

Great.  Don't you love OpenSource projects?

> 
> Thanks,
> Nick
> 

- Carsten

Re: dates before 1970

[Nick Dokos]

Carsten Dominik <carsten.dominik@gmail.com> wrote:


> > Ah, OK - thanks! I looked in current-time, saw the 0xffff mask and I
> > thought that the extra bits are truncated, but apparently not: I need to
> > go back and look at the C rules again.
> 
> I have no idea what you are talking about :)
> 

You think I do? :)

I was referring to the C code implementing current-time (which, btw, has been
changed in latest):

,----
| DEFUN ("current-time", Fcurrent_time, Scurrent_time, 0, 0, 0,
|        doc: /* Return the current time, as the number of seconds since 1970-01-01 00:00:00.
| The time is returned as a list of three integers.  The first has the
| most significant 16 bits of the seconds, while the second has the
| least significant 16 bits.  The third integer gives the microsecond
| count.
| 
| The microsecond count is zero on systems that do not provide
| resolution finer than a second.  */)
|   (void)
| {
|   EMACS_TIME t;
| 
|   EMACS_GET_TIME (t);
|   return list3 (make_number ((EMACS_SECS (t) >> 16) & 0xffff),
| 		make_number ((EMACS_SECS (t) >> 0)  & 0xffff),
| 		make_number (EMACS_USECS (t)));
| }
`----

Nick

Re: dates before 1970

[Achim Gratz]

Nick Dokos <nicholas.dokos@hp.com> writes:
> I was referring to the C code implementing current-time (which, btw, has been
> changed in latest):
>
> ,----
> | DEFUN ("current-time", Fcurrent_time, Scurrent_time, 0, 0, 0,
> |        doc: /* Return the current time, as the number of seconds since 1970-01-01 00:00:00.
> | The time is returned as a list of three integers.  The first has the
> | most significant 16 bits of the seconds, while the second has the
> | least significant 16 bits.  The third integer gives the microsecond
> | count.
> | 
> | The microsecond count is zero on systems that do not provide
> | resolution finer than a second.  */)
> |   (void)
> | {
> |   EMACS_TIME t;
> | 
> |   EMACS_GET_TIME (t);
> |   return list3 (make_number ((EMACS_SECS (t) >> 16) & 0xffff),
> | 		make_number ((EMACS_SECS (t) >> 0)  & 0xffff),
> | 		make_number (EMACS_USECS (t)));
> | }
> `----

I can't see how this code works correctly unless it is guaranteed that
EMACS_TIME is 32bit unsigned...
 

Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Waldorf MIDI Implementation & additional documentation:
http://Synth.Stromeko.net/Downloads.html#WaldorfDocs

Re: dates before 1970

[Carsten Dominik]

On Mar 14, 2011, at 7:12 PM, Achim Gratz wrote:

> Nick Dokos <nicholas.dokos@hp.com> writes:
>> I was referring to the C code implementing current-time (which, btw, has been
>> changed in latest):
>> 
>> ,----
>> | DEFUN ("current-time", Fcurrent_time, Scurrent_time, 0, 0, 0,
>> |        doc: /* Return the current time, as the number of seconds since 1970-01-01 00:00:00.
>> | The time is returned as a list of three integers.  The first has the
>> | most significant 16 bits of the seconds, while the second has the
>> | least significant 16 bits.  The third integer gives the microsecond
>> | count.
>> | 
>> | The microsecond count is zero on systems that do not provide
>> | resolution finer than a second.  */)
>> |   (void)
>> | {
>> |   EMACS_TIME t;
>> | 
>> |   EMACS_GET_TIME (t);
>> |   return list3 (make_number ((EMACS_SECS (t) >> 16) & 0xffff),
>> | 		make_number ((EMACS_SECS (t) >> 0)  & 0xffff),
>> | 		make_number (EMACS_USECS (t)));
>> | }
>> `----
> 
> I can't see how this code works correctly unless it is guaranteed that
> EMACS_TIME is 32bit unsigned...

Yes, I agree.  However, I am just realizing that using 32 bit unsigned
for (current-time) is fine for a loong time to come.  So the issue
if *Org* can handle dates outside the safe range is is not about
`current-time' at all, but about what encode-time, decode-time,
time-subtract, time-add etc do, what kind of integers they can handle.

- Carsten

> 
> 
> Achim.
> -- 
> +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+
> 
> Waldorf MIDI Implementation & additional documentation:
> http://Synth.Stromeko.net/Downloads.html#WaldorfDocs
> 
> 

- Carsten