Time zone trouble without time zones being involved

Time zone trouble without time zones being involved

[Michael Heerdegen]

Hello,

I'm using git to make a backup of the file I'm working with after every
save.  I want to display the age of the last backup made in the
mode-line.

To be able to control the time display format from Elisp, I decided to
ask git-log for the time since "the Epoch":

#+begin_src emacs-lisp
(string-to-number
 (helm-backup-exec-git-command "log" ref "--pretty=format:%ad"
                               "--date=format:%s" "-n1"
                               filename-for-git))
#+end_src

(where `helm-backup-exec-git-command' just calls "git" with the
specified args).

"%ad" means "author date", and "%s" means, according to "man strftime",
"The number of seconds since the Epoch, 1970-01-01 00:00:00 +0000 (UTC).
(TZ) (Calculated from mktime(tm).)"

After saving a file, this value is saved to the local variable
`helm-backup-time-of-latest-backup'.

In the mode-line, I show this:

#+begin_src emacs-lisp
   (helm-backup-time-of-latest-backup
    (:eval
     (progn
       (require 'timeclock)
       (propertize (concat " "
                           (timeclock-seconds-to-string
                            (time-to-seconds
                             (time-subtract (current-time)
                                            (seconds-to-time helm-backup-time-of-latest-backup))))
                           "h")
                   'face 'mode-line-shadow))))
#+end_src

The problem: since we have Daylight saving time, this always shows one
hour less than the expected value (directly after saving it shows
-0:59h).  This makes no sense to me, since I only work with "seconds
since Epoch" absolute time values that I thought would not depend on the
local time zone.  So either git or Emacs seems to make an error.

Do I miss something?  How can I prevent that wrong behavior?


Thanks,

Michael.

Re: Time zone trouble without time zones being involved

[Yuri Khan]

On Sun, Apr 3, 2016 at 12:11 PM, Michael Heerdegen
<michael_heerdegen@web.de> wrote:

> I'm using git to make a backup of the file I'm working with after every
> save.  I want to display the age of the last backup made in the
> mode-line.

[…snipped…]

Your method looks sane to me, although the mention of TZ and mktime
regarding %s bother me a bit.

> The problem: since we have Daylight saving time, this always shows one
> hour less than the expected value (directly after saving it shows
> -0:59h).  This makes no sense to me, since I only work with "seconds
> since Epoch" absolute time values that I thought would not depend on the
> local time zone.  So either git or Emacs seems to make an error.

Let’s determine which.

1. Mark the current UTC time, both as a human-readable string and as a
unixtime value (e.g. on GNU, “date --utc --rfc-3339=seconds” and “date
+%s”, respectively).
2. Make a commit.
3. Execute the same command, “git log --pretty=format:%ad --date=format:%s -n1”.
4. Evaluate (current-time) and convert the first two elements of the
resulting 4-element list to unixtime.
5. Check that the three unixtime values from steps 1, 3 and 4 are
reasonably close.
6. By the way, which OS are you on?

Re: Time zone trouble without time zones being involved

[Michael Heerdegen]

Hi Yuri,

sorry for the delay...


> Let’s determine which.

Ok.

> 1. Mark the current UTC time, both as a human-readable string and as a
> unixtime value (e.g. on GNU, “date --utc --rfc-3339=seconds” and “date
> +%s”, respectively).
> 2. Make a commit.
> 3. Execute the same command, “git log --pretty=format:%ad
> --date=format:%s -n1”.
> 4. Evaluate (current-time) and convert the first two elements of the
> resulting 4-element list to unixtime.
> 5. Check that the three unixtime values from steps 1, 3 and 4 are
> reasonably close.

I'm not sure how to do 4.  However, 1. and 3. already give me a
difference of ~ 60*60.

What does this tell us?


> 6. By the way, which OS are you on?

uname -a => "Linux drachen 4.4.0-1-amd64 #1 SMP Debian 4.4.6-1
(2016-03-17) x86_64 GNU/Linux".  Mostly Debian testing.


Thanks,

Michael.

Re: Time zone trouble without time zones being involved

[Yuri Khan]

On Wed, Apr 13, 2016 at 10:14 PM, Michael Heerdegen
<michael_heerdegen@web.de> wrote:

>> 1. Mark the current UTC time, both as a human-readable string and as a
>> unixtime value (e.g. on GNU, “date --utc --rfc-3339=seconds” and “date
>> +%s”, respectively).
>> 2. Make a commit.
>> 3. Execute the same command, “git log --pretty=format:%ad
>> --date=format:%s -n1”.
>> 4. Evaluate (current-time) and convert the first two elements of the
>> resulting 4-element list to unixtime.
>> 5. Check that the three unixtime values from steps 1, 3 and 4 are
>> reasonably close.
>
> I'm not sure how to do 4.  However, 1. and 3. already give me a
> difference of ~ 60*60.
>
> What does this tell us?

From what it looks like, either git or libc are misinformed about your
time zone.

I was hoping you’d post the actual values you see at each step. I
would then independently convert the unixtime into a date and time to
see they are sane. Your wall clock reading would be useful too.

(current-time) is documented thus in my copy of Emacs 24:

    Return the current time, as the number of seconds since 1970-01-01 00:00:00.
    The time is returned as a list of integers (HIGH LOW USEC PSEC).
    HIGH has the most significant bits of the seconds, while LOW has the
    least significant 16 bits.  USEC and PSEC are the microsecond and
    picosecond counts.

so you take the first element, multiply it by 65536, add the second element.

Example: I just now got this tuple: (22286 29507 190287 275000). 22286
* 65536 + 29507 = 1460564803. “date +%s” says 1460564877 so my Emacs
and the rest of my system agree and it took me just over a minute to
do the calculation.

>> 6. By the way, which OS are you on?
>
> uname -a => "Linux drachen 4.4.0-1-amd64 #1 SMP Debian 4.4.6-1
> (2016-03-17) x86_64 GNU/Linux".  Mostly Debian testing.

Good. What is your $TZ (if any), what does your /etc/timezone contain,
and is your tzdata package fresh enough?

Re: Time zone trouble without time zones being involved

[Michael Heerdegen]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> What is your $TZ (if any)

Not set.

> what does your /etc/timezone contain,

Europe/Berlin

> and is your tzdata package fresh enough?

Yes, up to date.


But your questions remind me that I have set

  LC_TIME="C"

in my environment, because I don't want to see German style time strings
everywhere.

Could simply that be the culprit (or do I still need to perform the
tests)?


Thanks,

Michael.

Re: Time zone trouble without time zones being involved

[Yuri Khan]

On Wed, Apr 13, 2016 at 11:16 PM, Michael Heerdegen
<michael_heerdegen@web.de> wrote:

> But your questions remind me that I have set
>
>   LC_TIME="C"
>
> in my environment, because I don't want to see German style time strings
> everywhere.
>
> Could simply that be the culprit (or do I still need to perform the
> tests)?

I’d think LC_TIME only affects the formats, not time zone calculations.

Tests, please :)

Re: Time zone trouble without time zones being involved

[Michael Heerdegen]

Hi Yuri,

> Tests, please :)

Good news - I managed to bring me to do this now!  Hope you are still
reading this...

Here are the results.  It is 20:44 here now after I run the tests
(Central European Daylight savings time, the time zone of Berlin).


> 1. Mark the current UTC time, both as a human-readable string and as a
> unixtime value (e.g. on GNU, “date --utc --rfc-3339=seconds” and “date
> +%s”, respectively).

date --utc --rfc-3339=seconds => 2016-05-14 18:38:17+00:00
date +%s                      => 1463251129


> 2. Make a commit.  3. Execute the same command, “git log
> --pretty=format:%ad --date=format:%s -n1”.

1463254759

> 4. Evaluate (current-time) and convert the first two elements of the
> resulting 4-element list to unixtime.

(let ((time (current-time))) (+ (* 65536 (car time)) (cadr time)))
==> 1463251305


Looks like git is 1 hour wrong...?


TIA,

Michael.

Re: Time zone trouble without time zones being involved

[Yuri Khan]

On Sun, May 15, 2016 at 12:53 AM, Michael Heerdegen
<michael_heerdegen@web.de> wrote:

> Here are the results.  It is 20:44 here now after I run the tests
> (Central European Daylight savings time, the time zone of Berlin).

So, UTC+1 in winter, UTC+2 in summer.

>> 1. Mark the current UTC time, both as a human-readable string and as a
>> unixtime value (e.g. on GNU, “date --utc --rfc-3339=seconds” and “date
>> +%s”, respectively).
>
> date --utc --rfc-3339=seconds => 2016-05-14 18:38:17+00:00
> date +%s                      => 1463251129

The unixtime works out to 2016-05-14 18:38:49+00:00, consistent with
the UTC time. Also consistent with your wall clock considering the
UTC+2 offset.

>> 2. Make a commit.  3. Execute the same command, “git log
>> --pretty=format:%ad --date=format:%s -n1”.
>
> 1463254759

1463254759 - 1463251129 = 3630 seconds, or 1 hour 30 seconds.

Does “git cat-file commit HEAD” show the same number in the “author” line?

>> 4. Evaluate (current-time) and convert the first two elements of the
>> resulting 4-element list to unixtime.
>
> (let ((time (current-time))) (+ (* 65536 (car time)) (cadr time)))
> ==> 1463251305

176 seconds from step 1.

> Looks like git is 1 hour wrong...?

Yes.


Here’s a repeatable recipe. You can report it as a Git bug.

0. Versions

    $ git --version
    git version 2.8.2

    $ dpkg -l tzdata | tail -1
    ii  tzdata  2016d-0ubuntu0.16.04 all  time zone and
daylight-saving time data

1. Initialize an empty Git repository:

    $ git init test
    $ cd test

2. Make a commit, using the Europe/Berlin time zone:

    $ TZ=Europe/Berlin git commit -m 'test' --allow-empty

3. Examine the timestamp recorded in the commit object:

    $ git cat-file -p HEAD | grep author
    author Yuri Khan <yurivkhan@gmail.com> 1463260938 +0200

4. Check that it corresponds to the current time:

    $ date +%s
    1463260977

5. Try to get the commit date in the unixtime format:

    $ TZ=Europe/Berlin git log --pretty=format:%ad --date=format:%s -1

Expected result: 1463260938 (same as recorded in the commit object).
Observed result: 1463264538 (3600s = one hour ahead).

For lulz, use another time zone:

    $ TZ=Asia/Novosibirsk git log --pretty=format:%ad --date=format:%s -1

Expected result: 1463260938 (unixtime is always UTC and should not
depend on TZ).
Observed result: 1463246538 (-14400s = 4 hours behind).

Not even specifying the UTC time zone helps:

    $ TZ=UTC git log --pretty=format:%ad --date=format:%s -1

Expected result: still 1463260938.
Observed result: 1463268138 (7200s = 2 hours ahead).

Re: Time zone trouble without time zones being involved

[Michael Heerdegen]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> >> 2. Make a commit.  3. Execute the same command, “git log
> >> --pretty=format:%ad --date=format:%s -n1”.
> >
> > 1463254759
>
> 1463254759 - 1463251129 = 3630 seconds, or 1 hour 30 seconds.
>
> Does “git cat-file commit HEAD” show the same number in the “author”
> line?

No:

| bash-4.3:micha:test-repo$ git log --pretty=format:%ad --date=format:%s -n1
| 1463254759
| bash-4.3:micha:test-repo$ git cat-file commit HEAD
| tree 45b5897f752b636a5dc6d101cb2a14dba4751cd0
| parent c56aab0f0de7335ffd579702b987e67ce0680af8
| author Michael Heerdegen <michael_heerdegen@web.de> 1463251159 +0200
| committer Michael Heerdegen <michael_heerdegen@web.de> 1463251159 +0200


> Here’s a repeatable recipe. You can report it as a Git bug.
> [...]

Thanks, I'll do this.


Regards,

Michael.

Re: Time zone trouble without time zones being involved

[Yuri Khan]

On Mon, May 16, 2016 at 9:47 PM, Michael Heerdegen
<michael_heerdegen@web.de> wrote:

> | bash-4.3:micha:test-repo$ git log --pretty=format:%ad --date=format:%s -n1
> | 1463254759
> | bash-4.3:micha:test-repo$ git cat-file commit HEAD
> | tree 45b5897f752b636a5dc6d101cb2a14dba4751cd0
> | parent c56aab0f0de7335ffd579702b987e67ce0680af8
> | author Michael Heerdegen <michael_heerdegen@web.de> 1463251159 +0200
> | committer Michael Heerdegen <michael_heerdegen@web.de> 1463251159 +0200

So, for the time being, as a workaround, you can use something like
"git cat-file commit HEAD | sed -nr 's/^author .* ([0-9]+)
[-+][0-9]+$/\1/p; t'" to get the UTC timestamp from the last commit.

Re: Time zone trouble without time zones being involved

[Michael Heerdegen]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> So, for the time being, as a workaround, you can use something like
> "git cat-file commit HEAD | sed -nr 's/^author .* ([0-9]+)
> [-+][0-9]+$/\1/p; t'" to get the UTC timestamp from the last commit.

Hmm, I hoped I can do without a shell.


Do you think the unixtime value as printed in

  git log --pretty=format:%ad --date=raw -n1

can be relied on?  Can I ignore the printed time zone delta?


[BTW, the doc of git-log tells that

  --date=raw shows the date in the internal raw Git format %s %z format.

but this seems to be a different %s because raw shows a different time]



Thanks,

Michael.

Re: Time zone trouble without time zones being involved

[Yuri Khan]

On Tue, May 17, 2016 at 8:57 PM, Michael Heerdegen
<michael_heerdegen@web.de> wrote:

> Hmm, I hoped I can do without a shell.
>
> Do you think the unixtime value as printed in
>
>   git log --pretty=format:%ad --date=raw -n1
>
> can be relied on?  Can I ignore the printed time zone delta?

In my limited testing, the timestamp seems to match the one in the commit.

The timestamp identifies the exact moment in universal time when the
commit was made. The time zone offset is only stored for the benefit
of tools which want to display commit times in their author and
committer’s time zones in effect at that moment (which might or might
not coincide with the user’s time zone offset in effect at the time of
viewing).

So yes, you can ignore the offset. But you still need something in
order to strip it off, either shell or string manipulation.

> [BTW, the doc of git-log tells that
>
>   --date=raw shows the date in the internal raw Git format %s %z format.
>
> but this seems to be a different %s because raw shows a different time]

I don’t even think any date arithmetic happens with --date=raw. The
information is already there in the commit data, in this exact format,
Git probably only extracts it with string operations.

Re: Time zone trouble without time zones being involved

[Michael Heerdegen]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> So yes, you can ignore the offset. But you still need something in
> order to strip it off, either shell or string manipulation.

FWIW, `string-to-number' is intelligent (or dumb) enough to work anyway:

  (string-to-number "123456789 foo")  => 123456789

so I did not have to change my code with respect to that.


Thanks,
Michael.