Tasks don't repeat correctly if system-time-locale is set to certain languages

Tasks don't repeat correctly if system-time-locale is set to certain languages

[Bruce V. Chiarelli]

Hello all,

I've noticed some unusual behavior with repeating entries when the
system-time-locale variable is set. Specifically:

It is Sunday, today, October 30th. I did not mark this task, which is
a habit, yesterday.

-- If I have (setq system-time-locale "hu_HU.utf8"), Hungarian, then
marking this task DONE

* TODO Anki basic reviews                                       :habit:study:
  SCHEDULED: <2016-10-29 szo .+1d>

v----becomes----v

* TODO Anki basic reviews                                       :habit:study:
  SCHEDULED: <2016-10-30 v .+1d>

Which is not correct. I marked it DONE today, so it should repeat tomorrow.

-- If I have (setq system-time-locale "es_MX.utf8"), Mexican Spanish,
then doing the same thing:

* TODO Anki basic reviews                                       :habit:study:
  SCHEDULED: <2016-10-29 szo .+1d>

v----becomes----v

* TODO Anki basic reviews                                       :habit:study:
  SCHEDULED: <2016-10-31 lun .+1d>

Which *is* correct. I have tried this with an unset
system-time-locale, and with it set to fa_IR, es_MX, en_GB, and hu_HU.
So far, hu_HU is the only one that behaves incorrectly. Note that it
doesn't seem to matter which language the day-of-the-week abbreviation
is already in, since every time I tried this, I reverted the file back
to the Hungarian line. Changing the date to <2016-10-29 Sat .+1d>
before marking it also had no effect.

Of course, I could just set the date locale to "C" or unset it, but
there's still a bug somewhere.

I am running the 1399f5 revision now, but I can reproduce this
behavior all the way back until version 7,

Cheers,
Bruce V C

Re: Tasks don't repeat correctly if system-time-locale is set to certain languages

[Nicolas Goaziou]

Hello,

"Bruce V. Chiarelli" <mano155@gmail.com> writes:

> I've noticed some unusual behavior with repeating entries when the
> system-time-locale variable is set. Specifically:
>
> It is Sunday, today, October 30th. I did not mark this task, which is
> a habit, yesterday.
>
> -- If I have (setq system-time-locale "hu_HU.utf8"), Hungarian, then
> marking this task DONE
>
> * TODO Anki basic reviews                                       :habit:study:
>   SCHEDULED: <2016-10-29 szo .+1d>
>
> v----becomes----v
>
> * TODO Anki basic reviews                                       :habit:study:
>   SCHEDULED: <2016-10-30 v .+1d>
>
> Which is not correct. I marked it DONE today, so it should repeat tomorrow.
>
> -- If I have (setq system-time-locale "es_MX.utf8"), Mexican Spanish,
> then doing the same thing:
>
> * TODO Anki basic reviews                                       :habit:study:
>   SCHEDULED: <2016-10-29 szo .+1d>
>
> v----becomes----v
>
> * TODO Anki basic reviews                                       :habit:study:
>   SCHEDULED: <2016-10-31 lun .+1d>
>
> Which *is* correct. I have tried this with an unset
> system-time-locale, and with it set to fa_IR, es_MX, en_GB, and hu_HU.
> So far, hu_HU is the only one that behaves incorrectly. Note that it
> doesn't seem to matter which language the day-of-the-week abbreviation
> is already in, since every time I tried this, I reverted the file back
> to the Hungarian line. Changing the date to <2016-10-29 Sat .+1d>
> before marking it also had no effect.
>
> Of course, I could just set the date locale to "C" or unset it, but
> there's still a bug somewhere.

I tend to think this is not a bug in Org mode, since it correctly work
with other locales, and we do not control locales.

Anyway, could you try bisecting and tell us when the bug was born?

Regards,

-- 
Nicolas Goaziou

Re: Tasks don't repeat correctly if system-time-locale is set to certain languages

[Bruce V. Chiarelli]

2016-10-31 8:23 GMT-07:00 Nicolas Goaziou <mail@nicolasgoaziou.fr>:
> Hello,
>
> "Bruce V. Chiarelli" <mano155@gmail.com> writes:
>
>> I've noticed some unusual behavior with repeating entries when the
>> system-time-locale variable is set. Specifically:
>>
>> It is Sunday, today, October 30th. I did not mark this task, which is
>> a habit, yesterday.
>>
>> -- If I have (setq system-time-locale "hu_HU.utf8"), Hungarian, then
>> marking this task DONE
>>
>> * TODO Anki basic reviews                                       :habit:study:
>>   SCHEDULED: <2016-10-29 szo .+1d>
>>
>> v----becomes----v
>>
>> * TODO Anki basic reviews                                       :habit:study:
>>   SCHEDULED: <2016-10-30 v .+1d>
>>
>> Which is not correct. I marked it DONE today, so it should repeat tomorrow.
>>
>> -- If I have (setq system-time-locale "es_MX.utf8"), Mexican Spanish,
>> then doing the same thing:
>>
>> * TODO Anki basic reviews                                       :habit:study:
>>   SCHEDULED: <2016-10-29 szo .+1d>
>>
>> v----becomes----v
>>
>> * TODO Anki basic reviews                                       :habit:study:
>>   SCHEDULED: <2016-10-31 lun .+1d>
>>
>> Which *is* correct. I have tried this with an unset
>> system-time-locale, and with it set to fa_IR, es_MX, en_GB, and hu_HU.
>> So far, hu_HU is the only one that behaves incorrectly. Note that it
>> doesn't seem to matter which language the day-of-the-week abbreviation
>> is already in, since every time I tried this, I reverted the file back
>> to the Hungarian line. Changing the date to <2016-10-29 Sat .+1d>
>> before marking it also had no effect.
>>
>> Of course, I could just set the date locale to "C" or unset it, but
>> there's still a bug somewhere.
>
> I tend to think this is not a bug in Org mode, since it correctly work
> with other locales, and we do not control locales.

I thought so too, to be honest, but I got my hands dirty and I think
I've figured out where the actual problem lies.

I did the bisect earlier, and the breaking change was right when the
code to handle native language day names was added as part of Org 8.0.
I got a bit disorganized and lost the exact commit, but I can try to
find it if need be. Anyway, I started the lisp debugger to trace what
was happening and found the real problem. I hope someone can confirm.

org-todo calls org-auto-repeat-maybe, which sees the ".+" style
repeater. It calls org-timestamp-change to move the timestamp up to
today. Point is left at the closing bracket. So far, so good.

org-timestamp-change sets origin-cat to 'after and origin to (point).
It then changes the timestamp to today as advertized.

Now these lines get evaluated

    (goto-char (cond
            ;; `day' category ends before `hour' if any, or at
            ;; the end of the day name.
            ((eq origin-cat 'day)
             (min (or (match-beginning 7) (- (match-end 5) 2)) origin))
            ((eq origin-cat 'hour) (min (match-end 7) origin))
            ((eq origin-cat 'minute) (min (1- (match-end 8)) origin))
            ((integerp origin-cat) (min (1- (match-end 0)) origin))
            ;; `year' and `month' have both fixed size: point
            ;; couldn't have moved into another part.
            (t origin))))

The since origin-cat is 'after, matching nothing else, we get
(goto-char origin).

This seems to be where the problem lies. When "<2016-10-29 szo .+1>"
becomes "<2016-10-31 h .+1>" (today), origin is now two characters
ahead of where it should be, now on the next line in fact.

So it returns fine at first, but when org-auto-repeat maybe calls
org-timestamp-change a second time to move the date into the future,
it throws the error "Not at a timestamp" and does nothing else. I
wasn't seeing this error until I was in the debugger, since
org-auto-repeat-maybe puts an "Entry repeats: ..." message in the echo
area. Oddly enough, that message shows the correct date since
org-last-changed-timestamp gets updated properly.

-BC

> Anyway, could you try bisecting and tell us when the bug was born?
>
> Regards,
>
> --
> Nicolas Goaziou

Re: Tasks don't repeat correctly if system-time-locale is set to certain languages

[Nicolas Goaziou]

"Bruce V. Chiarelli" <mano155@gmail.com> writes:

> org-todo calls org-auto-repeat-maybe, which sees the ".+" style
> repeater. It calls org-timestamp-change to move the timestamp up to
> today. Point is left at the closing bracket. So far, so good.
>
> org-timestamp-change sets origin-cat to 'after and origin to (point).
> It then changes the timestamp to today as advertized.
>
> Now these lines get evaluated
>
>     (goto-char (cond
>             ;; `day' category ends before `hour' if any, or at
>             ;; the end of the day name.
>             ((eq origin-cat 'day)
>              (min (or (match-beginning 7) (- (match-end 5) 2)) origin))
>             ((eq origin-cat 'hour) (min (match-end 7) origin))
>             ((eq origin-cat 'minute) (min (1- (match-end 8)) origin))
>             ((integerp origin-cat) (min (1- (match-end 0)) origin))
>             ;; `year' and `month' have both fixed size: point
>             ;; couldn't have moved into another part.
>             (t origin))))
>
> The since origin-cat is 'after, matching nothing else, we get
> (goto-char origin).
>
> This seems to be where the problem lies. When "<2016-10-29 szo .+1>"
> becomes "<2016-10-31 h .+1>" (today), origin is now two characters
> ahead of where it should be, now on the next line in fact.

I see. Thank you for the analysis.

Does adding the following branch in the `cond' above, before the
catch-all one, solves the issue?

  ((eq origin-cat 'after) (match-end 0))


Regards,

Re: Tasks don't repeat correctly if system-time-locale is set to certain languages

[Bruce V. Chiarelli]

2016-10-31 17:04 GMT-07:00 Nicolas Goaziou <mail@nicolasgoaziou.fr>:
> "Bruce V. Chiarelli" <mano155@gmail.com> writes:
>
>> org-todo calls org-auto-repeat-maybe, which sees the ".+" style
>> repeater. It calls org-timestamp-change to move the timestamp up to
>> today. Point is left at the closing bracket. So far, so good.
>>
>> org-timestamp-change sets origin-cat to 'after and origin to (point).
>> It then changes the timestamp to today as advertized.
>>
>> Now these lines get evaluated
>>
>>     (goto-char (cond
>>             ;; `day' category ends before `hour' if any, or at
>>             ;; the end of the day name.
>>             ((eq origin-cat 'day)
>>              (min (or (match-beginning 7) (- (match-end 5) 2)) origin))
>>             ((eq origin-cat 'hour) (min (match-end 7) origin))
>>             ((eq origin-cat 'minute) (min (1- (match-end 8)) origin))
>>             ((integerp origin-cat) (min (1- (match-end 0)) origin))
>>             ;; `year' and `month' have both fixed size: point
>>             ;; couldn't have moved into another part.
>>             (t origin))))
>>
>> The since origin-cat is 'after, matching nothing else, we get
>> (goto-char origin).
>>
>> This seems to be where the problem lies. When "<2016-10-29 szo .+1>"
>> becomes "<2016-10-31 h .+1>" (today), origin is now two characters
>> ahead of where it should be, now on the next line in fact.
>
> I see. Thank you for the analysis.
>
> Does adding the following branch in the `cond' above, before the
> catch-all one, solves the issue?
>
>   ((eq origin-cat 'after) (match-end 0))

It does! Wonderful.

-BC

Re: Tasks don't repeat correctly if system-time-locale is set to certain languages

[Nicolas Goaziou]

Hello,

"Bruce V. Chiarelli" <mano155@gmail.com> writes:

> 2016-10-31 17:04 GMT-07:00 Nicolas Goaziou <mail@nicolasgoaziou.fr>:

>> Does adding the following branch in the `cond' above, before the
>> catch-all one, solve the issue?
>>
>>   ((eq origin-cat 'after) (match-end 0))
>
> It does! Wonderful.

Applied. Thank you.

Regards,

-- 
Nicolas Goaziou