From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id QHaaGBj4fWMooAAAbAwnHQ (envelope-from ) for ; Wed, 23 Nov 2022 11:38:16 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id iASxFxj4fWPHagEAG6o9tA (envelope-from ) for ; Wed, 23 Nov 2022 11:38:16 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E654F3B568 for ; Wed, 23 Nov 2022 11:38:15 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oxn7n-00076S-9h; Wed, 23 Nov 2022 05:37:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oxn7S-00075v-HG for emacs-orgmode@gnu.org; Wed, 23 Nov 2022 05:36:43 -0500 Received: from mout01.posteo.de ([185.67.36.65]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oxn7Q-0008CE-3F for emacs-orgmode@gnu.org; Wed, 23 Nov 2022 05:36:42 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id D1890240026 for ; Wed, 23 Nov 2022 11:36:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1669199795; bh=OeR+mxT4+tYB/SVKNy/ar2Db8HsIYUmnqcDCkTABR1Q=; h=From:To:Cc:Subject:Date:From; b=GXnEciIhO8Rmi4Tp9dGkmLgRjip6+nobugnW2HklQMoeMJ8vAm+wjcS/sHf6ZmJpP +CBftS8vFIEakYqmnvWB0HwhQS+HvxhWHSQgnP430YeN4OstWfE9jeSacNt1kKecYv aQufJw2wgzQv6qavPgYKM6p6av/Hp0ZXpzygW4FWq+7EeQdcEgkueOxmOQHSn8LJjq lqhu3XXqpI+YT7vNST0mKPgOu9t5BtcSZVOyIEmNn0GBY4oZjwwhtkhSJLTeWjqnZ1 65ExAXjLG95bXavXVd+EutAOrC2zDZ5Ktl483bzFerGJ+Tl1+TR+rZxgOxlUhTPv7F 62slYb5MublYA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4NHHbz2NnGz6tmB; Wed, 23 Nov 2022 11:36:30 +0100 (CET) From: Ihor Radchenko To: Max Nikulin Cc: emacs-orgmode@gnu.org Subject: Re: test-org-table/sort-lines: Failing test on macOS In-Reply-To: References: <87ilkulwdy.fsf@localhost> <87y1tpejfm.fsf@localhost> <87wn7wdfis.fsf@localhost> <87pmdil0m0.fsf@localhost> <87k03pj8vw.fsf@localhost> <87leo3dc42.fsf@localhost> Date: Wed, 23 Nov 2022 10:37:08 +0000 Message-ID: <87k03mhs8b.fsf@localhost> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: pass client-ip=185.67.36.65; envelope-from=yantar92@posteo.net; helo=mout01.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-orgmode@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-orgmode-bounces+larch=yhetil.org@gnu.org Sender: emacs-orgmode-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1669199896; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=tiywvtIt8de3AOoPuIHtsMLdtKNPmJktaUqDOpGOpKk=; b=q/yYdajV48HXd4STEckyfpbH/PXssaCrvtsBLnDaOGCAbATY10c38xvUULcU8rtqP4OT9J 42RxqOYry6wysyjd3Kg9rSxDfs9Q1D5lGPXBpe+fJRSN0Vj6H/H9SLPz8Eym3h7jlV8gAA CaKItkko2CP35AKDAg7pjzQ8psHp7WFbMgZDhxU02lHzKXj8talqQWPBrr9M/MJ/lKA8dK iQhhFD1R9QtdjR2uSGk2J1tLxX8vEyiBykvrAhpsAhvownPT9ZgRRLmYpdW2VqvkMmtTq3 5tSLh/RPJRDgIOvYewzQ+u8CLBZFGGXNxKpjFMf3DEn8/mvoJChszGf1woY7Wg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1669199896; a=rsa-sha256; cv=none; b=jYST9A2Be2Xl80GGwRO8nRYlJtwxsQSLxrxK+11XvDcWJ2efHvpkiErvBw+g1XkeJFba6d 69iTZ3pDua38Fz4UEFjsTSVez/qvfD0b2jQVJJqjd4Ze8bDagLiSp0gppckwrYOeaGS1Xp 2DsgydqWmknBTuC6aKhKNZa0gQXXOLFfiO305Q97qlm5PJ50lUZc1Vwr6pCslG8E5211PM tEFze9aRAMJiospX3ha6H4/GFbzGV/ZF1GJBbMqOviZZ6bs6bMllBJzTv0J+vGLrPhksvk AxcBtwrt7oS2FA6HCL+Yoby/N6xPE7OUcLuG98MWon3twHSZPDYr6WNu6N3U4A== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=GXnEciIh; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.59 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.net header.s=2017 header.b=GXnEciIh; dmarc=pass (policy=none) header.from=posteo.net; spf=pass (aspmx1.migadu.com: domain of "emacs-orgmode-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="emacs-orgmode-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: E654F3B568 X-Spam-Score: -3.59 X-Migadu-Scanner: scn0.migadu.com X-TUID: Fz2IegtL8hk9 Max Nikulin writes: > On 22/11/2022 08:14, Ihor Radchenko wrote: >> Max Nikulin writes: >> >>>> 2. `org-sort-list' >>>> 5. `org-sort-entries' >>> `downcase' is used, not proper case folding, so a potential issue >> >> `downcase' is used to determine user input about sorting type. >> Not for sorting itself. > > See case-func variable. Its initialization depends on the IGNORE-CASE > argument. Strings to sort are passed either through `identity' or > through `downcase'. Thanks for the pointer. Now, I am getting more confused though. Do we even need to use `string-collate-lessp' then? Eli even argued that `string-collate-lessp' is strictly worse compared to more predictable approach. See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=59275#40 Do you remember any cases when users actually demanded locale-specific sorting? >>> IGNORE-CASE argument is not used, perhaps `downcase' is hidden in the code. >> >> I feel like we are slightly miscommunicating here. >> I mostly tried to list the uses of libc-sensitive sorting. Not >> specifically cases when we try to ignore the case. >> >> The problem is not limited to case-sensitive comparisons. Some systems >> may fail to implement specific locales and thus sorting may downgrade to >> simple string-lessp. > > When case folding is not involved, I consider `string-lessp' as a > graceful degradation. Despite locale rules are not applied, strings are > mostly sorted. Exceptions exist, but usually order is reasonable. > > Completely disregarding IGNORE-CASE argument of `string-collate-lessp' > on MacOS (that is not a heavily stripped embedded OS) is a bad surprise > for me. It was a surprise for me as well. Should be at least a bit more clear now as I updated the docstring of `string-collate-lessp'. However, I feel a bit lost about what to do on Org side. We can put a disclaimer in the manual and all that, but it still feels too complex. >>>> 6. Agenda sorting, when alphabetical sorting is involved >>> >>> `string-lessp' and `downcase' so even more severe locale-related issues >>> might be expected. >> >> Could you please elaborate? > > I admit that `downcase' may be an acceptable workaround since > `string-collate-lessp' may not work IGNORE-CASE, but I believe, when > available, `string-collate-lessp' should be the preferred option for > sorting. As I pointed above, Eli has an opposite opinion. I feel that my understanding of the topic is not sufficient to judge. Maybe we should ask users? (But who is even aware about these things happening under the hood?) > I have an idea of a compatibility wrapper for `string-collate-lessp' > with special treatment of ignoring case and bad libc implementation. > Apply `downcase' before passing arguments to `string-lessp'. It should > provide consistency, best user experience when locales works properly, > and graceful degradation otherwise. I hope, it is acceptable for Org > even though such trick is undesired for Emacs due to performance reasons. Macro idea sounds reasonable. Though I am still unsure which direction we need to go. > However I am afraid of compatibility shims after > > d3a9c424b 2022-08-16 17:15:27 +0800 Ihor Radchenko: org-encode-time: > Refactor into top-level `defmacro' What do you refer to? > I do not like that Emacs relies on locale support (and timezone as well) > in libc. It becomes a problem as soon as more than one locale should be > used in simultaneously. I agree that there are enough complications and > sometimes locale depends on the document (e.g. #+LANGUAGE:), sometimes > specific locale even restricted to a part of a document. It is tricky to > handle such cases, but current limitations are too strict (and defective > `string-collate-lessp' on MacOS is an example). The question is what can be done and, more importantly, how much effort will it take to implement and maintain an alternative. -- Ihor Radchenko // yantar92, Org mode contributor, Learn more about Org mode at . Support Org development at , or support my work at