format use inquiry

format use inquiry

[Jean-Christophe Helary]

I'd like to know if that kind of code is acceptable or not:

(format "http%s://elpa.gnu.org/packages/"
                                        (if (gnutls-available-p) "s" ""))

Jean-Christophe 

Re: format use inquiry

[Thien-Thi Nguyen]

[-- Attachment #1: Type: text/plain, Size: 577 bytes --]


() Jean-Christophe Helary <jean.christophe.helary@gmail.com>
() Tue, 20 Jun 2017 17:48:49 +0900

   (format "http%s://elpa.gnu.org/packages/"
            (if (gnutls-available-p) "s" ""))

I think this is fine, if a bit noisy.  You might also consider:

 (concat "a" nil "b")
 => "ab"

-- 
Thien-Thi Nguyen -----------------------------------------------
 (defun responsep (query)
   (pcase (context query)
     (`(technical ,ml) (correctp ml))
     ...))                              748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: format use inquiry

[Yuri Khan]

On Tue, Jun 20, 2017 at 3:48 PM, Jean-Christophe Helary
<jean.christophe.helary@gmail.com> wrote:
> I'd like to know if that kind of code is acceptable or not:
>
> (format "http%s://elpa.gnu.org/packages/"
>                                         (if (gnutls-available-p) "s" ""))

The standard localization argument against concatenating an "s" onto a
word does not apply here. But, one day, someone is going to grep for
occurrences of http:// to see if it’s still used anywhere.

That someone could be you.

Spell it out.

    (defvar …-elpa-root
      (if (gnutls-available-p)
          "https://elpa.gnu.org/packages/"
        "http://elpa.gnu.org/packages/")
      "…some doc…. http and https schemes are supported;
    https is conditional on GnuTLS.")

Re: format use inquiry

[Jean-Christophe Helary]

Thank you Yuri. I'm actually back to working on packages.el where there are some problematic string concatenations and I thought what you just proposed was a more acceptable way to handle such strings. Thank you for the confirmation.

Jean-Christophe 

> On Jun 21, 2017, at 0:02, Yuri Khan <yuri.v.khan@gmail.com> wrote:
> 
> On Tue, Jun 20, 2017 at 3:48 PM, Jean-Christophe Helary
> <jean.christophe.helary@gmail.com> wrote:
>> I'd like to know if that kind of code is acceptable or not:
>> 
>> (format "http%s://elpa.gnu.org/packages/"
>>                                        (if (gnutls-available-p) "s" ""))
> 
> The standard localization argument against concatenating an "s" onto a
> word does not apply here. But, one day, someone is going to grep for
> occurrences of http:// to see if it’s still used anywhere.
> 
> That someone could be you.
> 
> Spell it out.
> 
>    (defvar …-elpa-root
>      (if (gnutls-available-p)
>          "https://elpa.gnu.org/packages/"
>        "http://elpa.gnu.org/packages/")
>      "…some doc…. http and https schemes are supported;
>    https is conditional on GnuTLS.")

Re: format use inquiry

[Glenn Morris]

Jean-Christophe Helary wrote:

> I'd like to know if that kind of code is acceptable or not:
>
> (format "http%s://elpa.gnu.org/packages/"
>                                         (if (gnutls-available-p) "s" ""))

Yes, it is. But obviously I think that, since I wrote it.

Re: format use inquiry

[Jean-Christophe Helary]

> On Jun 21, 2017, at 7:53, Glenn Morris <rgm@gnu.org> wrote:
> 
> Jean-Christophe Helary wrote:
> 
>> I'd like to know if that kind of code is acceptable or not:
>> 
>> (format "http%s://elpa.gnu.org/packages/"
>>                                        (if (gnutls-available-p) "s" ""))
> 
> Yes, it is. But obviously I think that, since I wrote it.

Thank you for the comment.

Actually I've been checking package.el for code that is used to simulate natural language features, after finding a number of issues with plurals and singulars.

I started doing that at the same time the discussion on i18n and l10n started because when we have the i18n infrastructure in place we'll have to rethink the way messages are written.

The above example is not directly related to that effort since it won't be a string used for translation but it illustrates a strong tendency in package.el where you can find "s" added to a string if the related value is 2 or above, "ing" or "ed" added to a string depending on the status of the operation, etc. That makes for really smart code but it is absolutely untranslatable.

So a few weeks ago I set myself to straighten up the problematic strings and as far as the one I inquired about here is concerned I'd rather have the urls are fully spelled out because it makes the code more readable.

Would you object to that ?

Jean-Christophe

Re: format use inquiry

[Paul Eggert]

On 06/20/2017 05:20 PM, Jean-Christophe Helary wrote:
>> (format "http%s://elpa.gnu.org/packages/" (if (gnutls-available-p) 
>> "s" ""))
 > I'd rather have the urls are fully spelled out because it makes the 
code more readable.

I'd rather not see two copies of essentially-identical URLs, as that's a 
maintenance hassle. Perhaps we could use a function that replaces http: 
with https: as needed.

Re: format use inquiry

[Jean-Christophe Helary]

> On Jun 21, 2017, at 10:23, Paul Eggert <eggert@cs.ucla.edu> wrote:
> 
> On 06/20/2017 05:20 PM, Jean-Christophe Helary wrote:
>>> (format "http%s://elpa.gnu.org/packages/" (if (gnutls-available-p) "s" ""))
> > I'd rather have the urls are fully spelled out because it makes the code more readable.
> 
> I'd rather not see two copies of essentially-identical URLs, as that's a maintenance hassle. Perhaps we could use a function that replaces http: with https: as needed.

That makes sense. Thank you.

Jean-Christophe

Re: format use inquiry

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > (format "http%s://elpa.gnu.org/packages/"
  > >                                         (if (gnutls-available-p) "s" ""))

  > Yes, it is. But obviously I think that, since I wrote it.

I agree it is acceptable, but this looks like something that might be
needed in a number of places, so perhaps we should make a nicer
interface to do it.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: format use inquiry

[Jean-Christophe Helary]

> On Jun 22, 2017, at 10:57, Richard Stallman <rms@gnu.org> wrote:
> 
>>> (format "http%s://elpa.gnu.org/packages/"
>>>                                        (if (gnutls-available-p) "s" ""))
> 
>> Yes, it is. But obviously I think that, since I wrote it.
> 
> I agree it is acceptable, but this looks like something that might be
> needed in a number of places, so perhaps we should make a nicer
> interface to do it.

There is only one instance of such a query in package.el and no other package in the emacs distribution seems to use gnutls-available-p to add an "s" to http.

Paul suggested that it would be a maintenance hassle to keep 2 almost identical urls if we spelled them out as Yuri suggested but I checked other source packages and for ex auth-source-pass-tests.el explicitly spells out all the urls without resorting to smart formatting to save a few characters.

Also, considering the way %s is abused in other places package.el, for ex in:

 (message "%d package%s marked for upgrading."                                                               
        (length upgrades)                                                                                          
        (if (= (length upgrades) 1) "" "s"))))) 

I don't think it is good to keep the above code because it gives bad incentives to authors especially if work on i18n/l10n proceeds (even though http/https is not related to l10n).

So, I'm going to spell out the urls as I proceed with untangling code and translatable strings in package.el. I'll send a diff here when I'm done for evaluation.

Jean-Christophe 

RE: format use inquiry

[Drew Adams]

> Paul suggested that it would be a maintenance hassle to keep 2 almost
> identical urls if we spelled them out as Yuri suggested but I checked other
> source packages and for ex auth-source-pass-tests.el explicitly spells out
> all the urls without resorting to smart formatting to save a few characters.
> 
> Also, considering the way %s is abused in other places package.el, for ex
> in:
> 
>  (message "%d package%s marked for upgrading."
>         (length upgrades)
>         (if (= (length upgrades) 1) "" "s")))))
> 
> I don't think it is good to keep the above code because it gives bad
> incentives to authors especially if work on i18n/l10n proceeds (even though
> http/https is not related to l10n).
> 
> So, I'm going to spell out the urls as I proceed with untangling code and
> translatable strings in package.el. I'll send a diff here when I'm done for
> evaluation.

FWIW, I think this is a bit misguided.  It seems to
be putting translation/localization interests before
code-readability - IOW, making things simpler for some
tools or secondary uses of code, rather than keeping
them simple for someone to read.

I'm not arguing maintenance burden but readability
(which also affects maintenance burden).

Maybe some sacrifice of code simplicity is needed in
the interest of translation-help.  But I think another
approach should be sought than what you've proposed.

Code, especially Lisp code, is partly about expressing
something symbolically, to yourself and others who read
the code.

If you factor out the part of two things that is common,
that makes it easy to _see_ what is common and what is
different.

If you instead use two URLs that are almost identical
then you make a human reader scan each of them looking
for differences:

(if (gnutls-available-p)
    "https://elpa.gnu.org/packeges/";
  "http://elpa.gnu.org/packages/";)

(Did you spot the typo?)

Here the URLs are pretty short, so you might not
get a headache immediately, scanning and comparing.
But imagine the mental load if they were a lot longer.
Or if you had to scan and compare lots of them.
Spend an evening doing that and you might change your
mind about how great it is to duplicate all of that
common text.

You could bind a variable to name each one:

(let ((https "https://elpa.gnu.org/packages/")
      (http   "http://elpa.gnu.org/packages/"))
  (if (gnutls-available-p) https http))

(You can come up with better var names.)

That still makes someone scan and compare, but
it at least points out (names) what the expected
difference is.  I don't claim it doesn't suffer
from what I complain about above, but it seems a
bit less burdensome to me.

There might be other approaches.  None of this
matters much if we're talking only about someone's
preferred style here or there.  But if we're
talking about a wholesale change then maybe some
better approach can be found.

Just one opinion.

As for the question of messages that use singular
vs plural forms, I'd again point to Common Lisp's
`format', which addresses that kind of thing (at
least for English).

Re: format use inquiry

[Jean-Christophe Helary]

> On Jul 1, 2017, at 11:32, Drew Adams <drew.adams@oracle.com> wrote:
> But if we're
> talking about a wholesale change then maybe some
> better approach can be found.

I am certainly not talking about a *wholesale change* but about *1* instance in the emacs sources that uses that pattern.

Also, I don't think that *code* is especially bad, but considering what I have seen in the rest of the package.el code, this looks more like a way of thinking about natural language strings from the authors than about some kind of natural lispy rationalization.

Btw, I like your:

(let ((https "https://elpa.gnu.org/packages/")
     (http   "http://elpa.gnu.org/packages/"))
 (if (gnutls-available-p) https http))

That's something like that that I had in mind.

I am not conflating l10n and anything else btw. But it happens that authors who tangle code and strings in general do that all the time without considering whether the strings they create are internal or external strings. That leads to real problems: my first interest in package.el came from a number of singular cases that were not handled, then I checked the code and I found the external strings had *lots* of problems (and lots of "rationalizations" to save a few lines that ended up creating strings *totally* unlocalizable).

> Just one opinion.

And mine was equally just one opinion, based on greping about a 100 other .el files to find similar patterns, where I found that (seemingly) robust code would prefer redundancy to smarty formatting.

You may want to compare the tons of redundant urls in auth-source-pass-tests.el to that one instance where using format saved the code a few characters (and I'm really not sure it adds much in terms of readability).

> As for the question of messages that use singular
> vs plural forms, I'd again point to Common Lisp's
> `format', which addresses that kind of thing (at
> least for English).

Again ? Sorry if I missed the other references.

That -P flag for CL format contributes to producing just as ugly code as what we have here. Code should never be used to create natural language strings with syntactic expectation.

Here is the example given in "Common Lisp the Language, 2nd Edition"

(format nil "~D tr~:@P/~D win~:P" 7 1) => "7 tries/1 win" 
(format nil "~D tr~:@P/~D win~:P" 1 0) => "1 try/0 wins" 
(format nil "~D tr~:@P/~D win~:P" 1 3) => "1 try/3 wins"

How do you think that kind of strings can possibly be localized ?

Jean-Christophe 

RE: format use inquiry

[Drew Adams]

> > As for the question of messages that use singular
> > vs plural forms, I'd again point to Common Lisp's
> > `format', which addresses that kind of thing (at
> > least for English).
          ^^^^^^^^^^^
> That -P flag for CL format contributes to producing
> just as ugly code as what we have here.

Code using `format' is ugly - sure.  But it sure is
convenient too.  It's a bit like using Lisp `loop' or
using `find' in UNIX, GNU, etc.  Or using regexps.

For better _and_ worse, `format' is practically a
language unto itself - especially the more complex
`format' of Common Lisp.  Each of `loop', `find', and
regexps is a language, and in each case the result of
using it can be compact but esoteric code.

If you prefer to use conditional code with `concat'
etc. instead of `format', feel free.  I don't see why
existing code that uses `format' should be replaced
with code that does similar conditional processing
that is more explicit and verbose.

> Code should never be used to create natural language
> strings with syntactic expectation.

In that case, a powerful alternative should be developed
and proposed.  It's clear that `format' & compagnie are
_not_ the right tools for natural-language processing,
including for translation help.  They were not designed
for that.  There is nothing in Emacs Lisp that provides
such a tool, AFAIK.

If we start tearing apart existing Lisp code because it
handles messages, menus, titles, doc, etc. in a way that
does not facilitate natural-language treatment (including
localization), that could just create a mess.  Let simple
sleeping dogs lie.

Instead, someone interested in that aspect of things
would do well to work on creating powerful Emacs-Lisp
constructs that really _do_ facilitate natural-language
treatment.  IOW, try to come up with something that is
a reasonable alternative to the rudimentary constructs
that we use today.

That's probably a hard thing to do, but if that's where
someone's interest is, it could be a worthwhile endeavor.

And with decades of natural language processing and
localization research behind us now, perhaps there is
some existing code out there (perhaps not Emacs Lisp)
that could serve as inspiration, if not as a direct
model.

I don't think anyone would argue that `format' has
what it takes to help with handling multiple languages
cleanly.  I certainly wouldn't.  But it doesn't follow
that we should now try to recode existing uses of it
to recompose text into full words, in the interest of
some potential future localization.

Better would be to start localizing, and make whatever
changes are truly helpful immediately, here or there,
as needed.  IOW, demand-driven vs eager, proactive
change in ways that someone thinks might help.

> Here is the example given in "Common Lisp the Language,
> 2nd Edition"
> (format nil "~D tr~:@P/~D win~:P" 7 1) => "7 tries/1 win"
> (format nil "~D tr~:@P/~D win~:P" 1 0) => "1 try/0 wins"
> (format nil "~D tr~:@P/~D win~:P" 1 3) => "1 try/3 wins"
> 
> How do you think that kind of strings can possibly be localized ?

I said: "at least for English".  I was not proposing
`format', Common Lisp or otherwise, as a tool or
solution for localization.

But if some such code really did need to be localized
then the answer would presumably be to evaluate it - as
you have done, to see the effect/result, and then code
appropriately to produce an appropriate effect/result
for other languages.

Any way you look at it, in such a situation, regardless
of how the first-language treatment is coded, you would
presumably need to translate each of the possible outputs.

As long as there is no such built-in `format' handling
for other languages, their handling would need to be
done using some condition tests (if-then-else etc.).

That part would presumably follow what you would prefer.
But why should it be used also for the English part, if
a simpler expression is available for English?

And clearly such brute manipulation of strings for
natural language is a sledgehammer, whether it uses
`format' or explicit, verbose conditional code.  As you
point out, this is not the way to do natural-language
processing.

IOW, it is not `format' or its use that is the problem.
We don't have natural-language processing constructs
in Emacs Lisp.

In the absence of general natural-language help for
Emacs Lisp, a better short-term treatment might be for
someone to extend `format' (or similar) so that it can
handle this or that other language - IOW, to give
French, etc. a similar advantage wrt composing messages
that handle both singular and plural etc.

No, that would not be the right tool for localization
or general natural-language processing either.  But it
would at least give French etc. the same advantage of
compactness for formatting simple messages.

[One reason that something like `format', instead of
some more systematic, language-oriented construct,
has been used for English messages could be that
English is so irregular.  (1) It can be used, simply.
(2) It's not easy to come up with something more
systematic, for English.  It does the simple job,
and it's hard to do a more systematic job.

I expect that it would be simpler to handle
plurality, etc. in a more regular language, such as
French.  In English, there are very few useful rules
to take advantage of, so the treatment by CL `format'
is both (a) rudimentary and (b) about as good as it
can get while remaining simple.]

Anyway, again, I was not proposing Common-Lisp
`format' for any purpose of localization.  I'm a fan
of it for English, for simple things, and I think
Emacs might benefit from it or something similar.

My mention of it was essentially off-topic for your
thread about localization.  (But it could still be
somewhat on-topic for a thread about `format'.)

Re: format use inquiry

[Jean-Christophe Helary]

> On Jul 2, 2017, at 0:43, Drew Adams <drew.adams@oracle.com> wrote:

> My mention of (the -P flag in CL) was essentially off-topic for your
> thread about localization.  (But it could still be
> somewhat on-topic for a thread about `format'.)

This thread happens to be about "format", as the title and original exchanges indicate.

I was explicitly asking about a specific use of format in url creation that has no relation to l10n and got several replies ranging from "don't spell out the urls" to "do spell out the urls"...

After reviewing the whole emacs lisp code found in the distribution I found that no other files in the whole distribution used a format trick to add an "s" to "http" when creating "https" urls (especially *none* of the code that was heavily used for https etc.), and I also found that all the places where both http and https urls were present they were all fully spelled out.

So considering that even if the code is valid and considered acceptable, 1) general use of 40 years of elisp having not produced such distribution code and 2) the package.el code being ridden with such "smart uses" of format/concat and the like that in fact created errors in external strings (I did not check the internal strings), the conclusion was that such code should be fixed so that other developers who check it don't get bad ideas. Unless you have more powerful arguments, that kind of settles the issue for me.

In a different thread I also mentioned that I started straightening out strings in package.el because I had found plural/singular errors in it fully *knowing* that i18n is not available so that any "micro" l10n preparation of this type is bound to not be super useful *right now*, but currently that's the only part where I can contribute (I put aside the few lines of trivial code that I was allowed to contribute in ns-win and subr-x).

What I want to reach is a state where Emacs is fully localizable. There are many entry points and few people interested in producing relevant code, so I have to do with what I have (no knowledge of C but I'm starting to work on that, little knowledge of elisp, 20 years of localization work as an amateur in FOSS projects, 15 as a professional).

Last June, Philipp Stephani has implemented "field numbers for `format' so that argument indices can be explicitly specified" to take care of word order modifications (among other issues). I don't think a lot of people have started using that but that's an excellent and necessary start. We need a way to gettext to extract external strings, we need a way to display localized strings, we need to straighten out strings so that they are extractable by gettext and we need to localize them (including docstrings eventually). l10n itself is a huge endeavor: based on the state of the manuals, I'm estimating the volume at 500,000 words for the code alone, and 2,000,000 words total when including the manual. So the code that allows for that is actually a relatively small part of the work. But without the code, we're stuck.

So if you have an interest in the endeavor and concrete ideas, please come forward: do as Philipp did by producing relevant code, or do as I do, by fixing little things that will need to be fixed anyway.

Jean-Christophe