From: Attila Lendvai <attila@lendvai.name>
To: Simon Tournier <zimon.toutoune@gmail.com>
Cc: Guix Devel <guix-devel@gnu.org>
Subject: Re: [maintenance] Compressed JSON files and served file extension?
Date: Mon, 20 Nov 2023 14:03:13 +0000 [thread overview]
Message-ID: <himBVN7i-jtLo06KXENDJ1S-CMKIGL2Kq-ddcbriHxS9O2DRyFWT0ijV8LaIQEz20zaNOXwMgeyl1iZslfBmRaq5gH_0hSFGMD80-mS9i7s=@lendvai.name> (raw)
In-Reply-To: <861qe03rtj.fsf@gmail.com>
TL;DR the filename shouldn't contain the .gz extension, and the HTTP standard is crap ("If no Accept-Encoding field is present in a request, the server MAY assume that the client will accept any content coding.").
use curl --compressed
the details:
the Content-Encoding response header instructs the client on how to decode the __transfer payload__ that the server is serving.
i.e. proper HTTP clients should automatically decode the content as instructed by the Content-Encoding response header, or at the very least warn that they do not understand the response encoding...
but that should not happen, because the HTTP request can contain an Accept-Encoding header that tells the server what the client understands, and it defaults to unprocessed raw data ('identity')...
except that the standard allows the server to ignore the Accept-Encoding request header.
well, this is the theory, but both wget and curl don't automatically decode the content. curl at least can be instructed to do so, which arguably should be its default:
curl --compressed https://guix.gnu.org/sources.json | less
--verbose can be used to inspect the reques/response headers (printed to stderr):
curl --verbose https://guix.gnu.org/sources.json >/dev/null
curl --verbose --compressed https://guix.gnu.org/sources.json >/dev/null
here's a detailed discussion of this very question:
https://stackoverflow.com/questions/8364640/how-to-properly-handle-a-gzipped-page-when-using-curl
so, in an ideal world wget and curl should transparently decode the content according to the Content-Encoding response header, and nginx should not respond with a compressed content when the client is not sending an Accept-Encoding request header.
the pragmatic solution is to use curl --compressed in scripts, and/or add it to your ~/.curlrc:
# to automatically decode responses with some of
# the supported Content-Encoding
compressed
HTH,
--
• attila lendvai
• PGP: 963F 5D5F 45C7 DFCD 0A39
--
“We need people in our lives with whom we can be as open as possible. To have real conversations with people may seem like such a simple, obvious suggestion, but it involves courage and risk.”
— Thomas Moore (1940–)
next prev parent reply other threads:[~2023-11-20 14:04 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-12 13:15 [maintenance] Compressed JSON files and served file extension? Simon Tournier
2023-11-16 14:40 ` Ludovic Courtès
2023-11-20 9:49 ` Simon Tournier
2023-11-28 13:48 ` Simon Tournier
2023-11-28 15:57 ` Tomas Volf
2023-11-20 14:03 ` Attila Lendvai [this message]
2023-11-28 13:37 ` Simon Tournier
2023-11-28 15:50 ` Tomas Volf
2023-11-28 21:05 ` Attila Lendvai
2023-11-29 9:08 ` Simon Tournier
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='himBVN7i-jtLo06KXENDJ1S-CMKIGL2Kq-ddcbriHxS9O2DRyFWT0ijV8LaIQEz20zaNOXwMgeyl1iZslfBmRaq5gH_0hSFGMD80-mS9i7s=@lendvai.name' \
--to=attila@lendvai.name \
--cc=guix-devel@gnu.org \
--cc=zimon.toutoune@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.