From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Kangas Newsgroups: gmane.emacs.devel Subject: Re: emacs-29 9c0d7bb73b 2/2: Add automated tests for Eglot Date: Tue, 13 Dec 2022 11:23:52 -0800 Message-ID: References: <167080394233.4339.5863353994038854528@vcs2.savannah.gnu.org> <20221212001223.E9A9DC004BE@vcs2.savannah.gnu.org> <877cyvu1qo.fsf@gmx.de> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="21360"; mail-complaints-to="usenet@ciao.gmane.io" Cc: =?UTF-8?B?Sm/Do28gVMOhdm9yYQ==?= , emacs-devel@gnu.org To: Michael Albinus Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Dec 13 20:24:54 2022 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1p5Ata-0005OX-BN for ged-emacs-devel@m.gmane-mx.org; Tue, 13 Dec 2022 20:24:54 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p5Asq-0000tv-26; Tue, 13 Dec 2022 14:24:08 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p5Ase-0000t2-5w for emacs-devel@gnu.org; Tue, 13 Dec 2022 14:23:56 -0500 Original-Received: from mail-ot1-x329.google.com ([2607:f8b0:4864:20::329]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1p5Asc-0005U9-N4 for emacs-devel@gnu.org; Tue, 13 Dec 2022 14:23:55 -0500 Original-Received: by mail-ot1-x329.google.com with SMTP id db10-20020a0568306b0a00b0066d43e80118so475961otb.1 for ; Tue, 13 Dec 2022 11:23:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:mime-version:references:in-reply-to :from:from:to:cc:subject:date:message-id:reply-to; bh=IRS7qix9pHjyCn7DjPg2RiImch/EHDxPTXV1FeLsyHY=; b=cvnh3AwCR9DuA7eU8OMPSQWAgA/PISpaY0nzEyMXuHFzc22mAyexcwJSvYaZGt66WV LRdulQxVPhHRkalsPY2kxyeJYkrLVKBmxvrT0rRqMEMf8vRyzAT/TdnouH3Je+G9uzCw R4lCm1SGr7+olkpE6EepZ/O+ECJfcm1NUsoUvf1RvO85b8No5UugGWoQxgWQA3pCiIjp iOXzeJy7srYgc6RePOMB5txs2IbG0GltLgYRnuUO2dmXxT5Q3+c1Z1ZVS4vUXJs2A7z0 pGNP0gmSjcy2XJpzJx0QJF0QsbO7/7lipryfwzBWTdr4oGtjK96BeIac+MaGlEWCL9tx dO6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:mime-version:references:in-reply-to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=IRS7qix9pHjyCn7DjPg2RiImch/EHDxPTXV1FeLsyHY=; b=dqmwQrcx+yp1OrkSwJrXtR+Bys3aqMmlnR8PvCBL28eWDkuIEoLdQVUSkGW+cE9ucj nGT/Z4ejI7YKxFekEfcULpYf+RiPgCesu1RvDkHIrx6WHBIttQU+SEigl7fSeUY2E/KT YVX0HdgzdqWLRivUuCPaSyxUyCr7JQQtUM8L0h0gpPFIrDxcgvbue7WG4lGqWSwYrx5n wVi1XJwgTszhjBS8SHaYcitvW76wjx+QQwOttoDfx+UYuQgmjo6ttAPsq3cXHDabGrYG nDFJzy0lufydYW9axyxGXUCF7lrS9GwJyTq6scsJC42wX7Oz7PQfMv98El+iGMkiEoUJ JJ4Q== X-Gm-Message-State: ANoB5plWXFRj46Q3DeOZKkLR4bhbRlKRj+COs5rMiO5ux9S0KBR1MNPd GYKTBRKdi4V62JPe5ImQSoN532DmCRNSMPuVh8Y= X-Google-Smtp-Source: AA0mqf6X//0nSrxq+3R+a2PBOGyTHqjXFMPHkjie5V5LmUl8u5gYxt+YWCcIRxmn16ORE37a7OFFyLThzynxNhfnIFM= X-Received: by 2002:a9d:6203:0:b0:66e:f6cb:cb4e with SMTP id g3-20020a9d6203000000b0066ef6cbcb4emr8633289otj.105.1670959432883; Tue, 13 Dec 2022 11:23:52 -0800 (PST) Original-Received: from 753933720722 named unknown by gmailapi.google.com with HTTPREST; Tue, 13 Dec 2022 11:23:52 -0800 In-Reply-To: <877cyvu1qo.fsf@gmx.de> X-Hashcash: 1:20:221213:joaotavora@gmail.com::xEqKgQGSSgEWic4K:5U28 Received-SPF: pass client-ip=2607:f8b0:4864:20::329; envelope-from=stefankangas@gmail.com; helo=mail-ot1-x329.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:301353 Archived-At: Michael Albinus writes: >> I'd basically consider any test over half a second or a second or so as >> a candidate for being marked as :expensive. But we should do it >> judiciously, of course. (And even better if we can find a way to make >> slow tests faster.) > > That's a very restrictive rule. IIRC, we've discussed this a while ago, > and a recommendation was to regard tests lasting more than 10 seconds > being expensive. But I might remeber this wrong. It's just a rule of thumb that I use to decide which tests to consider marking. We should of course not do that in a blanket fashion, because we really do want some tests to always run, even if they take a long time. But the value of the test suite is decreased if it is slow by default, as people will avoid running it. We are already seeing a not insignificant number of new test failures regularly introduced on master and the release branch. This suggests to me that this might already be taking place. (When dealing with automatic merging, as I do, a broken test suite is a nuisance. I don't have any hard data unfortunately, but over the last 3-6 months it feels to me like the test suite has been broken more often than it's not. Perhaps this observation could be checked against the EMBA logs or similar.) >> passed 12/50 eglot--tramp-test (2.158451 sec) > > All eglot Tramp tests could be regarded as :expensive by default, I > guess. I don't know enough about Tramp to have an opinion, myself. But please do it if you think it makes sense, thanks.