From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS3215 2.6.0.0/16 X-Spam-Status: No, score=-3.3 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from mail-it1-x130.google.com (mail-it1-x130.google.com [IPv6:2607:f8b0:4864:20::130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 3AD3420248 for ; Wed, 27 Feb 2019 13:26:09 +0000 (UTC) Received: by mail-it1-x130.google.com with SMTP id z124so9810792itc.2 for ; Wed, 27 Feb 2019 05:26:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxfoundation.org; s=google; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-disposition:in-reply-to:user-agent; bh=BvoGQ+vY0KOrpWXBVF8NzSKlxpvbpmfxDhyeM2tkOKs=; b=fifuuEOHH8zJB1c7WX8TKS3uFc3Nob3h9ioPsLR9HnWM2XzFSBnS/czhMAjG1Mo/Su aF5AILjenHpXc2+WZJSQVvXhaGz3PgRRcZ3L71kcCSg74x0Bydb+t3hR64oTWpyo+GeY j1/Y1OQF9beLE/MwrcrQeFHF0eD3GIFe1W+kI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to:user-agent; bh=BvoGQ+vY0KOrpWXBVF8NzSKlxpvbpmfxDhyeM2tkOKs=; b=pmx7GmXqQV2dnlec0DXVCNa0QH+UI9BqMxDShynO24fXyNeZdluwwncxsehD1KUAau 8AMo+dm5hIHCUrGw76YiR2DoYymCSWbp+ukNDTJDR0At3tUOYdHQtLDaQDk3SEk9xFBW 8xkOD5F9gXLfST43nanGeOO6mhdsee9rNlqy9dC5J10GH8scow7mDYqd9pG4yGbmaG13 zx+1Oa0Dzu20lISp/n66oJkFm5kpervyeKadN/fnaxobkP9J9DQNZaf/JOfhIwyD8Zpy ztYw9m9XXNZzZ5YTpgEP7/1A1T0c68gkGz4gI1FL+YcsMTA6JUU5xmNEx01plkJVB8Ye gaOQ== X-Gm-Message-State: AHQUAuaASYcMb2mzPcnBruY6woDOkje521jUib0AewTws2qxLOc3RGdN oUSXyyhtmRfaG2BnANYtnFEW1Q== X-Google-Smtp-Source: AHgI3IbjosgwTVVlX43zkfJ1AS4Esix7JsEKBO38hStmTqtvHQj/MaEpayVBwEEhHgJDXcSaAPT1YA== X-Received: by 2002:a02:c78d:: with SMTP id n13mr1163727jao.57.1551273967282; Wed, 27 Feb 2019 05:26:07 -0800 (PST) Received: from pure.paranoia.local ([198.144.156.51]) by smtp.gmail.com with ESMTPSA id k12sm6294704iop.8.2019.02.27.05.26.05 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 27 Feb 2019 05:26:06 -0800 (PST) Date: Wed, 27 Feb 2019 08:26:04 -0500 From: Konstantin Ryabitsev To: Eric Wong Cc: meta@public-inbox.org Subject: Re: V2 shard roll-over Message-ID: <20190227132604.GA24441@pure.paranoia.local> Mail-Followup-To: Eric Wong , meta@public-inbox.org References: <20190212191116.GA8720@chatter.qube.local> <20190212192715.mtncyagjzusi3vtp@dcvr> <20190227002204.GA17917@dcvr> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20190227002204.GA17917@dcvr> User-Agent: Mutt/1.11.3 (2019-02-01) List-Id: On Wed, Feb 27, 2019 at 12:22:04AM +0000, Eric Wong wrote: > Eric Wong wrote: > > Konstantin Ryabitsev wrote: > > > Eric: > > > > > > I noticed today that the LKML shard 6 has grown over 1.1 GB, which is the > > > size of other shards (0-5). I'm wondering if it will roll over to shard 7 > > > automatically, or if there are other steps that need to be undertaken. > > > > It only counts bytes in *.pack files; so you might need to repack > > (or wait for gc to run via --auto). > > Btw, have you checked this? I've been wondering if 7 will show up, too. Yeah, I've repacked it, but we still haven't rolled over to 7. This is the latest on the server: $ git count-objects -v count: 18 size: 88 in-pack: 1036749 packs: 1 size-pack: 1169104 prune-packable: 0 garbage: 0 size-garbage: 0 $ ls -al objects/pack/pack-7d2041260250f79f5d2396f38959560e013c8d26.pack -r--r--r--. 1 archiver archiver 1168133236 Feb 27 13:10 objects/pack/pack-7d2041260250f79f5d2396f38959560e013c8d26.pack I'm looking at the code and I'm not entirely sure what PACKING_FACTOR is: my $PACKING_FACTOR = 0.4; ... rotate_bytes => int((1024 * 1024 * 1024) / $PACKING_FACTOR), Wouldn't that give us 2.7GB? (1024*1024*1024)/0.4 = 2,684,354,560 It's possible I'm not following the logic right. It looks to be the same code that properly sharded things on the initial import, so I'm not sure. -K