🚀 go-pugleaf

RetroBBS NetNews Server

Inspired by RockSolid Light RIP Retro Guy

Article View: fa.netbsd.tech.kern
Article #75

Re: pbulk using tmpfs

#75
From: David Laight
Date: Sun, 21 Apr 2013 08:58
45 lines
3819 bytes
On Sun, Apr 21, 2013 at 12:14:59AM +0000, Taylor R Campbell wrote:
>    Date: Sun, 21 Apr 2013 00:10:01 +0000
>    From: Taylor R Campbell <campbell+netbsd-tech-kern@mumble.net>
> 
>       Date: Sun, 21 Apr 2013 01:12:06 +0200 (CEST)
>       From: iMil <imil@home.imil.net>
> 
>       I've just witnessed it with a couple of packages:
> 
>       multimedia/vlc:
> 
>          CC       input/libvlccore_la-stream_memory.lo
>       /scratch/multimedia/vlc/work/.wrapper/bin/gcc: Cannot fork
>       /scratch/multimedia/vlc/work/.wrapper/tmp/logic: Cannot fork
>       /usr/pkg/bin/libtool: Cannot fork
> 
>       I guess I'm too short in memory to play that game :)
> 
>    You probably need to bump the maxproc ulimit (ulimit -p).  The default
>    soft limit is ridiculously low for modern machines, and does not scale
>    with the available RAM as it probably ought to.  I set it to 1044, the
>    default hard limit.
> 
> ...that is, assuming you are doing a highly parallel (high MAKE_JOBS
> or many pbulk slaves) build.  If not, it is curious that the failure
> should manifest in fork in particular and not in a variety of ways or
> in a way that is obviously related to memory exhaustion.

I'm fairly sure that fork work fail (cleanly) for 'lack of physical
memory' or 'lack of KVA', at least not without the world exploding
at the same time. So I'd guess you are just hitting the 'ulimit -p'
soft limit on the number of processes.

The default soft and hard limits for all the 'ulimit' values are 'wrong'
in just so many ways! Mostly because the 'hard' limits for each user
match the kernel hard limits - which are based on MAXUSERS (16).

I suspect some of the hard limits pre-date dynamic allocation of the
relevant structures, these days limiting on the dyanmic availability
of memory/swap and KVA would be more appropriate.

	David

-- 
David Laight: david@l8s.co.uk
Technical Details
Message-ID: <fa.KIMPpNHSN1BHONC2EhpRBsupBaY@ifi.uio.no>
Path: arc1-ca.pugleaf.net!hg5ni6880wib.1!nntp.google.com!goblin3!goblin1!goblin.stu.neva.ru!uio.no!nntp.uio.no!.POSTED!not-for-mail
References: <fa.7mNkL/xC2zPn/oZkVVxJHCq1jJI@ifi.uio.no> <fa.8O0xRXNTLjFmHekkPvrAb6aiH/I@ifi.uio.no>

Show Full Headers
Complete Article Headers
X-Google-Thread: 10c6f4,786e9905c90c1180
X-Google-NewGroupId: yes
X-Google-Attributes: gidb7e13b6095,domainid0,public,usenet
X-Google-Language: ENGLISH,ASCII-7-bit
X-Received: by 10.180.37.83 with SMTP id w19mr13528404wij.7.1366534695062;
        Sun, 21 Apr 2013 01:58:15 -0700 (PDT)
Path: arc1-ca.pugleaf.net!hg5ni6880wib.1!nntp.google.com!goblin3!goblin1!goblin.stu.neva.ru!uio.no!nntp.uio.no!.POSTED!not-for-mail
From: David Laight <david@l8s.co.uk>
Newsgroups: fa.netbsd.tech.kern
Subject: Re: pbulk using tmpfs
Date: Sun, 21 Apr 2013 08:58:14 UTC
Organization: Internet mailing list
Lines: 45
Sender: tech-kern-owner@NetBSD.org
Message-ID: <fa.KIMPpNHSN1BHONC2EhpRBsupBaY@ifi.uio.no>
References: <fa.7mNkL/xC2zPn/oZkVVxJHCq1jJI@ifi.uio.no> <fa.8O0xRXNTLjFmHekkPvrAb6aiH/I@ifi.uio.no>
NNTP-Posting-Host: mail-jess.uio.no
Mime-Version: 1.0
X-Trace: readme.uio.no 1366534694 26052 129.240.7.9 (21 Apr 2013 08:58:14 GMT)
X-Complaints-To: abuse@uio.no
NNTP-Posting-Date: Sun, 21 Apr 2013 08:58:14 +0000 (UTC)
User-Agent: Mutt/1.4.2.1i
Cc: iMil <imil@home.imil.net>, Jan Danielsson <jan.m.danielsson@gmail.com>,
 tech-kern@NetBSD.org, tech-pkg@NetBSD.org
To: Taylor R Campbell <campbell+netbsd-tech-kern@mumble.net>
Delivered-To: tech-kern@NetBSD.org
X-Virus-Scanned: amavisd-new at NetBSD.org
Mail-Followup-To: Taylor R Campbell <campbell+netbsd-tech-kern@mumble.net>,
 iMil <imil@home.imil.net>,
 Jan Danielsson <jan.m.danielsson@gmail.com>, tech-kern@NetBSD.org,
 tech-pkg@NetBSD.org
In-Reply-To: <20130421001340.49AE5604E9@jupiter.mumble.net>
List-Id: tech-kern.NetBSD.org
Original-Date: Sun, 21 Apr 2013 10:10:56 +0100
Original-Message-Id: <20130421091056.GH24004@snowdrop.l8s.co.uk>
Original-References: <20130421000841.CD4A6604E9@jupiter.mumble.net> <20130421001340.49AE5604E9@jupiter.mumble.net>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline