[Caml-list] Re: Memory fragmentation

Article View: fa.caml

Article #121

#121

From: SerP
Date: Fri, 20 Apr 2012 15:27

31 lines
4647 bytes

Ocaml 3.12.0., Lwt 2.3.1
On Apr 20, 2012 7:12 PM, "SerP" <serp256@gmail.com> wrote:

> Hi!
> We have developped a daemon on ocaml using the lwt lib(libev). It
> processes about 800 requests per second, but it increases to 2 Gb in memory
> for a hour of work. We have studied the problem for a long time, using
> mtrace, mallinfo and other tools, and we tried to change GC params. We
> found out that setting up GC.minor_heap_size=10Mb �
> major_heap_increment=2�b will make the daemon grow slower. mallinf� shows
> that the memory (1,5G) is allocated using mmap (blkhd field in mallinfo
> struct) and arena size - about 20M In this case, first calls of GC.compact
> compress the daemon to 200 Mb (live -110Mb) , and mallinfo show decreasing
> of total size of memory allocated by mmap. The daemon keeps growing, but it
> allocates the memory to arena (using brk), and calls of GC.compact leads to
> decrease of heap_bytes to 200M, but arena size (1.5 Gb) does not decrease,
> just doing less uordblks and more fordblks (fileds I.e., after first calls
> of GC.compact, the daemon starts using brk much more than mmap, and cant
> memory given back to the system.
>
> If Gc.allocation_policy is first fit, memory usage is stable, and it's
> grow very little, but worked very slowly.
>
> Any suggestions are very welcome. Thanks
>

-- 
Caml-list mailing list.  Subscription management and archives:
https://sympa-roc.inria.fr/wws/info/caml-list
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs

Technical Details

Message-ID: <fa.+H73UKud8lHoMnLtXf00xXEWw/s@ifi.uio.no>

Path:

arc1-ca.pugleaf.net!r9ni78447pbh.0!nntp.google.com!news2.google.com!goblin3!goblin1!goblin.stu.neva.ru!uio.no!nntp.uio.no!.POSTED!not-for-mail

References: <fa.BQPDkp4GTICgezqMipjyzfGHFCk@ifi.uio.no> <fa./QhU1susg5I2sxeCxgPZFPt/8n0@ifi.uio.no>

Show Full Headers

Complete Article Headers

X-Google-Thread: ff861,73392fc8a24639f4
X-Google-NewGroupId: yes
X-Google-Attributes: gidc0fe4b917e,domainid0,public,usenet
X-Google-Language: ENGLISH,ASCII
Received: by 10.68.129.169 with SMTP id nx9mr5965263pbb.2.1334935662340;
        Fri, 20 Apr 2012 08:27:42 -0700 (PDT)
Path: arc1-ca.pugleaf.net!r9ni78447pbh.0!nntp.google.com!news2.google.com!goblin3!goblin1!goblin.stu.neva.ru!uio.no!nntp.uio.no!.POSTED!not-for-mail
From: SerP <serp256@gmail.com>
Newsgroups: fa.caml
Subject: [Caml-list] Re: Memory fragmentation
Date: Fri, 20 Apr 2012 15:27:41 UTC
Organization: Internet mailing list
Lines: 39
Message-ID: <fa.+H73UKud8lHoMnLtXf00xXEWw/s@ifi.uio.no>
References: <fa.BQPDkp4GTICgezqMipjyzfGHFCk@ifi.uio.no> <fa./QhU1susg5I2sxeCxgPZFPt/8n0@ifi.uio.no>
NNTP-Posting-Host: mail-jess.uio.no
Mime-Version: 1.0
X-Trace: readme.uio.no 1334935661 18108 129.240.7.9 (20 Apr 2012 15:27:41 GMT)
X-Complaints-To: abuse@uio.no
NNTP-Posting-Date: Fri, 20 Apr 2012 15:27:41 +0000 (UTC)
To: caml-list@inria.fr
X-IronPort-AV: E=Sophos;i="4.75,453,1330902000"; 
   d="scan'208";a="154910381"
X-Authentication-Warning: walapai.inria.fr: sympa set sender to caml-list-owner@inria.fr using -f
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AtoBACl/kU/RVdW2kGdsb2JhbABDhWerSQgiAQEBAQkJDQcUBCOCCQEBAQMBEgIMIAEBGh4ECwsEBzshAQERAQUBHBkIGodeAQMGBZwXCopkZQZTgnMBhTcKGScNV4h2Bol1g02CC4EaiGGFModqizyDIT2EDw
X-IronPort-AV: E=Sophos;i="4.75,453,1330902000"; 
   d="scan'208";a="140962648"
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=googlemail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=de6BduPanxcH1dVCQJ7LUVSnG+Ms4LWNKA/lu5PIRK4=;
        b=oHcBlnepmq4uJppBStF7IOXzDA0ZMO1mQ0K9zhGPlkJceBCdd44gPYvJQLMIusT2Pe
         i+hycR5GcZ6QNZkDOlaTrPzv8cJLYHjeZWkovRkZxGE8Aej0a42eyDR9+1PhwV8+KwsT
         xUUT//jUuA2dS6kbU53Yjxi8WfnWxB8PhC8J5nZolClk5X3YdutBvY26Mj/rqve9s7X7
         wXp+VwMSj+uxZmvnHmrlqXsAOVucCmLYFwqzOq3aNs/End5BFAmt0HJeCgO70u+n9gu2
         uCXIezZHGylrUxKvfHsG0N+X5v2xpm7ySjD9BEEeX5x874COz34MQgyQV80mV74ZXkY8
         PbKQ==
In-Reply-To: <CAHgca+_OD0twkZ6WPc2bkQtEAS6Qnj1ogTDLE-S4esw6GKAV9A@mail.gmail.com>
X-Sequence: 3947
List-Id: <caml-list.inria.fr>
List-Archive: <https://sympa-roc.inria.fr/wws/arc/caml-list>
List-Help: <mailto:sympa@inria.fr?subject=help>
List-Owner: <mailto:caml-list-request@inria.fr>
List-Post: <mailto:caml-list@inria.fr>
List-Subscribe: <mailto:sympa@inria.fr?subject=subscribe%20caml-list>
List-Unsubscribe: <mailto:sympa@inria.fr?subject=unsubscribe%20caml-list>
X-FA-Processed: Removed HTML alternative
Original-Date: Fri, 20 Apr 2012 19:27:11 +0400
Original-Message-Id: <CAHgca+_5QrE_kMJpaGjWxTOWnnjJuRmvRxgY6Lbt9FwHky905Q@mail.gmail.com>
Original-References: <CAHgca+9-K71NynNVzS0Stnb2v5Tc6n5ecCnJHZorhuB8y8EdTw@mail.gmail.com>
 <CAHgca+_OD0twkZ6WPc2bkQtEAS6Qnj1ogTDLE-S4esw6GKAV9A@mail.gmail.com>
Content-Type: text/plain; charset=KOI8-R
Content-Transfer-Encoding: 8bit

Reply to Parent Message