cf-agent processes piling up waiting for LMDB

I'm seeing a build up of cf-agent procs on a RHEL6 host running CFEngine 3.6.3. Here's an lsof and gdb backtrace of one of the early processes. The host is a VM and may be having storage issues, will get more details on that.

<pre>

1. lsof -p 32741
   COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
   cf-agent 32741 root cwd DIR 253,0 4096 352257 /root
   cf-agent 32741 root rtd DIR 253,0 4096 2 /
   cf-agent 32741 root txt REG 253,1 414225 2395737 /var/cfengine/bin/cf-agent
   cf-agent 32741 root mem REG 253,1 57344 2392212 /var/cfengine/state/cf_lock.lmdb
   cf-agent 32741 root mem REG 253,0 383504 466954 /lib64/libfreebl3.so
   cf-agent 32741 root mem REG 253,0 65928 467323 /lib64/libnss_files-2.12.so
   cf-agent 32741 root mem REG 253,0 113432 467319 /lib64/libnsl-2.12.so
   cf-agent 32741 root mem REG 253,0 40400 466973 /lib64/libcrypt-2.12.so
   cf-agent 32741 root mem REG 253,0 113096 467125 /lib64/libaudit.so.1.0.0
   cf-agent 32741 root mem REG 253,0 14584 467093 /lib64/libutil-2.12.so
   cf-agent 32741 root mem REG 253,1 95063 2416753 /var/cfengine/lib/libz.so.1.2.8
   cf-agent 32741 root mem REG 253,1 57431 2416715 /var/cfengine/lib/libattr.so.1.1.0
   cf-agent 32741 root mem REG 253,0 1916528 466965 /lib64/libc-2.12.so
   cf-agent 32741 root mem REG 253,0 142464 467076 /lib64/libpthread-2.12.so
   cf-agent 32741 root mem REG 253,0 595800 467317 /lib64/libm-2.12.so
   cf-agent 32741 root mem REG 253,0 43832 467330 /lib64/librt-2.12.so
   cf-agent 32741 root mem REG 253,0 19536 467316 /lib64/libdl-2.12.so
   cf-agent 32741 root mem REG 253,0 52560 467068 /lib64/libnss_nis-2.12.so
   cf-agent 32741 root mem REG 253,0 55848 467236 /lib64/libpam.so.0.82.2
   cf-agent 32741 root mem REG 253,1 4268745 2416751 /var/cfengine/lib/libxml2.so.2.9.1
   cf-agent 32741 root mem REG 253,1 157841 2416744 /var/cfengine/lib/libpq.so.5.3
   cf-agent 32741 root mem REG 253,1 7230594 2416749 /var/cfengine/lib/libvirt.so.0.1001.3
   cf-agent 32741 root mem REG 253,1 753528 2416740 /var/cfengine/lib/libpcre.so.1.2.1
   cf-agent 32741 root mem REG 253,1 1662897 2416722 /var/cfengine/lib/libcrypto.so.0.9.8
   cf-agent 32741 root mem REG 253,1 365357 2416747 /var/cfengine/lib/libssl.so.0.9.8
   cf-agent 32741 root mem REG 253,1 166096 2416713 /var/cfengine/lib/libacl.so.1.1.0
   cf-agent 32741 root mem REG 253,1 2955175 2416746 /var/cfengine/lib/libpromises.so.3.0.6
   cf-agent 32741 root mem REG 253,0 154464 469389 /lib64/ld-2.12.so
   cf-agent 32741 root mem-r REG 253,1 320768 2392211 /var/cfengine/state/cf_lock.lmdb-lock
   cf-agent 32741 root mem REG 253,1 217016 1589255 /var/db/nscd/passwd
   cf-agent 32741 root mem REG 253,1 217016 1589257 /var/db/nscd/services
   cf-agent 32741 root mem REG 253,1 209313 2416724 /var/cfengine/lib/liblmdb.so
   cf-agent 32741 root 0r FIFO 0,8 0t0 676210295 pipe
   cf-agent 32741 root 1w FIFO 0,8 0t0 676210296 pipe
   cf-agent 32741 root 2w FIFO 0,8 0t0 676210296 pipe
   cf-agent 32741 root 4ur REG 253,1 320768 2392211 /var/cfengine/state/cf_lock.lmdb-lock
   cf-agent 32741 root 5u REG 253,1 57344 2392212 /var/cfengine/state/cf_lock.lmdb
   cf-agent 32741 root 6u REG 253,1 57344 2392212 /var/cfengine/state/cf_lock.lmdb

gdb backtrace:
Reading symbols from /usr/lib64/libfreebl3.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libfreebl3.so
0x00007f1fa0266054 in __lll_lock_wait () from /lib64/libpthread.so.0
Missing separate debuginfos, use: debuginfo-install audit-libs-2.2-2.el6.x86_64 glibc-2.12-1.80.el6_3.6.x86_64 nss-softokn-freebl-3.12.9-11.el6.x86_64 pam-1.1.1-10.el6_2.1.x86_64
(gdb) bt
#0 0x00007f1fa0266054 in __lll_lock_wait () from /lib64/libpthread.so.0
#1 0x00007f1fa0261388 in _L_lock_854 () from /lib64/libpthread.so.0
#2 0x00007f1fa0261257 in pthread_mutex_lock () from /lib64/libpthread.so.0
#3 0x00007f1fa1ecb252 in mdb_txn_renew0 (txn=0xc23670) at mdb.c:2202
#4 0x00007f1fa1ecb905 in mdb_txn_begin (env=0xb574c0, parent=0x0, flags=<value optimized out>, ret=0xa50c80) at mdb.c:2366
#5 0x00007f1fa1bf6aa1 in GetWriteTransaction (db=0x126dc40, txn=0x7fff92a3e908) at dbm_lmdb.c:122
#6 0x00007f1fa1bf732b in DBPrivDelete (db=0x7f1fa1e08040, key=0x7fff92a3e970, key_size=12728144) at dbm_lmdb.c:425
#7 0x00007f1fa1bf5ebc in DeleteDB (handle=0x7f1fa1db27e0, key=0x7fff92a3e970 "ce6d011fd796cb8826ce2db62a681240") at dbm_api.c:327
#8 0x00007f1fa1c1780a in RemoveLock (
name=0x8a6a90 "lock.fs_inputs.processes.comment.handle.signals.-atlgit01.proc_cf_monitord__norestart_6749_MD5=f554349cdb95bbf1ceddb052b989f0c0") at locks.c:262
#9 0x00007f1fa1c178ad in YieldCurrentLock (lock=...) at locks.c:780
#10 0x000000000041f170 in VerifyProcessesPromise ()
#11 0x000000000040ce9d in KeepAgentPromise ()
#12 0x00007f1fa1c0b565 in ExpandPromiseAndDo (ctx=0x8a66e0, pp=<value optimized out>, ActOnPromise=0x40ccb0 <KeepAgentPromise>, param=0x0)
at expand.c:207
#13 ExpandPromise (ctx=0x8a66e0, pp=<value optimized out>, ActOnPromise=0x40ccb0 <KeepAgentPromise>, param=0x0) at expand.c:161
#14 0x000000000040d36a in ScheduleAgentOperations ()
#15 0x000000000041c1ca in VerifyMethod ()
#16 0x000000000041c6f1 in VerifyMethodsPromise ()
#17 0x000000000040cfd1 in KeepAgentPromise ()
#18 0x00007f1fa1c0b565 in ExpandPromiseAndDo (ctx=0x8a66e0, pp=<value optimized out>, ActOnPromise=0x40ccb0 <KeepAgentPromise>, param=0x0)
at expand.c:207
#19 ExpandPromise (ctx=0x8a66e0, pp=<value optimized out>, ActOnPromise=0x40ccb0 <KeepAgentPromise>, param=0x0) at expand.c:161
#20 0x000000000040d36a in ScheduleAgentOperations ()
#21 0x000000000040db67 in KeepPromises ()
#22 0x000000000040ecf8 in main ()
</pre>