This has been discussed a few times on the mailing list. I run into this
problem more and more. It affects Cfengine's ability to work in large
Consider these examples.
A large template file
Suppose there is a large template file for ifcfg-eth0. It contains blocks and classes for 10,000 hosts.
[%CFEngine myhost10000:: %]
The above is a block for one host. For 10,000 hosts the file size is 100,000
lines. The agent takes over 4 minutes to process this template.
One option is to break this template into multiple smaller ones. I consider
this a poor work around. With policies of this scale, breaking one file into ten
reduces the simplicity of management.
There have been multiple discussions about list iteration and how the time to
do so increases dramatically beyond a few hundred. Such lists become more
common in large environments.
In the past flex or bison has been blamed for this 'limitation'. I believe it
is time to move beyond the limitation. Cfengine is touted as scaling better
than the competition. Indeed it does but, we now know that Cfengine has its own
scale limits. It's time we move beyond them.