Uploaded image for project: 'CFEngine Community'
  1. CFEngine Community
  2. CFE-1606

software_packages.csv is always regenerated, even when cache is recent

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.6.2
    • Component/s: Promise type: packages
    • Labels:
      None

      Description

      There seems to be a logic error in checking the cache of installed packages, so that even when the cache is new (and CFEngine reports so), the package list is read from the command instead of from the file, and the file is updated. This causes a severe performance decrease in packages promises with respect to previous versions, enough to prevent updating for 3.6 in large infrastructures.

      Sample file attached, and run logs below. Notice how the timestamp of /var/cfengine/state/software_packages.csv updates after each run. Note how CFEngine reports that the cache is out of date, but also reports that the timestamp is within the window.

      <pre>
      [root@localhost vagrant]# time cf-agent -KIv -f ./test-packages-3.6.cf | egrep -i 'state/software|Reading package'
      2014-07-30T08:09:24+0000 verbose: /default/test/packages/'gcc'[0]: Cache file '/var/cfengine/state/software_packages.csv' exists, but it is out of date (package_list_update_ifelapsed)
      2014-07-30T08:09:24+0000 verbose: /default/test/packages/'gcc'[0]: '/var/cfengine/state/software_update_timestamp_yum' is less than 240 minutes old. Not updating package list.
      2014-07-30T08:09:24+0000 verbose: /default/test/packages/'gcc'[0]: Reading package list from '/bin/rpm'

      real 0m1.665s
      user 0m1.063s
      sys 0m0.278s
      [root@localhost vagrant]# ls -lrt /var/cfengine/state/software_*
      rw------ 1 root root 0 Jul 30 07:47 /var/cfengine/state/software_update_timestamp_yum
      rw-rr- 1 root root 19257 Jul 30 08:09 /var/cfengine/state/software_packages.csv
      [root@localhost vagrant]# time cf-agent -KIv -f ./test-packages-3.6.cf | egrep -i 'state/software|Reading package'
      2014-07-30T08:10:29+0000 verbose: /default/test/packages/'gcc'[0]: Cache file '/var/cfengine/state/software_packages.csv' exists and is sufficiently fresh according to (package_list_update_ifelapsed)
      2014-07-30T08:10:29+0000 verbose: /default/test/packages/'gcc'[0]: '/var/cfengine/state/software_update_timestamp_yum' is less than 240 minutes old. Not updating package list.
      2014-07-30T08:10:29+0000 verbose: /default/test/packages/'gcc'[0]: Reading package list from '/bin/rpm'

      real 0m1.680s
      user 0m1.070s
      sys 0m0.279s
      [root@localhost vagrant]# ls -lrt /var/cfengine/state/software_*
      rw------ 1 root root 0 Jul 30 07:47 /var/cfengine/state/software_update_timestamp_yum
      rw-rr- 1 root root 19257 Jul 30 08:10 /var/cfengine/state/software_packages.csv
      </pre>

      Compare the run times with those from CFEngine 3.4.4 (the software_packages.csv file is not being updated every time):

      <pre>
      [root@localhost vagrant]# ls -lrt /var/cfengine/state/software_packages.csv
      rw-rr- 1 root root 18737 Jul 30 08:12 /var/cfengine/state/software_packages.csv
      [root@localhost vagrant]#
      [root@localhost vagrant]# time cf-agent -KI -f ./test-packages.cf

      real 0m0.236s
      user 0m0.152s
      sys 0m0.015s
      [root@localhost vagrant]# time cf-agent -KI -f ./test-packages.cf

      real 0m0.218s
      user 0m0.150s
      sys 0m0.010s
      [root@localhost vagrant]# ls -lrt /var/cfengine/state/software_packages.csv
      rw-rr- 1 root root 18737 Jul 30 08:12 /var/cfengine/state/software_packages.csv
      </pre>

        Attachments

        1. patch.diff
          0.7 kB
        2. test-packages-3.4.cf
          0.3 kB
        3. test-packages-3.6.cf
          0.3 kB

          Issue Links

            Activity

              People

              • Assignee:
                a10040 Kristian Amlie
                Reporter:
                zzamboni Diego Zamboni
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel