Uploaded image for project: 'Mender'
  1. Mender
  2. MEN-3471

mender-convert: replace pxz with with xz --threads=0

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Open
    • Priority: (None)
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Environment:
      using latest mender-convert and its docker instance
    • Days in progress:
      0

      Description

      xz-utils 5.2.x now has multi-thread support and significantly cuts the compression time for the same compressed size file.As it looks like mender-convert docker image is based upon 19.10, then xz-utils 5.2.x is installed and has multi-threading option available. 

      On a quad core cpu with hypertheading (8 threads) the following was my findings with a 20GB mender disk image. 

      with xz -k -9 foo.img (single-thread, 20GB img file => 557MB in):
      real 27m32.065s
      user 27m23.612s
      sys 0m7.816s
      
      with xz -k -9 --threads=0 foo.img  (8 threads, 20GB img file => 565MB in):
      real 7m42.419s
      user 48m31.505s
      sys 0m18.220s
      
      with pxz --best --force foo.img (? threads, 20GB img file => 565MB in):
      real 13m4.858s
      user 37m59.443s
      sys 0m47.760s
      
      with pxz --best --force -T8 (8 threads, 20GB img file => 565MB in):
      real 12m59.884s
      user 37m48.260s
      sys 0m46.631s
      

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dell.green@ideaworks.co.uk Dell Green
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Summary Panel