Uploaded image for project: 'Mender'
  1. Mender
  2. MEN-2335

HTTP 2.0 Golang library leaks memory (client needs vendor update)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: 1.6.0
    • Fix Version/s: 2.0.0
    • Labels:

      Description

      From mailing list:

      We distribute .mender from an nginx server.

      We recently changed server and as part of the upgrade, HTTP/2 was enabled on the new server.

      Using mender client 1.6, we saw updates starting to fail with Out of memory error in the mender client.

      Watching mender installing an update using top, I was surprised to see the RSS consumption grow up to 500+ MB for a .mender artifact of only 240MB.

      Since nothing else had changed, I tried installing the same artifact from the older server over HTTP 1.1. The RSS usage was a steady 10MB and the update succeeded.

      I then suspected a buggy HTTP/2 support in the mender http client. I disabled HTTP/2 on the new nginx server and retried an update: Steady 10MB RSS usage and success.

      I looked into the mender client git repos. There seems to be quite old http2 client that is vendored in. This http2 implementation is known to have memory leaks.

      Reference:
      https://github.com/golang/go/commit/125f52dfa8ce6df0380b0bdc66effb8afd697bda

      I would suggest updating the vendored http2 client in the mender client to fix this issue.

        Attachments

          Activity

            People

            • Assignee:
              lluis Lluís Campos
              Reporter:
              a10040 Kristian Amlie
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel