ticket

Table of Contents

Huge ticket update, but this is important, so sorry [~a10003] and [~a10041], but I think you will have to read through most of it. However, you can skip all the sections marked "Behind the scenes" and move to the next header if you want. Those sections are there for seeing more of the implementation details (and for me to remember).

It's good that we talked about this on the planning, because I realized that the partial update epic has much wider ramifications than I first anticipated. The thing we (I) did not consider was that doing a partial update also has consequences for the other "non-naming" fields in artifact_provides, such as rootfs_image_checksum, which is used by delta updates to track which version to install. If left as it is now, then any partial update you install (to the data partition, for example) would ruin this field and prevent any further delta updates.

But I've worked on it a while now, and I think I've converged on something quite good. I'm now considering the "naming part" of partial updates more of a side track, and the main problem to solve is actually how to keep and lose the right artifact_provides values when doing a partial update.

This has to take into account quite a lot of different use cases, which we discussed at the planning, and it's not so easy to see from the "spec" how my proposal actually solves them, so I'll try to explain it from the use case perspective, basically describing how each one is realized. I have focused heavily on minimizing user involvement, and automating as much as I possibly can, which you'll see from the "Actions required" sections.

1 Base case: rootfs updates, nothing else

1.1 Actions required when installing Mender

None

1.2 Actions required when generating Artifact

None

1.3 Effect for the user

Each rootfs update has an artifact_name, as usual, and also a rootfs-image.software_name, which is the name of that software component. In this scenario, the two are always the same.

1.4 Behind the scenes

  • Mender-artifact assigns rootfs-image.software_name the same value as artifact_name, no action required by the user.
  • Mender-artifact also assigns rootfs-image.checksum by default, although this value will not be used for anything in this scenario.

2 Application updates without rootfs

This is just a subset of the next scenario, so it is implicitly solved by it.

3 Application updates with Update Modules, but only one app per Module, and rootfs

3.1 Actions required when installing Mender

None

3.2 Actions required when generating Artifact

None

3.3 Effect for the user

  • When installing a single-file update, the artifact_name and rootfs-image.single-file.software_name fields are updated.
  • When installing a docker update, the artifact_name and rootfs-image.docker.software_name fields are updated.
  • When installing a rootfs-image, the artifact_name and rootfs-image.software_name fields are updated, and the rootfs-image.single-file.software_name and rootfs-image.docker.software_name fields are cleared

3.4 Behind the scenes

  • Mender-artifact assigns rootfs-image.software_name the same value as artifact_name.
  • The single-file-artifact-gen tool assigns rootfs-image.single-file.software_name the same value as artifact_name.
  • The docker-artifact-gen tool assigns rootfs-image.docker.software_name the same value as artifact_name.
  • All three Update Modules, rootfs-image, single-file and docker, instruct the client to use the default update behavior, which is to erase all artifact_provides that match <PREFIX>.*, where <PREFIX> is the part preceding .software_name in the name of the artifact.

4 Using single-file Update Module for several different apps, and rootfs

4.1 Actions required when installing Mender

None

4.2 Actions required when generating Artifact

Need to specify application domain

4.3 Effect for the user

  • Installing "app1" using single-file results in artifact_name and rootfs-image.app1.software_name being updated
  • Installing "app2" using single-file results in artifact_name and rootfs-image.app2.software_name being updated
  • When installing a rootfs-image, the artifact_name and rootfs-image.software_name fields are updated, and the rootfs-image.app1.software_name and rootfs-image.app2.software_name fields are cleared

4.4 Behind the scenes

  • Mender-artifact assigns rootfs-image.software_name the same value as artifact_name.
  • single-file-artifact-gen either sets rootfs-image.app1.software_name or rootfs-image.app2.software_name to the same value as artifact_name, depending on which one was selected by the user on the command line.
  • single-file-artifact-gen sets an additional field in the meta-data section of the artifact, clears_artifact_provides=rootfs-image.app1.* or clears_artifact_provides=rootfs-image.app2.*, depending on which one was selected by the user on the command line. This is used by the client at install time to erase values in that namespace only.

5 Using several different OOTB Update Modules to update the same app

This is the same case as the previous scenario, the user just uses the same app domain for several different Artifact generators.

6 Delta Updates

6.1 Actions required when installing Mender

None

6.2 Actions required when generating Artifact

None

6.3 Effect for the user

  • Installing a rootfs-image update updates artifact_name and rootfs-image.software_name fields.
  • Installing a mender-binary-delta update updates artifact_name and rootfs-image.software_name fields.
  • There is no mender-binary-delta.software_name nor rootfs-image.mender-binary-delta.software_name field.

6.4 Behind the scenes

  • Mender-artifact assigns rootfs-image.software_name the same value as artifact_name.
  • Mender-artifact assigns rootfs-image.checksum.
  • mender-binary-delta-generator assigns both artifact_name and rootfs-image.software_name the same value as the second input Artifact's artifact_name.
  • mender-binary-delta-generator assigns rootfs-image.checksum to the same value as the second input Artifact.
  • Both the rootfs-image and the mender-binary-delta Update Module instructs the client to use default update behavior which is to erase all artifact_provides that match <PREFIX>.*, where <PREFIX> is the part preceding .software_name in the name of the artifact. For both of them, this is rootfs-image.*.

7 Delta Updates and application updates at the same time

7.1 Actions required when installing Mender

None

7.2 Actions required when generating Artifact

  • None if single app
  • Need to specify app domain if more than one app (see earlier use cases)

7.3 Effect for the user

  • Installing a rootfs-image update updates artifact_name and rootfs-image.software_name fields.
  • Installing a mender-binary-delta update updates artifact_name and rootfs-image.software_name fields.
  • Installing a single-file update updates the app domain, like rootfs-image.app1.software_name or rootfs-image.app2.software_name.

7.4 Behind the scenes

  • Mender-artifact assigns rootfs-image.software_name the same value as artifact_name.
  • Mender-artifact assigns rootfs-image.checksum.
  • mender-binary-delta-generator assigns both artifact_name and rootfs-image.software_name the same value as the second input Artifact's artifact_name.
  • mender-binary-delta-generator assigns rootfs-image.checksum to the same value as the second input Artifact.
  • Both the rootfs-image and the mender-binary-delta Update Module instructs the client to use default update behavior which is to erase all artifact_provides that match <PREFIX>.*, where <PREFIX> is the part preceding .software_name in the name of the artifact. For both of them, this is rootfs-image.*.
  • single-file-artifact-gen either sets rootfs-image.app1.software_name or rootfs-image.app2.software_name to the same value as artifact_name, depending on which one was selected by the user on the command line.
  • single-file-artifact-gen sets an additional field in the meta-data section of the artifact, clears_artifact_provides=rootfs-image.app1.* or clears_artifact_provides=rootfs-image.app2.*, depending on which one was selected by the user on the command line. This is used by the client at install time to erase values in that namespace only. This also preserves rootfs-image.checksum.

8 Advanced: one application group being updated by several custom Update Modules

Examples is a custom Update Module which updates a Node application, and a separate Update Module which patches it. Let's call them node-application and node-patch.

8.1 Actions required when installing Mender

  • At least node-patch needs to implement PerformsFullUpdate Update Module API call (see "Behind the scenes" for technical details).

8.2 Actions required when generating Artifact

None

8.3 Effect for the user

  • Installing a node-application update updates artifact_name and rootfs-image.node-application.software_name fields.
  • Installing a node-patch update updates artifact_name and rootfs-image.node-application.software_name fields.
  • Installing a rootfs-image update updates rootfs-image.software_name and erases rootfs-image.node-application.software_name.

8.4 Behind the scenes

  • node-application-gen assigns a name to rootfs-image.node-application.software_name (it can be artifact_name, but this depends on the Update Module)
  • node-patch-gen assigns a name to rootfs-image.node-application.software_name (it can be artifact_name, but this depends on the Update Module)
  • The node-application-gen Update Module instructs the client to use the default update behavior, which is to erase all artifact_provides that match the rootfs-image.node-application.* glob pattern, and leave others alone.
  • node-patch Update Module implements PerformsFullUpdate, which returns ClearsArtifactProvides=rootfs-image.node-application.*, which in turns clears all node-application values, but leaves others (such as rootfs-image.*) alone.

9 App on data partition

9.1 Actions required when installing Mender

None

9.2 Actions required when generating Artifact

None (but see "Behind the scenes")

9.3 Effect for the user

  • Installing a directory update to the data partition updates artifact_name and data-part.directory.software_name.
  • Installing a directory update to the rootfs partition updates artifact_name and rootfs-image.directory.software_name.
  • Installing a rootfs-image update updates artifact_name and erases rootfs-image.directory.software_name. data-part.directory.software_name is left alone.

9.4 Behind the scenes

  • Mender-artifact assigns rootfs-image.software_name the same value as artifact_name.
  • The directory-artifact-gen tool assigns data-part.directory.software_name if the target directory is /data or subfolder of it, else it assigns rootfs-image.directory.software_name.
    • This logic is not full proof if the directory structure is different, so it should be possible to specify it on the command line as well, when generating the Artifact.
  • Both Update Modules instruct the client to use the default update behavior, which is to erase all artifact_provides that match <PREFIX>.*, where <PREFIX> is the part preceding .software_name in the name of the artifact.

10 Proxy and rootfs updates, using both full and patching updates

This is nearly the same use case as the previous one, essentially having two different domains, only instead of having rootfs-image and data-part domains, there are rootfs-image and proxy domains. Additionally the user will have to implement the needed Update Module API hook.

10.1 Actions required when installing Mender

The proxy Update Module needs to assign the artifact its name in the proxy.software_name namespace, not in the default rootfs-image.proxy.software_name namespace.

10.2 Actions required when generating Artifact

None

10.3 Effect for the user

  • Installing a proxy update updates proxy.software_name and leaves rootfs-image.software_name alone.
  • Installing a rootfs-image update updates rootfs-image.software_name and leaves proxy.software_name alone.

10.4 Behind the scenes

  • Mender-artifact assigns rootfs-image.software_name the same value as artifact_name.
  • The proxy-artifact-gen tool assigns proxy.software_name the same value as artifact_name by instructing mender-artifact not to use the default rootfs-image namespace.
  • Both Update Modules instruct the client to use the default update behavior, which is to erase all artifact_provides that match <PREFIX>.*, where <PREFIX> is the part preceding .software_name in the name of the artifact. This would be rootfs-image.* and proxy.*, respectively.

Author: Kristian Amlie

Created: 2020-03-09 Mon 15:38

Emacs 25.2.2 (Org mode 8.2.10)

Validate