Running fsck breaks mender-binary-delta

Even when running with the root filesystem read-only, running fsck (at least on ext4) updates the "Last Write Time" in the metadata of the filesystem. This results in the checksum not matching when attempting to do a binary-delta update.

root@Ubuntu:~# sha256sum /dev/sda3
a4cdd8ef2a1602e4ffa938d8e53fc380ee22a4d718a7d3ac85f4fd664da3028e  /dev/sda3
root@Ubuntu:~# fsck /dev/sda3
fsck from util-linux 2.31.1
e2fsck 1.44.1 (24-Mar-2018)
/dev/sda3: clean, 62428/475200 files, 330938/475136 blocks
root@Ubuntu:~# sha256sum /dev/sda3
526aa44513480630da834608a50f28da30b97c52665dc9518cbe70c4bb5327cf  /dev/sda3

For now, on Ubuntu specifically, I have worked around this by adding the following kernel command line parameters:

rd.skipfsck fsck.mode=skip

This doesn't seem like a good long-term solution though since it will skip the automatic fsck even on the writable partitions. Additionally, I can foresee other systems behaving differently but having similar issues.

Ideally we would find a way to account for this in the binary-delta code. The only difference seems to be exactly the "Last Write Time":

root@fedcms-Ubuntu-Network:~# dumpe2fs /dev/sda3 >/data/sda3.pre.txt
dumpe2fs 1.44.1 (24-Mar-2018)
root@fedcms-Ubuntu-Network:~# fsck /dev/sda3
fsck from util-linux 2.31.1
e2fsck 1.44.1 (24-Mar-2018)
/dev/sda3: clean, 62428/475200 files, 330938/475136 blocks
root@fedcms-Ubuntu-Network:~# dumpe2fs /dev/sda3 >/data/sda3.post.txt
dumpe2fs 1.44.1 (24-Mar-2018)
root@fedcms-Ubuntu-Network:~# diff /data/sda3.p*
28c28
< Last write time:          Wed Aug 26 17:10:25 2020
---
> Last write time:          Wed Aug 26 17:07:29 2020

Suggested way to fix:

• Add a new variable READ_ONLY_ROOTFS to mender-convert, defaulting to 0, and if 1, trigger the check
• If check is triggered, check that -O ^64bit -O ^has_journal is passed to mkfs.ext4