Due to the cache size limit for OTA generation, we used to split large
zip files linearly into pieces and do bsdiff on them. As a result, i) we
lose the advantage of imgdiff; ii) if there's an accidental order change
of some huge files inside the zip, we'll create an insanely large patch.
This patch splits the src&tgt more smartly based on the zip entry_name.
If the entry_name is empty or no matching source is found for a target
chunk, we'll skip adding its source and later do a bsdiff against the
whole split source image (this rarely happens in our use cases except
for the metadata inside a ziparchive).
After the split, the target pieces are continuous and block aligned,
while the sources pieces are mutually exclusive. (Some of the source
blocks may not be used if there's no matching entry_name in the target.)
Then we will generate patches accordingly between each split image
pairs.
Afterwards, if we apply imgpatch to each pair of split source/target
images and add up the patched result, we can get back the original
target image.
For example:
Input: [src_image, tgt_image]
Split: [src-0,tgt-0; src-1,tgt-1, src-2,tgt-2]
Diff: [ patch-0; patch-1; patch-2]
Patch: [(src-0,patch-0)=tgt-0; (src-1,patch-1)=tgt-1;
(src-2,patch-2)=tgt-2;]
Append: [tgt-0 + tgt-1 + tgt-2 = tgt_image]
Peformance:
For the small package in b/34220646, we decrease the patch size of
chrome.apk dramatically from 30M to 400K due to the order change of
two big .so files.
On two versions of angler, I also observe decent patch size decrease.
For chrome.apk, we reduced the size from 5.9M to 3.2M; and for
vevlet.apk from 8.0M to 6.5M.
Bug: 34220646
Test: recovery component test && apply imgdiff & imgpatch on two
chrome.apk
Change-Id: I145d802984fa805efbbac9d01a2e64d82ef9728b
Turn on -Wall for all modules. Also remove the obsolete file_cmp() in
apply_patch test and now() in wear_ui.
The only exception is lib_edify due to the unused functions in the
intermediate cpp files generated from the lex files. It will be handled
in a seperate CL.
Bug: 64939312
Test: mma, unit tests pass
Change-Id: Ic53f76b60b6401ab20db3d98130d674c08e3702f
The metadata file now can have multiple serial numbers in the format:
serialno=serialno1|serialno2|serialno3 ...
Verifier will pass the check if the device serial number matches any of
these numbers.
Bug: 64802465
Test: Create a metadata file with 1000 numbers and sideload in sailfish.
The checker detects both match and mismatch cases.
Change-Id: I3f12b75e15f4179df260778e37f4563d65db0fa8
1. Move the declaration of the Image classes to the header file to make
testing easier.
2. Also move rangeset.h to bootable/recovery to allow access in imgdiff.
Test: recovery component test
Change-Id: I68a863e60a3f2e7ae46ee48f48eb15391f5f4330
It fails to build recovery_component_test with the following errors:
out/soong/.intermediates/hardware/interfaces/boot/1.0/android.hardware.boot@1.0_genc++_headers/gen/android/hardware/boot/1.0/types.h:14:
error: undefined reference to 'android::hardware::hidl_string::hidl_string(android::hardware::hidl_string const&)'
out/soong/.intermediates/hardware/interfaces/boot/1.0/android.hardware.boot@1.0_genc++_headers/gen/android/hardware/boot/1.0/types.h:14:
error: undefined reference to 'android::hardware::hidl_string::operator=(android::hardware::hidl_string
const&)'
out/soong/.intermediates/hardware/interfaces/boot/1.0/android.hardware.boot@1.0_genc++_headers/gen/android/hardware/boot/1.0/types.h:14:
error: undefined reference to 'android::hardware::hidl_string::~hidl_string()'
libupdate_verifier includes <android/hardware/boot/1.0/IBootControl.h>,
which includes the 'types.h' above. In 'types.h', it defines struct
CommandResult that's using android::hardware::hidl_string.
Since libhidlbase doesn't have a static library target, remove
'LOCAL_FORCE_STATIC_EXECUTABLE := true', which isn't required for
running tests.
Test: mmma -j bootable/recovery
Bug: 64538692
Change-Id: Iaa7c08adc241128d787274fcaea9b363e7ff93f4
(cherry picked from commit 102016ce1f)
This is useful in imgdiff to maintain the block ranges of
splitted source image.
Bug: 34220646
Test: mma && unit tests pass
Change-Id: I6427f2ea50f0e3b0aa3dd01880ec0206679b7429
- Changed to std::string based implementation (mostly moved from the
former make_parents() in updater/install.cpp);
- Removed the timestamp parameter, which is only neeed by file-based OTA;
- Changed the type of mode from int to mode_t;
- Renamed dirCreateHierarchy() to mkdir_recursively().
Test: recovery_unit_test passes.
Test: No external user of dirCreateHierarchy() in code search.
Change-Id: I71f8c4b29bab625513bbc3af6d0d1ecdc3a2719a
This function has become obsolete since we've removed file-based OTA
support (it was needed by 'delete_recursive' edify function earlier).
Test: mmma -j bootable/recovery
Test: Code search shows no active user of the function.
Change-Id: If6faaa759d4c849b79acba4e6adb82baadc89f7a
This is to cover the code added by commit
5a1dee01df, where an O update_verifier
should not reject N care_map.txt.
Bug: 63544345
Test: recovery_component_test passes on marlin.
Change-Id: Ia944e16cba3cc635098b3ffd92842d725b570fec
receive_new_data may exit too early if the zip processor has sent all
the raw data. As a result, the last few 'new' commands will fail even
though the brotli decoder has more output in its buffer.
Restruct the code so that 'NewThreadInfo' owns the decoder state solely;
and receive_brotli_new_data is responsible for the decompression.
Also reduce the test data size to 100 blocks to avoid the test timeout.
Bug: 63802629
Test: recovery_component_test. on bullhead, apply full updates with and
w/o brotli compressed entries, apply an incremental update.
Change-Id: I9442f2536b74e48dbf7eeb062a8539c82c6dab47
Add a new writer that can decode the brotli-compressed system/vendor
new data stored in the OTA zip.
Brotli generally gives better compression rate at the cost of slightly
increased time consumption. The patch.dat is already compressed
by BZ; so there's no point to further compress it.
For the given 1.9G bullhead system image:
Size: 875M -> 787M; ~10% reduction of package size.
Time: 147s -> 153s; ~4% increase of the block_image_update execution time.
(I guess I/O takes much longer time than decompression.)
Also it takes 4 minutes to compress the system image on my local
machine, 3 more minutes than zip.
Test: recovery tests pass && apply a full OTA with brotli compressed
system/vendor.new.dat on bullhead
Change-Id: I232335ebf662a9c55579ca073ad45265700a621e
In a rare case, a random chunk will pass both the gzip header check
and the inflation process; but fail the uncompressed length check in the
footer. This leads to a imgdiff failure. So, we should treat this chunk
as 'normal' instead of 'inflated' while generating the patch.
Bug: 63334984
Test: imgdiff generates patch successfully on previous failing images.
Change-Id: Ice84f22d3653bce9756bda91e70528c0d2f264a0
package_extract_dir() was removed in go/aog/402383, and the
corresponding UpdaterTest should be removed as well.
Bug: 62918308
Test: mma && code search
Change-Id: Ibe9c473a5d41d2fa4d26abca5684e71b104891b0
When using AVB, PRODUCT_SUPPORTS_VERITY is not set so check for
BOARD_ENABLE_AVB as well. Also AVB sets up the root filesystem as
'vroot' so map that to 'system' since this is what is
expected. Managed to test at least that the code is at least compiled
in:
$ fastboot --set-active=_a
Setting current slot to 'a'...
OKAY [ 0.023s]
finished. total time: 0.023s
$ fastboot reboot
rebooting...
finished. total time: 0.050s
$ adb wait-for-device
$ adb logcat |grep update_verifier
03-04 05:28:56.773 630 630 I /system/bin/update_verifier: Started with arg 1: nonencrypted
03-04 05:28:56.776 630 630 I /system/bin/update_verifier: Booting slot 0: isSlotMarkedSuccessful=0
03-04 05:28:56.776 630 630 W /system/bin/update_verifier: Failed to open /data/ota_package/care_map.txt: No such file or directory
03-04 05:28:56.788 630 630 I /system/bin/update_verifier: Marked slot 0 as booted successfully.
03-04 05:28:56.788 630 630 I /system/bin/update_verifier: Leaving update_verifier.
Bug: None
Test: Manually tested on device using AVB bootloader.
Change-Id: I13c0fe1cc5d0f397e36f5e62fcc05c8dfee5fd85
This will help us to identify the patch corruption.
Meanwhile fix a wrong size parameter passed to bspatch.
(patch->data.size() into patch->data.size() - patch_offset).
Also remove the only usage of "ApplyBSDiffPatchMem()" and inline its
Sink function for simplicity.
Bug: 37855643
Test: Prints SHA1 for corrupted patch in imgdiff_test.
Change-Id: Ibf2db8c08b0ded1409bb7c91a3547a6bf99c601d
This allows writing native tests for non-A/B update_binary_command().
Prior to this CL, it was extracting the updater to a hard-coded
location (/tmp/update_binary) that's not available under the test
environment.
Test: recovery_component_test on angler and marlin respectively.
Test: Sideload OTA packages on angler and marlin respectively.
Change-Id: I78b9cc211d90c0a16a84e94e339b65759300e2a8
run_fuse_sideload() is passing the block size as the max_read
option, so it will only handle a request that involves at most two
blocks at a time. However, the minimal allowed value was set to 1024
prior to this CL, which is inconsistent with the kernel code
(fs/fuse/inode.c) that sets it to the greater of 4096 and the passed-in
max_read option. This would fail the calls with a block size / max_read
less than 4096 due to the wrongly computed block indices.
Note that we didn't observe real issue in practice, because we have been
using 64 KiB block sizes for both of adb and sdcard sideload calls. The
issue only shows up in my local CL (to come later) that uses 1024 block
size in run_fuse_sideload() tests.
Test: recovery_component_test
Test: adb sideload with the new recovery image on angler
Change-Id: Id9f0cfea13d0d193dcb7cd41a1553a23739545f2
The SpaceMode (applypatch -s) was used in amend script (cupcake) only,
which has been removed since commit
9ce2ebf5d300eba5f6086583b0941ef68a3e4b42 (platform/build). The later
(and current) edify script uses apply_patch_space().
Note that other modes (PatchMode, CheckMode) of applypatch executable
are mostly used by install-recovery.sh script.
Test: No active user of "applypatch -s".
Test: recovery_component_test
Change-Id: I1d689b7fedd3884077e88ed1d6c22f7a2198859d
Tianjie Xu
Tao Bao
David Zeuthen
Treehugger Robot