Commit Graph

90 Commits

Author SHA1 Message Date
Tianjie Xu 107a34f9fc Add support to decompress brotli compressed new data
Add a new writer that can decode the brotli-compressed system/vendor
new data stored in the OTA zip.

Brotli generally gives better compression rate at the cost of slightly
increased time consumption. The patch.dat is already compressed
by BZ; so there's no point to further compress it.

For the given 1.9G bullhead system image:
Size: 875M -> 787M; ~10% reduction of package size.
Time: 147s -> 153s; ~4% increase of the block_image_update execution time.
(I guess I/O takes much longer time than decompression.)

Also it takes 4 minutes to compress the system image on my local
machine, 3 more minutes than zip.

Test: recovery tests pass && apply a full OTA with brotli compressed
system/vendor.new.dat on bullhead

Change-Id: I232335ebf662a9c55579ca073ad45265700a621e
2017-07-07 16:08:18 -07:00
Tianjie Xu 14ebc1e5ae Fix a rare failure for imgdiff when random data equals gzip header
In a rare case, a random chunk will pass both the gzip header check
and the inflation process; but fail the uncompressed length check in the
footer. This leads to a imgdiff failure. So, we should treat this chunk
as 'normal' instead of 'inflated' while generating the patch.

Bug: 63334984
Test: imgdiff generates patch successfully on previous failing images.
Change-Id: Ice84f22d3653bce9756bda91e70528c0d2f264a0
2017-07-06 16:49:11 -07:00
Tianjie Xu e916cca8a9 Merge "Remove the obsolete package_extract_dir() test" 2017-06-23 04:46:14 +00:00
Tianjie Xu bb7e005a79 Remove the obsolete package_extract_dir() test
package_extract_dir() was removed in go/aog/402383, and the
corresponding UpdaterTest should be removed as well.

Bug: 62918308
Test: mma && code search
Change-Id: Ibe9c473a5d41d2fa4d26abca5684e71b104891b0
2017-06-22 17:35:06 -07:00
David Zeuthen 8ed9738b62 update_verifier: Support AVB.
When using AVB, PRODUCT_SUPPORTS_VERITY is not set so check for
BOARD_ENABLE_AVB as well. Also AVB sets up the root filesystem as
'vroot' so map that to 'system' since this is what is
expected. Managed to test at least that the code is at least compiled
in:

 $ fastboot --set-active=_a
 Setting current slot to 'a'...
 OKAY [  0.023s]
 finished. total time: 0.023s

 $ fastboot reboot
 rebooting...

 finished. total time: 0.050s

 $ adb wait-for-device

 $ adb logcat |grep update_verifier
 03-04 05:28:56.773   630   630 I /system/bin/update_verifier: Started with arg 1: nonencrypted
 03-04 05:28:56.776   630   630 I /system/bin/update_verifier: Booting slot 0: isSlotMarkedSuccessful=0
 03-04 05:28:56.776   630   630 W /system/bin/update_verifier: Failed to open /data/ota_package/care_map.txt: No such file or directory
 03-04 05:28:56.788   630   630 I /system/bin/update_verifier: Marked slot 0 as booted successfully.
 03-04 05:28:56.788   630   630 I /system/bin/update_verifier: Leaving update_verifier.

Bug: None
Test: Manually tested on device using AVB bootloader.
Change-Id: I13c0fe1cc5d0f397e36f5e62fcc05c8dfee5fd85
2017-05-24 14:14:11 -04:00
Tianjie Xu ce5fa5e538 Print SHA1 of the patch if bsdiff fails with data error
This will help us to identify the patch corruption.

Meanwhile fix a wrong size parameter passed to bspatch.
(patch->data.size() into patch->data.size() - patch_offset).

Also remove the only usage of "ApplyBSDiffPatchMem()" and inline its
Sink function for simplicity.

Bug: 37855643
Test: Prints SHA1 for corrupted patch in imgdiff_test.
Change-Id: Ibf2db8c08b0ded1409bb7c91a3547a6bf99c601d
2017-05-16 12:39:14 -07:00
Tao Bao 00d5757186 Add a binary path param to update_binary_command().
This allows writing native tests for non-A/B update_binary_command().
Prior to this CL, it was extracting the updater to a hard-coded
location (/tmp/update_binary) that's not available under the test
environment.

Test: recovery_component_test on angler and marlin respectively.
Test: Sideload OTA packages on angler and marlin respectively.
Change-Id: I78b9cc211d90c0a16a84e94e339b65759300e2a8
2017-05-08 12:16:45 -07:00
Tao Bao 4e8e56eaea Merge "fuse_sideload: Change the minimal block size to 4096." 2017-05-03 23:52:34 +00:00
Tao Bao ed13819a0d fuse_sideload: Change the minimal block size to 4096.
run_fuse_sideload() is passing the block size as the max_read
option, so it will only handle a request that involves at most two
blocks at a time. However, the minimal allowed value was set to 1024
prior to this CL, which is inconsistent with the kernel code
(fs/fuse/inode.c) that sets it to the greater of 4096 and the passed-in
max_read option. This would fail the calls with a block size / max_read
less than 4096 due to the wrongly computed block indices.

Note that we didn't observe real issue in practice, because we have been
using 64 KiB block sizes for both of adb and sdcard sideload calls. The
issue only shows up in my local CL (to come later) that uses 1024 block
size in run_fuse_sideload() tests.

Test: recovery_component_test
Test: adb sideload with the new recovery image on angler
Change-Id: Id9f0cfea13d0d193dcb7cd41a1553a23739545f2
2017-05-02 15:56:22 -07:00
Tao Bao b656a154ea Move sysMapFile and sysReleaseMap into MemMapping class.
Test: recovery_component_test
Test: recovery_unit_test
Test: Apply an OTA on angler.
Change-Id: I7170f03e4ce1fe06184ca1d7bcce0a695f33ac4d
2017-05-01 21:51:54 -07:00
Tao Bao 83b0780ddd Separate libupdate_verifier module and add testcases.
Enable -Wall and expose verify_image() for testing purpose.

Test: mmma bootable/recovery
Test: recovery_component_test
Change-Id: I1ee1db2a775bafdc1112e25a1bc7194d8d6aee4f
2017-04-27 08:57:23 -07:00
Tao Bao 8ab28082b7 applypatch: Remove the obsolete support for "applypatch -s".
The SpaceMode (applypatch -s) was used in amend script (cupcake) only,
which has been removed since commit
9ce2ebf5d300eba5f6086583b0941ef68a3e4b42 (platform/build). The later
(and current) edify script uses apply_patch_space().

Note that other modes (PatchMode, CheckMode) of applypatch executable
are mostly used by install-recovery.sh script.

Test: No active user of "applypatch -s".
Test: recovery_component_test
Change-Id: I1d689b7fedd3884077e88ed1d6c22f7a2198859d
2017-04-25 21:23:12 -07:00
Tao Bao a9dd77fa84 Merge "Add more tests for verify_package_compatibility()." 2017-04-24 18:08:55 +00:00
Tao Bao 0a3e4dca58 tests: Remove obsolete classes in applypatch_test.
ApplyPatchFullTest and ApplyPatchDoubleCacheTest were used for defining
testcases for file-based OTA. The testcases have already been removed by
commit 40e144dae8. This CL removes the
obsolete class defnitions.

Bug: 37559618
Test: recovery_component_test on angler and marlin respectively.
Change-Id: I3f4f1dfc8580cf010365e671de256f68bbc0d99a
2017-04-21 09:52:27 -07:00
Tao Bao f2784b6a43 Add more tests for verify_package_compatibility().
This now covers the actual calls to libvintf, and asserts we're getting
identical results through verify_package_compatibility() and by calling
libvintf directly.

We were missing the coverage and introduced the double free bug (fixed
by commit f978278995).

Bug: 37413730
Test: recovery_component_test passes.
Test: recovery_component_test fails w/o commit
      f978278995.
Change-Id: If5195ea1c583fd7c440a1de289da82145e80e23c
2017-04-19 17:10:34 -07:00
Tao Bao 8a7afcc6ed Add tests for read_metadata_from_package().
Test: recovery_component_test
Change-Id: I672a6a4f101c72e82b9f25f165dccd1c9520627b
2017-04-18 22:07:44 -07:00
Tao Bao bc4b1fe4c4 Add tests for update_binary_command().
Expose update_binary_command() through private/install.h for testing
purpose.

Also make minor clean-ups to install.cpp: a) adding more verbose logging
on ExtractToMemory failures; b) update_binary_command() taking
std::string instead of const char*; c) moving a few macro and global
constants into update_binary_command().

Bug: 37300957
Test: recovery_component_test on marlin
Test: Build new recovery and adb sideload on angler and sailfish.
Change-Id: Ib2d9068af3fee038f01c90940ccaeb0a7da374fc
2017-04-17 20:44:37 -07:00
Tao Bao 1d866050eb Verify the package compatibility with libvintf.
verify_package_compatibility() is added to parse the compatibility entry
(compatibility.zip) in a given OTA package. If entry is present, the
information is sent to libvintf to check the compatibility.

This CL doesn't actually call libvintf, since the API there is not
available yet.

Bug: 36597505
Test: Doesn't break the install with existing packages (i.e. w/o the
      compatibility entry).
Test: recovery_component_test
Change-Id: I3903ffa5f6ba33a5c0d761602ade6290c6752596
(cherry picked from commit 62e0bc7586)
2017-04-12 22:46:09 -07:00
Tianjie Xu 3a8d98dd90 Abort the update if there's not enough new data
Right now the update stuck in a deadlock if there's less new data than
expection. Add some checkers and abort the update if such case happens.
Also add a corresponding test.

Bug: 36787146
Test: update aborts correctly on bullhead && recovery_component_test passes
Change-Id: I914e4a2a4cf157b99ef2fc65bd21c6981e38ca47
2017-04-07 17:19:46 -07:00
Tao Bao ad87d9dc78 tests: Use get_bootloader_message_blk_device() to find /misc.
Since commit fb00d82f32 has added
get_bootloader_message_blk_device() as an API, switch the tests-local
implementation to it.

Test: recovery_component_test on angler.
Test: recovery_component_test on a local build that doesn't have /misc.
Change-Id: I4f5f542cb9ef58292c587a677da73d8822db7262
2017-04-04 20:35:10 -07:00
Tao Bao 3f2d35522a Merge "tests: Construct two bad packages at runtime for VerifierTest." 2017-03-29 00:37:05 +00:00
Tao Bao 7b0cda5b37 Merge "tests: Add a test for --wipe_ab into UncryptTest." 2017-03-28 22:12:35 +00:00
Tao Bao bdc8c1a264 tests: Add a test for --wipe_ab into UncryptTest.
Also factor out the common parts in {setup,clear}_bcb into a separate
function.

Test: recovery_component_test
Change-Id: I7b95cced925c8135e020dcb791ca2425d4f28449
2017-03-28 10:55:02 -07:00
Tao Bao c0e1c46a70 applypatch: Let Apply{BSDiff,Image}Patch accept std::function.
Test: mmma bootable/recovery system/update_engine
Test: recovery_component_test
Change-Id: I93c2caa87bf94a53509bb37f98f2c02bcadb6f5c
2017-03-28 10:14:53 -07:00
Tao Bao f7eb760fe7 applypatch: Change the ssize_t length parameters to size_t.
Mostly for applypatch family APIs like ApplyBSDiffPatch() and
ApplyImagePatch(). Changing to size_t doesn't indicate they would
necessarily work with very large size_t (e.g. > ssize_t), just
similar to write(2). But otherwise accepting negative length doesn't
make much sense.

Also change the return type of SinkFn from ssize_t to size_t. Callers
tell a successful sink by comparing the number of written bytes against
the desired value. Negative return values like -1 are not needed. This
also makes it consistent with bsdiff::bspatch interface.

Test: recovery_component_test
Test: Apply an incremental with the new updater.
Change-Id: I7ff1615203a5c9854134f75d019e266f4ea6e714
2017-03-28 10:13:38 -07:00
Tao Bao 217d9f9859 tests: Construct two bad packages at runtime for VerifierTest.
For the BadPackage tests from VerifierTest: one alters the footer, and
the other alters the metadata. Move the two tests to be based on
otasigned_v3.zip (they're based on otasigned_v1.zip previously). Also
construct the testdata files dynamically (to save the space and for
better readability).

Test: recovery_component_test
Change-Id: I7604d563f8b4fa0c55fec8730c063384158e3abc
2017-03-27 16:10:11 -07:00
Tao Bao b4c4f8c494 Merge "applypatch: Drop the support for patching non-EMMC targets." 2017-03-26 18:00:20 +00:00
Tao Bao db56eb073e updater: Fix the broken case for apply_patch_check().
It's valid to provide only 1 argument to apply_patch_check(). We
shouldn't fail the argument parsing.

Bug: 36541737
Test: recovery_component_test passes.
Test: recovery_component_test captures the failure without the fix.
Test: The previously failed update applies successfully.
Change-Id: Iee4c54ed33b877fc4885945b085341ec5c64f663
2017-03-23 06:52:45 -07:00
Tianjie Xu c444732540 Remove malloc in edify functions
And switch them to std::vector & std::unique_ptr

Bug: 32117870
Test: recovery tests passed on sailfish
Change-Id: I5a45951c4bdf895be311d6d760e52e7a1b0798c3
2017-03-22 14:20:57 -07:00
Tao Bao 3116ce4651 Add testcases for load_keys().
Test: recovery_component_test passes.
Change-Id: I6276b59981c87c50736d69d4af7647c8ed892965
2017-03-18 14:16:21 -07:00
Tao Bao 90d3f20c99 Merge "Drop the dependency on 'ui' in verify_file()." 2017-03-18 14:16:16 +00:00
Tao Bao 5e535014dd Drop the dependency on 'ui' in verify_file().
verify_file() has a dependency on the global variable of 'ui' for
posting the verification progress, which requires the users of
libverifier to provide a UI instance.

This CL adds an optional argument to verify_file() so that it can
post the progress through the provided callback function. As a result,
we can drop the MockUI class in verifier_test.cpp.

Test: recovery_component_test passes.
Test: verify_file() posts progress update when installing an OTA.
Change-Id: I8b87d0f0d99777ea755d33d6dbbe2b6d44243bf1
2017-03-17 14:07:12 -07:00
Tianjie Xu 881b08ecd2 Merge "Add a test to perform block_image_update" 2017-03-17 18:22:11 +00:00
Tianjie Xu 56ebe620a2 Add a test to perform block_image_update
Add the following tests:
stash src
bspatch stashed_src tgt
free stashed_src
(expected a successful update)

stash src
free stashed_src
fail_the_update
(expected stashed_src freed)

Bug: 36242722
Test: Test identified unfreed stashes correctly.
Change-Id: I5a136e8dc31774367972fbfe8c63cbc1ddb3a113
2017-03-16 23:06:37 -07:00
Tianjie Xu 12b90553d7 More cleanup to imgdiff & imgpatch
Also remove the utils in applypatch and replace them with the
corresponding libbase functions.

Test: recovery tests pass.
Change-Id: I77254c141bd3e7d3d6894c23b60e866009516f81
2017-03-16 12:09:49 -07:00
Tao Bao 40e144dae8 applypatch: Drop the support for patching non-EMMC targets.
Patching regular files is used in file-based OTA only, which has become
obsolete.

Bug: 35853185
Test: Apply an incremental that patches the boot.img.
Test: /system/bin/install-recovery.sh works.
Test: recovery_component_test passes.
Change-Id: Id44e42c4bc63f2162ecc8a6df1cb528b7ae6b0a9
2017-03-16 00:49:49 -07:00
Tao Bao 63d786cf22 updater: Remove some obsoleted functions for file-based OTA.
This CL removes the updater support for delete(), symlink(), rename(),
set_metadata() and set_metadata_recursive(). Such functions have been
removed from the generation script in commit
f388104eaacd05cfa075d6478369e1d0df5ddbf3 (platform/build).

Note: This CL also removes delete_recursive() which seems to have never
been supported in generation script.

Bug: 35853185
Test: recovery_component_test passes.
Change-Id: I51e1ec946fa73761118fa1eaa082423df6d588e9
2017-03-13 10:02:08 -07:00
Treehugger Robot a2e7a07031 Merge "Refractor the code for imgdiff" 2017-03-07 03:06:09 +00:00
Tianjie Xu 1ea84d6da9 Refractor the code for imgdiff
Put ImageChunk and some helper functions into a class. Also switch to
using std::vector instead of malloc.

Bug: 18606652
Test: imgdiff_test passed on host. Also generate a complete incremental OTA package.
The file content is the same and time consumption is similar.

Change-Id: Id603ada4e130ef521218400761a119001a86ca79
2017-03-06 15:41:02 -08:00
Tianjie Xu b694ba8aab Merge "Fix an error on bootloadermessager test teardown" 2017-03-03 21:43:14 +00:00
Tianjie Xu cace743c4d Fix an error on bootloadermessager test teardown
The test should not clear bcb during teardown on devices without
/misc.

Bug: 35712836
Test: The test tears down without errors after /misc removed from the fstab.
Change-Id: I42df89feb18fac5a435cd17eef97a6bad0f44545
2017-03-03 11:59:28 -08:00
Wei Wang b72a15a2ab Recovery Test: add SideloadTest to test FUSE support on target
FUSE FS is required in recovery sideload functionalites.
This CL is to add a native test to flag when FUSE is not
supported in the device kernel.

Bug: 35768196
Test: mma, run recovery_component_test on marlin and pass all
Change-Id: I43b6dbee658010df56ba4d4b0e91baa7fd1c4480
2017-03-02 15:38:22 -08:00
Tianjie Xu ca94856416 Skip BootloaderMessageTest, UncryptTest for devices without /misc
Skip these two tests if /misc partition is not found in fstab.

Bug: 35712836
Test: Both test skip correctly if there's no /misc in fstab.${hardware}.
Change-Id: I38417a8677030229a335e43eaef85ae70c4e0845
2017-02-28 17:55:27 -08:00
Tao Bao 9aa7ab5b10 tests: Add testcase for show_progress() and set_progress().
Test: recovery_component_test passes.
Change-Id: I4f00d0171cf86699e9ce747d07d7d44a01906e81
2017-01-05 17:50:10 -08:00
Tao Bao 3b828d879b Merge "imgdiff: Fix an edge case that leads to infinite loop." 2017-01-04 21:59:26 +00:00
Tao Bao 736d59c567 recovery: Fix the broken UI text.
UI text is broken (doesn't show any text during FDR) due to commit
d530449e54, which reordered the calls to
RecoveryUI::SetLocale() and RecoveryUI::Init().

Because Init() uses the locale info to load the localized texts (from
images), the locale must be set prior to that via SetLocale(). This CL
refactors Init() to take the locale parameter, and removes the odd
SetLocale() API.

Bug: 34029338
Test: 'Run graphics test' under recovery.
Change-Id: I620394a3d4e3705e9af5a1f6299285d143ae1b01
2017-01-03 22:40:03 -08:00
Tao Bao d37ce8f082 imgdiff: Fix an edge case that leads to infinite loop.
When the input image ends with the magic value sequence of 0x1f, 0x8b,
0x0b (optionally with 0x00), the image parsing code will be stuck in an
infinite loop.

Test: recovery_component_test passes.
Change-Id: Ie3629dfdc41360387b19cc3e0359c95ae4fb998e
2016-12-28 14:29:48 -08:00
Tao Bao 97555da4a6 Add tests for imgdiff.
Factor out libimgdiff static library for testing purpose.

This CL adds the imgdiff tests on host and on target both (similar to
libimgpatch). In practice, we only need imgdiff binary on host, and
libimgpatch on target. But they should build and pass tests on both
platforms.

Test: recovery_host_test passes; recovery_component_test passes.
Change-Id: I0eafb7faf727cdf70066310e845af6ee245d4f60
2016-12-19 16:53:03 -08:00
Tao Bao 2292db819b Add update_bootloader_message() to fix two-step OTAs.
This is a retry of commit 7e31f421a5.

Commit bd56f1590c switches to calling
write_bootloader_message(<options>) in get_args(), which
unintentionally resets the stage field thus breaks two-step OTAs.

This CL adds update_bootloader_message(<options>), which only sets
the command field (to "boot-recovery") and the recovery field (with
the specified options).

Bug: 33534933
Test: Apply a two-step package.
Test: recovery_component_test passes.
Change-Id: Ie0b1ed4053d2d3c97d9cb84310d616b28fcfc72e
2016-12-15 12:36:26 -08:00
Tao Bao 07d985b75b Merge "tests: Add tests for bootloader_message." 2016-12-15 20:28:22 +00:00