Commit Graph

42 Commits

Author SHA1 Message Date
Tao Bao
91a649ab62 updater: Add ABORT command.
This will be used for testing purpose only, replacing the previously
used "fail", to intentionally abort an update.

As we're separating the logic between commands parsing and execution,
"abort" needs to be considered as a valid command during the parsing.

Test: recovery_unit_test and recovery_component_test on marlin.
Change-Id: I47c41c423e62c41cc8515fd92f3c5959be08da02
2018-07-07 04:12:19 +00:00
Tao Bao
ffede3e8d9 updater: Remove the redundant check on line count.
Test: recovery_component_test on marlin.
Change-Id: I2ac2bd47469d1aec8a97a8c4ed0fe80ffd65c95b
2018-06-07 10:43:24 -07:00
Tao Bao
c0299edd46 tests: Add ResumableUpdaterTest.
This is a stress test that instantiates multiple testcases that
interrupt a BBOTA update at every transfer command. Each testcase
asserts the last_command_file after the interruption, verifies the
update resumability, then resumes the update and asserts the updated
image.

The transfer list in the testcase covers most of the transfer commands
(stash/free/move/bsdiff/zero/new), as well as some special pattern like
having duplicate stash ids.

This CL also addresses one issue in the updater code, by resetting the
stash_map before each run. The stash map should be valid only per
block_image_verify/block_image_update run. Having leftover may cause
issue in subsequent runs, in particular when calling block_image_verify
after a previous run of block_image_{update,verify}.

Test: Run recovery_component_test on marlin.
Change-Id: I6f9a0368d194a754ce41a9c9819c6d5be2657248
2018-06-04 16:19:45 +00:00
Tao Bao
7064aa2862 tests: Setup last_command_file for UpdaterTest.
Otherwise tests may interfere with each other by using the same / default
location.

Test: Run recovery_component_test on marlin.
Change-Id: I6b0455489f2fdce819009964dd92bfd9bfeb06ef
2018-05-24 21:49:58 -07:00
Tao Bao
bc4a6d59f6 tests: Refactor the common lines in UpdaterTest.
Move the common codes into RunBlockImageUpdate(). Also clean up the
partition updated marker after running each test.

Test: Run recovery_component_test on marlin.
Change-Id: Id4302e4da4c664231b737a1e83d2e164ef58ed97
2018-05-24 00:15:56 -07:00
Tao Bao
17054c051a otautil: Rename dir/sys/thermal utils.
Test: mmma -j bootable/recovery
Change-Id: I32ab98549e91f993364306e4a88dc654221b3869
2018-05-03 23:17:29 -07:00
Tao Bao
641fa97def Rename CacheLocation to Paths.
We have a general need for overriding more paths (e.g. "/tmp"), mostly
for testing purpose. Rename CacheLocation to Paths, and use that to
manage TEMPORARY_{INSTALL,LOG}_FILE.

Test: mmma -j bootable/recovery
Test: recovery_component_test
Change-Id: Ia8ce8e5695df37ca434f13ac4d3206de1e8e9396
2018-04-25 21:46:00 -07:00
Tianjie Xu
3bbb20f557 Add a singleton CacheLocation to replace the hard coded locations
This class allows us to set the following locations dynamically:
cache_temp_source, last_command_file, stash_directory_base.

In the updater's main function, we reset the values of these variables
to their default locations in /cache; while we can set them to temp
files in unit tests or host simulation.

Test: unit tests pass
Change-Id: I528652650caa41373617ab055d41b1f1a4ec0f87
2018-02-28 11:19:11 -08:00
Tianjie Xu
284752e2bc Log the last command to cache
When performing an update, save the index and cmdline of the current
command into the last command file if this command writes to the stash
either explicitly of implicitly. This mitigates the overhead to update
the last command file for every command. I ran a simple test on angler
and the time to update 1000 times is ~2.3 seconds.

Upon resuming an update, read the saved index first; then
  1. In verification mode, check if all commands before the saved index
     have already produced the expected target blocks. If not, delete the
     last command file so that we will later resume the update from the
     start of the transfer list.
  2. In update mode, skip all commands before the saved index. Therefore,
     we can avoid deleting stashes with duplicate id unintentionally;
     and also speed up the update.

If an update succeeds or is unresumable, delete the last command file.
Bug: 69858743
Test: Unittest passed, apply a failed update with invalid cmd on angler
and check the last_command content, apply a failed update with invalid
source hash and last_command is deleted.
Change-Id: Ib60ba1e3c6d111d9f33097759b17dbcef97a37bf
2018-02-06 16:16:49 -08:00
Treehugger Robot
3d24b7bca6 Merge "Use SuffixArrayIndexInterface opaque type instead of the underlying data pointer." 2017-10-24 17:20:14 +00:00
Alex Deymo
fa188268e4 Use SuffixArrayIndexInterface opaque type instead of the underlying data pointer.
bsdiff interface is changing such that it hides the suffix array
pointer from the public interface. This allows to use a different
suffix array data size depending on the input size, running much faster
in the normal case.

Bug: 34220646
Test: `make checkbuild`; Ran an incremental update generation on a non-A/B device.
Change-Id: I78e766da56cf28bc7774b8c8e58527bc11d919fb
2017-10-24 14:00:31 +02:00
Tianjie Xu
5450c84ba4 Finish the new data receiver when update fails
The thread to receive new data may still be alive after we exit
PerformBlockImageUpdate() upon failures. This caused memory corruption
when we run the unittest repeatedly. Set the receiver_available flag
to false and make sure the receiver exits when the update fails.

Bug: 65430057
Test: unittests passed with tsan
Change-Id: Icb232d13fb96c78262249ffbd29cdbe5b77f1fce
2017-10-20 11:14:56 -07:00
Tao Bao
09e468f84c Move rangeset.h and print_sha1.h into otautil.
Also drop the "bootable/recovery" path in LOCAL_C_INCLUDES from
applypatch modules.

Test: lunch aosp_{angler,bullhead,fugu,dragon,sailfish}-userdebug;
      mmma bootable/recovery
Change-Id: Idd602a796894f971ee4f8fa3eafe36c42d9de986
2017-10-10 15:52:11 -07:00
Tao Bao
1fc5bf353a Revert "Revert "Move error_code.h into otautil.""
This reverts commit 26436d6d60 to re-land
"Move error_code.h into otautil.".

This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".

This CL needs to land with device-specific module changes (e.g. adding
the dependency on libotautil).

Test: lunch aosp_{angler,bullhead,dragon,fugu,sailfish}-userdebug;
      mmma bootable/recovery
Change-Id: If193241801af2dae73eccd31ce57cd2b81c9fd96
2017-10-09 14:07:54 -07:00
Tao Bao
26436d6d60 Revert "Move error_code.h into otautil."
This reverts commit 623fe7e701.

Reason for revert: Need to address device-specific modules.

Change-Id: Ib7a4191e7f193dfff49b02d3de76dda856800251
2017-10-05 17:16:31 +00:00
Tao Bao
623fe7e701 Move error_code.h into otautil.
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".

Test: mmma bootable/recovery
Change-Id: Ia4649789cef2aaeb2785483660e9ea5a8b389c62
2017-10-04 08:55:24 -07:00
Tianjie Xu
79327ac21d Close cmd_pipe properly after updater test finishes
Otherwise the test may fail after a large number of iterations due to
file open failure.

Bug: 65430057
Test: run recovery_component_test on sailfish for 2000 iterations.
Change-Id: I0d456284d6064467038911d63eade95740cbec2c
2017-09-11 14:38:49 -07:00
Tianjie Xu
e45c8f0057 Merge "Fix a case when brotli writer fails to write last few blocks of data" 2017-07-21 20:59:24 +00:00
Tianjie Xu
6ed175d541 Fix a case when brotli writer fails to write last few blocks of data
receive_new_data may exit too early if the zip processor has sent all
the raw data. As a result, the last few 'new' commands will fail even
though the brotli decoder has more output in its buffer.

Restruct the code so that 'NewThreadInfo' owns the decoder state solely;
and receive_brotli_new_data is responsible for the decompression.

Also reduce the test data size to 100 blocks to avoid the test timeout.

Bug: 63802629
Test: recovery_component_test. on bullhead, apply full updates with and
w/o brotli compressed entries, apply an incremental update.

Change-Id: I9442f2536b74e48dbf7eeb062a8539c82c6dab47
2017-07-21 11:44:00 -07:00
Tianjie Xu
de6735e80c Fix the android-cloexec-* warnings in bootable/recovery
Add the O_CLOEXEC or 'e' accordingly.

Bug: 63510015
Test: recovery tests pass
Change-Id: I7094bcc6af22c9687eb535116b2ca6a59178b303
2017-07-19 12:17:41 -07:00
Tianjie Xu
107a34f9fc Add support to decompress brotli compressed new data
Add a new writer that can decode the brotli-compressed system/vendor
new data stored in the OTA zip.

Brotli generally gives better compression rate at the cost of slightly
increased time consumption. The patch.dat is already compressed
by BZ; so there's no point to further compress it.

For the given 1.9G bullhead system image:
Size: 875M -> 787M; ~10% reduction of package size.
Time: 147s -> 153s; ~4% increase of the block_image_update execution time.
(I guess I/O takes much longer time than decompression.)

Also it takes 4 minutes to compress the system image on my local
machine, 3 more minutes than zip.

Test: recovery tests pass && apply a full OTA with brotli compressed
system/vendor.new.dat on bullhead

Change-Id: I232335ebf662a9c55579ca073ad45265700a621e
2017-07-07 16:08:18 -07:00
Tianjie Xu
bb7e005a79 Remove the obsolete package_extract_dir() test
package_extract_dir() was removed in go/aog/402383, and the
corresponding UpdaterTest should be removed as well.

Bug: 62918308
Test: mma && code search
Change-Id: Ibe9c473a5d41d2fa4d26abca5684e71b104891b0
2017-06-22 17:35:06 -07:00
Tao Bao
b656a154ea Move sysMapFile and sysReleaseMap into MemMapping class.
Test: recovery_component_test
Test: recovery_unit_test
Test: Apply an OTA on angler.
Change-Id: I7170f03e4ce1fe06184ca1d7bcce0a695f33ac4d
2017-05-01 21:51:54 -07:00
Tianjie Xu
3a8d98dd90 Abort the update if there's not enough new data
Right now the update stuck in a deadlock if there's less new data than
expection. Add some checkers and abort the update if such case happens.
Also add a corresponding test.

Bug: 36787146
Test: update aborts correctly on bullhead && recovery_component_test passes
Change-Id: I914e4a2a4cf157b99ef2fc65bd21c6981e38ca47
2017-04-07 17:19:46 -07:00
Tao Bao
db56eb073e updater: Fix the broken case for apply_patch_check().
It's valid to provide only 1 argument to apply_patch_check(). We
shouldn't fail the argument parsing.

Bug: 36541737
Test: recovery_component_test passes.
Test: recovery_component_test captures the failure without the fix.
Test: The previously failed update applies successfully.
Change-Id: Iee4c54ed33b877fc4885945b085341ec5c64f663
2017-03-23 06:52:45 -07:00
Tianjie Xu
c444732540 Remove malloc in edify functions
And switch them to std::vector & std::unique_ptr

Bug: 32117870
Test: recovery tests passed on sailfish
Change-Id: I5a45951c4bdf895be311d6d760e52e7a1b0798c3
2017-03-22 14:20:57 -07:00
Tianjie Xu
56ebe620a2 Add a test to perform block_image_update
Add the following tests:
stash src
bspatch stashed_src tgt
free stashed_src
(expected a successful update)

stash src
free stashed_src
fail_the_update
(expected stashed_src freed)

Bug: 36242722
Test: Test identified unfreed stashes correctly.
Change-Id: I5a136e8dc31774367972fbfe8c63cbc1ddb3a113
2017-03-16 23:06:37 -07:00
Tao Bao
63d786cf22 updater: Remove some obsoleted functions for file-based OTA.
This CL removes the updater support for delete(), symlink(), rename(),
set_metadata() and set_metadata_recursive(). Such functions have been
removed from the generation script in commit
f388104eaacd05cfa075d6478369e1d0df5ddbf3 (platform/build).

Note: This CL also removes delete_recursive() which seems to have never
been supported in generation script.

Bug: 35853185
Test: recovery_component_test passes.
Change-Id: I51e1ec946fa73761118fa1eaa082423df6d588e9
2017-03-13 10:02:08 -07:00
Tao Bao
9aa7ab5b10 tests: Add testcase for show_progress() and set_progress().
Test: recovery_component_test passes.
Change-Id: I4f00d0171cf86699e9ce747d07d7d44a01906e81
2017-01-05 17:50:10 -08:00
Tao Bao
bedf5fc11c updater: Refactor set_stage() and get_stage() functions.
Add read_bootloader_message_from() and write_bootloader_message_to() to
allow specifying the BCB device (/misc).

Also add testcases for set_stage() and get_stage().

Test: recovery_component_test passes.
Test: Build a recovery image and apply a two-step OTA package.
Change-Id: If5ab06a1aaaea168d2a9e5dd63c07c0a3190e4ae
2016-11-18 12:04:48 -08:00
Tao Bao
1036d36542 updater: Add testcase for package_extract_dir().
Test: recovery_component_test passes.
Change-Id: I3af4707bc42c7331ca961be8b967a53de82ea25b
2016-11-17 23:02:13 -08:00
Tao Bao
d0f3088aa9 updater: Add "write_value()" function.
write_value(value, filename) writes 'value' to 'filename'. It can be
used to tune device settings when applying an OTA package. For example,
write_value("960000", "/sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq").

Bug: 32463933
Test: recovery_component_test passes.
Test: Apply an OTA package that contains a call to write_value(), and
      check the result.

Change-Id: Ib009ecb8a45a94353f10c59e2383fe1f49796e35
2016-11-16 15:51:53 -08:00
Tao Bao
ef0eb3b01b updater: Fix the wrong return value for package_extract_file().
'bool success = ExtractEntryToFile()' gives opposite result. Fix the
issue and add testcases.

Change the one-argument version of package_extract_file() to explicitly
abort for non-existent zip entry. Note that this is NOT changing the
behavior. Prior to this CL, it aborts from Evaluate() function, by
giving a general cause code. Now it returns kPackageExtractFileFailure.

BUg: 32903624
Test: recovery_component_test works.

Change-Id: I7a273e9c0d9aaaf8c472b2c778f7b8d90362c24f
2016-11-15 13:17:52 -08:00
Tao Bao
8992902aab updater: Add more testcase for symlink().
Clean up SymlinkFn() a bit. Also clean up the temp files created when
running the tests; otherwise non-empty TemporaryDir won't be removed.

Test: recovery_component_test passes.
Change-Id: Id3844abebd168c40125c4dcec54e6ef680a83c3a
2016-11-08 21:29:16 -08:00
Tianjie Xu
d75003d24c Make make_parent() to take const argument
Switch to use const std::string; and add corresponding tests.

Bug: 32649858
Test: Component tests pass
Change-Id: I640f3ec81f1481fa91aa310f8d4d96dac9649cb9
2016-11-08 10:11:03 -08:00
Tao Bao
a659d79b92 updater: Add a testcase for RenameFn().
Test: recovery_component_test passes.

Change-Id: Iba5a0fdf6c79e2bed6b30b8fc19a306c1ab29d8a
2016-11-03 23:35:42 -07:00
Tao Bao
0831d0b592 updater: Fix a bug in DeleteFn().
Also add a testcase for delete() function.

Test: recovery_component_test passes.
Change-Id: I064d1ad4693c3ed339d0a69eabadd08a61a2ea86
2016-11-03 23:28:36 -07:00
Tao Bao
51d516e913 updater: Fix an off-by-1 bug in file_getprop().
Also add a testcase for file_getprop().

Test: recovery_component_test passes.

Change-Id: I8eb2f9a5702b43997ac9f4b29665eea087b1c146
2016-11-03 14:58:54 -07:00
Tianjie Xu
aced5d9e4e Change StringValue to use std::string
Changing the field of 'Value' in edify to std::string from char*.
Meanwhile cleaning up the users of 'Value' and switching them to
cpp style.

Test: compontent tests passed.
Bug: 31713288

Change-Id: Iec5a7d601b1e4ca40935bf1c70d325dafecec235
2016-10-15 01:18:23 +00:00
Tao Bao
39119ad8ec edify: Some clean-ups to libedify.
- Remove dead declarations in expr.h: SetError(), GetError(),
  ClearError().
- Remove the declaration of Build() out of expr.h.
- Use std::unordered_map to implement RegisterFunction() and
  FindFunction(); kill FinishRegistration().
- Add a testcase for calling unknown functions.

Test: mmma bootable/recovery; recovery_component_test passes.
Change-Id: I9af6825ae677f92b22d716a4a5682f58522af03b
2016-10-12 23:29:59 -07:00
Tao Bao
361342cbd2 updater: Kill the duplicate PrintSha1() in install.cpp.
Also add a testcase for sha1_check().

Test: mmma bootable/recovery; recovery_component_test passes.

Change-Id: I4d06d551a771aec84e460148544f68b247a7e721
2016-10-10 20:36:17 -07:00
Tao Bao
0c7839ac14 Refactor libupdater into a seperate module.
So that we can write native tests for updater functions. This CL adds a
testcase for getprop() function.

Test: mmma bootable/recovery; Run recovery_component_test on device.

Change-Id: Iff4c1ff63c5c71aded2f9686fed6b71cc298c228
2016-10-10 16:49:50 -07:00