Commit Graph

315 Commits

Author SHA1 Message Date
Ethan Yonker
58f2132bc3 Merge AOSP android-9.0.0_r3
Fix conflicts and make it build in 5.1, 6.0, 7.1, 8.1, and 9.0

Change-Id: Ida0a64c29ff27d339b7f42a18d820930964ac6e4
2018-08-24 11:17:39 -05:00
Tom Marshall
bf4f24f227 Revert "updater: Remove some obsoleted functions for file-based OTA."
This reverts commit 63d786cf22.

These functions will be used for third party OTA zips, so keep them.

Change-Id: I24b67ba4c86f8f86d0a41429a395fece1a383efd
2018-05-25 18:48:06 +02:00
Tom Marshall
981118ec5f Revert "kill package_extract_dir"
This reverts commit 53c38b1538.

Change-Id: Id998923948fb51ef5640c8be8884801c4f8d90e3
2018-05-25 18:47:50 +02:00
big biff
a2c4fc29f3 Merge "otafault/tests: Replace pathmap with commands_recovery_local_path" into android-8.1 2018-05-13 16:51:56 +02:00
Tom Marshall
2c1a579cbb Revert "Remove the obsolete package_extract_dir() test"
This reverts commit bb7e005a79.

Change-Id: Ifcc2234ac9afd50384a6de93e828148bbd1fe077
2018-05-04 00:58:24 +02:00
Michael Bestas
43ee7e7354 otafault/tests: Replace pathmap with commands_recovery_local_path
* This is used everywhere else now in TWRP

Change-Id: I4dd0bab1980ea34fabfd2949474bd62ed07d1d3c
2018-04-22 21:57:35 +03:00
Tao Bao
454d8cb805 update_verifier: Support verifying product partition.
We have added the support for building /product partition in build
system (the CL in [1]), where /product is an optional partition that
contains system files. This CL adds the matching support if /product
needs to be verified during A/B OTA (i.e. listed in care_map file).

[1]: commit b7735d81054002961b681f4bdf296d4de2701135,
https://android-review.googlesource.com/c/platform/build/+/598454

Bug: 63974895
Test: Run update_verifier test on walleye.
Change-Id: Ia1c35e9583b8e66c98a4495b1f81a5ea7e65036f
(cherry picked from commit ec2e8c6c1e)
2018-03-23 13:27:53 -07:00
Tao Bao
d26f95287a tests: Add ApplyPatchModesTest.PatchModeEmmcTargetWithBsdiffPatch test.
/system/bin/applypatch on device is expected to work with bsdiff based
recovery-from-boot patch automatically. Adding a test to ensure that's
always the case.

Bug: 72731506
Test: Run recovery_component_test on marlin.
Change-Id: I56283cd3ce7cf0215cc3bb3619b206fa01d552c4
Merged-In: I56283cd3ce7cf0215cc3bb3619b206fa01d552c4
(cherry picked from commit d612b23dfd)
2018-03-20 11:38:16 +08:00
Yifan Hong
91b48f9161 recovery: add libhidl-gen-utils depedency
introduced as a depedency to libvintf.

Test: builds
Bug: 73556059
Change-Id: Ia51ba81ef462879481dcacb80d9ea9ea35e8b0bb
Merged-In: Ia51ba81ef462879481dcacb80d9ea9ea35e8b0bb
2018-03-13 10:56:51 -07:00
Tianjie Xu
3bbb20f557 Add a singleton CacheLocation to replace the hard coded locations
This class allows us to set the following locations dynamically:
cache_temp_source, last_command_file, stash_directory_base.

In the updater's main function, we reset the values of these variables
to their default locations in /cache; while we can set them to temp
files in unit tests or host simulation.

Test: unit tests pass
Change-Id: I528652650caa41373617ab055d41b1f1a4ec0f87
2018-02-28 11:19:11 -08:00
Tianjie Xu
572abbb81c Remove the assumption of target chunk size in imgdiff
In the split mode of imgdiff, we used to assume that the size of a split
target chunk is always greater than the blocksize i.e. 4096. This may
lead to the following assertion failure:
I0221 04:57:33.451323 818464 common.py:205 imgdiff F 02-21 04:57:33 821203 821203 imgdiff.cpp:999]
Check failed: tgt_size >= BLOCK_SIZE (tgt_size=476, BLOCK_SIZE=4096)

This CL removes the assumption and handles the edge cases.

Test: generate and verify the incremental update for TFs in the bug; unit test passes

Bug: 73757557
Bug: 73711365
Change-Id: Iadbb4ee658995f5856cd488f3793980881a59620
2018-02-23 10:44:05 -08:00
Tianjie Xu
284752e2bc Log the last command to cache
When performing an update, save the index and cmdline of the current
command into the last command file if this command writes to the stash
either explicitly of implicitly. This mitigates the overhead to update
the last command file for every command. I ran a simple test on angler
and the time to update 1000 times is ~2.3 seconds.

Upon resuming an update, read the saved index first; then
  1. In verification mode, check if all commands before the saved index
     have already produced the expected target blocks. If not, delete the
     last command file so that we will later resume the update from the
     start of the transfer list.
  2. In update mode, skip all commands before the saved index. Therefore,
     we can avoid deleting stashes with duplicate id unintentionally;
     and also speed up the update.

If an update succeeds or is unresumable, delete the last command file.
Bug: 69858743
Test: Unittest passed, apply a failed update with invalid cmd on angler
and check the last_command content, apply a failed update with invalid
source hash and last_command is deleted.
Change-Id: Ib60ba1e3c6d111d9f33097759b17dbcef97a37bf
2018-02-06 16:16:49 -08:00
Elliott Hughes
1d65c95fe8 StartsWith allows a std::string prefix now.
Bug: N/A
Test: builds
Change-Id: I5183ec8133f5dc9a81a438223c6d3d2ea11ef0ec
2017-12-20 12:36:31 -08:00
Ethan Yonker
ecbd3e8ba9 Merge up to android-8.1.0_r1 and fix conflicts
Change-Id: I2dc060134d15ec9f015a606cb24ef8276f6af1fc
2017-12-15 12:48:49 -06:00
Tao Bao
160514bf2b Load-balancing update_verifier worker threads.
Prior to this CL, the block verification works were assigned based on
the pattern of the ranges, which could lead to unbalanced workloads. This
CL adds RangeSet::Split() and moves update_verifier over.

a) For the following care_map.txt on walleye:
system
20,0,347,348,540,556,32770,33084,98306,98620,163842,164156,229378,229692,294914,295228,524289,524291,524292,524348,529059
vendor
8,0,120,135,32770,32831,94564,98304,98306

Measured the time costs prior to and with this CL with the following
script.

$ cat test_update_verifier.sh
  #!/bin/sh

  adb shell stop
  adb shell "cp /data/local/tmp/care_map.txt /data/ota_package/"
  for i in $(seq 1 50)
  do
    echo "Iteration: $i"
    adb shell "bootctl set-active-boot-slot 0"
    adb shell "echo 3 > /proc/sys/vm/drop_caches"
    adb shell "time /data/local/tmp/update_verifier"
    sleep 3
  done

Without this CL, the average time cost is 5.66s, while with the CL it's
reduced to 3.2s.

b) For the following care_map.txt, measured the performance on marlin:
system
18,0,271,286,457,8350,32770,33022,98306,98558,163842,164094,196609,204800,229378,229630,294914,295166,501547
vendor
10,0,42,44,85,2408,32770,32806,32807,36902,74242

It takes 12.9s and 5.6s without and with the CL respectively.

Fixes: 68553827
Test: recovery_unit_test
Test: Flash new build and trigger update_verifier. Check the balanced
      block verification.
Change-Id: I5fa4bf09a84e6b9b0975ee5f522724464181333f
2017-11-08 23:04:28 -08:00
Tao Bao
16b8b8fd1c Merge "otautil: Remove the aborts in RangeSet::Parse()." 2017-11-08 01:51:17 +00:00
Tao Bao
6798315327 otautil: Remove the aborts in RangeSet::Parse().
We used to CHECK and abort on parsing errors. While it works fine for
the updater use case (because recovery starts updater in a forked
process and collects the process exit code), it's difficult for other
clients to use RangeSet as a library (e.g. update_verifier).

This CL switches the aborts to returning empty RangeSet instead. Callers
need to check the parsing results explicitly.

The CL also separates RangeSet::PushBack() into a function, and moves
SortedRangeSet::Clear() into RangeSet.

Test: recovery_unit_test
Test: Sideload an OTA package with the new updater on angler.
Test: Sideload an OTA package with injected range string errors. The
      updater aborts from the explicit checks.
Change-Id: If2b7f6f41dc93af917a21c7877a83e98dc3fd016
2017-11-07 12:50:02 -08:00
Tianjie Xu
bf41f2aa23 Merge "Switch to bionic gtest in bootable/recovery" 2017-11-06 18:56:27 +00:00
Tao Bao
91a7aa4577 Clean up fuse_sideload and add a testcase.
This CL mainly changes:
a) moving the interface in struct provider_vtab to std::function;
b) code cleanup, such as moving the declaration closer to the uses,
   using explicit type conversion.

Test: recovery_component_test
Test: minadbd_test
Test: Sideload a package on marlin.
Change-Id: Id0e3c70f1ada54a4cd985b54c84438c23ed4687e
2017-11-03 18:30:41 -07:00
Tianjie Xu
a88cc5440e Switch to bionic gtest in bootable/recovery
We encountered segfaults in Imgdiff host tests due to the failure to
reset states of getopt. The problem can be solved by switching to use
bionic's gtest where a new process is forked for each test.

Also modify the recovery_component_test to make sure it runs in parallel.
Changes include:
  1. Merge the writes to misc partition into one single test.
  2. Change the hard coded location "/cache/saved.file" into a configurable
  variable.

Bug: 67849209
Test: recovery tests pass

Change-Id: I165d313f32b83393fb7922c5078636ac40b50bc2
2017-11-03 14:09:11 -07:00
Treehugger Robot
2cf44d08cf Merge "tests: Clean up the files in TemporaryDir." 2017-11-03 18:51:34 +00:00
Tianjie Xu
43f194c8bc Merge "Add libbrotli as a dependency for libbsdiff" 2017-11-02 17:50:40 +00:00
Tao Bao
54c1db4b3e tests: Clean up the files in TemporaryDir.
~TemporaryDir() calls rmdir(2) directly, which works with empty
directories only.

Test: Run recovery_host_test; No leftover on host.
Test; Run recovery_component_test on marlin; No leftover on device.
Change-Id: Ib510efb16eeda61b34161e2b386499e6cb79a4ca
2017-11-01 22:34:51 -07:00
Tianjie Xu
8c28081039 Add libbrotli as a dependency for libbsdiff
Bug: 34220646
Test: mma
Change-Id: If00285943fff8226f1bc7239db5570a277739904
2017-11-01 15:30:05 -07:00
Tianjie Xu
113fe05ee0 Fix the size mismatch in imgdiff
As we construct the deflate entries of the target zip file with
random data, the total size of the zip file may vary from case
to case. This leads to occasional failures in the split test for
deflate large apk files. This CL fixes the issue by adding two static
zip files in the testdata instead of generating them dynamically.

Bug: 67849209
Test: run the deflate_large_test repeatedly
Change-Id: Iaeffad9205adefa10c9f62f9f088c33c4360a650
2017-10-31 23:31:02 -07:00
Tao Bao
f29ed3e843 tests: Take the ownership of the FD when calling fdopen.
To avoid closing the same FD twice.

Test: recovery_component_test && recovery_host_test
Change-Id: I95195be8109101081410e9224efda535b2560e72
2017-10-26 12:05:57 -07:00
Alex Deymo
d8ecad7896 Merge "Use SuffixArrayIndexInterface opaque type instead of the underlying data pointer."
am: 3d24b7bca6

Change-Id: I18b7e9ca9ae299b8a028da63fc7d3e9e9eaef221
2017-10-24 17:29:30 +00:00
Treehugger Robot
3d24b7bca6 Merge "Use SuffixArrayIndexInterface opaque type instead of the underlying data pointer." 2017-10-24 17:20:14 +00:00
Alex Deymo
fa188268e4 Use SuffixArrayIndexInterface opaque type instead of the underlying data pointer.
bsdiff interface is changing such that it hides the suffix array
pointer from the public interface. This allows to use a different
suffix array data size depending on the input size, running much faster
in the normal case.

Bug: 34220646
Test: `make checkbuild`; Ran an incremental update generation on a non-A/B device.
Change-Id: I78e766da56cf28bc7774b8c8e58527bc11d919fb
2017-10-24 14:00:31 +02:00
Tianjie Xu
85fca1fc57 Merge "Finish the new data receiver when update fails"
am: 9831ef389b

Change-Id: I62be2406eede1f9e02ee4ca45ffca6fd2283ef0a
2017-10-20 21:11:20 +00:00
Tianjie Xu
5450c84ba4 Finish the new data receiver when update fails
The thread to receive new data may still be alive after we exit
PerformBlockImageUpdate() upon failures. This caused memory corruption
when we run the unittest repeatedly. Set the receiver_available flag
to false and make sure the receiver exits when the update fails.

Bug: 65430057
Test: unittests passed with tsan
Change-Id: Icb232d13fb96c78262249ffbd29cdbe5b77f1fce
2017-10-20 11:14:56 -07:00
Tao Bao
f63dbde3ef Merge "otautil: Move RangeSet implementation into rangeset.cpp."
am: bab1e8efc8

Change-Id: I5d41bdfe514e09c49bf2279ee6ca1b56ff5aab1f
2017-10-17 16:04:13 +00:00
Tao Bao
4568582002 otautil: Move RangeSet implementation into rangeset.cpp.
Since it has grown much larger, users of the header shouldn't compile
and carry their full copies.

Also add missing header includes in imgdiff.cpp and imgdiff_test.cpp.

Test: mmma bootable/recovery
Test: recovery_unit_test; recovery_component_test; recovery_host_test
Change-Id: I88ca54171765e5606ab0d61580fbc1ada578fd7d
2017-10-16 11:28:18 -07:00
Tao Bao
28f6938862 Merge "Move rangeset.h and print_sha1.h into otautil."
am: ff9b6f63a2

Change-Id: I4fea3584f64b273922ff20e9661c02a34ccca2f8
2017-10-11 17:15:26 +00:00
Tao Bao
09e468f84c Move rangeset.h and print_sha1.h into otautil.
Also drop the "bootable/recovery" path in LOCAL_C_INCLUDES from
applypatch modules.

Test: lunch aosp_{angler,bullhead,fugu,dragon,sailfish}-userdebug;
      mmma bootable/recovery
Change-Id: Idd602a796894f971ee4f8fa3eafe36c42d9de986
2017-10-10 15:52:11 -07:00
Tao Bao
28e8a2c43a Merge changes from topic "libedify-header"
am: abade5af15

Change-Id: I8a2c066f807b85139c24ec44f504d19549e1aa7d
2017-10-10 16:19:27 +00:00
Tao Bao
1fc5bf353a Revert "Revert "Move error_code.h into otautil.""
This reverts commit 26436d6d60 to re-land
"Move error_code.h into otautil.".

This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".

This CL needs to land with device-specific module changes (e.g. adding
the dependency on libotautil).

Test: lunch aosp_{angler,bullhead,dragon,fugu,sailfish}-userdebug;
      mmma bootable/recovery
Change-Id: If193241801af2dae73eccd31ce57cd2b81c9fd96
2017-10-09 14:07:54 -07:00
Tao Bao
b87edb00dc Merge "Revert "Move error_code.h into otautil.""
am: b23d29642d

Change-Id: Ib5b7fcf9c627bc1ee09b4abafc54e91c8e788edc
2017-10-05 17:27:02 +00:00
Tao Bao
26436d6d60 Revert "Move error_code.h into otautil."
This reverts commit 623fe7e701.

Reason for revert: Need to address device-specific modules.

Change-Id: Ib7a4191e7f193dfff49b02d3de76dda856800251
2017-10-05 17:16:31 +00:00
Tao Bao
73831e0636 Merge "Move error_code.h into otautil."
am: 916e155bab

Change-Id: I53265b03c12bfd022a634f8633f2d2e15a5e641b
2017-10-05 16:09:19 +00:00
Tao Bao
623fe7e701 Move error_code.h into otautil.
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".

Test: mmma bootable/recovery
Change-Id: Ia4649789cef2aaeb2785483660e9ea5a8b389c62
2017-10-04 08:55:24 -07:00
Tao Bao
8277230a98 Merge "otafault: Remove the use of LOCAL_WHOLE_STATIC_LIBRARIES."
am: 50f5a54a1e

Change-Id: I4f2d63757bb122cd027627321c1ab66c5e9284b6
2017-09-29 17:14:17 +00:00
Tao Bao
646b05a66c otafault: Remove the use of LOCAL_WHOLE_STATIC_LIBRARIES.
Commit d80a99883d has explanation of
potential issues.

Test: mmma bootable/recovery
Change-Id: I25ca9920952b7bbdd8a661d9dc90962431410bc4
2017-09-28 18:25:10 -07:00
Tianjie Xu
eb5e194012 Merge "Output split information for imgdiff when handling large apks"
am: 7f54fe8841

Change-Id: I97ad82d84b16b46e9c0c0f8198d3ab73d43031b0
2017-09-27 00:02:43 +00:00
Tianjie Xu
82582b4562 Output split information for imgdiff when handling large apks
Add a mandatory option in imgdiff to write the split info (i.e.
patch_size, tgt_size, src_ranges) to file when handling large apks.
Therefore, the caller of imgdiff can create split transfers based on
the info.

Bug: 63542719
Test: unit tests pass
Change-Id: I853d55d1f999fd576474faa81077f7307f4d856d
2017-09-21 11:42:16 -07:00
Tianjie Xu
2cb1fb45c0 Merge "Fix the dangling pointer when setting up arguments of imgdiff"
am: ca5a071307

Change-Id: I9411f0bce8d96c598db365d0257d4c9893e24104
2017-09-17 21:42:23 +00:00
Tianjie Xu
8ba7c45e0b Fix the dangling pointer when setting up arguments of imgdiff
Test: unit tests pass
Change-Id: If884e805ccd4df73671ab3436eb90860786ff6c9
2017-09-13 14:05:42 -07:00
Tianjie Xu
a11601da9b Merge "Close cmd_pipe properly after updater test finishes"
am: 64307daf5b

Change-Id: Ic384d9de3c559855cfe0bd122adb5e69fa996ad1
2017-09-12 19:47:11 +00:00
Tianjie Xu
79327ac21d Close cmd_pipe properly after updater test finishes
Otherwise the test may fail after a large number of iterations due to
file open failure.

Bug: 65430057
Test: run recovery_component_test on sailfish for 2000 iterations.
Change-Id: I0d456284d6064467038911d63eade95740cbec2c
2017-09-11 14:38:49 -07:00
Tianjie Xu
e24e90f403 Merge "Improve imgdiff for large zip files"
am: 9f48641784

Change-Id: I999525af80c5423da275025d977de10399140479
2017-09-08 20:46:47 +00:00