Commit Graph

285 Commits

Author SHA1 Message Date
Tianjie Xu 113fe05ee0 Fix the size mismatch in imgdiff
As we construct the deflate entries of the target zip file with
random data, the total size of the zip file may vary from case
to case. This leads to occasional failures in the split test for
deflate large apk files. This CL fixes the issue by adding two static
zip files in the testdata instead of generating them dynamically.

Bug: 67849209
Test: run the deflate_large_test repeatedly
Change-Id: Iaeffad9205adefa10c9f62f9f088c33c4360a650
2017-10-31 23:31:02 -07:00
Tao Bao f29ed3e843 tests: Take the ownership of the FD when calling fdopen.
To avoid closing the same FD twice.

Test: recovery_component_test && recovery_host_test
Change-Id: I95195be8109101081410e9224efda535b2560e72
2017-10-26 12:05:57 -07:00
Alex Deymo d8ecad7896 Merge "Use SuffixArrayIndexInterface opaque type instead of the underlying data pointer."
am: 3d24b7bca6

Change-Id: I18b7e9ca9ae299b8a028da63fc7d3e9e9eaef221
2017-10-24 17:29:30 +00:00
Treehugger Robot 3d24b7bca6 Merge "Use SuffixArrayIndexInterface opaque type instead of the underlying data pointer." 2017-10-24 17:20:14 +00:00
Alex Deymo fa188268e4 Use SuffixArrayIndexInterface opaque type instead of the underlying data pointer.
bsdiff interface is changing such that it hides the suffix array
pointer from the public interface. This allows to use a different
suffix array data size depending on the input size, running much faster
in the normal case.

Bug: 34220646
Test: `make checkbuild`; Ran an incremental update generation on a non-A/B device.
Change-Id: I78e766da56cf28bc7774b8c8e58527bc11d919fb
2017-10-24 14:00:31 +02:00
Tianjie Xu 85fca1fc57 Merge "Finish the new data receiver when update fails"
am: 9831ef389b

Change-Id: I62be2406eede1f9e02ee4ca45ffca6fd2283ef0a
2017-10-20 21:11:20 +00:00
Tianjie Xu 5450c84ba4 Finish the new data receiver when update fails
The thread to receive new data may still be alive after we exit
PerformBlockImageUpdate() upon failures. This caused memory corruption
when we run the unittest repeatedly. Set the receiver_available flag
to false and make sure the receiver exits when the update fails.

Bug: 65430057
Test: unittests passed with tsan
Change-Id: Icb232d13fb96c78262249ffbd29cdbe5b77f1fce
2017-10-20 11:14:56 -07:00
Tao Bao f63dbde3ef Merge "otautil: Move RangeSet implementation into rangeset.cpp."
am: bab1e8efc8

Change-Id: I5d41bdfe514e09c49bf2279ee6ca1b56ff5aab1f
2017-10-17 16:04:13 +00:00
Tao Bao 4568582002 otautil: Move RangeSet implementation into rangeset.cpp.
Since it has grown much larger, users of the header shouldn't compile
and carry their full copies.

Also add missing header includes in imgdiff.cpp and imgdiff_test.cpp.

Test: mmma bootable/recovery
Test: recovery_unit_test; recovery_component_test; recovery_host_test
Change-Id: I88ca54171765e5606ab0d61580fbc1ada578fd7d
2017-10-16 11:28:18 -07:00
Tao Bao 28f6938862 Merge "Move rangeset.h and print_sha1.h into otautil."
am: ff9b6f63a2

Change-Id: I4fea3584f64b273922ff20e9661c02a34ccca2f8
2017-10-11 17:15:26 +00:00
Tao Bao 09e468f84c Move rangeset.h and print_sha1.h into otautil.
Also drop the "bootable/recovery" path in LOCAL_C_INCLUDES from
applypatch modules.

Test: lunch aosp_{angler,bullhead,fugu,dragon,sailfish}-userdebug;
      mmma bootable/recovery
Change-Id: Idd602a796894f971ee4f8fa3eafe36c42d9de986
2017-10-10 15:52:11 -07:00
Tao Bao 28e8a2c43a Merge changes from topic "libedify-header"
am: abade5af15

Change-Id: I8a2c066f807b85139c24ec44f504d19549e1aa7d
2017-10-10 16:19:27 +00:00
Tao Bao 1fc5bf353a Revert "Revert "Move error_code.h into otautil.""
This reverts commit 26436d6d60 to re-land
"Move error_code.h into otautil.".

This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".

This CL needs to land with device-specific module changes (e.g. adding
the dependency on libotautil).

Test: lunch aosp_{angler,bullhead,dragon,fugu,sailfish}-userdebug;
      mmma bootable/recovery
Change-Id: If193241801af2dae73eccd31ce57cd2b81c9fd96
2017-10-09 14:07:54 -07:00
Tao Bao b87edb00dc Merge "Revert "Move error_code.h into otautil.""
am: b23d29642d

Change-Id: Ib5b7fcf9c627bc1ee09b4abafc54e91c8e788edc
2017-10-05 17:27:02 +00:00
Tao Bao 26436d6d60 Revert "Move error_code.h into otautil."
This reverts commit 623fe7e701.

Reason for revert: Need to address device-specific modules.

Change-Id: Ib7a4191e7f193dfff49b02d3de76dda856800251
2017-10-05 17:16:31 +00:00
Tao Bao 73831e0636 Merge "Move error_code.h into otautil."
am: 916e155bab

Change-Id: I53265b03c12bfd022a634f8633f2d2e15a5e641b
2017-10-05 16:09:19 +00:00
Tao Bao 623fe7e701 Move error_code.h into otautil.
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".

Test: mmma bootable/recovery
Change-Id: Ia4649789cef2aaeb2785483660e9ea5a8b389c62
2017-10-04 08:55:24 -07:00
Tao Bao 8277230a98 Merge "otafault: Remove the use of LOCAL_WHOLE_STATIC_LIBRARIES."
am: 50f5a54a1e

Change-Id: I4f2d63757bb122cd027627321c1ab66c5e9284b6
2017-09-29 17:14:17 +00:00
Tao Bao 646b05a66c otafault: Remove the use of LOCAL_WHOLE_STATIC_LIBRARIES.
Commit d80a99883d has explanation of
potential issues.

Test: mmma bootable/recovery
Change-Id: I25ca9920952b7bbdd8a661d9dc90962431410bc4
2017-09-28 18:25:10 -07:00
Tianjie Xu eb5e194012 Merge "Output split information for imgdiff when handling large apks"
am: 7f54fe8841

Change-Id: I97ad82d84b16b46e9c0c0f8198d3ab73d43031b0
2017-09-27 00:02:43 +00:00
Tianjie Xu 82582b4562 Output split information for imgdiff when handling large apks
Add a mandatory option in imgdiff to write the split info (i.e.
patch_size, tgt_size, src_ranges) to file when handling large apks.
Therefore, the caller of imgdiff can create split transfers based on
the info.

Bug: 63542719
Test: unit tests pass
Change-Id: I853d55d1f999fd576474faa81077f7307f4d856d
2017-09-21 11:42:16 -07:00
Tianjie Xu 2cb1fb45c0 Merge "Fix the dangling pointer when setting up arguments of imgdiff"
am: ca5a071307

Change-Id: I9411f0bce8d96c598db365d0257d4c9893e24104
2017-09-17 21:42:23 +00:00
Tianjie Xu 8ba7c45e0b Fix the dangling pointer when setting up arguments of imgdiff
Test: unit tests pass
Change-Id: If884e805ccd4df73671ab3436eb90860786ff6c9
2017-09-13 14:05:42 -07:00
Tianjie Xu a11601da9b Merge "Close cmd_pipe properly after updater test finishes"
am: 64307daf5b

Change-Id: Ic384d9de3c559855cfe0bd122adb5e69fa996ad1
2017-09-12 19:47:11 +00:00
Tianjie Xu 79327ac21d Close cmd_pipe properly after updater test finishes
Otherwise the test may fail after a large number of iterations due to
file open failure.

Bug: 65430057
Test: run recovery_component_test on sailfish for 2000 iterations.
Change-Id: I0d456284d6064467038911d63eade95740cbec2c
2017-09-11 14:38:49 -07:00
Tianjie Xu e24e90f403 Merge "Improve imgdiff for large zip files"
am: 9f48641784

Change-Id: I999525af80c5423da275025d977de10399140479
2017-09-08 20:46:47 +00:00
Tianjie Xu 2903cddb58 Improve imgdiff for large zip files
Due to the cache size limit for OTA generation, we used to split large
zip files linearly into pieces and do bsdiff on them. As a result, i) we
lose the advantage of imgdiff; ii) if there's an accidental order change
of some huge files inside the zip, we'll create an insanely large patch.

This patch splits the src&tgt more smartly based on the zip entry_name.
If the entry_name is empty or no matching source is found for a target
chunk, we'll skip adding its source and later do a bsdiff against the
whole split source image (this rarely happens in our use cases except
for the metadata inside a ziparchive).

After the split, the target pieces are continuous and block aligned,
while the sources pieces are mutually exclusive. (Some of the source
blocks may not be used if there's no matching entry_name in the target.)
Then we will generate patches accordingly between each split image
pairs.

Afterwards, if we apply imgpatch to each pair of split source/target
images and add up the patched result, we can get back the original
target image.

For example:
Input: [src_image, tgt_image]
Split: [src-0,tgt-0; src-1,tgt-1, src-2,tgt-2]
Diff:  [  patch-0;     patch-1;    patch-2]

Patch: [(src-0,patch-0)=tgt-0; (src-1,patch-1)=tgt-1;
(src-2,patch-2)=tgt-2;]
Append: [tgt-0 + tgt-1 + tgt-2 = tgt_image]

Peformance:
For the small package in b/34220646, we decrease the patch size of
chrome.apk dramatically from 30M to 400K due to the order change of
two big .so files.

On two versions of angler, I also observe decent patch size decrease.
For chrome.apk, we reduced the size from 5.9M to 3.2M; and for
vevlet.apk from 8.0M to 6.5M.

Bug: 34220646
Test: recovery component test && apply imgdiff & imgpatch on two
chrome.apk
Change-Id: I145d802984fa805efbbac9d01a2e64d82ef9728b
2017-09-05 15:09:58 -07:00
Tianjie Xu 44d0b008c6 Merge "Turn on -Wall for recovery modules"
am: 11f68b6b69

Change-Id: I5f0531de7880682c7dfe75a6274f41519e0c64d6
2017-08-29 17:34:28 +00:00
Tianjie Xu c89d1e7e2a Turn on -Wall for recovery modules
Turn on -Wall for all modules. Also remove the obsolete file_cmp() in
apply_patch test and now() in wear_ui.

The only exception is lib_edify due to the unused functions in the
intermediate cpp files generated from the lex files. It will be handled
in a seperate CL.

Bug: 64939312
Test: mma, unit tests pass
Change-Id: Ic53f76b60b6401ab20db3d98130d674c08e3702f
2017-08-28 21:56:33 -07:00
Tianjie Xu 7379a5290d Merge "Allow comparison against multi serial nums for A/B package" am: 3810046a55 am: e8b02d68e5
am: 968ebdeefd

Change-Id: I21de76c870001cb5faf145d5f39f8fb2cfa58a66
2017-08-22 23:23:07 +00:00
Tianjie Xu 3810046a55 Merge "Allow comparison against multi serial nums for A/B package" 2017-08-22 22:40:47 +00:00
Tianjie Xu 69b9649e9b Allow comparison against multi serial nums for A/B package
The metadata file now can have multiple serial numbers in the format:
serialno=serialno1|serialno2|serialno3 ...
Verifier will pass the check if the device serial number matches any of
these numbers.

Bug: 64802465
Test: Create a metadata file with 1000 numbers and sideload in sailfish.
      The checker detects both match and mismatch cases.

Change-Id: I3f12b75e15f4179df260778e37f4563d65db0fa8
2017-08-22 12:05:52 -07:00
Tianjie Xu fc569d1516 Merge "Move Image/ImageChunk/PatchChunk declaration into header files" am: b4bc57ed39 am: b127fddf09
am: f5e3cadeca

Change-Id: Ic9056467184e272eec5c0aead3d4a712033b0503
2017-08-19 04:34:03 +00:00
Tianjie Xu b127fddf09 Merge "Move Image/ImageChunk/PatchChunk declaration into header files"
am: b4bc57ed39

Change-Id: If254ed9e24bc0cafa19db9766ed36643ca0fed49
2017-08-19 04:27:34 +00:00
Tianjie Xu 57dd961995 Move Image/ImageChunk/PatchChunk declaration into header files
1. Move the declaration of the Image classes to the header file to make
testing easier.
2. Also move rangeset.h to bootable/recovery to allow access in imgdiff.

Test: recovery component test
Change-Id: I68a863e60a3f2e7ae46ee48f48eb15391f5f4330
2017-08-18 17:56:22 -07:00
Tao Bao 6f1c38c9ad Merge "tests: Add the missing dependency on libhidlbase." am: 05b2e982ad am: 43df6cfeee
am: 46c13f3f05

Change-Id: Iaa1bd4f403664a87153b9f6b647e04bc7798a33d
2017-08-10 06:10:14 +00:00
Tao Bao 43df6cfeee Merge "tests: Add the missing dependency on libhidlbase."
am: 05b2e982ad

Change-Id: Ie08dca777af1d12a304aebf11f4df25680ff939a
2017-08-10 06:03:45 +00:00
Tao Bao 3e2345e1fe tests: Add the missing dependency on libhidlbase.
It fails to build recovery_component_test with the following errors:

out/soong/.intermediates/hardware/interfaces/boot/1.0/android.hardware.boot@1.0_genc++_headers/gen/android/hardware/boot/1.0/types.h:14:
error: undefined reference to 'android::hardware::hidl_string::hidl_string(android::hardware::hidl_string const&)'
out/soong/.intermediates/hardware/interfaces/boot/1.0/android.hardware.boot@1.0_genc++_headers/gen/android/hardware/boot/1.0/types.h:14:
error: undefined reference to 'android::hardware::hidl_string::operator=(android::hardware::hidl_string
const&)'
out/soong/.intermediates/hardware/interfaces/boot/1.0/android.hardware.boot@1.0_genc++_headers/gen/android/hardware/boot/1.0/types.h:14:
error: undefined reference to 'android::hardware::hidl_string::~hidl_string()'

libupdate_verifier includes <android/hardware/boot/1.0/IBootControl.h>,
which includes the 'types.h' above. In 'types.h', it defines struct
CommandResult that's using android::hardware::hidl_string.

Since libhidlbase doesn't have a static library target, remove
'LOCAL_FORCE_STATIC_EXECUTABLE := true', which isn't required for
running tests.

Test: mmma -j bootable/recovery
Bug: 64538692
Change-Id: Iaa7c08adc241128d787274fcaea9b363e7ff93f4
(cherry picked from commit 102016ce1f)
2017-08-09 22:57:13 -07:00
Tianjie Xu 8b09001e47 Merge "Add implemention of SortedRangeSet" am: 64cba55fbc am: fbd4b10310
am: a09376ed90

Change-Id: I67b2f4e91133239d4eb35524c7f0ed41f18de8dc
2017-08-01 00:38:21 +00:00
Tianjie Xu fbd4b10310 Merge "Add implemention of SortedRangeSet"
am: 64cba55fbc

Change-Id: I2174e4f55c85fe57014b31625dbc2d06e41350be
2017-08-01 00:31:20 +00:00
Tianjie Xu 64cba55fbc Merge "Add implemention of SortedRangeSet" 2017-08-01 00:22:33 +00:00
Tianjie Xu b9e7fc7fa9 Add implemention of SortedRangeSet
This is useful in imgdiff to maintain the block ranges of
splitted source image.

Bug: 34220646
Test: mma && unit tests pass
Change-Id: I6427f2ea50f0e3b0aa3dd01880ec0206679b7429
2017-07-31 15:13:55 -07:00
Tao Bao 06db7363b0 Merge "otautil: Clean up dirCreateHierarchy()." am: 610712101b am: 5507c3d63c
am: 0863373376

Change-Id: Ia2326bca2d8a1344fccd2838e2b376cb9a843a82
2017-07-31 00:21:43 +00:00
Tao Bao 5507c3d63c Merge "otautil: Clean up dirCreateHierarchy()."
am: 610712101b

Change-Id: I95350c4b2aab36dd89ea7813f2eb63d407b5f8ac
2017-07-31 00:15:43 +00:00
Tao Bao ac3d1edca0 otautil: Clean up dirCreateHierarchy().
- Changed to std::string based implementation (mostly moved from the
  former make_parents() in updater/install.cpp);
- Removed the timestamp parameter, which is only neeed by file-based OTA;
- Changed the type of mode from int to mode_t;
- Renamed dirCreateHierarchy() to mkdir_recursively().

Test: recovery_unit_test passes.
Test: No external user of dirCreateHierarchy() in code search.
Change-Id: I71f8c4b29bab625513bbc3af6d0d1ecdc3a2719a
2017-07-27 10:33:07 -07:00
Tao Bao 9c62f67863 Merge "otautil: Delete dirUnlinkHierarchy()." am: 031661d4a9 am: 993cec5cc9
am: 633aeba594

Change-Id: I9833741fe5d785298d7f99f6b2c9539723f8fcd1
2017-07-25 14:35:50 +00:00
Tao Bao 993cec5cc9 Merge "otautil: Delete dirUnlinkHierarchy()."
am: 031661d4a9

Change-Id: I33a793646f6f11757afc20f16fd3a0f700dd09fb
2017-07-25 14:28:52 +00:00
Tao Bao 7934985e0c otautil: Delete dirUnlinkHierarchy().
This function has become obsolete since we've removed file-based OTA
support (it was needed by 'delete_recursive' edify function earlier).

Test: mmma -j bootable/recovery
Test: Code search shows no active user of the function.
Change-Id: If6faaa759d4c849b79acba4e6adb82baadc89f7a
2017-07-24 20:35:48 -07:00
Tao Bao 2357b0c745 Merge "tests: Add a test to cover legacy care_map.txt handling." am: 6eb23c594c am: dbe9da5331
am: 0148a6f9ab

Change-Id: Id568b7cb5043c444f88032368be57a6f128a44c7
2017-07-24 18:32:01 +00:00
Tao Bao dbe9da5331 Merge "tests: Add a test to cover legacy care_map.txt handling."
am: 6eb23c594c

Change-Id: Iaad43cb44f4c406e4c07c9399e70e96cbaa4db7b
2017-07-24 18:27:00 +00:00