Commit Graph

5157 Commits

Author SHA1 Message Date
Treehugger Robot
5ce9fe355c Merge "root: Fix an issue when volume length from fs_mgr is negative." 2017-11-20 18:47:56 +00:00
Jin Qian
cc10008887 root: Fix an issue when volume length from fs_mgr is negative.
fs_mgr passes negative length as offset for crypt_footer. We need
to compute the actual device size for this case in addition to
when length is zero.

Bug: 68949069
Change-Id: I803955dd9591ad7752cee0dea9b683be8f4fd4ba
2017-11-18 21:54:44 -08:00
Tianjie Xu
c7069cc922 Merge "Add libbrotli as a dependency when building applypatch binary" 2017-11-17 16:09:08 +00:00
Tianjie Xu
fd9c25e636 Merge "Switch imgdiff to libbase logging" 2017-11-17 05:53:58 +00:00
Tianjie Xu
6e293c99c1 Switch imgdiff to libbase logging
Also add a verbose option. And we won't print messages of 'info'
severity unless '-v' is present.

Test: run imgdiff and check the logs.
Change-Id: I1b90874baea8e72e2a2323a0b63bc5d35e653e6b
2017-11-15 22:35:39 -08:00
Tao Bao
7bd61f0cc0 Merge "minadbd: Remove two warning options." 2017-11-15 22:59:40 +00:00
Tao Bao
f87d20404d minadbd: Remove two warning options.
'-Wimplicit-function-declaration' is not needed (it's for C89) and
already enabled by -Wall.

For '-Wno-missing-field-initializers', don't see any existing case that
requires the flag.

Test: `mmma -j bootable/recovery` on aosp_{bullhead,marlin}-userdebug.
Change-Id: I46604723087ed9a7747f6cae31a95fc0074c6758
2017-11-15 11:34:26 -08:00
Treehugger Robot
d4b6e25fdd Merge "recovery: format f2fs with encrypt/quota" 2017-11-14 01:49:26 +00:00
Jaegeuk Kim
1a8bb0f542 recovery: format f2fs with encrypt/quota
Change-Id: Ia393b7b78b45f09964449ec0e255aa26bb3b8ddf
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2017-11-13 14:05:00 -08:00
Tao Bao
de07371b03 Merge "applypatch: Change the patch parameter to const Value& in Apply{BSDiff,Image}Patch." 2017-11-11 01:09:47 +00:00
Tao Bao
1e0941f4f6 applypatch: Change the patch parameter to const Value& in Apply{BSDiff,Image}Patch.
It used to be "const Value*", but nullptr won't be a valid input.

Test: recovery_host_test; recovery_component_test
Change-Id: I904b5689ac3e64504088bf0544c9fb5d45a52243
2017-11-10 12:18:34 -08:00
Alex Deymo
29cd3215ec Merge "Include bspatch.h from bsdiff/" 2017-11-10 09:54:11 +00:00
Tao Bao
992bccfef4 Merge "uncrypt: Move to Soong." 2017-11-09 23:09:33 +00:00
Tao Bao
a2a68a522e uncrypt: Move to Soong.
Test: mmma -j bootable/recovery
Change-Id: I405f2a70f51904c02c49a287c23cbc115a4c5132
2017-11-09 12:34:00 -08:00
Treehugger Robot
f1e70b93f1 Merge "Load-balancing update_verifier worker threads." 2017-11-09 20:14:08 +00:00
Tao Bao
160514bf2b Load-balancing update_verifier worker threads.
Prior to this CL, the block verification works were assigned based on
the pattern of the ranges, which could lead to unbalanced workloads. This
CL adds RangeSet::Split() and moves update_verifier over.

a) For the following care_map.txt on walleye:
system
20,0,347,348,540,556,32770,33084,98306,98620,163842,164156,229378,229692,294914,295228,524289,524291,524292,524348,529059
vendor
8,0,120,135,32770,32831,94564,98304,98306

Measured the time costs prior to and with this CL with the following
script.

$ cat test_update_verifier.sh
  #!/bin/sh

  adb shell stop
  adb shell "cp /data/local/tmp/care_map.txt /data/ota_package/"
  for i in $(seq 1 50)
  do
    echo "Iteration: $i"
    adb shell "bootctl set-active-boot-slot 0"
    adb shell "echo 3 > /proc/sys/vm/drop_caches"
    adb shell "time /data/local/tmp/update_verifier"
    sleep 3
  done

Without this CL, the average time cost is 5.66s, while with the CL it's
reduced to 3.2s.

b) For the following care_map.txt, measured the performance on marlin:
system
18,0,271,286,457,8350,32770,33022,98306,98558,163842,164094,196609,204800,229378,229630,294914,295166,501547
vendor
10,0,42,44,85,2408,32770,32806,32807,36902,74242

It takes 12.9s and 5.6s without and with the CL respectively.

Fixes: 68553827
Test: recovery_unit_test
Test: Flash new build and trigger update_verifier. Check the balanced
      block verification.
Change-Id: I5fa4bf09a84e6b9b0975ee5f522724464181333f
2017-11-08 23:04:28 -08:00
Alex Deymo
3afe5f5691 Include bspatch.h from bsdiff/
The function ApplyBSDiffPatch() defined in bspatch.cpp is declared in
applypatch.h, but it includes "bspatch.h" from the bsdiff/ project,
which is at least confusing. There is no "bspatch.h" in this repo, so
the include actually reffers to the one in bsdiff. This patch uses the
"bsdiff/bspatch.h" form instead to avoid confusion.

Bug: None
Test: It builds.

Change-Id: I6b6ffd6725b2b34ff644aed93683f69779103661
2017-11-08 12:29:58 +01:00
Tao Bao
16b8b8fd1c Merge "otautil: Remove the aborts in RangeSet::Parse()." 2017-11-08 01:51:17 +00:00
Tao Bao
6798315327 otautil: Remove the aborts in RangeSet::Parse().
We used to CHECK and abort on parsing errors. While it works fine for
the updater use case (because recovery starts updater in a forked
process and collects the process exit code), it's difficult for other
clients to use RangeSet as a library (e.g. update_verifier).

This CL switches the aborts to returning empty RangeSet instead. Callers
need to check the parsing results explicitly.

The CL also separates RangeSet::PushBack() into a function, and moves
SortedRangeSet::Clear() into RangeSet.

Test: recovery_unit_test
Test: Sideload an OTA package with the new updater on angler.
Test: Sideload an OTA package with injected range string errors. The
      updater aborts from the explicit checks.
Change-Id: If2b7f6f41dc93af917a21c7877a83e98dc3fd016
2017-11-07 12:50:02 -08:00
Tianjie Xu
bf41f2aa23 Merge "Switch to bionic gtest in bootable/recovery" 2017-11-06 18:56:27 +00:00
Tao Bao
73dca3e983 Merge "Clean up fuse_sideload and add a testcase." 2017-11-04 14:27:48 +00:00
Tao Bao
91a7aa4577 Clean up fuse_sideload and add a testcase.
This CL mainly changes:
a) moving the interface in struct provider_vtab to std::function;
b) code cleanup, such as moving the declaration closer to the uses,
   using explicit type conversion.

Test: recovery_component_test
Test: minadbd_test
Test: Sideload a package on marlin.
Change-Id: Id0e3c70f1ada54a4cd985b54c84438c23ed4687e
2017-11-03 18:30:41 -07:00
Treehugger Robot
3f06512714 Merge "recovery: remove make_ext4fs from updater" 2017-11-03 23:18:25 +00:00
Tianjie Xu
3d843868ae Add libbrotli as a dependency when building applypatch binary
libbrotli is now used in libbspatch.

Bug: 34220646
Test: mma
Change-Id: I8d3527141539ffdae112d9042355fd705caeb250
2017-11-03 15:52:22 -07:00
Treehugger Robot
0da645fefa Merge "recovery: fix library dependency" 2017-11-03 22:16:55 +00:00
Tianjie Xu
a88cc5440e Switch to bionic gtest in bootable/recovery
We encountered segfaults in Imgdiff host tests due to the failure to
reset states of getopt. The problem can be solved by switching to use
bionic's gtest where a new process is forked for each test.

Also modify the recovery_component_test to make sure it runs in parallel.
Changes include:
  1. Merge the writes to misc partition into one single test.
  2. Change the hard coded location "/cache/saved.file" into a configurable
  variable.

Bug: 67849209
Test: recovery tests pass

Change-Id: I165d313f32b83393fb7922c5078636ac40b50bc2
2017-11-03 14:09:11 -07:00
Jin Qian
502fd1c5e8 recovery: remove make_ext4fs from updater
Bug: 64395169
Change-Id: I6f6a4f82b225435c6ad5c828e110fa135e6f7579
2017-11-03 13:54:59 -07:00
Jin Qian
000148dbc9 recovery: fix library dependency
After removing some deadcode from libext4_utils, libz is optimized
out by linker. However, it's still required by libvintf. Moving libz
down the list fixed the build.

Bug: 64395169
Change-Id: I23ecd70c83af83a219faced59d8227dc3c4e43d5
2017-11-03 13:43:48 -07:00
Treehugger Robot
2cf44d08cf Merge "tests: Clean up the files in TemporaryDir." 2017-11-03 18:51:34 +00:00
Tao Bao
439263cb83 Merge "Add a repohook to clang-format changes." 2017-11-03 03:27:15 +00:00
Tao Bao
6b1aae376a Add a repohook to clang-format changes.
It would clang-format according to the local style file in
.clang-format, unless explicitly skipped with --no-verify.

An example output is as follows:

[COMMIT dda6b1ee4247] test
[FAILED] clang_format
    The following files have formatting errors:
        screen_ui.cpp
    You can run `/mnt/aosp/aosp-master/tools/repohooks/tools/clang-format.py --fix --clang-format /mnt/aosp/aosp-master/prebuilts/clang/host/linux-x86/clang-stable/bin/clang-format --git-clang-format /mnt/aosp/aosp-master/prebuilts/clang/host/linux-x86/clang-stable/bin/git-clang-format --style file --commit dda6b1ee424710760bbab4421e95239fa6a2b40d` to fix this
[COMMIT be69a2c4ba16] Add a repohook to clang-format the change.
[RUNNING 2/2] clang_format
An automatic fix can be attempted for the "clang_format" hook. Do you want to run it? (Yes/no)?
Fix successfully applied. Amend the current commit before attempting to upload again.

More details about repohooks can be found at:

  https://android.googlesource.com/platform/tools/repohooks/

Test: `repo upload` a CL.
Change-Id: Ie8203a317eb3be7acd5592e03374873997647aa0
2017-11-02 16:23:58 -07:00
Tianjie Xu
43f194c8bc Merge "Add libbrotli as a dependency for libbsdiff" 2017-11-02 17:50:40 +00:00
Tao Bao
54c1db4b3e tests: Clean up the files in TemporaryDir.
~TemporaryDir() calls rmdir(2) directly, which works with empty
directories only.

Test: Run recovery_host_test; No leftover on host.
Test; Run recovery_component_test on marlin; No leftover on device.
Change-Id: Ib510efb16eeda61b34161e2b386499e6cb79a4ca
2017-11-01 22:34:51 -07:00
Tianjie Xu
8c28081039 Add libbrotli as a dependency for libbsdiff
Bug: 34220646
Test: mma
Change-Id: If00285943fff8226f1bc7239db5570a277739904
2017-11-01 15:30:05 -07:00
Tianjie Xu
dec77a454e Merge "Fix the size mismatch in imgdiff" 2017-11-01 18:56:46 +00:00
Tianjie Xu
113fe05ee0 Fix the size mismatch in imgdiff
As we construct the deflate entries of the target zip file with
random data, the total size of the zip file may vary from case
to case. This leads to occasional failures in the split test for
deflate large apk files. This CL fixes the issue by adding two static
zip files in the testdata instead of generating them dynamically.

Bug: 67849209
Test: run the deflate_large_test repeatedly
Change-Id: Iaeffad9205adefa10c9f62f9f088c33c4360a650
2017-10-31 23:31:02 -07:00
Treehugger Robot
011af6e1a9 Merge "update_verifier: Fix the wrong computation with group_range_count." 2017-10-30 17:29:35 +00:00
Tao Bao
6ec94c023e update_verifier: Fix the wrong computation with group_range_count.
'group_range_count' doesn't properly consider the pair-wise range
structure. It may split the ranges into wrong pairs if it evaluates to
an odd number.

For example, for an input range string of "6,0,2,10,12,20,22" with 4
threads, group_range_count becomes 1. It would then try to verify (0,2),
(2,10), (10,12) and (12,20). Note that (2,10) and (12,20) are not valid
ranges to be verified, and with (20,22) uncovered.

Bug: 68343761
Test: Trigger update_verifier verification. Check the number of verified
      blocks against the one in care_map.txt.
Change-Id: I7c5769325d9866be06c45e7dbcc0c8ea266de714
2017-10-29 14:51:25 -07:00
Treehugger Robot
166479b259 Merge "tests: Take the ownership of the FD when calling fdopen." 2017-10-26 20:47:43 +00:00
Tao Bao
f29ed3e843 tests: Take the ownership of the FD when calling fdopen.
To avoid closing the same FD twice.

Test: recovery_component_test && recovery_host_test
Change-Id: I95195be8109101081410e9224efda535b2560e72
2017-10-26 12:05:57 -07:00
Treehugger Robot
1bc0117755 Merge "applypatch: Fix a memory leak in ApplyImagePatch()." 2017-10-24 21:59:26 +00:00
Tao Bao
fdec103335 applypatch: Fix a memory leak in ApplyImagePatch().
$ valgrind --leak-check=full out/host/linux-x86/nativetest64/recovery_host_test/recovery_host_test

==36755== 112 bytes in 1 blocks are definitely lost in loss record 4 of 16
==36755==    at 0x40307C4: malloc (valgrind/coregrind/m_replacemalloc/vg_replace_malloc.c:270)
==36755==    by 0x40C1669: operator new(unsigned long) (external/libcxxabi/src/cxa_new_delete.cpp:46)
==36755==    by 0x18D6A8: ApplyImagePatch(unsigned char const*, unsigned long, Value const*, std::__1::function<unsigned long (unsigned char const*, unsigned long)>, sha_state_st*, Value const*) (bootable/recovery/applypatch/imgpatch.cpp:62)
==36755==    by 0x18D02B: ApplyImagePatch(unsigned char const*, unsigned long, unsigned char const*, unsigned long, std::__1::function<unsigned long (unsigned char const*, unsigned long)>) (bootable/recovery/applypatch/imgpatch.cpp:134)
==36755==    by 0x160D15: GenerateTarget(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*) (bootable/recovery/tests/component/imgdiff_test.cpp:85)
==36755==    by 0x11FA7D: verify_patched_image(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (bootable/recovery/tests/component/imgdiff_test.cpp:96)
==36755==    by 0x12966C: ImgdiffTest_zip_mode_smoke_trailer_zeros_Test::TestBody() (bootable/recovery/tests/component/imgdiff_test.cpp:295)
==36755==    by 0x235EF9: testing::Test::Run() (external/googletest/googletest/src/gtest.cc:2455)
==36755==    by 0x236CBF: testing::TestInfo::Run() (external/googletest/googletest/src/gtest.cc:2653)
==36755==    by 0x2372D6: testing::TestCase::Run() (external/googletest/googletest/src/gtest.cc:2771)
==36755==    by 0x23EEE6: testing::internal::UnitTestImpl::RunAllTests() (external/googletest/googletest/src/gtest.cc:4648)
==36755==    by 0x23EB45: testing::UnitTest::Run() (external/googletest/googletest/src/gtest.cc:2455)

std::unique_ptr<z_stream, decltype(&deflateEnd)> strm(new z_stream(), deflateEnd);

Only the internally allocated buffers inside 'strm' would be free'd by
deflateEnd(), but not 'strm' itself.

This CL fixes the issue by moving 'strm' to stack variable. Note that we
only need to call deflateEnd() on successful return of deflateInit2().

Test: recovery_host_test && recovery_component_test
Change-Id: I39b9bdf62376b8029f95cab82c8542bfcb874009
2017-10-24 12:48:36 -07:00
Alex Deymo
d8ecad7896 Merge "Use SuffixArrayIndexInterface opaque type instead of the underlying data pointer."
am: 3d24b7bca6

Change-Id: I18b7e9ca9ae299b8a028da63fc7d3e9e9eaef221
2017-10-24 17:29:30 +00:00
Treehugger Robot
3d24b7bca6 Merge "Use SuffixArrayIndexInterface opaque type instead of the underlying data pointer." 2017-10-24 17:20:14 +00:00
Alex Deymo
fa188268e4 Use SuffixArrayIndexInterface opaque type instead of the underlying data pointer.
bsdiff interface is changing such that it hides the suffix array
pointer from the public interface. This allows to use a different
suffix array data size depending on the input size, running much faster
in the normal case.

Bug: 34220646
Test: `make checkbuild`; Ran an incremental update generation on a non-A/B device.
Change-Id: I78e766da56cf28bc7774b8c8e58527bc11d919fb
2017-10-24 14:00:31 +02:00
Dan Willemsen
3b4e2ebf2c Merge "libdrm_platform is now libdrm"
am: d830858952

Change-Id: I9614430880626ca25b9eced39ed83eb69073b3b5
2017-10-23 21:36:58 +00:00
Treehugger Robot
d830858952 Merge "libdrm_platform is now libdrm" 2017-10-23 21:25:17 +00:00
Dan Willemsen
0bda7a02c2 Merge "Build libminui with BOARD_VNDK_VERSION"
am: ec9ac55949

Change-Id: I18d3e50975bbff077cb435b798a71cf2384d5ba6
2017-10-21 00:16:05 +00:00
Dan Willemsen
ec9ac55949 Merge "Build libminui with BOARD_VNDK_VERSION" 2017-10-21 00:08:23 +00:00
Tianjie Xu
85fca1fc57 Merge "Finish the new data receiver when update fails"
am: 9831ef389b

Change-Id: I62be2406eede1f9e02ee4ca45ffca6fd2283ef0a
2017-10-20 21:11:20 +00:00