Commit Graph

372 Commits

Author SHA1 Message Date
andi34 0ef6492518 updater: Allow devices to suppress BLKDISCARD
* On some devices TRIM is disabled for security reasons. Don't fail
  flashing the ROM because discard isn't possible in this case.

Change-Id: I044619c3e0b01a496d967ef136501d0190240ad4
2019-03-23 14:39:53 +02:00
Simon Shields 2e9747f5b2 Revert "updater: Remove dead make_parents()."
This reverts commit 5902691764.

Change-Id: I69eadf1a091f6ecd45531789dedf72a178a055ba
2018-10-11 18:11:26 +02:00
Ethan Yonker 58f2132bc3 Merge AOSP android-9.0.0_r3
Fix conflicts and make it build in 5.1, 6.0, 7.1, 8.1, and 9.0

Change-Id: Ida0a64c29ff27d339b7f42a18d820930964ac6e4
2018-08-24 11:17:39 -05:00
Tom Marshall bf4f24f227 Revert "updater: Remove some obsoleted functions for file-based OTA."
This reverts commit 63d786cf22.

These functions will be used for third party OTA zips, so keep them.

Change-Id: I24b67ba4c86f8f86d0a41429a395fece1a383efd
2018-05-25 18:48:06 +02:00
Tom Marshall 981118ec5f Revert "kill package_extract_dir"
This reverts commit 53c38b1538.

Change-Id: Id998923948fb51ef5640c8be8884801c4f8d90e3
2018-05-25 18:47:50 +02:00
Jaegeuk Kim 2e5dc84d51 recovery: enable fsverity feature bit
Bug: 74604441
Bug 67380979
Change-Id: Iab1cc9aef356f0ddf6e2491578a2bd53009182ce
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2018-04-06 10:17:47 -07:00
Jaegeuk Kim 1224544806 install: mkfs.f2fs: specify sector size for target image size
The total sectors that we want to format is used in different meanings from
various users. This notifies its size based on 512 bytes explicitly.

Bug: 76407663
Change-Id: I20687b40a1733d3b459a45f8b64a338c37a7bc95
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2018-04-04 21:14:01 -07:00
Tianjie Xu 01daebbe68 Set the update locations to default in CacheLocation's constructor
Otherwise the applypatch executable will fail to back up the source
file to /cache when patching the recovery image.

Bug: 74198354
Test: run applypatch from boot to recovery
(cherry picked from commit b4e3a370bf)

Change-Id: I37b7fd88d66ab49ef953d4b7dca22577bd1472e1
2018-03-08 14:40:09 -08:00
Tianjie Xu 3bbb20f557 Add a singleton CacheLocation to replace the hard coded locations
This class allows us to set the following locations dynamically:
cache_temp_source, last_command_file, stash_directory_base.

In the updater's main function, we reset the values of these variables
to their default locations in /cache; while we can set them to temp
files in unit tests or host simulation.

Test: unit tests pass
Change-Id: I528652650caa41373617ab055d41b1f1a4ec0f87
2018-02-28 11:19:11 -08:00
Tianjie Xu c242084539 Fix the behavior of undefined commands in BlockImageVerify
In BlockImageVerify some commands are undefined, e.g. "erase", "new",
"zero". And we should not error out if the corresponding function
pointer of these commands is null; otherwise we will fail the
verification.

The old code is:
if (cmd->f != nullptr && cmd->f(params) == -1)
  return false;

In the last_command_file change the logic was wrongly modified to
if (cmd->f == nullptr)
  return false;
...
if (cmd->f(params) == -1)
  return false;

Test: sideload an incremental OTA twice on bullhead
Change-Id: I2561c365badb850da0e416629ccd61f0df7da5d7
2018-02-27 23:04:14 -08:00
Tianjie Xu 5419ad31e7 Reorder the functions in updater/install.cpp
There is no logical change to the file; merely the function definition
reorder and some comestic change to make the future review easier.

Test: mma
Change-Id: I7ffe952f8c78e840f10aa6bfad0c4b5a58e29896
2018-02-12 11:55:05 -08:00
Tianjie Xu 284752e2bc Log the last command to cache
When performing an update, save the index and cmdline of the current
command into the last command file if this command writes to the stash
either explicitly of implicitly. This mitigates the overhead to update
the last command file for every command. I ran a simple test on angler
and the time to update 1000 times is ~2.3 seconds.

Upon resuming an update, read the saved index first; then
  1. In verification mode, check if all commands before the saved index
     have already produced the expected target blocks. If not, delete the
     last command file so that we will later resume the update from the
     start of the transfer list.
  2. In update mode, skip all commands before the saved index. Therefore,
     we can avoid deleting stashes with duplicate id unintentionally;
     and also speed up the update.

If an update succeeds or is unresumable, delete the last command file.
Bug: 69858743
Test: Unittest passed, apply a failed update with invalid cmd on angler
and check the last_command content, apply a failed update with invalid
source hash and last_command is deleted.
Change-Id: Ib60ba1e3c6d111d9f33097759b17dbcef97a37bf
2018-02-06 16:16:49 -08:00
Tianjie Xu 5ad802839d Avoid overwrite of the error message in AbortFn
The AbortFn() used to overwrite the error message, hiding the real
failure reported in ErrorAbort(). And we will miss the failure in
the script patterns like 'blockimageupdate() || abort()'

We will ensure there's one line break at the end of ErrorAbort's
error message; and append to the existing error message when calling
abort().

Test: Message from ErrorAbort shows up in the log
Change-Id: I3aebd06629c5129330250c7fe5e8cdead2ae85bc
2018-01-29 11:42:59 -08:00
Ethan Yonker ecbd3e8ba9 Merge up to android-8.1.0_r1 and fix conflicts
Change-Id: I2dc060134d15ec9f015a606cb24ef8276f6af1fc
2017-12-15 12:48:49 -06:00
Jaegeuk Kim c1c7311b8a add sload.f2fs for recovery format
Change-Id: Iddfe54b2b36f2d531925cbe61c98dbfb4903c0d1
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2017-12-04 20:27:49 -08:00
Tianjie Xu 99b73be3a8 Detect interrupted update due to power off
An interrupted update may stash extra blocks in /cache, leading to a
failure when checking the cache size. We can save the incremented
retry_count in the BCB before installing the update; and distinguish
a fresh update from an interrupted one this way.

Bug: 68679601
Test: An interrupted update reapplies successfully.
Change-Id: Ic1403e1fd25a937c91ef34c14b92a0f6c8f1c0f4
2017-11-30 16:25:47 -08:00
Jaegeuk Kim 1a8bb0f542 recovery: format f2fs with encrypt/quota
Change-Id: Ia393b7b78b45f09964449ec0e255aa26bb3b8ddf
Signed-off-by: Jaegeuk Kim <jaegeuk@google.com>
2017-11-13 14:05:00 -08:00
Tao Bao 1e0941f4f6 applypatch: Change the patch parameter to const Value& in Apply{BSDiff,Image}Patch.
It used to be "const Value*", but nullptr won't be a valid input.

Test: recovery_host_test; recovery_component_test
Change-Id: I904b5689ac3e64504088bf0544c9fb5d45a52243
2017-11-10 12:18:34 -08:00
Tao Bao 6798315327 otautil: Remove the aborts in RangeSet::Parse().
We used to CHECK and abort on parsing errors. While it works fine for
the updater use case (because recovery starts updater in a forked
process and collects the process exit code), it's difficult for other
clients to use RangeSet as a library (e.g. update_verifier).

This CL switches the aborts to returning empty RangeSet instead. Callers
need to check the parsing results explicitly.

The CL also separates RangeSet::PushBack() into a function, and moves
SortedRangeSet::Clear() into RangeSet.

Test: recovery_unit_test
Test: Sideload an OTA package with the new updater on angler.
Test: Sideload an OTA package with injected range string errors. The
      updater aborts from the explicit checks.
Change-Id: If2b7f6f41dc93af917a21c7877a83e98dc3fd016
2017-11-07 12:50:02 -08:00
Jin Qian 502fd1c5e8 recovery: remove make_ext4fs from updater
Bug: 64395169
Change-Id: I6f6a4f82b225435c6ad5c828e110fa135e6f7579
2017-11-03 13:54:59 -07:00
Tianjie Xu 5450c84ba4 Finish the new data receiver when update fails
The thread to receive new data may still be alive after we exit
PerformBlockImageUpdate() upon failures. This caused memory corruption
when we run the unittest repeatedly. Set the receiver_available flag
to false and make sure the receiver exits when the update fails.

Bug: 65430057
Test: unittests passed with tsan
Change-Id: Icb232d13fb96c78262249ffbd29cdbe5b77f1fce
2017-10-20 11:14:56 -07:00
Tao Bao 99f0d9e52b Drop -Wno-unused-parameter.
The only one left is libedify. Will handle that in a separate CL.

Test: mmma bootable/recovery
Change-Id: I732a5f85229da90fd767bee2e46c5c95f529c396
2017-10-11 16:56:12 -07:00
Tao Bao 09e468f84c Move rangeset.h and print_sha1.h into otautil.
Also drop the "bootable/recovery" path in LOCAL_C_INCLUDES from
applypatch modules.

Test: lunch aosp_{angler,bullhead,fugu,dragon,sailfish}-userdebug;
      mmma bootable/recovery
Change-Id: Idd602a796894f971ee4f8fa3eafe36c42d9de986
2017-10-10 15:52:11 -07:00
Tao Bao 1fc5bf353a Revert "Revert "Move error_code.h into otautil.""
This reverts commit 26436d6d60 to re-land
"Move error_code.h into otautil.".

This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".

This CL needs to land with device-specific module changes (e.g. adding
the dependency on libotautil).

Test: lunch aosp_{angler,bullhead,dragon,fugu,sailfish}-userdebug;
      mmma bootable/recovery
Change-Id: If193241801af2dae73eccd31ce57cd2b81c9fd96
2017-10-09 14:07:54 -07:00
Tao Bao 0bf20d5133 Don't include "error_code.h" in edify/expr.h.
Use forward declartion to avoid pull in the module that contains
error_code.h (trying to move it into libotautil). Otherwise all the
modules that include "edify/expr.h" need to depend on the module that
exports error_code.h.

.cpp sources should include "error_code.h" explicitly to use the enums.

Test: lunch aosp_{angler,bullhead,dragon,fugu,sailfish}-userdebug;
      mmma bootable/recovery
Change-Id: Ic82db2746c7deb866e8cdfb3c57e0b1ecc71c4dc
2017-10-05 12:46:18 -07:00
Tao Bao 26436d6d60 Revert "Move error_code.h into otautil."
This reverts commit 623fe7e701.

Reason for revert: Need to address device-specific modules.

Change-Id: Ib7a4191e7f193dfff49b02d3de76dda856800251
2017-10-05 17:16:31 +00:00
Tao Bao 623fe7e701 Move error_code.h into otautil.
This way it stops requiring relative path ".." in LOCAL_C_INCLUDES
(uncrypt and edify). Soong doesn't accept non-local ".." in
"local_include_dirs".

Test: mmma bootable/recovery
Change-Id: Ia4649789cef2aaeb2785483660e9ea5a8b389c62
2017-10-04 08:55:24 -07:00
Tao Bao d33b2f86b7 otafault: Move headers under otafault/.
Test: mmma bootable/recovery
Change-Id: I3ceb72f703c7c2857d656c137d71baa1fccd8238
2017-09-29 10:29:53 -07:00
Ethan Yonker 8373cfe28c Update to AOSP 8.0 base
Change-Id: I29fe722b4eb9718765327902779046840a01433e
2017-09-08 07:14:59 -05:00
Tianjie Xu c89d1e7e2a Turn on -Wall for recovery modules
Turn on -Wall for all modules. Also remove the obsolete file_cmp() in
apply_patch test and now() in wear_ui.

The only exception is lib_edify due to the unused functions in the
intermediate cpp files generated from the lex files. It will be handled
in a seperate CL.

Bug: 64939312
Test: mma, unit tests pass
Change-Id: Ic53f76b60b6401ab20db3d98130d674c08e3702f
2017-08-28 21:56:33 -07:00
Tianjie Xu b127fddf09 Merge "Move Image/ImageChunk/PatchChunk declaration into header files"
am: b4bc57ed39

Change-Id: If254ed9e24bc0cafa19db9766ed36643ca0fed49
2017-08-19 04:27:34 +00:00
Tianjie Xu 57dd961995 Move Image/ImageChunk/PatchChunk declaration into header files
1. Move the declaration of the Image classes to the header file to make
testing easier.
2. Also move rangeset.h to bootable/recovery to allow access in imgdiff.

Test: recovery component test
Change-Id: I68a863e60a3f2e7ae46ee48f48eb15391f5f4330
2017-08-18 17:56:22 -07:00
Tianjie Xu fbd4b10310 Merge "Add implemention of SortedRangeSet"
am: 64cba55fbc

Change-Id: I2174e4f55c85fe57014b31625dbc2d06e41350be
2017-08-01 00:31:20 +00:00
Tianjie Xu b9e7fc7fa9 Add implemention of SortedRangeSet
This is useful in imgdiff to maintain the block ranges of
splitted source image.

Bug: 34220646
Test: mma && unit tests pass
Change-Id: I6427f2ea50f0e3b0aa3dd01880ec0206679b7429
2017-07-31 15:13:55 -07:00
Tao Bao ac634e3c9d Merge "updater: Remove dead make_parents()."
am: f5396eb8b2

Change-Id: Ief0557813324d49c5da64265dc49ab757ed3ae3f
2017-07-25 19:09:40 +00:00
Tao Bao 5902691764 updater: Remove dead make_parents().
Its former callers in RenameFn() and SymlinkFn() have been removed in
commit 63d786cf22.

Test: mmma -j bootable/recovery
Change-Id: I26ed126202554fc5840811ec7ae162da70593213
2017-07-25 08:42:52 -07:00
Tao Bao 248b4f9dca Merge "Remove the obsolete reference to /file_contexts."
am: 6b09b895be

Change-Id: I2b26ced9ffeb278ce01ff2ca7be4057b0ab6c61c
2017-07-23 04:03:34 +00:00
Tao Bao 338be53ed3 Remove the obsolete reference to /file_contexts.
This file no longer exists:
- /file_contexts has been split into plat_file_contexts and
  nonplat_file_contexts since commit
  b236eb6ca204cefcb926e19bd5682f9dcad4021d (system/sepolicy).
- It was named /file_contexts.bin prior to the split.

'-S file_contexts' is also no longer required by e2fsdroid, since commit
2fff6fb036cbbb6dedd7da3d208b312a9038a5ce (external/e2fsprogs). It will
load the file contexts via libselinux.

Test: Trigger the path by performing a data wipe for converting to FBE.
Change-Id: I179939da409e5c0415ae0ea0bf5ddb23f9e6331e
(cherry picked from commit 7af933b6a6)
2017-07-22 16:16:21 -07:00
Tianjie Xu d9759e8a7b Fix a case when brotli writer fails to write last few blocks of data
receive_new_data may exit too early if the zip processor has sent all
the raw data. As a result, the last few 'new' commands will fail even
though the brotli decoder has more output in its buffer.

Restruct the code so that 'NewThreadInfo' owns the decoder state solely;
and receive_brotli_new_data is responsible for the decompression.

Also reduce the test data size to 100 blocks to avoid the test timeout.

Bug: 63802629
Test: recovery_component_test. on bullhead, apply full updates with and
w/o brotli compressed entries, apply an incremental update.

Change-Id: Id429b2c2f31951897961525609fa12c3657216b7
(cherry picked from commit 6ed175d541)
2017-07-21 22:18:34 +00:00
Tianjie Xu 80acaab52f Merge "Fix a case when brotli writer fails to write last few blocks of data"
am: e45c8f0057

Change-Id: I337e8ec26f59a5245ab299080d7251331823e2da
2017-07-21 21:04:17 +00:00
Tianjie Xu e45c8f0057 Merge "Fix a case when brotli writer fails to write last few blocks of data" 2017-07-21 20:59:24 +00:00
Jin Qian 9203641742 Merge "recovery: replace make_ext4 with e2fsprogs"
am: f7c00ddaf6

Change-Id: I17167b59242c709b18d09fccb52058b0bc25bf09
2017-07-21 19:06:06 +00:00
Tianjie Xu 6ed175d541 Fix a case when brotli writer fails to write last few blocks of data
receive_new_data may exit too early if the zip processor has sent all
the raw data. As a result, the last few 'new' commands will fail even
though the brotli decoder has more output in its buffer.

Restruct the code so that 'NewThreadInfo' owns the decoder state solely;
and receive_brotli_new_data is responsible for the decompression.

Also reduce the test data size to 100 blocks to avoid the test timeout.

Bug: 63802629
Test: recovery_component_test. on bullhead, apply full updates with and
w/o brotli compressed entries, apply an incremental update.

Change-Id: I9442f2536b74e48dbf7eeb062a8539c82c6dab47
2017-07-21 11:44:00 -07:00
Jin Qian ded2dac082 recovery: replace make_ext4 with e2fsprogs
Execute mke2fs to create empty ext4 filesystem.
Execute e2fsdroid to add files to filesystem.

Test: enter recovery mode and wipe data
Bug: 35219933
Change-Id: I10a9f4c1f4754ad864b2df45b1f879180ab33876
(cherry picked from commit ac31808cd3)
2017-07-20 11:42:17 -07:00
Tao Bao f7cc0e0cfb Merge "Remove the obsolete reference to /file_contexts." 2017-07-14 00:04:12 +00:00
Tao Bao 7af933b6a6 Remove the obsolete reference to /file_contexts.
This file no longer exists:
- /file_contexts has been split into plat_file_contexts and
  nonplat_file_contexts since commit
  b236eb6ca204cefcb926e19bd5682f9dcad4021d (system/sepolicy).
- It was named /file_contexts.bin prior to the split.

'-S file_contexts' is also no longer required by e2fsdroid, since commit
2fff6fb036cbbb6dedd7da3d208b312a9038a5ce (external/e2fsprogs). It will
load the file contexts via libselinux.

Test: Trigger the path by performing a data wipe for converting to FBE.
Change-Id: I179939da409e5c0415ae0ea0bf5ddb23f9e6331e
2017-07-13 11:52:11 -07:00
Tianjie Xu 9f8fd32d95 Merge "Add support to decompress brotli compressed new data" am: 918e6ea1b2 am: 43bdf6cad6
am: 8375ebee37

Change-Id: I3eea508486f48d316644b68278f42976ffd4698d
2017-07-10 22:47:10 +00:00
Tianjie Xu 43bdf6cad6 Merge "Add support to decompress brotli compressed new data"
am: 918e6ea1b2

Change-Id: I4fd9cea71716ad1574ecb4bb7f612bc8734711c5
2017-07-10 22:33:37 +00:00
Tianjie Xu 107a34f9fc Add support to decompress brotli compressed new data
Add a new writer that can decode the brotli-compressed system/vendor
new data stored in the OTA zip.

Brotli generally gives better compression rate at the cost of slightly
increased time consumption. The patch.dat is already compressed
by BZ; so there's no point to further compress it.

For the given 1.9G bullhead system image:
Size: 875M -> 787M; ~10% reduction of package size.
Time: 147s -> 153s; ~4% increase of the block_image_update execution time.
(I guess I/O takes much longer time than decompression.)

Also it takes 4 minutes to compress the system image on my local
machine, 3 more minutes than zip.

Test: recovery tests pass && apply a full OTA with brotli compressed
system/vendor.new.dat on bullhead

Change-Id: I232335ebf662a9c55579ca073ad45265700a621e
2017-07-07 16:08:18 -07:00
Jin Qian 619b162d67 recovery: replace make_ext4 with e2fsprogs
Execute mke2fs to create empty ext4 filesystem.
Execute e2fsdroid to add files to filesystem.

Test: enter recovery mode and wipe data
Bug: 35219933
Change-Id: I10a9f4c1f4754ad864b2df45b1f879180ab33876
Merged-In: I10a9f4c1f4754ad864b2df45b1f879180ab33876
2017-06-27 00:34:43 +00:00