* For EXT2/3:
* Align mke2fs options with Android 9.0 fs_mgr_format.cpp.
* For EXT4:
* Note that make_ext4fs is still in use on Android 8.1 to workaround
a FDE encryption bug even though mke2fs has become the default
choice. The bug is fixed in Android 9.0+.
* For F2FS:
* Drop the broken "-r" option, it's never ported to O, explicitly
pass number of sectors instead.
* Keep all options aligned with Pie fs_mgr.
* Check existence before start wiping, as Wipe_EXT4 does.
* For all:
* Calculate block size at runtime, and reserve the space specified
via negative "length" option from fstab. Note that positive length
will be ignored.
* If the partition has crypto footer, and "length" is not specified
in fstab, automatically reverse CRYPT_FOOTER_OFFSET (16384).
* Remove the default crypto key location: footer, this is a bad
assumption since most partitions don't use encryption! We need to
know the real situation to decide whether to reserve crypto footer
on format.
* If the current action is "wipe" or "change filesystem", wipe crypto
footer or block device after mkfs if the partition is
unencrypted or undecrypted (e.g., wrong password).
* If the current action is "format data", unconditionally wipe crypto
footer or block device.
Change-Id: I7304a8ee703131ea4a08ab7c60334af28cac28b3
cryptfs.cpp based on CAF tag LA.UM.7.3.r1-05900-sdm845.0
Used CAF because AOSP no longer contains code for qcom's hardware
crypto.
Change-Id: I921cbe9bed70989f91449e23b5ac3ec1037b7b97
This update splits old MTP code and new MTP code from Google
into two trees, legacy and ffs. Depending on the SDK level,
the build system will select the correct version. The reason
for separating the versions out are due to older android trees
not supporting the updated MTP code from Google.
Most MTP code is from Google, with additions needed from
implementing the Java functions in C++ for TWRP and FFS.
We assume if you are in android-9.0 or above, your kernel
has support for FFS over MTP. Verify that your init.rc
is mounting the MTP FFS driver to the proper location.
Change-Id: I4b107b239bd9bc5699527f9c8c77d9079f264a7e
This patchset updates all the places we touch cache on AB devices
and non AB devices so that we can write to the appropriate
location. AB devices removed the cache partition and moved it to
/data/cache.
Change-Id: I485ec205a604f1f07ceb7700ae9c4e720fd55c3e
instead of hard-coded /system path
I updated most of the references I found,
but there might be more
For devices that have to mount system at /system_root,
this allows system to be bind mounted to /system
and detected properly by TWRP
Change-Id: I9f142fd8cec392f5b88e95476258dab9c21a9aac
Enable a property when mounting partitions required for decryption, and disable when they are unmounted.
This helps init scripts to react to such an event appropriately, for example by starting services critical for decryption that reside on said partitions.
Change-Id: Ib6fa1e5c903d9c800fb145d582ced6cc7e0ab5b9
* HTC U11 Oreo is using keymaster3 FDE encryption which requires
the new services:
1- /system/bin/hwservicemanager
2- /vendor/bin/hw/android.hardware.keymaster@3.0-service
3- /vendor/bin/qseecomd (instead of /system/bin/qseecomd)
So in addition to /vendor/lib and /vendor/lib64 also
symlink /system/vendor/bin to /vendor/bin.
* vold_decrypt services now have separate prefixes:
1- 'sys_' referring to /system/bin
2- 'ven_' referring to /vendor/bin
* The additional (hwservicemanager, keymaster-3-0) and modified
(qseecomd) .rc files have been updated in the vold_decrypt
directory.
Comments were added directly in the .rc files, please check
them.
* /etc/recovery.fstab needs to be temporarily moved since
vold will use it if it finds the '/sbin/recovery' file
(refer to fs_mgr for the fstab load code https://goo.gl/8KaZyf).
Since fs_mgr cannot parse TWRP style fstab, we 'hide' it
and attempt to create a symlink to /fstab.{ro.hardware}.
Also remove shell dependencies, code cleanup, new error codes:
* Critical sections of vold_decrypt should not rely on the external
shell (and the available binaries) provided by TWFunc::Exec_Cmd.
Doing so may lead to failures resulting from different shell
provided binaries not working properly, especially since busybox
can be inconsistent across different trees.
In particular the following functions have been changed:
* run_vdc() no longer uses daisy chained commands, instead
it now forks and executes vdc directly including a 30 second
built in timeout.
* Symlink_Firmware_Files() no longer relies on the shell 'find'
command to retrieve the list of firmware/vendor files and instead
uses a built in function, Find_Firmware_Files(), which traverses
the system partition to retrieve the list of files.
* The code has also been cleaned up a little for better consistency,
and vold_decrypt will now return various error codes for the
different failures, as defined in vold_decrypt.h, which allows the
gui_msg to be moved back to partitionmanager.cpp.
Notes regarding pre Android 8.0 builds:
* Service names in .rc files cannot exceed 16 characters (including
the prepended 'sys_' or 'ven_') in Android 7.1 and below, so a
service name such as 'sys_hwservicemanager' is out of the question
for 7.1 and below.
* hwservicemanager will check ACLs on 'hwservicemanager' and 'ITokenManager'
if they are even allowed to run, otherwise the interfaces will fail.
The policies have only been introduced in 8.0, and although it is possible
to manually add them to the 7.1 policies it's not recommended.
* Therefore the best course of action is to build in 8.0.
* SIDE NOTE: On the HTC U11 we are actually using omni-7.1 with some changes
in the device tree to support both Nougat and Oreo decryption, please
refer to:
1- https://gerrit.twrp.me/c/2756/ for the necessary sepolicy and
BoardConfig changes.
2- The Android.mk file for vold_decrypt was modified to truncate
greater than 16 character service names (as mentioned therein)
Other changes:
* TW_CRYPTO_SYSTEM_VOLD_DISABLE_TIMEOUT is now deprecated due to built-
in fork and timeout.
* Output_dmesg_to_recovery_log() is also deprecated so upon a failed
decryption the recovery.log will no longer append it, instead you can
just use 'adb shell dmesg' to check it. Nonetheless if a true debug
build is needed use the original TW_CRYPTO_SYSTEM_VOLD_DEBUG flag as
outlined in the original commit message (see below).
Usage info:
This is an update to the initial vold_decrypt, for more info refer to
71c6c50d0d
Change-Id: Id7129d125ae7f5dcba0779489825add718022ba3
Includes various minor fixes for building in Android 8 trees with r23+ tag
Update FBE extended header in libtar to version 2 and include the entire
ext4_encryption_policy structure now after translating the policy.
See this post for more details:
https://plus.google.com/u/1/+DeesTroy/posts/i33ygUi7tiu
Change-Id: I2af981e51f459b17fcd895fb8c2d3f6c8200e24b
Auto detect and support both the v1 and v2 fstab formats
Support putting TWRP style flags in a separate /etc/twrp.flags file
twrp.flags format is the same as twrp.fstab (v1 with TWRP flags)
Support using a wildcard in a block device and find all partitions:
/usb-otg vfat /dev/block/sda*
Support using sysfs entries (voldmanaged) and read uevents and scan for
wildcard partitions from uevent data. (twvold?)
May not be complete for some of the newer flags found in fstabs in newer
build trees and there is a slim chance of a crash if the user removes a
removable device while TWRP is performing actions. May need to add some
kind of mutex to prevent the 2 threads from causing this crash. We need
to start somewhere though and this change is pretty innocuous when not
using a v2 fstab.
Change-Id: I617d97c7db332cbe671a9d2b8ad98b3d9c4f03cc
Restore adb backup files that TWRP made to your PC.
Put files in your backup directory to see them.
e.g. /sdcard/TWRP/BACKUPS/<sn>
Change-Id: I2c57970d77b64c39a302159041456e761c185259
* When you format the system partition, it will remain
in a mounted state.
* Subsequently restoring a system_image (even though
successfully) the Update_System_Details() function
will not correctly update the system partition
details.
* Reproducible by:
1- Advanced wipe: System
2- Restore: System_Image
3- Reboot -> No OS prompt
* eg: [
~ # twrp get tw_min_system
tw_min_system = 50
~ # twrp get tw_backup_system_size
tw_backup_system_size = 8
~ # mount -o ro /system
~ # du -sh /system
3.5G /system
]
Change-Id: I99f75274816788dd38eccdd387f7ac691e1f3fab
If the user selects to skip digest creation, Backup_Partition always returned
false. This patch fixes this problem and somewhat cleans up the error
handling.
Change-Id: I1db0e285cd5ed2bd93756cd27c6f56b8415ffa86
This patch is to refactor twrpDigest using polymorphism
and inheritance to use the same call patterns for creating and
reading a digest. Now a library.
Use SHA2 from libcrypto. SHA2 is default if device has libcrypto.
Change string MD5 everywhere to use digest or Digest instead. Updated
string tags to digest. Translation will be required.
Switch out digest code into a driver class from partitionmanager.
SHA2 is better for digest creation due to decreased collision space
compared to MD5 and SHA1.
See https://en.wikipedia.org/wiki/SHA-2
Change-Id: I74b5546789990b12aa4ce2e389d25f80a3fe213f
* Partition information should be updated after successful
decryption (both userdata and adopted) and then logged
* Fix adopted storage nickname being ""
Change-Id: Idcdab84f339e932e62880089bad36e206920dd70
Set Backup_FileName on adb backup to fix restore.
Set read_from_adb after EOF to return properly
from newer adb client prompt.
Create setter and getter for Backup_FileName
for openrecoveryscript.cpp which is not a
friend class.
Fix condition for twrpTar::Get_Size() and adb backup.
Change-Id: I7eb8168fa3416366b996e220cecbc0e1bcec5e8e
If TWRP crypto fails to decrypt partition, mount the system
partition and use system's own vold to attempt decryption.
This provides a fallback for proprietary OEM encryption as well as
encryption methods which TWRP hasn't been updated for.
Requirements in device tree:
* fstab.{ro.hardware} in device/recovery/root
The fstab does not need to be complete, but it does need the
data partition and the encryption entries.
* 'TW_CRYPTO_USE_SYSTEM_VOLD := true' in BoardConfig
or
* 'TW_CRYPTO_USE_SYSTEM_VOLD := <list of services>'
Notes:
* Setting the flag to 'true' will just use system's vdc+vold
or
* Setting the flag with additional services, will also start them
prior to attempting vdc+vold decryption, eg: for qualcomm based
devices you usually need 'TW_CRYPTO_USE_SYSTEM_VOLD := qseecomd'
* For each service listed an additional import will be automatically
added to the vold_decrypt.rc file in the form of
init.recovery.vold_decrypt.{service}.rc
You will need to add any not already existing .rc files in
your device/recovery/root folder.
* The service names specified in the vold_decrypt.{service}.rc files
have to be named 'sys_{service}'
eg: 'service sys_qseecomd /system/bin/qseecomd'
* Any service already existing in TWRP as {service} or sbin{service} will
be stopped and restarted as needed.
* You can override the default init.recovery.vold_decrypt.rc file(s)
by placing same named ones in your device/recovery/root folder.
If you do, you'll need to manually add the needed imports.
* If /vendor and /firmware folders are temporarily moved and symlinked
to the folders and files in the system partition, the properties
'vold_decrypt.symlinked_vendor' and 'vold_decrypt.symlinked_firmware'
will be set to 1.
This allows for additional control in the .rc files for any extra
actions (symlinks, cp files, etc) that may be needed for decryption
by using: on property:vold_decrypt.symlinked_vendor=1 and/or
on property:vold_decrypt.symlinked_firmware=1 triggers.
Debug mode: 'TW_CRYPTO_SYSTEM_VOLD_DEBUG := true' in BoardConfig
* Specifying this flag, will enable strace on init and vdc, which will
create separate log files in /tmp for every process created, allowing
for detailed analysis of which services and files are being accessed.
* Note that enabling strace will expose the password in the logs!!
* You need to manually add strace to your build.
Thanks to @Captain_Throwback for co-authoring and testing.
Tested successfully on HTC devices:
M8 (KK through MM), M9 (MM and N), A9 (N), 10 (N), Bolt (N),
Desire 626s (MM), U Ultra (N)
HTC One X9 (MTK device)
And by Nikolay Jeliazkov on: Xiaomi Mi Max
Change-Id: I4d22ab55baf6a2a50adde2e4c1c510c142714227
Sometimes, usually because of proprietary binaries related to keymaster,
decrypt will hang waiting for the keymaster to initialize forever. This patch
enables a timeout so that we don't get stuck trying to decrypt forever.
A timeout is especially important when dealing with the default password
because the user has no option to cancel when TWRP tries to decrypt.
NOTE: This patch only adds a timeout for FDE. FBE will require some special
handling because we need access to some static data and that data is not
available across a fork.
Special thanks to nkk71 for cleaning up some issues in my patch set.
Change-Id: Iccf2fe769ac27a7dcd6bfebfe7d2e9eddd034308
* As a precaution, (non-permanently) disable MTP if TWRP crashes.
* Remove a duplicate logging of 'Starting MTP'.
* Combine and improve readability of conditions before MTP starts.
* Remove unnecessary Disable_MTP() call if conditions unsatisfied.
Change-Id: I61ac772fc123da5b84c3061fbcf2116531bc4e2f
The property set of sys.usb.config was not part of the original 'Only
change USB mode to mtp when ready for MTP' commit. Further, it adds an
additional USB off/on toggle to the MTP startup routine, potentially
confusing the host computer. The default init.recovery.usb.rc already
sets the usb functions to 'adb' during 'on fs'.
It is possible that devices could run into issues with MTP startup due
to sys.usb.config not yet being set. This is actually due to a poor
design decision in Enable_MTP(), where sys.usb.config would not be
configured for mtp if sys.usb.config had never been set. It is not
necessary to check for a previous value, just set it.
Fix the USB PID for adb-only mode in Disable_MTP().
Change-Id: I21b5e64c9bdcd3104372a8b4eb8ea50cf4561892
-Change backup display name to say that we are excluding storage
-Add warning message during backup of any partition with data media
-Also eliminate unused variable from twrpTar class
Maybe this will make the people in issue 276 happy, but probably not because
they already lost their data, or because the warning text is yellow instead of
red.
https://github.com/TeamWin/Team-Win-Recovery-Project/issues/276
Change-Id: I98303fe7f6b7a25fea029637c90145258d41ee46
Preserving the average BPS across partitions appears to have been
wiped out by the adb backup patch set. Restore it.
Change-Id: Id05c12ff61260b642ee7acd979132a2cba026d87
Rename twrpDU.* to exclude.*
Remove global variable for du and replace with partition specific
variables.
Use separate exclusion lists for backups and wiping.
Clean up some includes
Fix some parenthesis in twrp.cpp that I messed up.
Note: twrpTarMain command line utility compiles but probably does
not work correctly yet due to not properly setting part_settings
Change-Id: Idec9c3e6a8782ba53f3420fa79ba33394f4f85fb
PS2: full_filename is not a dir
PS3: use a consistent format of always assuming directory
is missing the trailing / + fix whitespace alignment
Change-Id: Ib963473ae10571b3d069b326d024ca04c7224dda
(cherry picked from commit fa4ff144374474c541351d153549ad11d4396614)
The PartitionSettings struct contains some data elements that are duplicates
of data elements in the TWPartition class that is contained within the
PartitionsSettings.Part element. We will eliminate this duplication to help
reduce the chances for programming bugs.
Specifically, this fixes problems where the current file system does not
match the backed up file system.
Change-Id: I02f236e72093362050556a2e53a09d1dbb9a269d
This patchset will fix issues with creating and checking
md5 checksums with single partitions and subpartitions.
Change-Id: Iddfaf46412e95635af958094726cf9e3eb5a4cc8
* Create data manager variable for 'tw_disable_free_space'
* Make configurable through Settings
* Fix the "two presses needed to enable" issue for the checkbox on the
backup screen
* No need to capitalize every word, but do specify precedes backup
Change-Id: Id436cef13e4ca9349618420aac03862ec4e3c35e
- Fixes the "Unsupported SD Card" error when partitioning w/TWRP
PS2:
Use hex code instead of GUID to make code more succinct
Update typecode for Linux swap to be consistent
PS3: Revert PS2 changes
Change-Id: I5aec780aa4dfb7bd9025ae8cbceda13bfaaa0b03
Functionality for client side to backup
tar and image streams over adbd to the client under backup.ab.
Using adb backup on the client side you can backup the partitions
TWRP knows about.
On the client side you can do the following:
adb backup -f <filename> --twrp <options> where options are
--compress: compress data
system: backup system
cache: backup cache
data: backup data
boot: backup boot
etc for each partition.
You can string multiple options,
i.e. adb backup -f <filename> --twrp --compress cache system data
adb backup in TWRP will take any option corresponding
to TWRP fstab partitions, e.g. efs boot as well.
If you do not specify the filename with the -f option,
adb will backup your data to a filename backup.ab on the client.
You can then rename the file and encrypt it with desktop tools.
If you don't want to use command line arguments:
adb backup --twrp
will bring up the gui and allow you to choose partitions
from the backup page.
To restore the backup use the following convention:
adb restore <filename>
Structures are used to store metadata in binary inside
of the file itself. If the metadata structure is modified,
update the adb version so that it will invalidate older
backups and not cause issues on restore. When restoring,
we currently do not support picking specific partitions.
It's all or nothing.
Change-Id: Idb92c37fc9801dc8d89ed2a4570e9d12e76facf8
The necessity to process fstab twice stems from mounting partitions
while still processing. Instead, wait to finish setup of /data, /cache
and storage parameters until after fstab has been processed (once).
Change-Id: Id77e1edbab5eb68a7cd4a1f34953d819a043d47a
There a few char->string conversions between functions which handle
different parts of fstab line processing, but there are no uses of
string functions. Pass char arrays/pointers around instead.
Change-Id: I976a9a54ee8dcfb6194cadcac6a34e467602003b
* If 'ro' fsflag detected, set Mount_Read_Only = true
* Only output human readable Mount_Options to log
* Match fsflags identically (not just first n chars) since there are
no fs_flags which take arguments
* Match new processing method introduced in TW fstab flag processing
Change-Id: Iefdb76016be90a131b0d627d0cd3f18d2eb1a008
The progress bar will now be updated during image backups, restores
and during image flashing (except for sparse images which will require
significant changes to libsparse, and except for mtd nand using
flash_utils).
The progress bar will now be updated mid-file for file systems (tar) so
the user will see changes even during large file backup / restore.
Add a new progress tracking class to simplify handling of progress bar
updates. The class will only update the progress bar 5 times a second to
reduce the CPU load from updating the GUI frequently which does affect
backup times.
Change-Id: Iff382faef3df1f86604af336c1a8ce8993cd12c5
Their parents will take care of them.
Good parents.
Fixes double restoration of subpartitions when they are
also set to backup in the recovery fstab.
Change-Id: I876c179135e0cb00754e9a8cfc8eac164c4b7fd4
