~TemporaryDir() calls rmdir(2) directly, which works with empty
directories only.
Test: Run recovery_host_test; No leftover on host.
Test; Run recovery_component_test on marlin; No leftover on device.
Change-Id: Ib510efb16eeda61b34161e2b386499e6cb79a4ca