Only disable the controllers not the whole fragment because
user might need to have entry for other features.
Fixes: 73664409
Merged-In: I98ed248cf33d11715dd523e711cbc68ebf128ef8
Change-Id: I98ed248cf33d11715dd523e711cbc68ebf128ef8
Signed-off-by: Weilun Du <wdu@google.com>
(cherry picked from commit 68a195ae93
)