This page lets you view, listen to and delete all of your Google Audio history. To delete all, find the 3 dots -> delete options -> advanced

Source: Google – Voice & Audio Activity