In order to complete this action a user must first be granted the "User is Allowed to Delete Tasks" permission. This can be helpful when you close out a case.

Navigate to the Tasks Tab of a case.

Click the select all check box on left-hand side of the the tasks list.

After selecting all desired tasks, click the three dots below the task search bar.

Select Delete and the tasks will be permanently deleted.