Yes, as long as it is not the only card backing an active subscription. To remove the current default card, set another card as the default first. Removing a card here also deletes it from Stripe — it is not kept anywhere else.