We already send NACKs when a packet is missing. Under high packet loss, however, the recovery packet might get lost two. Forward receiver NACKs to the sender, but only after a delay and after checking that the packet has not arrived in the meantime.
Now that we cache keyframes, it is worth keeping some history even when latency is low.