1 briefs tagged #positional-encoding.
The paper says floating-point underflow can make some ALiBi attention heads partially blind.