“Infini-attention” isn’t perfect memory, it’s highly compressed representations of the entire history. (arxiv.org/html/2404.07143v1).

Much like quantization. The longer the context gets the worse the compression makes recall.