BAT-LZ Out of Hell
Zsuzsanna Lipták, Francesco Masillo, and Gonzalo Navarro
Despite consistently yielding the best compression on repetitive text
collections, the Lempel-Ziv parsing has resisted all attempts at offering
relevant guarantees on the cost to access an arbitrary symbol. This makes it
less attractive for use on compressed self-indexes and other compressed data
structures. In this paper we introduce a variant we call BAT-LZ (for Bounded
Access Time Lempel-Ziv) where the access cost is bounded by a parameter given
at compression time. We design and implement a linear-space algorithm that, in
time O(n log^3 n), obtains a BAT-LZ parse of a text of length
n by greedily maximizing each next phrase length. The algorithm builds on a new linear-space data structure that solves 5-sided
orthogonal range queries in rank space, allowing updates to the coordinate
where the one-sided queries are supported, in O(log^3 n) time for both
queries and updates. This time can be reduced to O(log^2 n) if
O(n log n) space is used.
We design a second algorithm that chooses the sources for the phrases in a clever way, using an enhanced suffix tree, albeit no longer guaranteeing longest possible phrases. This algorithm is much slower in theory, but in practice it is comparable to the greedy parser, while achieving significantly superior compression.
We then combine the two algorithms, resulting in a parser that always chooses
the longest possible phrases, and the best sources for those. Our
experimentation shows that, on most repetitive texts, our algorithms reach an
access cost close to log_2 n on texts of length n, while incurring almost no loss in the compression ratio when compared with classical LZ-compression. Several open challenges are discussed at the end of the paper.