Abstract
The finite size of natural symbol sequences, e.g.DNA strands or texts in natural languages, is responsible for the worsening statistics with respect to increasing lengthnof thesubstrings, thus restricting the reliability of the results of higher-order entropy calculations tosmalln.A new method for the calculation of higher-order entropiesHnbased upon a theorem ofcoding theory is presented, allowing for reliable estimations far beyond. We tested the range ofvalidity of this method by means of symbol sequences with known entropies: Two stochasticprocesses (the underlying probability distribution being the equidistribution and the distributionof the nucleotides of the yeast chromosome III DNA sequence) and a Markov process of fifthorder with its transition probabilities also taken from this yeast DNA sequence.