Introduction

Maximum Likelihood Sequence Estimation (MLSE) over an FIR channel with unknown coefficients can be formulated as either a channel estimation problem (followed by MLSE), a joint sequence/channel estimation problem, or a direct sequence estimation problem. Since MLSE is our primary concern, we consider only the latter two formulations here. For joint estimation of both the transmitted sequence and channel coefficients, optimum methods have been developed both for non-time-varying (e.g. [1,2,3]) and slowly time-varying (e.g. [4]) channels. This approach can be effective, and the use of non-time-varying block processing algorithms has been proposed for a fast-changing channel (under the assumption that the channel is constant over the processing block). Direct estimation of the sequence (only) can result in improved estimator performance. However, the ML formulation for optimal sequence estimation over random channels, which requires marginalization over the unknown channel parameters, is generally intractable to solve directly.

The Expectation Maximization (EM) method is an approach to development of iterative ML and Maximum a Posterior (MAP) estimation algorithms, which was introduced into the signal processing community by Dempster, et al. [5]. For digital communications, the EM approach has been applied to the channel estimation problem [6,7,8]. It has also been employed for MLSE. For example, Georghiades, et al. [9] use EM to account for unknown phase given unsynchronized reception. Nelson and Poor [10] employ EM in multiuser applications where the other users' symbols are unknown. Zeger and Kobayashi [11] estimate GMSK modulated symbols in the presence of unknown multipath parameters. In these examples, either simple prior distributions on the unknown secondary parameters are assumed in order to develop algorithms, or a high SNR approximation is derived.

In this paper we develop a general EM algorithmic approach to MLSE over unknown random ISI channels. We use EM to marginalize over unknown channel parameters, incorporating knowledge of the FIR channel coefficients in the form of linear constraints and a prior distribution on the channel parameters. However, it should be noted that improved bit-error-rate performance, relative to joint sequence/channel estimators, can be realized even with only a noninformative prior (and no constraints). Algorithms are developed for a variety of channel models, including several that account for channel variation over short symbol blocks.