Toward a theory of information processing

Sinan Sinanovic, Don H. Johnson

    Research output: Contribution to journalArticlepeer-review

    28 Citations (Scopus)


    Information processing theory endeavors to quantify how well signals encode information and how well systems, by acting on signals, process information. We use information-theoretic distance measures, the Kullback–Leibler distance in particular, to quantify how well signals represent information. The ratio of distances calculated between two informationally different signals at a system's output and input quantifies the system's information processing properties. Using this approach, we derive the fundamental processing capabilities of simple system architectures that apply universally: the systems and the kinds of signals they process and produce do not affect our general results. Applications in array signal processing and in neural signal analysis illustrate how to apply the theory.
    Original languageEnglish
    Pages (from-to)1326-1344
    Number of pages19
    JournalSignal Processing
    Issue number6
    Early online date11 Dec 2006
    Publication statusPublished - Jun 2007


    • information theory
    • Kullback–Leibler distance
    • Information processing


    Dive into the research topics of 'Toward a theory of information processing'. Together they form a unique fingerprint.

    Cite this