Course Code and Title
Information and Coding Theory (EE3505)
Course Category
GCE/PME
Programe
BTech
Course Credit
3-0-0-3 (Lecture-Tutorial-Practical-Credit)
Prerequisite
Knowledge of probability theory, linear algebra, and basic mathematical analysis
Consent of Teacher
Required
Course Content
| Topic | Description |
|---|---|
| Information and entropy | Introduction to the fundamental concepts of information and entropy. |
| Inequalities: Data-Processing, Fano's, and Kraft's | Exploration of key inequalities in information theory. |
| Typical sequences and asymptotic equipartition property | Understanding of typical sequences and their applications in coding theory. |
| Source coding and data compression, Huffman codes | Study of source coding techniques, including Huffman coding for data compression. |
| Channel capacity, channel coding | Analysis of channel capacity and coding strategies for reliable communication. |
| Gaussian channel | Modeling and analysis of Gaussian communication channels. |
| Linear block codes: coding and decoding | Introduction to linear block codes and their encoding and decoding methods. |
| Convolutional codes: coding and decoding | Study of convolutional codes, including coding and decoding techniques. |
| Kolmogorov and computational complexity | Introduction to concepts of complexity in information theory. |
Learning Objectives
To develop an understanding of quantification of information and analytical tools necessary to apply information theory to modern engineering problems. To mathematically model and analyze communication channels. To understand and provide solutions to the key issues of compression and error correction of data. To convey the principles and applications of information theory.
Learning Outcomes
- Apply the concepts of entropy to quantify information.
- Mathematically model communication systems and the transfer of information in them.
- Analyze the performance of communication channels and information exchange efficiencies.
- Design codes for error correction and compression of data.
Textbook
- T. M. Cover, J. A. Thomas, "Elements of Information Theory", John Wiley & Sons.
References
- D. J. C. MacKay, "Information Theory, Inference, and Learning Algorithms", Cambridge University Press.
- R. G. Gallager, "Information Theory and Reliable Communication", John Wiley & Sons.
- M. Kelbert and Y. Suhov, "Information Theory and Coding by Example", Cambridge University Press.