Nguồn ngoài
Information and Entropy
This text explores the ultimate limits to communication and computation, with an emphasis on the physical nature of information and information processing. Topics include: information and computation, digital signals, codes and compression, applications such as biological representations of information, logic circuits, computer architectures, and algorithmic information, noise, probability, error correction, reversible and irreversible operations, physics of computation, and quantum computation.
- Tác giả
- Paul Penfield, Jr.
- Nhà xuất bản
- Massachusetts Institute of Technology
- Nguồn
- LibreTexts
- Giấy phép
- CC BY-NC-SA
- Ngôn ngữ
- EN
Mục lục· 162 mục
- 01: Bits
- 1.01: The Boolean Bit
- 1.02: The Circuit Bit
- 1.03: The Control Bit
- 1.04: The Physical Bit
- 1.05: The Quantum Bit
- 1.5.01: An Advantage of Qubits
- 1.06: The Classical Bit
- 1.07: Summary
- 02: Codes
- 2.01: Symbol Space Size
- 2.02: Use of Spare Capacity
- 2.2.01: Binary Coded Decimal (BCD)
- 2.2.02: Genetic Code
- 2.2.03: Telephone Area Codes
- 2.2.04: IP Addresses
- 2.2.05: ASCII
- 2.03: Extension of Codes
- 2.04: Fixed-Length and Variable-Length Codes
- 2.4.01: Morse Code
- 2.05: Detail- ASCII
- 2.06: Detail - Integer Codes
- 2.07: Detail- The Genetic Code
- 2.08: Detail- IP Addresses
- 2.09: Detail- Morse Code
- 03: Compression
- 3.01: Variable-Length Encoding
- 3.02: Run Length Encoding
- 3.03: Static Dictionary
- 3.04: Semi-adaptive Dictionary
- 3.05: Dynamic Dictionary
- 3.5.01: The LZW Patent
- 3.5.02: How does LZW work
- 3.06: Irreversible Techniques
- 3.07: Detail- LZW Compression
- 3.7.01: LZW Algorithm Example 1
- 3.7.02: LZW Algorithm Example 2
- 3.08: Detail- 2-D Discrete Cosine Transformation
- 3.8.01: Discrete Linear Transformations
- 3.8.02: Discrete Cosine Transformation
- 04: Errors
- 4.01: Extension of Symbol Model
- 4.02: How do Errors Happen
- 4.03: Detection vs. Correction
- 4.04: Hamming Distance
- 4.05: Single Bits
- 4.06: Multiple Bits
- 4.6.01: Parity
- 4.6.02: Rectangular Codes
- 4.6.03: Hamming Codes
- 4.07: Block Codes
- 4.08: Advanced Codes
- 4.09: Detail- Check Digits
- 05: Probability
- 5.01: Events
- 5.02: Known Outcomes
- 5.03: Unknown Outcomes
- 5.04: Joint Events and Conditional Probabilities
- 5.05: Averages
- 5.06: Information
- 5.07: Detail- Efficient Source Code
- 5.08: Properties of Information
- 5.09: Efficient Source Coding
- 5.10: Detail- Life Insurance
- 06: Communications
- 6.01: Source Model
- 6.1.01: Kraft Inequality
- 6.02: Source Entropy
- 6.2.01: Gibbs Inequality
- 6.03: Source Coding Theorem
- 6.04: Channel Model
- 6.05: Noiseless Channel Theorem
- 6.06: Noisy Channel
- 6.07: Noisy Channel Capacity Theorem
- 6.08: Reversibility
- 6.09: Detail- Communication System Requirements
- 07: Processes
- 7.01: Types of Process Diagrams
- 7.02: Probability Diagrams
- 7.2.01: Example- AND Gate
- 7.2.02: Example- Binary Channel
- 7.03: Information Loss and Noise
- 7.3.01: Example- Symmetric Binary Channel
- 7.04: Deterministic Examples
- 7.4.01: Error Correcting Example
- 7.05: Capacity
- 7.06: Information Diagrams
- 7.6.01: Notation
- 7.07: Cascaded Processes
- 08: Inference
- 8.01: Estimation
- 8.1.01: Symmetric Binary Channel
- 8.1.02: Non-symmetric Binary Channel
- 8.1.03: Berger's Burgers
- 8.1.04: Inference Strategy
- 8.02: Principle of Maximum Entropy - Simple Form
- 8.2.01: Berger's Burgers
- 8.2.02: Probabilities
- 8.2.03: Entropy
- 8.2.04: Constraints
- 8.2.05: Maximum Entropy Analytic Form
- 8.2.06: Summary
- 09: Principle of Maximum Entropy
- 9.01: Problem Setup
- 9.1.01: Bergers Burgers
- 9.1.02: Magnetic Dipole Model
- 9.02: Probabilities
- 9.03: Entropy
- 9.04: Constraints
- 9.05: Maximum Entropy Analytic Form
- 9.06: Maximum Entropy Single Constraint
- 9.6.01: Dual Variable
- 9.6.02: Probability Formula
- 9.6.03: The Maximum Entropy
- 9.6.04: Evaluating the Dual Variable
- 9.6.05: Examples
- 10: Physical Systems
- 10.01: Nature of Quantum Mechanics
- 10.02: Introduction to Quantum Mechanics
- 10.03: Stationary States
- 10.04: Multi-State Model
- 10.4.01: Energy Systems
- 10.4.02: Information Systems
- 11: Energy
- 11.01: Magnetic Dipole Model
- 11.02: Principle of Maximum Entropy for Physical Systems
- 11.2.01: General Properties
- 11.2.02: Differential Forms
- 11.2.03: Differential Forms with External Parameters
- 11.03: System and Environment
- 11.3.01: Partition Model
- 11.3.02: Interaction Model
- 11.3.03: Extensive and Intensive Quantities
- 11.3.04: Equilibrium
- 11.3.05: Energy Flow Work and Heat
- 11.3.06: Reversible Energy Flow
- 12: Temperature
- 12.01: Temperature Scales
- 12.02: Heat Engine
- 12.03: Energy-Conversion Cycle
- 13: Quantum Information
- 13.01: Quantum Information Storage
- 13.02: Model 1- Tiny Classical Bits
- 13.03: Model 2- Superposition of States (the Qubit)
- 13.04: Model 3- Multiple Qubits with Entanglement
- 13.05: Detail- Qubit and Applications
- 13.06: Bracket Notation for Qubits
- 13.6.01: Kets Bras Brackets and Operators
- 13.6.02: Tensor ProductComposite Systems
- 13.6.03: Entangled qubits
- 13.07: No Cloning Theorem
- 13.08: Representation of Qubits
- 13.8.01: Qubits in the Bloch sphere
- 13.8.02: Qubits and symmetries
- 13.8.03: Quantum Gates
- 13.09: Quantum Communication
- 13.9.01: Teleportation - Alice and Bobs story
- 13.9.02: Quantum Cryptography
- 13.10: Quantum Algorithms
- 13.10.01: Deutsch Josza
- 13.10.02: Grover
- 13.11: Quantum Information Science