Entropy (information theory)
GPTKB entity
Statements (30)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
concept
|
| gptkbp:appliesTo |
probability distributions
|
| gptkbp:category |
gptkb:Statistical_mechanics
gptkb:Probability_theory Information theory |
| gptkbp:describes |
uncertainty in a random variable
|
| gptkbp:field |
gptkb:information_theory
|
| gptkbp:form |
H(X) = -Σ p(x) log p(x)
|
| gptkbp:generalizes |
gptkb:Rényi_entropy
gptkb:Tsallis_entropy |
| gptkbp:hasUnit |
bit
hartley nat |
| https://www.w3.org/2000/01/rdf-schema#label |
Entropy (information theory)
|
| gptkbp:introduced |
gptkb:Claude_Shannon
|
| gptkbp:introducedIn |
1948
|
| gptkbp:maximizedBy |
gptkb:uniform_distribution
|
| gptkbp:minimizedBy |
deterministic distribution
|
| gptkbp:relatedTo |
gptkb:Kullback–Leibler_divergence
gptkb:Shannon_entropy mutual information thermodynamic entropy |
| gptkbp:symbol |
H
|
| gptkbp:usedIn |
gptkb:machine_learning
gptkb:statistical_mechanics coding theory cryptography data compression |
| gptkbp:bfsParent |
gptkb:Entropy_and_Information_Theory
|
| gptkbp:bfsLayer |
7
|