entropy (information theory)

GPTKB entity

Predicate	Object
gptkbp:instanceOf	concept statistical analysis
gptkbp:appliesTo	discrete random variables continuous random variables
gptkbp:continuousVersion	differential entropy
gptkbp:describes	uncertainty in a random variable
gptkbp:field	gptkb:information_theory
gptkbp:form	H(X) = -Σ p(x) log p(x)
gptkbp:generalizes	gptkb:Rényi_entropy gptkb:Tsallis_entropy
gptkbp:hasUnit	bit hartley nat
https://www.w3.org/2000/01/rdf-schema#label	entropy (information theory)
gptkbp:introduced	gptkb:Claude_Shannon
gptkbp:introducedIn	1948
gptkbp:maximizedWhen	all outcomes are equally likely
gptkbp:minimizedWhen	one outcome has probability 1
gptkbp:namedAfter	thermodynamic entropy
gptkbp:relatedTo	gptkb:Kullback–Leibler_divergence gptkb:Shannon_entropy conditional entropy joint entropy mutual information thermodynamic entropy
gptkbp:symbol	H
gptkbp:usedIn	gptkb:machine_learning coding theory cryptography data compression statistical inference
gptkbp:bfsParent	gptkb:Shannon's_source_coding_theorem
gptkbp:bfsLayer	6