AI-created image
Statements (219)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
gptkb:algorithm
gptkb:logic gptkb:stochastic_process statistical analysis decision process computational algorithm |
| gptkbp:abbreviation |
gptkb:MCMC
gptkb:MDP |
| gptkbp:alternativeName |
Markov_Chain_Monte_Carlo_method
Markov_chain_Monte_Carlo_algorithm Markov_chain_Monte_Carlo_method Markov_decision_process Markov_process hidden_Markov_model |
| gptkbp:application |
gptkb:statistical_mechanics
gptkb:protein_secondary_structure_prediction gptkb:Google_PageRank gptkb:reinforcement_learning computational chemistry finance game theory genetics natural language processing queueing theory robotics speech recognition weather prediction credit rating modeling network analysis queueing systems statistical inference stock market modeling handwriting recognition model selection parameter estimation image analysis gene prediction part-of-speech tagging |
| gptkbp:assumes |
gptkb:Markov_property
output independence |
| gptkbp:basedOn |
gptkb:Monte_Carlo_method
Markov chain |
| gptkbp:category |
gptkb:Monte_Carlo_methods
gptkb:Computational_statistics stochastic processes Bayesian inference Stochastic processes Markov processes numerical methods probabilistic algorithms simulation methods Statistical algorithms |
| gptkbp:characterizedBy |
state space
transition probabilities |
| gptkbp:component |
gptkb:public_policy
states actions rewards transition probabilities |
| gptkbp:describedBy |
Markov chain
|
| gptkbp:developedBy |
1950s
|
| gptkbp:field |
gptkb:artificial_intelligence
gptkb:reinforcement_learning control theory operations research |
| gptkbp:firstDescribed |
1906
1953 |
| gptkbp:form |
gptkb:probability_theory
stochastic processes tuple (S, A, P, R, γ) finite state automaton |
| gptkbp:generalizes |
Markov chain
|
| gptkbp:hasApplication |
gptkb:machine_learning
autonomous systems finance healthcare path planning inventory management queueing systems resource allocation |
| gptkbp:hasComponent |
transition probabilities
emission probabilities hidden states initial state distribution observable states |
| gptkbp:hasProperty |
gptkb:Markov_property
transience absorbing state aperiodicity ergodicity irreducibility memoryless recurrence state space stationary distribution transition matrix |
| gptkbp:hasType |
gptkb:continuous-time_Markov_chain
gptkb:continuous-time_Markov_process gptkb:discrete-time_Markov_chain gptkb:discrete-time_Markov_process gptkb:finite_Markov_decision_process gptkb:infinite_Markov_decision_process Markov chain |
| gptkbp:includes |
gptkb:Hamiltonian_Monte_Carlo
gptkb:Metropolis-Hastings_algorithm gptkb:Gibbs_sampling |
| gptkbp:introduced |
gptkb:Edward_Teller
gptkb:Nicholas_Metropolis gptkb:Arianna_W._Rosenbluth gptkb:Augusta_H._Teller gptkb:Marshall_N._Rosenbluth gptkb:Richard_Bellman |
| gptkbp:introducedIn |
1950s
1953 |
| gptkbp:inventedBy |
gptkb:Leonard_E._Baum
1960s |
| gptkbp:limitation |
autocorrelation
burn-in period slow mixing convergence diagnostics required |
| gptkbp:namedAfter |
gptkb:Andrey_Markov
|
| gptkbp:notableFor |
gptkb:Hamiltonian_Monte_Carlo
gptkb:Ising_model gptkb:Metropolis-Hastings_algorithm gptkb:Gibbs_sampling Bayesian inference statistical physics Slice sampling |
| gptkbp:notablePerson |
gptkb:Edward_Teller
gptkb:Nicholas_Metropolis gptkb:Stanislaw_Ulam gptkb:Arianna_W._Rosenbluth gptkb:Augusta_H._Teller gptkb:Marshall_N._Rosenbluth |
| gptkbp:objective |
find optimal policy
maximize expected reward |
| gptkbp:observedBy |
visible
|
| gptkbp:originatedIn |
gptkb:Los_Alamos_National_Laboratory
|
| gptkbp:output |
sampled values
empirical distribution |
| gptkbp:parameter |
action space
discount factor reward function state space transition function expectation-maximization |
| gptkbp:property |
gptkb:Markov_property
aperiodicity ergodicity irreducibility stationary distribution asymptotic convergence |
| gptkbp:purpose |
numerical integration
sampling from probability distributions approximating expectations |
| gptkbp:relatedTo |
gptkb:Monte_Carlo_integration
gptkb:Markov_property gptkb:Monte_Carlo_method gptkb:Poisson_process gptkb:stochastic_process gptkb:partially_observable_Markov_decision_process gptkb:semi-Markov_decision_process gptkb:stochastic_game Boltzmann machine Brownian motion Markov chain random walk birth-death process stochastic matrix importance sampling stochastic simulation rejection sampling |
| gptkbp:requires |
ergodicity
irreducibility stationary distribution |
| gptkbp:solvedBy |
gptkb:Baum-Welch_algorithm
gptkb:Q-learning gptkb:Viterbi_algorithm dynamic programming policy iteration value iteration Forward-backward algorithm |
| gptkbp:state |
hidden
|
| gptkbp:studiedBy |
gptkb:Andrey_Markov
|
| gptkbp:type |
generative model
|
| gptkbp:usedFor |
time series analysis
numerical integration sequence modeling sampling from probability distributions approximating posterior distributions approximating integrals estimating posterior distributions |
| gptkbp:usedIn |
gptkb:information_theory
gptkb:machine_learning gptkb:mathematics gptkb:probability_theory gptkb:statistical_mechanics Bayesian statistics biology computational biology computational physics computer science economics finance game theory natural language processing physics queueing theory robotics speech recognition statistics bioinformatics dynamic programming pattern recognition econometrics statistical physics |
| gptkbp:bfsParent |
gptkb:Markov_property
gptkb:machine_learning |
| gptkbp:bfsLayer |
4
|