Métricas do repositório
- Stars
- (495 stars)
- Métricas de merge de PR
- (Mesclagem média 1d 1h) (2 fundiu PRs em 30d)
Description
(This is to summarize my comments https://github.com/QuantEcon/QuantEcon.jl/pull/109#issuecomment-216474996, https://github.com/QuantEcon/QuantEcon.jl/pull/109#issuecomment-216727212, https://github.com/QuantEcon/QuantEcon.jl/pull/109#issuecomment-216764154.)
n: number of statesm: number of actions (Assume it is constant over the states)
Consider bellman_operator, where we compute
vals = R + beta * Q * vandTv = s_wise_max(vals).
Let me denote R + beta * Q * v by U(s, a) as a function of (s, a), to distinguish it from a Julia array; let me denote the probabilities by q(sa, s'), where sa is a state-action pair, while s' is a state tomorrow.
To compute s_wise_max, the data for U should be ordered in memory as
U(1, 1), ..., U(1, m), U(2, 1), ..., U(2, m), ..., U(n, 1), ..., U(n, m)
i.e., action should change faster.
So the same applies to R: action should change faster.
To compute Q * v, the data for q should be ordered in memory as
q(sa_1, 1), ..., q(sa_1, n), ..., q(sa_L, 1), ..., q(sa_L, n)
(where the state-action pairs are listed as sa_1, ..., sa_L)
i.e., state-tomorrow should change faster.
Given 1. and 2., for Q, state-tomorrow should change first, action second, and state-today third.