sampled location must volatile=True(stop gradient)

[machanic]

in ram.py

eps = (self.xp.random.normal(0, 1, size=m.data.shape)).astype(np.float32)
l = m.data + np.sqrt(self.var)*eps

ln_pi = -0.5 * F.sum((l-m)*(l-m), axis=1) / self.var  #log(location policy)

because according to jlindsey15/RAM#10
the sampled location l must not allow gradient flow, so I thought the above code which made ln_pi can't stop gradient , may have issue in it.

[machanic]

maybe modified to:

eps = (self.xp.random.normal(0, 1, size=m.data.shape)).astype(np.float32)
l = m.data + np.sqrt(self.var)*eps

l = chainer.Variable(l, volatile='on')
m = chainer.Variable(l, volatile='off')
ln_pi = -0.5 * F.sum((l-m)*(l-m), axis=1) / self.var  #log(location policy)

will be better.
but this will cause chainer report error:
ValueError: ON and OFF flags cannot be mixed.
How to solve it ?

[amasky]

I've already stopped the gradient flow via "l", and this line
l = m.data + np.sqrt(self.var)*eps
does this trick.

Because chainer.Variable retains computational history and then determine via which variable it computes backprops, we can stop backprops by simply take out numpy or cupy array from chainer.Variable (m.data) and define a new chainer.Variable (this is automatically done in chainer.functions). This case, "l" does not retain the history and ln_pi error propagates only via "m".