Update blog

Author: windshadow233

source/_posts/2025/06/策略梯度算法中梯度公式的推导.md

@@ -32,7 +32,9 @@ $$\max_{\theta\in\Theta}J(\theta)$$
 
 $$
 \begin{aligned}
-\nabla_\theta J(\theta)=\nabla_\theta\mathbb{E}_{s\in S}[V_{\pi_\theta}(s)]&=\nabla_\theta\mathbb{E}_{s\in S}\mathbb{E}_{a_t\sim\pi_\theta(*|s)}[Q(s,a_t)]\\
+\nabla_\theta J(\theta)&=\nabla_\theta\mathbb{E}_{s\in S}[V_{\pi_\theta}(s)]\\
+
+&=\nabla_\theta\mathbb{E}_{s\in S}\mathbb{E}_{a_t\sim\pi_\theta(*|s)}[Q(s,a_t)]\\
 
 &=\mathbb{E}_{s\in S}\nabla_\theta\mathbb{E}_{a_t\sim\pi_\theta(*|s)}[Q(s,a_t)]\\