Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dian xt ms #29

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open

Dian xt ms #29

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
31b49cb
DQN Model ms版本移植
AmiyaSX Feb 7, 2023
6948ace
优化算子使用
AmiyaSX Feb 7, 2023
b71dbcc
files organize
AmiyaSX Feb 12, 2023
fd5bfae
fix model_ms
AmiyaSX Feb 13, 2023
08c7fd6
remove redudant
AmiyaSX Feb 14, 2023
b1dc8d8
modify
AmiyaSX Feb 15, 2023
58e3c79
ppo model
AmiyaSX Mar 2, 2023
c3e4654
ppo_ms modify
AmiyaSX Mar 9, 2023
3a59633
modify ppo
AmiyaSX Mar 23, 2023
f401292
modify ppo model
AmiyaSX Mar 25, 2023
fd30d13
ppo modify
AmiyaSX Apr 8, 2023
092899a
add muzero model using mindspore
AmiyaSX Apr 14, 2023
1deed68
MuZero Model mindspore版本移植
AmiyaSX May 4, 2023
b9e29d7
code format
AmiyaSX May 9, 2023
8dd42ce
code reformat
AmiyaSX May 9, 2023
30f3af2
modify ppo
AmiyaSX May 9, 2023
3b61d7c
模型推理优化
AmiyaSX May 30, 2023
869b897
remove .idea
AmiyaSX May 30, 2023
907a549
update ppo yaml
AmiyaSX May 30, 2023
bfe0c11
bug fix
AmiyaSX May 30, 2023
eb8079c
fix for ms2.0 training
AmiyaSX Aug 23, 2023
57c537e
fix error
AmiyaSX Aug 24, 2023
06fea1d
add README&figs
AmiyaSX Aug 26, 2023
ba9061b
update
AmiyaSX Aug 31, 2023
941d56d
update
AmiyaSX Sep 12, 2023
5caf48d
update requirements
AmiyaSX Sep 12, 2023
c7fba1b
update ppo
AmiyaSX Sep 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update ppo yaml
  • Loading branch information
@AmiyaSX
AmiyaSX committed May 30, 2023
commit 907a54922e0b1b1ea5f8c11db05fc38dc85cdcfd
2 changes: 1 addition & 1 deletion examples/beamrider_ppo_ms.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ model_para:
VF_SHARE_LAYERS: True
activation: relu
hidden_sizes: [512]

USE_AMSGRAD: False
env_num: 10
speedup: False

Expand Down
4 changes: 2 additions & 2 deletions examples/breakout_ppo_ms.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@ model_para:
CRITIC_LOSS_COEF: 1.0
ENTROPY_LOSS: 0.003
LOSS_CLIPPING: 0.1
LR: 0.00025
LR: 0.0004
MAX_GRAD_NORM: 5.0
NUM_SGD_ITER: 4
SUMMARY: False
VF_SHARE_LAYERS: True
activation: relu
hidden_sizes: [256]

USE_AMSGRAD: False
env_num: 10
speedup: False

Expand Down
4 changes: 2 additions & 2 deletions examples/qbert_ppo_ms.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,14 @@ model_para:
CRITIC_LOSS_COEF: 1.0
ENTROPY_LOSS: 0.003
LOSS_CLIPPING: 0.1
LR: 0.00025
LR: 0.0005
MAX_GRAD_NORM: 5.0
NUM_SGD_ITER: 4
SUMMARY: False
VF_SHARE_LAYERS: True
activation: relu
hidden_sizes: [512]

USE_AMSGRAD: True
env_num: 10
speedup: False

Expand Down
2 changes: 1 addition & 1 deletion examples/spaceinvader_ppo_ms.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ model_para:
VF_SHARE_LAYERS: True
activation: relu
hidden_sizes: [512]

USE_AMSGRAD: False
env_num: 10
speedup: False

Expand Down
4 changes: 2 additions & 2 deletions xt/model/ppo/ppo_ms.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,10 @@ def __init__(self, model_info):
self.verbose = model_config.get('SUMMARY', SUMMARY)
self.vf_clip = Tensor(model_config.get('VF_CLIP', VF_CLIP))
self.dist = make_dist(self.action_type, self.action_dim)

self.amsgrad = model_config.get('USE_AMSGRAD', False)
super().__init__(model_info)
self.predict_net = self.PPOPredictPolicy(self.model, self.dist)
adam = Adam(params=self.predict_net.trainable_params(), learning_rate=self._lr, use_amsgrad=True, use_locking=True)
adam = Adam(params=self.predict_net.trainable_params(), learning_rate=self._lr, use_amsgrad=self.amsgrad, use_locking=True)
loss_fn = WithLossCell(self.critic_loss_coef, self.clip_ratio, self.ent_coef, self.vf_clip)
forward_fn = NetWithLoss(self.model, loss_fn, self.dist)
device_target = ms.get_context("device_target")
Expand Down