DQN_Agent in shm

[excellentiam-prog]

This is a work of art, it would be wonderful to find examples inspired by you with PPO and other agents.

I uncommented and tried to test the DQN_Agent in https://github.com/druce/rl/blob/master/Trading_with_RL.ipynb https://github.com/druce/rl/blob/master/Trading%20with%20RL.ipynb because even if it is slow, it still has great value for those who are learning, like myself. I only removed the “import resources” and its dependents because I am running it on Windows

I tried "
N_EPISODES = 10000
ticks_per_episode = 1256
nstocks = 1

env = Market(shm_market_gen,
nstocks=1,
episode_length=ticks_per_episode)

agent = DQN_Agent(state_size=nstocks*32,
action_size=2,
)

start_time = time.time()

for e in range(N_EPISODES):
agent.run_episode()
agent.score_episode(e, N_EPISODES)

if e and (e+1) % agent.save_interval == 0:
    agent.save()

elapsed_time = time.time() - start_time
print("Train time: ", elapsed_time)
agent.rlplot()"
Unfortunately, I got:

Traceback (most recent call last):
File "C:\Users\guest\AppData\Roaming\Python\Python310\site-packages\IPython\core\interactiveshell.py", line 3526, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "C:\Users\guest\AppData\Local\Temp\ipykernel_11800\4180752183.py", line 19, in
agent.run_episode()
File "C:\Users\guest\AppData\Local\Temp\ipykernel_11800\165701735.py", line 198, in run_episode
self.action = self.act(self.state.reshape([1, self.state_size]))
ValueError: cannot reshape array of size 8 into shape (1,32)"

[excellentiam-prog]

I did some changes fixing the "env = Market(shm_market_gen, lag=16, nstocks=1, episode_length=10)", lag 16 fixed the "shape (1,32)" issue and seemed that the model is working. However, it crashed one time in my pc on Wndows, and other time in colab, I think that is because of the high memory use in DQN_Agent with the memory being a dataframe, while in REINFORCE it is a list. The DQN_Agent is slow, but this is ok to me, the true problem is the quantity of memory