error while using LstmPolicy

[pirate-lofy]

I'm trying to implement impalaCNN with lstm output layer with PPO2, so I used LstmPolicy with impala as feature extractor, but it gives me this error

File "C:\Users\AppData\Local\Programs\Python\Python37\lib\site-packages\stable_baselines\common\tf_layers.py", line 143, in lstm
  weight_x = tf.get_variable("wx", [n_input, n_hidden * 4], initializer=ortho_init(init_scale))
File "C:\Users\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\ops\variable_scope.py", line 1500, in get_variable
  aggregation=aggregation)
File "C:\Users\BigNrz\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\ops\variable_scope.py", line 1243, in get_variable
  aggregation=aggregation)
File "C:\Users\BigNrz\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\ops\variable_scope.py", line 567, in get_variable
  aggregation=aggregation)
File "C:\Users\BigNrz\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\ops\variable_scope.py", line 519, in _true_getter
  aggregation=aggregation)
File "C:\Users\BigNrz\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow_core\python\ops\variable_scope.py", line 912, in _get_single_variable
  raise ValueError("The initializer passed is not valid. It should "
ValueError: The initializer passed is not valid. It should be a callable with no arguments and the shape should not be provided or an instance of `tf.keras.initializers.*' and `shape` should be fully defined.

when I traced this error I found that this line
_,n_input = [v.value for v in input_tensor[0].get_shape()] (in common/tf_layers.py line 141)
returns (1,None), then n_inputs will be None and it's not legal to define a tensorflow variable with None part in its shape.
here's the piece of code in the lstmPolicy

extracted_features = impala_gen(self.processed_obs)
input_sequence = batch_to_seq(extracted_features, self.n_env, n_steps)
masks = batch_to_seq(self.dones_ph, self.n_env, n_steps)
rnn_output, self.snew = lstm(input_sequence, masks, self.states_ph, 'lstm1', n_hidden=n_lstm,
                          layer_norm=layer_norm)

does anyone have a solution t

[Miffyli]

Possibly related issue #373 . I recommend you check it out if it offers help.

Using keras layers has given some headaches in the past, so that could be causing issues as well. Apart from that I can not tell what is the issue here.

then n_inputs will be None and it's not legal to define a tensorflow variable with None part in its shape.

It has been a while since I last time used TF, but back then a single None was allowed in tensor shape, which indicates dynamic shape (e.g. over batch). I do not think this is the issue.

[pirate-lofy]

Using keras layers has given some headaches in the past, so that could be causing issues as well. Apart from that I can not tell what is the issue here.

sorry, but I didn't use keras, it's the stable_baselines lstmPolicy implementation with tf

It has been a while since I last time used TF, but back then a single None was allowed in tensor shape, which indicates dynamic shape (e.g. over batch). I do not think this is the issue.

this is not allowed in tensorflow, I doubled checked..but I still can't figure out what is the problem here

[Miffyli]

sorry, but I didn't use keras, it's the stable_baselines lstmPolicy implementation with tf

My bad, the error message made me believe Keras things were being used.

Apart from this, I can not provide more help with this limited info, as default parts in stable-baseline work as expected. I have used custom cnn feature extractor with LSTMs without a problem.