Input 0 of layer "dense" is incompatible with the layer: expected min_ndim=2, found ndim=1. Full shape received: (None,)

[mrdbourke]

If you've upgraded to TensorFlow 2.7.0 (the default in Google Colab from November 2021), you might run into this issue whilst going through notebook 02 when running model_3.

ValueError: Exception encountered when calling layer "sequential_2" (type Sequential).
    
Input 0 of layer "dense" is incompatible with the layer: expected min_ndim=2, found ndim=1. Full shape received: (None,)

Original Code (errors after 2.7.0)

## Create model
# Set random seed
tf.random.set_seed(42)

# 1. Create the model (this time 3 layers)
model_3 = tf.keras.Sequential([
  tf.keras.layers.Dense(100), # add 100 dense neurons
  tf.keras.layers.Dense(10), # add another layer with 10 neurons
  tf.keras.layers.Dense(1)
])

# 2. Compile the model
model_3.compile(loss=tf.keras.losses.BinaryCrossentropy(),
                optimizer=tf.keras.optimizers.Adam(), # use Adam instead of SGD
                metrics=['accuracy'])

## Create data
# Set random seed
tf.random.set_seed(42)

# Create some regression data
X_regression = np.arange(0, 1000, 5)
y_regression = np.arange(100, 1100, 5)

# Split it into training and test sets
X_reg_train = X_regression[:150]
X_reg_test = X_regression[150:]
y_reg_train = y_regression[:150]
y_reg_test = y_regression[150:]

# Fit our model to the data
# -> Note: Before TensorFlow 2.7.0, this line would work <- 
model_3.fit(X_reg_train, y_reg_train, epochs=100)

[Fix 1] After TensorFlow 2.7.0 & using tf.keras.losses.BinaryCrossEntropy()

If using tf.keras.losses.BinaryCrossEntropy() (classification) and your input data is one dimensional ([0, 1, 2, 3, 4, 5...]), you need to specify the input_shape to the model.

# Set random seed
tf.random.set_seed(42)

# 1. Create the model (this time 3 layers)
model_3 = tf.keras.Sequential([
  # Before TensorFlow 2.7.0
  # tf.keras.layers.Dense(100), # add 100 dense neurons
  # After TensorFlow 2.7.0
  tf.keras.layers.Dense(100, input_shape=(None, 1)), # add 100 dense neurons with input_shape defined (None, 1) = look at 1 sample at a time
  tf.keras.layers.Dense(10), # add another layer with 10 neurons
  tf.keras.layers.Dense(1)
])

# 2. Compile the model
model_3.compile(loss=tf.keras.losses.BinaryCrossentropy(),
                optimizer=tf.keras.optimizers.Adam(), # use Adam instead of SGD
                metrics=['accuracy'])

And then expand the dimensions of the input:

# Set random seed
tf.random.set_seed(42)

# Create some regression data
X_regression = np.arange(0, 1000, 5)
y_regression = np.arange(100, 1100, 5)

# Split it into training and test sets
X_reg_train = X_regression[:150]
X_reg_test = X_regression[150:]
y_reg_train = y_regression[:150]
y_reg_test = y_regression[150:]

# Fit our model to the data
# Note: Before TensorFlow 2.7.0, this line would work
# model_3.fit(X_reg_train, y_reg_train, epochs=100)

# After TensorFlow 2.7.0
model_3.fit(tf.expand_dims(X_reg_train, axis=-1), 
            y_reg_train,
            epochs=100)

[Fix 2] After TensorFlow 2.7.0 & using tf.keras.losses.mae

If you're using tf.keras.losses.mae and your data is one dimensional ([1, 2, 3, 4, 5...]) you only have to expand the dimensions of the input data.

# Setup random seed
tf.random.set_seed(42)

# Recreate the model
model_3 = tf.keras.Sequential([
  tf.keras.layers.Dense(100),
  tf.keras.layers.Dense(10),
  tf.keras.layers.Dense(1)
])

# Change the loss and metrics of our compiled model
model_3.compile(loss=tf.keras.losses.mae, # change the loss function to be regression-specific
                optimizer=tf.keras.optimizers.Adam(),
                metrics=['mae']) # change the metric to be regression-specific

# Fit the recompiled model
model_3.fit(tf.expand_dims(X_reg_train, axis=-1),  # <- after 2.7.0
            y_reg_train, 
            epochs=100)

Why this happens

When TensorFlow 2.7.0 was released one of the major breaking changes was:

tf.keras:

• The methods Model.fit(), Model.predict(), and Model.evaluate() will no longer uprank input data of shape (batch_size,) to become (batch_size, 1). This enables Model subclasses to process scalar data in their train_step()/test_step()/predict_step() methods.
  Note that this change may break certain subclassed models. You can revert back to the previous behavior by adding upranking yourself in the train_step()/test_step()/predict_step() methods, e.g. if x.shape.rank == 1: x = tf.expand_dims(x, axis=-1). Functional models as well as Sequential models built with an explicit input shape are not affected.

This means that inputs with shape (batch_size,) won't automatically be upranked to become (batch_size, 1).

In notebook 02, X_reg_train is of shape (150, ) which is like (batch_size=150, ).

X_reg_train.shape
>>> (150,)

So for the model to work we have to uprank it on the last dimension to be (batch_size=150, 1):

tf.expand_dims(X_reg_train, axis=1).shape
>>> TensorShape([150, 1])

If you get any shape issues through the notebook and are running TensorFlow 2.7.0 this is likely one of the causes.

See this Google Colab notebook for more: https://colab.research.google.com/drive/1_dlrB_DJOBS9c9foYJs49I0YwN7LTakl?usp=sharing

The course notebooks have been updated to reflect these changes.

[PaoloTCS]

Thanks!

[01Andolos]

Hello
Thanks for explaining this. My tensor flow version is 2.9.2. I have wrote this code:
#create
model=tf.keras.Sequential([
tf.keras.layers.Dense(100),
tf.keras.layers.Dense(10),
tf.keras.layers.Dense(1)
])
#compile
model.compile(loss= tf.keras.losses.mae,
optimizer=tf.keras.optimizers.Adam(),
metrics=["mae"])
#fit
model.fit(tf.expand_dims(X, y, epochs=100) # I didn't add X_reg_train because colab says it doesn't recognize it

I get rid of the error about layers but now I get this error:
File "", line 13
model.fit(tf.expand_dims(X, y, epochs=1)
^
SyntaxError: unexpected EOF while parsing <<<<<<<<This error

Could you tell me how I can fix this? Thanks for your assistance.

[ranjeev3000]

Hi @01Andolos

You have got the error as your code has a syntax error. This has been very well explained by the author @mrdbourke .

Instead of writing model.fit(tf.expand_dims(X, y, epochs=1).

Try this: model.fit(tf.expand_dims(X, axis=-1),y, epochs =1)

[kkleiner]

wow, I was running into this problem while taking the mrdbourke youtube tensorflow course, I did a Google search, and landed on this page which has the fix! Nice!

[Justin-Jonany]

I don't think we should add the input_shape=(None, 1) because doing so, the model's input shape will be (none, none, 1) if you check the plot_model with show_shapes=True. This means that in the input layer, we are inputting an array rather than the numbers itself, i.e we're giving batches of array. Instead, we should specify the input shape as (1,) and so the model's input shape form plot_model would be (None, 1) which means we're giving it batches of integers