Does memorization in small models predict memorization in large models?

[uSaiPrashanth]

No description provided.

[StellaAthena]

We currently have the following correlation heat-map which indicates that the answer is "no." We should probably also make confusion matrices for the classifier that takes a small model and predicts memorization by the 13B model by assuming it is the same as the small model.

[lintangsutawika]

There seems to be a trend where (but weak) where the larger the model, the more it is able to predict if a sequence from 13B is memorized or not.

[StellaAthena]

The same comments about follow-ups I made on #29 apply here. The conclusions are different but the methodologies are the same.