I think the best way to tell whether GPT-3 has understanding of its data is by by asking questions related to the data but not explicit in the training dataset.

Maybe by asking it to clean up it's own training data?

By asking counterfactual questions.