reversal-curse · activeNot generalizing "A is B" to "B is A"
A model that learned a fact in one direction often cannot recall it when asked in the reverse direction.
Also called: directional recall, asymmetric knowledge
Tags: general, knowledge
A capable model that knows a relation between two entities can answer questions about it from either side, without needing to have seen both orderings in training.
Claims
- mechanismsingle paperModels fine-tuned on "A is B" fail to answer "B is A", and GPT-4 recalls celebrity parents far more often than the reverse.
- observationsingle paperGPT-4 could reliably name a well-known celebrity's parent, but was far less reliable naming the celebrity when given the parent — the same directional-recall asymmetry demonstrated in smaller fine-tuned models, showing up in a deployed model that wasn't fine-tuned for the test.
- mechanismsingle paperTraining on reversed sequences alongside forward text largely removes the effect.
Techniques
- Train on reversed texttrainingInclude reversed word or entity sequences in pretraining so facts are learned in both directions.
Capabilities are a way of carving up the subject, and carvings are arguable. Say so if this one is wrong — especially a proposed one, which a pipeline added because several papers used the same framing, not because anyone decided it was right.