Statements (22)
| Predicate | Object |
|---|---|
| gptkbp:instanceOf |
concept
|
| gptkbp:category |
mechanistic interpretability
|
| gptkbp:describedBy |
gptkb:A_Mathematical_Framework_for_Transformer_Circuits
|
| gptkbp:emergeIn |
transformer pretraining
|
| gptkbp:field |
gptkb:artificial_intelligence
gptkb:machine_learning |
| gptkbp:function |
pattern completion
induction of sequences |
| https://www.w3.org/2000/01/rdf-schema#label |
Induction Heads
|
| gptkbp:mechanismOfAction |
copying tokens based on previous context
|
| gptkbp:notableFor |
gptkb:OpenAI_GPT-2
|
| gptkbp:relatedTo |
circuit analysis
sequence modeling transformer models attention heads induction bias token copying |
| gptkbp:studiedBy |
gptkb:Tom_Lieberum
gptkb:Chris_Olah gptkb:Neel_Nanda |
| gptkbp:bfsParent |
gptkb:Transformer_Circuits
|
| gptkbp:bfsLayer |
8
|