Rotary Position Embedding (RoPE) introduced in paper: RoFormer: Enhanced Transformer with Rotary Position Embedding (Su et al., 2021).
Subject
Rotary Position Embedding (RoPE)
Predicate
introduced_in_paper
Object
RoFormer: Enhanced Transformer with Rotary Position Embedding (Su et al., 2021)
Primary source · preprint · 2021-04-20
RoFormer: Enhanced Transformer with Rotary Position Embedding — arXiv (Su, Lu, Pan, Murtadha, Wen, Liu)