Skip to content

Commit d51f103

Browse files
authored
Update README.md
1 parent 183931e commit d51f103

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,8 @@ For LongNet models ([Flash Attention](https://github.com/Dao-AILab/flash-attenti
144144
- [Retentive Network: A Successor to Transformer for Large Language Models](https://arxiv.org/abs/2307.08621)
145145
* created by `config = RetNetConfig(vocab_size=64000)` and `retnet = RetNetDecoder(config)`.
146146

147+
- [LongNet: Scaling Transformers to 1,000,000,000 Tokens](https://arxiv.org/abs/2307.02486)
148+
147149
Most of the features above can be used by simply passing the corresponding parameters to the config. For example:
148150

149151
```python

0 commit comments

Comments
 (0)