Researchers retrofit language models to read raw bytes instead of tokens
A Nature paper from Ai2 and university partners shows existing language models can be converted to read raw bytes using less than 1% of a typical pretraining budget.
A Nature paper from Ai2 and university partners shows existing language models can be converted to read raw bytes using less than 1% of a typical pretraining budget.