History of open alignment, part 1

History of open alignment, part 2

History of open alignment, part 3

History of open alignment, part 4

Synthetic data

1. The first main topic is "the race to reproduce open ChatGPT" and the likes of alpaca, koala, dolly, vicuna

which coincides with llama 1

then there's the reproducing core results phase, but people mostly using LoRA which I think hurt then

DPO came out as a paper in like July I think, was added places soon after, but didnt catch on for a while https://github.com/huggingface/trl/issues/405

Talks:

• John schulman at berkeley (distillation is hard!)
• John schulman at ICML (fine-tuning off user data and iterative RLHF is hard)

Papers / methods:

• QLoRA, Quanaco
• LIMA

Evaluation Tools:

• ChatBotArena May 2023
• MT Bench