More researchers are recognizing the significance of instruction data during the Supervised Fine-Tuning (SFT) stage. In June, I wrote a blog about data generation, but I believe it was somewhat superficial and insufficient. Since then, many new methods have emerged. Therefore, I aim to cover more papers I've read to discuss instruction data generation and selection.
Many studies have shown that large language models can stimulate their ability to follow instructions and generalize on more tasks during the fine-tuning stage. However, if we only rely on manual handwritten instruction data, it will consume a lot of human resources, and the quantity is limited.Therefore, it is essential to explore other automatic methods for generating instruction data.
