Summary of Why Fine Tuning is Dead w/Emmanuel Ameisen
Emmanuel Ameisen, in the video "Why Fine Tuning is Dead," challenges the traditional importance placed on fine-tuning in Machine Learning. He stresses the significance of Data Work, Engineering, Monitoring, and Evaluation in ML projects. Ameisen provides examples of fine-tuning models for tasks like language translation and code completion, highlighting its limitations compared to prompts and retrieval methods. The discussion also covers the challenges of staying updated with the latest models and the cost-effectiveness of fine-tuning. The subtitles further delve into the need for clear prompts, evaluating sets, and trends in model prices and context sizes. Ameisen suggests focusing on prompts and relevant examples before considering fine-tuning, indicating a potential decrease in its necessity with advancements in AI Engineering. The video overall advocates for a holistic approach to ML projects, combining fine-tuning with other crucial elements for success.
Notable Quotes
— 35:34 — « If you use API providers for models. And so like, theres like a strong, I think temptation for people that are just like interested in interesting problems, a temptation that I have and understand of like, no, like I want to like. get back and do the fun ML stuff. »
— 46:13 — « If they get faster like you just get your response immediately. And so I think theres like a really interesting question of like, obviously you cant extrapolate, you know, like any exponential or even like straight line forever. Theres always points at which they stop. »
— 49:35 — « Work on your prompt, find the examples that dont work, add them either as an example to your prompts or as one that you can retrieve and add conditionally and do this like 10 times. And then only after that consider anything else that tends to work really well. »
Category
Science and Nature