Microsoft’s Orca Learns From Complex Explanation Traces of GPT-4 to Significantly Enhance Smaller Models
In a new paper Orca: Progressive Learning from Complex Explanation Traces of GPT-4, a Microsoft research team introduces Orca, a 13-billion parameter model that learns explanation traces; step-by-step thought processes; and complex instructions from GPT-4 to significantly boosts SOTA instruction-tuned models.