You mentioned a need for specialist models, trained on task-specific data. Considering the high costs associated with training and fine-tuning SotA models, as well as recent research (https://arxiv.org/abs/2311.16452; https://arxiv.org/abs/2401.01943) suggesting that generalist training may be preferable overall, what are your thoughts on instead employing a combination of advanced prompting techniques (ICL, CoT, Ensembling) and traditional algorithms to improve accuracy?
To stick with an example you provided, a potential approach could look like this:
1) An LLM identifies the overall issue (diabetes mellitus)
2) An algorithm selects the appropriate checklist (pregnancy-induced? secondary issue? ...)
3) The LLM answers the checklist questions in a provided format (ICL)
4) Based on this structured response, the algorithm selects the appropriate code.
Very interesting read, thank you!
You mentioned a need for specialist models, trained on task-specific data. Considering the high costs associated with training and fine-tuning SotA models, as well as recent research (https://arxiv.org/abs/2311.16452; https://arxiv.org/abs/2401.01943) suggesting that generalist training may be preferable overall, what are your thoughts on instead employing a combination of advanced prompting techniques (ICL, CoT, Ensembling) and traditional algorithms to improve accuracy?
To stick with an example you provided, a potential approach could look like this:
1) An LLM identifies the overall issue (diabetes mellitus)
2) An algorithm selects the appropriate checklist (pregnancy-induced? secondary issue? ...)
3) The LLM answers the checklist questions in a provided format (ICL)
4) Based on this structured response, the algorithm selects the appropriate code.