well, and straight caching. If they know that 80% of people ask the same 10,000 ... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		VirusNewbie on Oct 10, 2023 \| parent \| context \| favorite \| on: GitHub Copilot loses an average of $20 per user pe... well, and straight caching. If they know that 80% of people ask the same 10,000 questions without a back and forth dialogue, it's not hard to just write a front end for that.

vineyardmike on Oct 10, 2023 [–]

This problem won’t work for code assistants. No way those queries have high repetition. Not when you’re uploading user files.

I assume the best strategy is to shrink the models and tune them more.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact