Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models ...
With a powerful enough PC, you don't need a cloud-hosted service to work with LLMs — you can download and run them locally on your own hardware. The hard part is standing up the infrastructure ...