Observability: Improve status codes consistency #834

Is your feature request related to a problem?
We need to improve observability in our application. There are inconsistencies in the status codes and error messages returned, which can lead to confusion and misinterpretation of application state.

Describe the solution you'd like

• Have Claude perform a sanity check on logs, status codes, and error messages.
• Fix status code inconsistencies in high usage endpoints like LLM call, Collection, Documents, LLM Chain and Evals. Inconsistency like below:
  • Change status code to 404 in app/crud/evaluations/batch.py:47 when success is false.
  • Change status code to 502 in backend/app/crud/assistants.py:68 for invalid responses from upstream servers.