Repository navigation
Routing: Environment variable for inference for Gemini #978
Description
Is your feature request related to a problem?
There are currently no flags to control the routing of Gemini model inferences through Google Vertex and Google AiStudio. This requires code changes for any ad-hoc routing adjustments, which is inefficient.
Describe the solution you'd like
Add an environment variable to allow easy routing to either google-gcp or google-aistudio.
Additional context
https://docs.google.com/document/d/1AT-vWVkF9iCLFoUc9HGRRwNAK-GM_9Qc0iHrDiomtX0/edit?usp=sharing
Original issue
Describe the current behavior
Currently there are no flags that could route Gemini model inferences via Google vertex and Google AiStudio.
It has to be via code changes if the ad-hoc routing changes are necessary.
Describe the enhancement you'd like
Add an env variable to efficiently route to either vertex or aistudio
Metadata
Metadata
Assignees
Labels
Type
Projects
- Status
Activity
[-]Gemini: Toggleable Default Routing for Gemini[/-][+]Routing: Environment variable for inference[/+][-]Routing: Environment variable for inference[/-][+]Routing: Environment variable for inference for Gemini[/+]