AiProviders with a dynamic model list / only visible with apiKey is never re-evaluated

[foppelfb]

The only place where $catalog->getModels() is executed is in the SymfonyAiCompilerPass Class during Dependency Injection in the method buildBridgeDefinition. It handles the missing list (no apiKey at this point) gracefully and adds the provider to the list of providers. The set Flag $catalogIsDynamic is not passed to the configuration though.

Additionally at no point the model List is re-evaluated once an apiKey is set.

The Problem in itself manifests in the Model [model] drop-down when creating an AI Provider Configuration (tx_aim_configuration), which is never populated in this case.

Additionally the models capabilities are never re-evaluated as well.

I don't see a good solution for this problem as long as the models are evaluated during DependencyInjection, and not during runtime with all configurations available. It probably needs to move into the runtime and be cached via the cachingframework..

Maybe if the catalogIsDynamic flag could be passed to the provider/bridge configuration during the DependencyInjection, to be checked at a later point..

I see this as a problem going forward especially in the DACH region as most of my customers choose to host AI themselves or buy the service from 3rd party hosters in the region, which will only provide an apiKey and potentially a static list of models.

[o-ba]

Hey @foppelfb ,

thanks for the report. You are right, $catalogIsDynamic gets computed but then dropped again in SymfonyAiCompilerPass.

One thing worth clarifying though: the model dropdown isn't actually stuck at compile-time for all dynamic providers. It already re-checks itself live every time the form loads. That already works for Ollama/LM Studio today, because for those, the "API key" field is really just a URL, and we do a plain HTTP call to /v1/models on it to list models.

The actual gap is: providers where that field holds a real secret API key instead of a URL. Right now there's no code that takes an API key, sends it as a Bearer token to some endpoint, and lists models that way, and no separate field to store a base URL alongside a real API key either if the two need to be configured together.

So before this can be built, could you share which specific provider/service you're hitting this with? Mainly:

• Does it have a fixed, known base URL, or is that configurable per customer?
• Does it expose an OpenAI-compatible /v1/models endpoint?

With that, we can build the authenticated fetch properly instead of guessing at the shape, the way it was already possible to reuse the known /v1/models convention for Ollama/LM Studio.

And, the model capabilities being unknown for these providers is real too, but it already falls back to "assume every non-image-generation capability", so it's a smaller issue than the empty dropdown.)

Best, Oli

[foppelfb]

Hi @o-ba ,

I have 2 examples where the models are hosted through an Open WebUI (https://openwebui.com) instance. In our personal example we run an ollama server as the model hosting backend, what my other example is using i don't know. It doesn't really matter anyway, because Open WebUI offers the api, including an api endpoint /api/models which is accessible via a Bearer Token only. Additionally it offers OpenAI compatible api endpoints, so /api/v1/models works as well.

It is worth knowing that the /api/models endpoint reflects what the owner of the Bearer Token is allowed to see, down to allowed available capabilities of a model. Open WebUI offers granular permissions here. Therefor it is worthwhile to have the model-list not fixed in the bridge (like the openai-bridge is doing), as it can be extended or reduced based on permissions on the server side.

I implemented my own bridge for our own instance (https://chat.smartbrew.ai) here https://github.com/sudhaus7/ai-smartbrew-platform with a fixed URL with only the API Key provided. (hence my earlier merge request for issue #21).

My not so nice solution for this problem is at the moment that my ModelCatalog will accept an Environment Variable with the API Key, and will use that if no apiKey is provided. This works and is my current basis going forward with the refactor of my other extensions, where I want to use b13/aim as the provider manager and for its unified api. I don't like the solution very much as it circumvents how the bridges usually work.

I understand the plain call to /v1/models, I saw that, but that gives you only the model id's and not the capabilities mapping in the system later, which I want/need.

As I said, I am aware this is not an easy issue to fix, without doubling the code of the CompilerPass into a Service Class that would do the same thing during runtime, if necessary. Plus caching issues, and source of truth issues.

all the best and thanks for your work,
Frank

[foppelfb]

Sorry for re-iterating some of the parts you already stated.