ExApp upgrade fails due to a init status report rejected as unauthorized

[kyteinsky]

Describe the bug

Upgrade of an ExApp context_chat_backend failed to automatically set everything up. The "version" key in the logs show the version of that app.
It doesn't seem to be an issue with the ex-app itself since /init is not handled in there.
A restart of the ex-app's container fixes the issue, maybe it was called too early?

{"timestamp": "2026-02-23T20:27:25.387258+00:00", "level": "INFO", "logger": "ccb.controller", "message": "App disabled", "filename": "[controller.py](http://controller.py/)", "function": "enabled_handler", "line": 81, "thread_name": "AnyIO worker thread", "pid": 77, "version": "5.2.0"}
{"timestamp": "2026-02-23T21:12:34.004647+00:00", "level": "INFO", "logger": "uvicorn.error", "message": "Started server process [104]", "filename": "[server.py](http://server.py/)", "function": "_serve", "line": 92, "thread_name": "MainThread", "pid": 104, "version": "5.3.0", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}
{"timestamp": "2026-02-23T21:12:34.005008+00:00", "level": "INFO", "logger": "uvicorn.error", "message": "Waiting for application startup.", "filename": "[on.py](http://on.py/)", "function": "startup", "line": 48, "thread_name": "MainThread", "pid": 104, "version": "5.3.0"}
{"timestamp": "2026-02-23T21:12:34.517840+00:00", "level": "INFO", "logger": "ccb.controller", "message": "App enable state at startup: False", "filename": "[controller.py](http://controller.py/)", "function": "lifespan", "line": 91, "thread_name": "MainThread", "pid": 104, "version": "5.3.0"}
{"timestamp": "2026-02-23T21:12:34.518918+00:00", "level": "INFO", "logger": "uvicorn.error", "message": "Application startup complete.", "filename": "[on.py](http://on.py/)", "function": "startup", "line": 62, "thread_name": "MainThread", "pid": 104, "version": "5.3.0"}
{"timestamp": "2026-02-23T21:12:34.519428+00:00", "level": "INFO", "logger": "uvicorn.error", "message": "Uvicorn running on http://127.0.0.1:23001/ (Press CTRL+C to quit)", "filename": "[server.py](http://server.py/)", "function": "_log_started_message", "line": 224, "thread_name": "MainThread", "pid": 104, "version": "5.3.0", "color_message": "Uvicorn running on \u001b[1m%s://%s:%d\u001b[0m (Press CTRL+C to quit)"}
{"timestamp": "2026-02-23T21:12:45.569155+00:00", "level": "INFO", "logger": "uvicorn.error", "message": "Started server process [78]", "filename": "[server.py](http://server.py/)", "function": "_serve", "line": 92, "thread_name": "MainThread", "pid": 78, "version": "5.3.0", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}
{"timestamp": "2026-02-23T21:12:45.569406+00:00", "level": "INFO", "logger": "uvicorn.error", "message": "Waiting for application startup.", "filename": "[on.py](http://on.py/)", "function": "startup", "line": 48, "thread_name": "MainThread", "pid": 78, "version": "5.3.0"}
{"timestamp": "2026-02-23T21:12:46.034904+00:00", "level": "INFO", "logger": "ccb.controller", "message": "App enable state at startup: False", "filename": "[controller.py](http://controller.py/)", "function": "lifespan", "line": 91, "thread_name": "MainThread", "pid": 78, "version": "5.3.0"}
{"timestamp": "2026-02-23T21:12:46.036239+00:00", "level": "INFO", "logger": "uvicorn.error", "message": "Application startup complete.", "filename": "[on.py](http://on.py/)", "function": "startup", "line": 62, "thread_name": "MainThread", "pid": 78, "version": "5.3.0"}
{"timestamp": "2026-02-23T21:12:46.038094+00:00", "level": "INFO", "logger": "uvicorn.error", "message": "Uvicorn running on http://127.0.0.1:23001/ (Press CTRL+C to quit)", "filename": "[server.py](http://server.py/)", "function": "_log_started_message", "line": 224, "thread_name": "MainThread", "pid": 78, "version": "5.3.0", "color_message": "Uvicorn running on \u001b[1m%s://%s:%d\u001b[0m (Press CTRL+C to quit)"}
{"timestamp": "2026-02-23T21:12:46.114695+00:00", "level": "INFO", "logger": "uvicorn.access", "message": "127.0.0.1:45972 - \"GET /heartbeat HTTP/1.1\" 200", "filename": "h11_impl.py", "function": "send", "line": 480, "thread_name": "MainThread", "pid": 78, "version": "5.3.0"}
{"timestamp": "2026-02-23T21:12:46.156925+00:00", "level": "INFO", "logger": "uvicorn.access", "message": "127.0.0.1:45972 - \"POST /init HTTP/1.1\" 200", "filename": "h11_impl.py", "function": "send", "line": 480, "thread_name": "MainThread", "pid": 78, "version": "5.3.0"}
{"timestamp": "2026-02-23T21:12:47.830611+00:00", "level": "ERROR", "logger": "uvicorn.error", "message": "Exception in ASGI application\n", "filename": "h11_impl.py", "function": "run_asgi", "line": 415, "thread_name": "MainThread", "pid": 78, "version": "5.3.0", "exc_info": "Traceback (most recent call last):\n  File \"/usr/local/lib/python3.11/dist-packages/nc_py_api/_exceptions.py\", line 68, in check_error\n    response.raise_for_status()\n  File \"/usr/local/lib/python3.11/dist-packages/niquests/models.py\", line 1597, in raise_for_status\n    raise HTTPError(http_error_msg, response=self)\nniquests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://cloud.example.com/ocs/v1.php/apps/app_api/apps/status/context_chat_backend?format=json\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.11/dist-packages/nc_py_api/ex_app/integration_fastapi.py\", line 190, in fetch_models_task\n    models[model][\"path\"] = __fetch_model_as_file(\n                            ^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.11/dist-packages/nc_py_api/ex_app/integration_fastapi.py\", line 232, in __fetch_model_as_file\n    nc.set_init_status(min(current_progress + progress_for_task, 99))\n  File \"/usr/local/lib/python3.11/dist-packages/nc_py_api/nextcloud.py\", line 433, in set_init_status\n    self._session.ocs(\n  File \"/usr/local/lib/python3.11/dist-packages/nc_py_api/_session.py\", line 222, in ocs\n    check_error(response, info)\n  File \"/usr/local/lib/python3.11/dist-packages/nc_py_api/_exceptions.py\", line 70, in check_error\n    raise NextcloudException(status_code, reason=response.reason, info=info, response=response) from e\nnc_py_api._exceptions.NextcloudException: [401] Unauthorized <request: PUT /ocs/v1.php/apps/app_api/apps/status/context_chat_backend>\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.11/dist-packages/nc_py_api/_exceptions.py\", line 68, in check_error\n    response.raise_for_status()\n  File \"/usr/local/lib/python3.11/dist-packages/niquests/models.py\", line 1597, in raise_for_status\n    raise HTTPError(http_error_msg, response=self)\nniquests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://cloud.example.com/ocs/v1.php/apps/app_api/apps/status/context_chat_backend?format=json\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n  File \"/usr/local/lib/python3.11/dist-packages/uvicorn/protocols/http/h11_impl.py\", line 410, in run_asgi\n    result = await app(  # type: ignore[func-returns-value]\n             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.11/dist-packages/uvicorn/middleware/proxy_headers.py\", line 60, in __call__\n    return await [self.app](http://self.app/)(scope, receive, send)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.11/dist-packages/fastapi/applications.py\", line 1134, in __call__\n    await super().__call__(scope, receive, send)\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/applications.py\", line 107, in __call__\n    await self.middleware_stack(scope, receive, send)\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/middleware/errors.py\", line 186, in __call__\n    raise exc\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/middleware/errors.py\", line 164, in __call__\n    await [self.app](http://self.app/)(scope, receive, _send)\n  File \"/usr/local/lib/python3.11/dist-packages/nc_py_api/ex_app/integration_fastapi.py\", line 324, in __call__\n    await [self.app](http://self.app/)(scope, receive, send)\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/middleware/exceptions.py\", line 63, in __call__\n    await wrap_app_handling_exceptions([self.app](http://self.app/), conn)(scope, receive, send)\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/_exception_handler.py\", line 53, in wrapped_app\n    raise exc\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/_exception_handler.py\", line 42, in wrapped_app\n    await app(scope, receive, sender)\n  File \"/usr/local/lib/python3.11/dist-packages/fastapi/middleware/asyncexitstack.py\", line 18, in __call__\n    await [self.app](http://self.app/)(scope, receive, send)\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/routing.py\", line 716, in __call__\n    await self.middleware_stack(scope, receive, send)\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/routing.py\", line 736, in app\n    await route.handle(scope, receive, send)\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/routing.py\", line 290, in handle\n    await [self.app](http://self.app/)(scope, receive, send)\n  File \"/usr/local/lib/python3.11/dist-packages/fastapi/routing.py\", line 119, in app\n    await wrap_app_handling_exceptions(app, request)(scope, receive, send)\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/_exception_handler.py\", line 53, in wrapped_app\n    raise exc\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/_exception_handler.py\", line 42, in wrapped_app\n    await app(scope, receive, sender)\n  File \"/usr/local/lib/python3.11/dist-packages/fastapi/routing.py\", line 106, in app\n    await response(scope, receive, send)\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/responses.py\", line 167, in __call__\n    await self.background()\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/background.py\", line 36, in __call__\n    await task()\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/background.py\", line 23, in __call__\n    await run_in_threadpool(self.func, *self.args, **self.kwargs)\n  File \"/usr/local/lib/python3.11/dist-packages/starlette/concurrency.py\", line 32, in run_in_threadpool\n    return await [anyio.to](http://anyio.to/)_[thread.run](http://thread.run/)_sync(func)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.11/dist-packages/anyio/to_thread.py\", line 63, in run_sync\n    return await get_async_backend().run_sync_in_worker_thread(\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py\", line 2502, in run_sync_in_worker_thread\n    return await future\n           ^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py\", line 986, in run\n    result = [context.run](http://context.run/)(func, *args)\n             ^^^^^^^^^^^^^^^^^^^^^^^^\n  File \"/usr/local/lib/python3.11/dist-packages/nc_py_api/ex_app/integration_fastapi.py\", line 199, in fetch_models_task\n    nc.set_init_status(current_progress, f\"Downloading of '{model}' failed: {e}: {format_exc()}\")\n  File \"/usr/local/lib/python3.11/dist-packages/nc_py_api/nextcloud.py\", line 433, in set_init_status\n    self._session.ocs(\n  File \"/usr/local/lib/python3.11/dist-packages/nc_py_api/_session.py\", line 222, in ocs\n    check_error(response, info)\n  File \"/usr/local/lib/python3.11/dist-packages/nc_py_api/_exceptions.py\", line 70, in check_error\n    raise NextcloudException(status_code, reason=response.reason, info=info, response=response) from e\nnc_py_api._exceptions.NextcloudException: [401] Unauthorized <request: PUT /ocs/v1.php/apps/app_api/apps/status/context_chat_backend>"}
{"timestamp": "2026-02-23T21:12:55.292689+00:00", "level": "INFO", "logger": "uvicorn.access", "message": "127.0.0.1:45512 - \"GET /enabled HTTP/1.1\" 200", "filename": "h11_impl.py", "function": "send", "line": 480, "thread_name": "MainThread", "pid": 78, "version": "5.3.0"}
...

{"timestamp": "2026-02-24T08:38:58.359818+00:00", "level": "INFO", "logger": "uvicorn.access", "message": "127.0.0.1:56712 - \"GET /enabled HTTP/1.1\" 200", "filename": "h11_impl.py", "function": "send", "line": 480, "thread_name": "MainThread", "pid": 78, "version": "5.3.0"}
{"timestamp": "2026-02-24T10:26:04.638108+00:00", "level": "INFO", "logger": "uvicorn.error", "message": "Started server process [78]", "filename": "[server.py](http://server.py/)", "function": "_serve", "line": 92, "thread_name": "MainThread", "pid": 78, "version": "5.3.0", "color_message": "Started server process [\u001b[36m%d\u001b[0m]"}
{"timestamp": "2026-02-24T10:26:04.638729+00:00", "level": "INFO", "logger": "uvicorn.error", "message": "Waiting for application startup.", "filename": "[on.py](http://on.py/)", "function": "startup", "line": 48, "thread_name": "MainThread", "pid": 78, "version": "5.3.0"}
{"timestamp": "2026-02-24T10:26:05.019656+00:00", "level": "INFO", "logger": "ccb.controller", "message": "App enable state at startup: True", "filename": "[controller.py](http://controller.py/)", "function": "lifespan", "line": 91, "thread_name": "MainThread", "pid": 78, "version": "5.3.0"}

Steps/Code to Reproduce

• install a version of an ex-app
• upgrade it

Expected Results

/

Actual Results

/

Setup configuration

nextcloud = 32.0.6
app_api = 32.0.0

[kyteinsky]

#9694762

[kyteinsky]

some docker logs from another instance:

INFO:     Started server process [40]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:23004 (Press CTRL+C to quit)
INFO:     127.0.0.1:50774 - "GET /heartbeat HTTP/1.1" 200 OK
INFO:     127.0.0.1:50774 - "POST /init HTTP/1.1" 200 OK
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/nc_py_api/_exceptions.py", line 68, in check_error
    response.raise_for_status()
  File "/usr/local/lib/python3.11/dist-packages/niquests/models.py", line 1597, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
niquests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://cloud.example.com/ocs/v1.php/apps/app_api/apps/status/context_chat_backend?format=json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/nc_py_api/ex_app/integration_fastapi.py", line 190, in fetch_models_task
    models[model]["path"] = __fetch_model_as_file(
                            ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/nc_py_api/ex_app/integration_fastapi.py", line 232, in __fetch_model_as_file
    nc.set_init_status(min(current_progress + progress_for_task, 99))
  File "/usr/local/lib/python3.11/dist-packages/nc_py_api/nextcloud.py", line 433, in set_init_status
    self._session.ocs(
  File "/usr/local/lib/python3.11/dist-packages/nc_py_api/_session.py", line 222, in ocs
    check_error(response, info)
  File "/usr/local/lib/python3.11/dist-packages/nc_py_api/_exceptions.py", line 70, in check_error
    raise NextcloudException(status_code, reason=response.reason, info=info, response=response) from e
nc_py_api._exceptions.NextcloudException: [401] Unauthorized <request: PUT /ocs/v1.php/apps/app_api/apps/status/context_chat_backend>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/nc_py_api/_exceptions.py", line 68, in check_error
    response.raise_for_status()
  File "/usr/local/lib/python3.11/dist-packages/niquests/models.py", line 1597, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
niquests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://cloud.example.com/ocs/v1.php/apps/app_api/apps/status/context_chat_backend?format=json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/uvicorn/protocols/http/h11_impl.py", line 410, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/fastapi/applications.py", line 1134, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/applications.py", line 107, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.11/dist-packages/nc_py_api/ex_app/integration_fastapi.py", line 324, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/middleware/exceptions.py", line 63, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.11/dist-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.11/dist-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/routing.py", line 716, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/routing.py", line 736, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/routing.py", line 290, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/fastapi/routing.py", line 119, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.11/dist-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.11/dist-packages/fastapi/routing.py", line 106, in app
    await response(scope, receive, send)
  File "/usr/local/lib/python3.11/dist-packages/starlette/responses.py", line 167, in __call__
    await self.background()
  File "/usr/local/lib/python3.11/dist-packages/starlette/background.py", line 36, in __call__
    await task()
  File "/usr/local/lib/python3.11/dist-packages/starlette/background.py", line 23, in __call__
    await run_in_threadpool(self.func, *self.args, **self.kwargs)
  File "/usr/local/lib/python3.11/dist-packages/starlette/concurrency.py", line 32, in run_in_threadpool
    return await anyio.to_thread.run_sync(func)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 2502, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/anyio/_backends/_asyncio.py", line 986, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/nc_py_api/ex_app/integration_fastapi.py", line 199, in fetch_models_task
    nc.set_init_status(current_progress, f"Downloading of '{model}' failed: {e}: {format_exc()}")
  File "/usr/local/lib/python3.11/dist-packages/nc_py_api/nextcloud.py", line 433, in set_init_status
    self._session.ocs(
  File "/usr/local/lib/python3.11/dist-packages/nc_py_api/_session.py", line 222, in ocs
    check_error(response, info)
  File "/usr/local/lib/python3.11/dist-packages/nc_py_api/_exceptions.py", line 70, in check_error
    raise NextcloudException(status_code, reason=response.reason, info=info, response=response) from e
nc_py_api._exceptions.NextcloudException: [401] Unauthorized <request: PUT /ocs/v1.php/apps/app_api/apps/status/context_chat_backend>
Config file already exists in the persistent storage ("/nc_app_context_chat_backend_data/config.yaml").
Embedder config:
{
  "base_url": "http://localhost:5000/v1",
  "workers": 1,
  "request_timeout": 1680,
  "model_name": "em_model",
  "auth": null,
  "remote_service": false,
  "llama": {
    "model": "multilingual-e5-large-instruct-q6_k.gguf",
    "n_batch": 512,
    "n_ctx": 8192,
    "n_gpu_layers": -1,
    "logits_all": false
  }
}
INFO:     127.0.0.1:45552 - "GET /enabled HTTP/1.1" 200 OK
10/1800 secs: [Embedding server] Waiting for the main app to be enabled/ready. Current enabled state: False
INFO:     127.0.0.1:45552 - "GET /enabled HTTP/1.1" 200 OK
20/1800 secs: [Embedding server] Waiting for the main app to be enabled/ready. Current enabled state: False

[kyteinsky]

nc_py_api needs to be updated to v0.24.2 to test the fix