Skip to content

request() drops query strings — yarl.URL.build(path=...) doesn't parse them, which blocks traversal of /json/palx data #2097

Description

@kmix27

WLED.request() builds the URL like this:

url = URL.build(scheme="http", host=self.host, port=80, path=uri)

If you try to send a query string viauri (e.g. /json/palx?page=1), yarl doesn't split that into path + query — it treats the whole thing as the path and percent-encodes the ?. So the attempted query string never actually makes it onto the wire as a query string:

>>> from yarl import URL
>>> url = URL.build(scheme="http", host="192.168.1.50", port=80, path="/json/palx?page=1")
>>> str(url)
'http://192.168.1.50/json/palx%3Fpage=1'
>>> url.query_string
''

Presented with such a request WLED's firmware doesn't parse a page param, it instead falls back to its default (page=0) every single time. Attempting to walk /json/palx page by page gives you nine+ responses of page=0.

Why /json/palx is useful and justifies a fix

/json/palx is barely documented, but it's the only way to get the raw gradient/color-stop data for palettes — including both default and custom. The regular /json palettes key only gives you built-in palette names, not their actual color data. If you want to do anything programmatic with palette colors (preview them, match one to a photo, harmonize between them, whatever), /json/palx is the only endpoint that has the data to enable that, and it's paginated (5 items/page on ESP8266, 8 elsewhere), so you have to walk pages to get the full set. Which is blocked today in python-wled by this bug.

I checked the firmware source and confirmed /json/palx is the only WLED JSON endpoint that paginates via a query param — everything else returns its full payload in one shot, so this fix only covers this one case at present, but it's inclusion would cover any future paginated endpoints that find their way into WLED firmware.

Proposed fix

Give request() its own query or query_string param defaulting to empty, and pass it to URL.build() separately:

async def request(
    self,
    uri: str = "",
    method: str = "GET",
    data: dict[str, Any] | None = None,
    query_string: str = "",
    # and/or 
    query: dict = {},

) -> Any:
    ...
    url = URL.build(
        scheme="http",
        host=self.host,
        port=80,
        path=uri,
        query_string=query_string,
        query=query
    )

Confirmed this actually works:

>>> url = URL.build(scheme="http", host="1.2.3.4", port=80, path="/json/palx", query_string="", query={} )
>>> str(url)
'http://1.2.3.4/json/palx'
>>> url.query_string
''
>>> qs_url = URL.build(scheme="http", host="1.2.3.4", port=80, path="/json/palx", query_string="page=5")
>>> q_url =  URL.build(scheme="http", host="1.2.3.4", port=80, path="/json/palx", query={"page":5})
>>> qs_url == q_url
True
>>> str(qs_url)
'http://1.2.3.4/json/palx?page=5'
>>> q_url.query_string
'page=5'

Fully backwards compatible — query_string defaults '', query defaults {} (None would throw) doesn't touch any existing call site, but makes /json/palx traversable through python-wled.

Happy to send a PR for this, just wanted to write up the root cause first in case there's a preference between (query_string: str vs. query: dict)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions