Release 0.6.6: autocomplete and client.events

.claude/skills/portal-app/SKILL.md

@@ -45,6 +45,8 @@ for item in res.items:
     print(item.catalog_id, item.data["name"])
 ```
 
+For an app that reads events, read each stream with `client.events.list(stream=...)`, or every stream when `streams` is missing; `user_id=` narrows it to one shopper. `client.complete(user_id=...)` ranks for a shopper from their stored events.
+
 With one id, `catalog_ids=[cat]` and `catalog_id=cat` are the same. If the app opens on no catalog and none arrives, fall back to `UNBOXAI_CATALOG_ID` or ask; don't guess an id. The API refuses a catalog the key can't read and catalogs that can't be ranked together; show its message rather than an empty result.
 
 ## Workflow

README.md

@@ -102,6 +102,34 @@ res = client.complete([View("paB08NYK61PJ")], user=shopper)
 
 If your site already sends events (the portal's browser script, or `POST /events`), pass the person's id instead of a history: `client.complete(user_id=uid, user=shopper)`. The API adds the events it has stored for that id to whatever `history` you pass. With the browser script, the id is in the `unbox_uid` cookie.
 
+From your server, record events with `client.events.track`, and read back what your account has stored with `client.events.list`, newest first:
+
+```python
+client.events.track(uid, View(shoe), user=shopper)
+client.events.track(uid, Order([shoe]), timestamp=paid_at)  # sent later than it happened
+
+client.events.list(user_id=uid, limit=20)
+client.events.list(stream="api", types=["order"])  # what your server sent; a stream is also a publishable key or an import
+```
+
+An event's catalog is `catalog_id=`, else the one an `Item` of a response names, else the client's default. Pass `timestamp=` whenever you send an event after it happened, so the history stays in order.
+
+## Search suggestions
+
+`autocomplete` suggests searches as the person types. BehaviorGPT ranks the items they most likely want given their history and the text so far, and each top item offers the first of its keywords that continues the text, allowing for a typo. So the same letters give different suggestions to different people:
+
+```python
+client.autocomplete("coffee m").texts
+# ['coffee maker', 'ninja coffee maker', 'coffee', 'keurig coffee machine', ...]
+
+client.autocomplete("gl", [Search("winter jacket")]).texts
+# ['gloves', 'mens gloves', 'winter gloves', ...]
+```
+
+Each suggestion has its `text`, the `item` it came from, and its `source`: `"keyword"`, or `"name"` for an item without keywords, which offers its name instead (`add_keywords` generates them, see [your own catalog](#your-own-catalog)). Nothing is recorded, so call it on each keystroke (after a short pause, about 150 ms, and cancel the previous call), and call `complete` with `Search(...)` once the person searches. Empty text suggests what comes after the history. It takes `catalog_id`, `catalog_ids`, `user` and `user_id` as `complete` does; `limit` is at most 20.
+
+When a few letters after earlier searches read as more of the same ("gl" after "golf clubs" ranks golf clubs), and too few of those items continue the text, the rest of the suggestions come from a second ranking of the text alone.
+
 ## Your own catalog
 
 The model can only return items it knows about. Out of the box the client uses `retail_catalog`, a sample retail catalog. To use the model on your own data, upload your catalog as a parquet file. A catalog doesn't have to be products for sale: artworks, articles, listings or anything else people browse and pick from work the same way.
@@ -120,7 +148,7 @@ To make your catalog the default, so calls need no `catalog_id`, set `UNBOXAI_CA
 
 A catalog holds up to 100,000 items, and each API key holds 2 catalogs. The file name is the catalog's name, so uploading the same name again refreshes that catalog.
 
-Once it is embedded, you can add or replace items, list and delete catalogs, and generate the keywords that `sections` uses to group results under titles such as "black coats":
+Once it is embedded, you can add or replace items, list and delete catalogs, and generate the keywords that `sections` uses to group results under titles such as "black coats", and `autocomplete` suggests:
 
 ```python
 client.add_items("new_items.parquet", catalog_id=job.catalog_id, wait=True)

src/behaviorgpt/_autocomplete.py

@@ -0,0 +1,119 @@
+"""Search suggestions read from a ranking: BehaviorGPT ranks the items the
+person is most likely to want given their history and what they have typed so
+far, and each item, best first, offers the first of its search terms that
+continues the text. So the suggestions follow the person, not only the text."""
+
+import unicodedata
+from difflib import SequenceMatcher
+from typing import Iterable, Iterator, Optional, Sequence
+
+from behaviorgpt.types import Item, Suggestion
+
+# a term matches when at most this share of the typed text has to change,
+# so "cofee m" still finds "coffee maker"
+MIN_MATCH = 0.7
+# a term this alike to an earlier suggestion is left out
+MAX_LIKENESS = 0.9
+# how many of an item's best terms may become its suggestion
+TERMS_PER_ITEM = 5
+
+
+def normalize(text: str) -> str:
+    """Lowercase, without accents, one space between words."""
+    decomposed = unicodedata.normalize("NFKD", text)
+    plain = "".join(c for c in decomposed if not unicodedata.combining(c))
+    return " ".join(plain.casefold().split())
+
+
+def match_score(typed: str, candidate: str) -> float:
+    """How well `candidate` continues `typed`: 1.0 when one of its words
+    starts with it ("kni" in "pocket knife"), less for each letter that has to
+    change, and 0.0 when nothing is close."""
+    q, c = normalize(typed), normalize(candidate)
+    if not q:
+        return 1.0
+    best = len(q)
+    for start in _word_starts(c):
+        best = min(best, _prefix_distance(q, c[start : start + len(q) + best]))
+        if best == 0:
+            break
+    return 1.0 - best / len(q)
+
+
+def suggest(
+    items: Iterable[Item],
+    typed: str,
+    limit: int,
+    earlier: Sequence[Suggestion] = (),
+) -> list[Suggestion]:
+    """`earlier` and then at most one suggestion per item, in the items' order:
+    the first of its terms that matches `typed` and is not alike to an earlier
+    one. An item without terms offers its name."""
+    found = list(earlier)
+    seen = [normalize(s.text) for s in found]
+    if len(found) >= limit:
+        return found[:limit]
+    for item in items:
+        for text, source in _candidates(item):
+            key = normalize(text)
+            if match_score(typed, text) < MIN_MATCH or _alike(key, seen):
+                continue
+            found.append(Suggestion(text=text, source=source, item=item))
+            seen.append(key)
+            break
+        if len(found) == limit:
+            break
+    return found
+
+
+def _candidates(item: Item) -> Iterator[tuple[str, str]]:
+    terms = list(_terms(item.data))[:TERMS_PER_ITEM]
+    if terms:
+        yield from ((term, "keyword") for term in terms)
+        return
+    name = item.data.get("name")
+    if isinstance(name, str) and name.strip():
+        yield name.strip(), "name"
+
+
+def _terms(data: dict) -> Iterator[str]:
+    """An item's search terms, best first: `top_terms` as the API returns
+    them, `{"term", "score"}` objects or (term, score) pairs."""
+    for entry in data.get("top_terms") or ():
+        term: Optional[object] = None
+        if isinstance(entry, dict):
+            term = entry.get("term", entry.get("text"))
+        elif isinstance(entry, (list, tuple)) and entry:
+            term = entry[0]
+        if isinstance(term, str) and term.strip():
+            yield term.strip()
+
+
+def _alike(key: str, seen: list[str]) -> bool:
+    return any(
+        key == other or SequenceMatcher(None, key, other).ratio() >= MAX_LIKENESS
+        for other in seen
+    )
+
+
+def _word_starts(text: str) -> Iterator[int]:
+    for i, char in enumerate(text):
+        if char.isalnum() and (i == 0 or not text[i - 1].isalnum()):
+            yield i
+
+
+def _prefix_distance(q: str, s: str) -> int:
+    """The fewest edits that turn `q` into the start of `s`."""
+    previous = list(range(len(s) + 1))
+    for i, qc in enumerate(q, 1):
+        current = [i]
+        for j, sc in enumerate(s, 1):
+            current.append(
+                min(
+                    previous[j] + 1,
+                    current[j - 1] + 1,
+                    previous[j - 1] + (qc != sc),
+                )
+            )
+        previous = current
+    return min(previous)