chore: clarify sort and partition semantics in table schemas

CHANGELOG.md

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
+- chore: clarify sort and partition semantics in table schemas
 - feat(tables): add partition_by and sorted_by configuration
 - chore(query-runs): clarify user_public_id documentation
 - feat(databases): add search parameter to list endpoint

docs/InformationSchemaApi.md

@@ -12,7 +12,7 @@ Method | HTTP request | Description
 
 List tables
 
-List discovered tables with optional filtering and pagination. Supports wildcard patterns (SQL %) for schema and table name filters. Set include_columns=true to include column definitions (omitted by default).
+List discovered tables with optional filtering and pagination. Supports wildcard patterns (SQL %) for schema and table name filters. Set include_columns=true to include column definitions (omitted by default). Every table carries its declared storage layout — `partition_by` and `sorted_by` — which is fixed when the table is created and cannot be changed afterwards. Both are always present; an empty array means none was declared. Only tables in a hotdata-managed database declare a layout here, so a table discovered from an external connection always reports empty arrays.
 
 ### Example
 

docs/TableInfo.md

@@ -9,7 +9,9 @@ Name | Type | Description | Notes
 **columns** | [**List[ColumnInfo]**](ColumnInfo.md) |  | [optional] 
 **connection** | **str** |  | 
 **last_sync** | **str** |  | [optional] 
+**partition_by** | [**List[TablePartitionKey]**](TablePartitionKey.md) | The table's partition keys, in the order they were declared when the table was created. Empty when the table is not partitioned.  A table's storage layout is fixed when the table is created and cannot be changed afterwards, so this is how to confirm a table really was created with the layout that was asked for. The field is always present: an empty array means \"no partitioning declared\", which is not the same as a response that omits the field entirely.  Reported for tables in a hotdata-managed database, which are the only ones whose layout is declared here. A table discovered from an external connection always reports an empty array — its layout belongs to the upstream system, so an empty array there means \"not known from here\", not \"confirmed unpartitioned\". | 
 **var_schema** | **str** |  | 
+**sorted_by** | [**List[TableSortKey]**](TableSortKey.md) | The table's sort keys, in the order they were declared when the table was created. Empty when no sort order was declared. Always present, and limited to tables in a hotdata-managed database, for the same reasons as `partition_by`. | 
 **synced** | **bool** |  | 
 **table** | **str** |  | 
 

docs/TableSortKey.md

@@ -7,8 +7,8 @@ One key of a table's sort order.  Rows are written in this order, which keeps th
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
 **column** | **str** |  | 
-**direction** | **str** | `asc` (the default) or `desc`. | [optional] 
-**nulls** | **str** | Where nulls are placed: `first` or `last`. Defaults to the SQL default for the chosen direction. | [optional] 
+**direction** | **str** | `asc` (the default) or `desc`. Null when the table was declared without an explicit direction for this key. | [optional] 
+**nulls** | **str** | Where nulls are placed: `first` or `last`. Defaults to the SQL default for the chosen direction. Null when the table was declared without an explicit placement for this key. | [optional] 
 
 ## Example
 

hotdata/api/information_schema_api.py

@@ -63,7 +63,7 @@ def information_schema(
     ) -> InformationSchemaResponse:
         """List tables
 
-        List discovered tables with optional filtering and pagination. Supports wildcard patterns (SQL %) for schema and table name filters. Set include_columns=true to include column definitions (omitted by default).
+        List discovered tables with optional filtering and pagination. Supports wildcard patterns (SQL %) for schema and table name filters. Set include_columns=true to include column definitions (omitted by default). Every table carries its declared storage layout — `partition_by` and `sorted_by` — which is fixed when the table is created and cannot be changed afterwards. Both are always present; an empty array means none was declared. Only tables in a hotdata-managed database declare a layout here, so a table discovered from an external connection always reports empty arrays.
 
         :param connection_id: Filter by connection ID
         :type connection_id: str
@@ -151,7 +151,7 @@ def information_schema_with_http_info(
     ) -> ApiResponse[InformationSchemaResponse]:
         """List tables
 
-        List discovered tables with optional filtering and pagination. Supports wildcard patterns (SQL %) for schema and table name filters. Set include_columns=true to include column definitions (omitted by default).
+        List discovered tables with optional filtering and pagination. Supports wildcard patterns (SQL %) for schema and table name filters. Set include_columns=true to include column definitions (omitted by default). Every table carries its declared storage layout — `partition_by` and `sorted_by` — which is fixed when the table is created and cannot be changed afterwards. Both are always present; an empty array means none was declared. Only tables in a hotdata-managed database declare a layout here, so a table discovered from an external connection always reports empty arrays.
 
         :param connection_id: Filter by connection ID
         :type connection_id: str
@@ -239,7 +239,7 @@ def information_schema_without_preload_content(
     ) -> RESTResponseType:
         """List tables
 
-        List discovered tables with optional filtering and pagination. Supports wildcard patterns (SQL %) for schema and table name filters. Set include_columns=true to include column definitions (omitted by default).
+        List discovered tables with optional filtering and pagination. Supports wildcard patterns (SQL %) for schema and table name filters. Set include_columns=true to include column definitions (omitted by default). Every table carries its declared storage layout — `partition_by` and `sorted_by` — which is fixed when the table is created and cannot be changed afterwards. Both are always present; an empty array means none was declared. Only tables in a hotdata-managed database declare a layout here, so a table discovered from an external connection always reports empty arrays.
 
         :param connection_id: Filter by connection ID
         :type connection_id: str

hotdata/models/table_info.py

@@ -21,6 +21,8 @@
 from pydantic import BaseModel, ConfigDict, Field, StrictBool, StrictStr
 from typing import Any, ClassVar, Dict, List, Optional
 from hotdata.models.column_info import ColumnInfo
+from hotdata.models.table_partition_key import TablePartitionKey
+from hotdata.models.table_sort_key import TableSortKey
 from typing import Optional, Set
 from typing_extensions import Self
 
@@ -31,10 +33,12 @@ class TableInfo(BaseModel):
     columns: Optional[List[ColumnInfo]] = None
     connection: StrictStr
     last_sync: Optional[StrictStr] = None
+    partition_by: List[TablePartitionKey] = Field(description="The table's partition keys, in the order they were declared when the table was created. Empty when the table is not partitioned.  A table's storage layout is fixed when the table is created and cannot be changed afterwards, so this is how to confirm a table really was created with the layout that was asked for. The field is always present: an empty array means \"no partitioning declared\", which is not the same as a response that omits the field entirely.  Reported for tables in a hotdata-managed database, which are the only ones whose layout is declared here. A table discovered from an external connection always reports an empty array — its layout belongs to the upstream system, so an empty array there means \"not known from here\", not \"confirmed unpartitioned\".")
     var_schema: StrictStr = Field(alias="schema")
+    sorted_by: List[TableSortKey] = Field(description="The table's sort keys, in the order they were declared when the table was created. Empty when no sort order was declared. Always present, and limited to tables in a hotdata-managed database, for the same reasons as `partition_by`.")
     synced: StrictBool
     table: StrictStr
-    __properties: ClassVar[List[str]] = ["columns", "connection", "last_sync", "schema", "synced", "table"]
+    __properties: ClassVar[List[str]] = ["columns", "connection", "last_sync", "partition_by", "schema", "sorted_by", "synced", "table"]
 
     model_config = ConfigDict(
         populate_by_name=True,
@@ -82,6 +86,20 @@ def to_dict(self) -> Dict[str, Any]:
                 if _item_columns:
                     _items.append(_item_columns.to_dict())
             _dict['columns'] = _items
+        # override the default output from pydantic by calling `to_dict()` of each item in partition_by (list)
+        _items = []
+        if self.partition_by:
+            for _item_partition_by in self.partition_by:
+                if _item_partition_by:
+                    _items.append(_item_partition_by.to_dict())
+            _dict['partition_by'] = _items
+        # override the default output from pydantic by calling `to_dict()` of each item in sorted_by (list)
+        _items = []
+        if self.sorted_by:
+            for _item_sorted_by in self.sorted_by:
+                if _item_sorted_by:
+                    _items.append(_item_sorted_by.to_dict())
+            _dict['sorted_by'] = _items
         # set to None if columns (nullable) is None
         # and model_fields_set contains the field
         if self.columns is None and "columns" in self.model_fields_set:
@@ -107,7 +125,9 @@ def from_dict(cls, obj: Optional[Dict[str, Any]]) -> Optional[Self]:
             "columns": [ColumnInfo.from_dict(_item) for _item in obj["columns"]] if obj.get("columns") is not None else None,
             "connection": obj.get("connection"),
             "last_sync": obj.get("last_sync"),
+            "partition_by": [TablePartitionKey.from_dict(_item) for _item in obj["partition_by"]] if obj.get("partition_by") is not None else None,
             "schema": obj.get("schema"),
+            "sorted_by": [TableSortKey.from_dict(_item) for _item in obj["sorted_by"]] if obj.get("sorted_by") is not None else None,
             "synced": obj.get("synced"),
             "table": obj.get("table")
         })

hotdata/models/table_sort_key.py

@@ -28,8 +28,8 @@ class TableSortKey(BaseModel):
     One key of a table's sort order.  Rows are written in this order, which keeps the values in each file within a narrow range and lets queries filtering on those columns skip files entirely. Most useful on columns you filter by ranges, such as a timestamp.
     """ # noqa: E501
     column: StrictStr
-    direction: Optional[StrictStr] = Field(default=None, description="`asc` (the default) or `desc`.")
-    nulls: Optional[StrictStr] = Field(default=None, description="Where nulls are placed: `first` or `last`. Defaults to the SQL default for the chosen direction.")
+    direction: Optional[StrictStr] = Field(default=None, description="`asc` (the default) or `desc`. Null when the table was declared without an explicit direction for this key.")
+    nulls: Optional[StrictStr] = Field(default=None, description="Where nulls are placed: `first` or `last`. Defaults to the SQL default for the chosen direction. Null when the table was declared without an explicit placement for this key.")
     __properties: ClassVar[List[str]] = ["column", "direction", "nulls"]
 
     model_config = ConfigDict(

test/test_information_schema_response.py

@@ -50,7 +50,18 @@ def make_instance(self, include_optional) -> InformationSchemaResponse:
                             ], 
                         connection = '', 
                         last_sync = '', 
+                        partition_by = [
+                            hotdata.models.table_partition_key.TablePartitionKey(
+                                column = '', 
+                                transform = '', )
+                            ], 
                         schema = '', 
+                        sorted_by = [
+                            hotdata.models.table_sort_key.TableSortKey(
+                                column = '', 
+                                direction = '', 
+                                nulls = '', )
+                            ], 
                         synced = True, 
                         table = '', )
                     ]
@@ -70,7 +81,18 @@ def make_instance(self, include_optional) -> InformationSchemaResponse:
                             ], 
                         connection = '', 
                         last_sync = '', 
+                        partition_by = [
+                            hotdata.models.table_partition_key.TablePartitionKey(
+                                column = '', 
+                                transform = '', )
+                            ], 
                         schema = '', 
+                        sorted_by = [
+                            hotdata.models.table_sort_key.TableSortKey(
+                                column = '', 
+                                direction = '', 
+                                nulls = '', )
+                            ], 
                         synced = True, 
                         table = '', )
                     ],

test/test_table_info.py

@@ -44,14 +44,36 @@ def make_instance(self, include_optional) -> TableInfo:
                     ],
                 connection = '',
                 last_sync = '',
+                partition_by = [
+                    hotdata.models.table_partition_key.TablePartitionKey(
+                        column = '', 
+                        transform = '', )
+                    ],
                 var_schema = '',
+                sorted_by = [
+                    hotdata.models.table_sort_key.TableSortKey(
+                        column = '', 
+                        direction = '', 
+                        nulls = '', )
+                    ],
                 synced = True,
                 table = ''
             )
         else:
             return TableInfo(
                 connection = '',
+                partition_by = [
+                    hotdata.models.table_partition_key.TablePartitionKey(
+                        column = '', 
+                        transform = '', )
+                    ],
                 var_schema = '',
+                sorted_by = [
+                    hotdata.models.table_sort_key.TableSortKey(
+                        column = '', 
+                        direction = '', 
+                        nulls = '', )
+                    ],
                 synced = True,
                 table = '',
         )