Skip to content

fix: validate embeddings against fitted dimensions (closes #34) - #45

Open
botbikamordehai2-sketch wants to merge 1 commit into
OpenTabular:mainfrom
botbikamordehai2-sketch:fix/issue-34-1786015735
Open

fix: validate embeddings against fitted dimensions (closes #34)#45
botbikamordehai2-sketch wants to merge 1 commit into
OpenTabular:mainfrom
botbikamordehai2-sketch:fix/issue-34-1786015735

Conversation

@botbikamordehai2-sketch

Copy link
Copy Markdown

What

fit() records embedding_dimensions_ for every embedding array passed, but transform() never checks anything against it. As a result, mismatched embeddings are accepted silently, allowing:

  • Wrong embedding width → result blocks have incompatible feature dimensions
  • Wrong number of rows → blocks have differing heights, causing shape errors or silent row misalignment downstream
  • Missing/extra embeddings → blocks are silently dropped or unexpected blocks appear

This leads to confusing failures far from the cause or, worse, incorrect computations.

Fix

Added _validate_embeddings() that checks, for each embedding passed to transform():

  • The set of embedding keys/indices matches those recorded during fit
  • Each embedding is a 2D array with a width equal to the fitted embedding_dimensions_
  • Each embedding has a row count equal to the number of samples in X

If any check fails, an IncompatibleParamsError is raised with a clear message naming the specific mismatch.

Closes #34

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(preprocessor): validate embeddings against the fitted dimensions

1 participant