Native Parquet scan skips Spark's type check on a root field that loses its file column to a field-id match #7207

Describe the bug

With spark.sql.parquet.fieldId.read.enabled=true, the native Parquet scan never checks the requested type of a root field whose file column another requested field claims by id. Spark checks it and fails the read.

Take a file with A (id 1, BIGINT) = 7, read with the schema [x (id 1) BIGINT, a (no id) STRUCT<v BIGINT>]. Spark's clipParquetGroupFields resolves each requested field on its own, so x selects A by id, a selects A by name, and clipping the primitive A against the struct a fails with ClassCastException: optional int64 A = 1 is not a group in both Spark readers. Comet gives A to x, the stronger claim, and reads a as a missing column, so it returns [7, null].

A primitive with a type that Spark can't read the column as behaves the same way. With a INT over A BIGINT, Spark's vectorized reader fails with SchemaColumnConvertNotSupportedException once it decodes a row group, because VectorizedColumnReader.readBatch calls getUpdater for every requested leaf. parquet-mr returns NULL, and Comet returns NULL.

Inside a struct, Comet raises both errors on main.

Steps to reproduce

In a CometTestBase suite on the default Spark 4.1.3 profile:

def withId(id: Int) =
  new MetadataBuilder().putLong(ParquetUtils.FIELD_ID_METADATA_KEY, id.toLong).build()
withSQLConf(SQLConf.PARQUET_FIELD_ID_WRITE_ENABLED.key -> "true") {
  spark
    .createDataFrame(
      spark.sparkContext.parallelize(Seq(Row(7L))),
      new StructType().add("A", LongType, true, withId(1)))
    .write
    .parquet(path)
}
val readSchema = new StructType()
  .add("x", LongType, true, withId(1))
  .add("a", new StructType().add("v", LongType), true)
withSQLConf(SQLConf.PARQUET_FIELD_ID_READ_ENABLED.key -> "true") {
  spark.read.schema(readSchema).parquet(path).collect()
}

On main at 330b29c:

Expected behavior

Comet should check a losing root field against the file column it selects, as it does for a nested field: fail at file open where Spark's clipping fails, and defer a type rejection to a non-empty row group the way RejectOnNonEmpty does.

Additional context

Found while reviewing #7196, which keeps the struct check for a nested field that loses its claim. The root reads such a field as missing before any type check runs. branch-1.2 has the same code as main here.

The shape needs field-id reads and a read schema in which a field without an id has the name of a file column that another field selects by id, so it should be rare in practice.