feat(avro): apply column default values when reading missing fields

[huan233usc]

What

Part 3 of 4 of Iceberg v3 column default-value support (POC #731), built on the
schema layer (#746) and the Parquet read path (#792).

When a column is present in the read (table) schema but absent from an Avro data
file — because the column was added after those rows were written — fill it with
the column's v3 initial-default instead of null.

Changes

• Avro projection (avro_schema_util.cc): when a field is missing from the
  file and carries an initial-default, project it as
  FieldProjection::Kind::kDefault, mirroring the generic / Parquet paths.
• Avro decode (avro_data_util.cc, avro_direct_decoder.cc): materialize
  the kDefault branch through an Avro-local AppendDefaultToBuilder helper.
  It reuses the shared ToArrowScalar conversion, while keeping Avro's
  row-by-row ArrayBuilder append behavior out of the shared Arrow utility.

Tests

• avro_data_test: AppendDefaultToBuilder appends a value and casts to the
  builder type; AppendDatumToBuilder fills missing required and optional
  default fields.
• avro_test: end-to-end — write an Avro file with an old schema, then read it
  through ReaderFactoryRegistry with an evolved schema carrying defaults
  (ReadMissingFieldsWithDefaults).

Stack

1. feat(schema): represent, serialize and validate v3 column default values (1/4) #746 — schema: represent / serialize / validate (merged)
2. feat(parquet): apply column default values when reading missing fields (2/4) #792 — read path: Parquet (merged)
3. this PR — read path: Avro
4. schema evolution: addColumn / updateColumnDefault (feat: support v3 column default values in UpdateSchema (3/4) #793)

Draft while the earlier PRs in the stack settle.

[Copilot]

Pull request overview

Adds Avro read-path support for Iceberg v3 initial-default column defaults, ensuring that when projected (table) schema fields are missing from older Avro files, readers materialize the specified defaults instead of null.

Changes:

• Avro schema projection: project missing fields with initial-default as FieldProjection::Kind::kDefault.
• Avro decode paths (generic + direct): materialize kDefault by appending the literal default into Arrow builders via a new Avro-local AppendDefaultToBuilder.
• Tests: add unit coverage for default appends and an end-to-end Avro read test exercising schema evolution with defaults.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/iceberg/avro/avro_schema_util.cc	Projects missing fields with initial-default as kDefault.
src/iceberg/avro/avro_data_util.cc	Implements and uses AppendDefaultToBuilder to materialize kDefault during generic Avro decoding.
src/iceberg/avro/avro_direct_decoder.cc	Materializes kDefault during direct decoding into Arrow builders.
src/iceberg/avro/avro_data_util_internal.h	Exposes AppendDefaultToBuilder for reuse across Avro decode implementations.
src/iceberg/test/avro_data_test.cc	Adds unit tests covering default materialization behavior.
src/iceberg/test/avro_test.cc	Adds end-to-end test for reading missing fields with defaults using an evolved schema.

[wgtmac]

Could this follow the ParquetExtraAttributes pattern and become AvroExtraAttributes with a default_scalar member? FieldProjection has a single attributes slot, so separate subclasses for future Avro attributes would not compose. A single Avro-specific container would also let the append path use checked_cast instead of a per-row dynamic_cast.

[huan233usc]

Done — renamed to AvroExtraAttributes with a default_scalar member, mirroring ParquetExtraAttributes. Since Avro now has a single attributes container, the append path uses checked_cast instead of a per-row dynamic_cast, and future Avro attributes can be added to the same struct rather than as competing subclasses.