chore: Use native impl of elt function

[kazantsev-maksim]

Which issue does this PR close?

• N/A

Rationale for this change

This PR migrates the elt string function from Comet's codegen-based dispatch path to the shared datafusion-spark implementation, registering it as a native scalar UDF.

What changes are included in this PR?

• Register the SparkElt scalar UDF in register_datafusion_spark_function so it is available to the DataFusion session context alongside the other Spark-compatible UDFs.
• Change CometElt from CometCodegenDispatch[Elt] to CometScalarFunction[Elt]("elt")

How are these changes tested?

Significantly expand the test coverage with many additional inputs to exercise edge cases

[comphead]

Thanks @kazantsev-maksim

CI pending

[kazantsev-maksim]

Thanks @comphead!

[andygrove]

I don't think the upstream elt implementation is Spark-compatible? It doesn't implement ANSI mode, for example.

From DF 54.1.0:

        // TODO: if spark.sql.ansi.enabled is true,
        //  throw ArrayIndexOutOfBoundsException for invalid indices;
        //  if false, return NULL instead (current behavior).
        if index < 1 || (index as usize) > num_values {
            builder.append_null();
            continue;
        }

[andygrove]

I don't think the upstream elt implementation is Spark-compatible? It doesn't implement ANSI mode, for example.

From DF 54.1.0:

        // TODO: if spark.sql.ansi.enabled is true,
        //  throw ArrayIndexOutOfBoundsException for invalid indices;
        //  if false, return NULL instead (current behavior).
        if index < 1 || (index as usize) > num_values {
            builder.append_null();
            continue;
        }

One option would be to mix in CodegenDispatchFallback and return Incompatible when expr.failOnError is true. It would be good to file an issue upstream for implementing ANSI support too.

[comphead]

I don't think the upstream elt implementation is Spark-compatible? It doesn't implement ANSI mode, for example.
From DF 54.1.0:

        // TODO: if spark.sql.ansi.enabled is true,
        //  throw ArrayIndexOutOfBoundsException for invalid indices;
        //  if false, return NULL instead (current behavior).
        if index < 1 || (index as usize) > num_values {
            builder.append_null();
            continue;
        }

One option would be to mix in CodegenDispatchFallback and return Incompatible when expr.failOnError is true. It would be good to file an issue upstream for implementing ANSI support too.

Thanks @andygrove for catching this. for ANSI we should start an epic there. The ANSI flag is already there, and we can try it on elt

[comphead]

Created apache/datafusion#23928