Skip to content

bug: Literal comparison orders -NaN as greater than +NaN, contradicting the documented total ordering #860

Description

@LuciferYang

Summary

Literal::operator<=> orders a negative NaN as greater than a positive NaN, the opposite of the total ordering the code documents and tests elsewhere. The helper CompareFloat (src/iceberg/expression/literal.cc) returns lhs_is_negative <=> rhs_is_negative for the both-NaN case, so -NaN <=> +NaN evaluates to true <=> false = greater. The adjacent comment states "-NAN < NAN", and FloatSpecialValuesComparison / DoubleSpecialValuesComparison assert the full ordering -NaN < -Infinity < ... < +Infinity < +NaN. The both-NaN sign branch contradicts both.

Root Cause

bool lhs_is_negative = std::signbit(lhs);
bool rhs_is_negative = std::signbit(rhs);
return lhs_is_negative <=> rhs_is_negative;

For lhs = -NaN (signbit = true) and rhs = +NaN (signbit = false), this is true <=> false = std::strong_ordering::greater, i.e. it reports -NaN > +NaN. A negative sign bit should sort below a positive one, so the operands must be swapped: rhs_is_negative <=> lhs_is_negative.

The existing FloatNaNComparison / DoubleNaNComparison tests only cover same-sign NaN pairs (qNaN vs sNaN, which are equivalent), so the mixed-sign case was never exercised.

Reproduction

auto neg_nan = Literal::Float(-std::numeric_limits<float>::quiet_NaN());
auto pos_nan = Literal::Float(std::numeric_limits<float>::quiet_NaN());

// Expected per the documented ordering: less
// Actual: greater
(neg_nan <=> pos_nan);

Proposed Fix

Swap the operands so a negative sign bit compares as less, consistent with the -NaN < +NaN comment and the special-values ordering tests:

return rhs_is_negative <=> lhs_is_negative;

I have the fix and a mixed-sign NaN regression test (float and double) ready and will open a PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions