[SPARK-55979][SQL] Required input attributes are missing from PartialMerge / Final BaseAggregateExec.references

[zhztheplayer]

What changes were proposed in this pull request?

The patch updates BaseAggregateExec.producedAttributes, from

  override def producedAttributes: AttributeSet =
    AttributeSet(aggregateAttributes) ++
    AttributeSet(resultExpressions.diff(groupingExpressions).map(_.toAttribute)) ++
    AttributeSet(aggregateBufferAttributes) ++
    AttributeSet(inputAggBufferAttributes.filterNot(child.output.contains))

to

  override def producedAttributes: AttributeSet =
    AttributeSet(aggregateAttributes) ++
    AttributeSet(resultExpressions.diff(groupingExpressions).map(_.toAttribute)) ++
    AttributeSet(aggregateBufferAttributes) ++
    AttributeSet(inputAggBufferAttributes) -- child.outputSet

The patch fixes bug described below by adding -- child.outputSet to exclude child's attributes from producedAttributes, ensuring producedAttributes doesn't include anything from the aggregate's input.

Why are the changes needed?

The current implementation has bug causing a AggregateExec in Final mode gives fewer attributes in references than it really needs. Our downstream project has a rule to prune the unused inputs, relying on the references call. This bug causes some needed inputs to be pruned accidentally.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Tested by newly added test case.

Was this patch authored or co-authored using generative AI tooling?

The test case was co-authored by AI and reviewed by PR author.

[HyukjinKwon]

Mind keeping the PR description template?

[zhztheplayer]

@HyukjinKwon updated. I will investigate the CI error tomorrow.

[zhztheplayer]

For example, in the golden file, the attributes not produced by this operator are now correctly shown in the references section.