feat: restore nested cross-resource references after Create and Update

[gustavodiaz7722]

Summary

A cross-resource reference (*Ref) is generated as a sibling of the concrete field it resolves into — spec.vpcConfig.subnetRefs next to spec.vpcConfig.subnetIDs. A resource manager builds its return value from an AWS API response, which has no concept of a reference, so rebuilding the containing struct drops every *Ref inside it.

That disables ClearResolvedReferences, which suppresses a resolved value only while it can still see the sibling:

if ko.Spec.VPCConfig != nil {
	if len(ko.Spec.VPCConfig.SubnetRefs) > 0 {   // false once the *Ref is gone
		ko.Spec.VPCConfig.SubnetIDs = nil
	}
}

So the spec patch deletes the declared *Ref and stores the resolved value in its place — what aws-controllers-k8s/community#2431 reports: a declared securityGroupRefs replaced by securityGroupIDs.

Reconciliation continues until the manifest is applied again, from Helm, Argo, Flux or kubectl apply. That apply restores the *Ref beside the now-stored value, and validateReferenceFields rejects the pair:

message: Reference resolution failed
reason: 'both resource reference wrapper and ID cannot be used together:
         VPCConfig.SubnetIDs,VPCConfig.SubnetRefs'

This PR adds an optional ReferenceEnsurer interface and invokes it on the object a resource manager hands back from Create and from Update, sourcing the references from the declared resource.

Fixes aws-controllers-k8s/community#2431. Pairs with aws-controllers-k8s/code-generator#738, which generates the method.

Backwards compatible

ReferenceEnsurer is deliberately separate from ReferenceManager and reached through a type assertion, so every controller generated before the method existed still satisfies AWSResourceManager and compiles unchanged. Those controllers take the existing path untouched and opt in by regenerating. TestReconcilerUpdate_WithoutEnsurerIsUnaffected pins that.

Why these two call sites

The restoration runs where a manager returns an object whose spec is about to be patched back, not at patchResourceMetadataAndSpec, because the source has to be the declared, reference-resolved resource — and that is only in scope on these paths.

lateInitializeResource patches with the AWS-observed latest as its base, so hooked into the shared patch path the restoration would have been handed a source carrying no references at all. TestReconcilerUpdate_LateInitializeIsNotAffectedByEnsureReferences keeps that boundary asserted.

Why desired and not reconcileDesired

reconcileDesired is what gets handed to rm.Update, and a resource manager may mutate the object it is given. apigateway's ApiKey sdkUpdate assigns desired.ko.Spec.StageKeys straight from the UpdateApiKey response (sdk.go:331), so by the time Update returns it is no longer a record of what the user declared. applyIgnoredFields likewise merges observed values into it for a resource carrying the ignore-field-drift annotation. Only desired is clean, and TestReconcilerUpdate_EnsuresReferencesAfterUpdate asserts it is what gets passed.

What the generated method does

Detailed in aws-controllers-k8s/code-generator#738. Summarised here because it bounds this PR's blast radius:

• top-level *Ref — nothing emitted; it cannot be lost, since every write path starts from a DeepCopy of the object it was handed.
• reached through structs — only the reference field is assigned, so every concrete value the service reported stands. This is the whole of what is emitted, and it codifies a pattern eks/cluster, lambda/function and opensearchservice/domain already hand-maintain in sdk_*_post_set_output hooks.
• reached through a list — nothing emitted; no fixed address to assign to, and no sound way to pair an observed element with a declared one. These behave exactly as they do today.

So 117 struct-nested references across 37 resources in 25 controllers change behaviour; the 315 top-level and 38 list-nested ones are untouched.

Testing

Four tests in pkg/runtime/reconciler_test.go:

Test	Asserts
TestReconcilerUpdate_EnsuresReferencesAfterUpdate	runs once after Update, sourced from desired, and the returned object is what gets patched
TestReconcilerCreate_EnsuresReferencesAfterCreate	same on the create path
TestReconcilerUpdate_LateInitializeIsNotAffectedByEnsureReferences	never invoked on the late-init patch
TestReconcilerUpdate_WithoutEnsurerIsUnaffected	a manager not implementing the interface is untouched

Full runtime suite passes. Verified on a cluster against real generated code (lambda and ec2 regenerated against code-generator#738): a Function declaring vpcConfig.securityGroupRefs/subnetRefs keeps both in its stored spec through create and across repeated resyncs, with no resolved IDs written and no Reference resolution failed condition.

Not addressed

The read path. This runs after Create and after Update, not after ReadOne. Two spec writes consume a ReadOne-derived object and so can still delete a nested *Ref: the AdoptionPolicy_Adopt branch of Sync, and deleteResource. Both use the stored CR as the patch base and the ReadOne result as the target. Confirmed on a cluster for the adoption path, which is why the existing sdk_read_one_post_set_output hooks in the three controllers above must stay. Covering it is a separate change: the unresolved desired on those paths is a valid source, since a *Ref is user-declared and resolution only fills the concrete sibling.

The delta. It is computed against the raw ReadOne result, which this does not touch. A dropped struct-nested *Ref was invisible to the delta — the generated delta compares *Ref fields only at the top level — so it never drove a redundant Update and there is nothing left to fix for the shapes this PR covers.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[ack-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: gustavodiaz7722
Once this PR has been reviewed and has the lgtm label, please assign michaelhtm for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[gustavodiaz7722]

/retest

[ack-prow]

@gustavodiaz7722: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ec2-controller-test	c2f3d04	link	true	/test ec2-controller-test

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.