feat: generate EnsureReferences to preserve nested cross-resource references

[gustavodiaz7722]

Summary

A cross-resource reference (*Ref) is generated as a sibling of the concrete field it resolves into — spec.vpcConfig.subnetRefs next to spec.vpcConfig.subnetIDs. A resource manager builds its return value from an AWS API response, which has no concept of a reference, so rebuilding the containing struct drops every *Ref inside it.

That disables ClearResolvedReferences, which suppresses a resolved value only while it can still see the sibling:

if ko.Spec.VPCConfig != nil {
	if len(ko.Spec.VPCConfig.SubnetRefs) > 0 {   // false once the *Ref is gone
		ko.Spec.VPCConfig.SubnetIDs = nil
	}
}

So the spec patch deletes the declared *Ref and stores the resolved value in its place — what aws-controllers-k8s/community#2431 reports: a declared securityGroupRefs replaced by securityGroupIDs.

Reconciliation continues until the manifest is applied again, from Helm, Argo, Flux or kubectl apply. That apply restores the *Ref beside the now-stored value, and validateReferenceFields rejects the pair:

message: Reference resolution failed
reason: 'both resource reference wrapper and ID cannot be used together:
         VPCConfig.SubnetIDs,VPCConfig.SubnetRefs'

This PR generates an EnsureReferences method that restores the missing reference from the declared resource.

Fixes aws-controllers-k8s/community#2431. Addresses the struct-nested half of aws-controllers-k8s/community#2361.

Requires the runtime change

aws-controllers-k8s/runtime#267 defines the optional ReferenceEnsurer interface and calls the method after Create and after Update. Without it this method is inert but harmless. Controllers generated before it existed are unaffected and opt in by regenerating.

What is emitted

Reference	Emitted	Why
top-level (spec.xRef)	nothing	every write path starts from a DeepCopy, and the *Ref is a sibling of the concrete field, so nothing rebuilds it
through structs (spec.a.xRef, spec.a.xRefs)	assign the reference	it has one fixed address; every value the service reported stands
through a list (spec.l[].xRef)	nothing	it has no fixed address; out of scope, see below

A reference field that is itself a list (*Refs, whose concrete sibling is a list of scalars) belongs in the struct row: the list is the leaf, not part of the path, so nothing has to be indexed to reach it.

if desiredKO.Spec.VPCConfig != nil && latestKO.Spec.VPCConfig != nil &&
	len(desiredKO.Spec.VPCConfig.SubnetRefs) > 0 && len(latestKO.Spec.VPCConfig.SubnetRefs) == 0 {
	latestKO.Spec.VPCConfig.SubnetRefs = desiredKO.Spec.VPCConfig.SubnetRefs
}

Only the reference is written, so nothing the service populated is touched.

This codifies an existing pattern

Restoring a nested *Ref from the declared resource is not new — three controllers already hand-maintain this assignment for want of a generated equivalent. eks/cluster:

// templates/hooks/cluster/sdk_create_post_set_output.go.tpl
if desired.ko.Spec.ResourcesVPCConfig.SubnetRefs != nil {
	ko.Spec.ResourcesVPCConfig.SubnetRefs = desired.ko.Spec.ResourcesVPCConfig.SubnetRefs
}
if desired.ko.Spec.ResourcesVPCConfig.SecurityGroupRefs != nil {
	ko.Spec.ResourcesVPCConfig.SecurityGroupRefs = desired.ko.Spec.ResourcesVPCConfig.SecurityGroupRefs
}

lambda/function does the same for VPCConfig, and opensearchservice/domain for VPCOptions — the latter with a comment naming this issue directly:

// To prevent https://github.com/aws-controllers-k8s/community/issues/2431

The generated code is the same assignment with stricter guards: it nil-checks the container on both objects and only writes when the target is actually missing the reference, so it cannot clobber a reference the service did report. What changes is that every controller with struct-nested references gets the behaviour without hand-writing it.

Scope

Classifying each reference by the shape of the path to its *Ref, across the controllers with a generated references.go:

Shape	References	Resources	Controllers	This PR
top-level	315	157	53	not needed
struct-nested	117	37	25	fixed
list-nested	38	21	12	unchanged

Testing

Eight unit tests in pkg/generate/code/resource_reference_test.go: top-level emits nothing, struct-nested single ref, struct-nested list-of-refs, list path emits nothing, a resource mixing struct- and list-nested, indent level, and the two rejection paths (a reference within a map, and a model missing an ancestor field).

Regenerated ec2-controller and lambda-controller; references.go diffs are purely additive, output is gofmt-clean, and both build in full. lambda/function restores Code.S3BucketRef, VPCConfig.SecurityGroupRefs and VPCConfig.SubnetRefs — the shape #2431 was filed for. Verified on a cluster that a Function declaring those refs keeps them through create and across repeated resyncs, with no resolved IDs written to the spec.

Not addressed

References reached through a list. No fixed address to assign to, and no sound way to pair an observed element with a declared one: an AWS response need not preserve request order. These behave exactly as they do today.

The read path. EnsureReferences runs after Create and after Update, not after ReadOne. The AdoptionPolicy_Adopt branch of Sync and deleteResource both patch the spec from a ReadOne-derived object and so can still delete a nested *Ref; verified on a cluster for the adoption path. The sdk_read_one_post_set_output hooks in lambda/function, eks/cluster and opensearchservice/domain must therefore stay — only their create-path halves are subsumed.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[ack-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: gustavodiaz7722
Once this PR has been reviewed and has the lgtm label, please assign jlbutler for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[gustavodiaz7722]

/retest

[gustavodiaz7722]

/retest

[ack-prow]

@gustavodiaz7722: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
acm-controller-test	8ccc75d	link	true	/test acm-controller-test
ec2-controller-test	8ccc75d	link	true	/test ec2-controller-test

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.