Disable replication on Close

[samliok]

Making this change means that if we call Close on replication state, we don't need to worry about calling AdvanceTime or frankly any of the other methods on it.

[yacovm]

I don't understand what goes wrong without this change.

Is there any scenario that can be demonstrated that fails or gets stuck without this change?

[samliok]

I don't understand what goes wrong without this change.

Is there any scenario that can be demonstrated that fails or gets stuck without this change?

I was concerned that our shutdown for non-validator and validator could break depending on the ordering of the Close calls. Haven't written a test to show it breaking, but the potential issue is what if we close replication and then the epoch/non-validator is still processing replication messages. It will be trying to update the state of a closed struct, so maybe this would cause an ungraceful shutdown.

func (n *NonValidator) Stop() {
	n.Logger.Info("Shutting down non-validator", zap.Stringer("ID", n.ID))
	n.cancelCtx()
	n.sequenceReplicator.Close()
	n.verifier.Close()
}

func (e *Epoch) Stop() {
	e.Logger.Info("Shutting down node")
	e.finishFn()
	e.monitor.Close()
	e.blockVerificationScheduler.Close()
	e.buildBlockScheduler.Close()
	e.timeoutHandler.Close()
	e.replicationState.Close()
}