[SPARK-58114][PYTHON] Consolidate VALUE_NOT_ANY_OR_ALL into VALUE_NOT_ALLOWED

[marcuslin123]

What changes were proposed in this pull request?

Remove the specialized VALUE_NOT_ANY_OR_ALL PySpark error condition and replace its single use site (DataFrame.dropna's how validation, in both classic and Spark Connect) with the more general VALUE_NOT_ALLOWED condition, which already exists and expresses the same thing via an allowed_values parameter.

Before:

VALUE_NOT_ANY_OR_ALL: "Value for `<arg_name>` must be 'any' or 'all', got '<arg_value>'."

After (reusing the existing general condition):

VALUE_NOT_ALLOWED: "Value for `<arg_name>` has to be amongst the following values: <allowed_values>."

Why are the changes needed?

VALUE_NOT_ANY_OR_ALL is a narrow, single-purpose error condition that duplicates what VALUE_NOT_ALLOWED already provides generically. Consolidating reduces the number of error conditions to maintain and keeps dropna's error consistent with other "value must be one of a fixed set" validations across PySpark (e.g. between, profiler options).

Does this PR introduce any user-facing change?

Yes, a minor change to the error message for df.dropna(how=<invalid>). Previously: "Value for how must be 'any' or 'all', got 'foo'." Now: "Value for how has to be amongst the following values: ['any', 'all']." The raised exception type (PySparkValueError) is unchanged.

How was this patch tested?

Updated the existing regression test in test_stat.py to assert the VALUE_NOT_ALLOWED condition and its parameters. Test passes:
python/run-tests --testnames "pyspark.sql.tests.test_stat DataFrameStatTests.test_dropna"

Was this patch authored or co-authored using generative AI tooling?

Generative AI tooling (Claude Code) was used as an assistive tool for implementation guidance.

[uros-b]

Is this superseded by #57236? @marcuslin123 @HyukjinKwon, cc @zhengruifeng @dongjoon-hyun

[uros-b]

Is this superseded by #57236?

If so, let's please consider closing the current PR.