feat: probe without TLS certificate verification by default

CHANGELOG.md

@@ -11,6 +11,39 @@ Version numbers follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html
 
 ---
 
+## [0.4.0] — 2026-08-18
+
+### Added
+
+- `HeadersReport.tls_verified` field (default `True`): `False` when the final
+  response was served over HTTPS without certificate verification (the
+  default probe mode). Exposed in the `--json` output and rendered as a
+  warning banner in the terminal report and in file exports. The flag is
+  derived from the final post-redirect URL, so an unverified `http://`
+  request that lands on `https://` is flagged and a probe that ends on
+  plain HTTP is not.
+- CLI: `--tls-verify` flag to opt into strict certificate verification
+  (TLS errors then abort with exit code 2, as before). `--no-tls-verify`
+  remains valid and states the default explicitly.
+
+### Changed
+
+- The header probe no longer verifies TLS certificates by default
+  (`assess(..., verify_tls=False)`): no extra request is ever spent on a
+  certificate verification attempt, and self-signed, expired, or wrong-host
+  certificates cannot block header assessment (previously any certificate
+  failure aborted with exit code 2). The pre-existing `http://` fallback on
+  a refused HTTPS connection and the HEAD→GET retry on 405 are unchanged.
+  Certificate posture is tlsvalidator's domain; the grade is unaffected and
+  the report carries `tls_verified=False` instead. With `verify_tls=True`
+  TLS errors still abort and are never silently retried, and TLS errors
+  never trigger the `http://` fallback.
+- `fetch_headers()` suppresses urllib3's `InsecureRequestWarning` for
+  unverified fetches — the unverified channel is already reported explicitly,
+  so the per-request stderr warning added only noise.
+
+---
+
 ## [0.3.2] — 2026-08-12
 
 ### Removed
@@ -310,7 +343,8 @@ Version numbers follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html
 
 ---
 
-[Unreleased]: https://github.com/NC3-TestingPlatform/headersvalidator/compare/v0.3.2...HEAD
+[Unreleased]: https://github.com/NC3-TestingPlatform/headersvalidator/compare/v0.4.0...HEAD
+[0.4.0]: https://github.com/NC3-TestingPlatform/headersvalidator/compare/v0.3.2...v0.4.0
 [0.3.2]: https://github.com/NC3-TestingPlatform/headersvalidator/compare/v0.3.1...v0.3.2
 [0.3.1]: https://github.com/NC3-TestingPlatform/headersvalidator/compare/v0.3.0...v0.3.1
 [0.3.0]: https://github.com/NC3-TestingPlatform/headersvalidator/compare/v0.2.3...v0.3.0

README.md

@@ -11,7 +11,7 @@ $ headersvalidator check example.com
 ```
 
 ![Python](https://img.shields.io/badge/python-%3E%3D3.11-blue)
-![Tests](https://img.shields.io/badge/tests-421%20passing-brightgreen)
+![Tests](https://img.shields.io/badge/tests-435%20passing-brightgreen)
 ![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen)
 ![License](https://img.shields.io/badge/license-GPLv3-lightgrey)
 
@@ -109,8 +109,12 @@ headersvalidator check https://example.com
 # Adjust request timeout (seconds, default 10)
 headersvalidator check example.com --timeout 15
 
-# Skip TLS certificate verification (internal hosts)
-headersvalidator check https://intranet.local --no-tls-verify
+# The probe does not verify TLS certificates by default, so self-signed or
+# invalid certificates cannot block the assessment and no extra request is
+# spent on a verification attempt (the existing HTTP fallback when port 443
+# is closed and the HEAD→GET 405 retry are unchanged); the report is
+# flagged. Opt into strict verification with:
+headersvalidator check https://example.com --tls-verify
 
 # Output as JSON (for CI/CD pipelines)
 headersvalidator check example.com --json
@@ -151,7 +155,7 @@ from headersvalidator.reporter import print_full_report
 report = assess(
     "example.com",
     timeout=10.0,      # optional: request timeout in seconds
-    verify_tls=True,   # optional: set False to skip TLS verification
+    verify_tls=False,  # optional: default False — set True for strict verification
 )
 
 print_full_report(report)
@@ -171,6 +175,8 @@ print(report.score)         # 0–100
 print(report.url)           # "https://example.com"
 print(report.final_url)     # effective URL after redirects
 print(report.status_code)   # HTTP status code
+print(report.tls_verified)  # False = final HTTPS response fetched without
+                            # certificate verification (the default probe mode)
 
 # Iterate over all results
 for result in report.results:
@@ -302,7 +308,7 @@ pytest tests/test_checker.py
 pytest tests/test_checker.py::TestHSTS -v
 ```
 
-The test suite has **421 tests** and maintains **100% statement coverage**.
+The test suite has **435 tests** and maintains **100% statement coverage**.
 
 All HTTP network I/O (`requests.head`, `requests.get`) is mocked at the
 `fetch_headers` boundary — no test touches a real server or the internet.

docs/SECURITY_VERDICT.md

@@ -42,6 +42,16 @@ Total penalty is mapped to a letter grade:
 | 41 – 60        | **D**  | Poor — significant security exposure |
 | > 60           | **F**  | Critical — immediate remediation required |
 
+> **TLS certificate posture does not affect the grade.** The probe does not
+> verify certificates by default (no request is spent on a verification
+> attempt and self-signed or invalid certificates cannot block the
+> assessment; the pre-existing HTTP fallback on a refused HTTPS connection
+> is unchanged); the report is flagged
+> instead (`tls_verified: false`, plus a warning banner). Certificate posture
+> is assessed and graded by **tlsvalidator**, not here — this module grades
+> only the headers themselves, exactly as it already does for targets served
+> over plain HTTP.
+
 ---
 
 ## Severity levels explained

headersvalidator/__init__.py

@@ -5,7 +5,7 @@
 try:
     __version__ = version("headersvalidator")
 except PackageNotFoundError:  # pragma: no cover – only when package not installed
-    __version__ = "0.3.2"
+    __version__ = "0.4.0"
 
 # NullHandler so library users who have not configured logging
 # do not see "No handler found" warnings (PEP 3118 / logging HOWTO).

headersvalidator/assessor.py

@@ -36,7 +36,7 @@ def assess(
     url: str,
     *,
     timeout: float = HTTP_TIMEOUT,
-    verify_tls: bool = True,
+    verify_tls: bool = False,
     user_agent: str | None = None,
     progress_cb: Callable[[str], None] | None = None,
 ) -> HeadersReport:
@@ -50,10 +50,15 @@ def assess(
     :param url: Target URL.  Scheme is optional; ``https://`` is assumed.
         If the HTTPS connection is refused (port 443 closed), the function
         automatically retries over ``http://`` so that plain-HTTP→HTTPS
-        redirect chains are followed.  TLS errors are never silently retried.
+        redirect chains are followed.
     :param timeout: Per-request socket timeout in seconds.
-    :param verify_tls: If ``False``, TLS certificate errors are ignored
-        (useful for internal or self-signed hosts).
+    :param verify_tls: Certificate verification is **off by default**: one
+        probe per target, and a self-signed or otherwise invalid certificate
+        cannot block header assessment (certificate posture is tlsvalidator's
+        domain).  The report is flagged with ``tls_verified=False`` whenever
+        the final response was served over HTTPS without verification.
+        Pass ``True`` for strict verification — TLS errors then abort the
+        assessment and are never silently retried.
     :param user_agent: Override the default headersvalidator User-Agent string.
     :param progress_cb: Optional callable invoked with a short status string
         at key milestones during the assessment (for CLI spinner integration).
@@ -69,16 +74,23 @@ def assess(
         progress_cb(f"Fetching headers for {url} …")
 
     # ---- Network I/O (single point — easy to mock in tests) ----------
-    # If https:// fails with a connection error (port closed, not an TLS error),
+    # The probe does not verify TLS certificates by default: one request per
+    # target, and a self-signed or otherwise invalid certificate cannot block
+    # header assessment. The module already grades headers fetched over plain
+    # HTTP, so an unverified TLS channel is no weaker than what is accepted
+    # elsewhere, and certificate posture belongs to tlsvalidator.
+    # If https:// fails with a connection error (port closed, not a TLS error),
     # retry with http:// so that plain-HTTP→HTTPS redirect chains are followed.
+    unverified = not verify_tls  # the successful fetch ran without verification
     try:
         response = fetch_headers(
             url, timeout=timeout, verify_tls=verify_tls, user_agent=user_agent
         )
     except requests.exceptions.ConnectionError as exc:
-        # SSLError is a subclass of ConnectionError — do NOT fall back for TLS
-        # errors; those indicate an active connection that failed at the TLS
-        # layer and should be surfaced as-is.
+        # SSLError (only possible with verify_tls=True) is a subclass of
+        # ConnectionError — do NOT fall back for TLS errors: the caller asked
+        # for strict verification, and the host is reachable, so an http://
+        # fallback would assess the wrong endpoint.
         if isinstance(exc, requests.exceptions.SSLError) or not url.startswith(
             "https://"
         ):
@@ -99,6 +111,11 @@ def assess(
         except requests.exceptions.RequestException:
             raise exc  # re-raise the original HTTPS error
 
+    # verify=False propagates across the redirect chain, so judge the channel
+    # the graded headers actually arrived on: flag only when the *final*
+    # response was served over HTTPS without certificate verification.
+    tls_verified = not (unverified and str(response.url).startswith("https://"))
+
     # ---- Extract normalised headers ----------------------------------
     headers = extract_headers(response)
     logger.debug("Received %d headers from %s", len(headers), response.url)
@@ -113,6 +130,7 @@ def assess(
         status_code=response.status_code,
         final_url=response.url,
         results=results,
+        tls_verified=tls_verified,
     )
 
     logger.info(

headersvalidator/cli.py

@@ -41,8 +41,15 @@ def check(
     timeout: float = typer.Option(
         10.0, "--timeout", "-t", help="Request timeout in seconds."
     ),
-    no_tls_verify: bool = typer.Option(
-        False, "--no-tls-verify", help="Skip TLS certificate verification."
+    tls_verify: bool = typer.Option(
+        False,
+        "--tls-verify/--no-tls-verify",
+        help=(
+            "Verify TLS certificates. Off by default so a single probe is "
+            "sent and invalid or self-signed certificates cannot block the "
+            "assessment; with verification on, TLS errors abort with exit "
+            "code 2."
+        ),
     ),
     json_output: bool = typer.Option(False, "--json", help="Output results as JSON."),
     fail_on_warn: bool = typer.Option(
@@ -72,7 +79,7 @@ def check(
     from headersvalidator.reporter import print_full_report
 
     try:
-        report = assess(url, timeout=timeout, verify_tls=not no_tls_verify)
+        report = assess(url, timeout=timeout, verify_tls=tls_verify)
     except requests.RequestException as exc:
         console.print(f"[red]Error:[/red] Could not reach {url!r}: {exc}")
         raise typer.Exit(code=2)
@@ -248,6 +255,7 @@ def _print_json(report) -> None:
         "status_code": report.status_code,
         "status": report.status.value,
         "score": report.score,
+        "tls_verified": report.tls_verified,
         "results": [
             {
                 "name": r.name,

headersvalidator/http_utils.py

@@ -8,10 +8,12 @@
 from __future__ import annotations
 
 import logging
+import warnings
 
 import requests
 from requests import Response
 from requests.exceptions import RequestException
+from urllib3.exceptions import InsecureRequestWarning
 
 logger = logging.getLogger("headersvalidator")
 
@@ -38,8 +40,9 @@ def fetch_headers(
     :param user_agent: Custom ``User-Agent`` header value.  Defaults to the
         headersvalidator UA string from
         :data:`headersvalidator.constants.USER_AGENT`.
-    :param verify_tls: Whether to verify TLS certificates.  Set ``False``
-        only for internal or self-signed hosts.
+    :param verify_tls: Whether to verify TLS certificates.  The assessor
+        probes with ``False`` by default (single request; certificate
+        posture is tlsvalidator's domain).
     :returns: The full HTTP response (headers accessible via
         ``response.headers``).
     :rtype: requests.Response
@@ -56,23 +59,29 @@ def fetch_headers(
 
     logger.debug("HEAD %s (timeout=%.1fs)", url, timeout)
     try:
-        response = requests.head(
-            url,
-            headers=headers,
-            timeout=timeout,
-            allow_redirects=True,
-            verify=verify_tls,
-        )
-        if response.status_code == 405:
-            logger.debug("HEAD returned 405; retrying with GET %s", url)
-            response = requests.get(
+        with warnings.catch_warnings():
+            if not verify_tls:
+                # An unverified fetch is the deliberate, report-flagged
+                # default probe mode; urllib3's per-request
+                # InsecureRequestWarning on stderr adds nothing to that.
+                warnings.simplefilter("ignore", InsecureRequestWarning)
+            response = requests.head(
                 url,
                 headers=headers,
                 timeout=timeout,
                 allow_redirects=True,
                 verify=verify_tls,
-                stream=True,  # Don't download the body
             )
+            if response.status_code == 405:
+                logger.debug("HEAD returned 405; retrying with GET %s", url)
+                response = requests.get(
+                    url,
+                    headers=headers,
+                    timeout=timeout,
+                    allow_redirects=True,
+                    verify=verify_tls,
+                    stream=True,  # Don't download the body
+                )
         logger.info(
             "Fetched %s → %s (final URL: %s)",
             url,

headersvalidator/models.py

@@ -93,6 +93,13 @@ class HeadersReport:
     results: list[HeaderResult] = field(default_factory=list)
     """One HeaderResult per evaluated header."""
 
+    tls_verified: bool = True
+    """False when the final response was served over HTTPS without
+    certificate verification — the default probe mode (``verify_tls=False``).
+    Headers received over an unverified channel could in principle be
+    attacker-influenced; the grade is not affected (certificate posture is
+    tlsvalidator's domain)."""
+
     # ------------------------------------------------------------------
     # Aggregate convenience properties
     # ------------------------------------------------------------------

headersvalidator/reporter.py

@@ -48,6 +48,13 @@
     VerdictSeverity.INFO: "dim",
 }
 
+# Plain-text body of the unverified-TLS warning banner; also used to size
+# file exports so the banner is never wrapped.
+_TLS_UNVERIFIED_NOTICE = (
+    "⚠ TLS certificate verification disabled — "
+    "headers were fetched over an unverified HTTPS connection."
+)
+
 # Grade letter → Rich colour string
 _GRADE_STYLE: dict[str, str] = {
     "A+": "bold bright_green",
@@ -80,6 +87,11 @@ def print_full_report(report: HeadersReport, console: Console | None = None) ->
             f"  [dim]→ redirected to[/dim] [cyan]{report.final_url}[/cyan]",
             highlight=False,
         )
+    if not report.tls_verified:
+        con.print(
+            f"  [bold yellow]{_TLS_UNVERIFIED_NOTICE}[/bold yellow]",
+            highlight=False,
+        )
     con.print()
     _print_results_table(report, con)
     actions = extract_verdict_actions(report)
@@ -287,6 +299,7 @@ def save_report(path: str, report: HeadersReport) -> None:
     header_floor = max(
         len(f"HTTP Headers Report — {report.url}"),
         len(f"  → redirected to {report.final_url}"),
+        len(f"  {_TLS_UNVERIFIED_NOTICE}") if not report.tls_verified else 0,
     ) + 4
     width = max(content_width, header_floor)
     file_console = Console(record=True, highlight=False, width=width, file=StringIO())

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "headersvalidator"
-version = "0.3.2"
+version = "0.4.0"
 description = "HTTP response header validator — RFC 9110, RFC 9111, OWASP, IANA"
 readme = "README.md"
 requires-python = ">=3.11"