Pure-Python opentelemetry-proto (drop the protobuf dependency) [1/4 RFC]

[ocelotl]

What

Replace the google.protobuf-generated message classes in opentelemetry-proto with a hand-written pure-Python protobuf implementation, and drop the protobuf dependency (protobuf>=5.0, <8.0) entirely.

The public opentelemetry.proto.* import namespace is preserved: real code lives under a private opentelemetry._proto package, and the existing opentelemetry.proto.* modules become thin re-export shims (from opentelemetry._proto... import *). Downstream code importing opentelemetry.proto.trace.v1.trace_pb2 etc. keeps working unchanged.

Why

protobuf ships a compiled C extension (upb) and enforces a narrow, moving version range. That causes real friction:

• Injection / zero-code instrumentation conflicts — when the SDK is injected into an arbitrary target process (K8s operator/injector), the bundled protobuf can collide with the version the target app pins, producing hard-to-diagnose ABI/version errors. A pure-Python encoder is safe to inject.
• Dependency-resolution friction — the >=5.0,<8.0 range regularly clashes with other libraries (gRPC stacks, ML tooling, cloud SDKs).
• Wheel/ABI portability — no compiled component means it runs anywhere CPython runs.

The OTLP exporters only ever serialize protobuf; they never parse arbitrary messages. The only decode needed is the export-service response, which is empty. So a small, auditable encode-only implementation is sufficient — this is not a general-purpose protobuf runtime.

Scope of this PR

This is the base of a 4-PR stacked series that makes the whole OTLP export path dependency-light. Merge order:

1. opentelemetry-proto → pure-Python (this PR)
2. opentelemetry-exporter-otlp-proto-common → pure-Python backend
3. opentelemetry-exporter-otlp-proto-http → pure-Python + stdlib urllib (drops requests)
4. opentelemetry-exporter-otlp-proto-grpc → pure-Python gRPC (drops grpcio)

Because this org's fork model can't host intermediate base branches upstream, each PR targets main and its diff is cumulative (this PR's changes appear in all four). Isolated per-package diffs are viewable via the fork compare links noted in PRs 2–4.

Known gaps / discussion points (why this is a draft)

• profiles signal not ported. The pure-Python impl currently covers common, resource, trace, metrics, logs and the collector service messages, but not opentelemetry.proto.profiles. That namespace is removed here and would need to be added before this could replace the current package for profiles users.
• Codegen vs hand-maintenance. The message classes should ideally be generated from the same .proto sources the project vendors, so schema changes stay mechanical. This PR carries a hand-written implementation; agreeing on a generator is part of the discussion.
• Structure. The _proto (real) + proto (shim) split is one option; collapsing into proto directly is another. Happy to go either way.
• Conformance. Output should be byte-for-byte identical to the protobuf-generated serialization; a differential test against the current encoders would be worth adding.
• CI will not be green as-is (tests and docs need adaptation) — this is intentionally an RFC to gauge interest in the approach.

The reference implementation this is derived from has been running in a downstream distribution.

---

Stack (merge in order): #5503 (proto) → #5504 (common) → #5505 (http) → #5506 (grpc)