Achhina's Digital Garden

GSSAPI, SPNEGO, Kerberos - HTTP Negotiate Auth

5 min read

GSSAPI, SPNEGO, Kerberos — HTTP Negotiate Auth

Layering of the Negotiate auth stack, the Python library landscape, and the long-standing concurrency bug in requests-kerberos / requests-gssapi.

Protocol layering

  • Kerberos — actual auth protocol: KDC, TGT, service tickets, AP-REQ/AP-REP.
  • GSSAPI (RFC 2743, C bindings RFC 2744) — not a wire protocol. A generic interface contract: mechanisms produce opaque tokens; the application ferries them over its own transport. Kerberos is one pluggable mechanism (RFC 4121).
  • SPNEGO (RFC 4178, obsoletes 2478) — Simple and Protected GSS-API Negotiation Mechanism. A pseudo-mechanism inside GSSAPI that negotiates the real mechanism in-band. “Simple”: client optimistically bundles its preferred mech token (usually the Kerberos AP-REQ) in the first negotiation token → zero extra round trips in the common case. “Protected”: post-handshake MIC over the mech list detects downgrade attacks (e.g. stripping Kerberos to force NTLM).
  • HTTP Negotiate (RFC 4559) — the transport binding: tokens ride in Authorization: Negotiate <b64> / WWW-Authenticate: Negotiate <b64>.

Stack for one request: HTTP Negotiate → SPNEGO → Kerberos.

GSSAPI state machine

stateDiagram-v2
    [*] --> NoCredential
    NoCredential --> HaveCredential : GSS_Acquire_cred (ccache/TGT)
    HaveCredential --> Loop : Init_sec_context(∅) → token₁
    state "Context Establishment Loop" as Loop {
        [*] --> Awaiting
        Awaiting --> Stepping : receive tokenₙ
        Stepping --> Awaiting : CONTINUE_NEEDED → tokenₙ₊₁
        Stepping --> [*] : COMPLETE
    }
    Loop --> Established : session keys derived,\nmutual auth verified
    Loop --> Failed : GSSError (bad ticket,\nclock skew, SPN mismatch)
    state Established {
        [*] --> Ready
        Ready --> Ready : Wrap/Unwrap, GetMIC/VerifyMIC
    }
    Established --> [*] : Delete_sec_context

Key lifecycle facts:

  • The security context is the loop’s state → inherently per-handshake lifetime.
  • Kerberos over HTTP: token₁ = AP-REQ (sent in the 401 retry); server AP-REP on the final response is the last step() → COMPLETE. Mutual auth = client verifying that AP-REP.
  • Established context holds session keys → enables message-level Wrap/Unwrap (this is what pywinrm uses to encrypt SOAP payloads over plain HTTP).
  • Per-message Negotiate is stateless per request on RFC-compliant servers, but Windows behaves non-RFC-compliantly: persistent connections reuse the established context across requests (connection-oriented auth). Context lifetime is per-request except in the WinRM/Windows world.

Python library map

LibraryLayerNotes
requests-kerberosHTTP plugin (requests auth hook)v0.15 engine = pyspnego; maintenance mode
requests-gssapiHTTP pluginsuccessor; engine = python-gssapi; keeps a requests-kerberos compat shim
pyspnegocross-platform Negotiate dispatcherdelegates → python-gssapi (Unix), SSPI/sspilib (Windows), own pure-Python NTLM; implements SPNEGO framing logic
python-gssapibindings to system GSSAPI (MIT/Heimdal)mechanism-agnostic, Unix-centric
pykerberos / winkerberoslegacy thin wrapperspre-0.15 requests-kerberos engines

Dependency arrow: pyspnego → python-gssapi (optional extra pyspnego[kerberos]), not the other way — protocol layering (SPNEGO under GSSAPI) and library dependency graph point in opposite directions because pyspnego sits above both platform interfaces (GSSAPI and SSPI).

The concurrency bug

Mechanism. requests auth objects are invoked once at prepare time (__call__) and register self.handle_response as a response hook. The 401→token→retry handshake is faked as a nested synchronous send inside the hook. Because the hook is a bound method, all handshake state lands on the shared auth instance:

  • self.pos — body stream offset, saved at prepare, used to rewind the (consumed) streaming body before the 401 retry. One slot, all in-flight requests.
  • self._context — GSSAPI contexts keyed by hostname. Two concurrent handshakes to the same host clobber each other → invalid Negotiate None headers, mutual auth failures.
  • v0.15 added more shared state: self.auth_done, CBT/channel-binding values populated after first response.

Root cause framing: all mutable state has request lifetime, not authenticator lifetime. Thread-local narrows the race (fixes threads, not event-loop/greenlet interleaving) but doesn’t fix the scoping. Correct fix: key state by PreparedRequest (reachable in the hook via response.request) or capture it in a per-request closure. Contrast: httpx’s auth interface is a generator (coroutine-shaped) → handshake state is stack-local → safe by construction.

History (verified 2026-07-17):

  • Issue #113 (mkomitee, Mar 2018) — original author files the exact diagnosis + fix (index context by PreparedRequest). Cross-filed as requests-gssapi #8.
  • PR #114 — working fix (user-confirmed), never merged. Blockers:
    1. WinRM: pywinrm reaches into auth.context[host] for the established context’s session key to encrypt SOAP payloads; cipher state is tied to request state + Kerberos sequence numbers. Host-keyed forever-lived context was a load-bearing side channel. Fix removed winrm_wrap/winrm_unwrap → jborean93 objected (no deprecation cycle).
    2. Governance: mkomitee could merge but not release; project went dormant.
    3. gssapi side: frozencemetery willing to merge, blocked by the requests-kerberos compat shim (signature change to generate_request_header → TypeError in shim).
  • michael-o tested the request-keyed patch under 18 threads: still failed — pos (and target_name) remain shared. Confirms partial fix insufficient; all mutable state must be request-scoped.
  • Current status: unfixed in both libraries. But v0.15 renamed context → _context (“internal use only”) and modern WinRM stack does encryption via pyspnego itself → the 2018 blocker has largely dissolved.

Patch design notes (for upstream PR)

  • Scope per-request: pos, security context, CBT/auth_done — via WeakKeyDictionary keyed on PreparedRequest (avoids the memory leak mkomitee flagged; nitzmahone suggested weakrefs) or per-request closure hooks.
  • Gotcha: handle_401 does response.request.copy() before re-sending — stashed state must be carried across the copy explicitly.
  • Target requests-gssapi first (active maintainer); preserve the compat shim signature to avoid the 2019 blocker.
  • Related prior art: HTTPDigestAuth in requests fixed the same class of bug with threading.local() — weaker fix, but the precedent to cite.

RFCs

Graph View