AI Inference Server Security: Lessons From the NEAR AI SGLang Incident

Confidential computing does not remove the need to secure the software running inside an inference server. The NEAR AI SGLang incident makes this boundary concrete. An inference security assessment should identify what the enclave protects, which code processes requests and what downstream authority a compromised workload could reach.

What NEAR AI disclosed

In its November 9 report, NEAR AI says an SGLang remote-code-execution vulnerability gave an unauthorized party execution on one inference server and access to infrastructure. The report describes selected legacy-system data and two Cloud customers’ API keys in logs; it also says requests processed on the accessed server on November 8 could have been observed, including confidential requests.

NEAR says it closed the access, fixed the vulnerability across inference servers and rotated affected authority. It explains that confidential computing protects against the operator and host, but not a vulnerability in software running inside the environment. This article does not assign an unconfirmed SGLang CVE or invent a fixed-version number. Obtain the applicable upstream or provider-specific correction for your deployment.

NEAR AI’s report separates host protection from in-server execution

Draw the deployment before testing a single prompt

List the public gateway, inference API, management interfaces, workers, model and adapter sources, cache storage, logging systems and cloud identities. Mark where authentication and authorization actually occur. Keep the deployment’s network path separate from a vendor’s architectural description; a private design can still be deployed with an unintended reachable control surface.

그리고 vLLM security documentation is a useful separate example, not evidence about SGLang’s implementation. It documents route-specific authentication limits, recommends an allowlisting reverse proxy, warns against production development endpoints and treats plugins and writable caches as trusted-code concerns. Apply each product’s own documentation rather than assuming one API key protects every interface in every engine.

Boundary	Question for the owner	Safe acceptance evidence
Gateway to inference	Which callers and request classes are allowed?	Synthetic allowed and denied requests with correlated gateway and service decisions.
Inference to administration	Can a normal caller reach operational controls?	Route inventory and isolated rejection cases; no destructive operation.
Worker to cloud services	What identity and resources can the workload use?	Scoped synthetic resource test and effective permission review.
Model, adapter, cache and plugin inputs	Who can introduce executable or interpreted artifacts?	Origin, revision, write permissions and controlled update process.
Request to logs or analytics	Which contents leave the serving boundary?	Synthetic marker trace and retention/visibility record.

Verify the management plane independently

Begin with configuration and route inventory. Confirm what the proxy exposes and whether the backend can be reached through an alternate path. Use only documented, non-destructive operations in a controlled environment. A successful authenticated inference request proves that request worked; it does not establish that administrative routes require the same identity.

For a bounded pilot, define a normal application identity and a separate operator identity. The application identity should perform its intended inference operation but fail the approved management test before any state change. Record both decisions and the actual route. Do not use a model’s refusal text as proof that a backend control blocked the action.

Proposed deployment acceptance plan; follow each engine’s own docs

Test data handling and downstream authority with synthetic fixtures

Use a distinctive non-sensitive request marker to trace the permitted processing path. Record which logs, queues or analytics datasets may retain it and what the documented deletion process covers. Treat unavailable evidence as a limit; the absence of a marker in one log does not prove it was never stored elsewhere.

Review the serving workload’s cloud and internal-service identity. Define the minimum resource operations it needs, then test an unrelated synthetic resource through the supported assessment method. The goal is to establish an authorization boundary without granting extra privileges or replaying an intrusion. Keep credentials out of transcripts and reports.

Buy and close an assessment by boundary

Serving engine / exact artifact / configuration revision
Gateway and backend route inventory
Enclave or host boundary: documented protection and limits
Application identity vs operator identity
Artifact sources and cache writers
Downstream resource access: allowed / denied / unknown
Synthetic data route and retention evidence
Fix evidence, active runtime and normal-service retest

Require a provider to distinguish software correction, network containment, identity revocation and historical investigation. A restarted service is one observation. It does not establish all four outcomes. Revisit these boundaries when the engine, plugin set, model-loading route or deployment configuration changes.

Take the gateway, management and identity boundary map into any assessment pilot. When considering 펜리전트, confirm the supported surface and required evidence for each authorized case.

The official Penligent 3.0 video below demonstrates the general product workflow. It illustrates product use; the article’s assessment proposals and source findings are evaluated separately.

Method: AI-assisted review of the linked primary sources, checked November 12, 2026, with an original assessment framework. No exploit, production target or customer environment was tested for this article.