Root cause analysis / Investigation

api.x.ai

Reference RCA-8AF499244A029279

xAI API endpoint

Measurement point: Stealth NYC, 32 Avenue of the Americas.

Published investigation. Updates run automatically.

01 / Network path

Network path.

Endpoint measurements / ICMP IPv4

Latency over time.

Successful endpoint RTT recordings from up to one hour before and after the event.

Enter time range

Times use the selected timezone; choices include recorded samples and intervals across the available chart.

Use the recordings table below to view latency values.

Successful RTT recording Event interval

02 Nov 2026 15:05 – 02 Nov 2026 17:34 UTC. Hover or tap to inspect; drag across the chart to zoom. On touchscreens, first choose Select time range. Escape or Reset zoom restores the full range. Missing recordings are gaps; this is not an application availability or packet-loss chart.

Select a recording to explore nearby path observations.

View recorded values
Successful ICMP IPv4 RTT recordings.
Recorded at (UTC)RTT
1.381 ms
1.323 ms
1.340 ms
1.369 ms
1.639 ms
1.356 ms
1.346 ms
1.341 ms
1.314 ms
1.298 ms
1.427 ms
1.361 ms
1.343 ms
1.312 ms
1.337 ms
1.491 ms
1.343 ms
1.336 ms
1.333 ms
1.330 ms
1.605 ms
1.312 ms
1.459 ms
1.649 ms
1.411 ms
1.343 ms
1.373 ms
1.338 ms
1.371 ms
1.439 ms
1.334 ms
1.334 ms
1.467 ms
1.381 ms
1.319 ms
1.368 ms
1.410 ms
1.334 ms
1.356 ms
1.447 ms
9.871 ms
1.395 ms
1.310 ms
1.336 ms
1.364 ms
1.369 ms
1.320 ms
1.375 ms
1.436 ms
1.366 ms
1.414 ms
1.342 ms
1.316 ms
1.449 ms
1.333 ms
1.350 ms
1.350 ms
1.405 ms
1.324 ms
1.322 ms
1.365 ms
1.315 ms
1.330 ms
1.310 ms
1.355 ms
1.361 ms
1.390 ms
1.334 ms
1.444 ms
1.403 ms
1.401 ms
1.360 ms
1.372 ms
1.340 ms
1.472 ms
1.353 ms
8.268 ms
1.386 ms
1.363 ms
1.368 ms
1.421 ms
1.328 ms
1.472 ms
1.340 ms
1.530 ms
1.636 ms
1.355 ms
1.425 ms
1.396 ms
1.694 ms
1.466 ms
1.357 ms
1.358 ms
1.406 ms
1.298 ms
1.369 ms
1.294 ms
1.364 ms
1.387 ms
1.364 ms
7.437 ms
1.483 ms
1.324 ms
1.315 ms
1.343 ms
1.357 ms
4.319 ms
1.334 ms
1.336 ms
1.362 ms
1.373 ms
1.453 ms
1.393 ms
1.293 ms
1.316 ms
1.340 ms
1.353 ms
1.375 ms
1.381 ms
3.005 ms
1.390 ms
1.421 ms
1.464 ms
1.333 ms
1.333 ms
12.722 ms
21.024 ms
14.968 ms
1.326 ms
7.880 ms
1.348 ms
1.300 ms
6.477 ms
1.378 ms
1.625 ms
4.105 ms
1.345 ms
1.382 ms
7.096 ms
1.469 ms
1.336 ms
2.753 ms
1.365 ms
11.555 ms
1.412 ms
1.347 ms
1.343 ms
1.506 ms
1.390 ms
6.354 ms
1.397 ms
1.319 ms
1.336 ms
1.419 ms
1.345 ms
1.409 ms
1.435 ms
1.352 ms
1.353 ms
1.451 ms
1.358 ms
1.423 ms
1.330 ms
1.964 ms
1.331 ms
2.030 ms
1.413 ms
1.371 ms
1.808 ms
3.708 ms
1.447 ms
3.778 ms
1.923 ms
2.299 ms
1.371 ms
1.393 ms
1.345 ms
1.367 ms
1.419 ms
1.468 ms
1.359 ms
1.388 ms
1.437 ms
1.367 ms
1.493 ms
1.370 ms
1.424 ms
1.430 ms
1.374 ms
1.352 ms
1.342 ms
1.498 ms
1.408 ms
1.400 ms
1.318 ms
1.357 ms
1.364 ms
1.564 ms
1.364 ms
1.510 ms
1.331 ms
1.328 ms
1.403 ms
1.407 ms
1.437 ms
1.390 ms
1.396 ms
1.290 ms
1.343 ms
1.447 ms
1.386 ms
1.363 ms
1.331 ms
1.334 ms
1.392 ms
1.393 ms
1.314 ms
1.316 ms
1.330 ms
1.344 ms
1.396 ms
1.420 ms
1.774 ms
1.310 ms
1.377 ms
1.479 ms
1.305 ms
1.338 ms
3.917 ms
1.351 ms
6.347 ms
1.416 ms
14.174 ms
1.394 ms
11.337 ms
1.373 ms
1.348 ms
1.331 ms
1.381 ms
1.334 ms
1.725 ms
1.323 ms
1.323 ms
1.417 ms
1.352 ms
4.822 ms
1.321 ms
1.307 ms
1.404 ms
1.405 ms
1.385 ms
1.359 ms
2.848 ms
1.347 ms
1.394 ms
1.431 ms
1.339 ms
1.337 ms
1.400 ms
1.382 ms
1.341 ms
1.350 ms
1.418 ms
1.636 ms
2.356 ms
1.464 ms
1.356 ms
1.387 ms
1.336 ms
1.370 ms
2.946 ms
2.806 ms
1.373 ms
1.349 ms
1.358 ms
1.339 ms
1.388 ms
1.352 ms
1.451 ms
1.336 ms
3.282 ms
2.947 ms
1.325 ms
4.105 ms
1.359 ms
1.365 ms
1.424 ms
1.383 ms
1.369 ms
1.415 ms
1.371 ms
1.343 ms
1.327 ms
1.380 ms
8.601 ms
1.421 ms
1.430 ms
1.361 ms
1.364 ms
Network path / recorded observations

Network path observations.

17 observations

Choose a measurement type to see the closest of the available sampled observations to the selected chart point. Each probe is separate and may follow a different path.

Before

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.129 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.189 msReply RTT AS8002 · Stealth
Before

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.160 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.183 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hopNo replyNo comparable RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop4.611 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination0.926 msReply RTT AS13335 · Cloudflare
Before

ICMP traceroute · Recorded address 1

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.133 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.198 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop2.965 msReply RTT AS13335 · Cloudflare
Before

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.150 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.202 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop2.466 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop3.995 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.159 msReply RTT AS13335 · Cloudflare
Before

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.208 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.184 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hopNo replyNo comparable RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop2.035 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Probe hopNo replyNo comparable RTT ASN unavailable
  7. TTL 7 Destination17.440 msReply RTT AS13335 · Cloudflare
During

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.290 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.338 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop61.679 msReply RTT AS13335 · Cloudflare
During

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.144 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.185 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop1.025 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop1.555 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.461 msReply RTT AS13335 · Cloudflare
During

ICMP traceroute · Recorded address 1

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.203 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.175 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop19.878 msReply RTT AS13335 · Cloudflare
During

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.199 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.208 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hopNo replyNo comparable RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop2.249 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination19.410 msReply RTT AS13335 · Cloudflare
During

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.124 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.177 msReply RTT AS8002 · Stealth
During

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.209 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.186 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hopNo replyNo comparable RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop8.879 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.395 msReply RTT AS13335 · Cloudflare
After

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.130 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.180 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop18.234 msReply RTT AS13335 · Cloudflare
After

TCP traceroute / 443 · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.143 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.667 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop17.722 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop4.238 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Probe hopNo replyNo comparable RTT ASN unavailable
  7. TTL 7 Destination1.920 msReply RTT AS13335 · Cloudflare
After

ICMP traceroute · Recorded address 2

Recorded trace · destination not reached by this trace


to UTC

  1. TTL 1 Probe hop0.276 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.281 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop16.478 msReply RTT AS13335 · Cloudflare
After

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.192 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.181 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop17.865 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop14.483 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.319 msReply RTT AS13335 · Cloudflare
After

ICMP traceroute · Recorded address 1

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.134 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.183 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hop15.937 msReply RTT AS13335 · Cloudflare
  4. TTL 4 Probe hop1.668 msReply RTT AS13335 · Cloudflare
  5. TTL 5 Probe hop1.784 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.378 msReply RTT AS13335 · Cloudflare
After

TCP traceroute / 443 · Recorded address 2

Recorded trace · destination replied


to UTC

  1. TTL 1 Probe hop0.175 msReply RTT AS8002 · Stealth
  2. TTL 2 Probe hop0.186 msReply RTT AS8002 · Stealth
  3. TTL 3 Probe hopNo replyNo comparable RTT ASN unavailable
  4. TTL 4 Probe hopNo replyNo comparable RTT ASN unavailable
  5. TTL 5 Probe hop2.282 msReply RTT AS13335 · Cloudflare
  6. TTL 6 Destination1.584 msReply RTT AS13335 · Cloudflare

RTT is the round trip to each responding hop, not the delay of the preceding link. TTL order may omit routers and does not prove direct links. A missing reply is not proof of packet loss. ASN labels are dated address associations; they do not establish the forwarding path or the operator at the event time. Up to three recorded traces per method and period are selected across the archive. Where no individual trace is available, a comparison may summarize a window; MTR observations aggregate multiple probes. Address aliases are independent across measurement types.

Same method / same recorded address

Compare the traces.

Compare one selected trace from before the event with one during it. Values are individual replies, not a statistical comparison.

ICMP traceroute · Recorded address 1

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.133 msAS8002 · Stealth0.203 msAS8002 · StealthSame responder
20.198 msAS8002 · Stealth0.175 msAS8002 · StealthSame responder
32.965 msAS13335 · Cloudflare19.878 msAS13335 · CloudflareSame responder
ICMP traceroute · Recorded address 2

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.129 msAS8002 · Stealth0.290 msAS8002 · StealthSame responder
20.189 msAS8002 · Stealth0.338 msAS8002 · StealthSame responder
3Not recorded61.679 msAS13335 · CloudflareNot established
TCP traceroute / 443 · Recorded address 1

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.150 msAS8002 · Stealth0.209 msAS8002 · StealthSame responder
20.202 msAS8002 · Stealth0.186 msAS8002 · StealthSame responder
32.466 msAS13335 · CloudflareNo replyNot established
4No replyNo replyNot established
53.995 msAS13335 · Cloudflare8.879 msAS13335 · CloudflareDifferent responders
61.159 msAS13335 · Cloudflare1.395 msAS13335 · CloudflareSame responder
TCP traceroute / 443 · Recorded address 2

Before: UTC · During: UTC

Same destination address and probe method. A matching TTL does not establish the same forwarding path.
TTLBefore RTTDuring RTTResponding address
10.208 msAS8002 · Stealth0.144 msAS8002 · StealthSame responder
20.184 msAS8002 · Stealth0.185 msAS8002 · StealthSame responder
3No reply1.025 msAS13335 · CloudflareNot established
4No replyNo replyNot established
52.035 msAS13335 · Cloudflare1.555 msAS13335 · CloudflareDifferent responders
6No reply1.461 msAS13335 · CloudflareNot established
717.440 msAS13335 · CloudflareNot recordedNot established

RTTs include the return path. Different responders, missing replies and independent probe flows limit what a change can localize. MTR snapshots remain separate in the replay.

Event timeline.

Latency anomaly. Times shown are UTC.

Event opened
UTC
Last event observation
UTC
Recovery observed
UTC

Recorded recovery describes this event, not current reachability.

02 / Assessment

Assessment.

Investigation stateRecovered
Path assessment

Elevated latency observed.

Endpoint round-trip time was elevated during part of the event. The available path observations do not identify where the increase occurred.

What we observed

  • A sustained increase in measured round trip time was observed.
  • Stored traces were compared using matching source and destination addresses.
  • Repeated replies from a comparable hop showed increased round trip time.

Recording counts describe collected measurements, not application availability or packet loss. Exact packet paths and the return path remain unverified.

Measurements at a glance.

Baseline RTT1.346 msUsual round trip before the event
Last recorded event RTT1.468 msLast successful event recording
Event p95 RTT11.613 ms95th percentile of successful recordings
Change from baseline+0.122 msLast event RTT compared with the baseline
Successful / expected recordings60 / 60Recorded endpoint samples during the event

RTT is round trip time; p95 describes the upper range of successful recordings. Endpoint measurements differ from individual trace replies and are not an uptime percentage.

Record sources and timestamps
Analysis updated
UTC
Public snapshot published
UTC
Public observation window
– UTC

Event and source timestamps are UTC. The latency chart uses its selected timezone. Page refresh times are separate from measurement timestamps.

03 / Method and limits

Method and limits.

How to interpret the sampled path and its limits

Corroborating the path.

Comparisons use matching destinations before and during the event. ICMP traces use echo probes. TCP/443 traces send SYN probes to port 443 only; they do not complete a TCP handshake or perform a TLS or HTTP exchange.

The Stealth gNMI interface verifies the active route-table choice and maps its primary next hop to the Stealth logical interface, configured attachment and, where applicable, the known LAG member set at each snapshot. Stable Stealth replies, together with mapped interface evidence showing no corresponding errors or sustained capacity pressure, reduce the likelihood of a Stealth forwarding issue. An 80% load warning warrants investigation; it does not prove congestion. Lower interval averages can conceal bursts or queue pressure. Optical readings assess the physical link, not congestion.

What remains unseen.

Probe reply handling can differ from packet forwarding. A slow intermediate reply that disappears at later hops does not locate a forwarding problem. A destination that does not respond to a trace is not thereby shown to be unavailable.

Stealth does not use ECMP in this monitored scope. The route snapshot does not identify which physical LAG member carried a particular independent probe, attest line-card forwarding at that packet instant, prove the exact application packet flow or observe the return path. RTT includes the return path, which may differ. No remote reverse traceroute is available. Tunnels and silent routers may hide parts of the path. Address registration alone does not establish the forwarding operator or responsibility.

References: ICMP reply handling / RFC 1812 · Round trip measurements / RFC 2681 · Full methodology