Open the monitoring dashboards of most organisations and they look complete: application response times, error rates, logs, traces, CPU and memory use for every host. Look one layer down and the picture thins out. The temperature of each CPU core, the corrected error count of each memory module, the media health of each drive, the output of each power supply, fan speeds, inlet and outlet air temperatures, firmware versions, the coolant flow to each rack: these are often collected somewhere, by someone, and seen by nobody when it matters. In AI data centers, where hardware faults are frequent and expensive, the layer below the operating system is where many incidents start.
Why the gap exists
The reasons are historical more than technical.
In-band monitoring stops where the operating system stops. Agents running inside the OS report what the OS can see. They cannot report a server that has crashed, hung or lost power, and they see little of the hardware's own sensors and logs.
Hardware data lives on another network. Baseboard management controllers sit on a separate management network, speak IPMI or Redfish, and are often configured once at installation and forgotten. Each server vendor has its own management tool, and many sites run several.
Facility data belongs to another team. PDUs, UPS systems, cooling units and leak sensors report to a building management system or DCIM tool run by facilities staff, with no link to the IT tools.
GPU data needs its own collector. GPU health and performance come through NVIDIA's DCGM or the equivalent for other accelerators, which is often deployed by the AI platform team for their own use.
The result is several partial views, each with its own alarms, and no single place where a person can see the whole server and everything around it.
What usually goes unwatched
| Layer | Usually watched | Often missed |
|---|---|---|
| Application | Latency, errors, throughput | Correlation with hardware events |
| Operating system | CPU, memory, disk space, network traffic | Machine check errors, kernel hardware warnings |
| Server hardware | "Server up or down" from ping | ECC counts, PSU redundancy, fan failures, RAID state, firmware drift |
| GPU | Utilisation from a basic counter | Xid errors, memory remapping, thermal throttling, NVLink errors |
| Rack and power | Total power per room | Per-PDU and per-outlet load, phase balance, breaker alarms |
| Cooling | Room temperature | Inlet temperature per rack, CDU flow and ΔT, leak sensors |
None of the missed items is exotic. Each is available from standard interfaces. The gap is in collection, retention and correlation.
A worked example
An inference cluster's latency rises slowly over two weeks. The application team sees it and investigates the software: recent deployments, traffic changes, model versions. They find nothing.
Below the operating system, the story was visible from the start. A cooling unit at the end of one row had a failing fan. Inlet temperatures for the racks in that row rose by 4 °C. GPU temperatures in those racks rose further, and several GPUs began thermal throttling during peak hours. Requests routed to those servers were served more slowly, which raised the average latency for the whole service.
The data existed in three systems: the building management system had the cooling unit's fan alarm, the BMCs had the inlet temperatures, and DCGM had the throttling events. No one connected them until a facilities engineer replaced the fan during routine maintenance, and the latency problem disappeared.
Closing the gap
Most existing tools can stay. Closing the gap takes a few deliberate steps.
Map coverage first. For each layer in the table above, list which systems collect what, how often, and who looks at it. Gaps become obvious.
Collect out of band for every server. Use Redfish or IPMI to gather sensors, event logs and inventory from every BMC on a regular schedule, and keep enough history to see trends. This also reports on servers that are down.
Bring in facility data. Connect PDUs, UPS units, cooling units, CDUs and leak sensors through standard protocols such as SNMP, Modbus or BACnet, and map each device to the racks it serves.
Add GPU health. Deploy DCGM collection on every GPU server and keep the health and error fields alongside utilisation.
Keep history long enough to see trends. Many hardware problems develop over weeks: a memory module's corrected errors climb, a fan slowly loses speed, a drive's spare blocks run down. Retention of a few days, common for high-frequency metrics, hides these patterns. Keep daily summaries of hardware health for at least a year, so that a slow drift stands out against the server's own past and against identical servers.
Link everything to an asset record. Each reading should know which server, rack, row and power feed it belongs to. Without that, data cannot be correlated.
Correlate alarms. Group related alarms from different layers into one incident with a probable root cause, so the team sees one problem instead of thirty alerts.
Measure the improvement
Two numbers show whether the gap is closing: the share of hardware incidents that were detected by monitoring before users or applications noticed, and the average time from first signal to action. Both should improve steadily as coverage grows. A third, the share of alarms that turned out to be noise, keeps the effort honest.
CloudSino DCOS collects hardware health from server BMCs over IPMI and Redfish, covering power supplies, fans, disks, memory, CPUs, GPUs and temperatures, and it keeps reporting when the operating system is down. On AI computing center sites, CloudSino adds GPU data from NVIDIA DCGM and liquid cooling readings, and SmartBSM correlates the resulting alarms into incidents with a probable root cause. Ask us for a coverage review of your own environment.
Frequently asked questions
What is out-of-band server monitoring?
Out-of-band monitoring collects hardware data directly from a server's baseboard management controller over a separate management network, using IPMI or Redfish. Because the BMC runs independently of the operating system, it can report sensors, event logs and inventory even when the server has crashed, hung or lost its OS. In-band agents cannot see a machine in that state.
What hardware metrics do data center monitoring tools usually miss?
The most commonly missed metrics sit below the operating system: corrected memory error counts, power supply redundancy, fan failures, RAID state, firmware drift, GPU Xid errors and thermal throttling, per-outlet PDU load, rack inlet temperatures, CDU flow and leak sensors. All are available through standard interfaces. The usual gap is that nobody collects, keeps or connects them.
How long should I keep server hardware health data?
Keep daily summaries of hardware health for at least a year. Many faults build slowly over weeks, such as rising corrected memory errors, a fan losing speed or a drive using up its spare blocks. A few days of retention, typical for high-frequency metrics, hides those trends. Long history lets you compare a server against its own past and against identical machines.