Nodes drift out of spec over time, someone applies a manual fix, a package updates unexpectedly, a setting gets changed by hand. We catch this with a weekly automated compliance scan that checks every Gaudi node against the approved baseline:
If drift is found in the driver or firmware, we fix it through the standard upgrade process (see Keeping Software and Firmware Current) with a change ticket. If the OS/kernel itself has drifted, the node gets re-imaged from scratch rather than patched live it's faster and more reliable than trying to reconcile a broken state by hand.