====== Bringing a New Node Online ====== When a Gaudi node is added to a cluster, it goes through the same repeatable process every time: - **Physical setup:** rack, cable, and power the node; connect it to both the management network and the RoCEv2 fabric. - **BIOS/firmware config:** set PXE boot order and enable IOMMU/SR-IOV/NUMA interleaving per the Gaudi profile. - **Register it:** add the node's MAC address and IPMI details to our provisioning system (BCM) under the ''gaudi2-ubuntu22'' or ''gaudi3-ubuntu22'' profile. - **Image it:** PXE-boot the base OS image, then run our configuration management playbook (installs monitoring agents, hardening baseline from [[gaudi-os-patching-hardening|OS Patching and Hardening]], standard packages). - **Install the Gaudi driver stack:** as in [[gaudi-software-firmware|Keeping Software and Firmware Current]]. - **Validate:** run ''habana-validate'' to confirm the card and fabric are healthy. - **Add to the scheduler:** register the node in the Slurm/Kubernetes Gaudi pool. - **Log it:** add the node to the asset register (serial number, location, firmware version, install date). **Taking a node out of service** follows the same steps in reverse: drain it, confirm no jobs or data are still on it, revoke its access credentials, wipe its storage (see [[gaudi-rma|Hardware Replacement (RMA)]]), remove it from BCM, and update the asset register. ---- [[gaudi-security|← Previous]] | [[gaudi-guide|Guide Index]] | [[gaudi-configuration-consistency|Next →]]