User Tools

Site Tools


wiki:ai:nvidia_gpu_operator_installation_and_upgrade

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
wiki:ai:nvidia_gpu_operator_installation_and_upgrade [2026/07/20 12:42] ssureshwiki:ai:nvidia_gpu_operator_installation_and_upgrade [2026/07/20 16:53] (current) ssuresh
Line 198: Line 198:
  
  
-=== Phase 5 — CUDA VectorAdd GPU test ===+===== Phase 5 — CUDA VectorAdd GPU test =====
  
 <code -> <code ->
Line 224: Line 224:
 {{:wiki:ai:screenshot_2026-07-14_at_7.05.33 pm.png|}} {{:wiki:ai:screenshot_2026-07-14_at_7.05.33 pm.png|}}
  
-==== Phase 6 — Upgrade the Operator==== +===== Phase 6 — Upgrade the Operator =====
  
 **Concept: ** **Concept: **
Line 505: Line 505:
  
 {{:wiki:ai:screenshot_2026-07-15_at_10.57.56 pm.png|}} {{:wiki:ai:screenshot_2026-07-15_at_10.57.56 pm.png|}}
 +
 +
 +===== PART III — ROLLBACK PLAN =====
 +
 +**Before any upgrade — capture your safety net**
 +
 +<code ->
 +helm list -n gpu-operator                                             # note operator + version
 +helm get values <RELEASE_NAME> -n gpu-operator > working-values.yaml  # save WORKING config (incl. k3s toolkit.env)
 +helm history <RELEASE_NAME> -n gpu-operator                           # note current revision
 +</code>
 +
 +{{:wiki:ai:screenshot_2026-07-15_at_11.18.24 pm.png|}}
 +
 +
 +**Roll back the Operator release**
 +
 +<code ->
 +helm history <RELEASE_NAME> -n gpu-operator          # see revisions
 +helm rollback <RELEASE_NAME> -n gpu-operator          # previous revision
 +# or a specific revision:
 +helm rollback <RELEASE_NAME> 2 -n gpu-operator
 +helm rollback <RELEASE_NAME> 1 -n gpu-operator
 +</code>
 +
 +{{:wiki:ai:screenshot_2026-07-15_at_11.19.40 pm.png|}}
  
 **Simulation cleanup** **Simulation cleanup**
Line 516: Line 542:
 </code> </code>
  
-=== PART III — ROLLBACK PLAN ===+{{:wiki:ai:screenshot_2026-07-15_at_10.59.54 pm.png|}}
  
 +**Cleanup (avoid a surprise bill!)**
  
 +  * Pause between sessions: EC2 → Stop (compute stops billing; small EBS cost remains; public IP changes on restart unless you use an Elastic IP).
 +  * Done for good: EC2 → Terminate (deletes instance + disk).
 +  * Release any Elastic IP if you allocated one (billed even while unused).
wiki/ai/nvidia_gpu_operator_installation_and_upgrade.1784551369.txt.gz · Last modified: by ssuresh