Senior Network, Firewall & Voice Infrastructure SME

Job Summary Deploy and validate Infini Band fabrics supporting distributed AI training and HPC workloads. Troubleshoot issues involving fabric discovery, subnet management, link state, routing, partitioning, congestion, credit starvation, error counters, and host channel adapter behavior. Diagnose job failures and performance degradation related to collective communication, NCCL transport selection, RDMA pathing, and fabric imbalance. Validate switch, HCA, firmware, and cable consistency during cluster bring-up and expansion. Use low-level fabric tooling to isolate bad links, flapping ports, unhealthy endpoints, topology mismatches, or subnet manager instability. Partner with Linux, deployment, and AI validation teams to drive root cause analysis from application symptom to fabric source. Define and execute pre-flight and post-change validation workflows for IB cluster readiness. Document recurring fault patterns and create remediation playbooks for operational teams. WHAT WE NEED TO SEE7+ years in HPC or high-performance network environments, including direct Infini Band operations experience. Strong working knowledge of IB architecture, subnet management, link training, routing, partitions, congestion behavior, and performance diagnostics. Experience with RDMA, NCCL-related network dependencies, and multi-node AI workload sensitivity to transport issues. Strong troubleshooting skill using fabric health, counter, topology, and endpoint tools. Experience with firmware and driver alignment across HCAs, switches, and Linux hosts. Ability to triage complex issues that span host configuration, fabric state, and application communication behavior. PREFERRED EXPERIENCEExperience supporting DGX, GPU superpod, or equivalent AI cluster environments. Familiarity with UFM, telemetry pipelines, and automated fabric validation. Experience correlating IB anomalies with AI training performance outcomes.