docs: create work/Nexus_1_Build

This commit is contained in:
Administrator 2026-02-19 20:45:23 +00:00 committed by John Smith
parent 50539767a0
commit 9195c5f77c

715
work/Nexus_1_Build.md Normal file
View file

@ -0,0 +1,715 @@
---
title: C9300GX-1 Build
description:
published: true
date: 2026-02-19T20:45:10.926Z
tags:
editor: markdown
dateCreated: 2026-02-19T20:45:10.926Z
---
# AT1EU-NEXUS-1 — Cisco Nexus 9300 Configuration
## Overview
AT1EU-NEXUS-1 is the **primary** switch in a vPC pair (role priority 10, lower = preferred). It runs NX-OS 10.3(7) and forms a vPC domain with AT1EU-NEXUS-2. The two switches share a vPC peer-link (Po10) across Eth1/4748, and use out-of-band management (mgmt0 at 192.168.0.1) for the vPC peer-keepalive path.
**Key roles of this switch:**
- vPC primary (role priority 10)
- STP root bridge for management/native VLANs (priority 8192 for VLANs 1, 66)
- Layer 3 gateway for Vlan502 (Atom VRF, IP 15.0.2.121/24)
- NTP master (stratum 3)
- Upstream connections: 500e-X1 (Po3), 500e-X2 (Po4), 9300 (Po124)
- Storage connections: AFF300-A (Po127), AFF300-B (Po128), FAS2750-A (Po129), FAS2750-B (Po130), A70-A (Po131), A70-B (Po132)
- Compute connections: UCS-A (Po125), UCS-B (Po126)
---
## Cut-and-Paste Configuration
```
version 10.3(7) Bios:version 07.71
switchname AT1EU-NEXUS-1
! --- QoS: Jumbo Frame Policy ---
policy-map type network-qos JUMBO
class type network-qos class-default
mtu 9216
! --- VDC Resource Limits ---
vdc AT1EU-NEXUS-1 id 1
limit-resource vlan minimum 16 maximum 4094
limit-resource vrf minimum 2 maximum 4096
limit-resource port-channel minimum 0 maximum 511
limit-resource m4route-mem minimum 58 maximum 58
limit-resource m6route-mem minimum 8 maximum 8
! --- Features ---
feature nxapi
feature bash-shell
feature scp-server
cfs eth distribute
feature udld
feature interface-vlan
feature lacp
feature vpc
feature lldp
feature telemetry
! --- RBAC ---
role name network-ro
rule 2 permit read
rule 1 permit command show running-config
! --- Users ---
username admin password 5 $5$MFJCIC$AJyskD7vdoVFKK5cTS2lO20omFL4XFrgqNB94qDA5Z2 role network-admin
ssh key rsa 2048
! --- Banner ---
banner motd ^
********************* DOD NOTICE AND CONSENT BANNER *************************
* You are accessing a U.S. Government (USG) Information System (IS) that is *
* provided for USG-authorized use only. By using this IS (which includes any*
* device attached to this IS), you consent to the following conditions: *
*-The USG routinely intercepts and monitors communications on this IS for *
* purposes including, but not limited to, penetration testing, COMSEC *
* monitoring, network operations and defense, personnel misconduct (PM), *
* law enforcement (LE), and counterintelligence (CI) investigations. *
*-At any time, the USG may inspect and seize data stored on this IS. *
*-Communications using, or data stored on, this IS are not private, are *
* subject to routine monitoring, interception, and search, and may be *
* disclosed or used for any USGauthorized purpose. *
*-This IS includes security measures (e.g., authentication and access *
* controls) to protect USG interests--not for your personal benefit or *
* privacy. *
*-Notwithstanding the above, using this IS does not constitute consent to *
* PM, LE or CI investigative searching or monitoring of the content of *
* privileged communications, or work product, related to personal *
* representation or services by attorneys, psychotherapists, or clergy, and *
* their assistants. Such communications and work product are private and *
* confidential. See User Agreement for details. *
************************ POC: SIL Network Team ****************************
^
! --- SSH ---
ssh ciphers aes256-gcm
! --- DNS & Domain ---
ip domain-lookup
ip name-server 15.0.2.128 15.0.2.129 15.32.2.128
ip domain-name atom.dev use-vrf Atom
ip name-server 15.0.2.128 15.0.2.129 15.32.2.128 use-vrf Atom
! --- RADIUS ---
radius-server host 15.0.11.68 key 7 "V1P-jaynmv" authentication accounting
radius-server host 15.32.11.68 key 7 "V1P-jaynmv" authentication accounting
aaa group server radius NETMAN_RADIUS
server 15.0.11.68
server 15.32.11.68
use-vrf Atom
! --- Management ACL ---
ip access-list SWITCH_MGMT
10 permit ip 15.0.11.150/32 any log
20 permit ip 15.0.11.151/32 any log
30 permit ip 15.32.2.154/32 any log
40 permit ip 15.0.2.154/32 any log
50 permit ip 15.32.2.1/32 any log
60 permit ip 15.0.2.1/32 any log
70 permit ip 15.0.2.2/32 any log
80 permit ip 15.0.11.47/32 any log
90 permit ip 15.32.11.45/32 any log
93 permit ip 15.32.11.150/32 any log
100 deny ip any any log
! --- System QoS ---
system qos
service-policy type network-qos JUMBO
copp profile strict
! --- SNMP ---
snmp-server user admin network-admin auth sha 042F64DB5D2E0D40DF543D6A00495F1F18F9DD5FED7B priv aes-128 00540CF9793F282ED96D666B110B00753FC3F269E964 localizedV2key
snmp-server host 15.0.2.188 traps version 3 priv at-sw-svc
snmp-server enable traps config ccmCLIRunningConfigChanged
rmon event 1 log trap public description FATAL(1) owner PMON@FATAL
rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 log trap public description ERROR(3) owner PMON@ERROR
rmon event 4 log trap public description WARNING(4) owner PMON@WARNING
rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO
! --- NTP ---
ntp server 15.0.0.9 prefer use-vrf Atom key 123
ntp server 15.32.0.9 prefer use-vrf Atom key 125
ntp source-interface Vlan502
ntp authenticate
ntp authentication-key 123 md5 pz5yamz 7
ntp trusted-key 123
ntp logging
ntp master 3
! --- AAA ---
aaa authentication login default group NETMAN_RADIUS local
aaa authentication login console group NETMAN_RADIUS local
aaa accounting default group NETMAN_RADIUS local
system default switchport
no ip source-route
! --- VLANs ---
vlan 1-2,8,10,12,66,85,100-103,107-108,121-124,129-130,142-143,145-146,148-150,153,157-158,188,305,321,323,340,342,349,353,374,382,501-502,504-505,549,551,559,562-563,600,611,660-661,667-668,672-673,697-698,701-702,704-710,720-722,724,727,740,750-751,772,777,800-802,804,814,820-823,905,1051,1127,1129,1160-1161,1551,1559-1560,1670-1674,1720-1722,1800-1802,1814-1817,1862,1865,1870-1871
vlan 1882-1883,1885,1905,3563,3965
vlan 2
name TEST_CLUS_COMM
vlan 8
name FP_Test1
vlan 10
name NESS_BOX_TRANSIT
vlan 12
name FP_Test2
vlan 66
name NATIVE_VLAN
vlan 85
name NESS_Temp
vlan 100
name migration
vlan 101
name iscsi_csv
vlan 102
name iscsi_boot
vlan 103
name Netapp_XFER
vlan 107
name Test
vlan 108
name NET_TEST_NET
vlan 121
name Atom_Backup
vlan 123
name storage
vlan 124
name Admin_iSCSI
vlan 130
name SIL_SNAPMIRROR
vlan 143
name Secman_Storage
vlan 146
name Foxhound_Storage
vlan 150
name iscsi
vlan 153
name Javelin(L4)
vlan 157
name GNext_Storage
vlan 158
name Ness_Storage
vlan 188
name JASON_NFS
vlan 321
name ATOM_Backup
vlan 323
name AT-vServer
vlan 340
name ucs_test
vlan 342
name MadHatter_SVM_Mgmt
vlan 349
name Rock_SVM3_Mgmt
vlan 353
name Javlin_SVM
vlan 374
name Rock_Backup_Mgmt
vlan 382
name Darrin_User
vlan 501
name MGMT
vlan 502
name Atom_User2
vlan 504
name Commvault_Test
vlan 505
name NETAPP_SNAP
vlan 549
name WDS
vlan 551
name L4_User
vlan 559
name Victory_WS_L4
vlan 562
name Brace(L3)_User
vlan 667
name Britt_Test
vlan 668
name RockTesters(L4)_User
vlan 672
name GTRI_User
vlan 673
name VDI(L5)
vlan 701
name MH_L3_DATA_HLCI
vlan 702
name MH_L4_DATA_HLCI
vlan 704
name Legacy-704
vlan 705
name Legacy-705
vlan 706
name Legacy-706
vlan 707
name Legacy-707
vlan 708
name Legacy-708
vlan 709
name Legacy-709
vlan 710
name Legacy-710
vlan 721
name GTRI_JAVELIN_L4-721
vlan 740
name NETMAN
vlan 750
name l4_secman
vlan 751
name Secman_DMP-751
vlan 777
name FTD1010_TSHOOT
vlan 804
name FH_L4_HLCI
vlan 814
name Rock_L4
vlan 820
name GNext_User
vlan 821
name GNext_Sentris
vlan 822
name GNext_VPX
vlan 823
name GNext_VDA
vlan 905
name Rock_(L4)
vlan 1051
name IP_SEC_1010
vlan 1127
name Vic_Storage
vlan 1551
name Services(L3)_User
vlan 1559
name Victory(L3)_User
vlan 1670
name BigTen_User
vlan 1671
name Victory_DMP-1671
vlan 1672
name VIC_VDI
vlan 1673
name Victory_Sentris
vlan 1720
name Javelin(L3)_User
vlan 1721
name GTRI_JAVELIN_L3-1721
vlan 1722
name Victory_VDI-1722
vlan 1800
name Foxhound(L3)_User
vlan 1801
name FH_L3_DATA_HLCI
vlan 1814
name ROCK_L3_MLS
vlan 1815
name ServMan_User
vlan 1870
name AT1EU-JavelinCoop(L3)_User
vlan 1883
name NESS_User
vlan 1885
name NESS_Client
vlan 1905
name Rock(L3)_User
vlan 3563
name Brace_User
vlan 3965
name V3E_DEV_HOST
! --- Spanning Tree ---
spanning-tree port type edge bpduguard default
spanning-tree port type edge bpdufilter default
spanning-tree port type network default
spanning-tree vlan 1,66 priority 8192
spanning-tree vlan 2,100-102,107-108,121-123,129,142,145,148-150,153,305,323,340,353,382,501-502,505,549,551,562-563,600,611,660-661,667-668,672,697-698,701-702,704-710,720-722,724,727,750,772,800-802,804,814,905,1127,1129,1160-1161,1551,1559-1560,1670,1672-1673,1720-1721,1800-1802,1814-1817,1862,1865,1870-1871,1882,1905,3563,3965 priority 24576
spanning-tree vlan 3-65,67-99,103-106,109-120,124-128,130-141,143-144,146-147,151-152,154-304,306-322,324-339,341-352,354-381,383-500,503-504,506-548,550,552-561,564-599,601-610,612-659,662-666,669-671,673-696,699-700,703,711-719,723,725-726,728-749,751-771,773-799,803,805-813,815-904,906-1126,1128,1130-1159,1162-1550,1552-1558,1561-1669,1671,1674-1719,1722-1799,1803-1813,1818-1861,1863-1864,1866-1869,1872-1881,1884-1904,1906-3562,3564-3964,3966-3967 priority 0
spanning-tree vlan 1883 priority 4096
! --- VRF ---
vrf context Atom
ip domain-name atom.dev
ip name-server 15.0.2.128 15.0.2.129 15.32.2.128
ip route 0.0.0.0/0 15.0.2.254
vrf context management
! --- Port-Channel Load Balance ---
port-channel load-balance src-dst ip-l4port-vlan
! --- vPC Domain ---
vpc domain 1
peer-switch
role priority 10
peer-keepalive destination 192.168.0.2 source 192.168.0.1
delay restore 150
peer-gateway
auto-recovery
! --- SVI ---
interface Vlan1
interface Vlan502
no shutdown
vrf member Atom
no ip redirects
ip address 15.0.2.121/24
no ipv6 redirects
! --- Port-Channels ---
interface port-channel3
description //Trunk 500e X1
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard enable
spanning-tree guard root
mtu 9216
vpc 3
interface port-channel10
description //Trunk Peer - Allow STP
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type network
vpc peer-link
interface port-channel124
description //Trunk 9300
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type normal
spanning-tree bpduguard disable
spanning-tree guard root
mtu 9216
no lacp suspend-individual
vpc 124
interface port-channel125
description //Trunk UCS-A
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard disable
spanning-tree guard root
mtu 9216
vpc 125
interface port-channel126
description //Trunk UCS-B
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard disable
spanning-tree guard root
mtu 9216
vpc 126
interface port-channel127
description //Trunk AFF300-A
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree guard root
mtu 9216
vpc 127
interface port-channel128
description //Trunk AFF300-B
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree guard root
mtu 9216
vpc 128
interface port-channel129
description //Trunk FAS 2750-A
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard enable
spanning-tree guard root
mtu 9216
storm-control broadcast level 99.00
storm-control unicast level 99.00
switchport block unicast
vpc 129
interface port-channel130
description //Trunk Fas 2750-B
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard enable
spanning-tree guard root
mtu 9216
storm-control broadcast level 99.00
storm-control unicast level 99.00
switchport block unicast
vpc 130
interface port-channel131
description //Trunk A70-A
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree guard root
mtu 9216
vpc 131
interface port-channel132
description //Trunk A70-B
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree guard root
mtu 9216
vpc 132
! --- Breakout Ports (100G -> 4x25G) ---
int e1/1 - 26
shutdown
exit
interface breakout module 1 port 1 map 25g-4x
interface breakout module 1 port 5 map 25g-4x
! --- Physical Interfaces: Breakout (UCS/A70) ---
interface Ethernet1/1/1
description //Trunk 6554-1:25
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard enable
spanning-tree guard root
mtu 9216
channel-group 125 mode active
no shutdown
interface Ethernet1/1/2
description //Trunk 6554-1:26
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard enable
spanning-tree guard root
mtu 9216
channel-group 125 mode active
no shutdown
interface Ethernet1/1/3
description //Trunk 6554-2:27
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard enable
spanning-tree guard root
mtu 9216
channel-group 126 mode active
no shutdown
interface Ethernet1/1/4
description //Trunk 6554-2:28
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard enable
spanning-tree guard root
mtu 9216
channel-group 126 mode active
no shutdown
interface Ethernet1/5/1
description //Trunk A70-A
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard enable
spanning-tree guard root
mtu 9216
channel-group 131 mode active
no shutdown
interface Ethernet1/5/2
description //Trunk A70-A
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard enable
spanning-tree guard root
mtu 9216
channel-group 131 mode active
no shutdown
interface Ethernet1/5/3
description //Trunk A70-B
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard enable
spanning-tree guard root
mtu 9216
channel-group 132 mode active
no shutdown
interface Ethernet1/5/4
description //Trunk A70-B
switchport mode trunk
switchport access vlan 67
switchport trunk native vlan 66
switchport trunk allowed vlan 2-66,68-4094
spanning-tree port type edge trunk
spanning-tree bpduguard enable
spanning-tree guard root
mtu 9216
channel-group 132 mode active
no shutdown
!
! --- Bulk Disabled Ports ---
int e1/3/1-4,e1/7/1-4,e1/11/1-4,e1/13-23
description //Disabled access
switchport access vlan 67
switchport trunk native vlan 66
spanning-tree port type edge
spanning-tree bpduguard enable
spanning-tree guard root
storm-control broadcast level 99.00
storm-control unicast level 99.00
switchport block unicast
udld enable
shutdown
! --- Management Interface ---
interface mgmt0
vrf member management
ip address 192.168.0.1/24
icam monitor scale
! --- Console & VTY ---
line console
exec-timeout 5
line vty
session-limit 4
exec-timeout 0
access-class SWITCH_MGMT in
! --- Logging ---
logging ip access-list cache entries 8001
logging logfile LOG_FILE 6 size 4096
logging server 15.0.2.146 2
logging server 15.0.2.222 6
logging level authpri 6
intersight use-vrf Atom
```
---
## Configuration Explanation
### Platform & Global Settings
Running NX-OS 10.3(7) with a Jumbo MTU QoS policy (9216 bytes) applied globally via `system qos`. IP source-route is disabled. SSH is restricted to AES256-GCM ciphers. CoPP is set to strict for control-plane protection.
### VDC Resource Limits
Standard resource limits for a single-VDC 9300 — up to 4094 VLANs, 4096 VRFs, and 511 port-channels.
### Features Enabled
`nxapi`, `bash-shell`, `scp-server`, `udld`, `interface-vlan`, `lacp`, `vpc`, `lldp`, `telemetry`, and CFS Ethernet distribution for vPC.
### Authentication & Access Control
RADIUS authentication via two servers (15.0.11.68 and 15.32.11.68) in the `NETMAN_RADIUS` group, using the `Atom` VRF. AAA fallback is local. VTY access is restricted to the `SWITCH_MGMT` ACL (specific management host IPs only, with a deny-all default). VTY timeout is 0 (no timeout — note this differs from NEXUS-2 which uses 5 minutes).
### NTP
Two NTP servers in the Atom VRF (preferred) with MD5 authentication. NTP source is Vlan502. This switch acts as NTP master stratum 3.
### SNMP
SNMPv3 with SHA auth and AES-128 privacy. Traps sent to 15.0.2.188. RMON events configured for severity levels 15.
### VLANs
Approximately 200 VLANs are defined, covering storage (iSCSI, NFS, SnapMirror), compute (UCS, HLCI workloads), management, user, and VDI segments. VLAN 66 is the native VLAN; VLAN 67 is the unused/quarantine access VLAN for disabled ports.
### Spanning Tree
STP is configured with global edge/bpduguard and bpdufilter defaults for access ports, and network type for uplinks. This switch holds STP root priority 8192 for VLANs 1 and 66, making it the root for those VLANs. Most production VLANs are set to priority 24576 (secondary root). Unused VLANs are set to priority 0 (disabled from becoming root).
### VRF & Routing
A single non-default VRF `Atom` carries the management/user traffic with a default route to 15.0.2.254. Vlan502 (`Atom_User2`) is the L3 gateway SVI at 15.0.2.121/24.
### vPC Domain
- **Domain:** 1
- **Role Priority:** 10 (primary)
- **Peer-link:** Po10 (Eth1/4748), `spanning-tree port type network`
- **Peer-keepalive:** mgmt0, destination 192.168.0.2, source 192.168.0.1
- **Options:** `peer-switch`, `peer-gateway`, `auto-recovery`, 150-second restore delay
- **vPC members:** Po3 (500e-X1), Po4 (500e-X2), Po124 (9300), Po125 (UCS-A), Po126 (UCS-B), Po127 (AFF300-A), Po128 (AFF300-B), Po129 (FAS2750-A), Po130 (FAS2750-B), Po131 (A70-A), Po132 (A70-B)
### Port-Channel Load Balancing
`src-dst ip-l4port-vlan` — distributes traffic based on source/destination IP, L4 port, and VLAN for optimal flow distribution.
### Physical Interfaces
- **Ports 1/11/26:** Shut down as a group first, then individual interfaces are re-configured. Ports 1, 5, and 9 are broken out as 4x25G sub-interfaces.
- **Eth1/1/11/1/4:** 25G breakout ports to UCS 6554 FIs → Po125/Po126
- **Eth1/5/11/5/4:** 25G breakout ports to A70 storage arrays → Po131/Po132
- **Eth1/241/25, 1/451/46:** 9300 uplink → Po124 (4-link LACP)
- **Eth1/26:** 500e-X1 → Po3
- **Eth1/18:** 500e-X2 → Po4
- **Eth1/471/48:** vPC peer-link → Po10
- **Eth1/531/54:** AFF300-A/B → Po127/Po128
- **Eth1/21/3:** FAS2750 → Po129/Po130
- **Disabled ports:** Placed in VLAN 67, bpduguard enabled, storm-control, UDLD, unicast block — shutdown
### Logging
Syslog to 15.0.2.146 (severity 2) and 15.0.2.222 (severity 6). Local log file `LOG_FILE` at severity 6. ACL hit caching configured for 8001 entries.