diff --git a/Work/Nexus_1_Build.md b/Work/Nexus_1_Build.md deleted file mode 100644 index 881ef2e..0000000 --- a/Work/Nexus_1_Build.md +++ /dev/null @@ -1,715 +0,0 @@ ---- -title: C9300GX-1 Build -description: -published: true -date: 2026-02-19T20:46:00.149Z -tags: -editor: markdown -dateCreated: 2026-02-19T20:45:10.926Z ---- - -# AT1EU-NEXUS-1 — Cisco Nexus 9300 Configuration - -## Overview - -AT1EU-NEXUS-1 is the **primary** switch in a vPC pair (role priority 10, lower = preferred). It runs NX-OS 10.3(7) and forms a vPC domain with AT1EU-NEXUS-2. The two switches share a vPC peer-link (Po10) across Eth1/47–48, and use out-of-band management (mgmt0 at 192.168.0.1) for the vPC peer-keepalive path. - -**Key roles of this switch:** -- vPC primary (role priority 10) -- STP root bridge for management/native VLANs (priority 8192 for VLANs 1, 66) -- Layer 3 gateway for Vlan502 (Atom VRF, IP 15.0.2.121/24) -- NTP master (stratum 3) -- Upstream connections: 500e-X1 (Po3), 500e-X2 (Po4), 9300 (Po124) -- Storage connections: AFF300-A (Po127), AFF300-B (Po128), FAS2750-A (Po129), FAS2750-B (Po130), A70-A (Po131), A70-B (Po132) -- Compute connections: UCS-A (Po125), UCS-B (Po126) - ---- - -## Cut-and-Paste Configuration - -``` -conf t -switchname AT1EU-NEXUS-1 - -! --- QoS: Jumbo Frame Policy --- -policy-map type network-qos JUMBO - class type network-qos class-default - mtu 9216 - -! --- VDC Resource Limits --- -vdc AT1EU-NEXUS-1 id 1 - limit-resource vlan minimum 16 maximum 4094 - limit-resource vrf minimum 2 maximum 4096 - limit-resource port-channel minimum 0 maximum 511 - limit-resource m4route-mem minimum 58 maximum 58 - limit-resource m6route-mem minimum 8 maximum 8 - -! --- Features --- -feature nxapi -feature bash-shell -feature scp-server -cfs eth distribute -feature udld -feature interface-vlan -feature lacp -feature vpc -feature lldp -feature telemetry - -! --- RBAC --- -role name network-ro - rule 2 permit read - rule 1 permit command show running-config - -! --- Users --- -username admin password 5 $5$MFJCIC$AJyskD7vdoVFKK5cTS2lO20omFL4XFrgqNB94qDA5Z2 role network-admin -ssh key rsa 2048 - -! --- Banner --- -banner motd ^ -********************* DOD NOTICE AND CONSENT BANNER ************************* -* You are accessing a U.S. Government (USG) Information System (IS) that is * -* provided for USG-authorized use only. By using this IS (which includes any* -* device attached to this IS), you consent to the following conditions: * -*-The USG routinely intercepts and monitors communications on this IS for * -* purposes including, but not limited to, penetration testing, COMSEC * -* monitoring, network operations and defense, personnel misconduct (PM), * -* law enforcement (LE), and counterintelligence (CI) investigations. * -*-At any time, the USG may inspect and seize data stored on this IS. * -*-Communications using, or data stored on, this IS are not private, are * -* subject to routine monitoring, interception, and search, and may be * -* disclosed or used for any USGauthorized purpose. * -*-This IS includes security measures (e.g., authentication and access * -* controls) to protect USG interests--not for your personal benefit or * -* privacy. * -*-Notwithstanding the above, using this IS does not constitute consent to * -* PM, LE or CI investigative searching or monitoring of the content of * -* privileged communications, or work product, related to personal * -* representation or services by attorneys, psychotherapists, or clergy, and * -* their assistants. Such communications and work product are private and * -* confidential. See User Agreement for details. * -************************ POC: SIL Network Team **************************** -^ - -! --- SSH --- -ssh ciphers aes256-gcm - -! --- DNS & Domain --- -ip domain-lookup -ip name-server 15.0.2.128 15.0.2.129 15.32.2.128 -ip domain-name atom.dev use-vrf Atom -ip name-server 15.0.2.128 15.0.2.129 15.32.2.128 use-vrf Atom - -! --- RADIUS --- -radius-server host 15.0.11.68 key 7 "V1P-jaynmv" authentication accounting -radius-server host 15.32.11.68 key 7 "V1P-jaynmv" authentication accounting -aaa group server radius NETMAN_RADIUS - server 15.0.11.68 - server 15.32.11.68 - use-vrf Atom - -! --- Management ACL --- -ip access-list SWITCH_MGMT - 10 permit ip 15.0.11.150/32 any log - 20 permit ip 15.0.11.151/32 any log - 30 permit ip 15.32.2.154/32 any log - 40 permit ip 15.0.2.154/32 any log - 50 permit ip 15.32.2.1/32 any log - 60 permit ip 15.0.2.1/32 any log - 70 permit ip 15.0.2.2/32 any log - 80 permit ip 15.0.11.47/32 any log - 90 permit ip 15.32.11.45/32 any log - 93 permit ip 15.32.11.150/32 any log - 100 deny ip any any log - -! --- System QoS --- -system qos - service-policy type network-qos JUMBO -copp profile strict - -! --- SNMP --- -snmp-server user admin network-admin auth sha 042F64DB5D2E0D40DF543D6A00495F1F18F9DD5FED7B priv aes-128 00540CF9793F282ED96D666B110B00753FC3F269E964 localizedV2key -snmp-server host 15.0.2.188 traps version 3 priv at-sw-svc -snmp-server enable traps config ccmCLIRunningConfigChanged -rmon event 1 log trap public description FATAL(1) owner PMON@FATAL -rmon event 2 log trap public description CRITICAL(2) owner PMON@CRITICAL -rmon event 3 log trap public description ERROR(3) owner PMON@ERROR -rmon event 4 log trap public description WARNING(4) owner PMON@WARNING -rmon event 5 log trap public description INFORMATION(5) owner PMON@INFO - -! --- NTP --- -ntp server 15.0.0.9 prefer use-vrf Atom key 123 -ntp server 15.32.0.9 prefer use-vrf Atom key 125 -ntp source-interface Vlan502 -ntp authenticate -ntp authentication-key 123 md5 pz5yamz 7 -ntp trusted-key 123 -ntp logging -ntp master 3 - -! --- AAA --- -aaa authentication login default group NETMAN_RADIUS local -aaa authentication login console group NETMAN_RADIUS local -aaa accounting default group NETMAN_RADIUS local -system default switchport -no ip source-route - -! --- VLANs --- -vlan 1-2,8,10,12,66,85,100-103,107-108,121-124,129-130,142-143,145-146,148-150,153,157-158,188,305,321,323,340,342,349,353,374,382,501-502,504-505,549,551,559,562-563,600,611,660-661,667-668,672-673,697-698,701-702,704-710,720-722,724,727,740,750-751,772,777,800-802,804,814,820-823,905,1051,1127,1129,1160-1161,1551,1559-1560,1670-1674,1720-1722,1800-1802,1814-1817,1862,1865,1870-1871 -vlan 1882-1883,1885,1905,3563,3965 -vlan 2 - name TEST_CLUS_COMM -vlan 8 - name FP_Test1 -vlan 10 - name NESS_BOX_TRANSIT -vlan 12 - name FP_Test2 -vlan 66 - name NATIVE_VLAN -vlan 85 - name NESS_Temp -vlan 100 - name migration -vlan 101 - name iscsi_csv -vlan 102 - name iscsi_boot -vlan 103 - name Netapp_XFER -vlan 107 - name Test -vlan 108 - name NET_TEST_NET -vlan 121 - name Atom_Backup -vlan 123 - name storage -vlan 124 - name Admin_iSCSI -vlan 130 - name SIL_SNAPMIRROR -vlan 143 - name Secman_Storage -vlan 146 - name Foxhound_Storage -vlan 150 - name iscsi -vlan 153 - name Javelin(L4) -vlan 157 - name GNext_Storage -vlan 158 - name Ness_Storage -vlan 188 - name JASON_NFS -vlan 321 - name ATOM_Backup -vlan 323 - name AT-vServer -vlan 340 - name ucs_test -vlan 342 - name MadHatter_SVM_Mgmt -vlan 349 - name Rock_SVM3_Mgmt -vlan 353 - name Javlin_SVM -vlan 374 - name Rock_Backup_Mgmt -vlan 382 - name Darrin_User -vlan 501 - name MGMT -vlan 502 - name Atom_User2 -vlan 504 - name Commvault_Test -vlan 505 - name NETAPP_SNAP -vlan 549 - name WDS -vlan 551 - name L4_User -vlan 559 - name Victory_WS_L4 -vlan 562 - name Brace(L3)_User -vlan 667 - name Britt_Test -vlan 668 - name RockTesters(L4)_User -vlan 672 - name GTRI_User -vlan 673 - name VDI(L5) -vlan 701 - name MH_L3_DATA_HLCI -vlan 702 - name MH_L4_DATA_HLCI -vlan 704 - name Legacy-704 -vlan 705 - name Legacy-705 -vlan 706 - name Legacy-706 -vlan 707 - name Legacy-707 -vlan 708 - name Legacy-708 -vlan 709 - name Legacy-709 -vlan 710 - name Legacy-710 -vlan 721 - name GTRI_JAVELIN_L4-721 -vlan 740 - name NETMAN -vlan 750 - name l4_secman -vlan 751 - name Secman_DMP-751 -vlan 777 - name FTD1010_TSHOOT -vlan 804 - name FH_L4_HLCI -vlan 814 - name Rock_L4 -vlan 820 - name GNext_User -vlan 821 - name GNext_Sentris -vlan 822 - name GNext_VPX -vlan 823 - name GNext_VDA -vlan 905 - name Rock_(L4) -vlan 1051 - name IP_SEC_1010 -vlan 1127 - name Vic_Storage -vlan 1551 - name Services(L3)_User -vlan 1559 - name Victory(L3)_User -vlan 1670 - name BigTen_User -vlan 1671 - name Victory_DMP-1671 -vlan 1672 - name VIC_VDI -vlan 1673 - name Victory_Sentris -vlan 1720 - name Javelin(L3)_User -vlan 1721 - name GTRI_JAVELIN_L3-1721 -vlan 1722 - name Victory_VDI-1722 -vlan 1800 - name Foxhound(L3)_User -vlan 1801 - name FH_L3_DATA_HLCI -vlan 1814 - name ROCK_L3_MLS -vlan 1815 - name ServMan_User -vlan 1870 - name AT1EU-JavelinCoop(L3)_User -vlan 1883 - name NESS_User -vlan 1885 - name NESS_Client -vlan 1905 - name Rock(L3)_User -vlan 3563 - name Brace_User -vlan 3965 - name V3E_DEV_HOST - -! --- Spanning Tree --- -spanning-tree port type edge bpduguard default -spanning-tree port type edge bpdufilter default -spanning-tree port type network default -spanning-tree vlan 1,66 priority 8192 -spanning-tree vlan 2,100-102,107-108,121-123,129,142,145,148-150,153,305,323,340,353,382,501-502,505,549,551,562-563,600,611,660-661,667-668,672,697-698,701-702,704-710,720-722,724,727,750,772,800-802,804,814,905,1127,1129,1160-1161,1551,1559-1560,1670,1672-1673,1720-1721,1800-1802,1814-1817,1862,1865,1870-1871,1882,1905,3563,3965 priority 24576 -spanning-tree vlan 3-65,67-99,103-106,109-120,124-128,130-141,143-144,146-147,151-152,154-304,306-322,324-339,341-352,354-381,383-500,503-504,506-548,550,552-561,564-599,601-610,612-659,662-666,669-671,673-696,699-700,703,711-719,723,725-726,728-749,751-771,773-799,803,805-813,815-904,906-1126,1128,1130-1159,1162-1550,1552-1558,1561-1669,1671,1674-1719,1722-1799,1803-1813,1818-1861,1863-1864,1866-1869,1872-1881,1884-1904,1906-3562,3564-3964,3966-3967 priority 0 -spanning-tree vlan 1883 priority 4096 - -! --- VRF --- -vrf context Atom - ip domain-name atom.dev - ip name-server 15.0.2.128 15.0.2.129 15.32.2.128 - ip route 0.0.0.0/0 15.0.2.254 -vrf context management - -! --- Port-Channel Load Balance --- -port-channel load-balance src-dst ip-l4port-vlan - -! --- vPC Domain --- -vpc domain 1 - peer-switch - role priority 10 - peer-keepalive destination 192.168.0.2 source 192.168.0.1 - delay restore 150 - peer-gateway - auto-recovery - -! --- SVI --- -interface Vlan1 - -interface Vlan502 - no shutdown - vrf member Atom - no ip redirects - ip address 15.0.2.121/24 - no ipv6 redirects - -! --- Port-Channels --- -interface port-channel3 - description //Trunk 500e X1 - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard enable - spanning-tree guard root - mtu 9216 - vpc 3 - -interface port-channel10 - description //Trunk Peer - Allow STP - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type network - vpc peer-link - -interface port-channel124 - description //Trunk 9300 - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type normal - spanning-tree bpduguard disable - spanning-tree guard root - mtu 9216 - no lacp suspend-individual - vpc 124 - -interface port-channel125 - description //Trunk UCS-A - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard disable - spanning-tree guard root - mtu 9216 - vpc 125 - -interface port-channel126 - description //Trunk UCS-B - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard disable - spanning-tree guard root - mtu 9216 - vpc 126 - -interface port-channel127 - description //Trunk AFF300-A - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree guard root - mtu 9216 - vpc 127 - -interface port-channel128 - description //Trunk AFF300-B - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree guard root - mtu 9216 - vpc 128 - -interface port-channel129 - description //Trunk FAS 2750-A - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard enable - spanning-tree guard root - mtu 9216 - storm-control broadcast level 99.00 - storm-control unicast level 99.00 - switchport block unicast - vpc 129 - -interface port-channel130 - description //Trunk Fas 2750-B - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard enable - spanning-tree guard root - mtu 9216 - storm-control broadcast level 99.00 - storm-control unicast level 99.00 - switchport block unicast - vpc 130 - -interface port-channel131 - description //Trunk A70-A - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree guard root - mtu 9216 - vpc 131 - -interface port-channel132 - description //Trunk A70-B - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree guard root - mtu 9216 - vpc 132 - -! --- Breakout Ports (100G -> 4x25G) --- -int e1/1 - 26 - shutdown -exit -interface breakout module 1 port 1 map 25g-4x -interface breakout module 1 port 5 map 25g-4x - - -! --- Physical Interfaces: Breakout (UCS/A70) --- -interface Ethernet1/1/1 - description //Trunk 6554-1:25 - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard enable - spanning-tree guard root - mtu 9216 - channel-group 125 mode active - no shutdown - -interface Ethernet1/1/2 - description //Trunk 6554-1:26 - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard enable - spanning-tree guard root - mtu 9216 - channel-group 125 mode active - no shutdown - -interface Ethernet1/1/3 - description //Trunk 6554-2:27 - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard enable - spanning-tree guard root - mtu 9216 - channel-group 126 mode active - no shutdown - -interface Ethernet1/1/4 - description //Trunk 6554-2:28 - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard enable - spanning-tree guard root - mtu 9216 - channel-group 126 mode active - no shutdown - -interface Ethernet1/5/1 - description //Trunk A70-A - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard enable - spanning-tree guard root - mtu 9216 - channel-group 131 mode active - no shutdown - -interface Ethernet1/5/2 - description //Trunk A70-A - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard enable - spanning-tree guard root - mtu 9216 - channel-group 131 mode active - no shutdown - -interface Ethernet1/5/3 - description //Trunk A70-B - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard enable - spanning-tree guard root - mtu 9216 - channel-group 132 mode active - no shutdown - -interface Ethernet1/5/4 - description //Trunk A70-B - switchport mode trunk - switchport access vlan 67 - switchport trunk native vlan 66 - switchport trunk allowed vlan 2-66,68-4094 - spanning-tree port type edge trunk - spanning-tree bpduguard enable - spanning-tree guard root - mtu 9216 - channel-group 132 mode active - no shutdown - - - -! -! --- Bulk Disabled Ports --- -int e1/3/1-4,e1/7/1-4,e1/11/1-4,e1/13-23 - description //Disabled access - switchport access vlan 67 - switchport trunk native vlan 66 - spanning-tree port type edge - spanning-tree bpduguard enable - spanning-tree guard root - storm-control broadcast level 99.00 - storm-control unicast level 99.00 - switchport block unicast - udld enable - shutdown - -! --- Management Interface --- -interface mgmt0 - vrf member management - ip address 192.168.0.1/24 - -icam monitor scale - -! --- Console & VTY --- -line console - exec-timeout 5 -line vty - session-limit 4 - exec-timeout 0 - access-class SWITCH_MGMT in - - -! --- Logging --- -logging ip access-list cache entries 8001 -logging logfile LOG_FILE 6 size 4096 -logging server 15.0.2.146 2 -logging server 15.0.2.222 6 -logging level authpri 6 - -intersight use-vrf Atom -``` - ---- - -## Configuration Explanation - -### Platform & Global Settings -Running NX-OS 10.3(7) with a Jumbo MTU QoS policy (9216 bytes) applied globally via `system qos`. IP source-route is disabled. SSH is restricted to AES256-GCM ciphers. CoPP is set to strict for control-plane protection. - -### VDC Resource Limits -Standard resource limits for a single-VDC 9300 — up to 4094 VLANs, 4096 VRFs, and 511 port-channels. - -### Features Enabled -`nxapi`, `bash-shell`, `scp-server`, `udld`, `interface-vlan`, `lacp`, `vpc`, `lldp`, `telemetry`, and CFS Ethernet distribution for vPC. - -### Authentication & Access Control -RADIUS authentication via two servers (15.0.11.68 and 15.32.11.68) in the `NETMAN_RADIUS` group, using the `Atom` VRF. AAA fallback is local. VTY access is restricted to the `SWITCH_MGMT` ACL (specific management host IPs only, with a deny-all default). VTY timeout is 0 (no timeout — note this differs from NEXUS-2 which uses 5 minutes). - -### NTP -Two NTP servers in the Atom VRF (preferred) with MD5 authentication. NTP source is Vlan502. This switch acts as NTP master stratum 3. - -### SNMP -SNMPv3 with SHA auth and AES-128 privacy. Traps sent to 15.0.2.188. RMON events configured for severity levels 1–5. - -### VLANs -Approximately 200 VLANs are defined, covering storage (iSCSI, NFS, SnapMirror), compute (UCS, HLCI workloads), management, user, and VDI segments. VLAN 66 is the native VLAN; VLAN 67 is the unused/quarantine access VLAN for disabled ports. - -### Spanning Tree -STP is configured with global edge/bpduguard and bpdufilter defaults for access ports, and network type for uplinks. This switch holds STP root priority 8192 for VLANs 1 and 66, making it the root for those VLANs. Most production VLANs are set to priority 24576 (secondary root). Unused VLANs are set to priority 0 (disabled from becoming root). - -### VRF & Routing -A single non-default VRF `Atom` carries the management/user traffic with a default route to 15.0.2.254. Vlan502 (`Atom_User2`) is the L3 gateway SVI at 15.0.2.121/24. - -### vPC Domain -- **Domain:** 1 -- **Role Priority:** 10 (primary) -- **Peer-link:** Po10 (Eth1/47–48), `spanning-tree port type network` -- **Peer-keepalive:** mgmt0, destination 192.168.0.2, source 192.168.0.1 -- **Options:** `peer-switch`, `peer-gateway`, `auto-recovery`, 150-second restore delay -- **vPC members:** Po3 (500e-X1), Po4 (500e-X2), Po124 (9300), Po125 (UCS-A), Po126 (UCS-B), Po127 (AFF300-A), Po128 (AFF300-B), Po129 (FAS2750-A), Po130 (FAS2750-B), Po131 (A70-A), Po132 (A70-B) - -### Port-Channel Load Balancing -`src-dst ip-l4port-vlan` — distributes traffic based on source/destination IP, L4 port, and VLAN for optimal flow distribution. - -### Physical Interfaces -- **Ports 1/1–1/26:** Shut down as a group first, then individual interfaces are re-configured. Ports 1, 5, and 9 are broken out as 4x25G sub-interfaces. -- **Eth1/1/1–1/1/4:** 25G breakout ports to UCS 6554 FIs → Po125/Po126 -- **Eth1/5/1–1/5/4:** 25G breakout ports to A70 storage arrays → Po131/Po132 -- **Eth1/24–1/25, 1/45–1/46:** 9300 uplink → Po124 (4-link LACP) -- **Eth1/26:** 500e-X1 → Po3 -- **Eth1/18:** 500e-X2 → Po4 -- **Eth1/47–1/48:** vPC peer-link → Po10 -- **Eth1/53–1/54:** AFF300-A/B → Po127/Po128 -- **Eth1/2–1/3:** FAS2750 → Po129/Po130 -- **Disabled ports:** Placed in VLAN 67, bpduguard enabled, storm-control, UDLD, unicast block — shutdown - - -### Logging -Syslog to 15.0.2.146 (severity 2) and 15.0.2.222 (severity 6). Local log file `LOG_FILE` at severity 6. ACL hit caching configured for 8001 entries. -