Data centre

Rack row architecture

A row of racks whose top-of-rack switches serve the servers and dual-home to an aggregation pair on two separate power feeds.

Rack R1Rack R2Rack R3tor1 ↔ agg1tor1 ↔ agg2tor2 ↔ agg1tor2 ↔ agg2tor3 ↔ agg1tor3 ↔ agg2tor1 ↔ srv1tor2 ↔ srv2tor3 ↔ srv3pduA ↔ srv1pduA ↔ srv2pduA ↔ srv3pduB ↔ srv1pduB ↔ srv2pduB ↔ srv3Aggregation switch A10.20.0.1Aggregation switch A — 10.20.0.1Aggregation switch B10.20.0.2Aggregation switch B — 10.20.0.2Top-of-rack switch R110.20.1.1Top-of-rack switch R1 — 10.20.1.1Top-of-rack switch R210.20.1.2Top-of-rack switch R2 — 10.20.1.2Top-of-rack switch R310.20.1.3Top-of-rack switch R3 — 10.20.1.3Rack R1 servers10.20.10.0/24Rack R1 servers — 10.20.10.0/24Rack R2 servers10.20.11.0/24Rack R2 servers — 10.20.11.0/24Rack R3 servers10.20.12.0/24Rack R3 servers — 10.20.12.0/24PDU A — feed APDU A — feed APDU B — feed BPDU B — feed B
Legend

Current step

Rack row and top-of-rack switches

Each cabinet holds servers that patch into the top-of-rack switch mounted at the top of the rack.

Airflow
Front to back, contained hot aisle
Rack units
42 U, 12 servers per rack
Every server draws power from feed A and feed B at the same time, and every top-of-rack switch uplinks to both aggregation switches, so a single power supply or a single uplink can fail without taking the rack offline. Air flows front to back into a contained hot aisle.

Documentation example: addresses use the RFC 5737 ranges and ASNs use the reserved private range. Nothing here connects to a real network.

Tags

  • datacenter
  • rack
  • top-of-rack
  • power
  • redundancy

Lessons using this diagram