Common Datacenter Specifications
This page will outline our preferred specifications for use in datacenters.
Overall Notes
- We prefer to provide our own enclosures, as well as everything within those enclosures (including power strips).
- We prefer a caged deployment over a shared row deployment.
Cabling Specifications
| Color | Cable Spec | Network |
|---|---|---|
| Blue | Cat6 | Production |
| Green | Cat5e | Management |
| Orange | Cat5e | Serial |
| Red | Cat5e | Out of Band Access |
| Yellow | 9/125 Singlemode Fiber | All fiber connections |
- Fiber note: LC to LC in rack, SC to LC for peer/transit links.
- Cabling Notes:
- We use pre-made patches in rack (5'/7'/10') for mgmt and production network.
- We use custom on-site assembled patches for serial.
- These are nearly always cross-rack and must be custom measured.
- The pin out differs to work for serial console (cisco/juniper/netapp/sun) without an adapter:
- End Device Side: 568B
- Serial Console Side: pin 4 white-green, pin 5 green, pin 6 blue (remainder immaterial)
Cabling Options
- Gear Mounted Top of Rack
- 12 3M Fiber or 12 10' CAT6 Ethernet Cable
- 24 2M Fiber or 24 7' CAT6 Ethernet Cable
- 5 1M Fiber or 5 5' CAT6 Ethernet Cable
- Gear Mounted Mid-Rack
- 10 1M Fiber or 10 5' CAT6 Ethernet Cable
- 32 2M Fiber or 32 7' CAT6 Ethernet Cable
Power Specifications
US Based Sites
- 208V 3 Phase power @ 30 Amps - Two circuits per enclosure for redundancy.
- Sites in use: eqiad, sdtpa (not all racks redundant power)
- Power connector used for 30AMP 3Phase is NEMA L21-30P(plug) / L21-30R(eceptacle).
- Sites not in use: pmtpa (208V 3 Phase @ 20 Amps, redundant) & ULSFO(redundant).
- Power connector used for 20AMP 3Phase is NEMA NEMA L21-20P(plug) / L21-20R(eceptacle).
- ServerTech 0U SMART PDU
- 42 c13 power ports per feed for a total of 84 ports.
- Model: CS-84VDY-L2130
- Also have to order (2) EMTH-1-1 to have two temp/humidity leads, and one Kit-MB-01 (two button mount kit - can use Rittal equivalent)
- In our present deployments, we use SMART over SWITCHED due to the higher possible rate of failure of the switched power ports over non-switched power ports.
- We have a rack with switched power, due to legacy equipment that could not out of band power cycle; though now none of that hardware is still in service.
- RobH thinks we may need to re-investigate the failure rates on switched power, and possibly use it more, since it can be used to remotely clear service processors out of unrecoverable error states,
- Power cords for servers are c13/c14 type.
Enclosure Specifications
- Rittal TS-8 Enclosure
- Rittal 9962172
- Sheet Steel, 800MM W by 2200MM H by 1050MM D. Sand Texture, Black, Perforated front/rear doors, standard rear wall/door, perforated with 4 round cable entry in roof.
- We have to get three kinds, right side end of row, left side end of row, and enclosures within the row.
- Inner enclosure: partition wall left side
- Right end enclosure: partition left, sidewall right
- Left end enclosure: sidewall left
- Rittal TS - External Baying Connector
- Rittal 8800490
- One kit gangs together two enclosures
- Rittal Roof Mount Cable Manager for 32" Wide TS Enclosure
- Rittal 9970016
- Rittal Cable D Rings
- Rittal 7111900
- 4.7" x 3.5", pack of 10
- Rittal Button Mount Brackets
- Rittal 9971067
Network Specifications
EQIAD
- Core Routers
- Juniper MX80 (Caching Sites)
- Juniper MX480 (Primary Sites)
- Out of Band Access
- Juniper SRX100B
- Central Management Switch
- Juniper EX4200-48T
- Rack-level Access Switches
- Juniper EX4200-48T - for non-10G racks
- Juniper EX4550 - for 10G racks
- Juniper sfp+ Uplink Modules for EX4200
- Rack-level Management Switches
- Central mgmt switch handles one connection to each rack, rack mgmt switch can be stupid.
- Netgear FS750T2 - 48 port switch
- We use as a dumb switch, we don't configure anything.
- Transceivers
- XFP: Finisar FTLX1412M3BCL
- SFP+: Finisar FTLX1471D3BCL
- SFP-T: Memorydealers SFP-1GE-FE-E-T-MD
Serial Console
- Opengear CM4148
- 48 port serial console server
- We attach serial to every power strip and every bit of network kit (excluding the rack level mgmt switches)
Cameras
- Axis P1344 IP Camera