The PowerEdge R740 servers represent a great performance and storage base in a 2U box. The server can be configured with up to 2 Scalable Intel CPUs and 24 DDR4 DIMMs (or 12 NVDIMM), but where they really shine is in the way they approach storage. While the R740 offers up to 16 storage bays, the xd offers up to 32 2.5 "bays, 24 of which can be NVMe.The R740xd also offers some unique storage arrangements over typical front-loading bays, including bays of central and rear loading to fit all the additional storage space in the same space of 2U.The design allows users to adapt their storage needs to their application by being able to mix NVMe, SSD and HDD in the same chassis, creating levels storage within the chassis The R740xd also supports up to 192 GB of NVDIMM In addition, the R740xd has the ability to boot from internal M.2 SSDs RAID through an additional card, freeing up the most accessible space for load storage Both versions are good for SDS, service providers and VDI, with total storage and NVMe is the key difference, it is also new in the R740 / R740xd the biggest support for GPU or FPGA. Both are compatible with up to three 300W cards or six 150W cards. In this generation, Dell EMC designed the BIOS to automatically record the airflow required by each card and provide an individually adapted airflow through a function called multi-vector cooling.
With every update of any server line, there are new CPUs, more RAM and better storage options and networks. What differentiates many companies, however, is the complete management of the product's life cycle. Within reason, any server with the same hardware specifications will have roughly the same score. The difference, however, quickly becomes apparent with the quality of the hardware, the breadth of the support software and how easy it is for the system to be deployed quickly in a given environment. This is a key area in which Dell EMC stands apart from others in the market. Dell EMC offers users key tools such as LifeCycle Controller, iDRAC, OpenManage Mobile and others. We have taken advantage of many of these tools in our own environment, and again and again, we have been impressed by how simple and mature the platform has become over time.
The new PowerEdge servers have support for software defined storage (SDS) integrated from the beginning, which allows them to use cases such as hyperconverged infrastructure. In its own line of business products, Dell EMC leverages the R740 with pre-built and validated solutions, such as Ready Nodes for ScaleIO or VSAN, as well as the PowerEdge XC line. The R740xd allows configurations that take advantage of all external drive bays for the SDS product itself, keeping the boot segment in an internal m.2 SSD.
The new Dell EMC PowerEdge R740xd is available now and is highly customizable. For this review, we leveraged an individual R740xd with a near top-end configuration, as well as a cluster of 12 R740xds with a more modest configuration.
The single R740xd we are using is built with the following:
Dual Intel Xeon Platinum 8180 CPUs
384GB of DDR4 2667MHz RAM (32GB x 12)
4 x 400GB SAS SSDs
2 x 1.6TB NVMe SSDs
Mellanox ConnectX-4 Lx Dual Port 25GbE DA/SFP rNDC
LCD bezel with Quick Sync 2 and OpenManage capabilities
iDRAC 9 Enterprise
Dell EMC PowerEdge R740xd Server specifications:
Form factor: 2U Rackmount
Processors: up to 2 Intel Scalable CPUs or up to 28 cores
Memory: 24x DDR4 RDIMM, LR-DIMM (3TB max)
NVDIMM support: up to 12 or 192GB
Drive Bays
Front bays:
Up to 24 x 2.5” SAS/SSD/NVMe, max 153TB
Up to 12 x 3.5” SAS, max 120TB
Mid bay:
Up to 4 x 3.5” drives, max 40TB
Up to 4 x 2.5” SAS/SSD/NVMe, max 25TB
Rear bays:
Up to 4 x 2.5” max 25TB
Up to 2 x 3.5” max 20TB
Storage Controllers
Internal controllers: PERC H730p, H740p, HBA330, Software RAID (SWRAID) S140
Boot Optimized Storage Subsystem: HWRAID 2 x M.2 SSDs 120GB, 240 GB
External PERC (RAID): H840
External HBAs (non-RAID): 12 Gbps SAS HBA
Ports
Network daughter card options: 4 x 1GE or 2 x 10GE + 2 x 1GE or 4 x 10GE or 2 x 25GE
Front ports: VGA, 2 x USB 2.0, dedicated IDRAC Direct Micro-USB
Rear ports: VGA, Serial, 2 x USB 3.0, dedicated iDRAC network port
Video card:
VGA
Up to 8 x Gen3 slots, Up to 4 x16
GPU Options:
Nvidia Tesla P100, K80, K40, Grid M60, M10, P4, Quadro P4000.
AMD S7150, S7150X2
Supported OS
Canonical Ubuntu LTS
Citrix XenServer
Microsoft Windows Server with Hyper-V
Red Hat Enterprise Linux
SUSE Linux Enterprise Server
VMware ESXi
Power
Titanium 750W, Platinum 495W, 750W, 1100W,
1600W, and 2000W
48VDC 1100W, 380HVDC 1100W, 240HVDC 750W
Hot plug power supplies with full redundancy
Up to 6 hot plug fans with full-redundancy, high-performance fans available
Design and Build
The new PowerEdge Servers have been redesigned not only to make them look flawless (what they really do), but also to reflect how users and applications interact with them. On the other side of the front is the new bezel that has Quick Sync support with its OpenManage wireless capabilities. The same design in the new servers is also combined with the new Dell EMC storage offerings, including systems such as the Unity 450F all-flash array. Below the bezel, there are 24 2.5 "bays that support SATA, SAS, Nearline SAS and NVMe (if configured to do so).
Where others in the market are finding ways to reduce costs and eliminate components in favor of cost reduction, an item that Dell EMC has maintained as an option for the R740xd and the R740 is its front panel. Some might be saying "Who cares?" But the small LCD screen and its three buttons are incredibly useful in a data center environment. For example, in a scenario where you can not access iDRAC remotely, the configuration of the management network has changed and you do not want to turn the server on and off to start manually with a cart and keyboard, the front panel is very useful. On a Dell EMC server, you can go through the small interface for the iDRAC configuration, and you can change the IP address to DHCP from static through the front panel. Without that functionality still in place, on many systems you would need to restart it to manually change it. On the R740xd, this is completely out of band through different controls.
Taking the top cover off shows the inner workings and the massive attention to detail that Dell EMC has put into the new PowerEdge servers. Many of the server components can be swapped out easily if the need arises, and clutter is kept to a minimum to improve airflow. In the system we reviewed, you are able to see the dual-slot m.2 boot SSD card, two RAID cards, as well as two PCIe pass-through adapters for the NVMe slots in front.
The rNDC slot is leveraged for the onboard primary network interface. This can be pre-populated with a number of offerings, ranging from a quad-port 1GbE NIC up to dual-port 25Gb offerings from both Mellanox and Broadcom. None of the options take away from one of the server's available PCIe slots, keeping them completely open for other uses. As we've shown in our rNDC upgrade guide, this bay is easy to upgrade and quite helpful at keeping networking devices out of the main PCIe slots.
Management
The PowerEdge R740xd offers a wide range of management options, including some traditional, as well as others that fit in the palm of your hand. The R740xd can be deployed by leveraging Dell EMC’s OpenManage Mobile app or locally like previous generation servers. The abilities of OpenManage Mobile can really make a difference, especially when you are setting up several servers in one data center, or you just want to get it finished on the floor without going back and forth to your desk or bringing a crash cart. Leveraging pre-built profiles to rapidly deploy a server with nothing more than a cell phone dramatically speeds up a process that frequently requires a crash cart in a datacenter.
This blends in a mix of handheld access for quick status checks or system polling, or more advanced functionality and iKVM work without worrying about connecting any wires or crash carts. The very short range (5 to 10 feet from the server in a datacenter environment) also helps to minimize the chance of anyone hopping onto the system without noticing. When your work is completed, turning off the wireless radio disables any further access.
Performance
When comparing the R740xd to the prior generation systems, compute and storage potential have skyrocketed. With Intel Broadwell updates, the top-spec CPU offered in the R730 series (E5-2699v4) offers 96.8GHz in a dual processor configuration. With the Intel Scalable line inside the PowerEdge R740xd, though, the top-end CPU (Platinum 8180) pushes that number to 139.66GHz. At face value, that's a 44% jump, but it doesn't even start to look at improvements in clockspeed at those higher core counts or DRAM-clockspeed improvements. On the storage side, NVMe SSDs have also taken on a bigger role inside the R740xd configurations, with offerings now topping out at 24 NVMe SSDs, where four used to be the peak on the R730xd.
As we look at the improvements made on the latest generation Dell EMC PowerEdge server, we will touch on both local performance, as well as clustered performance across a group of eight servers leveraging storage from a Dell EMC Unity 450F All-Flash array in a later review. This review layout is geared to help interested buyers see how these servers perform well equipped in single instances, as well as how they interact in a highly virtualized environment within a Dell EMC ecosystem. Bringing together all these systems are Mellanox ConnectX-4 25Gb rNDC NICs, as well as Dell EMC Networking Z9100 100G switches.
In our section looking at local system performance, we have a well-equipped R740xd which we are testing with two different NVMe combos. One is with two Samsung 1.6TB PM1725a NVMe SSDs, while the second is using four Toshiba 1.6TB PX04P NVMe SSDs. With the Intel Platinum 8180 CPUs inside, we had plenty of CPU cycles to throw at our storage workloads, giving us a chance to show the difference moving from two to four NVMe SSDs within the same application workload. Additionally, we also push storage to the brink inside an ESXi 6.5 environment with a multiple-worker vdbench test, with multiple workloads geared at simulating basic four-corners testing up to VDI traces.
Sysbench MySQL Performance
Our first local-storage application benchmark consists of a Percona MySQL OLTP database measured via SysBench. This test measures average TPS (Transactions Per Second), average latency, and average 99th percentile latency as well.
Each Sysbench VM is configured with three vDisks: one for boot (~92GB), one with the pre-built database (~447GB), and the third for the database under test (270GB). From a system resource perspective, we configured each VM with 16 vCPUs, 60GB of DRAM and leveraged the LSI Logic SAS SCSI controller.
Sysbench Testing Configuration (per VM)
CentOS 6.3 64-bit
Percona XtraDB 5.5.30-rel30.1
Database Tables: 100
Database Size: 10,000,000
Database Threads: 32
RAM Buffer: 24GB
Test Length: 3 hours
2 hours preconditioning 32 threads
1 hour 32 threads
We compared the performance of two Sysbench runs on the PowerEdge R740xd, one with 4VMs hosted on two NVMe SSDs and another with 4VMs with a dedicated NVMe SSD for each VM. In both of these tests, the CPU load wasn't brought to the breaking point of 100%. We saw a split of about 60% and 80% CPU utilization for the two benchmarks, meaning there was still room to grow with additional VMs and more DRAM. The first with two NVMe SSDs hosting the Sysbench VMs, aggregate TPS came to 11,027, and in the second test with four NVMe SSDs, the aggregate TPS increased to 13,224. This contrasts to a measurement of 10,683TPS from the PowerEdge R630 we benchmarked about a year ago with E5-2699v4 CPUs and four NVMe SSDs as well.
SQL Server Performance
StorageReview’s Microsoft SQL Server OLTP testing protocol employs the current draft of the Transaction Processing Performance Council’s Benchmark C (TPC-C), an online transaction processing benchmark that simulates the activities found in complex application environments. The TPC-C benchmark comes closer than synthetic performance benchmarks to gauging the performance strengths and bottlenecks of storage infrastructure in database environments.
Each SQL Server VM is configured with two vDisks: 100GB volume for boot and a 500GB volume for the database and log files. From a system resource perspective, we configured each VM with 16 vCPUs, 64GB of DRAM and leveraged the LSI Logic SAS SCSI controller. While our Sysbench workloads tested previously saturated the platform in both storage I/O and capacity, the SQL test looks for latency performance.
This test uses SQL Server 2014 running on Windows Server 2012 R2 guest VMs, and is stressed by Dell's Benchmark Factory for Databases. While our traditional usage of this benchmark has been to test large 3,000-scale databases on local or shared storage, in this iteration we focus on spreading out four 1,500-scale databases evenly across our servers.
SQL Server Testing Configuration (per VM)
Windows Server 2012 R2
Storage Footprint: 600GB allocated, 500GB used
SQL Server 2014
Database Size: 1,500 scale
Virtual Client Load: 15,000
RAM Buffer: 48GB
Test Length: 3 hours
2.5 hours preconditioning
30 minutes sample period
Similar to how we ran our Sysbench benchmark, we tested configurations with 2 NVMe SSDs as well as 4 NVMe SSDs. With 4VMs spread out over 2 drives, we saw aggregate TPS in Benchmark Factory measure 12,631, whereas with 4 NVMe SSDs, this measured 12,625. While this is a bit counter-intuitive, with our particular configuration of the benchmark, latency measured below shows the real story.
Our last section of local performance testing focuses in on synthetic-workload performance. In this area, we leveraged four NVMe SSDs in VMware ESXi 6.5, and evenly spread out 16 worker VMs, each with two 125GB vmdks mounted to measure a storage footprint of 4TB. This type of test is useful to show what real-world storage metrics look like with the overhead associated with a virtualized environment.
When it comes to benchmarking storage arrays, application testing is best, and synthetic testing comes in second place. While not a perfect representation of actual workloads, synthetic tests do help to baseline storage devices with a repeatability factor that makes it easy to do apples-to-apples comparison between competing solutions. These workloads offer a range of different testing profiles ranging from "four corners" tests, common database transfer size tests, as well as trace captures from different VDI environments. All of these tests leverage the common vdBench workload generator, with a scripting engine to automate and capture results over a large compute testing cluster. This allows us to repeat the same workloads across a wide range of storage devices, including flash arrays and individual storage devices. On the array side, we use our cluster of Dell PowerEdge R730 servers:
Profiles:
4K Random Read: 100% Read, 128 threads, 0-120% iorate
4K Random Write: 100% Write, 64 threads, 0-120% iorate
64K Sequential Read: 100% Read, 16 threads, 0-120% iorate
64K Sequential Write: 100% Write, 8 threads, 0-120% iorate
Synthetic Database: SQL and Oracle
VDI Full Clone and Linked Clone Traces
Looking at peak read performance, the Dell EMC PowerEdge R740xd offered sub-millisecond latency 4K read performance up to just over of 800K IOPS, starting at 0.21ms. At its peak, the R740xd measured 978k IOPS at a latency of 3.8ms.
Switching to 64K peak read, the R740xd started off at 0.27ms latency and stayed below 1ms until it hit around 150K IOPS. It peaked just over 170K IOPS with 3ms of latency. The R740xd finished with a bandwidth of 10.644GB/s.
For 64K sequential peak write, the R740xd started off at 0.14ms and stayed under 1ms until it hit just over 65K IOPS. The R740xd hit its peak at 93K IOPS with a latency of 2.7ms. The R740xd also had a bandwidth of 5.83GB/s at its peak.
The SQL 80-20 saw the R740xd start with a latency of 0.2ms and stay under 1ms until it moved over 481K IOPS. The R740xd peaked nearly at 538K IOPS with 1.7ms latency.
With the Oracle Workload, the R740xd started with a latency of 0.2ms and stayed under 1ms until just over 400K IOPS. The R740xd peaked at 470K IOPS with a latency of 2.5ms.
With the Oracle 80-20, the R740xd started off at a latency of 0.2ms and stayed under 1ms until it was just under 529K IOPS. It peaked at 533K IOPS with a latency of 1.14ms.
Switching over to VDI Full Clone, the boot test showed the R740xd starting with a latency of 0.21ms and staying under 1ms until around 490K IOPS. The R740xd peaked at 539K IOPS with a latency of 1.9ms.
The VDI Full Clone initial login started off at 0.17ms latency and stayed under 1ms until around 175K IOPS. The R740xd peaked at 218K IOPS with a latency of 4.1ms.
The VDI Full Clone Monday login started off at 0.2ms latency, staying under 1ms until over 180K IOPS. It peaked at 215K IOPS with 2.36ms.
Moving over to VDI Linked Clone, the boot test showed performance staying under 1ms up to roughly 350k IOPS, and later topping out at a peak of 376K IOPS with an average latency of 1.36ms.
In the Linked Clone VDI profile measuring Initial Login performance, we saw sub-ms latency up till around 130k IOPS, where it further increased to 154K IOPS at 1.64ms at its peak.
In our last profile looking at VDI Linked Clone Monday Login performance, we see the 1ms barrier transition happening at around 109K IOPS, where the workload continued to increase to its peak at 151K IOPS and 3.36ms average latency.
The new Dell EMC PowerEdge R740xd is the "extreme disk" version of the R740. Within its 2U space, it can hold up to 32 2.5 "drives, including up to 24 NVMe drives.This server can help unlock the potential of all that high-performance storage by taking up to two scalable Intel processors and up to 3 TB of memory Dell EMC not only stopped with hardware upgrades The new server comes with integrated SDS support, making it ideal for use cases that need to take advantage of HCI The server is modular and highly configurable to satisfy almost all the needs of the customers.
In our application performance benchmarks, we tested one Dell EMC PowerEdge R740xd with 4VM hosted on two NVMe SSDs, and tested another with 4VM with a dedicated NVMe SSD for each VM. For Sysbench, test 4 NVMe had a score of 13,224 TPS, average latency of 10 ms and scenario latency of worst scenario of 21ms, while comparative test 2 NVMe had 11,028 TPS, average latency of 12ms and worst case latency 24ms stage For our SQL Server test, the 4 NVMe tests achieved an aggregate TPS score of 12.625 and an aggregate latency of 4 ms. Test 2 NVMe produced an aggregate TPS score of 12,631.8 and an aggregate latency of 6.5ms.
In our VDBench workload analysis, the R740xd really shone in a virtualized ESXi 6.5 environment. In our 4K randomized synthetic test, we saw less than millisecond performance in reading up to 800,000 IOPS, and in writing up to 730,000 IOPS. In the 64K sequential read, the R740xd had a latency of less than milliseconds to 150,000 IOPS and ended with a bandwidth of 10,644 GB / s. To write 64K, the server had a performance of less than milliseconds up to 65,000 IOPS and a bandwidth of 5.83GB / s. In our SQL workload, we saw again a solid performance of less than milliseconds (up to 700,000 IOPS, 600,000 IOPS and 481,000 IOPS for workload, 90-10, 80-20, respectively), but the most impressive thing was that the performance exceeded with latency between 1.29ms and 1.7ms with a performance well in excess of 500,000 IOPS in each. Oracle's workload also showed a solid performance of less than milliseconds with the 90-10 running the full benchmark in less than 1 ms, reaching a maximum of 636,000 IOPS. The R740xd reached a maximum of 539,000 IOPS, 218,000 IOPS and 215,000 IOPS in full clone (with a maximum latency of 1.9ms, 4.1ms and 2.36ms). And in our linked cloning reference points, the server reached a maximum of 376,000 IOPS, 154,000 IOPS and 151,000 IOPS (with a maximum latency of 1.36ms, 1.64ms and 3.36ms).
Dell EMC is clearly excited about the launch of the new line of servers and specifically with the R740xd, the centerpiece of the PowerEdge line. We have logged many weeks with the new systems, and thirteen R740xds make up the central backbone of our test lab. From the work we have done, the servers have impressed everywhere, from the management capacity through iDrac and OpenManaage Mobile to the performance with the NVMe bays. With all the additional flexibility that the x7 flavor of the R740 offers, it's no surprise that Dell EMC is using it as the backbone of several of its SDS offerings, including Ready vSAN nodes, ScaleIO ready nodes, direct storage nodes, VxRail and the XC740xd (Nutanix) for example. In total, the PowerEdge R740xd is the most complete server offering we've seen to date in terms of build quality, system design, storage flexibility, performance and manageability, which makes it a clear leader in the space and our first Editor's Choice in the server category.
No comments:
Post a Comment
Note: only a member of this blog may post a comment.