Handle up to 1.65x More Inference Work with Hugging Face – BERT Large Using Pytorch on Microsoft Azure Ddsv5 Virtual Machines over Ddsv4 VMs

Hugging Face – BERT Large

  • Handle up to 1.46x More Inference Work with 32-vCPU Ddsv5 VMs vs. Ddsv4 VMs.

  • Handle up to 1.65x More Inference Work with 48-vCPU Ddsv5 VMs vs. Ddsv4 VMs.

author-image

作者

PDF

Enjoy Stronger Performance with New Ddsv5 VMs Featuring 3rd Gen Intel® Xeon® Scalable Processors

Companies increasingly rely on machine learning inference workloads for a range of business activities. Inference is extremely compute-intensive, making it paramount to understand the performance potential of any VMs you are considering when shopping for a cloud solution to host your inference workloads. The latest Microsoft Azure Ddsv5-series VMs enabled by 3rd Gen Intel® Xeon® Scalable processors—available in a range of vCPU counts—can deliver excellent performance. For applications that benefit from high vCPU counts and large amounts of memory, such as inference, these Microsoft Azure Ddsv5-series VMs are a great choice.

We used the Hugging Face - BERT Large inference workload to measure the inference performance of two sizes of Microsoft Azure VMs. We found that new Ddsv5 VMs enabled by 3rd Gen Intel Xeon Scalable processors delivered up to 1.65x more inference work as Ddsv4 VMs with older processors.

Achieve More Inference Work with 32-vCPU VMs

Choosing Microsoft Azure Ddsv5 VMs with newer processors for your inference workloads rather than older VMs can improve performance per VM. In Hugging Face - BERT Large testing of 32-vCPU VMs, Azure Ddsv5 VMs enabled by 3rd Gen Intel Xeon Scalable processors handled up to 1.46x more inference work than a Ddsv4 VM enabled by previous-generation processors (see Figure 1).

Figure 1. Relative Hugging Face - BERT Large performance of the 32-vCPU Azure Ddsv5 VM and 32-vCPU Azure Ddsv4 VM types.

Achieve More Inference Work with 48-vCPU VMs

In Hugging Face – BERT Large testing of 48-vCPU VMs, Azure Ddsv5 VMs enabled by 3rd Gen Intel® Xeon® Scalable processors handled up to 1.65x more inference work than a Ddsv4 VM enabled by previous generation processors (see Figure 2).

Figure 2. Relative Hugging Face - BERT Large performance of the 48-vCPU Azure Ddsv5 VM and 48-vCPU Azure Ddsv4 VM types.

Learn More

To begin running your DL inference workloads on Microsoft Azure Ddsv5 virtual machines with 3rd Gen Intel Xeon Scalable 
processors, visit https://intel.com.tw/microsoftazure.

For pricing information, visit https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/#pricing.