{"id":412,"date":"2021-08-13T14:25:26","date_gmt":"2021-08-13T14:25:26","guid":{"rendered":"https:\/\/terrabioappdev.wpenginepowered.com\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/"},"modified":"2023-12-27T04:54:55","modified_gmt":"2023-12-27T04:54:55","slug":"speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus","status":"publish","type":"post","link":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/","title":{"rendered":"Speed up your workflows with N2 instances sporting Intel&#8217;s 2nd Generation Xeon CPUs"},"content":{"rendered":"<p><span style=\"font-weight: 400;\">Terra&#8217;s workflow management system is designed to handle most of the computing infrastructure &#8220;busy work&#8221; on your behalf so that you can focus on your work rather than dealing with IT minutia. Sometimes that means we have to limit some options for practical reasons, yet we&#8217;re always looking for ways to increase your freedom to choose what&#8217;s right for you.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Most recently, we opened up a new option to allow you to use a newer model of Intel CPUs than what the system normally uses by default to execute workflow tasks. Until now, the system automatically assigned all workflow execution to a family of virtual machines (VMs) called &#8220;N1 instances&#8221;, which use CPUs from older generations of Intel processors. Now you&#8217;ll be able to request &#8220;N2 instances&#8221; instead, which use 2nd Generation Intel\u00ae Xeon\u00ae Scalable Processors, codenamed &#8220;<\/span><a href=\"https:\/\/www.intel.com\/content\/www\/us\/en\/products\/platforms\/details\/cascade-lake.html\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">Cascade Lake<\/span><\/a><span style=\"font-weight: 400;\">&#8220;. According to recent benchmarks, the Cascade Lake CPUs are capable of delivering substantial performance advantages at a lower cost for genomics workloads, so we&#8217;re excited to share the news that you can now opt into using them for your workflows in Terra.<\/span><\/p>\n<p>&nbsp;<\/p>\n<h3><span style=\"font-weight: 400;\">Benchmarking results on GATK Best Practices workflows<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Our colleagues in the GATK development team at the Broad Institute have a long-standing collaboration with an Intel team that focuses on optimizing genomics tools and pipelines, so they hooked us up with some benchmarking results that illustrate where the Cascade Lake CPUs make a big difference in the GATK Best Practices workflows.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Specifically, they compared the runtime and cost of all 24 tasks in the Broad Institute&#8217;s <\/span><a href=\"https:\/\/github.com\/broadinstitute\/warp\/tree\/develop\/pipelines\/broad\/dna_seq\/germline\/single_sample\/wgs\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">production implementation<\/span><\/a><span style=\"font-weight: 400;\"> of the GATK Best Practices Pipeline for WGS Germline Variant Calling. Table 1 below provides a summary of the tasks in the pipeline, the number of VMs each task is distributed across, and the minimum number of CPUs, memory (DRAM), and disk required per VM for that task.<\/span><\/p>\n<p>&nbsp;<\/p>\n<table style=\"height: 699px;\" width=\"513\">\n<tbody>\n<tr>\n<td><b>Task<\/b><\/td>\n<td><b>Number\u00a0 \u201cshards,\u201d or VMs<\/b><\/td>\n<td><b>Min vCPU\u00a0<\/b><\/td>\n<td><b>Min DRAM (GB)<\/b><\/td>\n<td><b>Min <\/b><b>Local Disk (GB)<\/b><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">SortSampleBam<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">4.9<\/span><\/td>\n<td><span style=\"font-weight: 400;\">379<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">MarkDuplicates<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">22.5<\/span><\/td>\n<td><span style=\"font-weight: 400;\">352<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">ApplyBQSR<\/span><\/td>\n<td><span style=\"font-weight: 400;\">19<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">3.4<\/span><\/td>\n<td><span style=\"font-weight: 400;\">253<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">GatherBamFiles<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">3.0<\/span><\/td>\n<td><span style=\"font-weight: 400;\">174<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">HaplotypeCallerGATK4<\/span><\/td>\n<td><span style=\"font-weight: 400;\">50<\/span><\/td>\n<td><span style=\"font-weight: 400;\">2<\/span><\/td>\n<td><span style=\"font-weight: 400;\">6.5<\/span><\/td>\n<td><span style=\"font-weight: 400;\">114<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">CheckContamination<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">2<\/span><\/td>\n<td><span style=\"font-weight: 400;\">7.5<\/span><\/td>\n<td><span style=\"font-weight: 400;\">110<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">BaseRecalibrator<\/span><\/td>\n<td><span style=\"font-weight: 400;\">18<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">6.0<\/span><\/td>\n<td><span style=\"font-weight: 400;\">110<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">CrossCheckFingerprints<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">2.0<\/span><\/td>\n<td><span style=\"font-weight: 400;\">97<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">CollectAggregationMetrics<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">7.0<\/span><\/td>\n<td><span style=\"font-weight: 400;\">84<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">CollectReadgroupBamQuality<\/span><span style=\"font-weight: 400;\">Metrics<\/span><\/td>\n<td><span style=\"font-weight: 400;\">24<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">7.0<\/span><\/td>\n<td><span style=\"font-weight: 400;\">84<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">CalculateReadGroupChecksum<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">2.0<\/span><\/td>\n<td><span style=\"font-weight: 400;\">81<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">ValidateVCF<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">6.8<\/span><\/td>\n<td><span style=\"font-weight: 400;\">40<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">SamToFastqAndBwaMemAndMba<\/span><\/td>\n<td><span style=\"font-weight: 400;\">24<\/span><\/td>\n<td><span style=\"font-weight: 400;\">16<\/span><\/td>\n<td><span style=\"font-weight: 400;\">14.0<\/span><\/td>\n<td><span style=\"font-weight: 400;\">39<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">CollectVariantCallingMetrics<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">3.0<\/span><\/td>\n<td><span style=\"font-weight: 400;\">37<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">MergeVCFs<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">3.0<\/span><\/td>\n<td><span style=\"font-weight: 400;\">33<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">CollectUnsortedReadgroupBamQuality <\/span><span style=\"font-weight: 400;\">Metrics<\/span><\/td>\n<td><span style=\"font-weight: 400;\">24<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">7.0<\/span><\/td>\n<td><span style=\"font-weight: 400;\">25<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">CollectQualityYieldMetrics<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">3.0<\/span><\/td>\n<td><span style=\"font-weight: 400;\">23<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">GatherBqsrReports<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">3.4<\/span><\/td>\n<td><span style=\"font-weight: 400;\">20<\/span><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">CreateSequenceGroupingTSV<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">2.0<\/span><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">ScatterIntervalList<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">2.0<\/span><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">GetBwaVersion<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1.0<\/span><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><span style=\"font-weight: 400;\">SumFloats<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><span style=\"font-weight: 400;\">1<\/span><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td><i><span style=\"font-weight: 400;\">Total<\/span><\/i><\/td>\n<td><span style=\"font-weight: 400;\">181<\/span><\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">They ran the workflow three times on N1 instances (N1-standard-2; N1-standard-16) and three times on N2 instances (N2-standard-2; N2-standard-16) on the Broad&#8217;s NA12878 30X coverage human whole genome sequence (WGS) test dataset.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Figure 1 below shows the median runtime for these two different instance types. As expected, the N2 instances with the later generation Intel Xeon Scalable processors have a faster runtime overall. The end-to-end pipeline executes in 24.9 hours compared to 32.9 hours when run on N1 instances.\u00a0<\/span><\/p>\n<p><img fetchpriority=\"high\" decoding=\"async\" class=\"wp-image-1071 alignleft\" src=\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/N1vN2-300x183.png\" alt=\"Runtype of 2 different types\" width=\"387\" height=\"236\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">A closer look at per-task runtimes and cost (not shown here) tells us that the bulk of the speedup comes down to the two most compute-intensive tasks, SamToFastqAndBwaAndMba (genome mapping and formatting) and HaplotypeCaller (variant calling), which have the longest runtimes in the pipeline. This makes sense, since we know for example that the HaplotypeCaller task benefits from improved hardware accelerations on the latest Intel chips thanks to the use of the Intel Genomics Kernel Library, which includes accelerated implementations of the PairHMM and Smith-Waterman algorithms in HaplotypeCaller.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The overall cost of the pipeline is lower for N2 instances as well, with a median cost measured at $5 per WGS sample for the N2 instances vs $6 per WGS sample for the N1 instances (preemptible instances were used in both cases).<\/span><\/p>\n<p><span style=\"font-weight: 400;\">These results suggest that N2 instances are the more cost-effective and performant option compared to N1 instances. Opting to use N2 instances may be an attractive option for shaving off time and expense from your workflows, especially for the more compute-intensive tasks.<\/span><\/p>\n<p>&nbsp;<\/p>\n<h3><span style=\"font-weight: 400;\">How to request N2 instances for your workflows\u00a0<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Terra&#8217;s workflow execution system, which uses the Cromwell workflow manager and the Workflow Description Language (WDL) under the hood, allows us to specify what kind of compute resources should be allocated for executing each step of our workflow.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">For example, if we look at the WDL code for the <\/span><a href=\"https:\/\/github.com\/broadinstitute\/warp\/blob\/8988d9a490ba026fc2470d40886d9f50ac7920d0\/tasks\/broad\/GermlineVariantDiscovery.wdl#L130\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">HaplotypeCaller task<\/span><\/a><span style=\"font-weight: 400;\"> mentioned in the benchmarking section above, the block of code that specifies the relevant runtime requirements looks like this:<\/span><\/p>\n<pre><span style=\"font-weight: 400;\">runtime {<\/span>\n\n<span style=\"font-weight: 400;\">\u00a0\u00a0\u00a0\u00a0docker: gatk_docker<\/span>\n\n<span style=\"font-weight: 400;\">\u00a0\u00a0\u00a0\u00a0preemptible: preemptible_tries<\/span>\n\n<span style=\"font-weight: 400;\">\u00a0\u00a0\u00a0<\/span><b>\u00a0memory: \"6.5 GiB\"<\/b>\n\n<span style=\"font-weight: 400;\">\u00a0\u00a0<\/span><b>\u00a0\u00a0cpu: \"2\"<\/b>\n\n<span style=\"font-weight: 400;\">\u00a0\u00a0\u00a0\u00a0bootDiskSizeGb: 15<\/span>\n\n<span style=\"font-weight: 400;\">\u00a0\u00a0\u00a0\u00a0disks: \"local-disk \" + disk_size + \" HDD\"<\/span>\n\n<span style=\"font-weight: 400;\">}<\/span><\/pre>\n<p><span style=\"font-weight: 400;\">This amounts to a request for a VM with<\/span><b> 2 CPUs<\/b><span style=\"font-weight: 400;\"> and <\/span><b>at least<\/b> <b>6.5 GB of memory<\/b><span style=\"font-weight: 400;\">, plus some additional characteristics we won&#8217;t cover in detail here. When you launch the workflow, Terra will translate these minimum requirements into a specific machine type request to the computing platform.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Where it gets a little complicated is that cloud vendors (here, Google Cloud) offer a lot of different machine types, further categorized into instance types such as N1 and N2 (there are multiple others), that are designed to satisfy a wide range of different computational needs. This introduces a lot of complexity into the question of how best to interpret the basic requirements encoded in the WDL workflow. For practical reasons, we reduce that complexity by having Terra use only a subset of instance types.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Specifically, on Google Cloud, Terra uses only N1 instances by default. The closest match to our example request in the <\/span><a href=\"https:\/\/cloud.google.com\/compute\/docs\/general-purpose-machines#n1-standard\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">list of N1 instances<\/span><\/a><span style=\"font-weight: 400;\"> is the <\/span><b>n1-standard-2<\/b><span style=\"font-weight: 400;\"> machine type, which has 2 CPUs and 7.5 GB of memory.<\/span><\/p>\n<p><img decoding=\"async\" class=\"wp-image-1073 alignleft\" src=\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/table-of-instances-300x101.png\" alt=\"n1-standard-2 machine type\" width=\"490\" height=\"165\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">This means that when you launch the workflow, the system will create the VM(s) for running that HaplotypeCaller task based on the n1-standard-2 machine type.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">However, as noted above you can now choose to use N2 instance types instead, which have similar technical profiles to the N1 instances but are equipped with the newer generation of Intel\u00ae Xeon\u00ae Scalable Processors, aka Cascade Lake CPUs.\u00a0\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The not-so-obvious trick here is that to request an N2 instance for a task, you&#8217;ll need to specify <\/span><b>cpuPlatform: &#8220;Intel Cascade Lake&#8221;<\/b><span style=\"font-weight: 400;\"> in the task&#8217;s runtime block. Going back to our earlier example, this is what the runtime block would look like for our HaplotypeCaller task in the WDL workflow:<\/span><\/p>\n<pre><span style=\"font-weight: 400;\">runtime {<\/span>\n\n<span style=\"font-weight: 400;\">\u00a0\u00a0\u00a0\u00a0docker: gatk_docker<\/span>\n\n<span style=\"font-weight: 400;\">\u00a0\u00a0\u00a0\u00a0preemptible: preemptible_tries<\/span>\n\n<span style=\"font-weight: 400;\">\u00a0\u00a0\u00a0<\/span><b>\u00a0memory: \"6.5 GiB\"<\/b>\n\n<span style=\"font-weight: 400;\">\u00a0\u00a0<\/span><b>\u00a0\u00a0cpu: \"2\"<\/b>\n\n<b>\u00a0\u00a0\u00a0\u00a0cpuPlatform: \"Intel Cascade Lake\"<\/b>\n\n<span style=\"font-weight: 400;\">\u00a0\u00a0\u00a0\u00a0bootDiskSizeGb: 15<\/span>\n\n<span style=\"font-weight: 400;\">\u00a0\u00a0\u00a0\u00a0disks: \"local-disk \" + disk_size + \" HDD\"<\/span>\n\n<span style=\"font-weight: 400;\">}<\/span><\/pre>\n<p><b>That one line setting the cpuPlatform parameter is all it takes to make the switch.\u00a0<\/b><\/p>\n<p><span style=\"font-weight: 400;\">Now Terra will request the closest match in the <\/span><a href=\"https:\/\/cloud.google.com\/compute\/docs\/general-purpose-machines#n2-standard\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">list of N2 instances<\/span><\/a><span style=\"font-weight: 400;\">: the <\/span><b>n2-standard-2<\/b><span style=\"font-weight: 400;\"> machine type, which has 2 CPUs and 8 GB of memory.<\/span><\/p>\n<p><img decoding=\"async\" class=\"wp-image-1072 alignleft\" src=\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/table-of-instances-n2-300x101.png\" alt=\"n2-standard-2 machine type\" width=\"425\" height=\"143\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">The cpuPlatform parameter refers to the processor family that is used in N2 instances, so this is a slightly indirect way of getting what we want&#8230; but whatever works, right?<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The downside of being so specific about the cpuPlatform you want is that if there are no machines matching that requirement available at the time you make the request, your task will get queued until one becomes available. At times of peak utilization, that could end up delaying your workflow more than what you gain from using the upgraded hardware. We don&#8217;t have exact numbers on how many N2 instances are available relative to N1 instances in Google Cloud, and that proportion is likely to be <\/span><a href=\"https:\/\/cloud.google.com\/compute\/docs\/regions-zones\/#available\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">zone-dependent<\/span><\/a><span style=\"font-weight: 400;\">, but it&#8217;s fair to guess that there are probably more N1 instances than N2 instances available at this time.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">So for now, a conservative approach would be to use N2 instances for tasks where you know they make a big difference to cost and runtime (like the mapping and variant calling steps revealed by the GATK workflow benchmarks), and stick with N1 instances (ie not specify a cpuPlatform in task runtime blocks) for the rest. Over time we expect that the number of newer machines will increase and it will eventually make sense to switch over to N2 instances completely.\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<hr \/>\n<h3><b>Acknowledgments<\/b><\/h3>\n<p><span style=\"font-weight: 400;\">We are grateful to the Intel team, specifically <\/span><span style=\"font-weight: 400;\">Priyanka Sebastian, Keith Mannthey, and Marissa Powers, for running the GATK benchmarks and helping us characterize the benefits of using N2 instances.<\/span><\/p>\n<hr \/>\n<h3><\/h3>\n<h3><\/h3>\n<h3><\/h3>\n<h3><b>Resources<\/b><\/h3>\n<p><a href=\"https:\/\/support.terra.bio\/hc\/en-us\/articles\/360046944671\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">Default runtime attributes for workflow submissions<\/span><\/a><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<p><a href=\"https:\/\/support.terra.bio\/hc\/en-us\/articles\/4402240831771-June-16-2021\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">Release notes announcing support for N2 instances<\/span><\/a><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<p><a href=\"https:\/\/cromwell.readthedocs.io\/en\/develop\/RuntimeAttributes\/#cpuplatform\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">Cromwell documentation for the cpuPlatform parameter<\/span><\/a><span style=\"font-weight: 400;\"> (introduced in version 64)<\/span><\/p>\n<p><a href=\"https:\/\/terra.bio\/reduce-computing-costs-by-tailoring-resource-allocations-in-workflows\/\"><span style=\"font-weight: 400;\">Blog post about optimizing resource allocations<\/span><\/a><\/p>\n<p><a href=\"https:\/\/www.youtube.com\/watch?v=rUBrJNqLyfU\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">Workflows on Terra: Quickstart video<\/span><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Terra&#8217;s workflow management system now allows you to use a newer model of Intel CPUs than what the system normally uses by default to execute workflow tasks. You&#8217;ll be able to request &#8220;N2 instances&#8221; instead of &#8220;N1 instances&#8221;, which use 2nd Generation Intel\u00ae Xeon\u00ae Scalable Processors, codenamed &#8220;Cascade Lake&#8221;.<\/p>\n","protected":false},"author":4,"featured_media":416,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[12,43,119,32],"tags":[],"class_list":["post-412","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-analysis","category-features","category-most-recent","category-workflows"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.0 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Speed up your workflows with N2 instances sporting Intel&#039;s 2nd Generation Xeon CPUs - Terra<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Speed up your workflows with N2 instances sporting Intel&#039;s 2nd Generation Xeon CPUs - Terra\" \/>\n<meta property=\"og:description\" content=\"Terra&#039;s workflow management system now allows you to use a newer model of Intel CPUs than what the system normally uses by default to execute workflow tasks. You&#039;ll be able to request &quot;N2 instances&quot; instead of &quot;N1 instances&quot;, which use 2nd Generation Intel\u00ae Xeon\u00ae Scalable Processors, codenamed &quot;Cascade Lake&quot;.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/\" \/>\n<meta property=\"og:site_name\" content=\"Terra\" \/>\n<meta property=\"article:published_time\" content=\"2021-08-13T14:25:26+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-27T04:54:55+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1675464244.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"627\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Geraldine Van der Auwera\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Geraldine Van der Auwera\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/\"},\"author\":{\"name\":\"Geraldine Van der Auwera\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/ad0522d0b331a5e08fa1733f65086ee2\"},\"headline\":\"Speed up your workflows with N2 instances sporting Intel&#8217;s 2nd Generation Xeon CPUs\",\"datePublished\":\"2021-08-13T14:25:26+00:00\",\"dateModified\":\"2023-12-27T04:54:55+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/\"},\"wordCount\":1329,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/terra.bio\/#organization\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1675464244.png\",\"articleSection\":[\"Analysis\",\"Features\",\"Most Recent\",\"Workflows\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/\",\"url\":\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/\",\"name\":\"Speed up your workflows with N2 instances sporting Intel's 2nd Generation Xeon CPUs - Terra\",\"isPartOf\":{\"@id\":\"https:\/\/terra.bio\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1675464244.png\",\"datePublished\":\"2021-08-13T14:25:26+00:00\",\"dateModified\":\"2023-12-27T04:54:55+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#primaryimage\",\"url\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1675464244.png\",\"contentUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1675464244.png\",\"width\":1200,\"height\":627,\"caption\":\"shutterstock_1675464244\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/terra.bio\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Speed up your workflows with N2 instances sporting Intel&#8217;s 2nd Generation Xeon CPUs\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/terra.bio\/#website\",\"url\":\"https:\/\/terra.bio\/\",\"name\":\"Terra\",\"description\":\"Science at Scale\",\"publisher\":{\"@id\":\"https:\/\/terra.bio\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/terra.bio\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/terra.bio\/#organization\",\"name\":\"Terra\",\"url\":\"https:\/\/terra.bio\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp\",\"contentUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp\",\"width\":287,\"height\":318,\"caption\":\"Terra\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/ad0522d0b331a5e08fa1733f65086ee2\",\"name\":\"Geraldine Van der Auwera\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/d73bdaf6740465b385e0e3b290786d8cb9d9d548eadec23364254ba06c85204b?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/d73bdaf6740465b385e0e3b290786d8cb9d9d548eadec23364254ba06c85204b?s=96&d=mm&r=g\",\"caption\":\"Geraldine Van der Auwera\"},\"sameAs\":[\"https:\/\/app.terra.bio\/\"],\"url\":\"https:\/\/terra.bio\/author\/geraldinevanterra\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Speed up your workflows with N2 instances sporting Intel's 2nd Generation Xeon CPUs - Terra","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/","og_locale":"en_US","og_type":"article","og_title":"Speed up your workflows with N2 instances sporting Intel's 2nd Generation Xeon CPUs - Terra","og_description":"Terra's workflow management system now allows you to use a newer model of Intel CPUs than what the system normally uses by default to execute workflow tasks. You'll be able to request \"N2 instances\" instead of \"N1 instances\", which use 2nd Generation Intel\u00ae Xeon\u00ae Scalable Processors, codenamed \"Cascade Lake\".","og_url":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/","og_site_name":"Terra","article_published_time":"2021-08-13T14:25:26+00:00","article_modified_time":"2023-12-27T04:54:55+00:00","og_image":[{"width":1200,"height":627,"url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1675464244.png","type":"image\/png"}],"author":"Geraldine Van der Auwera","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Geraldine Van der Auwera","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#article","isPartOf":{"@id":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/"},"author":{"name":"Geraldine Van der Auwera","@id":"https:\/\/terra.bio\/#\/schema\/person\/ad0522d0b331a5e08fa1733f65086ee2"},"headline":"Speed up your workflows with N2 instances sporting Intel&#8217;s 2nd Generation Xeon CPUs","datePublished":"2021-08-13T14:25:26+00:00","dateModified":"2023-12-27T04:54:55+00:00","mainEntityOfPage":{"@id":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/"},"wordCount":1329,"commentCount":0,"publisher":{"@id":"https:\/\/terra.bio\/#organization"},"image":{"@id":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#primaryimage"},"thumbnailUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1675464244.png","articleSection":["Analysis","Features","Most Recent","Workflows"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/","url":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/","name":"Speed up your workflows with N2 instances sporting Intel's 2nd Generation Xeon CPUs - Terra","isPartOf":{"@id":"https:\/\/terra.bio\/#website"},"primaryImageOfPage":{"@id":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#primaryimage"},"image":{"@id":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#primaryimage"},"thumbnailUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1675464244.png","datePublished":"2021-08-13T14:25:26+00:00","dateModified":"2023-12-27T04:54:55+00:00","breadcrumb":{"@id":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#primaryimage","url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1675464244.png","contentUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1675464244.png","width":1200,"height":627,"caption":"shutterstock_1675464244"},{"@type":"BreadcrumbList","@id":"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/terra.bio\/"},{"@type":"ListItem","position":2,"name":"Speed up your workflows with N2 instances sporting Intel&#8217;s 2nd Generation Xeon CPUs"}]},{"@type":"WebSite","@id":"https:\/\/terra.bio\/#website","url":"https:\/\/terra.bio\/","name":"Terra","description":"Science at Scale","publisher":{"@id":"https:\/\/terra.bio\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/terra.bio\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/terra.bio\/#organization","name":"Terra","url":"https:\/\/terra.bio\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/#\/schema\/logo\/image\/","url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp","contentUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp","width":287,"height":318,"caption":"Terra"},"image":{"@id":"https:\/\/terra.bio\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/terra.bio\/#\/schema\/person\/ad0522d0b331a5e08fa1733f65086ee2","name":"Geraldine Van der Auwera","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/d73bdaf6740465b385e0e3b290786d8cb9d9d548eadec23364254ba06c85204b?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d73bdaf6740465b385e0e3b290786d8cb9d9d548eadec23364254ba06c85204b?s=96&d=mm&r=g","caption":"Geraldine Van der Auwera"},"sameAs":["https:\/\/app.terra.bio\/"],"url":"https:\/\/terra.bio\/author\/geraldinevanterra\/"}]}},"_links":{"self":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts\/412","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/comments?post=412"}],"version-history":[{"count":0,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts\/412\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/media\/416"}],"wp:attachment":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/media?parent=412"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/categories?post=412"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/tags?post=412"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}