{"id":509,"date":"2022-03-17T19:17:59","date_gmt":"2022-03-17T19:17:59","guid":{"rendered":"https:\/\/terrabioappdev.wpenginepowered.com\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/"},"modified":"2023-12-27T04:55:18","modified_gmt":"2023-12-27T04:55:18","slug":"ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs","status":"publish","type":"post","link":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/","title":{"rendered":"Ten simple rules \u2014 #3 Understand hardware and regulatory limitations and trade-offs"},"content":{"rendered":"<p><i><span style=\"font-weight: 400;\">This blog post is part of a series based on the paper &#8220;<a href=\"https:\/\/journals.plos.org\/ploscompbiol\/article?id=10.1371\/journal.pcbi.1009757\">Ten simple rules for large-scale data processing<\/a>&#8221; by Arkarachai Fungtammasan et al. (PLOS Computational Biology, 2022). Each installment reviews one of the rules proposed by the authors and illustrates how it can be applied when working in Terra. In this installment, we touch on questions of costs and benefits, regulatory constraints, and opportunities to leverage the scalability of cloud infrastructure.<\/span><\/i><\/p>\n<hr \/>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">This series has been cruising along quite smoothly, with two fairly self-explanatory rules so far \u2014 &#8220;<\/span><a href=\"https:\/\/terra.bio\/ten-simple-rules-1-dont-reinvent-the-wheel\/\"><span style=\"font-weight: 400;\">Don&#8217;t reinvent the wheel<\/span><\/a><span style=\"font-weight: 400;\">&#8221; and &#8220;<\/span><a href=\"https:\/\/terra.bio\/ten-simple-rules-2-document-everything\/\"><span style=\"font-weight: 400;\">Document everything<\/span><\/a><span style=\"font-weight: 400;\">&#8221; \u2014 but this third rule might feel like a hard turn onto bumpy terrain. Less of a rule, perhaps, and more a checklist of considerations?\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Buckle up as we tackle some of the key factors that Arkarachai Fungtammasan and colleagues recommend should go into choosing a computing platform for large-scale data processing.\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<h3><span style=\"font-weight: 400;\">Spoiler: you should probably use the cloud<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">To kick us off, the authors define the choice of such a platform as a &#8220;multi-objective optimization&#8221; problem. This sounds complicated, yet at its simplest this boils down to balancing costs \u2014 &#8220;with respect to computing cost or quota, waiting time, and implementation time&#8221; \u2014 against the value of the output, e.g. the scientific insights that you&#8217;ll be able to extract from the data.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Looking at that enumeration of what counts as a cost, it&#8217;s great to see &#8220;waiting time and implementation time&#8221; taken explicitly into account. So many discussions in this space focus primarily on the monetary costs of using on-premises computing vs. cloud infrastructure, and leave out the question of how long it takes for researchers to actually get work done. It&#8217;s important to remember that people&#8217;s time is valuable too, both at a personal level and from the point of view of opportunity cost for their organization. Public cloud infrastructure tends to yield very clear benefits on this front, because their whole point is to offer access to compute resources that are readily available on demand.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">If you&#8217;re currently working with shared on-premises computing infrastructure, think about how much time you&#8217;ve spent waiting for your jobs to make it out of the queue, only to find out that you made a small mistake and need to tweak a parameter, then resubmit everything? What if you could get your work done in days instead of weeks, months instead of years?\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Then there&#8217;s the closely related question of scale. The biggest driver of change in this space has been the need for greater scalability to cope with the flood of data brought on by a decade of intense technological development and the falling costs of data generation.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">In this context, scalability doesn&#8217;t mean &#8220;just&#8221; being able to process a lot of data; it also means that you&#8217;re able to easily reapply the same methods to other data, or re-process cohorts of data to take advantage of new computational innovations. And as the authors point out in their paper:<\/span><\/p>\n<blockquote><p><span style=\"font-weight: 400;\">&#8220;[&#8230;] the investment in infrastructure for repeatability and selecting and rerunning certain subsets of the data becomes much more valuable as the processing task grows.&#8221;<\/span><\/p><\/blockquote>\n<p><span style=\"font-weight: 400;\">We&#8217;ve seen this play out with Terra itself, as over time more research consortia have flocked to the platform to take advantage of its <\/span><a href=\"https:\/\/terra.bio\/calling-variants-from-telomere-to-telomere-with-the-new-t2t-chm13-genome-reference\/\"><span style=\"font-weight: 400;\">large-scale workflow processing capabilities<\/span><\/a><span style=\"font-weight: 400;\"> and built-in support for <\/span><a href=\"https:\/\/terra.bio\/achieve-full-computational-reproducibility-by-sharing-your-published-analysis-as-a-terra-workspace\/\"><span style=\"font-weight: 400;\">reproducibility and collaboration<\/span><\/a><span style=\"font-weight: 400;\">. I personally find it very exciting that a wide range of organizations are choosing to <\/span><a href=\"https:\/\/terra.bio\/from-infrastructure-projects-to-connected-communities\/\"><span style=\"font-weight: 400;\">invest in shared, scalable infrastructure<\/span><\/a><span style=\"font-weight: 400;\"> rather than each building out their own, because it means we&#8217;re able to go much further together, and deliver economies of scale that benefit the entire ecosystem.\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<h3><span style=\"font-weight: 400;\">Navigating regulatory constraints<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">This brings us to actual rules that you have to understand and comply with, or face potentially severe penalties, if you&#8217;re working with sensitive or identifiable data. The specifics vary substantially based on what is the applicable jurisdiction, but generally speaking every locality has policies that govern certain types of data, particularly human clinical data:<\/span><\/p>\n<blockquote><p><span style=\"font-weight: 400;\">&#8220;Certain types of clinical data can require a computing platform to meet specific standards for data security and access control.&#8221;<\/span><\/p><\/blockquote>\n<p><span style=\"font-weight: 400;\">The specific requirements may involve the need for accreditation from a relevant authority; for example, in the US, there is a government program called FedRAMP that authorizes software and service providers for use by federal agencies. Terra is one of a very small number of life sciences data platforms to be <\/span><a href=\"https:\/\/terra.bio\/terra-security-affirmed-by-fedramp\/\"><span style=\"font-weight: 400;\">authorized under the FedRAMP program<\/span><\/a><span style=\"font-weight: 400;\"> to make controlled-access datasets available to authorized researchers on behalf of certain NIH agencies.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">More globally, many jurisdictions also set limitations on where data can be stored and processed:\u00a0<\/span><\/p>\n<blockquote><p><span style=\"font-weight: 400;\">&#8220;[&#8230;] many countries have data locality policies prohibiting data transfer out of the country.&#8221;<\/span><\/p><\/blockquote>\n<p><span style=\"font-weight: 400;\">These policies can complicate use of cloud infrastructure, and solutions vary depending on the level of stringency adopted by a particular country. You should always check the applicable policies before uploading data to any platform.<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Yet the authors also note the following:<\/span><\/p>\n<blockquote><p><span style=\"font-weight: 400;\">&#8220;Performing computing where the data is located [\u2026] may make it easier to meet regulatory guidelines and avoids the cost and transfer time associated with moving large datasets to different locations.&#8221;<\/span><\/p><\/blockquote>\n<p><span style=\"font-weight: 400;\">From the point of view of cloud infrastructure, this is the flip side of the previous point: if the cloud platform you&#8217;re interested in offers data residency in the &#8220;right&#8221; jurisdiction relative to your needs, it can indeed be a lot easier to give external collaborators access to your data and analyses by inviting them into the platform, rather than trying to figure out how to send them copies of the data while complying with all applicable data residency requirements.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Terra currently only guarantees data residency within the US, though we recently released new features that make it possible to <\/span><a href=\"https:\/\/terra.bio\/choose-your-storage-and-compute-locations-in-terra\/\"><span style=\"font-weight: 400;\">specify data storage and processing locations in specific regions of Google Cloud<\/span><\/a><span style=\"font-weight: 400;\"> (including one in Canada, with additional non-US regions coming soon) for the purpose of optimizing costs. We are working toward support for data residency in other countries as a future improvement.<\/span><\/p>\n<p>&nbsp;<\/p>\n<h3><span style=\"font-weight: 400;\">Technical questions to ask in practical evaluation<\/span><\/h3>\n<p><span style=\"font-weight: 400;\">Finally, the authors of the &#8220;Ten Simple Rules&#8221; paper close this rule with some questions that feel like the tie-breakers you would apply after having narrowed down your options based on the criteria we covered above. Here&#8217;s what the answers look like for Terra.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><b>Is computing available on a first-come-first-serve basis?<\/b><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Essentially, yes, but since Terra is built on the cloud, the resources you need are typically ready to go whenever you are. If you request a specific resource that is in particularly high demand (like <\/span><a href=\"https:\/\/terra.bio\/speed-up-your-machine-learning-work-with-gpus\/\"><span style=\"font-weight: 400;\">certain types of GPUs<\/span><\/a><span style=\"font-weight: 400;\">), you may have to wait for an instance to become available. Keep in mind that the cloud providers monitor usage of these resources and periodically expand their inventory based on demand, so these tend to be transient limitations.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><b>Are there hard limits on resources per task or per user?<\/b><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">The cloud provider (not Terra) imposes some <\/span><a href=\"https:\/\/cloud.google.com\/compute\/quotas\"><span style=\"font-weight: 400;\">resource quotas<\/span><\/a><span style=\"font-weight: 400;\">; for example, a limitation on the number of virtual machines you can have running at the same time. Most of these quotas are fairly reasonable, and in practice you only start hitting them when operating at a very large scale. If and when you reach that point, you can get the provider to raise your quotas by <\/span><a href=\"https:\/\/cloud.google.com\/docs\/quota#requesting_higher_quota\"><span style=\"font-weight: 400;\">making a request<\/span><\/a><span style=\"font-weight: 400;\"> that establishes you&#8217;re a bona fide researcher (as opposed to a shady bitcoin mining bot farm).\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">As a point of interest, the Terra system itself uses some internal queuing logic to keep everything running smoothly for everyone even when someone submits tens of thousands of samples. Fun fact: this feature is called &#8220;<\/span><a href=\"https:\/\/cromwell.readthedocs.io\/en\/stable\/cromwell_features\/HogFactors\"><span style=\"font-weight: 400;\">hog factors<\/span><\/a><span style=\"font-weight: 400;\">&#8221; because it prevents any one user from &#8220;hogging&#8221; the submission queue. Like many guardrails, this was originally spurred by a particular incident in which someone submitted a huge number of workflows on a Friday, exceeding their Google Cloud quota \u2014 so their workflows were queued by Google, and everyone else&#8217;s jobs were held up all weekend even though no one else was exceeding their own quota. Yikes! Since then, we&#8217;ve implemented the &#8220;hog factors&#8221; as well as a number of additional load balancing measures; the result is that Terra can enable individuals to submit very large processing requests without affecting anybody else&#8217;s work.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><b>How do the specific features of hardware, network, and storage impact execution time?<\/b><span style=\"font-weight: 400;\">\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">Terra is set up to offer a lot of flexibility regarding the specific computing resources you can request, so if you have a need for speed, you can typically throw more powerful hardware at your problem.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">For workflows, you can specify the amount of processors, memory and storage space per task, which allows you to customize resource use at a very granular level. This provides ample <\/span><a href=\"https:\/\/terra.bio\/reduce-computing-costs-by-tailoring-resource-allocations-in-workflows\/\"><span style=\"font-weight: 400;\">opportunity for optimization<\/span><\/a><span style=\"font-weight: 400;\">. Interestingly, selecting more powerful instances is not always more expensive; <\/span><a href=\"https:\/\/terra.bio\/speed-up-your-workflows-with-n2-instances-sporting-intels-2nd-generation-xeon-cpus\/\"><span style=\"font-weight: 400;\">in some cases<\/span><\/a><span style=\"font-weight: 400;\">, even though the cost per minute of operation is higher, you can end up paying less overall if the job takes less time to complete.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400;\">For interactive applications, the basic usage is similar: you can dial the hardware specifications of your <\/span><a href=\"https:\/\/support.terra.bio\/hc\/en-us\/sections\/360004143932-Cloud-Environments-Analysis\"><span style=\"font-weight: 400;\">Cloud Environment<\/span><\/a><span style=\"font-weight: 400;\"> up or down depending on what you need. And the system will <\/span><a href=\"https:\/\/support.terra.bio\/hc\/en-us\/articles\/360029761352-Preventing-runaway-costs-with-Cloud-Environment-autopause-\"><span style=\"font-weight: 400;\">turn off any idle instances<\/span><\/a><span style=\"font-weight: 400;\"> on your behalf, so you won&#8217;t accrue any major charges while you&#8217;re away on vacation!<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400;\">Whew, this was a big one. We&#8217;ll return next week with the more straightforward Rule #4: &#8220;Automate your workflows!&#8221;<\/span><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Factors to consider when choosing a platform for large-scale data analysis, inspired by \u201cTen simple rules for large-scale data processing\u201d (Fungtammasan 2022)<\/p>\n","protected":false},"author":4,"featured_media":510,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[53,32],"tags":[],"class_list":["post-509","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-getting-started","category-workflows"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.0 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Ten simple rules \u2014 #3 Understand hardware and regulatory limitations and trade-offs - Terra<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Ten simple rules \u2014 #3 Understand hardware and regulatory limitations and trade-offs - Terra\" \/>\n<meta property=\"og:description\" content=\"Factors to consider when choosing a platform for large-scale data analysis, inspired by \u201cTen simple rules for large-scale data processing\u201d (Fungtammasan 2022)\" \/>\n<meta property=\"og:url\" content=\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/\" \/>\n<meta property=\"og:site_name\" content=\"Terra\" \/>\n<meta property=\"article:published_time\" content=\"2022-03-17T19:17:59+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-27T04:55:18+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/understand_OG.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"627\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Geraldine Van der Auwera\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Geraldine Van der Auwera\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"8 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/\"},\"author\":{\"name\":\"Geraldine Van der Auwera\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/ad0522d0b331a5e08fa1733f65086ee2\"},\"headline\":\"Ten simple rules \u2014 #3 Understand hardware and regulatory limitations and trade-offs\",\"datePublished\":\"2022-03-17T19:17:59+00:00\",\"dateModified\":\"2023-12-27T04:55:18+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/\"},\"wordCount\":1587,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/terra.bio\/#organization\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/understand_OG.png\",\"articleSection\":[\"Getting Started\",\"Workflows\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/\",\"url\":\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/\",\"name\":\"Ten simple rules \u2014 #3 Understand hardware and regulatory limitations and trade-offs - Terra\",\"isPartOf\":{\"@id\":\"https:\/\/terra.bio\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/understand_OG.png\",\"datePublished\":\"2022-03-17T19:17:59+00:00\",\"dateModified\":\"2023-12-27T04:55:18+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#primaryimage\",\"url\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/understand_OG.png\",\"contentUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/understand_OG.png\",\"width\":1200,\"height\":627,\"caption\":\"understand_OG\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/terra.bio\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Ten simple rules \u2014 #3 Understand hardware and regulatory limitations and trade-offs\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/terra.bio\/#website\",\"url\":\"https:\/\/terra.bio\/\",\"name\":\"Terra\",\"description\":\"Science at Scale\",\"publisher\":{\"@id\":\"https:\/\/terra.bio\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/terra.bio\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/terra.bio\/#organization\",\"name\":\"Terra\",\"url\":\"https:\/\/terra.bio\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp\",\"contentUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp\",\"width\":287,\"height\":318,\"caption\":\"Terra\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/ad0522d0b331a5e08fa1733f65086ee2\",\"name\":\"Geraldine Van der Auwera\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/d73bdaf6740465b385e0e3b290786d8cb9d9d548eadec23364254ba06c85204b?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/d73bdaf6740465b385e0e3b290786d8cb9d9d548eadec23364254ba06c85204b?s=96&d=mm&r=g\",\"caption\":\"Geraldine Van der Auwera\"},\"sameAs\":[\"https:\/\/app.terra.bio\/\"],\"url\":\"https:\/\/terra.bio\/author\/geraldinevanterra\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Ten simple rules \u2014 #3 Understand hardware and regulatory limitations and trade-offs - Terra","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/","og_locale":"en_US","og_type":"article","og_title":"Ten simple rules \u2014 #3 Understand hardware and regulatory limitations and trade-offs - Terra","og_description":"Factors to consider when choosing a platform for large-scale data analysis, inspired by \u201cTen simple rules for large-scale data processing\u201d (Fungtammasan 2022)","og_url":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/","og_site_name":"Terra","article_published_time":"2022-03-17T19:17:59+00:00","article_modified_time":"2023-12-27T04:55:18+00:00","og_image":[{"width":1200,"height":627,"url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/understand_OG.png","type":"image\/png"}],"author":"Geraldine Van der Auwera","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Geraldine Van der Auwera","Est. reading time":"8 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#article","isPartOf":{"@id":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/"},"author":{"name":"Geraldine Van der Auwera","@id":"https:\/\/terra.bio\/#\/schema\/person\/ad0522d0b331a5e08fa1733f65086ee2"},"headline":"Ten simple rules \u2014 #3 Understand hardware and regulatory limitations and trade-offs","datePublished":"2022-03-17T19:17:59+00:00","dateModified":"2023-12-27T04:55:18+00:00","mainEntityOfPage":{"@id":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/"},"wordCount":1587,"commentCount":0,"publisher":{"@id":"https:\/\/terra.bio\/#organization"},"image":{"@id":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#primaryimage"},"thumbnailUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/understand_OG.png","articleSection":["Getting Started","Workflows"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/","url":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/","name":"Ten simple rules \u2014 #3 Understand hardware and regulatory limitations and trade-offs - Terra","isPartOf":{"@id":"https:\/\/terra.bio\/#website"},"primaryImageOfPage":{"@id":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#primaryimage"},"image":{"@id":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#primaryimage"},"thumbnailUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/understand_OG.png","datePublished":"2022-03-17T19:17:59+00:00","dateModified":"2023-12-27T04:55:18+00:00","breadcrumb":{"@id":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#primaryimage","url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/understand_OG.png","contentUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/understand_OG.png","width":1200,"height":627,"caption":"understand_OG"},{"@type":"BreadcrumbList","@id":"https:\/\/terra.bio\/ten-simple-rules-3-understand-hardware-and-regulatory-limitations-and-trade-offs\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/terra.bio\/"},{"@type":"ListItem","position":2,"name":"Ten simple rules \u2014 #3 Understand hardware and regulatory limitations and trade-offs"}]},{"@type":"WebSite","@id":"https:\/\/terra.bio\/#website","url":"https:\/\/terra.bio\/","name":"Terra","description":"Science at Scale","publisher":{"@id":"https:\/\/terra.bio\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/terra.bio\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/terra.bio\/#organization","name":"Terra","url":"https:\/\/terra.bio\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/#\/schema\/logo\/image\/","url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp","contentUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp","width":287,"height":318,"caption":"Terra"},"image":{"@id":"https:\/\/terra.bio\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/terra.bio\/#\/schema\/person\/ad0522d0b331a5e08fa1733f65086ee2","name":"Geraldine Van der Auwera","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/d73bdaf6740465b385e0e3b290786d8cb9d9d548eadec23364254ba06c85204b?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d73bdaf6740465b385e0e3b290786d8cb9d9d548eadec23364254ba06c85204b?s=96&d=mm&r=g","caption":"Geraldine Van der Auwera"},"sameAs":["https:\/\/app.terra.bio\/"],"url":"https:\/\/terra.bio\/author\/geraldinevanterra\/"}]}},"_links":{"self":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts\/509","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/comments?post=509"}],"version-history":[{"count":0,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts\/509\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/media\/510"}],"wp:attachment":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/media?parent=509"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/categories?post=509"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/tags?post=509"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}