{"id":218,"date":"2020-07-17T13:16:16","date_gmt":"2020-07-17T13:16:16","guid":{"rendered":"https:\/\/terrabioappdev.wpenginepowered.com\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/"},"modified":"2023-12-27T04:54:13","modified_gmt":"2023-12-27T04:54:13","slug":"faster-creation-of-notebook-environments-with-google-compute-engine-vms","status":"publish","type":"post","link":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/","title":{"rendered":"Faster creation of Notebook environments with Google Compute Engine VMs"},"content":{"rendered":"<p><strong>Summary:<\/strong> We know that the 4 minutes it takes to create a cloud environment to perform a Jupyter Notebook analysis can feel like a long time. To reduce this time and save you cost, Terra has added support for using standard Google Compute Engine Virtual Machines (GCE VMs) as the underlying compute\/runtime.<\/p>\n<p>Terra researchers frequently use Jupyter Notebooks for genomic analyses, but may not often think about the virtual machine (VM) used for the underlying Notebook compute. However, the VM can be a very important factor in not only how fast you can create a Notebook environment but the types of applications you can use in Terra. When Terra was originally created, we chose a Spark VM for the underlying Notebook compute so you could perform GWAS analysis with the Hail python library. But as Terra has expanded, researchers now use Jupyter Notebooks for multiple use cases beyond Hail, and others require new tools like RStudio and Galaxy which don\u2019t run as efficiently on Spark. That\u2019s why we are thrilled to introduce support for GCE VMs, a first step in faster cloud environment creation and the integration of new Terra applications.<\/p>\n<p>There are multiple advantages to using GCE VMs. Compared to the current Spark VM in Terra, GCE VMs are created in 50% of the time due to fewer installation steps, saving you precious time for your analyses. They also reduce the costs for running and paused VMs by $0.01 per CPU. But perhaps the best part of the GCE VMs is that they provide support for detachable persistent disks, a more durable storage solution that will be required to support new applications like RStudio. Persistent disks will also allow you to save your analysis environment set up and your output files even after you delete your VM (you can read more about them in an upcoming blog).<\/p>\n<p>Based on these benefits, we are now providing GCE VMs as a \u201cStandard\u201d option for your Jupyter Notebook\u2019s underlying compute while continuing to support Hail and Spark. When you create a new cloud environment for Jupyter in Terra, you will have the choice of either a \u201cStandard VM\u201d or a \u201cSpark\u201d VM\/cluster. There are two ways to make the selection:<\/p>\n<ol>\n<li>You can choose the appropriate environment from the \u201cRuntime Configuration\u201d window\u2019s dropdown menu. Depending on the environment you choose, Terra will recommend which VM type to select. For example, if you choose the Default environment, Terra will automatically select \u201cStandard VM\u201d. If you select the Hail environment, Terra will automatically select Spark; you can then choose to create either a single \u201cSpark master node\u201d, or a Spark cluster with worker nodes.<\/li>\n<li>You can choose the VM by simply toggling the \u201cRuntime type\u201d dropdown; select either Standard or Spark VMs.<\/li>\n<\/ol>\n<p>&nbsp;<\/p>\n<p><img decoding=\"async\" src=\"\/wp-content\/uploads\/2020\/11\/Runtime_Config800.png\" alt=\"Runtime_Config800.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>When you choose the Standard VM option, you will find that cloud environments for Jupyter create in ~2 minutes as opposed to the ~4 minutes required to create a Spark VM\/cluster (although sometimes this time fluctuates depending on Google).<\/p>\n<p>We\u2019re currently working on providing support for detachable persistent disks as the next step to improve the analysis experience on Terra.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>We know that the 4 minutes it takes to create a cloud environment to perform a Jupyter Notebook analysis can feel like a long time. To reduce this time and save you cost, Terra has added support for using standard Google Compute Engine Virtual Machines (GCE VMs) as the underlying compute\/runtime. Terra researchers [&#8230;]<\/p>\n","protected":false},"author":9,"featured_media":172,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[43,44,45],"tags":[48,57],"class_list":["post-218","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-features","category-interactive-analysis","category-notebooks","tag-jupyter","tag-speed"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.0 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Faster creation of Notebook environments with Google Compute Engine VMs - Terra<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Faster creation of Notebook environments with Google Compute Engine VMs - Terra\" \/>\n<meta property=\"og:description\" content=\"We know that the 4 minutes it takes to create a cloud environment to perform a Jupyter Notebook analysis can feel like a long time. To reduce this time and save you cost, Terra has added support for using standard Google Compute Engine Virtual Machines (GCE VMs) as the underlying compute\/runtime. Terra researchers [...]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/\" \/>\n<meta property=\"og:site_name\" content=\"Terra\" \/>\n<meta property=\"article:published_time\" content=\"2020-07-17T13:16:16+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-27T04:54:13+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Warp_Speed.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"627\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Adrian Sharma\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Adrian Sharma\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/\"},\"author\":{\"name\":\"Adrian Sharma\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/9438c11587bf259749fbb93c71b38718\"},\"headline\":\"Faster creation of Notebook environments with Google Compute Engine VMs\",\"datePublished\":\"2020-07-17T13:16:16+00:00\",\"dateModified\":\"2023-12-27T04:54:13+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/\"},\"wordCount\":532,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/terra.bio\/#organization\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Warp_Speed.png\",\"keywords\":[\"jupyter\",\"speed\"],\"articleSection\":[\"Features\",\"Interactive Analysis\",\"Notebooks\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/\",\"url\":\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/\",\"name\":\"Faster creation of Notebook environments with Google Compute Engine VMs - Terra\",\"isPartOf\":{\"@id\":\"https:\/\/terra.bio\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Warp_Speed.png\",\"datePublished\":\"2020-07-17T13:16:16+00:00\",\"dateModified\":\"2023-12-27T04:54:13+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#primaryimage\",\"url\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Warp_Speed.png\",\"contentUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Warp_Speed.png\",\"width\":1200,\"height\":627,\"caption\":\"Warp_Speed\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/terra.bio\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Faster creation of Notebook environments with Google Compute Engine VMs\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/terra.bio\/#website\",\"url\":\"https:\/\/terra.bio\/\",\"name\":\"Terra\",\"description\":\"Science at Scale\",\"publisher\":{\"@id\":\"https:\/\/terra.bio\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/terra.bio\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/terra.bio\/#organization\",\"name\":\"Terra\",\"url\":\"https:\/\/terra.bio\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp\",\"contentUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp\",\"width\":287,\"height\":318,\"caption\":\"Terra\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/9438c11587bf259749fbb93c71b38718\",\"name\":\"Adrian Sharma\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/d36fc2e1efc6e9f2aea368747a36c0439ca989f4590a67a66364dce59ee9a701?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/d36fc2e1efc6e9f2aea368747a36c0439ca989f4590a67a66364dce59ee9a701?s=96&d=mm&r=g\",\"caption\":\"Adrian Sharma\"},\"url\":\"https:\/\/terra.bio\/author\/asharma\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Faster creation of Notebook environments with Google Compute Engine VMs - Terra","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/","og_locale":"en_US","og_type":"article","og_title":"Faster creation of Notebook environments with Google Compute Engine VMs - Terra","og_description":"We know that the 4 minutes it takes to create a cloud environment to perform a Jupyter Notebook analysis can feel like a long time. To reduce this time and save you cost, Terra has added support for using standard Google Compute Engine Virtual Machines (GCE VMs) as the underlying compute\/runtime. Terra researchers [...]","og_url":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/","og_site_name":"Terra","article_published_time":"2020-07-17T13:16:16+00:00","article_modified_time":"2023-12-27T04:54:13+00:00","og_image":[{"width":1200,"height":627,"url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Warp_Speed.png","type":"image\/png"}],"author":"Adrian Sharma","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Adrian Sharma","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#article","isPartOf":{"@id":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/"},"author":{"name":"Adrian Sharma","@id":"https:\/\/terra.bio\/#\/schema\/person\/9438c11587bf259749fbb93c71b38718"},"headline":"Faster creation of Notebook environments with Google Compute Engine VMs","datePublished":"2020-07-17T13:16:16+00:00","dateModified":"2023-12-27T04:54:13+00:00","mainEntityOfPage":{"@id":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/"},"wordCount":532,"commentCount":0,"publisher":{"@id":"https:\/\/terra.bio\/#organization"},"image":{"@id":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#primaryimage"},"thumbnailUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Warp_Speed.png","keywords":["jupyter","speed"],"articleSection":["Features","Interactive Analysis","Notebooks"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/","url":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/","name":"Faster creation of Notebook environments with Google Compute Engine VMs - Terra","isPartOf":{"@id":"https:\/\/terra.bio\/#website"},"primaryImageOfPage":{"@id":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#primaryimage"},"image":{"@id":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#primaryimage"},"thumbnailUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Warp_Speed.png","datePublished":"2020-07-17T13:16:16+00:00","dateModified":"2023-12-27T04:54:13+00:00","breadcrumb":{"@id":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#primaryimage","url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Warp_Speed.png","contentUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Warp_Speed.png","width":1200,"height":627,"caption":"Warp_Speed"},{"@type":"BreadcrumbList","@id":"https:\/\/terra.bio\/faster-creation-of-notebook-environments-with-google-compute-engine-vms\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/terra.bio\/"},{"@type":"ListItem","position":2,"name":"Faster creation of Notebook environments with Google Compute Engine VMs"}]},{"@type":"WebSite","@id":"https:\/\/terra.bio\/#website","url":"https:\/\/terra.bio\/","name":"Terra","description":"Science at Scale","publisher":{"@id":"https:\/\/terra.bio\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/terra.bio\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/terra.bio\/#organization","name":"Terra","url":"https:\/\/terra.bio\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/#\/schema\/logo\/image\/","url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp","contentUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp","width":287,"height":318,"caption":"Terra"},"image":{"@id":"https:\/\/terra.bio\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/terra.bio\/#\/schema\/person\/9438c11587bf259749fbb93c71b38718","name":"Adrian Sharma","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/d36fc2e1efc6e9f2aea368747a36c0439ca989f4590a67a66364dce59ee9a701?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d36fc2e1efc6e9f2aea368747a36c0439ca989f4590a67a66364dce59ee9a701?s=96&d=mm&r=g","caption":"Adrian Sharma"},"url":"https:\/\/terra.bio\/author\/asharma\/"}]}},"_links":{"self":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts\/218","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/comments?post=218"}],"version-history":[{"count":0,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts\/218\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/media\/172"}],"wp:attachment":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/media?parent=218"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/categories?post=218"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/tags?post=218"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}