{"id":571,"date":"2022-06-13T15:00:28","date_gmt":"2022-06-13T15:00:28","guid":{"rendered":"https:\/\/terrabioappdev.wpenginepowered.com\/moving-away-from-multi-regional-storage-buckets\/"},"modified":"2023-12-27T04:55:32","modified_gmt":"2023-12-27T04:55:32","slug":"moving-away-from-multi-regional-storage-buckets","status":"publish","type":"post","link":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/","title":{"rendered":"Moving away from multi-regional storage buckets"},"content":{"rendered":"

Adam Mullen, Software Product Manager in the Data Sciences Platform at the Broad Institute, is responsible for Terra Workspaces. In this blog, he announces a change to the default storage bucket setting for new Terra workspaces that will address the upcoming Google Cloud pricing changes.<\/em><\/p>\n

\n

Sep 2022 Update:<\/span><\/strong><\/p>\n

We’ve negotiated a pricing extension with Google to keep the current storage pricing through March 15th 2024 for Broad billing accounts<\/span><\/i>.\u00a0 <\/span><\/i>Non-Broad billing account managers should reach out to their Google representatives to determine if they have received the pricing extension or are eligible to receive the pricing extension and discuss options.<\/span><\/em><\/p>\n

Given this pricing extension, we have decided to postpone a solution for migrating multi-region buckets until summer 2023.\u00a0 More details will be announced in a follow-up communication around that time. <\/span><\/em><\/p>\n

\n

 <\/p>\n

Google Cloud recently <\/span>announced<\/span><\/a> upcoming changes (starting Oct 1, 2022<\/span>) <\/span>to the pricing structure of a number of services, including multi-regional storage buckets. Crucially, they will start charging egress fees \u2014 which they normally charge when data stored in one region needs to be transferred to another region for computing \u2014 on data stored in multi-regional buckets.<\/span><\/p>\n

Multi-regional buckets provide “<\/span>geo-redundancy<\/span><\/a>“, meaning that the data they contain is copied in data centers located in multiple regions. This offers a degree of protection against data unavailability or loss in case of large-scale failure (such as a disaster affecting an entire region), which can be an attractive feature for some use cases.\u00a0<\/span><\/p>\n

Until this announced pricing change, multi-regional buckets also offered the opportunity to use the data in multiple compute regions without incurring egress fees. That is why, until now, all Terra workspaces were created with a multi-regional bucket (unless you <\/span>specified a region<\/span><\/a> at time of creation).<\/span><\/p>\n

Under the new pricing structure, however, it will no longer be advantageous to use multi-regional buckets for Terra workspaces by default. <\/b>Using a multi-regional storage bucket would lead to egress fees every time you run any analysis on data stored in the bucket.\u00a0<\/span><\/p>\n

 <\/p>\n

Going forward: single-region buckets for all new Terra workspaces<\/span><\/h3>\n

To address this emerging issue, we are switching the default bucket setting for new workspaces from “US multi-regional” to “us-central1”<\/span>, which is the same region Terra uses by default for compute. This will protect you from incurring egress fees related to the region of your workspace buckets that would otherwise result from Google Cloud’s pricing change. In fact, you may notice a slight drop in your storage costs, since single-region buckets are a little cheaper than multi-regional buckets.<\/span><\/p>\n

You will of course still be able to <\/span>select a different storage region<\/span><\/a> when you create a workspace, and the “US multi-regional” option will remain available, with a notice of caution regarding the pricing policy that will be displayed upon selection of that option. There’s also a <\/span>documentation article<\/span><\/a> that details the trade-offs involved.<\/span><\/p>\n

 <\/p>\n

Migrating existing Terra workspaces to single-region storage<\/span><\/h3>\n

There are already over 30,000 Terra workspaces with multi-regional buckets in existence, all of which will be subject to Google Cloud’s new pricing structure starting in October 2022. For those, we are currently developing a migration protocol that will allow us to convert existing workspaces to using single-region storage buckets without breaking any links.\u00a0<\/span><\/p>\n

In order to minimize burden on individual users, we envision doing this on an opt-out basis: we will migrate all workspaces UNLESS the workspace owner requests their workspace(s) be excluded from the migration. We have not yet determined the exact timeframe for this; however, we are committed to providing ample notice, and will send out a separate communication announcing the specifics of the migration plan in due time.\u00a0<\/span><\/p>\n

As always, we are undertaking these changes with the goal of serving the best interests of the research community, and we are open to your suggestions, questions, and concerns. Don’t hesitate to reach out to our support team in the <\/span>Terra community forum<\/span><\/a> or privately through the <\/span>Helpdesk<\/span><\/a> as you prefer.\u00a0<\/span><\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Adam Mullen, Software Product Manager in the Data Sciences Platform, explains a change to the default storage bucket setting for new Terra workspaces that will address the upcoming Google Cloud pricing changes to a number of services, including multi-regional storage buckets.<\/p>\n","protected":false},"author":39,"featured_media":572,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[55,119,14],"tags":[],"class_list":["post-571","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cost-control","category-most-recent","category-workspaces"],"acf":[],"yoast_head":"\nMoving away from multi-regional storage buckets - Terra<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Moving away from multi-regional storage buckets - Terra\" \/>\n<meta property=\"og:description\" content=\"Adam Mullen, Software Product Manager in the Data Sciences Platform, explains a change to the default storage bucket setting for new Terra workspaces that will address the upcoming Google Cloud pricing changes to a number of services, including multi-regional storage buckets.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/\" \/>\n<meta property=\"og:site_name\" content=\"Terra\" \/>\n<meta property=\"article:published_time\" content=\"2022-06-13T15:00:28+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-27T04:55:32+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1447431287.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"627\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Adam Mullen\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Adam Mullen\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/\"},\"author\":{\"name\":\"Adam Mullen\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/7ed4c75930d7f12c74a537228ba198bb\"},\"headline\":\"Moving away from multi-regional storage buckets\",\"datePublished\":\"2022-06-13T15:00:28+00:00\",\"dateModified\":\"2023-12-27T04:55:32+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/\"},\"wordCount\":657,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/terra.bio\/#organization\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1447431287.jpg\",\"articleSection\":[\"Cost Control\",\"Most Recent\",\"Workspaces\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/\",\"url\":\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/\",\"name\":\"Moving away from multi-regional storage buckets - Terra\",\"isPartOf\":{\"@id\":\"https:\/\/terra.bio\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1447431287.jpg\",\"datePublished\":\"2022-06-13T15:00:28+00:00\",\"dateModified\":\"2023-12-27T04:55:32+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#primaryimage\",\"url\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1447431287.jpg\",\"contentUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1447431287.jpg\",\"width\":1200,\"height\":627,\"caption\":\"multi-regional storage buckets\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/terra.bio\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Moving away from multi-regional storage buckets\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/terra.bio\/#website\",\"url\":\"https:\/\/terra.bio\/\",\"name\":\"Terra\",\"description\":\"Science at Scale\",\"publisher\":{\"@id\":\"https:\/\/terra.bio\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/terra.bio\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/terra.bio\/#organization\",\"name\":\"Terra\",\"url\":\"https:\/\/terra.bio\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp\",\"contentUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp\",\"width\":287,\"height\":318,\"caption\":\"Terra\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/7ed4c75930d7f12c74a537228ba198bb\",\"name\":\"Adam Mullen\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/d99bce128215a5e5784e7fbafa1578cb3a6389f12c120959116875c308142c3f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/d99bce128215a5e5784e7fbafa1578cb3a6389f12c120959116875c308142c3f?s=96&d=mm&r=g\",\"caption\":\"Adam Mullen\"},\"url\":\"https:\/\/terra.bio\/author\/amullen\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Moving away from multi-regional storage buckets - Terra","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/","og_locale":"en_US","og_type":"article","og_title":"Moving away from multi-regional storage buckets - Terra","og_description":"Adam Mullen, Software Product Manager in the Data Sciences Platform, explains a change to the default storage bucket setting for new Terra workspaces that will address the upcoming Google Cloud pricing changes to a number of services, including multi-regional storage buckets.","og_url":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/","og_site_name":"Terra","article_published_time":"2022-06-13T15:00:28+00:00","article_modified_time":"2023-12-27T04:55:32+00:00","og_image":[{"width":1200,"height":627,"url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1447431287.jpg","type":"image\/jpeg"}],"author":"Adam Mullen","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Adam Mullen","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#article","isPartOf":{"@id":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/"},"author":{"name":"Adam Mullen","@id":"https:\/\/terra.bio\/#\/schema\/person\/7ed4c75930d7f12c74a537228ba198bb"},"headline":"Moving away from multi-regional storage buckets","datePublished":"2022-06-13T15:00:28+00:00","dateModified":"2023-12-27T04:55:32+00:00","mainEntityOfPage":{"@id":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/"},"wordCount":657,"commentCount":0,"publisher":{"@id":"https:\/\/terra.bio\/#organization"},"image":{"@id":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#primaryimage"},"thumbnailUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1447431287.jpg","articleSection":["Cost Control","Most Recent","Workspaces"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/","url":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/","name":"Moving away from multi-regional storage buckets - Terra","isPartOf":{"@id":"https:\/\/terra.bio\/#website"},"primaryImageOfPage":{"@id":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#primaryimage"},"image":{"@id":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#primaryimage"},"thumbnailUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1447431287.jpg","datePublished":"2022-06-13T15:00:28+00:00","dateModified":"2023-12-27T04:55:32+00:00","breadcrumb":{"@id":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#primaryimage","url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1447431287.jpg","contentUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/shutterstock_1447431287.jpg","width":1200,"height":627,"caption":"multi-regional storage buckets"},{"@type":"BreadcrumbList","@id":"https:\/\/terra.bio\/moving-away-from-multi-regional-storage-buckets\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/terra.bio\/"},{"@type":"ListItem","position":2,"name":"Moving away from multi-regional storage buckets"}]},{"@type":"WebSite","@id":"https:\/\/terra.bio\/#website","url":"https:\/\/terra.bio\/","name":"Terra","description":"Science at Scale","publisher":{"@id":"https:\/\/terra.bio\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/terra.bio\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/terra.bio\/#organization","name":"Terra","url":"https:\/\/terra.bio\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/#\/schema\/logo\/image\/","url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp","contentUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp","width":287,"height":318,"caption":"Terra"},"image":{"@id":"https:\/\/terra.bio\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/terra.bio\/#\/schema\/person\/7ed4c75930d7f12c74a537228ba198bb","name":"Adam Mullen","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/d99bce128215a5e5784e7fbafa1578cb3a6389f12c120959116875c308142c3f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d99bce128215a5e5784e7fbafa1578cb3a6389f12c120959116875c308142c3f?s=96&d=mm&r=g","caption":"Adam Mullen"},"url":"https:\/\/terra.bio\/author\/amullen\/"}]}},"_links":{"self":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts\/571","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/users\/39"}],"replies":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/comments?post=571"}],"version-history":[{"count":0,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts\/571\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/media\/572"}],"wp:attachment":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/media?parent=571"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/categories?post=571"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/tags?post=571"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}