{"id":166,"date":"2020-10-28T14:17:05","date_gmt":"2020-10-28T14:17:05","guid":{"rendered":"https:\/\/terrabioappdev.wpenginepowered.com\/new-resources-for-unlocking-the-power-of-terras-data-tables\/"},"modified":"2023-12-27T04:54:01","modified_gmt":"2023-12-27T04:54:01","slug":"new-resources-for-unlocking-the-power-of-terras-data-tables","status":"publish","type":"post","link":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/","title":{"rendered":"New resources for unlocking the power of Terra’s Data tables"},"content":{"rendered":"

“This is not a question. I just wanted to say that I’ve avoided learning how to use Terra Data tables for a long time, primarily because I’ve used workflows that require sample sheet files, but I finally got to try it out and I must say it was quite powerful.”<\/em><\/strong> — A computational scientist, in a recent comment to the Terra helpdesk.<\/p>\n

Data tables are probably the Terra feature that has been the most misunderstood and underutilized until now, despite being stunningly useful when you know how to use them effectively. Let’s take a few minutes to go over why data tables are so great, then I’ll point you to some brand new tutorials that will help you master the key functionalities involved.<\/p>\n

In a nutshell, Terra’s data tables are designed to allow you to describe your dataset in a structured way so that you can launch workflows on batches of data of arbitrary size nearly effortlessly, and have the results associated with the appropriate data entries automatically.<\/p>\n

For example, let’s say that you have a very simple data table in which each row represents a biological sample. The columns include a sample identifier (‘sample_id’), links to where the sequencing data files are stored (‘bam’ and ‘bam_index’), and maybe some additional metadata related to the experiment.<\/p>\n

\"Screen_Shot_2020-10-28_at_1.24.07_AM.png\"<\/p>\n

Terra’s workflow configuration interface allows you to point the workflow system to that table and basically just say “run this workflow on every sample in this table, taking whatever is in the column called ‘bam’ as the data input for each one” (or do the same thing but for a subset of the data). Conveniently, when the processing is finished, the output files will be added to the relevant rows in the table, so you don’t have to go digging for them anywhere — they will automatically be associated with the original data.<\/p>\n

This is conceptually very simple, but incredibly powerful because it scales to just about any cohort size that you might want to process.<\/p>\n

Where things can get a little more complicated is that real datasets will usually involve multiple data entities <\/em>organized in separate tables.<\/em> For example, your samples are typically derived from study participants; samples and participants are two different data entities that are related to each other but possess different kinds of attributes<\/em> (metadata, associated data files, etc), so you’ll want to store them in different tables. Yet you’ll still want to link them together by including a back-reference to the relevant participant in each sample’s row.<\/p>\n

\"data_model__cancer_.png\"<\/p>\n

The formal description of the various entities involved in your experimental design and how they relate to each other is called the data model<\/em>.<\/p>\n

There’s a whole raft of benefits that comes with having a well-structured data model in place. I’m not going to go further into detail about that here, but hopefully, I’ve sparked your interest enough that you want to learn more, yes?<\/p>\n

If you haven’t worked with data tables yet, or if you have tried and struggled, be sure to check out our brand new tutorials on the basic use of data tables<\/a>, importing or adding your own data tables<\/a>, and working with “sets”<\/a>.<\/p>\n

Stay tuned also for a follow-up blog post in which I’ll cover some more advanced topics related to data tables. On the docket: what happens when you import data from various repositories, how we’re thinking about data model harmonization for federated analyses and an early implementation of namespaces for managing data table attributes.<\/p>\n","protected":false},"excerpt":{"rendered":"

“This is not a question. I just wanted to say that I’ve avoided learning how to use Terra Data tables for a long time, primarily because I’ve used workflows that require sample sheet files, but I finally got to try it out and I must say it was quite powerful.” — A computational scientist, in […]<\/p>\n","protected":false},"author":4,"featured_media":167,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[18,24,25],"tags":[27,29,30],"class_list":["post-166","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data","category-data-management","category-data-model","tag-data-management","tag-features","tag-tutorials"],"acf":[],"yoast_head":"\nNew resources for unlocking the power of Terra's Data tables - Terra<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"New resources for unlocking the power of Terra's Data tables - Terra\" \/>\n<meta property=\"og:description\" content=\""This is not a question. I just wanted to say that I've avoided learning how to use Terra Data tables for a long time, primarily because I've used workflows that require sample sheet files, but I finally got to try it out and I must say it was quite powerful." -- A computational scientist, in [...]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/\" \/>\n<meta property=\"og:site_name\" content=\"Terra\" \/>\n<meta property=\"article:published_time\" content=\"2020-10-28T14:17:05+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-27T04:54:01+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Hands_keyboard_security.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"627\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Geraldine Van der Auwera\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Geraldine Van der Auwera\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/\"},\"author\":{\"name\":\"Geraldine Van der Auwera\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/ad0522d0b331a5e08fa1733f65086ee2\"},\"headline\":\"New resources for unlocking the power of Terra’s Data tables\",\"datePublished\":\"2020-10-28T14:17:05+00:00\",\"dateModified\":\"2023-12-27T04:54:01+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/\"},\"wordCount\":608,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/terra.bio\/#organization\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Hands_keyboard_security.png\",\"keywords\":[\"Data Management\",\"Features\",\"Tutorials\"],\"articleSection\":[\"Data\",\"Data Management\",\"Data Model\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/\",\"url\":\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/\",\"name\":\"New resources for unlocking the power of Terra's Data tables - Terra\",\"isPartOf\":{\"@id\":\"https:\/\/terra.bio\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Hands_keyboard_security.png\",\"datePublished\":\"2020-10-28T14:17:05+00:00\",\"dateModified\":\"2023-12-27T04:54:01+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#primaryimage\",\"url\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Hands_keyboard_security.png\",\"contentUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Hands_keyboard_security.png\",\"width\":1200,\"height\":627,\"caption\":\"Hands_keyboard_security\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/terra.bio\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"New resources for unlocking the power of Terra’s Data tables\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/terra.bio\/#website\",\"url\":\"https:\/\/terra.bio\/\",\"name\":\"Terra\",\"description\":\"Science at Scale\",\"publisher\":{\"@id\":\"https:\/\/terra.bio\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/terra.bio\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/terra.bio\/#organization\",\"name\":\"Terra\",\"url\":\"https:\/\/terra.bio\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp\",\"contentUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp\",\"width\":287,\"height\":318,\"caption\":\"Terra\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/ad0522d0b331a5e08fa1733f65086ee2\",\"name\":\"Geraldine Van der Auwera\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/d73bdaf6740465b385e0e3b290786d8cb9d9d548eadec23364254ba06c85204b?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/d73bdaf6740465b385e0e3b290786d8cb9d9d548eadec23364254ba06c85204b?s=96&d=mm&r=g\",\"caption\":\"Geraldine Van der Auwera\"},\"sameAs\":[\"https:\/\/app.terra.bio\/\"],\"url\":\"https:\/\/terra.bio\/author\/geraldinevanterra\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"New resources for unlocking the power of Terra's Data tables - Terra","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/","og_locale":"en_US","og_type":"article","og_title":"New resources for unlocking the power of Terra's Data tables - Terra","og_description":"\"This is not a question. I just wanted to say that I've avoided learning how to use Terra Data tables for a long time, primarily because I've used workflows that require sample sheet files, but I finally got to try it out and I must say it was quite powerful.\" -- A computational scientist, in [...]","og_url":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/","og_site_name":"Terra","article_published_time":"2020-10-28T14:17:05+00:00","article_modified_time":"2023-12-27T04:54:01+00:00","og_image":[{"width":1200,"height":627,"url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Hands_keyboard_security.png","type":"image\/png"}],"author":"Geraldine Van der Auwera","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Geraldine Van der Auwera","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#article","isPartOf":{"@id":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/"},"author":{"name":"Geraldine Van der Auwera","@id":"https:\/\/terra.bio\/#\/schema\/person\/ad0522d0b331a5e08fa1733f65086ee2"},"headline":"New resources for unlocking the power of Terra’s Data tables","datePublished":"2020-10-28T14:17:05+00:00","dateModified":"2023-12-27T04:54:01+00:00","mainEntityOfPage":{"@id":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/"},"wordCount":608,"commentCount":0,"publisher":{"@id":"https:\/\/terra.bio\/#organization"},"image":{"@id":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#primaryimage"},"thumbnailUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Hands_keyboard_security.png","keywords":["Data Management","Features","Tutorials"],"articleSection":["Data","Data Management","Data Model"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/","url":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/","name":"New resources for unlocking the power of Terra's Data tables - Terra","isPartOf":{"@id":"https:\/\/terra.bio\/#website"},"primaryImageOfPage":{"@id":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#primaryimage"},"image":{"@id":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#primaryimage"},"thumbnailUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Hands_keyboard_security.png","datePublished":"2020-10-28T14:17:05+00:00","dateModified":"2023-12-27T04:54:01+00:00","breadcrumb":{"@id":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#primaryimage","url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Hands_keyboard_security.png","contentUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Hands_keyboard_security.png","width":1200,"height":627,"caption":"Hands_keyboard_security"},{"@type":"BreadcrumbList","@id":"https:\/\/terra.bio\/new-resources-for-unlocking-the-power-of-terras-data-tables\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/terra.bio\/"},{"@type":"ListItem","position":2,"name":"New resources for unlocking the power of Terra’s Data tables"}]},{"@type":"WebSite","@id":"https:\/\/terra.bio\/#website","url":"https:\/\/terra.bio\/","name":"Terra","description":"Science at Scale","publisher":{"@id":"https:\/\/terra.bio\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/terra.bio\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/terra.bio\/#organization","name":"Terra","url":"https:\/\/terra.bio\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/#\/schema\/logo\/image\/","url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp","contentUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp","width":287,"height":318,"caption":"Terra"},"image":{"@id":"https:\/\/terra.bio\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/terra.bio\/#\/schema\/person\/ad0522d0b331a5e08fa1733f65086ee2","name":"Geraldine Van der Auwera","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/d73bdaf6740465b385e0e3b290786d8cb9d9d548eadec23364254ba06c85204b?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d73bdaf6740465b385e0e3b290786d8cb9d9d548eadec23364254ba06c85204b?s=96&d=mm&r=g","caption":"Geraldine Van der Auwera"},"sameAs":["https:\/\/app.terra.bio\/"],"url":"https:\/\/terra.bio\/author\/geraldinevanterra\/"}]}},"_links":{"self":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts\/166","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/comments?post=166"}],"version-history":[{"count":0,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts\/166\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/media\/167"}],"wp:attachment":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/media?parent=166"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/categories?post=166"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/tags?post=166"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}