{"id":250,"date":"2019-07-19T18:52:22","date_gmt":"2019-07-19T18:52:22","guid":{"rendered":"https:\/\/terrabioappdev.wpenginepowered.com\/workflow-worries-to-working-wdls-in-under-a-week\/"},"modified":"2023-12-27T04:54:21","modified_gmt":"2023-12-27T04:54:21","slug":"workflow-worries-to-working-wdls-in-under-a-week","status":"publish","type":"post","link":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/","title":{"rendered":"Workflow worries to working WDLs in under a week"},"content":{"rendered":"
How insightful feedback about the GATK showcase led to a flurry of updates and the creation of a suite of format conversion tools<\/em><\/h5>\n

Last June, just as we were putting the final touches on the teaching materials for a 4-day GATK workshop that we were planning to run entirely on Terra, we received a very insightful email from the workshop host, Matt Bashton of the University of Newcastle in the UK. A long-time user of the GATK, Matt had been trying out the Best Practices workspaces that we maintain on behalf of the GATK team in the Terra Showcase<\/a>. In his email, he detailed\u00a0a list of nine issues he saw as blocking researchers like him from moving their work onto Terra.<\/p>\n

The workshop was scheduled for the following week, so\u00a0my initial response was a bit of panic. Even though the workshop mainly uses Terra as a convenient environment for running the hands-on tutorials, it’s a unique opportunity for us to introduce the Best Practices workspaces as the recommended way to try out and evaluate the official GATK workflows. That wasn’t going to be terribly effective if there were major flaws in the resources involved.<\/p>\n

Reading down the detailed list Matt had sent us, I understood his concerns:\u00a0for example, he was absolutely right that it wasn\u2019t clear how to convert sequence data to the uBAM format many GATK workflows used as input on the platform. Or how to deal with interleaved FASTQ files. Some issues we were already aware of, such as the lack of support for WDL 1.0 which had been blocking the publication of new and updated GATK workflows on Terra. In the case of WDL 1.0, resolution was imminent (in fact it\u2019s now supported as of July 18th!), but others were a month or more away from being addressed. So while we wanted to make some concrete improvements in time for the workshop, my immediate question was, what could we possibly do in less than a week??<\/p>\n

Turns out, quite a bit more than I was giving our team credit for.<\/p>\n

That Friday afternoon, four members of our Support team sprang into action. It started with a phone call meeting between two team members in the office, one working remotely and me on my drive up to Maine for the weekend (don\u2019t worry, I pulled off the road at a stop before taking the call). We went over the list of issues one by one and within an hour had a list of action items for the four of us to tackle before the workshop.<\/p>\n

By that evening we were working away. We had a very basic file format conversion workspace already, so one team member went through those workflows, updating them to make sure they were configured to be runnable back-to-back (in the right order) without additional tweaks. Another developed an entirely new WDL that converts an interleaved FASTQ input into paired FASTQ files. I worked on the documentation to clarify the purpose and requirements of each workflow, define their input and output formats, and so on. As we finished each part of the action plan, our fourth teammate updated the tools, data, and workspaces in the public showcase.<\/p>\n

Within five days we had completely revamped the barebones file format conversion workspace that we had started from. It now contained one completely new and four significantly updated workflows, as well as a clearer title and description card:<\/p>\n

\"Revamped_FW_card_Screen_Shot.png\"<\/p>\n

We shared the new workspace<\/a><\/strong> with Matt, and his feedback was deeply rewarding: \u201cThanks so much for creating this workspace it\u2019s exactly what I was looking for I suspect it will really help with on-boarding of new people.\u201d Nailed it! We don’t often pull a weekender like that — this was honestly an extreme case (seriously, we know about work-life balance) — but sometimes it’s just worth it, you know?<\/p>\n

Of course, we know that we\u2019re not done yet. In our line of work, you’re never really done. There’s always more to do to clear the path, lay down tracks and make the ride smoother. For my part, the next project I plan to tackle is making all the GATK workspaces follow the same template – more modular, with clearer descriptions and titles.<\/p>\n

I wanted to share this story to encourage all of you to give us your feedback too. Be like Matt. Tell us what you need, what’s blocking you. I can’t promise we’ll always have the solutions to all your troubles, but I can promise we’ll do our best every time.<\/p>\n","protected":false},"excerpt":{"rendered":"

How insightful feedback about the GATK showcase led to a flurry of updates and the creation of a suite of format conversion tools. Last June, just as we were putting the final touches on the teaching materials for a 4-day GATK workshop that we were planning to run entirely on Terra, we received […]<\/p>\n","protected":false},"author":54,"featured_media":251,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[42,43,50,32],"tags":[84,85],"class_list":["post-250","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-community","category-features","category-team","category-workflows","tag-cromwell","tag-wdl"],"acf":[],"yoast_head":"\nWorkflow worries to working WDLs in under a week - Terra<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Workflow worries to working WDLs in under a week - Terra\" \/>\n<meta property=\"og:description\" content=\"How insightful feedback about the GATK showcase led to a flurry of updates and the creation of a suite of format conversion tools. Last June, just as we were putting the final touches on the teaching materials for a 4-day GATK workshop that we were planning to run entirely on Terra, we received [...]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/\" \/>\n<meta property=\"og:site_name\" content=\"Terra\" \/>\n<meta property=\"article:published_time\" content=\"2019-07-19T18:52:22+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-27T04:54:21+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Cloud_people_electric.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"627\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Allie Cliffe\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Allie Cliffe\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/\"},\"author\":{\"name\":\"Allie Cliffe\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/991fd50107233fb428223aaa5f292bef\"},\"headline\":\"Workflow worries to working WDLs in under a week\",\"datePublished\":\"2019-07-19T18:52:22+00:00\",\"dateModified\":\"2023-12-27T04:54:21+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/\"},\"wordCount\":778,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/terra.bio\/#organization\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Cloud_people_electric.png\",\"keywords\":[\"cromwell\",\"wdl\"],\"articleSection\":[\"Community\",\"Features\",\"Team\",\"Workflows\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/\",\"url\":\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/\",\"name\":\"Workflow worries to working WDLs in under a week - Terra\",\"isPartOf\":{\"@id\":\"https:\/\/terra.bio\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Cloud_people_electric.png\",\"datePublished\":\"2019-07-19T18:52:22+00:00\",\"dateModified\":\"2023-12-27T04:54:21+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#primaryimage\",\"url\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Cloud_people_electric.png\",\"contentUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Cloud_people_electric.png\",\"width\":1200,\"height\":627,\"caption\":\"Cloud_people_electric\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/terra.bio\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Workflow worries to working WDLs in under a week\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/terra.bio\/#website\",\"url\":\"https:\/\/terra.bio\/\",\"name\":\"Terra\",\"description\":\"Science at Scale\",\"publisher\":{\"@id\":\"https:\/\/terra.bio\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/terra.bio\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/terra.bio\/#organization\",\"name\":\"Terra\",\"url\":\"https:\/\/terra.bio\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp\",\"contentUrl\":\"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp\",\"width\":287,\"height\":318,\"caption\":\"Terra\"},\"image\":{\"@id\":\"https:\/\/terra.bio\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/991fd50107233fb428223aaa5f292bef\",\"name\":\"Allie Cliffe\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/terra.bio\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/81a0a851dbdad36c85e60e9065cc6fa35168bd2d3324584f00d70b54d8e575dc?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/81a0a851dbdad36c85e60e9065cc6fa35168bd2d3324584f00d70b54d8e575dc?s=96&d=mm&r=g\",\"caption\":\"Allie Cliffe\"},\"url\":\"https:\/\/terra.bio\/author\/acliffe\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Workflow worries to working WDLs in under a week - Terra","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/","og_locale":"en_US","og_type":"article","og_title":"Workflow worries to working WDLs in under a week - Terra","og_description":"How insightful feedback about the GATK showcase led to a flurry of updates and the creation of a suite of format conversion tools. Last June, just as we were putting the final touches on the teaching materials for a 4-day GATK workshop that we were planning to run entirely on Terra, we received [...]","og_url":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/","og_site_name":"Terra","article_published_time":"2019-07-19T18:52:22+00:00","article_modified_time":"2023-12-27T04:54:21+00:00","og_image":[{"width":1200,"height":627,"url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Cloud_people_electric.png","type":"image\/png"}],"author":"Allie Cliffe","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Allie Cliffe","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#article","isPartOf":{"@id":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/"},"author":{"name":"Allie Cliffe","@id":"https:\/\/terra.bio\/#\/schema\/person\/991fd50107233fb428223aaa5f292bef"},"headline":"Workflow worries to working WDLs in under a week","datePublished":"2019-07-19T18:52:22+00:00","dateModified":"2023-12-27T04:54:21+00:00","mainEntityOfPage":{"@id":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/"},"wordCount":778,"commentCount":0,"publisher":{"@id":"https:\/\/terra.bio\/#organization"},"image":{"@id":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#primaryimage"},"thumbnailUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Cloud_people_electric.png","keywords":["cromwell","wdl"],"articleSection":["Community","Features","Team","Workflows"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/","url":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/","name":"Workflow worries to working WDLs in under a week - Terra","isPartOf":{"@id":"https:\/\/terra.bio\/#website"},"primaryImageOfPage":{"@id":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#primaryimage"},"image":{"@id":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#primaryimage"},"thumbnailUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Cloud_people_electric.png","datePublished":"2019-07-19T18:52:22+00:00","dateModified":"2023-12-27T04:54:21+00:00","breadcrumb":{"@id":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#primaryimage","url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Cloud_people_electric.png","contentUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Cloud_people_electric.png","width":1200,"height":627,"caption":"Cloud_people_electric"},{"@type":"BreadcrumbList","@id":"https:\/\/terra.bio\/workflow-worries-to-working-wdls-in-under-a-week\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/terra.bio\/"},{"@type":"ListItem","position":2,"name":"Workflow worries to working WDLs in under a week"}]},{"@type":"WebSite","@id":"https:\/\/terra.bio\/#website","url":"https:\/\/terra.bio\/","name":"Terra","description":"Science at Scale","publisher":{"@id":"https:\/\/terra.bio\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/terra.bio\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/terra.bio\/#organization","name":"Terra","url":"https:\/\/terra.bio\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/#\/schema\/logo\/image\/","url":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp","contentUrl":"https:\/\/terra.bio\/wp-content\/uploads\/2023\/12\/Terra-Bio-App@2x.webp","width":287,"height":318,"caption":"Terra"},"image":{"@id":"https:\/\/terra.bio\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/terra.bio\/#\/schema\/person\/991fd50107233fb428223aaa5f292bef","name":"Allie Cliffe","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/terra.bio\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/81a0a851dbdad36c85e60e9065cc6fa35168bd2d3324584f00d70b54d8e575dc?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/81a0a851dbdad36c85e60e9065cc6fa35168bd2d3324584f00d70b54d8e575dc?s=96&d=mm&r=g","caption":"Allie Cliffe"},"url":"https:\/\/terra.bio\/author\/acliffe\/"}]}},"_links":{"self":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts\/250","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/users\/54"}],"replies":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/comments?post=250"}],"version-history":[{"count":0,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/posts\/250\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/media\/251"}],"wp:attachment":[{"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/media?parent=250"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/categories?post=250"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/terra.bio\/wp-json\/wp\/v2\/tags?post=250"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}