What is Create a Derived Table?
Create a Derived Table is a service that brings dbt, Git, and data access controls together to allow you to deploy tables derived from data available on the Analytical Platform; straight to Athena, in a reproducible way, and with scheduled table updates. All you’ll need to do is submit the SQL to derive your tables along with a few configuration files in a GitHub PR. dbt is at the core of Create a Derived Table and it’s packed full of features for transforming data using SQL so you’ll need to get familiar with certain bits of dbt syntax. To learn more about dbt, take a look at their documentation. Some of the basics about working with dbt are covered below, but you can also sign up and work through dbt’s own training courses for free. The training uses their own user interface instead of Create a Derived Table but it’s still relevant here.
We’re still in beta so we’d love to get some of you using Create a Derived Table to get your feedback to help guide data modelling best practice in the Ministry of Justice and make sure we can continue to improve the user experience.