Links

dbt Cloud

Follow these steps to set up a metadata integration from dbt Cloud to Rasgo
These instructions will help you set up the integration to sync metadata from dbt Cloud to Rasgo. If you're just wondering what will be sync'd from dbt Cloud, scroll down to the bottom of this page.

1. Setup a dbt Cloud job

Setup a new job in dbt Cloud to compile the SQL and docs for all dbt models in your project.
The dbt cloud job does not need to run the models, just compile the SQL.
When setting up the new job:
  • Choose your Production environment
  • Check the box for Generate Docs
  • In Commands, add this command: dbt compile --full-refresh
  • In Triggers -> Schedule, Choose to run on schedule, and have it run every day at exactly hour 8 (which is midnight PST; 3 AM EST)
  • Click Save
  • All done! You've made a new job

2. Run your new job

This job needs to run and generate metadata before Rasgo can import it. To run the job, click the green Run Now button, and wait until it completes and shows a green Success status for the run.

3. Copy your API key, account ID, and job ID

To set up the integration in Rasgo, you'll need 3 things:
  1. 1.
    Metadata Only service account token for dbt Cloud (dbt docs setup instructions)
  2. 2.
    dbt Cloud account ID
  3. 3.
    dbt Cloud job ID
To get the IDs, you can pull them directly out of the URL. The URL will be structured like this:
https://cloud.getdbt.com/next/deploy/{{account_id}}/projects/{{project_id}}/jobs/{{job_id}}
Just copy those two numbers, the account_id and job_id, for later and you're ready to go.

4. Add the integration in Rasgo

  • Log into Rasgo and go to the account management screen: https://app.rasgoml.com/profile
  • Find the dbt Connect section at the bottom
  • Click Connect, and paste in the API key, account ID, and job ID
  • Click Connect

5. Rasgo automatically ingests your dbt metadata every night

The integration is set up and good to go! Rasgo will now ingest your dbt metadata every night to make sure it stays updated in Rasgo.
Here is the metadata that Rasgo will ingest from dbt Cloud:
  • dbt models -> Rasgo datasets
    • Description
    • Lineage
    • Columns
      • Column descriptions
    • SQL
  • dbt metrics -> Rasgo metrics
    • Metric definition