Elektrine lite

← Feed

@noamross@ecoevo.social

2026-05-04 22:49 UTC

@defuneste @jonocarroll @tylermorganwall I don't know a lot about dbt but it seems some intermediate steps might never need to actually be materialized or saved unless explicitly asked for, if the engine is smart enough to combine the sql statements. At some point you need to do something else like fit a model or make a graphic, though. How does that work in dbt? Can you call scripts?

Replies (2)

  • @defuneste@fosstodon.org 2026-05-04 22:56

    @noamross @jonocarroll @tylermorganwall I tested and it with PG mostly and it cached data. This is also one issue with it: not everything is transparent. I think it does different stuff depending on the engine used. This is cool since it is an abstraction layer but sometimes this is also a default. Your sql files are in particular directory and dbt call them for you. I do not think the framework is that flexible. I could be wrong since I did not try all the jinja magic directly.

    Open ##2044473

  • @jonocarroll@fosstodon.org 2026-05-04 22:59

    @noamross @defuneste @tylermorganwall this is one of the points I was trying to make in the Analysis section - I think dbt's scope ends there and expects an analyst to pick up the 'analysis ready' data in a separate project. {targets} can combine all the steps and keep building, but with the trade-off of maybe taking on too much responsibility. Hence my question of 'what is too big?'

    Open ##2044474