Elektrine lite

← Feed

@noamross@ecoevo.social

2026-05-04 22:43 UTC

@defuneste @jonocarroll @tylermorganwall I think an interesting thing to think about is pipelines where intermediate steps may or may not be "virtual"? Is a dbt step a materialized table or a view? Are multiple steps able to be combined into a single, efficient query via the SQL engine? Targets and R have some challenges taking advantage of some of these situations because of the core object-and-serialization model.

Replies (1)

  • @noamross@ecoevo.social 2026-05-04 22:49

    @defuneste @jonocarroll @tylermorganwall I don't know a lot about dbt but it seems some intermediate steps might never need to actually be materialized or saved unless explicitly asked for, if the engine is smart enough to combine the sql statements. At some point you need to do something else like fit a model or make a graphic, though. How does that work in dbt? Can you call scripts?

    Open ##2044472