Skip to main content

Data Model Optimization

Data Model Optimization helps you detect and eliminate duplicate or redundant logic across your data models. When multiple pipelines perform similar transformations, Zingle identifies merge opportunities and lets you consolidate them via pull request.


The optimization dashboard

Navigate to Data Model Optimization in the sidebar. The main view shows:

  • Total estimated savings across all identified opportunities
  • Number of affected models that could be consolidated
  • Duplicate logic detected — count of redundant transformation patterns

Click into any opportunity to see details.


Reviewing an optimization opportunity

Each opportunity card shows:

FieldDescription
Opportunity IDUnique identifier
Affected modelsList of models with overlapping logic
Duplicate patternDescription of the redundant transformation
Estimated savingsCost reduction from merging
Suggested actionMerge recommendation

Merging models

  1. Open the merge view

    Click into an optimization opportunity and select Merge. This opens a dedicated merge interface showing:

    • Before lineage — current state with redundant models
    • After lineage — proposed consolidated state
    • Data comparison — side-by-side data preview
    • SQL comparison — diff of the transformation logic
  2. Review the proposed changes

    Examine:

    • The consolidated SQL to ensure correctness
    • The updated lineage to verify no downstream dependencies break
    • Data samples to confirm output equivalence
  3. Edit if needed

    If the suggested merge isn't quite right, edit the suggestion directly in the interface before committing.

  4. Create the merge PR

    Select a GitHub repository and click Create PR. Zingle:

    1. Creates a branch for the merge
    2. Commits the consolidated model and removes redundant files
    3. Opens a PR with full context on what was merged and why

When to use this

  • After a period of rapid model creation — teams often build similar models independently
  • During quarterly reviews — identify consolidation opportunities to reduce warehouse spend
  • Before scaling — clean up technical debt before adding more complexity