Azure architecture

Metadata-driven data warehouse in your own Azure tenant. Azure Data Factory loads sources into Yres Dataplatform, with SCD2 history, for consumption in Power BI or Fabric.

Sources & IRControl planeOrchestrationAzure data planePower BI / Fabric

Sources & connectivity

CLOUD SOURCES · EXAMPLES

SAP S/4HANA

OData · cloud-hosted

Exact Online

Direct connector · REST API

AFAS Profit

GetConnectors · REST API

Microsoft Teams

Microsoft Graph

Salesforce

CRM · REST API

On-premises · no inbound

SQL Server

Local network; only through the self-hosted IR.

INTEGRATION RUNTIMES

Azure Integration Runtime

Azure-managed · cloud sources

Self-hosted IR

On-prem VM · outbound HTTPS 443
Second node for HA: optional
Shared DEV · TEST · PROD

COPY

Control plane

Yres tenant · SaaS

Users

Business users: reports & monitoring
Developers: sources, tables & changes
Administrators: roles, environments & secrets

Microsoft Entra ID

Azure SSO · frontend login
RBAC: standard and custom CRUD roles

Yres-frontend

Monitoring

Targets, runs, steps & rollback

Data sources

Wizards: source, table & load type

Load Management

Workflows, triggers & persisted views

Changes & Releases

Bundle changes through DTAP

Lineage & impact

Tables, views & procedures; no column lineage

Admin

Users, roles, environments & scaling

↓

Via App Registration · Owner on the resource group

Azure Management APIADF APIKey Vault APIAzure DevOps APISQL

Azure Data Factory

Data plane · customer tenant

`Dynamic Workflow YRES`

Metadata determines source, table, load type and database capacity.

StartRegister the run

Open the run and record the start. Scale the database up if needed.

→

WorkloadRead metadata

Metadata determines sources, tables, load type and settings.

→

Per tableUp to 5 at a time

Copy source data to STAGE.
Process new and changed records into history.
Optionally write mutations to the Data Lake.
Clean up or keep STAGE.
Record status and row counts.

→

ExposeRefresh views

Materialise reporting views in the right order.

→

FinishClose the run

Scale the database back down, close the run and start the refresh.

Source pipelines · a selection of 30+ connectorsCredentials via Key Vault

SAPOData

ExactREST

AFASREST

GraphTeams

SQL Servervia SHIR

SalesforceREST

Load types: FULL · DELTA · DELTAIMAGE · IMAGE · OVERWRITE · RELOAD · ADDITIONAL

LOAD & MERGE

Azure SQL

Yres Dataplatform

Data plane · customer tenant

DATA & HISTORY

STAGE

Hold source data temporarily, including the load date.
Hashes of key and content detect new and changed records.

↓

ODS · HIS layer

Add new records and store changed records as a new version.
SCD2 keeps validity and version history; the current version stays recognisable.
History schema: ODS by default, configurable per customer.

↓

Exposed

Publish reporting views and materialised views.

FRAMEWORK & CUSTOMISATION

LoadManagement

Source metadata, table selection and key management

Config

Settings and database capacity

Monitoring

Load status, steps and row counts

CustomYres

Customer-specific objects and logic

Optional

Data Lake Gen2

Store mutations per table as Parquet: inserts, updates and deletes.
Keep hashes and dates for downstream processing.

READS EXPOSED

Power BI or Fabric

Reads from Exposed

SEMANTIC MODELS

Sales & Finance

SAP + Exact

Report A

Report B

HR & Projects

AFAS

Report A

Report B

Operations

Teams + SQL Server

Report A

Report B

CRM & Pipeline

Salesforce

Report A

Report B

Refresh

Post-load from ADF via the Power BI REST API.

Service principal (Entra ID). Secret in Key Vault.

Azure resources per environment

Resource group · customer tenant

Resource group

…-$ · placeholder

Data Factory

Orchestration

Azure SQL

Yres Dataplatform

Key Vault

kv-yres-… · placeholder

Data Lake Gen2

Optional

Azure Blob Storage

Blob storage

Security & identity

Azure Key Vault

Source credentials & Power BI secret. ADF reads them through its managed identity; the frontend stores no source secrets.

App Registration

Single tenant · Owner on the resource group to assign roles to managed identities.
API permissions: Azure Service Management user_impersonation, Microsoft Graph User.Read, Key Vault & Azure DevOps.

Monitoring & operations

Track load processes

Record status, durations, row counts and errors in SQL.

Health checks & archiving

Check the configuration and archive old history in a controlled way.

DEVadf · Yres Dataplatform · kv

→

TESTadf · Yres Dataplatform · kv

→

PRODadf · Yres Dataplatform · kv

One self-hosted IR, shared by all three environments through a linked IR. Changes & Releases carry structure from DEV to PROD; metadata decides per environment what loads. Each environment has its own ADF, SQL database and Key Vault.