Skip to content

Commit 539c627

Browse files
elliot-huffmanelliot-huffman
committed
Include Additional Details
Add additional charts. Correct existing charts. Add additional information about the update service. Add links to the component map table. Change from bold section headers to actual headers.
1 parent 9585f4b commit 539c627

File tree

1 file changed

+114
-79
lines changed

1 file changed

+114
-79
lines changed

‎docs/Data-Gateway/Architecture/Infrastructure.md‎

Lines changed: 114 additions & 79 deletions
Original file line numberDiff line numberDiff line change
@@ -7,51 +7,54 @@ This page describes how Data Gateway is composed and how the core services inter
77
## Platform topology
88

99
```mermaid
10-
flowchart LR
10+
flowchart TD
11+
12+
%% Entra ID Trust Boundary
13+
subgraph CustomerTrustBoundary[Customer Environment]
14+
Client(["API Client/SDK"])
15+
Users([Users])
16+
end
17+
18+
%% Entra ID Trust Boundary
19+
subgraph EntraTrustBoundary[Microsoft Global Cloud]
20+
Entra{"Entra ID (IDP)"}
21+
end
1122
12-
%% External actors (stadium)
13-
U([Users]):::external
14-
IDP([Entra ID - authentication]):::external
23+
%% GitHub Trust Boundary
24+
subgraph GitHubTrustBoundary[GitHub Pages]
25+
Dashboard((Data Gateway UI))
26+
end
1527
1628
%% SHI Cloud trust boundary
17-
subgraph CLOUD[SHI Cloud]
29+
subgraph ShiTrustBoundary[SHI Cloud]
1830
direction LR
1931
20-
%% Application components
21-
UI[Data Gateway UI]:::component
22-
API[Data Gateway API]:::component
32+
API{{Data Gateway API}}
2333
2434
%% Data services
25-
subgraph DATA[Data services]
35+
subgraph DataStorage[Data Services]
2636
direction TB
27-
SQL[["Azure SQL Database / Processed data"]]:::datastore
28-
BULK[["Azure Blob Storage / Bulk data"]]:::datastore
29-
UPKG[["Azure Blob Storage / Update packages"]]:::datastore
30-
ATAB[["Azure Table Storage / Update config"]]:::datastore
37+
AzSql[("Relational Data (Azure SQL DB)")]
38+
GeneralBlob[/"Bulk data (Azure Blob)"\]
39+
UpdateBlob[/"Update packages (Azure Blob)"\]
40+
AzTable[["Update config (Azure Table)"]]
41+
OpenAi(["LLM Service (Azure OpenAI)"])
3142
end
3243
end
3344
34-
%% Data flows
35-
U -->|HTTPS| UI
36-
UI -->|OAuth2 sign in| IDP
37-
IDP -->|Tokens OIDC OAuth2| UI
38-
UI -->|HTTPS JWT| API
39-
API --> SQL
40-
API --> BULK
41-
API --> UPKG
42-
API --> ATAB
43-
44-
%% Styling
45-
classDef boundary fill:#f8fafc,stroke:#94a3b8,stroke-width:1px,rx:8,ry:8,color:#0f172a;
46-
classDef component fill:#eef2ff,stroke:#6366f1,stroke-width:1px,rx:6,ry:6,color:#111827;
47-
classDef datastore fill:#ecfdf5,stroke:#10b981,stroke-width:1px,rx:6,ry:6,color:#064e3b;
48-
classDef external fill:#fff7ed,stroke:#fb923c,stroke-width:1px,rx:18,ry:18,color:#7c2d12;
49-
50-
class CLOUD boundary
51-
class DATA boundary
52-
class UI,API component
53-
class SQL,BULK,UPKG,ATAB datastore
54-
class U,IDP external
45+
%% Relationships
46+
Users --> | HTTPS | Dashboard
47+
Client --> | Public/Secret Client Auth | Entra
48+
Entra --> | Auth Code/Access Tokens | Client
49+
Client --> API
50+
Dashboard --> | Public Client Auth | Entra
51+
Entra --> | Auth Code/Access Tokens | Dashboard
52+
Dashboard --> | HTTPS + JWT | API
53+
API --> | TDS + JWT | AzSql
54+
API --> | HTTPS + JWT | GeneralBlob
55+
API --> | HTTPS + JWT | UpdateBlob
56+
API --> | HTTPS + JWT | AzTable
57+
API --> | HTTPS + JWT | OpenAi
5558
```
5659

5760
!!! note "Key points"
@@ -62,84 +65,116 @@ flowchart LR
6265
- **Azure Table Storage** for update service configuration
6366
- **LicenseGPT** - Chat interactions are transient. Prompts are sent to the API and responses are returned in-session, never stored.
6467

65-
## Update service (channels & rings)
68+
## SHIELD Update Service
6669

6770
Data Gateway controls update delivery using **channels** (e.g., stable, beta, alpha) and **rings** (e.g., ring 0, ring 1). Configuration is stored in **Azure Table Storage**; package files are stored in **Azure Blob Storage**.
6871

72+
### Example Channel Configuration
73+
74+
The below channel configuration demonstrates how the update system can be configured where various tenants are assigned by default to a ring in a channel.
75+
The tenant's default channel can be overridden by API call. If this happens it defaults to ring 0 on the other channel.
76+
In the below diagram, the channel is not named, all channels follow the below architecture.
77+
There can be an unlimited number of rings in a channel. `N...` represents all numbers above 1.
78+
79+
The Alpha channel is RBAC gated and is not available by default. SHI has to approve Alpha access per-tenant.
80+
81+
```mermaid
82+
flowchart TD
83+
84+
%% Tenant Configs
85+
CxTenant1[/Tenant 1\]
86+
CxTenant2[/Tenant 2\]
87+
CxTenant3[/Tenant 3\]
88+
DevTenant1[/Dev Tenant 1\]
89+
DevTenant2[/Dev Tenant 2\]
90+
91+
%% Channel Example
92+
subgraph Channel["Channel"]
93+
Ring0(("Ring 0<br>Latest"))
94+
Ring1(("Ring 1<br>Latest"))
95+
RingN(("Ring N...<br>Previous"))
96+
end
97+
98+
%% Available Versions
99+
Versions["Latest: 3.0.0<br>Previous: 2.5.0"]
100+
101+
%% Relationships
102+
Versions --> Channel
103+
Ring0 --> CxTenant1
104+
Ring1 --> CxTenant2
105+
RingN --> CxTenant3
106+
Ring0 --> DevTenant1
107+
Ring0 --> DevTenant2
108+
```
109+
110+
### Data Flow
111+
112+
The below chart demonstrates how the data storage systems relate to each other and how the configurations flow to Data Gateway.
113+
69114
```mermaid
70115
flowchart LR
71116
72117
%% Tenant and API
73-
TEN([Tenant]):::external
74-
API[Data Gateway API]:::component
118+
Client([SHIELD])
119+
API{{Data Gateway API}}
75120
76121
%% Configuration in Table Storage
77-
subgraph CFG[Azure Table Storage - update config]
78-
79-
%% spacer line to avoid overlap with title
80-
space[" "]:::transparent
81-
82-
CHAN[["Channels (latest, previous)"]]:::datastore
83-
RING[["Rings (latest true or false)"]]:::datastore
84-
TCONF[["Tenant config (default channel, ring, alpha allowed)"]]:::datastore
122+
subgraph AzureTableStorage[Azure Table Storage]
123+
ChannelConfig[["Channels"]]
124+
RingConfig[["Rings"]]
125+
TenantConfig[["Tenant config"]]
85126
end
86127
87128
%% Packages in Blob
88-
PKG[["Azure Blob Storage - update packages"]]:::datastore
89-
90-
TEN -->|Request version| API
91-
API -->|Read| TCONF
92-
API -->|Read| CHAN
93-
API -->|Read| RING
94-
API -->|Resolve effective version| PKG
95-
API -->|Version or package stream| TEN
96-
97-
%% Styling
98-
classDef boundary fill:#f8fafc,stroke:#94a3b8,stroke-width:1px,rx:8,ry:8,color:#0f172a;
99-
classDef component fill:#eef2ff,stroke:#6366f1,stroke-width:1px,rx:6,ry:6,color:#111827;
100-
classDef datastore fill:#ecfdf5,stroke:#10b981,stroke-width:1px,rx:6,ry:6,color:#064e3b;
101-
classDef external fill:#fff7ed,stroke:#fb923c,stroke-width:1px,rx:18,ry:18,color:#7c2d12;
102-
classDef transparent fill:#ffffff00,stroke:#ffffff00;
103-
104-
class API component
105-
class PKG,CHAN,RING,TCONF datastore
106-
class TEN external
107-
class CFG boundary
129+
PackageStorage[/"Update Packages (Azure Blob)"\]
130+
131+
%% Relationships
132+
Client --> | Request Version | API
133+
API --> | Request Config | TenantConfig
134+
TenantConfig --> | Default channel, Ring, Alpha allowed | API
135+
API --> | Request Config | ChannelConfig
136+
ChannelConfig --> | Latest Version, Previous Version | API
137+
API --> | Request Config | RingConfig
138+
RingConfig --> | Is Latest Available | API
139+
API --> | Request Package | PackageStorage
140+
PackageStorage --> | Package Stream | API
141+
API --> | Version or Package Stream | Client
108142
```
109143

110-
**Update selection process**
144+
### Update Selection Process
111145

112146
1. Tenant defaults define which channel and ring are used, and whether alpha builds are allowed.
113-
2. If you explicitly request a channel and it’s permitted, the API applies it; otherwise the tenant default applies.
147+
2. If you explicitly request a channel and it's permitted, the API applies it; otherwise the tenant default applies.
114148
3. The ring setting (latest or previous) determines the version within that channel.
115149
4. The API returns the resolved version and streams the corresponding package from Blob Storage.
116150

117-
## Data flow lifecycle
151+
### Data Flow Lifecycle
118152

119153
1. **Authenticate** - The UI authenticates with Entra ID and receives a token.
120154
2. **Authorize** - The UI calls the API over HTTPS with the JWT.
121155
3. **Process** - The API validates the token and performs reads/writes against SQL, Blob, or Table Storage.
122156
4. **Respond** - The API returns data or confirmation.
123157
5. **Render** - The UI presents the result to the user. *(LicenseGPT chat is not stored.)*
124158

125-
## Security and reliability
159+
### Security and Reliability
126160

127161
- **Identity** - Entra ID is the single identity provider.
128162
- **Transport** - All calls use HTTPS.
129163
- **Isolation** - Clients never access data stores directly; all access is via the API.
130164
- **Controlled rollout** - Channels and rings enable predictable, staged updates.
131165
- **Resilience** - Operations are designed to handle transient faults.
132166

133-
## Component map
134-
135-
| Component | Backing service(s) | Role |
136-
|---------------------|-------------------------------|----------------------------------------------------------------------|
137-
| Data Gateway UI | Web app (browser) | Entry point for user interactions (Tenant Manager, LicenseGPT) |
138-
| Data Gateway API | HTTPS, Entra ID auth | Authenticates tokens and brokers access to data stores |
139-
| Processed data | **Azure SQL Database** | Relational/reporting data; tenant metadata |
140-
| Bulk report data | **Azure Blob Storage** | JSON/report payloads |
141-
| Update packages | **Azure Blob Storage** | Versioned package archives |
142-
| Update config | **Azure Table Storage** | Channels, rings, and tenant update settings |
167+
### Component Map
168+
169+
| Component | Backing service(s) | Role |
170+
|---------------------|---------------------------------|----------------------------------------------------------------------|
171+
| Data Gateway UI | Web Browser, [GitHub Pages](https://docs.github.com/en/pages) | Entry point for user interactions (Tenant Manager, LicenseGPT) |
172+
| Data Gateway API | HTTPS, [Entra ID auth](https://learn.microsoft.com/en-us/entra/identity-platform/application-model#multitenant-apps) | Authenticates tokens and brokers access to data stores |
173+
| Processed Data | [Azure SQL Database](https://learn.microsoft.com/en-us/azure/azure-sql/database) | Relational/reporting data; tenant metadata |
174+
| Bulk report Data | [Azure Blob Storage](https://learn.microsoft.com/en-us/azure/storage/blobs/) | JSON/report payloads |
175+
| Update Packages | [Azure Blob Storage](https://learn.microsoft.com/en-us/azure/storage/blobs/) | Versioned package archives |
176+
| Update Config | [Azure Table Storage](https://learn.microsoft.com/en-us/azure/storage/tables/) | Channels, rings, and tenant update settings |
177+
| LicenseGPT | [Azure OpenAI Services](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/overview) | Provides LLM and Embeddings to the Data Gateway service |
143178

144179
## See also
145180

0 commit comments

Comments
 (0)