SMP Forecast IQ

AI-powered System Marginal Price forecasting for Malaysian SMB energy buyers
Built with Microsoft Fabric · Azure ML · FastAPI · Next.js · Azure OpenAI
Developed with GitHub Copilot

---

Table of Contents

1. Demo Overview
2. Business Use Case
3. Azure Services Used
4. Architecture
5. Folder Structure
6. Local Setup
7. Azure Deployment
8. Data Source Configuration
9. How to Run Ingestion
10. How to Train the Model
11. How to Deploy the Azure ML Endpoint
12. How to Run the Backend
13. How to Run the Frontend
14. Demo Storyline
15. Known Limitations
16. Production Roadmap

---

1. Demo Overview

SMP Forecast IQ is a full-stack demonstration platform that transforms publicly available Malaysian National Energy Market (NEM) data into a boardroom-ready SMP forecasting command center.

The platform delivers:

Capability	Description
Multi-horizon Forecasting	1-day, 7-day, 30-day, and 90-day SMP forecasts with confidence intervals
Scenario Simulator	Interactive what-if analysis across demand growth, fuel prices, generation mix, and reserve margin
Driver Analysis	SHAP-based feature importance explaining which market signals are driving the forecast
AI Market Analyst	Natural-language Q&A on market conditions and forecast rationale, powered by Azure OpenAI GPT-4o
Data Cockpit	Real-time view of data source health, ingestion status, and quality metrics
Demo Mode	Fully offline mode using synthetic Malaysia-calibrated data — no live API dependency

The entire platform — from data ingestion to dashboard — was built with GitHub Copilot in a matter of weeks, demonstrating the developer productivity gains from AI-assisted engineering.

---

2. Business Use Case

Malaysia's NEM sets a System Marginal Price (SMP) every 30 minutes. That price determines what large industrial and commercial buyers pay for electricity. It fluctuates from approximately RM 100/MWh at off-peak to RM 1,800/MWh during scarcity events, with a long-run average near RM 184/MWh.

For a facility consuming 1 MW continuously:

Metric	Value
Annual electricity spend at average SMP	~RM 1.6 million
Value of a 3% procurement optimisation	~RM 48,000/year
Value of a 10% procurement optimisation	~RM 160,000/year

Why SMP is Hard to Predict Manually

Six interconnected drivers affect SMP simultaneously:

Driver	Impact on SMP	Source
Electricity Demand	High demand → higher SMP	Suruhanjaya Tenaga (ST)
Gas Price (JKM LNG)	Gas at ~42% of generation mix	PETRONAS / JKM
Coal Price (Newcastle)	Coal at ~38% of mix	Newcastle benchmark
Reserve Margin	Below 20% → scarcity pricing	ST outage registry
Generation Mix	Renewables displace gas generation	ST metering data
Time of Day / Season	Peak-hour premium of 40–60%	Historical NEM data

An analyst tracking all six dimensions in spreadsheets would need hours per day. SMP Forecast IQ replaces that manual work with a model that re-runs every 30 minutes as new data arrives, and surfaces the result in an executive-ready dashboard.

Target Personas

• Finance Director / CFO — Budget visibility for electricity cost exposure
• Procurement Manager — Contract timing and hedging decisions
• Operations Lead — Flexible load shifting to off-peak windows
• Energy Consultant — Client advisory backed by data-driven analysis

---

3. Azure Services Used

Layer	Service	Purpose
Data Pipeline	Microsoft Fabric / Azure Data Factory	Scheduled ingestion of NEM + fuel price data
Storage	ADLS Gen2 / Fabric Lakehouse	Bronze / Silver / Gold medallion data lake
Machine Learning	Azure Machine Learning	Model training, registry, managed online endpoint
ML Ops	Azure ML Model Monitor	Data drift detection, retraining triggers
Backend	Azure App Service	Hosts the FastAPI REST API (Python 3.11)
AI	Azure OpenAI (GPT-4o)	Natural-language market explanations
Frontend	Azure Static Web Apps	Hosts the Next.js 14 dashboard
Delivery	Azure CDN	Edge-caches static frontend assets globally
Identity	Microsoft Entra ID	RBAC · OAuth 2.0 · Managed Identity
Secrets	Azure Key Vault	API keys, endpoint URIs — zero secrets in code
Observability	Application Insights	Request tracing, custom metrics, alerts
Observability	Log Analytics Workspace	Centralised logs, KQL queries, dashboards
IaC	Azure Bicep	All infrastructure version-controlled in infra/
CI/CD	GitHub Actions	Lint → build → test → deploy pipeline

Why This Stack?

This solution is intentionally built on the Microsoft Azure Well-Architected Framework:

• Reliability — Retry + fallback + synthetic data ensures the dashboard always loads
• Security — Managed Identity throughout; no secrets in code or environment variables
• Cost — Consumption-based ML endpoint; autoscale App Service; CDN caches frontend
• Performance — In-process model fallback delivers < 50 ms inference latency
• Operational Excellence — Automated drift monitoring and blue/green model rollout

---

4. Architecture

┌────────────────────────────────────────────────────────────────────┐
│  PUBLIC DATA SOURCES                                                │
│  Malaysia NEM SMP · System Demand · Gas Price (JKM) · Coal Price  │
│  (Newcastle) · Grid Reserve Margin                                 │
└───────────────────────────┬────────────────────────────────────────┘
                            │  Fabric Data Pipeline / Python ingestion
                            ▼
┌────────────────────────────────────────────────────────────────────┐
│  FABRIC LAKEHOUSE  /  ADLS Gen2                                     │
│  Bronze (raw) → Silver (cleaned) → Gold (44-feature table)        │
│  Data quality gates at every layer · Azure Key Vault              │
└───────────────────────────┬────────────────────────────────────────┘
                            │  Azure ML Pipeline
                            ▼
┌────────────────────────────────────────────────────────────────────┐
│  AZURE MACHINE LEARNING                                             │
│  XGBoost training · Model Registry · Managed Online Endpoint      │
│  4 horizons: 1D / 7D / 30D / 90D · Model Monitor (drift alerts)  │
└───────────────────────────┬────────────────────────────────────────┘
                            │  HTTPS / Managed Identity
                            ▼
┌────────────────────────────────────────────────────────────────────┐
│  FASTAPI BACKEND  (Azure App Service)                               │
│  9 endpoints · Azure OpenAI GPT-4o · Managed Identity             │
│  In-process joblib fallback if ML endpoint is unavailable          │
└───────────────────────────┬────────────────────────────────────────┘
                            │  HTTPS REST
                            ▼
┌────────────────────────────────────────────────────────────────────┐
│  NEXT.JS 14 FRONTEND  (Azure Static Web Apps · CDN)                │
│  8 pages: Landing · Cockpit · Forecast · Drivers · Scenario       │
│  AI Analyst · Storyline · Architecture                            │
└───────────────────────────┬────────────────────────────────────────┘
                            ▼
                    Executive User

Observability: Application Insights + Log Analytics
Identity:      Microsoft Entra ID · Managed Identity · Key Vault
IaC:           Azure Bicep (infra/main.bicep)
CI/CD:         GitHub Actions

For full architecture detail — including the security model, monitoring KQL queries, model lifecycle, data quality checks, and fallback design — see docs/architecture.md.

---

5. Folder Structure

smp-forecast-iq/
│
├── backend/                     FastAPI REST API
│   ├── main.py                  App entry point, CORS, middleware
│   ├── config.py                Pydantic settings (reads .env)
│   ├── requirements.txt
│   ├── .env.example             Template — copy to .env
│   ├── routers/
│   │   ├── forecast.py          GET /forecast
│   │   ├── data.py              GET /data/status
│   │   ├── explain.py           GET /explain
│   │   ├── ai_explain.py        POST /ai-explain
│   │   ├── metrics.py           GET /metrics
│   │   └── demo.py              GET /demo/storyline
│   └── services/
│       ├── forecast_service.py  Calls Azure ML endpoint or joblib fallback
│       ├── model_service.py     Loads / caches local model bundles
│       ├── explain_service.py   SHAP computation
│       ├── ai_explainer.py      Azure OpenAI GPT-4o integration
│       └── data_service.py      Reads data_status.json
│
├── frontend/                    Next.js 14 + TypeScript + Tailwind CSS
│   ├── src/
│   │   ├── app/                 App Router pages
│   │   │   ├── page.tsx         / — Landing page + KPI cards
│   │   │   ├── cockpit/         /cockpit — Data source health
│   │   │   ├── forecast/        /forecast — SMP chart + procurement signal
│   │   │   ├── drivers/         /drivers — Feature importance
│   │   │   ├── scenario/        /scenario — What-if simulator
│   │   │   ├── analyst/         /analyst — AI Market Analyst chat
│   │   │   ├── storyline/       /storyline — Demo walkthrough
│   │   │   └── architecture/    /architecture — Solution architecture
│   │   ├── components/
│   │   │   ├── ui/              Card, Badge, Nav, Skeleton
│   │   │   ├── charts/          ForecastChart (Recharts)
│   │   │   └── dashboard/       MetricsPanel, ScenarioPanel
│   │   ├── lib/
│   │   │   ├── api.ts           API client with mock fallback
│   │   │   ├── mock.ts          Synthetic data for offline demo
│   │   │   ├── storyline.ts     Canonical demo storyline data
│   │   │   └── utils.ts         Formatters, design helpers
│   │   └── types/
│   │       └── index.ts         All TypeScript interfaces
│   ├── .env.local.example
│   └── package.json
│
├── ml/                          Data + ML pipeline scripts
│   ├── ingest_public_data.py    Ingest NEM SMP, demand, fuel prices
│   ├── generate_synthetic_energy_data.py  Malaysia-calibrated synthetic data
│   ├── feature_engineering.py  Build 44-feature training dataset
│   ├── train_baseline_model.py  Local XGBoost/LightGBM training
│   ├── train_automl_azureml.py  Azure ML AutoML training job
│   ├── evaluate_model.py        Model evaluation + metrics report
│   ├── deploy_model_endpoint.py Deploy to Azure ML Managed Online Endpoint
│   ├── score.py                 Scoring script used by the endpoint
│   └── artifacts/               Trained model bundles (gitignored)
│
├── infra/                       Azure infrastructure
│   ├── main.bicep               Bicep template — all Azure resources
│   ├── parameters.json          Deployment parameters
│   ├── deploy_azure.ps1         One-click deploy (PowerShell / Windows)
│   ├── deploy_azure.sh          One-click deploy (Bash / macOS / Linux)
│   └── .env.azure.example       Template — generated by deploy scripts
│
├── data/
│   ├── raw/                     Raw ingested CSVs (gitignored)
│   ├── curated/                 Feature-engineered Parquet (gitignored)
│   └── data_status.json         Per-source ingestion metadata
│
├── sample-data/                 Checked-in sample CSVs for offline use
│
└── docs/
    ├── architecture.md          Full architecture documentation
    ├── demo-storyline.md        15-minute demo script
    ├── data-sources.md          Data source reference
    └── azureml-setup.md         Azure ML workspace setup guide

---

6. Local Setup

Prerequisites

Requirement	Version
Python	3.10 or higher
Node.js	18 or higher
npm	9 or higher

1. Clone the repository

git clone https://github.com/<your-org>/smp-forecast-iq.git
cd smp-forecast-iq

2. Backend — install dependencies

cd backend
pip install -r requirements.txt

3. Backend — configure environment

cp .env.example .env

Key variables in backend/.env:

# Set to true for offline demos (no Azure required)
DEMO_MODE=true

# Azure ML endpoint (optional — leave blank to use local model)
AZUREML_ENDPOINT_URL=
AZUREML_ENDPOINT_KEY=

# Azure OpenAI (optional — AI Analyst will be disabled if not set)
AZURE_OPENAI_ENDPOINT=https://<your-resource>.openai.azure.com/
AZURE_OPENAI_DEPLOYMENT=gpt-4o
AZURE_OPENAI_API_KEY=       # Leave blank to use Managed Identity

# Allowed frontend origins
CORS_ORIGINS=http://localhost:3000

4. Frontend — install dependencies

cd frontend
npm install

5. Frontend — configure environment

Create frontend/.env.local:

NEXT_PUBLIC_API_URL=http://localhost:8000
NEXT_PUBLIC_USE_MOCK=true

Set NEXT_PUBLIC_USE_MOCK=true to run the frontend entirely from local mock data without needing the backend running.

6. Generate a local model (first time only)

# From project root — generates synthetic data + trains local model
python ml/generate_synthetic_energy_data.py
python ml/feature_engineering.py
python ml/train_baseline_model.py

Trained model bundles are saved to ml/artifacts/.

---

7. Azure Deployment

Prerequisites

• Azure CLI installed and logged in (az login)
• Contributor access to the target subscription
• Azure ML workspace provisioned (see docs/azureml-setup.md)

One-click deploy

Windows (PowerShell):

cd infra
.\deploy_azure.ps1 `
  -SubscriptionId "<your-subscription-id>" `
  -ResourceGroup "smp-forecast-iq-rg" `
  -Location "southeastasia"

macOS / Linux (Bash):

cd infra
chmod +x deploy_azure.sh
./deploy_azure.sh \
  --subscription-id "<your-subscription-id>" \
  --resource-group "smp-forecast-iq-rg" \
  --location southeastasia

The script:

1. Creates the resource group and provisions all resources via infra/main.bicep
2. Assigns Managed Identity roles (Azure ML, OpenAI, Key Vault, Storage)
3. Writes all output values to infra/.env.azure
4. Prints next-step instructions

Resources provisioned by Bicep

Resource	SKU / Tier
Log Analytics Workspace	PerGB2018 · 30-day retention
Application Insights	Web · linked to Log Analytics
Azure Key Vault	Standard
Azure Storage Account (ADLS Gen2)	LRS
Azure ML Workspace	Basic
Azure Container Registry	Basic
Azure App Service Plan	B1
Azure App Service (Backend)	Python 3.11
Azure Static Web App (Frontend)	Free tier

After deployment, copy the generated env file to the services:

cp infra/.env.azure backend/.env
cp infra/.env.azure frontend/.env.local

---

8. Data Source Configuration

SMP Forecast IQ ingests data from five public Malaysia NEM sources. Each source is attempted in order (Direct HTTP → Manual file → Synthetic fallback) so the pipeline never blocks.

Source	Provider	Frequency	Ingestion Mode
SMP (System Marginal Price)	singlebuyer.com.my	30-min	HTTP / Synthetic
System Demand	singlebuyer.com.my	30-min	HTTP / Synthetic
Supply Mix	singlebuyer.com.my	Daily	HTTP / Synthetic
Generation Profile	gso.org.my	10-min	HTTP / Synthetic
Fuel Prices (Gas, Coal, Oil)	World Bank / ST	Daily	HTTP / CSV / Synthetic

Providing a manual fuel price file

If the live fuel price API is unavailable, supply a CSV:

date,gas_price_myr_per_mmbtu,coal_price_usd_per_t,oil_price_usd_per_bbl
2026-01-01,16.5,130.2,78.4
2026-01-02,16.6,130.5,78.9

Pass it to the ingestion script with --fuel-csv path/to/fuel_prices.csv.

Demo / offline mode

Set DEMO_MODE=true in backend/.env to activate synthetic data for all sources. The dashboard will display an amber "Synthetic data active" indicator. No external API calls are made.

For full data source documentation, see docs/data-sources.md.

---

9. How to Run Ingestion

All scripts run from the project root directory.

# Full ingestion — all sources, today's date range
python ml/ingest_public_data.py

# Specific date range
python ml/ingest_public_data.py --start 2026-04-01 --end 2026-05-07

# Specific sources only
python ml/ingest_public_data.py --sources smp demand gso

# Provide manual fuel price CSV
python ml/ingest_public_data.py --fuel-csv sample-data/fuel_prices.csv

# Force synthetic data (Mode C) — no external API calls
python ml/ingest_public_data.py --mode C

# Force direct HTTP only — no synthetic fallback (Mode A)
python ml/ingest_public_data.py --mode A

Output locations:

Path	Content
data/raw/	Raw CSVs, one file per source per run
data/curated/	Cleaned, standardised CSVs and Parquet files
data/data_status.json	Per-source ingestion metadata and quality flags

After ingestion, run the feature engineering step:

python ml/feature_engineering.py
# Output: data/curated/smp_training_dataset.parquet

---

10. How to Train the Model

Option A — Local training (no Azure required)

Trains four separate XGBoost / LightGBM models, one per forecast horizon. Quantile regression models are trained separately for 10th and 90th percentile confidence bounds.

# Train all four horizons (1d, 7d, 30d, 90d)
python ml/train_baseline_model.py

# Train specific horizons only
python ml/train_baseline_model.py --horizons 1d 7d

# Custom input dataset path
python ml/train_baseline_model.py --input data/curated/smp_training_dataset.parquet

# Adjust train/test split (default: 80/20)
python ml/train_baseline_model.py --train-frac 0.75 --verbose

Output — ml/artifacts/:

File	Content
model_1d.joblib	Trained model bundle for day-ahead horizon
model_7d.joblib	7-day horizon
model_30d.joblib	30-day horizon
model_90d.joblib	90-day horizon
metrics.json	MAE, RMSE, MAPE, R², directional accuracy per horizon
feature_importance.json	Top-30 SHAP feature importances per horizon
sample_forecast.json	96-period test-set predictions with CI bounds

Option B — Azure ML AutoML (recommended for production)

Requires an Azure ML workspace configured (see docs/azureml-setup.md) and azure-ai-ml installed:

pip install azure-ai-ml azure-identity

Set the required environment variables:

AZURE_SUBSCRIPTION_ID=<your-subscription-id>
AZURE_RESOURCE_GROUP=smp-forecast-iq-rg
AZURE_ML_WORKSPACE=<your-workspace-name>

Then run:

# Validate config without submitting jobs
python ml/train_automl_azureml.py --dry-run

# Submit AutoML jobs for all horizons
python ml/train_automl_azureml.py

# Submit for specific horizons with 60-min timeout per job
python ml/train_automl_azureml.py --horizons 1d 7d --timeout 60

# Use hourly aggregation (recommended for 30d/90d horizons)
python ml/train_automl_azureml.py --aggregate-hourly --register

AutoML searches over LightGBM, XGBoost, Prophet, ARIMA, TCNForecaster, and ElasticNet. The best model per horizon is automatically registered in the Azure ML Model Registry.

---

11. How to Deploy the Azure ML Endpoint

Prerequisites

• Azure ML workspace provisioned
• Models trained and available in ml/artifacts/ or the ML Model Registry
• azure-ai-ml package installed

pip install azure-ai-ml azure-identity

Deploy

# Dry-run — validate config, print deployment plan (no Azure calls)
python ml/deploy_model_endpoint.py --dry-run

# Deploy all four horizon models to a Managed Online Endpoint
python ml/deploy_model_endpoint.py

# Deploy specific horizons only
python ml/deploy_model_endpoint.py --horizons 1d 7d

# Local FastAPI inference server (no Azure ML required)
python ml/deploy_model_endpoint.py --local-only

# Local server on a custom port
python ml/deploy_model_endpoint.py --local-only --port 9000

The deployment script:

1. Registers each model bundle in the Azure ML Model Registry
2. Creates (or reuses) a ManagedOnlineEndpoint named smp-forecast-endpoint
3. Creates a blue deployment with SKU Standard_DS2_v2
4. Sets 100% traffic to blue
5. Writes the scoring URI and API key to ml/artifacts/deployment_report.json

After deployment, update backend/.env:

AZUREML_ENDPOINT_URL=<scoring-uri from deployment_report.json>
AZUREML_ENDPOINT_KEY=<api-key from deployment_report.json>

---

12. How to Run the Backend

cd backend

# Development server with auto-reload
uvicorn main:app --reload --port 8000

# Production-style (no reload)
uvicorn main:app --host 0.0.0.0 --port 8000

The API will be available at http://localhost:8000. Interactive docs are at http://localhost:8000/docs.

API Endpoints

Method	Path	Description
GET	/health	Liveness check — returns version, uptime, mode
GET	/data/status	Per-source ingestion status and quality flags
POST	/forecast	SMP forecast for a given horizon and model
POST	/forecast/scenario	Scenario forecast with parameter deltas
GET	/explain	SHAP feature importance for last forecast
POST	/ai-explain	GPT-4o natural-language explanation
GET	/ai-explain/briefing	Executive briefing summary
GET	/metrics	Model accuracy metrics (MAE, RMSE, MAPE)
GET	/demo/storyline	Structured demo walkthrough metadata

Quick smoke test

curl http://localhost:8000/health
curl http://localhost:8000/data/status
curl -X POST http://localhost:8000/forecast \
  -H "Content-Type: application/json" \
  -d '{"horizon": "1d", "model": "gradient_boosting"}'

---

13. How to Run the Frontend

cd frontend

# Development server with hot reload
npm run dev

Open http://localhost:3000 in your browser.

Build for production

npm run build
npm start

Environment variables

Variable	Default	Description
NEXT_PUBLIC_API_URL	http://localhost:8000	Backend base URL
NEXT_PUBLIC_USE_MOCK	true	Use local mock data (no backend needed)

Set NEXT_PUBLIC_USE_MOCK=false to connect to a live backend. The app falls back to mock data gracefully if the backend is unreachable.

Pages

Route	Page	Description
/	Landing	KPI overview, procurement signal, Azure stack banner
/cockpit	Data Cockpit	Source health, ingestion timeline, generation mix
/forecast	Forecast Center	SMP chart, CI bands, horizon selector, model comparison
/drivers	Driver Analysis	SHAP feature importance, RadarChart, key insights
/scenario	Scenario Simulator	Interactive sliders, delta cards, AI explanation
/analyst	AI Market Analyst	GPT-4o natural-language Q&A on market conditions
/storyline	Demo Storyline	Step-by-step demo guide + full briefing view
/architecture	Architecture	Solution architecture with 6 tabbed sections

---

14. Demo Storyline

Audience: Finance Director, CFO, Procurement Manager
Duration: 15 minutes
Mode: Live with backend, or DEMO_MODE=true for offline

Recommended flow

#	Page	Key Point	Duration
1	Data Cockpit	"We ingest 5 real-time Malaysia NEM feeds — SMP, demand, generation mix, fuel prices."	2 min
2	Forecast (1D)	"Here is tomorrow's SMP forecast with 80% confidence interval. Procurement signal: HOLD."	3 min
3	Forecast (30D/90D)	"Extend the horizon — 30-day view shows the seasonal ramp heading into the peak quarter."	1 min
4	Drivers	"Gas price and reserve margin are the top drivers today. Click any feature to see its impact curve."	2 min
5	Scenario Simulator	"What if gas prices rise 15%? Drag the slider — the model re-runs in real time."	3 min
6	AI Market Analyst	"Ask in plain English: 'Should we buy forward contracts this week?' — GPT-4o answers with citations."	2 min
7	Architecture	"Built on Azure in weeks with GitHub Copilot. Zero secrets in code. Full blue/green ML deployment."	2 min

Talking points per persona

For the CFO:

"This gives you a daily forward view of your single largest variable cost. A 3% improvement in procurement timing is worth RM 48,000 per megawatt per year."

For the Procurement Manager:

"The scenario simulator lets you stress-test your contract assumptions before you sign — gas price up 20%, coal supply disruption, what does that do to your hedging position?"

For the Operations Lead:

"The 1-day forecast tells your team exactly which half-hour slots are cheap tomorrow — that's your flexible load shift window."

For the full scripted demo, see docs/demo-storyline.md.

---

15. Known Limitations

Data

Limitation	Detail
Live API availability	The Malaysia NEM public endpoints (singlebuyer.com.my, gso.org.my) are scraped, not formally licensed. They may change structure or become unavailable. The synthetic fallback handles this gracefully.
Fuel price freshness	Gas (JKM) and coal (Newcastle) price APIs have up to 24-hour latency. Intraday SMP spikes driven by sudden fuel cost moves may not be captured.
Fuel price history	The coal/gas price series goes back to 2018. Earlier periods use approximate regional averages.
Holiday calendar	Malaysian public holidays are hardcoded to 2025–2026. Extend ml/feature_engineering.py for future years.

Model

Limitation	Detail
Extreme scarcity events	The model is trained on historical data. It may underestimate SMP during unprecedented outage events (e.g., simultaneous loss of two major plants).
90-day horizon accuracy	Long-horizon forecasts have wider confidence intervals and lower directional accuracy. Use for trend awareness, not precise contract pricing.
No intraday update cycle	The current pipeline runs daily. A production version should re-ingest and re-forecast every 30 minutes as new SMP half-hours are published.
Single-node endpoint	The Azure ML endpoint is deployed as a single instance for demo cost purposes. A production deployment would use autoscaling with a minimum of 2 replicas.

Platform

Limitation	Detail
Authentication	The frontend has no login wall in the current build. Production would add Entra ID authentication via the Static Web App built-in auth.
Multi-tenancy	The platform is designed for a single organisation. Multi-tenant support (separate data, separate models per client) requires additional data isolation work.
Mobile layout	The dashboard is optimised for desktop (1280px+). Some chart interactions do not work well on small screens.

---

16. Production Roadmap

Items are prioritised by business impact and listed in recommended delivery order.

Priority	Item	Effort	Description
P0	30-min ingest schedule	Medium	Schedule ingest_public_data.py as an Azure Functions timer trigger or Fabric notebook run every 30 minutes so the forecast is always fresh
P0	Entra ID authentication	Low	Add Static Web App built-in auth with Entra ID. Map to SMP-Admin / SMP-Analyst / SMP-Viewer roles
P1	Streaming ingest	High	Replace batch ingestion with Event Hubs + Stream Analytics for true real-time SMP updates with < 2-min latency
P1	AutoML production training	Medium	Promote train_automl_azureml.py from script to a scheduled Azure ML pipeline. Auto-register best model after each weekly run
P1	Blue/green promotion gate	Medium	Implement the approval gate: auto-promote new model only if MAPE improves vs current production. Slack/Teams notification for manual override
P2	Contract advisor module	High	Add a procurement module that ingests forward contract prices and recommends buy/hedge/wait based on the forecast and scenario analysis
P2	Email alert digest	Low	Daily 6 AM email digest: yesterday's SMP outcome vs forecast, today's procurement signal, top driver change
P2	Multi-tenant isolation	High	Fabric workspace per client, separate model registry, Row-Level Security on shared analytics layer
P3	Mobile-responsive UI	Medium	Rebuild chart layouts for viewport < 768px. Add PWA manifest for mobile home-screen install
P3	Renewable integration	Medium	Add solar irradiance and wind speed forecasts as exogenous inputs. Malaysia's increasing solar penetration will grow their influence on SMP
P3	Explainable AI dashboard	Medium	Dedicated SHAP waterfall charts per individual forecast period, not just aggregate importance
P3	Backtesting report	Low	Automated walk-forward backtest report comparing model forecast vs actual SMP for the past 90 days, updated weekly

---

Contributing

This project was built as a customer demonstration of Azure AI capabilities. Pull requests are welcome for bug fixes and the roadmap items above.

Before contributing:

1. Ensure npm run build passes in frontend/ with 0 errors
2. Ensure python ml/train_baseline_model.py --dry-run exits with code 0
3. Do not commit infra/.env.azure, ml/artifacts/*.joblib, or data/raw/

---

License

This project is provided as a demonstration and reference implementation. See LICENSE for terms.

---

Built with GitHub Copilot · Powered by Microsoft Azure · Malaysia NEM