Microsoft Fabric · OneLake · PySpark · Delta Lake · Power BI · Apache Airflow · dbt · GCP · PostgreSQL · Streamlit · Google Apps Script · Tableau Public

An end-to-end financial-market analytics platform that ingests daily price, company-profile, trading-activity, and news-sentiment data for the three most actively traded stocks. The project transforms raw Alpha Vantage REST API responses into governed analytical datasets and delivers them through Power BI, a live Streamlit application, and Tableau Public.

The platform now has two complete implementations. The original version demonstrates an open-source, containerized data stack on Google Cloud. The newer version re-engineers the same use case in Microsoft Fabric using a OneLake lakehouse, PySpark notebooks, Delta tables, Fabric Data Factory, and Power BI.

Explore the project

• Live Streamlit dashboard: jinghaodata.engineer
• Microsoft Fabric and Power BI repository: github.com/chenjinghao/ms_fabric_project
• Power BI dashboard PDF: View the dashboard
• Power BI demonstration: Watch on YouTube
• Tableau Public dashboard: Open Tableau
• Original GCP/Airflow/dbt backend: View the repository
• Streamlit frontend: View the repository
• Original Streamlit/Tableau demonstration: Watch on YouTube

Project overview

Both implementations process the same core business data:

• Daily price and trading-volume history
• Company profiles and financial metadata
• Analyst ratings
• Relevant market news and bullish/bearish sentiment
• Business-ready measures including 100-day price and volume comparisons

The project emphasizes more than dashboard creation. It demonstrates ingestion, replayable raw storage, schema enforcement, incremental processing, tested transformations, orchestration, monitoring, secure secret management, semantic modeling, and multi-channel analytics delivery.

Architecture overview

The two data-platform implementations operate independently. The Tableau delivery bridge reads the modeled PostgreSQL marts after the scheduled Airflow/dbt processing window.

1. Microsoft Fabric and Power BI implementation

Stack: Microsoft Fabric, OneLake, Fabric Lakehouse, Fabric Data Factory, PySpark, Delta Lake, SQL Analytics Endpoint, Power BI, DAX

Lakehouse and transformations

• Built an automated pipeline that ingests stock prices, company profiles, and news sentiment from REST APIs into a OneLake lakehouse.
• Implemented a Bronze–Silver–Gold medallion architecture across four modular PySpark notebooks.