SKILL.md
Harvard Artifacts Collection Analytics ETL
Skill by ara.so — Data Skills collection.
Overview
This project provides an end-to-end data engineering solution for the Harvard Art Museums API. It demonstrates a production-ready ETL pipeline that extracts artifact data, transforms nested JSON structures into relational tables, loads data into SQL databases (MySQL/TiDB), and provides interactive analytics through a Streamlit dashboard with Plotly visualizations.
Architecture Flow: API → ETL → SQL → Analytics → Visualization
Installation
# Clone the repository
git clone https://github.com/Manali0711/Harvard-Artifacts-Collection-Data-Engineering-Analytics-App.git
cd Harvard-Artifacts-Collection-Data-Engineering-Analytics-App
# Install dependencies
pip install -r requirements.txt
Harvard Artifacts Collection Analytics ETL
Skill by ara.so — Data Skills collection.
Overview
This project provides an end-to-end data engineering solution for the Harvard Art Museums API. It demonstrates a production-ready ETL pipeline that extracts artifact data, transforms nested JSON structures into relational tables, loads data into SQL databases (MySQL/TiDB), and provides interactive analytics through a Streamlit dashboard with Plotly visualizations.
Architecture Flow: API → ETL → SQL → Analytics → Visualization
Installation
# Clone the repository
git clone https://github.com/Manali0711/Harvard-Artifacts-Collection-Data-Engineering-Analytics-App.git
cd Harvard-Artifacts-Collection-Data-Engineering-Analytics-App
# Install dependencies
pip install -r requirements.txt