SKILL.md
Harvard Artifacts ETL & Analytics App
Skill by ara.so — Data Skills collection.
This project is an end-to-end data engineering application that demonstrates real-world ETL pipelines using the Harvard Art Museums API. It extracts artifact data, transforms it into relational structures, loads it into SQL databases, and provides interactive analytics through a Streamlit dashboard.
What This Project Does
- API Integration: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting
- ETL Pipeline: Transforms nested JSON into normalized relational tables
- SQL Storage: Stores data in MySQL/TiDB with proper schema design
- Analytics: Executes 20+ predefined SQL queries for insights
- Visualization: Creates interactive dashboards with Plotly charts
Installation
Harvard Artifacts ETL & Analytics App
Skill by ara.so — Data Skills collection.
This project is an end-to-end data engineering application that demonstrates real-world ETL pipelines using the Harvard Art Museums API. It extracts artifact data, transforms it into relational structures, loads it into SQL databases, and provides interactive analytics through a Streamlit dashboard.
What This Project Does
- API Integration: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting
- ETL Pipeline: Transforms nested JSON into normalized relational tables
- SQL Storage: Stores data in MySQL/TiDB with proper schema design
- Analytics: Executes 20+ predefined SQL queries for insights
- Visualization: Creates interactive dashboards with Plotly charts