SKILL.md
Harvard Art Museum Data Pipeline
Skill by ara.so — Data Skills collection.
This project provides an end-to-end data engineering solution for collecting, transforming, storing, and analyzing artifact data from the Harvard Art Museums API. It demonstrates production-ready ETL pipelines, SQL analytics, and interactive visualization using Streamlit.
What It Does
The Harvard Art Museum Data Pipeline:
- Extracts artifact data from the Harvard Art Museums API with pagination and rate limiting
- Transforms nested JSON into normalized relational tables (metadata, media, colors)
- Loads data into MySQL/TiDB Cloud with batch inserts for performance
- Analyzes data using predefined SQL queries for business insights
- Visualizes results through interactive Streamlit dashboards with Plotly charts
Architecture
Harvard Art Museum Data Pipeline
Skill by ara.so — Data Skills collection.
This project provides an end-to-end data engineering solution for collecting, transforming, storing, and analyzing artifact data from the Harvard Art Museums API. It demonstrates production-ready ETL pipelines, SQL analytics, and interactive visualization using Streamlit.
What It Does
The Harvard Art Museum Data Pipeline:
- Extracts artifact data from the Harvard Art Museums API with pagination and rate limiting
- Transforms nested JSON into normalized relational tables (metadata, media, colors)
- Loads data into MySQL/TiDB Cloud with batch inserts for performance
- Analyzes data using predefined SQL queries for business insights
- Visualizes results through interactive Streamlit dashboards with Plotly charts