SKILL.md
Harvard Artifacts Data Engineering Analytics
Skill by ara.so — Data Skills collection
This project provides an end-to-end data engineering and analytics application for the Harvard Art Museums API. It demonstrates real-world ETL pipelines, SQL database design, analytical queries, and interactive visualization using Streamlit.
What This Project Does
The application implements a complete data pipeline:
- Extract: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting
- Transform: Processes nested JSON into relational database tables (metadata, media, colors)
- Load: Batch inserts transformed data into MySQL/TiDB Cloud
- Analyze: Executes 20+ predefined SQL queries for insights
- Visualize: Renders interactive dashboards with Plotly charts in Streamlit
Installation
Harvard Artifacts Data Engineering Analytics
Skill by ara.so — Data Skills collection
This project provides an end-to-end data engineering and analytics application for the Harvard Art Museums API. It demonstrates real-world ETL pipelines, SQL database design, analytical queries, and interactive visualization using Streamlit.
What This Project Does
The application implements a complete data pipeline:
- Extract: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting
- Transform: Processes nested JSON into relational database tables (metadata, media, colors)
- Load: Batch inserts transformed data into MySQL/TiDB Cloud
- Analyze: Executes 20+ predefined SQL queries for insights
- Visualize: Renders interactive dashboards with Plotly charts in Streamlit