AI导哈

数据与研究

harvard-artifacts-data-engineering-app

reason-machines/data-skills

Build ETL pipelines and analytics dashboards using the Harvard Art Museums API with SQL storage and Streamlit visualization

  • 1975 次安装
  • 5 星标
  • 收录于

安装命令

npx skills add https://github.com/reason-machines/data-skills --skill harvard-artifacts-data-engineering-app

SKILL.md

Harvard Artifacts Data Engineering App

Skill by ara.so — Data Skills collection.

This project demonstrates end-to-end data engineering using the Harvard Art Museums API. It extracts artifact data, transforms it into relational tables, loads it into SQL databases (MySQL/TiDB), and provides interactive analytics through Streamlit dashboards with Plotly visualizations.

What It Does

  • API Integration: Fetches paginated artifact data from Harvard Art Museums API
  • ETL Pipeline: Transforms nested JSON into normalized relational tables
  • SQL Storage: Creates and populates artifactmetadata, artifactmedia, and artifactcolors tables
  • Analytics Queries: 20+ predefined SQL queries for artifact insights
  • Interactive Visualization: Streamlit dashboard with Plotly charts

Installation

Harvard Artifacts Data Engineering App

Skill by ara.so — Data Skills collection.

This project demonstrates end-to-end data engineering using the Harvard Art Museums API. It extracts artifact data, transforms it into relational tables, loads it into SQL databases (MySQL/TiDB), and provides interactive analytics through Streamlit dashboards with Plotly visualizations.

What It Does

  • API Integration: Fetches paginated artifact data from Harvard Art Museums API
  • ETL Pipeline: Transforms nested JSON into normalized relational tables
  • SQL Storage: Creates and populates artifactmetadata, artifactmedia, and artifactcolors tables
  • Analytics Queries: 20+ predefined SQL queries for artifact insights
  • Interactive Visualization: Streamlit dashboard with Plotly charts

Installation

相关 Skills

apache-airflow-orchestrationreason-machines/data-skillsExpert knowledge of Apache Airflow for building, scheduling, and monitoring data pipelines and workflows智能体工作流2395datatalks-data-engineering-zoomcampreason-machines/data-skillsFree 9-week data engineering course covering Docker, Terraform, Kestra, BigQuery, dbt, Spark, and Kafka with hands-on projects数据与研究2392llm-public-opinion-analytics-assistantreason-machines/data-skillsMulti-platform hot search crawler and LLM-powered public opinion analysis system with clustering, sentiment analysis, and multi-channel push notifications数据与研究2384roblox-mm2-analytics-toolkitreason-machines/data-skillsAnalytics and inventory management toolkit for Roblox Murder Mystery 2 gameplay optimization数据与研究2380mm2-analytics-roblox-trackerreason-machines/data-skillsAnalyze Murder Mystery 2 gameplay data, track inventory, and optimize strategy using this Roblox analytics toolkit数据与研究2359mm2-roblox-analytics-toolkitreason-machines/data-skillsMurder Mystery 2 gameplay analytics, inventory tracking, and strategy optimization toolkit for Roblox数据与研究2358