querying-aws-s3

Query AWS S3 System Tables

Overview

Works best with the AWS MCP server for sandboxed execution and audit logging. All commands below use the AWS CLI and work in any environment with configured AWS credentials. Use IAM roles or temporary credentials; avoid long-lived access keys.

Amazon S3 Metadata provides continuously-updated Apache Iceberg tables that capture object-level metadata for general-purpose buckets. S3 Storage Lens exports aggregated storage and activity metrics as Iceberg tables. Both are read-only, stored in the AWS-managed aws-s3 table bucket, and queryable via Amazon Athena.

System tables are preferred over raw S3 APIs (list-objects-v2, head-object) because:

• list-objects-v2 paginates at 1000 objects/page — inefficient for large buckets (millions or billions of objects). The inventory table answers SELECT COUNT(*) in seconds at any scale.
• list-objects-v2 cannot identify who uploaded an object, from which IP, or when something was deleted. Only the journal table has requester, source_ip_address, and delete event tracking.
• Filtering by tag requires get-object-tagging per object. The inventory table has object_tags as a queryable map column.

Decision Tree

Query AWS S3 System Tables

Overview

Works best with the AWS MCP server for sandboxed execution and audit logging. All commands below use the AWS CLI and work in any environment with configured AWS credentials. Use IAM roles or temporary credentials; avoid long-lived access keys.

Amazon S3 Metadata provides continuously-updated Apache Iceberg tables that capture object-level metadata for general-purpose buckets. S3 Storage Lens exports aggregated storage and activity metrics as Iceberg tables. Both are read-only, stored in the AWS-managed aws-s3 table bucket, and queryable via Amazon Athena.

System tables are preferred over raw S3 APIs (list-objects-v2, head-object) because:

• list-objects-v2 paginates at 1000 objects/page — inefficient for large buckets (millions or billions of objects). The inventory table answers SELECT COUNT(*) in seconds at any scale.
• list-objects-v2 cannot identify who uploaded an object, from which IP, or when something was deleted. Only the journal table has requester, source_ip_address, and delete event tracking.
• Filtering by tag requires get-object-tagging per object. The inventory table has object_tags as a queryable map column.

Decision Tree