Reports
Reports generated by the system are stored in the reports/ folder inside your duracloud-$ID-managed bucket. For a full description of all folders in the managed bucket, see Managed Resources.
NOTE: AWS runs inventory once daily, your hosting provider may have no control over that timing, so your reports might not reflect exactly what you were expecting, or you may receive errors. These errors may be because of moving content around in your account and the inventory process has not caught up with the changes you made.
Report types
Checksum verification
Automatically generated by the system. Compares checksums between your active bucket and its replication bucket to detect data integrity issues. This report runs twice a year on a schedule.
reports/0000-00-00-LATEST/checksums/<bucket>_checksum-report.csv— most recent reportreports/YYYY-MM-DD/checksums/<bucket>_checksum-report.csv— date-stamped archive
Each report includes rows for each object within a bucket, including folder structures, with the following fields:
| Field | Description |
|---|---|
| bucket | Bucket name |
| key | Filenamke |
| version_id | AWS-generated identifier |
| status | Typically will state “ok”; may state “error” |
| checksum_algorithm | CRC64NVME |
| checksum_source | Checksum for item created or uploaded by user |
| checksum_replication | Checksum for same item in -repl bucket; the values in these two columns should match |
If there are any errors in the “status” column, contact your hosting provider for more information.
A note about the version_id column: When you overwrite files, both versions are available for a set period, and the old version is restorable in case you accidentally overwrote a file.
Checksum inventory
A per-object listing of checksum metadata built from S3 inventory data. Unlike the verification report, this records the checksum value for each object rather than comparing source and replica. Folder structures are not included as separate rows on this report.
reports/0000-00-00-LATEST/checksums/<bucket>_checksum-inventory.csv— most recent reportreports/YYYY-MM-DD/checksums/<bucket>_checksum-inventory.csv— date-stamped archive
Each row includes the object key, its CRC64NVMe or SHA256 checksum (when present), and a status:
ok— checksum metadata retrieved successfully (note: this is not checksum verification)not_found— object was not foundmissing_checksum— object exists but has no checksum recordederror— other failure
Requesting a checksum inventory
To trigger a checksum inventory for a bucket, upload a .txt file named after the bucket to the checksums/ folder in your request bucket. The file’s contents do not matter — only the filename is used.
aws s3 cp duracloud-$ID-mybucket.txt s3://duracloud-$ID-request/checksums/dcp-$ID-mybucket.txt
The report will appear in the managed bucket once processing is complete (typically within a few minutes). If you mistyped the bucket name, no report will appear, and you should double-check you spelled everything correctly.
Inventory manifest
A full listing of all objects in a bucket, generated daily by AWS.
reports/0000-00-00-LATEST/manifests/<bucket>.csv— most recent reportreports/YYYY-MM-DD/manifests/<bucket>.csv— date-stamped archive
The fields included in this report:
| Field | Description |
|---|---|
| bucket | Bucket name |
| key | Filename |
| size | Object size in bytes |
| last_modified_date | UTC timestamp |
| storage_class | Options include Standard, Intelligent Tiering, depending on if the object has recently been accessed/downloaded |
| replication_status | Should state “completed” to indicate object has been added to the -repl bucket |
This report does not include rows for folder structures.
Storage
An interactive HTML report generated weekly showing storage usage across all buckets in your stack.
reports/0000-00-00-LATEST/storage/<stack>.html— most recent reportreports/YYYY-MM-DD/storage/<stack>.html— date-stamped archive
Open the .html file in your browser to view:
The top row of the report includes
- Total Buckets - how many buckets are associated with your AWS account
- Total Files - across all buckets
- Total Size - how much space all files are taking up
- Total Prefixes - how many folders you’ve created within buckets
- Capacity Used - how much of your subscription you have used (will show a % of however many TBs)
- Data Transfer Out - how much data has been transferred out (or downloaded) from your account
Additional information includes
- Big Picture (Aggregated totals) — storage usage across all buckets, graphs for bucket size and file distribution
- Per bucket totals — usage broken down by individual bucket
- Per bucket / per prefix totals — usage by folder (prefix) within each bucket
Accessing reports
These reports are available no matter how you access your AWS account. The HTML storage reports will open in your default browser after you download.
Cyberduck
- Connect to S3 (see Connecting to S3).
- Navigate to
duracloud-$ID-managed→reports→0000-00-00-LATEST. - Open the relevant subfolder (
checksums,manifests, orstorage). - Right-click the file and select Download, Download As, or Download To.
SFTPGo
- Log in to the web interface (see Connecting to S3).
- Navigate to
managed→reports→0000-00-00-LATEST. - Open the relevant subfolder (
checksums,manifests, orstorage). - To download a single file, click directly on its filename, or check the box next to it and use Actions → Download.
- To download multiple files, check the boxes next to them and use the Actions menu → Download. Selected items will be zipped automatically.
AWS CLI
Download the latest inventory manifest for a bucket:
aws s3 cp s3://duracloud-$ID-managed/reports/0000-00-00-LATEST/manifests/duracloud-$ID-mybucket.csv .
Download the latest checksum inventory for a bucket:
aws s3 cp s3://duracloud-$ID-managed/reports/0000-00-00-LATEST/checksums/duracloud-$ID-mybucket_checksum-inventory.csv .
Download the latest storage report:
aws s3 cp s3://duracloud-$ID-managed/reports/0000-00-00-LATEST/storage/duracloud-$ID.html .
List all available reports in a folder:
aws s3 ls s3://duracloud-$ID-managed/reports/0000-00-00-LATEST/checksums/
Sync an entire dated archive locally:
aws s3 sync s3://duracloud-$ID-managed/reports/ ./reports/
- Right-click to rename files when downloading to avoid overwriting reports from previous dates.