Amazon S3
1. Buckets:
Buckets form the top-level namespace for Amazon S3, and bucket names are global. This means that your bucket names must be unique across all AWS accounts, much like Domain Name System (DNS) domain names, not just within your own account. Bucket names can contain up to 63 lowercase letters, numbers, hyphens, and periods.You can create and use multiple buckets; you can have up to 100 per account by default
-
存儲桶名稱的長度必須為至少 3 個字符，且不能超過 63 個字符。
-
存儲桶名稱必須是一系列的一個或多個標簽。相鄰標簽通過單個句點 (.) 分隔。存儲桶名稱可以包含小寫字母、數字和連字符。每個標簽都必須以小寫字母或數字開頭和結尾。
-
存儲桶名稱不得采用 IP 地址格式 (例如，192.168.5.4)。
-
當通過 SSL 使用虛擬托管式存儲桶時，SSL 通配符證書僅匹配不包含句點的存儲桶。要解決此問題，請使用 HTTP 或編寫自己的證書驗證邏輯。我們建議您不在存儲桶名稱中使用句點 (“.”)。
針對 buckets 的屬性/properties有： Versioning/logging/Static website hosting/Tags/Transfer acceleration/Events/Requester pays
2. Objects
對象的大小可以是0-5TB, Buckets下Objects數量沒有限制。
Metadata：systems metadata and user metadata. user metadata可選，如可以做tag。
針對Object的屬性有：Storage class/Encrytion/Metadata
3. Keys
A key can be up to 1024 bytes of Unicode UTF-8 characters, including embedded slashes（/）, backslashes(\), dots(.), and dashes(~).
key是在bucket範圍內有效， bucket+key是S3 object的唯一標識符。
4. Object URL
樣例 bucket=cloudifierpublic
https://s3.eu-central-1.amazonaws.com/clodifierpublic/dog1.jpg
https://clodifierpublic.s3.eu-central-1.amazonaws.com/dog1.jpg
https://clodifierpublic.s3.amazonaws.com/dog1.jpg 同樣生效，會重定向到https://clodifierpublic.s3.eu-central-1.amazonaws.com/dog1.jpg
5. Durability and availability
Amazon S3 standard storage is designed for 99.999999999% (11個9) durability and 99.99%（4個9） availability
RRS:Reduced Redundancy Storage (RRS) at a lower cost. RRS offers 99.99% durability with a lower cost of storage than traditional Amazon S3 storage.
*RRS 定價已經高於S3 standerd定價，aws是要放棄RRS類型。
6. Data Consistency
S3提供的是最終一致性系統（eventually consistent）
http://www.allthingsdistributed.com/2008/12/eventually_consistent.html http://mark311.github.io/%E5%88%86%E5%B8%83%E5%BC%8F/%E6%9C%80%E7%BB%88%E4%B8%80%E8%87%B4%E6%80%A7/2014/10/18/eventually-consistent.html
7. S3 inventory(清單)
S3存儲清單是S3 提供的一項存儲管理工具，S3存儲清單可以每天或每周輸出指定S3存儲桶或存儲桶中指定前綴的對象及其相關元數據信息的列表，並以CSV文件的格式存儲在指定的S3存儲桶中。存儲清單遵循最終一致性模型，即列表中可能沒有最近添加或刪除的對象信息，如果需要確認某一個對象的狀態，我們可以使用HEAD Object REST API（或命令行，SDK）來獲取該對象的元數據。對於存儲桶中有海量文件的用戶而言，存儲清單可以方便的幫助用戶了解當前存儲桶中的文件列表而不是像過去那樣需要頻繁調用GET Bucket API（每次返回最多1000個對象），從而加速一些業務工作流及大數據作業等等。
8. 資源訪問的權限控制 ACL & Bucket Plicies and user policies
resource-based policies and user policies/基於資源的策略和用戶策略兩類
存儲桶和對象的權限是相互獨立的。對象不繼承其存儲桶的權限。例如，如果您創建了一個存儲桶並授予一個用戶寫入權限，您將無法訪問此用戶的對象，除非此用戶明確地授予您訪問權限。(ACL)。作用在Bucket上的policy對Bucket裏面的object有效。
ACL 可以附加到Object和Bucket上。 Policy只有Bucket Policy。存儲桶策略 – 對於存儲桶，您可以通過添加存儲桶策略向其他 AWS 賬戶或 IAM 用戶授予對相應存儲桶及其中對象的權限。任何對象權限都僅應用於存儲桶擁有者創建的對象。存儲桶策略補充 (在很多情況下取代) 基於 ACL 的訪問策略。
在運行時將所有相關訪問策略 (用戶策略、存儲桶策略、ACL) 轉換為一組策略以進行評估。
何時使用存儲桶 ACL: 存儲桶 ACL 的唯一建議的使用案例是授予 Amazon S3 日誌傳輸組寫入權限，以便將訪問日誌對象寫入您的存儲桶 (請參閱服務器訪問日誌記錄 (p. 553))。如果希望 Amazon S3 將訪問日誌傳輸到您的存儲桶，您需要向日誌傳輸組授予對存儲桶的寫入權限。向日誌傳輸組授予必要權限的唯一方法是通過存儲桶 ACL。
Canonical user ID概念
AWS assigns two unique IDs to each AWS account:
-
An AWS account ID 12-digit number, such as 123456789012
-
A canonical user ID such as
79a59df900b949e55d96a1e698fbacedfd6e09d98eacf8f8d5218e7cd47ef2be.
You can use canonical user IDs in an Amazon S3 bucket policy for cross-account access, which means an AWS account can access resources in another AWS account. For example, to grant another AWS account access to your bucket, you specify the account‘s canonical user ID in the bucket‘s policy
9. 加密
服務器端加密三種加密方式互斥，SSE-S3（Server-Side Encryption）, SSE-KMS, SSE-C。 SSE-KMS使用CMK（customer master keys）加密，
使用客戶端加密保護數據（Client-Side Encryption）：Option 1: Using an AWS KMS–Managed Customer Master Key (CMK)；Option 2: Using a Client-Side Master Key
10. Static Website Hosting
bucket-name.s3-website-region.amazonaws.com或bucket-name.s3-website.region.amazonaws.com 取決於不同region，
http://webcasetest.s3-website.eu-central-1.amazonaws.com/， 法蘭克福節點表達方式
https://docs.aws.amazon.com/zh_cn/general/latest/gr/rande.html#s3_website_region_endpoints
bucket policy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::webcasetest/*"
}
]
}
11. Transfer acceleration:
假定您當前有一個使用 PUT 對象 (該對象在 PUT 請求中使用主機名 mybucket.s3.amazonaws.com) 的 REST API 應用程序。要加速 PUT 操作，您只需將請求中的主機名更改為 mybucket.s3-accelerate.amazonaws.com。要重新使用標準上傳速度，只需將名稱更改回 mybucket.s3.amazonaws.com
12. S3 advanced features
prefixes and delimiters / Object lifecycle/ storage classes( S3 standard / S3 standard - Infrequent Access (Standeard-IA) /S3 RRS /Glacier / Versioning/ MFA delete/ Pre-Signed URLS/ Multipart upload/ Range gets/ Cross-Region Repliaction/ Logging/ Event Notifications(SNS Topic, SQS Queue, Lambda Function)
13. 其他最佳實踐：
Another common pattern is to use Amazon S3 as bulk “blob” ((binary large object, BLOB) storage for data, while keeping an index to that data in another service, such as Amazon DynamoDB or Amazon RDS. This allows quick searches and complex queries on key names without listing keys continually.
Amazon S3 will scale automatically to support very high request rates, automatically re-partitioning your buckets as needed. If you need request rates higher than 100 requests per second, you may want to review the Amazon S3 best practices guidelines in the Developer Guide. To support higher request rates, it is best to ensure some level of random distribution of keys, for example by including a hash as a prefix to key names.
14. 命令行
aws s3api list-buckets
aws s3api list-objects --bucket clodifierpublic
aws s3api get-object --bucket clodifierpublic --key dog1.jpg c:dogshow.jpg
15. S3 工具
https://s3browser.com
http://tntdrive.com
S3 Select 和 Glacier Select – 檢索對象子集 : https://aws.amazon.com/cn/blogs/china/s3-glacier-select/
如何實現 S3 數據跨區域高效可靠傳輸: https://aws.amazon.com/cn/blogs/china/amazon-s3-depth-practice-series-ii-how-to-achieve-efficient-and-reliable-transmission-of-s3-data-across-regions/?nc1=b_rp
參考材料
中文Blog https://aws.amazon.com/cn/blogs/china/tag/amazon-s3/
S3 手冊 https://docs.aws.amazon.com/zh_cn/AmazonS3/latest/dev/Welcome.html
Amazon S3