In one of our previous articles we described the quickest way to save on Amazon S3. Let’s discover how to reduce costs on Alibaba Cloud OSS by avoiding incomplete multipart uploads and non-actual artifacts in the bucket.
Problem description: Incomplete multipart uploads in Alibaba Cloud
Using Alibaba Cloud Object Storage Service (OSS) you can store and protect any amount of data. Millions of companies derive benefit from cost-efficiency, scalability, reliability and performance of this cloud storage.
Alibaba Cloud provides customers with several storage options for various use cases, like Object Storage Service (OSS), Apsara File Storage NAS (NAS), Elastic Block Storage (EBS) and Storage Capacity Unit (SCU). OSS got widespread adoption, so it could be assumed, if your company uses Alibaba Cloud, most likely, it uses OSS.
The artifacts are usually uploaded to OSS, during a CI/CD workflow. A proper artifact retention policy is aimed to avoid storage of non-actual artifacts. But let’s talk about another simple and extremely efficient way of resource optimization, which is applied to non-actual artifacts in your Alicloud OSS buckets. Retention for incomplete multipart upload (MPU) objects helps get rid of non-actual artifacts and prevent paying for idle cloud resources.
The part of the data which has already been uploaded to OSS will still be stored there, even if that specific upload session was not completed in a right way. Some may argue that upload failures are not a typical case during an R&D process to take into consideration. But the analysis and expertise shows that a bucket used for storing artifacts, which was created over one year, usually contains up to 20 percent of incomplete MPU objects of total storage.
How to check incomplete MPU in Alibaba Cloud OSS?
The Alicloud user interface shows incomplete MPU objects right in the bucket interface.
Navigate to bucket page – https://oss.console.aliyun.com/bucket/<region>/<bucket_name>/object
You will see files which are stored in the current bucket. On this page you will see how many incomplete MPU you currently have right now. Button called “Parts”.
For checking how much storage you spend on an incomplete MPU you can open “parts” and click to “Start Statistics“. It will show you the total size for every incomplete MPU.
If you see that incomplete multipart uploads bytes are a significant part of the total storage, you should go and optimize it.
Free cloud cost optimization. Lifetime
How to optimize incomplete MPU storage in Alibaba Cloud OSS?
The best way to ensure that a specific bucket won’t have incomplete MPU storage issue is to set Lifecycle Policy for it – https://www.alibabacloud.com/help/en/doc-detail/31904.htm
OSS Lifecycle management is a very powerful though complicated process, but creation of MPU policy is quite short and simple.
You can use cloud shell console to do that (check the Cloud Shell icon in the top menu of the Alibaba Cloud console), as well as using alicloud Command Line Interface (aliyun).
To set a MPU retentions, do the following:
- Ensure that you have aliyun installed and configured.
- Prepare the following mpu-retention.xml file:
<?xml version=”1.0″ encoding=”UTF-8″?>
<LifecycleConfiguration>
<Rule>
<ID>MPU Retention</ID>
<Prefix></Prefix>
<Status>Enabled</Status>
<AbortMultipartUpload>
<Days>7</Days>
</AbortMultipartUpload>
</Rule>
</LifecycleConfiguration>
It describes the policy to abort all incomplete upload parts which were not active within the last 7 days.
- Run the following command to apply the MPU retention policy to your-bucket:
aliyun oss lifecycle -e <<bucket-endpoint>> –method put oss://<<your-bucket>> mpu-retention.xml
Now you have set the MPU retention rule for your bucket and may be sure that you won’t see the issue again. Please note that the lifecycle rule takes effect 48 hours after it is configured or modified.
Overlooked resources are contributed to a company cloud bill, and users don’t even expect that they’re paying for them.
💡 Find the ways of identifying and cleaning up orphaned snapshots to keep MS Azure and Alibaba Cloud costs under control → https://hystax.com/finops-best-practices-how-to-find-and-cleanup-orphaned-and-unused-snapshots-in-microsoft-azure-and-alibaba-cloud
