Integrating AWS S3 with MuleSoft

AWS is one of the most popular cloud platforms. Check a detailed post on AWS here:

AWS

AWS S3

Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance.

Benefits

  • Industry-leading performance, scalability, availability, and durability
  • Wide range of cost-effective storage classes
  • Unmatched security, compliance, and audit capabilities
  • Easily manage data and access controls
  • Query-in-place and process on-request

The AWS S3 Connector

Amazon S3 Connector v5.6.xRequires Mule 4.1.1 or higher

Anypoint Connector for Amazon S3 (Amazon S3 Connector) provides connectivity to the Amazon S3 API, enabling you to interface with Amazon S3 to store objects, download and use data with other AWS services, and build applications that require internet storage. The Amazon S3 connector is built using the SDK for Java.


Connector Configurations

  1. First import the AWS S3 connector from Exchange

  2. Create a Connector configuration as below

Now, we will use this configuration to use some of the available operations.


Some available operation in the connector

  • List Buckets – Returns a list of all Amazon S3 buckets that the authenticated sender of the request owns.

    This returns all the buckets in given account. So, we can add some data weave to filter or change the output to be responded back
  • Create Bucket – Creates a new bucket. The connector must not be configured as anonymous for this operation to succeed. Bucket names must be unique across all of Amazon S3, that is, among all Amazon S3 users.
  • Delete Bucket – Deletes the specified bucket. All objects ┬áin the bucket must be deleted before the bucket can be deleted. This can be overridden by setting force = true.
    Below configuration won’t delete unless we delete all contents in it.

    We have an option of force delete where bucket will be deleted even if there are some contents in it. Below configuration will work


    Note – The force option supports dataweave as input so we can handle it dynamically based on certain conditions.
  • List Objects – Lazily lists all objects for a given prefix. Because S3 does not limit the number of objects, this list can retrieve an arbitrary amount of objects

    List objects allows us to add prefix and markers etc. We can add a dataweave code as shown above to selectively display certain content.
  • Create Object – Uploads an object to S3. Supported contents are input streams, strings, byte arrays, and files.

    The Create Object configurations allows a lot of tweaks. We can set some object to be publicly available using Canned ACL options. Here, I am keeping it private
  • Delete Object – Deletes a given object. Only the owner of the bucket containing the version can perform this operation.

Some common use cases

Automate common business operations by integrating Amazon S3 with other business applications such as ERP, analytics systems, and data warehouse systems. Some examples are:

  • Build apps with native cloud-based storage – Connect your mobile apps to scalable Amazon S3 buckets to store files, images, and so on.
  • Backup and archive critical data – Leverage the Amazon S3 connector to seamless integrate with your ERP, CRM, EDI, and fulfillment systems and archive necessary data.
  • Drive business intelligence and optimize operational outcomes – Leverage S3 as a data lake and extract valuable insights using query-in-place, analytics, and machine learning tools.

I will cover more operations in a separate blog! Cheers.

 

14 thoughts on “Integrating AWS S3 with MuleSoft

  1. Fantastic items from you, man. I have have in mind your stuff previous to and you’re simply extremely excellent. I really like what you’ve got right here, certainly like what you are saying and the best way through which you say it. You’re making it entertaining and you still care for to stay it sensible. I can’t wait to read far more from you. That is actually a wonderful site.

  2. Generally I don’t read post on blogs, but I would like to say that this write-up very forced me to try and do so! Your writing style has been surprised me. Thanks, very nice post.

  3. I think this is one of the most vital info for me. And i’m glad reading your article. But wanna remark on few general things, The website style is wonderful, the articles is really great : D. Good job, cheers

  4. Hmm is anyone else having problems with the pictures on this blog loading? I’m trying to find out if its a problem on my end or if it’s the blog. Any responses would be greatly appreciated.

  5. Greetings from Ohio! I’m bored to tears at work so I decided to check out your website on my iphone during lunch break. I enjoy the knowledge you provide here and can’t wait to take a look when I get home. I’m amazed at how fast your blog loaded on my phone .. I’m not even using WIFI, just 3G .. Anyways, amazing site!

Leave a Reply

Your email address will not be published. Required fields are marked *