Configuring SFTP ETL with Make (Integromat)

Make (Formerly known as Integromat) is a no-code automation platform that integrates with most cloud applications globally. Using Couchdrop’s cloud SFTP service, you can create a fully functioning extract, transform, load (ETL) process where you can either process customers' data on upload or send your vendor data based on an action.

Couchdrop – More than a simple SFTP server

Couchdrop’s SFTP offers not only a traditional SFTP server, but also modern features such as webhooks, API, and advanced security for additional users or your platform as a whole. You can restrict users to specific directories and have an array of webhooks and actions based on each user's directory. Couchdrop SFTP also enables you to connect your cloud storage platform as the storage backend, regardless of if this is Azure Blob, SharePoint, S3, Dropbox, or anything in between.

A great example would be a finance company that receives financial information from external clients into their Couchdrop instance via SFTP or inbox. From there a webhook would be sent to an automation system which will then pull down the file that was uploaded and perform ETL operations as required.

Using Couchdrop and Make for ETL

This article will run you through an example of a workflow using Couchdrop’s SFTP and Make. The following steps are required in our scenario: 

• Step 1. Create Webhook connection
• Step 2. Copy Webhook URI to Couchdrop SFTP
• Step 3. Configure JSON Parser module
• Step 4. Configure SFTP module
• Step 5. Conduct operation on file

Make modules required

• Webhook
• SFTP
• JSON Parser

Step 1. Create Webhook connection

Create the webhook connection and copy the address to the clipboard.

Step 2. Copy Webhook URI to Couchdrop

In your Couchdrop portal either locate the directory you wish to receive webhook alerts on and access under its properties or amend at the global account level.

Step 3. Configure JSON Parser module

Add the JSON Parse module and create a data structure. To do this, use the generator and paste the below sample Couchdrop webhook output.

Sample Couchdrop SFTP webhook output:

In Make, Add the object (value) from the webhook as the string.

Step 4. Configure SFTP module

For this scenario, we have selected the Get File operation from the SFTP module. Configure your SFTP connection using yoursubdomain.couchdrop.io as the host and your Couchdrop username and password.

For the File Path select the filename object from the parsed JSON. 

Step 5. Conduct operation on file

For this scenario, we are going to upload the file to a different SFTP location and then delete it from the source. This is where your ETL operation would initiate.

To confirm that the file arrived as expected, you can check Couchdrop's Audit logs in your account.

Simplify SFTP ETL with Couchdrop and Make

While we used a simple Get, Upload, and Delete sequence with Make, there are several other options to make more intricate or complex workflows if you need to. Couchdrop can also easily move a file between your cloud storage, rename the file in the process, and then archive the original. While Couchdrop supports over 30 storage connections, when combined with Make that number can be significantly increased. 

Want to try a workflow like this for yourself? You can try Couchdrop free for 14 days with no credit card required. Simply sign up and get started and see if Couchdrop is right for you. Start your free trial now.