As a data activation platform, one of the questions we’re often asked by prospective customers is “But what if I want to support real-time?” In some cases, which we’ll get into later, real-time is necessary. But in 80% of the cases in marketing, the answer is that real-time isn’t necessary and microbatching or a frequent schedule can accomplish the task. Read on to learn more about data processing options and when real-time is worth the hype.

Common Data Processing Options

Real-Time

Real-time refers to data transfers that happen in a matter of milliseconds from a data source like Snowflake Data Cloud to a destination like Facebook ads. But, real-time can lead to significantly higher data processing costs and you’re often struggling with data that is inaccurate. 

Microbatching

Microbatching, or micro-batch processing, is a process by which data requests are queued up into a small group and then this micro-batch is moved onto the next stage. As you would expect, microbatching is faster than traditional batch processing, because the batches are smaller. But, because the requests are piled up, microbatching is slower than real-time.

Scheduled Batch

Scheduled batch data processing is just that: you have a set schedule of when data is processed, sometimes that’s every 15 minutes or maybe once an hour. Obviously a schedule is slower than both real-time and microbatching, but can often lead to lower data costs and more accurate data. 

Doesn’t real-time provide a better customer experience?

As businesses continue to focus on the customer experience, it is totally normal for marketers and executives to cling onto the real-time fallacy expecting it to be a crucial part of providing a great customer experience. When, in actuality, some real-time marketing can come across as quite creepy. For example, if my friend refers me to a meal service, I expect to get the welcome email right away, but when I then get a Facebook ad from that same company a minute later, it can be off-putting. 

Real-time: Necessary vs Unnecessary

When do I need real-time? 

Let’s look over some use cases where real-time data sync is needed. 

• Sending customer order confirmation emails from Mailchimp
• Recognizing unusual activity for fraud detection
• Creating an internal alert when your website is down

When is real-time unnecessary?

Now, let’s look at some situations where real-time isn’t necessary and can become quite costly. 

• Sending an email from Marketo when the shipping label is printed
• Adding a new subscriber to a Facebook ad campaign from Snowflake
• Sending an SMS reminder for tomorrow’s meeting

Making the right decision for your business

So, while it may be tempting to just push for all data syncs to happen in real time, it can be costly, inaccurate and unnecessary. Instead, start with looking at different campaigns and marketing activities and decide where real-time is truly the best option. From there, you can separate out the rest of the activities into micro-batching or a scheduled cadence. 

Taking these considerations into account, GrowthLoop supports a litany of sync frequencies from real-time to cadences like hourly or weekly. Interested in learning more about GrowthLoop and how we can help you activate your Data Cloud to 30+ destinations? Contact GrowthLoop today to speak with our experts.