Marketing
How to A/B test WhatsApp messages
Two versions of a reminder, a welcome message or a follow-up, sent to customers picked at random, will tell you more than any opinion in the room. Here is how to plan a WhatsApp A/B test, how many customers it needs, and how to run it in a MonoChat flow.
MonoChat Team Updated: 6 min read
On this page
To A/B test a WhatsApp message, write two versions that differ in one thing, send each to customers picked at random, and compare one result, such as replies, bookings or payments, once each version has reached enough people. Opinions about which wording works are cheap; a test that sends both versions in the same week to the same kind of customer settles it.
This guide covers what to test, how many customers you need, the WhatsApp rules that shape the test, and how to run it in a MonoChat flow.
What is worth testing on WhatsApp
Test the messages you send often, where a small improvement adds up:
| Message | What to change | What to measure |
|---|---|---|
| Appointment reminder | Time of sending, a “reply 1 to confirm” button versus a link | Confirmations, no-shows |
| Abandoned cart or payment reminder | With or without a discount, first message after 1 hour or 24 hours | Payments |
| Welcome message on a new chat | A question first versus a menu first | Replies, handovers to the team |
| Lead follow-up after an ad form | A personal question versus a brochure | Replies, booked calls |
| Review request after a visit | Asking the same day versus two days later | Reviews left |
Change one thing at a time. If version B has a new text, a new button and a new sending time, you will not know which of the three made the difference.
How many customers you need
A test needs enough customers in each group for the difference to be more than luck. The number depends on how often the result happens today and how small a change you want to detect. With a 10% baseline (one customer in ten confirms, pays or replies) and the aim of spotting a rise to about 13%, Evan Miller’s sample size calculator asks for about 2,545 people per version.
Three practical rules follow:
- Small volume, bold changes. A clinic sending 300 reminders a month cannot detect a three-point change, but it can see whether a new reminder halves its no-shows.
- Decide the end before you start. Pick the number of customers per version, or the number of weeks, in advance. Stopping the moment one version looks ahead is the most common way to fool yourself.
- Run both versions at the same time. Sending A in one week and B in the next mixes your message with everything else that changed that week: a holiday, a sale, the weather.
The WhatsApp rules that apply
Each version is its own approved template
Outside the 24-hour customer service window, a business can only start a conversation with a template Meta has approved. An A/B test of a reminder or a campaign therefore needs two templates, each reviewed and approved. Write both so that Meta puts them in the same category: if one version reads as an offer and the other as a plain update, one can come back as marketing and the other as utility, and you are comparing different prices as well as different words.
Every delivered template costs money
Meta charges per delivered template message, by category and by the customer’s country. A test is not extra spend if you run it on messages you would send anyway, so test the reminder you already send rather than adding a new send. The bulk message cost calculator estimates what a send costs.
Marketing needs consent from everyone in both groups
A random split does not change who you may message. Both groups must come from customers who agreed to receive that kind of message, and an opt-out in version B has to be honoured as much as one in version A. Our opt-in guide covers how to collect it.
How to measure the result
Pick one primary result before the test starts, and make sure you can count it per version:
- Replies or button taps: visible in the conversation itself.
- Bookings, payments or orders: counted in your booking tool, store or MonoChat payment links.
- Opt-outs and blocks: check them too. A version that sells more but makes customers block you is not a winner.
The simplest way to count per version is to tag every customer with the version they received. Then you filter customers by tag and count the result in each group. Our guide to customer segmentation with tags shows how tags are used for this.
How to do it in MonoChat
MonoChat is an official Meta Business Partner. Its flows have a Random Split node for A/B tests, and since version 3.8.38 (October 2026) the branch a customer took can be saved and used later in the same flow.
1. Start from the event that already sends the message. For a reminder, that may be a webhook from your booking or store system; for a welcome message, New Message Received; for an ad lead, Lead Received.
2. Add a Random Split node. It starts with two outputs, A and B, weighted 50 each. Each time the flow runs, one output is picked at random with a chance equal to its weight divided by the sum of the weights, so 70 and 30 send about 70% and 30% of the runs down each branch. To try a risky change on a small share first, use 90 and 10. The pick is not remembered: a customer who runs the flow again is drawn again.
3. Save the branch. Turn on Save Flow Variable and enter a Flow Variable Name, for example variant. The name of the output the run took is saved as flow.variant.
4. Send each version. Connect output A to a Send Template Message node with template A, and output B to one with template B. Inside an open customer service window you can use a normal message instead.
5. Tag the customer. Add a Set Customer Tag node after the split with a tag such as reminder-test-{{flow.variant}}; variables are allowed in the tag, so customers end up tagged reminder-test-A or reminder-test-B.
6. Use the branch later if you need to. A Flow Variable Value condition can check flow.variant, so a follow-up after a Wait node can match the version the customer saw.
While the test runs, Activity in the last 7 days on the flow canvas shows how many runs went down each link, which is a quick check that the split is working. When it ends, filter customers by the two tags and compare the result. Keep the winning version, and remove the split or set the losing output’s weight so low it rarely runs.
Replies land in the same MonoChat inbox as the rest of your WhatsApp, Instagram and Messenger chats, where your team or an AI agent answers them. To build flows, see workflow automation; for one-off campaigns to a list, see bulk messaging.
To start, create a free account at app.monochat.ai, connect your WhatsApp number and compare plans on the pricing page.
Sources
Put this into practice with MonoChat