How to Split Test Restaurant Loyalty Messages Without Double Counting Guests

To compare two loyalty messages without counting overlapping guests twice, give each eligible guest one stable identifier, assign that identifier to exactly one group before sending, and measure both groups over the same period. Count the guest once in the assigned group even if they have multiple contact details or place several orders.

Keep the reward, audience rules, channel and timing the same. Change the message you want to evaluate. Then compare delivery, redemption and a clearly defined contribution measure against the number of guests originally assigned to each group.

Start with one guest, not one contact record

A restaurant may have an email address from an online order, a phone number from a reservation and a loyalty account for the same person. Treating each record as a separate guest can put that person in both groups. It can also make one redemption appear to support both messages.

Choose an identifier that stays attached to the guest throughout the test. A loyalty account ID may work if your records reliably connect that account to the contact details and orders you want to measure. An email address alone may be less dependable if guests use more than one address or share an inbox.

Before assignment, review records that appear to belong to the same person. Merge them only when your normal identification process supports the match; a shared phone number or surname is not proof by itself. If you cannot resolve a likely duplicate, leave it out of the test rather than guess.

Access to contact information is not the same as permission to market. Build the eligible audience using the permissions and preferences relevant to the channel. Also exclude guests who cannot use the offer, employees or test accounts, and anyone already scheduled to receive a conflicting promotion during the measurement window.

Ask a vendor how its marketing software identifies a guest across contact records, suppresses duplicates and exports the assigned audience. Check whether you can retain the group assignment alongside delivery and order records. A count on a campaign dashboard is useful, but you also need to understand what that count represents.

Assign the cohort before either message goes out

Freeze an eligible guest list at a stated cutoff. Randomly assign each unique guest ID to group A or group B, save the assignment, and do not move guests between groups after seeing delivery or sales results. Record the assignment method and the sizes of both groups.

For a fair message comparison, use the same channel, offer value, redemption rules and send schedule. For example, one email subject line might emphasize the reward while the other emphasizes the occasion. Both emails should lead to the same offer. If one message also has a larger discount, you cannot tell whether the wording or the discount drove a difference.

Set the observation window in advance. Both groups should have the same opportunity to receive and use the offer, including the same start and end times. Plan how you will handle late orders, cancellations, refunds and guests who opt out after assignment.

Use the originally assigned group as the main denominator. A guest whose email bounces stays in that group for the main comparison, even though they did not receive the message. Otherwise, a group with worse delivery can look artificially strong because its unreachable guests disappear from the calculation. Delivery and redemption among delivered messages are useful secondary measures.

Check the workflow before a live send

Run a simulation with dummy guest records. Include one person represented by two contact records, one guest who opts out and one order with a refund. Confirm that each eligible test identity appears in no more than one group, the opt-out is suppressed, and the refunded order follows your planned reporting rule.

The simulation should not trigger a real payment, guest message or courier dispatch. Ask how to disable those actions before trying it. A separate live service pilot, if you choose one, uses actual eligible guests and normal operating controls; do not describe its sends or orders as simulated.

Use one worksheet for delivery, redemption and contribution

The following figures are hypothetical, not customer results. Assume 200 unique, eligible guests are randomly split into two groups of 100. Each group receives a different email message for the same redeemable reward, with the same send and measurement windows. A redemption means a guest actually used the reward, not merely that their points or spending were equivalent to its value.

The contribution calculation includes all measured orders linked to assigned guests during the window, whether or not the reward was redeemed. Sales are net of recorded refunds. Costs include variable food and packaging, redeemed rewards, message charges, an allocated fixed campaign cost and recorded exception costs. Other operating costs are excluded. These amounts are assumptions for comparing this workflow, not Fleksa pricing.

Message A delivered to 90% of its assigned guests and B to 88%. Redemption was 18% and 16% of assigned guests, respectively. A guest who redeemed twice still counts as one redeemer; both qualifying orders and both reward costs belong in the financial totals.

For A, subtract $2,160, $36, $10, $20 and $4 from $3,600 to get $1,370. The same calculation gives B $1,345. Contribution is a partial operating measure here, not profit: it excludes costs such as labor and rent. The measured sales also include purchases guests might have made without either message, so these totals do not establish campaign incrementality.

Before using your own numbers, decide which costs are available and whether they are recorded consistently for both groups. Include actual reward costs rather than the face value automatically, and avoid subtracting a discount twice if sales are already recorded after that discount. Payment processing fees, if included, are an expense; they are not sales or deposits. For a closer look at reward economics, use the restaurant loyalty reward cost guide.

Set a decision boundary before reading the results

In the hypothetical worksheet, suppose the owner decided in advance to change the standard message only if one group produced at least $40 more measured contribution than the other across equally sized groups. A leads B by $25, so the practical decision is to keep the current message and collect more evidence rather than switch based on this result.

The $40 boundary is a business rule, not a statistical test. Choose your own threshold based on the work and risk of changing a campaign. If group sizes differ, compare contribution per assigned guest and translate your threshold to that same basis. Keep the cost definitions and observation periods consistent.

A small sample can produce a difference by chance. One large catering order, a refund or a few delivery failures can move the result. Look at the underlying order distribution and note unusual events before drawing a conclusion. Do not remove an inconvenient order after the fact unless it meets a rule established before the send.

If the result is inconclusive, plan another comparison with a larger eligible audience or a longer, equally applied observation period. Avoid repeatedly checking results and stopping as soon as a preferred message pulls ahead. If you need a formal confidence estimate or a reliable sample-size plan, ask someone qualified in experiment design to help before the next send.

Make the reporting requirements part of the buying decision

Before committing to a tool, ask the vendor to walk through a duplicate guest record from audience creation to reporting. Can you assign one guest to one group, freeze or export that assignment, and keep it separate from delivery status? Can you identify unique redeemers and connect orders, refunds and reward costs to the right group without counting an order twice?

Also ask what happens when a guest changes an email address, opts out or redeems through a different ordering channel. Find out which records can be exported and which calculations you must complete elsewhere. The answers matter more than a dashboard label if you need to explain your decision later.

Consider the staff work as well as software charges: preparing the audience, resolving duplicates, checking permissions and reconciling orders all take time. Fleksa’s plans page lists current pricing; check the details for your setup. Pricing reference checked November 2026. Ask for a written quote that separates platform charges, messaging charges and payment processing from other operating costs.

Bring your intended audience rules, sample worksheet and reporting questions to a conversation about product fit. Discuss your requirements with Fleksa and ask to see how the proposed workflow would handle assignment, suppression and export before you schedule a live campaign.

Frequently asked questions

Should a guest who receives no message stay in the comparison?

Yes, keep that guest in their originally assigned group for the main per-assigned-guest comparison. Report delivery separately so you can see whether reach affected the result.

Can I compare two different rewards instead of two messages?

You can compare the complete offers, but the result will not isolate the effect of wording. Record the different reward costs and redemption rules before interpreting the outcome.

Does the higher-contribution group prove the campaign caused more sales?

No. Comparing A with B helps assess the difference between those messages. Estimating sales caused by messaging itself requires a suitable no-message comparison and a separate measurement plan.