The safest manager questions for an AI pilot are usually narrow lookups and simple calculations with a known source, a clear answer and a manager who can check it. Questions that ask what the restaurant should do deserve a higher bar: they depend on context, trade-offs and consequences that may not be in the data.
Start by testing question types separately. A tool that correctly finds an order count may still invent missing labor costs or offer a confident staffing recommendation from incomplete sales data. The buying decision is not whether AI is generally useful. It is which questions your team can safely put in front of it, under what limits.
Sort manager questions by the work involved
Factual lookup asks the tool to find and report a value already recorded in an agreed source. For example: “How many orders were recorded for Tuesday dinner?” The answer should identify the period and source, not just give a number. Before testing, define whether canceled orders count and what hours make up dinner.
A lookup can still go wrong. The assistant might use the wrong date, confuse order count with guest count, or answer from an old report. If the manager cannot find the same number in the underlying record, the answer is hard to trust.
Calculation combines known values using an agreed formula. A manager might ask, “What were average recorded sales per order on Tuesday dinner?” The expected answer depends on the sales measure and order count chosen. The assistant should show enough arithmetic for the manager to reproduce it and flag a missing input rather than guess.
For example, if a hypothetical report shows $840 in recorded sales across 28 orders, the average is $30 per order. That is not profit, average spend per guest or a measure of cash received after fees. The figure includes only the sales and orders in the report; it does not subtract fixed costs, variable costs or exception costs such as refunds and chargebacks. Whether any particular adjustment is already reflected depends on the report definition.
Recommendation asks the tool to interpret information and suggest an action: “Should I schedule another server next Tuesday?” Sales alone cannot settle that question. Reservations, expected party sizes, staff availability, service standards and the manager's judgment may all matter. An answer can sound reasonable while overlooking the one constraint that changes the decision.
Some questions cross categories. “What were sales last Tuesday, and should I cut a shift?” contains a lookup and a recommendation. Test and score both parts. Do not let a correct sales number make an unsupported staffing suggestion look safe.
Build a small set with known answers
Choose questions from actual management routines, then prepare a controlled evaluation set before asking vendors to demonstrate their tools. Agree on the source, date range, definitions and expected answer for each case. A manager should be able to check the answer without relying on the AI's explanation alone.
For sales and order lookups, start with the reports your team already uses. Fleksa's partner portal provides dashboard, order, menu and reservation views, including sales and order summaries for a selected period; available views and configuration vary. Ask which fields the tool can actually access. A separate payroll, inventory or accounting tool may still be needed for questions involving those records. The restaurant POS and other source systems should be identified by name in your test worksheet.
Include ordinary cases, ambiguous wording and deliberately missing inputs. For a calculation, write down the formula and expected unit. For a recommendation, record the context a manager would need and the situations in which the assistant must ask for more information or refer the decision to a person.
Hypothetical evaluation: An owner prepares 12 questions using a controlled report and written operating assumptions. Four are lookups, four are calculations and four request recommendations. In each group, three have enough information for the task and one deliberately omits a necessary input. The owner defines a supported answer for each complete case and an appropriate missing-data response for each incomplete case.
| Question type | Example and expected behavior | Hypothetical results | Pilot decision |
|---|---|---|---|
| Lookup | Find Tuesday dinner order count; identify the period and source. If the date is missing, ask for it. | 3 of 3 complete answers accurate; missing-date case handled correctly. | Include within the defined report and period rules. |
| Calculation | Calculate sales per order using the agreed fields; request an absent order count. | 2 of 3 complete answers accurate; missing-count case handled correctly. | Hold out until corrected and retested. |
| Recommendation | Suggest whether to add a server; identify relevant constraints and refer the decision to a manager. | 2 of 3 complete cases met the review rule; the incomplete case received a recommendation without a needed input. | Exclude recommendations from the pilot. |
The table is a decision worksheet, not a claim about a product. Here, “accurate” means the answer matches the prepared source or formula and uses the correct units. For recommendation cases, the reviewer checks the documented constraints and escalation behavior rather than treating one staffing choice as the only correct answer.
Before running the set, agree on the acceptance boundary. In this example, a question type needs all three complete cases to pass and its missing-data case to be handled correctly. Recommendation cases also require the assistant to leave the decision with a manager. The set is small, so passing it supports only a narrow pilot, not a conclusion that the tool is reliable for every variation.
Score accuracy, missing data and escalation separately
Keep a row for each prompt, the source answer, the assistant's answer and the reviewer's decision. Save the wording used: changing “orders placed” to “orders completed” can change the answer. If a response contains several claims, check each one. A correct total alongside an invented source is not a clean pass.
Accuracy
For lookups, compare the value, date range, status and source. For calculations, check the inputs, formula, arithmetic and units. Record the specific failure rather than simply marking the response wrong. A wrong date filter calls for a different fix from a wrong division.
In the hypothetical worksheet, calculations fail the stated boundary because one of the three complete cases is wrong. The fact that the assistant correctly asked for the missing count does not repair that error. Ask the vendor how the failure would be addressed, then use fresh cases as well as the original ones in a retest.
Missing-data handling
Remove a necessary input on purpose. Does the assistant ask a useful follow-up, clearly say it cannot calculate, or silently fill the gap? A reasonable response to an absent order count is to request the count or explain which report is needed. It should not manufacture a number to finish the calculation.
Also test an ambiguous term. If “sales” could mean a gross figure or a figure after adjustments, the assistant should ask which definition applies or state a supported definition that the manager can verify. Deposits, sales and payment fees are different measures; do not accept an answer that treats them as interchangeable.
Escalation
Decide in advance when the assistant should stop and involve a manager. For a staffing recommendation, that might be when schedule constraints or expected demand are unavailable. Review whether the response actually names the gap and leaves the decision to the manager, rather than presenting a guess as a plan.
Escalation can be correct even when the assistant gives no numerical answer. Conversely, a warning at the end does not make an unsupported recommendation a pass. In the hypothetical set, the incomplete recommendation case fails because the assistant proceeds despite a necessary missing input. That failure supports excluding the category from this pilot.
Set boundaries, then run a supervised pilot
Turn the worksheet into a short operating rule. In the example, managers may ask for the tested order lookup using the agreed report, date and status definitions. Calculations and recommendations remain outside scope until they meet a revised test. A manager checks the source before using an answer in a report or decision.
Make the boundary visible to everyone involved. List allowed questions, source systems, who reviews answers and where staff should report an error. Also specify what the assistant must not do: for this pilot, it provides information only. It does not change schedules, send guest messages, place orders or take payments.
First run simulated checks in a controlled environment with sample records or otherwise isolated data. Confirm that the exercise cannot trigger a real card payment, guest message or courier dispatch. A live service pilot is different: it uses real operating data during service and needs an assigned reviewer, a stop rule and an ordinary fallback process if the tool is wrong or unavailable.
Review failures by type. If the tool repeatedly chooses the wrong date, narrow the accepted date format or hold the lookup out. If it cannot show the calculation inputs, keep that calculation with a person. If it gives advice when information is missing, exclude that recommendation type rather than relying on staff to catch every instance.
Revisit scope when reports, definitions or access change. A pass on Tuesday order counts does not automatically cover refunds, reservations or a new location. For related questions about matching sales and labor periods, see the guide to comparing labor hours with sales by shift.
Next, select a handful of manager questions, write their source answers and review rules, and compare the workflow with your current management software. Then discuss your requirements with Fleksa, including the views available to your team and the current product fit for your pilot.
Frequently asked questions
How many questions should an AI pilot test?
Use enough cases to cover each intended question type, including normal, ambiguous and missing-data cases. A small set can screen for obvious failures, but it cannot establish reliability across every shift and report.
Can AI recommend staffing levels during the pilot?
Only include that question type if it meets your documented review and escalation rules. A manager should retain the decision, and the assistant should not recommend a plan when necessary context is missing.
What should happen when the assistant gives a wrong answer?
Record the prompt, source, response and failure. Pause the affected question type if it crosses your stop rule, use the normal manual process and retest after the issue is addressed.
