What Arbor AI Can't Do: Limitations, Hallucinations, and Feedback

Arbor's AI features, like Ask Arbor, have limitations such as no access to certain data (e.g., meal records, payments) and cannot act autonomously, requiring human approval for actions. The AI may produce incorrect answers ("hallucinations"), but safeguards like RAG help improve accuracy. Users can provide feedback directly in Ask Arbor or via the MIS interface to help enhance the system.

Limitations

Arbor's AI features, like Ask Arbor don't have access to all of the data on your MIS. There is data, like meal records or payment information, that Arbor AI does not have the tools to access.

If you Ask Arbor to report on something about Meals, for example, it will let you know that it does not have access to this information, rather than trying to make something up.

Another thing Arbor AI can't do is take action on your behalf without oversight. Arbor AI can never take action autonomously; a human always needs to be in the loop to approve an action. Even Auto Absence only creates pending absence requests that are not finalised untill a human approves them.

In the case of Ask Arbor, which is able to perform certain actions for you too - like send customised comms, create interventions, log absences, or find and book free rooms - you still need to be the one to take the final action, like send the email draft.

These guardrails help prevent Arbor's AI features from "going rogue", but limit what Ask arbor can be used for nonetheless.

Hallucinations

Like all LLMs, Ask Arbor and other AI features can sometimes "hallucinate," producing plausible but incorrect answers.

This occurs because the AI aims to respond even without accurate information.

To minimise this, we use safeguards like RAG (Retrieval Augmented Generation) and an agentic approach, giving the AI access to tools with set parameters, to help ensure our AI features output reliable, substantiated and accurate answers.

Providing Feedback

Your feedback is crucial to us! It really helps us improve.

In Ask Arbor, you can do this without leaving the conversation:

• Use the Nailed it or Missed the mark icons: in the chatbot to rate the quality of responses as good or bad, respectively.
  
  

For feedback on other AI features, or anything else in the MIS, for that matter, you can use the Share product idea button in the left-hand menu of any MIS page.