In a previous job, we built our AI assistant so that it could operate our UI in the front-end and it was very powerful.
An example: - user intent is to update an attribute for a component part number A21445
- user can click a chat bubble icon in lower right and chat to the assistant
- user describes their intent - "help me update the description for part number A21445
- system replies by informing the user it will open the right screen, opening a part/component editing UI, with the right part loaded, with the cursor positioned in the description field, and the assistant stays open for further assistance; or;
- system replies that it found the part and can update the description, shows the current description for the user, asks "what description do you want?"
- user enters updated description
- system confirms the change is correct
- user confirms the change is correct
- part/component description is updated without even opening the UI
FWIW, it's great that you have cmd-K and also I've seen those kind of search boxes get more smarts like being able to type "part:A21445" to go directly to a specific UI.
I just suggested the above as we learned some interesting user experiences became possible when our AI assistant had the ability to control our UI directly on behalf of a specific user.
An example in the app I worked in (a web based data pipeline tool):
- "Hey assistant, can you help me add some SQL transformation logic to dataflow ABC, to process the customer data?"
- system uses metadata and knowledge of the UI to open the right dataflow, select the right type of UI to open to enable the user to add a SQL query in the right place, maybe even autogenerate the initial SQL query - this all from the main home page of the app, from a side panel chat assistant.
- net result feels like talking to the assistant to operate the app, almost no clicks required.
I hope that makes sense.
Instead of arranging things in a logical hierarchy, and enabling quick navigation thru keystrokes - just toss out making the ui make sense because the chatbot can solve all.
Your job is to make a good product. AI-as-interface is just slapping a layer of randomized language parsing in between the user and bad ui.
I’m not actually saying no chatbots. What I am saying is using a chatbot to solve a usability problem is a massive middle finger to anyone who might have to use your product for 8 hours a day. Make it good, then make it idiot-proof. Don’t make it for idiots only.
we've started trying to work through adding agents like this: https://x.com/barbinbrad/status/1903047303180464586
the trouble is that there are 1000s of possible mutations -- and the quality of an agent tends to diminsh with the amount of "tools" you give it. i need to figure out the right abstraction for this.