Living Documents: Designing the Next Evolution of AI User Experience
The rise of Large Language Models (LLMs) has led to a surge in AI-powered features, particularly embedded chat boxes, across various products. However, the true potential lies in integrating AI capabilities into more sophisticated User Interfaces (UIs). At Elicit, we're pioneering this approach by developing a tool that leverages LLMs to create dynamic, editable documents that encapsulate the vast expanse of scientific knowledge on any given topic.
This article delves into the concept of "Living Documents" as an AI UX pattern, exploring the use case motivation and the intricate product design challenges involved.
What are Living Documents?
Living Documents are complex, editable documents powered by AI, designed to provide a comprehensive understanding of a specific topic by synthesizing information from various sources, such as scientific literature. Unlike static documents, they evolve and adapt as new information becomes available, offering a continuously updated view of the subject matter.
The Challenges of Designing Living Documents
Creating Living Documents using LLMs presents a unique set of challenges that go beyond traditional software development. These challenges can be broadly categorized into:
- Decomposing AI Interactions: Unlike linear chat interactions or single request-response cycles, Living Documents involve thousands of individual language model actions assembled into an editable table. This requires careful consideration of how to handle varying response times and potential failures of the language model backends.
- Managing Heavy LLM Workloads: The interface must remain fast and responsive, even with the massive parallelized set of compute operations running in the background.
- Visualizing Cost and Avoiding Surprise: LLM call costs can be unpredictable. It's crucial to provide users with clear indications of operation costs, avoiding unexpected billing surprises.
Use Case: Systematic Review
To illustrate the value of Living Documents, consider the use case of systematic review. In industries like medical device development, a systematic review involves surveying all existing literature on a disease and its treatments. This process traditionally requires a team working for months, manually collecting papers, reading them, and extracting data into a massive spreadsheet.
Elicit streamlines this process by using AI-powered semantic search to populate an in-browser data grid with papers and data extractions. This allows researchers to quickly filter, analyze, and verify information, potentially saving weeks or months of work.
Key UX Considerations for Living Documents
1. Decomposing AI Interactions
- Variable Response Times: Display results as they come in, using subtle placeholders to indicate work in progress.
- Flagging Low-Confidence Answers: Allow users to quickly identify and rephrase data extraction questions for more confident AI responses.
- Drilling into Source Quotes: Enable users to verify AI answers by providing access to relevant quotes from the source papers.
2. Managing Batch Workloads
- Responsive Interface: Maintain a light, fluid interface that responds to user interactions immediately.
- Careful Timeout Management: Set timeouts that accommodate potential delays in language model calls.
- Prioritize Onscreen Results: Focus processing power on the data that is currently visible to the user.
- Graceful Error Handling: Quietly manage retries to avoid distracting the user, while surfacing general system struggles.
3. Visualizing Cost
- Credit System: Implement a credit system that allows users to understand and manage their AI usage costs.
- Expected Cost Indicators: Display credit icons next to actions to indicate the estimated cost (small, medium, or heavy).
- Exact Estimates: When possible, provide precise credit estimates for specific actions.
- No Billing for Retries: Only charge for successful results, absorbing the cost of retries and fault tolerance.
Open Questions and Future Directions
As Living Documents evolve and scale, several open questions remain:
- Eager Loading vs. Virtualization: Determine the optimal approach for loading and displaying large datasets.
- Separate Batch Mode vs. Realtime: Consider whether to introduce a batch mode for large-scale operations.
- Navigating Large Documents: Improve navigation within the data grid, especially on smaller screens.
- Better Cost Visibility: Enhance cost transparency and provide users with a clearer mental model of query costs.
- Adapting to New Language Models: Incorporate new language models and balance cost versus quality.
- Multi-User Collaboration: Enable seamless collaboration among team members on Living Documents.
The Future of AI UX
Living Documents represent a significant step forward in AI UX, showcasing how LLMs can be leveraged to create powerful tools for knowledge synthesis and analysis. By addressing the challenges of decomposing AI interactions, managing workloads, and visualizing costs, we can unlock the full potential of AI to augment human intelligence and accelerate discovery.
As we continue to explore this frontier, we believe that Living Documents will serve as a valuable example of a new form of AI UX, paving the way for innovative applications across various industries.
