Turning Unstructured Data into Insights

The AI Application Platform for Unstructured Data

Instabase is an enterprise-grade AI Application Platform that turns unstructured data into insights, instantly. They solve one of the most acute problems businesses face – slow or inaccurate decisioning due to unstructured data. Their engineering team was looking for a database that would allow them to deliver data insights to their users with a high degree of accuracy and rapid performance. They chose Weaviate because of the flexibility that a leading open-source tool gave them while hitting Instabase’s critical performance metrics better than any other database they tested. Built-in features like hybrid search and distance metrics, as well as support from an engaged developer community, made Weaviate the clear choice for Instabase.

Challenge

Instabase processes over 500K highly varied documents per day and needs to scale to the ever-growing demand of their customers. They needed a database solution that would allow them to index, store, and retrieve massive volumes of data while delivering results with an incredibly high level of accuracy to their users. “Accuracy determines the amount of savings any large institution can get,” said Shaunak Godbole, Head of Infrastructure Engineering at Instabase. “If the results aren’t accurate or take too long to surface, a human needs to get involved, and the cost savings goes away. So accuracy and speed are critical for us.”  Additionally, Instabase needed to support customers in highly regulated environments. European customers needed to ensure their data didn’t leave certain countries, while financial institutions didn’t want their data to leave their on-premises servers. They needed a database solution that could be deployed anywhere while maintaining a high level of performance. If accuracy, speed, and flexible deployment criteria were not met, Instabase would not be able to reduce the need for human intervention in complex data workflows for their customers.

Why Instabase Chose Weaviate

Instabase wants to empower their customers to focus on making fast and accurate decisions. They achieve this by classifying, extracting, and validating information from highly unstructured data. “The collaboration with Weaviate’s team, the community, and the results of our performance tests made Weaviate an easy choice,” notes Godbole. “In terms of performance, nobody needed convincing – the benchmarks spoke for themselves.”

Weaviate checked all of Instabase’s boxes:

• Powerful performance: Instabase had their own benchmarks for the use cases they knew they had to solve really, really well. These benchmarks focus on retrieval accuracy and latency, and the queries are complex aggregation and composition where both dense and sparse searches are required. Each use case had different benchmarks, and Weaviate was a clear winner with high retrieval accuracy and low latency for vector search. Once the benchmarking team was convinced Weaviate was up for the challenge, teams across the organization worked together to bring Weaviate into production.
• High adaptability: By using an AI-native open-source vector database, Instabase could meet their customers wherever they operated, whether in the cloud or on-prem. This allowed for maximum flexibility to fulfill the strict deployment needs of organizations in highly regulated regions and industries.
• Modular architecture: Instead of having to build out their own capabilities, Instabase developers saved time by using Weaviate’s out of the box features like hybrid search and distance metrics, as well as easy integrations with popular large language models (LLMs).
• Strong support: Weaviate has cultivated an engaged, open source community with over 6M downloads and tens of thousands of organizations using the platform. Over 5K developers have direct access to the experts and comprehensive documentation they need to quickly problem solve as they build their applications. In addition, Weaviate’s core team was very collaborative with Instabase and was able to provide short, medium and long-term solutions to existing and anticipated challenges.

“Accuracy determines the amount of savings any large institution can get. If the results aren’t accurate or take too long to surface, a human needs to get involved, and the cost savings are greatly reduced. So accuracy and speed are critical for us.”

Shaunak Godbole, Head of Infrastructure Engineering at Instabase