TMCnet News
Gretel Releases World's Largest Open Source Text-to-SQL Dataset to Accelerate AI Model TrainingSAN FRANCISCO, April 04, 2024 (GLOBE NEWSWIRE) -- Gretel, the leader in synthetic data, today released the world’s largest open source Text-to-SQL dataset to unlock new possibilities for AI in the enterprise. Available on Hugging Face and released under the Apache 2.0 license, Gretel’s dataset consists of over 100,000 high-quality synthetic Text-to-SQL samples with SQL metadata and spans 100 verticals. With access to Gretel’s open-source, high-quality synthetic dataset, developers can train AI models that empower business users to extract value from critical enterprise data sources, expediting AI initiatives across the enterprise. "Access to quality training data is one of the biggest obstacles to building with generative AI. Everything Gretel does is designed to address this issue head-on, and contributing to the open-source community is no exception," said Alex Watson, co-founder & Chief Product Officer at Gretel. "By providing developers with high-quality, synthetic Text-to-SQL data, we're enabling them to create AI models that can understand natural language queries and generate SQL queries. This empowers users across the organization to easily access and derive insights from complex databases, data warehouses, and data lakes, without needing to learn SQL or rely on technical teams. We’re excited for developers to take our dataset for a spin, and build upon it.” Growing demand for AI training data Most text-to-SQL datasets today are manually curated and annotated, limiting their size, applicability and utility. This process is expenive, labor intensive, and cumbersome. For instance, the Spider text-to-SQL dataset, consisting of 7k samples, was annotated by 11 college students at Yale, and took a total of 1,000 hours to complete — an incredible amount of effort for a relatively small dataset in the context of large language models. Furthermore, the vast majority of existing Text-to-SQL datasets lack a natural language explanation of what their SQL code does. Gretel’s dataset includes an explanation field providing a plain-english description of the SQL code, which helps end users quickly understand the output and realize its value. Filling a gap in the open source community With Gretel’s Text-to-SQL dataset released under the Apache 2.0 license, AI developers can build conversational applications that open up a range of new opportunities for businesses across industries. For instance:
Elevating data quality in the age of AI The quality of Gretel’s dataset was assessed and compared to other Text-to-SQL datasets using an independent service and the LLM-as-a-judge technique. A full synopsis of Gretel’s holistic quality evaluation is available on the blog. A deeper dive into the data, including data previews, examples, and statistics, can be found on Hugging Face. To learn more about Gretel’s platform and capabilities, visit gretel.ai. About Gretel Media Contact |