Open-sourcing 4 solutions from the Enterprise RAG Challenge

We’ve written earlier about the first round of our Enterprise RAG Challenge. In this friendly challenge different AI Assistants competed in answering questions based on the annual reports of public companies. Teams from the different countries competed in building such assistants. There were even a few commercial solutions in the mix.

We are open sourcing code of four solutions from this leaderboard, including the winning one.

You can find code for the solutions marked with “TTA” (TimeToAct) on Github. Solutions include description of the approach, code itself, sometimes even the log of failed experiments.

Code is real, without any cleanups and beautification.

In short:

  1. Daniel - winning and surprisingly simple solution that used checklist pattern with structured outputs. First place.
    → Check out Daniel's solution
     

  2. Felix - multi-agent solution with ChatGPT-4o. 12th place.
    → Check out Felix' solution
     

  3. Maria - solution using OpenAI Assistants API. 13th place.
    → Check out Maria's solution
     

  4. Pedro - locally-capable solution using openchat-3.5-0106. ninth place.
    → Check out Pedro's solution  
     

What's next?

Next round of Enterprise RAG Challenge will take place later this fall, with a bigger audience. The exact time depends on the organisation process within the TIMEOTACT GROUP. Perhaps, around November.

In the next round, question generator will be rebalanced, so that:

  1. There are less questions that don't have an answer (agents must respond N/A to these)

  2. There is more variability in the questions, so that “brute force” approach with checklists+structured outputs will not be able to win the competition so easily.

Questionnaires for the participants will also be reworked, so that we all together could learn more about approaches for Enterprise AI that work well in practice.

Insights

These are the proud winners of the Enterprise RAG Challenge

Discover the winners of the Enterprise RAG Challenge! Explore top RAG solutions, watch the official announcement, and see how AI-driven retrieval and LLMs shaped the best-performing models.

Insights

Team-Leaderboard of the Enterprise RAG Challenge

The team-leaderboard includes all submitted entries – including those submitted after the Ground Truth was released. Therefore, we consider this ranking an unofficial overview.

Blog 3/11/25

Answering Business Questions with LLMs

8th place in Enterprise RAG Challenge 2025: Answering Business Questions with LLMs

Blog

How I Won the Enterprise RAG Challenge

In this article, Ilia Ris describes the approach that helped him achieve first place in both prize categories and the overall SotA leaderboard.

Blog 1/21/25

AI Contest - Enterprise RAG Challenge

TIMETOACT GROUP Austria demonstrates how RAG technologies can revolutionize processes with the Enterprise RAG Challenge.

Headerbild zu IBM Watson Assistant
Technologie

IBM Watson Assistant

Watson Assistant identifies intention in requests that can be received via multiple channels. Watson Assistant is trained based on real-live requests and can understand the context and intent of the query based on the acting AI. Extensive search queries are routed to Watson Discovery and seamlessly embedded into the search result.

Blog 7/22/24

So You are Building an AI Assistant?

So you are building an AI assistant for the business? This is a popular topic in the companies these days. Everybody seems to be doing that. While running AI Research in the last months, I have discovered that many companies in the USA and Europe are building some sort of AI assistant these days, mostly around enterprise workflow automation and knowledge bases. There are common patterns in how such projects work most of the time. So let me tell you a story...

Wissen 8/30/24

LLM-Benchmarks August 2024

Instead of our general LLM benchmarks, we present the first benchmark of different AI architectures in August.

Kollaboration mit dem modernen Helpdesk
Produkt 4/5/23

Backup and recovery solutions for the cloud - HYCU in focus

Experience the ultimate cloud experience with the strong partnership of CLOUDPILOTS and HYCU. Our pioneering solutions are designed to meet all your cloud needs.

Headerbild zu Operationalisierung von Data Science (MLOps)
Service

Operationalization of Data Science (MLOps)

Data and Artificial Intelligence (AI) can support almost any business process based on facts. Many companies are in the phase of professional assessment of the algorithms and technical testing of the respective technologies.

On-Premises in die Atlassian Cloud bei Oetiker nahtlos integriert
Referenz

Seamless migration from on-premises to the Atlassian cloud

Oetiker, a leading supplier of connecting solutions for the automotive industry, migrated to the cloud with catworkx Jira and Confluence in 2021.

Referenz

smartOPTIMO migrates its Atlassian solutions to the cloud

smartOPTIMO advises on energy solutions and uses Jira for tickets and Confluence for collaboration. The cloud migration increases efficiency and ensures communication.

Unternehmen

CLOUDPILOTS Software & Consulting GmbH

CLOUDPILOTS consults and supports companies in the transformation process of business processes and applications to the Cloud. Also, it assists in the implementation of Cloud based IT services (SaaS).

Blog 7/22/24

Let's build an Enterprise AI Assistant

Let’s take the basic principles of building AI assistants for a spin with a product case that we worked on: using AI to support enterprise sales pipeline.

Blog 5/16/24

Common Mistakes in the Development of AI Assistants

We share how failures when implementing AI occurr and what can be learned from them for future projects: So that AI assistants can be implemented more successfully in the future!

Wissen 4/14/23

General Data Protection Regulation of idea management

Walldorf-based dacuro GmbH provides the external data protection officer for companies, helps with the fulfillment of documentation obligations and advises on all aspects of data protection. Fulfilling the requirements of the GDPR without blocking everyday life is the claim of dacuro GmbH. The team of lawyers and IT specialists provides support for all GDPR challenges, whether they are of a legal or technical nature.

Blog 11/5/24

AIM Hackathon 2024: Sustainability Meets LLMs

Focusing on impactful AI applications, participants addressed key issues like greenwashing detection, ESG report relevance mapping, and compliance with the European Green Deal.

Blog 2/21/22

The Power of Event Sourcing

This is how we used Event Sourcing to maintain flexibility, handle changes, and ensure efficient error resolution in application development.

Blog 10/30/24

Second Place - AIM Hackathon 2024: Trustpilot for ESG

The NightWalkers designed a scalable tool that assigns trustworthiness scores based on various types of greenwashing indicators, including unsupported claims and inaccurate data.

Blog 2/3/25

ChatGPT & Co: LLM Benchmarks for January

Find out which large language models outperformed in the January 2025 benchmarks. Stay informed on the latest AI developments and performance metrics.

Bleiben Sie mit dem TIMETOACT GROUP Newsletter auf dem Laufenden!