logo

FoodFacts

How Altostruct Transformed Foodfacts' Data Accessibility on AWS

increased data scalability
99 % data accuracy

FoodFacts is a company that, with the help of AI and algorithms, uses intelligent food data to power the revolution towards a more healthy and sustainable food system. 

In the fast-paced world of food technology, FoodFacts was determined to make a difference. They identified a crucial pain point: the difficulty consumers faced in accessing accurate and comprehensive food data. Therefore, FoodFacts embarked on a journey to create a platform that provided accessible, actionable and understandable food information.

Altostruct helped FoodFacts create a robust and scalable solution to ensure that their users always had access to the most up-to-date food information by utilising AWS Lambda combined with SQS.

Problem description

Foodfacts recognized the critical need to update and maintain high-volume product data from external APIs efficiently. Ensuring data accuracy and reducing latency was paramount for delivering valuable information to their users. However, managing the data retrieval, processing, and updating processes posed a significant challenge. Ultimately, a more efficient data processing pipeline would reduce the risk of errors and increase the data fetching accuracy.

Solution

The proposed solution? Harness the capabilities of AWS Lambda and SQS (Simple Queue Service) to optimize the data processing pipeline. Foodfacts implemented a series of Lambda functions and SQS queues to efficiently fetch, process, and update product data.

Here's how we did it: Each step of the data fetching pipeline was designed as separate Lambda functions, with SQS queues serving as intermediaries between them. This architecture was intentionally designed to be loosely coupled, meaning that a failure in one part of the pipeline would only affect that specific component, leaving the rest of the system unaffected. In case of a failure, the problematic entries were routed to a Dead Letter Queue (DLQ), where they later could be processed and analyzed. This approach ensured that Foodfacts could consistently provide their users with the most current and accurate food data available. The use of DLQ and automated error handling led to a 90% reduction in manual intervention for dealing with data processing errors.

Conclusion

The implementation of AWS Lambda and SQS has launched a new era of data processing for Foodfacts, marked by exceptional results. This optimized data pipeline not only ensures 100% data accuracy but also offers a significant boost in data update speed. The architecture's resilience to failures and the automatic handling of problematic entries with use of DLQ led to a 90% reduction in manual intervention for dealing with data processing errors. 

This accomplishment underscores the power of innovative solutions and the strategic utilization of AWS services, particularly AWS Lambda and SQS, to enhance efficiency and data accuracy. As Foodfacts continues to evolve and grow, this optimized data processing framework serves as a solid foundation for delivering up-to-date food information to their users, thus creating more value for both their business and their customers.