As part of a team in the AngelHack 48H global hackathon, I contributed to the development of an AI-powered tool to analyze Heineken's marketing campaigns. The hackathon involved over 200 developers from India, Singapore, Vietnam, Korea, and China. Our tool focused on using object detection (YOLOv8) to identify and count beer products (Heineken and competitors) and leveraging a Large Language and Vision Assistant (LAVA) for sentiment analysis to assess customer satisfaction and promotional staff efficiency.

Key Contributions & Impact

• Object Detection Model Training (YOLOv8): Collaborated in labeling over 1,000 images to create a training dataset for YOLOv8. Trained the YOLOv8 model to detect and count Heineken beer products and those of competitors in images, providing insights into product visibility and market presence.
• Sentiment Analysis with LAVA: Utilized LAVA (Large Language and Vision Assistant) to perform sentiment analysis on images and associated text (if available) from the marketing campaign. This helped assess customer satisfaction levels and the effectiveness of promotional staff interactions.
• UI and Dashboard Development: Contributed to the development of a user interface and dashboard to allow users to upload images and visualize the campaign analysis results. This provided a user-friendly way to interact with the AI tool and gain insights.

Challenges & Solutions

• Rapid Data Labeling: Labeling a large dataset (1,000+ images) within the 48-hour hackathon timeframe was a significant challenge. Solution: Efficiently divided the labeling tasks among team members. Used streamlined annotation tools and techniques to accelerate the process. Focused on labeling the most relevant objects (beer products) accurately.
• Integrating YOLOv8 and LAVA: Combining the outputs of the object detection model (YOLOv8) and the sentiment analysis model (LAVA) to provide a holistic campaign analysis. Solution: Developed a pipeline to process images through both models and combine their results. For example, detected objects could be used as context for the sentiment analysis performed by LAVA.
• Time Constraints of a Hackathon: Developing a fully functional AI tool and UI within the 48-hour time limit. Solution: Prioritized core features and focused on building a Minimum Viable Product (MVP). Utilized agile development principles and rapid prototyping techniques. Leveraged pre-trained models and existing libraries to accelerate development.

Lessons Learned

• Object Detection with YOLOv8: Deepened my understanding of object detection techniques and the practical application of YOLOv8.
• Sentiment Analysis with LAVA: Gained experience with using a Large Language and Vision Assistant (LAVA) for sentiment analysis.
• Importance of Data Labeling: Reinforced the critical importance of high-quality data labeling for training effective AI models.