RecyKool is a multimodal, agentic AI system designed to reduce downtime in Waste Management’s conveyor sorting system. The platform combines an AI chatbot interface on Apple Vision Pro, an NVIDIA Omniverse Isaac Sim digital twin of the sorting line, material characterization to identify items that may jam equipment, and gesture recognition to detect unsafe worker movements. Together, these components provide real-time operational insight that helps improve line reliability, minimize stoppages, and support safer working conditions.
Group members, responsibilities, and profiles
RecyKool is a multimodal AI system designed to support operations within Waste Management’s conveyor sorting environment. The platform is organized into three core components. The AI Assistant provides a conversational interface backed by EOD reports and equipment manuals, allowing users to query system behavior and receive grounded guidance. A vision-language model (VLM) handles material characterization and flagging, including detection of jam-causing items and recognition of unsafe worker behavior. A digital twin component models the conveyor system, enabling simulation and providing additional context for understanding real-time conditions and operational issues.
Supports text and image-based querying, connects user requests to retrieved documentation, and uses knowledge graph reasoning to provide more grounded operational guidance.
Provides an Omniverse-based facility simulation concept for understanding conveyor activity, camera input, and real-time monitoring in a visual operational context.
Uses a vision-language model approach to identify materials that may cause jams or disruptions in the sorting line.
Explores unsafe action detection using real and synthetic data to support earlier awareness of hazardous worker behavior.
Capabilities developed during the project
The main tools, models, platforms, and frameworks used across the AI assistant, detection models, digital twin, synthetic data pipeline, and XR interface.
System walkthroughs
Screenshots, video, and project links