Latest events

AIML Special Presentation: Several Recent Results in 4D Reconstruction and Generative Modelling

Miguel Balbin — Mon, 02 Dec 2024 10:45:52 +1030

Dr Golyanik’s talk focuses on several recent approaches for generative modelling and 3D reconstruction of non-rigid scenes from various conditioning signals and inputs, such as textual prompts, RGB images, or event streams. He also outlines promising directions for how the field could benefit from modern and upcoming quantum hardware.

AIML Special Presentation: From Instructional Diagrams to Real-World Assembly

Miguel Balbin — Thu, 05 Dec 2024 13:04:55 +1030

Abstract: From instructional diagrams to real-world assembly, understanding and bridging the gap between diagrammatic instructions and practical actions is a significant challenge. To address this, we introduce the Ikea Assembly in the Wild (IAW) Dataset, which comprises 183 hours of diverse furniture assembly videos and nearly 8,300 corresponding illustrations from assembly manuals, annotated with their ground truth alignments. Leveraging this dataset, we tackle three key tasks: First, we align segments of in-the-wild assembly videos with corresponding instructional diagrams using a supervised contrastive learning approach that captures subtle diagrammatic details. Second, we predict the precise start and end times of the steps outlined in the manuals within the video sequences, enabling accurate temporal grounding. Finally, we demonstrate a furniture assembly pipeline where furniture parts are selected and their 6D poses are predicted based on cues from the instructional diagrams. Our methods significantly advance multimodal alignment and learning for practical assembly, achieving state-of-the-art performance on tasks such as retrieval, temporal grounding, and 3D part assembly.

AIML Research Seminar: What Makes AI Safe for Human Decision-Making?

Miguel Balbin — Tue, 10 Dec 2024 12:00:55 +1030

Abstract: Lana is a cognitive psychology student at AIML investigating medical decision-making with AI. She incorporates knowledge of human factors, expertise, and cognitive science with areas of bioethics and AI safety. She will cover the work of her PhD, findings, and future directions.

AIML Connect Fridays: Towards the 3D Human Foundation Agent

Miguel Balbin — Fri, 13 Dec 2024 16:14:21 +1030

This talk will describe current progress on building a 3D Human Foundation Agent (HFA) that can perceive the world and the humans in it. The HFA is a digitally embodied agent that understands human behavior and responds to it using its “motor system” to translate its goals into 3D actions. The Human Foundation Agent must (1) perceive human movement in 3D, (2) understand the goals, implications, and emotions inherent in that movement, and (3) plan and generate natural motor activity to (4) drive a digital or physical embodiment that interacts with real or virtual humans in real or virtual 3D worlds. This talk will focus on current progress and the path to building HFAs through 3D human motion capture from video, synthetic training data, generative behaviour modelling, AI-driven graphics, and large vision-language models that are fine-tuned to understand 3D humans. HFAs will radically change how people interact with machines. So much so that a child born today will have trouble imagining a world in which technology doesn’t understand their motions and behaviours.