Home

Held in conjunction with CVPR 2024 (Seattle), June 18 PM, 2024
Main Theme: Embedded Vision for Sustainable Development
Organized by: Tse-Wei Chen, Branislav Kisacanin, Marius Leordeanu, Ahmed Nabil Belbachir

Description

Embedded vision is an active field of research, bringing together efficient learning models with fast computer vision and pattern recognition algorithms, to tackle many areas of robotics and intelligent systems that are enjoying an impressive growth today. Such strong impact comes with many challenges that stem from the difficulty of understanding complex visual scenes under the tight computational constraints required by real-time solutions on embedded devices. The Embedded Vision Workshop will provide a venue for discussing these challenges by bringing together researchers and practitioners from the different fields outlined above. Such a topic is directly aligned with the topics of interest of the CVPR community.

---

Invited Talk #1

Title: Physical Modeling Approach for Simple and Energy-Efficient Analog Neuromorphic Computers
Abstract: Analog computing is a promising paradigm for realizing unprecedented energy-efficient artificial intelligence. However, due to its analog nature, the design and operation of circuits are more challenging than their digital counterparts. Furthermore, the nonideal characteristics of analog devices hinder the simple design of analog circuits without compromising the behavior discrepancy between the target model (such as artificial neural networks) and the analog hardware. One remedy for this problem is to reverse the design process from top-down (model to hardware) to bottom-up (hardware to model). In the bottom-up approach, the physical model of the hardware is directly trained. If the physical model is accurate, the discrepancy between the model and the hardware can be suppressed.
In this talk, we demonstrate that this physical-modeling approach can efficiently reduce the complexity of analog circuits, while not impairing learning performance. Additionally, within the framework of spiking neural networks, we show that a newly proposed learning method can efficiently train physical neural networks.
Biography: Yusuke Sakemi received the B.S. degree in physics from Keio University, Tokyo, Japan, in 2010, and the M.S. and Ph.D. degrees in physics from The University of Tokyo, Tokyo, in 2012 and 2015,respectively. From 2015 to 2021, he was a researcher at NEC Corporation. Since 2022 he has been a senior research scientist at the Research Center for Mathematical Engineering (RCME) at Chiba Institute of Technology. He is an Affiliated Faculty at International Research Center for Neuro intelligence (IRCN) in the University of Tokyo. His current research interests include neuromorphic engineering and machine learning, and open-source circuit design.

---

Invited Talk #2

Title: Real Time for Urban Streetscape Video-Based Applications
Abstract: Dense urban environments need technological solutions that would dramatically improve the safety of pedestrians and traffic flows. Safety-critical applications demand non-negotiable accuracy, reliability and robustness, which are hard to meet by fully autonomous systems. We explore technologies that integrate infrastructure-installed sensors such as cameras, high bandwidth communications devices, and AI-enabled embedded and edge-based computing and actuation to support smart-cities of the future. For this problem, meeting real time latencies to close observation/control loops is an underappreciated challenge. We present the results of the experiments capturing real-time demands and constraints, using the testbed deployed in Manhattan, NYC. The conclusions motivate the directions of research for future computing and communications systems.
Biography: Zoran Kostic is a Professor in the Electrical Engineering Department at Columbia University in New York City. He received the Ph.D. degree in Electrical Engineering from the University of Rochester and the Dipl. Ing. degree from the University of Novi Sad. Prior to joining Columbia, he worked on research and development in Bell Laboratories, AT&T Research, Thomson Multimedia, The Mathworks and Broadcom, and taught at several academic institutions.
Zoran’s present interests cover the applications of deep learning applications in smart cities and medicine, parallel/heterogeneous computing, internet of things, and mobile/wireless systems. Zoran’s industrial expertise spans mobile data systems and wireless communications, from algorithms and VLSI modem design to architecture of TCP/IP-based cellular networks in early OFDM-based systems. He worked on software-defined radios, display processing and openVG graphics engines. His work covered both research and SW/HW system development, which resulted in notable publications record, three dozen patents and critical contributions to successful products. As a Senior Principal Scientist, he was the system architecture lead for two multi-core mobile SoC product lines. Zoran has experience in Intellectual Property consulting. Dr. Kostic is a senior member of the IEEE. He served as an associate editor of the IEEE Transactions on Communications and IEEE Communications Letters.

---

Invited Talk #3

Title: Building Moxie – an embedded AI companion
Abstract: In the age of pervasive AI and massive LLMs, the development of consumer-grade products present a unique set of challenges. This talk describes the process of creating Moxie, an embedded AI companion, within the constraints of consumer electronics cost limitations. Unlike conventional LLM assistants, Moxie is conceived as an integrated audio-visual experience integrating a physical presence that enhances the interaction and the emotional connection. Moxie’s design aims to perform a balance in the landscape of hybrid embedded-cloud systems, striking a delicate balance between local processing and cloud connectivity while ensuring optimal responsiveness without compromising cost-effectiveness. The talk outlines Moxie’s journey from ideation to production, highlighting the challenges of the endeavour.
Biography: Mario E. Munich is Chief Technology Officer at Embodied, Inc. where he manages research & development, and engineering efforts, including hardware, software, and manufacturing with a focus on human-robot interaction, natural language processing and artificial intelligence. With 20+ years of experience developing and releasing innovative consumer robotic products, he most recently served as Senior Vice President of Research & Development at iRobot Corp. There, he managed research and technology development efforts with special focus on robot navigation, computer vision, and artificial intelligence. He previously worked as the Chief Technology Officer of Evolution Robotics where he developed object recognition, and navigation and mapping algorithms for consumer robotics. He received his degree in Electronic Engineering with honors from the National University of Rosario, Argentina, and earned his M.S. and Ph.D. degrees in Electrical Engineering from the California Institute of Technology, Pasadena. His PhD work focused on the development of novel Human-Machine Interfaces using video technology and computer vision techniques. His research interests include computer vision, machine learning, artificial intelligence, sensors for robots, autonomous navigation, and human-robotic interaction.

---

Invited Talk #4

Title: Perception/control co-design for autonomous vehicles
Abstract: This talk will cover two important challenges in the use of computer vision and neural networks to guide autonomous vehicles, both aerial and ground.  We will discuss the time required to analyze an image and the error rates of that analysis.  We will show how these computer vision tasks affect the overall design of the vehicle.
Biography: Marilyn Wolf is Elmer E. Koch Professor of Engineering and Director for ORED Engineering and Technology Initiatives at the University of Nebraska– Lincoln. She received her BS, MS, and PhD in electrical engineeringfrom Stanford University in 1980, 1981, and 1984, respectively.  She was with AT&T Bell Laboratoriesfrom 1984 to 1989.  She was on the faculty of Princeton University from 1989 to 2007 and was Farmer Distinguished Chair at Georgia Tech from 2007 to 2019.  Her researchinterests included embedded computing, embedded video and computer vision, and VLSI systems.She has received the IEEE Kirchmayer Graduate Teaching Award, the IEEE Computer Society Goode Memorial Award, the ASEE Terman Award and IEEE Circuits and Systems Society Education Award.She is a Fellow of the IEEE and ACM and an IEEE Computer Society Golden Core member.

---

Invited Talk #5

Title: Design Space Exploration for ML System Design with Hardware-in-the-loop
Abstract: Practical applications of deep learning are on a path to commoditization. What are implications for the future of ML system design? Will ML system components such as data, models, training and optimization regimens become fungible entities, understood only by their external performance characteristics with substantial disregard to their internal workings or structure? What opportunities and challenges come with this for application design for edge targets? How do performance tradeoffs (cost to train, cost to infer, accuracy, speed, energy usage, carbon footprint) relate to a system design perspective? In this talk, I’ll show how we combine thinking from Product Lifecycle Management and Component Based Software Engineering as a step towards shifting the focus of DL application development from single-use performance optimization to prescriptive performance-based design.
Biography: Dr. Jan Ernst currently serves as Director of AI at Latent AI, a US-based startup. He received his PhD from the University of Erlangen-Nürnberg, has an extensive background in applied R&D and led many innovations from “back of the napkin sketch” to industrial products in mobility, energy, automation and operations. His current focus is making AI on the Edge efficient for anyone, not just experts.

---

Important Dates

Paper submission: March 15, 2024 March 22, 2024 (Pacific Time)
Demo abstract submission: March 15, 2024 March 22, 2024 (Pacific Time)
Notification to the authors: April 7, 2024
Camera ready paper: April 14, 2024

Please refer to Submission page for details.
CMT Submission website:
https://cmt3.research.microsoft.com/EVW2024

---

Topics

• Lightweight and efficient computer vision algorithms for embedded systems
• Hardware dedicated to embedded vision systems (GPUs, FPGAs, DSPs, etc.)
• Software platforms for embedded vision systems
• Neuromorphic computing
• Applications of embedded vision systems in general domains: UAVs (industrial, mobile and consumer), Advanced assistance systems and autonomous navigation frameworks, Augmented and Virtual Reality, Robotics.
• New trends and challenges in embedded visual processing
• Analysis of vision problems specific to embedded systems
• Analysis of embedded systems issues specific to computer vision
• Biologically-inspired vision and embedded systems
• Hardware and software enhancements that impact vision applications
• Performance metrics for evaluating embedded systems
• Hybrid embedded systems combining vision and other sensor modalities
• Embedded vision systems applied to new domains

---

Committee

---

Sponsors

---

Steering Committee:
Marilyn Claire Wolf, University of Nebraska-Lincoln
Martin Humenberger, NAVER LABS Europe
Roland Brockers, Jet Propulsion Laboratory
Swarup Medasani, MathWorks
Stefano Mattoccia, University of Bologna
Jagadeesh Sankaran, Nvidia
Goksel Dedeoglu, Perceptonic
Margrit Gelautz, Vienna University of Technology
Branislav Kisacanin, Nvidia
Sek Chai, Latent AI
Zoran Nikolic, Nvidia
Ravi Satzoda, Nauto
Stephan Weiss, University of Klagenfurt

Program Committee:
Alina Marcu, University Politehnica of Bucharest
Antonio Haro, eBay
Arun Visweswaraiah, Nvidia
Branislav Kisacanin, Nvidia
Burak Ozer, Verificon Corporation
Dongchao Wen, Inspur Electronic Information Industry Co., Ltd.
Faycal Bensaali, Qatar University
Florin Condrea, Institute of Mathematics of the Romanian Academy
Linda Wills, Georgia Institute of Technology
Martin Kampel, Vienna University of Technology, Computer Vision Lab
Matteo Poggi, University of Bologna
Mihai Cristian Pîrvu, University Politehnica of Bucharest