InfoSys 2023 Congress
March 13, 2023 to March 17, 2023 - Barcelona, Spain

  • ICNS 2023, The Nineteenth International Conference on Networking and Services
  • ICAS 2023, The Nineteenth International Conference on Autonomic and Autonomous Systems
  • ENERGY 2023, The Thirteenth International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies
  • WEB 2023, The Eleventh International Conference on Building and Exploring Web Based Environments
  • DBKDA 2023, The Fifteenth International Conference on Advances in Databases, Knowledge, and Data Applications
  • SIGNAL 2023, The Eighth International Conference on Advances in Signal, Image and Video Processing
  • BIOTECHNO 2023, The Fifteenth International Conference on Bioinformatics, Biocomputational Systems and Biotechnologies

InfoWare 2023 Congress
March 13, 2023 to March 17, 2023 - Barcelona, Spain

  • ICCGI 2023, The Eighteenth International Multi-Conference on Computing in the Global Information Technology
  • ICWMC 2023, The Nineteenth International Conference on Wireless and Mobile Communications
  • VEHICULAR 2023, The Twelfth International Conference on Advances in Vehicular Systems, Technologies and Applications
  • INTERNET 2023, The Fifteenth International Conference on Evolving Internet
  • COLLA 2023, The Thirteenth International Conference on Advanced Collaborative Networks, Systems and Applications
  • INTELLI 2023, The Twelfth International Conference on Intelligent Systems and Applications
  • VISUAL 2023, The Eighth International Conference on Applications and Systems of Visual Paradigms
  • HUSO 2023, The Ninth International Conference on Human and Social Analytics
  • BRAININFO 2023, The Eighth International Conference on Neuroscience and Cognitive Brain Information

NexComm 2023 Congress
April 24, 2023 to April 28, 2023 - Venice, Italy

  • ICDT 2023, The Eighteenth International Conference on Digital Telecommunications
  • SPACOMM 2023, The Fifteenth International Conference on Advances in Satellite and Space Communications
  • ICN 2023, The Twenty-Second International Conference on Networks
  • ICONS 2023, The Eighteenth International Conference on Systems
  • MMEDIA 2023, The Fifteenth International Conference on Advances in Multimedia
  • PESARO 2023, The Thirteenth International Conference on Performance, Safety and Robustness in Complex Systems and Applications
  • CTRQ 2023, The Sixteenth International Conference on Communication Theory, Reliability, and Quality of Service
  • ALLDATA 2023, The Ninth International Conference on Big Data, Small Data, Linked Data and Open Data
  • SOFTENG 2023, The Ninth International Conference on Advances and Trends in Software Engineering

DigitalWorld 2023 Congress
April 24, 2023 to April 28, 2023 - Venice, Italy

  • ICDS 2023, The Seventeenth International Conference on Digital Society
  • ACHI 2023, The Sixteenth International Conference on Advances in Computer-Human Interactions
  • GEOProcessing 2023, The Fifteenth International Conference on Advanced Geographic Information Systems, Applications, and Services
  • eTELEMED 2023, The Fifteenth International Conference on eHealth, Telemedicine, and Social Medicine
  • eLmL 2023, The Fifteenth International Conference on Mobile, Hybrid, and On-line Learning
  • eKNOW 2023, The Fifteenth International Conference on Information, Process, and Knowledge Management
  • ALLSENSORS 2023, The Eighth International Conference on Advances in Sensors, Actuators, Metering and Sensing
  • SMART ACCESSIBILITY 2023, The Eighth International Conference on Universal Accessibility in the Internet of Things and Smart Environments

DataSys 2023 Congress
June 26, 2023 to June 30, 2023 - Nice, Saint-Laurent-du-Var, France

  • AICT 2023, The Nineteenth Advanced International Conference on Telecommunications
  • ICIW 2023, The Eighteenth International Conference on Internet and Web Applications and Services
  • ICIMP 2023, The Eighteenth International Conference on Internet Monitoring and Protection
  • SMART 2023, The Twelfth International Conference on Smart Cities, Systems, Devices and Technologies
  • IMMM 2023, The Thirteenth International Conference on Advances in Information Mining and Management
  • INFOCOMP 2023, The Thirteenth International Conference on Advanced Communications and Computation
  • MOBILITY 2023, The Thirteenth International Conference on Mobile Services, Resources, and Users
  • SPWID 2023, The Ninth International Conference on Smart Portable, Wearable, Implantable and Disability-oriented Devices and Systems
  • ACCSE 2023, The Eighth International Conference on Advances in Computation, Communications and Services

ComputationWorld 2023 Congress
June 26, 2023 to June 30, 2023 - Nice, Saint-Laurent-du-Var, France

  • SERVICE COMPUTATION 2023, The Fifteenth International Conference on Advanced Service Computing
  • CLOUD COMPUTING 2023, The Fourteenth International Conference on Cloud Computing, GRIDs, and Virtualization
  • FUTURE COMPUTING 2023, The Fifteenth International Conference on Future Computational Technologies and Applications
  • COGNITIVE 2023, The Fifteenth International Conference on Advanced Cognitive Technologies and Applications
  • ADAPTIVE 2023, The Fifteenth International Conference on Adaptive and Self-Adaptive Systems and Applications
  • CONTENT 2023, The Fifteenth International Conference on Creative Content Technologies
  • PATTERNS 2023, The Fifteenth International Conference on Pervasive Patterns and Applications
  • COMPUTATION TOOLS 2023, The Fourteenth International Conference on Computational Logics, Algebras, Programming, Tools, and Benchmarking
  • BUSTECH 2023, The Thirteenth International Conference on Business Intelligence and Technology

NetWare 2023 Congress
September 25, 2023 to September 29, 2023 - Porto, Portugal

  • SENSORCOMM 2023, The Seventeenth International Conference on Sensor Technologies and Applications
  • SENSORDEVICES 2023, The Fourteenth International Conference on Sensor Device Technologies and Applications
  • SECURWARE 2023, The Seventeenth International Conference on Emerging Security Information, Systems and Technologies
  • AFIN 2023, The Fifteenth International Conference on Advances in Future Internet
  • CENICS 2023, The Sixteenth International Conference on Advances in Circuits, Electronics and Micro-electronics
  • ICQNM 2023, The Seventeenth International Conference on Quantum, Nano/Bio, and Micro Technologies
  • FASSI 2023, The Ninth International Conference on Fundamentals and Advances in Software Systems Integration
  • GREEN 2023, The Eighth International Conference on Green Communications, Computing and Technologies

NexTech 2023 Congress
September 25, 2023 to September 29, 2023 - Porto, Portugal

  • UBICOMM 2023, The Seventeenth International Conference on Mobile Ubiquitous Computing, Systems, Services and Technologies
  • ADVCOMP 2023, The Seventeenth International Conference on Advanced Engineering Computing and Applications in Sciences
  • SEMAPRO 2023, The Seventeenth International Conference on Advances in Semantic Processing
  • AMBIENT 2023, The Thirteenth International Conference on Ambient Computing, Applications, Services and Technologies
  • EMERGING 2023, The Fifteenth International Conference on Emerging Networks and Systems Intelligence
  • DATA ANALYTICS 2023, The Twelfth International Conference on Data Analytics
  • GLOBAL HEALTH 2023, The Twelfth International Conference on Global Health Challenges
  • CYBER 2023, The Eighth International Conference on Cyber-Technologies and Cyber-Systems

TrendNews 2023 Congress
September 25, 2023 to September 29, 2023 - Porto, Portugal

  • CORETA 2023, Advances on Core Technologies and Applications
  • DIGITAL 2023, Advances on Societal Digital Transformation

SocSys 2023 Congress
November 13, 2023 to November 17, 2023 - Valencia, Spain

SoftNet 2023 Congress
November 13, 2023 to November 17, 2023 - Valencia, Spain

  • ICSEA 2023, The Eighteenth International Conference on Software Engineering Advances
  • ICSNC 2023, The Eighteenth International Conference on Systems and Networks Communications
  • CENTRIC 2023, The Sixteenth International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services
  • VALID 2023, The Fifteenth International Conference on Advances in System Testing and Validation Lifecycle
  • SIMUL 2023, The Fifteenth International Conference on Advances in System Simulation
  • SOTICS 2023, The Thirteenth International Conference on Social Media Technologies, Communication, and Informatics
  • INNOV 2023, The Twelfth International Conference on Communications, Computation, Networks and Technologies
  • HEALTHINFO 2023, The Eighth International Conference on Informatics and Assistive Technologies for Health-Care, Medical Support and Wellbeing

IARIA Congress 2023, The 2023 IARIA Annual Congress on Frontiers in Science, Technology, Services, and Applications
November 13, 2023 to November 17, 2023 - Valencia, Spain

 

 


ThinkMind // ICAS 2022, The Eighteenth International Conference on Autonomic and Autonomous Systems // View article icas_2022_1_60_20026


A Multimodal AI Approach for Intuitively Instructable Autonomous Systems: A Case Study of an Autonomous Off-Highway Vehicle

Authors:
Abdellatif Bey Temsamani
Anil Kumar Chavali
Ward Vervoort
Tinne Tuytelaars
Gorjan Radevski
Hugo Van Hamme
Kevin Mets
Matthias Hutsebaut-Buysse
Tom De Schepper
Steven Latré

Keywords: AI based autonomous systems; Multimodal AI; Natural language processing (NLP); deep learning; neural networks; reinforcement learning

Abstract:
In current production shop floors, a fleet of production machines and AGVs form a full manufacturing system with a high degree of automation. These current manufacturing systems need to deal with high variability of products and production tasks. Every task, however, requires a proper reconfiguration & control that is often done manually requiring complex settings and long configuration time. With AI techniques the reconfiguration of these systems to deal with a new task can be made more intuitive. In this paper we present the upgrade of an autonomous system, used for manufacturing assets handling and transportation, with AI features that make it easy to reconfigure in order to deal with high variability of assets and missions. Visual and spoken information is used to instruct and guide the autonomous vehicle using an AI multimodal framework where first, spoken language, with different local dialects, is translated to digital instructions, that can be associated to visual information to form control instructions to the autonomous vehicle. Different AI models, respectively for spoken language understanding, visual perception, vision based navigation are associated through a multimodal AI framework to intuitively control the AGV to perform a specific task. Beside the challenges related to the integration of these models in the AGV platform, other challenges related to dealing with variabilities of dialects, objects, surroundings and ambient conditions are partly tackled in this research.

Pages: 31 to 39

Copyright: Copyright (c) IARIA, 2022

Publication date: May 22, 2022

Published in: conference

ISSN: 2308-3913

ISBN: 978-1-61208-966-9

Location: Venice, Italy

Dates: from May 22, 2022 to May 26, 2022

SERVICES CONTACT
2010 - 2022 © ThinkMind. All rights reserved.
Read Terms of Service and Privacy Policy.