The Role of Large Language Models (LLMs) in AI Planning
Artificial intelligence (AI) planning involves the creation of a sequence of actions to achieve specific goals in the development of autonomous systems that perform complex tasks, such as robotics and logistics. Large language models (LLMs) have shown great promise in various areas focused on natural language processing and code generation. However, using LLMs for AI planning presents challenges, particularly when attempting to generate complete plans.
The challenge lies in achieving soundness and completeness within AI planning when LLMs are involved. This involves methods that are more scalable and effective than traditional approaches, which often rely on human experts to guide the planning phase. The goal is to automate this process with minimal loss of accuracy and reliability in LLMs.
Various challenges have been studied using different approaches, some showing promise while others remain inefficient. One approach involves treating LLMs as world models that define the search space for planning tasks. Another approach includes using LLMs to generate entire plans or planning models that automated systems evaluate. However, these methods have needed more reliability and efficiency due to a strong dependence on human feedback.
To address these challenges, researchers from Cornell University and IBM Research introduced AutoToS—a system designed to automatically generate sound and complete search components without human oversight. AutoToS aims to improve components of LLM-generated searches by using unit tests and automated debugging processes, providing assurances for soundness and completeness.
In this methodology, the system extracts successor functions and a goal test from the LLM before automatically testing them using generic and domain-specific unit tests. If any elements do not satisfy conditions for soundness or completeness, AutoToS provides detailed feedback to the LMM for code revisions—an iterative process until all generated components are fully validated.
AutoToS was critically examined across several benchmark problems within domains such as BlocksWorld, PrOntoQA, Mini Crossword, 24 Game, Sokoban—achieving 100% accuracy with significantly fewer feedback iterations compared to traditional methods.
With feedback soundness and completeness playing a key role in achieving correct results with minimal intervention from humans—AutoToS is presented as a state-of-the-art system that guarantees scalable solutions with correctness longality—in turn opening up new possibilities for developments within AI planning across various domains.
