Workshop on Planning in the Era of LLMs (LM4Plan @ AAAI 2025)

User: What are 10 steps that I should take towards planning a conference workshop on 'Planning in the Era of Large Language Models'?
GPT4: Organizing a conference workshop on "Planning in the Era of Large Language Models" is an exciting endeavor. The topic is increasingly relevant, and there's a lot to consider both in terms of content and logistics. Here's a 10-step plan to get you started:

Step 1: Clearly Define Objectives and Scope
Start by identifying the objectives you want the workshop to achieve. Is it an academic forum, a practical skills-building workshop, or a networking event? The scope could range from ethical considerations to technical aspects, or could encompass all of these.

Step 2: Identify Your Target Audience
Who would benefit the most from your workshop? Academics, industry professionals, or students? The audience you target will inform many of your subsequent decisions, from speakers to the types of activities and presentations.

Step 3: Assemble a Team
Organizing a workshop is a complex task that usually requires a team. This may include co-chairs, volunteers, and administrative support. You'll need people to handle logistics, outreach, content creation, and day-of-event activities.

Step 4: Secure Funding and Sponsorships
The financial aspect is critical. You'll need to prepare a budget that includes venue costs, speakers' fees (if applicable), catering, equipment rentals, and marketing. Sponsorships can help cover some of these costs and add credibility to your event.

Step 5: Decide on a Date and Location
Choose a date that doesn't conflict with other major events in the same field. The location should be convenient for your target audience. Due to the global relevance of large language models, consider also offering a virtual participation option.

Step 6: Call for Speakers and Topics
Put out a call for papers or speakers, depending on the format of your workshop. You may also personally invite key figures in the field. Decide on the type of content you want (e.g., lectures, panel discussions, hands-on sessions).

Step 7: Promote the Event
Use social media, academic journals, and community outreach to promote the event. You could also partner with universities, companies, and online forums to get the word out.

Step 8: Review Submissions and Confirm Speakers
If you've called for papers or speaker proposals, you'll need a review process. This may involve peer reviews or an advisory committee. Confirm your speakers well in advance and provide them with any logistical information they may need.

Step 9: Logistics and Rehearsal
Prepare all the logistical details such as registration, badges, technical setup, and any workshop materials. If possible, do a dry run a day before the workshop to ensure that everything runs smoothly.

Step 10: Conduct the Workshop and Collect Feedback
On the day of the event, make sure you have a team to manage different aspects, from registration to managing Q&A sessions. After the event, send out surveys to collect feedback for future improvements.

Remember to document the proceedings and share them with participants afterward, and perhaps consider publishing a summary or report to reach a wider audience and contribute to the field.

Workshop Details

  • Venue: AAAI’25 Workshop, Pennsylvania Convention Center, Philadelphia, Pennsylvania, USA
  • Date: March 3, 2025
  • Location: Room 117

Preliminary Schedule

Start time End time Agenda Item Title  
9:00 9:05 Opening remarks    
9:05 9:50 Invited talk Speaker: Manling Li, Title: From Large Language Models to Large Agent Models: MDP formulation of Agent Training  
9:50 9:58 paper talk Predicate Invention from Pixels via Pretrained Vision-Language Models  
9:58 10:06 paper talk S3E: Semantic Symbolic State Estimation With Vision-Language Foundation Models  
10:06 10:14 paper talk Transition Function Prediction in AI Planning Using LLMs  
10:14 10:22 paper talk Creating PDDL Models from Javascript using LLMs: Preliminary Results  
10:22 10:30 paper talk Bootstrapping Object-level Planning with Large Language Models  
10:30 11:00 Coffee    
11:00 11:45 Posters Session I Starts during the coffee break  
11:45 12:30 Invited talk Speaker: Christian Muise, Title: What model understanding can do in our quest to bridge planning and natural language  
12:30 14:00 Lunch    
14:00 14:45 Invited talk Speaker: Shuang Li, Title: How Vision and Language Models Are Changing Decision-Making  
14:45 15:30 Posters Session II Continues into the coffee break  
15:30 16:00 Coffee    
16:00 16:08 paper talk ACPBench: Reasoning about Action, Change, and Planning  
16:08 16:16 paper talk ACPBench Hard: Unrestrained Reasoning about Action, Change, and Planning  
16:16 16:24 paper talk Conversational Goal-Conflict Explanations in Planning via Multi-Agent LLMs  
16:24 16:32 paper talk Planning in the Dark: LLM-Symbolic Planning Pipeline without Experts  
16:32 16:40 paper talk Chasing Progress, Not Perfection: Revisiting Strategies for End-to-End LLM Plan Generation  
16:40 16:48 paper talk Learning to Reason with Transformers via Search Inductive Biases: A Proposal  
16:48 16:56 paper talk A Roadmap to Guide the Integration of LLMs in Hierarchical Planning  
17:00 18:00 Panel Moderator: Christian Muise.  

List of papers accepted as orals (titles, no author info)

  • A Roadmap to Guide the Integration of LLMs in Hierarchical Planning
  • ACPBench Hard: Unrestrained Reasoning about Action, Change, and Planning
  • ACPBench: Reasoning about Action, Change, and Planning
  • Bootstrapping Object-level Planning with Large Language Models
  • Conversational Goal-Conflict Explanations in Planning via Multi-Agent LLMs
  • Creating PDDL Models from Javascript using LLMs: Preliminary Results
  • Chasing Progress, Not Perfection: Revisiting Strategies for End-to-End LLM Plan Generation
  • Learning to Reason with Transformers via Search Inductive Biases: A Proposal
  • Planning in the Dark: LLM-Symbolic Planning Pipeline without Experts
  • Predicate Invention from Pixels via Pretrained Vision-Language Models
  • S3E: Semantic Symbolic State Estimation With Vision-Language Foundation Models
  • Transition Function Prediction in AI Planning Using LLMs

List of accepted papers as posters (titles, no author info), split into two sessions

Poster session I

  • Adaptive Iterative Feedback Prompting for Obstacle-Aware Path Planning via LLMs
  • APT: Architectural Planning and Text-to-Blueprint Construction Using Large Language Models for Open-World Agents
  • Benchmark Real-time Adaptation and Communication Capabilities of Embodied Agent in Collaborative Scenarios
  • CAPTAIN: Continuous Automated Planning Through Autonomous Internet Navigation
  • DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planning
  • Factored State Sampling
  • Language and Planning in Robotic Navigation: A Multilingual Evaluation of State-of-the-Art Models
  • Look Further Ahead: Testing the Limits of GPT-4 in Path Planning
  • Planning with Vision-Language Models and a Use Case in Robot-Assisted Teaching
  • SFF Rendering-Based Uncertainty Prediction using VisionLLM

Poster session II

  • A Critical Assessment of LLMs for Solving Multi-Step Problems: Preliminary Results
  • CSG-Driver: Common Sense Guided Autonomous Driving under Legal Compliance and Practical Flexibility in Dilemma Situations
  • Evaluating the Meta- and Object-Level Reasoning of Large Language Models for Question Answering
  • Leveraging Large Language Models for Automated Planning and Model Construction: A Survey
  • Leveraging LLMs for Generating Document-Informed Hierarchical Planning Models: A Proposal
  • Mutual Enhancement of Large Language and Reinforcement Learning Models through Bi-Directional Feedback Mechanisms: A Planning Case Study
  • Planning AI Assistant for Emergency Decision-Making (PlanAID): Framing Planning Problems and Assessing Plans with Large Language Models
  • Starjob: Dataset for LLM-Driven Job Shop Scheduling
  • Towards Siloed LLM-based Systems for Mission-critical Planning
  • Towards Zero-Shot, Controllable Dialog Planning with LLMs


Call for Papers

 #### Overview

Large Language Models (LLMs) are a disruptive force, changing how research was done in many sub-areas of AI. Planning is one of the last bastions that remain standing. The focus of this workshop is on the questions in the intersection of these areas. Some of the specific areas we would like to gain a better understanding in include: what LLMs can contribute to planning, how LLMs can/should be used, what are the pitfalls of using LLMs, what are the guarantees that can be obtained.

#### Workshop Topics

We are welcoming paper submissions on the following topics:

  • Planning directly with pre-trained or fine-tuned LLMs.
  • LLMs for (partial) model elicitation.
  • LLMs for generating structured planning problem descriptions.
  • LLMs for search guidance or search pruning.
  • Validation/verification of plans, policies, or models.
  • Planning for LLMs.
  • Using LLMs to develop interfaces for planning-based systems.
  • Using LLMs as a proxy for user preferences.
  • Generalization in planning and generalized planning with LLMs.
  • Using LLMs to develop interfaces for planning-related problems.
  • Other applications of LLMs in planning.
  • Other applications of large vision-language models (VLMs) in planning.
  • Planning for LLMs and VLMs.
#### Important Dates Paper submission deadline: **November 29th, 2024, AoE** (extended, final) Paper acceptance notification: **December 11th, 2024, AoE** AAAI will be in-person this year. Authors of accepted workshop papers are expected to register for the workshop, physically attend the conference and present in person. #### Submission Details We solicit workshop paper submissions relevant to the above call of the following types: Long papers – up to 8 pages + unlimited references / appendices Short papers – up to 4 pages + unlimited references / appendices Paper submissions should be made through [OpenReview](https://openreview.net/group?id=AAAI.org/2025/Workshop/LM4Plan). The review process is single blind by default, so please include author details. If the paper is currently under review at some other venue, we do allow an anonymized submission. To clarify, we do not enforce anonymity for submissions but do support it - authors' names will not be displayed on OpenReview during the review period. It is entirely up to the authors to decide whether they wish to anonymize the PDFs they submit. Please format submissions in AAAI style (see instructions in the [Author Kit](https://aaai.org/authorkit25) ). Authors submitting papers rejected from other conferences, please ensure you do your utmost to address the comments given by the reviewers. Some accepted long papers will be invited for contributed talks. All accepted papers (long as well as short) will be given a slot in the poster presentation session.


Organizing Committee
Sarath Sreedharan, Colorado State University
Jiayuan Mao, MIT
Wenlong Huang, Stanford
Michael Katz, IBM Research
Subbarao Kambhampati, Arizona State University

Please send your inquiries to llmforplanning@gmail.com