As robots are increasingly deployed in real-world environments, users expect to issue high-level navigation instructions in natural language and have the robot handle the details. A key challenge is that directly applying Large Language Models (LLMs) to path planning over 3D scene graphs exposes three fundamental failure modes: context windows overflow as map size grows (prompt complexity scales as O(|V|), where |V| is the number of graph nodes), geometric hallucinations produce paths over nonexistent edges, and transmitting scene graphs to cloud APIs risks leaking sensitive spatial data about private environments. We present PathPlanLLM, a neurosymbolic framework that resolves all three limitations by shifting the LLM’s role from planner to data engineer. Rather than serializing the scene graph into the prompt, PathPlanLLM provides the LLM only with a lightweight structural schema, the set of object category labels (typically 85-120 for indoor scenes) and uses a three-stage pipeline: a Parser that maps natural language queries to structured intents, a Semantic Linker that resolves categories to unique node identifiers, and a Generative Code Executor (GCE) that prompts the LLM to synthesize, execute, and iteratively debug Python code against a local, sandboxed scene graph. This architecture achieves O(∣Cl) prompt scaling (sublinear in graph size), privacy by design (no instance-level spatial data ever leaves the local system), and formal path validity (every returned path is executionverified or rejected). We evaluate PathPlanLLM on a 404node hierarchical scene graph across 30 test cases spanning eleven query categories, achieving a 97% success rate and outperforming both baselines by a significant margin using the same Qwen2.5-Coder 14B model.
ISBN for host publication: 979-8-3195-4747-7