Semantically Calibrated Evidence Composition for CT Vision-Language Learning
arXiv preprint, 2026 · Paper
Composes anatomy-indexed findings with whole-volume context and calibrates the evidence against a diagnostic summary.
Guoliang You
Postdoctoral Researcher · University of Pennsylvania
My research studies multimodal AI, from vision-language foundation models to agents that reason and act. My current focus is multimodal AI for healthcare, including vision-language pretraining for 3D medical images, multimodal LLMs, and evidence-grounded clinical agents. This direction builds on my Ph.D. research in embodied and autonomous systems, where I studied multi-sensor perception, vision-language reasoning, planning, and reinforcement learning. I remain involved in collaborative work on embodied AI, including vision-language-action learning and language-guided robotics.
I received my Ph.D. in Computer Technology from the University of Science and Technology of China (USTC), where I worked in the LINKE Lab with Prof. Yanyong Zhang and Assoc. Prof. Jianmin Ji. I earned my B.E. in Applied Physics from Anhui University of Science and Technology.
Research
My current work has two primary directions in healthcare, alongside continuing collaborations in embodied AI:
- Medical vision-language learning · 3D imaging, anatomy-grounded pretraining, clinical report generation
- Clinical reasoning and agents · multimodal LLMs, diagnostic QA, traceable evidence, tool use and verification
- Embodied AI collaborations · vision-language-action learning and language-guided robotics, building on earlier work in sensor fusion, planning and reinforcement learning; exploring applications in healthcare
Publications
Authors are listed in publication order; Guoliang You is in bold and * marks equal contribution. Preprints are identified separately from peer-reviewed publications.
Click an illustration to view the hand-drawn overview or the original paper figure.
Medical AI
Learning How Much, Not Just What: Cross-Patient Burden Order for CT Vision-Language Pretraining
arXiv preprint, 2026 · Paper
Learns lower-to-higher disease-burden relationships across patients without longitudinal pairs.
Learning Anatomy-Grounded CT Vision-Language Representations with Organ-Hierarchical Report Knowledge
arXiv preprint, 2026 · Paper
Links 3D visual features to clinically structured, organ-level report knowledge.
Anatomy-Grounded CT Report Generation Guided by Organ-Hierarchical Clinical Knowledge
Journal of Imaging Informatics in Medicine supplement; SIIM CAIMI, 2026 · Abstract
Uses organ-aware visual tokens and clinical knowledge to guide CT report generation.
Fine-Grained Vision-Language Pretraining with Organ-Conditioned Pattern Tokens for CT Understanding
arXiv preprint, 2026 · Paper
Models radiological patterns conditioned on the organ in which they occur.
Auditable Agentic AI for Evidence-Grounded Thyroid Ultrasound Diagnosis and Reporting
Collaborative arXiv preprint, 2026 · Paper
Studies traceable evidence and multi-step reasoning for thyroid ultrasound diagnosis.
Agents and evaluation
WebGameBench: Requirement-to-Application Evaluation for Coding Agents via Browser-Native Games
arXiv preprint, 2026 · Paper
Evaluates the delivered game through real browser interaction, beyond whether its code builds.
Embodied AI and autonomous systems
RaCFormer: Towards High-Quality 3D Object Detection via Query-Based Radar-Camera Fusion
CVPR, 2025 · Paper
Fuses radar and camera evidence through object queries for 3D detection.
GraspCoT: Integrating Physical Property Reasoning for 6-DoF Grasping under Flexible Language Instructions
ICCV, 2025 · Paper
Reasons about object properties to ground flexible language instructions in robotic grasps.
VLMPlanner: Integrating Visual Language Models with Motion Planning
ACM Multimedia, 2025 · Paper
Connects visual-language scene understanding with motion planning for autonomous driving.
LFP: Efficient and Accurate End-to-End Lane-Level Planning via Camera-LiDAR Fusion
arXiv preprint, 2024 · Paper
Combines camera and LiDAR features for lane-level planning in autonomous driving.
Perception Helps Planning: Facilitating Multi-Stage Lane-Level Integration via Double-Edge Data Structures
IEEE Robotics and Automation Letters, 2024 · Paper
Uses structured perception output to connect lane-level mapping and route planning.
P3O: Transferring Visual Representations for Reinforcement Learning via Prompting
ICME, 2023 · Paper
Transfers pretrained visual features into reinforcement learning through prompting.
CalibWorkflow: A General MLLM-Guided Workflow for Centimeter-Level Cross-Sensor Calibration
ACM Multimedia, 2025 · Paper
Uses a multimodal LLM to guide a cross-sensor calibration workflow.
RayFormer: Improving Query-Based Multi-Camera 3D Object Detection via Ray-Centric Strategies
ACM Multimedia, 2024 · Paper
Places and samples object queries along camera rays to reduce ambiguous features.
CELLmap: Enhancing LiDAR SLAM through Elastic and Lightweight Spherical Map Representation
ICRA, 2025 · Paper
Builds a compact spherical map representation for LiDAR localization and mapping.
EdgeCalib: Multi-Frame Weighted Edge Features for Automatic Targetless LiDAR-Camera Calibration
IEEE Robotics and Automation Letters, 2024 · Paper
Matches weighted edge features across frames to calibrate LiDAR and camera without a target.
Rotation Initialization and Stepwise Refinement for Universal LiDAR Calibration
arXiv preprint, 2024 · Paper
Initializes multi-LiDAR rotation with a spherical descriptor, then refines extrinsic calibration.
ElectricSight: 3D Hazard Monitoring for Power Lines Using Low-Cost Sensors
arXiv preprint, 2025 · Paper
Combines image-based hazard detection with 3D scene priors to estimate power-line clearance.
Experience
University of PennsylvaniaPostdoctoral Researcher · Philadelphia, PA
LimX DynamicsResearch Intern, multimodal foundation models and vision-language-action learning · Beijing
University of Science and Technology of China (USTC)Ph.D. in Computer Technology, LINKE Lab · HefeiAdvisors: Prof. Yanyong Zhang and Assoc. Prof. Jianmin Ji
AI Research Institute, Hefei Comprehensive National Science CenterResearch Intern, reinforcement learning and sim-to-real autonomous driving · Hefei
Anhui University of Science and TechnologyBachelor's degree in Applied Physics · Huainan
Beyond papers
Some of the most memorable parts of my research happened away from a desk: building platforms, testing them outside, and learning what changes when a model meets a physical system. These photos are from that part of my journey.
