Robotic Inspection of Photovoltaic Hotspots
Two industrial robots revived from a lab closet, a YOLOv8 model trained on 1,682 thermal images, and mAP@0.5 = 0.985 on hotspot detection.
Role — Solo researcher — hardware restoration through model deployment
- Drone-based PV inspection is fast but misses underside faults and close-range detail. A robotic arm can inspect panels from angles drones can't reach — if the perception pipeline is accurate enough to trust.
- Perception pipeline + robotic inspection platform
- Thermal faults degrade solar output and can cascade. Beyond the energy application, the project is a full-lifecycle proof: dead hardware to deployed AI, with the metrics to show for it.
- Solo project under Dr. Kevin Wedeward, Fort Lewis College (summer research internship).
Overview
A solo research project under Dr. Kevin Wedeward: a lab-validated system for detecting thermal faults in photovoltaic panels using a 7-DoF Sawyer arm and a custom-trained YOLOv8 model. The arm performs automated multi-angle thermal inspections — including underside components that drone-based inspection misses. The project spanned the full lifecycle: first reviving two industrial robots dormant since 2018 (tracing a Sawyer boot failure to a dead CMOS battery that re-enabled Secure Boot, then rebuilding the OS on a new SSD), then building the ROS Noetic / MoveIt / Gazebo environment, engineering a Blender synthetic-data pipeline, and training and validating the detector on live hardware. Placed 2nd at FLC's Physics & Engineering Symposium.
Two data sources — Blender-rendered synthetic imagery and real multi-angle thermal captures — train a YOLOv8 detector, which runs against the Sawyer's live thermal feed as the arm sweeps panels from angles drones can't reach. Edge cases discovered on hardware feed the next training round.
- Blender synthetic renders + real thermal captures
- YOLOv8 training
- Hotspot detector
- Sawyer 7-DoF multi-angle sweep
- Live inspection & localization
Contributions
- Revived two non-functional industrial robots with no vendor support — diagnosed the Sawyer's boot failure (dead CMOS battery → BIOS reset → incompatible Secure Boot), disassembled the controller, and rebuilt the OS on a new SSD with fresh Intera software.
- Built the complete ROS Noetic workstation and control environment; resolved firmware–SDK version conflicts and configured MoveIt and Gazebo for motion planning and simulation.
- Engineered a synthetic-data pipeline in Blender + Python to generate training imagery at scale; explored NVIDIA Omniverse for scalable generation.
- Trained a YOLOv8 hotspot detector on 1,682 thermal images and validated it live on Baxter/Sawyer hardware with real-time thermal feeds.
Evidence & evaluation
Evidence
Detection metrics
attachedmAP@0.5 = 0.985 · 98% detection accuracy · trained on 1,682 thermal images.
Live hardware validation
attachedValidated on Baxter/Sawyer hardware with real-time thermal feeds.
Pipeline diagram
attachedData-to-deployment pipeline in the gallery.
Edge-case gallery
pendingCurate the inputs that break the detector — honest operating envelope.
Metrics
0.985
98%
1,682 images
2nd place
Limitations
- Lab-validated, not field-deployed — panel variety, weather, and mounting geometry in the field remain untested.
- Synthetic data closed the volume gap, but the synthetic-to-real distribution shift for rare fault types isn't yet quantified.
Lessons & tradeoffs
- The first 100+ hours were hardware revival, not AI — embodied systems reward system-level thinking and patience with legacy machines.
- Synthetic data is leverage: Blender scripting turned a data-starved problem into a data-rich one, but only after the real captures defined what 'realistic' meant.
