Active development2025 — presentAuthor

Kurat

Autonomous conversational robot

A companion robot split into three brains — perception, cognition, action — so that any one of them can fail and recover without taking the robot down with it.

Problem

A robot that talks, sees and moves has three subsystems with completely different failure characteristics and timing requirements. Speech recognition stalling should not stop the robot navigating; a lost localization fix should not end a conversation.

Coupling them into one pipeline means the slowest and least reliable component sets the behaviour of the whole robot.

What I built

Three subsystems with independent failure and recovery, running on one embedded platform.

Perception — YOLOv8 detection and ORB-SLAM3 localization on RealSense D435i RGB-D
Cognition — speech via Whisper, and the conversation system on top
Action — Nav2 navigation on the mechanical platform
Modular autonomy boundaries so each brain degrades on its own

Architecture

The three-brain split is the architecture: perception, cognition and action communicate through defined interfaces rather than a shared pipeline, so a failure in one is a degraded capability rather than a stopped robot. All three run on a Jetson Orin Nano with a RealSense D435i as the primary sensor.

Three brains, so one can fail

The split is the architecture. Each brain degrades on its own rather than taking the robot down — speech stalling should not stop navigation, and a lost localization fix should not end a conversation.

Perception
YOLOv8 detection and ORB-SLAM3 localization on RealSense D435i RGB-D.
Localization lost — navigation degrades, conversation continues
Cognition
Whisper speech recognition and the conversation system.
Speech stalls — the robot keeps moving
Action
Nav2 navigation on the mechanical platform.
Path blocked — recovery, while the other two brains carry on
Each recovers independently

What runs where

All of it on a single Jetson Orin Nano, which is the constraint that shapes what can run concurrently.

Brain	Models	Sensor
Perception	YOLOv8, ORB-SLAM3	RealSense D435i RGB-D
Cognition	Whisper, conversation system	Microphone
Action	Nav2	Odometry and the chassis

Limitations and failure modes

Active development — the conversation and navigation subsystems are not yet validated together at length.
Running four models on a single Jetson Orin Nano constrains what can run concurrently.

Gallery

Kurat — RealSense D435i and Jetson Orin Nano on a four-wheel chassis

Attribution

Built by me: Kurat is mine — architecture, integration, hardware and software.
Based on external research: YOLOv8, Whisper, ORB-SLAM3 and Nav2 are upstream, integrated here.

SMR300
Autonomy stack for a 300 kg industrial AMR
SO101 Manipulation and SmolVLA Experiments
Policy training and evaluation on the LeRobot SO101 platform