Two customizations sit on top of the vendor's platform: a conversational layer that answers patrons and drives the robot, and an RFID layer that reads the books. Neither is supplied by PUDU. This is what they are, where they run, and what a first PoC has to contain to prove them.
The governing constraint
PUDU exposes two completely separate integration surfaces, and everything we want to build lands on the second one. Getting this right up front is the difference between a realistic estimate and a proposal that unravels in delivery.
The Cloud API is fleet orchestration only. Signed REST — dispatch a task, read status, fetch map points, receive callbacks. It has no access to peripherals, no microphone, no text-to-speech, and no arbitrary screen rendering. Callbacks time out at 5 seconds with 3 retries, so no model inference can happen inside one.
The OS SDK is an Android library we embed in our own app, running on the robot's head unit. PdCoreSDK gives us navigation, map and shelf data, tray detection, lights and the LED matrix. This is the only layer where voice and RFID can live.
So the deliverable is an application, not an integration. We are writing an Android app that ships on the robot, plus a backend behind it. The Cloud API is a supporting actor used for dispatch and reporting.
Scenarios
Five scenarios cover the service. Two of them are enough to prove both customizations, and those two are the PoC — the rest are deliberately deferred so the first build stays falsifiable.
A patron asks for a book or a subject. The robot resolves it against the live catalogue, then physically walks them to the right bay and shows the call number on screen.
Out of hours the robot runs a cruise route down the aisles. The RFID antenna reads spines continuously; every read is stamped with the robot's pose and reconciled against the catalogue. Misfiled and missing items land in a report before opening.
getCurrentPose() → read → bay mapping → diff vs ILS → exception reportA patron places a book on the tray. Tray vision detects the placement, RFID identifies exactly which title it is, check-in posts to the ILS, and the robot routes to the correct sorting trolley.
"Something like Sapiens but about economics." Keyword search fails at this; semantic retrieval over blurbs and subject headings does not. This is the one place retrieval genuinely earns its keep.
Hours, fines, membership, renewals, events. A small document corpus — if it is under a few dozen pages, prompt-stuffing beats standing up retrieval infrastructure for it.
Architecture
Capability audit
Taken directly from the PUDU Cloud API and OS SDK documentation. Anything marked unconfirmed is a live question for the call — several of them can move the estimate materially in either direction.
| Capability | Source | Status | Note |
|---|---|---|---|
| Navigation to a named point | PUDU | Exists | custom_call cloud-side; newDestinationNaviTask() on-robot. BellaBot Pro supported. |
| Map points and shelf data | PUDU | Exists | /map-service/v1/open/point and getRetailShelves(). Point types are catering-flavoured; needs a library naming convention. |
| Multi-tray delivery tasks | PUDU | Exists | delivery_task, priority 1–5, queueing, per-tray destinations. |
| Tray occupancy detection | PUDU | Exists | Vision-based. Detects that something was placed, never which book — that gap is exactly what RFID fills. |
| Robot pose stream | PUDU | Unconfirmed | poseFl exists; sampling rate and accuracy undocumented. Determines whether a read maps to a bay or only to an aisle. |
| Text-to-speech | Us | Absent | No TTS anywhere in either API. Audio endpoints are KettyBot-only and play pre-recorded named clips. |
| Microphone / speech input | Us | Absent | Not exposed. Must go through raw Android — needs mic permission and audio focus for a third-party app. |
| Existing PUDU voice assistant | PUDU | Unconfirmed | They demo "AI-powered voice interaction". If its backend is pointable at our endpoint, most of the voice scope disappears. |
| Arbitrary screen content | Us | Constrained | Cloud custom_content allows image, video or QR only. Rich UI needs our app rendering to the Android display. |
| RFID / NFC / barcode | Us | Absent | Zero mentions across the entire documentation set. Fully greenfield: hardware, mounting, driver, power. |
| Third-party APK installation | PUDU | Unconfirmed | Hard dependency. If a custom app cannot be installed and held in the foreground, the entire approach fails. |
| API credentials | PUDU | Gated | Docs restrict issuance to distributors and self-operated merchants, reviewed by a regional lead. May require vendor sponsorship. |
Work packages
Effort is relative sizing, not calendar time — real durations depend on the vendor answers and on which ILS the library runs. Packages marked for the PoC are the minimum set that proves both customizations on real hardware.
| ID | Work package | Effort | In PoC | Depends on |
|---|---|---|---|---|
| WP-01 | On-robot app shell. Android project, PdCoreSDK init and connection handling, navigation task wrapper, patron screen UI, kiosk/foreground behaviour. |
L | Yes | SDK access, dev robot |
| WP-02 | Voice I/O. Mic capture with wake word or push-to-talk, streaming ASR, TTS playback, barge-in, audio focus arbitration against the stock assistant. | L | Yes | Q5, Q6 — may shrink to S |
| WP-03 | Orchestrator and tool schemas. LLM tool-calling over the robot and catalogue APIs, dialogue state, grounding and refusal rules, latency budget. | M | Yes | WP-04 |
| WP-04 | ILS adapter. Catalogue search, item status, holdings and location lookup, check-in/out. Normalises whichever ILS the library runs into one internal contract. | M | Yes | ILS API access |
| WP-05 | Shelf-to-map mapping. Call-number ranges mapped to bays, bays mapped to PUDU map points, plus the map authoring pass on site. | M | Yes | Site survey, WP-04 |
| WP-06 | RFID hardware integration. Reader and antenna selection, mounting and mast design, power, USB or BLE link, EMI validation against LiDAR and depth sensors. | L | Yes | Q8–Q11, vendor NRE |
| WP-07 | Read reconciliation service. Pose-stamped tag reads, deduplication, read-to-bay attribution, diff against catalogue, misfiled and missing exception reporting. | M | Yes | WP-05, WP-06 |
| WP-08 | Cloud API integration. Signed request layer, task dispatch, status polling, callback receiver with async hand-off inside the 5s window. | S | Yes | Credentials |
| WP-09 | Retrieval index. Embeddings over blurbs and subject headings for discovery, plus the policy corpus. Explicitly excludes availability and location. | S | No | Content licensing |
| WP-10 | Safety and operations. Fallback when the model or network fails, escalate-to-staff path, conversation logging, an eval set for answer quality, privacy review for mic and camera. | M | Partial | — |
PoC boundary
The PoC exists to answer two questions and nothing else: can our app own the robot's interaction surface, and can RFID reads be attributed to a shelf accurately enough to be useful. Everything not serving those two questions is deferred.
Critical path
Nothing in WP-01, 02, 06 or 07 can be validated without physical hardware. Peripheral behaviour, audio focus, EMI and pose accuracy are all empirical questions the documentation does not answer. This is the longest lead item — raise it first on the call.
The entire architecture assumes we can install and foreground a custom APK with mic and USB access. If PUDU will not permit it, the approach has to be redesigned around a companion device and the estimate changes fundamentally.
Issuance is restricted to distributors and self-operated merchants. We may need PUDU to sponsor us or to resell through them. No India or Middle East node is listed — confirm where our deployment routes and what that means for latency and data residency.
Sitting entirely on our side, and more likely to be the schedule risk than anything PUDU-related. If the library runs Koha, Alma, Sierra or SirsiDynix, check the API surface and rate limits now rather than after the vendor call.
The vendor has done chassis modifications before — their own materials show a custom airport variant. But an RFID mount is new work: NRE cost, minimum order quantity and lead time all need to land before the PoC can be scheduled.
Vendor call
Q5 and Q7 have the largest effect on scope. If their existing voice stack is pluggable, WP-02 collapses from a large package to a small one.
ApiAppKey and ApiAppSecret as a systems integrator, or must it be sponsored or resold through you? Which regional node serves our deployment?sdk_demo available to us, under what licence, and which BellaBot Pro firmware version is required?getRetailShelves() available on BellaBot Pro, and how are shelves authored in the map editor?poseFl sampling rate and pose accuracy? This directly determines RFID read attribution granularity.