feat(camera): add camera management functionality and data flow documentation
- Implemented camera slice in Redux store for managing camera state. - Added new tables in PostgreSQL for camera devices, lending, and audit logs. - Created camera-related schemas in contracts for validation. - Developed camera data flow documentation outlining ingest and read processes. - Introduced async thunks for fetching, creating, lending, returning, and updating cameras.
This commit is contained in:
1 parent
7923ba1b8d
commit
d3b6c5032f
11 files changed
+2524
-114
No files matched your search
@@ -0,0 +1,100 @@
|
||||
# Camera Data Flow
|
||||
|
||||
This document describes the current camera ingest and read flow.
|
||||
|
||||
## Overview
|
||||
|
||||
- Active cameras auto-generate ingest events every 1 second.
|
||||
- Events are enqueued to Redis Stream `telemetry:stream`.
|
||||
- A Redis Stream worker consumes events, retries failed writes up to 3 times, and dead-letters failures.
|
||||
- Worker writes:
|
||||
- Mongo raw telemetry events
|
||||
- Redis presence key with TTL
|
||||
- Postgres latest current state on every event
|
||||
- Postgres analytics snapshot at most once every 5 seconds per device
|
||||
- UI reads camera status by polling every 1 second.
|
||||
|
||||
## System Flow Diagram
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph CameraLane[Camera Lane]
|
||||
C1[Active Cameras]
|
||||
T1[Auto Simulator in ingest-api\nEvery 1 second]
|
||||
C1 --> T1
|
||||
end
|
||||
|
||||
subgraph IngestAPI[ingest-api Koa]
|
||||
E1[Build telemetry event]
|
||||
Q1[XADD telemetry:stream]
|
||||
API[POST /telemetry]
|
||||
E1 --> Q1
|
||||
API --> Q1
|
||||
end
|
||||
|
||||
T1 --> E1
|
||||
|
||||
subgraph Redis[Redis]
|
||||
S1[(Stream: telemetry:stream)]
|
||||
G1[Consumer Group Worker\nRetries up to 3]
|
||||
DLQ[(Stream: telemetry:stream:dlq)]
|
||||
P1[(Presence key device:id TTL)]
|
||||
end
|
||||
|
||||
Q1 --> S1
|
||||
S1 --> G1
|
||||
G1 -->|on repeated failure| DLQ
|
||||
|
||||
subgraph Stores[Storage Writes by Worker]
|
||||
M1[(Mongo telemetry raw events)]
|
||||
PG1[(Postgres device_current_state\nLatest status)]
|
||||
PG2[(Postgres device_status_snapshots\nAt most once per 5 sec per device)]
|
||||
end
|
||||
|
||||
G1 --> M1
|
||||
G1 --> P1
|
||||
G1 --> PG1
|
||||
G1 --> PG2
|
||||
|
||||
subgraph ReadPath[Read Path]
|
||||
UI[Web /devices]
|
||||
CAMAPI[GET /cameras\nGET /cameras/:id/latest]
|
||||
UI -->|poll every 1 second| CAMAPI
|
||||
CAMAPI --> PG1
|
||||
CAMAPI --> P1
|
||||
end
|
||||
```
|
||||
|
||||
## Event Timeline Diagram
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Cam as Camera Simulator
|
||||
participant Ingest as ingest api
|
||||
participant Stream as Redis Stream
|
||||
participant Worker as Stream Worker
|
||||
participant Mongo as MongoDB
|
||||
participant Pg as Postgres
|
||||
participant Web as Web UI
|
||||
|
||||
loop every second for each active camera
|
||||
Cam->>Ingest: create telemetry event
|
||||
Ingest->>Stream: XADD telemetry stream
|
||||
Ingest-->>Cam: accepted when enqueue succeeds
|
||||
end
|
||||
|
||||
loop worker processing
|
||||
Worker->>Stream: XREADGROUP
|
||||
Worker->>Mongo: insert raw telemetry
|
||||
Worker->>Pg: upsert latest device current state
|
||||
Worker->>Pg: insert snapshot if older than 5 seconds
|
||||
Worker->>Stream: XACK
|
||||
Note over Worker,Stream: retry up to three times then write to DLQ
|
||||
end
|
||||
|
||||
loop web refresh every second
|
||||
Web->>Ingest: GET cameras
|
||||
Ingest->>Pg: read latest state
|
||||
Ingest-->>Web: return camera rows with latest status
|
||||
end
|
||||
```
|
||||
Reference in new issue
Block a user