feat: initialize ingest-api with Koa and telemetry handling
- Create ingest-api with Koa framework for handling device telemetry. - Implement telemetry ingestion with MongoDB, Redis, and PostgreSQL. - Add API endpoints for telemetry and health checks. - Set up testing with Vitest for unit and end-to-end tests. - Configure Prettier and ESLint for code formatting and linting. - Create Docker Compose setup for PostgreSQL, MongoDB, and Redis services. - Establish contracts package for shared Zod schemas and TypeScript types. - Implement React frontend with device status and greeting functionalities. - Add VS Code setup documentation for onboarding developers.
This commit is contained in:
1 parent
534fa6dc51
commit
7923ba1b8d
70 files changed
+3085
-3434
No files matched your search
@@ -0,0 +1,93 @@
|
||||
# Architecture
|
||||
|
||||
Copy this layout for later device + dashboard projects.
|
||||
|
||||
## Lanes
|
||||
|
||||
- **Device lane:** `apps/ingest-api` (Koa) — firmware / gateways POST JSON.
|
||||
- **User lane:** `apps/crud-api` (Express) — React, humans, billing later.
|
||||
- **Shared types:** `packages/contracts` (Zod only). No Koa, Express, Mongo, or trest.
|
||||
|
||||
Frontend people can run `yarn workspaces focus web` and do not need the private trest registry.
|
||||
|
||||
## Data flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph devices [Device lane]
|
||||
D[Device / demo ping]
|
||||
I[Koa ingest-api :3001]
|
||||
D -->|POST /telemetry JSON| I
|
||||
end
|
||||
|
||||
subgraph stores [Stores]
|
||||
M[(MongoDB raw telemetry)]
|
||||
R[(Redis hot key TTL)]
|
||||
P[(PostgreSQL device_current_state)]
|
||||
A[(PostgreSQL analytics — not in this repo)]
|
||||
end
|
||||
|
||||
subgraph humans [User lane]
|
||||
W[React web :5173]
|
||||
C[Express crud-api :3000]
|
||||
W -->|axios GET /greeting GET /devices/:id/status| C
|
||||
end
|
||||
|
||||
I -->|insert event| M
|
||||
I -->|SET device:id EX 15s| R
|
||||
I -->|UPSERT current row| P
|
||||
M -.->|ETL cron later| A
|
||||
C -->|SELECT snapshot| P
|
||||
C -->|GET device:id| R
|
||||
```
|
||||
|
||||
Near-real-time product path (Postgres + Express is enough at this scale):
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant D as Device
|
||||
participant I as ingest-api Koa
|
||||
participant PG as Postgres
|
||||
participant C as crud-api Express
|
||||
participant W as React
|
||||
D->>I: POST /telemetry
|
||||
I->>PG: UPSERT device_current_state
|
||||
W->>C: GET /devices/dev-1/status poll 1s
|
||||
C->>PG: SELECT by device_id
|
||||
C->>W: lastSeen payload source postgres
|
||||
```
|
||||
|
||||
Redis is compared on the Devices page. It is not required for the UI to look live.
|
||||
|
||||
## Demo dual-write (do not copy blindly)
|
||||
|
||||
Ingest writes Mongo first, then Redis and Postgres best-effort. A 200 can still mean Redis or Postgres missed. Production should derive those from a queue. This handler is a teaching shortcut.
|
||||
|
||||
Redis TTL expiry means “no recent ping.” Redis restart looks like everyone is offline. Postgres still has `device_current_state`.
|
||||
|
||||
Ingest may only upsert that snapshot table. CRUD owns greeting / future users and orgs.
|
||||
|
||||
Hourly Mongo → Postgres analytics is the dotted ETL box. Not implemented.
|
||||
|
||||
## Ports
|
||||
|
||||
| Service | Port |
|
||||
| ---------- | ----- |
|
||||
| crud-api | 3000 |
|
||||
| ingest-api | 3001 |
|
||||
| web | 5173 |
|
||||
| postgres | 5432 |
|
||||
| mongo | 27017 |
|
||||
| redis | 6379 |
|
||||
|
||||
## Run
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
yarn install
|
||||
yarn dev:crud
|
||||
yarn dev:ingest
|
||||
yarn dev:web
|
||||
```
|
||||
|
||||
Open `http://127.0.0.1:5173/devices`, click **Ping dev-1**, watch Postgres and Redis columns. Wait 15s without pinging: Redis goes offline, Postgres keeps the last row.
|
||||
Reference in new issue
Block a user