# DATA_GUIDE.md — the Mumbai ops export

This guide describes the shape of the files in `data/`. It is written by us, the
Sage AI team, the way we'd hand the export to a colleague: here are the columns,
here's what they mean, here are the units. It does **not** tell you what to do
with the data, and it does **not** tell you which rows are good. That part is the
job.

All four files cover one week of Cityflo Mumbai North operations, service dates
**Monday 2026-06-15 through Friday 2026-06-19**. Use whichever subset your chosen
domain needs — you do not need all four.

## Conventions used across the export

- **Currency:** any monetary figure is in Indian Rupees (INR). None of these
  particular files carry fare amounts, but pass and refund context lives in the
  tickets.
- **Timestamps:** ISO 8601 with an explicit UTC offset. Cityflo Mumbai runs on
  IST, so a well-formed timestamp ends in `+05:30`. Parse them as
  timezone-aware datetimes; don't assume every string is well-formed just
  because most are.
- **Routes:** referenced by a stable id like `R-12` and by a human label like
  `Route 12`. The same route is the same id all week.
- **Vehicles:** Indian number plates, e.g. `MH-12-7781`. A plate can appear on
  more than one trip and more than one day.
- **Devices:** the on-bus GPS/telemetry unit that produced a trip's actual
  times, e.g. `D-22`. Devices are physical hardware; a flaky unit tends to show
  up on more than one trip.
- **Depots:** Andheri, Borivali, Vashi, Thane.

A note Priya passed along with the export, reproduced here neutrally: the GPS
feed has been intermittent on a couple of buses this week, and there's a belief
on the team that a few timestamps came through wrong. We have not cleaned
anything. The export is the raw export.

---

## `data/trips.csv`

One row per scheduled trip for the week. ~140 rows.

| column | meaning |
|---|---|
| `trip_id` | unique id for the trip, `TRIP_001`… |
| `service_date` | the operating day, `YYYY-MM-DD` |
| `route_id` | route id, e.g. `R-12` |
| `route_label` | human label, e.g. `Route 12` |
| `vehicle_plate` | the vehicle that ran the trip, e.g. `MH-12-7781` |
| `device_id` | the on-bus telemetry device that reported the actual times, e.g. `D-22` |
| `scheduled_departure` | planned departure, ISO 8601 `+05:30` |
| `actual_departure` | reported actual departure, ISO 8601 |
| `scheduled_arrival` | planned arrival, ISO 8601 `+05:30` |
| `actual_arrival` | reported actual arrival, ISO 8601 |
| `booked_seats` | seats sold on the trip |
| `capacity` | vehicle capacity, 30 or 40 |

Lateness is not a column. There is no `late` flag and no agreed threshold in
this file — what counts as "late" is a decision, not a lookup. If you compute
lateness, you compute it from the scheduled and actual times yourself.

## `data/occupancy.csv`

Point-in-time occupancy snapshots taken against trips. ~90 rows.

| column | meaning |
|---|---|
| `snapshot_id` | unique id for the snapshot, `SNAP_001`… |
| `trip_id` | the trip this snapshot is for (foreign key into `trips.csv`) |
| `service_date` | operating day |
| `route_id` | route id |
| `vehicle_plate` | vehicle |
| `occupancy_pct` | reported occupancy as a percentage (0–100) |
| `boarded` | reported headcount that boarded |
| `capacity` | vehicle capacity, 30 or 40 |

`occupancy_pct` and `boarded` are two separate readings of the same thing from
the same snapshot; we report both as they arrived.

## `data/tickets.csv`

Inbound support tickets for the week. ~40 rows.

| column | meaning |
|---|---|
| `ticket_id` | unique id, `TKT_001`… |
| `created_at` | when the ticket was filed, ISO 8601 `+05:30` (from the rider's device) |
| `channel` | how it came in: `app`, `whatsapp`, `email`, `call` |
| `route_id` | route the rider referenced, if any (can be blank) |
| `rider_text` | the rider's own words, free text. Some are in Hinglish. |
| `category_guess` | an automatic tag — `refund`, `safety`, `lost_item`, `driver_behaviour`, `app_bug`, `general`. It is a best-effort auto-classification and is **not** always right. |
| `status` | `open`, `pending`, `resolved` |

`rider_text` is exactly what the rider typed, untouched. `category_guess` was
applied by an older auto-tagger and is provided for convenience only — treat it
as a hint, not ground truth.

## `data/ops_log.txt`

The free-text overnight ops log, one timestamped IST line per entry, in the
phrasing the depot shift writes it. ~50 lines. It's a running narrative: most of
it is routine depot housekeeping, and a few lines are things that actually
matter for the morning. There is no structure beyond the leading timestamp —
it reads like what it is, a log a human typed through the night.

---

## A note on realism

This is synthetic data, but it's modelled on the real shape of a Cityflo Mumbai
ops export: the volumes, the route and depot names, the kind of phrasing riders
and depot staff actually use, and the kind of mess a 7am CSV export actually
carries. We did not sand the edges off. Real operational data has gaps,
disagreements between two fields that should agree, a device having a bad week,
the same thing reported twice, and a clock somewhere that isn't set right. We
left all of that in, because handling it is the work. Run the data before you
trust it.