# Data migrations

Changes every environment needs (new fields, indexes, menu nodes, seed rows, backfills)
go here, not in `OneTimeCronController`. Deploys apply them automatically - nobody runs
them by hand on dev or prod.

`php yii mongodb-migrate/up` applies every file in this folder not yet recorded in the
`migration` collection, in filename order.

Use `mongodb-migrate`, never plain `migrate`: `php yii migrate` is Yii's SQL migration tool
and fails with `Failed to instantiate component or class "db"` - this app has no SQL
connection.

Locally, from the host:

```bash
docker exec -w /var/www/html/ng-crm/yiiapp optimacrmphp php yii mongodb-migrate/up --interactive=0
```

or inside the container (`docker exec -it optimacrmphp bash`):

```bash
php yii mongodb-migrate/up
```

## When they run

| Environment | Where                     | When                                              |
|-------------|---------------------------|---------------------------------------------------|
| Prod        | `scripts/deploy.prod.sh`  | after the blue/green flip, once the old colour is stopped |
| Dev         | `scripts/deploy.dev.sh`   | every deploy, after the RBAC migrations           |
| Local       | by hand (command below)   | after pulling code that adds a migration          |

On prod only the new release is serving by the time they run, so old code can neither
see a change (e.g. a menu node for a route it lacks) nor race it (e.g. writing rows a
backfill is fixing). A failure there does not roll back the deploy: it finishes with
status `DONE_WITH_WARNINGS`, and the migration - still unrecorded - is retried on the
next deploy. On dev a failure stops the deploy.

Because migrations run after the new code is live, the new code must cope with the
migration not having run yet (e.g. a field that is still missing on old rows).

## Writing one

```bash
php yii mongodb-migrate/create add_payments_menu
```

Rules:

1. **Idempotent.** Mongo cannot roll back a half-applied migration; the fix is to
   correct it and run again. A second run must change nothing (check before insert,
   `$set` rather than `$inc`, skip rows already done).
2. **No arguments, no agency-specific data.** One-off operational scripts (seeding one
   client, anonymising one agency) stay console commands run by hand.
3. **Ids from `Counters::getNextIdentityCounter`**, stepping past ids already in use -
   counters can lag hand-inserted rows.
4. `down()` returns false unless a real reversal exists.
5. Removing or renaming data is a two-deploy job: ship the code that stops reading it,
   then remove it in a later deploy.

## Useful commands

```bash
php yii mongodb-migrate/new            # pending, not yet applied
php yii mongodb-migrate/history        # applied, newest first
php yii mongodb-migrate/mark <version> # treat everything after <version> as pending again
```

## Restoring a database

The history lives in the same database as the data, so a full restore rewinds both
together: anything applied after the backup becomes pending again. Run them straight
after restoring rather than waiting for the next deploy:

```bash
php yii mongodb-migrate/up --interactive=0
```

After a *partial* restore (a few collections, not the history), the history can claim a
migration ran that the restored data no longer has. `mark` back to before it, then `up` -
safe because migrations are idempotent.