# Firebase Test Lab Integration

Automated testing on real physical Android devices using Firebase Test Lab.
Ensures the Kordant Android app works correctly across a diverse device matrix
including Pixel, Samsung, and Xiaomi devices.

## Architecture

```
firebase-test-lab/
├── README.md                         # This file
├── test_matrix_config.yaml           # Device matrix and test configuration
├── robo_script.json                  # Robo crawl script (guided UI navigation)
├── run_robo_tests.sh                 # Run Robo exploratory tests
├── run_instrumentation_tests.sh      # Run instrumentation (UI) tests
├── run_all_tests.sh                  # Run full test suite
└── download_results.sh               # Download and analyze test results
```

## Prerequisites

1. **Google Cloud Project** with Firebase enabled
2. **Blaze plan** (pay-as-you-go) — Test Lab is free for the first 100 device-minutes/day on physical devices
3. **gcloud CLI** installed and authenticated
4. **Service account** with `Firebase Test Lab Admin` role

### Installation

```bash
# Install gcloud CLI (macOS)
brew install --cask google-cloud-sdk

# Authenticate
gcloud auth login
gcloud auth application-default login

# Verify
gcloud firebase test android models list
```

### Firebase Project Setup

1. Create a Firebase project at https://console.firebase.google.com
2. Enable the Blaze (pay-as-you-go) plan
3. Optionally link to Google Play Console for deep Play Store integration
4. Create a service account and download JSON key:
   - IAM & Admin → Service Accounts → Create Service Account
   - Role: `Firebase Test Lab Admin` (roles/firebase.testlab.admin)
   - Create and download JSON key

## Device Matrix

The app is tested on 5 devices across 2 orientations and 2 locales
(20 device/locale/orientation combinations total):

| Device | Model ID | API | Screen | RAM | Target |
|--------|----------|-----|--------|-----|--------|
| Pixel 6 | `Pixel6` | 33 | 1080×2400 | 8GB | Primary target |
| Pixel 4 | `Pixel4` | 30 | 1080×2280 | 6GB | Older device |
| Galaxy S21 | `GalaxyS21` | 31 | 1080×2400 | 8GB | Samsung |
| Redmi Note 8 | `RedmiNote8` | 29 | 1080×2340 | 4GB | Xiaomi / budget |
| Aquest M2 | `AquestM2` | 28 | 720×1280 | 2GB | Low-end / minimum spec |

**Orientations:** portrait, landscape
**Locales:** en_US (English US), es_ES (Spanish Spain)

## Running Tests

### 1. Build the app

```bash
cd android
./gradlew :app:assembleProdRelease :app:assembleProdDebugAndroidTest
```

### 2. Run Robo Tests (exploratory crash detection)

Robo tests automatically crawl the app UI without requiring any test code.
They detect crashes, ANRs, and UI rendering issues.

```bash
cd android/firebase-test-lab
./run_robo_tests.sh --project-id kordant-android
```

Options:
- `--project-id` — Firebase project ID (default: `kordant-android`)
- `--app-aab` — Path to AAB (preferred, more accurate)
- `--app-apk` — Path to APK (fallback)
- `--robo-script` — Path to robo crawl script
- `--timeout` — Max crawl time in seconds (default: 600)
- `--dry-run` — Preview command without executing

### 3. Run Instrumentation Tests (UI tests with assertions)

Runs the existing UI test suite (AuthFlowTest, DashboardUITest, ServiceUITests, etc.)
across the full device matrix.

```bash
cd android/firebase-test-lab
./run_instrumentation_tests.sh --project-id kordant-android
```

Options:
- `--project-id` — Firebase project ID
- `--app-apk` — Path to app APK (auto-detected)
- `--test-apk` — Path to test APK (auto-detected)
- `--dry-run` — Preview command without executing

### 4. Run Full Test Suite

```bash
cd android/firebase-test-lab
./run_all_tests.sh --project-id kordant-android
```

Options:
- `--skip-build` — Skip Gradle build step
- `--skip-robo` — Skip Robo tests
- `--skip-instr` — Skip instrumentation tests
- `--dry-run` — Preview commands without executing

### 5. Download Results

```bash
cd android/firebase-test-lab
./download_results.sh --project-id kordant-android --download-all
```

This downloads:
- Test result XMLs (JUnit format)
- Screenshots (PNG)
- Test videos (MP4)
- Performance metrics (JSON)
- Logcat output
- Crawl maps (Robo test UI exploration paths)

## CI Integration

The GitHub Actions workflow at `.github/workflows/firebase-test-lab.yml`
runs automatically on pushes and PRs that modify Android code.

### CI Pipeline Flow

1. **Build job** — Compiles release and test APKs
2. **Robo Tests job** — Runs crash/ANR detection on all 20 device configurations
3. **Instrumentation Tests job** — Runs UI test suite on all 20 device configurations
4. **Test Summary job** — Collects results and determines pass/fail

### GitHub Secrets Required

| Secret | Description |
|--------|-------------|
| `GCP_SA_KEY_TEST_LAB` | Service account JSON key with Test Lab admin role |
| `FIREBASE_PROJECT_ID` | Firebase project ID (default: `kordant-android`) |

### Adding to CI

The workflow triggers on:
- Push to `main` with Android changes
- PR to `main` with Android changes
- Manual trigger via `workflow_dispatch`

To block release builds on test failures, add the test-summary job as a
required check in your branch protection rules.

## Robo Test Script

The `robo_script.json` file guides the Robo crawler through the app's
critical user flow:

1. Wait for splash screen
2. Click "Get Started" on the onboarding screen
3. Click "Sign In" on the login screen
4. Enter email and password
5. Submit sign-in
6. Navigate through: Dashboard → Services → Alerts → Settings

This ensures the crawler reaches authenticated screens. The test user
credentials are injected via `${ROBO_ID}` for unique user per test run.

## Test Accounts

Robo tests support test accounts for automatic login during the crawl.
Configure credentials securely:

```bash
gcloud firebase test android run \
  --type robo \
  --app app.apk \
  --device model=Pixel6,version=33,locale=en_US,orientation=portrait \
  --test-accounts username=test@kordant.com,password=$ROBO_PASSWORD
```

For CI, store credentials in GitHub Secrets and pass as environment variables.

## Analyzing Results

### In Firebase Console

1. Navigate to https://console.firebase.google.com/project/YOUR_PROJECT/testlab
2. View test matrices grouped by history name
3. Click on a matrix to see per-device results
4. Watch test videos to identify UI issues
5. Review screenshots for visual regressions
6. Check performance metrics for responsiveness

### Performance Budget

| Metric | Target | Device |
|--------|--------|--------|
| Cold start | < 1500ms | Pixel 6 |
| Warm start | < 1000ms | Pixel 6 |
| Robo crawl | Complete in < 10min per device | All |
| No crashes | 0 crashes | All |
| No ANRs | 0 ANRs | All |

### Device-Specific Issues to Watch

- **Low-end devices (API 28, 2GB RAM):** Check for OOM, slow rendering, lag
- **Xiaomi:** Check for MIUI-specific permission quirks
- **Samsung:** Check for One UI theme compatibility
- **Landscape:** Verify proper layout adaptation
- **Spanish locale:** Check for text truncation or layout overflow

## Troubleshooting

### "Permission denied" when running scripts

```bash
chmod +x android/firebase-test-lab/*.sh
```

### "No authenticated account" error

```bash
gcloud auth login
gcloud auth application-default login
```

### "Project not found" error

Verify the project exists and has the Blaze plan enabled:
```bash
gcloud projects list
gcloud firebase test android models list --project YOUR_PROJECT_ID
```

### "Quota exceeded" error

Firebase Test Lab has daily quotas. Check usage in the Firebase Console.
The free tier provides 100 device-minutes/day on physical devices.

### Test APK not found

Build the test APK first:
```bash
cd android
./gradlew :app:assembleProdDebugAndroidTest
```

## Best Practices

1. **Run Robo tests first** — They're free-form and catch crashes without test code
2. **Always test on low-end devices** — Many issues only appear on 2GB RAM devices
3. **Review screenshots** — Visual issues are common across device families
4. **Watch videos of failures** — The video shows exactly what led to the crash
5. **Run on release builds** — Debug builds may mask issues
6. **Use AAB for Robo tests** — More accurate representation of Play Store installs
7. **Set --fail-fast for CI** — Stop on first failure to save device-minutes
8. **Archive results** — Keep screenshots and videos for regression comparison