significant android work

This commit is contained in:
2026-06-02 00:04:30 -04:00
parent 542172d1e8
commit 6c4d77bbec
53 changed files with 5182 additions and 587 deletions

View File

@@ -0,0 +1,271 @@
# Firebase Test Lab Integration
Automated testing on real physical Android devices using Firebase Test Lab.
Ensures the Kordant Android app works correctly across a diverse device matrix
including Pixel, Samsung, and Xiaomi devices.
## Architecture
```
firebase-test-lab/
├── README.md # This file
├── test_matrix_config.yaml # Device matrix and test configuration
├── robo_script.json # Robo crawl script (guided UI navigation)
├── run_robo_tests.sh # Run Robo exploratory tests
├── run_instrumentation_tests.sh # Run instrumentation (UI) tests
├── run_all_tests.sh # Run full test suite
└── download_results.sh # Download and analyze test results
```
## Prerequisites
1. **Google Cloud Project** with Firebase enabled
2. **Blaze plan** (pay-as-you-go) — Test Lab is free for the first 100 device-minutes/day on physical devices
3. **gcloud CLI** installed and authenticated
4. **Service account** with `Firebase Test Lab Admin` role
### Installation
```bash
# Install gcloud CLI (macOS)
brew install --cask google-cloud-sdk
# Authenticate
gcloud auth login
gcloud auth application-default login
# Verify
gcloud firebase test android models list
```
### Firebase Project Setup
1. Create a Firebase project at https://console.firebase.google.com
2. Enable the Blaze (pay-as-you-go) plan
3. Optionally link to Google Play Console for deep Play Store integration
4. Create a service account and download JSON key:
- IAM & Admin → Service Accounts → Create Service Account
- Role: `Firebase Test Lab Admin` (roles/firebase.testlab.admin)
- Create and download JSON key
## Device Matrix
The app is tested on 5 devices across 2 orientations and 2 locales
(20 device/locale/orientation combinations total):
| Device | Model ID | API | Screen | RAM | Target |
|--------|----------|-----|--------|-----|--------|
| Pixel 6 | `Pixel6` | 33 | 1080×2400 | 8GB | Primary target |
| Pixel 4 | `Pixel4` | 30 | 1080×2280 | 6GB | Older device |
| Galaxy S21 | `GalaxyS21` | 31 | 1080×2400 | 8GB | Samsung |
| Redmi Note 8 | `RedmiNote8` | 29 | 1080×2340 | 4GB | Xiaomi / budget |
| Aquest M2 | `AquestM2` | 28 | 720×1280 | 2GB | Low-end / minimum spec |
**Orientations:** portrait, landscape
**Locales:** en_US (English US), es_ES (Spanish Spain)
## Running Tests
### 1. Build the app
```bash
cd android
./gradlew :app:assembleProdRelease :app:assembleProdDebugAndroidTest
```
### 2. Run Robo Tests (exploratory crash detection)
Robo tests automatically crawl the app UI without requiring any test code.
They detect crashes, ANRs, and UI rendering issues.
```bash
cd android/firebase-test-lab
./run_robo_tests.sh --project-id kordant-android
```
Options:
- `--project-id` — Firebase project ID (default: `kordant-android`)
- `--app-aab` — Path to AAB (preferred, more accurate)
- `--app-apk` — Path to APK (fallback)
- `--robo-script` — Path to robo crawl script
- `--timeout` — Max crawl time in seconds (default: 600)
- `--dry-run` — Preview command without executing
### 3. Run Instrumentation Tests (UI tests with assertions)
Runs the existing UI test suite (AuthFlowTest, DashboardUITest, ServiceUITests, etc.)
across the full device matrix.
```bash
cd android/firebase-test-lab
./run_instrumentation_tests.sh --project-id kordant-android
```
Options:
- `--project-id` — Firebase project ID
- `--app-apk` — Path to app APK (auto-detected)
- `--test-apk` — Path to test APK (auto-detected)
- `--dry-run` — Preview command without executing
### 4. Run Full Test Suite
```bash
cd android/firebase-test-lab
./run_all_tests.sh --project-id kordant-android
```
Options:
- `--skip-build` — Skip Gradle build step
- `--skip-robo` — Skip Robo tests
- `--skip-instr` — Skip instrumentation tests
- `--dry-run` — Preview commands without executing
### 5. Download Results
```bash
cd android/firebase-test-lab
./download_results.sh --project-id kordant-android --download-all
```
This downloads:
- Test result XMLs (JUnit format)
- Screenshots (PNG)
- Test videos (MP4)
- Performance metrics (JSON)
- Logcat output
- Crawl maps (Robo test UI exploration paths)
## CI Integration
The GitHub Actions workflow at `.github/workflows/firebase-test-lab.yml`
runs automatically on pushes and PRs that modify Android code.
### CI Pipeline Flow
1. **Build job** — Compiles release and test APKs
2. **Robo Tests job** — Runs crash/ANR detection on all 20 device configurations
3. **Instrumentation Tests job** — Runs UI test suite on all 20 device configurations
4. **Test Summary job** — Collects results and determines pass/fail
### GitHub Secrets Required
| Secret | Description |
|--------|-------------|
| `GCP_SA_KEY_TEST_LAB` | Service account JSON key with Test Lab admin role |
| `FIREBASE_PROJECT_ID` | Firebase project ID (default: `kordant-android`) |
### Adding to CI
The workflow triggers on:
- Push to `main` with Android changes
- PR to `main` with Android changes
- Manual trigger via `workflow_dispatch`
To block release builds on test failures, add the test-summary job as a
required check in your branch protection rules.
## Robo Test Script
The `robo_script.json` file guides the Robo crawler through the app's
critical user flow:
1. Wait for splash screen
2. Click "Get Started" on the onboarding screen
3. Click "Sign In" on the login screen
4. Enter email and password
5. Submit sign-in
6. Navigate through: Dashboard → Services → Alerts → Settings
This ensures the crawler reaches authenticated screens. The test user
credentials are injected via `${ROBO_ID}` for unique user per test run.
## Test Accounts
Robo tests support test accounts for automatic login during the crawl.
Configure credentials securely:
```bash
gcloud firebase test android run \
--type robo \
--app app.apk \
--device model=Pixel6,version=33,locale=en_US,orientation=portrait \
--test-accounts username=test@kordant.com,password=$ROBO_PASSWORD
```
For CI, store credentials in GitHub Secrets and pass as environment variables.
## Analyzing Results
### In Firebase Console
1. Navigate to https://console.firebase.google.com/project/YOUR_PROJECT/testlab
2. View test matrices grouped by history name
3. Click on a matrix to see per-device results
4. Watch test videos to identify UI issues
5. Review screenshots for visual regressions
6. Check performance metrics for responsiveness
### Performance Budget
| Metric | Target | Device |
|--------|--------|--------|
| Cold start | < 1500ms | Pixel 6 |
| Warm start | < 1000ms | Pixel 6 |
| Robo crawl | Complete in < 10min per device | All |
| No crashes | 0 crashes | All |
| No ANRs | 0 ANRs | All |
### Device-Specific Issues to Watch
- **Low-end devices (API 28, 2GB RAM):** Check for OOM, slow rendering, lag
- **Xiaomi:** Check for MIUI-specific permission quirks
- **Samsung:** Check for One UI theme compatibility
- **Landscape:** Verify proper layout adaptation
- **Spanish locale:** Check for text truncation or layout overflow
## Troubleshooting
### "Permission denied" when running scripts
```bash
chmod +x android/firebase-test-lab/*.sh
```
### "No authenticated account" error
```bash
gcloud auth login
gcloud auth application-default login
```
### "Project not found" error
Verify the project exists and has the Blaze plan enabled:
```bash
gcloud projects list
gcloud firebase test android models list --project YOUR_PROJECT_ID
```
### "Quota exceeded" error
Firebase Test Lab has daily quotas. Check usage in the Firebase Console.
The free tier provides 100 device-minutes/day on physical devices.
### Test APK not found
Build the test APK first:
```bash
cd android
./gradlew :app:assembleProdDebugAndroidTest
```
## Best Practices
1. **Run Robo tests first** — They're free-form and catch crashes without test code
2. **Always test on low-end devices** — Many issues only appear on 2GB RAM devices
3. **Review screenshots** — Visual issues are common across device families
4. **Watch videos of failures** — The video shows exactly what led to the crash
5. **Run on release builds** — Debug builds may mask issues
6. **Use AAB for Robo tests** — More accurate representation of Play Store installs
7. **Set --fail-fast for CI** — Stop on first failure to save device-minutes
8. **Archive results** — Keep screenshots and videos for regression comparison

View File

@@ -0,0 +1,225 @@
#!/usr/bin/env bash
# =============================================================================
# Download Firebase Test Lab Results
# =============================================================================
# Downloads and organizes test results from Firebase Test Lab, including
# screenshots, videos, performance metrics, and test reports.
#
# Usage:
# ./download_results.sh [options]
#
# Options:
# --project-id Firebase project ID (default: kordant-android)
# --matrix-id Specific matrix ID to download (optional, downloads latest)
# --output-dir Output directory (default: ./test_results)
# --download-all Download all artifacts including screenshots and videos
# --help Show this help message
# =============================================================================
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# Default values
PROJECT_ID="${FIREBASE_PROJECT_ID:-kordant-android}"
MATRIX_ID=""
OUTPUT_DIR="${SCRIPT_DIR}/test_results"
DOWNLOAD_ALL=false
# ============================================================
# Parse arguments
# ============================================================
while [[ $# -gt 0 ]]; do
case "$1" in
--project-id)
PROJECT_ID="$2"
shift 2
;;
--matrix-id)
MATRIX_ID="$2"
shift 2
;;
--output-dir)
OUTPUT_DIR="$2"
shift 2
;;
--download-all)
DOWNLOAD_ALL=true
shift
;;
--help)
grep "^#" "$0" | grep -v "^#!/" | sed 's/^# //'
exit 0
;;
*)
echo "Unknown option: $1"
echo "Usage: $0 --help"
exit 1
;;
esac
done
# ============================================================
# Validate gcloud
# ============================================================
if ! command -v gcloud &> /dev/null; then
echo "Error: gcloud CLI is not installed."
exit 1
fi
# ============================================================
# Find the GCS bucket for test results
# ============================================================
echo "🔍 Finding Firebase Test Lab results bucket..."
echo "Project ID: $PROJECT_ID"
# Get the storage bucket name from the Firebase project
# Test Lab results are stored in gs://<project-id>-test-lab-<random-suffix>
RESULTS_BUCKET=$(gsutil ls 2>/dev/null | grep "${PROJECT_ID}-test-lab-" || echo "")
if [ -z "$RESULTS_BUCKET" ]; then
echo "No test lab bucket found via gsutil. Trying gcloud to list recent matrices..."
echo ""
fi
# ============================================================
# List recent test matrices
# ============================================================
echo "📋 Recent test matrices:"
echo ""
RECENT_MATRICES=$(gcloud firebase test android matrices list \
--project "$PROJECT_ID" \
--limit 10 \
--format="table(matrixId, state, gcsPath, createTime)" 2>/dev/null || echo "No matrices found.")
echo "$RECENT_MATRICES"
echo ""
# ============================================================
# If no matrix ID specified, get the latest
# ============================================================
if [ -z "$MATRIX_ID" ]; then
MATRIX_ID=$(gcloud firebase test android matrices list \
--project "$PROJECT_ID" \
--limit 1 \
--format="value(matrixId)" 2>/dev/null || echo "")
fi
if [ -z "$MATRIX_ID" ]; then
echo "No test matrices found. Run tests first."
exit 1
fi
echo "Selected matrix: $MATRIX_ID"
echo ""
# ============================================================
# Get GCS path for this matrix
# ============================================================
MATRIX_INFO=$(gcloud firebase test android matrices describe "$MATRIX_ID" \
--project "$PROJECT_ID" \
--format="json" 2>/dev/null || echo "{}")
GCS_PATH=$(echo "$MATRIX_INFO" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('gcsPath',''))" 2>/dev/null || echo "")
if [ -z "$GCS_PATH" ]; then
echo "Error: Could not determine GCS path for matrix $MATRIX_ID"
exit 1
fi
echo "GCS Path: $GCS_PATH"
echo ""
# ============================================================
# Create output directory
# ============================================================
mkdir -p "$OUTPUT_DIR"
# ============================================================
# Download results summary (JUnit XML)
# ============================================================
echo "📄 Downloading test results summary..."
echo " Output: $OUTPUT_DIR/"
# Download the test_results.xml (JUnit format)
gsutil -m cp "$GCS_PATH/**/test_result.xml" "$OUTPUT_DIR/" 2>/dev/null || true
gsutil -m cp "$GCS_PATH/**/test_results.xml" "$OUTPUT_DIR/" 2>/dev/null || true
# Download the performance metrics
gsutil -m cp "$GCS_PATH/**/performance_metrics.json" "$OUTPUT_DIR/performance/" 2>/dev/null || true
# Download the logcat output (if available)
gsutil -m cp "$GCS_PATH/**/logcat" "$OUTPUT_DIR/logcat/" 2>/dev/null || true
# ============================================================
# Download screenshots and videos (if --download-all)
# ============================================================
if [ "$DOWNLOAD_ALL" = true ]; then
echo ""
echo "🖼️ Downloading screenshots and videos..."
# Download all PNG screenshots
mkdir -p "$OUTPUT_DIR/screenshots"
gsutil -m cp "$GCS_PATH/**/*.png" "$OUTPUT_DIR/screenshots/" 2>/dev/null || true
SCREENSHOT_COUNT=$(find "$OUTPUT_DIR/screenshots" -name "*.png" 2>/dev/null | wc -l | tr -d ' ')
echo " Screenshots downloaded: $SCREENSHOT_COUNT"
# Download all MP4 videos
mkdir -p "$OUTPUT_DIR/videos"
gsutil -m cp "$GCS_PATH/**/*.mp4" "$OUTPUT_DIR/videos/" 2>/dev/null || true
VIDEO_COUNT=$(find "$OUTPUT_DIR/videos" -name "*.mp4" 2>/dev/null | wc -l | tr -d ' ')
echo " Videos downloaded: $VIDEO_COUNT"
# Download crawl maps (Robo test output)
mkdir -p "$OUTPUT_DIR/crawl_maps"
gsutil -m cp "$GCS_PATH/**/*.json" "$OUTPUT_DIR/crawl_maps/" 2>/dev/null || true
CRAWL_COUNT=$(find "$OUTPUT_DIR/crawl_maps" -name "*.json" 2>/dev/null | wc -l | tr -d ' ')
echo " Crawl maps downloaded: $CRAWL_COUNT"
fi
# ============================================================
# Generate report
# ============================================================
echo ""
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "📊 Results Summary"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
# Parse matrix info for summary
MATRIX_STATE=$(echo "$MATRIX_INFO" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('state','UNKNOWN'))" 2>/dev/null || echo "UNKNOWN")
echo "Matrix State: $MATRIX_STATE"
echo ""
# Show outcome summary per device
echo "$MATRIX_INFO" | python3 -c "
import sys, json
d = json.load(sys.stdin)
tests = d.get('testExecutions', [])
for t in tests:
device = t.get('device', {})
model = device.get('androidModelId', '?')
version = device.get('androidVersionId', '?')
state = t.get('state', '?')
outcome = t.get('outcome', {}).get('summary', '?')
print(f' {model} (API {version}): {state} - {outcome}')
" 2>/dev/null || echo " (Could not parse individual device results)"
echo ""
echo "Output directory: $OUTPUT_DIR"
echo ""
echo "View in Firebase Console:"
echo " https://console.firebase.google.com/project/$PROJECT_ID/testlab/histories"
# ============================================================
# Check for failures
# ============================================================
if echo "$MATRIX_STATE" | grep -qi "fail\|error\|invalid"; then
echo ""
echo "⚠️ Test matrix has failures! Review the results."
exit 1
else
echo ""
echo "✅ Test matrix completed successfully!"
exit 0
fi

View File

@@ -0,0 +1,247 @@
#!/usr/bin/env bash
# =============================================================================
# Run All Firebase Test Lab Tests
# =============================================================================
# Orchestrates the full Firebase Test Lab test suite:
# 1. Robo exploratory tests (crash detection without test code)
# 2. Instrumentation tests (UI tests with assertions)
#
# This script builds the app, runs both test types sequentially, and
# reports results.
#
# Prerequisites:
# 1. gcloud CLI installed and authenticated
# 2. Firebase project with Blaze plan enabled
# 3. Service account with Firebase Test Lab admin role
# 4. Java 17+ for Android builds
#
# Usage:
# ./run_all_tests.sh [options]
#
# Options:
# --project-id Firebase project ID (default: kordant-android)
# --skip-build Skip the Gradle build step
# --skip-robo Skip Robo tests (run instrumentation only)
# --skip-instr Skip instrumentation tests (run Robo only)
# --dry-run Print commands without executing
# --help Show this help message
# =============================================================================
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Default values
PROJECT_ID="${FIREBASE_PROJECT_ID:-kordant-android}"
SKIP_BUILD=false
SKIP_ROBO=false
SKIP_INSTR=false
DRY_RUN=false
# ============================================================
# Parse arguments
# ============================================================
while [[ $# -gt 0 ]]; do
case "$1" in
--project-id)
PROJECT_ID="$2"
shift 2
;;
--skip-build)
SKIP_BUILD=true
shift
;;
--skip-robo)
SKIP_ROBO=true
shift
;;
--skip-instr)
SKIP_INSTR=true
shift
;;
--dry-run)
DRY_RUN=true
shift
;;
--help)
grep "^#" "$0" | grep -v "^#!/" | sed 's/^# //'
exit 0
;;
*)
echo "Unknown option: $1"
echo "Usage: $0 --help"
exit 1
;;
esac
done
# ============================================================
# Timestamp helper
# ============================================================
timestamp() {
date "+%Y-%m-%d %H:%M:%S"
}
# ============================================================
# Validate prerequisites
# ============================================================
echo "=========================================="
echo "Firebase Test Lab - Full Test Suite"
echo "=========================================="
echo "Started at: $(timestamp)"
echo "Project ID: $PROJECT_ID"
echo ""
# Check gcloud
if ! command -v gcloud &> /dev/null; then
echo "Error: gcloud CLI is not installed."
echo "Install it from: https://cloud.google.com/sdk/docs/install"
exit 1
fi
# Check authentication
if ! gcloud auth application-default print-access-token &> /dev/null; then
echo "Warning: Application default credentials not set."
echo "Run: gcloud auth application-default login"
echo ""
# Check if user is authenticated at all
if ! gcloud auth list --filter=status:ACTIVE --format="value(account)" &> /dev/null; then
echo "Error: No active gcloud account. Run: gcloud auth login"
exit 1
fi
fi
# ============================================================
# Step 1: Build the app (if not skipped)
# ============================================================
if [ "$SKIP_BUILD" = false ]; then
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "📦 Step 1: Building Android APKs"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""
cd "$PROJECT_DIR/android"
# Determine Java version
JAVA_VERSION=$(java -version 2>&1 | head -1 | grep -oP 'version "\K[^"]+' || echo "unknown")
echo "Java version: $JAVA_VERSION"
if [ "$DRY_RUN" = true ]; then
echo "[DRY-RUN] Would run: ./gradlew :app:assembleProdRelease :app:assembleProdDebugAndroidTest"
else
echo "Building release APK and test APK..."
./gradlew :app:assembleProdRelease :app:assembleProdDebugAndroidTest
echo ""
echo "Build completed. APK locations:"
find "$PROJECT_DIR/android/app/build/outputs" -name "*.apk" -type f 2>/dev/null | while read -r apk; do
size=$(stat -f%z "$apk" 2>/dev/null || stat -c%s "$apk" 2>/dev/null || echo "?")
echo " $(basename "$apk") ($(echo "scale=1; $size/1048576" | bc) MB)"
done
fi
echo ""
else
echo "⏭️ Build step skipped."
echo ""
fi
# ============================================================
# Step 2: Run Robo tests
# ============================================================
ROBO_RESULT=0
if [ "$SKIP_ROBO" = false ]; then
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "🤖 Step 2: Running Robo Tests"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""
ROBO_SCRIPT="${SCRIPT_DIR}/run_robo_tests.sh"
if [ "$DRY_RUN" = true ]; then
echo "[DRY-RUN] Would run: $ROBO_SCRIPT --project-id $PROJECT_ID"
else
if [ -f "$ROBO_SCRIPT" ]; then
bash "$ROBO_SCRIPT" --project-id "$PROJECT_ID" || ROBO_RESULT=$?
else
echo "Warning: $ROBO_SCRIPT not found, skipping Robo tests."
ROBO_RESULT=0
fi
fi
echo ""
else
echo "⏭️ Robo tests skipped."
echo ""
fi
# ============================================================
# Step 3: Run instrumentation tests
# ============================================================
INSTR_RESULT=0
if [ "$SKIP_INSTR" = false ]; then
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo "🧪 Step 3: Running Instrumentation Tests"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
echo ""
INSTR_SCRIPT="${SCRIPT_DIR}/run_instrumentation_tests.sh"
if [ "$DRY_RUN" = true ]; then
echo "[DRY-RUN] Would run: $INSTR_SCRIPT --project-id $PROJECT_ID"
else
if [ -f "$INSTR_SCRIPT" ]; then
bash "$INSTR_SCRIPT" --project-id "$PROJECT_ID" || INSTR_RESULT=$?
else
echo "Warning: $INSTR_SCRIPT not found, skipping instrumentation tests."
INSTR_RESULT=0
fi
fi
echo ""
else
echo "⏭️ Instrumentation tests skipped."
echo ""
fi
# ============================================================
# Step 4: Summary
# ============================================================
echo "=========================================="
echo "📊 Test Suite Summary"
echo "=========================================="
echo "Finished at: $(timestamp)"
echo ""
if [ "$SKIP_ROBO" = false ]; then
if [ $ROBO_RESULT -eq 0 ]; then
echo "✅ Robo Tests: PASSED"
else
echo "❌ Robo Tests: FAILED (exit code $ROBO_RESULT)"
fi
fi
if [ "$SKIP_INSTR" = false ]; then
if [ $INSTR_RESULT -eq 0 ]; then
echo "✅ Instrumentation: PASSED"
else
echo "❌ Instrumentation: FAILED (exit code $INSTR_RESULT)"
fi
fi
echo ""
echo "View all results in Firebase Console:"
echo " https://console.firebase.google.com/project/$PROJECT_ID/testlab"
echo ""
# Determine overall exit code
if [ "$SKIP_ROBO" = false ] && [ $ROBO_RESULT -ne 0 ]; then
exit $ROBO_RESULT
fi
if [ "$SKIP_INSTR" = false ] && [ $INSTR_RESULT -ne 0 ]; then
exit $INSTR_RESULT
fi
exit 0

View File

@@ -0,0 +1,239 @@
#!/usr/bin/env bash
# =============================================================================
# Run Robo Tests on Firebase Test Lab
# =============================================================================
# This script runs Robo exploratory tests on Firebase Test Lab across the
# configured device matrix. Robo tests automatically crawl the app UI to
# find crashes and ANRs without requiring instrumented test code.
#
# Prerequisites:
# 1. gcloud CLI installed and authenticated (gcloud auth login)
# 2. Firebase project created and Blaze plan enabled
# 3. Google Play Console linked to Firebase project (optional, for deep links)
# 4. Service account with Firebase Test Lab admin role
#
# Usage:
# ./run_robo_tests.sh [options]
#
# Options:
# --project-id Firebase project ID (default: kordant-android)
# --app-aab Path to app AAB (default: auto-detected)
# --app-apk Path to app APK (default: auto-detected)
# --robo-script Path to robo script JSON (default: robo_script.json)
# --timeout Max robo crawl time in seconds (default: 600)
# --dry-run Print gcloud command without executing
# --help Show this help message
#
# Reference: https://cloud.google.com/sdk/gcloud/reference/firebase/test/android/run
# =============================================================================
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"
# Default values
PROJECT_ID="${FIREBASE_PROJECT_ID:-kordant-android}"
APP_PATH=""
ROBO_SCRIPT="${SCRIPT_DIR}/robo_script.json"
MAX_CRAWL_TIME=600
DRY_RUN=false
# Device matrix from test_matrix_config.yaml (generated via gcloud --device flags)
# Each device runs with each orientation and locale combination
declare -a DEVICE_ARGS=(
# Pixel 6 - Primary target device (API 33)
"--device model=Pixel6,version=33,locale=en_US,orientation=portrait"
"--device model=Pixel6,version=33,locale=en_US,orientation=landscape"
"--device model=Pixel6,version=33,locale=es_ES,orientation=portrait"
"--device model=Pixel6,version=33,locale=es_ES,orientation=landscape"
# Pixel 4 - Older Pixel device (API 30)
"--device model=Pixel4,version=30,locale=en_US,orientation=portrait"
"--device model=Pixel4,version=30,locale=en_US,orientation=landscape"
"--device model=Pixel4,version=30,locale=es_ES,orientation=portrait"
"--device model=Pixel4,version=30,locale=es_ES,orientation=landscape"
# Samsung Galaxy S21 - Popular Samsung device (API 31)
"--device model=GalaxyS21,version=31,locale=en_US,orientation=portrait"
"--device model=GalaxyS21,version=31,locale=en_US,orientation=landscape"
"--device model=GalaxyS21,version=31,locale=es_ES,orientation=portrait"
"--device model=GalaxyS21,version=31,locale=es_ES,orientation=landscape"
# Xiaomi Redmi Note 8 - Budget device (API 29)
"--device model=RedmiNote8,version=29,locale=en_US,orientation=portrait"
"--device model=RedmiNote8,version=29,locale=en_US,orientation=landscape"
"--device model=RedmiNote8,version=29,locale=es_ES,orientation=portrait"
"--device model=RedmiNote8,version=29,locale=es_ES,orientation=landscape"
# Low-end device - Minimum spec target (API 28, 2GB RAM equivalent)
"--device model=AquestM2,version=28,locale=en_US,orientation=portrait"
"--device model=AquestM2,version=28,locale=en_US,orientation=landscape"
"--device model=AquestM2,version=28,locale=es_ES,orientation=portrait"
"--device model=AquestM2,version=28,locale=es_ES,orientation=landscape"
)
# ============================================================
# Helper: Print usage
# ============================================================
usage() {
grep "^#" "$0" | grep -v "^#!/" | sed 's/^# //'
exit 0
}
# ============================================================
# Parse arguments
# ============================================================
while [[ $# -gt 0 ]]; do
case "$1" in
--project-id)
PROJECT_ID="$2"
shift 2
;;
--app-aab)
APP_PATH="$2"
shift 2
;;
--app-apk)
APP_PATH="$2"
shift 2
;;
--robo-script)
ROBO_SCRIPT="$2"
shift 2
;;
--timeout)
MAX_CRAWL_TIME="$2"
shift 2
;;
--dry-run)
DRY_RUN=true
shift
;;
--help)
usage
;;
*)
echo "Unknown option: $1"
echo "Usage: $0 --help"
exit 1
;;
esac
done
# ============================================================
# Auto-detect APK/AAB path if not provided
# ============================================================
if [ -z "$APP_PATH" ]; then
# Prefer AAB for more accurate Play Store representation
APP_PATH=$(find "$PROJECT_DIR/android/app/build/outputs/bundle" -name "*-release.aab" 2>/dev/null | head -1)
# Fall back to APK if AAB not found
if [ -z "$APP_PATH" ]; then
APP_PATH=$(find "$PROJECT_DIR/android/app/build/outputs/apk" -name "*-release.apk" 2>/dev/null | head -1)
fi
# Last resort: any APK
if [ -z "$APP_PATH" ]; then
APP_PATH=$(find "$PROJECT_DIR/android/app/build/outputs/apk" -name "*.apk" 2>/dev/null | head -1)
fi
fi
if [ -z "$APP_PATH" ]; then
echo "Error: Could not find app APK or AAB."
echo ""
echo "Build the app first:"
echo " ./gradlew :app:assembleProdRelease"
echo " # or for AAB:"
echo " ./gradlew :app:bundleProdRelease"
exit 1
fi
if [ ! -f "$ROBO_SCRIPT" ]; then
echo "Warning: Robo script not found at $ROBO_SCRIPT"
echo "Robo will run without guided script (fully autonomous crawl)."
ROBO_SCRIPT=""
fi
# ============================================================
# Determine type flag based on file extension
# ============================================================
if [[ "$APP_PATH" == *.aab ]]; then
TYPE_FLAG="--type robo"
APP_FLAG="--app-package com.kordant.android"
AAB_FLAG="--aab \"$APP_PATH\""
APK_FLAG=""
else
TYPE_FLAG="--type robo"
APP_FLAG=""
AAB_FLAG=""
APK_FLAG="--app \"$APP_PATH\""
fi
# ============================================================
# Print configuration
# ============================================================
echo "=========================================="
echo "Firebase Test Lab - Robo Tests"
echo "=========================================="
echo "Project ID: $PROJECT_ID"
echo "App: $APP_PATH"
echo "Robo Script: ${ROBO_SCRIPT:-none (fully autonomous)}"
echo "Max Crawl Time: ${MAX_CRAWL_TIME}s"
echo "Devices: ${#DEVICE_ARGS[@]} configurations"
echo ""
# ============================================================
# Build gcloud command
# ============================================================
GCLOUD_CMD="gcloud firebase test android run \
$TYPE_FLAG \
--project \"$PROJECT_ID\" \
$APP_FLAG \
$AAB_FLAG \
$APK_FLAG \
--timeout 60m \
--max-crawl-time $MAX_CRAWL_TIME \
--record-video \
--performance-metrics \
--results-history-name \"Kordant Android Robo Tests\""
# Add robo script if available
if [ -n "$ROBO_SCRIPT" ]; then
GCLOUD_CMD="$GCLOUD_CMD --robo-script \"$ROBO_SCRIPT\""
fi
# Add device configurations
for device in "${DEVICE_ARGS[@]}"; do
GCLOUD_CMD="$GCLOUD_CMD $device"
done
echo "Command:"
echo "$GCLOUD_CMD"
echo ""
if [ "$DRY_RUN" = true ]; then
echo "DRY RUN - Command not executed."
exit 0
fi
# ============================================================
# Execute
# ============================================================
echo "Starting Robo tests..."
echo ""
eval "$GCLOUD_CMD"
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ]; then
echo ""
echo "✅ Robo tests completed successfully!"
echo "View results in Firebase Console: https://console.firebase.google.com/project/$PROJECT_ID/testlab"
echo "Review crawl maps, screenshots, and videos for each device."
else
echo ""
echo "❌ Robo tests failed with exit code $EXIT_CODE"
echo "View results in Firebase Console: https://console.firebase.google.com/project/$PROJECT_ID/testlab"
fi
exit $EXIT_CODE