88 lines
2.6 KiB
Markdown
88 lines
2.6 KiB
Markdown
# 20. Firebase Test Lab Integration
|
|
|
|
meta:
|
|
id: android-production-20
|
|
feature: android-production
|
|
priority: P2
|
|
depends_on: []
|
|
tags: [testing, device-farm, quality]
|
|
|
|
objective:
|
|
- Integrate Firebase Test Lab for automated testing on a wide range of physical Android devices
|
|
|
|
deliverables:
|
|
- Firebase Test Lab configuration
|
|
- Robo test configuration
|
|
- Instrumentation test execution
|
|
- Test results and reports
|
|
|
|
steps:
|
|
1. Set up Firebase Test Lab:
|
|
- Create Firebase project
|
|
- Link to Google Play Console
|
|
- Enable Blaze plan (pay-as-you-go)
|
|
- Install gcloud CLI
|
|
2. Configure test matrix:
|
|
- Select device models:
|
|
- Pixel 6 (API 33)
|
|
- Pixel 4 (API 30)
|
|
- Samsung Galaxy S21 (API 31)
|
|
- Xiaomi Redmi (API 29)
|
|
- Low-end device (API 28, 2GB RAM)
|
|
- Select orientations: portrait, landscape
|
|
- Select locales: en_US, es_ES
|
|
3. Run Robo tests:
|
|
- Upload APK/AAB
|
|
- Configure Robo crawl (timeout, login credentials)
|
|
- Run on device matrix
|
|
- Review crawl map and screenshots
|
|
4. Run instrumentation tests:
|
|
- Upload test APK
|
|
- Configure test runner
|
|
- Execute UI tests on device matrix
|
|
- Collect test results and videos
|
|
5. Integrate with CI:
|
|
- Add gcloud commands to GitHub Actions
|
|
- Run on release builds
|
|
- Block release on critical failures
|
|
- Upload results as artifacts
|
|
6. Analyze results:
|
|
- Review screenshots from all devices
|
|
- Watch test videos for failures
|
|
- Check performance metrics
|
|
- Document device-specific issues
|
|
7. Fix issues:
|
|
- Address failures on specific devices
|
|
- Fix OEM-specific bugs
|
|
- Optimize for low-end devices
|
|
|
|
tests:
|
|
- Device: All selected devices pass tests
|
|
- Regression: No new failures compared to previous run
|
|
- Performance: App responsive on all devices
|
|
|
|
acceptance_criteria:
|
|
- Firebase Test Lab project configured
|
|
- Test matrix with 5+ different devices
|
|
- Robo tests running on each release build
|
|
- Instrumentation tests running on device matrix
|
|
- CI integration for automated execution
|
|
- Test results archived
|
|
- Screenshots and videos from all test runs
|
|
- Device-specific issues identified and fixed
|
|
- No crashes on any tested device
|
|
- Performance acceptable on low-end device
|
|
|
|
validation:
|
|
- Run tests via gcloud → all devices complete
|
|
- Review results → screenshots show correct UI
|
|
- Watch videos → no ANRs or crashes
|
|
- Check performance metrics → within budget
|
|
- CI pipeline → Test Lab results green
|
|
|
|
notes:
|
|
- Firebase Test Lab is free for first 100 device-minutes/day (physical)
|
|
- Robo tests are great for exploratory testing without writing tests
|
|
- Physical devices are more reliable than virtual devices
|
|
- Test on low-end devices to catch performance issues
|