Clever Geek Handbook

Upgrade Android project with GitHub Actions. Part 2

Running UI Tests on GitHub Actions

We continue to deal with the automation of an Android project on GitHub Actions, in this part:





  • Let's start a new project for UI tests in Firebase Test Lab





  • Set up the integration of GitHub Actions and Test Lab





  • Let's see how you can run UI tests in a workflow on CI / CD.









If you missed the first part of the story, which dealt with Unit tests in an Android project, you can start with it.





To run unit tests, we just need to have a configured Java environment. All tests pass very quickly inside the JVM, everything is simple, and the situation of flacky tests is almost excluded. Such tests should be 70-80% of the total number of tests in the project, first of all it is worth covering the business logic with them.





But sometimes you want more - tests that will simulate the actions of real users and compare expectations with reality.





UI- . -, . , . , UI- , , /- . , - - . , .





- Firebase Test Lab





Firebase Test Lab - Google, . 10 5 . 1$ - , .





Test Lab :





  1. checkout  Java-





  2. unit-. , UI-.





  3. Gradle- UI-





  4. APK-, .





  5. Firebase Test Lab .





  6. command line gcloud,   Test Lab APK.





  7. , , workflow GitHub Actions.





Firebase GitHub.





 https://console.firebase.google.com  Google-.





, .









Google- , Test Lab . , .





, GitHub Actions Test Lab. 





“ ” (“Project settings”), “ ” (“Service accounts”). “ ” (“Manage service account permissions”).





, CI/CD GitHub Actions. UI- “”. .









“”. , Firebase 403.





ERROR: (gcloud.firebase.test.android.run) Unable to access the test environment catalog: ResponseError 403: Not authorized for project ***

“”





CI/CD . “ ” (“Create key”).





JSON, . , , - . private_key.





, JSON . Base64.





:





1) 





base64 github-actions-sample-key.json > base64-key.txt

github-actions-sample-key.json - JSON, base64-key – , .





2)  https://www.base64encode.org/





GitHub Secrets GitHub.





Firebase base64-key.





Project ID Firebase. Project number.





, GitHub Actions Test Lab. workflow giithub/workflows.





workflow, UI- Test Lab .





ERROR: (gcloud.firebase.test.android.run) User [github-actions-ci-cd@***.iam.gserviceaccount.com] does not have permission to access project [***:initializeSettings] (or it may not exist): Cloud Tool Results API has not been used in project 254361894337 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/toolresults.googleapis.com/overview?project=254361894337 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

- API toolresults.googleapis.com, , API . “Enable APIs and services”. 





API . Cloud Tool Result API .





- , workflow.





name: UI_tests_on_release
 
on:
  pull_request:
    branches:
      - 'main'
 
jobs:
  assemble_ui_test_artifacts:
    if: startsWith(github.head_ref, 'release/') == true
    name: Build artifacts
    runs-on: ubuntu-20.04
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-java@v1
        with: {java-version: 1.8}
 
      - name: Build APK for UI test after Unit tests
        run: |
          ./gradlew test
          ./gradlew assembleDebug
          ./gradlew assembleDebugAndroidTest
 
      - name: Upload app-debug APK
        uses: actions/upload-artifact@v2
        with:
          name: app-debug
          path: app/build/outputs/apk/debug/app-debug.apk
 
      - name: Upload app-debug-androidTest APK
        uses: actions/upload-artifact@v2
        with:
          name: app-debug-androidTest
          path: app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk
 
  run_ui_tests_on_firebase:
    runs-on: ubuntu-20.04
    needs: assemble_ui_test_artifacts
    steps:
      - uses: actions/checkout@v2
      - name: Download app-debug APK
        uses: actions/download-artifact@v1
        with:
          name: app-debug
 
      - name: Download app-debug-androidTest APK
        uses: actions/download-artifact@v1
        with:
          name: app-debug-androidTest
 
      - name: Firebase auth with gcloud
        uses: google-github-actions/setup-gcloud@master
        with:
          version: '290.0.1'
          service_account_key: ${{ secrets.FIREBASE_KEY }}
          project_id: ${{ secrets.FIREBASE_PROJECT_ID }}
 
      - name: Run Instrumentation Tests in Firebase Test Lab
        run: |
          gcloud firebase test android models list
          gcloud firebase test android run --type instrumentation --use-orchestrator --app app-debug/app-debug.apk --test app-debug-androidTest/app-debug-androidTest.apk --device model=Pixel2,version=28,locale=en,orientation=portrait





,





1





name: UI_tests_on_release
 
on:
  pull_request:
    branches:
      - 'main'
 
jobs:
  assemble_ui_test_artifacts:
    if: startsWith(github.head_ref, 'release/') == true
    name: Build artifacts
    runs-on: ubuntu-20.04
    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-java@v1
        with: {java-version: 1.8}

workflow, . Pull request main , release/.





checkout Java 8.





2





- name: Build APK for UI test after Unit tests
  run: |
    ./gradlew test
    ./gradlew assembleDebug
    ./gradlew assembleDebugAndroidTest
 
- name: Upload app-debug APK
  uses: actions/upload-artifact@v2
  with:
    name: app-debug
    path: app/build/outputs/apk/debug/app-debug.apk
 
- name: Upload app-debug-androidTest APK
  uses: actions/upload-artifact@v2
  with:
    name: app-debug-androidTest
    path: app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk

unit- APK - app-debug.apk app-debug-androidTest.apk. ? APK - , APK instrumentation-, .





upload-artifact@v2.





, APK , .





3





run_ui_tests_on_firebase:
  runs-on: ubuntu-20.04
  needs: assemble_ui_test_artifacts
  steps:
    - uses: actions/checkout@v2
    - name: Download app-debug APK
      uses: actions/download-artifact@v1
      with:
        name: app-debug
 
    - name: Download app-debug-androidTest APK
      uses: actions/download-artifact@v1
      with:
        name: app-debug-androidTest

Job workflow (assemble_ui_test_artifacts), , .





.





needs: assemble_ui_test_artifacts

action download-artifact@v1 APK Job.





4





- Test Lab .





- name: Firebase auth with gcloud
  uses: google-github-actions/setup-gcloud@master
  with:
    version: '290.0.1'
    service_account_key: ${{ secrets.FIREBASE_KEY }}
    project_id: ${{ secrets.FIREBASE_PROJECT_ID }}
 
- name: Run Instrumentation Tests in Firebase Test Lab
  run: |
    gcloud firebase test android models list
    gcloud firebase test android run --type instrumentation --use-orchestrator --app app-debug/app-debug.apk --test app-debug-androidTest/app-debug-androidTest.apk --device model=Pixel2,version=28,locale=en,orientation=portrait

action setup-gcloud, ID Base64 , , .





.





 gcloud firebase test android models list  SDK. , .





, , , . .





workflow , .





, , UI- ! Test Lab . Project ID ( ID) .









orchestrator, --use-orchestrator.





UI- , . 





--num-flaky-test-attempts - Flaky .





--network-profile - . , .





:





https://cloud.google.com/sdk/gcloud/reference/firebase/test/android/run





 





https://firebase.google.com/docs/test-lab/android/instrumentation-test#sharding





https://github.com/Flank/flank





- :





1) UI- Test Lab , MacOS.  https://github.com/ReactiveCircus/android-emulator-runner





2) UI-. Android SDK .











UI- GitHub Actions :)







More articles:


All Articles