imterah/element-portable
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add test reporter to prevent stale screenshots (#12743)

Browse source 
* Split up slow Playwright tests

To optimise parallelism

Deals with:

```
Slow test file: read-receipts/redactions.spec.ts (5.4m)
Slow test file: read-receipts/new-messages.spec.ts (3.9m)
Slow test file: read-receipts/high-level.spec.ts (3.6m)
Slow test file: read-receipts/editing-messages.spec.ts (3.1m)
Slow test file: read-receipts/reactions.spec.ts (2.2m)
Slow test file: crypto/crypto.spec.ts (2.4m)
Slow test file: settings/appearance-user-settings-tab/appearance-user-settings-tab.spec.ts (1.2m)
Slow test file: composer/composer.spec.ts (1.1m)
Slow test file: crypto/verification.spec.ts (1.1m)
```

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Move around snapshots

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Add test reporter to prevent stale screenshots

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix test

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove darwin screenshots which should not have been checked in

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Fix absolute vs relative path mismatch

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Revert "Remove darwin screenshots which should not have been checked in"

This reverts commit 1e189977fa9ec873339fc02b2b231a314809b2d5.

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Revert "Revert "Remove darwin screenshots which should not have been checked in""

This reverts commit 5144b9b28e31ca543b2c5d02820c3f957dbd8c04.

* Iterate

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove stale screenshots

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Revert "Remove stale screenshots"

This reverts commit 9beae9974557c1ffa99c2372da280bb0da407bd1.

* Apply same sanitization as Playwright for file name consistency

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* add dev dep

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Remove stale screenshots

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>

* Discard changes to playwright/flaky-reporter.ts

* Update end-to-end-tests.yaml

---------

Signed-off-by: Michael Telatynski <7t3chguy@gmail.com>
This commit is contained in:
Michael Telatynski 2024-07-15 19:23:20 +01:00 • committed by GitHub
parent 7863de653a
commit e6d9eccf1b
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
21 changed files with 131 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

5
.github/workflows/end-to-end-tests.yaml vendored
View file

@ -190,13 +190,14 @@ jobs:
- name: Merge into HTML Report
if: inputs.skip != true
run: yarn playwright merge-reports --reporter=html,./playwright/flaky-reporter.ts ./all-blob-reports
run: yarn playwright merge-reports --reporter=html,./playwright/flaky-reporter.ts,./playwright/stale-screenshot-reporter.ts ./all-blob-reports
env:
# Only pass creds to the flaky-reporter on main branch runs
GITHUB_TOKEN: ${{ github.ref_name == 'develop' && secrets.ELEMENT_BOT_TOKEN || '' }}
# Upload the HTML report even if one of our reporters fails, this can happen when stale screenshots are detected
- name: Upload HTML report
if: inputs.skip != true
if: always() && inputs.skip != true
uses: actions/upload-artifact@v4
with:
name: html-report