One of the main contributors of this project[0], was the core contributor (creator?) of Puppeteer[1], but then I guess left Google to join Microsoft and work on this[2][3].
[0] - https://github.com/aslushnikov
[1] - https://github.com/puppeteer/puppeteer/
[2] - https://github.com/microsoft/playwright/graphs/contributors
[3] - https://github.com/microsoft/playwright/graphs/contributors
- Cypress: https://www.cypress.io
- Playwright aws lambda: https://github.com/PauloGoncalvesBH/running-playwright-on-aw...
- Checkly: https://www.checklyhq.com
Meanwhile Playwright's API is just promises. Everybody knows how to compose promises.
The Cypress approach does in theory completely eliminate latency-induced flakiness, but in my experience, Playwright/Puppeteer's usage of bi-directional real-time protocols to communicate with the browser makes latency low enough that latency-induced flakiness is no longer a practical concern, especially when paired with the ability to selectively run test logic in client-side JS in the page with 0 latency.
Selenium did suffer from latency-induced flakes all the time due to its slow uni-directional request/response model. I personally believe the Cypress model is an over-correction for Selenium PTSD, and isn't making as good a set of tradeoffs compared to Playwright/Puppeteer.
Having said that, I would love to support Cypress if I had infinite time and focus.
Side note: Selenium is not on the menu, even with its large install base.
We are aiming for where the puck is going and it’s going to Playwright.
Full disclaimer: I’m CTO at Checkly.
https://blog.checklyhq.com/cypress-vs-selenium-vs-playwright...
Cypress vs Selenium vs Playwright vs Puppeteer speed comparison
doesn't playwright use puppeteer for chromium-based browsers(even edge-based), I thought it's just a wrapper for puppeteer with extra support for firefox.
I switched to Playwright 6 months ago - I'm much happier now and the switch also allowed me to delete 3/4 of the helper code that I needed before.
https://docs.cypress.io/guides/references/trade-offs#Permane...
- Playwright is more lightweight. Can be good or bad on what you expect. But I definitely prefer Playwright.
I don't know if it's Selenium specifically or some problem with the .NET binding, but I figure Microsoft must have better .NET integration so it will at least eliminate that possible source of problems.
- Hard-coded waits in your code, like "Thread.sleep(1000)". A better alternative is to replace hard-coded waits with something that waits for an element or value to appear on the page. i.e. click on a button and wait for a 'Success' message to appear. Puppeteer and Playwright both have good constructs for doing this.
- Needless complexity in the tests. Conditionals in particular are a code-smell and indicate there's something needlessly complex about the test.
- No test data management strategy. The more assumptions you can make about the state of your application, the simpler your tests become. Ideally tests are running in an environment that nothing else is touching, and you're seeding data into that environment before tests run. I personally don't believe in mocking data in regression tests since that quickly becomes hard to manage.
We spend a lot of time thinking about these issues at my company and wrote a guide that covers other common regression testing issues in more detail here: https://reflect.run/regression-testing-guide/
We don't do any timed waits, all of our waits are for an element or value to appear, but these waits never complete sometimes, non-deterministically. We then added a long 5 min timeout on these waits because we know the test will never complete at that point. It's always fine in manual testing though, and if we don't run the browser in headless mode and watch it work. Very frustrating.
Sometimes the HTTP requests themselves timeout after a few minutes, but this never happens in manual testing either. That's actually the most common issue these days, and this happens non-deterministically too. This is what I meant by "flaky".
// Expect an element "to be visible".
await expect(page.locator('text=Learn more').first()).toBeVisible();
> I don't know if it's Selenium specifically or some problem with the .NET binding
If the execution in .NET is slow then I suppose it could be .NET. But it could be (and often is) the suite design. You must wait for /everything/ before interacting with it because the code execution is quicker than the page.
Large Se/Webdriver suites are often a PIA. I find it's nice to write them with Python or Ruby so they can be debugged interactively with the an interactive shell.
That's what I do, but the wait for an element in certain tests times out after a few minutes, even though the elements are clearly visible, and manual use never has an issue.
From other comments it sounds like Puppeteer and Playwright are better on this, so will look into switching.
With great care and effort you can make your tests reliable (especially if you are happy to allow a "best of 3" type test strategy to allow for 1 flake and 2 passes) though. Prodigious use of the wait (i.e. stdlib polling) primitives seems to give you the most bang for your buck.
I am note sure if this is just the nature of web automation, or if selenium is just crap? My gut is to say it is selenium's fault since we never get the same issues when using javascript in the DOM or in an extension)...maybe it is the browser APIs I guess? U have no idea but if this playwright is any better than that would be superb.
We have an AI model that understands the page structure as humans do. So we can do this "Click on 'Sign in' on the 'Login' page".
We have a no-code tool as well which adapts to the changes. But we want to generate the code for people who want to keep things internally.
Would love to discuss it more: m@preflight.com Our website: https://preflight.com
In Playwright that manifests itself as the "auto-wait" feature: https://playwright.dev/docs/actionability
You can do this kind of thing with Selenium too but it wasn't designed in from the very start of that project.
You can also give the browser a virtual clock so that you can use time based timeouts and give every test a timeout of 3 hours, but those 3 hours only take milliseconds in real time. That approach gets CPU expensive if your site has any background polling scripts or animation, because obviously the animation will end up animating a lot during the test!
Yes, this is what I've done but the elements non-deterministically do or do not appear according to selenium, and then the wait times out. This happens for dozens of tests across dozens of pages with no issue with manual use, so something fishy is going on.
No-code is the best in my opinion :D https://preflight.com
We have done all the ground work. Like: - Concurrency - Adapt to the changes. Our selectors are like this: "Click on 'Login' button in the 'Sign in' form" - Update the tests with an HTML/Video player etc
I just want to plug Playwright by Microsoft as I've been using it over the past month and have had a really great experience with it: https://playwright.dev It's built by the founders of Puppeteer which came out of the Chrome team. Some things I like about it:
1. It's reliable and implements auto-waiting as described in the article. You can use modern async/await syntax and it ensures elements are a) attached to the DOM, visible, stable (not animating), can receive events, and are enabled: https://playwright.dev/docs/actionability
2. It's fast — It creates multiple processes and runs tests in parallel, unlike e.g. Cypress.
3. It's cross-browser — supports Chrome, Safari, and Firefox out-of-the-box. 4. The tracing tools are incredible, you can step through the entire test execution and get a live DOM that you can inspect with your browser's existing developer tools, see all console.logs, etc...
5. The developers and community are incredibly responsive. This is one of the biggest ones — issues are quickly responded to and addressed often by the founders, pull requests are welcomed and Slack is highly active and respectful.
My prior experience with end-to-end tests was that they were highly buggy and unreliable and so Playwright was a welcome surprise and inspired me to fully test all the variations of our checkout flow.
I've seen some sites that behave differently when the browser is being automated. E.g., if I access fanfiction.net from a browser being automated with Selenium it gets stuck in an endless Cloudflare CAPTCHA loop. Accordingly I've come to prefer automation methods that are less revealing to the site.
We’ve seen a consistent growth of interest in people wanting to use Playwright for browser automation (and testing).
Pros of Playwright/Puppeteer:
Reuse existing HTML/CSS knowledge
Cons: Requires an external service or shelling out to an external process
Pros of using a pdf lib:
Probably better performance, simpler architecture by being in-process.
Cons: Ad-hoc language for designing the PDF.
I used one based on docker, and the bottleneck was actually sending the html, css you want to print (if it's not already served over http). I used a shared docker volume to write to from one process (python) and read from another (the node pupetter).
It all comes down to, load html, wait to load, save to pdf. Very simple, fast, and reliable. More so than weasyprint for example.
There's so much potential to use playwright in CI/CD with GitHub Actions cron jobs. Really enjoying it so far.
We use TestCafe at work for this purpose. I personally hate TestCafe as it's is an absurd unfocused mess of a browser remote, but it lets me control my browser by navigating to a URL which no other browser remote system does.
test.use({
...devices['iPhone 13 Pro'],
locale: 'en-US',
geolocation: { longitude: 12.492507, latitude: 41.889938 },
permissions: ['geolocation'],
})https://github.com/microsoft/playwright/blob/0d277fa589e9508...
edit: ah puppeteer was a Google project. I forgot
Would love to help any of your testing needs
All of the above are uploaded automatically as Github artefact as part of auto-generated Github Action!
Br
The freemium service provides access to compute-heavy machine learning models running on GPUs.
Hackers blast 50-100 requests in the same second, which clog the servers and block legitimate users.
We reported IPs to AWS and use Cloudflare "Super Bot Fight Mode" to thwart attacks, but the hackers still break through.
We don't require accounts, but could impose account requirements if this helps.
Any suggestions?
The one event that cannot be automated is cursor movement/position. Put a check into your event handlers that check that the cursor is actually over the event target.
Are you saying block form submission unless the cursor is over the event target?
If so:
* How to handle legitimate requests from mobile users?
* How to handle form submissions with the "return" key?
Are there proxy examples you could point us to?
Thanks for your help.
The freemium service provides access to machine learning models on GPU instances, served with FastAPI.
Each request invokes a compute-intensive ML model, but perhaps there is something wrong with the FastAPI configuration as well?
It is possible, but this degrades the experience for legitimate users.
We prefer solving this without impacting/taxing normal users if possible.
It is possible, but this degrades the experience for legitimate users.
We prefer solving this without impacting/taxing normal users if possible.