AFT (Automation Framework Team) should not take developers' false catches

What does It mean by "developers' false catches" ?

It means those implementations or application unexpected behaviors which should not be automated with automation framework.

Why those should not be the candidates for automation? It is observed that adding these kinds of requirements in automation may set short ROI for organizations but it never becomes for long terms – You may ask “why” again, below are couple of examples which put more lights on this question.

Case-Study : #1
“Product Team has new requirements in build 1.2.x like objects inputs are changed with prefix or suffix strings and that affects more than 1000 automated test-case(s)”

Due to limited resources and short dead line to certify build(1.2.x) of the product, Product lead comes to AFT and asks for help. He requests to update Automation Framework in such a way that manages prefix and suffix inputs and his team does not have to update their test-case(s). As part of the service team (AFT), we provided solution at framework level. These solutions may help them to certify product but it also carries live wires to take cares are as

1. As per Automation Framework development perspective,
Automation Framework becomes dependent to product builds.
According to Standard Automation Framework protocols, framework should not depend on product; Independent framework can be useful to automate other products as well and such framework would have maintenance and complexity low which helps to give consistent behavior for automation projects.
2. As per Product team perspective,
Product team loses track for test-case(s) on product behavior. Since, Automation Framework injects input data for build 1.2.x ; There becomes no control to update those input data by product team and they lost transparency of test-case(s) with test-data inputs which may raise high effort require to map test-data inputs with testcase(s) per builds for both product team and Automation Framework Team.


Case Study #2
"Product Team has more than 800 test-case(s) automated which works fine with couple of synchronization points (like status as “Processing On Server”). Next iteration, team gets new build and say for “Processing On Server” is broken."

We always face some issues with product to automate. Some are very common when application behavior changed and automation framework has synchronization issues but they concern when they becomes blocker issues for team to certify product. I see lots of organization does not care for this kind of issues since they are not affecting to their core functionality to certify build manually. Because of automation works on synchronizing objects behavior; those issues become “real matters” to resolve them. In one of the experience I have, these issues (broken synchronization points or application unexpected behaviors) made automation in batch executions hanged and crashed the application.

In such condition, product team wants to certify build and request to resolve this kind of issues at automation framework level. As part of solution provider team(AFT), we resolves these issues at automation framework level and helped them to certify build.

But, Why these issues should be resolved at application level not automation framework level? Because

1. As per Automation Framework perspective,
To resolve these kind of issues, automation framework injected with codes to handle broken synchronization points. This becomes un-necessary degrade test-case execution performance and pulls garbage code to automation framework project.
2. As per Product team perspective,
Most of the time, These issues are related to performance of the application and so they should be fixed at application level with priority in QA cycle.

Concluding as …
AFT supports product teams to unblock them to certify builds by giving hot fixes or work-around at framework level but however it is the product teams' responsibility to have priorities to fix those issues (application unexpected behaviors - false catches) at application level and deliver quality product.

After all, we deliver product only; not product with automation framework to handle product’s uncertainties :)

Please, send your precious comments here,

With Regards,
Nimesh - VN.

[Photo: In this Nov. 19, 1978 file photo, Philadelphia Eagles' Herm Edwards (46) pounces on a ball fumbled by New York Giants quarterback Joe Pisarcik (9), right foreground, in the last minutes of the game. Credit: G. Paul Burnett]