What are the most common mistakes when automating tasks?

Some of the common mistakes we make: - Not getting clear understanding of existing manual process. - Which leads to having confusion in choosing the best tool to automate the task. - We need to test the automate process thoroughly for any bugs or issues.

In my experience, I found that not understanding the manual test cases is just one area where automated execution suffers but not understanding requirements and goals of automation is very common and most of the automation testers do not go about it. Secondly, thinking about ROI and setting up management expectations are other aspects. However, not understanding the requirements and goals is more disruptive for the quality of the product under test. Sometimes, I found that tester misses the validations or rather enough validation to cover the requirement. This leads to partial testing. Thirdly the bugs found via automation were not captured due to less focus on script failure analysis and more pressure from delivery.

Not understanding where flexibility is going to be critical. Building a prototype and not scoping time for the necessary iteration so it can actually become the thing you / your team envisioned

The absolute first thing to consider is " isthe task worth automating." Too often we focus on perfecting the process and not solving the problem. Failure to consider the desired outcome results in an automated process that fails faster than what it replaced. What are we trying to do? What factors have changed since we created the process? Are we answering the "so what" question? If we automate the "Thursday Report", is it really a win if no one uses or needs it? By scoring it as a win for automation, we simply derive no business value faster. Leadership must define the "why" of a process and that can change over time. Engineers can solve any problem; executives must ensure we give them problems worth solving.

Getting proper requirements and understanding them is very important. Once you have the requirements one can do some google search ask chatgpt on how to automate the task :- implementation part is easy but first one should understand the requirements very clearly.

"Quis custodiet ipsos custodes?" says Juvenal, in this case "who tests the tests?" Too often the testers will restrict themselves only to the requirements given them and will avoid any audits of a given test's value or utility. This is safe, but of very limited value. It's not just testing, but your test group must view this as a full contact sport, and go in with the attitude that "the sonofabachelor WILL break somewhere, and soon we'll know exactly where and how." This is true for automated tests as well. Just because you have an automated test doesn't mean it's worth the half cent of electricity it takes to run it. Never ever allow yourself to believe your tests are working and adequate. Entropy does not rest. Nor may you.

Testing is very important in automation. - have four environments (development, QA, staging, and production). - promote the scripts and tools to production after thoroughly going through all of the above stages. - Have a QA team. - Set up observability (logging, matrices, events, and tracing).   

In vielen Bereichn sind sehr wohl Bestrebungen und Leitlienien vorhanden zu testen und zu validieren. Diese Bestrebung wird aber oftmals durch faktische terminiliche Situationen oder organisatorische Fakten salop gesagt untergraben.

QA is a vital part of any project. Companies that do not have dedicated QA teams for large projects often struggle getting their projects to fulfill a deadline in a timely manner. Remember, an error caught during dev is cheaper than an error caught in prod by a factor of 10 and sometimes even as much as 100. On my experience, having a reliable QA team ends up saving money despite being a heavy investment.

There are always inputs and considerations that are not included before hand and are only revealed by actually practice running automation scripts/scenarios. To paraphrase Von Moltke: "No test plan ever survives contact with the software."

Leider ist viel Theorie und Praxis aus der Pharmabranche, die in weiten Teilen anderer Branchen noch nicht so ganz angekommen sind. Das korrekte Dokumentieren ist absolut ein wichtiger Bestandteil eines Automation Projekts. Diesen Mehrwert dem Endkunden zu quantifizieren leider oft ein sehr schwieriges Unterfangen. Vor allem wenn man in sehr individuellen Prozessen die sich nicht einfach auf Standards definieren lassen konfrontiert ist. Gute und dokumentierte Arbeit hat seinen Preis, der nicht von jedem Kunden estimiert wird. Es wird da oftmals über den Preis entschieden, what you pay is what you get. Das Bewusstsein ist im kommen aber noch nicht präsent.

Over a period of time even with good vendor documentation, I have observed this is no substitute for personnel performing maintenance to make documentation "improvements" readily available to others, sort of a wiki style but leave a signature at the end of said document. We used to use Palm Tungsten E or Palm TX and render our documents searchable by indexing and word search! This system worked fantastic instead of putting this in Edcoms on computer database. In that case, there was little to no updating.

Every system evolves whether its a COTS or custom solution. Both require automated O&M. Standards like OMG’s IT-CISQ provide a viable approach.

This would be my number one. Too often automation projects concentrate on the technology, but an organization must align people, tools and the processes that tie them together to truly leverage and reap the business benefits of automation. This must be done in the planning stage and not as an after-effect. Properly done, it enhances the business agility that is essential in today's rapidly changing landscape. There is no one-size-fits-all, it depends on the organization's business and culture.

This should be number one or two on the list. Human related issues take many forms: - Inconsistency and bias mean that your data will have a lot of noise. - If you don't educate people on how to use the output of the automation, chances are they won't use it right and say it doesn't work. - If you don't level expectations, people will expect the automation to work perfectly all the time. Which is fine with industrial robots but when you reach the level of advanced computer vision or AI, that is not a viable expectation. - Also, a simple thing but when you monitor how people work to learn to reproduce their work or to benchmark their work alongside the automation... they won't work as they normally do so your data is skewed again.

While building automated processes for my company I spend more time trying to account for the human factor then I ever do in just creating what starts out to look like a simple problem to solve.

Ist immer eine Gratwanderung zwischen Neuerziehung von Mitarbeitern und alte Gewohnheiten belassen. Das Wichtigste ist dabei mit allen Beteiligten eines Projekts über die Bedürfnisse und auch über die aktuelle Handhabung zu kommunizieren. "Wir haben das schon immer so gemacht." Kenne ich zur genüge. Und ja auch respektieren, dass die Mitarbeiter langjährige Erfahrung mit der Anlange haben. Lösungen gemeinsam suchen und nicht ein Neues tolles Konzept auf's Auge drücken. Das kann sich nur Microsoft leisten.

The most common mistake done in automated testing is not considering the use cases in the perspective of the user. Most of the test conditions are done at sub system level and not at integrated system level, which misses those realtime corner cases.

Without lifecycle cost and risk models, no major project should initiate. This is the 2nd most critical step. Function Point Analysis is a key tool for this phase.

One mistake would be to ignore the energy, cost and time required to implement such a system. Automation is a project in itself that must be specified carefully

Automation of tasks should enhance the productivity and efficiency and not just merely shift from the old task and create another equally inefficient unproductive task.

It's crucial to regularly review and update the cost-benefit analysis as the project progresses. For instance, if you're automating an inventory management system, continually assessing the automation's impact on inventory accuracy, restocking efficiency, and overall supply chain management is essential. Comparing these metrics against the initial projections can reveal whether the automation is delivering the expected value or if adjustments are needed. Additionally, staying open to alternative methods and hybrid solutions can provide a balanced approach, ensuring that automation enhances rather than detracts from overall system performance and business goals.

Failure to have a human system integration assessment of the task that is to be automated may spell trouble. The important ones are safety and human factors engineering. Engineers can design the best software and hardware in an environment but fail to factor how the human is part of the system. Automation is not just turn on a switch and watch the machine go.

I've worked in environments where automation can become recursive. A product is made and needs to be validated. The validation process is automated. However, as the scope of the product increases (i.e., increased use-case permutations), the (automated) validation burden also increases. More features are then added to the automated validation system. However, this increases the validation burden which will eventually spawn automated validation. This is a consequence of forward progress. However, it should be factored into project plans. Fundamental design processes (e.g., requirements, cost-benefits) become much more important, so as to not distract from core activities. Done correctly, automation enables progress.

It is important to remember that throwing money at common issues is often seen as an option. However, do understand the value of fighting an inconvenience with money is only as effective as the product being created. Remember to always look at automative solutions with a discerning eye when it comes to understanding the scope and the hidden prices it comes with when deploying projects and plan ahead for spending 1/2 of the project cost on maintenance.

If a process is not good, clear, or efficient, automation will merely make it faster, not better. A robust evaluation must be made of the process to maximize the benefit of the process transition.

While automated tests are an identified requirement, top management don't always understand why it's a big requirement for the entreprise. Mostly when you report costs, human resources, needed time, documentation etc... Therefore it's important to give insights to them