There’s more to DevOps than meets the eye. Experts decode all things DevOps, Test Case Automation, and finally, the tools vs tester conundrum.
There’s something about the likes of AI, Machine Learning, Big Data, Bitcoins, or Agile–Buzzwords that are on everyone’s lips, understood by few–and more often than not–used by businesses just to earn brownie points in the SEO department. Recently, #TrundlTalks with Sérgio Freire, Solution Architect & Testing Advocate, Xray and Bruno Conde, Product Manager, Xray shed light on one such buzzword: DevOps. Excerpts follow…
DevOps: Beyond the buzzword
DevOps is essentially a culture, a mindset, and a set of practices and principles that enable product owners, Development QA, IT operations, etc not only to work amongst each other but also ensure that the overall organization succeeds. Best viewed as a holistic approach, DevOps is successfully implemented by working towards a common goal and enables a fast flow of planned working towards production while achieving top-notch stability, reliability, availability and security.
DevOps: The 3 core principles
Experts opine that there are three principles driving DevOps for teams to achieve desired outcomes. These principles are as follows:
- Flow
- Feedback
- Continuous learning & experimentation
Flow: Accelerates the delivery of work from Development to Operations and in the hand of customers.
Feedback: This enables us to create more systems that are safer to work with. Feedback is instrumental in shaping decisions, and having short feedback loops are helpful.
Continuous learning and experimentation: It is important to nurture a culture of trust and a scientific approach to organizational improvement.
To sum up, the idea is to not shy away from the kind of disruption which paves the way for excellence.
The DevOps dilemma: What it is, what it is not
It is also not uncommon to spot resumes in which candidates ‘flaunt’ their skills with a badge of sorts: “I am a DevOps guy”. Contrary to popular notion, DevOps is neither a set of tools nor a role, point experts Bruno and Sérgio.
Another misconception is that DevOps is a process or methodology. It is neither a task nor a set of tasks, and certainly not just about having Developers and Operations people within the same team. The potential of DevOps extends beyond that.
The cornerstone of DevOps lies in the true ownership of the value that is created when some features are built. Sometimes, you see people just building a feature and they move it to the Ops team or to someone that will put it to production. And here, we are talking about the true ownership of the code which demands collaboration within the team and striving for perfection through continuous learning. That also requires one to get things to production in an easy and systematic way.
“It is pivotal to measure what’s happening in production by monitoring at length what is happening there, which also means that we need to have easy ways to roll back, or some way of feature-flagging to easily disable or enable the feature as per preference,” points Sérgio, who feels that although the process in its entirety requires tooling as well as infrastructure tooling, it’s much more than that.
It’s important to note that the purpose of having a tooling system in place is not to make the team feel disconcerted in any way. Everyone should be able to monitor what is happening in production and easily push the code to production–sans any awkwardness.
For example, if you spot a typo somewhere in a certain word–think of it as a bug that can be easily fixed with a simple change of code. This should be as easy as going there in the code, doing a commit, and putting it to production. Knowing what is happening in production should also be straightforward, and is a key component of DevOps.
This can be accomplished with intra-team collaboration as we need to have improved knowledge. “We need to not only continuously improve in tooling, but also how we look at the software,” avers Sérgio.
Test Case Automation:
Blending seamlessly with teams’ ways of working
According to Bruno, Test Case Automation is a key aspect of software teams in general, and specifically Agile teams, because it provides a short feedback loop of problems and everything that happens within the Development life cycle.
For instance, imagine a developer who is building the next big feature in a given product, and is ready to commit and create a pull request with that implementation. If you have your automated tests executing it that moment, and if you detect any problem, you’re able to fix the same immediately. This is how Automated Tests can help within an Agile team if you have automated tests in place–just like regression tests from previous releases or if they are automated tests for this sprint. When you pull the new feature, your automated tests will be executed and you’ll provide you immediate feedback on how your feature will interact with all of the other features in the product. If a bug or problem is detected, you can immediately fix it and bring value to it.
This is important for all kinds of tests, and not just functional tests. Hence, if you find that your feature is degrading the performance because the automated tests are telling you that when you created your pull requests, you can immediately act upon the problem. Therefore, it is important to have automated tests in this context.
Exploratory Testing
Another aspect is Exploratory Testing. Sergio and Bruno talk about test automation as a means to provide valuable feedback, and point that it also freezes the team to perform Exploratory Testing which is crucial to look at the parts that we don’t cover in the automated test. This is instrumental in evaluating the missing elements, thereby gaining time, reducing costs, and leveraging the benefit of performing these more human-based testing crucial to understanding how the product works.
Xray: Nailing the traceability game
Traceability is a critical feature for a Test Management tool and an area where Xray delivers outstandingly. There are reports that tell you where you can go from the user story, and notify about defects for the user story too.
Sample this: How many tests do you have covering the user story? How many executions do you have for this specific version under consideration? How many defects do you have from the executions? Thanks to Xray’s in-built reports, one can instantly leverage this traceability from the requirements to the tests, the execution, defects, and the likes.
Watch webinar on how to use Xray in Jira for test management
Since Xray uses issue types, you can immediately jump from the user story that you’re seeing and become aware if one of the tests is failing. Next, you can go into the failing test, move on to the failed execution and see the defects right there.
Since you are in Jira, viewing the codes is also an option. If you have a failing user story, simply go to the user story, see the code and check its traceability–even for the test case. If it is an automated one, you can also go to Xray and track where the test case is stored and easily track the code.
Traceability apart, Bruno thinks the ‘Requirements status’ provided by Xray also needs to be highlighted. Through the traceability feature, you can see the user story’s connection with test cases, which are in turn, connected with the results and the defects, but Xray provides much more. It can provide you a level of abstraction within the reports.
Suppose, you need to learn how your project is doing for this version in terms of testing, and Xray draws up a list of all the requirements for that specific version and all the statuses for the requirements. For each requirement, you can see which one is okay in terms of testing, and which is not (indicative of some of the tests are failing). With the given report, you can immediately identify the status and if you’re ready for release or not.
Irrespective of test types–Manual, Automated or Exploratory–Xray will calculate all of the statuses for the user stories automatically. You can then analyze the test results for a specific environment, such as a specific browser. This is important as sometimes you can face a problem in one of the browsers or in one of the mobile devices and the traceability in Xray can actually deep dive into that level.
Can tools replace testers?
Tools provide a systematic way to get things done by helping avoid repetitive tasks performed by people, making them pivotal in amplifying existing human skills. Does this trigger the speculation of tools replacing humans? Not in the near future, opine the expert duo. Critical thinking is something that only human beings are endowed with, which means a tester’s unique skills–those that help unravel the unknown and bridge the gaps–make him indispensable.
The tester can collaborate with their Dev & Ops counterparts. A software can add better testability and help with monitoring features to improve imperfections, and this requires true collaboration. A tester, however, can use one of the many observability tools available out there to seek answers to unresolved questions. Additionally, the right kind of insight is also available for later analysis. This is also about easy operability of the software for frictionless deployment, restart, rollback, or even enabling/ disabling the features spoken about earlier.
Tester: Talent, skills and valued qualities
A tester must figure out what is important, and of relevance–objectives that s/he cannot really achieve by him or herself, and demands collaboration with the DevOps team. The tools can be used more prudently and effectively to gain insights about pertinent criteria and discover the true value of what they are delivering.
As per Sérgio, communication is a highly valued skill in testers, besides critical thinking and technical prowess. They can go in deployment scripts or any scripts or tools present, and shouldn’t be hesitant to touch them. They should also try to understand code as much as possible because the more knowledge they possess about the architecture, tools and the code, the better their prospects at exposing the risks.
Know your speakers:
- Sérgio Freire is a Solution Architect and Testing Advocate at Xray. He is a tech enthusiast as well as an ardent advocate of innovation in the field of frameworks, languages, technology, methodologies, and processes. He is responsible for exploring, prototyping, and explaining, and works with different development teams in the organization on their implementation.
- Bruno Conde is the Product Manager at Xray. His association with Xray spans 9 years, and he is currently responsible for bringing the best features to the product.
Know your host:
- Sajit Nair (SAFe® Program Consultant (SPC) | Atlassian Community Leader | Program Manager at Trundl Inc) is an Agile consultant working with technical teams to deliver quality solutions with expertise in Atlassian PM toolsets like JIRA and Confluence.
