Automation to test IoT systems across tiers

This https://www.statista.com/statistics/471264/iot-number-of-connected-devices-worldwide/ page estimates the number of internet-connected devices worldwide from 2015 to 2025.

Suppose you’re working on an IoT watch. Wouldn’t it be great to have automation to verify a functional requirement that crosses tiers with just one check?

For example, a functional requirement could be that a notification on your phone can be dismissed on the watch, and the phone displays correctly that it was dismissed.

Suppose a dev is preparing to check in a code change set that touches one or both of these tiers. You need to verify, before the change set is committed so that others are affected by changes, that the notification requirement is still met.

Clearly, you need automation, and automation that crosses tiers, at least in simulation.

Can you do this?

It turns out, that using structured data, e.g., XML, works great for this because the different tiers are nicely presented in the structure.

This http://www.metaautomation.net/metaautomation-samples page links to samples on GitHub that demonstrate how an ordered-tree hierarchy (the Hierarchical Steps pattern of MetaAutomation) enables automation to pay attention and make data available to further automation.

Sample 3 http://www.metaautomation.net/metaautomation-samples#sample3 shows how this can be done across any number of tiers. The check verifies a functional requirement, and the artifact of the check run is an XML document with a supplied grammar. Any tier can drive the SUT on that tier to manipulate and gather data, and all that data is stored and available to further quality automation or to be transformed into a nice, browsable view of what the SUT is doing.

If this is not done, the above requirement could be broken for days at least until somebody notices it, which breaks people’s experience with the product (at least, on the team developing and dogfooding it) and perhaps hiding other quality issues as well. The further from the breaking change that the break is discovered, the harder it is to fix it and the costlier the break is to the team.

Speed is important, as is cross-tier capability. MetaAutomation shows how.

No Comments

Add a Comment

Sign up here for emails with bites of wisdom on quality automation and MetaAutomation

Recent Blog Posts

  • The differences: Manual Test vs. quality automation

    In my last post I describe out the two kinds of automation that fit in the quality automation space.

    People who do quality automation (at least, the part of quality automation that drives and … more

  • The two halves of quality automation

    Quality automation is the domain (or problem space) of driving the SUT, measuring and recording data on SUT behavior and communicating that data to the business. I also use “quality automation” to … more

  • Fixing the false negative problem

    False negatives happen when these three things happen in order:

    Operations (ops) promotes the software to the next level, or ships it to end-users

    Someone (or, some automated process) discovers a … more

  • Fixing the false positive problem

    With all the quality automation that is your responsibility, a run of a check failed. It is your job to check it out.

    After 30 minutes or so of investigation, you find that the failure happened … more