End-to-end testing for a modern IoT device

The eternal question in the mind of a test engineer is, 'Why didn't they start testing earlier?' It is always a good idea to take testing issues into consideration from the beginning of a project. Proper planning significantly reduces the risk of having to return to the drawing board at a later stage, which saves money and enables a faster time to market.

"Testing should be at the core of the new product development process but is often not considered a high enough priority”, says Markku Konttajärvi, Senior Project Manager at Etteplan, who has worked in the testing field since 2004. "I have seen projects where a year or more is invested in designing a new product without properly testing the prototypes. When the projects finally move into the testing phase, it quickly becomes evident that the product can't fulfil the requirements, and they have to take many big steps back."

There should also be a logic and coherence to the testing process. Each product development project is different in terms of its aims, features and the technologies used, and this should also be reflected in the testing requirements. It is important to plan what needs to be tested and at what stage.

Etteplan's internal development project focused on developing an IoT prototype device that used various location technologies, cloud services, and connectivity technologies.

This is the framework that we used to plan the testing for the new device:

We asked Markku to go through the plan one stage at a time and explain why some other tests were not included in this particular project.Device evaluation with evaluation kits. "We did some initial prototyping using the evaluation kits and platforms from the module or component manufacturers. These platforms help in the early stage software development and support decision-making for the implementation phase, including possible technology choices."

Software-related unit testing. "Here, the units or components of the software code were reviewed and tested so that they perform as expected."

PCB simulation. "The PCB simulation phase included Resonant mode analysis, Cross talk and DC-drop-analysis. Based on these simulations we made some modifications to the layout design. For example, antenna simulations weren’t done because there were no specific frequency requirements.”

PCB wake-up testing. "The focus was on verifying the basic electronic functionality of the printed circuit boards.”

Functional testing. "We tested the hardware and software interaction, making sure that the data flowed correctly between the PCBs and externally so that the device can be controlled and the modules function properly."

System testing. "The scope of the testing widened significantly at this stage. The device was tested as part of a bigger system that included the positioning base station, cloud services, and mobile phone network. We made sure that the device could be configured and controlled remotely and that it communicated accurately with the other devices."

Field verification. "This is where we finally tested the positioning accuracy of the device in real-world conditions. We also discovered that the prototype's casing let in rainwater when it was incorrectly installed on the roof of a car. This resulted in better installation instructions on the device."

Preliminary pre-compliance testing, Electro-magnetic (EMC) testing. "The device was tested against different EMC measurements to ensure that it would comply with the relevant regulations. It's also worth noting that at Etteplan, we also do quality assurance in the form of reviews throughout every project."

The goal of our development project was clearly defined: to create a technology demo that could be tailored to the needs of specific customers. Here are Markku's comments on the tests that we decided to leave out and the reasons why they were not required:

Production testing. "We only produced 15 prototype devices, so doing large-scale, automated production testing was not relevant. PCB wake-up testing was considered to be enough of production testing for these prototypes." 

Antenna testing. "We decided to use ready-made antennas in this project, and we consulted Etteplan's antenna specialist to make sure that they were positioned correctly."

Environmental testing. "In this project, no requirements were set related to standards, so we did not undertake environmental testing."

Customer acceptance testing. "In this project the customer did their own verifications after system delivery."

From meticulous planning to rigorous implementation, Etteplan has the experience, competence, and modern facilities to deliver a comprehensive range of testing services.

"We are confident that our wide SW and HW testing expertise and compliance testing can fulfil all the end-to-end system testing and quality assurance tasks required for an advanced IoT device. Companies that start the testing process at an early stage save time and money, and often end up with a higher quality final product," says Markku Konttajärvi.