API Strategy: Testing

Figure 1: API Testing Pyramid
  • Unit Testing
  • Mutation Testing
  • Contract Validation
  • Functional Testing
  • Consumer-driven Testing
  • Specification Compliance

Mutation Testing

Figure 2: Mutation Testing Flow
Figure 3: Why unit test coverage is not good enough
Figure 4: Pitest — An open source mutation testing framework

Contract Validation

Figure 5: API Contract Validation mechanism
Figure 6: DREDD — An API Contract validation tool
Figure 7: Sample Contract validation scenario using DREDD

Consumer-driven Testing

Figure 8: Consumer-driven testing
  • Forces you to think about API backwards compatibility
  • Let’s you evolve APIs independently, giving more autonomy to teams
  • Ensures that you can deploy microservices independently
  • Low-cost integration tests
  • Gives real examples of how your services interact
  • Allows you to visualize the relationships between services
  • Informs you which versions of your applications can be deployed safely together
Figure 9: Consumer-driven testing using PACTFLOW
  1. API producer and consumer teams talk to each other and agree on consumer-driven testing using a tool like PACTFLOW. The consumer team writes the consumer test and generates the PACT file. Unit testing is part of testing the expected functionality of an API.
  2. The consumer team publishes the PACT file via a pact broker client (API or CLI).
  3. The PACT file is stored on the pact broker server and is available to the provider team.
  4. The provider team implements/modifies the API as per contract documented in the PACT file. The unit tests are designed to make sure the expected behavior is equal to the actual behavior.
  5. The provider team sends verification to the pact broker.
  6. The broker marks the pact as VERIFIED by annotating the PACT file.
Figure 10: Example artifacts involved in the “pactPublish” phase of consumer-driven testing
Figure 11: Screenshot showing the status of a PACT after “pactPublish” is successful
Figure 12: Example artifacts involved in the “verifyPact” phase of consumer-driven testing
Figure 13: Screenshot showing the status of a PACT post “verifyPact” step

Specification Compliance

Figure 14: Pipeline based API specification compliance

References:

  1. Muneeb S. Ahmad, “Major Disciplines of an API Program”, Medium, 2022 [Online]. Available: https://muneebsa.medium.com/major-disciplines-of-an-api-program-2ad0d7c8f183
  2. Goran Petrovic, “Mutation Testing”, Google Testing Blog, 2021 [Online]. Available: https://testing.googleblog.com/2021/04/mutation-testing.html
  3. Alexander S. Gillis, “mutation testing”, TechTarget, 2019 [Online]. Available: https://www.techtarget.com/searchitoperations/definition/mutation-testing
  4. Nathan Thompson, “Mutation Testing: A Tale of Two Suites”, Etsy Engineering [Online]. Available: https://www.etsy.com/codeascraft/mutation-testing-a-tale-of-two-suites/
  5. Apiaryio, “DREDD”, Github, 2022 [Online]. Available: https://github.com/apiaryio/dredd
  6. Pact Foundation, “Effective Pact Setup Guide”, GitHub, 2022 [online]. Available: https://github.com/pact-foundation/pact.io/tree/documentation/pact_nirvana

--

--

--

Muneeb Ahmad is Microservices Architect at IBM pursuing passion in ABC (Ai, Blockchain, Cloud)

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

MicroStrategy Education Newsletter | July 2021

Python program to compute the GCD of two numbers.

Simple Docker Based Virtual Private Server (Notes to myself)

Top 5 marketplace platforms: Pick the best one to build a marketplace website

Top 5 marketplace platforms to pick for eCommerce website

Digital transformation — Tackling complexity in changing software landscape

Gear Monthly Updates: November

Poco officially launches NFT marketplace and partnership announcement with Babylons

Do You Know the Tight and Loose Coupling in the Oop?

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Muneeb S. Ahmad

Muneeb S. Ahmad

Muneeb Ahmad is Microservices Architect at IBM pursuing passion in ABC (Ai, Blockchain, Cloud)

More from Medium

Migrations In Entity Framework Core

Automatically document and deploy your API with SLATE and GitLab

Performance, Load and Stress Tests

Big Picture of Spring Cloud Gateway