Heavy Memory Usage of the Regression Test in XCTest

While recently wrestling with a regression test for my Swift project using the XCTest framework, I ventured into the tumultuous world of a heavily multithreaded application. Ensuring the absence of timing issues was like defusing a bomb – one wrong wire, and boom! So, what's a developer to do? Run the code tens of thousands of times, of course (or even more for the marathon runners of applications), and keep your fingers crossed for any anomalies.

My trusty XCTest framework was the stage, and my test code, which cheerfully returned results asynchronously, demanded I master the classic XCTest dance of expectation(...) and waitForExpectations(...) moves. All seemed well as the test passed with flying colors for a few thousand iterations. However, as I ambitiously cranked up the numbers, strange gremlins began to pop up. The memory usage skyrocketed, adding a whopping 90 MB per 10,000 iterations – definitely not a feature I had intended to implement!

I dove into the depths of Instruments, yet no memory leaks waved back at me. Puzzling! Yet, I noticed waitForExpectations was having a bit too much fun allocating objects like NSString left and right. My online detective work on Google and StackOverflow turned up zilch – it seemed I was charting uncharted waters.

Refusing to be defeated, me and my digital buddy ChatGPT kept on going into the uncharted territory with me as novis to Swift while ChatGPT blind beyond 20 ft. I scrutinized the allocation patterns revealed by Instruments with a detective's eye. This careful observation led to a pivotal "Eureka!" moment. It dawned on me to employ an autoreleasepool within the scope of each iteration. This strategic move, akin to a well-crafted chess play, was not just a stroke of luck but a calculated decision based on the insights gleaned. And, like a charm, this adept adjustment worked wonders, effectively dispelling the memory bloat gremlins back into the abyss from whence they came.


Here is a sample test code anyone can run in our XCTest project, to reproduce the issue and verify the solution.

1.     func testAsyncOperationInLoop() {
2.         let iterationCount = 50000  // Number of iterations in the loop
3.         
4.         for i in 1...iterationCount {
5.             autoreleasepool {
6.                 
7.                 // Create a new expectation for each iteration of the loop
8.                 let expectation = self.expectation(description: "Async operation \(i)")
9.                 
10.                 if i > 0 && i % (10000) == 0 {
11.                     print("breaking for a breath")
12.                 }
13.                 
14.                 // Simulate an asynchronous operation
15.                 DispatchQueue.global().asyncAfter(deadline: .now() + 1.0/100000) {
16.                     expectation.fulfill()  // Fulfill the expectation once the operation is completed
17.                 }
18.                 
19.                 // Wait for the expectation to be fulfilled, with a timeout
20.                 waitForExpectations(timeout: 2) { error in
21.                     if let error = error {
22.                         XCTFail("waitForExpectations errored: \(error)")
23.                     }
24.                 }
25.             }
26.         }
27.         
28.         print("All iterations completed")
29.     }