NestedBTO: A Timestamp-Based STM Protocol for Closed Nested Transactions with Opacity Guarantees

Software Transactional Memory (STM) provides a high-level abstraction for managing concurrency, alleviating the complexity of traditional lock-based programming. While nesting support is critical for enabling composability and reducing the vulnerability window of long transactions, existing STM protocols for closed nesting often fall short in either offering rigorous correctness guarantees or demonstrating practical performance benefits. This paper introduces NestedBTO, a new STM protocol for closed nested transactions that uses timestamp ordering to allow child transactions to be executed in parallel while maintaining consistency according to the opacity criterion. NestedBTO satisfies opacity for all transactional histories, as demonstrated by our formal proof of correctness. We conduct extensive experimental evaluations using application-level workloads STAMP and concurrent red-black tree and hashtable benchmarks to assess its effectiveness. The results reveal that NestedBTO achieves substantial improvements in throughput and abort rates compared to other state-of-the-art STM, i.e., NesTM. These findings demonstrate that NestedBTO offers a scalable and practical solution for high-performance transactional memory systems.