Open Access Article

Title: Enhancing IoT security in the post-quantum era: a hybrid approach to protection from impersonation attacks

Authors: Adel Hassan; Isam Ishaq; Jorge Munilla

Addresses: Faculty of Engineering and Information Technology, Arab American University, Jenin, Palestinian Territory ' Faculty of Engineering, Al-Quds University, Jerusalem, Palestinian Territory ' Faculty of Telecommunication Engineering School, University of Málaga, Málaga, Spain

Abstract: The imminent advent of quantum computing threatens the long-term security of classical public-key cryptography, creating critical challenges for resource-constrained internet of things (IoT) environments. Hybrid cryptographic frameworks combining elliptic curve cryptography (ECC) with post-quantum cryptography (PQC) primitives have emerged as a transitional solution; however, their security properties remain insufficiently validated under semi-trusted infrastructure assumptions. This paper presents a comprehensive security evaluation of a hybrid ECC-KEM authentication and key agreement framework targeting IoT deployments. Using a dual-verification methodology that integrates formal symbolic analysis with the Tamarin-Prover and practical AI-assisted testing via the APSVer tool, we identify a critical impersonation vulnerability that enables a semi-trusted third party (TTP) to reconstruct PUF-based authentication material. To mitigate this flaw, we introduce a protocol enhancement incorporating ML-DSA post-quantum digital signatures and device-isolated key derivation. Formal verification confirms that the enhanced protocol achieves mutual authentication, session key secrecy, replay resistance, and effective isolation of the TTP. Experimental performance analysis demonstrates that the proposed security improvements introduce only a 12%-15% computational overhead, preserving feasibility for constrained IoT devices. The proposed framework provides a practical, quantum-resilient authentication architecture suitable for next-generation secure IoT deployments.

Keywords: post-quantum cryptography; PQC; hybrid ECC-KEM; key agreement; PUF-based authentication; Tamarin-Prover; APSVer; ML-DSA; ML-KEM.

DOI: 10.1504/IJITST.2026.156451

International Journal of Internet Technology and Secured Transactions, 2026 Vol.13 No.9, pp.1 - 42

Received: 15 Mar 2026
Accepted: 22 Jun 2026

Published online: 18 Sep 2026 *