How does MTProto 2.0 secure communication in Telegram and what are its key cryptographic primitives?

Telegram relies on its proprietary cryptographic protocol called MTProto. The latest iteration, MTProto 2.0, introduced critical enhancements over version 1.0 to mitigate theoretical cryptographic vulnerabilities like chosen-ciphertext attacks.

Core Cryptographic Primitives of MTProto 2.0

MTProto 2.0 employs symmetric encryption combined with key exchange mechanisms to protect communication pipelines:

• AES-256 in IGE Mode: Used for symmetric message payload encryption.
• SHA-256: Used as the hashing function for message key calculation and integrity checking.
• Diffie-Hellman Key Exchange: Used during chat initialization to establish shared secret keys without exposing them over untrusted networks.

Key Integrity Verification Code Example

Below is a conceptual Python representation of how MTProto calculates the message key digest to verify incoming payload integrity:

import hashlib

def verify_message_key(msg_key, payload_bytes, auth_key):
    # MTProto 2.0 derives msg_key from a SHA-256 hash of payload + auth_key slice
    combined_data = payload_bytes + auth_key[88:120]
    computed_hash = hashlib.sha256(combined_data).digest()
    computed_msg_key = computed_hash[8:24]
    
    if computed_msg_key == msg_key:
        return True
    return False

Security Discussion

Why has the academic security community questioned Telegram's custom cryptographic choices compared to standardized protocols like TLS 1.3 or the Signal Protocol, and how does MTProto 2.0 defend against replay attacks?