Implementing RoPE (Rotary Positional Embedding) in C#

What is RoPE?

RoPE stands for Rotary Positional Embeddings.  RoPE was introduced to provide positional information directly inside the Attention mechanism.

Instead of:

Embedding + Position

RoPE modifies:

Query
Key

vectors before Attention calculation which is a major difference.

Why Modern LLMs Use RoPE

RoPE provides several benefits like: 

• Better Long Context Understanding
• Relative Position Awareness
• Better Generalization
• Efficient Computation

Most modern LLMs use RoPE like:

• Llama
• Mistral
• Qwen
• DeepSeek
• GPT-style architectures

RoPE has become the industry standard positional encoding technique.

C# Implementation:

using System;
class Program
{
    static void Main()
    {
        double[] vector={1.0, 0.0};
        double angle=Math.PI/4;

        double x=vector[0];
        double y=vector[1];

        double rotatedX=x*Math.Cos(angle)-y*Math.Sin(angle);
        double rotatedY=x*Math.Sin(angle)+y*Math.Cos(angle);

        Console.WriteLine($"Real Vector:({x},{y})");
        Console.WriteLine($"Rotated Vector:({rotatedX:F3},{rotatedY:F3})");
    }
}