Steps for Using DeepSpeed on Windows 11

git clone https://github.com/microsoft/DeepSpeed.git

Then you need to modify some code:

1. Open the file in the directory: csrc/quantization/pt_binding.cpp

Change line 203's

std::vector<long int>

to:

std::vector<long long>

2. Open the file in the directory: csrc/transformer/inference/csrc/pt_binding.cpp
On line 534:

auto prev_key = torch::from_blob(workspace + offset,
                                     {bsz, heads, all_tokens, k},
                                     {hidden_dim * InferenceContext::Instance().GetMaxTokenLength(),
                                      k * InferenceContext::Instance().GetMaxTokenLength(),
                                      k,
                                      1},
                                     options);

Add unsigned, i.e., change to:

auto prev_key = torch::from_blob(workspace + offset,
                                     {bsz, heads, all_tokens, k},
                                     {hidden_dim * (unsigned)InferenceContext::Instance().GetMaxTokenLength(),
                                      k * (unsigned)InferenceContext::Instance().GetMaxTokenLength(),
                                      k,
                                      1},
                                     options);

And change line 1570's

auto intermediate_gemm =
        at::from_blob(intermediate_ptr, {input.size(0), input.size(1), mlp_1_out_neurons}, options);

to:

auto intermediate_gemm =
        at::from_blob(intermediate_ptr, {input.size(0), input.size(1), (int)mlp_1_out_neurons}, options);

3. Find "x64 Native Tools Command Prompt for VS 2022" in the Start menu, right-click it, and run build_win.bat as Administrator.