Discord.Net sending audio with ffmpeg only works once (the first time)

Context:

So I am writing a small bot that should play local audio files on command. I am using Discord.Net, ffmpeg, opus and libsodium. I have a Speak() function that should, in Theory,

Open ffmpeg -> encode .mp3 -> create PCM stream -> pump the encoded .mp3 to the output(PCM Stream).

That looks like this:

public async Task Speak(IGuild guild, Sound.SoundName soundName)
{
  IAudioClient client;
  if (ConnectedChannels.TryGetValue(guild.Id, out client))
  {
    using (var ffmpeg = CreateStream(sound.Filename))
    using (var stream = client.CreatePCMStream(AudioApplication.Voice, 98304))
    using (var output = ffmpeg.StandardOutput.BaseStream)
    {
     try
     {
       await output.CopyToAsync(stream);                            
     }
     catch (Exception ex) { Console.WriteLine("Error " + ex.Message); Console.WriteLine($"- {ex.StackTrace}"); }
     finally { await stream.FlushAsync(); }
     Console.WriteLine("Spoken!");
   }
}

private Process CreateStream(string _path) 
{
  var process = Process.Start(new ProcessStartInfo
                 {
                   FileName = "ffmpeg",
                   Arguments = $"-hide_banner -loglevel panic -i \"{_path}\" -ac 2 -f s16le -ar 48000 pipe:1",
                   UseShellExecute = false,
                   RedirectStandardOutput = true,
                 });
  return process;
}

And it does this, but only one time. A bit more specific, When my bot joins a Voice Channel, it saves the VoiceChannel ID and the IAudioClient in a private static readonly ConcurrentDictionary<ulong, IAudioClient> ConnectedChannels = new ConcurrentDictionary<ulong, IAudioClient>();. After that it automatically calls the Speak() function to play a hello.mp3 audio. This Process is done like this:

[Command("join", RunMode = RunMode.Async)]
[RequireUserPermission(GuildPermission.MentionEveryone)]
public async Task JoinChannel(IVoiceChannel channel = null) 
{
 // Get the Voice channel
 channel = channel ?? (Context.User as IGuildUser)?.VoiceChannel;
 if (channel == null) { await Context.Channel.SendMessageAsync("You are not in a VoiceChannel"); return; }

 // Saving the AudioClient
 var audioClient = await channel.ConnectAsync();
 Console.WriteLine($"Connected to channel {channel.Name}");

 ConnectedChannels.TryAdd(channel.Guild.Id, audioClient);

 await Task.Delay(1000);
 await Speak(Context.Guild, Sound.SoundName.hello);
}

This works pefectly fine, The audio plays.

Now that the bot is connected to a VoiceChannel and the IAudioClient is saved to the Dictionary. I should be able to call the Speak() function whenever i want and from Whereever i want, as long as the bot is in the VoiceChannel, right?

No it doesnt.

and that takes me to my

Problem:

After the bot is now sitting silently in the voice channel i call a "speak" command that looks in code like this:

[Command("speak")]
public async Task speakSingle() 
{
  await Speak(Context.Guild, Sound.SoundName.Random);
}

But the bot remains silent, even though the speak indicator in Discord lights up! What am I missing? I dont get it. Is it sending an empty stream?
Even when I disconnect the bot from the VoiceChannel an reconnect it It wont send audio. The only thing that helps is Reconectiong the Bot From the Server.
I am pretty new to C# and Streams and async programing. So could somebody help me out finding the problem and fix it?

Errors (CommandPromt Ouput):

1#
This occures when i run the "speak" command, im getting a NullReferenceException for "Discord.WebSocket.pdb not loaded" in VisualStudio. Allthough I couldn't find anything thats null...

17:34:17 Audio #1    System.Exception: WebSocket connection was closed
 ---> Discord.Net.WebSocketClosedException: The server sent close 4008: "Rate limited."
   at Discord.Net.WebSockets.DefaultWebSocketClient.RunAsync(CancellationToken cancelToken)
   --- End of inner exception stack trace ---
   at Discord.ConnectionManager.<>c__DisplayClass29_0.<<StartAsync>b__0>d.MoveNext()

2#
This occures when the bot rejoined the Voice Channel and automatically executes the Speak() function.

Error A task was canceled.
-    at Discord.Audio.Streams.BufferedWriteStream.WriteAsync(Byte[] data, Int32 offset, Int32 count, CancellationToken cancelToken)
   at Discord.Audio.Streams.OpusEncodeStream.WriteAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancelToken)
   at System.IO.Stream.CopyToAsyncInternal(Stream destination, Int32 bufferSize, CancellationToken cancellationToken)
   at Bot.Modules.VoiceChannel.Speak(IGuild guild, SoundName soundName) in Y:\Dokumente\Coding\C#\Vs_Studio\Bot\Bot\Modules\VoiceChannel.cs:line 90

This is my first Question on StackOverflow and i hope i provieded enough context, if not tell me please.