ResourceManager GetResourceSet gets slow

I have an older two-language site (c#/js using resx-files) where the language of a specific page is determined not by the users prefered language but by how the post showed on the page is created - to get the correct language to the client using AJAX I have the following code;

var rm = new ResourceManager(typeof(T));
using (var rs = rm.GetResourceSet(CultureInfo.CurrentUICulture, true, true))
{
    var result = rs.Cast<DictionaryEntry>().ToDictionary(x => x.Key.ToString(), x => x.Value.ToString());
    return result;
}

After a while, it can take weeks or even months, GetResourceSet can get very slow, easily 10-15 seconds, and I need to restart the App Pool. The "fun" part is that this only happens for one of the two languages and it is the least used. This cannot be provoked in dev nor test environments. No errors occurs, it just gets slow.

I guess it is because of some lock in GetResourceSet but how can I fix this issue?