VSTS Self-hosted Build Agent Xamarin build error

Trying to get our CI to build our Xamarin.Android apps, I have made a Task for this, and it builds fine the first 2 or 3 times but after that, it constantly throws errors like the one below

C:\agent_work\1\s*\obj\Xamarin\110\lp\104\jl\res\values-sq\values.xml(1): error APT2254: xml parser error: no element found. [C:\agent_work\1\s\Implementations*.csproj

[error]C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Xamarin\Android\Xamarin.Android.Aapt2.targets(157,3): Error APT2260: resource style/Theme.AppCompat.Light.DarkActionBar (aka ****:style/Theme.AppCompat.Light.DarkActionBar) not found.

our styles xml:

<resources>

  <!-- Base application theme. -->
  <style name="AppTheme" parent="Theme.AppCompat.Light.DarkActionBar">
    <!-- Customize your theme here. -->
    <item name="colorPrimary">@color/colorPrimary</item>
    <item name="colorPrimaryDark">@color/colorPrimaryDark</item>
    <item name="colorAccent">@color/colorAccent</item>
    <item name="android:titleTextStyle">@android:color/white</item>
    <item name="android:textColor">@android:color/white</item>
    <item name="windowActionBar">false</item>
    <item name="windowNoTitle">true</item>
  </style>

  <style name="AppTheme.NoActionBar">
    <item name="windowActionBar">false</item>
    <item name="windowNoTitle">true</item>
  </style>

  <style name="AppTheme.AppBarOverlay" parent="ThemeOverlay.AppCompat.Dark.ActionBar" />

  <style name="AppTheme.PopupOverlay" parent="ThemeOverlay.AppCompat.Light" />
  <style name="Theme.Splash" parent="android:Theme">
    <item name="android:windowBackground">@drawable/drawable_splashscreen</item>
    <item name="android:windowNoTitle">true</item>
  </style>
</resources>

EDIT: I already have on the pipeline:

1. All nuget are restored, and set to not use cache (even with cache it still fails)
2. All Obj/Bins folders are deleted before build (has been confirmed)
3. Clean and Restore are called before Build also.
4. Full Post Build Cleanup task