Can you legally dynamic_cast to non-polymorphic base class of a polymorphic class

In this answer, the following scenario came up:

#include <cassert>

struct A {};

struct B { virtual ~B(){} };

struct AA{};
template <class T>
struct C : A, T {};

int main()
{
  B * b = new C<B>;
  AA * aa = new C<AA>;
  assert(dynamic_cast<A*>(b));
  assert(dynamic_cast<A*>(aa)); //this line doesn't compile, as expected
}

On g++ 4.8.4 (Ubuntu), this compiles and the assert passes. My question is, is that really legal? I feel like you shouldn't be able to dynamic_cast to a non-polymorphic class at all, but I freely admit that I'm not an expert in what's happening here.

When I tried the opposite direction:

dynamic_cast<B*>((A*)(new C<B>));

it fails to compile, stating that "source type is not polymorphic". I feel like that's a clue, but it still seems a stretch to find the non-polymorphic base class that belongs to a class that the current pointer is a base of (did that sentence make sense?).