Do I really have to worry about alignment when using placement new operator?

I read this When should I worry about alignment? but I am still do not know if I have to worry about not aligned pointer returned by placement new operator - like in this example:

class A {
public:
   long double a;
   long long b;
   A() : a(1.3), b(1234) {}
};

char buffer[64];

int main() {
   // (buffer + 1) used intentionally to have wrong alignment
   A* a = new (buffer + 1) A(); 
   a->~A();
}

__alignof(A) == 4, (buffer + 1) is not aligned to 4. But everything works fine - full example here: http://ideone.com/jBrk8

If this depends on architecture then I am using: linux/powerpc/g++ 4.x.x.

[UPDATE] Just after posting this question I read this article: http://virtrev.blogspot.de/2010/09/memory-alignment-theory-and-c-examples.html. Maybe the only drawbacks in my case would be performance penalty, I mean unaligned access cost more than aligned?