Typedef private struct prototype in source file

In my class I have the need to keep a pointer to a structure which is defined in a library I use to implement it. Since this library is only used within the implementation file I would like to avoid including it in the header directly. At the same time I want to avoid polluting the namespace. Thus I would like to do:

/* HEADER */
class Foo {
    private:
        struct ImplementationDetail;
        ImplementationDetail * p;
};
/* SOURCE */
#include <Library.h>
using Foo::ImplementationDetail = Library::SomeStruct;

But this doesn't work, and I'm currently falling back on PIMPL:

/* HEADER */
class Foo {
    private:
        struct ImplementationDetail;
        ImplementationDetail * p_;
};
/* SOURCE */
#include <Library.h>
struct ImplementationDetail {
    Library::SomeStruct * realp_; 
}

Is there a way to avoid the double dereference? Is the reason for my non-working first solution due to unknown pointer sizes?