It can be done.
TCP is actually handled by your OS, not by userspace code. So the best thing you may do is to kindly ask your OS about it via the (system) calls it exposes.
As such, OS knows and is prepared for parallel calls from many user-space threads. It is protected by an appropriate locking mechanism to ensure serializability.
What you should or should not be concerned about is performance and error handling. What do you do if the Windows server you have does not accept TCP connection? What would you do if it did accept, however TCP retransmission happens over and over again and you can't pass a message? What do you do if the Windows service forcibly terminates the connection? There are many questions to ask in distributed systems.
Performance is another story. You haven't mentioned it, so I won't spend much time on it. Let me know if I should though, I'll update the post.
Now about Python specifics. Due to the "global interpreter lock" in the CPython, multithreading along with its Thread does not make your code truly parallel; at best you get single-treaded multitasking a-ka simplest possible concurrency. That shouldn't be a big problem for you, since TCP is inherently IO-bound.
The simplest you can do is to spawn a dedicated Thread per session. That kinda implies long-living sessions, otherwise, it is a resource waster and performance killer. Again, you did not mention the requirements, so I won't suggest anything.
