Redux+Websockets: Why manage this using middleware?

There are couple of pros with placing such logic in middle-wares instead of actual components.
The main reasons in my opinion are:

• Each connected component will instantiate a WebSocket or you will need a global declaration of the connection which will be independently from the store, in other words, not part of the redux flow.
• Middle-Wares have access to the store and are part of the redux flow.
• You also get access to the entire store, hence you can pass forward more data then initially dispatched.
• You decouple the components from the web-socket logic, hence you can centralize your code and reuse your components.

All in all, there are no special reasons that are specific regarding using web-sockets with middle-wares, using middle-wares have great advantages in general.

Edit
As a followup to your comment

How would you suggest managing a case where you might want a particular component to initialize a websocket connection, but want to manage whether it's already connected, etc... would it be just as simple as having a flag in the store that says it's connected or is there a better approach?

As i said, I wouldn't initialize a web-socket connection within a component, rather I would do it in the entry point of my application. like index.js for example.
If your concern is to make sure you won't try to connect when there is already a connection, then a simple socketStart method that get invoked at the point when you create the store and initialize all your App data, you can pass it a callback that will do the rendering and store update through dispatch.
A simple example (keep in mind this is a pseudo code ):

Our Socket-start Method:

export function socketStart(store, callback) {
  // this is only a pseudo code!

  // register to server -> client events
  _socketClient.someFunction = (data) => {
    store.dispatch({ type: "Server_Invoked_Some_Function", data });
  }

  _socketClient.someFunction2 = (data) => {
    store.dispatch({ type: "Server_Invoked_Some_Function2", data });
  }

  // connect to the server via the web-socket client API
  _socketClient.connect(() => callback());
}

We can use it in our index.js file:

let store = createStore(
  // your reducers here...
  // ...
  applyMiddleware(socketMiddleware) // our web socket middleware
)

// the callback will invoked only if the connection was successful
// the React-Dom's render function is our callback in this case
socketStart(store, () => { render(<App />, document.getElementById("root")); });

With middleware, you can easily unfold/relay messages between Redux and Web Socket. Also, you can use Redux middleware without React, that means you can write API using Redux on server-side code (probably with Redux saga).

I would agree lifetime management as a React component is easier than a Redux middleware. But if you want to reconnect (destroy/recreate), you will need to use key props to make the reconciler to consider it as a new object, which is a little bit weird.

You can look at redux-websocket-bridge, which unfold Web Socket messages into Redux actions, and relay Redux actions to Web Socket.

On your Web Socket server, you send an action:

ws.on('connection', conn => {
  conn.send(JSON.stringify({ 
    type: 'GREETING', 
    payload: { now: Date.now() } 
  }));
});

You will get the GREETING action on your Redux reducer. And vice versa, when you want to relay an action to Web Socket, you mark your action with meta.send with true:

this.props.dispatch({ 
  type: 'SIGN_IN',
  meta: { send: true },
  payload: { authToken: '...' }
});