Vite use HMR in production

I have a Vue 3 app in production mode that should be extendable via plugins (basically just Vue components) by users. To do so it would be great if a user doesn't have to clone the whole app and run it in dev mode. Instead the production app (running on the users computer) should load the plugin from a vite dev server.

App

// main.js
import plugin from 'http://127.0.0.1:4173/src/index.js'

const app = createApp(App)
app.use(plugin)
app.mount('#app')

// App.vue

<template>
  <PluginComponent></PluginComponent>
</template>

Plugin

// index.js (served as 'http://127.0.0.1:4173/src/index.js' by vite)

import PluginComponent from './PluginComponent.vue'

export default {
  install(app) {
    app.component('PluginComponent', PluginComponent)
  }
}

// PluginComponent.vue

<template>
  <div>Plugin</div>
</template>

<style scoped>
div {
  background: yellow;
}
</style>

This actually works, but HMR is only triggered for global styles and scripts defined in the PluginComponent. Everything scoped like scoped CSS or the component itself won't update. I realised that the dev server for the plugin uses a different hmrId/data-v for the PluginComponent than the App, so I guess this could be the problem.

Has anyone tried a setup like this or maybe is there a better solution for my scenario?