Windows Desktop Flutter: Error when using flutter libserialport package

I am trying to convert an existing Qt application to Flutter App. The existing application communicates with a hardware over serial port. Sends and receives commands and messages.

I am using flutter's libserialport package to achieve the communication.

The application UI has a simple button as of now which calls following function when clicked.

void displayPorts()
{
  _name = SerialPort.availablePorts;
  print(_name);
  sp = SerialPort(_name[2]);
  sp.config.baudRate = 9600;
  sp.config.bits = 8;
  sp.config.stopBits = 1;
  sp.config.parity = 0;
  sp.config.xonXoff = 0;
  sp.config.setFlowControl(0);
  sp.config.dtr = 0;
  sp.config.rts = 0;
  sp.open(mode: 3);
  if(!sp.openReadWrite()){
    print(SerialPort.lastError);
  }
  if(sp.isOpen){
    print("open and write");
    sp.write(utf8.encode("A"));
  }

  print('DN: ${sp.deviceNumber}'); 
  print('\tDescription: ${sp.description}');
  print('\tManufacturer: ${sp.manufacturer}');

}
if(sp.isOpen){
    print("open and write");
    sp.write(utf8.encode("A"));
  }

I am getting following error on running this: "flutter: SerialPortError: Access is denied., errno = 5"

The application is able to read available ports and display the information as per print messages above.

But it is not able to read/write on the port(The print statement inside if check of isOpen is not printed). I have also attached an oscilloscope on the output of USB/Serial converter and can not any message being thrown out.

Any pointers to possible solution. I am running the application from a user login with admin rights. I did not get any relevant links on google search.