Record Level Data truncation on Mainframe server while doing SFTP from spark server

From this presentation Dovetail SFTP Webinar on slide 21:

ls /+recfm=fb,lrecl=80

it seems to me there is one '/' too many in your code.

From the error message, I think the SFTP server has the current path in the UNIX file system. You do not set the data set high level qualifier (HLQ) for the data set, do you? I can't see it in the code. Again from the above presentation, do a cd before the ls:

cd //your-hlq-of-choice

This will do two things:

1. Change the current working directory to the MVS data set side.
2. Set the HLQ to be used.

Sorry I cannot test myself; I do not know scala.

First, what SFTP server is running on z/OS? If it is the one provided with z/OS (not Dovetail) the command you are executing isn't supported and you will receive a message like Can't ls: "/+recfm=fb,lrecl=80" not found. Which would be valid because that is not valid file. Everything to the right of the / would be considered part of the filename.

I converted your code to Java as I'm not familiar with Scala and didn't have time to learn it. Here was my code sample I used.

import com.jcraft.jsch.JSch;
import java.util.Properties;
import java.util.Vector;

class sftptest {
  static public void main(String[] args) {

    String username = "ibmuser";
    String host = "localhost";
    int port = 10022;              // Note, my z/OS is running in a docker container so I map 10022 to 22
    JSch jsch = new JSch(); 
    String sourceFile = "/";
    String destinationPath ="/";
    String privateKey = "myPrivateKey";


    try {
      jsch.addIdentity(privateKey);                   //add private key path and file
      com.jcraft.jsch.Session session = jsch.getSession(username, host, port);
      session.setConfig("PreferredAuthentication","password");
      session.setConfig("MaxAuthTries", "2");
      System.out.println("Created SFTP Session");

      Properties sftpSessionConfig = new Properties();
      sftpSessionConfig.put("StrictHostKeyChecking","no");
      session.setConfig(sftpSessionConfig);
      session.connect(); //Connect to session
      System.out.println("Connected to SFTP Session");
      
      com.jcraft.jsch.ChannelSftp channel = (com.jcraft.jsch.ChannelSftp) session.openChannel("sftp");
      channel.connect();
      // com.jcraft.jsch.Channel sftpChannel = (ChannelSftp) channel;
      //    channel.ls("/+recfm=fb,lrecl=3000"); //set lrecl and recfm ---> THROWING ERROR HERE
      //    channel.ls("/"); //set lrecl and recfm ---> THROWING ERROR HERE
      Vector filelist = channel.ls("/");
      for(int i=0; i<filelist.size();i++){
          System.out.println(filelist.get(i).toString());
      }


    //  channel.put(sourceFile, destinationPath, com.jcraft.jsch.ChannelSftp.APPEND);  //Push file from local to mainframe
    } catch (Exception e) {
      System.out.println("Exception "+e.getMessage());
    }
  }
}

For my case I did use an ssh key and not a password. The output with your ls method is:

Created SFTP Session
Connected to SFTP Session
Exception No such file

dropping the + and everything to the right you get:

Created SFTP Session
Connected to SFTP Session
drwxr-xr-x    2 OMVSKERN SYS1         8192 May 13 01:18 .
drwxr-xr-x    7 OMVSKERN SYS1         8192 May 13 01:18 ..
-rw-r--r--    1 OMVSKERN SYS1            0 May 13 01:18 file 1
-rw-r--r--    1 OMVSKERN SYS1            0 May 13 01:18 file 2

The main issue is that the z/OS appears to not support the syntax you are using which is provided by a specific SFTP implementation by Dovetail.

If you do not have Dovetail I recommend that since you are sending CSV files that are generally variable in length that you send them as a USS file so that the lines will be properly translated and will be of variable length. Transfer them to USS (regular Unix on z/OS) and then copy them to an MVS file that has a RECFM of VB. Assuming the file is already allocated you could do a cp myuploadedFile.csv "//'MY.MVS.FILE'"