How do I add a folder to a GitHub repo using JGIT without cloning?

Question

How do I add a folder with item to a repo (in this case a GitHub repo) using JGIT without having to clone and such? Would I just create a local repo with the folder then merge it with the remote one?

The reason I don't want to have to clone is that the repo is going to get very large and I don't want it to take forever to clone. Also, this is inside a Spigot plugin command, but I don't think that would change anything. It gives me an error when I run the command.

One last thing is that when I copy the folder to the repo folder. It just creates a folder called "Minecraft Servers" and none of the files in it. What am I doing wrong?

Source code:

public class SaveServer implements CommandExecutor {

    private final CrucialPlugin plugin;

    public SaveServer(CrucialPlugin plugin) {
        this.plugin = plugin;
    }


    @Override
    public boolean onCommand(CommandSender sender, Command command, String label, String[] args) {

        if (sender instanceof Player) {
            Player player = (Player) sender;

            if (args.length == 0) {
                player.sendMessage(ChatColor.RED + "You need to enter some arguments");
                player.sendMessage(ChatColor.YELLOW + "To save the server network to the GitHub repo: /saveserver ");
            } else if (args.length == 1) {
                if (plugin.getConfig().getString("Repository URL") != null && plugin.getConfig().getString("GitHub username") != null && plugin.getConfig().getString("GitHub password") != null) {
                    Path repofolder = Path.of("Server Network Repo");
                    File test = new File(String.valueOf(repofolder));
                    if (Files.exists(repofolder)) {
                        test.delete();
                    }
                    player.sendMessage(ChatColor.GREEN + "Saving server with caption: " + args[0] + "...");
                    File curdir = new File(System.getProperty("user.dir"));
                    Path networkFolder = curdir.toPath().getParent();
                    Path finalFolder = Path.of(repofolder + File.separator + "Minecraft Servers");
                    Date date = new Date();
                    char subColon = '\uA789';
                    String filename = date.toString().replace(':', subColon) + "-" + args[0].toUpperCase().replace("_", " ") + ".txt";

                    File txt = new File(networkFolder + File.separator + filename);
                    try {
                        txt.createNewFile();
                        Git.init().setDirectory(new File("Server Network Repo")).call();
                        Git git = Git.open(new File("Server Network Repo"));
                        git.add().addFilepattern(networkFolder.toString()).call();

                        Files.copy(networkFolder, finalFolder, REPLACE_EXISTING);
                        // add remote repo:
                        RemoteAddCommand remoteAddCommand = git.remoteAdd();
                        remoteAddCommand.setName("origin");
                        remoteAddCommand.setUri(new URIish(plugin.getConfig().getString("Repository URL")));
                        remoteAddCommand.call();

                        // push to remote:
                        PushCommand pushCommand = git.push();
                        pushCommand.setCredentialsProvider(new UsernamePasswordCredentialsProvider(plugin.getConfig().getString("GitHub username"), plugin.getConfig().getString("GitHub password")));
                        pushCommand.call();
                    } catch (GitAPIException | IOException | URISyntaxException e) {
                        player.sendMessage(ChatColor.RED + "Unable to retrieve repository.");
                    }
                    player.sendMessage("Filename test: " + filename);
                } else {
                    player.sendMessage(ChatColor.RED + "You need to fill in all the fields under Save Server in the configuration file before you can use this feature.");
                }
            } else {
                player.sendMessage(ChatColor.RED + "Too many arguments");
                player.sendMessage(ChatColor.YELLOW + "To save the server network to the GitHub repo: /saveserver ");
            }
        }
        return true;
    }
}

user_1374322 · Answer

The problem seems to be solvable by using shallow clone and sparse checkout, which are git features that allow to clone repository without actually pulling the entire contents and history. The problem is that as of April 2022, the feature is not implemented in jGit. So depending on your circumstances, you may settle with some sort of compromise:

If you have control over the environment, where the code is going to be executed, you may try invoking the shell commands and call real git instead of jgit. This of course reduces the portability of your solution, and puts additional configuration burden on the environment, (expect git to be installed, expect it to be in correct version, expect code running with permissions sufficient to call system commands, git authentication now happens outside Java code, etc.).
If you intend to only push specific updated file into the top of the main branch, and you do not care about all git features, like branching, diffing and merging, but simply use a very specific github repository as a sort of configuration storage that you sometimes need to update with Java, then you may want to skip git altogether, and use Github REST API instead. You would then send your updated file and the commit information, such as message and author as a http POST request, and let Github transform it into a commit that other users could use with regular git client. There are ready to use Java wrappers for the API, e.g. this project. Of course, if you just happen to use Github, and the actual code is expected to work with any git repository and server backend, then this method will not work. It also has the downside of creating additional dependency on the API that is not as stable as regular git is, so there is a greater chance of something getting broken because of changes in Github.
Bite the problem from yet another angle and try to minimize the repository size. For example, give up git history and make each commit an amend commit force pushed to overwrite the previous one. Probably a top 10 item on git antipatterns list and clear abuse of what git is intended to do, but if it solves your problem, then it's all for the best. Not solving the problem of repository that is large due to being full of large files, and not just because of the long history.
Hire a team and solve this and/or this. Many people will benefit, so it's a moral choice to make. No downsides as far as I can tell ;).

How do I add a folder to a GitHub repo using JGIT without cloning?

1 Answers