rsync / git / ssh problem macos 11.3.1 / Big Sur

Viewed 1048

16.6.21 UPDATE

I Found the cause of this problem. It was my VPN Client (CISCO Anyconnect). There were some Packet Filters installed with the Client and therefore not all Packages were sent properly to the server.

QUESTIONE

I already searched google for hours to find a solution for my problem, but none of the written solutions helped me.

Last Friday, I did a complete fresh install of Mac OS Big Sur on my Mac Mini (2020). I was setting up my development environment to my needs and it looked like it was working really good until yesterday.

Yesterday was trying to push some commits to our GitLab server... while pushing a error came up

client_loop: send disconnect: Broken pipe
send-pack: unexpected disconnect while reading sideband packet
fatal: the remote end hung up unexpectedly

I tried to fix the error by setting several arguments in my ./ssh/config file, like:

Host *
  ServerAliveInterval 60
  ServerAliveCountMax 5
  IPQoS=throughput

and many more.

3 hours later I was really annoyed and pushed the commits via https not via ssh anymore.

A bit later I had to push a local WordPress installation to our staging environment. Suddenly nearly the same error occurred again

rsync: [sender] write error: Broken pipe (32)
rsync error: unexplained error (code 255) at io.c(823) [sender=3.2.3]

The strange thing is:
While downloading via git clone or rsync (both via ssh) everything works well. Also connecting via ssh to the server works really well BUT! when I begin to upload a larger amount of files I'm getting those errors (also tried with a plain rsync command).

Could this be caused by different OpenSSH versions on our servers (OpenSSH_7.2p2 Ubuntu-4ubuntu2.10, OpenSSL 1.0.2g 1 Mar 2016) and my machine (OpenSSH_8.6p1, OpenSSL 1.1.1k 25 Mar 2021)?

Anyone has any advices for me?

3 Answers

It works again. I don't know how or why, but it works. I did a clean reinstall of macos (again) and now it seems to work.

I still can't pinpoint what exactly it was, but i noticed the ssh disconnect issues after the latest Update. I could pinpoint it down to the Macbook SSH Libraries , because i did the following steps:

  • Connected from another another source (Terminus on iPhone, another Mac and Debian running on NUC) within the network without any problems (same ssh target destinations in all source device examples)
  • Disabled temporarily all network adblocking, firewall and filtering
  • added a wired connection to the "problem macbook"
  • tried all the ssh config options below , with the default openSSH and LibreSLL Version from MacOS
  • Connected to different Target Servers from the "problem Mac" and the disconnects appeared suddenly everywhere with different timings (only with the problem Mac).

I assume, that the problem lies with the default Big Sur OpenSSH and LibreSSL Version. After i installed OpenSSH with brew install openssh with the following config, everything works like before.

I hope it also solves your problem, because it took me couple of days :(

My Machine

  • MacOS 11.3.1
  • MacBook Pro Retina 15, Mid 2015
  • Intel i7

Steps to fix:

  1. brew install openssh
  2. Add config to ~/.ssh/config
Host *

    IPQoS none
    TCPKeepAlive no
    ServerAliveInterval 60
    ServerAliveCountMax 5

Play around with the .ssh/config config values, to match your needs and don't make them to aggressive

ssh -V after brew install openssh: OpenSSH_8.6p1, OpenSSL 1.1.1k 25 Mar 2021

Could this be caused by different openssh versions on our servers

Check if using an old PEM SSH key format would work better, considering OpenSSH 7.8 has switch to a new format by default:

ssh-keygen -t rsa -P "" -m PEM -f ~/.ssh/key2

Register that key2.pub to your GitLab server, and try at least a ssh -Tv git@myGitLabServer

Check also, as mentioned here, your SSH confog for any non-standard directive, like RemoteCommand

Related