Nginx -Gitlab: could not clone over https

user_6565711 0
Viewed 1182

I starting setting up a gitlab-runner, before I only try to clone, pull, push, etc. over ssh. With ssh it was no problem, so I think it is a problem with nginx. I try some settings in nginx, but not clearly sure what will be need. Do anybody know what to set, to get data? Website is also running fine.

The nginx output while cloning git repo ci https:

        172.17.0.1 - - [20/Jul/2017:21:13:39 +0000] "GET /server/nginx.git/info/refs?service=git-upload-pack HTTP/1.1" 401 26 "-" "git/2.7.4"
    172.17.0.1 - user [20/Jul/2017:21:13:39 +0000] "GET /server/nginx.git/info/refs?service=git-upload-pack HTTP/1.1" 401 26 "-" "git/2.7.4"
172.17.0.1 - - [20/Jul/2017:21:13:42 +0000] "POST /heartbeat HTTP/1.1" 200 5 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0"
    172.17.0.1 - - [20/Jul/2017:21:13:46 +0000] "GET /ocs/v2.php/apps/notifications/api/v2/notifications HTTP/1.1" 200 74 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:54.0) Gecko/20100101 Firefox/54.0"
    172.17.0.1 - user [20/Jul/2017:21:13:47 +0000] "GET /server/nginx.git/info/refs?service=git-upload-pack HTTP/1.1" 200 415 "-" "git/2.7.4"
    172.17.0.1 - user [20/Jul/2017:21:13:47 +0000] "POST /server/nginx.git/git-upload-pack HTTP/1.1" 500 0 "-" "git/2.7.4"

git response:

error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500 Internal Server Error
fatal: The remote end hung up unexpectedly

git workhorse error message

    2017-07-22_11:19:45.43536 2017/07/22 11:19:45 error: POST "/server/nginx.git/git-upload-pack": handleUploadPack: ReadAllTempfile: open /tmp/gitlab-workhorse-read-all-tempfile358528589: permission denied
2017-07-22_11:19:45.43551 git.dropanote.de 172.10.11.97:43758 - - [2017-07-22 11:19:45.349933226 +0000 UTC] "POST /server/nginx.git/git-upload-pack HTTP/1.1" 500 0 "" "git/2.7.4" 0.085399

nginx config

    ## GitLab
##
## Modified from nginx http version
## Modified from http://blog.phusion.nl/2012/04/21/tutorial-setting-up-gitlab-on-debian-6/
## Modified from https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
##
## Lines starting with two hashes (##) are comments with information.
## Lines starting with one hash (#) are configuration parameters that can be uncommented.
##
##################################
##        CONTRIBUTING          ##
##################################
##
## If you change this file in a Merge Request, please also create
## a Merge Request on https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests
##
###################################
##         configuration         ##
###################################
##
## See installation.md#using-https for additional HTTPS configuration details.

upstream gitlab-workhorse {
  server 172.10.11.66:8181;
  keepalive 32;
}

## Redirects all HTTP traffic to the HTTPS host
server {
  ## Either remove "default_server" from the listen line below,
  ## or delete the /etc/nginx/sites-enabled/default file. This will cause gitlab
  ## to be served if you visit any address that your server responds to, eg.
  ## the ip address of the server (http://x.x.x.x/)
  listen 0.0.0.0:80;
  listen [::]:80 ipv6only=on;
  server_name url.tdl; ## Replace this with something like gitlab.example.com
  server_tokens off; ## Don't show the nginx version number, a security best practice

    location /.well-known/acme-challenge {
        root /tmp;
    }

    location / {
        return 301 https://$http_host$request_uri;
        access_log  /var/log/nginx/gitlab_access.log;
        error_log   /var/log/nginx/gitlab_error.log;
    }
}

## HTTPS host
server {
  listen 0.0.0.0:443 ssl;
  listen [::]:443 ipv6only=on ssl;
  server_name url.tdl; ## Replace this with something like gitlab.example.com
  server_tokens off; ## Don't show the nginx version number, a security best practice
  root /opt/gitlab/embedded/service/gitlab-rails/public;



  ## Strong SSL Security
  ## https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html & https://cipherli.st/
  ssl on;
    ssl_certificate linkto/fullchain.pem;
    ssl_certificate_key linkto/privkey.pem;

    add_header Strict-Transport-Security "max-age=15768000; includeSubDomains" always;

  # GitLab needs backwards compatible ciphers to retain compatibility with Java IDEs
  ssl_ciphers "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4";
  ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
  ssl_prefer_server_ciphers on;
  ssl_session_cache shared:SSL:10m;
  ssl_session_timeout 5m;



  location / {
    client_max_body_size 0;
    gzip off;

    ## https://github.com/gitlabhq/gitlabhq/issues/694
    ## Some requests take more than 30 seconds.
    proxy_read_timeout      3000;
    proxy_connect_timeout   3000;
    proxy_redirect          off;

    proxy_http_version 1.1;

    proxy_set_header    Host                $http_host;
    proxy_set_header    X-Real-IP           $remote_addr;
    proxy_set_header    X-Forwarded-Ssl     on;
    proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
    proxy_set_header    X-Forwarded-Proto   $scheme;
    proxy_pass http://gitlab-workhorse;
  }
}
gitlab nginx
3 Answers

With Git 2.35 (Q1 2022), "git upload-pack"(man) (the other side of "git fetch"(man)) should be more robust.

It used a 8kB buffer but most of its payload came on 64kB "packets".
The buffer size has been enlarged so that such a packet fits.

This is a contribution directly from GitLab staff (Jacob Vosmaer).

See commit 55a9651 (14 Dec 2021) by Jacob Vosmaer (jacobvosmaer).
(Merged by Junio C Hamano -- gitster -- in commit d9fc3a9, 05 Jan 2022)

upload-pack.c: increase output buffer size

Signed-off-by: Jacob Vosmaer

When serving a fetch, git upload-pack(man) copies data from a git pack-objects(man) stdout pipe to its stdout.
This commit increases the size of the buffer used for that copying from 8192 to 65515, the maximum sideband-64k packet size.

Previously, this buffer was allocated on the stack.
Because the new buffer size is nearly 64KB, we switch this to a heap allocation.

On GitLab.com we use GitLab's pack-objects cache which does writes of 65515 bytes.
Because of the default 8KB buffer size, propagating these cache writes requires 8 pipe reads and 8 pipe writes from git-upload-pack, and 8 pipe reads from Gitaly (our Git RPC service).
If we increase the size of the buffer to the maximum Git packet size, we need only 1 pipe read and 1 pipe write in git-upload-pack, and 1 pipe read in Gitaly to transfer the same amount of data.
In benchmarks with a pure fetch and 100% cache hit rate workload we are seeing CPU utilization reductions of over 30%.

VonC_63090
Related