Gitlab 8.0 behind reverse proxy: fix failed git-http-push

23.9.2015

With the release of Gitlab 8.0, the way how git push/pull/... events via http(s) are handled has changed. A new gitlab-git-http-server process has been introduced that by default listens on port 8181. If you run gitlab behind a reverse proxy and/or are not using the nginx reverse proxy that runs in a default gitlab-ce installation, this might cause trouble and require you to change the gitlab and reverse proxy configurations (as it did in my case). Problem and solution are documented here:

Errors you get

When pushing to remote, I got:

$ git push origin master             
Username for 'https://gitlab.points-of-interest.cc': USERNAME
Password for 'https://USERNAME@gitlab.points-of-interest.cc': 
error: Cannot access URL https://gitlab.points-of-interest.cc/points-of-interest/testor.git/, return code 22
fatal: git-http-push failed

When pulling, you get:

$ git pull origin master             
Username for 'https://gitlab.points-of-interest.cc': USERNAME
Password for 'https://USERNAME@gitlab.points-of-interest.cc': 
fatal: Couldn't find remote ref master
Unexpected end of command stream

Also, when cloning, I received no error message, but an empty repository, in which I know is data on the remote.

How to solve

The problem was that the new gitlab-git-http-server process handling these requests has to forwarded as well by your reverse proxy, and, depending on your configuration, also has to be configured to listen on the right port.

Changes to the reverse proxy

Gitlab provides a recipe for Apache as reverse proxies and you see that this has changed from 7.* to 8:

Apache:

$ diff gitlab-ssl-apache2.4.conf gitlab-8.0-ssl-apache2.4.conf
47a48,50
>       #Allow forwading to gitlab-git-http-server
>       ProxyPassReverse http://127.0.0.1:8181
>       #Allow forwading to GitLab Rails app (Unicorn) 
55a59,63
>   #Forwad request ending with .git to gitlab-git-http-server
>   RewriteCond %{REQUEST_URI} .*\.(git)
>   RewriteRule .* http://127.0.0.1:8181%{REQUEST_URI} [P,QSA]
> 
>   #Forwad forward any other requests to GitLab Rails app (Unicorn)

For nginx, there is a solution in a comment on the respective issue on gitlab.com. This, however suggest, unix sockets, which did not work in my configuration, hence I propose a tcp solution: Add to your nginx configuration:

    location ~ [-\/\w\.]+\.git\/ {

      proxy_read_timeout      300;
      proxy_connect_timeout   300;
      proxy_redirect          off;
      proxy_buffering off;

      proxy_set_header    Host                $http_host;
      proxy_set_header    X-Real-IP           $remote_addr;
      proxy_set_header    X-Forwarded-For     $proxy_add_x_forwarded_for;
      proxy_set_header    X-Forwarded-Proto   $scheme;

     proxy_pass http://localhost:8181;
    }

Changes to the gitlab.rb

For both, the nginx and apache situation, you need to make sure that gitlab-git-http-server runs on port 8181 and not the unix sockets. This can be done using your gitlab.rb to configure gitlab as documented here. Adding these lines should work:

gitlab_git_http_server['enable'] = true
gitlab_git_http_server['ha'] = false
gitlab_git_http_server['repo_root'] = "/var/opt/gitlab/git-data/repositories"
gitlab_git_http_server['listen_network'] = "tcp"
gitlab_git_http_server['listen_addr'] = "127.0.0.1:8181"

After that, a gitlab-ctl reconfigure and restart of your proxy should help.