Farewell SFTP, Hello Gitea

Yesterday we announced that we’ll discontinue using SFTP by the end of April. For those of you who didn’t catch the news, we’ve replaced SFTP with a simple and straightforward web app based on Gitea, a solid open-source project with a vibrant community.

To some of you, this might sound like just another update. But in the lines that follow, you’ll come closer to understanding the dynamics of this change. We’ve summarised the history of SFTP, and how the evolution of web standards has left the protocol behind, and the reasons behind our pursuit of a new, better alternative. Next, you’ll find the apps we gave a try as replacements, with Gitea the absolute winner. And finally, we’ll tell you more about the process of introducing Gitea, and the benefits it presents.

#The rise and fall of SFTP

If we want to talk about SFTP, we absolutely have to start with FTP. File Transfer Protocol is the father of all protocols, being developed in the early ’70s, way before the TCP and IP were born. As the name implies, it allows file and data transfers between a computer and a server over a network connection (the client-server model).

At the time when FTP was developed, security wasn’t envisioned in the form we know today, let alone the complex security challenges we’ve come to meet—thus, drawbacks such as data transparency and an increased vulnerability to attacks were pretty common. The solution to this problem was to fabricate a protocol of data transfer made through a secure connection. Fast-forward 30 years and the SFTP was conceived. Of course, in computer-era years, this means ages.

SFTP, short for SSH File Transport Protocol, is a network protocol that provides a trio of file access, transfer, and management between a client and a server, done via a secure connection, with an emphasis on security. It was developed as a SSH Protocol extension by the Internet Engineering Task Force, IETF in short, over a time span of almost 10 years. Note that the secure connection is done through the SSH protocol. Moreover, it requires an encrypted public key authentication, thus “sealing” the transfer. Another perk is that it’s packet-based, making transfers faster, as opposed to FTP which is text-based.

SFTP has been in use for about two decades, a period in which the depth and complexity of the web environment have evolved at a fast pace. While its popularity peaked, it has also addressed the need for a secure and fast transfer connection. Nowadays, we see some cracks in the giant SFTP, as its capabilities don’t support the present web standards.

Back in 2011, our team started using SFTP since day one because it was the web-wide standard. Everybody knew how to use it, and it was the easier alternative for changing or updating source code, compared to git-driven development, which is more robust and consequently it would’ve been harder for our clients to understand and use.

Usability was paramount to us so we’ve had to create two components which we open-sourced – a Git-based file system – GitFS and a Git WordPress plugin – Gitium, specially adapted to WordPress needs. These allowed us to reconcile beginner developers that were first-time SFTP users, with experienced developers using Git. We also needed to make sure ninja-style edits from the WordPress dashboard all worked on the same code base and also reduced reconciliation and file management effort and redundancy.

#SFTP Disadvantages

Seeing that SFTP is losing popularity, we compiled a list of why this decrease is happening. These are some of the disadvantages of using SFTP:

• Not scalable and hard to manage, as the underlying SSH wasn’t designed to fit the upcoming changes
• Lacking a versioning system, with overrides occurring when two or more people are working at the same time (this problem was solved, at a high engineering cost, in the form of GitFS and Gitium)
• Difficult to use on the go, either on tablet or mobile – to this day, we still haven’t found a cross-platform SFTP app to recommend light-heartedly to customers
• Unreliable, due to networking glitches that can happen during file edits – this has been a source of subtle errors that need hard debugging work from everyone.

So, after years and years of being in a love-hate relationship with SFTP, it was time:

We started looking for alternatives. Web editing apps used in cPanel-style dashboards are not really much better without a powerful versioning platform behind, so those bare apps have been excluded from the beginning. But there was other interesting stuff waiting for us.

#The new kids on the block: Gitolite, Gogs, Gitea

The web keeps evolving at a fast pace, and new players with it show up every day. Since 2005 when Git was first introduced, the development game went through a 180 degrees change. It’s no wonder the creator of the Linux kernel is the key person in the development of Git.

Git introduced a distributed model of managing projects in a simple and efficient way. When the biggest fish in the sea like Google, Facebook and Twitter are using it, then you know you should, too. The benefits of using Git are infinite, ranging from a flexible workflow, to data integrity and security.

Github does a heck of a job in hosting a variety of code from individuals and companies, but when you’re a managed hosting service provider, you surely want to room the code on your servers.

Nowadays, the market is booming with self-hosted Git repository managers—Gitolite, Gogs, Gitea, Phabricator, Bitbucket Server, Github Enterprise, RhodeCode just to name a few. Trying to find the perfect fit for us, we set out to explore Gitlab, Gitolite, Gogs, and Gitea.

#Gitlab drawbacks

Back in 2014, we dived into Gitlab. We found some drawbacks which gave us second thoughts:

• The sheer size of it was mind-numbing
• Closed community
• It was written in Ruby, which for Python lovers like us, it just wasn’t appealing
• It wasn’t scalable
• It consumed too many resources.

#Our fair share of Gitolite

Next, we turned our attention to Gitolite. Being a lighter version of Gitlab, we gave it a try and stuck with it until recently.

We used Gitolite for a series of benefits which, at the time, made all the difference:

• Built-in web interface for admins
• Easy to configure using one git repo
• Built-in replication via git mirroring.

We hosted it on a subdomain, which didn’t interfere with our core activities. Around it, we had to develop pyolite in order to create users and manage repos easily.

Nonetheless, during the 4 years of using it, we also found some weak points:

• It’s hard to maintain the configuration, as you can’t automate management
• You can’t display it for all your users
• It’s hard to configure and maintain HTTP/HTTPS clone option
• It lacks project management elements like issues, branches, and commits.
• It doesn’t have a code editor

Once we decided on switching our entire infrastructure on Kubernetes, we knew that Gitolite wasn’t enough, and we started searching again.

#Our best choice, Gitea

We were buzzing to research Gitea, because of its open community and continuous updates. Gitea is actually a fork of Gogs. We had a look at Gogs, too, but its divided community and reluctance to accept contributions from other developers made it a no-go for us.

Gitea provided the perfect answers to our questions, so we picked Gitea for our next journey. All the WordPress websites that we host now run on Gitea, a move which was, much to our surprise, very smooth.

Working with Gitea and using it, we found a series of much-requested benefits:

• Easy to manage repos
• Uses HTTP / HTTPS, Git and SSH protocols
• Periodically updated
• Issue management
• Time track for code
• Visual, simple to use
• Encourages collaboration by integrating pull requests and branches
• A vibrant community.

#How Gitea works with our platform

Once a site is added on our dashboard, a repo is automatically created for it. The dashboard app clones the repo and gets the code from the standard site file layout – Gitium base, which is a public repo on GitHub, and all our instances start from there.

The code gets pushed to the repo on Gitea, and deploy hooks are added (for every push on Gitea, a deploy on wp-admin and purge assets cache are made). A deploy key is created, which is a unique key for each instance, so that every change from wp-admin can get on Gitea.

All the collaborators from our dashboard are added in Gitea. An oxygen-bot collaborator will be also added, in order for our team to have access to the users’ code.

In order to access a repo, a user has to add his/her SSH key on our dashboard, and use HTTPS to clone the repo by introducing his/her Presslabs account credentials.

#What’s next?

We contributed with pull requests on the Gitea project and Gitea Go SDK, and want to continue doing so, in order to add edit support, multiple files uploads, folder removal, and allow admins to clone any repos. On our part, we look forward to hearing the feedback and improve.

Integrating Gitea in our platform was a decision that we made with our users in mind, to help them get a better visual overview of the source code and better handle changes made by their teams.

For more information regarding the usage of Gitea, please check our docs page, or drop us a line at ping@presslabs.com.