At a virtual presentation streamed worldwide, the Nextcloud team introduced the availability of Nextcloud Hub 21.
Nextcloud Hub 21 brings a wide range of improvements in file handling performance with Files High Performance Back-end and important new collaboration features in Text, Talk, Groupware and Files. In this blog, we go into a bit more detail of the performance improvements.
For Nextcloud 21, we wanted to do something that benefits all users: make it faster.
To increase the capacity of especially large installations and keep Nextcloud snappy, our team profiled and optimized various core areas of Nextcloud. We made all kinds of improvements and you can watch our release video above to get some more details.
With the COVID crisis continuing to force employees to work from home, Nextcloud server administrators need to keep adding capacity. Nextcloud 21 will reduce the load on servers, enabling more employees to work without requiring additional hardware. At the same time, the new collaboration features improve the day to day productivity and support team members in getting more work done in less time.
We did a number of improvements that reduce the amount of code loaded on each page, for example handling the translations and theming more efficiently. We also worked hard to optimize our use of the database, as this is an important limit for large deployments. Many common database queries were reduced or even outright eliminated. The move to PHP 8 will also give a nice performance boost for systems that have it installed!
Besides this generic performance work, we also looked at some specific scenarios where servers would run into performance limitations. Nextcloud Text, widely used in many organizations for note taking and basic writing, creates load on the server by checking in for changes constantly. We reduced this load by 25% and introduced a ‘idle mode’ where a document not used for 30 minutes stops polling the server. We also optimized object storage handling, made loading the trash view twice as fast, optimized LDAP, installation and upgrading, the app store and much more.
High Performance Back-end 🚀
The performance improvements above should impact nearly every server, giving a better user experience and enabling system administrators to grow the user base without having to add more hardware. But there is another, big change that we have been working on, and it requires a little background.
Nextcloud offers near-perfect scalability
By using the traditional LAMP stack with Linux, Apache, Mysql and PHP, Nextcloud builds on some of the most popular technologies on the web. This has many benefits. Our platform is easy to deploy and maintain and scales extremely well. The scaling, in particular, is a huge benefit of PHP over practically every other technology out there: to double the performance of a PHP server, you can simply have twice as many cores and things will work without any changes needed. Not only is the scaling you get pretty much 100%, unlike other languages, this scaling continues beyond a single server! You can put a second server next to it and let it handle requests. This is possible because PHP handles each request entirely independently from every other request. The result is that it is easy to create a cluster that offers Nextcloud to hundreds of thousands of users on a single instance.
The scalability of Nextcloud itself is thus near-infinite, but file systems and databases are not. The improvements we did in this release will help to scale further, but inevitably they will become the limiting factor. That is why we invented Global Scale! This innovation allows multi-continental deployments with hundreds of millions of users. We already deployed the first Global Scale setup in early 2018 and our largest customers use it with tens of millions of users today. Scalability is certainly not something Nextcloud lacks!
Taking advantage of new technologies
But every technical choice has two sides to the coin, and some emerging technologies on the internet provide other, unique benefits. To make sure our users always get the best of both worlds, Nextcloud 21 introduces a new, completely optional component: our High Performance Back-end (HPB) for Files.
The new HPB is a binary written in Rust and enables Nextcloud to keep a permanent connection open to our clients. So what impact does this have?
Faster notifications and syncing
First, you no longer have to wait! Currently, the client checks every 30 seconds to see if there have been changes on the server. With the open connection, the server notifies the client of changes and it can begin downloading new files right away. So when a colleague asks “did you get the file I just shared”, you no longer have to say: “wait, it just started syncing!”
This does not only impact file syncing, however. It also means you get instantly notified of new chat messages in Talk or an incoming call. New files shared with you. A colleague commenting on a file. A Deck card running into its deadline. And much more! In many of these cases, the faster response time makes a real difference.
90% less load from polling
Server administrators will also be delighted. One check every 30 seconds equates to 2880 times per day. We reduced this number by 90%, only checking in once per five minutes.
To give you an idea of the impact of this change, we did an analysis of traffic at the German DFN Cloud, with more than 3 dozen educational and research institutes managed by the TU Berlin. In a 15 minute period, more than 250.000 requests are made by desktop clients just asking if there were any new files to download! This accounted for over 2/3rd of their network connections. You can imagine the impact of reducing that traffic by 90%. And that is just the desktop client – we also implemented this in the web interface, which also checks every 30 seconds! A user with 3 tabs open is polling the server 3 times every 30 seconds, 10 tabs means a ping every 3 seconds, all day long. You can imagine what impact it can have to reduce this by 90%.
At a scale of millions of users, milliseconds start to add up. After analyzing application and caching server, storage and database behavior, our team was able to significantly reduce the impact of common operations. The High Performance Back-end, on the other hand, provides a completely new way of reducing server load while bringing a new level of responsiveness to users.
— Roeland Douma, Server Engineering and Support Lead
The HPB for Files requires a reverse-proxy setup and Redis, among other pieces of infrastructure. You can find instructions here.
“When we have welcoming communities of contributors, open source software gets better and more useful to everyone.” Limor Fried, Electrical Engineer, Inventor and Founder of open-source hardware company Adafruit We believe in this ideal and love to work with our community. We are always looking to involve more people in Nextcloud, bringing in their ideas, […]
“Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat […]
Over the last years Nextcloud Talk has developed in a fantastic productivity tool, enabling teams across the globe to communicate and collaborate in chat rooms, video meetings and webinars.
Hot on the heels of Nextcloud Hub 4, our desktop client now enables users who are running the latest Nextcloud to take advantage of its improved End-to-end encryption features!
After a complaint filed by Nextcloud on behalf of a coalition of dozens of European cloud tech providers in November 2021, the German Bundeskartellamt (federal antitrust authorities) has now begun an official investigation into Microsoft to assess if the company has a dominant position in the market.
Nextcloud users know the importance of integrating different systems and tools to create a seamless workflow. Nextcloud Enterprise allows you to integrate with Microsoft environments for file storage, user directory, Outlook, Sharepoint, Windows Desktop, MS Office online server, and Teams. And now, we are excited to announce a new addition to our lineup: the Nextcloud […]
Nextcloud, a leading provider of open-source collaboration software has partnered with epiKshare to deliver Nextcloud One — a fully managed, secure and compliant cloud solution hosted in Germany.
The Nextcloud Conference is not your average event - it's a community meetup that brings together Nextcloud enthusiasts, contributors, developers, users and industry experts from all over the world.
We save some cookies to count visitors and make the site easier to use. This doesn't leave our server and isn't to track you personally!
See our Privacy Policy for more information. Customize