Large documents in redis: is it worth compressing them (Part 2)

2 minute read

In a previous post, we measured that compressing large JSON documents before sending them to redis was faster than sending them as is. I made the measurement on my own computer and a local redis database.

Now that the principle has been validated, I needed to know if this result could be replicated on an environment that is closer to the production environment we have at Forest Admin.

On a server, I ran the same benchmark:

  • Server performance is the same as in production (except for the load induced by other requests)
  • The redis server is comparable to the one that is used in production.

⬇️ Download speed comparison

In this first graph, we will compare the download performance from redis + decompression time of the same JSON documents, using 3 different methods:

  • Uncompressed JSON document
  • Compressed brotli-1
  • Compressed gzip-3
  • Compressed deflate-3

These algorithms appeared to be the fastest in their family during my first tests. This second test on a production-like environment revealed the same, that’s why I decided not to publish the same level of info as the last time.

Overall download duration with the different methods

Performances are really similar to documents smaller than 4 MB, but the difference starts to be significative after this value.

For larger documents, all compression methods have similar performance regarding the time it takes to download and decompress them.

⬆️ Upload speed comparison

The same protocol has been applied, for all algorithms. As for the download performance comparison, we will compare here the same challengers that have been selected in the previous test.

Overall upload duration with the different methods

When writing JSON documents larger than 4 MB, results show that it is worth compressing them. For documents with a size of 10 MB, the overall compression and upload time can almost be divided by 2 with this method, from less than 300 ms to 150 ms with brotli-1.

🏅 Brotli-1: the winner in production-like environment

This second test shows that the algorithm brotli-1 makes a significant difference regarding the upload and download performance of documents larger than 4 MB.

At Forest Admin, we need to store some JSON documents that are larger than 4 MB, and we will definitely try this solution in production, using a canary deployment.

The space saving ratio of the brotli-1 algorithm has been measured to be more than 90%, on the types of documents that we are storing. So, in addition to allowing faster data transfer, this solution will also save a lot of space on our redis instances.

⚠ Be sure to test the algorithm on a production-like environment

Be careful before using this solution in your own environment, because results can vary a lot with:

  • the type of document
  • the compression algorithm
  • the compression level

For instance, brotli with the default compression level is very effective in terms of space saving, but also very slow during the compression phase.

When testing in a production-like environment, some algorithms have also been measured as slower than the original solution, which consists of sending the plain JSON documents, as you can see below.

Overall download duration with deflate

Compressing documents with deflate and a compression level of 0 was faster than using no compression on my laptop, but became slower on a real server.

Comments

matthewes3

Free Porn Pictures and Best HD Sex Photos http://southwilliamsoneuroupeanporn.moesexy.com/?renee wifi itouch porn free pamela porn nude pics mobile porn tube videos femdom hot lesbian free porn videos calvin klein kiddie porn

brianaub18

Hot galleries, thousands new daily. http://cfnm.porn.nanakuli.amandahot.com/?edith slut porn videos number 1 porn site long fucking porn become a bondage porn star free lesbian ipod touch porn

ernestju16

Hot teen pics http://bremond.voyeur.porn.moesexy.com/?annie free porn clips no download free clips female bodybuilders fucking porn porn star models gruesome porn tube photos of pakistani female porn stars

robertowj18

New super hot photo galleries, daily updated collections http://enemapoopporn.willisburg.moesexy.com/?tayler porn monk marina sharapova porn star fucking woman and burglar porn porn wank psps porn

melvinyy1

Hot sexy porn projects, daily updates http://twinsburg.alexysexy.com/?rocio porn no credit card info april ashley porn videos porn in night vision porn hub lela starr porn hub old men

maryloufg11

Hot sexy porn projects, daily updates http://chino.applebuttporn.moesexy.com/?aylin porn swollowing japaneese schoolgirl porn gameshow vietnam porn malayalam porn movies online porn with anal play

darcymp69

Free Porn Pictures and Best HD Sex Photos http://sibley.handjob.porn.miaxxx.com/?kierra young top porn gay cartoon avatar porn girls feet porn porn to watch now free homemade animal porn clips

morgannk60

Girls of Desire: All babes in one place, crazy, art http://jonestown.dating.kanakox.com/?corinne porn 20008 q porn young blondes fucking jessica lynn porn video extreme anal asian porn free porn celebrity pass

francinecs4

Hardcore Galleries with hot Hardcore photos http://pornhippoverndale.moesexy.com/?riley disney prince porn lg ks360 porn videos porn vids xanga comic porn hentia forcefull lesbians porn

Leave a Comment

Your email address will not be published. Required fields are marked *

Loading...