How Meta deals with high volume of data and what your start-up NEEDS to learn from it!

 

In our first blog article, we spoke about the importance of big data for start-up businesses and different types of free big data software that entrepreneurs can use to process large amounts of complicated data (Fischer, 2020). This data is becoming larger and larger as revealed in a study conducted by Lyman and Varian which revealed that data stored in digital media devices at the time of the study as compared to data from 2002 is up 92% while the size of the data itself is calculated to be five exabytes (Tsai et al, 2015). The problem is as revealed in this study is that data is so large that it is difficult for those analysing it to find what they are looking for. Tradional methods of analysing data like sampling for example may have been useful in the past but as shown by the Lyman and Varians study, the volume of data is so large that tradional methods will no longer do the trick. The three Vs (volume, velocity and variety) would then be developed by Doug Laney as a means to define what big data actually is (Kitchin and McArdle, 2016). 

1. Volume is defined as the large size of the data. 
2. Velocity defines how the data is created in real time. 
3. Variety defines the structure of the data, if there is a structure at all.

In today's instalment of Start-ups on a budget, we will look at social media network Meta (previously known as Meta) and how they utilised big data software to deal with their large volume of data. Meta was once a start-up company just like you are now so take notes as one day your business may achieve similar success and will need ways to analyse large forms of data.

As Meta grew bigger and bigger it became evident that tradional ways of analysing data would no longer be applicable to them. Let us look at  how large Meta was in 2014. In 2014, Meta had processed data that consisted of 10 billion messeages, 4.5 billion "like" actions and 350 billion photo uploads per day (Marr, 2014). It is also thought that Meta in 2016 had 350 billion images (Optalix, 2022). These numbers are huge which shows the large volumes of data companies like Meta are now faced with which is why tradional forms of data no longer work. This data is so large in fact that it could no longer be held in a computer but in cloud devices. We are talking about terabytes and petabytes of data (Hairiri et al, 2019). 

So what big data softwares does Meta use to analyse these terabytes and petabytes of data? Meta would go on to create there own platform titled Scuba as a means to manage data (Hewage et al, 2018, p. 94).

Scuba is an effective platform for Meta as it can process large data very quickly. Scuba has the ability to process millions of rows of data per second while also disposing of data that is not useful (Hewage et al, 2018, p. 94). Scuba processes data from every user which is very impressive considering the fact that there is nearly a billion users on Meta so can you imagine how many quiries a day this software deals with (Abraham et al, 2013). At the time of this article in 2018, Meta owned more then hundreds of servers with the capacity of 144 GB Ram (Hewage et al, 2018, p. 94). The data management tool Scuba enabled this data to be stored in the memory of these servers (Hewage et al, 2018, p. 94).  

This is the part of my look into Scuba I found most interesting that it stores data in the memory of the users. As a user of Meta myself I was ofcourse very interested. After some digging I learned how this actually effects the user. So have you ever received a notification about a memory you have as it relates to an old post? Maybe its an anniversary with a friend or a birthday. Sometimes you may even receive a notification about a video that Meta created for you highlighting these. Well this is Meta using Scuba to send you these notifcations (Rangaiah, 2021).

Let us take a further look into some use cases to understand why Scuba is so effective for Meta. 

Performance montoring as mentioned previously is one of Scuba's biggest attributes but how exactly does the software do this. The scuba dashboard shows the operator metrics about the user like for example CPU load on servers amongst others (Abraham et al, 2013). All these metrics are displayed on graphs to give the operator a visual representation and the graphs update frequentuly in real time so can be compared. A use of this can be if there is a bug due to a faulty code. the operator can go through different colmns until they find the problematic block of code and then fill out the bug report (Abraham et al, 2013).

The performance monitoring tool that Scuba provides Meta is fantastic. As stated previously Meta due to its high amount of users is creating a high volume of data a day. In the early stages of your business you will be able to monitor the performance of your website manuelly since you wont have a high mount of data however as your start-up matures and grows more data will be present to you. Meta was once a start-up just like your company is now so bear that in mind and take notes. A big data software like Scuba could be exactly what your business needs when you do reach this mature stage and manuel analysation of raw data like this is no longer effective.

Another use case if that of trend analysis. The operator can identify spikes in word freuqncies used by users. This data can come from a range of countries, ages and genders and what is even more impressive is that the data can be generated in seconds (Abraham et al, 2013).

Trend analysis is ofcourse important to any business regardless of what field you are in. Discovering what your database is talking about is effective as you can then tailor whatever content you create towards that database. It is something I believe start-ups struggle with when it comes to gaining exposure. Making sure your content is search engine optimised to its highest degree is very important as you want to ensure the content you create can be seen by the most amount of people possible. This is why a trend analysis tool as Scuba provides to Meta is very important. 

So there we have it. The social media network Meta is growing everyday which makes it more difficult to define what data is effective since there is such a high abundance of it. This is why Meta developed Scuba which as outlined above helps them monitor the performance of the site along with analysing trends from there users amongst others. What is most impressive is that all of this data is analysed in mere seconds after being created and displayed on graphs which allows Meta to resolve problems with the site as quickly as possible.  Now as a Start-up you, of course, do not have the funds needed to create your own big data tool like Meta did with Scuba howver do not worry as your friends at Start-ups on a Budget have you covered. Click on the link below to see our first blog which gives you links to free big data softwares you can use now to improve your business. 

https://startupsonabudget.blogspot.com/2022/02/the-importance-of-big-data-and-why-all.html?fbclid=IwAR2IYlyTiN3e4uSi-4EeyL-idEfgtgaH1FIo2mZYdQ6N3r0fTTUujbmnfBQ

Thanks for reading and follow our social media accounts below to keep up to date with the next release of our weekly blog which helps you digitally market your start up business.

Facebook Link - https://www.facebook.com/Marketing-Trends-for-Start-Ups-on-a-Budget-109366131666054

Instagram Link - https://www.instagram.com/marketingtrendsforstartups/

YouTube Link -  https://www.youtube.com/channel/UCf33jKkYwiplFKNxVvSLS5w 

#bigdata #bigdatavolume #Meta #Facebook  #freebigdatatools #start-up #start-upbusinesses #Startuper #thestartup #upstart 

Author is Jason Flood

References

Abraham, L. Borkar, V. Merl, D. Subramanian, S. Allen, J. Chopra, B. Metzler, J. Wiener, J. Barykin, O. Chopra, B. Metzler, J. Wiener, J. Barykin, O. Gerea, C. Reiss, D. Zed, O. 'Scuba: Diving into Data at Facebook'. VIDB. Available at: http://www.vldb.org/pvldb/vol6/p1057-wiener.pdf (Accessed: 24th February 2022).

Fischer, F. (2020) ‘What is your definition of Big Data? Researchers’ understanding of the phenomenon of the decade. A journal article at the National Library of Medicine. National Library of Medicine, 15(2). Available at doi: 10.1371/journal.pone.0228987 (Accessed: 08 February 2022). 

Hariri, R.H., Fredericks, E.M. & Bowers, K.M. Uncertainty in big data analytics: survey, opportunities, and challenges. J Big Data 6, 44 (2019). https://doi.org/10.1186/s40537-019-0206-3 

Kitchin, R. Mc Ardle G. (2016) 'What makes Big Data, Big Data? Exploring the ontological characteristics of 26 datasets", Maynooth University, 3(1).

Marr B (2014) Big data: The 5 vs everyone must know. March 6. Available at: https://www.linkedin.com/pulse/20140306073407-64875646-big-data-the-5-vs-everyone-must-know (accessed 4 September 2015). 

Optalix (2022) 'What are the 3 Vs of big data? Available at: https://www.optalitix.com/guides/what-are-the-3-vs-of-big-data/ (Accessed:23rd January 2020) 

Rangaiah, M. (2021) 'How Facebook uses Big Data to enhance customer experience'. Available at: https://www.analyticssteps.com/blogs/how-facebook-uses-big-data-enhance-customer-experience (Accessed: 03th February 2021).

Tsai, CW., Lai, CF., Chao, HC. et al. Big data analytics: a survey. Journal of Big Data 2, 21 (2015). https://doi.org/10.1186/s40537-015-0030-3