{"pages_text_array":["Introduction to Big DataLearning ObjectivesAt the end of this text, you should present the  following learnings: Define big data.Discuss the Vs of big data and implications.Point out the  types of data related to big data.IntroductionSince the beginning, man  has stored data for  himself and for others, through drawings on the rocks and rock art. This record was made with  the aim of making some decision or enabling access to knowledge. As societies became more  complex, the volume of data storage This led to the  construction of libraries and the later  invention of printing by Johannes Gutenberg around 1450. The abacus itself, a mechanical  instrument of Chinese origin created in the 5th century BC, stored information about numbers  and helped with computing. Later,  the emergence of the internet for information exchange,  during World War II and the Cold War (1945\u01521991), made it even more necessary data  storage for further analysis. Over time, various ways of storing this information were  developed: mainframes, floppy  disks, tapes, hard drives, NAS (Network -Attached Storage),  cluster environment, pen drives, CDs, DVDs. In modern society, data began to be produced  from different sources, whether in social networks (photos, videos, messages), in online  purchases, in deli very applications, in distance education courses, in transactions with  currencies and digital banks. In addition, there was the replacement of roles, such as physical  agenda, medical records, request for exams, for the digital context.       In this sense, companies realized the value of storing and processing strategic data. Thus, the  new power race became clear: data started to be seen as the new oil. As a result, we can  observe a gradual growth in the production and storage of data througho ut history, until we  reach the context of big data. In this chapter, you will study the concept and characteristics  inherent to big data. the main types of data that are related to context.1 The data society and  what defines big dataModern culture started to produce and store more data. With a  computer or a smartphone in our hands, we now have access to a greater volume of  information. Thus, the massive growth of sending photos, videos, audio and text messages  made the social relationship become digital. It  was in this scenario that the concept of big data  emerged. According to Mauro, Greco and Grimaldi (2015, online document, our translation),  big data is defined as follows: \ufb01Big Data represents information assets characterized by high  volume, speed and var iety, which require specific technologies and analysis methods to be  transformed in value [...]\ufb02.From the growth of hundreds of Terabytes of data, the context of  big data began to be systematized.The definition of this term is based on five principles: spe ed,  volume, variety, veracity and value .You will see that such principles always go together in this  context. Thus, big data is a broad term that deals with several areas and composes the various  related studies. In the academic area, departments were cre ated focused on engineering and  data science. , in order to compose the sets of knowledge and studies that the area demanded.  Soon, several professions related to this area also emerged. The data engineer deals with  acquisition, storage and disposal strate gies. level of data. The data scientist and the machine  learning engineer (in English, machine learning) make up the context of exploratory analysis,  pattern recognition and predictive analysis, as well as other related contexts. similar to  software engine ering DevOps, but focused on the context of the data.Introduction to big data2      ","The exponential production of data with the internet of thingsThe internet of things has  emerged with remarkable potential, causing the context of connected devices to exponen tially  increase agricultural production produces voluminous data every second, with monitoring in  the chicken coop, monitoring the temperature and ambient humidity, among others. As a  result, a large volume of data is produced. The production of refrigerat ors, air conditioners,  fans, electric pans and other connected devices made daily life permeated by the internet of  things .Thus, with so much data produced, it is necessary to organize a storage and processing  structure for decision making. The term \ufb01inte rnet of things\ufb02 refers to the interconnection of  intelligent devices, which produce, consume and transmit data. make up the context, in  addition to several boards and embedded systems.The production of data by peopleThe  biggest producers of given Away     s in the world are human beings themselves. Previously, each could only create small notes for  themselves or within a small group. Now, we have a massive online file sharing environment at  our disposal. As we walk, we produce data through of our GPS positions, which are transmitted  in real time via applications. Our speech produces data that is analyzed by virtual assistants. If  we are hospitalized, our breathing will produce data, through sensors, for the medical record.  the use of social networks is increasing , generating immense amounts of data. The sending of  daily e -mails with advertisements and for closing deals, the allocation of photos of travel in the  cloud and several other situations of our daily life generate data, in tremendous volume and  speed.So, i n the age of big data, to live is to produce data.3Introduction to big data       The production of public data by governmentsGovernments also produce a tremendous range  of data, on the most diverse fronts: health, infrastructure, transport, education, tourism ,  economy, bids, contracts, among others. Federal Government website and are commonly  consumed by entities that In addition, the market seeks to carry out, from this data, various  predictive analyses. On the other hand, governments also use each other's da ta, 2 The Vs of  big data and its impact on technologies and society Big data has changed the way companies  see their data. Currently, each piece of information about their own business and customer  has become crucial in decision making. In the academic con text, more and more processing  and data analysis. In this sense, the characteristics of big data and its five Vs showed the  systematization of the context, offering a vision of how studies and technological solutions  should be. those proposed for the area.  See below for more information on each of these  aspects.Volume The reference to the size of the data produced and the need to store it  encompasses the volume of big data. Currently, we are not talking about Terabytes, but about  Zettabytes or of Brotonbyte s.Speed The speed in data production can be seen, for example,  from the perspective of social networks, where we have millions of messages exchanged per  minute.Imagine that a million people sent 10 messages only in the morning, that is, in the first  six hours of your day. In that case, we would already have Introduction to big data4      ","10 million pieces of data to be stored. The reality, however, is much greater. The production of  data is fast, whether in monitoring, through sensors, or in the data that pe ople themselves  produce. VarietyThe multiplicity of file types within of big data is, in fact, a punctual  characteristic. When we started to produce, mostly, digital data, we transformed physical tasks  into online data. This data can be agendas, purchase o rders and deliveries, sending text  messages , audio, video and image. This variety can be composed and stored, for example, in  the HDFS file structure of Apache Hadoop, and managed by its various services, such as Hive,  Hbase, Spark, among others. Veracity The composition of the veracity of the data in big data is  a characteristic part of data quality and continuous improvement. We cannot use data that do  not represent the problem or that have a bias. In this context, data science deals with cleaning  and org anizing the data, in order to increase the framework for the context of data quality  comes from The Dama guide to the data management body of knowledge that help in data  governance.ValorThe first step that occurred in big data was the need to store the dat a, for  only later see what to do with them. This was because it was realized that, with the rise of  predictive analytics, having a lot of data about a given context could be invaluable. .The use of  data for companies was already used in business intelligen ce, which became known for the  theories of the data warehouse and the respective specificities, with techniques to create data  structures and enter the dashboards. However, predictive analysis has gained immense  importance , since everyone wants to predict  the future based on several variables in a  context.5Introduction to big data       Other VsAccording to Taleb, Serhani and Dssouli (2019), there are still other Vs involved. Some  of them are variability, which consists of the constant change of security issue s. Data ingestion  and storage in Apache HadoopData can take different forms. structured as spreadsheets, in  ERP systems, they can be semi -structured or unstructured, like data from social networks, or  they can come from a network of wireless sensors that p roduce information such as  temperature, humidity or pressure (Figure 1).Figure 1.The various data types of  "],"pdf_complete_text":"Introduction to Big DataLearning ObjectivesAt the end of this text, you should present the  following learnings: Define big data.Discuss the Vs of big data and implications.Point out the  types of data related to big data.IntroductionSince the beginning, man  has stored data for  himself and for others, through drawings on the rocks and rock art. This record was made with  the aim of making some decision or enabling access to knowledge. As societies became more ...