"Big data" usually refers to those data sets that are large in number, hard to collect, process, and analyze, and also those that are stored in the traditional infrastructure for a long period of time. The "big" here has several layers of meaning. It can describe the size of the organization, and more importantly, it defines the scale of the IT infrastructure in the enterprise. The industry places endless expectations on big data applications – the more value the business information accumulates – the more we just need a way to tap those values.
Perhaps people’s impressions of big data come mainly from the cheapness of storage capacity, but in fact, companies are creating a lot of data every day, and more and more, and people are trying to find valuable data from vast data. Business intelligence. On the other hand, the user will also save the data that has been analyzed, because the old data can be compared with the new data collected in the future, there are still potential use possibilities.
Why big data? Why is it now?
Compared with the past, we have to face more data types in addition to the ability to store more data. Sources of these data include online transactions, social networking activities, automatic sensors, mobile devices, and scientific instruments. In addition to those fixed sources of data production, various trading activities may also speed up the accumulation of data. For example, the explosive growth of social multimedia data stems from new online transactions and records. Data is always growing, but only the ability to store large amounts of data is not enough, because it does not guarantee that we can successfully search for commercial value.
Data is an important production factor
In the information age, data has become an important production factor, just like capital, labor, raw materials and other elements, and as a general demand, it is no longer limited to the application of certain special industries. Companies from all walks of life are collecting and using a large amount of data analysis results to reduce costs as much as possible, improve product quality, increase production efficiency, and create new products. For example, analyzing data collected directly from the product test site can help companies improve their designs. In addition, a company can also surpass its competitors by analyzing customer behavior in depth and comparing large amounts of market data.
Storage technology must keep up
With the explosive growth of big data applications, it has spawned its own unique architecture, but also directly promote the development of storage, networking and computing technology. After all, dealing with this special demand for big data is a new challenge. The development of hardware is ultimately driven by software requirements. For this example, it is clear that the demand for big data analytics applications is affecting the development of data storage infrastructure.
On the other hand, this change is not an opportunity for storage vendors and other IT infrastructure vendors. With the continuous increase in the amount of structured data and unstructured data, and the diversification of sources of analysis data, the design of the storage system has been unable to meet the needs of big data applications. Storage vendors have realized this and they have begun to modify the architectural design of block- and file-based storage systems to accommodate these new requirements. Here, we will discuss the properties related to big data storage infrastructure and see how they meet the challenge of big data.
Capacity issues The “big capacity†mentioned here can usually reach the scale of the PB data. Therefore, the massive data storage system must also have the corresponding level of expansion capability. At the same time, the expansion of the storage system must be simple, you can increase the capacity by adding modules or disk cabinets, or even without downtime. Based on this demand, customers now increasingly favor the storage of the Scale-out architecture. The characteristic of Scale-out cluster structure is that each node not only has a certain storage capacity, but also has internal data processing capabilities and interconnected devices. Compared with the chimney architecture of traditional storage systems, the Scale-out architecture can achieve seamless smoothing. Expansion to avoid storage silos.
In addition to the large scale of data, "big data" applications also mean having a large number of files. Therefore, how to manage the metadata accumulated in the file system layer is a difficult problem. If it is handled improperly, it will affect the system's scalability and performance, and traditional NAS systems have this bottleneck. Fortunately, the object-based storage architecture does not have this problem. It can manage billions of files in one system, and it will not suffer from metadata management as traditional storage. Object-based storage systems also have wide-area scalability capabilities that can be deployed at multiple locations to form a cross-region large-scale storage infrastructure.
Delay problem "big data" applications also have real-time problems. Especially related to applications related to online transactions or finance. For example, the online advertising promotion service of the online clothing sales industry needs to analyze the browsing records of customers in real time and accurately place advertisements. This requires that the storage system must be able to support the above features while maintaining a high response speed, because the result of the response delay is that the system will push "expired" advertising content to customers. In this scenario, the scale-out architecture of the storage system can play an advantage because each node has processing and interconnection components, and the processing capacity can be increased simultaneously while increasing capacity. Object-based storage systems, on the other hand, can support concurrent data streams to further increase data throughput.
There are many "big data" application environments that require high IOPS performance, such as HPC high-performance computing. In addition, the popularity of server virtualization has also led to the demand for high IOPS, just as it has changed the traditional IT environment. In order to meet these challenges, various models of solid-state storage devices have emerged, ranging from simple internal caches in servers, to scalable storage systems with all solid-state media, and so on.
Concurrent access Once companies realize the potential value of big data analytics applications, they will incorporate more data sets into the system for comparison and allow more people to share and use the data. In order to create more business value, companies often analyze multiple data objects from different platforms. The storage infrastructure including the global file system can help users solve data access problems. The global file system allows multiple users on multiple hosts to concurrently access file data, which may be stored in multiple locations. Different types of storage devices.
Security issues Applications in certain special industries, such as financial data, medical information, and government intelligence, have their own security standards and confidentiality requirements. Although these are not different for IT managers, they all must be complied with. However, big data analysis often requires multiple types of data to refer to each other. In the past, this type of data was not mixedly accessed. Data applications have also spawned new security issues that need to be considered.
Perhaps people’s impressions of big data come mainly from the cheapness of storage capacity, but in fact, companies are creating a lot of data every day, and more and more, and people are trying to find valuable data from vast data. Business intelligence. On the other hand, the user will also save the data that has been analyzed, because the old data can be compared with the new data collected in the future, there are still potential use possibilities.
Why big data? Why is it now?
Compared with the past, we have to face more data types in addition to the ability to store more data. Sources of these data include online transactions, social networking activities, automatic sensors, mobile devices, and scientific instruments. In addition to those fixed sources of data production, various trading activities may also speed up the accumulation of data. For example, the explosive growth of social multimedia data stems from new online transactions and records. Data is always growing, but only the ability to store large amounts of data is not enough, because it does not guarantee that we can successfully search for commercial value.
Data is an important production factor
In the information age, data has become an important production factor, just like capital, labor, raw materials and other elements, and as a general demand, it is no longer limited to the application of certain special industries. Companies from all walks of life are collecting and using a large amount of data analysis results to reduce costs as much as possible, improve product quality, increase production efficiency, and create new products. For example, analyzing data collected directly from the product test site can help companies improve their designs. In addition, a company can also surpass its competitors by analyzing customer behavior in depth and comparing large amounts of market data.
Storage technology must keep up
With the explosive growth of big data applications, it has spawned its own unique architecture, but also directly promote the development of storage, networking and computing technology. After all, dealing with this special demand for big data is a new challenge. The development of hardware is ultimately driven by software requirements. For this example, it is clear that the demand for big data analytics applications is affecting the development of data storage infrastructure.
On the other hand, this change is not an opportunity for storage vendors and other IT infrastructure vendors. With the continuous increase in the amount of structured data and unstructured data, and the diversification of sources of analysis data, the design of the storage system has been unable to meet the needs of big data applications. Storage vendors have realized this and they have begun to modify the architectural design of block- and file-based storage systems to accommodate these new requirements. Here, we will discuss the properties related to big data storage infrastructure and see how they meet the challenge of big data.
Capacity issues The “big capacity†mentioned here can usually reach the scale of the PB data. Therefore, the massive data storage system must also have the corresponding level of expansion capability. At the same time, the expansion of the storage system must be simple, you can increase the capacity by adding modules or disk cabinets, or even without downtime. Based on this demand, customers now increasingly favor the storage of the Scale-out architecture. The characteristic of Scale-out cluster structure is that each node not only has a certain storage capacity, but also has internal data processing capabilities and interconnected devices. Compared with the chimney architecture of traditional storage systems, the Scale-out architecture can achieve seamless smoothing. Expansion to avoid storage silos.
In addition to the large scale of data, "big data" applications also mean having a large number of files. Therefore, how to manage the metadata accumulated in the file system layer is a difficult problem. If it is handled improperly, it will affect the system's scalability and performance, and traditional NAS systems have this bottleneck. Fortunately, the object-based storage architecture does not have this problem. It can manage billions of files in one system, and it will not suffer from metadata management as traditional storage. Object-based storage systems also have wide-area scalability capabilities that can be deployed at multiple locations to form a cross-region large-scale storage infrastructure.
Delay problem "big data" applications also have real-time problems. Especially related to applications related to online transactions or finance. For example, the online advertising promotion service of the online clothing sales industry needs to analyze the browsing records of customers in real time and accurately place advertisements. This requires that the storage system must be able to support the above features while maintaining a high response speed, because the result of the response delay is that the system will push "expired" advertising content to customers. In this scenario, the scale-out architecture of the storage system can play an advantage because each node has processing and interconnection components, and the processing capacity can be increased simultaneously while increasing capacity. Object-based storage systems, on the other hand, can support concurrent data streams to further increase data throughput.
There are many "big data" application environments that require high IOPS performance, such as HPC high-performance computing. In addition, the popularity of server virtualization has also led to the demand for high IOPS, just as it has changed the traditional IT environment. In order to meet these challenges, various models of solid-state storage devices have emerged, ranging from simple internal caches in servers, to scalable storage systems with all solid-state media, and so on.
Concurrent access Once companies realize the potential value of big data analytics applications, they will incorporate more data sets into the system for comparison and allow more people to share and use the data. In order to create more business value, companies often analyze multiple data objects from different platforms. The storage infrastructure including the global file system can help users solve data access problems. The global file system allows multiple users on multiple hosts to concurrently access file data, which may be stored in multiple locations. Different types of storage devices.
Security issues Applications in certain special industries, such as financial data, medical information, and government intelligence, have their own security standards and confidentiality requirements. Although these are not different for IT managers, they all must be complied with. However, big data analysis often requires multiple types of data to refer to each other. In the past, this type of data was not mixedly accessed. Data applications have also spawned new security issues that need to be considered.
Solar Street Light,LED Solar Light,Solar Street Light,Solar Garden Light
Jiangmen Dilin Lighting High-Tech Co., Ltd. , https://www.jmdilinlighting.com