The security industry faces new challenges in the era of big data

Since the birth of the first computer in 1946, in just over half a century, information technology has swept the world with its extensive influence and tremendous vitality. It has become the most brilliant, fastest-growing and most influential human impact in the history of science and technology development. Of science and technology. Over the past few years, with the simultaneous development of the Internet and information technology, various industries have accumulated a large amount of data. According to the world's leading information technology research and consulting firm Gartner report, the important factors that drive the development of big data come from two aspects: First, the consumer field, such as online shopping and social media applications generate a large amount of data; the other hand, from urban infrastructure construction. Security is one of them.

Multi-type security data hits

In the security industry, there are many types of data involved, and in terms of data structure types, it includes various unstructured, structured, and semi-structured information. Unstructured data mainly refers to video recordings and picture recordings, such as surveillance video recordings, alarm recordings, summary recordings, vehicle mount images, face captures, and alarm captures. Structured data includes alarm records and system log records. O&M data records, summary analysis of structured description records, and various related information databases such as population information, geographic data information, vehicle management information, etc. Semi-structured data such as face modeling data, fingerprint records, etc.

There are several sources of this information, one is the information generated within the security system, such as various video recordings, snapshots, system operation and maintenance data, log records, summary records, etc.; others are collected or integrated through external systems. Such as population information, geographic data information, face database data, vehicle management data, etc. These data, as a whole, constitute the big data foundation of the security system or security field, and have the following characteristics: The most notable is that the security data volume is huge and continues to expand rapidly. With the continuous networking and integration of video surveillance image systems, With the continuous improvement of video resolution and frame rate, the storage of real-time video data has jumped from the TB level to the PB level; secondly, there are many types of security data, such as the above-mentioned videos, pictures, geographic information, etc., and With the continuous integration of various security systems and related information systems, there will be more and more data types; in addition, the overall value of security data contained in the data is low, but the value is high. Take video surveillance data as an example. In the continuous 24/7 monitoring process, the useful data may only have one or two minutes or even one or two seconds. At the same time, these information updates are particularly fast, security data is generated in large numbers every moment, and information is updated all the time.

To quickly and efficiently screen effective information from these small amounts of information, security vendors need the following two foundations to ensure: First, the processing power and efficiency of unstructured information needs to be improved, video images are accurately and quickly processed, and face features are built. Modules and other data, to extract useful information from the inside, and can be a representation of the information, complete the extraction and storage of data information on the storage level of big data, in order to carry out subsequent data information retrieval, analysis and mining business. Secondly, by using and improving current big data processing technologies and platform frameworks, it provides a quick search mechanism for security data information and forms a targeted mass security data information processing architecture from these large amounts of structured and semi-structured. Rapid retrieval and analysis of information.

Application of Big Data in Security Industry

Security data is still dominated by unstructured information such as video and video, so it has its own characteristics in the analysis and processing of big data. In addition to conventional information retrieval, the security industry is more focused on information retrieval of graphic images; current IT Big data generally focuses on analyzing and retrieving text data. Therefore, there are two main types of processing and analysis tools for security in the security community. One is the processing and analysis of unstructured information such as video images. Tools, including video intelligence analysis tools, video summary tools, image clarity tools, video clarity tools, video transcoding tools, video editing tools and more. The other is the big data analysis and processing tool for structured, semi-structured information. In this respect, the security industry has drawn on the IT industry's architecture and experience in processing big data. It can be used for structured and semi-structured data. Enable fast and accurate data analysis and mining.

Looking at the entire security field, Safe City is the largest system with the most complex business. It involves security, surveillance, command communications, investigation, law enforcement, and social services. Video access scales from thousands to tens. With the increasing demand for high-definition, intelligent, and networked security monitoring, the volume of data generated by Pingan City's projects every day is growing at an alarming rate.

However, the application of big data technology in various fields of security is not as mature as the IT and Internet industries. The related information collision and mining needs have not yet been able to form a model. In the face of increasingly large data, how to do a comprehensive application and accurate analysis of massive amounts of video data is the pain point of Ping An City's big data, and it also determines that it needs big data technology to support it. In all kinds of safe city construction projects, relying on big data analysis technology to extract effective security information from massive video images has long become the consensus of the security community.

Faced with the advent of the era, according to the advent of the era, the company continued to explore in the fields of data collection, convergence, and application. It took the lead in introducing the industry's first comprehensive solution for public security image information applications. The program is based on an integrated application platform for image information and corresponding management and control software. Can integrate various internal and social image information resources of the public security into the image resource database, and organize and store the audio and video image information concerned by various departments and police breeds. At the same time, an index summary is created to facilitate evidence search and case correlation analysis. The image resource library also has perfect query, information comparison, archiving and storage functions, and adopts dynamic management methods to ensure that the information is timely, accurate, and effective, so as to facilitate the province's resource networking and sharing.

After completing the collection and organization of large amounts of data, the entire program can realize the functions of video data resource review, intelligent analysis and application, online supervision, public security prevention and control, intelligent transportation, image investigation and judgment, information reporting, and image resource library. The value information behind the massive video surveillance data quickly feeds back information to aid decision making.

Taking intelligent transportation as an example, the smart bayonet subsystem introduces deep data mining and analysis technology into the traditional security bayonet, and implants several sets of license plate map detection methods. Through follow-up vehicle correlation analysis, path correlation analysis, and regional data collision analysis, etc. Combining data mining functions with practical applications, the analysis of bayonet data looks for regularities among them, narrows the scope of investigations, and improves the public security traffic police's early warning capabilities and case handling efficiency. When the vehicle continuously passes through multiple intersections, the system can display all vehicle passing information in the time sequence in the list according to the license plate number and the license plate color within a specified time range, and linkage the electronic map. Presents its trajectory for automatic behavioral analysis.

In the process of public security handling of cases, the image investigation and decision-making subsystem has been highly praised in the public security industry for allowing the handling of cases to be liberated from massive data. The image investigation and judgment subsystem carries out data and information flow according to the process of map detection and judgment. A variety of intelligent image processing software performs unified transcoding, summary processing, video editing, video feature extraction, and image clarity processing on video images, and simultaneously tactics and procedures. Combine organically to form a complete set of technical warfare methods and complete the quick comparison analysis of the case in different situations, allowing ** to intelligently locate key points in massive information, maximizing the processing results and shortening the video analysis time This will effectively reduce the time for manual viewing of videos, reduce manpower investment, and promote the rapid development of case video analysis in an intelligent way, providing an important method and basis for cracking cases.

Big Data Test Storage Capacity

From most current security system architectures, the transmission channels of security big data are mainly networks. Generally, the network convergence and hierarchical storage mechanisms are adopted. The network is generally a Gigabit network, and storage is also based on general disk array storage. In order to adapt to the integration of security information, security data analysis and information mining needs, security systems gradually tend to be interconnected and integrated, and security data information is gradually converged and integrated. There has been centralized management of 10 Gigabit network and cloud storage as video. Image data storage platform trends. At present, cloud storage has become the core technology of security big data, and all manufacturers are also working hard to improve their R&D capabilities.

Compared with traditional storage devices, cloud storage is not only a hardware, but a complex system composed of multiple components such as network devices, storage devices, servers, application software, public access interfaces, access networks, and client programs. Cloud storage can provide disk-array-like bare space services, as well as provide storage-based services. For example, cloud storage in the safe city system can provide storage services for multiple subsystems, including video surveillance, bayonet police, image resource library, map detection analysis, etc. It can complete the daily video recording resources, resources such as videos and pictures involved in various cases, and the unified storage of resources such as vehicle capture pictures generated by the bayonet police.

From the application architecture point of view, the difference between cloud storage systems and traditional distributed storage systems is not obvious, and it is more essential to distinguish the entity's internal software architecture. The cloud storage system adopts a large-scale distributed parallel file system. Based on a large number of servers and storage devices, it builds a large-scale storage cluster, provides hundreds of PBs of storage capacity, and is capable of online capacity expansion. The overall cost of the capacity storage system is much lower than that of the traditional storage architecture, and it has good scalability and flexibility.

The asymmetrical architecture in which the cloud storage system separates metadata and storage data, and achieves up to tens of Gbps transmission rates and hundreds of petabytes of storage capacity under normal hardware conditions through load balancing and concurrent data access strategies. The trend of user application development is timely and on-demand dynamic expansion. Unlike a stand-alone file system, a distributed file system does not manage this data on a disk and is managed by the upper-level operating system. Instead, it is stored on a server cluster. The servers in the cluster do their best and cooperate with each other. Provides the entire file system service.

The cloud storage system has a built-in object-based data management strategy to ensure the security and reliability of data in the event of a local failure of the system, completely eliminating single points of failures in the storage system, and combining automatic fault detection and fast failure recovery technologies to ensure that users Applications continue to operate steadily, while reducing the difficulty of deployment and management. Take a cloud storage system as an example. Traditional storage devices usually use RAID for redundant backup. When a hard disk is damaged, the RAID reconstruction time usually takes more than ten hours. If the hard disk is broken during this period, the data cannot be saved. Restored, so maintenance personnel need to rush to the scene to replace the hard disk at the first time, 7 × 24 hours standby; cloud storage using a distributed file system, data storage and backup no longer rely on the ability of a single device, when the hard disk is broken The rest of the hard disk will be automatically reconstructed to recover damaged data. This only takes ten minutes. Maintenance personnel only need to regularly check the hard disk for damage and replace the new hard disk. Maintenance is very simple.

Opportunities and challenges coexist

The more data is huge, the more business information is accumulated, the greater the value. Big data that is characterized by mass, diversity, and rapidity is not as easy to manage and analyze as traditional database data. It also poses higher requirements for the entire IT and communications industry.

For example, security issues, data security is a problem in the entire IT industry, and the security industry is no exception. The security vulnerabilities encountered in the IT field and all kinds of damage and attacks will also be encountered in the field of security. As for the security field, the amount of data is large, and the amount of information contained in image information is more, and it also involves personal privacy, public security and other issues. Therefore, the system security must not be ignored, but the relevant technology and equipment of the equipment manufacturers The legal norms cannot be achieved overnight.

At present, the security industry has entered a big data era with one foot, but it has not yet fully entered. The reason why this is said is that the current data volume of the security system is indeed expanding, the emergence of high-definition video, the continuous advancement of security system networking and integration, and the increase in storage technology and capacity, resulting in huge data volume, highlighting the characteristics of big data Such a security system is destined to be a large collection of data, but at present, a large amount of video data is still independent and fragmented. The video recording data is scattered in various industries and units and is independent of the system. It has not been used to achieve true networking and sharing, and the industry has not formed a common method for data mining and utilization.

In addition, in the relevant IT infrastructure, intelligent analysis and retrieval of video image information, organization and management of security data, data analysis and mining algorithm modeling and implementation, the need to fully introduce innovative technologies. Security companies need to strengthen their internal security, improve their R&D capabilities, strengthen their technical reserves, respond to the impact of larger data volumes, and improve their ability to handle and utilize big data, especially at the present time. The increased video intelligence applications and the accuracy of video intelligence analysis are issues that the security industry needs to address before it enters the era of big data.