Big data illustration

Deep learning from big data

Solving the world’s problems through artificial intelligence technology

The amount of data that is generated globally every year now exceeds the total volume of information created throughout the history of the world. Terms such as “data rich” and an “abundance of data” are now a gross understatement in this day and age. We humans are now bombarded with a never-ending stream of electronic communications, sensory information and in many cases redundant, repetitive data noise that demands processing and filtering.

This article was first published in Impact @ Griffith Sciences magazine, find out more… 

Sea of DataAt the other end of the spectrum, that is in the arena of business, science and research, the shear volume of data now available presents immeasurable opportunities. But just as the overwhelming amount of data that we humans experience on a day-to-day basis requires processing and filtering, so does the data which presents itself for the purpose of providing statistics to support a business case, or a data stream that enables a software system to make seemingly intelligent decisions.

Big Data v Open Data

Types of data, which are becoming popular in all aspects of society, include “Open Data” and “Big Data”. Open data can be defined as information that is made publicly available and “Open”, frequently associated with data disseminated through government agencies. This means that the public can download information about anything from the location of parking spots in a city all the way to (in more bold governments) crime data in the form of maps and hotspots in a particular region. The data is made public to provide businesses, researchers and local industry the opportunity to develop software or systems to enhance their operations, support their R&D efforts or assist with business processes.

On the other hand, “Big Data” is big by name and by nature. It represents the new wave of data that is large in volume, velocity and variety (3 “Vs”) as defined by the Gartner Group earlier this century. The first “v” (volume) is self explanatory. Data sets now exist which are in the order of petabytes (1015 bytes or 1000 terabytes = 1000,000 gigabytes); this is extraordinarily large in anyone’s language. The second “v” (velocity) refers to the speed at which data presents and itself, and lastly the third “v” (variety) relates to the diverse sources that data now comes from. All of these ”Vs” demand clever technologies for processing.


Artificial Intelligence meets Big Data

Artificial (or Computational) Intelligence (AI/CI) has emerged as one of the most revolutionary areas of computer science. Although decades in the making, the prospect of one day building computers that can think for themselves, are self aware and make seemingly intelligent decisions like their human counterparts, is now closer than ever. Images of machines taking over the world in a high-tech war or robots turning against humans is frequently espoused in movies and stories as the dreary future, which is the product of AI gone awry. The truth of the matter is that AI in conjunction with Big Data is contributing to significant advances in science, engineering, medicine and the environment, which is saving lives, saving millions of dollars and saving assets.

The key to the world’s future success is in the development of cutting-edge technology that can address the greatest problems facing humanity through interdisciplinary research. One of the most exciting revolutions in AI technology is the advent of the “Artificial Neural Network” or ANN. This is a software model which was conceived in 1943 and was widely adopted in the mid-1980s for solving a variety of problems. ANNs are touted as simple representations of the human brain, exhibiting the capacity to learn, having the ability to recognise patterns automatically and making intelligent decisions. In this decade, the resurgence of ANNs can be attributed to the algorithms behind such concepts as “Deep Learning”. This approach to “training” an ANN relies on its ability to learn high-level concepts from data to perform automated processes such as recognition of objects. These new learning models are becoming the ultimate AI technology, which can exploit the potential of Big Data for solving some of the most pressing problems in the world.

What we are doing

At Griffith’s School of ICT, interdisciplinary research using AI technology in the area of Coastal management has generated camera-based technology for analysing large amounts of digital video images of beach scenes to count and understand the behaviour of persons on the shore along Australia’s coast. This technology is accurate to 80-90% and can detect, from very low-resolution video, whether a person is walking/running along the beach or entering the ocean. The application of this work can assist in beach safety and support the work of our lifeguards to detect swimmers in danger.

Another exciting area in the School of ICT is the research work being undertaken merging the fields of IT and Engineering for predicting the deterioration of bridges in Queensland (10,000 in total) using ANN technology. This research work, undertaken in collaboration between the City of Gold Coast and the Queensland State Government, has produced predictive models trained using historical bridge data to ensure the safety and effective maintenance of important State assets. The research in this area has generated effective models to predict the deterioration of bridges up to 20-30 years into the future with the ability to save millions of dollars through prioritisation of asset maintenance regimes.

Finally, research work, which intersects information technology and the environment, has produced accurate and sophisticated AI models for the prediction of flood events on the Gold Coast. Using a variety of data from sensors located at numerous catchments around the Gold Coast, rainfall data and other complementary information, ANNs have been trained to understand historical trends and are able to perform predictive analytics for forecasting flood events up to three hours ahead, which provides sufficient time for undertaking effective disaster management. This work being undertaken at Griffith’s School of ICT is scalable and can be extended across the State and indeed the Nation.

The use of breakthrough predictive analytics and AI technology for use with “Big Data” opens up countless opportunities for solving the problems faced in modern society. As can be seen from the few examples above, technology underpins all aspects of science and its power to harness the massive amounts of data, which is the key for providing solutions to the big issues facing humanity in the 21st century and beyond.

By Professor Michael Blummenstein

Griffith’s School of ICT

Keep up to date and subscribe to Impact @ Griffith Sciences