Everything You Need to Know About Text Mining

Written in collaboration with Mary D.

Text mining is widely recognized by companies as one of the major tools that are provided by A.I. technology to extract valuable ‘structured” data from text and help businesses filter through and condense valuable project-oriented information. If this sounds a little too ‘sci-fi” for you, that is exactly the point. Its capacity to recognize and respond to human speech and mimic the neural pathway activities of the human brain to develop independent cognitive abilities and behavioural responses is only the beginning, especially as more technological advancements are introduced into the field and artificial intelligence software are gaining higher momentum and demand among multiple industries with connections to the IT/Computer/Mobile industry—from the transport, banking, social services, government, and medical sectors among many more. While the first attempts to develop intelligent thinking machines can be traced historically to Raymond Lull in the 14th Century—and “automatons” are present in the ancient mythology of Greco-Roman, Egyptian and Babylonian/Mesopotamian literature—text mining goes back to the WWII era, when governments started adopting “content analysis” and assigning numerical codes to public concepts and ideas that were found in the media, including newspapers, magazines, letters, documents…etc. with the objective of analyzing and monitoring trends in mass-behaviour by tracking the levels of popularity and development of those concepts/ideas. This practice has also developed into another branch known as open source intelligence, used by governments and the intelligence communities to sift through all the pertinent information that is available on the World Wide Web for reasons of national security and especially in response to the current major crises facing the modern world. Unfortunately for the general public, it is also something that has led to the infringement of personal security and civil rights as demonstrated by its current use as a global spy network set up by agencies like the NSA and the CIA.

⇥ (read more)