I just read a very interesting article on “How New Intelligence Will Tame The Information Explosion” on CNBC.com. The article written by Steve LaValle and Jim Bramante from IBM describes how one in three business leaders cannot make the right decision. The reason? According to LaValle and Bramante, even though there is an abundance of information around, there is scarcity in getting the right data at the right time for making good decisions.
“New Intelligence” is vital for today’s CEO and other decision makers to make agile and accurate decisions. It’s simply impossible to drive a company to success based on intuition and a closed group of advisers. You need to assemble the right proof points, analyze them, and make your move. And this can’t be a one-off effort. You need to build it into your management process to continuously have your finger on the pulse, ready to alter your business direction at any given moment.
Two critical components necessary for “New Intelligence”
First (and most important): the data. No decision-making is better than the data behind it.
Second: Analyzing and reporting of the data. Fortunately, there is an ever growing set of very good analysis and reporting tools on the market today. Products from large companies like IBM Cognos, Oracle Hyperion, SAP Business Objects, Microsoft Fast, Autonomy, as well as numerous pure-play vendors like ClaraBridge, Corda, Attensity, and QlikTech.
One source of data that has become increasingly important today is web data. By web data I specifically mean data from public websites, including those of your competitors as well as from business partners.
Most web data is hidden behind a human browser interface and inaccessible by traditional enterprise applications. As a result, companies have been doing either manual cut-and-paste of the data or writing fragile “Web scrapers” in technologies like PERL or Python.
Unfortunately, those data acquisition methods are simply not scalable. Manual cut-and-paste can only access a fraction of the needed data while “Web scrapers” can only deal with static HTML pages and are insufficient to address the increasing amount of JavaScript and AJAX powered sites.
This is exactly what we have addressed at Kapow Technologies with the Kapow Web Data Server 7.0, the newest release of our flagship product. Using our proprietary, scalable, HTML parser and JavaScript engine in a point-and-click development environment, it is now a breeze to extract or service-enable any web data, even behind the most complex and dynamic web applications. Typically, it only takes as much effort and time as it would to click through a website once. After that, all the critical web data is automatically collected or accessed to empower your critical business decisions and drive your business forward.
I encourage you to check it out and let us know what you think.
By: Stefan Andreasen 
