The Web Data Services Blog

Last week I was in Washington DC with John Yapaola, our CEO, to receive our ComputerWorld Honors Laureate Medal based on our nomination by Morgan Stanley.

Morgan Stanley has been a long time customer of Kapow Technologies and in 2008 they even became one of our investors.

Morgan Stanley’s nomination highlights in detail how the Kapow Web Data Server is critical to the research team’s ability to “advance their stock recommendations [and] manage data access and extraction for its complex research projects in orders of magnitude more efficiently compared with previous methods”.  Access to real-time web data simplifies the complexities of analyzing business opportunities and provides a significant advantage for Morgan Stanley.

Morgan Stanley continues to evolve and redefine the potential and impact of Web Data Services, and we are proud to have them as a customer and investor.

By:  Stefan Andreasen

Kate Gosselin on Dancing with the Stars Photo Credit: ABC

Can Social Media be used to predict the outcome of Reality TV shows such as American Idol and Dancing with the Stars?  We created Reality Buzz based on our real-time automated web data collection platform to find out.

Jennifer Zaino over at Semantic Web wrote a nice article that captures the essence of Reality Buzz and our process of using real-time social media web data to build intelligence in to predictive analytics:  Taking Sentiment Analysis to Dancing with the Stars and American Idol

Check it out.  And if you have the need to automate the access, collection, harvesting, scrubbing, grabbing or scraping or real-time web data to improve market or competitive analysis to improve your strategic decision making, we’re here to help.

By:  Rick Kawamura

Together with my product management team, I am currently out speaking with our customers to gather input for future product roadmap priorities.

It’s always great to meet up with our customers, but in particular, what’s been so rewarding this time is hearing one amazing story after another from our customers on the huge time and cost savings they’ve achieved with the Kapow Web Data Server.

Let me share two of their stories.

First, I talked to a large telecommunications company, a long time customer of ours, who has been using Kapow on 80 data automation projects. One project, with expected revenue of $4.9 million over 3 years, included the assembly of 129,000 listings into a directory, all to be finished within 6-8 weeks.

Performed manually, one listing took 15 minutes to complete, so one person was expected to complete 30 listings per day, or 1200 listings in 8 weeks. Basically 100 employees were required to finish the project on time using traditional manual order-entry methodologies.

Using Kapow they successfully built data automation “robots” that took 44 data fields from the source application, transformed it into 69 target fields, and then automatically loaded them into the target system. The Kapow robots completely automated the interaction with the source and target application Web front-ends, as well as automatically performed all the advanced transformations to convert the source data format to the target data format. With robots they were able to complete 5000 automatic listings per day with a 95% success rate and finish the project in only 4 weeks.

This resulted in an astonishing total savings of $1.3 million and at the same time eliminated the risk of the project not getting done in time.

The second company, a business information provider, was extracting energy related data from a dozen government web sites using manual cut-and-paste. With Kapow they built automated data acquisition robots that saved 2868 hours per year of manual labor for each data source. For 12 data sources this resulted in an astonishing savings of 34,416 hours per year.

I love to hear these stories and it’s always very satisfying to hear a customer tell you directly how much they like and benefit from your solutions.

By:  Stefan Andreasen

While web data, especially social media data, grows exponentially, the vast array of opportunities for using real-time web data to improve analysis and decision making is limited only by your imagination.  These days, companies must incorporate Web data into their intelligence and analysis tools in order to compete. In some industries it’s a matter of survival.

Real-time data is where the answers are.  It’s where market and customer trends are immediately identifiable.  It’s where deals will be won and where winners will claim their trophies.

Reality Buzz

We recently built something 30M Americans can relate to – a way to predict American Idol and other reality show results based on data harvested from popular social media sites.

Every week on American Idol, contestants perform, their fans dial-in their support for their favorites, and the next day contestants are voted off the show.  During the performances, and for several hours after, fans tweet about and discuss their favorites online, showing support for the ones they want to see voted through to the next show.

We scrape thousands of pieces of web data from twitter, Facebook, forums and discussion sites around the web, apply sentiment analysis, analyze the data, and make predictions about the person(s) to be eliminated from the show, all in the span of a few hours.

We built the robots (automated web data collection processes) in a matter of hours.  Now they are automated to collect the data, transform unstructured data into structured data, and load it into a MySQL database.

In the last two weeks starting with the top 12 contestants, we’ve successfully predicted the American Idol contestant to be eliminated hours before the elimination show aired.  For more information on our latest predictions and to learn more, please visit Reality Buzz on Facebook.

Imagine what you could do for your business with Kapow’s Web Data Server and a few hours creating Kapow robots.  Real-time web data can fuel predictive analytics capabilities to give your company an unfair advantage.

Over 400 Kapow customers are jumping in with both feet.  What’s stopping you?  Learn more on our Kapowtech.com website, or contact us for a Free Trial of the Kapow Web Data Server.

By:  Rick Kawamura

“Information, thanks to the internet, is growing at breakneck, exponential speeds. This has proven to be both an opportunity and a curse for businesses. With all this data abundance, organizations of all sizes struggle to access and act upon it in a timely and cost effective way. Just think about the impact to your business if you could automatically add high-value Web data to your market intelligence, pricing intelligence, financial intelligence or any other business intelligence application. Until recently, this seemed like an impossible feat, or at least cost prohibitive based on the man hours involved.”

The above is an outtake of an article I recently wrote for ebizQ, the full article is available here and provides more insight into the emerging web data services market, common web data access challenges and real world use cases and benefits of web data services. Feel free to check it out and let me know what you think.

Web Data – Why Top Business Leaders Depend On It And You Should To

By:  Ron Yu

Here at Kapow Technologies we just finished an outstanding quarter having signed our 400th enterprise customer, closed 87 deals, and doubled our subscription revenue mix.  We’re proud to see our hard work paying off despite tough economic times.  For more details check out our press release.

More and more businesses are seeing the critical value Kapow’s solutions and technology play in creating efficient, agile businesses.

In December I visited an executive at one of our new global enterprise customers who heads a business unit heavily dependent on business applications to perform process automation, data analysis and reporting.

To better serve the business, his company made several efforts over a 10 year period to consolidate data integration on a common platform. First Corba, then SOA. Unfortunately neither of these initiatives lived up to their promise to deliver business applications, business data and process automation at the pace needed to support line of business and growth of the company.  The economic downturn actually created an opportunity to better serve IT business needs by leveraging the Kapow Web Data Server to serve and deliver on LOB’s ever changing needs for additional data sources and more precise, timely data. They simply could not afford to wait for traditional coding approaches to “maybe” deliver on their IT needs in 6 or twelve months. They needed to get things done much more quickly and that is exactly what they are doing with Kapow.

More and more business leaders are putting front-office applications in the cloud or developing them themselves “on demand”.

Welcome to the new world of efficient business IT powered by Kapow Technologies.

By:  Stefan Andreasen

Earlier this week we announced great results for Q2 FY2010.  See the highlights.  At Kapow, we see the voracious appetite of businesses to harness the power of the internet by leveraging real-time web data into their core enterprise information systems.

I met up with Joe Bugajski, analyst at The Burton Group, this morning and he conveyed that making sense out of the world of “unstructured” data remains unsolved.  Also that information managers continue to face challenges combining “external” data with the data they work with every day.

Web Data Services certainly fills that void.  With over 400 customers and Fortune 1000 companies across every industry and geography, forward thinking executives are finding ways to use public Web data to keep an eye on competition, reduce risk, automate manual processes and even roll out new products and services.   Stay tuned for our webinar with Deutsche Boerse, Europe’s #2 stock exchange, to see for yourself how real-time web data services was put into action to create a power financial instrument for energy commodities traders to time the market!

By:  Ron Yu

At Kapow Technologies we launched our first SaaS product Kapow OnDemand about a year ago and today we have many great companies running it for pricing intelligence, web portals, mobile enablement and more. During the same period we also had an increasing amount of people using either our non-commercial community, openkapow, or a free trial of Kapow OnDemand. Many came back and said “I love this product, it’s exactly what I’m looking for, but my project and budget is not big enough for my own hosted solution like Kapow OnDemand.  Do you have a multi-tenant pay-as-you-go SaaS offering instead?”

As always, we strive to deliver products our customers want, so we looked around for the ideal partner to launch such a product.

Today I am very proud to announce we are launching Kapow Web Data Services 7.0.0, a new pay-as-you-go SaaS service together with StrikeIron.  The new offering is based on a true SaaS multi-tenant implementation of our well proven and robust Kapow OnDemand hosted solution.

Say “goodbye” to cutting and pasting from your web browser into Microsoft Excel. Now you can wrap any website or web application into RSS feeds or REST web services for rapid, automated and structured access of any external web data source. Deliver that feed or service to your choice of application, Business Intelligence tool or RSS feed reader. With the StrikeIron Live Data for Excel you can even get real-time access to Web Data from Microsoft Excel.

Bob Brauer, Founder and President of StrikeIron, wrote a nice blog post on the announcement.
We are excited about our partnership with Bob’s company, and StrikeIron has some great complementary data services.  And now together with Kapow Technologies they have the most complete SaaS based Data Services offering on the market.

Check out our free trial of Kapow Web Data Services 7.0.0 over on StrikeIron and let us know what you think.

By:  Stefan Andreasen

We received very nice coverage by Dana Gardner in his blog post “Web data gains some due respect as Kapow eases it into mission critical enterprise uses” at ZDNet.com.  Dana details the essential role web-based content plays in business intelligence (BI) and service-oriented and web-oriented applications (SOA/WOA), and also highlights the need for fast and easy integration of the data.  Great article, Dana!

By:  Stefan Andreasen