The inevitable nature of big companies is to lock customers into their suite of products. For instance, Oracle started with databases, now bundles financial and integration software with their database. Customers are told it’s easier to use their add-on, value-add products than use another vendor and get into compatibility issues. Add-on, value-add products are sometimes free, enticing you even more.

So it comes to no surprise that Microsoft has announced they have tightly bundled their BI software, PerformancePoint Server, with their document management software, Sharepoint. Joined at the hip, you cannot buy one without getting the other.

Comparatively to other BI vendors, Microsoft has the only bundled BI, document management, workflow, collaboration software on the market (I think). IBM Cognos, SAP Business Objects, and Oracle Hyperion are behind with this endeavour – they haven’t tightly integrated BI with their other software.

Each of these competitors to Microsoft have similar software breadth but Microsoft’s tight integration is a step ahead. It may take years for IBM, SAP, and Oracle to properly integrate BI, document management, workflow and collaboration.

But what about the customers? On the surface, the tight bundling can be viewed as positive. You could be using Sharepoint and now want some BI reporting. Or maybe you have PerformancePoint reports and need to manage documents.

Tackling both BI and document management pieces is challenging.  Without considering planning, governance, and skillsets, you will most certainly encounter serious setbacks.  Or worse your vision is restricted to an unimpressive rollout.

Here are some problems you may encounter with tight bundling of BI and document management.

1. Standardizing on Microsoft, Oracle, IBM, SAP technologies requires highly paid IT resources and/or consultants (not great for this economic climate)
2. Very different skillsets are required to build Sharepoint sites vs PerformancePoint reports (need to hire more staff and/or pay for more IT training)
3. Goverenance and best practices are very different for managing BI vs documents (different knowledge needed, so twice the effort)
4. Locked into sole-source agreements (difficult switching BI or document mgmt products for a better and more cost effective product in the future)

Putting those concerns aside, there are positives in Microsoft’s story.  Microsoft’s price point is unbeatable when compared to other on-premise vendors.  This can offset additional costs (see points above) and make your entrance into Document Management and Business Intelligence a little easier.

“Predictive analysis can be an extremely useful tool for many different types of businesses. In fact, where there is any type of data warehousing there should be implementation of a business intelligence program that includes predictive analysis. However, in order to learn how your business can profit from this facet of business intelligence, you are going to have to understand exactly how and why predictive analysis works.

The main idea of predictive analysis is to use current and past data to predict future events. The goal of the statistical techniques used in predictive analysis is to determine market patterns, identify risks, and predict potential opportunities for growth. In addition, data relationships can be reordered to determine the most plausible outcome of possible solutions and patterns can be recognized that might have the power to alter the outcome of a probable event.

One of the most important aspects to reliable predictive analysis is data quality. The information provided by predictive analysis can only be as effective as the abundance and accuracy of data available. Data quality is absolutely necessary to the process of predictive analysis. In order to attain accurate business intelligence, companies must maintain quality data.  Predictive analysis requires both past and current data about many different things including customers, businesses, products, and the economy. All of this information is used to draw relationships and patterns between sets of data. If the data is accurate and well maintained, then the business intelligence produced will be high quality as well.

In the past, predictive analysis was mainly used for newly emerging technologies. However, in recent years these practices have quickly started to become common for mainstream businesses. There are a few differences between the ways that these techniques are currently used and how they were used in the past. One of the main reasons for these differences is why companies use predictive analysis. In the past, these techniques were used for long-term analysis of market and consumer trends. However, in recent years, the mainstream implementation of predictive analysis techniques has tended to focus more on immediate, tactical uses. Because of the “real-time” nature of this business intelligence, more and more companies are using predictive analysis as standard in making predictions about particular industry markets and consumer trends.

Some of the industries that have started utilizing these business intelligence techniques include telecom, insurance, pharmaceutical, and financial industries. All of the companies in these different business sectors have been able to use predictive analysis to make the right decisions to move their businesses in a positive direction. These processes can help with economic predictions as well as predicting the behavior of businesses and consumers. This type of information, made available in an efficient manner by business intelligence, is understandably invaluable. It can turn a simple prediction into intelligence that is more precise than even the most educated guesses. Predictive analysis with appropriate attention paid to data quality has made it easier than ever for businesses to make accurate market and consumer predictions and thus smarter decisions for the growth of their company.”

Service Orient or Be Doomed

Wikinomics

Mavericks At Work

When you talk with a person who has worked for successful companies and spent time as a venture capitalist sitting on multiple boards, you tend to actively listen to the stories and glean what insight you can. When I spoke with Bruce Armstrong, CEO of Kickfire, it was such a conversation.

Bruce is looking to take Kickfire and disrupt the data warehouse market. Their product, as Bruce put it, helps bridge MySQL (which doesn’t understand data warehousing) with data warehousing (which doesn’t understand open source). He is now in the midst of a David and Goliath battle with Teradata — his former employer.

Question: Hi Bruce. Care to describe this battle with Teradata?

Answer: Actually, Tom, we believe the battle is less with Teradata’s existing high-end data warehouse business and more for the vast, under-served data warehouse “mass” market out there. As such, we don’t plan on taking the battle directly to Teradata or anyone else, but rather create a new market by providing affordable, easy-to-deploy data warehouse and data mart appliances for the mass market. The mass market for data warehousing includes rapidly growing small medium businesses as well as departments in larger organizations who simply can’t afford either their own Teradata machine or in a lot of cases even the charge-backs from central IT to time share on a Teradata machine. So, we hope to be complementary to Teradata, but realize there may be some skirmishes for budget dollars in the market today!

Question: You started as employee #16 at Teradata who boasts Walmart as a customer. Was that your start that brought you to running Kickfire today?

Answer: Yes, I was at Teradata for 15 years, Tom. I worked my way up from programmer to President after we went public and were acquired by NCR/ATT. I spent almost 10 years in the field working with customers – including Walmart – so I developed a pretty good idea of what they are looking for at the high end. After Teradata, I was GM of the Server Group at Sybase where one of the products I launched was Sybase IQ, the first column-store database on the market. Later, I was EVP of Sales Marketing at Broadbase – the second column-store database on the market and a pioneer of data marts. Since then, I have been involved in doing due diligence for a lot of the data warehouse startups, including Netezza, Greenplum, and Vertica.

Question: What did you learn from Teradata back then that you are applying to Kickfire today?

Answer: While I learned many things at Teradata that are applicable to Kickfire, the key concept for me was just how different the data warehousing high end is from the mass market. At the high end, once you’re able to break into the market with a new approach, it quickly becomes less about technology and more about services and support. Large customers just simply demand more hand holding. In the mass market, it’s just the opposite. Customers in the mass market expect the product to be highly packaged – easy to buy, install, and manage. They simply can’t afford lots of services and support, so the product needs to deliver high performance in a plug-and-play way.

Question: You have years of history in the data warehouse industry. Care to explain what makes you think the data warehouse industry needs changing?

Answer: While I believe every sector of the data warehouse market is looking for more performance, faster deployments, and lower cost, our goal at Kickfire is not necessarily to change the data warehouse industry but rather to enable more people in the market to take advantage of the benefits that come from data warehousing. Again, we’re going after an under-served market with rapidly growing small and medium sized businesses and departments in enterprises that simply can’t afford Teradata or Netezza. The change that’s required to enable the mass market is to get high-end performance out of a mass-market machine. Kickfire achieves this through our patented parallel-processing SQL chip. In a single chip, we are able to get the performance of dozens of general purpose CPUs. So, in the same way that NVIDIA has radically changed the dynamics of the graphics industry by encoding the graphics processing language in silicon, Kickfire has the same opportunity in the data warehouse industry.

Question: There have been recent acquisitions. Sun purchased MySQL. Then Oracle purchased Sun. Should the MySQL community be concerned?

Answer: We do not think so. We believe that with Solaris, Java, and MySQL, Oracle now has the kind of assets they will need to truly make Microsoft worry. With MySQL, Oracle will finally be able to undercut Microsoft as well as to dominate the high end in databases. As such, we think Oracle is going to continue to invest in MySQL. In fact, Oracle has already proven to be a good steward for InnoDB, the most popular MySQL storage engine they bought several years ago.

Question: About Kickfire, how does this chip technology bridge MySQL with data warehousing?

Answer: MySQL has become the de facto standard for online businesses by virtue of its open source business model and the fact that it has increasingly become production-ready for transaction processing applications. For data warehousing applications, though, MySQL remains very primitive. As such, customers struggle with database volumes as small as 50GB when it comes to reporting and analytics. The primary technical issue with MySQL regarding data warehousing is that MySQL does not have any query parallelization capabilities. This is where Kickfire comes in: using MySQL’s pluggable storage engine API, Kickfire takes over a query from MySQL and provides parallel-processing with the SQL chip allowing queries to run 10x – 1000x faster.

Question: Are you getting any response from the MySQL or data warehouse communities? Are they seeing potential benefit?

Answer: Yes! We have a large and rapidly growing pipeline of opportunities in the MySQL community. One of our early customers, Mamasource, has been using MySQL for years to drive their online community website. However, when it came to analyzing the clickstream data about their community in order to improve the user experience and increase ad revenue, MySQL hit the wall at less than 30GB of data. With Kickfire, Mamasource is now able to run their queries on average 20x faster and up to 600x faster for complex queries. More importantly, Mamasource can now scale their data warehouse to over 300GB, which allows them to increase from just one month of data to a full year.

Question: It’s hard to ignore Microsoft. How does Microsoft with SQL Server fit in the Kickfire world view?

Answer: Microsoft and Oracle are the predominant databases in the mass market today. As MySQL continues to penetrate the market (at a rate of 70,000 downloads a day, by the way), Kickfire will be brought into more and more Microsoft and Oracle shops. As such, one could imagine us building more specific features to co-exist with those databases in addition to MySQL.

Question: You spent time as a Venture Capitalist. We won’t hold that against you but what are your take-aways from being a VC?

Answer: My time as a venture capitalist was really more about board work than investing. As an operator, the VC firm I was with was interested in my ability to help identify interesting spaces, provide input on business plans and teams, and then work with portfolio companies from a board perspective in order to help generate value for shareholders. A lot of what I did was to dive deep into each function (sales, marketing, development, etc) of the portfolio companies and provide input to the boards (including the CEO’s) on where things could improve. Having served on 20+ boards, I got a pretty good idea of what works and what doesn’t work in start-ups and rapidly growing companies. The lessons I learned were really quite simple:

1. make sure you have a big market opportunity;
2. understand very clearly who your customer is and why they should buy from you;
3. build a very high quality product;
4. regularly assess your market position and look for ways to change the game in your favor;
5. don’t be afraid to tell the truth about the business – there’s always more than one way to create a market leader.

Question: It’s been a pleasure talking with you Bruce. Do you have any additional links or information about Kickfire you want to share?

Answer: Thank you – the pleasure’s all mine. Please come visit us at www.kickfire.com!

I’m tired of reading posts about business intelligence needing “an army of consultants“, concerns about “losing control over our BI system to a third party“, and only 13% achieved their business intelligence objectives.

Or Boris Evelson’s quote, “Business intelligence is still an art much more than a science.“

Or Hound of the BI-skervilles stating that data is safer inside a company’s firewall than outside in a professional data center.

Based on these comments, the way business intelligence is being done today just doesn’t make the grade.   There are all kinds of statistics to support this.

It’s hard to hear about the proverbial glass ceiling with the Business Intelligence (and data warehousing) industry. This ceiling acts as an invisible force stopping the industry from moving up or beyond it’s current paradigm. Perhaps the holy grail is on the other side; or maybe this is the end. And really there are only two choices: assume the ceiling cannot be broken and live happily or innovate around it breaking past the barrier and into more chaos and opportunity.

Shawn Dolley mentioned Netezza’s new blog, Previously Impossible. They quote a survey that shows just over 8 percent of employees actually use BI tools. And this is for BI-using organizations! The article also talks about the inflated usage statistics by BI vendors (they want to show value and sell more user licenses – usage is a must-have metric).

Many people mention the high cost of BI/DW. I wouldn’t mind that so much if there was equal value in the results, the ROI. I think any MBA would be hard pressed to find positive ROI with only 8 percent of employees using such a costly system. However, companies pay for large BI/DW initiatives either out of necessity or ignorance.

Let’s say it’s out of necessity with the mounds of data being captured daily.

Now what. There is a need for information but there is still much pain with BI pre and post implementation. Some vendors will jump on the “low cost” wagon with starter kits, blueprints, configuration tools, cheaper hardware, etc. The problem I hope more people are looking at, the real glass ceiling problem, is addressing the question, “why are so few using BI?“

Look at how many people use Google to research or find information. Google’s search success is because it’s easy and quick! When my father who is 73 can find stuff that’s a testimonial! Alternatively BI is still too confusing for the general employee. Dashboards. Analytical cubes. Reports. Forecasts. Anytime you need more than a few hours training (and re-training when you haven’t used it for a month), it’s too much. This, of course, assumes “BI for the masses”.

So will BI ever get to the ease and simplicity where my father could determine the best deal of potato chips and dip mix combination near his location?

Two Sides of the Same Coin

We’ve worked in the field of Business Intelligence for years, cumulatively decades. During that time, we’ve helped clients save millions of dollars.

We’ve also been in management positions as the recipient of information from Business Intelligence systems. During those times, we saw business people frustrated, limited, and spending millions of dollars on systems they didn’t have to.

You Are Not Alone

We believe passionately in efficient and effective businesses; and business people being empowered with their own information. Some of you might be thinking, “We have a Data Warehouse and cubes and reports! We have what we need so what are you talking about?”

Excellent question but first I want to share with you WHY I’m even bringing this up. We sat with a prospective client today talking about their current problems. I mentioned they weren’t alone, that I’ve seen this in many organizations, and proceeded to mention a few examples. Then it dawned on me.

I don’t want to tell people “you’re not alone” anymore. I don’t want to share examples of business disempowerment. Instead, I want to tell them they are one of the few left doing it that way!

So the best way to bring about this kind of change is to give away the knowledge. This is scary in an “information is power” industry but the gains/benefits/change out weigh the competitive edge we may lose.

Getting Back to Saving Millions

So what exactly are we giving away? We are sharing with you three ways to start the change within your own organization. We have five ways but we’re keeping two for some competitive advantage!

The goals are to a) save money by reducing costs and automating repetitive tasks, and b) make better decisions with readily available, fresh, accurate information.

Here are three specific ways that can change your organization and save money:

1. Identify significant points of interest

It’s more than asking what reports they need. People require different types of information to base their decisions upon. Look for places where it’s difficult to find answers. Analysts, key individuals, management are good places to start. They may not even know the points of interest they are missing so focus on what’s known first. When points of interest are identified, better decisions can be made.

2. Free people to do more creative work

When someone is doing repetitive tasks, the same task over and over, this is a candidate for automation. When tasks are automated, it frees people to do other creative work or spending time analyzing and producing answers. Find tasks that copy data into spreadsheets, use side of desk documents or databases, or manually aggregate numbers from reports. There is enough work for people without having to do repetitive tasks.

3. Aggregate One or More Sources

Bringing sources of data together will have roadblocks. Privacy of information,who controls the data, who has access to the data are hurdles that can be overcome with the right tack and skill. The benefit is being able to manipulate and translate the data into information. Data quality improvement and aggregating multiple sources can give more accurate answers but are not always needed.

Common sense should prevail for this point. And lots of it! I’m reminded of the adage, “when you’re a hammer, everything looks like a nail”. So keep in mind a centralized data warehouse isn’t always the answer. If one already exists, adding to it isn’t always the answer either. Think: efficiency, value versus cost, and that the business is constantly changing. Typically data warehouses are unable to keep up to business change.

And here’s a bonus point for 3.5 ways that we’re giving away.

Different Tools for Different Roles

Finding the available tools in the Business Intelligence market is hard. There are so many. However, choosing the ‘best fit’ tool can be even harder when you include the politics, costs, and persuasive sales people. We’ll talk more about this in a later post.

You may have realized while reading this post that these techniques do more than just save money, they also help your decision makers. We find when we go through this process with clients, they come away with a focus on what they really need to know and the importance of giving decision makers and analysts access to the information.

The rumors could be more than rumors for the SaaS BI company that focused on analytics for Salesforce.com. LucidEra used a variety of open source BI software to address a market need. People wanted SaaS BI and were paying for it. So what was the problem?

If you haven’t heard, the rumor is that LucidEra is no longer in business.

I want to say rumor until someone from the company verifies it. However I’ve heard from a couple sources now, which is good enough for me to publish this post. Plus their blog is down. Some Salesforce.com customers are looking for BI vendors again. And senior executives have moved onto other companies.

What went wrong?

Rumor #1: LucidEra had more servers than NetSuite!

One advantage that SaaS should bring to the table is ease of scalability. Without that you’re just on-premise software in someone else’s data center adding servers for new customers. SaaS architecture needs to have automated scalability.

Rumor #2: Not a multi-tenant architecture.

A few things could have taken them away from a pure multi-tenant architecture. The open source BI software, like Mondrian, may not have lended itself well to multi-tenancy. (Maybe someone technically savvy could verify or refute this?) Another issue can be the data model design. Replicating data models for each new customer isn’t multi-tenancy. I’m not sure why or where LucidEra moved away from a pure SaaS model. But that’s what I’m hearing.

Rumor #3: Customers loved them and their product.

If you want to please your customers, find out how LucidEra did their sales and customer support. They are an example of how a SaaS business needs excellent sales and customer support AND savvy engineers and architects.

At least this casualty wasn’t because of the economy or so the rumors go.