It’s a love thing…

0

Posted on : 30-06-2011 | By : Ben Stinner | In : Business Intelligence

It’s been three years that me and the blog have been together (truthfully, the date was on the 15th, and I didn’t forget honey, I was just really busy with work, so you understand now don’t you?… Ok, I’ll get you flowers icon smile It’s a love thing… )

I have to admit that looking back at older posts is sorta weird, cause I think my own voice in my writing has changed since then. But then again, I guess you never stop changing your own voice in your writing, just as you never stop changing yourself.
I also really learned to appreciate the blog more, as time passed. Just a few weeks back, Microsoft had organized a meeting for their IT bloggers. I was there. And it really made me look at all I got from the blog – both professionally and personally. You don’t expect that from a blog, or at least I didn’t…

Still, I know that you, yes you, my dear reader, have also contributed to this. I have gotten along the years quite a few comments on my posts that lead to new posts. That’s because you helped me with your comments look into something interesting or challenging. That’s how my post about Recursive Sum for a Matrix in SSRS started, back in the day.
Also, I understood, you really like free stuff – so it’s not just me (yay! icon smile It’s a love thing… )
And that you’re also interested in data.

I would really like to hear from you personally. What is it you would like to read about most in this blog?
Are you looking to read more about best practices of design and development? More about the fundamentals of MDX (or perhaps DAX)? More about Report Builder in its newer versions? More free stuff? More videos and less text?
Post me and the blog a comment – we’d love to hear from you.

Ok, now enough with the gushing – hit it Fatboy Slim!

 It’s a love thing….com/~r/EllaMaschiach/~4/9tqeL0fIn-k” height=”1″ width=”1″/

Article source: http://www.biblogs.com/2010/05/29/its-a-love-thing%E2%80%A6/

VM Ware Networking Options

0

Posted on : 30-06-2011 | By : Ben Stinner | In : Business Intelligence

Your VMware Fusion virtual machines talk to your network using a virtual network adapter. Inside your virtual machine, the guest operating system (Windows, Linux, etc.) believes that it is equipped with an ordinary (wired) Ethernet card. But Fusion patches this card to your Mac’s regular network connection, regardless of whether it is wireless or wired.

Using Fusion’s Virtual Machine menu, you can choose among bridged mode, NAT mode, and host-only mode. NAT mode is the default.

e0b28 network options VM Ware Networking Options

We’ll examine each of these network modes in turn.

What is bridged mode?

Your home or office network is probably equipped with a router for talking to the Internet. Bridging is a network term that describes extending a network without using a router. When you place your virtual machine’s virtual network adapter into bridged mode, your local wired or wireless network is effectively extended to your virtual machine. Your virtual machine becomes a peer of all the other computers on that network.

 VM Ware Networking Options

Although your virtual machine will connect to your local network using the same hardware your Mac uses, the virtual machine will retain its own fully independent network identity. Mac OS’s networking features cannot tell the difference between your virtual machine and a PC on your local network; Mac OS interacts with them over the network in exactly the same way.

Many users’ Macs get their IP address from their wireless base station or router, using a piece of software built into those devices called a DHCP server. In bridged mode, if the operating system inside your virtual machine (Windows, Linux, etc.) is configured to request an IP address from a DHCP server, your virtual machine will get its IP address from the same DHCP server your Mac uses.

What is NAT mode?

NAT is a network technology that protects one network from another. For example, your router probably also acts as a firewall: it protects your Mac by preventing unknown computers on the Internet from connecting directly to it. The term “NAT” is an abbreviation for “Network Address Translation”; the protection afforded by a firewall works (in part) by the router representing your Mac on the Internet. Your router substitutes its own address for your Mac’s.

 VM Ware Networking Options

VMware Fusion’s NAT mode is the same technology, but one layer closer to you. NAT mode protects your virtual machine from the other computers around it by placing the virtual machine on an isolated virtual network. Whenever your virtual machine wants to communicate with your office network or the Internet, it does so through a software firewall. This firewall is a component of VMware Fusion that runs inside Mac OS.

Notice that, in this situation, your virtual machine is not on the same network as your Mac and your router. Compare this to the diagram of bridged mode, in which your virtual machine is a peer of your Mac and your router.

In NAT mode, your virtual machine will get its IP address from a DHCP server supplied by VMware Fusion that runs in Mac OS, just as the firewall does. But this IP address will only be used for relaying communication between your virtual machine and the software firewall; that firewall will represent your virtual machine for its network communication with the outside world. Another way to think about this: from the Internet’s perspective, your virtual machine is sharing your Mac’s IP address.

What is host-only mode?

In host-only mode, your virtual machine is not only protected from your local network and the Internet, but also locked out of them. The virtual machine’s network world is wholly within your Mac.

 VM Ware Networking Options

Just as in NAT mode, your virtual machine will get its IP address from a DHCP server supplied by VMware Fusion that runs in Mac OS.

How do I choose the right network mode for me?

The right network mode for your virtual machine depends on how you plan to use it. In an office or home-office environment, with network printers and file sharing, bridged mode is probably best, particularly because bridged mode allows your virtual machine to use Apple’s Bonjour technology for finding printers. Windows’s workgroup features need bridged mode; only in bridged mode will a Windows virtual machine’s Network Neighborhood show the other PCs on your local network.

NAT mode is ideal when you want your virtual machine to be protected from other computers on your local network, such as when you are using a public wireless connection. Because, in NAT mode, your virtual machine shares your Mac’s IP address for purposes of external communication, it’s especially ideal for use when you are using a paid wireless service, such as in a coffeehouse or hotel. The use of bridged mode would require you to pay twice: once for your Mac and once for your virtual machine.

Host-only mode is useful for environments where your Mac has no network connection at all, or when you wish your virtual machine to be completely isolated from the rest of the Internet.

Here’s a way to think about the network choices:

Yes! But, unless you have configured your virtual machine to use fixed IP addresses, you must tell the operating system inside your virtual machine to release and renew its IP address. Renewing your IP address after you change network modes will automatically contact the correct DHCP server: the one on your local network if you changed to bridged mode, and the one provided by VMware Fusion if you changed to NAT or host-only mode.

To release and renew IP addresses within Windows, open a command prompt using StartRuncmd . Then, in the command-prompt window, give first this command:

ipconfig/release

then this command:

ipconfig/renew

Rebooting the virtual machine will also cause it to obtain a fresh IP address.

Article source: http://www.biblogs.com/2010/05/29/vm-ware-networking-options/

Changing up SAS Work for the Intelligence Platform

0

Posted on : 30-06-2011 | By : Ben Stinner | In : Analytics

There is a wide array of papers (from SAS, conferences, on blog posts, etc) discussing the need to move WORK (the SAS temporary files) off to other locations.
In the SAS Intelligence Platform it’s the same way, however there are a BUNCH of SAS configuration files to start worrying with. This is great because you can put WORK for one compute service in a separate location from another. You can also define the configuration for the entire system in one location.

The MAIN thing – (C:Program FilesSASSASFoundation9.2sasv9.cfg) applies options to the rest of the system. However this runs first, so you can overwrite this in any of the below configurations.

Each of the server configuration folders has two configuration files. sasv9.cfg and sasv9_usermods.cfg.
http://support.sas.com/documentation/cdl/en/bisag/60945/HTML/default/viewer.htm#a003147155.htm
SAS recommends that you modify or add options only to the sasv9_usermods.cfg (when available) and store these modifications in a backup location.

References:
http://support.sas.com/rnd/papers/sgf07/sgf2007-iosubsystem.pdf
http://support.sas.com/resources/papers/ETLperformance07.pdf
http://support.sas.com/rnd/emi/EbiApm92/sas92.ebiapm.win.pdf

Article source: http://feedproxy.google.com/~r/RealBusinessIntelligenceForRealUsers/~3/aZxJmX1y6Zc/changing-up-sas-work-for-intelligence.html

Mixing Business and Spirituality

0

Posted on : 30-06-2011 | By : Ben Stinner | In : Business Planning

We don’t often think about how we mix business and spirituality, mostly because the two don’t seem to share any common bonds. Business is about driving revenue, boosting sales, and making money. Spirituality is about finding a moral center, a place in this world, and what we can do to elevate our own sense of inner peace. Yet it is important, especially within our modern fast-paced culture, to find a balance between business and spirituality.

Can business and spirituality coexist?

The question has been raised time and time again and depending on your personal background and upbringing, you may find completely different answers than someone else. We have come to understand modern business as being a cutthroat, take-no-prisoners ideology and showing any compassion for others or a sense of limits to what you are willing to do to win at business as is considered a weakness.

Yet nations around the world have been founded on the foundations of spirituality. Some might call this religion, but the two are closely related and it seems as though mankind alters religion into a political tool when it suits them, though it is the spirituality that underlies those beliefs that define these nations.

As younger generations grow up, go to school and college, earn their degrees and then enter the workforce, they have been inundated with messages about the importance of money. It drives them, it motivates them, and ultimately it becomes their compass in life. While this trend is certainly not new to the world of business and entrepreneurship, it is fast becoming something of a plague in modern society.

Why can’t business and spirituality coexist? It has happened with some of the best and strongest companies in the world. The basic groundwork for dealing with this issue comes down to each individual and the importance they place on making money.

Boosting business is not evil

Just because you have a strong desire to elevate your business, make it more profitable, and to become a leader doesn’t make you an evil person. There are many businessmen and businesswomen who have worked hard and become successful while holding onto their strong sense of spirituality. It all comes down to a basic choice.

How important it the bottom line to you and your business? What are your limits when it comes to achieving your goals for your business? If you are a spiritual person yet you wish to make a million dollars with your business in five years, there’s nothing wrong with that. Unless you begin to conduct yourself in a way that leads you off the path you have chosen for your spiritual strength and salvation.

Merging business and spirituality requires foresight, a plan, and being able to hold fast to that plan. When you do that, there is nothing that can stop you from achieving your dreams.

Article source: http://ezinearticles.com/6350459

10 Things to Consider Before Planning for What’s Next in Your Business

0

Posted on : 30-06-2011 | By : Ben Stinner | In : Business Planning

Suppose you just won a trip around the world. What are you busy doing, now that you’ve won? You’re planning, right? You’re checking out the weather, the accommodations, the best routes, the times to see the sights you want to see, the rules and restrictions, and all the details so that your trip will be the best it can be. You won’t just jump on the plane or ship without any planning. You’ll plan out exactly where you want to go, what stops to make, how you want to get there, and what you want to do when you get there. You’re thinking about what you will take with you, who you’ll take with you, and you are making contingency plans in case parts of your ideal plans don’t work out.

The same should true for your company and its “journey” to what’s next. You and your team will get the maximum enjoyment, results, and success from a journey that is planned. So if you haven’t done it already, now is the time to do the homework and research necessary to create a business plan that sees your company through to what’s next. Take time to consider the following:

 

  1. What are the market demands for what your company offers?
  2.  

  3. Does the market need educating about what it needs from your company?
  4.  

  5. What changes, suggestions, or requests have your clients made that you may want to implement in your services?
  6.  

  7. Do you have staffing needs or changes to make?
  8.  

  9. Does your organization’s mission statement still ring true or does it need to be modified?
  10.  

  11. Are your company’s values clear?
  12.  

  13. What past goals were not met? Are these goals still important? You may have a “pet project” your team really wants to do, but is that something the consumer needs or wants?
  14.  

  15. Think about your company’s strengths and weaknesses. What are the opportunities and possible obstacles? What’s working well, and what isn’t?
  16.  

  17. Do best case-worst case scenario thinking. Start developing responses to “what ifs” that might occur. For example, what if your industry experiences an unexpected, unprecedented change in demand for your services? Consider such things as changes in regulations or demographics. What potential strategies might work?
  18.  

  19. Is it time to just shake it all up and take a fresh, new, unpredictable approach? Sometimes companies stay with what works because, well, because that’s what works! Even “what works” can get stale and predictable… and your clients can lose interest. It could be time to turn your company on its head and get a whole new look! Think about what elements make it successful and what elements could be restyled, refurbished or refreshed without impacting the workability and success of your company.

Have fun and be creative! Don’t limit your thinking to what your company has always done. Share your wildest ideas with your team and listen to all of the team’s ideas. You may come up with a future for your company that is so exciting you’ll want to get started right now writing the strategic plan for getting there!

Article source: http://ezinearticles.com/6383358

5 Reasons Why NOT to Write a Business Plan

0

Posted on : 30-06-2011 | By : Ben Stinner | In : Business Planning

Looking back I don’t think I’ve ever written specifically about business plans before and the more I think about it the more I realise why, because they’re usually a pretty boring subject.

Let’s be honest, business plans are not fun things to write or to read, so I’ve quite happily avoided the subject up to now.

So why the sudden change of heart?

Well, my view on them is a little unconventional and I figured it’s time I told you why.

Admittedly I can think of a few reasons why you might write a business plan, but I can think of whole lot more reasons why you shouldn’t waste your time on them.

The fact that business plans are quite boring to write isn’t a good reason not to write one. However, here are my top five other reasons why you shouldn’t waste your time on one:

Business Plans are Guesswork

Almost everything you write in your business plan will be guesswork. Even worse, there’ll be some pretty inflated and optimistic claims in there that won’t reflect reality at all. Here’s some reasons why you’re going to be doing a lot of guessing:

o You don’t really know how many widgets you’re going to sell until you try to sell some.

o You don’t know what price point to use until you test to see what your market will accept.

o You don’t know whether your customer even wants your product/service until you find a market and ask them.

o You don’t know how to market your business until you try various methods, split test and see what works.

o You don’t know how new tools like Twitter might transform you business, if they aren’t yet invented when you write your five year plan.

So we’ve established that a business plan is largely a work of fiction. What other reasons are there not to write one?

They Give You a False Sense of Security

I don’t mind admitting I’ve written business plans in the past for some pretty bad ideas. Yet, I managed to make the business sound really viable, enough to fool even myself.

This is a dangerous thing to do because having a plan in place which looks great can give you a false sense of security.

With all the guesswork involved you’ll have a free rein to make your plan reflect what you want to happen, rather than what is likely to happen. This can lead you to think you have a great idea when maybe you don’t.

If you use your plan to convince other people it’s a great idea then the problem can escalate. Now those you seek approval from also think it sounds like a good idea, all based on fiction.

Those who have read anything I’ve written before will already know that I believe guesswork doesn’t contribute to a good business mindset and should be avoided whenever possible.

It’s best to just take action, try it out on a small scale with minimal cost and you’ll know very quickly whether it’s a good idea or a terrible idea.

Writing a plan doesn’t make your idea any better, it jut makes it sound better.

No-one Reads Business Plans

Besides yourself while you’re writing it, who are you expecting to read your shiny new 80 page business plan?

An investor might pretend they’ve read it all but the chances are they’ll only read the salient points. They could have established these anyway with a five minute conversation with you.

If you ever have time to read your plan yourself after you’ve written it then you’re probably not spending enough time actually running your business.

You can refer back to specific sections, but these will probably be outdated and obsolete just weeks after you’ve written them.

If you do refer back to it, the optimism you had when you wrote it might inspire you, until you realise the path your business has actually taken is nothing like what you planned.

This isn’t a failure on your part to follow the plan, it’s often a good thing. Let me explain why in my next point.

Your Business Should Be Flexible, Not Rigid

Your business needs to be flexible and dynamic. It needs to adapt to your customers needs, to the changes in the market, to the economic climate, to the available technology.

It needs to evolve as you experiment with advertising media, copy, headlines, price points, offers, social networking and split testing.

As you create a market you’ll be surveying them to find out what they really want from your business and then you’ll be creating exactly that.

You’ll be creating goals that you want to achieve and tracking them weekly or monthly using a spreadsheet.

The common factors here are flexibility, experimentation and adaptation.

Now tell me where a rigid five year business plan fits into this scenario? It doesn’t, because a rigid plan is the last thing you need.

Business Plans are a Distraction

Depending on what sort of business plan you’re writing it could take from a day to a week to write. That’s valuable time you could be spending actually creating and trying out your business.

I’m all for knowing exactly what you want from your business and creating one which supports the lifestyle you want. But let’s not be getting goals and plans mixed up here.

You should definitely have goals, and you should definitely be tracking your progress against them.

However kidding yourself that you know at the very beginning exactly how you’re going to achieve them isn’t going to help, because you can’t possibly know without experimenting first.

Allow yourself to be flexible, experiment and work out what takes you to your goals the faster. That’s a much better use of your time than writing a rigid plan that’ll be out of date a week after you wrote it.

I know my views on this might be a little controversial but I feel it’s only right that I share my thoughts with you. The time you save by not writing a lengthly plan will allow you to concentrate on more important things, like making your business actually happen.

Now my guess is, if there’s one thing that’s been annoying you the most as you read this it’s probably that if your business needs external funding then you do need a business plan. I agree, if you have no choice then you have to create on, that’s obvious.

If this is the case for you then write whatever your potential investors want to hear, as long as it sounds realistic.

Don’t spend forever on it though because when your funding’s in the bag you should put you business plan in your bottom draw and start the real work, figuring out what actually works and doing more of that.

 

Source: http://businessmindsetexpert.com/blog/

Article source: http://ezinearticles.com/6386819

Live Blogging Data Integration in Large Organizations

0

Posted on : 30-06-2011 | By : Ben Stinner | In : Business Intelligence

This is live blogging from the Donald Farmer seminar from the SQL BI convention in Tel Aviv. I would really like to thank Hamada who lent me his netstick so I can blog – Thanks Hamada!

Business Intelligence is about people not about IT.
Your data is always connected to real world and in direct connection to it. It’s important to know your business for you to design a good Decision Support system.
BI has always been in the middle of the spectrum – tactical decisions. CEO and operational workers will usually look just at your BI reports, they won’t create them themselves.

For the CEO, you have strategic decisions – Dashboards and Scorecards. Getting data aggregated.
For Operational Workers, you need embedded reports, analytic APIs so that they can get the info from where they’re working (no special knowledge needed). Getting the data to the right place.

SSIS for the operational level. SSAS to model the operational data giving a consistent view of the data for the entire organization. PerformancePoint Services giving the high level strategic view.

Decisions flow from the strategic level down to the operational end. Traditional BI from the tactic level may move up to the strategic level and change their decision.

Questions you need to ask yourself: Who am I giving the info to? What level of info do they need? How will they use the info to make decisions?

Prototype a BI project. Test the accuracy and performance of the DW and also see that people actually use the solution you developed. Do Beta testing to see how people use the reports – look at the query log their using and tune the DW according to that.
An outer user should be treated differently only from the security aspect, but except for that you treat them just as any other user. This is unless you just give them a report rather than let them use freely the data.

Operational – won’t learn new application – reports, embedded functions, mobile BI
Tactical – reports, cubes, annonationas and SharePoint workflow.
Starategic – PerformancePoint Services, annonationas and SharePoint workflow.

SharePoint is the largest growing server in Microsoft. SharePoint is growing because people need to share info and work on it together in a managed way. SharePoint 2010 has social features as well, as collaborating through social networks has become more and more important. Where you’re collaborating and where you’re sharing, is now also where you use your Business Intelligence, which is also integrated into SharePoint.

1. IT oversight – how much control IT has

2. Organizational alignments – coordination with the goals of the organization

3 S of a DWH:
Scale
Standards – built to. Critical. Laws for instance about HR DWH (for instance) and who do you allow against it (Security).
Skills

DWH and Reports – high on 1 + 2
Spreadsheets desktop databases – low on 1 + 2

Between the two there exist:
Shared sheets, Databases
Departmental Data Marts

Sharepoint as a way to bind all of these together.

New to SQL Server 2008 R2 – Master Data. Master Data is not a DW. DW is for analysis and hold data of current state + history. Master Data is about the current state of the Business Entity – is for operational use. Master Data – keeps the data synchronized for all the departments.

Next version of SQL Server in 2011 will include more Data Quality capabilities based on abilities acquired from Zoomix.

ETL – Extratct Transform Load. Moving data in the organization. SSIS.
EAI – Enterprise Application Integration. Moving messages in the organization. Biztalk.

Biztalk and SSIS work both on 3 layers: Data, Logic, Presentation. Coordinate on all levels.

Request and Reply – with ETL updates. Cross reference table is updated by an ETL process in batch mode.

 Live Blogging Data Integration in Large Organizations Live Blogging Data Integration in Large Organizations

Article source: http://www.biblogs.com/2010/05/30/live-blogging-data-integration-in-large-organizations/

SSIS 2008 improvements over SSIS 2005

0

Posted on : 30-06-2011 | By : Ben Stinner | In : Business Intelligence

This is continued live blogging from the Donald Farmer seminar from the SQL BI convention in Tel Aviv of Data Integration in Large Organizations.

SSIS 2008 improvements over SSIS 2005

C# scripting can reference all .net assemblies
VSA replaced by VSTA

Pipeline Scale Up

Tuning SSIS through looking at the buffers on the Data Flow.
In SSIS 2005 you would add buffers and break up flows

SSIS Tip:
Tools Options BI Designers IS Designers:
Control Flow Data Connect
Data Flow Data Connect
Check them both to connect the components automatically

Use Row Sampling component to do performance testing.

One buffer structure goes all through the process.
We load data in the beginning but we don’t need it to move in memory. The problem is when one buffers slows down the process, it slows down for all the buffers before him as well.
In 2008 we have benchmarks for each buffer and if needed, create buffers artificially (rather than manually as we had to in 2005).

Asynchronous component – the input and output are different. We don’t know what have in the middle.
Sort – asynchronous, creates a new buffer as you can exclude a column.
The other components don’t need to be held up when an asynchronous component doesn’t run quickly, as they don’t depend on it.
Advanced Editor for the component: SynchronousInputID exists for synchronous components (and not for asynchronous).

ADO .net support – for improved source UI and added destination adapter (enabling you to use SSIS to load data into providers other than SQL Server).
The performance is still 30% slower because of the use of ADO .net provider, but it enables greater flexibility.
Use OLEDB providers as is, preferably don’t use .Net OLEDB providers through ADO .net providers unless you have to.
When you connect using the ADO .net, you will only be getting a list of tables. If the data isn’t in shape of tables, you won’t see anything in the dropdown list.

Use ADO .net if you have to use ODBC, or if the component you’re using requires it, or if you need it for your Script component.
Your default should still be OLEDB.

Import Export Wizard is much smarter:
ADO .net support for source and destination
Data types conversion – does only accurate conversions

Starting Import Export from the program menu enables straight execution. Starting it in SSIS Package requires execution to be done manually.

Persistent cache usage reducing DB and memory usage.
Cache in Lookup enabling more control over sharing and lifetime of reference data.
Cache potential cons – requiring more disk storage and data in it available only for SSIS.

New to SQL Server 2008 that’s useful for SSIS 2008:
SQL Merge Syntax
Change Data Capture CDC (works only against SQL Server or through partner product against other sources)

Enhanced supportability:
In case SSIS crashes, SuperDump collects all the data needed to analyze why it crashed. Enables quicker time to analyze and treat the problem.

Data Profiling which enables you to react to the data quality. Enables analysis of the amount of nulls in a column, the uniqueness of a field etc.

Community Samples on Codeplex – code for you to use freely (SharePoint List adapter for instance).

More connectivity out of the box in SSIS 2008. The Attunity connector which is inside SSIS 2008 moves data into Oracle quicker than with the connector Oracle gives you.

 SSIS 2008 improvements over SSIS 2005 SSIS 2008 improvements over SSIS 2005

Article source: http://www.biblogs.com/2010/05/30/ssis-2008-improvements-over-ssis-2005/

SSIS 2008 Performance and Scalability

0

Posted on : 30-06-2011 | By : Ben Stinner | In : Business Intelligence

This is continued live blogging from the Donald Farmer seminar from the SQL BI convention in Tel Aviv of Data Integration in Large Organizations.

 

 

The importance of measuring to improve performance

Understand and measure hardware:
How many CPU cores?
How much memory?
How fast is the IO subsystem?

Understand the potential bottlenecks:
Understand the limits of the Source System (usually a bottleneck as it’s usually slower than the new system)

Gains from the source system:
Find better drivers
Configure the driver
Optimize IO and network configuration

Measure Speed per Connection: Row / sec = Row Count / Time of the data flow

Improving the speed of the Source
Use multiple connections (not possible for a text file, but is possible for a DB with a query that takes CustID 1-1000 and then 1000- 2000 etc)
Convert the data from string to the relevant type (like int) already at the source – can create better performance.

FastParse – doesn’t have any error handling. We have to guarantee that the integers are indeed integers else we won’t get the error and it just won’t execute. We gain better speed.

Don’t assume you know where performance is best. Test it to see where it will be best – source, transformation or destination.

SSIS will take all the memory possible for it. There is no memory management for it at the current version, though it is planned for future release.

 

Tuning the Data Flow

Use the NOLOCK hint to remove locking overhead in large table scans
SELECT only columns you need

Network Tuning
Change the network packet size in the connection manager – put in the max of 32767, cut it by half and see how it changed performance.

Make data types as narrow as possible
Casting data types is expensive, especially in dates
Think of money, float and decimal – consider how effective they are.

SQL destination – guarantees great performance if the DB is on the same server as the SSIS.

Use partitions and partitions SWITCH load data in parallel and in great performance and then use SWITCH to connect all the tables together.

Even if you use split, you don’t have to union all them. You can load the data in parallel to the DB. This is dependant of having an index on the table.

Simplify the logic as much as possible.

Change the Design
Don’t Sort unless you absolutely have to (a time you do need – joining data when one source is text file that needs to be sorted). Usually you don’t need to use Sort. Use the capabilities of the Source System (indexes, set based operations).
Sometimes T-SQL is faster – taking advantage of Indexes or set operations: an Update by T-SQL, group by, sum.

Delta detection: if more than 10% of your data has changed – reload it!

Do minimally logged operation if possible.

Designing for parallelism
Partition the problem: partition source data, partitioning the target table.
Design to stay in memory.

Schedule it correctly.
Right now, we’re using SQL Agent, suppose to get a scheduler for the next version of SSIS.

Lookup in SSIS gives good performance, close in performance to T-SQL.

 SSIS 2008 Performance and Scalability SSIS 2008 Performance and Scalability

Article source: http://www.biblogs.com/2010/05/30/ssis-2008-performance-and-scalability/

BI costs Fortune 500 millions

0

Posted on : 30-06-2011 | By : Ben Stinner | In : Analytics

I don’t normally like to bring bad news up, although the Business Intelligence industry desperately needs a new guiding light. However there is always something to be learned from failures and bad news. Just like the first “F” I received in university… a wake up call for me that kicked me in the ass!

Dynamic Markets did market research, posted by TradingPattern.net and found that “BI failed to impact decision making in US and UK companies.

TradingPattern.net posts about industry risks and value creation. 218 operational execs and front line mgmt were surveyed. Some quick stats for you:

  • 76% were forced to make decisions because not all the info was available in time.
  • 63% believe that BI reports are simply reference documents used to justify after the decision is made.
  • 70% do not receive reports that provide predictions about problems or potential opportunities.

Okay, those aren’t great stats for an industry that promotes itself as “providing timely, accurate results to make fact-based decisions.” A more alarming stat is the cost to companies — the impact of inadequate intelligence.

$478,868 in lost revenue!

That really hits home for me. That doesn’t include the cost to implement a BI solution in the first place. Now if we take a step back and look at the industry. There are a plethora of tools, consultants, methodologies, and training courses but for the Fortune 500, BI is hurting companies more than it is helping.

Now there may be some very valid reasons for this. Fortune 500 companies are large, very complex business with multiple departments all intermingled. They have high volumes of data that BI needs to distribute over a large geography. Plus just understanding their business to build a BI solution would be a tremendous effort. But as my papi would say, “you’re giving into excuses.”

So if we assume these stats are true and speak to the Business Intelligence industry in generalities, then we should be trying to figure out, in big leaps and bounds, how to make this work. There needs to be a vision… perhaps many visions on taking BI forward. Other software industries have gone through regular change and come out better for it. Isn’t it time for Business Intelligence to go through it’s paradigm shift?

Article source: http://biforbusinesspeople.blogspot.com/2008/07/bi-costs-fortune-500-millions.html