I had a few conversations, actually more like 6 different conversations, pertaining to BI (Business Intelligence), OLAP (Online Analytical Processing), Cube or UDM (Unified Dimensional Model) Processing, ETL (Extract Transform Load), normalization and de-normalization for Reporting Purposes, and other such related topics.  It seems that there are tons of BI Projects starting up around the area.  Some of the projects are small “it’s just a possibility” type of project to the fully ramped we’re going to “toss a few million at this and get some serious BI out of it” type of project.

With all these conversations I decided to do a quick write up of what BI is all about.  I figured, what better way to refresh the memory of experience than to do some writing on the topic.

First I gotta knock out the crazy acronyms.  There are a bunch.

- AMO – Analysis Management Objects.  API for Analysis Services (SSAS). - OLTP – Online Transaction Processing.  OLTP is the storage medium for databases that are used in production environments for day to day input, output, delete, and update of data. - OLAP – Online Analytical Processing.  OLAP is the storage medium specific to processing of BI reports and such.  It is geared not toward input and output, but solely output and query reporting via MDX. - KPI – Key Performance Indicator.  A primary indicator for a business, usually displayed via a dashboard or similar report for change reference. - BIDS – Business Intelligence Development Studio.  Basically this is Visual Studio 05 with the appropriately installed templates and such for building, maintaining, and developing data OLAP Cubes. - ETL – Extract Transform Load.  The process of moving, copying, transforming, or otherwise changing data during movement from one location into another; usually from an OLTP source to an OLAP source.

- SSAS – SQL Server Analysis Services. - SSIS – SQL Server Integration Services. - XMLA – XML for Analysis.  Used for scripting SSAS. - MDX – Multidimensional Expressions.  Similar to SQL in syntax but designed for execution against OLAP Cubes for reporting demands. - ROLAP – Relational Online Analytical Processing.  Data is stored in a relational database for the cubes. - MOLAP – Multidimensional Online Analytical Processing.  Data is stored in a proprietary structure for fast retrieval. - HOLAP – Hybrid Online Analytical Processing.  Data is stored in a mix of relational and multidimensional mediums. - BI – Business Intelligence.  A general description of the particular part of the industry that focuses on OLAP Warehousing, Reporting, and related services and solutions. - DSV – Data Source View.  A collection of tables located in the data warehouse, usually a representation of logical keys and relationships. - UDM – Unified Dimensional Model. The measure groups and dimensions that define BI data.  Synonymous with a cube.

 There are also a few key concepts that one must understand.  I’m just going to discuss and define these concepts below, in a somewhat haphazard way as I think of them.

A dimension is a way data is cut, what appears vertically or horizontally on a report, to display information based on data.  A dimension can be broken out by time, linked, parent-child, standard or other means.  The data the dimensions cut are stored via a fact table.  A fact table stores all the detailed values for the measures or facts.  A fact table is usually a denormalized table with the bulk of the data to be reported on.  The main thing to know, is that each dimension is a way to break out the data, across various measurements such as time.  Another key word to know is that when data forms grouped attributes they are referred to as a natural hierarchy.

One can also have a dimension table that has a break down of information to use as a dimension cut across the data.  For instance a product could be displayed in the fact table, and the individual rows could be aggregated across the product types.

One odd thing, at least for newcomers to the BI world, is how often a relational database table has an application key but a surrogate key is assigned the row once it is in the OLAP Cube or Warehouse.  The reason this is often, if not always done, is because the application key could lose its uniqueness once brought across ETL or stored over a period of times to represent points of data.  If this occurs one needs a surrogate key to prevent confusion about which row is which.

I’ve puttered out at this point, and am contemplating posting this incomplete write up at a wonderful 1:00am.  But instead I’ll sleep on it so I’ll probably post it around 8am or so.  I’ll be writing a follow up to complete out the basic concepts and ideas behind BI.  In the near future, maybe even via a screen captured VLOG entry I’ll post how to actually setup and prepare a sample OLAP Cube for reporting via BIDS.  In the meantime stay tuned for the awesome power of the Snowflake and Star Designs!

I got an e-mail tonight just a few minutes ago, I am absolutely stoked beyond all recognition.  I jumped out of my chair and scared the cats even, and I think my girl friend thought I’d busted a gasket too.  The Inspiron 1720, Intel Core 2 Duo T7300, 2.0GHz, 800Mhz, 4M L2 Cache has shipped!  I can’t wait, just a few more days now!

YEAH!  Vista, VS 08 Beta, Silverlight, and more – here I come! 

After almost a year of frustration with personal and professional projects utilizing TFS (Team Foundation Server), I have decided to return my personal projects to Subversion.  The time, effort, and the costs are far to high for me to use TFS on my personal projects.  It is somewhat sad, as the potential with TFS is massive, but the core element, the source control, has failed me too many times and ate up too much time for me at this point.

With that in mind, I present my most recent re-diving into Subversion for my source control.  I’ll be giving TFS another go in a few months, especially when VS 2008 and SQL Server 2008 are released, but for now it is good ole’ reliable Subversion.

To get Subversion up and running nice and quick like, follow these turbo boosted instructions.

First download the following:

1. Head over to the main Subversion site at Tigris and download the latest copy of Subversion. - After that I grabbed a download of the latest TortoiseSVN for Explorer Integration. - Last I picked up ankhsvn also on Tigris.  If you want to keep up with happenings for ankhSVN check out Arild Fines Blog.

Once those are downloaded get to installing.  An even faster way to setup Subversion and TortoiseSVN is also available via SVN-1.  It is a single click install basically, well that’s how it’s advertised but there are some extremely easy steps that need clicked through.  If you want to use this instead of installing the items separately that is fine, just make sure to grab ankhSVN above to follow the rest of the steps.  I personally like to install the packages separately so that I am intimately familiar with what is and is not actually being installed.

1. Install Subversion.- Install TortoiseSVN.- Install ankhSVN.

With those installed it is now time to setup the repository and get some source code under source control.  The following instructions are also shown in this video.

1. Create a new directory, ideally something like SubversionRepository so it is easy to tell what the directory is for.- Right click on the newly created folder and select “Create Repository Here”.- A prompt will come up asking to choose either “Native File System (FSFS)” or “Berkeley database(BDB).  I always choose the native file system.  When complete you will receive a dialog that states, “The Repository was successfully created.”- I always check the repository, just to make sure I had everything clicked on correctly.  In the repository directory you should see the following folders;  conf, dav, db, hooks, locks, and the following files;  format and README.txt.  DO NOT PUT THINGS IN THIS DIRECTORY, JUST LEAVE IT AS IS!- To do a full import of items into the source repository navigate to the directory that has the project files in it.- Right click either on the root, or individual files and select “Tortoise SVN” and then the “Import” option.- A dialog will appear.  Enter the proper Url, the image has an example of a physical file location, and click “OK”.- Another dialog will come up to show the status of the files being imported.  When it finishes just click “OK” again.  The files and folders are now in the repository.- - Check out the drive space now, I do it with DiskView Visualizer since it’s ridiculously easy to get a view of space changes in a matter of seconds.  Now that you’ve added something there should be a few MB in the db directory, up from the default standard load db directory of about~100k or so.

Now let’s say someone else needs to get a copy of the project folders to work with.  This is unbelievably easy, so don’t think it didn’t happen once you’ve done it once.

1. First right click anywhere in a folder, on a drive within the Windows Explorer.  On the right click menu “SVN Checkout…” should be available, select it.- The check out dialog should now appear.  Enter the information pertinent to your installation and project like what appears in the screen shot.  Click “OK” when you’ve entered all of the information. - If you haven’t created the check out directory you will be prompted by TortoiseSVN to create them. - Once the checkout is finished, you will be provided a dialog as shown in the following screen shot.  Click “OK”. - After all this you will see the directory that was added in the folder structure as displayed in the following screen shot.

My next Subversion Tutorial will cover some of the steps to update, check file differences, and commit changes back to the repository.  Stay tuned! 

Technorati Tags: TFS, Team Foundation Server, Visual Studio, Subversion, Tortoise, ankh, SVN

del.icio.us Tags: TFS, Team Foundation Server, Visual Studio, Subversion, Tortoise, ankh, SVN

I’ve got at least three uses for video;  Filming my drifting efforts at PIR, getting screen captures for samples I’d like to present via Visual Studio per a recent discussion I’ve had, and of course for general editing, manipulation, and entertainment value of filming things from planes, trains, to goofy people stunts or whatever.  It’s all rather interesting, and becoming a more and more vested interest of mine.  I believe at some point I’ll need to plunk down the bills for a video camera and other gear.

On the topic of screen capture, screen casting, and hosting of said video I dug up a cool tool for just these topics.  Techsmith has a tool called Camtasia Studio for recording, enhancing, and sharing screen captures and recordings.  They also have a screen cast video media hosting service with seamless integration into Camtasia Studio.  Techsmith of course makes the almost famous SnagIt screen capture utility too.  I’ve used this for years and if their Camtasia Studio is anything like SnagIt in quality it will definitely be worth it.

Looking at this for a short bit also made me realize this effort might even integrate into some of my other personal projects I’ve been working on with a friend of mine.  Some of the functional needs of what I’m searching for a central hosting “vehicle” I’ll call it.  Something along the lines of Flash or Silverlight.  I’m mid-research on these topics but hope to finally dig up something soon.

So stay tuned, as soon as I can muster the time together, I’ll give a screen cast a shot.  Heck, maybe I’ll even get my ugly mug displayed in there too. 

I posted an example a few days ago regarding ‘ref’ and ‘out’ parameter keywords.  My main purpose was to present a “tip o’ the day” via a code example of ref and out parameters similar to some questions I had been posed earlier in the week.  One of my stellar coworkers pointed out that the examples are fine and all, but really don’t point out what really occurs when trying to assign objects versus base value types, and properties of those objects etc.  With that he threw together some great examples that I nabbed from him (with permission of course).  Thanks Shane for the snippets. 

So check out these examples…

1 using System.Diagnostics;

    2 using System.Reflection;

    3

    4 namespace ConsoleInterface

    5 {

    6     public class TestParam

    7     {

    8         private int _TestProp;

    9

   10         public int TestProp

   11         {

   12             get { return _TestProp; }

   13             set { _TestProp = value; }

   14         }

   15     }

   16

   17     public class TestingRefAndOutParameters

   18     {

   19         public void CallingMethodChangeProp()

   20         {

   21             TestParam testParam = new TestParam();

   22

   23             testParam.TestProp = 2;

   24             Debug.WriteLine(MethodInfo.GetCurrentMethod().Name + ” Start testParam.TestProp=” + testParam.TestProp);

   25

   26             TestMethod(testParam);

   27             Debug.WriteLine(MethodInfo.GetCurrentMethod().Name + ” End testParam.TestProp=” + testParam.TestProp);

   28         }

   29

   30         public void TestMethod(TestParam testParam)

   31         {

   32             testParam.TestProp = 4;

   33         }

   34

   35         //**********************************

   36

   37         public void CallingMethodChangePropChangeObj()

   38         {

   39             TestParam testParam = new TestParam();

   40

   41             testParam.TestProp = 2;

   42             Debug.WriteLine(MethodInfo.GetCurrentMethod().Name + ” Start testParam.TestProp=” + testParam.TestProp);

   43

   44             TestMethodSetObj(testParam);

   45             Debug.WriteLine(MethodInfo.GetCurrentMethod().Name + ” End testParam.TestProp=” + testParam.TestProp);

   46         }

   47

   48         public void TestMethodSetObj(TestParam testParam)

   49         {

   50             testParam.TestProp = 4;

   51

   52             TestParam testParam2 = new TestParam();

   53

   54             testParam2.TestProp = 3;

   55             testParam = testParam2;

   56         }

   57

   58         //**********************************

   59

   60         public void CallingMethodChangePropChangeObjByRef()

   61         {

   62             TestParam testParam = new TestParam();

   63

   64             testParam.TestProp = 2;

   65             Debug.WriteLine(MethodInfo.GetCurrentMethod().Name + ” Start testParam.TestProp=” + testParam.TestProp);

   66

   67             TestMethodS etObjByRef(ref testParam);

   68             Debug.WriteLine(MethodInfo.GetCurrentMethod().Name + ” End testParam.TestProp=” + testParam.TestProp);

   69         }

   70

   71

   72         public void TestMethodSetObjByRef(ref TestParam testParam)

   73

   74         {

   75             testParam.TestProp = 4;

   76

   77             TestParam testParam2 = new TestParam();

   78             testParam2.TestProp = 3;

   79             testParam = testParam2;

   80         }

   81

   82         //**********************************

   83

   84         public void CallingMethodChangePropChangeObjByOut()

   85         {

   86             TestParam testParam = new TestParam();

   87

   88             testParam.TestProp = 2;

   89             Debug.WriteLine(MethodInfo.GetCurrentMethod().Name + ” Start testParam.TestProp=” + testParam.TestProp);

   90

   91             TestMethodSetObjOut(out testParam);

   92             Debug.WriteLine(MethodInfo.GetCurrentMethod().Name + ” End testParam.TestProp=” + testParam.TestProp);

   93         }

   94

   95         public void TestMethodSetObjOut(out TestParam testParam)

   96         {

   97             testParam = new TestParam();

   98             testParam.TestProp = 4;

   99

  100             TestParam testParam2 = new TestParam();

  101             testParam2.TestProp = 3;

  102

  103             testParam = testParam2;

  104         }

  105     }

  106 }

As you work through each of these methods, realize what is going on with the object, value type, or other parameter manipulations as the method executes.  You’ll notice that even though something is set to another value, outside of the method it is not particularly going to have the value changed.  Take the following method for example:

   48         public void TestMethodSetObj(TestParam testParam)

   49         {

   50             testParam.TestProp = 4;

   51

   52             TestParam testParam2 = new TestParam();

   53

   54             testParam2.TestProp = 3;

   55             testParam = testParam2;

   56         }

In the code, when reading it directly, it would seem that based on the second assignment of testParam = testParam2, that the TestProp property would be assigned the value of 3.  But because of the way parameters are passed in C# one actually ends up with 4 instead of 3.  One must know the differences between value and reference type passing, for more information check out this great write up or check out the MSDN article on “Passing Parameters” to really dig into what is going on.  Another great write up on what is getting put where in memory.

I hope these examples worked to display more effectively what is functionally happening with ‘ref’, ‘out’ and general parameter passing in C#.