I've Returned, In My Mind, Back to Work

One of the notable things I heard at work today which I thought was just hilarious, is that Microsoft still spends significant time and effort publishing marketing crap with FLASH!!!

I’m all ready to jump on the Silverlight band wagon, hell, I already have.  But Flash, why are they still using Flash?  The stuff I’ve seen they could easily do with Silverlight in about the same time with an editing tool like Blend.  In addition to that they could help themselves by getting this pushed across their site.

Since I was snooping around Todd Anglin’s Site, he has a good entry about it too, “Memo to Microsoft Marketing:  Silverlight is out of Beta“.  I almost can’t believe it.

…however…

…all of this silliness from Microsoft brings me to a thought that has popped into my head on a semi-frequent basis lately…

Microsoft doesn’t just produce bloated software anymore, they have become excessively bureaucratically bloated also.  Not in a normal corporate way like they have in the past but in a, “bad for the company it is gonna start sinking the stock price if they don’t become performant”, type of way.  Come on Microsoft, get lean, but don’t be stupid about it and just whack half your employees or anything.  Just wise up about hiring, stop hiring so much, and for the love of sanity let those that aren’t good start leaving or moving on.  I understand there has to be some serious turnover in some departments.

The MVC is top notch, the extensions are great, the CTPs rock, Visual Studio 2008 is great too.  So now the operating system team, the marketing groups, and others need to start opening up and performing in a similar way instead of publishing crap in Flash like Hey Genius (even though it is kinda neat), and…  come on… Defy All Challenges!!!!  Maybe by the time readers click on those links it’ll be re-posted in Flash?

Anyway, Microsoft, keep up the good work and the arrays of confusion!  ARGH!  I look forward to continued efforts and readings on ASP.NET, MVC, and all the good stuff that you produce.  As for the silly out of sync marketing efforts I’ll continue to enjoy those as canon fodder for every geek group in the world.

Thanks Microsoft.  Big Smile [:D]

Index, Fragmentation, Statistics – A List

This entry is going to be a collected list of practices for index, fragmentation, statistics, and general VLDB table and view maintenance.

First off, a few obvious index practices.

  1. If a non-clustered index is not used, toss it.  When a table has a record inserted, the duplicate of that is inserted into the non-clustered index.  Each time that record is deleted the same must occur in the index and the same goes for updates.  Most horrendously, if a clustered index is created then the non-clustered index has to be rebuilt.  Major IO hog, if it isn’t used, no point in having it.  For information on how to check if a non-clustered index is being used, check out this blog entry on the SQL Server Storage Engine.  At some point I might shrink that into a short entry and post it as a tip o’ the day.  I’ve had to reference it a number of times and would rather just the quick bullet points.- When checking VLDBs, and especially VVVLDBs, for consistency in sizes exceeding hundreds of gigabytes or even multiple terabytes there are a few options that one can follow:
    1. Use the WITH PHYSICAL_ONLY option for CHECKDB.  It will run similar to DBCC CHECKALLOC and read and audit every allocated page in the database.  It will skip logical errors, inter-page checks, and some other things like the DBCC CHECKCATALOG.- Break up the checks by partition.  The DBCC CHECKFILEGROUP run on a read-only filegroup every week or two works great.  On a read-write filegroup DBCC CHECKFILEGROUP should be run every day or every few days.- Break up the checks into smaller groups by doing DBCC CHECKCATALOG or DBCCCHECKTABLE.  Break them into different groups and then segment the job execution of these checks to different maintenance windows.- Logical fragmentation only affects read-ahead performance, only a rebuild/defrag of indexes will help.- Low page density affects UI throughput and memory usage, which could be a sign of page-splits.  Check this frequently.- If defrag is done instead of rebuild, make sure to update stats.- Be cautious when doing large index maintenance jobs if log shipping or DBM is used, index rebuilds are always full-logged when DBM is present.  (DBM = Database Mirroring)

A few items mentioned by Peter Sampson – Technical Director at Centerstance;  Blogs: Moving to Mac and SQL Janitor, over a recent pint o’ Beer at Bailey’s Taproom.

  1. Regardless of OLTP or OLAP almost any table in SQL Server should…
    1. always have a clustered index on the table.- have a clustered index design that either very rarely or never has the clustered index updated.  SQL Server does a DELETE and an UPDATE if you update a column with a clustered index!- use non-clustered indexes in conjunction with the clustered index since they are interleaved.- DBCC DBREINDEX rebuilds the table and indexes depending on the parameters passed.  This is best done on a regular schedule, often by setting a job to execute the task.- FILLFACTOR is a critical parameter for DBREINDEX, make sure it isn’t just set to 100 or some other erroneous value that will cause problems.  FILLFACTOR sets the free space to leave during the rebuild of the Index B-tree structures and is a reverse representation.  100 == no free space, 90 = 10% free, and so on.  For read-only tables do a FILLFACTOR = 95 or perhaps 90.  For other stuff FILLFACTOR = 85 is a good starting point since it gives you some head room for INSERTs and UPDATEs after you run DBREINDEX.- DBCC UPDATE STATS collects new statistics but does not remove fragmentation.  It takes less time than DBREINDEX and can be useful if you’ve done a lot of transactions and don’t quite have time for DBREINDEX.- Peter:  “Auto-update stats I no longer trust to provide accurate statistics over an extended period of time.  Got burned up with this, leave auto-update stats enabled but supplement with regular DBREINDEX or UPDATE STATS as needed.“

Someone asked me during a discussion, and it is often forgotten what VLDB stands for.  To alleviate confusion, VLDB stands for very large database.  When it has extra Vs it means it is REALLY frikkin huge!

A last few links for reference material:

Best practice when optimizing indexes on SQL Server 2005

Best SQL Server indexing strategies

TOP 10 SQL Server Indexing Tips to Improve Performance

Hope that’s useful.

Technorati Tags: FILLFACTOR,DBCC,DBREINDEX,INSERT,UPDATE,DELETE,Index,Defrag,Index Rebuild,DBM,Database Mirroring,CHECKALLOC,CHECKCATALOG,CHECKFILEGROUP,Index Best Practices,Index Strategy,SQL Server,SQL Server 2005,VLDB,VVLDB

del.icio.us Tags: FILLFACTOR,DBCC,DBREINDEX,INSERT,UPDATE,DELETE,Index,Defrag,Index Rebuild,DBM,Database Mirroring,CHECKALLOC,CHECKCATALOG,CHECKFILEGROUP,Index Best Practices,Index Strategy,SQL Server,SQL Server 2005,VLDB,VVLDB

BI Applications?

It seems that everywhere one looks in the ETL/OLAP/BI Industry Space there are no real canned applications.  When I say canned application I’m talking about an application that from the ground up is all inclusive, managing the database, the ETL, the warehouse and any possible staging, and finally the Cube itself and reporting that sits on top of it.  Let me break that down real quick:

  • Database- Staging- Warehouse- Cube- Reporting

…and of course, one can’t forget…

  • ETL – Extraction, Transform, and Loading

I have as of yet seen something, able to be delivered with a single MSI, or even something that is easily installed in one or two steps.  Of course, I saw all this, and would love for someone to comment on a product that is easily installed that utilizes all of these things in one install package.  Especially if it uses Microsoft’s Technology Stack.

In addition to not having seen this yet, I keep reading about the prospective gain in market share of applications that are supposedly just like this.  To that I ask, “How does one scale a canned application”.  The amount of logic and decision branching the application would have to make during the course of the application scaling is practically impossible to do well.  Sure, one can put in some basic checks, but beyond that I would suspect a kind of diminishing return.  The simple fact is, that business intelligence still needs a significant presence of skilled ETL, Report, and Database Developers.

I’m not really complaining, just sort of stating the obvious.  As I dig more into this industry, I can really see how difficult it is to find people that are skilled in the full technology stack or even theory of the entire technology ideology behind business intelligence.

So with the end of the year coming up, I think what I’ll do is just set a new years resolution to get something built with a business intelligence engine as the basis.  Personally or work related doesn’t matter, I just need to create something in this space that brings together these points.

Vista Slaughtered

Vista really is a lacking Microsoft Product.  I’m not sure what the hell happened in comparison to Windows 2000 and XP.  Both of those launches went rough, as usual, but not noticeably so.  Vista however is just a dog within the industry these days.  I’m sure it is even ceding a large number of customers to Apple and I couldn’t blame them.

Hopefully SP1 will get this dog running right.

Some rather entertaining quotes top list garnerings published on The Insider: Tech pundits take dim view of Vista.

Michael Gartenburg’s quote I found entertaining.

“I’m starting to think of Vista a little like sushi. I don’t love sushi. If it’s around, if it’s prepared well, if it’s fresh, I’ll eat it. I rarely go out of my way to eat it, but if there’s an alternative, I’ll usually eat that. If you ask me if I like sushi, I’ll usually say it’s not bad.”

Technorati Tags: Vista,Windows Vista

del.icio.us Tags: Vista,Windows Vista

The Oncoming Microsoft ASP.NET MVC – Part 1

The ASP.NET MVC is officially released in the .NET 3.5 Extensions CTP.  This is part one of a multi-part series I’m going to do on getting a basic application up and running with good practice and solid test driven development process.  Enjoy.

First we’ll go over a quick review.  What is MVC?  MVC stands for the model view controller pattern.

  • The Model in an MVC pattern are the components of the application that are responsible for maintaining state.- The View in this pattern are the parts responsible for displaying the application’s user interface.- The Controller in this pattern are responsible for handling end user interaction.  Manipulating the model and choosing a view to render for the UI is also part of the responsibilities of the controllers.

In each of the above parts of the pattern, you could have multiple models, views, or controllers depending on the need and setup of the application.

Some of the absolutely kick ass benefits of the MVC pattern include;

  • To me, one of the largest benefits of the MVC pattern is the separation of concerns.  I’ve ranted and raved about the need for a separation of concerns in some of my other blog entries.  This is fundamental to maintainable, easy to understand, and easy to read code.  Without a clear separation of concerns it is easy to toss out all three; maintainability, understanding, and readability.- The new MVC does not use the post back model for server interactions.  What is used is interactions with a controller class which enables the above mentioned separation of concerns and testing.- Unit testing and test driven development are easily implemented with the MVC pattern.  Especially so with this particular implementation of the pattern.  All the contracts within the framework are interface declared and easily mock-able.  Unit testing the application can be done without needing to run the controllers within the ASP.NET process making them turbo fast.- Dependency injection and inversion of control container models can be integrated and used.  This could include Windsor, Spring.NET, NHibernate, probably even StructureMap or others.

So with all this awesomeness released finally I’ve set out to build my first ASP.NET MVC How To.  Since I’ve been needing to rebuild my Transportation Tracker Application I decided that would be my first ASP.NET MVC Candidate for building.  This quick tutorial is going to assume that you have Visual Studio 2008 installed and the .NET 3.5 Extensions CTP mentioned above installed already.  In addition to these things I’m also assuming that we all have Rhino Mocks installed.  If not, get it.

Spring.NET Framework

Go hit up the Spring.net Framework.  Download that and install it, making sure to have Visual Studio 2008 closed.  Once you’ve got that installed we’re ready to go.  Click on the images to see the full size versions.

Creating the Project

First step after all that installation mess is to get a project and the proper test project created!  Just open up Visual Studio 2008 and click on file, then new so the new project dialog appears.  Name the project whatever you’d like to at this point.  Same thing with these images, click em’ if you want to see the full size ones.

Once you’ve created the project with the test project, then create some initial model, view, and controller files that will be the basis of the application.  The first few I created are the interfaces for the model classes;

Following the interfaces I created the model class files.  At this point I have a Solution Explorer Tree that looks like the image below.

Next create the necessary controllers for the application.  I’m going to break this apart into each separate section of the application that I will have data manipulation.  Each of these sections is basically a separate page with specific flow and data characteristics on each page.

Creating new files and code 03

Note when you are creating the MVC Controller Classes that there is a template item specifically for creating these that cut out some of the work for you, albeit a small amount.  What you get is a few attributes marking the class and initial void that is put in place for you.

public class TripController : Controller {

    [ControllerAction]     public void Index()     {         //Add code action logic here     } }

After adding each of the controllers I’ve got; LocationController, TransitModeController, TripController, TripPurposeController, TripTypeController, and the HomeController that was already generated by the template for me.  Since I’m trying to maintain my test driven development methodologies these days, I’m going to add a quick few pieces of code so that I can get something building and add some tests.

I added the following tests for the HomeController to assure that the proper items where setup and displaying properly.  Of course since I’m doing TDD, and I have none of this implemented yet, none of the tests will pass.  The following is my t est code;

using System; using System.Text; using System.Collections.Generic; using System.Linq; using Microsoft.VisualStudio.TestTools.UnitTesting; using MvcApplication.Controllers;

namespace MvcApplicationTest.Controllers {     [TestClass]     public class TransportationControllerTests     {         [TestMethod]         public void HomeControllerSelectsCorrectViewTripType()         {             HomeControllerDouble controller = new HomeControllerDouble();             controller.TripTypes();             Assert.AreEqual(“TripType”, controller.SelectedView);         }

        [TestMethod]         public void HomeControllerSelectsCorrectViewTrip()         {             HomeControllerDouble controller = new HomeControllerDouble();             controller.Trips();             Assert.AreEqual(“Trip”, controller.SelectedView);         }

        [TestMethod]         public void HomeControllerSelectsCorrectViewTripPurpose()         {             HomeControllerDouble controller = new HomeControllerDouble();             controller.TripPurpose();             Assert.AreEqual(“TripPurpose”, controller.SelectedView);         }

        [TestMethod]         public void HomeControllerSelectsCorrectViewTransitModes()         {             HomeControllerDouble controller = new HomeControllerDouble();             controller.TransitModes();             Assert.AreEqual(“TransitMode”, controller.SelectedView);         }

        [TestMethod]         public void HomeControllerSelectsCorrectViewHome()         {             HomeControllerDouble controller = new HomeControllerDouble();             controller.Index();             Assert.AreEqual(“Index”, controller.SelectedView);         }

        [TestMethod]         public void HomeControllerSelectsCorrectViewAbout()         {             HomeControllerDouble controller = new HomeControllerDouble();             controller.About();             Assert.AreEqual(“About”, controller.SelectedView);         }

        [TestMethod]         public void HomeControllerSelectsCorrectLocation()         {             HomeControllerDouble controller = new HomeControllerDouble();             controller.Location();             Assert.AreEqual(“Location”, controller.SelectedView);         }

        private class HomeControllerDouble : HomeController         {             public HomeControllerDouble()                 : base()             { }

            public string SelectedView { get; private set; }             public object RenderedViewData { get; private set; }

            protected override void RenderView(string viewName               , string masterName               , object viewData)             {                 this.SelectedView = viewName;                 this.RenderedViewData = viewData;             }         }     } }

Now that there are some tests to verify each of the different areas I’m going to move on to wiring these sections up.  After editing the HomeController.cs the code looks like the following;

using System; using System.Web; using System.Web.Mvc;

namespace MvcApplication.Controllers {     public class HomeController : Controller     {         [ControllerAction]         public void Index()         {             RenderView(“Index”);         }

        [ControllerAction]         public void About()         {             RenderView(“About”);         }

        [ControllerAction]         public void TransitModes()         {             RenderView(“TransitMode”);         }

        [ControllerAction]         public void Trips()         {             RenderView(“Trip”);         }

        [ControllerAction]         public void TripPurpose()         {             RenderView(“TripPurpose”);         }

        [ControllerAction]         public void TripTypes()         {             RenderView(“TripType”);         }

        [ControllerAction]         public void Location()         {             RenderView(“Location”);         }     } }

Now we have a compiling, unit test executing, running application.  Albeit there isn’t a whole bunch of use to the thing yet.  Next thing we’ll do is setup some testing that will utilize the Rhino Mocks framework.  Keep in mind while working through this that we’re doing test driven development.  We get a test written, it tosses a red light, we then fix it so that we get a green light.  On and on goes the process.  The first test I’m going to write is for the TransitMode section of the application.  Create a file in the unit tests project folder called TransitModeControllerTest.cs.  After you’ve completed the code for this file, change the create the TransitModeController.cs file in the Controllers Directory.

using System; using System.Collections.Generic; using System.Web; using System.Web.Mvc; using MvcApplication.Models;

namespace MvcApplication.Controllers {     public class TransitModeController : Controller     {         ITransitMode transitModeRepository;

        public TransitModeController(ITransitMode repository)         {             this.transitModeRepository = repository;         }

        [ControllerAction]         public void TransitModesView()         {             // TODO: Add functional code after test is created.         }     } }

Now that you have skeleton of the controller created we have to get a red light test.  Make before going on to the test that the interface is created.  The code for it should look like the listing below.

using System.Collections.Generic;

namespace MvcApplication.Models {     public interface ITransitMode     {         void Create(TransitMode transitMode);

        IList ListTransitModes();     } }

Open that TransitModeControllerTest.cs file back up and add the code listed below.  Once completed run the test, and accordingly it should fail.

using System; using System.Text; using System.Collections.Generic; using System.Linq; using Microsoft.VisualStudio.TestTools.UnitTesting; using MvcApplication.Controllers; using MvcApplication.Models; using Rhino.Mocks;

namespace MvcApplicationTest.Controllers {     [TestClass]     public class TransitModeControllerTest     {         [TestMethod]         public void TransitModeControllerPassesCorrectViewData()         {             MockRepository mocks = new MockRepository();             ITransitMode transitModeRepository = mocks.DynamicMock();             SetupResult               .For(transitModeRepository.ListTransitModes())               .IgnoreArguments()               .Return(new List(new TransitMode[] { new TransitMode(), new TransitMode(), new TransitMode() }));             mocks.ReplayAll();

            TransitModeControllerDouble controller = new TransitModeControllerDouble(transitModeRepository);             controller.TransitModesView();             IList transitModes = (IList)controller.RenderedViewData;             Assert.AreEqual(3, transitModes.Count, “Expected three modes.”);         }

        private class TransitModeControllerDouble : TransitModeController         {             public TransitModeControllerDouble(ITransitMode transitMode)                 : base(transitMode)             { }

            public string SelectedView { get; private set; }             public object RenderedViewData { get; private set; }

            protected override void RenderView(string viewName               , string masterName               , object viewData)             {                 SelectedView = viewName;                 RenderedViewData = viewData;             }         }     } }

That gives us our red light test.  Now for the green light.  Add the code in bold below to the TransitModeController.cs file.  Running the test again will give you a green light test.  For now that gives us a good start.

using System.Collections.Generic; using System.Web.Mvc; using MvcApplication.Models;

namespace MvcApplication.Controllers {     public class TransitModeController : Controller     {         ITransitMode transitModeRepository;

        public TransitModeController(ITransitMode repository)         {             this.transitModeRepository = repository;         }

        [ControllerAction]         public void TransitModesView()         {             IList transitModes = transitModeRepository.ListTransitModes();             RenderView(“TransitModes”, transitModes);         }     } }

That’s it for now.  Merry Christmas, enjoy the code, and I’ll take it from this point forward for my next entry!

Technorati Tags: ASP.NET,ASP,C#,MVC,Model View Controller

del.icio.us Tags: ASP.NET,ASP,C#,MVC,Model View Controller

kick it on DotNetKicks.com