Monitor SQL Server CDC with Customer Data Collector

I’ve posted several times over the years here @MSSQLDUDE on the benefits of SQL Server Management Data Warehouse (MDW) for baselining your SQL Server performance and sending those measurements to a central server for monitoring. MDW makes use of the SQL Server Data Collector feature and is a feature you get out of the box. Here are some of my writings on MDW here @MSSQLDUDE: http://en.search.wordpress.com/?q=mdw&site=mssqldude.wordpress.com.

There is a debate among SQL Server DBAs about the worthiness of MDW and Data Collectors because they can appear to be a “heavy burden” on your server and provide overhead that you may not wish to occur on your production server. That being said, the way things work in our world and jobs in databases is a give and take with performance, monitoring, scalability and providing functionality to our business users & customers to allow them to do their jobs. Any monitoring and baselining product incur a small performance cost to your servers. There are a number of options out there in the market and many DBAs write their own scripts to store DMV data into tables and files to provide similar functionality.

What I wanted to point you all to is something that I think proves the value of Data Collectors (and MDW) through extending the functionality with custom data collectors.

I’ve been working on a CDC project recently in a large BI/DW scenario and one thing that you notice when working on CDC instead of longer-run SQL Server features like Replication, is that there is not much support in SSMS or other built-in SQL Server tools for CDC.

But what is really nice is this custom data collector for CDC that is included on MSDN from this terrific article onMonitoring the Change Data Capture Process. Scroll down to the Data Collector portion and the code to paste into SSMS is right there for you to create and start your data collector. This is a great way to monitor your CDC log processes.

BTW, you have to make an update to the XML in code from MSDN … Look for the SELECT @parameters code and replace the entire SELECT … XML statement with this:

SELECT @paramters = CONVERT(xml, 
    N'<ns:TSQLQueryCollector xmlns:ns="DataCollectorType">
        <Query>
          <Value>SELECT * FROM UserDB.sys.dm_cdc_log_scan_sessions</Value>
          <OutputTable>cdc_log_scan_data</OutputTable>
        </Query>
      </ns:TSQLQueryCollector>');

Enjoy! Best, Mark

System Center for the SQL Server DBA: Wrap-Up

I’m going to call the series on System Center for the SQL Server DBA complete with this one last post today. First, here is the full series to date … my apologies for the lack of post naming consistency!!

System Center for the SQL Server DBA Intro

The Continuing Story of System Center for the SQL Server DBA

Monitor SQL Server from System Center Operations Manager

SQL Server management pack for SCOM

System Center for the SQL Server DBA Part 2: DPM

System Center Data Warehouse for SQL Server DBAs

What I wanted to point you all to is an update to my methodology of using SQL Server 2012 business intelligence leveraging in-memory analytical models to provide very important insights into your SQL Server environment with System Center Operations Manager. The link is here to my blog on this on SQL Server Pro Magazine.

By leveraging the SCOM data warehouse, you can create compelling reports quickly with a semantic model that includes SQL Server performance counters, server health and history for long-term history and trends. Use this to create your server performance baselines, alerts, trends, etc.

The take away for SQL Server DBAs is that it is very, very important to gain that level of insight into your databases.

Thanks!! Mark

Continuing Series: System Center for the SQL Server DBA (SCOM Data Warehouse)

Not really a new part to my multi-part series on System Center for the SQL Server DBA … I’m going to post more on the other products in the System Center Suite shortly, but I want to go back to the bread & butter tool for the SQL Server DBA in System Center: Operations Manager. OpsMgr (or SCOM) is the tool where SQL Server DBAs can view alerts, events and overall health monitoring of your entire environment including application & web monitoring, server monitoring as well as databases. With the SQL Server Management Pack add-on to SCOM, you get a large set of SQL Server counters added to your monitoring tool that you can put into dashboards and reports.

Where I want to focus today is the data warehouse feature that ships with SCOM. When events come into the SCOM server from agents, they are logged in both the operational SQL Server database schemas as well as the data warehouse schema. The DW schema keeps history that gives you a wealth of information to report against from recent performance and health to historic reports.

In the box with System Center and the SQL Server MP, you get several reports already built into the catalog for you. To extend those, you use a SQL Server reporting tool like Report Builder against the data warehouse schema. For the current project that I’m working on to extend SCOM for SQL DBAs, I used the new SQL Server 2012 reporting tools with PowerPivot and Power View (see below).

In order for you to build reports from the SCOM DW, you are going to have to poke around at the documentation for the data model, reverse engineer existing RDL reports (which you will find on the SCOM server) and read the reporting guide and blogs at these links:

http://technet.microsoft.com/en-us/library/gg508713.aspx

http://blogs.technet.com/b/momteam/archive/2008/02/26/operations-manager-report-authoring-guide.aspx

Once I had a good enough understanding of the data model, I was able to build a tabular model cube in PowerPivot by connecting directly to the SQL Server DW tables and then stored the cube in SSAS. You don’t need to use the in-memory analytics tools that I show above to report against a cube, but having some sort of a semantic model in tabular, PowerPivot, SSAS UDM, report models, etc. will make the process of building and maintaining SCOM reports much easier. Otherwise, you are relegated to creating on-the-fly joins or using the data model views, which limits you to what is presented. Certainly, in my mind, building a cube will provide much more meaningful business intelligence based on events & alerts that you collect into System Center from your entire SQL Server environment.

It’s a SQL Server Weekend!

Tomorrow is SQL Saturday Philly in our Microsoft Technology Center in Malvern, PA!!

Here is the SlideShare link to my presentation on Microsoft Cloud BI

This is my blog post from today on SQL Server Pro Magazine BI Blog for Microsoft Cloud BI

Here is the SlideShare link to my presentation on Microsoft DW for SQL Server DBAs

Have a great SQL Server weekend all!!  Br, Mark

SQL Server 2012 Database Backups in Azure

SQL Server 2012 inlcudes new & updated features to SQL Server Management Studio (SSMS) that provide out-of-the-box integration into the Windows Azure platform-as-a-service Cloud service from Microsoft. It goes beyond just SQL Azure integration as you can now also connect into Windows Azure storage accounts where you can store files and blobs, including DACPAC & BACPAC files, essentially providing DBAs with out-of-the-box backup-to-cloud capabilities.

From a DBA’s perspective, this can be very beneficial because this would allow you to take your SQL Server backups and post them into the Azure cloud where files are automatically protected and replicated for high availability. This will also eliminate the need for you to maintain infrastructure for backups locally on your site. You would utilize Azure Storage for maintenance, retrieval and disaster recovery of your database backups. Here is a link with more details on Azure Storage.

Here are the steps to backup from SSMS 2012 to Windows Azure:

  1. First thing to note is that you will need to sign-up for a Windows Azure account and configure a storage container. You can click here for a free trial.
  2. Now, on SSMS, choose a database to backup. But instead of the normal Backup task, select “Export Data-Tier Application”. This is going to walk you through the process of exporting the schema with the data as a “BACPAC” file output.

3. On the next screen in the wizard, you will select the Azure storage account and container where SQL Server will store the export. Note that it will first backup the database (schema & data) to a local file and then upload it to the cloud for you.

4. Once the process is complete, you will see your exported backup as a BACPAC in your storage container. To restore a BACPAC, you right-click on the file from your container and select “Import Data-Tier Application”.

BTW, this process is identical to the way that you can export & import databases in SQL Azure. You can also easily now move your databases (schema and/or data) to and from SQL Server and SQL Azure with these BACPACs. This is not a full-service TLOG and data file backup like SQL Server native backups. This is more of a database export/import mechanism. But that is pretty much the most interactive that a DBA will get with SQL Azure anyway because you do not perform any TLOG maintenance on SQL Azure database. Microsoft takes care of that maintenance for you.

SQL Server 2012 AlwaysOn Availability Groups In Action

I think of SQL Server 2012 AlwaysOn Availability Groups as both high availability and scale-out database. This is because you can identify multiple secondary servers in your Availability Group that can be marked for read-only. When you connect to the Availability Group Listener, SQL Server will redirect the request of any connection marked as READONLY to the secondary read-only database server automatically for you. Additionally, you can run common administrative functions that take up cycles and locks on your production database server such as backups and DBCC checks to a readable secondary, again making for a higher-performance scaled-out SQL Server infrastructure.

I just posted a short 10-minute video on our Microsoft East Region DW BI database group here: http://www.youtube.com/watch?v=TF9_Kn2Iqy8 which shows my AlwaysOn configuration from the perspective of an end-user writing and viewing reports. This highlights the high-availability capabilities that AlwaysOn AG’s provide, above and beyond the now-deprecated Database Mirroring feature.

One last thing that I want to re-emphasize which I call-out in the video a few times, is how easy it was set-up AlwaysOn and then Availability Groups. I didn’t have to configure shared storage for the cluster configuration, which makes it very easy to demonstrate on a VM, because I don’t need a SAN. There is no shared storage involved with AlwaysOn and the entire demo in the video is performed on Hyper-V.

Philly Code Camp 2012.1 SQL Server Presentation

Hi all! Many thanks for joining me at the SQL Server 2012 what’s new session at Philly Code Camp this past Saturday in Abington, PA at the beautiful Penn State campus! Here is the Slideshare link to the slides that I used to walk you through the new features: http://www.slideshare.net/kromerm/whats-new-in-sql-server-2012-for-philly-code-camp-20121. Enjoy! Br, Mark