Tuesday, December 8, 2015

T-SQL Tuesday #73: IOPS on Azure VM's Naughty to Nicer

Hello Dear Reader!  Last week I posted the invitation to T-SQL Tuesday #73.  The premise? 

As you work with SQL Server look around you.  Is your environment Naughty or Nice?  If it is Naughty what’s wrong with it?  What would you do to fix it?  Do you have a scrooge that is giving you the Christmas chills?  Perhaps you have servers of past, present, and future haunting you.  Maybe you are looking at SQL Server 2016 like some bright shining star in the east.

I don’t have an environment of my own.  Not one that I get to interact with every day.  Recently I’ve had several experiences where some friends with various companies were having issues and the root cause was all hardware.  Memory, SAN, or CPU one had become a bottleneck.

Several had to do with I/O.  One had issues with their SAN, another had issues with their code (causing unnecessary I/O), and another on their Azure VM.

“So Balls,” you say, “What does this have to do with Naughty & Nice?”

As always Dear Reader, thanks for keeping me on track.  For this week I’m going to tackle how IOp’s can affect your SQL Server.   I’ll be combining this with the pseudo code to mimic the issue my client had.  Then a little on Azure VM’s, and how Premium Storage made a big difference. 

THE SET UP

I’ve got a really bad query.  This is from years past.  Lots of unnecessary row by row logic.  Duplicate logic.  It’s bad. 

Sufficient to say, I’m rewriting it with pseudo code because I don’t want to risk insulting the guilty.  I will post the stats though.  We were on an Azure D13 Series VM, and we moved to a DS 13 Series VM.  This fixes nothing by itself.  Click here to see the machine specs.  See below for a quick table.  We will be using the DS3 VM.  We will move from standard storage to premium storage on the same machine to highlight our fix.


SIZE – AZURE CLASSIC PORTAL\CMDLETS & APIS
CPU CORES
MEM
NICS (MAX)
MAX. DISK SIZES – VIRTUAL MACHINE
MAX. DATA DISKS (1023 GB EACH)
CACHE SIZE (GB)
MAX. DISK IOPS & BANDWIDTH
Standard_DS3\same
4
14
4
OS = 1023 GB
8
172
12,800
OUR DEMO MACHINE
Local SSD disk = 28 GB
128 MB per second
Standard_DS13\same
8
56
8
OS = 1023 GB
16
288
25,600
Local SSD disk = 112 GB
256 MB per second

A DS Series VM gives us the ability to use Premium Storage.  We will use that turn bad code into, faster bad code.  Fixing the code comes later.  The first thing I will do is run CrystalDiskmark against my disk. 

Our G drive is standard storage. 


M drive is our premium storage.



The M drive is a 1 TB Premium Storage Drive.  I have done nothing fancy here.  Just attached and formatted.  As you can see we received a little bit better than 128 MB per second sequential in my baseline.

Our really bad query?  It runs in 19:48 seconds on our regular storage.  When I found it in the wild it actually ran for 4 hours.  I could only replicate so much.  Sufficed to say, for this example it will do.



Without tuning the query, but moving the database to premium storage, dropping all data from memory, and rerunning the query we reduced the time to 5:49 seconds.  A roughly 72% improvement.




Premium storage can go faster.  It is a matter of configuring the VM & the Storage to hit the IOP's limits you need.  For example, this is a screen shot from the DS 13 VM I was talking about earlier.  



With a DS 14 we would have been able to go above 512 MB per second.  Premium storage on DS Series VM's is some serious good stuff.

THE WHOLE TRUTH


The query is bad.  Really bad.  I can fix the whole thing by converting it from row by row logic to set based logic.  I can get it down to 2 seconds using the merge statement.

We did.  It was a big fix.  Originally we were hoping to get the process from 4 hours down to 2 hours.  After I was done we had it down to 5 minutes.

I say this often when teaching.  I can throw super-fast hardware at bad code and make it run faster.  But there is still room for improvement.  Fixing the query is for another day.   Today we keep our IO on the ball.   Why is it so fast?


CPU
Memory
SSD
Spinning Disk
Nanosecond
Nanosecond
Microsecond
Millisecond

You can only get as fast as your slowest component.  Premium storage uses SSD's.  We are hastening our retrieval of data from our disks by a factor of x10.  In this case it made our query 72% faster.  There are other things to fix.  The IO issue is one of them.


“So Balls,” you say, “Keep our IO on the ball.  Really?!”


Sorry Dear Reader, I couldn’t resist.  As always, Thank You for stopping by.

Thanks,

Brad





Tuesday, December 1, 2015

T-SQL Tuesday #73 Invitation – Naughty or Nice?




Hello Dear Reader!  This is the first Tuesday of the month and you know what that means.  It’s time to announce the T-SQL Tuesday Topic of the month!  This is your opportunity to participate in the largest SQL Blog party on the intrawebs. 

T-SQL Tuesday is an event started by Adam Machanic(@AdamMachanic| Blog) back in 2009.  The basic idea one blogger hosts the party and others participate.  We announce the topic the first Tuesday of the month, December 1st 2015 for today, and everyone will post their blogs on the second Tuesday of the month, December 8th 2015 for the actual posts.  This month the host is none other than……….. drum roll………….. ME!

Since it is the holiday season, I wanted to keep this holiday themed.  I couldn’t help but think of the approaching Christmas holiday.  Every year I get my children to make a list of presents that they would like to get from Santa Clause.  If we are out and about and they start to misbehave I remind them about their list and that only Nice children get presents.

“So Balls,” you say, “What in the ever loving wide wide world of sports does this have to do with SQL Server?!?”

As you work with SQL Server look around you.  Is your environment Naughty or Nice?  If it is Naughty what’s wrong with it?  What would you do to fix it?  Do you have a scrooge that is giving you the Christmas chills?  Perhaps you have servers of past, present, and future haunting you.  Maybe you are looking at SQL Server 2016 like some bright shining star in the east.

First and foremost the rules. 

Rule 1: Don’t get yourself fired.  If your boss is a scrooge don’t name names, don’t hint to hard.  It’s okay to protect yourself.  You want to write about events we can look back on and reflect over, not events HR would *love* to know about.  Perhaps a reminder of SQL Server's past would be better than that of SQL present.

Rule 2: Some Time next Tuesday using GMT, here’s a link to a GMT time convertor, publish your blog post.  For example in the US that would cover 8 pm Monday to 8 pm Tuesday.

Rule 3: Make sure that you include the Image at the top of the page helping to identify your post as a T-SQL Tuesday blog.  Then come back here and post a link in the comments so I can find them.  Before the end of the week I'll do a round up of all the blogs. 

Extra Credit!

Tweet your blog with the hash tag #tsql2sday, use SQL Server 2016, & go read someone else’s blog on the subject!

As Always Dear Reader, Thanks for stopping by and I’ll see you next Tuesday!

Thanks,


Brad

Thursday, June 25, 2015

Top 3 Questions Before Submitting to a SQL Saturday

Hello Dear Reader!  We are coming up upon the 9th SQL Saturday for Orlando.  Orlando is where SQL Saturday began, and the home to some pretty amazing speakers.

We, the SQL Saturday Orlando Team, are hard at work getting ready for the event.  Our Call for Speakers ends on July 10th, just 15 days from now.  

So hurry up and get those submissions in!

"So Balls", you say, "How do I deal with the rejection if I don't get selected?  How do I submit a Presentation? Why would I want to submit to be a Speaker?"

Great questions Dear Reader.  As always you are on top of your game!  Let's dive right in!


1. HOW DO I DEAL WITH THE REJECTION IF I DON'T GET SELECTED?


Simple answer, in Orlando you don't.  SQL Saturday was invented and founded by SQL MVP & SQL Saturday Orlando Committee member Andy Warren(@SQLAndy | Blog).  The main thought behind it was to grow the next generation of SQL Server Speakers.  In 2007 Andy saw the need to grow the pool of professionals that spoke at SQL Server events.  He knew the value of expanding the community.

My first SQL Saturday was #49 in Orlando as an attendee.  Since then I've been on the Speaker Selection Committee since 2011.  When I started I asked my friend, and one half of my future law firm of Biguns & Balls, Jack Corbett (@UncleBiguns | Blog) what do I need to know about this position?  How do I pick who makes it and who doesn't?  His answer (and I'm paraphrasing, it was 4 years ago).

"You make room for everyone.  This is about growing the community.  You, Kendal, Andy, none of you speak unless there is room.  Everybody who submits get's a spot.  We encourage new Speakers, friends we haven't seen present in a while, MVP's, people from Microsoft, we cast a net and we give the community a place to gather and share."

And we do.

I'm proud to say that my co-partner in crime on the Speaker Committee SQL MVP Rodney Landrum (@SQLBeat | Blog)  and I have never rejected anyone.  We've helped tweak abstracts, had experience speakers sit with new speakers, and provide direct feedback after their presentation.  We've done what we needed to in order to help them give them a spot.  In all that time we've never rejected anyone and we are not about to start now.

2. HOW DO I SUBMIT A PRESENTATION

Don't worry, it's easy.  I'll give you some quick advice, but I walk through this in a lot of detail in a blog that I had written for SQL Saturday 85, click here to review.

First write an abstract, NOT a presentation.  Why you may ask?  Because it takes  a lot of time to write your presentation.  It takes a couple minutes to an hour to write an Abstract.

What's an Abstract?  It is the description of your presentation.  Here are a couple sample abstracts below that I had previously written about.  By the way, I never used these, I never developed a presentation on them so you could have at them Dear Reader.

The Top 10 Things Your Developers Should Know

It’s not easy being a DBA, heck it’s not easy being a Developer.  Too often DBA’s and Developers are put at odds, are we two different species, are they from Venus and we’re from Mars, Why wasn't the Green Lantern the movie more successful?  There are a lot of questions and you want Answers!  Come and find out the top 10 things you Developers should know about SQL Server and a friendly way you can present them.

So there we go, we start off with the statement, “It’s not easy being a DBA, heck it’s not easy being a developer.”  We don’t want to alienate anybody you get just as many developers, managers, and other folks at a SQL Saturday as you do DBA’s.  And then the Question Why wasn't the Green Lantern Movie more successful Why do we have such a hard time communicating?  Then present the solution, and suggest a Take-Away-Item that people will get by attending your presentation. There are a lot of questions and you want Answers! Come and find out the top 10 things you Developers should know about SQL Server and a friendly way you can present them.”  

Notice I made a couple light hearted jokes in the abstract, you don’t have to do that. That's more my thing.  So let’s do one more that is a little more serious.

Surrogate Keys vs. Natural Keys – The Great Debate

An essential part of Database Design is looking at the keys that you’ll have in your tables, You want to make sure that you’ve got a Primary key right?  But do you want that key to be Natural or Surrogate? What is the difference between Natural and Surrogate Keys, What is the Sort order on a Page, and How can they affect performance?  What do I want for an OLTP system vs. an OLAP system?  Come find out as we take a step by step process that will build and compare each!

No jokes this time Dear Reader, kind of like an anti-mullet, no Party in the back and all business up front.   

Once your abstract get's selected THEN you write the presentation.  In order to do that you have to submit.

You do NOT need to be on Twitter, LinkedIn, or have a blog.  Those are optional, but they do help with your personal branding.  Long term you should consider those things.  But you have time to do that!

Submit to SQL Saturday 442 Orlando by Clicking Here.


3. WHY WOULD I WANT TO BE A SPEAKER?


If you liked his presentation,
go buy Simon's book
It's important to start with Why.  I cannot tell you your why.  I can't.  It is for you to decide.  I can tell you my Why.  That's mine not your's, and I would encourage you to listen to Simon Sinek's great presentation on that.  It's only 18 minutes, and it has the potential to change your perspective and by proxy your life.  Go ahead and watch.  I'll wait.

Okay did you go listen?  Seriously!  After I said the whole change your life and perspective thing!  Really!?!  Go watch, click here.

Okay, if you still didn't do it I'm not going to turn into The Monster at The End Of This Book.

My Why.  I love empowering myself and people with knowledge.  I do this by trying to make complex concepts simple and relatable.  I just happen to do this through presenting, writing, and consulting.  Would you like to come to my presentation?  If you watched the video, you see exactly what I did there.

If you don't know your Why, finding it is half the fun.  Being a speaker can help you along that path as well.

Networking and opportunities are certainly a part of the package, but they are a result of What you do not Why you do it.  Look at your Why and if it leads you here we'd love to have you!

You don't have to speak.  You can be an attendee, a volunteer, or just come for the coffee, donuts, and conversations.  After all this is all about community!

WRAP IT UP

What are you doing still reading this blog!  Go submit, and if Orlando is not close to you head over to SQLSaturday.com and find a SQL Saturday near you, I'm sure they'd love to have you speak as well!


As always, Thanks for stopping by.

Thanks,

Brad

Thursday, January 22, 2015

Looking Out for Others Helps You Lead


I'm sitting on a plane. I'm in an exit row, still a big guy in a little seat. I go to sit down and notice the seat to my left has water on the seat. Not so deep that it is a puddle. 5 to 10 big drops. Enough that you wouldn't want to sit in it. The person who this seat will belong to hasn't made an appearance yet.

I sit down and try to politely wave and smile at a flight attendant to get her attention. I think she sees me, but she doesn't respond by acknowledging me.  I want to fix this before the seat is filled.

It would be a rotten way to start a trip by finding tour seat with water on it. It's a modern conundrum.  How do I attract attention on a plane when the line is boarding, while not becoming that guy who attracts the wrong kind of attention.

I wave again and try to smile as I think I am acknowledged.  Still didn't work.

The call attendant light!  That's the ticket. I hit my button, eyes locked on the flight attendant. BINGO!  She sees me.

She mouth the words from across the plane, "Did you push the button?".

"Yes", I mouth back.

"Hit the button again", she says across the plane.

Okay I think, now I can simply point this out and get it taken care of. The water will be cleaned up. Everything will be right and in order before the seat gets filled. Wrong.

After I turn off the light the flight attendant goes back to talking with the people at the front of the plane. She must have assumed I did it on accident. Again I am discouraged.  I mean it's not even my seat. If I just wait when the person arrives they can hold up the line and at that point it will have to be fixed.

I can't leave it like that.

It is something simple, but it is so much more than that.  As a kid I loved Superman. I was always a fan. I probably slept in a cape more times than I didn't. I tried too, at least. My Mom was convinced that I would strangle myself in my sleep, if she caught me wearing one she would make me take it off.  And I would. Until she left the room.

Link to original
The thing about Superman was he helped people. That was important, even as a child I knew this. I think at that age we all do. We are supposed to look out for other people. We should help them. As an adult we tend to get jaded about that.

We realize there is only so much that we can do. We can't help everyone.  We cannot donate all of our time or all of our money to charity if we have a family or children at home. It wouldn't be wise, as a matter of fact that would be wrong.  We have to take care of our families first.  So we begin to make realizations over time. The innocence of our childhood fade away.  We realize we cannot save everyone.

During my brief time as a football coach for my boys a couple years ago I had a phrase I fell in love with. You play how you practice, you practice how you play.  Simple but true.

I still tell my kids that on a weekly if not daily basis. If you have a good strong work ethic at practice you carry that over to the game. If you quit at practice you will probably quit in the game. If you are sloppy at home or lazy about getting things done... you practice how you play.

If we cannot look out for others over simple things, how can be expected to look out for our coworkers, our customers, or our clients.  It's simple, but it is more than that. We have a responsibility to make the world better.

Period. Not just better in business. Better.  Period. End of sentence.  It doesn't matter if the goal is to lead employees, a team, or any effort that we under take in life.  We have the chance to effect change for the better in every action we take.   We can act with others in mind, and by doing so we will make things better for them.  You practice how you play.

This thought was fresh in my head because of work.

I lead a team of highly skilled highly talented consultants. We love fresh and new ideas. At Pragmatic Works, my job, we have had a very innovative approach to raises and reviews that allowed people to control their own destiny. Like many ideas it looked good on paper, but after several years we could see cracks in the system.

For the past four months we've been working to replace to old and determine the new. This isn't new to us.  Every year we take the feedback from our teams and we look at how we can do things better, how we can make changes that benefit us personally and professionally.  However, this year marked our largest set of changes we've introduced since I've been with the company.

The core thing that drove us was how we would grow, encourage, and support our team.  How do we help them?  Not just internally for the company, but how do we help them technically and professionally.  If people are here for years or only a short while, the goal is to help them leave better than when they arrived.  We need to help make you better.  As leaders for our company it is our job to help our employees.  Period.

We sat down with people and had a lot of 1 on 1 meetings discussing ideas and asking for feedback.

We took that feedback and kept some things and threw out others. We increased vacation, changed the review process, increased the benefits to help people be active in the SQL Community, and other technology communities.  We are working on redoing our mentoring program and some other very cool things to help people grow.


After all if you are in the business of helping people, which we are; what good are you if you cannot help the people who work for you.  You practice how you play.

"So Balls", you say.  "There was this water in a seat...."

Thank you Dear Reader.   So I stood up, waved to the flight attendant, and in my most polite southern accent I asked if I could get a napkin to clean up the water in the seat next to me.


She walk over to me with a puzzled look, as though I had spoken is some foreign tongue. As we both looked at the water she realized why I had been waving. She rushed off to get a couple napkins to clean it up. About 20 minutes later I began to laugh inside.  The seat next to me was still empty.

What if it was all for nothing and no one sat down.

Before long an elderly woman boarded late and took the seat next to me. Her seat was nice and dry.  I think my kids would be happy with me, after all you practice how you play.

As always Dear Reader Thank you for stopping by.

Thanks,

Brad