Hello Dear Reader! This past Thursday night I took my son Zachary to see Spider-Man Homecoming, it was an amazing movie with a fantastic cast. Both of us are BIG comic book guys, and we enjoyed it immensely. As a matter of fact we both enjoyed it so much that I couldn’t help but wonder how the rest of the world felt! You’ve probably seen different ways to view Twitter sentiment analysis, but I wanted to do this today using Azure Logic Apps to create a real-time Power BI Dashboard, with a live data source using Azure SQL Database. We’ll how we did this in detail in a later blog post, but first let’s look at the data!
Want to read a little more? Head over to our, Josh Luedeman (@Twitter) and myself, blog over at MSDN! Click here for the Deeper Insights Blog.
Monday, July 10, 2017
Wednesday, March 1, 2017
Microsoft: 1 Year in, how's it going?
Hello Dear Reader, one year ago today it was leap year, February 29th 2016. So I'm a year older, the big four zero.... 40. A little wiser? Maybe.
Despite my New Year's resolution to get fatter and work out less, I've lost 21 pounds and I'm working out 3-5 times a week. CURSE YOU NEW YEARS RESOLUTIONS!!! YOU NEVER COME TRUE!!!!
Also I left my old job as a Data Platform Lead, ie 'Executive Manager', and went back into the field as a Premier Services for Developer Consultant. Oh and I did this by going to work for Microsoft.
That's right Microsoft.
"So Balls", you say "Is this Micro-soft that makes the ultra-micro-soft pillow cases? or is this the REAL Microsoft?"
Good question Dear Reader! Yes this is the real deal. I really work for Microsoft, also I feel the need to smile whenever I say that.
WAS THIS THE RIGHT DECISION?
I don't want to make this seem like being an executive or a manager were a bad experience. They were not. I love the people who were on my team at Pragmatic Works. Some have left, some have stayed, and others joined Microsoft. I love the people I worked with, and I still do.
I would relish the chance to Manage again, but it would have to be the right scenario. I was asked to manage teams many times in my career before PW. I refused. It wasn't that I didn't want to. I knew that if I had to lead the team I was a part of, I would have to
Adam Jorgensen (@WAdamJ), who has probably changed his twitter handle.....again...... the second I linked to it in a blog post, was a FANTASTIC boss, mentor, and friend. There are a few habits I picked up from Adam that continue to help me.
My first year I felt like I was spinning my wheels. I was more efficient, but I was still doing some of the same things and didn't know a better way to do it. Adam asked me how many books I read on SQL when that was my field. I stopped to count and he said, "A lot right? The number doesn't matter. It's the concept. You read a lot, studied more, and experimented to become an expert right?". He was right*. He then asked me, "How many books have you read on management?" I was floored. But I also didn't know what to read.
Adam created a spreadsheet that had a lot of book recommendations. He gave it to me in a spreadsheet and prioritized what he felt the most important books where. Some I loved, some I make everyone read, and others... were terrible.
I learned, I grew. I liked it. I also realized what kind of management books I enjoyed and that lead me to more books, more ideas, more new concepts.
Moving from a Consultant to a Manager was like moving from a wide receiver or running back to a Football Coach. You don't play on the field. You put people in place to play positions, you can even put a talented lead consultant in the field, your quarterback, by you don't do the work. That's not your job.
You do the opposition research now. What does the field look like? What is the goal? How do we develop the right tools so our team can win?
Going back to being a position player was a little scary. I never gave up being technical, but I didn't know if I would enjoy it like I used to.
*By the way Adam likes it when you agree with him try to do that. But don't tell him that's why you are doing that, for some reason he doesn't like that.
SO HOW DO YOU LIKE IT?
I love it. I love this job. I sometimes pinch myself. I've called my friend Jorge Segarra (@SQLChicken) sometimes just to thank him. My thank you generally starts something like this, "Dear Sir, Thank You for you kindness, generous patience, and pure grit for the number of times you called me, asked me to join you, and for all the times I said NO. Thank you for not giving up on me."
I could go on and on, but let's start to break down what is so great. I didn't number these, because it's hard. I tried. I picked my top five. There were more. I ordered, re-ordered, and then said the heck with it. So here they are. The Corporate Culture, Freedom, My Managers, Health Benefits & Corporate Giving, and The Ability to Grow as a Professional.
The Corporate Culture. The company is driven by a Growth Mindset, you've seen these changes. The quality of products are better. The collaboration is better. It's no longer Microsoft vs. the world. Gone are the days when your Microsoft app works great as long as you use additional Microsoft products X, Y, and Z and woe-be-on-to-you if you used anything non-Microsoft.
This all comes from a growth mindset of our Senior Leadership Team that views different competitive advantages as challenges to get better. The recognition that a Microsoft product is not always the best answer, and the zeal to learn from competitors and our mistakes to constantly grow. We aren't just sitting still here. There is constant progress to be made, it starts with reinventing ourselves with a clear mission dedicated to help others achieve more.
Freedom. Freedom is a great and scary thing. It means that I get to be responsible for me and my time, but it also means I am responsible. My manager doesn't care if I'm at my desk. If I'm on the road two weeks in a row I may have an errand to run in the middle of the day. He's not constantly checking up on me. He encourages me to be active on social media, trusts that I will perform for my client and job, and allows me to manage that myself. The expectation is that I'm a responsible professional, and I get treated that way. It's simple, but pretty darn revolutionary given what I've experienced throughout most of my professional career. I pick my engagements, I pick my schedule, and I pick where and when I travel.
The Health Care Benefits & Corporate Giving. I not going to go into the benefits. Its not that I couldn't talk about some of them, but I don't want to accidentally cross any lines. I typically discount benefits in negotiations. Company X will say, take this job because we offer this, and this accounts to us paying this much money. I've been at plenty of companies with "GREAT HEALTHCARE OPTIONS". None of them come close to crazy good benefits. Companies use them as negotiating tools. I discount them because at the end of the day none of them were much better than the others. That is until this one. If you've ever worked at Microsoft you know they are crazy good. I don't know if the people who have worked at Microsoft their whole careers realize how good they have it. Personally, I'm extremely satisfied.
The Corporate culture of giving is also amazing. A couple of times a year the whole company spends lots of free time volunteering, creating, and offering up things for others to purchase just to give the profits to charity. We have multiple giving campaigns and Microsoft matches what you give up to over $10,000 per employee. There are whole charities that have great causes from building sustainable housing for those in need, to raising money for cures to diseases and prevention, to disaster relief. Some of them almost entirely funded internally by Microsoft employees. This is selflessness at its very best. There are so many people here doing good you cannot help but be inspired by example or by proxy. I feel very strongly about this, because Microsoft does not beat their chest and say "Look how great we are!", if you didn't see this from the inside you may not even know it occurs. This culture has been around since they existed. This is built into the place. Watching people do amazing things, and knowing my 'work' helps bring in revenue that in turn is donated to these great causes in some way, shape, or form is inspiring.
The Managers. I have two managers. One that I directly report to, but they both have decided to co-own our group. Remember all the good things I said about my friend Adam. Take that, wrap it in Microsoft culture, and put almost two decades of experience behind it. They manage a team of All Stars from App Platform to Data all with a developer focus. I've worked with some of these folks before, and even managed some of them.
To see them thrive in this environment helps me see places where at PW I could have done better, or how things could have been done for the better. It is a very interesting learning experience. I managed Bradley Schacht (@BradleySchacht), Jorge Segarra, Josh Luedeman (@JoshLuedeman), Gareth Swanepoel (@GarethSwan), and others are at Microsoft now. All thriving. It is a very cool thing to watch, to see new managers get the chance to work with them.
The really great thing is my Managers realizes this. We talk often as Managers and I'm asked for feedback and perspective. When I have it I give it, when they deserve praise I give that as well. The best feeling is the mutual respect that is given. Never once have I heard the phrase, "Well things are different here you aren't at <insert former job> you are at Microsoft and we do things this way". That comes from the corporate culture, and is embodied by how we communicate. There is always respect and a desire to learn what we can from different experiences. My managers are really incredible, and it inspires me even for when my next opportunity to lead presents itself. Dan and Niel, Thank you both!
The Ability to Grow as a Professional. One of my first conversations with Niel during my on-boarding was about how to find my next job at Microsoft. I thought, "Well that interview must have not gone as well as I thought this guy already wants to get rid of me!" The conversation revolved around finding a manager that inspires you. Finding someone that you want to work with, and making sure that the fit is right. The best part about the conversation was realizing here I am on my first day, and my manager believes in me so much that he's already trying to help me figure out what will be my next step at Microsoft. Internally I've watched people I admire transition from CSS to the Tiger Team or to the SQL Server Product Group or from the Product Group to AzureCAT. The fact is as long as you want to grow as a professional, you have a place to go within Microsoft. I didn't even mention the massive investments they make to help each of us grow technically, but I'm starting to run a bit long and if you've hung in there I appreciate your time.
WRAP IT UP
Dear Reader, I'm home. I hope to be here for as long as I can.
As always, Thank You for stopping by.
Thanks,
Brad
Tuesday, February 28, 2017
How Do I Add an R Package to SQL Server from In-Database R?
Hello Dear Reader! I'm working on some really fun stuff with in-database R using SQL Server 2016.
I needed to add a couple packages to my R engine so I could use them and call them from an R script. I realized I had not added any external packages to my R instance so this was a perfect opportunity to write a blog! So here is a quick blog on how you add an R Package to SQL Server. This entire tutorial is if you have internet. If you need to do this for a server, which probably doesn't have access to internet look at this MSDN article, Install Additional R Packages on SQL Server.
First we need to find our instance default install path for SQL Server. In this case I'm a comic book geek and my named instances are JARVIS and STARK. STARK is my 2016 instance so we'll need to find that path.
In this case my path is C:\Program Files\Microsoft SQL Server\MSSQL13.STARK\R_SERVICES\bin\x64 , then I want to hold down SHIFT and right click on the Rgui.exe, then select Run as administrator.
After this opens we need to create a variable to hold the location were we would like the package to be installed. Then we can install the package we want to use, in this case the plyr package.
lib.SQL <- "C:\\Program Files\\Microsoft SQL Server\\MSSQL13.STARK\\R_SERVICES\\library"
install.packages("plyr", lib=lib.SQL)
Notice I adjusted the path for my named instance.
After it installs our window should look like this.
Now I can run a simple test script loading the plyr package.
If I get this error the package did not load.
Msg 39004, Level 16, State 20, Line 4
A 'R' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004.
Msg 39019, Level 16, State 1, Line 4
An external script error occurred:
Error in library(plyr) : there is no package called 'plyr'
Calls: source -> withVisible -> eval -> eval -> library
Error in ScaleR. Check the output for more information.
Error in eval(expr, envir, enclos) :
Error in ScaleR. Check the output for more information.
Calls: source -> withVisible -> eval -> eval -> .Call
Execution halted
Msg 11536, Level 16, State 1, Line 4
EXECUTE statement failed because its WITH RESULT SETS clause specified 1 result set(s), but the statement only sent 0 result set(s) at run time.
If I the statement compiles, everything is fine and I get a result set.
As always Thanks for stopping by.
Thanks,
Brad
I needed to add a couple packages to my R engine so I could use them and call them from an R script. I realized I had not added any external packages to my R instance so this was a perfect opportunity to write a blog! So here is a quick blog on how you add an R Package to SQL Server. This entire tutorial is if you have internet. If you need to do this for a server, which probably doesn't have access to internet look at this MSDN article, Install Additional R Packages on SQL Server.
First we need to find our instance default install path for SQL Server. In this case I'm a comic book geek and my named instances are JARVIS and STARK. STARK is my 2016 instance so we'll need to find that path.
In this case my path is C:\Program Files\Microsoft SQL Server\MSSQL13.STARK\R_SERVICES\bin\x64 , then I want to hold down SHIFT and right click on the Rgui.exe, then select Run as administrator.
After this opens we need to create a variable to hold the location were we would like the package to be installed. Then we can install the package we want to use, in this case the plyr package.
lib.SQL <- "C:\\Program Files\\Microsoft SQL Server\\MSSQL13.STARK\\R_SERVICES\\library"
install.packages("plyr", lib=lib.SQL)
Notice I adjusted the path for my named instance.
After it installs our window should look like this.
Now I can run a simple test script loading the plyr package.
execute sp_execute_external_script
@language=N'R'
,@script =N'
library(plyr);
OutputDataSet<-InputDataSet'
,@input_data_1=N'select 1'
WITH RESULT SETS undefined;
If I get this error the package did not load.
Msg 39004, Level 16, State 20, Line 4
A 'R' script error occurred during execution of 'sp_execute_external_script' with HRESULT 0x80004004.
Msg 39019, Level 16, State 1, Line 4
An external script error occurred:
Error in library(plyr) : there is no package called 'plyr'
Calls: source -> withVisible -> eval -> eval -> library
Error in ScaleR. Check the output for more information.
Error in eval(expr, envir, enclos) :
Error in ScaleR. Check the output for more information.
Calls: source -> withVisible -> eval -> eval -> .Call
Execution halted
Msg 11536, Level 16, State 1, Line 4
EXECUTE statement failed because its WITH RESULT SETS clause specified 1 result set(s), but the statement only sent 0 result set(s) at run time.
If I the statement compiles, everything is fine and I get a result set.
As always Thanks for stopping by.
Thanks,
Brad
Friday, February 3, 2017
The High Availability Conversation Part 1: Introduction
Hello Dear Reader!
Twitter is a fantastic tool if you use SQL Server. It is a great way to network, find
presentations, interact with experts, and #SQLHelp offers some of the best
technical forum conversation on the subject.
I was on Twitter and my friend Kenneth Fisher asked a question.
Kenneth’s question was interesting. Let’s define a couple terms up front. HA in this case is High Availability. It is the capability for a database or data
store to maintain availability and connectivity to a Graphical User Interface,
Web Service, or some other data consuming application despite a localized
outage or failure of the primary system.
High Availability is different from Disaster Recovery, or
DR. Disaster Recovery is required when a
disaster, natural or man-made, prevents the use of resources within a data
center and the hardware within to support regular business processes. In this blog, we will be addressing HA only
as that was what Kenneth had asked about.
Asked for further explanation Kenneth didn’t need HA at his
environment, more on this in a moment, he was looking for a rounded point of
view. I had a couple replies and then
started direct messaging, DM`ing, Kenneth to share my insight. This is a conversation you must have with the
business.
A keen understanding
of how business logic translates into technology workflows requires both IT and
business leaders making the right decisions together. Over this series I’m going to give you a
couple examples on when HA was needed. Some
in which HA was not. Later we will discuss different HA technologies and
understanding when each one is right for you.
First I will define some key things to cover in a High Availability
Conversation between the business and IT.
THE HIGH AVAILABILITY
CONVERSATION
There are three types of conversations: technical, business,
and budget. Business comes first. Our first goal is to understand what the
business is facing. Some of this you may
know in advance. If so review the
information to ensure business and IT communication.
·
Describe the business process –
o
What product or service are we providing?
o
What type of employees participate in the
process?
o
What does each employee need to get from the
process?
·
Diagram a workflow of how a customer interaction
occurs
·
Add the steps that are IT specific
·
Critical days, weeks, months, holidays, or times
of the day for the specific business process
·
Understand any current paint points
First make sure you understand the business process
surrounding any technical system. Draw
shapes on dry erase boards, connect them to business process, make sure you
understand who the business owner is for each technical system. Force
self-examination and business awareness.
Add your IT system to the diagram.
Highlight business and IT dependencies.
Talk to teams to understand what happens when a system goes down.
You can use Viso, PowerPoint, a dry erase board, or whatever
you like. Make a visual representation based on the findings from the
meeting. Confirm that everyone can agree
on it, or use it to solicit their teams for additional vetting. You
may not get it right on the first run.
That is okay. This could be an
interview process depending on the complexity of your IT environment. There are
some standard technical terms that you should use
·
SLA – Service Level Agreement. This is the expected agreement between the
business and IT for overall system availability. This may represent business hours. In some cases, Service Providers may have
contractual SLA’s. If your business has
an SLA with customers there could be a financial penalty for not meeting the
SLA.
·
RPO – Recovery Point Objective. This defines the amount of data that can be
lost on a system in and not cause impact or harm to the business. The point at which all data or transactions
must be recoverable. The price of any
solution increases with lower RPO’s.
·
RTO – Recovery Time Objective. This is the amount of time it takes to get a
system online in the event of an outage.
The time to recover will vary greatly on size, volume, and type of data
discussed. If you seek to reduce
recovery time there are strategies or technical solutions that can be
utilized. The complexity of these
solutions can add to the overall cost.
REFINING HA: USING
THE THREE BEARS TECHNIQUE
After the first business meeting the technical team should
meet to discuss their options. I like to
develop three options when working with IT Managers. This is where we, the technical staff, align
a few things.
·
Proposed Technical Solution
·
Cost of Technical Solution
·
Cost to Maintain Technical Solution
·
Migration from the existing platform to the New
There are many ways to address HA. Some could be solutions using Third party
vendor products that you may already utilize, others can be hardware or
virtualization options, and other can be database specific. IT the architecture, size of the corporation,
and business need for HA will shape these decisions.
Notice I said three options for IT Managers. When you meet with the business you should
have a cohesive vision. When I started
my IT career finding a manager with IT experience was extremely rare. It is increasingly becoming more common. As IT knowledge permeates the business world
it will be more common to give the three options to the business for
collaboration. If you are lucky you are
in an environment like that today.
For this step you need to take all business requirements and
align them to technical justification for a solution. Producing three options may seem
difficult. Think of this like the old
children’s story Goldilocks and the Three Bears. One of these technical solutions will be too
cold, too much latency and not viable for 100% of business scenarios. One of these solutions will be too hot, this
is the spare-no-expense-throw-everything-you’ve-got-at-it-as-close-to-five-9’s-as-possible
option. One will be just right. It will fit the business needs, not be cost
prohibitive, and be maintainable solution.
You’re natural first reaction will be, “Why Three options! Why not just
propose the right one?”.
Valid point Dear Reader.
The reason is simple. This is a
high stakes mental exercise. The
business, jobs, profitability, security of our data, and many other things may
rest on our solutions. What makes a good
IT person great? Their mind. Our ability to make virtual skyscrapers and
complex virtual super highways out of thin air.
There may be a natural reaction to jump at a ‘best’ quick solution. Take your time, think outside of the
box. You will find if you push yourself
to consider multiple options, one may supplant the ‘best’ quick solution. While too cold and too hot may not work in
this situation, they may next time.
We’re done, right?
Wrong.
If we are changing an existing system, how are we going to get there? We need to understand if there are production outages windows we must avoid. For example in Retail or Manufacturing there are normally Brown out and Blackout windows for IT. Leading up to Black Friday, a United States shopping holiday, most Retail environments have IT Blackouts. This means no changes can be made in production unless they are to fix an existing bug. Everything must wait. Leading up to a Blackout window there is a Brown out window. During a Brown out no changes can be made to existing server infrastructures, this can limit the ability to allocate more storage from SAN’s, networking equipment, or networking changes in general.
If we are changing an existing system, how are we going to get there? We need to understand if there are production outages windows we must avoid. For example in Retail or Manufacturing there are normally Brown out and Blackout windows for IT. Leading up to Black Friday, a United States shopping holiday, most Retail environments have IT Blackouts. This means no changes can be made in production unless they are to fix an existing bug. Everything must wait. Leading up to a Blackout window there is a Brown out window. During a Brown out no changes can be made to existing server infrastructures, this can limit the ability to allocate more storage from SAN’s, networking equipment, or networking changes in general.
If we are performing an upgrade to use a new HA technology,
when will that happen? How do we
migrate? What are the steps? Have we coordinated with the application
development teams? They will need to
regression test their applications when a migration occurs. This means changes in the development,
QA/UAT, and eventually the production environments. Additional coordination will require the
server and SAN teams. Depending on the
size of your IT organization this may be a large effort or the act of shouting
over your cubical wall.
THE HA CONVERSATION II: THE SOLUTION & THE
RESPONSE
We’ve spoken with the business. We understand their process. We drew it up, passed it around, and found a
few new things along the way. We met as
an IT team. We produced tentative
solutions. Took the ideas to Management
and now we have a winner. We’ve
coordinated with our counterparts to make sure we knew what a tentative
timeline would look like. Our next
step? Present the solution to the
business.
After this meeting the goal is to move forward. To get the project going. We’ve spent a lot of work
on this, and you
want to see it implemented. Sometimes
that happens. Sometimes it doesn’t. There are a lot of factors that can go into
why something doesn’t happen. Because
they can be legion, and it distracts from our overall topic, I’m going to skip
why things do not happen. It is an
intriguing topic that I may blog on later.
When we move forward the goal it to do so in a collaborative
way. A good solution addresses business
needs, technical requirements, project timelines, and budget. There may be changes, there may be wrinkles
that occur along the way. What looked
good on paper doesn’t always translate to technical viability. It is important to maintain the communication
amongst key stake holders along the way.
Alright Dear Reader, this has been a good kick off towards
our topic. If you have questions or
comments please sound off below! I look
forward to hearing from you. Next we
will review a couple real-life examples of when HA was needed and when it
wasn’t. As always, Thank You for
stopping by.
Thanks,
Brad
Subscribe to:
Posts (Atom)