Troubleshooting SharePoint Diagnostics Logging

One of my clients called me this morning with a SharePoint logging problem.  Her server(s) were creating blank log files, 0KB in size without any errors or apparent issue.

I started with some basics:

1) Is there adequate disk space on the server(s)?
2) Is the “Windows SharePoint Timer Service” running?

I then restarted the timer service to ensure that the credentials were still valid.  But that didn’t help.

Next, I wanted to validate what my logging levels were set to, so I ran the following STSADM command:

stsadm -o listlogginglevels

This revealed that everything was set to “None” from the original defaults.

I next ran the following command to reset values to defaults:

stsadm -o setlogginglevel -default

That did it!

SharePoint Saturday Michigan – March 13, 2010 Announced!

I am excited to announce that we will be hosting a SharePoint Saturday event on March 13th at Washtenaw Community College.  We are currently seeking sponsors and additional speakers.  For more information, please see: http://www.sharepointsaturday.org/michigan.

The Cloud and Microsoft’s Business Productivity Online Suite

The Cloud

The Cloud is a metaphor for the term “cloud computing”.  One of the first web-based applications was SalesForce.com, starting in 1999, even before that there was Hotmail.  With the advent of technologies such as AJAX and the launch of Gmail in 2005 (later followed by Google Apps) the concept of web-based applications has become more widespread.

If you have had any involvement in the IT world in 2009, you surely have seen offerings from major vendors, such as Amazon (EC2) and Microsoft (BPOS) touting cloud-based services.  In 2010, service offerings will continue to grow and you are likely to run into in some form or another.

What is BPOS?

Our focus will be Microsoft’s Business Productivity Online Suite, or BPOS.  It is their suite of hosted messaging and collaboration solutions.

Offerings

The suite includes the following functionality which is well known to the client/server environment, but now available as a service:

• Email                                                                     ->                            Exchange Online
• Collaboration                                                     ->                            Office SharePoint Online
• Instant Messaging                                           ->                            Office Communications Online
• Meetings                                                             ->                            Office Live Meeting

Why Should I Care?

Web-based applications have not displaced boxed software as the primary means for delivering functionality, however, for those entrusted with crafting IT strategy for business, it is a trend that cannot be ignored.  Furthermore, Intel’s investment in the Atom processor platform, the increasing popularity of net books, and increasing power and functionality of PDA and smart phones should provide some clue that the concept is sure to grow, not fade away.

In addition, the possibility to lower the total cost of “ownership” through aggressive pricing and promotion to gain market share, guaranteed SLAs, and minimized management and maintenance are sure to garner of cost-conscious IT leaders.

Is BPOS For Me?

The advent of cloud computing, web-based applications and an ever demanding economic climate increase the need for flexibility and mobility.  From these new needs come the concepts of the “desk-less worker” and “mobile sales force”.

No longer does productivity require us to be tethered to our desks and bound by the traditional physical boundaries of “brick and mortar”.  Nowadays, our offices are the coffee shop down the street, the airport terminal, a home office, anywhere and everywhere we are needed.

The most obvious benefit to moving email, collaboration portals, instant messaging and conferencing to the cloud is in the reduced cost to manage infrastructure.  This includes eliminating the need to purchase server hardware, agents, software licensing and FTEs required to ensure proper maintenance through patching, backups, monitoring and administration.

The pricing for BPOS breaks down as a monthly per user cost.

• Exchange Online                                              ->                            $10/user/month
• SharePoint Online                                           ->                            $7.25/user/month
• Office Communications Online                  ->                            $2.50/user/month
• Office Live Meeting                                        ->                            $4.50/user/month
• Exchange Online Deskless Worker           ->                            $2/user/month
• SharePoint Online Deskless Worker        ->                            $2/user/month
• Deskless Worker Suite                                   ->                            $3/user/month

The next time Microsoft releases a set of critical updates, wouldn’t it be nice to let someone else worry about coordinating downtime, testing and applying patches?

But there are also benefits beyond having servers live on someone else’s network.  Spending less time to size, architect and deploy any or all of the above helps you roll out new functionality quicker.

Lastly, Microsoft is not bashful about touting its three nines (99.9%) uptime SLA and their cash-backed guarantee.  Where most vendors mitigate issues stemming from outages with credit towards future service, Microsoft “puts its money where its mouth is” and pays you should they fail to meet their SLA.

Is BPOS Not For Me?

While it has many great benefits, BPOS, is not solution for all.  The specifics of your environment will dictate whether it makes sense to consider it.

First, BPOS has a five (5) user license minimum, which for some very small business may not make sense because the unused user licenses may offset cost benefit.

Every company has some instance of a legacy or custom application that requires the typical client/server setup.  So while quite “long in the tooth”, these applications may still provide a critical function to your business and may not allow for migrating to a cloud environment.

There is also a feeling that if it’s “in the cloud”, I don’t really “own” it.  Reality is that if you read through the EULA (end user license agreement) the next time you install a Microsoft product, rather than just “clicking next”, you will see that you really don’t own that copy of Office 2007 Professional, you merely paid for the right to install it and use it.  However, this may be an extremely minimal issue once the initial shock of realizing what the EULA states, and most companies would realize how little value there is in the traditional boxed software model.

In environments where cost constraints force open source, free software models to prevail the cost issue remains.  If your company runs MDaemon because Microsoft Exchange is too expensive, chances are even the minimal per-user costs of Exchange on BPOS are still too high.  Granted these situations are few and far between, but they are out there.

What Happens to IT After We Move to the Cloud?

Isn’t the cloud just another way to “outsource” my IT staff?  In a way, yes.  So what happens to my IT staff when I move services to BPOS?  Think of BPOS as an opportunity to re-direct effort into other areas where BPOS cannot provide value; legacy applications, building content within SharePoint, BI, dashboards, reporting, end user training, planning and developing proactive plans for technology, and over getting out of “fire fighting” mode.

BPOS offers a great opportunity to reduce IT operational costs and provide better services.  However, I will reiterate some key points about my earlier piece on outsourcing e-mail that similarly will ensure success with a BPOS deployment; 1) ensure that additional costs do not offset savings, 2) that visible change to end users is minimized and where unavoidable, training is prepared and 3) that where people resources are involved, management sees this as an opportunity to better utilize skills instead of simply lowering head count.

Web Part Error: One of the properties of the Web Part has an incorrect format.

I had a call from a client this morning who was experiencing errors when editing instances of an InfoPath forms and in search results.  The full error was:

Web Part Error: One of the properties of the Web Part has an incorrect format. Windows SharePoint Services cannot deserialize the Web Part.  Check the format of the properties and try again.

My first thought was “weird”.  I have come across this error before, but only with custom developed web parts.  Being that this the edit form and search functionality are using all out-of-the-box features, this seemed strange for me. As I started to Google for some ideas, I found a couple of articles of interest:

1) OK, your C: drive could be full…that is easy to fix, you can see Hannah Scott’s post for details: http://bytelab.blogspot.com/2007/05/moss-webpart-errors-due-to-log-file.html

2) Alternatively, Shai Petel has a great blog for troubleshooting this error: http://kwizcom.blogspot.com/2008/06/unable-to-add-selected-web-parts.html

Under “Reason 5″, Shai mentions that after deploying DLLs to the GAC you should perform an IIS reset to register safe controls.  Sure enough, Windows Updates had inadvertently been pushed to the SharePoint environment.  I performed an iisreset /noforce and the error disappears from our InfoPath form and search.

Your licence for Office Server Search has expired

Earlier this week, a client asked me to assist with an issue they came across when performing a search on their SharePoint portal.  The portal was available, content was accessible yet when they searched they would receive the following error:

Your licence for Office Server Search has expired.

As I began to research the error, I several ideas but none that manifested in the exact way my issue did.  Here is what I found:

1)  UAC causing issues with configuration wizard: http://social.msdn.microsoft.com/Forums/en/searchserverinstallconfigandadminstration/thread/61e6fec0-f4dd-4473-84d9-2144b67149c8

2) Hotfixes KB 960010 and 960011: http://social.technet.microsoft.com/Forums/en-US/sharepointsearch/thread/96bde6cd-578c-4e72-b81f-af3e9b905415/

3) Another post pointing to re-running the configuration wizard: http://share-point.blogspot.com/2008/04/moss2007-license-has-expired-error.html

Because my environment had SP2 (12.0.0.6421) I wondered if this was a manifestation of the 180-day trial issue: http://blogs.msdn.com/sharepoint/archive/2009/05/21/attention-important-information-on-service-pack-2.aspx

Sure enough, when I went to Central Administration > Operations > Convert License Type I saw:

To resolve this, you can install KB971620: http://support.microsoft.com/kb/971620

Or, you can fix it manually…

1. Click Start, point to All Programs, point to Administrative Tools, and then click SharePoint 3.0 Central Administration.
2. Click the Operations tab.
3. Click Convert License Type.
4. In the Enter the Product Key text box, type your product identification number, and then click OK.
5. The License Synchronizer Job will run on all computers in the farm after a few moments. After all the computers have updated their license from the timer job, the Convert License Type page will reflect the correct license.

Voila!

By the way, the SP2 bits have been updated to resolve this issue.

Hope that helps!

Error: “Access is denied. (Exception from HRESULTS: 0×80070005 (E_ACCESSDENIED))” When Creating Crawl Schedule

As I was wrapping up configuration of the SSP for a new farm, I wanted to configure the full and incremental crawl schedules when I received an error.

Access is denied. (Exception from HRESULTS: 0×80070005 (E_ACCESSDENIED))

This new farm has SP2 and the October 2009 Cumulative Update installed so I was surprised to see that the fix for this has been around for quite a bit, yet this is my first instance of coming across this error in this manifestation of an old issue.

http://support.microsoft.com/default.aspx/kb/926959

 

SharePoint 2010 Downloads

Now that the SharePoint Conference in Las Vegas has come and gone, the mad rush to get ramped up on SharePoint 2010 begins.  For all of you looking to get a head start on the beta, click here:  http://www.microsoft.com/downloads/en/results.aspx?pocId=&freetext=sharepoint%202010&DisplayLang=en

How to Troubleshoot Alerts in SharePoint

I spent a little time last week troubleshooting alerts for one of our clients.  I had built this farm a little over a year ago and all of the sudden, alerts stopped working.  To give you some background, the farm is rather simple; 2 WFEs connecting to a their enterprise SQL cluster.

Of course, I asked the obvious questions:

1)  Is this happening for just a single user or everyone that has subscribed to alerts?

Answer: This is happening to everyone.

2) Did you check your spam filter at the server and your Junk Mail in Outlook?

Answer: Yes, neither place shows “lost” alerts.

3) Is this on new or old alerts?

Answer: Both

First, I connected to Central Administration and validated the outgoing mail server settings by going to “Central Administration > Operations > Outgoing E-Mail Settings” and checking  “Outbound SMTP server”.

My next step was to test the mail relay to this server, which simply involves connecting to the mail server via a telnet session on port 25.  For details on how to send a test message via telnet, see the following article:  http://support.microsoft.com/kb/153119.

Not expecting anything to have changed, I connected to one of my WFEs and since the test was successful I proceeded to test by creating another alert, changing an item for which alerts had been configured.

I also found the following article that has a fairly extensive testing process for validating alert functionality: http://msmvps.com/blogs/bradley/archive/2009/03/22/how-to-troubleshoot-alerts-in-wss-3-0-moss.aspx.

As I was stepping through the article confirming what I had already done, I came to step (k).

I ran the following query on my SQL server:

select * from timerlock

The output confirmed the issue as I saw which of my WFEs was responsible for processing the timer service.  It turns out that the relay to their Exchange environment had been changed to include only one of the WFEs.  All I basically needed to do to resolve the problem was re-add the relay for the second WFEs and within minutes alerts were flowing again.

After the fact, I also found this article which explains the process of testing alerts with NetMon to validate what is “going down the pipe”: http://blogs.msdn.com/vedvyas/archive/2009/09/02/troubleshooting-moss-alerts.aspx.  While it did not help in my situation, it would certainly be helpful had my issue not had been the relay but an ACL somewhere in-between my SharePoint farm and the mail server.

Take My Email, Please

Introduction

Gartner Inc. estimates that 80 percent of enterprises with fewer than 300 employees could save money by outsourcing email.

Last month, I wrote a brief article for our “Knowledge Transfer” newsletter focusing on the potential risks of outsourcing IT operations.  For firms that have decided that outsourcing is a viable way to lower IT operational costs, one of the first areas of focus is email.

The ability to quickly share information via email has made it an invaluable asset for information workers.  If has effectively replaced letters, faxes, phone conversations as a means of conveying information.  However, despite all of the benefits, email is a management nightmare for IT professionals.  Not only does the increasing business reliance on email add to the complexity of systems required to maintain it; but the evolution of security threats and stringent legal requirements can add costs far beyond the cost of hardware and licensing.

For many, outsourcing email (and all of the responsibilities that come with it) can represent a substantial opportunity to reduce costs.  Gartner Inc. estimates that 80 percent of enterprises with fewer than 300 employees could save money by outsourcing email¹.

Many Choices

For those considering outsourcing their email, there are a myriad of options.  If you are currently running and content with Microsoft Exchange, the latest option is Microsoft’s SaaS solution, or Microsoft Exchange Online.  If you would like to host your Exchange through a third-party, but not necessarily Microsoft, Microsoft Partner Hosting providers include; Rackspace, Intermedia, among many others.

If Exchange is not your “cup of tea”, there are plenty of other hosted, or cloud solutions to outsource your email; vendors include Rackspace, Gmail and Zimbra.  Lastly, everyone has heard of Google’s Gmail for personal email; they now offer an enterprise email solution.

Why?

Outsourcing your email infrastructure allows your business to lower costs by eliminating licensing requirements, reducing the number of servers required to host critical business functions, eliminating the need to periodically dedicate substantial resources to maintenance such as patching and upgrades.

Why Not?

For some organizations, outsourcing email may not be so easy.  Integration with other systems such as CRM or ERP may be limited when moving to a third party.  The size of your deployment may help you leverage some concessions that allow you to make their systems work; however, this is on an individual basis.

Costs

It’s possible that for some users, you may be able to lower costs of email by up to 75%.

The ongoing cost of outsourcing your email will vary by vendor and also depends on the specific features requirements of your organization.  Expect that for the average “information worker” the cost of our sourcing their email account is about $10 per month².

Whereas some studies estimate the cost of hosting email internally costs up to $27 per month for the average “information worker” and up to $40 per month for a “mobile executive”³.

Gotchas

Outsourcing should be viewed as not only an opportunity to reduce costs, but to rid yourself of burdensome tasks which will free your employees to concentrate on more valuable responsibilities that make better use of their skills.

By its nature, there is an ugly side to outsourcing in that it allows you to shed responsibilities.  This sometimes may include reducing staffing requirements.  Ramifications can range from poor public opinion to an unexpected loss of talent through attrition as employees fear that “they are next”.  Either way it’s a complex “layer 8” issue that cannot be ignored.  Outsourcing should be viewed as not only an opportunity to reduce costs, but to rid yourself of burdensome tasks which will free your employees to concentrate on more valuable responsibilities that make better use of their skills.

Moving critical business data to a third party provider raises concerns about privacy and data protection.  To prevent your email outsourcing solution from being a “black box” it is important to find a provider that comes with the best references and also conducts pro-active monitoring and testing to ensure they meet SLAs.

Deviating from basic vendor offerings may cause you to incur additional costs.  It is important to understand how integration with LOB software, Blackberry services may be affected.  Lastly, there may be one-time costs involved with configuration and migration of your email data.  Once your email is hosted remotely, you may find that you need to increase the bandwidth of your connectivity to the vendor.

Bottom Line

Outsourcing email offers a great opportunity to reduce IT operational costs.  However, the key to successfully achieving the potential cost reductions careful planning and attention to details is required to 1) ensure that additional costs do not offset savings, 2) that visible change to end users is minimized and where unavoidable, training is prepared and 3) that where people resources are involved, management sees this as an opportunity to better utilize skills instead of simply lowering head count.

References

¹ http://www.itmanagement.com/features/corporate-email-outsourcing-052908

² http://sharepointlotus.net/content/fully-loaded-cost-corporate-email-and-how-take-it-down

³ http://blogs.forrester.com/information_management/2009/06/calculating-the-fully-loaded-costs-of-email-its-bigger-than-you-think.html



To Outsource or Not to Outsource…

…there are many potential risks to outsourcing which can substantially outweigh the benefits…

As the world flattens (sorry, Thomas Friedman reference) the popularity of outsourcing increases. But there are many potential risks and hidden expenses which can substantially outweigh the benefits, so it is crucial for management to consider these risks when considering moving key IT functions to an outside provider.

In the manufacturing sector, the trend has been to outsource labor-intensive functions of the supply chain to China. In the service industry, the trend has been to outsource non-essential functions to India.

In difficult times, our initial reaction is to rein in spending through workforce reductions or layoffs, or move certain business processes to a lower-cost provider. Over the last decade, outsourcing commoditized IT skills has become fairly common – but the ROI from these practices remains elusive.

Reduced Quality

The loss of control over day-to-day operational functions can produce less-than-desirable effects in how the work product measures up to organizational standards. This requires additional resources to review quality and resolve issues before the product is issued or shipped.

Public Opinion

The second negative effect is more ethereal, but can still have a drastic impact on profits.

Actual services may meet all guidelines and requirements, but clients may still perceive a lower level of quality simply because they know the service is being performed overseas. Previous experience with language barriers can generate negative sentiment towards your company. These barriers are falling, but overseas outsourcing can still pose an issue for many customers who either want to interact with American workers, or who perceive outsourcing as indicative of financial instability.

Protection of Law

Despite the tightest wording in confidentiality and service level agreements, it may be difficult to enforce breaches of contract in a foreign court. Once third-parties are exposed to confidential client information, intellectual property or patented information, you have no guarantee the information will not be stolen.

Ultimately, the decision to outsource requires an evaluation of internal skills, and an honest assessment of your firm’s tolerance for risk. Even when management decides that outsourcing provides value, it still must consider hidden costs related to mitigation of risks that could narrow the savings gap.