Errors importing SharePoint web parts and Health Monitor alerts

This is a draft, but to get the notes down I am publishing it and will update it master with the details.

We had been receiving errors alerts in the Health Monitor after upgrading NeoStream’s P4E product line to SharePoint 2013. Although the error alerts were front and center in the monitor. It wasn’t breaking anything so we ignored them.

I recently started mucking around with importing sites from PowerShell and cane across some errors that had the same smell about them.

These errors seemed situational at first, occurring for a while and then disappearing. It turns out that the error “failed to import unknown web part”, which would last for a while in PowerShell then go away, would actually go away the moment I opened a specific SharePoint web part page in the browser. Something was amiss and sharepoint corrected it when the page opened.

I ran a couple scripts to diagnose the web parts in error and nailed it down to a custom web part in the product. This webpart was pretty benign itself, and after checking that it wasn’t the configuration of the web part I cracked open the code.

The answer was immediately apparent…

Something that alot of people don’t know is that sharepoint operates in a few different processes, including the IIS worker process, the SharePoint Timer service, and (when you use PowerShell) in PowerShell. Well of these processes and more, only one of them actually has a SPContext.Current, which is created in the ASP.Net pipeline based on the http request. This means that any code calling methods or properties on the SPContext.Current which is outside of ASP.Net, will fail with an object ref error.

So what’s that got to do with web parts since they are UI and wouldn’t need to be loaded outside of the website? Well SharePoint tries to validate the registered web parts (reporting the results in the monitor) and configure the web parts when importing via PowerShell. These processes require loading the data, which sharepoint uses the binary serializer for, which actually creates an instance of an object. Now it doesn’t raise any control lifecycle events which would make up most of the code but it does call the parameter less constructor and initialize any fields.

So if you have any code called from the constructor that calls SPContext.Current, that doesn’t check for it to be null then and exception will occur during deserializing. This exception will also occur if you initialize any fields from SPContext.Current, which was the cause in this case with:

private SPWeb spWeb = SPContext.Current.Web;

So making that a read only property, solved that as those aren’t touched by the serializer.

Advertisement

Drobo Mini and thin Intel Solid State Drives Getting Stuck

So I bought a Drobo Mini and stuck a couple Intel SSDs in it only to learn that the Drobo Mini isn’t compatible with the 520 series… Boo, but what can yah do, its a valid enough reason they don’t work.

So I would then use the drives elsewhere (they are still great drives), but when trying to eject them I found myself in a very frustrating situation. The Intel drives are thinner than usual and so to make them fit, Intel includes a plastic frame attached to the top of the drive. Its enough to make them fit snuggly in a drive bay without adding weight since the frame only wraps the outside edges.

Well the Drobo Mini has an (assuming its needed but still) annoying strip of flexible metal in it that holds the drive snug. Unfortunately it is not as wide as the drive and once loaded drops into the frame on the drive, holding it in place. Holds it snug nicely, however, when trying to remove the drive, this strip of metal prevents the drive from sliding out. I could slid it out enough to disconnect the drive but any further and the frame would hit the metal springy strip and stop.

The Drobo Mini is mini for a reason… Its packed in tight, with barely any space around the drive openings. With the drive part way out I had more room but the only thing flat and long enough to fit was a butter knife (as I cringe again at the thought) but as I could get part of the metal raised out of the way it was wide enough and flexible enough that the other corner stayed down. Two knives were just two hard to maneuver, so I took a break to let my mind process things.

I needed something wider and thin as paper. Nothing came to mind that would be thin enough but strong enough to lift the metal strip. Then it occurred to me that that it only had to be strong enough to not crumple since if wide enough it could ride along the frame and, with the frame supporting it, lift the metal strip up and out of the way of the frame.

I cut a piece of a cereal box out, almost as wide as the drive but twice as long. Slid the cardboard carefully along side the top of the drive and as far back as it would go, and success.

The cardboard held flat and was enough to lift the metal strip up and I was able to remove the drive.

Issues with .Net Assembly Binding Redirection between .Net 2.0, 3.0 and 3.5 to .Net 4.0 and 4.5 (relating to SharePoint 2010 and 2013)

After about three hours of uncertainty as to why it didn’t work… a last ditch test and a 3am eureka moment converged to the following:

So a bit of backstory… the company I am working for, without going into the sensitive details, was writing a console app that we wanted to work against both SharePoint 2010 and SharePoint 2013.  Well there’s lots of hacky was to do that, including two different code brancehs which then need to be kept in sync or one code branch but doubling up project files and solutions in the folders, one pointing to SP2010 and one at SP2013… We use TFS build services and so any option would work, but most a management nightmares… so on being asked I suggested we use the assembly binding redirection (just like SharePoint itself does for loading webparts from older versions), as it’s the easiest to maintain.

So we tried… and after a good number of hours by another dev, and then by myself also for three hours, so couldn’t figure out why it didn’t work.  It would compile fine, and using the FUSLOGVW.EXE we were able to see the redirect was occuring, but then it wasn’t able to find the SharePoint 2013 assemblies (version 15).  All we had was a cryptic message at the end of the fusion log about “GAC Lookup was unsuccessful”.

So we ended the day with a big WTF?!?!

Well, so about 2 mins after I logged off, I decided to try one more thing and got a build working by changing the target framework to be .Net 4.5… a step forward.  Although this helped for SharePoint 2013 all of a sudden, it stopped working for SharePoint 2010, again complaining about not being able to find SharePoint.  Odd… but it’s something.

Then at 3am in the morning, the EUREKA moment happened… I figured out ‘why’, and then the details to make it all come together. The problem is that .Net 4.0 and above is a different GAC location now (it’s now in the place it should have always been).  If the project is a .Net Framework 3.5 project then it’s looking in the wrong GAC for the SP2013 assemblies (since they are .Net 4.5).  This was the whole reason it didn’t work from the beginning; when compiled to the right target framework, you don’t even need your own the binding redirection entries since SharePoint actually includes the minimum ones as policies in the .Net 4.5 GAC.  Having made the project a .Net 4.5 project it would have worked fine on SP2013 without any config settings as it would have looked in the correct GAC.

There is a catch though that would seem to be SharePoint specific.  SharePoint 2010 actually checks to ensure the app is targetting the ‘supported’ .Net 3.5 framework.  What this means is that to run against SP 2010 the EXE needs to target a .Net v3.5, while to run against SP 2013 it needs to target .Net v4.5 (or v4.0, but v4.5 is better).  The project can reference the SP2010 assemblies and, if targeting to the correct framework, will automatically adapt for the basic Microsoft.SharePoint assembly (there is a ‘but’ here that didn’t affect our specific situation right now).

So the next question is, how do we have one project target two different framework versions… answer, we don’t.  Luckily for us, MSBuild allows us to pass in overriding values for many things, one of which being the TargetFrameworkVersion (and in support of that, the ToolsVersion if needed).  And so we can instead make two build definitions in TFS (or simply two MSBuild commands if you are using some other automated build tool), one that does the default (or we could explicitly target .Net 3.5) and the other that passes additional arguments to MSBuild to explicitly target .Net 4.5. (it might be possible that we could also do one build definition with two outputs, one for each framework also). Now when the builds are kicked off, the exact same code, project file, and solution is used, but the MSBuild overrides the version and gives us one for SP 2010 and one for SP 2013.

Here are the details of the changes to make.  Open the original solution and ensure that it’s targeting .Net 3.5, and that the SharePoint reference is pointing to the SP 2010 (v14) DLL. Create one build defintion to do the defaults, and then a second build definition and add additional MSBuild arguments (in TFS you can also add them per queued build if you wanted to test this first) of:

/tv:4.0 /p:TargetFrameworkVersion=v4.5

* Note, both build defintions would point to the same SharePoint 2010 build server, for the compile to work, and you will also need to ensure that the .net 4.5 is installed to the SharePoint 2010 build server also (this doesn’t affect SharePoint, but is required to pick the 4.5 target when compiling).

Now, when you kick off each build it should create one as a SP 2010 compatible EXE and one as a SP 2013 compatible EXE (if we wanted we could use more arguments to alter the EXE output name to make them distinct also, such as “MyToolForSharePoint2010.exe” and “MyToolForSharePoint2013.exe”).

One code base… one thing to maintain, driven by configuration.

 

Some useful references:

http://msdn.microsoft.com/en-us/library/bb383985(v=vs.90).aspx

http://stackoverflow.com/questions/12003475/override-target-framework-from-command-line

The time travellers ego…

I was just watching “Back to the Future 3”, and it was at the part where they were discussing the driving their car into the future on track that didn’t yet exist. The explanation was sound at first, sure the track doesn’t exist now but in 100 years it will so we will appear and glide along…

But the earth is rotating, fast, about 1,674.4 km/h it would seem. So maybe you take that into account and travel to the exact same time of day in the future (or past), which the movie might imply by only having a date on the time circuit’s.

But the earth is also moving, fast, about 108,000 km/h through space, orbiting the sun. This means that the earths position in space tomorrow at this same time of day will 2,592,000 km away from where it is now. So only if the orbit of the earth around the sun is perfectly consistent, the best you could do is travel forward by years, so that you appear on the earth as it passes by at the perfect moment and is at the same point in its daily rotation as to pit you at the right spot on the surface.

But there’s more… Just as the earth orbits the sun in our solar system, our solar system orbits in the Milky Way Galaxy. This galactic year, as it’s called, takes anyway from 200 to 250 million earth years. That’s a pretty big margin of error, and even if you got it perfect, you would only be able to travel forward or back in steps of galactic years. That might explain why so many cartoons show people jumping back into the past and landing face to face with a T-Rex.

Now without doing a bunch of research, I would assume based on the various theories of universal expansion and contraction that our galaxy is also moving, either towards, away from, or around some other central point. So this again would mean that the odds of you moving to another moment in time where the train tracks are in the exact same point in the universe are slim to none.

So either ol’ Doc Brown was not only able to move through time but also calculate and reposition the delorean to another point in the universe based on “approximate” measurements at the high end, all while also ensuring that their forward movement was maintained relative to the surface of the earth as it spun and moved through space as though in a huge gyroscope, or… the earth is the center point for all of space time, and so all changes in time would be relative to the earth as a fixed point in space… Talk about thinking you’re the center of the universe.

All that said, it is a great movie.

Packaging and Deployment – ‘missing types’ compiler errors using Visual Studio 2012 and SharePoint 2010 tools

For a while now, the team and I have been dealing with a odd behavior in Visual Studio 2012 and SharePoint 2010, disruptive to the point that I actually scripted out the whole build and packaging process from the command line just to package up our WSPs (SharePoint packages).  Basically it happened like this… we has a solution with a combination of Class Library projects and SharePoint projects, and were able to compile the projects individually or as a solution in Visual Studio, but when trying to package or deploy them to SharePoint via Visual Studio we would receive a variety of errors about missing types.  These missing types were all part of the class projects that were referenced by the SharePoint projects, providing a variety of utility classes and such, which all compiled fine when simply doing a build, but were reported as missing when we attempted to create SharePoint packages.

Well with no useful information from Visual Studio, the issue just sat on the side lines until either time or frustration found me.  The fact that we could compile from the command line using MSBUILD only added to the confusion, but gave us an opportunity to ignore the issue as long as possible.  But alas, all good things must come to an end, and I found myself in a situation where I needed to actually deploy from within Visual Studio, and so I looked again at the issue.

Well with a bit of speculation, and some criticism of the SharePoint tools for Visual Studio, here’s what seemed to be going on.  The SharePoint project templates and tools are designed for moving forward, and are not friendly to changes or reworking.  This is obvious in the management of features and packages, with the unfriendly two column UI for adding and removing items.  An important thing in SharePoint packages is sequence, however only one of the many item list in the SharePoint tool UIs provides buttons for reordering and unfortunately this is not the one.  In the case of this issue, it appears to be the package additional assemblies.  As time went by and code was refactored and moved to enhance reuse and organization, more project assemblies were added to the various packages.  However, the sequence of the assemblies isn’t easy to manipulate and can intermittently cause issues packaging issues when code a lower level project is changed, as the project additional assemblies appear to be recompiled during packaging either in a weird order or with weird references.

Without spending an inordinate amount of time looking into exactly why, what seemed to help mitigate the issue, was to open the package files in notepad and reorder the additional assemblies such that they would be added in the same order that they would naturally be compiled.  This seemed to clear up the issue, allowing to the SharePoint package projects to be published and deployed again.

So the lesson here would seem to be, make sure you monitor and reorder your package additional assemblies…

Feature contents is another one to watch for order, but that’s a different story.

XSLTListViewerWebPart with Custom List view

Locate the Views node in the list definition schema.xml file, add a new view entry, giving it a new unique baseviewid (using an already in use baseviewid will inherit the others settings without allowing you to customize anything set it the previous views using that id*); I generally start with 101 and increment from there. Set all the view settings you want, such as sorting, filters, columns, etc, and save it.

In the elements.xml file of the module containing the web part page you want to add the custom view to, add a new view node (or use an existing one) and set the BaseViewID of the view to the ID you used in the schema.xml file.

Now if you want to do this when you don’t have a list definition (schema.xml) to edit it is not supported out of the box completely in SharePoint templates. I will discuss further shortly, as we had a specific set of customizations to make and most solutions where maintenance heavy (hard-coded or implementation specific), where I instead implemented a generic method to reuse for any view.

SharePoint – An unexpected error has occurred or Object Reference Error in a site collection after reattaching a Content Database

After removing a content database from a SharePoint 2010 web application, so that I can restore a backup (for development purposes), upon reattaching the content database I was being plagued with an assortment of error message.

These included when trying to load the site, seeing both an “An unexpected error has occurred” and a simple server information block returned, and upon checking the logs seeing:

“System.NullReferenceException: Object reference not set to an instance of an object.    at Microsoft.SharePoint.SPSite.PreinitializeServer(SPRequest request)”

This would occur on a single web front end server in the development farm (yep, I have multiple WFEs in my development environment), while the other WFEs would load fine.

After a couple different investigations, I found out that one cause of single server site corruption after reattaching a SP content db is… it’s the PowerShell SharePoint CmdLets.

If you are running PowerShell and are using the SharePoint CmdLets (SPSite for example) to connect to that site (for the content database in question), if you try to reattach the content database while that PowerShell session is still open, the reattach goes fine, but the servers where PowerShell was open will fail load the site thereafter… even after reboots and all.

You need to close the PowerShell window and then reattach the content database (either by the Central Admin UI or from a new PowerShell window if you are using scripts).

Additionally, the W3SVC and SPADMIN and SPTIMER services can all cause the same problem, so I have found that restarting those also (I bundled it up into a single PowerShell cmdlet) ensures that I never saw this issue again.

Here, There, and Anywhere (part 1)

Technology is an amazing thing… throughout the years it has made our lives “better”.  I double quote the “better” because it is a very personal thing, where better might mean more free time, more luxury, more information, and in my case better means more open sky.

This year my wife and I decided to turn our plans for the future into our plans for the day.  We sold our house in Calgary, a city of over a million people, and headed to the open prairies of Saskatchewan.  While in a very small town right now, our final destination will be 6 acres of country side living about 10 minute drive from Yorkton, Saskatchewan.  As we meticulously plan out our new acreage home to be built next year, I think about how our ‘retirement dreams’ is a lifestyle we can enjoy in our younger years and provide to our children, all thanks to Technology.

A few years ago I decided to leave the full-time employment world and get back into contracting, seeing how much of that I could do working from home… well it can be a bit too quiet.  Although you spend a lot of time with the headphones on while writing software, there is a huge aspect of the work that is collaborative and social.  When I started a contract with NeoStream Technologies in Calgary, I was back to working out of an office (planned to be part time, but became full time).  Well when it comes to fun places to work, NeoStream certainly ranks at the top, and I say fun in a broad sense of what I consider fun… great people, bleeding edge tech, intriguing challenges, and a very social approach to business.

So with the first contract renewal and the approaching sale of our home, the realization that I would miss the in person interaction that working in an office affords.  Enter ‘Microsoft Lync’… the communication system NeoStream had recently incorporated for phone and messaging… add a few web cams, and viola, ‘Virtual Baydon’ was born.  Eight hours a day, five days a week, with two way video and voice, I can see and be seen by all the team members as if I was still sitting at my desk.  A camera was added to the product room, so conversations during meetings are engaging and interactive, and with desktop sharing of our onscreen planning board, everyone and everything is visible to all.

A few things were still missing, mainly due to the locations of certain events such as the morning huddle (an all staff standup meeting where we hear daily updates and announce our top tasks for the day), and our product sprint meetings.  These are in areas not covered by a camera, so recently NeoStream added a large screen TV and camera to the open area for morning huddles, allowing anyone to join in, including people working offsite or from home.  We even recently started having people working in other parts of the office join into the ‘Virtual Baydon’ session, so that they could contribute to conversations going on in the bull pen while continuing their work at their own desks.

I will delve into more details next time on both the work and home sides of things, but for now I must say, I didn’t expect working remotely to go so well.  Apart from not being able to go for coffee, not much is different… but I am definitely enjoying not having the commute.

Disable Password Prompts In Microsoft Word and Excel Automation

It’s not an easy task to automate MS Office, and especially to run it as a service or within a ASP.Net web site, but it is possible and with a little bit of hard work and research you can actually have a very stable interaction.  This is the first of my many "things to note" and "gotcha’s" I found when dealing with MS Office Automation, and it’s a long list.

Disabling Password Prompts in MS Word and Excel Automation, seemed to be a hot topic in a lot of forums with few insights to an answer, but here is the trick that I figured out.

First you have to accept that fact that you likely aren’t going to be able to open the password protected file, and that Word or Excel is going to fail the open request, so you have a COM Exception to catch.  The nice thing is that the Error Code returned from the exception is unique for password issue so you can catch it specifically and deal with it (either tell the user or simply ignore it, whatever you want).  Again I will post a sample with the codes soon, but if you try to open a password protected file and hit cancel you will see the exception returned so you can get the code from there.

Now onto the second and most important point, the brilliant engineers at Microsoft figured that if a document isn’t password protected then there is no point in even trying to use any password supplied.  So basically… enter a garbage password into the various password fields (something around 8 chars seems fine; too many though and it causes another exception all the time).  Both word and Excel behave the same way, only using the password when the document is protected, and so normal documents will open fine since they won’t use the password, and documents that require a password will use the garbage one you provided instead of prompting, and will simply fail with ‘Invalid password’ which you can catch and handle.

   1:  const uint COM_CODE_BADPASSWORD = 0x800A1520;

   2:  const string OPEN_OPTION_MOCK_PASSWORD = "GARBAGE1";

   3:   

   4:  object missingValue = Type.Missing;

   5:  object fileNameObject = fileName;

   6:  object confirmConversions = false;

   7:  object openAsReadOnly = true;

   8:  object addToRecentFiles = false;

   9:  object passwordDocumentObject = OPEN_OPTION_MOCK_PASSWORD;

  10:  object passwordTemplateObject = OPEN_OPTION_MOCK_PASSWORD;

  11:  object revertObject = missingValue;

  12:  object writePasswordDocumentObject = OPEN_OPTION_MOCK_PASSWORD;

  13:  object writePasswordTemplateObject = OPEN_OPTION_MOCK_PASSWORD;

  14:  object formatObject = missingValue;

  15:   

  16:  object doNotSaveChanges = WdSaveOptions.wdDoNotSaveChanges;

  17:   

  18:  try

  19:  {

  20:      // Using OpenOld is documented to be more compatible with older versions of Word.

  21:      // Word only tries the password if it needs it.

  22:      Document document = wordApplication.Documents.OpenOld(

  23:          ref fileNameObject,

  24:          ref confirmConversions,

  25:          ref openAsReadOnly,

  26:          ref addToRecentFiles,

  27:          ref passwordDocumentObject,

  28:          ref passwordTemplateObject,

  29:          ref revertObject,

  30:          ref writePasswordDocumentObject,

  31:          ref writePasswordTemplateObject,

  32:          ref formatObject

  33:          );

  34:   

  35:      // Do stuff with the document.

  36:   

  37:      document.Close(ref doNotSaveChanges, ref missingValue, ref missingValue);

  38:   

  39:  }

  40:  catch (COMException ex)

  41:  {

  42:      if (((uint) comException.ErrorCode) == COM_CODE_BADPASSWORD)

  43:      {

  44:          Debug.WriteLine("Detected a password protected Word document.");

  45:      }

  46:      else

  47:      {

  48:          throw;

  49:      }

  50:  }

You may ask, that’s fine for Word and Excel, but what about the other office apps… well unfortunately I haven’t tried Publisher or Access since I didn’t have need at the time, but I did learn that PowerPoint doesn’t expose any way to enter a password for opening the presentation, so the old Dialog Killer thread was the only resolution we had, looking for visible dialog windows with a particular name and sending them a WM_CLOSE message (I say ‘visible’ dialog since we were also dealing with Acrobat which keeps invisible dialog boxes with the particular name we were monitoring, and visible only seems to work 100% of the time).

My next post will be on using the Automatic Recovery options to open documents and some very frustrating issues I had with those.

Using Find and Replace in Visual Studio .Net with Regular Expressions to Bulk Change Properties and Methods (or anything else)

This is a really quick sample of using Regular Expression in Visual Studio .Net to perform and intelligent Find and Replace.  Documented mainly so I don?t forget it.

The need for this arose when I was extracting properties from a class, so that the data of the class was separate from the functionality.

Unlike normal regular expressions, Visual Studio doesn?t let you reference groups derived in the expression, instead a different syntax is used as {?} to tag parts of the string, and then in your replacement expression you refer to those tagged parts by \n (0 being everything, and then 1, 2, 3, etc for the individual tagged parts).

Find

public {[A-Za-z\<\>\[\]]*} {[A-Za-z]*} \{ get\; private set\; \}

Replace With

public \1 \2 \{ get \{ return someClass\.\2\; \} private set \{ someClass.\2 \= value\; \} \}

Note* Be sure to check ?Use: Regular expressions? in the Find and Replace dialog box.

For more details see: http://msdn.microsoft.com/en-us/library/2k3te2cs(VS.80).aspx