PSConfig failed with attempt to register null pointer at …… SharePoint 2019

PSConfix.exe failed in SharePoint 2019 today. Below are some entries from the ULS logs.

Error messages in console:
Failed to upgrade SharePoint Products
An exception of type Microsoft.SharePoint.Upgrade.SPUpgradeException was thrown.
Upgrade logs ending with “–error “show :
 07/03/2019 11:18:33.76    PSCONFIG (0x982C)    0x3CAC    SharePoint Foundation                                                      Upgrade    SPHierarchyManager    ajyw5    INFO    SPWebApplication    Name=XYZEXT    00000000-0000-0000-0000-000000000000
07/03/2019 11:18:33.76    PSCONFIG (0x982C)    0x3CAC    SharePoint Foundation Upgrade    SPHierarchyManager    ajyw5    ERROR    Attempt to register null pointer at: at Microsoft.SharePoint.Upgrade.SPHierarchyManager.AddNextLevelObjects(Object current, IEnumerable nextObjects) at Microsoft.SharePoint.Upgrade.SPHierarchyManager.Grow(SPTree`1 root, Boolean bRecursing, SPDelegateManager delegateManager)
SPUpgradeSession    an59w    DEBUG    Begin processing children of [SPWebApplication Name=XYZEXT] for upgrade.
SPWebApplicationSequence    ajy60    DEBUG    Web application [SPWebApplication Name=XYZEXT] has 1 IisSettings object
SPWebApplicationSequence    ajy61    DEBUG    Searching for sites with Server Comment = XYZEXT
SPWebApplicationSequence    ajy66    DEBUG    Adding a iis web site object with instance Id = [815745826]
SPHierarchyManager    ajyw5    ERROR    Attempt to register null pointer at: at    Microsoft.SharePoint.Upgrade.SPHierarchyManager.AddNextLevelObjects(Object current, IEnumerable   nextObjects) at Microsoft.SharePoint.Upgrade.SPHierarchyManager.Grow(SPTree`1 root, Boolean bRecursing, SPDelegateManager

Logs show it failed at  “Begin processing children of [SPWebApplication Name=XYZEXT]“, most recent change we made last week was addition of few new content databases in to this web application. So started looking at the new content databases and noticed that “Manage Content Databases” in CA shows total of 25 databases
Another check was with

$WA=Get-SPWebApplication XYZEXT
$WA.ContentDatabases.Count
26

Notice it shows 26 content databases in the Config database listed as content databases for this web application.
That was something new but now logs make little more sense at “Attempt to register null pointer at: at Microsoft.SharePoint.Upgrade.SPHierarchyManager.AddNextLevelObjects”
Since the results is coming from the Config database lets grab the ID of our web application and Log in to SQL and explore Config database.

$WA=Get-SPWebApplication XYZEXT
$WA.Id
Guid
----
255ea172-62b5-4196-a335-c92ae4088b9b

In Config database I use the SQL query to see the information about the web application we grab the ID above.

SELECT * FROM [SP2019_Config].[dbo].[Objects]
Where ID like '255ea172-62b5-4196-a335-c92ae4088b9b'

Output:

We need to look in to the properties of this object you can right click, copy and paste in note pad or use the following SQL query and click on the output  to see the properties.

SELECT cast (Properties as xml ) FROM [SP2019_Config].[dbo].[Objects]
Where ID like '255ea172-62b5-4196-a335-c92ae4088b9b'

Output:

Once click you will see all the databases associated with this web application and the content database with <fldtype=“null“ />. This seems like bad database refernce and causing our PSConfig to fail. You can check with ID no such database exist. 

Grab that ID from here and let’s go back to the SharePoint PS. Delete this fake content database using the ID and run the PSConfig again.

To remove the bad content db ID:

 Remove-SPContentDatabase 9a15bfd6-e5bd-4340-abec-e06e86fa60fe

Command above threw an error message, so  I used reference of web application and same behavior but when check the count of databases associated with this web application it was 25 (instead of 26), which is correct number based on our initial test. I beleive it was deleted with the first command but you can use any of them.

Remove-SPContentDatabase 9a15bfd6-e5bd-4340-abec-e06e86fa60fe
$WA=Get-SPWebApplication XYZEXT
$WA.ContentDatabases.Delete("9a15bfd6-e5bd-4340-abec-e06e86fa60fe") 
$WA.ContentDatabases.Count

Go back to SQL and run the same query again you will see the content database with <fldtype=“null“ /> is gone now.

Hope this will help someone.

DelegateControl: Exception thrown while adding control. The file name you specified could not be used. SharePoint 403 forbidden.

Came across a weird issue today some of the users were getting 403 forbidden when access SharePoint sites. Find out one of the WFEs was acting up and showing 403 all the times when user’s request hit that bad server.

Did IIS reset, reboot, added app pool account in “local security policies”  at “Impersonate a client after authentication” but none of them worked.

I find lots of access denied entries in ULS logs but no account was mentioned. Below are some entries from the ULS.

Failed to open the file ‘C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\Template\Features\Publishing\Pages\schema.xml’.
Getting Error Message for Exception Microsoft.SharePoint.SPException: Invalid file name.  The file name you specified could not be used.  It may be the name of an existing file or directory, or you may not have permission to access the file. —>
System.Runtime.InteropServices.COMException: Invalid file name.  The file name you specified could not be used.  It may be the name of an existing file or directory, or you may not have permission to access the file
An unexpected error has been encountered in this Web Part.  Type: Microsoft.SharePoint.WebPartPages.XsltListViewWebPart, Error: Invalid file name.  The file name you specified could not be used.  It may be the name of an existing file or directory, or you may not have permission to access the file.
Getting Error Message for Exception System.Web.HttpUnhandledException (0x80004005): Exception of type ‘System.Web.HttpUnhandledException’ was thrown. —> Microsoft.SharePoint.SPException: Invalid file name
DelegateControl: Exception thrown while adding control ‘ASP._controltemplates_15_metadatanavtree_ascx’: Microsoft.SharePoint.SPException: Invalid file name.  The file name you specified could not be used.  It may be the name of an existing file or directory, or you may not have permission to access the file.

Resolution:
After reviewing log files I started looking at the permissions at the folder level and notice that two accounts have no permissions at all on any folder in 15hive. “Network Service” & “ServerName\Users”
Added these 2 accounts at the 15hive folder, gave read and execute permissions and everything started working, no more forbidden 403.

Still not sure how those 2 accounts got removed but during my search for this issue I could find this solution anywhere. Hope this will help some running into the same issue.

 

Inventory : Get the size of each document in SharePoint.

The script below can be used to get the size of each document in a web application but in our case, we had thousands of site collections in a single web application so I wrote this script to read from a csv file to get the size of each document from a selected set of the site collection.

Script also provides some other useful information like created by and created date. You can also set the limit on size for the reporting purpose.

List document size per site collection from csv file. 

  <# Function will read the sites from csv file and generate a list of the document larger then the size (MB) mentioned #>

   Add-PSSnapin Microsoft.SharePoint.Powershell
   Function Get-DocSize()
   {
   $URLs=Get-Content E:\scripts\GetAttachmentSize\Sites.csv #<-csv file path 
       foreach($URL in $URLs)
       {
          $site = New-Object Microsoft.SharePoint.SPSite $Url
              foreach($web in $site.AllWebs)
              {
                  foreach($list in $web.Lists) 
                  {
                      if($list.BaseType -eq "DocumentLibrary") 
                      {
                          foreach($item in $list.Items) 
                          {
                              if($item.file.Length/1MB -gt 72) #<-- Change size here 
                              {
                                  $data = @{
                                      "Web Application" = $site.WebApplication
                                      "Site" = $site.Url
                                      "Web" = $web.Url
                                      "list" = $list.Title
                                      "Item Title" = $item.Title
                                      "Item Name"  =$item.Name
                                      "File Size" = $item.File.Length/1MB
                                       }
                                New-Object PSObject -Property $data
                             }}
               }}  }
                    #$web.Dispose();
                   # $site.Dispose()
                }}
                Get-DocSize | Out-GridView

List document size from all site collections in a web application.  

      Add-PSSnapin Microsoft.SharePoint.Powershell
      Function Get-DocSize($URL)
      {
          $Webapp=Get-SPWebApplication $URL
          foreach($Site in $Webapp.Sites)
          { 
             foreach($web in $site.AllWebs)
             {
                foreach($list in $web.Lists) 
                {
                     if($list.BaseType -eq "DocumentLibrary") 
                     {
                         foreach($item in $list.Items) 
                         {
                             if($item.file.Length/1MB -gt 20) #<-Set the size here
                             {
                                 $data = @{
                                     "Web Application" = $site.WebApplication
                                     "Site" = $site.Url
                                     "Web" = $web.Url
                                     "list" = $list.Title
                                     "Item Title" = $item.Title
                                     "Item Name"  =$item.Name
                                     "File Size" = $item.File.Length/1MB
                                     }
                             New-Object PSObject -Property $data
                            }}
                 }}  }
           $web.Dispose()
           $site.Dispose()
          }}
     Get-DocSize "http://sandbox" | Out-GridView

Hope this will help someone looking for the same…

The workflow was canceled by System Account – Out of the box workflows stopped working in SharePoint.

The workflow was canceled by System Account – Out of the box workflows stopped working in SharePoint.

Issue description:
After installing September.Net Security patches, all workflows in SharePoint stopped working and when click on status of the workflow to see the details it shows

Workflow was canceled by System Account.

Error message in ULS logs. 
10/05/2018 15:27:29.96 w3wp.exe (0x4E6C) 0x7C2C SharePoint Foundation Legacy Workflow Infrastructure 72fs Unexpected RunWorkflow: Microsoft.SharePoint.SPException: <Error><CompilerError Line=”-1″ Column=”-1″ Text=”Type System.CodeDom.CodeBinaryOperatorExpression is not marked as authorized in the application configuration file.” /><CompilerError Line=”-1″ Column=”-1″ Text=”Type System.CodeDom.CodeBinaryOperatorExpression is not marked as authorized in the application configuration file.” /><CompilerError Line=”0″ Column=”0″ Text=”Activity ‘ID5’ validation failed: Property &quot;Condition&quot; has invalid value. Condition expression is invalid. The condition expression can not be null.” /><CompilerError Line=”0″ Column=”0″ Text=”Activity ‘ID128’ validation failed: Property &quot;Condition&quot; has invalid value. Condition expression is invalid. The condition expression can not be null.” /></Error> at Microsoft.SharePoint.Workflow.SPNoCodeXomlCompiler.LoadXomlAssembly(SPWorkflowAssociation association, SPWeb web)

Resolution:
As mentioned in the error message “System.CodeDom.CodeBinaryOperatorExpression” is not marked as authorized in the application configuration file”, to resolve this issue we need to add those assemblies’ types as authorizedType in each web.config.

To resolve this issue, add the references below into web.config for each web application where workflows are broken just before the ending tag for “</targetFx>” in </authorizedTypes>.

 

For SharePoint 2013, 2016 & 2019:

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeBinaryOperatorExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodePrimitiveExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeMethodInvokeExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeMethodReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeFieldReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeThisReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodePropertyReferenceExpression” Authorized=”True” />

For SharePoint 2010,

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeBinaryOperatorExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodePrimitiveExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeMethodInvokeExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeMethodReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeFieldReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodeThisReferenceExpression” Authorized=”True” />

<authorizedType Assembly=”System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089″ NameSpace=”System.CodeDom” TypeName=”CodePropertyReferenceExpression” Authorized=”True” />

For more details: https://blogs.msdn.microsoft.com/rodneyviana/2018/09/13/after-installing-net-security-patches-to-address-cve-2018-8421-sharepoint-workflows-stop-working/