SharePoint 2010: an unexpected error has occurred

Problem

Helpdesk notified that the entire SharePoint Server 2010 farm was inaccessible: connecting to the farm resulted in the following error being displayed in the browser:

Troubleshooting

1. Checked farm servers:
• all up and running.
2. Remoted into the application server and launched Central Administration:
• connecting to this site resulted in same error.
3. Remoted into the database server and launched SQL Server Management Studio:
• SQL Server service running.
4. On a web front end server, launched Server Manager and checked the Application log:
• Group of 8306 errors occuring at 2:10 AM: An exception occurred when trying to issue security token: This implementation is not part of the Windows Platform FIPS validated cryptographic algorithms..
• A large volume of 5586, 6398, 8088, 5240, 6772 and other errors began appearing at approximately 11:20 PM.
5. Checked development farms:
• same error appearing on connection; same errors appearing in Application logs.
6. Checked FIPS-related security settings:
1. Computer Configuration > Windows Settings > Security Settings > Local Policies > Security Options > System cryptography: Use FIPS compliant algorithms for encryption... this was set to Enabled.

Solution

1. The setting change was pushed out at the corporate NOC level (above us), thus, we could not change it directly.
2. Instead, department sysadmin added a new instance of the FIPS policy setting to the AD container but configured it at a higher priority.  Thus, after the corporate policy set the FIPS setting to Enabled, the department policy set the FIPS setting back to DISABLED.
3. Forced the update to the servers in this container.
4. Reset IIS on farm servers.
5. Check security settings: security setting now DISABLED.

References

• System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing
• "System cryptography: Use FIPS compliant algorithms for encryption, hashing, and signing" security setting effects in Windows XP and in later versions of Windows
• Disabling the FIPS Algorithm Check
• FIPS and SharePoint
• DISABLE FIPS ALGORITHMS BEFORE INSTALLING SHAREPOINT 2010
• Open the Local Group Policy Editor