The http://trac.sitecore.net/LinkProvider/ Sitecore Shared Source project provides many of the features described in this post.

I spent February 18 through 20 with a customer experiencing performance issues with a Sitecore 5.3.1 solution. We identified a configuration problem within the first few hours, but I stayed on the project for a little while to monitor the solution.

As most Sitecore developers are aware, Sitecore is thoroughly optimized for performance, relying heavily on caching. Caching consumes memory, and caching requirements for a solution with significant data or load can exceed available memory. Out of memory conditions can cause exceptions and can cause ASP.NET to crash or restart, resulting in errors, delays, and outages.

Sitecore provides a memory monitor hook to flush caches when memory exceeds a given threshold. You configure the memory monitor hook using the /configuration/sitecore/hooks section of web.config:

<hooks>
  <hook type="Sitecore.Diagnostics.HealthMonitorHook, Sitecore.Kernel" />
  <hook type="Sitecore.Diagnostics.MemoryMonitorHook, Sitecore.Kernel">
    <param desc="Threshold">500MB</param>
    <param desc="Check interval">00:00:05</param>
    <param desc="Minimum time between log entries">00:01:00</param>
    <ClearCaches>true</ClearCaches>
    <GarbageCollect>true</GarbageCollect>
    <AdjustLoadFactor>true</AdjustLoadFactor>
  </hook>
</hooks>

The first parameter (the first <param> element) to the constructor for the Sitecore.Diagnostics.MemoryMonitorHook class specifies the threshold for the memory monitor hook, which is 500MB by default in Sitecore 5.3.1. Sitecore frequently checks the amount of memory consumed by ASP.NET and if it exceeds this limit, then Sitecore will flush caches. The following message repeated frequently in the logs should have made this pretty obvious:

WARN  Memory usage exceeded the MemoryMonitor threshold. All caches have been cleared and a forced GC has been induced.
WARN  Memory used before/after GC: 554,590,208 / 554,590,208

If you see a message like this, your cache may be too small. If you see it constantly, your cache may be way too small, especially if the before/after values are the same.

The problem for this customer was that the amount of memory consumed by ASP.NET was always greater than 500MB. Therefore, Sitecore was flushing caches constantly. It's somewhat amazing that this didn't completely overwhelm the content delivery and the database servers.

The solution is quite simple: either set the memory monitor threshold to an appropriate value, or set the ClearCaches property to false. In this case, we set the size of the cache to 1792MB and afterwards were unable to reproduce the issue.

Note that in Sitecore 6, the default value for the memory monitor threshold is higher (800MB), but it shouldn't matter anyway because cache clearing is disabled by default:

<hooks>
  <hook type="Sitecore.Diagnostics.HealthMonitorHook, Sitecore.Kernel" />
  <hook type="Sitecore.Diagnostics.MemoryMonitorHook, Sitecore.Kernel">
    <param desc="Threshold">800MB</param>
    <param desc="Check interval">00:00:05</param>
    <param desc="Minimum time between log entries">00:01:00</param>
    <ClearCaches>false</ClearCaches>
    <GarbageCollect>false</GarbageCollect>
    <AdjustLoadFactor>false</AdjustLoadFactor>
  </hook>
</hooks>

I would like to thank Alexey Romaniuha of the Sitecore Support team for his identification of the issue.

The Sitecore browser-based debugger by default opens the home page of the published Web site using the publishing target database (Web). If you want to debug another page, you can navigate to that page, or enter its URL. If you want to debug in another database, you can add the sc_database query string parameter. Or you can add a command to the Content Editor ribbon to debug the selected item in the selected database:

1. Compile the class into your project.
2. Update the type attribute of the /configuration/command element in /App_Config/commands.config with name system:debug to reference this class and the assembly that contains it. Remember to hide all files again after adding this file to the Visual Studio project, or debugging in Visual Studio will hang.
3. In the Sitecore Desktop, use the database selection icon to select the Core database.
4. In the Core database, open the Content Editor and duplicate the /sitecore/content/Applications/Content Editor/Ribbons/Chunks/Publish/Preview item as an item named Debug (or if you prefer the Debug command on the Reviewing tab instead of the Publish tab, duplicate /sitecore/content/Applications/Content Editor/Ribbons/Chunks/Proofing/Spellcheck as Debug).
5. In the Data section of the new Debug item, set Header to Debug, Icon to Software/16x16/debug_run.png, Click to system:debug, and Tooltip to Start the debugger.
6. In the Sitecore Desktop, be sure to use the database selection icon to select the Master database.
7. Open the Content Editor, navigate to any item with layout details, click the Publish tab, and then in the Publish group, click the Debug command (or on the Review tab in the Proofing group).

After step 2, the default Debug command on the Sitecore menu will take you to the Master database instead of the publishing target database (Web).

You can download the code from http://resources.thedotnetcms.com/sitecorejohn/debug.zip.