Solving Least Privilege Problems with the Application Compatibility Toolkit

Exclusive offer: get 50% off this eBook here
Least Privilege Security for Windows 7, Vista and XP

Least Privilege Security for Windows 7, Vista and XP — Save 50%

Secure Microsoft Windows desktops with least privilege security for regulatory compliance and business agility with this security book and eBook for Windows 7, Vista and XP

$35.99    $18.00
by Russell Smith | July 2010 | Enterprise Articles Microsoft

The launch of Windows XP in 2001 heralded the long-awaited transition of consumer-orientated editions of Windows to the NT codebase. Microsoft designed the Windows Application Compatibility Infrastructure as part of Windows XP to help system administrators and home users solve compatibility problems with applications that were designed to run in Windows 98 or earlier versions of the 9x range.

In this article by Russell Smith, author of Least Privilege Security for Windows 7, Vista and XP, we will learn:

  • How the Application Compatibility Infrastructure uses shims to modify incompatible applications on the fly
  • Why using shims provides the best balance between compatibility and security
  • How to create shims using Application Compatibility Toolkit 5.5
  • Distributing compatibility databases to devices across the enterprise

(For more resources on Microsoft products, see here.)

Quick compatibility fixes using the Program Compatibility Wizard

In small enterprise environments or on home computers, compatibility fixes can be applied without using the Application Compatibility Toolkit through the user interface. If you want to know how the Windows Application Compatibility Infrastructure works in detail or if you want to deploy fixes to multiple devices, skip straight to the second part of this article: Achieving application compatibility in enterprise environments.

Applying compatibility modes to legacy applications

Compatibility fixes can be grouped together to form compatibility modes. Windows XP, Vista, and Windows 7 all come with a default set of compatibility modes out of the box. System administrators can also use Compatibility Administrator to create their own compatibility modes.

Compatibility modes can be applied directly from the user interface either by using the Program Compatibility Wizard in Windows XP, or by right-clicking on an executable file, or a shortcut to an executable file, and selecting Properties from the menu.

Windows XP contains the following compatibility modes:

  • Windows 95
  • Windows 98 / Windows ME
  • Windows NT 4.0 (Service Pack 5)
  • Windows 2000

As shown in the following screenshot, the Compatibility tab in Vista and Windows 7(left) looks a little different from Windows XP (right).

Privilege level in Vista and Windows 7
Note that the Run this program as an administrator option will be grayed out if an application manifest is included with the program.

Program Compatibility Wizard

Included in Windows Vista and later, the Program Compatibility Wizard allows users or system administrators to test legacy applications running against different compatibility modes.

  1. To launch the Program Compatibility Wizard in Windows XP, select Start | Accessories | Program Compatibility Wizard.

    To launch the Program Compatibility Wizard in Windows Vista, type the following command into the Search box on the Start menu:
    %systemroot%\System32\mshta.exe res://acprgwiz.dll/compatmode.hta
    To launch the Program Compatibility Wizard in Windows 7, search for program compatibility troubleshooter in Start | Help and Support.

  2. Click Next on the welcome screen in Help and Support Center.
  3. Leave the default I want to choose from a list of programs option selected and click on Next.
  4. Select the application you want to test, in this case Maxthon, from the Select a program list and click on Next.

  5. Choose a compatibility mode from the list, such as Microsoft Windows 95 or Windows 2000, and click on Next.

    Compatibility modes set using the Program Compatibility Wizard only apply to the currently logged in user. Compatibility settings in Vista and Windows 7 can be applied to the logged in user or to all users.Administrative privileges are required to apply compatibility settings for a given application to all users of a system.

  6. If required, select display settings (256 colors or 640 x 480 screen resolution) to be used with the program and then click on Next.
  7. Click Next to test the compatibility mode against your application.

The application will launch and you can test it to see if the selected compatibility mode resolves the identified problems. Once you've finished testing, close the application. Back in the Help and Support Center, choose to either apply the compatibility settings or try different settings.

Browsing compatibility settings for each user
If compatibility modes are set on programs for specific users, you can use Compatibility Administrator to browse the settings for each user under the Per User Compatibility Settings node. This node only appears if per user settings are found on the system.

Program Compatibility Assistant

Introduced in Windows Vista, the Program Compatibility Assistant helps to automate the process of applying compatibility fixes to legacy applications by monitoring for known problems when programs are running. This feature runs as a service, and prompts users to apply suggested fixes for applications, either during the setup phase or when running an installed application. The Program Compatibility Assistant can detect the following problems automatically:

  • Errors when launching setup programs
  • Failures in install routines
  • Failures caused by User Account Control
  • An install needing to run as an administrator
  • A control panel applet requiring administrative privileges
  • Errors caused because a component is not present in the current version of Windows
  • Notifying users about unsigned drivers on 64-bit versions of Windows
  • Matching applications against a list of programs with known problems and notifying the user at program startup

The Program Compatibility Assistant intercepts an installation routine, prompting the user to try again using recommended settings.

Disabling the Program Compatibility Assistant

The Program Compatibility Assistant is intended to help home users resolve problems with legacy applications. In an enterprise environment, to avoid potential problems with messages generated by the Program Compatibility Assistant, you should consider disabling the service in Group Policy. The Turn off Program Compatibility Assistant setting can be found in the Group Policy Management Editor under Computer or User Configuration | Policies | Administrative Templates | Windows Components | Application Compatibility.

Excluding executables from the Program Compatibility Assistant

The Program Compatibility Assistant doesn't monitor programs that have an application manifest that marks them as compatible with Vista or Windows 7. However, if you want to exclude a program but don't want to disable the Program Compatibility Assistant completely, you can create a REG_MULTI_SZ registry value named ExecutablesToExclude under the following key:
HKLM\ Software\Microsoft\Windows NT\CurrentVersion\Compatibility Assistant

Least Privilege Security for Windows 7, Vista and XP Secure Microsoft Windows desktops with least privilege security for regulatory compliance and business agility with this security book and eBook for Windows 7, Vista and XP
Published: July 2010
eBook Price: $35.99
Book Price: $59.99
See more
Select your format and quantity:

(For more resources on Microsoft products, see here.)

Achieving application compatibility in enterprise environments

As new versions of Windows are being developed, improvements and changes mean that some applications designed for earlier editions of the operating system may experience compatibility problems in the later versions. Rather than restricting innovation and development of the OS in favor of compatibility, the Windows Application Compatibility Infrastructure was designed to allow the OS to move forward while retaining compatibility with legacy applications.

Compatibility fixes

Compatibility fixes, sometimes also known as shims, target specific legacy applications and allow them to run in current versions of the operating system. The advantage of using shims is that rather than maintaining legacy code in the operating system to ensure that all applications run without modification, which in turn makes the OS inherently more complicated and insecure, the legacy application code is modified by a shim instead of the OS.

Modifying applications using shims

Shims intercept Win32 API calls from legacy applications, as defined by system administrators, and then modify the call before passing the code to Windows for execution.

A legacy application fails to run in Windows Vista:

A legacy application successfully runs in Windows Vista:

Shims are bundled with the operating system in the system database, but are not part of Windows code itself. As such, shims are updated by Microsoft via Windows Update.

Enhancing security using compatibility shims

As far as Windows is concerned, the modified code presented to it by the shim is code from the application itself. Therefore, it's important to note that any application code modified by a shim runs in the same security context as the application. As shim code is not part of the operating system, it can't be used to bypass Windows security features.

It's common place for system administrators to either grant administrative rights to users or loosen Access Control Lists (ACLs) on files, directories, or registry keys to force legacy applications to run on newer versions of Windows. This increases the attack surface and makes it more likely that systems will fall prey to some kind of security threat. Shims on the other hand, don't require users to hold any additional privileges or modification of security principals. Therefore, shims are inherently more secure than workarounds such as relaxing ACLs.

One example of improved security when using shims is if an application needs to write to a protected area of the file system. Rather than loosening the ACLs on the given directory to ensure that write operations made by the application succeed, a shim can be used to redirect the request to a virtualized copy of the protected directory.

Deciding whether to use a shim to solve a compatibility problem

From a security standpoint, a shim is always preferable to loosening ACLs on security principals for solving compatibility problems. While failed I/O operations made by applications are the most common reason why legacy programs fail in Windows XP, shims provide a solution for many additional compatibility problems that loosening ACLs alone cannot solve.

Vendor support

Bear in mind, when considering whether to deploy shims in your organization, that because the application's behavior will essentially be modified, you may invalidate any support agreement with the vendor. If vendor support is absolutely critical for your line-of-business application, you should check in advance whether support will be provided if a shim is used to solve a compatibility problem. Better still, ask the vendor if they will modify the application to support the current version of Windows.

In-house applications

In the same way that shims are preferable to loosening ACLs, modifying the original application code is better than using a shim, as it removes an extra layer of complexity. Applications developed in-house should be modified by the developers and then redeployed. Shims can be used as a temporary solution if compatibility problems with line-of-business applications are likely to block an upgrade project for a significant time period.

Kernel-mode applications

Shims cannot be used to solve compatibility problems with drivers and other software that hooks deep into the operating system, such as antivirus programs. Shims run in user-mode, so drivers and kernel-mode software must be modified at source if there are compatibility problems with the latest version of Windows.

Creating shims for your legacy applications

Custom databases are created using Compatibility Administrator, part of the Application Compatibility Toolkit, and contain:

  • Compatibility fixes (shims): Compatibility fixes can be applied to specific applications installed on devices across your organization.
  • Custom compatibility modes: Custom compatibility modes can be added to the default list that ships with Windows.
  • Apphelp messages: Apphelp messages inform users about compatibility problems with specific software and optionally block programs from running.

Shims can be packaged with each application as you deploy software on your network, or you can manage a central database that contains all the shims required for your applications and deploy the database to every machine.

Compatibility Modes are collections of shims used to address problems with legacy applications that were designed to run on a specific platform.

Viewing the fixes included in a compatibility mode
Follow the instructions to add a compatibility fix to a custom database using Compatibility Manager, but instead of pressing Ctrl+P to add a fix, press Ctrl+L to add a compatibility mode. At the bottom of the Create a Custom Compatibility Mode dialog box, click on Copy Mode. The fixes included in the selected compatibility mode are then displayed under Compatibility fixes for the mode on the right.

On small networks, where only a handful of legacy applications require shims for compatibility purposes, it may be feasible to package a custom database with each application. This requires a developer or system administrator who is familiar with application packaging technology, and each package must be tested thoroughly. Shims can be difficult to track if they're embedded into application installer packages. If many applications require shims, a more scalable solution is to maintain a single central custom database and update it on client machines as necessary. Your custom database can be deployed using a Group Policy startup script. Very large organizations can manage several sets of custom databases and deploy them to defined categories of users for ease of administration.

Custom databases must be deployed under the security context of a user with administrative privileges.

You could consider adding a custom database to the OS image that your organization uses to deploy Windows. Custom databases then only need to be updated using Group Policy when a change is made to the central database.

Solving compatibility problems with shims

This section describes the compatibility fixes that are most commonly used to solve application compatibility problems caused by least privilege security.

You should note that you cannot create your own compatibility fixes, only compile collections of fixes (custom databases) modified to work with specific applications in addition to the default system database of customized fixes.

LUA compatibility mode fixes in Windows XP

The LUARedirectFS and LUARedirectReg compatibility fixes provide the two main workarounds for forcing legacy applications that write to protected areas of the file system and registry to work correctly using a standard user account.

LUARedirectFS

If a legacy application tries to write to a protected system folder such as c:\windows or c:\program files, the LUARedirectFS compatibility fix can be used to redirect all writes and subsequent read operations to the user's profile: %systemDrive%\Documents and Settings\%username%\LocalAppData\Redirected\drive\filepath. When configuring this fix for a specific application, you need to launch and run the program during the configuration process so that the Compatibility Administrator can detect which files should be redirected.

LUARedirectFS_Cleanup

Intended for use when an application is removed from a computer, the LUARedirectFS_Cleanup compatibility fix can be used to remove application files that were redirected to user profiles.

LUARedirectReg

Similar to LUARedirectFS, LUARedirectReg can be used to detect when a legacy application tries to write to the protected HKEY_LOCAL_MACHINE registry hive and redirect the write and subsequent read operations to the user's HKEY_CURRENT_USER hive. The application being fixed must be launched and run during the configuration process so that Compatibility Administrator can detect which keys should be redirected.

LUARedirectReg_Cleanup

For use when an application is being uninstalled, the LUARedirectReg_Cleanup fix removes redirected registry keys from users' profiles.

LUATrackFS

The LUATrackFS compatibility fix is for monitoring which files and folders a legacy application accesses when run. This information is recorded in a file so that developers can modify the application to work with a standard user account.

Creating your own custom database

Start by installing the Application Compatibility Toolkit 5.5 (ACT). The only component of ACT required to create a custom database is the Compatibility Administrator. There's no need to install SQL, SQL Express, or AppVerifier to run the Compatibility Administrator tool.

Compatibility fixes are specific to different versions of Windows, so if you want to create a custom database with shims for Windows XP, you should create your custom database using Compatibility Administrator installed on Windows XP.

Windows XP-specific compatibility fixes are not available when Compatibility Administrator is installed on Vista or Windows 7.

Maxthon on Windows XP

A legacy version of Maxthon, a replacement shell for Internet Explorer, doesn't retain customizations made by standard users because settings are stored in files in the protected Program Files directory. Let's create a custom database with compatibility fixes configured to target Maxthon that redirects file and registry I/O operations to virtualized per-user file and registry stores.

  1. Log in to Windows XP as an administrator and select Start | All Programs | Microsoft Application Compatibility Toolkit 5.5 | Compatibility Administrator.
  2. In the left pane of Compatibility Administrator, you'll see System Database, which contains Applications (see the following information box), Compatibility Fixes, and Compatibility Modes, and a new custom database.

    The Applications folder in the System Database contains customized compatibility fixes that address known problems with commonly used programs.

  3. Under Custom Databases, select New Database and press Ctrl+R. Type Maxthon and press Enter. Now, press Ctrl+P to create a new application fix.
  4. In the Create new Application Fix dialog box, type Maxthon in the Name of the program to be fixed field.
  5. To the right of Program file location, click on Browse and locate the Maxthon executable file in the Program Files directory. Select it and click on Open.
  6. In the Create new Application Fix dialog box, click on Next.
  7. We're going to apply specific compatibility fixes to Maxthon, so under Operating System Modes select None and in the additional compatibility modes list on the right select LUA and then click on Next.
  8. As we have already selected LUA (Least privilege User Account) in the previous dialog box, the appropriate compatibility fixes, LUARedirectFS, and LUARedirectReg are already selected for us. Therefore, no configuration is required on this screen and we can click on Next to continue.

    LUARedirectFS and LUARedirectReg are specific to Windows XP. Starting in Vista, UAC File and Registry Virtualization provides automatic redirection. The CorrectFilePaths or VirtualRegistry compatibility fixes can be used for more granular control over redirection.

  9. On the Matching Information screen, we can control how the shim identifies the Maxthon executable. Let's leave the default settings and click Next.
  10. Select Yes, Customize these fixes now and then click Finish.
  11. Click Next on the Monitor the Program screen.
  12. In the Test run application dialog box, click OK to launch Maxthon.
  13. Once Maxthon has been launched, select Maxthon Options from the Options menu, check Allow only one instance of Maxthon, and click OK.

  14. Now, close Maxthon. On the Exclude File Extensions screen, leave the File extensions to be excluded field blank and click Next.
  15. The Edit the File Redirection List screen shows us all the files that Compatibility Administrator has determined to be redirected to a per-user location. Let's select all the files listed and click Next.
  16. Click Finish on the Redirection Locations screen.

    As Compatibility Administrator didn't detect any failed registry I/O operations, there was no screen showing a list of registry key redirections.

  17. Maxthon will now appear under the Applications node in the Maxthon custom database. Let's save the Maxthon database by pressing Ctrl+S. Name the database Maxthon and save it to your desktop.
  18. Right-click on the Maxthon database in Compatibility Administrator and select Install from the menu.
  19. Click OK in the Compatibility Administrator dialog box. This confirms that the database was successfully installed.
  20. To test whether the shim is being applied, we need to start Maxthon as a standard user and see if it retains our customizations. Right-click on the Maxthon shortcut on your desktop, or the maxthon.exe file in c:\program files and select Run as... from the menu.
  21. In the Run As dialog box, select The following user and then type the username and password of a local or AD user who is a standard user on the machine.

  22. Maxthon will start. Select Maxthon Options from the Options menu, check or uncheck Allow only one instance of Maxthon, and click OK.
  23. Close Maxthon and then restart it using Run As and the same limited user account.
Least Privilege Security for Windows 7, Vista and XP Secure Microsoft Windows desktops with least privilege security for regulatory compliance and business agility with this security book and eBook for Windows 7, Vista and XP
Published: July 2010
eBook Price: $35.99
Book Price: $59.99
See more
Select your format and quantity:

(For more resources on Microsoft products, see here.)

Working with other commonly used compatibility fixes

The compatibility fixes demonstrated in this section are available in all versions of Windows, starting with XP, unless otherwise stated.

ForceAdminAccess

Problem The application has a hard-coded check to see if the logged in user is a member of the local Administrators group.
Fix Apply the ForceAdminAccess compatibility fix using Compatibility Administrator.

  1. Log in to Windows as a standard user. Start Standard User Analyzer (SUA) in Developer and Tester Tools in the Application Compatibility Toolkit under All Programs in the Start menu.
  2. On the App Info tab, click on Browse and locate the CorporateApp executable file.

    AppVerifier
    Remember that AppVerfier, which comes with the Application Compatibility Toolkit 5.5, must be installed to use Standard User Analyzer.

  3. Click Launch, enter administrator credentials into the Run As dialog box, and click OK to elevate the SUA service. If SUA prompts that All existing AppVerifier logs will be deleted, do you want to continue?, click Yes.
  4. Enter administrator credentials into the Run As dialog box and click OK again to elevate the CorporateApp executable file.
  5. Run CorporateApp and make sure that the function that checks for administrative privileges runs. Now, close CorporateApp.
  6. Return to the Standard User Analyzer program and you'll see that the issue count on the Token tab has increased to 1.
  7. Click the Token tab to see detailed information about the problem. Double-click the entry on the Token tab and three extra windows will appear with details about the problem.

In the Detailed Information box, you'll see that SUA has identified that CorporateApp calls the IsUserAnAdmin API to check for administrative privileges. This information is important, because the ForceAdminAccess check fix in Windows XP only intercepts the CheckTokenInformation API. The ForceAdminAccess fix in Vista and Windows 7 can intercept the following APIs that might be used by applications to check whether the user is a member of the local Administrators group:

  • AccessCheck
  • CheckTokenMembership (CheckTokenInformation in Windows XP)
  • RegOpenKeyExA
  • GetTokenInformation
  • IsUserAnAdmin
  • NetUserGetInfo
  • SetActivePwrScheme

ForceAdminAccess
The ForceAdminAccess fix will not intercept the IsUserAnAdmin API in Windows XP. So, it can only be used to solve our problem with CorporateApp in Vista and Windows 7.

  1. Right-click on Compatibility Administrator in the Start menu under All Programs Microsoft Application Compatibility Toolkit 5.5| and then click on Run as from the menu.
  2. In the Run As dialog box, click on The following user: and enter the username and password of a local or domain administrator account.
  3. Under the Custom Databases node in Compatibility Administrator, click New Database, press Ctrl+R, and rename it as CorporateApp.
  4. Click on the CorporateApp database and press Ctrl+P to create a new fix. Under Program information, enter the application's details including the path to the main executable file.
  5. Click on Next to continue. As we know the specific compatibility fix that we want to apply, select None under Operating System Modes on the Compatibility Modes screen and then click Next.
  6. On the Compatibility Fixes screen, scroll down the list of fixes and check ForceAdminAccess.

  7. Now, click Parameters and then type an asterisk (*) in the Module name field and click Add.

    Files located in C:\windows\system32 are excluded from the compatibility infrastructure by default. So if an application calls an API from a file located in the system32 folder, any shims you've configured will be ignored. While this default configuration doesn't usually cause a problem, applications coded with Visual Basic (VB) 6 or earlier may ignore shims as the VB runtime is located in system32. CorporateApp is a Visual Basic application, so we must ensure that all modules launched from the system32 folder are included.

  8. Click OK in the Options for ForceAdminAccess dialog box and then click on Next to continue in the Create new Application Fix wizard.
  9. Accept the default settings on the Matching Information screen by clicking Finish.
  10. Select the CorporateApp database in Compatibility Administrator and click the Save icon at the top of the window.
  11. Save the database as corporateapp.sdb to C:\.
  12. To test the application, right-click the CorporateApp database in Compatibility Administrator and select Install from the menu.
  13. Click OK in the confirmation dialog box.

You can now run CorporateApp as a standard user on this system and the hard-coded check for administrative privileges will not prevent the program from running.

CorrectFilePaths

Problem The application tries to create or write to a file in a protected system directory, such as C:\Program Files, where the application is installed.
Fix Apply the CorrectFilePaths compatibility fix using Compatibility Administrator.

Log in as a standard user but start Compatibility Administrator with administrative privileges, as described in the steps for configuring ForceAdminAccess fix and follow these instructions:

  1. Click Open at the top of Compatibility Administrator and open the CorporateApp database (c:\corporateapp.sdb).
  2. Expand the CorporateApp database under the Custom Databases node and click CorporateApp in the Applications folder in the left pane.
  3. Click CorporateApp.exe at the top of the right pane. Now select Edit | Modify | Edit Application Fix.
  4. Click Next past the Program information screen, and Next again past Compatibility Modes.
  5. On the Compatibility Fixes screen, scroll down the list of fixes and check CorrectFilePaths.
  6. Click Parameters and then type the path of the file to be redirected and its new location as follows: old path;new path into the Command line field. For example, CorporateApp is trying to create and write to a file called config.ini in c:\program files\CorporateApp. We need to redirect the file to a location where standard users have permission to write. So an example redirection might be as follows:
    %programfiles%\CorporateApp\config.ini;%userappdata%\config.ini

    Multiple file redirections
    One CorrectFilePaths application fix can contain multiple redirections on the command line separated by a space.

  7. Once you've added the file redirection, type an asterisk (*) in the Module name field and click Add to include all modules in the system32 folder.
  8. Click OK in the Edit Application Fix dialog box and then click Next to continue in the Create new Application Fix wizard.
  9. Accept the default settings on the Matching Information screen by clicking Finish.
  10. Click Save at the top of the Compatibility Administrator window and then right-click the CorporateApp database and select Install from the menu.
  11. Click OK in the confirmation dialog box.

VirtualRegistry

The VirtualRegistry fix is one of the most flexible compatibility fixes and can be modified by specifying command-line options. In this section, we'll look at the functions of VirtualRegistry that are of most interest in solving problems related to least privilege security.

ADDREDIRECT

The ADDREDIRECT option was introduced in Windows Vista. It allows you to redirect a specific registry key to a different location as specified in the command-line options. Let's have a look at how to add command-line options to the VirtualRegistry fix. In this example, we'll redirect a registry key that reads and writes to the protected HKEY_LOCAL_MACHINE hive to the HKEY_CURRENT_USER hive, which standard users have read and write access to.

Follow these instructions, logged in as a standard user but running Compatibility Administrator with administrative privileges:

  1. Click Open at the top of Compatibility Administrator and open the CorporateApp database (c:\corporateapp.sdb).
  2. Expand the CorporateApp database under the Custom Databases node and click CorporateApp in the Applications folder in the left pane.
  3. Click on CorporateApp.exe at the top of the right pane. Now select Edit | Modify | Edit Application Fix.
  4. Click Next past the Program information screen, and Next again past Compatibility Modes.
  5. On the Compatibility Fixes screen, scroll down the list of fixes and check VirtualRegistry.
  6. Click Parameters at the top of the screen and type the following option into the command line field:
    ADDREDIRECT (HKLM\SOFTWARE\Microsoft\PCHealth\ErrorReporting\DW^HKCU\Software\DemoApp\DW)
  7. Type an asterisk (*) in the Module name field and click Add to include all modules in the system32 folder. Click OK to continue.

You'll now see the command line added to VirtualRegistry on the Compatibility Fixes screen and you can continue as per the instructions in the Create your own custom database section.

RedirectFiles and RedirectRegistry
The RedirectFiles and RedirectRegistry compatibility fixes are designed specifically for Internet Explorer. CorrectFilePaths and VirtualRegistry should be used to redirect files and registry keys.

Working with custom databases

When updating or maintaining a central custom database in your organization, you may need to deploy or redeploy the database, and merge new applications fixes.

Adding new shims to your custom database (merging custom databases)

If you decide to maintain a central custom database that contains all your application shims, it's likely that you'll need to update it from time to time when new programs are deployed. Compatibility Administrator allows you to open two databases simultaneously and copy and paste fixes between them. Let's create a new application fix and add it to a custom database. First you should follow the steps described in Creating your own custom database to add a new shim to a custom database and save it. You cannot create a shim without a database.

  1. In Compatibility Administrator, open the database that contains the shim you want to add to the central database, in this case Shim to merge, and open the central database.
  2. In this example, we want to merge the fix for Demoapp into the central database that already contains a fix for Maxthon. Right-click on the Demoapp fix under Applications and select Copy from the menu.
  3. Now, right-click the Applications folder under the central database node an select Paste from the menu.
  4. The fixes for Demoapp and Maxthon now appear together in the central database and you can click Save at the top of Compatibility Administrator to save changes to the central database.

Temporarily disabling compatibility fixes

If an application is updated to address a known compatibility problem that one of your shims addresses, you may want to temporarily disable a fix in the custom database while testing the new version of the program.

  1. Click Open at the top of Compatibility Administrator, locate your central database .sdb file and click Open.
  2. Select the fix you want to disable in the Applications folder in the left pane and then select Disable Entry from the Database menu at the top of the window.

  3. To re-enable the disabled fix, repeat the procedure as described earlier, but select Enable Entry from the Database menu instead.

Installing a custom database from Compatibility Administrator

When developing shims using Compatibility Administrator, you will need to test if the fixes solve the issue(s) at hand. To do this on the same machine on which Compatibility Administrator is running, you must install the custom database that contains the fixes.

  1. To install a custom database locally, right-click on your database in the left pane of Compatibility Administrator and select Install from the menu.
  2. To uninstall a custom database, expand the Installed Databases node in the left pane of Compatibility Administrator, right-click on the database to remove, and select Uninstall from the menu.

Database locations
The System Database, which is included as part of the operating system and contains shims that resolve known compatibility problems with popular applications, is located in the AppPatch directory in c:\ windows and is updated by Microsoft via Windows Update. The AppPatch directory contains a subdirectory called Custom, which contains any custom databases installed on a machine.

Deploying a database to multiple devices

While Compatibility Administrator can be used to install or remove a database on the machine where it's installed, Microsoft provides a command-line utility called sdbinst.exe for deploying custom databases across multiple machines.

Finding the GUID of custom database

Once a custom database has been saved in Compatibility Administrator, it's assigned a Globally Unique Identifier (GUID) that can be referenced from the command line when installing or removing databases. To find the GUID of a given database, right-click the database in the left pane of Compatibility Administrator and select Properties from the menu. The database's GUID will be shown in the Database Properties window.

SDBINST command-line switches

There are four basic switches that can be used with sdbinst.exe and they are outlined in the following table:

Switch Description
-n "filename" Used to specify the filename of the custom database to install or uninstall.
-g GUID Used to specify the custom database by GUID to install or uninstall.
-q Silent install. No information is presented to the user during an install or uninstall operation. Used mainly for scripts.
-u Performs an uninstall operation as opposed to the default install.

For example, we can install our custom central database from the command line using the following switches:

sdbinst.exe -g {1810c392-765d-457f-873e-8e176ee09298}

Alternatively, we can uninstall the same database by filename silently:

sdbinst.exe -u -q -n "\\networkshare\centralshimdb.sdb

SDBINST must be run with administrative privileges to install and remove custom databases.

Distributing or updating a custom database using Group Policy

Custom databases can be included as part of your organization's standard image. Alternatively, SDBINST can be distributed using a Group Policy computer startup script. The following batch file calls reg.exe and sdbinst.exe, both command-line programs that are included in Windows XP and later by default, to first check the existence of a custom database by verifying the presence of an uninstall registry key, and if the registry key is not found, install the database:

reg.exe query "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion
\Uninstall\{GUID}.sdb" &&(goto:eof)
sdbinst.exe -q %~dp0centralshimdb.sdb
:eof
exit

The %~dp0 variable
%~dp0 sets the working directory to the same path as the location of the batch file. This enables us to drop the centralshimdb.sdb file into the same location as the batch file called from the Group Policy startup script without having to specify the exact path.

If you want to update an installed custom database, as long as the new version of the database has the same GUID as the one to be replaced, sdbinst.exe will automatically remove the database and install the new version. In an update situation, you will need to remove the logic in the batch file that checks for the presence of an installed database with the specified GUID. This can be easily achieved by adding a hash (#) in front of the reg.exe command to temporarily prevent it from running.

#reg.exe query "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion
\Uninstall\{GUID}.sdb" &&(goto:eof)
sdbinst.exe -q %~dp0centralshimdb.sdb
:eof
exit

Now that we've got our batch file ready, let's create a Group Policy Object and add it as a computer startup script. First, save the batch file in Notepad and call it installdb.bat.

  1. Open the Group Policy Management Console as a domain administrator from Administrative Tools on the Start menu and expand your Active Directory forest and domain in the left pane.
  2. Locate your Clients OU, right-click it, and select Create a GPO in this domain, and Link it here from the menu.
  3. In the New GPO dialog box, type App Compat DB startup script in the Name field and click OK.
  4. Expand the Clients OU in the left pane, right-click the App Compat DB startup script GPO, and select Edit from the menu.
  5. In the Group Policy Management Editor window, expand Policies under Computer Configuration in the left pane.
  6. Expand Windows Settings and click Scripts (Startup/Shutdown).
  7. Double-click Startup in the right pane and click Show Files....
  8. A new window will open with the location of script files for this Group Policy Object. Copy the installdb.bat and centralshimdb.sdb to the Startup folder and then close the window.
  9. Click Add... in the Startup Properties dialog box and then Browse... to the right of the Script Name field.
  10. Select the installdb.bat file in the Browse dialog box and click Open.
  11. Click OK in the Add a Script dialog box and the installdb.bat file should now appear in the Startup Properties window.
  12. Click OK in the Startup Properties dialog box and close the Group Policy Management Editor.

Now if you restart a machine in Clients OU, the Startup Script will run as the computer boots.

Summary

In this article, we've discussed all the basic concepts of the Windows Application Compatibility Infrastructure and how to manage compatibility fixes in an enterprise environment. Before continuing to the next chapter you should:

  • Understand how to use Compatibility Administrator to create a custom database
  • Be able to deploy and update a custom database to multiple devices across a network using Group Policy in conjunction with the sdbinst.exe command-line utility
  • Know how to update and add shims to a custom database using Compatibility Administrator
  • Be familiar with the most common compatibility fixes used for solving least privilege security problems in legacy applications and how to deploy them

Further resources on this subject:


About the Author :


Russell Smith

Russell Smith is an independent IT consultant who specializes in management and security of Microsoft-based IT systems. An MCSE with more than ten years experience, his recent projects include Active Directory Security Consultant for the UK Health Service National Programme for Information Technology (NPfIT) and Exchange Architect for Wipro Technologies.  Russell is a regular contributor to industry journals Windows IT Professional, Security Professional VIP, CDW's Biztech Magazine and a contributing author to Supporting and Troubleshooting Applications on a Microsoft Windows Vista Client for Enterprise Support Technicians from Microsoft's Official Academic Course (MOAC) series of books published by Wiley and Sons. Russell also has extensive experience as an IT trainer.

Books From Packt


Refactoring with Microsoft Visual Studio 2010
Refactoring with Microsoft Visual Studio 2010

Microsoft Visio 2010 Business Process Diagramming and Validation
Microsoft Visio 2010 Business Process Diagramming and Validation

Applied Architecture Patterns on the Microsoft Platform
Applied Architecture Patterns on the Microsoft Platform

.NET Compact Framework 3.5 Data Driven Applications
.NET Compact Framework 3.5 Data Driven Applications

Microsoft Silverlight 4 Data and Services Cookbook
Microsoft Silverlight 4 Data and Services Cookbook

Microsoft Dynamics NAV 2009 Application Design
Microsoft Dynamics NAV 2009 Application Design

Microsoft Office Live Small Business: Beginner’s Guide
Microsoft Office Live Small Business: Beginner’s Guide

WCF 4.0 Multi-tier Services Development with LINQ to Entities
WCF 4.0 Multi-tier Services Development with LINQ to Entities


Code Download and Errata
Packt Anytime, Anywhere
Register Books
Print Upgrades
eBook Downloads
Video Support
Contact Us
Awards Voting Nominations Previous Winners
Judges Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software
Resources
Open Source CMS Hall Of Fame CMS Most Promising Open Source Project Open Source E-Commerce Applications Open Source JavaScript Library Open Source Graphics Software