In this chapter, we will cover:
Gathering requirements by asking the right questions
Analyzing requirements and creating a design
Installing and using the BizTak Best Practices Analyzer
Validating BizTalk installation with the BizTalk Benchmark Wizard tool
Automating performance analysis by using the PAL tool
Managing the SSO system
Configuring MSDTC for multi-server BizTalk platforms
Having a stable, robust, (highly) available BizTalk environment is important. Many times BizTalk is installed, solutions are architected, and then the environment is not up to its task. System boundaries are reached fast and solutions do not perform well and make BizTalk throttle, as a result of excessive memory use or flooding of the system with messages. There are many ways to prevent this by using tools that are available, together with some best practices provided by Microsoft. They deliver a quite extensive BizTalk optimization guide (http://www.microsoft.com/download/en/details.aspx?id=10855) and operation guide (http://www.microsoft.com/download/en/details.aspx?id=6282).
Knowing your environment and its boundaries will provide insight on how to prevent your BizTalk environment from becoming a hotspot. For doing this, you have to go through the following steps:
Process of setup
Configure and test
Adjust and test
It starts with setting up your environment, but prior to that you will need to assess the requirements. You will only be able to do so by asking the right questions and analyzing them well. When you deploy BizTalk on the determined hardware and software, you will be able to validate, test, and tune it using the following:
During assessing and analyzing the requirements, it is possible that you will end up with a design of a scaled out BizTalk configuration. This will affect some of the essential components in a BizTalk runtime, the Microsoft Distributed Transaction Coordinator (MSDTC) and the Master Secret Server. In a multi-server BizTalk environment, MSDTC is vital and needs to get set up and configured properly before one starts configuring BizTalk features, such as group, Business Rule Engine (BRE), or Business Activity Monitoring (BAM).
During configuration of these BizTalk features, databases such as the BizTalk MessageBox (BizTalkMsgBoxDb), BizTalk Management (BizTalkMgmtDb), BizTalk Tracking (BizTalkDTADb), BizTalk Rule Engine (BizTalkRuleEngineDb), BAM Primary Import (BAMPrimaryImport), and others (BAMStarSchema) are created on the database server.
The Master Secret Server plays an essential role in aiding the BizTalk runtime in securing information for the receive locations and send ports. Some cases in which high availability is required, SQL Server and the host instances, need to be clustered for certain adapters such as FTP, POP3, and MSMQ. The reason for clustering the adapters is to prevent message duplication. The FTP protocol, for instance, does not support file locking (http://kentweare.blogspot.com/2009/04/clustering-biztalk-hosts.html).
In this chapter, requirements gathering and assessment will be described in a manner that could aid you when undergoing this process at the client side. Once you have prepared the deployment of BizTalk, you can go through the Analyzing requirements and creating a design, Configuring MSDTC for multi-server BizTalk platforms, and Managing the SSO system recipe later in this chapter. To validate, test and tune your BizTalk environment; you can read the recipes mentioned earlier.
Although, this is not an exact recipe, asking questions to obtain requirements for your BizTalk environment is important. Having a clear view and understanding of the requirements enables you to deploy the desired BizTalk environment that meets expectations of the customer. What are the right questions you may ask yourself? Well, there is quite a large area in general you basically need to cover with questions. These questions will be around the following topics:
Organize the sessions, and/or the workshop(s) to discuss the BizTalk architecture (environment), functionality, and non-functional requirements, where you do a series of interviews with appropriate stakeholders. This way you will be able to retrieve the necessary requirements and information for a BizTalk environment. You will need to focus on business first and IT later. You will notice that each business will have a different set of requirements on integration of data and processes. Some of these are listed as follows:
Business is able to have the access of information from anywhere any time
Have the proper information to present to the proper people
Have the necessary information available when needed
Manage knowledge efficiently and be able to share it with the business
Change the information when needed
Automate the business process that is error-prone
Automate the business process to reduce the processing time of orders, invoices, and so on
Regarding the business requirements, BizTalk will have certain workloads, and with the business you determine if you want BizTalk to aid in automating processes, exchange of information with partners, maintaining business rules, visibility of psychical events, and/or integration with different systems. One important factor to reckon with bringing BizTalk into an organization is risk-associated with transitioning to its platform. This risk can be of a technical, operational, political, and financial nature. BizTalk solutions have to operate correctly, meet the business requirements, and be accepted by stakeholders within the organization and should not be too expensive.
With IT, you focus more on the technical side of the BizTalk Environment such as, "What messages in size, format, and encoding are sent to the BizTalk system or what does it need to output?" You should consider security around it, when information going to or coming from trading partners is confidential. Encryption and decryption of data such as, "What processes that are automated need to interact with internal and external systems?" or "How are you going to monitor messages that are going in and out?" can come into play. Support needs to be set up properly to keep BizTalk and its solutions healthy. Solutions need to be developed and tested, preferably using different environments such as test and acceptance. For that, you will need an agreed deployment process with IT. These are factors to reckon with and need to be addressed when interviewing or talking to IT stakeholders within the organization.
Categorize your stakeholders into two categories—business and IT. Create a communication plan and list of questions related to areas mentioned earlier. With the list of questions you can assign each question to a person you think can answer it. This way you ask the right questions to the right people. The following table shows a sample of roles belonging to business and/or IT. It could be that you identify more roles depending on your situation:
|
Category |
Role |
|---|---|
|
Business |
CEO, CIO, Security Officer, Business Analyst, Enterprise Architect, and Solution Architect. |
|
IT |
IT Manager, Enterprise Architect, Solution Architect, System/Application Architect, System Analyst, Developer, System Engineer, and DBA. |
Having the roles clear belonging to either business, IT, or both, you will then need to have a list of questions and assign these to the appropriate role. You can find an example list of questions associated to a particular role in the following table:
|
Question |
Role |
|---|---|
|
Will BizTalk integrate with systems in the enterprise? Which consumers and host systems will it integrate with? |
Enterprise Architect, Solution Architect |
|
What are the applicable workloads? |
Enterprise Architect |
|
Is BizTalk going to be strategic for integration with internal/external systems? |
CEO, CIO, Enterprise Architect, and Business Analyst |
|
Number of messages a day/hour |
Enterprise Architect |
|
What are the candidate processes to automate with BizTalk? |
Business Analyst, Solution Architect |
|
What communication protocols are required? |
Enterprise Architect, Solution Architect |
|
Choice of Microsoft platform—Operating System, SQL Server Database |
Enterprise Architect, Security Officer, Solution Architect, System Engineer, and DBA |
|
Encryption algorithm for data |
Enterprise Architect, Security Officer, Solution Architect, and System Engineer |
|
Is Secure Socket Layer required for communication? |
Enterprise Architect, Security Officer, Solution Architect, and System Engineer |
|
What kind of certificate store is there? |
Enterprise Architect, Security Officer, Solution Architect, and System Engineer |
|
Is the support for BizTalk going to be outsourced |
CEO, IT Manager |
The best approach to gather the requirements is to view it as a project or a part of the project. You can use a methodology such as PRINCE2.
Projects in Controlled Environments (PRINCE) is a project management method. It covers the management, control, and organization of a project. PRINCE2 is the second major release of it. More information is available at http://www.prince2.com/.
The Microsoft BizTalk Server website provides a lot of information. Especially, the Production Information section provides detailed information on system requirements, roadmap, and the FAQs. The latter sections provide details on pricing, licensing, and so on. Go to http://www.microsoft.com/biztalk/en/us/default.aspx.
Analyzing requirements and creating a design for the BizTalk landscape is the next step forward before planning and installing. With the gathered requirements, you can make decisions on how to design a BizTalk environment(s). If BizTalk is used for the first time in an enterprise environment capacity, planning and server allocation is something to focus on. Once you gather requirements and ask questions, you will have a clear picture of where the platform will be hosted and whether it needs to be scaled up or out. If everything gets placed on one big server, it will introduce a serious single point of failure. You should try to avoid this scenario. Therefore, separating BizTalk from the SQL Server is the first thing you will do in your design, each on a separate hardware preferably.
Depending on availability requirements, you will probably cluster the SQL Server. Besides that, you can choose to scale out BizTalk into a multiserver group, because of availability requirements and if the expected load cannot be handled by one BizTalk instance. You can opt for installing BizTalk and SQL separately first and then scale-out after performing benchmark tests. You can scale vertically (scaleup) by increasing the number of processors and the amount of memory each server uses, or you can scale horizontally (scaleout) by adding more servers to your BizTalk Server configuration. Other options you can consider during your design are as follows:
Having multiple MessageBox databases
Separate BizTalk databases
These options are best visualized by the scale-out poster from Microsoft (http://www.microsoft.com/download/en/details.aspx?id=13103).
Based on the requirements, you can consider isolating the BizTalk hosts to be able to manage BizTalk applications better and divide the load. By separating send, receive, and processing functionality in different hosts, you will benefit from better memory and thread management.
Note
If you expect a high load of large messages or orchestrations that would consume large amounts of resources, you should isolate send and/or receive adapters. Another consideration is to separate a host to handle tracking and relieve processing hosts from it.
So far we have discussed scalability and design decisions you could consider. There are some other design considerations for a BizTalk environment such as security, tracking, fault tolerance, load balancing, choice of license, and support for virtualization (http://support.microsoft.com/kb/842301). BizTalk security can be enhanced by deploying Secure Socket Layer (SSL), IPSec Tunneling, the Inter Security and Acceleration (ISA) server, and certificate services included with the Windows Server 2008. With the BizTalk Server, you can apply access control, implement least rights to limit access, and provide integrated security through Enterprise Single Sign-On (http://msdn.microsoft.com/en-us/library/aa577802%28v=bts.70%29.aspx). Furthermore, you can protect and secure applications and data by authenticating the sender of a message and authorizing the receiver of a message (refer to Chapter 4, Securing your Message Exchange).
Tracking messages in BizTalk messages can be useful to see what messages come in and out of the system, or for auditing, troubleshooting, or archiving purposes. Tracking of messages within BizTalk is a process by which parts of a message such as the body, properties, and metadata are stored in a database. These parts can be viewed by running queries from the Group Hub page in the BizTalk Server Administration console.
Note
It is important that you decide, or take up into the design, what needs to be tracked based on the requirements.
There are some considerations to make regarding tracking. Tracking everything is not the smart thing to do, as each time a message is touched in BizTalk; a copy is made and stored. Focus on scope by tracking only on a specific port, which is better for performance and keeps the database uncluttered. For the latter, it is important that the data purge and archive job is configured properly. As mentioned earlier, it is worth considering a dedicated host for tracking.
Fault tolerance and load balancing for BizTalk can be achieved through clustering, separating hosts as described earlier, implement a Storage Area Network (SAN) to house the BizTalk Server databases, cluster Enterprise Single Sign-On (SSO) Master Secret Server, and configuring the Internet Information Services (IIS) web server for isolated host instances and the BAM Portal web page to be highly available using Network Load Balancing (NLB) or other load balancing devices. The best way to implement this is to follow the steps in the Checklist: Providing High Availability with Fault Tolerance or Load Balancing document found on MSDN (http://msdn.microsoft.com/en-us/library/gg634479%28v=bts.70%29.aspx).
Another important topic regarding your BizTalk environment is costs and based on requirements you will choose the Branch, Standard, or Enterprise Edition. The editions differ not only in price, but also in functionality. As with the Standard Edition, it is not possible to support scenarios for high availability, fault tolerance, and is limited on CPU and applications. The Branch Edition is even more limited and is designed for hub and spoke deployment scenarios including Radio Frequency Identification (RFID). With any version, you probably want to consider whether or not to virtualize. With virtualization in mind, licensing can be difficult.
With the Standard Edition, you need a license for each virtual processor used by the virtual OS environment, regardless of whether the number of virtual processors is less than, or greater than, the number of physical processors on the server. With the Enterprise Edition, if you license all physical CPUs on the server you can run any number of instances in the physical or virtual OS environment. With both of these, a virtual processor is assumed to have the same number of cores as the physical processor. Using less than the number of cores available in the physical processor still counts as a full virtual processor (http://www.microsoft.com/biztalk/en/us/editions.aspx).
Last, but not least, you need to consider how to support your BizTalk environment. It is worth considering the System Center Operation Manager to monitor your BizTalk environment using management packs for the SQL Server, Windows Server, and BizTalk Server 2010. The management pack for the BizTalk Server 2010 provides two views, one for the enterprise IT administrator and one for the BizTalk Server administrator. The first will be monitoring the state and health of the various enterprise deployments, the machines hosting the SQL Server databases, machines hosting the Enterprise SSO service, host instance machines, IIS, network services, and is interested in the overall health of the "physical deployment" of a BizTalk Server setup. The BizTalk Server Administrator will be monitoring the state and health of various BizTalk Server application artifacts, such as orchestrations, send ports, receive locations, and is interested in monitoring and tracking the BizTalk Server's health. If necessary, he/she can carry out corrective measures to keep applications running as expected.
What you have read so far are considerations, which are useful while analyzing requirements and preparing your design. You need to take a considerable amount of time for analyzing requirements to be able to create a solid design for your BizTalk environment. There is a wealth of information provided by Microsoft in this book. It will be worth investing time now as you will lose a lot time and money if your applications do not perform or the system cripples under load while receiving the process.
To analyze the requirements, you will need to categorize them to certain topics mentioned in the Gathering requirements by asking the right questions recipe. You will then go over each requirement and decide how it can be met best. For each requirement, you will consider what the best option is and capture that in your design for the BizTalk setup. The BizTalk design will be a Word document, where you capture your design, considerations, and decisions.
During analysis of each requirement, you will capture your considerations and decisions in a word document. Besides that, you will also describe the situation at the enterprise where the BizTalk environment will be deployed. You will find an example structure of a design document for a Development, Test, Acceptance, and Production (DTAP) environment, as follows, where you can place all the information:
Introduction
1.2 Current situation
1.3 IT landscape
2.1 Considerations/Issues
3.1 DTAP landscape
3.2 Scope
3.3 MS BizTalk and SQL Server editions
3.4 SQL Database Server
4.1 Operating systems
4.2 Windows Server
4.3 Backup
4.4 Antivirus
4.5 Windows update
4.6 Security Settings
5.1 Backup procedure
5.2 Restore procedure
6.1 Development environment
6.2 Development server
6.3 Developer machine
Test
8.1 SQL Server clustering
8.2 BizTalk group
8.3 Acceptance server
9.1 SQL Server clustering
9.2 BizTalk group (load balancing)
9.3 Production server
10.1 Groups and accounts
10.2 SCOM
10.3 Single Sign-On
Hosts
11.1 In process hosts
11.2 Isolated hosts
11.3 Trusted and untrusted hosts
Resources
Appendix A Redistributable CAB Files
Design decisions are the important parts of your document. Here, you summarize all your design decisions and reference them to each corresponding chapter/section in the document, where a decision is described; you also note issues around your design.
Analyzing requirements is an important task, which should not be taken lightly. Knowing architectural patterns, for instance, can help you choose the right technology and create the appropriate design. It can be that the BizTalk Server is not the right fit for the purpose. The following resources can aid you in analyzing the requirements:
Architectural Patterns: Packt has published a book called Applied Architecture Patterns on Microsoft Platform that can aid you in analyzing the requirements by selecting the right technology.
Wiki TechNet article: Refer to the Recommendations for Installing, Sizing, Deploying, and Maintaining a BizTalk Server Solution article at http://social.technet.microsoft.com/wiki/contents/articles/666.aspx.
Microsoft BizTalk Server 2010 Operations Guide: Microsoft has created a BizTalk Server 2010 Operations Guide for anyone involved in the implementation and administration of a BizTalk solution, particularly IT professionals. You can find it online (http://msdn.microsoft.com/en-us/library/gg634499%28v=bts.70%29.aspx) or you can download it from http://www.microsoft.com/downloads/en/details.aspx?FamilyID=4ef9eebb-b3f4-4534-b733-3eb2cb83d867&displaylang=en.
Microsoft volume licensing brief: Licensing Microsoft Server Products in Virtual Environments is an interesting white paper from Microsoft. It describes licensing models under virtual environments for the server operating systems and server applications. It can help you understand how to use Microsoft server products with virtualization technologies, such as Microsoft Hyper-V technology, Microsoft Virtual Server 2005 R2, or third-party virtualization solutions that are provided by VMWare and Parallels. You can download from the URL: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=9ef7fc47-c531-40f1-a4e9-9859e593a1f1&displaylang=en.
Microsoft poster scale-out configurations: Microsoft has published a poster (normal or interactive) that can be downloaded describing typical scenarios and commonly used options for scaling out the BizTalk Server 2010's physical configurations. This post clearly illustrates how to scale for achieving high availability through load balancing and fault tolerance. It also shows how to configure for high-throughput scenarios.
A normal poster can be obtained from the URL: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=2b70cbfc-d158-45a6-8bbd-99782d6747dc.
An interactive poster created in Silverlight can be obtained from the URL: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=7ef9ae69-9cc8-442a-8193-831a414dfc30.
The Best Practices Analyzer (BPA) examines a BizTalk Server 2010 deployment and generates a list of issues pertaining to best practice standards for BizTalk Server deployments. This tool is designed to assess the configuration of a BizTalk installation. The BPA performs configuration-level verification by gathering data from different information sources, such as Windows Management Instrumentation (WMI) classes, SQL Server databases, and registry entries and presents a report to the user. Under the hood, it uses the data to evaluate the deployment configuration. It does not modify any system settings and is not a self-tuning tool. The tool is there to deliver support in achieving the best suitable configuration and report issues or possible issues, that could potentially harm the BizTalk environment.
The latest version of the BPA tool (V1.2) can be obtained from the Microsoft download center (http://www.microsoft.com/downloads/en/details.aspx?FamilyID=93d432fe-1370-4b6d-aaa8-a0c43c30f5ab&displaylang=en) and must be installed on the BizTalk machine. As a user, you need an account that has local administrative rights, that is a member of the BizTalk Server Administrators group, and a member of the SSO Administrators group to be able to run the BPA.
Note
You may need to explicitly set some WMI permissions before you can use the BPA in a distributed environment, where the SQL Server is not installed on the same computer as the BizTalk Server. This is because when the BPA tries to connect to a remote computer running the SQL Server, WMI may not have sufficient access to determine whether the SQL Server Agent is running. This may result in incorrect BPA evaluations.
To run the Best Practices Analyzer, perform one of the following:
Start the BizTalk Server Best Practices Analyzer from the Start menu. Go to Start | Programs | Microsoft BizTalk Server Best Practices Analyzer.
Open Windows Explorer and navigate to the Best Practices Analyzer installation directory (by default,
c:\Program Files\BizTalkBPA\) and double-click onBizTalkBPA.exe.Open a command prompt, change to the installation directory, and then enter
BizTalkBPACmd.exe.
The following steps need to be performed to do the analysis:
As soon as you start the BPA, it will check for updates. The user can decide whether or not to check for updates for newer versions of the configuration:
If a newer version is found, you are able to download the latest updates. The next step is to perform a scan by clicking on Start a scan:
After starting the scan,
starts datawill be gathered from different information sources as described earlier.After the scan has been completed, the user can decide to view the report of the performed scan:
You can click View a report of this Best Practices scan and the report will be generated. After generation of the report, several tabs will appear:
Critical Issues
All Issues
Non-Default Settings
Recent Changes
Baseline
Informational Items
When the BPA is running, it gathers information and evaluates them to best practice rules from the Microsoft product group and support. A report is presented to the user providing information on issues, non-default settings, changes, and so on. The report enables you to take action and apply the necessary changes to resolve identified issues. The BPA can be run again to verify that it adheres to all the necessary best practices. This shows the value of the tool when assessing the deployed BizTalk environment before it is operational. When BizTalk becomes operational, the MessageBox Viewer (MBV) has more value.
The BPA is very useful and gives you information that helps you to tune BizTalk and to keep it healthy. There are more tools that can help in sustaining a healthy environment overall. The Microsoft SQL Server 2008 R2 BPA is a diagnostic tool that provides information about a server and a Microsoft SQL Server 2008 or Microsoft SQL Server 2008 R2 instance installed on that server.
The Microsoft SQL Server 2008 R2 Best Practices Analyzer can be downloaded from http://www.microsoft.com/download/en/details.aspx?id=15289.
There are a couple of analyzers provided by Microsoft that do a good job helping you and the system engineer to put out a healthy, robust, and stable environment:
Best Practices Analyzer: http://technet.microsoft.com/en-us/library/dd759260.aspx
Microsoft Baseline Configuration Analyzer 2.0: http://www.microsoft.com/download/en/details.aspx?id=16475
Microsoft Baseline Security Analyzer 2.1.1: http://www.microsoft.com/download/en/details.aspx?id=19892
The BizTalk Benchmark Wizard (BBW) is a tool to validate a BizTalk installation and was built by Mikael Håkansson and Ewan Fairweather. It is intended to verify your BizTalk installation and is a useful tool to prove the performance characteristics prior to deploying the first solutions. You may also want to use it before you are about to scale out your environment, to make sure you are really using all resources, before investing in additional hardware and licenses. The BBW performs load to BizTalk Server in relation to specific scenarios. During the execution of the test, counter information is collected and benchmarked against collected statistics relevant to your BizTalk Server environment.
Microsoft provides guidance on performance optimization through the performance optimization guide that provides in-depth information for optimizing the performance of a BizTalk Server.
Note
Even though the BizTalk Server 2010 Optimization Guide (http://www.microsoft.com/download/en/details.aspx?id=10855) is very useful while designing a BizTalk system, it does not provide any expected performance numbers related to specific environments.
This is where the BBW comes into the picture. After having configured the BizTalk group and applied some of the best practices by using the BPA, you can test your BizTalk configuration.
The tool is not a load or analyzing tool; it doesn't give advice or hints like the BPA. However, if the test fails, you can analyze the data using the PAL tool. You can run the tool on either a single-server or a multi-server installation. Regardless of the number of BizTalk Servers in your group, you should not run it with more than two "active" servers (http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=2290), as it will otherwise not be covered by the benchmark values.
Note
This tool should be used after the BizTalk Server has been installed and before any solutions are deployed to the environment. This will ensure that you are getting consistent and clean results from the BBW.
The latest version of the Benchmark Wizard tool can be obtained from CodePlex (http://bbw.codeplex.com/) and must be installed on the BizTalk machine. Follow the steps given next:
In a browser, such as Internet Explorer, navigate to the BBW download location (http://bbw.codeplex.com/).
On the BBW download page, download and run the self-extracting file.
Open a command prompt window and navigate to
[installation folder]\Artefacts\BizTalk. By default, the installation folder isC:\Program Files\Blogical\BizTalk Benchmark Wizard.In this folder, you will find the
InstallHosts.vbsfile. Execute it using the following parameters:UserName: The name of the user account running the service instancesPassword: The password of the user account running the service instancesReceive Host: The name of the server where you want to run the receive host instanceSend Host: The name of the server where you want to run the send host instanceProcessing Host: The name of the server where you want to run the process host instance
If you have a single box installation, your script command might look similar to the following:
cscript InstallHosts.vbs "BizTalk Application Users" "\MyUser" "MyPassword" "BtsServer1" "BtsServer1" "BtsServer1"The result will be as depicted in the following diagram:
If you have a multi-server installation, your script command might look similar to the following:
cscript InstallHosts.vbs "MyDomain\BizTalk Application Users" "MyDomain\MyUser" MyPassword" "BtsServer1" "BtsServer2" "BtsServer2"The result will be as depicted in the following diagram:
Execution of the script in a command line will result in creation of the hosts and the result of these actions is displayed in the following screenshot:
Open the BizTalk Administration Console, point to the Applications node and import the
BizTalk Benchmark Wizard.msifound in the folder[Installation folder]\Artefacts\BizTalk.Run the
BizTalk Benchmark Wizard.msion all BizTalk servers to add the assemblies to the Global Assembly Cache (GAC).Go to Start | All Programs | BizTalk Application Wizard to start the application.
When the installation of the BBW is successful, you can start it from C:\Program Files\Blogical\BizTalk Benchmark Wizard and follow the steps given next:
You will be welcomed first and then you will have to set up some prerequisites such as the BizTalk Management Database:
The user can opt to collect data for further analysis but this is optional. Collection of datasets enables the creation of a log file that contains all information of a Benchmark run. This log can be analyzed later using, for instance, the PAL tool:
Then the user gets to select one of the two scenarios—Messaging or Orchestration. Each scenario has a set of tested environments as follows:
Single server (2*Quad CPU, 4GB RAM)
1*BTS (1*Quad CPU. 4GB RAM) + 1*SQL(1*Quad CPU, 8GB RAM)
2*BTS (2*Quad CPU. 8GB RAM) + 2*SQL(2*Quad CPU, 16GB RAM)
The user can select the environment that best resembles his own environment. The next screen will display what will happen given the chosen scenario:
As a user, you will have to select where each host instance resides on each machine, if applicable. In case of a single-server installation, all the hosts are on the same machine:
One of the following steps is to configure the Indigo Service, a console application hosting service which will be called from the BizTalk Send port. You will have to either host the service on a separate server or go to the folder
[Installation folder]\Artefacts\IndigoServiceand then right-click IndigoService. In the BBW, you will have to either follow instructions and/or fill the Server name and click Test the Indigo Service . There is also the option of collecting performance counters:
As the user clicks Run Test, the tool continues to start ports and orchestrations. It will also start the Perfmon collector sets if the user has chosen to create those (refer to step 2).
As the test proceeds, the user can monitor the counter values through the gauges (Avg Received msgs/sec and Avg Processed msgs/sec). The default test duration is 30 minutes, with a warm-up of two minutes. In this case, five minutes has been chosen.
After warming up, the test will run and you will see the gauges moving certain values.
Finally, the user is presented with a result, which is either Succeeded, Acceptable, or Failed. After the test is run, you can save a report:
By completing the BizTalk Benchmark wizard installation, the following artifacts have been created:
Three hosts –
BBW_RxHost,BBW_PxHost,BBW_TxHostThree host instances
Two adapter handlers for NetTcp
One BizTalk application
Two Receive ports
Two Send hosts
One Orchestration
Hosts and instances are created through scripts and ports, orchestration is created through installing the msi files. The user can start the wizard and, at the end, run a test based on the selections made. During the test, loadgen will generate the xml messages that will be sent over NetTcp to BizTalk. The messages will be published in the MessageBox database and picked up either by the subscribing send port or orchestration. Finally, the messages will be sent to the backend web service through the WCF-NetTcp adapter.
The following section deals with KPIs in detail for certain scenarios.
The test run failed as it did not meet the KPIs, at least not all of them. Meeting the KPIs is not easy. The following is the list of KPIs for each scenario. They are based on separate machines for BizTalk and the SQL Server instance, Intel Xeon CPUs with multiple cores, x64 bit platforms and software (refer to the Test Environment table).
KPIs for the scenario of Messaging a single-message and multi-message box are stated in the following table:
|
Number of BTS |
CPU BizTalk Server |
Number of SQL Servers |
CPU SQL Server |
Msg/Sec Received |
Msg/Sec Processed |
|---|---|---|---|---|---|
|
1 |
1 Quad |
1 |
1 Quad |
160 |
200 |
|
1 |
1 Quad |
1 |
1 Quad |
280 |
350 |
|
1 |
1 Quad |
1 |
2 Quads |
390 |
490 |
|
1 |
1 Quad |
1 |
2 Quads |
560 |
700 |
|
2 |
1 Quad |
1 |
2 Quads |
620 |
770 |
|
2 |
2 Quads |
1 |
2 Quads |
730 |
910 |
|
2 |
2 Quads |
1 |
4 Quads |
780 |
980 |
KPIs for the scenario of an orchestration single-message box are stated in the following table:
|
Number of BTS |
CPU BizTalk Server |
Number of SQL Servers |
CPU SQL Server |
Msg/Sec Received |
Msg/Sec Processed |
|---|---|---|---|---|---|
|
1 |
1 Quad |
1 |
1 Quad |
110 |
140 |
|
1 |
1 Quad |
1 |
1 Quad |
170 |
210 |
|
1 |
1 Quad |
1 |
2 Quads |
190 |
240 |
|
1 |
1 Quad |
1 |
2 Quads |
220 |
270 |
|
2 |
1 Quad |
1 |
2 Quads |
230 |
290 |
|
2 |
2 Quads |
1 |
2 Quads |
260 |
320 |
|
2 |
2 Quads |
1 |
4 Quads |
300 |
370 |
Each has a Windows 2008 Enterprise x64 Edition as the operating system, with 64 bit CPUs having four cores:
|
Type |
Model |
CPU Type |
Number of CPU |
Logical Disks |
Software |
|---|---|---|---|---|---|
|
Database |
DL875 |
Intel Xeon |
8 * 2,4 Ghz |
2 x 72gb 10k* |
SQL Server 2008 SP1 |
|
BTS Host Receive |
R805 |
Intel Xeon |
2 * 2,33 Ghz |
2 x 72gb 10k SAS |
BizTalk Server 2009 |
|
BST Host Send |
R805 |
Intel Xeon |
2 * 2,33 Ghz |
2 x 72gb 10k SAS |
BizTalk Server 2009 |
|
Load |
R805 |
Intel Xeon |
2 * 2,33 Ghz |
2 x 72gb 10k SAS |
BizTalk Benchmark Wizard |
|
Backend |
R805 |
Intel Xeon |
2 * 2,33 Ghz |
2 x 72gb 10k SAS |
Indigo Service |
Following is the configuration of the test environment for Storage: EMC Clarion CX-240 (five solid state drives):
Global tracking enabled
Partitioning of the
TempDb(SQL Server system database),BizTalkDTADbandBizTalkMsgBoxDbto as many files as CPUsSeparation of the BizTalk MessageBox database into multiple file groups or files
Disabled throttling on send and processing hosts
Updated the thread settings (CLR Hosting) on all hosts (there you will find the update install path of
BBW artefacts\registry settings)
The Performance Analysis of Logs (PAL) tool is a powerful tool that reads in a performance monitor counter log (any known format) and analyzes it using known thresholds. Basically, it automates the analysis of performance counter logs, for instance, one that is generated by the Benchmark Wizard. At the end, you can generate an HTML or XML-based report that graphically charts important performance counters and throws alerts (in red), when thresholds are exceeded. The thresholds are originally based on thresholds defined by the Microsoft product teams, including the BizTalk Server, and members of Microsoft support.
Before PAL, it was quite hard to analyze logs, which had to be done manually and required quite extensive knowledge of the Windows Architecture. It could also require a lot time to analyze the logs and so investment in money. There are tools that could help in the analysis such as the Microsoft System Center Operation Manager, but might not collect critical data that is necessary or the Microsoft Server 2008 Performance toolkit, which is limited for Windows 2003. There was not really a tool that could analyze log file(s) easy and fast. PAL does just that, it analyzes logs and does not require much time, and is free. This tool has the following benefits:
Consolidated guidance, central repository of guidance gathered from multiple sources
Log file data access layer using the Microsoft Log Parser
Analyzes more data points, breaks down the data into smaller slices, and analyzes them individually
Dynamically changing thresholds, learning environment asking questions
Reusable (open source); code is open source
Extensibility of thresholds; add, edit, or delete thresholds
The PAL tool is available through CodePlex version 2.0.8 (http://pal.codeplex.com/) and it requires the Microsoft Log Parser. The latter is a powerful, versatile tool that provides universal query access to text-based data such as log files, XML files, and CSV files, as well as key data sources on the Windows operating system such as the event log, the registry, the filesystem, and the Active Directory Service. Log Parser version 2.2 is available at http://www.microsoft.com/downloads/en/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en. Other prerequisites are as follows:
Microsoft .NET Framework 3.5 Service pack 1
Microsoft Chart Controls for Microsoft .NET Framework 3.5
PowerShell 2.0
You can start PAL from <install path>\PAL\PAL v2.0.7\PALWizard, where the install path is presumably C:\Program Files.
You will now go through a Wizard, where the following steps will be performed:
Select the log files to analyze.
Select the time frame for the analysis.
Select the appropriate threshold file.
Answer questions.
Select the analysis interval.
Select the output options.
Execute or add to the queue.
Select the template.
With the following steps you can perform analysis of a BizTalk Benchmark Wizard log file:
With the PAL tool you can select, for instance, the Counter Log generated by the BizTalk Benchmark wizard. These files can usually be found at
<install path Benchmark Wizard>\Blogical\BizTalk Benchmark Wizard\COUNTERLOGS\<machine name>\<machine name>_BizTalk Server\<sequence number>\<machine name>or\<machine name>_SQL_Server\<sequence number>\<machine name>:
Next, you can select the appropriate Threshold File. You will select the Microsoft BizTalk Server 2006/2009/2010 file. You are able to edit this file if you wish to do so:
Next, you will find the Questions tab, where you need to answer questions on your environment regarding the memory, processor, and platform:
In the next tab, you will be able to select theOutput Options:
After determining the output options, you should select Output Directory and which format you want your report in—HTML or XML:
Now, you will be able to queue the analysis, if there is any. This can be done by using the Queue tab:
Finally, you can execute the analysis. After the analysis, you can open the report and investigae the possible issues.
PAL analyzes the performance log data using the BizTalk thresholds. The tool aids in performance analysis by automatically interpreting data in the log file instead of doing it manually. Thresholds used in the BizTalk threshold file include operating system thresholds (CPU, disk, memory, and network), Microsoft SQL Server counter thresholds, and BizTalk counter thresholds. Focus lies on host throttling, adapter latency, service instance statistics (suspended, dehydrated, and so on), database sizes, and memory usage.
The generated report is separated by categories and each category has a collection of analyses. Each analysis focuses on a specific performance counter. An alert is raised when thresholds are exceeded, by yellow (warning) or red (critical) color. When critical alerts are present, there will be a description of the analysis, a description of the thresholds used, and a link for more information on the topic.
Note
The analyses in the report contain content describing the purpose of the analysis, why the thresholds are there, and references for more information. To learn more about interpreting the PAL report for the BizTalk analysis, read the article available at (BizTalk 2006 related, but still applicable to 2010):
Besides the BizTalk instances, you can also analyze the SQL Server environment.
On MSDN blogs, you will find a story of around three parts using PAL for the SQL Server. The PAL tool is useful in analyzing bottlenecks in the BizTalk environment, but you can also analyze the performance in the SQL Server. For more details, refer to the following documents:
Performance Analysis of Logs (PAL) Tool: Part 1: http://blogs.msdn.com/b/temenosonsql/archive/2010/04/30/performance-analysis-of-logs-pal-tool-part-1.aspx
Performance Analysis of Logs (PAL) Tool: Part 2: http://blogs.msdn.com/b/temenosonsql/archive/2010/05/03/performance-analysis-of-logs-pal-tool-part-2.aspx
Performance Analysis of Logs (PAL) Tool: Part 3: http://blogs.msdn.com/b/temenosonsql/archive/2010/05/03/performance-analysis-of-logs-pal-tool-part-3.aspx
The creator of the PAL tool is Clint Huffman and he has a blog related to his tool at http://blogs.technet.com/b/clinth/.
The BizTalk Server and another Microsoft Server product, Host Integration Server (HIS), both support an extension of the Windows Enterprise Security integration called Enterprise SSO. You will notice that Enterprise SSO is one of the BizTalk features during installation. Enterprise SSO in total is provided by a set of processes that run on network servers to provide the following services for heterogeneous systems:
User account and password mapping and caching
SSO to multiple Windows domains and host security systems
Password synchronization to simplify administration
The services mentioned earlier are mandatory for the BizTalk Server, even if you do not require them. The BizTalk Server uses the SSO to help secure information for the receive locations. When the Enterprise SSO service gets started, it retrieves the encryption key called master secret from the Master Secret Server. The Master Secret Server is another Enterprise SSO service that has an additional subservice that distributes and maintains the master secret. What the Enterprise SSO service does is that it caches the master secret after it has been retrieved. Every 60 seconds the service synchronizes the master secret with the Master Secret Server.
As you can see, the Master Secret Server plays an important role like MSDTC (refer to the Configuring MSDTC for multi-server BizTalk platforms recipe later in this chapter). Regardless of whether you will use the Enterprise SSO service for credential mapping or not, it has to be available in any kind of BizTalk configuration.
With the Microsoft Management Console
(MMC) or command line ssomanage utility, you are able to manage the SSO system. With either of these tools, you can update the SSO database, adding, deleting, and managing applications, and administer user mappings. In the MMC, you will find all programs of your operating system. Refer to the following screenshot:
The command line ssomanage is available in C:\Program Files\Common files\Single Sign On. You will also find the ssoconfig command-line tool at the specified location, which is a utility to configure your password synchronization settings.
The following steps describe how to work with the ssomanage and ssoconfig commands:
You can start
ssomanagefrom the command line and with the commandssomanage -?. You will see all the functions, as shown in the following screenshot:
You can change the global information in the SSO database, such as the Master Secret Server identification, the account names, and so on. This information can be updated by using the
–updatecommand providing theupdatefile containing this information. Refer to the following command line:ssomanage –updatedb <update file>, where <update file> is the path and name of the fileThe
updatefile (xml) will have the following format:<sso> <globalnfo> <ssoAdminAccount>YourDomain\Accountname</ssoAdminAccount> <ssoAffiliateAdminAccount> YourDomain \Accountname</ssoAffiliateAdminAccount> <secretServer>ServerName</secretServer> <auditDeletedApps>1000</auditDeletedApps> <auditDeletedMappings>1000</auditDeletedMappings> <auditCredentialLookups>1000</auditCredentialLookups> <ticketTimeout>2</ticketTimeout> <credCacheTimeout>60</credCacheTimeout> </globalInfo> </sso>The
ssoconfigcommand can be started from the command line and with the commandssoconfig -?. You will see all the functions again, as shown in the following screenshot:
One of the common commands used with
ssoconfigis the restoreSecret for restoring the SSO master secret as a part of the recovery scenario. For restoring the SSO master secret, you should type the following command:ssoconfig –restoreSecret <backup file>
The backup file has the name of the master secret file that you backed up during configuration.
See How to Update the SSO database document at http://msdn.microsoft.com/en-us/library/aa559867.aspx.
With the
ssomanage functions, you can find out, for instance, which SSO server is used, what is the SSO administrator account, and if everything is correctly enabled. ssomanage also plays a role during clustering of the Master Secret Server (http://msdn.microsoft.com/en-us/library/aa561823.aspx ).
With the functions of ssoconfig, you can get to know where SSO database is created or upgraded, and also where the SSO master secret is restored in case it has become unavailable.
Besides the ssoconfig command-line tool, now there is also an MMC Snap-in available and you are able to troubleshoot SSO with command-line tools. Finally, you will find high availability options for a multi-machine BizTalk environment on Microsoft TechNet.
SSO configuration application MMC Snap-in: It provides the ability to add and manage applications, add and manage key value pairs, as well as import and export configuration applications so that they can be deployed to different environments. You can download the MMC Snap-in from Microsoft (http://www.microsoft.com/downloads/en/details.aspx?displaylang=en&FamilyID=94e07de1-1d33-4245-b430-9216979cd587).
It also provides a client-side class that makes accessing the SSO system to retrieve your key/value pairs easy.
Troubleshoot Enterprise SSO: To troubleshoot your SSO environment, it may be useful to walk through certain items described in the troubleshoot enterprise single sign-on table found on MSDN (http://msdn.microsoft.com/en-us/library/aa953861%28v=bts.70%29.aspx).
High availability installation options: Microsoft TechNet provides high availability options for Enterprise SSO in a multicomputer BizTalk deployment (http://technet.microsoft.com/en-us/library/aa578263%28BTS.70%29.aspx).
Through MSDN, you can find information on how to use SSO (http://msdn.microsoft.com/en-us/library/aa561654.aspx).
In a multiserver environment, the BizTalk Server runtime requires MSDTC. The runtime operations need MSDTC support to ensure that the operations are transactionally consistent. The runtime also depends upon some other components, such as SSO, BizTalk host instances, and any SQL Server instances that are connected to the BizTalk Server. In case MSDTC is not functioning properly, these components will get affected.
MSDTC is a component inside Component Services, as shown in the following screenshot:
As you can imagine, it is important in a multi-server environment to have MSDTC properly setup and configured. Especially, the security settings need to be set appropiately. This recipe will show how to use DTCPing and DTCTester to aid you in validating and troubleshooting the MSDTC configuration.
To validate the connection between the BizTalk Server and the SQL Server machines, you can use DTCPing and detect it if a firewall is blocking the access. Distributed Transactions (specifically OleTx transactions) use the Microsoft Remote Procedure Call (MSRPC) protocol to talk to MSDTC on the other machine. You have to make sure that the two machines are able to communicate with each other using the MSRPC protocol. With the DTCPing tool (http://www.microsoft.com/downloads/en/details.aspx?FamilyID=5e325025-4dcd-4658-a549-1d549ac17644&DisplayLang=en) running on both the machines, you can test whether the normal Remote Procedure Calls (RPC) communication is working or not. After testing, download and unzip the file to a destination folder.
Another tool is the DTCTester (http://support.microsoft.com/kb/293799) utility to verify transaction support between two computers, if the SQL Server is installed on one of the computers (multi-server BizTalk environment). The DTCTester utility uses ODBC to verify the transaction support against a SQL Server database.
After you have downloaded the tool, you can run the self-extracting file and place it somewhere on your machine. You will have to create an Open Database Connectivity (ODBC) data source for your SQL Server through the ODBC utility in the Control Panel. DTCTester is a 32 bit tool, and accesses the ODBC for the 32 bit client. If you have a 64 bit environment, do not use the ODBC client that is in the administration console. To access the 32 bit client, you will have to open up c:\windows\syswow64\odbcad32.exe.c:\windows\syswow64\odbcad32.exe.
The following steps describe how to work with DTCPing:
Run the DTCPing tool and in the Remote Server Name type in the right server name and click PING. If everything works fine, you should see the following message being returned by the tool:
The DTCPing only works with NetBIOS name.
Refer to the following screenshot:
From the second machine you would see that the ping has been received. Refer to the following screenshot:
From this machine you can ping back to the other machine and the test is complete. Refer to the following screenshot:
DTCTester can be started from the command line, where it has been installed. Execute the following from the command line:
dtctester <dsn name><user name><password>Replace the values in brackets as appropriate for your environment:
When running DTCPing, it will provide output of any success or error messages obtained when attempting to communicate between servers. A ping from one server will be sent to another server, where an instance of DTCPing is running. Result of the ping is displayed in a running instance of DTCPing and from this instance a ping is sent back to the other server. The result of this ping will also be displayed. A user can verify if there are issues with communication between the two servers.
To be able to verify if an actual transaction in a database can be performed, DTCTester comes into play. When running DTCTester, it establishes a connection to the SQL Server by using a Data Source Name (DSN). It also provides the username and password that you provide on the command line by using the default network library. Then, a temporary table is created and the connection is enlisted in the transaction. An insert on a temporary table is done and this transaction is committed. The transaction is verified by the execution of a select statement. Finally, the connection is closed.
There is a clear distinction between both tools. DTCPing focuses on connectivity and DTCTester on transaction. An analogy to these tools is that you could see DTCPing as authentication and DTCTester as authorization.
There are some resources regarding MSDTC and BizTalk you can benefit from:
Troubleshooting MSDTC issues with the DTCPing tool: If you encounter errors during testing with the DTCPing tool, then you can report it to the Distributed Services Support Team. It has a blog entry with details on possible errors and how to resolve them (http://blogs.msdn.com/b/distributedservices/archive/2008/11/12/troubleshooting-msdtc-issues-with-the-dtcping-tool.aspx).
Troubleshooting MSDTC issues in general: You will find more information on the Microsoft website on troubleshooting MSDTC at http://msdn.microsoft.com/en-us/library/aa561924%28v=bts.20%29.aspx.
BizTalk and MSDTC: You can get more information at http://soa-thoughts.blogspot.com/2010/01/biztalk-and-msdtc.html.
