The receive ports on the left side consist of adapters, the End Point Manager (EPM), and receive pipelines, and are responsible for bringing messages into BizTalk from a source system and storing them in the BizTalk MessageBox.
Orchestrations in the center are an optional feature of BizTalk in that you can choose to leverage orchestrations to process messages, as required. Messages that are processed by orchestrations are retrieved from the BizTalk MessageBox, and any messages sent from an orchestration are stored in the BizTalk MessageBox.
The send ports on the right side consist of adapters, the End Point Manager (EPM), and send pipelines, and are responsible for collecting outbound messages from the BizTalk MessageBox and transmitting them to the destination system.
Adapters provide connectivity for BizTalk to and from the outside world.Once an adapter has received the message, it’s then responsible for passing that message to the BizTalk Messaging Engine for processing. The Messaging Engine will process the message in a pipeline after receiving a message and before sending a message.
Subscriptions
Once a message has been processed by the adapter and associated pipeline, it needs to be routed to an interested party. A BizTalk orchestration or send port can register its interest in a specific message via subscriptions.
When a message is processed, metadata is derived from the message content, the transport adapter, and the BizTalk port that the message was received over and is written to the context of the message. These context properties are evaluated against the subscriptions in order to determine how to route the message.
These subscriptions are held within the Subscription table of the BizTalkMsgBoxDb SQL database and are evaluated when a message is published to the MessageBox by the BizTalk Message Agent, which is run as part of the BizTalk host.
Message Box
Once a message arrives at the Message Agent, subscribers are evaluated, the message is inserted, and references are then added into the host instance queue, ready for processing by the subscriber hosted by the host instance.
Publisher
Message Agent:-
Subscriber
Each host instance polls the database at a regular interval to look for new work in the queue. Dequeue threads, bts_DequeueMessages_<HostInstanceName> stored procedure is then used by these dequeue threads to retrieve messages for processing. Each call to bts_DequeueMessages will retrieve as many messages as possible for processing up to a configurable limit.
Why Isolated Host
There are several good reasons for this split. In-bound SOAP messages are handled by Internet Information Server (IIS), not the BizTalk Server engine, unlike other adapters. Also, you can think of messages being received from IIS as being far less trusted than those from other adapters; in many cases they will be received over an untrusted network, such as the Internet.
BizTalk Throttling:
Host Level:
A message is received through a receive location defined in a given receive port. This message is processed by the pipeline associated with the receive location, and if there are any inbound maps associated with the receive port they are executed. The resulting message is then published to the MessageBox database. The MessageBox evaluates active subscriptions and routes the message to those orchestrations, and send ports with matching subscriptions. Orchestrations may process the message and publish messages through the MessageBox to a send port where it is pushed out to its final destination.
BizTalk Server Jobs:
To fail over a service or application, RC the service or application in the left pane, point to Move it to another node and click to select the destination node.
BizTalk uses MSDTC for different communication and it’s therefor important that this is configured correctly, depending on the clustering method. For active / passive we have to cluster the DTC. But when it comes to an active / active cluster we need to make sure all active machines have their own versions of MSDTC running, it should not be a part of the cluster as a resources.
Trusted and Untrusted Hosts
If the MessageBox database receives a message from a host that you did not identify as authentication trusted, the MessageBox database will overwrite the PID with the guest ID, and the SSID with the service account that the host instance is running. BizTalk Server enables hosts identified as authentication trusted to indicate that the sender of a message that the trusted host is queuing to the MessageBox database is an entity other than the trusted host itself. The primary purposes of authentication trust are to enable pipelines to resolve to a Product ID (PID) and pass that PID along to consuming services for use in authorization and outbound party resolution, and to enable the transmission of the sender Windows Security ID (SSID) along to consuming services for use in orchestration action authorization.
A BizTalk host is a logical container representing one or more BizTalk Server run-time instances. Hosts are virtual process boundaries that provide an administrative and a security context for running instances of a BizTalk application. A BizTalk Host provides a container for items such as adapter handlers, receive locations (including pipelines), and orchestrations.
32 bit vs. 64 Bit Hosts
A 32-bit host instance can only address to a maximum of 2 GB. That limits the amount of memory that a process can take. Therefore, as a guideline, it is recommended to configure hosts as 64-bit hosts by not selecting 32-bit only as a 64-bit hosts can address a much higher memory space (up to 7 TB). However, the FTP, POP3 and SQL adapters are not supported to run on 64-bit host instances, and should be run in 32-bit host instances.
Host is nothing but the logical container of host instance. from which we can create host instance. Host instance is a WinNT service.
A BizTalk Server Host is a logical set of zero or more BizTalk Server runtime processes in which you deploy items such as adapter handlers, receive locations (including pipelines), and orchestrations.
A host instance is the process where the message processing, receiving, and transmitting occurs You install a host instance on each server running BizTalk Server that has one or more hosts mapped to that server.
Security Groups
When a host instance processes a message, it actually retrieves the physical message data from the Spool, MessageParts, Parts, and Fragments tables.
Storing only a reference to any message on the host queues enables each message to be processed by multiple hosts but be stored only once.
Orchestrations in the center are an optional feature of BizTalk in that you can choose to leverage orchestrations to process messages, as required. Messages that are processed by orchestrations are retrieved from the BizTalk MessageBox, and any messages sent from an orchestration are stored in the BizTalk MessageBox.
The send ports on the right side consist of adapters, the End Point Manager (EPM), and send pipelines, and are responsible for collecting outbound messages from the BizTalk MessageBox and transmitting them to the destination system.
Subscriptions
Once a message has been processed by the adapter and associated pipeline, it needs to be routed to an interested party. A BizTalk orchestration or send port can register its interest in a specific message via subscriptions.
When a message is processed, metadata is derived from the message content, the transport adapter, and the BizTalk port that the message was received over and is written to the context of the message. These context properties are evaluated against the subscriptions in order to determine how to route the message.
These subscriptions are held within the Subscription table of the BizTalkMsgBoxDb SQL database and are evaluated when a message is published to the MessageBox by the BizTalk Message Agent, which is run as part of the BizTalk host.
Message Box
Once a message arrives at the Message Agent, subscribers are evaluated, the message is inserted, and references are then added into the host instance queue, ready for processing by the subscriber hosted by the host instance.
Publisher
Message Agent:-
- Executes bts_InsertProperty to store Properties in MessageProps Table
- Executes bts_FindSubscriptions
- Executes bts_InsertMessage to insert the BizTalk message into the Spool SQL table, along with basic metadata about the message, such as the number of parts
- bts_InsertMessage calls int_InsertPart stored procedure, which places the raw body of the message into the Parts table.
- int_InsertPart calls int_EvaluateSubscriptions inserts a reference to the message into the relevant Host Queue SQL table, follows the naming convention <HostName>Q.
Subscriber
Each host instance polls the database at a regular interval to look for new work in the queue. Dequeue threads, bts_DequeueMessages_<HostInstanceName> stored procedure is then used by these dequeue threads to retrieve messages for processing. Each call to bts_DequeueMessages will retrieve as many messages as possible for processing up to a configurable limit.
Why Isolated Host
There are several good reasons for this split. In-bound SOAP messages are handled by Internet Information Server (IIS), not the BizTalk Server engine, unlike other adapters. Also, you can think of messages being received from IIS as being far less trusted than those from other adapters; in many cases they will be received over an untrusted network, such as the Internet.
BizTalk Throttling:
Host Level:
- Resource Based:- CPU, Memory, Severity
- Rate Based:- Publishing, Deliverying
- Orchestration Based:- Dehydration Behaviour, Time Based, Subscription
Host Instance:
- .Net CLR Setting :- Threading Setting (IO and Worker Threads)
- Orchestration Memory Throttling :- Physical and Virtual. (Optimal and Maximal Settings)
Message Life Cycle
A message is received through a receive location defined in a given receive port. This message is processed by the pipeline associated with the receive location, and if there are any inbound maps associated with the receive port they are executed. The resulting message is then published to the MessageBox database. The MessageBox evaluates active subscriptions and routes the message to those orchestrations, and send ports with matching subscriptions. Orchestrations may process the message and publish messages through the MessageBox to a send port where it is pushed out to its final destination.
BizTalk Server Jobs:
- Backup BizTalk Server:
- Full database backups of the BizTalk Server databases.
- Backs up the BizTalk Server database logs.
- Specifies for how long the backup history is kept.
- Monitor BizTalk Server:Scans for any known issues with the BizTalkMgmtDb, BizTalkMsgBoxDb, and BizTalkDTADb databases. This includes orphaned instances.
- MessageBox_Message_ManageRefCountLog_BizTalkMsgBoxDb: Manages the reference count logs for messages and determines when a message is no longer referenced by a subscriber. This job runs in an infinite loop and deletes the entries from the two individual message reference count logs. This job also calls the MessageBox_Message_Cleanup_BizTalkMsgBoxDb job. The job will always display a status of Failure after the first failure.
- MessageBox_DeadProcesses_Cleanup_BizTalkMsgBoxDb: Detects when a BizTalk Server host instance (BTSNTSvc.exe) has stopped responding. The job then releases the work from the host instance so a different host instance can finish the tasks.
- MessageBox_Message_Cleanup_BizTalkMsgBoxDb: Removes all messages that are not referenced by any subscribers in the BizTalkMsgBoxDb database tables.\
- MessageBox_Parts_Cleanup_BizTalkMsgBoxDb: Removes all message parts that are no longer referenced by a message in the BizTalkMsgBoxDb database tables. All messages are composed of one or more message parts that contain the message data.
- PurgeSubscriptionsJob_BizTalkMsgBoxDb: Purges unused subscription predicates from the BizTalkMsgBoxDb database.
- Rules_Database_Cleanup_BizTalkRuleEngineDb:Purges old audit data from the Rule Engine (BizTalkRuleEngineDb) database every 90 days,his job also purges old history data (deploy/undeploy notifications) from the Rule Engine (BizTalkRuleEngineDb) database every 3 days.
- TrackedMessages_Copy_BizTalkMsgBoxDb:Copies the message bodies of tracked messages from the BizTalkMsgBoxDb database to the Tracking (BizTalkDTADb) database.
- TrackingSpool_Cleanup_BizTalkMsgBoxDb:Purges inactive tracking spool tables to free database space.
- Full Back Up Job
- Custom DB backup, if required.
- Custom databases that you want to back up, which you can find at C:\Program Files (x86)\Microsoft BizTalk Server 2013\SchemacBackup_Setup_All_Tables.sql and Backup_Setup_All_Procs.sql. This creates the necessary procedures, table, and role and assigns permissions to the stored procedure.
- Go to the adm_OtherBackupDatabases table from the BizTalkMgmtDb database and add a row for each of your custom database you want to add to the backup job.
- Enable Log shipping [BizTalk Server Administrator group]
- Process that allows SQL server to move transactional log from one SQL server to another standby SQL server.
- To create the infrastructure for Log Shipping. These scripts are located in the Schema subfolder for BizTalk (%SystemDrive%\Program Files\Microsoft BizTalk Server 2013\Schema) LogShipping_Destination_Schema.sql and LogShipping_Destination_Logic.sql then execute bts_ConfigureBizTalkLogShipping stored procedure.
- BTS Log Shipping Get Backup History
- BTS Server Log Shipping Restore Databases
- BTS Log Shipping Restore To Mark(Disabled)
To fail over a service or application, RC the service or application in the left pane, point to Move it to another node and click to select the destination node.
BizTalk uses MSDTC for different communication and it’s therefor important that this is configured correctly, depending on the clustering method. For active / passive we have to cluster the DTC. But when it comes to an active / active cluster we need to make sure all active machines have their own versions of MSDTC running, it should not be a part of the cluster as a resources.
Trusted and Untrusted Hosts
If the MessageBox database receives a message from a host that you did not identify as authentication trusted, the MessageBox database will overwrite the PID with the guest ID, and the SSID with the service account that the host instance is running. BizTalk Server enables hosts identified as authentication trusted to indicate that the sender of a message that the trusted host is queuing to the MessageBox database is an entity other than the trusted host itself. The primary purposes of authentication trust are to enable pipelines to resolve to a Product ID (PID) and pass that PID along to consuming services for use in authorization and outbound party resolution, and to enable the transmission of the sender Windows Security ID (SSID) along to consuming services for use in orchestration action authorization.
A BizTalk host is a logical container representing one or more BizTalk Server run-time instances. Hosts are virtual process boundaries that provide an administrative and a security context for running instances of a BizTalk application. A BizTalk Host provides a container for items such as adapter handlers, receive locations (including pipelines), and orchestrations.
32 bit vs. 64 Bit Hosts
A 32-bit host instance can only address to a maximum of 2 GB. That limits the amount of memory that a process can take. Therefore, as a guideline, it is recommended to configure hosts as 64-bit hosts by not selecting 32-bit only as a 64-bit hosts can address a much higher memory space (up to 7 TB). However, the FTP, POP3 and SQL adapters are not supported to run on 64-bit host instances, and should be run in 32-bit host instances.
Host is nothing but the logical container of host instance. from which we can create host instance. Host instance is a WinNT service.
A BizTalk Server Host is a logical set of zero or more BizTalk Server runtime processes in which you deploy items such as adapter handlers, receive locations (including pipelines), and orchestrations.
A host instance is the process where the message processing, receiving, and transmitting occurs You install a host instance on each server running BizTalk Server that has one or more hosts mapped to that server.
Security Groups
- SSO Administrators
- SSO Affiliate Administrators
- BizTalk Server Administrators
- BizTalk Server Operators
- BizTalk Application Users
- BizTalk Isolated Host Users
BRE Execution Cycle:
BizTalk business rule engine (BRE) is a wonderful tool to extend dynamics and flexibility of business. It allows business rules to be tailored on the fly with no hazards and downtime.
Business rules execution cycle works in three stages –
Match
BizTalk business rule engine (BRE) is a wonderful tool to extend dynamics and flexibility of business. It allows business rules to be tailored on the fly with no hazards and downtime.
Business rules execution cycle works in three stages –
Match
- Conditions of all rules are evaluated together at once.
- If there are conditions which are shared across rules then those conditions are evaluated only once.
- If some condition is based on fact and that particular fact is not supplied then that rule is ignored.
- In this stage, rules are checked against their predefined priority to establish exact sequence of execution. Priorities are put on rules when you desire one rule to be executed before another. Rules with higher priority number are executed first. Rules with same priority number can be executed in any sequence.
- During conflict resolution, the sequence of rule execution alters in agenda based on their priority number.
- Actions in the resolved rules (as queued in agenda) are executed in this stage. Actions belonging to a rule are treated as a block and executed from top to bottom. Then next rule in picked according to sequence in agenda.
Two uses of ESSO
ESSO is used to securely store send port and receive location configuration information for BizTalk. Stores encrypted user credentials, which can then be transmitted to various systems.
----------------------
The Master Secret Server
SSO uses the SSODB database to store all encrypted credentials. Information is encrypted and decrypted from this database using a key called the master secret key.
The following figure shows a BizTalk Server database tier with high availability through active/passive server clustering.
----------
SQL old adapter relied on using Updategrams for update, insert and delete statements against tables and required SQL statements or
stored procedures to use the FOR XML mode to return results so that the data was returned to BizTalk Server in XML format. Updategrams
caused both BizTalk maps and schemas to be more complicated than necessary
The new adapter does not require any of these extra steps and creates the schemas that follow the standard Insert, Update, Delete, and
Select statements against tables and views and also to execute Stored Procedures and Functions.
---------------------
Do not deploy BizTalk Applications using old MSI files (MSI generated in BizTalk 2006, R2 or 2009), first, migrate the project,
rebuild the project using VS 2010 and then deploy the Application.
Depending on your needs, options or opportunities, you should consider using the WCF adapters:
WCF-SQL Adapter instead of SQL Adapter
WFC-WSHttp or WCF-BasicHttp instead of using SOAP Adapter.
-------------------------------
BTS.soap_envelope_1__1.Fault
When a host instance processes a message, it actually retrieves the physical message data from the Spool, MessageParts, Parts, and Fragments tables.
Storing only a reference to any message on the host queues enables each message to be processed by multiple hosts but be stored only once.
When a message is routed only to one subscriber, once it has finished processing a reference will immediately be inserted into the MessageZeroSum table,which is used to mark the physical messages for clean up by a job running within SQL Server Agent. This means that these types of messages will be cleaned up very quickly from the MessageBox database.
For messages going to multiple subscribers, once the message finishes processing, the BizTalk service will not remove the message from the database or insert a reference in the MessageZeroSum table, because it does not know whether there are any more subscribers to the message (for example, if it is referenced in another queue). The reference counts (or number of subscribers) for these messages are maintained by a combination of the following tables: MessageRefCountLog1, MessageRefCountLog2, and MessageRefCountLogTotals. The ActiveRefCountLog table is also used as an auxiliary table.
A job running within SQL Server Agent is used to aggregate the reference counts in these tables and determine which messages can be deleted. The Message IDs that can be deleted are then inserted into the MessageZeroSum table, from which another SQL Server Agent job will clean them up.
