BMC Control-M 7: A Journey from Traditional Batch Scheduling to Workload Automation
Formats:
save 20%!
save 37%!
Free Shipping!
| Also available on: |
|
- Implement and utilize a world class enterprise batch scheduling and workload automation tool in the best possible ways with this book and e-book
- Hands-on implementation and administration of a Control-M environment
- Easily develop Control-M job flows to meet simple and complex business requirements
- Written in a conversational manner and supported by a lot of practical, easy-to-follow examples accompanied by screenshots
Book Details
Language : EnglishPaperback : 534 pages [ 235mm x 191mm ]
Release Date : October 2012
ISBN : 1849682569
ISBN 13 : 9781849682565
Author(s) : Qiang Ding
Topics and Technologies : All Books, Enterprise
Table of Contents
PrefaceChapter 1: Get to Know the Concept
Chapter 2: Exploring Control-M
Chapter 3: Building the Control-M Infrastructure
Chapter 4: Creating and Managing Batch Flows with Control-M GUI
Chapter 5: Administrating the Control-M Infrastructure
Chapter 6: Advanced Batch Scheduling and Management
Chapter 7: Beyond Everyday Administration
Chapter 8: Road to Workload Automation
Index
- Chapter 1: Get to Know the Concept
- Introduce batch processing
- The history of batch processing
- Batch processing versus interactive processing
- Time-based batch- and event-driven batch
- Is this the end for batch processing?
- Running batch processing tasks
- Automating batch processing
- Basic elements of a job
- What to trigger
- When to trigger (Job's scheduling criteria)
- Dependencies (Job's predecessors and dependents)
- More advanced features of scheduling tools
- Ability to generate notifications for specified events
- Ability to handle an external event-driven batch
- Intelligent scheduling – decision-making based on predefined conditions
- Security features
- Additional reporting, auditing, and history tracking features
- Centralized enterprise scheduling
- Challenges in today's batch processing
- Processing time
- Batch window length
- Batch monitoring and management
- Cross-time zone scheduling
- Resource utilization
- Maintenance and troubleshooting
- Reporting
- Reacting to changes
- The solution
- Processing time and resource utilization
- Batch monitoring and management
- Cross-time zone scheduling
- Maintenance and troubleshooting
- Reporting
- Reacting to changes
- From batch scheduling to workload automation
- Batch scheduling: Static scheduling
- The Workload Automation concept
- Dynamic batch processing with virtualization technology and Cloud computing
- Integration with real-time system, workload reusability
- Summary
- Chapter 2: Exploring Control-M
- Control-M overview
- Control-M road map
- Key features
- Supported platforms
- The Control-M way
- Control-M job
- Submitting jobs
- Post processing
- From the user's perspective - Control-M/Enterprise Manager
- Control-M Enterprise Manager GUI Client
- Control-M Desktop
- Control-M Configuration Manager
- Reporting Facility
- Control-M's Optional Features
- Control-M Control Modules
- Control-M/Forecast and BMC Batch Impact Manager
- BMC Batch Discovery
- Control-M Architecture and Components
- Control-M/Enterprise Manager
- Control-M/Enterprise Manager Server Components
- Control-M/Server
- Control-M/Server processes
- Control-M/Agent
- AG: Agent Listener, Request Handler
- AT: Agent Tracker
- AR: Agent Router Process
- UT: Utility Process
- Agentless Technology
- Control-M/Control Modules
- How do Organizations Work With Control-M?
- Where to Start?
- General Product information
- Official Education and Certification
- Getting a Job in Control-M
- Summary
- Chapter 3: Building the Control-M Infrastructure
- Three ages to workload automation
- Stone age
- Iron age
- Golden age
- Planning the Batch environment
- Control-M sizing consideration
- Total number of batch jobs run per day
- Total number of job execution hosts
- Number of datacenters
- Amount of concurrent GUI users
- Use Control-M/Agent or go Agentless
- Production, development, and testing
- Control-M high availability requirements
- Control-M in a clustered environment
- Control-M/Server mirroring and failover
- Control-M node group
- High availability by virtualization technology
- Pre-installation technical considerations
- Environment compatibility
- Choices of database
- System configuration requirements
- Storage space related considerations for Control-M
- Firewall requirements
- Between Control-M/Enterprise Manager Clients and Server Components
- Between Control-M/Enterprise Manager Server Components and Control-M/Server
- Between Control-M/Server and Control-M/Agent
- Agentless remote hosts
- Database
- Last things to make sure of before the installation starts
- Installation
- Install Control-M/Enterprise manager server components
- Download and execute the check_req script
- Create a Linux user and allocate space for Control-M/EM
- Configuring the system to meet installation requirements
- Preparing the installation media
- Installation
- Post-installation tasks
- Install Control-M/Enterprise manager clients
- Preparing the installation media
- Installation
- Post-installation tasks
- Installing Control-M/Server
- Installation in Linux environment
- Installation in a Windows environment
- Installing Control-M/Agent
- Installation in Linux environment
- Installation in a Windows environment
- Summary
- Chapter 4: Creating and Managing Batch Flows with Control-M GUI
- The Control-M way – continued
- Contents of a job definition
- "What" #1: job type
- "What" #2: task type
- "Who" #1 – owner of the job
- "Who" #2 – author of the job
- "Where" #1 – job's execution host
- "Where" #2 – storing job definitions
- "When" #1 – job's scheduling date
- "When" #2 – time frame for job submission
- "When" #3 – cyclic jobs
- "When" #4 – manual confirmation jobs
- "When" #5 – job condition
- "When" #6 – resource and job priority
- "When" #7 – time zone
- What happens right after the job's execution is completed?
- Autoedit facility
- Lifecycle of a job
- Write/Load, Upload/Download, Order/Force, and Hold
- State of a job
- New Day Procedure (NDP)
- Active job ordering
- Active job cleaning
- Control-M Date and Odate
- User Daily
- Working with Control-M Desktop and EM GUI Client
- Control-M Desktop – the Workspace
- Control-M/EM GUI client – Active ViewPoint
- Defining and running jobs
- Creating the first job – Hello World!
- Write, Upload, and Order the job
- Monitor and Control the Job
- Modifying and rerunning the job
- A more complicated job flow
- Defining SMART table, application, and group
- Building cyclic jobs
- Utilizing the Autoedit facility
- Linking jobs with job conditions
- Defining Global Conditions
- Creating calendars
- Adding job post-processing and job steps
- Working with Resources
- Having a "Start" job
- Summary
- Chapter 5: Administrating the Control-M Infrastructure
- Additional component installations
- Installation of BIM and Forecast
- Installation
- Post-installation tasks
- Configuring BIM web interface
- Installation of Control Modules
- Pre-installation considerations
- Installation – Control-M for database
- Installation – Control Module for Advanced File Transfer
- Installation – Control-M Business Process Integration Suite
- Post-installation tasks
- Expanding and updating the batch environment
- Ongoing installation of Control-M/Agents and Control Modules
- Installing multiple Control-M/Agents on the same host
- Defining Agentless remote hosts
- Unix/Linux remote host (using SSH)
- Windows remote host (using WMI)
- Applying Control-M fix packs and patches
- When to apply fix packs and patches
- How to apply fix packs and patches
- Fix pack and patch installations in our environment
- Installing additional Control-M GUI clients
- Frequent administration tasks
- Stop/start components
- Manually stop/start components
- Configuring automatic startup script
- Defining additional GUI users and groups
- Authorization of configuration items
- Customizing Control-M GUI
- Summary
- Chapter 6: Advanced Batch Scheduling and Management
- Importing existing batch processing tasks
- Importing CRON jobs into our environment
- For Host ctm-demo-linux-01
- For Host ctm-demo-linux-02
- Enhance the file processing batch flow
- Control-M filewatch
- Technical background
- Defining filewatch job
- Advanced file transfer
- Technical background
- Implementing AFT jobs
- Control-M for database
- Technical background
- Implementing Database CM jobs
- Advanced batch management
- ViewPoints
- Viewing jobs in Active ViewPoint
- Performing job actions in Active ViewPoint
- The Time Machine – Archived ViewPoint
- Creating ViewPoint
- Forecasting
- Job scheduling plan forecasting
- Forecast ViewPoint
- Managing batch flows as services
- Defining services
- Monitoring services
- Control-M reporting facility
- Type of reports
- Creating a report
- Automated reporting
- The emreportcli utility
- Reporting job
- Summary
- Chapter 7: Beyond Everyday Administration
- GUI alternative—command-line utilities
- Control-M/Server utilities
- Control-M/Agent utilities
- Securing the environment
- User authentication: External authentication
- Configuring LDAP parameters
- Converting existing GUI users to authenticate with LDAP
- Associating EM user groups with LDAP groups
- User privilege: Control-M/Server security
- Defining group-level security
- Defining user-level security
- Enabling Control-M/Server security
- Job ordering and submission: User exit
- Job execution: Control-M/Agent security
- Security for Windows Control-M/Agents
- Security for Unix/Linux Control-M/Agents
- Control-M/Server utility authorizations
- Inter-component communication—firewall
- Between Control-M/EM server components and GUI clients
- Between Control-M/Server and Agents
- Inter-component Communication—SSL
- Implementing SSL
- Auditing
- Enabling and configuring auditing
- Producing auditing report
- Control-M mirroring and failover
- Pre-implementation tasks
- Installing and configuring the secondary Control-M/Server
- Configuring Control-M/Agents
- Testing the secondary Control-M/Server
- Initializing mirroring and failover
- Switching to mirroring and failover
- Switching over to mirroring
- Switching over to failover
- Recovering from mirroring and failover
- Recovering from mirroring
- Recovering from failover
- Perfecting Control-M
- Housekeeping
- Active environment-related housekeeping
- Database-related housekeeping
- Filesystem-related housekeeping
- Component status checking
- NDP tuning
- Things happening during NDP
- Shortening NDP
- Other configurations items
- Control-M/EM: MaxOldDay and MaxOldTotal
- Control-M/EM: Default AverageTime
- Control-M/Server: New Day Time
- Control-M/Server: Simple Mail Transfer Protocol parameters
- Control-M/Server: shout destination tables
- Summary
- Chapter 8: Road to Workload Automation
- Integrating Control-M with business processes
- Building the environment
- Interacting with BPI interfaces
- Technical background
- Defining an account
- Triggering job ordering
- Taking parallel processing to the next level
- Merging the two file processing job flows
- Enabling parallel processing
- Implementing control module for BPI jobs
- Technical background
- Defining accounts
- Creating jobs
- Updating the quantitative resource
- End-to-end testing
- Managing batch jobs as workloads
- Running jobs in node groups
- Technical background
- Creating node groups
- Making necessary changes to the environment and jobs
- Putting into action
- Defining and managing workloads
- Technical background
- Defining workloads
- Putting into action
- Into the Cloud
- Technical background
- Defining accounts
- Defining jobs
- Modifying the file processing job flow
- Defining CM for Cloud jobs
- End-to-end testing
- Summary
Qiang Ding
Submit Errata
Please let us know if you have found any errors not listed on this list by completing our errata submission form. Our editors will check them and add them to this list. Thank you.
Errata
- 10 submitted: last submission 19 Feb 2013Errata type: typo | Page number: Preface (page 18) | Errata date: 19-2-2013
The 1st paragraph reads: Control-M is one of the world's most widely used enterprise class batch workload automation product produced by BMC Software. With a strong knowledge of Control-M, you will be able to use the tool to meet ever growing batch processing needs. However, there has been no book that can guide you to implement and manage this powerful tool successfully until now. With this book, you will quickly master Control-M, and be able to call yourself "a Control-M" specialist! The 4th paragraph reads: You will start off with the history and concept of batch processing and how recently it got evolved into workload automation, and then get an understanding of how Control-M fits into the big picture. Then we will look more in depth at the technical details of the tool - How to plan, install, use, as well as manage a Control-M environment, and finally look at how to leverage the tool to meet the already sophisticated and ever growing business demand. Throughout the book, you will learn important concepts and features of Control-M through detailed explanations and examples, as well as learn from the author's experience, over many years. By the end of the book, you will be set up to work efficiently with this tool, and also understand how to utilize the latest features of Control-M.
Errata type: typo | Preface (page 19) | Errata date: 19-2-2013
The 4th paragraph reads: Chapter 4, Creating and Managing Batch Flows with Control-M GUI , looks at the important job scheduling concepts of Control-M in depth and applies them by defining some simple jobs in Control-M Desktop, and manages them using the Control-M/EM GUI client. Then it goes one step further by defining a complete batch to meet a scheduling requirement. The 5th paragraph reads: Chapter 5, Administrating the Control-M Infrastructure , starts with installing the additional Control-M components – BIM and Forecast, followed by discussing the tasks involved in expanding and updating the Control-M environment. It talks about the different methods of performing regular installation tasks, such as applying fix packs and installing Control-M/EM GUI clients, as well as demonstrates how to define Agentless remote hosts in both Linux and Windows environments. Towards the end, it explains how to perform some must-know administration tasks, including stop/start Control-M components, define Control-M/EM user authorizations, and customize the GUI.
Errata type: typo | Preface (page 20) | Errata date: 19-2-2013
The 1st paragraph reads: Chapter 7, Beyond Everyday Administration, focuses on the administration side of Control-M but in more depth. It starts with looking at the command-line utilities that can be used to affect the active environment. More importantly, it reviews the different security options provided by Control-M, as well as demonstrates the Control-M mirroring and failover. After securing the environment, it takes us to the new level by perfecting the Control-M environment. The 2nd paragraph reads: Chapter 8, Road to Workload Automation , makes a number of improvements to the file processing job flow from both the integration and performance aspects, by using the cutting-edge Control-M add-on features. These include enabling the job to have exposure to external applications, by using the BPI web service interface, integration with ESB with BPI web service and message queue jobs, rendering the processing truly parallel, and implementing load balancing. Towards the end, it turns our processing flow into workloads, and taps into the power of cloud computing for limitless processing. Errata type: typo | Preface (page 21) | Errata date: 19-2-2013 The 2nd last paragraph reads: To send us general feedback, simply send an e-mail to feedback@packtpub.com, and mention the book title in the subject of your message.
Errata type: typo | Preface (page 25) | Errata date: 19-2-2013
The 1st heading reads: An introduction to batch processing The last paragraph, 2nd last sentence reads: In this manner, lots of data could be processed at once, rather than frequently re-stacking the program card multiple times for each small amount of input data. Errata type: typo | Preface (page 27) | Errata date: 19-2-2013 The 1st pargraph reads: Traditionally Batch jobs are known to process large amounts of data at once. Therefore, it is not practical to expect the output to be given immediately. But there is still a predefined deadline for every batch processing, either it is set by the business requirement (also known as SLA – Service Level Agreement) or simply because it needs to finish so that the dependent batch processing tasks can start. For example, a group of batch jobs of an organization need to generate the payroll data and send it to the bank by Monday 5am, so the bank can have enough time to process it and ensure the funds get transferred to each of the organization's employee's account by 8am Tuesday morning. Errata type: typo | Preface (page 28) | Errata date: 19-2-2013 The 1st paragraph, last sentence reads: CICS ( Customer Information Control System) is a typical mainframe application designed for handling high-volume online transactions. On the other hand, there is personal computer started to gain popularity which designed and optimised to work primarily in interactive mode. The 2nd pargraph reads: In reality, we often see that batch processing and transaction processing share the same computing facility and data source in an enterprise class computing environmnet. As interactive processing aims at providing a fast response for user requests generated on a random basis, in order to ensure that there are sufficient resources available on the system for processing such requests, the resource intensiv batch jobs that used to occupy the entire computing facility 24/7 had to be set to run only during a time frame when user activities were low, which at the time was is more likely to be during night. This is why we often hear a more seasoned IT person with mainframe background call it an overnight batch. Errata type: typo | Preface (page 29) | Errata date: 19-2-2013 The last bullet point reads: 7:00am: The system starts to get hit by online traffic again. The 2nd last paragraph reads: In this example, programs for batch mode processing are set to run only when online transactions are low. This allows online processing to have the maximum system resources during its peak hours. During online processing's off peak hours, batch jobs can use up the entire system to perform resource-intensive processing such as sales reconciliation or reporting.
Errata type: typo | Page 13 | Errata date: 19-2-2013
The 2nd paragraph reads: The timeframe allocated for batch processing is called a batch window. The concept sounds simple, but there are many factors that need to be taken into consideration before defining the batch window. Those factors include what time the resource and input data will be available for batch processing, how much data needs to be processed each time, and how long the batch jobs will take to process them. In case the batch processing fails to complete within the defined time window, not only does the expected batch output fail to be delivered on time, but the next day's online processing may also get affected. Here are some of the scenarios: The 1st bullet point under the 2nd paragraph reads: Online requests start to come in at its usual time, but the backend batch processing is still running. As the system resources such as CPU, memory, and IO are still occupied by the over-running batch jobs, the resource availability and system response time for online processing are significantly impacted. As a result, online users see responses given slowly and get timeout errors. The 3rd bullet point under the 2nd paragraph reads: In extreme cases, online transactions are based on the data processed by the previously run batch. Therefore, the online transactions cannot start at all unless the previous batch processing is completed. This can happen with banks, as you often hear them say the bank cannot open tomorrow morning if the overnight batch fails. Errata type: typo | Page 14 | Errata date: 19-2-2013 The 1st paragraph, 3rd sentence reads: Using the online shopping example again; during the day, orders get generated by online users. The 1st paragraph, last sentence reads: With event-triggered batch processing, the online business is able to offer a customer a near instant response on their order status, and therefore, provide a better shopping experience. The 3rd paragraph, last sentence reads: However, it's yet to completely replace batch processing. Errata type: typo | Page 16 | Errata date: 19-2-2013 The 1st paragraph reads: Whether running the business process in real time or batches, the system designer should take many factors into consideration, such as: The 2nd paragraph, last sentence reads: Large orgnisations today are still investing in batch processing and continuously trying to figure out how to make it run better. The last paragraph, first two sentence read: To understand batch processing further we need to begin from its source - mainframe computers. Job Control Language ( JCL ) was introduced as a basic tool on the mainframe computers for defining how a batch program should be executed. A JCL is considered to be a job, also called a job card (name inherited from punched cards).
Errata type: typo | Page 19 | Errata date: 19-2-2013
The 1st bullet point reads: By 12:00am, there are ten orders generated in total. Program PROCESS_ORDER is set to trigger at this time of the day to process them. For each order, the program scans the inventory to make sure each item on the order is in stock and creates the file called daily build list. The 2nd bullet point reads: According to the daily build list, the program MAIL_CONFIRMED_ORDER sends out an e-mail to each customer to confirm if his or her order is ready to be built. The 4th bullet point reads: Program GENERATE_BACKORDER_LIST will create a list of orders that cannot be built due to missing parts; the list also gets stored in a file called backorder list. Errata type: typo | Page 20 | Errata date: 19-2-2013 The 1st paragraph, 2nd sentence reads: Jobs are scheduled o that they can run immediately one after another, and keep the computing resource occupied without idle time. The 1st paragraph, last sentence reads: Using our online computer shop example by breaking down processing into tasks and running tasks that are not interdependent in parallel, we can s ignificantly shorten the processing time. In the screenshot, the left-most vertical heading reads "35 hours to process" and Step 2 in teh 1st block reads: Send Order confirmation email Errata type: typo | Page 21 | Errata date: 19-2-2013 The 1st paragraph reads: As the modern computing batch processing is far more complicated than just simply feeding punched cards in sequence into the mainframe as it was in old days, a lot more factors need to be taken into consideration when running batch processing due to its time consuming and task-intensive nature.Batch scheduling tools were designed to automate such processing tasks, thus reducing the possibility of human mistake and security concerns. The 1st bullet point in the last paragraph reads: What, how, and where to trigger – The executable program's physical location on the file system Errata type: typo | Page 23 | Errata date: 19-2-2013 The 2nd paragraph, 1st sentence reads: A many to one relationship refers to one child job that depends on many parent jobs (for example, to perform an end of job stream backup or data transmission). Errata type: typo | Page 24 | Errata date: 19-2-2013 The 1st paragraph reads: Driven by business and user demand, more sophisticated features have been provided with modern scheduling tools apart from automating the execution of batch jobs, such as: The 3rd bullet point in the first paragraph reads: Intelligent scheduling – decision making based on pre-defined conditions The last paragraph reads: Event-driven batch jobs are defined to run only when the expected external event has occurred. In order to have this capability, special interfaces are developed within a batch scheduling tool to detect such an event. Detecting a file's creation or arrival is a typical interface for event trigger batch. Some batch schedulers also provide their own application programming interface ( API) or have the ability to act as a web service or through message queue to accept an external request. The user needs to specify beforehand what event to trigger, which job or action within the batch scheduling tool, and during what time frame. So during the defined time frame, the scheduler will listen to the event and trigger the corresponding batch job or action accordingly. External events can sometimes be unpredictable and this can happen at any time. The batch scheduling tools also need to have the ability to limit the number of concurrent running event-triggered batch jobs to prevent the machine from overloading during peak time periods. Errata type: typo | Page 25 | Errata date: 19-2-2013 The 1st paragraph, last sentence reads: With this feature, a lot of the repetitive manual actions for handling events can be automated, such as: The 2nd paragraph reads: This feature avoids the human response time for handling such tasks and minimized the possible human mistakes. It significantly contributes to shortening the overall batch processing time. However, this is not a one-size-fits-all approach, as there are chances that the events really need a human decision to take place. This approach can free the user from repetitive tasks, but can also increase maintenance overhead, such as each time when the output of a program is changed, the condition for automatic reaction more likely needs to be changed accordingly. The last paragraph, 2nd last sentence reads: Batch scheduling tools eliminated this concern by providing additional security features from the batch processing prospective, that is, providing user authentication for accessing the batch scheduling console, and grouping users into different levels of privileges according to their job role. Errata type: typo | Page 26 | Errata date: 19-2-2013 The 1st paragraph, 1st sentence reads: In recent years, IT has become more and more sophisticated to meet the ever-growing business requirements. Errata type: typo | Page 27 | Errata date: 19-2-2013 The last paragraph, 1st sentence reads: Most of these machines will have their own batch jobs running to serve a particular need.
Errata type: typo | Page 32 | Errata date: 19-2-2013
The 1st paragraph reads: When jobs are running on different platforms, they can be monitored on a machine by machine basis only. The user can see which job completed and which job failed, but unable to see everything as a complete business process flow. Many business processes today require thousands of jobs to complete and these jobs may be spread on hundreds of machines. It is not practical for operators to track each step of the batch flow by logging on every machine that is involved in the processing, not only because this approach is labor intensive, but also because different skill sets are required for people who are in charge for batch jobs running on each different environment. Also, it is difficult to find out what the consequences would be if one job needs to be started late or if some jobs need to be disabled for a given day. Errata type: typo | Page 33 | Errata date: 19-2-2013 The 2nd paragraph reads: Just think from the maintenance point as well, all these failures were caused by a rename to the CSV files. Without seeing the whole picture, the person who made the modification did not know there's a downflow reporting job, or that many other parties outside his department may rely on these files for further processing. The last paragraph, 3rd sentence reads: Without an overall view of the entire batch environment from a business process point of view, along with a lack of standardization and documentation, IT will become a hindrance to the business transition. Errata type: typo | Page 34 | Errata date: 19-2-2013 The 3rd paragraph, 2nd last sentence reads: Once the job is submitted, the centralized scheduling platform will wait for the response from the remote host and, in the meantime, submit other jobs that are meeting their scheduling criteria. Errata type: typo | Page 35 | Errata date: 19-2-2013 The 1st paragraph, 3rd last sentence reads: In this case, jobs can be triggered immediately when its parent job(s) are completed. The last paragraph reads: We often hear from more senior IT people talking about their "good old days" - when batch operators had a list of all jobs expected to be executed on the day, together with a list of machines where these jobs were located. They had to manually logon each machine to check the previous job's completion state, then logon another machine to execute the next job, Centralized scheduling allowed users to monitor and manage cross platform batch jobs from a single point of control. Users no longer need to estimate which job should run next by looking at their spreadsheet, because jobs that belong to a single business process are grouped into a visual batch flow. Users can see exactly where the execution is up to in the batch flow. Centralized scheduling also provides a uniform job management interface. The people in charge of managing the batch jobs no longer need to have in-depth knowledge of the job's running environment to be able to perform simple tasks such as rerun a job, delay a job's execution, or deploy a new job. Errata type: typo | Page 39 | Errata date: 19-2-2013 The 1st paragraph reads: Based on what we know so far, in this section we will be looking at where batch scheduling is going in the very short term future. The 2nd paragraph, 1st sentence reads: As discussed earlier in this chapter, for many years, batch scheduling was primarily static scheduling. The 3rd paragraph, last sentence reads: Even if these idle machines are allowed to be used for temporary batch job execution; it is not worth the effort to re-configure the batch jobs to run on these hosts only as a one off thing. Errata type: typo | Page 41 | Errata date: 19-2-2013 The 1st paragraph reads, 1st sentence: Virtualization technology has become common in IT environments in the recent years. Errata type: typo | Page 43 | Errata date: 19-2-2013 The 2nd paragraph reads, last sentence: This system may low rapid changes to be made, but users are more likely to suffer poor performance when it comes to processing large amounts of data at once. The spelling in the tip box is Wikipedia and NOT Wikipida. The last paragraph reads, 3rd last sentence: The same idea can also be applied to a data backup process. Errata type: typo | Page 44 | Errata date: 19-2-2013 The 1st paragraph reads, last sentence: A request can be sent to trigger the batch processing through web services or messaging queue when needed, then the batch scheduler runs the batch processing in black box and return the desired outcome to the requester. Errata type: typo | Page 45 | Errata date: 19-2-2013 The last paragraph reads: Later on in this book, you will see how the challenges are addressed and how the ideal workload automation concept is achieved in the amazing feature of Control-M. Stay tuned! We are just about to let the real game begin!
Errata type: typo | Page 48 | Errata date: 19-2-2013
The 3rd paragraph, 2nd last sentence reads: In the recent years, BMC software made Control-M its BSM strategy by integrating Control-M with BMC Atrium Configuration Management Database (CMDB). Errata type: typo | Page 49 | Errata date: 19-2-2013 The last paragraph, last sentence reads: In the evolutionary release of version 7, Control-M enhanced its features by adopting the workload automation concept to become more resource-aware and gain ability to manage dynamic workload in real-time and take advantage of virtualization and cloud computing. Errata type: typo | Page 49 | Errata date: 19-2-2013 The 5th bullet point reads: According to the predefined rules, automatically perform actions when a job execution event has occurred (for example, a job failed with a certain return code, or the job didn't run at the expected time) Errata type: typo | Page 51 | Errata date: 19-2-2013 The 3rd paragraph, last sentence reads: Let's begin with understanding some basic Control-M concepts and terms. The last paragraph, 1st sentence reads: As we discussed in the previous chapter, most batch jobs are not standalone. Errata type: typo | Page 54 | Errata date: 19-2-2013 The 1st paragraph, 1st sentence reads: Control-M is well known for its powerful, post processing ability. In Control-M, post processing activities are the tasks to be performed automatically right after a job's completion. The 2nd paragraph, last sentence reads: The complexity of the heterogeneous batch environment is encapsulated to the people who manage the batch jobs, that is, without having platform-specific knowledge the user can define, monitor, and manage job executions on any of the supported systems through the centralized GUI front end in a uniform way. Errata type: typo | Page 55 | Errata date: 19-2-2013 The 1st paragraph, last sentence reads: Alternatively, users can manage jobs by exception, by using the Alert Window provided as part of the GUI Client. Errata type: typo | Page 55 | Errata date: 19-2-2013 The 2nd paragraph, 2nd sentence reads: It provides a uniform job editing interface to define and modify job definitions regardless of what operating system it is going to be running on. Errata type: typo | Page 58 | Errata date: 19-2-2013 The 2nd paragraph, 1st sentence reads: In order to monitor and control components, firstly users need to define the manageable components into CCM and set the Desired State for each of them. The last paragraph, 1st sentence reads: The Reporting Facility is the key tool for extracting batch related information from Control-M and analyzing the batch environment to support future decision making such as forecasting batch processing volume or planning and optimising batch processing duration. Errata type: typo | Page 59 | Errata date: 19-2-2013 The last paragraph, 2nd sentence reads: This feature allows pre-defined reports to be produced automatically via Control-M when the pre-defined command line is scheduled as a job. Errata type: typo | Page 60 | Errata date: 19-2-2013 The 3rd paragraph, 2nd bullet point reads: Constructing the command line execution for a particular application requires sophisticated arguments, some of which are only determined at run time, for example, locate and process a randomly generated file. The last paragraph reads: As an enterprise wide centralized scheduling platform, apart from scheduling standard jobs on the systems listed in supported platforms, with additional softwa package's support (Control-M Control Modules), Control-M can automate special job types such as file transfer, web services, messaging, java applications, as well as being able to expand its schedule capability into major applications such as SAP, Oracle E-Business Suit, Informatica, and PeopleSoft. Errata type: typo | Page 62 | Errata date: 19-2-2013 The 1st paragraph, 1st sentence reads: The purpose of Control Modules is to make application specific job scheduling easy, therefore necessary parts of the processing that run in those applications can be integrated with the traditional type batch jobs as much as possible to allow the end-to-end business process to be presented in Control-M. Errata type: typo | Page 61 | Errata date: 19-2-2013 The last paragraph, last sentence reads: Reacting to each individual job failure with the same approach without seeing the bigger picture may not be the most effective way to ensure critical services meet their SLA.
Errata type: typo | Page 63 | Errata date: 19-2-2013
1st paragraph, 1st sentence reads: Referring to the example we mentioned in the Control-M/Forecast section, now the 15000 interrelated jobs are designed to deliver 80 different business services. 2nd paragraph, last sentence reads: Batch Impact Manager also can trigger alerts and work in conjunction with BMC Remedy ITSM to automatically trigger an incident ticket. 3rd paragraph reads: Batch Impact Manager uses job statistical information in reference to the sequence of job execution within the batch service to calculate how much a job's delay (or completion quicker than usual) can affect the completion time of the business service, as well as suggesting how quickly a failed job needs to be fixed in order to ensure the business service completes within its SLA. Last paragraph reads: When is the best time window to schedule an outage change on a production machine that will bring minimal impact to critical batch services? What is the actual business impact if the outage change did not finish within agreed change window? These are the type of questions regularly asked by change management. Given that business processes today commonly require task intensive and complex cross platform batch processing, it can be very challenging to provide an accurate answer to these kinds of questions. By performing an outage change without knowing these facts, an organization may be at the risk of suffering business loss and getting a penalty for missing the SLA. Errata type: typo | Page 64 | Errata date: 19-2-2013 1st paragraph, 2nd sentence reads: Once all the batch services CIs and associated computer resources CIs are imported into the CMDB, the people who do not have visibility to the batch environment (for example, the change manager) can look up the CI of the computer resource that requires the outage to find out if there are any batch service CIs associated with it, and are therefore able to estimate potential impacts to the business services. 2nd paragraph, 1st sentence reads: By knowing the associated batch services, the users who have access to Control-M can use Control-M/Forecast to find out the best time window for an outage that will bring no impacts or minimal impact to the business service. Errata type: typo | Page 65 | Errata date: 19-2-2013 Last paragraph reads: With this highly scalable architecture, users can choose to run their batch environment on a single scheduling engine or divide the workload onto multiple scheduling engines, but still have the ability to monitor and manage the jobs within a single GUI—as long as every Control-M/Server is configured to communicate with the same Control-M/Enterprise Manager. Errata type: typo | Page 66 | Errata date: 19-2-2013 1st bullet point reads: Naming Service Errata type: typo | Page 67 | Errata date: 19-2-2013 2nd paragraph reads: Control-M Configuration Server component was introduced in Control-M/ Enterprise Manager 6.3.01 as a background service to support CCM GUI front-end and perform backend administration tasks like cleaning historical data from the database on regular intervals. 3rd paragraph, last sentence reads: Control-M/Enterprise Manager Configuration Agent does the start-up, shutdown, and recycle for these components according to the process Desired State flag that is set by the user in CCM. Errata type: typo | Page 68 | Errata date: 19-2-2013 1st paragraph, 2nd sentence reads: For GUI clients, it does most of the heavy lifting part of displaying jobs (reduce the workload on each GUI clients) as well as handling user requests that come from each active GUI client session. 2nd paragraph, 2nd sentence reads: The gateway process enables the communication between the two layers so jobs can be displayed on GUI clients in real time and user actions can be immediately passed to Control-M/Server. 3rd paragraph reads: A dedicated gateway process is required to be defined for each Control-M/Server to handle the communication. 4th paragraph reads: The Control-M/Enterprise Manager GUI Client provides an alert window for the user to track exceptions in the batch environment. Such exceptions can be a job's failure, a job has been running too long, or Control-M/Agent has gone unavailabe. The alert information gets transmitted from Control-M/Server to Control-M/ Enterprise Manager by the gateway process. The global Alert Server is in charge for processing and presenting these alert notifications. 5th paragraph reads: In a multi Control-M/Server environment, jobs residing in different Control-M/ Servers might be related and to be executed in a pre-defined log ical order. the Global Condition Server handles the job dependencies between jobs running on different datacenters (Control-M/Servers). Errata type: typo | Page 69 | Errata date: 19-2-2013 1st paragraph, 1st two sentences read: In Control-M, a job condition is a string defined as part of the definition and the Global conditions are distinguished from normal conditions by global condition prefix. Users needs to predefine the condition prefix, as well as its source datacenter and destination. Last paragraph, last sentence reads: Control-M/Server's built-in command line and menu driven utilities can be used as an alternative for job monitoring and management when Control-M/Enterprise Manager is temporarily unavailable. Errata type: typo | Page 69 | Errata date: 19-2-2013 1st paragraph, 1st sentence reads: SL process is one of the most critical processes in Control-M/Server. 4th paragraph reads: Apart from relying on Control-M/Agents to provide job updates in real time, TR process also performs a Track All periodically to all running jobs. This feature is to manage cases where job status updates are not received due to connection with Control-M/Agent being temporarily lost or TR process (or Control-M/Server) being down just at the time when the original job status updates were sent from Control-M/Agent. 5th paragraph, last sentence reads: It also maintains the communication with Control-M/Agents— checks if they are available or not, and update the status into Control-M/Server's job execution node availability record. Errata type: typo | Page 72 | Errata date: 19-2-2013 2nd paragraph reads: Firstly, it is in charge of generating information to be sent to Control-M/Enterprise Manager reflex active batch workload status changes happened in the Control-M/ Server. On the other hand, it also passes requests from Control-M/Enterprise Manager such as the user requests initiated from GUI Clients or when a global condition has been newly created. 3rd paragraph, 3rd sentence reads: From the user's point of view, they may obtain better response time during peak usage when multiple CS processes are presented. 4th paragraph, last sentence reads: The LG process handles these requests by triggering the actual process residing on the Control-M server and sending the execution result back to Control-M/Agent through the NS process. 5th paragraph reads: The WD process can do many different things, they are all optional.
Errata type: typo | Page 73 | Errata date: 19-2-2013
1st Paragraph reads: Thirdly, the WD process performs Control-M/Server User Exits. We will introduce "User Exits" in Chapter 6 , Advanced Batch Scheduling and Management. 2nd Paragraph reads: The RT process is also one of the most important processes within Control-M/Server however, it hardly requires our attention. It acts like a network router to reduce the number of communication channels between Control-M/Server processes, therefore reducing the complexity of Control-M/Server inter-process communication. It listens on an Internal Process Communication ( IPC ) port that allows all the Control-M/ Server processes we talked about to connect with each other. 3rd Paragraph reads: The CA process was introduced in Control-M 6.3.01. Its job role is similar to Control-M/Enterprise Manager Configuration Agent—to set the Control-M/Server Up or Down according to the desired state set by user in CCM. Of course it only communicates with the "master" - SU process and lets it do the work of bring other processes up or down. It also retrieves Control-M/Agent status and sends the information to Control-M/Enterprise Manager to be displayed on CCM. It allows users to recycle Control-M/Agents and change their configurations through CCM. 4th Paragraph reads: Control-M/Agent is a collection of processes that are constantly running on the job execution host to act on job submission requests from Control-M/Server. Control-M/ Agent submits the actual executable program to operating system for execution, monitors the program execution, and provides the execution result back to Control-M/Server. Control-M/Agent is also the base for Control Modules, as well as handling the communication between Control-M/Server and Agentless remote hosts. Errata type: typo | Page 74 | Errata date: 19-2-2013 3rd Paragraph reads: Before version 6.2.01, Control-M/Agent on the Unix/Linux systems relied on the operating system's INETD daemon for listening to server-to-agent communication. For each request that comes from Control-M/Server, the INETD process will create a new instance of the AG process to handle it. 4th Paragraph reads: We can tell the purpose of this process from its name—tracking job status. During jobs running, a record of each job is created within Control-M/Agent. The TR process is in charge for monitoring the actual job process and updating the status record. For completed jobs, the TR process will acknowledge Control-M/Server and mark the corresponding status record as completed on the Agent side. 5th Paragraph reads: The TR process also performs post processing related to job's output file, by scannin the file and searching for required strings for which Control-M/Server will later trigger post processing events. 6th Paragraph reads: The AR process is activated only when persistent connection mode is used. In a persistent mode, the AR process listens to the Server-to-Agent port and maintains the connection with Control-M/Server. Once a request is received, AR will route the request to the AG process. It also listens on requests from the AT process and routes them back to the job request's source— the Control-M/Server. Errata type: typo | Page 75 | Errata date: 19-2-2013 1st Paragraph reads: As we mentioned while introducing Control-M/Server's LG process, there are a number of Control-M/Server command line utilities that are allowed to be invoked from Control-M/Agent. The UT process is in charge for processing and submitting such utility requests from Control-M/Agent to Control-M/Server. 3rd Paragraph reads: From the user's point of view, defining and monitoring jobs running on an Agentless Remote Host has very little difference as compared to working with Control-M/ Agent hosts. From a management point of view, the connection status of an Agentless Remote Host can be monitored from CCM. Control-M also allows users to convert a Control-M/Agent host into an Agentless Remote Host or vice versa. Errata type: typo | Page 76 | Errata date: 19-2-2013 2nd Paragraph reads: In fact, all jobs submitted are handled by CMs. Normal jobs such as executing a command or running an executable are handled by the OS CM, which comes as part of Control-M/Agent . The Control Modules we discussed earlier are the application CMs—CM for SAP, CM for Oracle E-Business Suite, CM for PeopleSoft, and so on. Application CMs are extensions of Control-M/Agent, and therefore have to be installed on top of Control-M/Agent. In other words, Agentless remote hosts cannot handle application CM jobs. 3rd Paragraph reads: In an ideal environment, Control-M should be used as a centralized batch scheduling tool across the entire organization. These organizations are more likely to have a significant amount of complex batch processing running across multiple machines. To some extent, we could say that the higher the number of jobs that are running, the more complex they are (that is, dependencies, scheduling rules, and number of execution hosts involved), and the more RIO Control-M will be present. That's why we see so many Control-M customers that are banks, insurance companies, large manufacture companies, or telecommunication service providers that are Control-M customers. Errata type: typo | Page 77 | Errata date: 19-2-2013 last 3 sentences read: So if any errors occur while the job is running in production, the operators can perform a series of manual actions to the job or escalate the problem according to the instructions. Once the verification is done, th e schedulers will raise change request and schedule these jobs in production. Operators will then monitor and manage these jobs according to their severity and perform actions on failed jobs according to their runbook. Errata type: typo | Page 78 | Errata date: 19-2-2013 1st pragrapgh, last sentence reads: For failed jobs, the Administrators may have to work with the Developers and Schedulers for RCA ( root cause analysis ), to fix the problem, and to provide a summary to Incident Management and the Application owners. 3rd Paragraph, 2nd last sentence reads: The primary location for Control-M-related information is the BMC software website and some online communities. 4th Paragraph, last sentence reads: However, we will not be able to find any detailed Control-M technical information from these pages because they are only available under a secured website which requires sign up with company support ID. 1st bullet point of last paragraph reads: Control-X Yahoo group: The Control-X Yahoo group is the most well-known Control-M discussion group. It was founded in Nov 5, 1998. By the time this book is published, there will be over 1000 registered members. In order to join the group, one has to be a Yahoo member first then request access to the group. It normally takes two to three days for a new member's approval. URL: http://tech.groups.yahoo.com/group/Control-X/ Errata type: typo | Page 79 | Errata date: 19-2-2013 3rd bullet point at from the start of the page reads: LinkedIn groups: There are a number of Control-M related groups on LinkedIn. We will find that a lot of members are either working in BMC software or working at a Control-M customer site. There are also recruitment people who have joined these groups and they occasionally make job related posts. 1st paragraph reads: BMC Education Services offers various Control-M related courses and certification. There is a great chance that the training information has already been updated on the BMC site by the time this copy of the book is sold. Since the basic ideas and fundamentals are always the same, going through these courses will help us choose the right education path. Last paragraph reads: BMC only offers one Control-M related certification—BMC Certified Administrator—BMC Control-M for Distributed Systems 6.4. Certification is not a prerequisite for getting a Control-M-related job, but obtaining this certification will increase your career prospects and help you become more competitive in the field, especially if you are looking at working for a BMC partner company in certain countries. To prepare for the certification exam, you need to attend a BMC Education instructor-led training course first. The required courses for the exam are: Errata type: typo | Page 80 | Errata date: 19-2-2013 1st paragraph reads: The Boot Camp course is a five day instructor led course. It is well and truly a high-intensity course—9 to 11 hours a day, five days straight in a classroom with reading assignments for each evening. The class covers topics in the scheduling and administration courses plus hands-on labs for implementation related exercises. The certification exam is included in the book camp course. It is an online multiple choice exam, but it can only be taken at the local BMC training center. Students can take the exam twice within three weeks after the Boot Camp course, of course retests are only for those who failed the first time. 4th paragraph reads: With every BMC Control-M customer, there will be potential for a Control-M related position. The role might focus more on the functionality side such as schedulers and operators, or could be more towards the administration side or doing project work. I have seen some companies that do not have a dedicated Control-M administrator or scheduler/operator. The administration role maybe handled by a DBA or Unix/ Windows system administrator. In this case if we are looking for a DBA or system administrator role with some Control-M skill on the side it would be really helpful. Errata type: typo | Page 81 | Errata date: 19-2-2013 1st paragraph reads: A lot of people who are working on Control-M today are from mainframe background. These people may not have the best knowledge about midrange environment or scripting language, but they would have a good understanding of Control-M concepts. Since they have been working in the field for a long time, they also have good relationships and connections within the field t o get the help they need. On the other hand, there are fresh men who come from a Linux/Windows or DBA background and this group of people are good at getting their way around the system with installations and are much more confinable with writing scripts, but may find it hard to use the Control-M concepts and terminologies at the very beginning. 2nd paragraph reads: On the positive side, there is a reasonable amount of large organizations around the world who use Control-M and they treat Control-M as a very critical application. But this may create a problem too. It largely depends on our physical location. In some countries and cities there are more Control-M customers therefore more chance for a job vacancy. Whereas if someone lives in a city that has no Control-M customers at all, or only a very small number of Control-M customers, relocation needs to be considered. Final words for this topic are: just understand your own strength, be flexible, go with your passion and the job will come. Errata type: typo | Page 81 | Errata date: 19-2-2013 1st paragraph reads: We went through a lot of knowledge points during this chapter. In this chapter we looked at the features of Control-M, important concepts, and reviewed the architecture of Control-M. I hope you are not overwhelmed. As you start using and managing Control-M, you will become more familiar with the concepts and the architectural side of the things. And you might find it useful to re-visit this chapter later on during the installation of Control-M and troubleshooting issues.
Sample chapters
You can view our sample chapters and prefaces of this title on PacktLib or download sample chapters in PDF format.
- Understand the concepts of traditional batch processing and workload automation
- Plan and implement Control-M infrastructure by using the right technology in the right place
- Familiarize yourself with the GUI interface and define Control-M job flows to meet both simple and complex batch processing needs
- Get hands-on experience with various types of Control-M add-on modules (Control Modules), and understand how they can simplify and enhance existing batch flows
- Complete advanced administration tasks and performance tuning using best practices
- Take one step further into the world of workload automation by using the latest features available in Control-M
Control-M is one of the most widely used enterprise class batch workload automation platform. With a strong knowledge of Control-M, you will be able to use the tool to meet ever growing batch needs. There has been no book that can guide you to implement and manage this powerful tool successfully... until now. With this book you will quickly master Control-M and be able to call yourself "a Control-M" specialist!
“BMC Control-M 7: A Journey from Traditional Batch Scheduling to Workload Automation” will lead you into the world of Control-M and guide you to implement and maintain a Control-M environment successfully. By mastering this workload automation tool, you will see new opportunities opening up before you.
With this book you will be able to take away and put into practice knowledge from every aspect of Control-M – implementation, administration, design and management of Control-M job flows, and more importantly how to move into workload automation and let batch processing utilize the cloud.
You will start off with batch processing and workload automation, and then get an understanding of how Control-M meets these needs. Then we will look more in depth at the technical details of Control-M, and finally look at how to work with it to meet critical business needs. Throughout the book, you will learn important concepts and features, as well as learn from the Author’s experience, accumulated over many years. By the end of the book you will be set up to work efficiently with this tool and also understand how to utilize the latest features of Control-M.
This book is written in a direct to-the-point style to provide everything you need to start with Control-M. From this book, you will not only learn the technical aspects of Control-M, but also understand how things should be done in real environments and why things are done that way.
Each chapter of the book starts with a short description of what you will learn and a list of things you are expected to master by the end of it. The book can be read straight through, from cover to cover. It emphasizes the things you need to know, in a logical progression, along with advice for further reading or other tasks you might be interested in.
This book is suitable for professionals who are beginning to use Control-M, but who also have some general IT experience, such as understanding the concepts of computer system architecture, operating systems, databases, and basic computer networking. Some entry level skills in scripting languages will be helpful along the way.
For those who are from a mainframe background or moving from other batch scheduling tools, you can use this book as a starting point.

