Business Process Management with JBoss jBPM

Chapter 1. Introduction

Business Process Management is one of the hottest topics in the fast-moving world of business analysis and enterprise application development. Yet, it is curiously difficult to pin down as a defined field of work. You don't see job listings for "Process Developer" and there are few, if any, official courses that you can take in Business Process Management.

The answer to this conundrum lies in the almost accidental way in which BPM has come about, and in the speed with which the technology marketing machine swings into action these days: usually before the technology is properly understood. Business Process Management is at the start of the "hype curve" and it will be some time before its key concepts become common currency among enterprise managers.

We will try to cut through this hype and the resultant barriers to adoption by presenting a practical step-by-step approach to the successful implementation of business process management. We won't spend a great deal of time on theory in this book; instead we will concentrate on building something of value to your business. Having said that, we won't simply throw together any old business process management system, we will advocate a project lifecycle approach, so that we implement business process management in the right way.

So what is Business Process Management anyway? Well, hopefully you are coming to this book with some idea of the answer to that question. However, Business Process Management, or BPM as we shall call it henceforth, means different things to different people. Each person's definition probably has some element that falls in the intersection of a Venn diagram of definitions, at the centre of which is the truth. One of the first things we have to do is to define what BPM means for us, so that we may set expectations about what will be achieved by reading and implementing the suggestions in this book.

This introductory chapter will lay the ground work for the rest of the book. In it, we shall cover:

The business process management approach to developing software
What a business process is and why you want to manage it
Typical business scenarios ripe for BPM
How this book will work:
- The solution we'll build
- Our suggested project lifecycle
- Our example business scenario
- Our example BPM suite

The BPM approach to software development

Business Process Management is the natural evolution and convergence of several powerful forces within the fields of software development methodology, enterprise application technology, and management theory. These underlying forces have all matured and converged at the right time for a productive fusion, which we know as business process management.

Evolution of software development methodologies

Traditional software development methodologies owe much to their engineering roots. The waterfall approach to software development was designed with the idea that building a piece of software is like building a bridge: the better your design and blueprint, the sturdier the end result. In reality, this approach falls very far short of perfection.

Developing enterprise application software is about delivering value to a business, and the business expresses that value as a set of business requirements. The problem with the waterfall approach, and the difference from bridge construction, is that unlike the laws of physics and the construction properties of metal and concrete, business requirements are subject to change. Businesses cannot afford to stay still: if they don't adapt to the marketplace then they will not survive. So business requirements are necessarily a shifting target.

Unfortunately, this is not the only problem with the traditional software development methodologies. There is also the problem of business requirements "dissonance". This is where the layers of end users, analysts, and developers create a chain of Chinese whispers, resulting in software that fails to resemble the original requirement. Each link in the chain puts its own interpretation on the requirement, until the end result is horribly different from what the business originally needed. This requirements dissonance can easily be visualized:

In recent years, the traditional waterfall approach to software development has been superseded by other, more adaptable methodologies. These methodologies attempt to break down the requirements dissonance by taking out the middle man as much as possible, and by creating prototypes early on, and then iterating them towards the final version. This allows for an iterative approach to software development, far removed from the "build a bridge" traditional approach:

The most prominent of these newer development methodologies is Agile. On the right project, there is no doubt that Agile development can deliver valuable software more successfully, and more quickly, than the waterfall approach.

Nevertheless, Agile development and its ilk do have serious drawbacks and limitations. The first and most obvious limitation is that the Agile development methodology does away with the Business Analyst. This is an important drawback, because often the BA's interpretation of the requirements is more logical and more far-sighted than that of the end user who specifies the original requirement. This can mean that the developer can be led up blind alleys by an end user who doesn't have the necessary perspective.

There is also the problem that although we are removing some layers of interpretation, the layer of interpretation that we are leaving in place is the one that causes the most significant dissonance: the developer still has to interpret what the end user means.

This can mean that time is unnecessarily wasted on honing a prototype that starts off a long way from what the business needs. Indeed, some Agile developments have turned into one extremely long prototyping process, with an end result never being reached. This is an expensive way to develop software.

So what is the ideal, and where is Business Process Management in relation to this? For some idealists, the best situation would be one where the business users can build the software tools they need for themselves, without having to rely on developers or analysts. Unfortunately, although programming languages are becoming simpler all the time, we are still light years away from them being abstracted enough for an end user to build their own software. Software development is still hard.

Nevertheless, BPM does go some way towards this ideal, and given the right scenario, it can successfully deliver valuable software in extremely short time scales. In a similar fashion to Agile, BPM relies on cutting out the middle man as much as possible, only this time the emphasis is on a strong partnership between the end user and the BA working on iterations towards the final software:

The reinstatement of the Business Analyst has several advantages:

Firstly, the BA is skilled in the interpretation of requirements, and so their business process models are likely to be close to the original requirement.
Secondly, a BA's models are far easier for an end user to understand than code or even prototype software, allowing for closer collaboration and faster development.
Thirdly, the BA has the long term view and business skills to steer the end user's expression of their requirements in the most beneficial direction.
Last, and by no means least, models can usually be produced much more quickly than working software. As the working software produced by a BPM system is initially generated from the BA's process model, this is an extraordinarily fast method of software development.

Don't be tempted to think that this means developers are no longer required, however. The reality of BPM development is that it makes the working relationship between end user, BA, and developer much more symbiotic and productive, but does not make any of those roles redundant. BPM is a partnership approach to software development; on one hand between the end user and the BA, and on the other between the BA and the developer. The skills of a developer are very much still required to take a BPM system all the way to implementation. Where a business process calls for the integration of other systems, that integration work will almost certainly involve an interface built by a developer. And while the software that is generated by the BPM suite is good, it does still require some development to make it properly fit for purpose.

It would be foolhardy to suggest that the BPM approach is the right one in every software development scenario, but it is a formidable new challenger to other development methodologies. Later on in this chapter, we'll consider some of the scenarios where a BPM approach is the most appropriate.

The emergence of key technologies

Workflow software has been around since the early 90s, if not before. These systems were most often used in document management scenarios, where a document (for example, an insurance claim form) was passed between different departments as work was done on it. This worked well because the workflow system only had to maintain a pointer to the document in order to pass it down the process chain. Where things got more difficult was when the workflow system met other, task-specific systems.

Most mature processes involve the coordination of several systems. For example, we might have one system to record our insurance claim, another to work out what payment is due on the claim, and yet another to make the payment to the end customer. Before the advent of internet technologies, and specifically XML, it would have been a mammoth and fearsomely complicated task to integrate and tie together these task-specific systems and their proprietary programming languages and data formats within the context of a process. This was quite often attempted, however, and the result was usually a development and maintenance nightmare. More code ended up being written to handle the interfaces than was actually needed to process the work.

Thankfully, XML emerged from the internet revolution as a simple way for systems to talk to each other without them having to know about each other's proprietary data formats. Many task-specific systems now implement XML web services, making the task of integrating them into a process relatively simple.

As a result, business process management can certainly be viewed as a repackaging of the workflow software that was available in the 90s, but the reality is that those old tools could never have delivered the same value as BPM, because the technology landscape has fundamentally changed in the interim.

Meanwhile—management theory

The third leg of the tripod that has raised BPM up to its prominent position is the focus on process in management theory since the 1980s.

What is a business process and why do we want to manage it?

What do we mean by "business process"? We typically mean a collection of business activities that takes one or more kinds of input and creates an output that is of value to the business. It is the focus by management theorists on the elements of this definition that has led us to BPM. This quotation from W. Edwards Deming, founder of the quality movement, is illuminating:

"If you can't describe what you are doing as a process, you don't know what you're doing."

Any business process improvement project is an attempt to answer the fundamental question of "How do we organize our activities so that we can minimize inputs, maximize outputs, and maximize value?".

Business process improvement and re-engineering

There are several strands of management theory that are built around this fundamental question, and there are some striking examples where these theories have been effectively put into practice. Think of Jack Welch, who turned General Electric from a struggling manufacturing company to a highly profitable service-based company. Amongst other initiatives, this successful transformation can be attributed to radical business process re-engineering, and adoption of Six Sigma quality practices. Think too of Michael Dell, whose company of the same name changed the playing field of PC making and retail through a relentless focus on process improvement and ruthless process efficiency.

In business process re-engineering and improvement thinking, processes are viewed as organizational building blocks with as much (if not more) significance as functional areas and geographic territories. Business process re-engineering emerged in the 1980s with the idea that sometimes radical redesign and reorganization of these process building blocks was necessary to lower costs and increase the quality of service, and that IT was the key enabler for that radical change. The trouble with this radical approach is that it is too difficult to achieve in the real world. Mature organizations often simply cannot wipe the slate clean, and re-organize themselves without the instinctive memory of past processes and procedure creeping back in. Ultimately, business process re-engineering initiatives came to be viewed as nothing more than a cover up for downsizing efforts.

Business process improvement initiatives have been more successful, although they have been hampered by the lack of a comprehensive solution. Good-quality process design would be let down by sketchy IT support that couldn't be adapted. A business process would be designed around system constraints rather than systems doing exactly what the process required.

Nevertheless, many of the elements of business process improvement have proven to be useful and have not been discarded. Business process modeling has certainly increased businesses' ability to understand their operations and to make rational decisions about how best to organize their activities. Also, the definition and measurement of process metrics have given concrete, meaningful, and achievable targets for managers to work towards. The business is now more involved than ever before in the specification and delivery of IT programs.

From this convergence, BPM emerges

BPM is the final piece of the puzzle that allows business process initiatives to be fully successful. BPM espouses the incremental approach of business process improvement, but the IT delivery phase is supported by custom-designed tools that reduce the effect of requirements dissonance by allowing the delivery to be driven by the business.

In its simplest form, workflow software is generated from the process maps that are modeled by the Business Analyst. This workflow software is then the end user's "front end" to the process, and it controls the execution of the process in the live environment. Other software is then used to report on the operation of the process within the workflow software, allowing for dashboarding of key performance indicators. These dashboards can in turn be used to drive ongoing process improvement decisions.

Business process management isn't just one piece of software or one analysis technique: it is a suite of software, a framework of analysis techniques, and a defined project lifecycle. The Business Analyst, with their unique perspective on both business and technology, are in the happy position of having the right relationships and the right skill set to drive BPM initiatives in the enterprise.

Business process management: a definition

So now that we understand the background to BPM, it's about time we attempted a definition:

Note

Business Process Management involves the graphical modeling of a business process, from which workflow software can be generated, which in turn will control the live operation of the process, interacting with both humans and other applications. Further software measures the execution of the process in the live environment in order to permit ongoing analysis and iterative improvements.

Key benefits of BPM

The buzzwords and hype that are currently circulating around BPM are presenting serious barriers to adoption. What's needed is a clear expression of the benefits of BPM. BPM delivers efficiency, control, and agility to the business that implements it in the right way. These three key areas of promised benefit can be further broken down as:

Increases in productivity and effectiveness—a BPM system's task list makes sure that everyone is always working on the highest priority item, speeding the process along.
Increased process compliance and governance—users of a BPM system have no choice but to follow the process that the system is built on.
A more agile business that can change and adapt more quickly—because a BPM system is driven by a process model rather than by pure code, generally it is easier to effect system change, and therefore business change.
Increased ability to scale best practices across a changing organization—once defined and built, a BPM system doesn't care if it has 10 or 100 users. Organizations that try to scale out a ten-man operation to a 100 person one often run into difficulties because the process becomes so difficult to control without software support.
Improved communication, cooperation, coordination, handoffs—BPM systems are all about moving work from one team to another, reducing the need for teams to be skilled in communication and cooperation.
Improved resource utilization—resources that aren't pulling their weight are very visible to management because everything that happens in the process can be reported on.
Improved visibility of process pipeline—managers can easily report on everything that is in the course of being processed.
More accurate operational forecasts—because managers have such good visibility of their process pipeline, they can more easily plan their operations.
Greater process throughput—a well-oiled process running at maximum efficiency means that it will produce more of whatever the process is designed to produce.
Higher quality output—because process compliance is assured, and because the process was designed in line with best practice, it stands to reason that the output of that process will be of high quality.
Shorter process cycle times—with everybody who is involved in the process working at maximum efficiency, the total time it takes to run the process from start to finish will be reduced.
Minimized cost of inputs—because the process that underpins the BPM system has been defined and because the BPM system leads the process actors through that process, there is a reduced need for high quality, high cost staff to ensure the process runs smoothly.
Lower total process cost—the reduction in cycle time, the improvement in quality, and the minimized cost of inputs ensure that the total cost of running the process is reduced.
Faster new hire ramp-up—all new hires need to do is follow what the BPM system tells them to do.
More satisfied customers—the BPM system ensures customers get a higher quality good or service more quickly, and more consistently than they would otherwise.

Typical business scenarios ripe for BPM

Another barrier to adoption of BPM is a lack of understanding of the problems that BPM is designed to solve. Despite the persuasive benefits listed above, we must be clear from the beginning that BPM isn't the right solution in every circumstance. The following scenarios are good indicators of when BPM might be an appropriate solution:

People don't comply with a defined process.
The pipeline of work is unpredictable despite consistent customer demand.
The actors in the process don't have meaningful targets for how much or how fast they need to process.
Processes are carried out by disparate teams.
Elements of the process have been outsourced.
A business's reliance on a particular process has grown very quickly and best practice has not been adopted properly.
Task-specific systems are not coordinated, causing breakdowns in the process chain.

Similarly, there are clearly situations where BPM is not appropriate:

BPM is not appropriate for task-specific, procedural requirements: for example, calculating tax on an invoice.
The business is so small that controlling the process would impose a disproportionate burden on its operation.

How this book works

This book is a full toolkit for someone who wants to implement BPM in the right way. This toolkit is particularly aimed at Business Analysts, although Project Managers, IT managers, developers, and even business people can expect to find useful tools and techniques in here. We will present the project framework, analysis techniques, and templates, BPM technology and example deliverables that you need to successfully bring a BPM solution into your organization.

The book itself is structured to reflect the project lifecycle that we advocate. Each chapter represents a phase in the project. Each chapter will talk through the theory involved in that phase, explain the techniques or the technology, and then show you how it is done with an example. Every chapter has specific deliverables that fit in with the respective project phase, and these deliverables will be worked through in the example. Templates for the deliverables and the working example can be found in the download for this book.

The solution we'll build

As we go through the project phases, we will put together our example BPM system. The process that we will manage will be a realistic scenario and the solution could be used in real life. The BPM system we'll build will be stand alone, without proper interfaces to other systems, although we will simulate an interface, so we show how it can be done. The solution could certainly be developed much further, and in the final chapter we'll see some pointers for how this could be done, but even without further development, the solution is fully working and useful. The most important thing is that we go through the project steps so that the solution we build is functional and effective.

Introducing our suggested project lifecycle

The book, and our suggested project lifecycle, is divided into six distinct phases:

Understand the target process—to start off, we need to scope our target process, put together our project team, and then set about analyzing the process and building our first model for business sign off.
Develop the process—now that we have our process model, we need to install our BPM suite and build our model within it.
Prototype the process workflow user interface—once we've developed the process model in the BPM suite, we can generate a prototype user interface in order to run a proof of concept with our users.
Iterate the workflow prototype—our proof of concept will turn up numerous process changes and user interface requirements that we need to capture, prioritize, and implement.
Pilot and implement the workflow—we can now run a full-scale user acceptance test, and develop our key performance indicators that we'll track in the last phase. We can then put our process live.
Ongoing process improvement—now that the process is in the live environment, we can monitor its execution and investigate opportunities for further improvement.

Introducing our example business scenario

Any business process can be modeled, but some processes are more suited to business process management than others. For our worked example, the process we will use will be drawn from the music recording industry: "Produce music products". As we'll see, this process fulfils many of the criteria we defined above for a business scenario that is apt for a BPM solution. It also gives us the opportunity to demonstrate all the capabilities of our BPM suite, so that you can adapt the solution for your own processes.

Introducing our example BPM suite

There are many BPM vendors in the marketplace at the moment, and many of them offer the full suite of tools that we are looking for. One option is to bring in a vendor straightaway, but given the barriers to adoption that we've discussed in this chapter, it is likely that a BPM project will have to prove that the concept is valid and the solution can achieve a return on investment before a vendor is engaged.

Fortunately, there are some open-source tools available to us that will, at least, allow us to prove the concept for minimal investment, and in fact are certainly good enough to provide a solution that is comparable to the best that the vendors have to offer. For our BPM suite we need a graphical process modeler, a workflow user interface generation tool, a workflow application server, and a process metric reporting tool. All of the following tools are free to download and use.

JBoss jBPM

The first element in our open-source BPM suite is the jBPM development environment provided by JBoss. This Integrated Development Environment, or IDE, is based on another open-source tool, Eclipse, which is widely used for Java development. The JBoss jBPM IDE gives us not only our graphical process modeler, but can also generate the workflow user interface for us. Here is a screenshot showing the user interface of the IDE:

JBoss

The next element in our BPM suite is the JBoss Application Server, which is used to serve our workflow application to our end users. This workflow application is effectively a website, which the end users use to complete and record their process tasks. This is how it looks:

SeeWhy business intelligence platform

Finally, once we have our process in operation, we need to measure its execution. Our final tool in our BPM suite is therefore the SeeWhy Business Intelligence reporting toolset. SeeWhy is administered and configured over the Web using a browser-based console called the Desktop:

The second element of the SeeWhy platform is the Navigator, which is where our users will actually view the reports created by the platform. Here is a screenshot of the SeeWhy Navigator showing some business process metric reporting in action: