Data | 0 articles | Tech News, Tutorials & Expert Insights

article-image-sap-netweaver-accessing-mdm-system

28 Sep 2011

4 min read

SAP Netweaver: Accessing the MDM System

28 Sep 2011

(For more resources on SAP, see here.) Accessing an MDM server involves mounting and unmounting operations which we discuss in the following section. Mounting and unmounting an MDM server MDM server installations are accessible on the console only after they have been mounted. Multiple servers can be mounted within a single console session. We have a choice of mounting only those servers which need to be accessed. The server may or may not be in a running state when mounted in your console session. No password is required to mount a server in your console session even if it is password protected. The MDM Console provides the option of saving the list of currently mounted servers to an MDM Console Settings file. We can load this settings file in the console session and automatically get the previously saved server(s) list mounted as a group. An MDM server can be mounted by multiple MDM Consoles. Once an MDM server is started from any console, it runs on the machine where it is installed and is seen as running by all MDM Consoles that have mounted it. We can mount an MDM server as follows: Right-click on the root node (SAP MDM Servers) in the hierarchy pane tree and choose Mount MDM Server… from the context menu: Alternatively you many select the root node (SAP MDM Servers) and choose MDM Servers | Mount MDM Server… from the main menu: MDM opens the Mount MDM Server dialog prompting for the MDM Server input as displayed next: In the drop-down list input the region displaying the text Enter or select an MDM Server, type the name of the MDM server (typically the name of the machine on which the server is running) you want mounted or select it from the drop-down list. Alternatively (for non-Windows installations), type the name or IP address of any remote machine into the edit box in the Mount MDM Server dialog. Click on the OK button: The drop-down list of MDM Servers shows only those servers that you have previously mounted. If a specific server is not in the list, click on … (Browse) button to open the Select MDM Server dialog (see next) and select the machine on which the MDM Server has been installed from the list of Windows machines visible on the local network. On successful mounting of the MDM server, you will see a new server node added in the tree structure of the hierarchy pane. Depending on the state of the MDM server, the corresponding icon is displayed in the tree node. The different states and the respective icons of the server node are listed in the following table: Status icon State of MDM server MDM server is stopped MDM server is running MDM server is in one of the following states*: Server Inaccessible Communication Error Start Server Failed Invalid If the MDM server is inaccessible via the console even after the server has been started, you can try unmounting and remounting the MDM server in the console to restore connectivity. Next we see how to unmount an already mounted MDM server: In the hierarchy tree, right-click on the MDM server that you want to unmount and choose Unmount MDM Server from the context menu. Alternatively, you may unmount the server by first selecting its node in the tree and then clicking on MDM Servers | Unmount MDM Server from the main menu. Unmounting an MDM server is also possible by using the MDM Servers pane (top-right) when the root node (SAP MDM Servers) is selected in the hierarchy tree. Then you can right-click on the MDM Server in the objects pane and select Unmount MDM Server from the context menu. The MDM server node disappears from the tree in the hierarchy pane. Unmounting a running MDM server while it is still running keeps the MDM repositories mounted and loaded even while the unmounted server remains disconnected from the console session. Unmounting and again re-mounting an MDM server within the same MDM Console session requires the MDM server's password to be re-entered to perform any server-level operations (like starting and stopping the server).

0
0
2205

article-image-sap-netweaver-mdm-scenarios-and-fundamentals

Packt

16 Sep 2011

11 min read

SAP NetWeaver: MDM Scenarios and Fundamentals

Packt

16 Sep 2011

11 min read

0
0
2725

article-image-nhibernate-3-creating-sample-application

Packt

06 Sep 2011

8 min read

NHibernate 3: Creating a Sample Application

Packt

06 Sep 2011

8 min read

0
0
4616

article-image-oracle-e-business-suite-entering-and-reconciling-bank-statements

Packt

23 Aug 2011

4 min read

Oracle E-Business Suite: Entering and Reconciling Bank Statements

Packt

23 Aug 2011

4 min read

Oracle E-Business Suite 12 Financials Cookbook Take the hard work out of your daily interactions with E-Business Suite financials by using the 50+ recipes from this cookbook. Entering bank statements Bank statements are downloaded from the bank to a local directory. Once the file is received, the bank account balance and statement information can be loaded into the bank statement open interface tables, using the bank statement loader program or a custom loader program. The files can also be loaded automatically using an interface program or using the XML Gateway. Bank statements can also be entered manually. In this recipe, we will look at how to enter bank statements. Getting ready The bank statement shown next has been loaded into the open interface table: Let's review the transactions in the open interface: Select the Cash Management responsibility. Navigate to Bank Statements | Bank Statement Interface Lines. Select 95-6891-3074 in the Account field. Click on the Lines button to view the transactions in the interface tables. How to do it... Let's list the steps required to automatically enter the bank statements from the import and AutoReconciliation program: Select the Cash Management responsibility. Navigate to Other | Programs | Run, or select View | Requests from the menu. Click on the Submit a New Request button. Select Single Request from the Options. Click on the OK button. In the Submit Request form, select Bank Statement Import & AutoReconciliation from the list of values. Please note that we could run the Bank Statement Import program, to run only the import. Select the Parameters field, and select Kings Cross as the Bank Branch Name, select 95-6891-3074 as the Bank Account Number, and select 20110314-0001 as the parameter for the Statement Number From and the Statement Number To fields. Accept the default values for the remaining fields. Click on the OK button. We can schedule the program to run periodically, for example, every day. Click on the Submit button to submit the request. Let's review the imported bank statements: Navigate to Bank Statement | Bank Statements and Reconciliation. The imported statement is displayed. Click on the Review button. (Move the mouse over the image to enlarge it.) In the Bank Statement window, select the Lines button. The imported lines are displayed. How it works... Bank statements can be imported automatically, using a SQL*Loader script against the bank file to populate the bank statement open interface. The bank statement information is then imported into the Bank Statement windows using the Bank Statement Import program. There's more... Now, let's look at how to enter statements manually. Entering bank statements manually Let's enter the bank statement for the 15th of March manually. The lines on the statement are as follows: Payment of 213.80. Receipt of 3,389.89 from A.C. Networks. Credit of 7,500.00 for Non Sufficient Funds for the receipt from Advantage Corp. Bank Transfer payment of 1,000.00. Select the Cash Management responsibility. Navigate to Bank Statement | Bank Statements and Reconciliation. (Move the mouse over the image to enlarge it.) In the Reconcile Bank Statements window, click on the New button. In the Account Number field, enter 95-6891-3074, the other details are automatically entered. In the Date field enter 15-MAR-2011. In the Statement Number field enter 20110314-0002. In the Control Totals region, let's enter control totals based on our bank statement. The Opening Balance of 125,727.21 is entered based on the previous opening balance. In the Receipts field, enter 3,389.89 and 1 in the Lines field. In the Payments field, enter 8,713.80 and 3 in the Lines field. The Closing Balance of 98,495.56 is entered automatically. Let's enter the bank statement lines: Click on the Lines button. (Move the mouse over the image to enlarge it.) In the Bank Statements Lines form, enter 1 in the Line field. Select Payment as the Type. Enter 100 as the code. In the Transaction Date field, enter 15-MAR-2011. In the Amount field, enter 213.80. Select the next line, and enter 2 in the Line field. Select Receipt as the Type. Enter 200 as the code. In the Transaction Date field, enter 15-MAR-2011. In the Amount field, enter 3,389.89. Select the Reference tab, and enter A.C. Networks. Select the next line, and enter 3 in the Line field. Select NSF as the Type. Enter 500 as the code. In the Transaction Date field, enter 15-MAR-2011. In the Amount field, enter 7,500.00. Select the Reference tab, and enter Advantage Corp. Select the next line, and enter 4 in the Line field. Select Payment as the Type. Enter 140 as the code. In the Transaction Date field, enter 15-MAR-2011. In the Amount field, enter 1,000.00. Save the record.

0
0
5218

article-image-oracle-e-business-suite-adjusting-items-inventory-and-classifying-items

Packt

19 Aug 2011

4 min read

Oracle E-Business Suite: Adjusting Items in Inventory and Classifying Items

Packt

19 Aug 2011

4 min read

Oracle E-Business Suite 12 Financials Cookbook Adjusting Items in Inventory Item quantities can be adjusted in Inventory. In this recipe, we will use miscellaneous transactions to adjust Items. Let's try to enter transactions on some of the controls we have set up. We will try and enter a Miscellaneous Transaction for five paper widgets into Inventory. How to do it... Navigate to Transactions | Miscellaneous Transactions. In the Type field, select Miscellaneous receipt from the list of values. Click on the Transaction Lines button to enter the receipt. Search for the PRD20001 Item in the Item field. Select the Subinventory list of values and the field should be automatically populated with ACME-FG. In the Locator field, enter A1.1.2. The system should display an Error message to indicate that an invalid locator has been entered. Click on OK and enter A1.1.1–the system should accept this value. Enter a value of 5 in the Quantity field. In the account field, enter 01-000-1410-0000-000. This is the account that will be charged for the Inventory transaction. Select the Lot / Serial button. Enter the Lot number–LN10001. The expiration date is generated based on the setting in the Item definition. Enter the quantity of 5. Click on the Serial button. Enter SN10001 in the Start Serial Number field and press Tab on the keyboard. The SN10005 should be automatically populated in the End Serial Number field. Click on the Done button. Click on the Done button again in the next screen. Save the record. There's more... Let's search for the Items in Inventory. Searching for Items We will use the material workbench to search for the Items: Navigate to On-hand | Availability | On-hand Quantity. Enter PRD20001 in the Item / Revision field. Click on the Find button. Expand the Organizations tree to show LN10001. Review the Item details. Close the form. Classifying Items Items are grouped into logical classifications through categories. Categories can be further grouped into category sets. A default category set can be assigned to a functional area. When an Item is created, it is automatically added to the default category set. The groupings are mainly used for reporting purposes. Let's look at how to classify Items using categories. How to do it... Let's list the steps required to create category codes: Navigate to Setup | Items | Categories | Category Codes. Click on the New button to enter the Category. In the Structure Name, select Item Categories. In the Category field, enter BOOKS.MISC. In the Description field, enter Other Books. Save the record. Let's now create the Category Set, add the Category Codes to a new set called ACME Books, and assign it to the PRD20001 Item: Navigate to Setup | Items | Categories | Category Sets. Click on the New button to enter the category set. In the Name field, enter ACME Books. Enter a description, for example, ACME Paper Books. Select Item Categories for Flex Structure. Select Controlled At as the Org level. Enter BOOKS.MISC as the Default Category. Select the checkbox Allow Multiple Item Category Assignments. Add the following Category Codes to the list: BOOKS.MISC BOOKS.NEW BOOKS.USED Save the record. Let's now assign the categories to the PRD20001 Item: Navigate to Items | Organization Items. From the Menu, select Tools and then Categories. Select ACME Books in the Category Set. Enter BOOKS.NEW in the Category field. Save the record. How it works... The structure of the Item category is defined in the Item Flexfield structure and the values are held in the individual Value Sets. The combination of the individual values forms the category code. For example, the structure we used previously is made of two segments, defined in the Flexfield structure. The segments are Family and Class. BOOKS is a value in Family and MISC, NEW, and USED are individual values in Class.

0
0
8920

article-image-oracle-e-business-suite-creating-bank-accounts-and-cash-forecasts

Packt

19 Aug 2011

3 min read

Oracle E-Business Suite: Creating Bank Accounts and Cash Forecasts

Packt

19 Aug 2011

3 min read

Oracle E-Business Suite 12 Financials Cookbook Take the hard work out of your daily interactions with E-Business Suite financials by using the 50+ recipes from this cookbook Introduction Oracle E-business suite The liquidity of an organization is managed in Oracle Cash Management; this includes the reconciliation of the cashbook to the bank statements, and forecasting future cash requirements. In this article, we will look at how to create bank accounts and cash forecasts. Cash management integrates with Payables, Receivables, Payroll, Treasury, and General Ledger. Let's start by looking at the cash management process: The Bank generates statements. The statements are sent to the organization electronically or by post. The Treasury Administrator loads and verifies the bank statement into cash management. The statements can also be manually entered into cash management. The loaded statements are reconciled to the cash book transactions. The results are reviewed, and amended if required. The Treasury Administrator creates the journals for transactions in the General Ledger. Creating bank accounts Oracle Cash Management provides us with the functionality to create bank accounts. In this recipe, we will create a bank account for a bank called Shepherd Bank, for one of their branches called Kings Cross branch. Getting ready Log in to Oracle E-Business Suite R12 with the username and password assigned to you by the system administrator. If you are working on the Vision demonstration database, you can use OPERATIONS/WELCOME as the USERNAME/PASSWORD. We also need to create a bank before we can create the bank account. Let's look at how to create a bank and the branch: Select the Cash Management responsibility. Navigate to Setup | Banks | Banks.(Move the mouse over the image to enlarge it.) In the Banks tab, click on the Create button. Select the Create new bank option. In the Country field, enter United States. In the Bank Name field, enter Shepherds Bank. In the Bank Number field, enter JN316. Click on the Finish button. Let's create the branch and the address: (Move the mouse over the image to enlarge it.) Click the Create Branch icon: The Country and the Bank Name are automatically entered. Click on the Continue button.(Move the mouse over the image to enlarge it.) In the Branch Name field, enter Kings Cross. Select ABA as the Branch Type. Click on the Save and Next button to create the Branch address.(Move the mouse over the image to enlarge it.) In the Branch Address form, click on the create button. In the Country field, enter United States. In the Address Line 1 field, enter 4234 Red Eagle Road. In the City field, enter Sacred Heart. In the County field, enter Renville. In the State field, enter MN. In the Postal Code field, enter 56285. Ensure that the Status field is Active. Click on the Apply button. Click on the Finish button.

0
0
6802

article-image-oracle-e-business-suite-creating-items-inventory

Packt

18 Aug 2011

7 min read

Oracle E-Business Suite: Creating Items in Inventory

Packt

18 Aug 2011

7 min read

Oracle E-Business Suite 12 Financials Cookbook Take the hard work out of your daily interactions with E-Business Suite financials by using the 50+ recipes from this cookbook Read more about this book Oracle E-Business Suite 12 Financials is a solution that provides out-of-the-box features to meet global financial reporting and tax requirements with one accounting, tax, banking, and payments model, and makes it easy to operate shared services across businesses and regions. In this article by Yemi Onigbode, author of Oracle E-Business Suite 12 Financials Cookbook, we will start with recipes for creating Items. We will cover: Creating Items Exploring Item attributes Creating Item templates Exploring Item controls (For more resources on Oracle, see here.) Introduction An organization's operations include the buying and selling of products and services. Items can represent the products and services that are purchased and sold in an organization. Let's start by looking at the Item creation process. The following diagram details the process for creating Items: The Item Requester (the person who requests an Item) completes an Item Creation Form, which should contain information such as: Costing information Pricing Information Item and Product Categories Details of some of the Item attributes The inventory organization details Once complete, a message is sent to the Master Data Manager (the person who maintains the master data) to create the Item. The message could be sent by fax, e-mail, and so on. The Master Data Manager reviews the form and enters the details of the Item into Oracle E-Business Suite by creating the Item. Once complete, a message is sent to the Item Requester. The Item Requester reviews the Item setup on the system. Let's look at how Items are created and explore the underlying concepts concerning the creation of Items. Creating Items Oracle Inventory provides us with the functionality to create Items. Sets of attributes are assigned to an Item. The attributes define the characteristics of the Item. A group of attributes values defines a template, and a template can be assigned to an Item to automatically define the set of attribute values. An Item template defines the Item Type. For example, a Finished Good template will identify certain characteristics that define the Item as a finished good, with attributes such as "Inventory Item" and "Stockable" with a value of "Yes". Let's look at how to create an Item in Oracle Inventory. We will also assign a Finished Good template to the Item. Getting ready Log in to Oracle E-Business Suite R12 with the username and password assigned to you by the System Administrator. If you are working on the Vision demonstration database, you can use OPERATIONS/WELCOME as the USERNAME/PASSWORD: Select the Inventory Responsibility. Select the V1 Inventory Organization. How to do it... Let's list the steps required to create an Item: Navigate to Items | Master Items. Please note that Items are defined in the Master Organization. Enter the Item code, for example, PRD20001. Enter a description for the Item: Select Copy From from the tools menu (or press Alt+T). We are going to copy the attributes from the Finished Good template: We can also copy attributes from an existing Item. Enter Finished Good and click on the Apply button (or press Alt+A) and click on the Done button. Save the Item definition by clicking on the Save icon (or press Ctrl+S). How it works... Items contain attributes and attributes contain information about an Item. Attributes can be controlled centrally at the Master Organization level or at the Inventory Organization level. There's more... Once the Item is created, we need to assign it to a category and an inventory organization. Assigning Items to inventory organizations For us to be able to perform transactions with the Item in the inventory, we need to assign the Item to an inventory organization. We can also use the organization Item form to change the attributes at the organization level. For example, an Item may be classified as raw materials in one organization and finished goods in another organization. From the Tools menu, select Organization Assignment. Select the inventory organization for the Item. For example, A1–ACME Corporation. Click on the Assigned checkbox. Save the assignment. Assigning Items to categories When an Item is created, it is assigned to a default category. However, you may want to perform transactions with the Item in more than one functional area, such as Inventory, Purchasing, Cost Management, Service, Engineering, and so on. You need to assign the Item to the relevant functional area. A category within a functional area is a logical classification of Items with similar characteristics. From the Tools menu, select Categories. Select the Categories Set, Control Level, and the Category combination to assign to the Item: Save the assignment. Exploring Item attributes There are more than 250 Item attributes grouped into 17 main attribute groups. In this recipe, we will explore the main groups that are used within the financial modules. How to do it... Let's explore some Item attributes: Search for the Finished Good Item by navigating to Items | Master Items: Click on the Find icon. You then enter the Item code and click on the Find button to search for the Item. Select the tabs to review each of the attributes group: In the Main tab, check that the Item Status is Active. We can also enter a long description in the Long Description field. The default value of the primary Unit of Measure (UOM) can be defined in the INV: Default Primary Unit of Measure profile option. The value can be overwritten when creating the Item. The Primary UOM is the default UOM used in other modules. For example, in Receivables it is used for invoices and credit memos. In the Inventory tab, check that the following are enabled: Inventory Item: It enables the Item to be transacted in Inventory. The default Inventory Item category is automatically assigned to the Item, if enabled. Stockable: It enables the Item to be stocked in Inventory. Transactable: Order Management uses this flag to determine how returns are transacted in Inventory. Reservable: It enables the reservation of Items during transactions. For example, during order entry in Order Management. In the Costing tab, check that the following are enabled: Costing: Enables the accounting for Item costs. It can be overridden in the Cost Management module, if average costing is used. Cost of Goods Sold Account: The cost of goods sold account is entered. This is a general ledger account. The value defaults from the Organization parameters. In the Purchasing tab, enter a Default Buyer for the purchase orders, a List Price, and an Expense Account. Check that the following are enabled: Purchased: It enables us to purchase and receive the Item. Purchasable: It enables us to create a Purchase Order for the Item. Allow Description Update: It enables us to change the description of the Item when raising the Purchase Order. RFQ Required: Set this value to Yes to enable us to require a quotation for this Item. Taxable: Set this value to Yes with the Input Tax Classification Code as VAT–15%. This can be used with the default rules in E-Tax. Invoice Matching: Receipt Required–Yes. This is to allow for three-way matching. In the Receiving tab, review the controls. In the Order Management tab, check that the following are enabled: Customer Ordered: This enables us to define prices for an Item assigned to a price list. Customer Orders Enabled: This enables us to sell the Item. Shippable: This enables us to ship the Item to the Customer. Internal Ordered: This enables us to order an Item via internal requisitions. Internal Orders Enabled: This enables us to temporarily exclude an Item from internal requisitions. OE Transactable: This is used for demand management of an Item. In the Invoicing tab, enter values for the Accounting Rule, Invoicing Rule, Output Tax Classification Code, and Payment Terms. Enter the Sales Account code and check that the Invoiceable Item and Invoice Enabled checkboxes are enabled.

0
0
16621

article-image-terms-and-concepts-related-mdx

Packt

09 Aug 2011

8 min read

Terms and Concepts Related to MDX

Packt

09 Aug 2011

8 min read

MDX with Microsoft SQL Server 2008 R2 Analysis Services Cookbook More than 80 recipes for enriching your Business Intelligence solutions with high-performance MDX calculations and flexible MDX queries in this book and eBook Parts of an MDX query This section contains the brief explanation of the basic elements of MDX queries: members, sets, tuples, axes, properties, and so on. Regular members Regular dimension members are members sourced from the underlying dimension tables. They are the building blocks of dimensions, fully supported in any type of drill operations, drillthrough, scopes, subqueries, and probably all SSAS front-ends. They can have children and be organized in multilevel user hierarchies. Some of the regular member's properties can be dynamically changed using scopes in MDX script or cell calculations in queries - color, format, font, and so on. Measures are a type of regular members, found on the Measures dimension/hierarchy. The other type of members is calculated members. Calculated members Calculated members are artificial members created in a query, session, or MDX script. They do not exist in the underlying dimension table and as such are not supported in drillthrough and scopes. In subqueries, they are only supported if the connection string includes one of these settings: Subqueries = 1 or Subqueries = 2. See here for examples: http://tinyurl.com/ChrisSubquerySettings They also have a limited set of properties compared to regular members and worse support than regular members in some SSAS front-ends. An often practiced workaround is creating dummy regular members in a dimension table and then using MDX script assignments to provide the calculation for them. They are referred to as "Dummy" because they never occur in the fact table which also explains the need for assignments. Tuples A tuple is a coordinate in the multidimensional cube. That coordinate can be huge and is often such. For example, a cube with 10 dimensions each having 5 attributes is a 51 dimensional object (measures being that extra one). To fully define a coordinate we would have to reference every single attribute in that cube. Fortunately, in order to simplify their usage, tuples are allowed to be written using a part of the full coordinate only. The rest of the coordinate inside the tuple is implicitly evaluated by SSAS engine, either using the current members (for unrelated hierarchies) or through the mechanism known as strong relationships (for related hierarchies). It's worth mentioning that the initial current members are cube's default members. Any subsequent current members are derived from the current context of the query or the calculation. Evaluation of implicit members can sometimes lead to unexpected problems. We can prevent those problems by explicitly specifying all the hierarchies we want to have control over and thereby not letting the implicit evaluation to occur for those hierarchies. Contrary to members and sets, tuples are not an object that can be defined in the WITH part of the query or in MDX script. They are non-persistent. Tuples can be found in sets, during iteration or in calculations. They are often used to set or overwrite the current context, in other words, to jump out of the current context and get the value in another coordinate of the cube. Another important aspect of tuples is their dimensionality. When building a set from tuples, two or more tuples can be combined only if they are built from the same hierarchies, specified in the exact same order. That's their dimensionality. You should know that rearranging the order of hierarchies in a tuple doesn't change its value. Therefore, this can be the first step we can do to make the tuples compatible. The other thing is adding the current members of hierarchies present only in the other tuple, to match the other tuple's dimensionality. Named sets A named set is a user-defined collection of members, more precisely, tuples. Named sets are found in queries, sessions, and MDX scripts. Query-based named sets are equivalent to dynamic sets in MDX script. They both react to the context of subquery and slicer. Contrary to them, static sets are constant, independent of any context. Only the sets that have the same dimensionality can be combined together because what we really combine are the tuples they are built from. It is possible to extract one or more hierarchies from the set. It is also possible to expand the set by crossjoining it with hierarchies not present in its tuples. These processes are known as reducing and increasing the dimensionality of a set. Set alias Set aliases can be defined in calculations only, as a part of that calculation and not in the WITH part of the query as a named set. This is done by identifying a part of the calculation that represents a set and giving a name to that expression inside the calculation, using the AS keyword. This way that set can be used in other parts of the calculation or even other calculations of the query or MDX script. Set aliases enable true dynamic evaluation of sets in a query because they can be evaluated for each cell if used inside a calculated measure. The positive effect is that they are cached, calculated only once and used many times in the calculation or query. The downside is that they prevent block-computation mode because the above mentioned evaluation is performed for each cell individually. In short, set aliases can be used in long calculations, where the same set appears multiple times or when that set needs to be truly dynamic. At the same time, they are to be avoided in iterations of any kind. Axis An axis is a part of the query where a set is projected at. A query can have up to 128 axes although most queries have 1 or 2 axes. A query with no axis is also a valid query but almost never used. The important thing to remember is that axes are evaluated independently. SSAS engine knows in which order to calculate them if there is a dependency between them. One way to create such a dependency is to refer to the current member of a hierarchy on the other axis. The other option would be to use the Axis() function. Some SSAS front-ends generate MDX queries that break the axes dependencies established through calculations. The workaround calculations can be very hard if not impossible. Slicer The slicer, also known as the filter axis or the WHERE clause, is a part of the query which sets the context for the evaluation of members and sets on axes and in the WITH part of the query. The slicer, which can be anything from a single tuple up to the multidimensional set, interacts with sets on axes. A single member of a hierarchy in slicer forces the coordinate and reduces the related sets on axes by removing all non-existing combinations. Multiple members of the same hierarchy are not that strong. In their case, individual members in sets on axes overwrite the context of the slicer during their evaluation. Finally, the context established by the slicer can be overwritten in the calculations using tuples. Subquery The subquery, also known as the subselect, is a part of the query which executes first and determines the cube space to be used in the query. Unlike slicer, the subquery doesn't set the coordinate for the query. In other words, current members of all hierarchies (related, unrelated, and even the same hierarchy used in the subquery) remain the same. What the subquery does is it applies the VisualTotals operation on members of hierarchies used in the subquery. The VisualTotals operation changes each member's value with the aggregation value of its children, but only those present in the subquery. Because the slicer and the subquery have different behaviors, one should not be used as a replacement for the other. Whenever you need to set the context for the whole query, use the slicer. That will adjust the total for all hierarchies in the cube. If you only need to adjust the total for some hierarchies in the cube and not for the others, subquery is the way to go; specify those hierarchies in the subquery. This is also an option if you need to prevent any attribute interaction between your subquery and the query. The areas where the subquery is particularly good at are grouping of non-granular attributes, advanced set logic, and restricting members on hierarchies. Cell properties Cell properties are properties that can be used to get specific behaviors for cells. For example: colors, font sizes, types and styles, and so on. Unless explicitly asked for, only the Cell_Ordinal, Value, and Formatted_Value properties are returned by an MDX query. Dimension properties Dimension properties are a set of member properties that return extra information about members on axes. Intrinsic member properties are Key, Name, and Value; the others are those defined by the user for a particular hierarchy in the Dimension Designer. In client tools, dimension properties are often shown in the grid next to the attribute they are bound to or in the hint over that attribute.

0
0
1700

article-image-oracle-integration-and-consolidation-products

Packt

09 Aug 2011

11 min read

Oracle Integration and Consolidation Products

Packt

09 Aug 2011

11 min read

Oracle Information Integration, Migration, and Consolidation The definitive book and eBook guide to Oracle information integration and migration in a heterogeneous world Data services Data services are at the leading edge of data integration. Traditional data integration involves moving data to a central repository or accessing data virtually through SQL-based interfaces. Data services are a means of making data a 'first class' citizen in your SOA. Recently, the idea of SOA-enabled data services has taken off in the IT industry. This is not any different than accessing data using SQL, JDBC, or ODBC. What is new is that your service-based architecture can now view any database access service as a web service. Service Component Architecture (SCA) plays a big role in data services as now data services created and deployed using Oracle BPEL, Oracle ESB, and other Oracle SOA products can be part of an end-to-end data services platform. No longer do data services deployed in one of the SOA products have to be deployed in another Oracle SOA product. SCA makes it possible to call a BPEL component from Oracle Service Bus and vice versa. Oracle Data Integration Suite Oracle Data Integration (ODI)Suite includes the Oracle Service Bus to publish and subscribe messaging capabilities. Process orchestration capabilities are provided by Oracle BPEL Process Manager, and can be configured to support rule-based, event-based, and data-based delivery services. The Oracle Data Quality for Data Integrator, Oracle Data Profiling products, and Oracle Hyperion Data Relationship Manager provide best-in-class capabilities for data governance, change management hierarchical data management, and provides the foundation for reference data management of any kind. ODI Suite allows you to create data services that can be used in your SCA environment. These data services can be created in ODI, Oracle BPEL or the Oracle Service Bus. You can surround your SCA data services with Oracle Data Quality and Hyperion Data Relationship to cleanse your data and provide master data management. ODI Suite effectively serves two purposes: Bundle Oracle data integration solutions as most customers will need ODI, Oracle BPEL, Oracle Service Bus, and data quality and profiling in order to build a complete data services solution Compete with similar offerings from IBM (InfoSphere Information Server) and Microsoft (BizTalk 2010) that offer complete EII solutions in one offering The ODI Suite data service source and target data sources along with development languages and tools supported are: Data sourceData targetDevelopment languages and toolsERPs, CRMs, B2B systems, flat files, XML data, LDAP, JDBC, ODBCAny data sourceSQL, Java, GUI The most likely instances or use cases when ODI Suite would be the Oracle product or tool selected are: SCA-based data services An end-to-end EII and data migration solution Data services can be used to expose any data source as a service. Once a data service is created, it is accessible and consumable by any web service-enabled product. In the case of Oracle, this is the entire set of products in the Oracle Fusion Middleware Suite. Data consolidation The mainframe was the ultimate solution when it came to data consolidation. All data in an enterprise resided in one or several mainframes that were physically located in a data center. The rise of the hardware and software appliance has created a 'what is old is new again' situation; a hardware and software solution that is sold as one product. Oracle has released the Oracle Exadata appliance and IBM acquired the pure database warehouse appliance company Netezza, HP, and Microsoft announced work on an SQL Server database appliance, and even companies like SAP, EMC, and CICSO are talking about the benefits of database appliances. The difference is (and it is a big difference) that the present architecture is based upon open standards hardware platforms, operating systems, client devices, network protocols, interfaces, and databases. So, you now have a database appliance that is not based upon proprietary operating systems, hardware, network components, software, and data disks. Another very important difference is that enterprise software COTS packages, management tools, and other software infrastructure tools will work across any of these appliance solutions. One of the challenges for customers that run their business on the mainframe is that they are 'locked into' vendor- specific sorting, reporting, job scheduling, system management, and other products usually only offered from IBM, CA, BMC, or Compuware. Mainframe customers also suffer from a lack of choice when it comes to COTS applications. Since appliances are based upon open systems, there is an incredibly large software ecosystem. Oracle Exadata Oracle Exadata is the only database appliance that runs both data warehouse and OLTP applications. Oracle Exadata is an appliance that includes every component an IT organization needs to process information—from a grid database down to the power supply. It is a hardware and software solution that can be up and running in an enterprise in weeks instead of months for typical IT database solutions. Exadata provides high speed data access using a combination of hardware and a database engine that runs at the storage tier. Typical database solutions have to use indexes to retrieve data from storage and then pull large volumes of data into the core database engine, which churns through millions of rows of data to send a handful of row results to the client. Exadata eliminates the need for indexes and data engine processing by placing a lightweight database engine at the storage tier. Therefore, the database engine is only provided with the end result and does not have to utilize complicated indexing schemes, large amounts of CPU, and memory to produce the end results set. Exadata's capabilities to run large OLTP and data warehouse applications, or a large number of smaller OLTP and data warehouse applications on one machine make it a great platform for data consolidation. The first release of Oracle Exadata was based upon HP hardware and was for data warehouses only. The second release came out shortly before Oracle acquired Sun. This release was based upon Sun hardware, but ironically not on Sun Sparc or Solaris (Solaris is now an OS option). The Exadata source and target data sources along with development languages and tools supported are: Data sourceData targetDevelopment languages and toolsAny (depending upon the data source this may involve an intensive migration effort)Oracle ExadataSQL, PL/SQL, Java The most likely instances or use cases when Exadata would be the Oracle product or tool selected are: A move from hundreds of standalone database hardware and software nodes to one database machine A reduction in hardware and software vendors, and one vendor for hardware and software support Keepin It Real The database appliance has become the latest trend in the IT industry. Data warehouse appliances like Netezza have been around for a number of years. Oracle has been the first vendor to offer an open systems database appliance for both DW and OLTP environments. Data grid Instead of consolidating databases physically or accessing the data where it resides, a data grid places the data into an in-memory middle tier. Like physical federation, the data is being placed into a centralized data repository. Unlike physical federation, the data is not placed into a traditional RDBMS system (Oracle database), but into a high-speed memory-based data grid. Oracle offers both a Java and SQL-based data grid solution. The decision of what product to implement often depends on where the corporations system, database, and application developer skills are strongest. If your organization has strong Java or .Net skills and is more comfortable with application servers than databases, then Oracle Coherence is typically the product of choice. If you have strong database administration and SQL skills, then Oracle TimesTen is probably a better solution. The Oracle Exalogic solution takes the data grid to another level by placing Oracle Coherence, along with other Oracle hardware and software solutions, into an appliance. This appliance provides an 'end-to-end' solution or data grid 'in a box'. It reduces management, increases performance, reduces TCO, and eliminates the need for the customer having to build their own hardware and software solution using multiple vendor solutions that may not be certified to work together. Oracle Coherence Oracle Coherence is an in-memory data grid solution that offers next generation Extreme Transaction Processing (XTP). Organizations can predictably scale mission critical applications by using Oracle Coherence to provide fast and reliable access to frequently used data. Oracle Coherence enables customers to push data closer to the application for faster access and greater resource utilization. By automatically and dynamically partitioning data in memory across multiple servers, Oracle Coherence enables continuous data availability and transactional integrity, even in the event of a server failure. Oracle Coherence was purchased from Tangosol Software in 2007. Coherence was an industry-leading middle tier caching solution. The product only offered a Java solution at the time of acquisition, but a .NET offering was already scheduled before the acquisition took place. The Oracle Coherence source and target data sources along with development languages and tools supported are: Data sourceData targetDevelopment languages and toolsJDBC, any data source accessible through Oracle SOA adaptersCoherenceJava, .Net The most likely instances or use cases when Oracle Coherence would be the Oracle product or tool selected are: When it is necessary to replace custom, hand-coded solutions that cache data in middle tier Java or .NET application servers Your company's strengths are in application servers Java or .NET Oracle TimesTen Oracle TimesTen is a data grid/cache offering that has similar characteristics to Oracle Coherence. Both of the solutions offer a product that caches data in the middle tier for high throughput and high transaction volumes. The technology implementations are much different. TimesTen is an in-memory database solution that is accessed through SQL and the data storage mechanism is a relational database. The TimesTen solution data grid can be implemented across a wide area network (WAN) and the nodes that make up the data grid are kept in sync with your back end Oracle database using Oracle Cache Connect. Cache Connect is also used to automatically refresh the TimesTen database on a push or pull basis from your Oracle backend database. Cache Connect can also be used to keep TimesTen databases spread across the global in sync. Oracle TimesTen offers both read and update support, unlike other database in- memory solutions. This means that Oracle TimesTen can be used to run your business even if your backend database is down. The transactions that occur during the downtime are queued and applied to your backend database once it is restored. The other similarity between Oracle Coherence and TimesTen is that they both were acquired technologies. Oracle TimesTen was acquired from the company TimesTen in 2005. The Oracle TimesTen source and target data sources along with development languages and tools supported are: Data sourceData targetDevelopment languages and toolsOracleTimesTenSQL, CLI The most likely instances or use cases when Oracle TimesTen would be the Oracle product or tool selected are: For web-based read-only applications that require a millisecond responseand data close to where request is made For applications where updates need not be reflected back to the user in real-time Oracle Exalogic A simplified explanation of Oracle Exalogic is that it is Exadata for the middle tier application infrastructure. While Exalogic is optimized for enterprise Java, it is also a suitable environment for the thousands of third-party and custom Linux and Solaris applications widely deployed on Java, .NET, Visual Basic, PHP, or any other programming language. The core software components of Exalogic are WebLogic, Coherence, JRocket or Java Hotspot, and Oracle Linux or Solaris. Oracle Exalogic has an optimized version of WebLogic to run Java applications more efficiently and faster than a typical WebLogic implementation. Oracle Exalogic is branded with the Oracle Elastic cloud as an enterprise application consolidation platform. This means that applications can be added on demand and in real-time. Data can be cached in Oracle Coherence for a high speed, centralized, data grid sharable on the cloud. The Exalogic source and target data sources along with development languages and tools supported are: Data sourceData targetDevelopment languages and toolsAny data sourceCoherenceAny language The most likely instances or use cases when Exalogic would be the Oracle product or tool selected are: Enterprise consolidated application server platform Cloud hosted solution Upgrade and Consolidation of hardware or software Oracle Coherence is the product of choice for Java and .NET versed development shops. Oracle TimesTen is more applicable to database-centric and shops more comfortable with SQL.

0
0
4765

article-image-performing-common-mdx-related-tasks

Packt

05 Aug 2011

6 min read

Performing Common MDX-related Tasks

Packt

05 Aug 2011

6 min read

MDX with Microsoft SQL Server 2008 R2 Analysis Services Cookbook More than 80 recipes for enriching your Business Intelligence solutions with high-performance MDX calculations and flexible MDX queries in this book and eBook Skipping axis There are situations when we want to display just a list of members and no data associated with them. Naturally, we expect to get that list on rows, so that we can scroll through them nicely. However, the rules of MDX say we can't skip axes. If we want something on rows (which is AXIS(1) by the way), we must use all previous axes as well (columns in this case, which is also known as AXIS(0)). The reason why we want the list to appear on axis 1 and not axis 0 is because a horizontal list is not as easy to read as a vertical one. Is there a way to display those members on rows and have nothing on columns? Sure! This recipe shows how. Getting ready Follow these steps to set up the environment for this recipe: Start SQL Server Management Studio (SSMS) or any other application you use for writing and executing MDX queries and connect to your SQL Server Analysis Services (SSAS) 2008 R2 instance (localhost or servernameinstancename). Click on the New Query button and check that the target database is Adventure Works DW 2008R2. How to do it... Follow these steps to get a one-dimensional query result with members on rows: Put an empty set on columns (AXIS(0)). Notation for empty set is this: {}. Put some hierarchy on rows (AXIS(1)). In this case we used the largest hierarchy available in this cube – Customer hierarchy of the same dimension. Run the following query: SELECT { } ON 0, { [Customer].[Customer].[Customer].MEMBERS } ON 1 FROM [Adventure Works] How it works... Although we can't skip axes, we are allowed to provide an empty set on them. This trick allows us to get what we need – nothing on columns and a set of members on rows. There's more… Notice that this type of query is very convenient for parameter selection of another query as well as for search. See how it can be modified to include only those customers whose name contains the phrase "John": SELECT { } ON 0, { Filter( [Customer].[Customer].[Customer].MEMBERS, InStr( [Customer].[Customer].CurrentMember.Name, 'John' ) > 0 ) } ON 1 FROM [Adventure Works] In the final result, you will notice the "John" phrase in various positions in member names: The idea behind If you put a cube measure or a calculated measure with a non-constant expression on axis 0 instead, you'll slow down the query. Sometimes it won't be so obvious, sometimes it will. It will depend on the measure's definition and the number of members in the hierarchy being displayed. For example, if you put the Sales Amount measure on columns, that measure will have to be evaluated for each member in the rows. Do we need those values? No, we don't. The only thing we need is a list of members; hence we've used an empty set. That way, the SSAS engine doesn't have to go into cube space. It can reside in dimension space which is much smaller and the query is therefore more efficient. Possible workarounds In case of a third-party application or a control which has problems with this kind of MDX statement (i.e. expects something on columns and is not working with an empty set), we can define a constant measure (a measure returning null, 0, 1 or any other constant) and place it on columns instead of that empty set. For example, we can define a calculated measure in the MDX script whose definition is 1, or any other constant value, and use that measure on the columns axis. It might not be as efficient as an empty set, but it is a much better solution than the one with a regular (non-constant) cube measure like the Sales Amount measure. Handling division by zero errors Another common task is handling errors, especially division by zero type of errors. This recipe offers a way to solve that problem. Not all versions of Adventure Works database have the same date range. If you're not using the recommended version of it, the one for the SSAS 2008 R2, you might have problems with queries. Older versions of Adventure Works database have dates up to the year 2006 or even 2004. If that's the case, make sure you adjust examples by offsetting years in the query with a fixed number. For example, the year 2006 should become 2002 and so on. Getting ready Start a new query in SQL Server Management Studio and check that you're working on Adventure Works database. Then write and execute this query: WITH MEMBER [Date].[Calendar Year].[CY 2006 vs 2005 Bad] AS [Date].[Calendar Year].[Calendar Year].&[2006] / [Date].[Calendar Year].[Calendar Year].&[2005], FORMAT_STRING = 'Percent' SELECT { [Date].[Calendar Year].[Calendar Year].&[2005], [Date].[Calendar Year].[Calendar Year].&[2006], [Date].[Calendar Year].[CY 2006 vs 2005 Bad] } * [Measures].[Reseller Sales Amount] ON 0, { [Sales Territory].[Sales Territory].[Country].MEMBERS } ON 1 FROM [Adventure Works] This query returns 6 rows with countries and 3 rows with years, the third row being the ratio of the previous two, as its definition says. The problem is that we get 1.#INFM on some cells. To be precise, that value (the formatted value of infinity), appears on rows where the CY 2005 is null. Here's a solution for that. How to do it... Follow these steps to handle division by zero errors: Copy the calculated member and paste it as another calculated member. During that, replace the term Bad with Good in its name, just to differentiate those two members. Copy the denominator. Wrap the expression in an outer IIF() statement. Paste the denominator in the condition part of the IIF() statement and compare it against 0. Provide null value for the True part. Your initial expression should be in the False part. Don't forget to include the new member on columns and execute the query: MEMBER [Date].[Calendar Year].[CY 2006 vs 2005 Good] AS IIF ([Date].[Calendar Year].[Calendar Year].&[2005] = 0, null, [Date].[Calendar Year].[Calendar Year].&[2006] / [Date].[Calendar Year].[Calendar Year].&[2005] ), FORMAT_STRING = 'Percent' The result shows that the new calculated measure corrects the problem – we don't get errors (the rightmost column, compared to the one on its left): How it works... A division by zero error occurs when the denominator is null or zero and the numerator is not null. In order to prevent this error, we must test the denominator before the division and handle the case when it is null or zero. That is done using an outer IIF() statement. It is enough to test just for zero because null = 0 returns True. There's more... SQLCAT's SQL Server 2008 Analysis Services Performance Guide has lots of interesting details regarding the IIF() function: http://tinyurl.com/PerfGuide2008 Additionally, you may find Jeffrey Wang's blog article useful in explaining the details of the IIF() function: http://tinyurl.com/IIFJeffrey Earlier versions of SSAS If you're using a version of SSAS prior to 2008 (that is, 2005), the performance will not be as good. See Mosha Pasumansky's article for more info: http://tinyurl.com/IIFMosha

0
0
2149

article-image-how-perform-iteration-sets-mdx

Packt

05 Aug 2011

5 min read

How to Perform Iteration on Sets in MDX

Packt

05 Aug 2011

5 min read

MDX with Microsoft SQL Server 2008 R2 Analysis Services Cookbook More than 80 recipes for enriching your Business Intelligence solutions with high-performance MDX calculations and flexible MDX queries in this book and eBook Iteration is a very natural way of thinking for us humans. We set a starting point, we step into a loop, and we end when a condition is met. While we're looping, we can do whatever we want: check, take, leave, and modify items in that set. Being able to break down the problems in steps makes us feel that we have things under control. However, by breaking down the problem, the query performance often breaks down as well. Therefore, we have to be extra careful with iterations when data is concerned. If there's a way to manipulate the collection of members as one item, one set, without cutting that set into small pieces and iterating on individual members, we should use it. It's not always easy to find that way, but we should at least try. Iterating on a set in order to reduce it Getting ready Start a new query in SSMS and check that you're working on the right database. Then write the following query: SELECT { [Measures].[Customer Count], [Measures].[Growth in Customer Base] } ON 0, NON EMPTY { [Date].[Fiscal].[Month].MEMBERS } ON 1 FROM [Adventure Works] WHERE ( [Product].[Product Categories].[Subcategory].&[1] ) The query returns fiscal months on rows and two measures: a count of customers and their growth compared to the previous month. Mountain bikes are in slicer. Now let's see how we can get the number of days the growth was positive for each period. How to do it... Follow these steps to reduce the initial set: Create a new calculated measure in the query and name it Positive growth days. Specify that you need descendants of current member on leaves. Wrap around the FILTER() function and specify the condition which says that the growth measure should be greater than zero. Apply the COUNT() function on a complete expression to get count of days. The new calculated member's definition should look as follows, verify that it does. WITH MEMBER [Measures].[Positive growth days] AS FILTER( DESCENDANTS([Date].[Fiscal].CurrentMember, , leaves), [Measures].[Growth in Customer Base] > 0 ).COUNT Add the measure on columns. Run the query and observe if the results match the following image: How it works... The task says we need to count days for each time period and use only positive ones. Therefore, it might seem appropriate to perform iteration, which, in this case, can be performed using the FILTER() function. But, there's a potential problem. We cannot expect to have days on rows, so we must use the DESCENDANTS() function to get all dates in the current context. Finally, in order to get the number of items that came up upon filtering, we use the COUNT function. There's more... Filter function is an iterative function which doesn't run in block mode, hence it will slow down the query. In the introduction, we said that it's always wise to search for an alternative if available. Let's see if something can be done here. A keen eye will notice a "count of filtered items" pattern in this expression. That pattern suggests the use of a set-based approach in the form of SUM-IF combination. The trick is to provide 1 for the True part of the condition taken from the FILTER() statement and null for the False part. The sum of one will be equivalent to the count of filtered items. In other words, once rewritten, that same calculated member would look like this: MEMBER [Measures].[Positive growth days] AS SUM( Descendants([Date].[Fiscal].CurrentMember, , leaves), IIF( [Measures].[Growth in Customer Base] > 0, 1, null) ) Execute the query using the new definition. Both the SUM() and the IIF() functions are optimized to run in the block mode, especially when one of the branches in IIF() is null. In this particular example, the impact on performance was not noticeable because the set of rows was relatively small. Applying this technique on large sets will result in drastic performance improvement as compared to the FILTER-COUNT approach. Be sure to remember that in future. More information about this type of optimization can be found in Mosha Pasumansky's blog: http://tinyurl.com/SumIIF Hints for query improvements There are several ways you can avoid the FILTER() function in order to improve performance. When you need to filter by non-numeric values (i.e. properties or other metadata), you should consider creating an attribute hierarchy for often-searched items and then do one of the following: Use a tuple when you need to get a value sliced by that new member Use the EXCEPT() function when you need to negate that member on its own hierarchy (NOT or <>) Use the EXISTS() function when you need to limit other hierarchies of the same dimension by that member Use the NONEMPTY() function when you need to operate on other dimensions, that is, subcubes created with that new member Use the 3-argument EXISTS() function instead of the NONEMPTY() function if you also want to get combinations with nulls in the corresponding measure group (nulls are available only when the NullProcessing property for a measure is set to Preserve) When you need to filter by values and then count a member in that set, you should consider aggregate functions like SUM() with IIF() part in its expression, as described earlier.

0
0
7814

article-image-getting-started-oracle-information-integration

Packt

02 Aug 2011

14 min read

Getting Started with Oracle Information Integration

Packt

02 Aug 2011

14 min read

Oracle Information Integration, Migration, and Consolidation: RAW The definitive book and eBook guide to Oracle Information Integration and Migration in a heterogeneous world Read more about this book (For more resources on Oracle, see here.) Why consider information integration? The useful life of pre-relational mainframe database management system engines is coming to an end because of a diminishing application and skills base, and increasing costs.—Gartner Group During the last 30 years, many companies have deployed mission critical applications running various aspects of their business on the legacy systems. Most of these environments have been built around a proprietary database management system running on the mainframe. According to Gartner Group, the installed base of mainframe, Sybase, and some open source databases has been shrinking. There is vendor sponsored market research that shows mainframe database management systems are growing, which, according to Gartner, is due primarily to increased prices from the vendors, currency conversions, and mainframe CPU replacements. Over the last few years, many companies have been migrating mission critical applications off the mainframe onto open standard Relational Database Management Systems (RDBMS) such as Oracle for the following reasons: Reducing skill base: Students and new entrants to the job market are being trained on RDBMS like Oracle and not on the legacy database management systems. Legacy personnel are retiring, and those that are not are moving into expensive consulting positions to arbitrage the demand. Lack of flexibility to meet business requirements: The world of business is constantly changing and new business requirements like compliance and outsourcing require application changes. Changing the behavior, structure, access, interface or size of old databases is very hard and often not possible, limiting the ability of the IT department to meet the needs of the business. Most applications on the aging platforms are 10 to 30 years old and are long past their original usable lifetime. Lack of Independent Software Vendor (ISV)applications: With most ISVs focusing on the larger market, it is very difficult to find applications, infrastructure, and tools for legacy platforms. This requires every application to be custom coded on the closed environment by scarce in-house experts or by expensive outside consultants. Total Cost of Ownership (TCO): As the user base for proprietary systems decreases, hardware, spare parts, and vendor support costs have been increasing. Adding to this are the high costs of changing legacy applications, paid either as consulting fees for a replacement for diminishing numbers of mainframe trained experts or increased salaries for existing personnel. All leading to a very high TCO which doesn't even take into account the opportunity cost to the business of having inflexible systems. Business challenges in data integration and migration Once the decision has been taken to migrate away from a legacy environment, the primary business challenge is business continuity. Since many of these applications are mission critical, running various aspects of the business, the migration strategy has to ensure continuity to the new application—and in the event of failure, rollback to the mainframe application. This approach requires data in the existing application to be synchronized with data on the new application. Making the challenge of data migration more complicated is the fact that legacy applications tend to be interdependent, but the need from a risk mitigation standpoint is to move applications one at a time. A follow-on challenge is prioritizing the order in which applications are to be moved off the mainframe, and ensuring that the order meets both the business needs and minimizes the risk in the migration process. Once a specific application is being migrated, the next challenge is to decide which business processes will be migrated to the new application. Many companies have business processes that are present, because that's the way their systems work. When migrating an application off the mainframe, many business processes do not need to migrate. Even among the business processes that need to be migrated, some of these business processes will need to be moved as-is and some of them will have to be changed. Many companies utilize the opportunity afforded by a migration to redo the business processes they have had to live with for many years. Data is the foundation of the modernization process. You can move the application, business logic, and work flow, but without a clean migration of the data the business requirements will not be met. A clean data migration involves: Data that is organized in a usable format by all modern tools Data that is optimized for an Oracle database Data that is easy to maintain Technical challenges of information integration The technical challenges with any information integration all stem from the fact that the application accesses heterogeneous data (VSAM, IMS, IDMS, ADABAS, DB2, MSSQL, and so on) that can even be in a non-relational hierarchical format. Some of the technical problems include: The flexible file definition feature used in COBOL applications in the existing system will have data files with multi-record formats and multi-record types in the same dataset—neither of which exist in RDBMS. Looping data structure and substructure or relative offset record organization such as a linked list, which are difficult to map into a relational table. Data and referential integrity is managed by the Oracle database engine. However, legacy applications already have this integrity built in. One question is whether to use Oracle to handle this integrity and remove the logic from the application. Finally, creating an Oracle schema to maximize performance, which includes mapping non-oracle keys to Oracle primary and secondary keys; especially when legacy data is organized in order of key value which can affect the performance on an Oracle RDBMS. There are also differences in how some engines process transactions, rollbacks, and record locking. General approaches to information integration and migration There are several technical approaches to consider when doing any kind of integration or migration activity. In this section, we will look at a methodology or approach for both data integration and data migration. Data integration Clearly, given this range of requirements, there are a variety of different integration strategies, including the following: Consolidated: A consolidated data integration solution moves all data into a single database and manages it in a central location. There are some considerations that need to be known regarding the differences between non- Oracle and Oracle mechanics. Transaction processing is an example. Some engines use implicit commits and some manage character sets differently than Oracle does, this has an impact on sort order. Federated: A federated data integration solution leaves data in the individual data source where it is normally maintained and updated, and simply consolidates it on the fly as needed. In this case, multiple data sources will appear to be integrated into a single virtual database, masking the number and different kinds of databases behind the consolidated view. These solutions can work bidirectionally. Shared: A shared data integration solution actually moves data and events from one or more source databases to a consolidated resource, or queue, created to serve one or more new applications. Data can be maintained and exchanged using technologies such as replication, message queuing, transportable table spaces, and FTP. Oracle has extensive support for consolidated data integration and while there are many obvious benefits to the consolidated solution, it is not practical for any organization that must deal with legacy systems or integrate with data it does not own. Therefore, we will not discuss this type any further, but instead concentrate on federated and shared solutions. Data migration Over 80 percent of migration projects fail or overrun their original budgets/ timelines, according to a study by the Standish Group. In most cases, this is because of a lack of understanding of some of the unique challenges of a migration project. The top five challenges of a migration project are: Little migration expertise to draw from: Migration is not an industry-recognized area of expertise with an established body of knowledge and practices, nor have most companies built up any internal competency to draw from. Insufficient understanding of data and source systems: The required data is spread across multiple source systems, not in the right format, of poor quality, only accessible through vaguely understood interfaces, and sometimes missing altogether. Continuously evolving target system: The target system is often under development at the time of data migration, and the requirements often change during the project. Complex target data validations: Many target systems have restrictions, constraints, and thresholds on the validity, integrity, and quality of the data to be loaded. Repeated synchronization after the initial migration: Migration is not a one-time effort. Old systems are usually kept alive after new systems launch and synchronization is required between the old and new systems during this handoff period. Also, long after the migration is completed, companies often have to prove the migration was complete and accurate to various government, judicial, and regulatory bodies. Most migration projects fail because of an inappropriate migration methodology, because the migration problem is thought of as a four stage process: Analyze the source data Extract/transform the data into the target formats Validate and cleanse the data Load the data into the target However, because of the migration challenges discussed previously, this four stage project methodology often fails miserably. The challenge begins during the initial analysis of the source data when most of the assumptions about the data are proved wrong. Since there is never enough time planned for analysis, any mapping specification from the mainframe to Oracle is effectively an intelligent guess. Based on the initial mapping specification, extractions, and transformations developed run into changing target data requirements, requiring additional analysis and changes to the mapping specification. Validating the data according to various integrity and quality constraints will typically pose a challenge. If the validation fails, the project goes back to further analysis and then further rounds of extractions and transformations. When the data is finally ready to be loaded into Oracle, unexpected data scenarios will often break the loading process and send the project back for more analysis, more extractions and transformations, and more validations. Approaching migration as a four stage process means continually going back to earlier stages due to the five challenges of data migration. The biggest problem with migration project methodology is that it does not support the iterative nature of migrations. Further complicating the issue is that the technology used for data migration often consists of general-purpose tools repurposed for each of the four project stages. These tools are usually non-integrated and only serve to make difficult processes more difficult on top of a poor methodology. The ideal model for successfully managing a data migration project is not based on multiple independent tools. Thus, a cohesive method enables you to cycle or spiral your way through the migration process—analyzing the data, extracting and transforming the data, validating the data, and loading it into targets, and repeating the same process until the migration is successfully completed. This approach enables target-driven analysis, validating assumptions, refining designs, and applying best practices as the project progresses. This agile methodology uses the same four stages of analyze, extract/transform, validate and load. However, the four stages are not only iterated, but also interconnected with one another. An iterative approach is best achieved through a unified toolset, or platform, that leverages automation and provides functionality which spans all four stages. In an iterative process, there is a big difference between using a different tool for each stage and one unified toolset across all four stages. In one unified toolset, the results of one stage can be easily carried into the next, enabling faster, more frequent and ultimately less iteration which is the key to success in a migration project. A single platform not only unifies the development team across the project phases, but also unifies the separate teams that may be handling each different source system in a multi-source migration project. Architectures: federated versus shared Federated data integration can be very complicated. This is especially the case for distributed environments where several heterogeneous remote databases are to be synchronized using two-phase commit. Solutions that provide federated data integration access and maintain the data in the place wherever it resides (such as in a mainframe data store associated with legacy applications). Data access is done 'transparently' for example, the user (or application) interacts with a single virtual or federated relational database under the control of the primary RDBMS, such as Oracle. This data integration software is working with the primary RDBMS 'under the covers' to transform and translate schemas, data dictionaries, and dialects of SQL; ensure transactional consistency across remote foreign databases (using two-phase commit); and make the collection of disparate, heterogeneous, distributed data sources appear as one unified database. The integration software carrying out these complex tasks needs to be tightly integrated with the primary RDBMS in order to benefit from built-in functions and effective query optimization. The RDBMS must also provide all the other important RDBMS functions, including effective query optimization. Data sharing integration Data sharing-based integration involves the sharing of data, transactions, and events among various applications in an organization. It can be accomplished within seconds or overnight, depending on the requirement. It may be done in incremental steps, over time, as individual one-off implementations are required. If one-off tools are used to implement data sharing, eventually the variety of data-sharing approaches employed begin to conflict, and the IT department becomes overwhelmed with an unmanageable maintenance, which increases the total cost of ownership. What is needed is a comprehensive, unified approach that relies on a standard set of services to capture, stage, and consume the information being shared. Such an environment needs to include a rules-based engine, support for popular development languages, and comply with open standards. GUI-based tools should be available for ease of development and the inherent capabilities should be modular enough to satisfy a wide variety of possible implementation scenarios. The data-sharing form of data integration can be applied to achieve near real-time data sharing. While it does not guarantee the level of synchronization inherent with a federated data integration approach (for example, if updates are performed using two-phase commit), it also doesn't incur the corresponding performance overhead. Availability is improved because there are multiple copies of the data. Considerations when choosing an integration approach There is a range in the complexity of data integration projects from relatively straightforward (for example, integrating data from two merging companies that used the same Oracle applications) to extremely complex projects such as long-range geographical data replication and multiple database platforms. For each project, the following factors can be assessed to estimate the complexity level. Pretend you are a systems integrator such as EDS trying to size a data integration effort as you prepare a project proposal. Potential for conflicts: Is the data source updated by more than one application? If so, the potential exists for each application to simultaneously update the same data. Latency: What is the required synchronization level for the data integration process? Can it be an overnight batch operation like a typical data warehouse? Must it be synchronous, and with two-phase commit? Or, can it be quasi-real-time, where a two or three second lag is tolerable, permitting an asynchronous solution? Transaction volumes and data growth trajectory: What are the expected average and peak transaction rates and data processing throughput that will be required? Access patterns: How frequently is the data accessed and from where? Data source size: Some data sources of such volume that back up, and unavailability becomes extremely important. Application and data source variety: Are we trying to integrate two ostensibly similar databases following the merger of two companies that both use the same application, or did they each have different applications? Are there multiple data sources that are all relational databases? Or are we integrating data from legacy system files with relational databases and realtime external data feeds? Data quality: The probability that data quality adds to overall project complexity increases as the variety of data sources increases. One point of this discussion is that the requirements of data integration projects will vary widely. Therefore, the platform used to address these issues must be a rich superset of the features and functions that will be applied to any one project.

0
0
2387

Packt

01 Aug 2011

8 min read

Oracle Tools and Products

Packt

01 Aug 2011

8 min read

0
0
7298

article-image-integrating-kettle-and-pentaho-suite

Packt

14 Jul 2011

13 min read

Integrating Kettle and the Pentaho Suite

Packt

14 Jul 2011

13 min read

0
0
6683

article-image-getting-started-oracle-goldengate

Packt

12 Jul 2011

8 min read

Getting Started with Oracle GoldenGate

Packt

12 Jul 2011

8 min read

What is GoldenGate? Oracle GoldenGate is Oracle's strategic solution for real-time data integration. GoldenGate software enables mission-critical systems to have continuous availability and access to real-time data. It offers a fast and robust solution for replicating transactional data between operational and analytical systems. Oracle GoldenGate captures, filters, routes, verifies, transforms, and delivers transactional data in real-time, across Oracle and heterogeneous environments with very low impact and preserved transaction integrity. The transaction data management provides read consistency, maintaining referential integrity between source and target systems. As a competitor to Oracle GoldenGate, data replication products, and solutions exist from other software companies and vendors. These are mainly storage replication solutions that provide a fast point in time data restoration. The following is a list of the most common solutions available today: EMC SRDF and EMC RecoverPoint IBM PPRC and Global Mirror (known together as IBM Copy Services) Hitachi TrueCopy Hewlett-Packard Continuous Access (HP CA) Symantec Veritas Volume Replicator (VVR) DataCore SANsymphony and SANmelody FalconStor Replication and Mirroring Compellent Remote Instant Replay Data replication techniques have improved enormously over the past 10 years and have always been a requirement in nearly every IT project in every industry. Whether for Disaster Recovery (DR), High Availability (HA), Business Intelligence (BI), or even regulatory reasons, the requirements and expected performance have also increased, making the implementation of efficient and scalable data replication solutions a welcome challenge. Oracle GoldenGate evolution GoldenGate Software Inc was founded in 1995. Originating in San Francisco, the company was named after the famous Golden Gate Bridge by its founders, Eric Fish and Todd Davidson. The tried and tested product that emerged quickly became very popular within the financial industry. Originally designed for the fault tolerant Tandem computers, the resilient and fast data replication solution was in demand. The banks initially used GoldenGate software in their ATM networks for sending transactional data from high street machines to mainframe central computers. The data integrity and guaranteed zero data loss is obviously paramount and plays a key factor. The key architectural properties of the product are as follows: Data is sent in "real time" with sub-second speed. Supports heterogeneous environments across different database and hardware types. "Transaction aware" —maintaining its read-consistent and referential integrity between source and target systems. High performance with low impact; able to move large volumes of data very efficiently while maintaining very low lag times and latency. Flexible modular architecture. Reliable and extremely resilient to failure and data loss. No single point of failure or dependencies, and easy to recover. Oracle Corporation acquired GoldenGate Software in September 2009. Today there are more than 500 customers around the world using GoldenGate technology for over 4000 solutions, realizing over $100 million in revenue for Oracle. Oracle GoldenGate solutions Oracle GoldenGate provides five data replication solutions: High Availability Live Standby for an immediate fail-over solution that can later re-synchronize with your primary source. Active-Active solutions for continuous availability and transaction load distribution between two or more active systems. Zero-Downtime Upgrades and Migrations Eliminates downtime for upgrades and migrations. Live Reporting Feeding a reporting database so as not to burden the source production systems with BI users or tools. Operational Business Intelligence (BI) Real-time data feeds to operational data stores or data warehouses, directly or via Extract Transform and Load (ETL) tools. Transactional Data Integration Real-time data feeds to messaging systems for business activity monitoring, business process monitoring, and complex event processing. Uses event-driven architecture and service-oriented architecture (SOA). The following diagram shows the basic architecture for the various solutions available from GoldenGate software: We have discovered there are many solutions where GoldenGate can be applied. Now we can dive into how GoldenGate works, the individual processes, and the data flow that is adopted for all. Oracle GoldenGate technology overview Let's take a look at GoldenGate's fundamental building blocks; the Capture process, Trail files, Data pump, Server collector, and Apply processes. In fact, the order in which the processes are listed depicts the sequence of events for GoldenGate data replication across distributed systems. A Manager process runs on both the source and the target systems that "oversee" the processing and transmission of data. All the individual processes are modular and can be easily decoupled or combined to provide the best solution to meet the business requirements. It is normal practice to configure multiple Capture and Apply processes to balance the load and enhance performance. Filtering and transformation of the data can be done at either the source by the Capture or at the target by the Apply processes. This is achieved through parameter files. The capture process (Extract) Oracle GoldenGate's capture process, known as Extract, obtains the necessary data from the databases' transaction logs. For Oracle, these are the online redo logs that contain all the data changes made in the database. GoldenGate does not require access to the source database and only extracts the committed transactions from the online redo logs. It can, however, read archived redo logs to extract the data from long-running transactions. The Extract process will regularly checkpoint its read and write position, typically to a file. The checkpoint data insures GoldenGate can recover its processes without data loss in the case of failure. The Extract process can have one of the following statuses: STOPPED STARTING RUNNING ABENDED The ABENDED status stems back to the Tandem computer, where processes either stop (end normally) or abend (end abnormally). Abend is short for an abnormal end. Trail files To replicate transactional data efficiently from one database to another, Oracle GoldenGate converts the captured data into a Canonical Format which is written to trail files, both on the source and the target system. The provision of the source and target trail files in the GoldenGates architecture eliminates any single point of failure and ensures data integrity is maintained. A dedicated checkpoint process keeps track of the data being written to the trails on both the source and target for fault tolerance. It is possible to configure GoldenGate not to use trail files on the source system and write data directly from the database's redo logs to the target server data collector. In this case, the Extract process sends data in large blocks across a TCP/IP network to the target system. However, this configuration is not recommended due to the possibility of data loss occurring during unplanned system or network outages. Best practice states, the use of local trail files would provide a history of transactions and support the recovery of data for retransmission via a Data Pump. Data pump When using trail files on the source system, known as a local trail, GoldenGate requires an additional Extract process called Data pump that sends data in large blocks across a TCP/IP network to the target system. As previously stated, this is best practice and should be adopted for all Extract configurations. Server collector The server collector process runs on the target system and accepts data from the source (Extract/Data Pump). Its job is to reassemble the data and write it to a GoldenGate trail file, known as a remote trail. The Apply process (Replicat) The Apply process, known in GoldenGate as Replicat, is the final step in the data delivery. It reads the trail file and applies it to the target database in the form of DML (deletes, updates, and inserts) or DDL*. (database structural changes). This can be concurrent with the data capture or performed later. The Replicat process will regularly checkpoint its read and write position, typically to a file. The checkpoint data ensures that GoldenGate can recover its processes without data loss in the case of failure. The Replicat process can have one of the following statuses: STOPPED STARTING RUNNING ABENDED * DDL is only supported in unidirectional configurations and non-heterogeneous (Oracle to Oracle) environments. The Manager process The Manager process runs on both source and target systems. Its job is to control activities such as starting, monitoring, and restarting processes; allocating data storage; and reporting errors and events. The Manager process must exist in any GoldenGate implementation. However, there can be only one Manager process per Changed Data Capture configuration on the source and target. The Manager process can have either of the following statuses: STOPPED RUNNING GGSCI In addition to the processes previously described, Oracle GoldenGate 10.4 ships with its own command line interface known as GoldenGate Software Command Interface (GGSCI). This tool provides the administrator with a comprehensive set of commands to create, configure, and monitor all GoldenGate processes. Oracle GoldenGate 10.4 is command-line driven. However, there is a product called Oracle GoldenGate Director that provides a GUI for configuration and management of your GoldenGate environment. Process data flow The following diagram illustrates the GoldenGate processes and their dependencies. The arrows largely depict replicated data flow (committed transactions), apart from checkpoint data and configuration data. The Extract and Replicat processes periodically checkpoint to a file for persistence. The parameter file provides the configuration data. As described in the previous paragraphs, two options exist for sending data from source to target; these are shown as broken arrows: Having discovered all the processes required for GoldenGate to replicate data, let's now dive a little deeper into the architecture and configurations.

0
0
6535

How-To Tutorials - Data

SAP Netweaver: Accessing the MDM System

SAP NetWeaver: MDM Scenarios and Fundamentals

NHibernate 3: Creating a Sample Application

Oracle E-Business Suite: Entering and Reconciling Bank Statements

Oracle E-Business Suite: Adjusting Items in Inventory and Classifying Items

Oracle E-Business Suite: Creating Bank Accounts and Cash Forecasts

Oracle E-Business Suite: Creating Items in Inventory

Terms and Concepts Related to MDX

Oracle Integration and Consolidation Products

Performing Common MDX-related Tasks

Trending Topics

How to Perform Iteration on Sets in MDX

Getting Started with Oracle Information Integration

Oracle Tools and Products

Integrating Kettle and the Pentaho Suite

Getting Started with Oracle GoldenGate

Create a Free Account To Continue Reading

Sign in to activate your 7-day free access