EBS Implementation Audit

With some imagination, Data Analytics can be used to aid decision making in almost any aspect of business.

Synopsis

We were invited by a Fortune 100 client to audit an implementation of Oracle EBS in a new market whose go-live was imminent. We analyzed the testing coverage, conversions, setups and configurations, transactions and error logs to determine the quality of the implementation and whether it was ready to go live. Among a large number of issues and discrepancies, we identified five critical areas in which the implementation could fail. All conclusions were data driven and irrefutable. We also provided strategies to recover the system if these areas failed. True to our prediction, the five areas we identified had issues after go-live, but the client was able to prevent major problems because they had been forewarned.

Background

A Fortune 100 client was implementing Oracle EBS in a new and key market segment. They had used EBS for their other markets for a few years and wanted to expand to this new market. Towards the end of the implementation, when go-live was in sight, we were invited to audit the implementation and provide input to the go / no-go decision. This document focuses only on the use of data and analytics for auditing the implementation.

Discovery

We entered during the integration-testing phase, when development and configuration had been completed. The first warning signs were that, despite the advanced stage of the project, the project team was extremely busy and documentation was difficult to get. However, we had access to the last round of testing, the user-testing environment and the existing production environment.

Audit Approach

We collected data from the system as well as from people to start our analysis:

• Project documents such as Requirements, Design documents, Test Strategy and Plans, and the Traceability Matrix
• Issue logs
• Interviews with key participants

Configurations and Setups: The quality and comprehensiveness of the application configuration and setups were analyzed by comparing them with global setups, localizations and setups required for specific business requirements. The setups in the testing environment were compared to the existing production environment and to the expected configuration for market-specific localizations.

Data migration: The quality of the data migration was audited by comparing the percentage of successful conversions. We compared the converted data column by column to global and similar markets, and checked how the master and transaction data performed in the end-to-end processes.

Extent of testing: We compared the test environment to production to evaluate whether all significant production scenarios were adequately tested.

Transactions: We compared end-to-end transactions in testing to production to check the coverage of business processes — analyzing parts, customers, order types, line types and vendors, whether all sources were used, and how many transactions ran successfully end to end.

Information Proxies: Sources of the various transactions in a test environment can be used as a proxy for the extent and coverage of testing. For example, sources of posted journal entries show the number of upstream transactions completed; sources of sales orders show the extent of manual quote conversions and EDI testing; sources of AP transactions show whether all interfaces were tested.

Errors: Concurrent-program error logs and custom error logs were analyzed to understand failures and possible root causes.

Controls: We analyzed document and code review, issue logging and resolution, code versioning and promotion, and how often (and how late in the test cycle) custom programs were modified and whether those changes were authorized.

Findings and Conclusions

We found and reported a large number of issues and discrepancies in the data, testing and setups, along with recommendations for fixing critical issues and for future implementations. The following show two of the most compelling findings.

Relying on data, we were able to isolate five of the numerous programs for extensive deep dives. Because everything was backed with data, the client found it easy to accept the findings as facts rather than opinion. It was clear that the process areas covered by these five customizations were not ready to go live. Our recommendation was to go live but have backup manual processes ready to cover these five programs, and to prepare a plan to stabilize them.

As it turned out, the go-live was largely successful; however, the five programs we had identified had to be redesigned and redeployed.

← Back to all stories