Enhancing the Effectiveness of Software Development Projects

September 2007

By providing independent assessments of software development projects, internal auditors can make a significant contribution to IT project success and effectiveness.

Constantine Photopoulos
Partner
The SOX Group

After 12 months of hard work, the moment has finally arrived. Today the Heartland Company, an IT software developer, will complete Company AMA's new client database. This much-anticipated tool will enable AMA to use client data in new ways by exporting information in multiple formats and enabling its use from any remote location. However, two days before the due date, the project manager noticed the export function didn't work as expected. "No big deal," he thought. "This should be fixed in less than an hour." Seven days later, the software developer — who was hired to work on this project after the original subcontractor didn't meet timeline deliverables — was still trying to figure out what was wrong with the code. Now the project is five days behind schedule and the company's reputation hangs in a balance. If only a software development project audit had been performed, the problem would have been caught earlier.

Software development project audits help organizations to determine whether software initiatives follow established development methodologies and procedures, meet organizational needs, and include adequate security and management controls. Planning, methodology assessments, reports of project results, and post-implementation reviews are four key steps to success in this area. Following is a description of these steps and how they can enhance the functionality of any business application.

PLANNING

Planning is an important aspect of any audit because it helps internal auditors determine which projects need to be reviewed and how to proceed with the audit. In addition, reviewing the annual audit plan enables auditors to determine which software development initiatives are under way so that they can become a valuable member of the project team.

After reviewing the audit plan, auditors should contact the project manager and business owner to inform them of their wish to be involved in the project. During this meeting, auditors should also discuss the criticality and risk of the application chosen for review and any compliance requirements that need to be met. As a general guideline, auditors should try to become involved in the project as close as possible to the start date to ensure that issues are addressed early and that adequate controls are built into the application. This is because it is more expensive to retrofit fixes and controls into existing applications than to ensure they are implemented correctly in the first place.

Furthermore, becoming involved early in the project will enable management to keep auditors updated on any project meetings and planning sessions, as well as help auditors to obtain the documentation they need to perform the audit. This, in turn, will help to establish the senior management support needed to allow the audit to be successful and the recommendations to be implemented.

Once a project is selected, the audit should be planned by identifying:

The kind of review to be performed (i.e., pre-implementation, parallel or concurrent, or post-implementation).
Any existing legal, regulatory, and policy aspects the company must comply with.
The review's scope.
The review's start and end dates.
The process for reporting observations and recommendations, as well as for following up on agreed actions.

METHODOLOGY ASSESSMENTS

The second step in the audit is to determine whether the organization has a formal software development life cycle (SDLC) document — a structured methodology that guides the development of application systems. The SDLC typically begins with the project's initiation; extends throughout the project's feasibility study and business requirements and functional specifications phases; continues with the review of the different development stages (i.e., systems design and development, testing, data conversion and migration, and implementation); and ends with an assessment of the entire project after its implementation.

Throughout the entire SDLC, auditors need to ensure that relevant tasks and activities are described clearly, a measurable end product or deliverable is defined, and the individual or role responsible for the execution of the phase is identified along with the required approvals and sign-offs before proceeding to the next phase. Finally, auditors need to review any documents that are available as part of the SDLC. Two important documents include checklists that cover each phase of the process and templates that are associated with project deliverables. If an SDLC methodology is not available, the audit should be conducted against the organization's current best practices with a recommendation that an SDLC be developed and implemented to guide future software projects.

Project Initiation
Software development projects are usually initiated through a formalized project request by management. Auditors need to ensure that the project request is present and includes the following:

An overview of the business problem being addressed or the company need.
The desired system features.
The expected system users.
A high-level cost and benefit estimate.
The criticality of the proposed system.
The timeframe for completion.
The approval of the senior executive who is authorized to initiate requests.

As a general best practice, IT departments should have a formalized system to receive, organize, and prioritize user requests. The auditor's role is to determine if the timeframe within which the request was approved was adequate and whether the users were notified of any decision in a timely manner.

Feasibility Study
As a preliminary assessment, feasibility reviews determine a project's viability, whether or not to proceed with system development, and any alternative approaches for the application's creation. This is generally required for major development or enhancement projects to ensure the solution chosen is economically and technologically appropriate. When reviewing a feasibility study, auditors need to determine if it addresses:

The software's business objectives.
The project's priority and impact.
Any alternative courses of action and their evaluation based on technical, financial, and operational factors, such as time and cost estimates, resource requirements, a cost and benefit analysis, and risks.
The proposed solution, which can include a recommendation that the project is not developed.
The start date and completion timeframes.
Reviews and signoffs by appropriate management.

Business Requirements
As part of the SDLC, the project team needs to have a clear picture of the requirements that need to be met. These requirements should describe current and future business needs to ensure they are understood and addressed before developing the system. Additionally, these requirements should detail the business functions that the system is required to support, usually expressed in terms of broad outcomes rather than specific functions the system must perform. During this phase, auditors need to identify if business requirements document current and future business needs clearly; identify security, control, and privacy issues; include feedback from all potential user areas; and have user and IT management approvals.

Functional Specifications
The functional specifications document describes in detail a product's intended functionality from a technical point of view. In this document, business requirements are analyzed and converted to produce a preliminary design of the proposed system. As part of the review process, auditors need to determine whether functional specifications include the following elements:

Expected data input, processing, and output.
A mockup and design of user interface screens.
Reports, including audit and logging reports.
Interfaces to other systems.
Internal control and security requirements.
Transaction volumes.
New hardware and software requirements.
Infrastructure and support considerations.
IT management reviews and approvals.

System Design and Development
Before actual programming begins, a system design document needs to be created that provides a detailed technical description of the proposed application and gives the development team overall guidance on the actual software architecture. This document also should describe the desired software features in detail. As part of their work, auditors need to review the system design document and determine whether:

Inputs are defined in detail, including screen designs.
Outputs and reports are fully defined.
The functional logic of program modules is documented.
Data entities, relationships, and flows are documented.
Controls are in place to prevent unauthorized access and to maintain audit trails of user activity.
Responsible departments signed off on the design.

Any changes to functionality during the system design phase should require the functional specifications document to be updated and reviewed by the user. This should be addressed through the established change management process. As far as the actual programming effort is concerned, auditors need to make sure that development takes place in an environment that is separate from the production environment. Finally, if the programming effort is partially or completely outsourced to a third party, auditors need to assess the adequacy of and compliance with outsourcing controls.

Testing
A testing strategy must be developed and followed for new or updated applications. At a minimum, testing should consist of the following unit testing (i.e., testing that validates whether individual programs or modules are working properly), system testing (i.e., testing conducted on a complete system to evaluate its compliance with specified requirements), and user acceptance testing (i.e., testing performed by end users to determine whether the software meets business requirements).

Throughout the testing phase, auditors need to identify whether written test plans and scripts are available and adequate. For instance, test plans should include the test's set-up and scope, testing procedures and data, expected results, and sign-off from the appropriate staff. In addition, auditors need to determine whether testing is performed in an environment that is separate from the production environment, whether test results are logged, and whether tests having unexpected results are retested adequately. Auditors also need to keep in mind that nothing should be installed in the production environment until it is successfully examined in a test environment and formally approved by the business user.

Data Conversion and Migration
If the tool requires data to be converted from an existing system, auditors need to obtain and review the conversion plan. At a minimum, auditors need to determine if the source data is identified, verified, and correctly mapped to the new system; data conversion and migration is tested to confirm it is complete, accurate, and valid; and any data conversion and migration mismatches have been logged and corrected.

Implementation
In this phase, the new system is installed and made operational in the production environment. This phase should be initiated after successful user acceptance testing and sign off. During the review, auditors need to determine whether:

A detailed implementation plan was developed and followed.
A rollback contingency plan is available.
The approval of business, development, and operations management was obtained.
User and operation documentation is available.
User training took place prior to the system's implementation.
Support personnel are available and accessible to users.
Software backup procedures are in place.

REPORTING AUDIT RESULTS

While conducting the audit, auditors can submit interim reports as issues are identified or in a final document that lists all the issues raised during the review. In either case, the audit plan should include timely notification of any control weaknesses so that they can be resolved prior to system implementation.

Audit reports should address any risks and issues, including their causes, effects, and suggested mitigation actions; IT control strengths and weaknesses; and lessons learned that can be applied to future projects. In addition, auditors need to discuss their findings with management and obtain their commitment to ensure that corrective actions are implemented and deadlines for remedying significant deficiencies are followed. To this end, auditors should follow up after an appropriate period of time prior to the product's live date to make sure that issues were resolved satisfactorily.

POST-IMPLEMENTATION REVIEWS

A post-implementation review is performed to determine whether the system adequately meets identified business requirements and needs. After a few months of live operation, auditors should be able to determine if the expected benefits of the new system were realized and whether users are satisfied with the new system.

At this stage, auditors need to review the problems that occurred throughout the project and whether any corrective actions taken were adequate. If differences remain between expectations and actual results, auditors need to determine if they were due to any misunderstanding of business needs or incomplete user requirements; inadequate design, programming, or testing; and the need for additional user training. Each of these items can help auditors to evaluate the current situation and offer guidelines for future projects.

MOVING FORWARD

By following a systematic approach like the one discussed in this article, auditors can provide an accurate and objective assessment of software development projects and indirectly increase the likelihood of their success. In addition, audit observations and recommendations can prevent the organization from repeating mistakes in future projects as well as produce useful software development best practice guides. Thus, auditors can become an integral part of the software development process, help organizations to meet project deadlines, and optimize the productivity of software tools. This, is turn, helps the organization to make sure software tools are effective and are not a drain on the company's budget.

Constantine Photopoulos is a partner and senior IT auditor with The SOX Group, a New York-based consulting firm specializing in IT security and compliance. His background includes extensive auditing and software development experience in large organizations.

Improving Audit skill and add value of audited projects
Hi Constantine, Thanks for your valuable input. It would be great if you could publish some sample audit report (removing sensitive data and keeping it subjective) in a manner to understand and improve our skill to next level,please. Sushil
Posted By: Sushil Vishwakarma
2011-07-20 8:39 AM

Internal Auditor is pleased to provide you an opportunity to share your thoughts about the articles posted on this site. Some comments may be reprinted elsewhere, online, or offline. We encourage lively, open discussion and only ask that you refrain from personal comments and remarks that are off topic. Internal Auditor reserves the right to edit/remove comments.

To make something bold:
<strong>Text to bold</strong>

To make something italic:
<em>Text to italicize</em>

To make a hyperlink:
<a href="URL">Text to link</a>