Vol. 10, January 10, 2007

Print Article Print Entire Issue

11 Steps to an Effective FTP Audit

Identifying and automating file transfer protocol activities are two of the steps organizations can take to protect sensitive data that is transmitted through this increasingly used technology.

Scott Myers
President, Software Assist Corp.

File transfer protocol (FTP) is a widely used procedure for transmitting files from one computer platform to another. It is used heavily in multisystem environments and often performs a critical role in a company's day-to-day operations. Although most FTP servers have the ability to accept secured connections, these connections are the exception, not the rule. As levels of FTP activity continue to grow, auditing FTP use becomes more important every day. Performing an effective FTP audit that incorporates recommendations to enhance file transfer activities can give organizations a head start by identifying security data problems before they occur.

WHY AUDIT FILE TRANSFER ACTIVITIES?

FTP is a convenient way to transmit data from one system to another and is an integral aspect of business operations in many organizations. It is not uncommon for large organizations to make thousands, and in some cases, millions of FTP transmissions daily. Furthermore, FTP use eases the way data is sent and received from one system to another. Virtually every operating system has a built-in FTP client that enables FTP server connections. All of the popular Internet browsers also have been supporting FTP connections for some time.

Despite these advantages, the problem with FTP is that the overwhelming majority of file transfer activities are unsecured, which may result in the exposure of login information and unencrypted data traveling in plain text format. This unencrypted data can be captured and viewed by a network or packet sniffer running on any computer onthe network between the FTP client and FTP server. In addition, FTP use makes it easy to send files to people outside the company's network. All that is required is read-level access to data and an Internet connection for someone to be able to send the data to an FTP server virtually anywhere in the world.

As a result, new compliance rules are directing internal auditors to take a closer look at their organization's use and management of FTP activities and the controls used to protect individuals from the disclosure of sensitive information. This tougher compliance landscape, combined with the alarming number of recent data breaches, has created a pressing need for internal auditors to take a closer look at data security procedures and identify and address the exposure that FTP use creates.

PERFORMING THE FTP AUDIT

Compliance in today's highly regulated IT environment requires companies to ensure the security of sensitive data is maintained throughout the file transfer process and that only intended data is transmitted to the designated destination. Consequently, auditors are responsible for determining if proper controls are in place and regular FTP audits are performed. More specifically, regulations are asking internal auditors to perform regular, end-to-end reviews of all FTP transmissions and regular network audits to detect new FTP servers. In addition, compliance regulations are requiring that companies maintain a long-term historical record of all file transfer activity, preferably in a manner that facilitates auditing and management of large volumes of FTP activity; ensure sensitive data is transmitted securely; and develop and implement plans that explain what steps need to be taken in the event of a data exposure.

Performing an effective FTP audit that addresses these and other compliance requirements will help organizations stay one step ahead of the game when a breach occurs and prevent certain risks from occurring. The following 11 steps can help auditors conduct a more effective and comprehensive FTP audit. For a list of audit exceptions to these 11 steps, read FTP Audit Exceptions (PDF, 2 KB).

1. Identify and Locate All FTP Servers Used Companywide.

Auditors need to work with network administrators and other IT staff to identify and locate all FTP servers by running a network scanner monthly. Free network scanning tools, such as FTP Auditor can be employed to look for rogue FTP servers. These tools should scan networks, identify FTP servers, and check to see if these servers allow anonymous login to take place. While trying to locate all FTP servers in use, the auditor may find servers on the network that were not set up by or are not managed by the IT department.

2. Determine if All FTP Transactions Are Logged.

Once the auditor identifies the FTP servers used by the company, he or she should make sure that FTP logs are enabled on each server. If the log functionality is not enabled, there will be no record of what data was sent and received through the server. In servers using IBM's z/OS mainframe operating system, FTP activity is logged in System Management Facility (SMF) records. For guidance on how to tell whether FTP activity is being logged on an z/OS FTP server, read SMF Logging of FTP Activity (PDF, 14 KB).

Environments using a distributed system server may have different FTP servers in use. Each FTP server can use a different method for indicating whether logs are enabled. To determine how the activity is being logged, auditors should check the documentation for the FTP servers the organization are running. Documentation for most FTP servers is publicly available on the Internet. If an FTP server is found that has no documentation, auditors should recommend having it replaced by an FTP server that meets company standards.

3. Identify and Locate Log Files for All FTP Servers.

Log files maintained by FTP servers provide an important audit trail. In the event there is an FTP-related breach, the task force that is established to deal with the breach needs to access and understand the information in the logs to explain what happened, identify who was responsible, and assess the impact of the breached data.Various FTP servers store log data differently, often in distinct formats. Some FTP servers log information in a recognizable or "eye-readable" text format, while others log data in binary formats. Table 1 describes the different default log formats used on today's most common FTP server platforms.

Table 1: Default log files based on common FTP server types
(Note: It also is possible with many FTP servers to override the location and file name of the log file.)

4. Monitor Files Containing Sensitive Data.

One of the most critical items auditors need to examine is whether sensitive data is being transmitted through an FTP server. Examples of sensitive data include patient health records, customer financial data, and company intellectual property. Auditing sensitive data transmission across country borders is complicated further by the difference in privacy regulations from one country to another. Because some countries forbid the transmission of privacy data all together, special policies need to be implemented to allow for these activities to take place.

Currently, the only effective way to identify files containing sensitive data is to look at the file name and check the details on all sensitive data transmissions. Auditors need to determine who is initiating the transmissions of sensitive data, where the sensitive data is going to or coming from, and whether the transmissions are secured.

It is important for the auditor to work with the company's application software group to develop a list of the datasets and file names containing the sensitive data. This will enable auditors to determine whether sensitive data transmissions are taking place to and from an authorized location. Once an organization has identified where its sensitive data is located, it will be in a position to begin auditing its use and file transmission access.

5. Recommend Using a Central Location for FTP Data Analysis and Archiving.

The large volume of FTP log files in use may pose data management challenges. FTP use logs can be in various formats and can be located on different platforms across the organization. Merging all FTP files into a central repository will facilitate end-to-end auditing of FTP use. Because the organization may have servers operating in different time zones, it may be important to reset the dates and times in the FTP use data to a common time zone when comparing file transfer activity across the enterprise. In addition, the IT department will need to demonstrate all FTP activity is being logged and archived companywide. Auditors should recommend that IT staff demonstrate they can provide reports of historic FTP use if the need arises.

6. Consider the Use of Real-time Monitoring, Alerting, and Automation of FTP Activity.

Monitoring FTP use in real time enables the organization to generate alerts when suspicious activity takes place and tie in FTP processes into overall automation efforts. Real-time monitoring also can be useful in identifying FTP hacking attempts, unsecured or unapproved transmissions of sensitive data, and failed FTP transmissions that might affect production processing.

Furthermore, real-time monitoring can provide an important missing component in data center automation by generating alerts that can be used to escalate problems for human intervention, start processes based on the successful or failed completion of an FTP transmission, and inform stakeholders when FTP events impact their area of interest. For more information on these benefits, read Advantages of Real-time Monitoring (PDF, 2 KB). 

7. Schedule Regular IT Audits of FTP Use.

Auditors should recommend that IT departments perform regularly scheduled, comprehensive, end-to-end reviews of FTP use. At first, audits should focus on validating that FTP use is taking place in line with company policy and that sensitive data is transmitted to and from approved locations using a secured connection. Later on, audits can focus on what changes, if any, have taken place.

Generally, these audits will involve large volumes of FTP activity data, so a manual analysis may not be effective. Auditors may recommend the use of software tools to simplify the task by enabling interactive analysis of the FTP data and supporting exception analysis. These reviews should answer:

1. What sensitive data is being transmitted via the FTP?
2. Who is initiating the transmissions of sensitive data?
3. Are secured connections used to transmit all sensitive data?
4. Is the sensitive data coming from and going to an approved location only?
5. Are there indications of any hacking attempts?
6. Who are the largest FTP users?
7. Do all FTP transactions comply with corporate policy?
8. Are FTP errors and delays impacting negatively production processing and service-level agreements?

8. Review Anonymous FTP Connections.

Most FTP servers allow anonymous access — the person specifies a username of "anonymous" when logging onto the FTP server and supplies an e-mail address as a password. Using an anonymous FTP connection is a convenient way to make data available without having to maintain a burdensome list of usernames and passwords. Anonymous connections also are enabled to save time when setting up the FTP server.

The trade-off, however, is that anyone with network access to log into the FTP server supporting the anonymous connection can download any of the files that are available to anonymous users. In fact, the e-mail address that is supplied as a password is not validated, so any text string that follows an e-mail address format generally will suffice. In addition, depending on how the FTP server's security settings are configured, data can be uploaded to the server that might be available universally. FTP servers that are set up by individual business units are more likely to be configured this way than servers that are set up by the IT department.

An FTP server that supports anonymous FTP connections requires a higher level of care and monitoring to ensure that no sensitive or protected data resides in the folders that are accessible by the server. If uploading is allowed, further monitoring must be performed to ensure that no sensitive or protected data is uploaded to server folders. If an anonymous FTP connection is not critical to the business operation, auditors should recommend that this option be disabled. Auditors also should recommend that the anonymous login option be left in place only if the business benefit outweighs the risk it creates.

9. Recommend Implementing a Secured FTP.

The company's goal should be to have all FTP transmissions sent using a secured connection. Using secured FTP connections will ensure that login information and data are not accessible to network sniffing tools. Because nearly all FTP servers support secured socket layer or transport layer security connections (i.e., protocols for transmitting private documents via the Internet or for ensuring privacy between applications, respectively), having a secure FTP connection is only a matter of converting the FTP users to these technologies. An alternative approach is to replace the FTP with a managed file transfer solution — a program that enables users to move data of any type securely over the Internet. Although this is a more expensive option, it may be appropriate in certain circumstances.

10. Review FTP Server Settings.

Auditors should review all FTP server settings by ensuring that access to sensitive data is controlled properly through the user access settings. Generally, it is best to grant FTP server users the minimum access they need to get the job done. Furthermore, because FTP servers are popular targets for password cracker programs, IT departments need to restrict access by Internet protocol address whenever possible and disable hacked accounts as soon as possible to ensure unauthorized users don't get logged in. Auditors also need to verify strong passwords are required where supported by the FTP server and FTP sessions time out automatically if they remain idle for a period of time. If possible, auditors need to monitor that account lockouts are enabled so accounts will be disabled automatically after experiencing an excessive number of login failures.

11. Verify That FTP Server Settings Follow Change Management Procedures.

After the auditor has reviewed all FTP server settings, it is important for the organization to account for any setting changes in the future. Therefore, whenever possible, auditors should identify whether FTP server settings are secured properly and under some form of change management control. Change management control systems maintain logs indicating when settings were changed and by whom. By doing this, the organization ensures that an audit trail is present in the event that FTP server settings are changed.  

FTP AUDITS ARE IMPORTANT

Data breaches are not cheap. The cost of a data breach can be financially crippling to an organization, especially when all the fines, penalties, and profit losses are taken into account. As many high-profile companies have learned recently, a little preparation and ongoing monitoring can go a long way toward keeping the company's name from being tarnished by negative press.

Data breaches also are affecting the audit profession. Internal auditors are being held accountable for ensuring data assets are secured and protected properly. By doing their job thoroughly and requiring IT departments to monitor FTP activities regularly, auditors can help companies ensure that the proper steps have been taken to minimize the chances of a data breach. Therefore, audits of FTP activity need to take place whether or not the organization has encountered any problems in the past.

As the number of security threats continues to grow, organizations need to be proactive to deter as many risks as possible. Completing these 11 steps will enable the organization to be in a better position to ensue FTP activities are not used as a means to gain access to corporate data assets.

Scott Myers is the president of Software Assist Corp. and has more than 20 years of experience developing z/OS data center software. For the past three years, Myers has focused his attention on FTP issues.

_______________________________________________________________________________________