Data retention is the collection, storage, and management of data. Businesses, organizations, and governments have policies, regulations, and laws that define how data must be stored and for how long. The drivers for data retention programs include compliance, disaster recovery, and the need to feed analytic engines.
Data retention requirements can be grouped into four categories.
- Government regulations, such as those set forth by the FTC and IRS
- International standards as set for by ISO (the International Organization for Standardization), such as ISO/IEC 27040, ISO 9001 ISO 17068:2017, and 27001
- Industry regulations, such as GDPR, PCI-DSS, and CCPA
- Internal policies, such as version control and employee record retention
Let’s jump in and learn:
Data Retention Policies Worldwide
Globally, data retention requirements range from very specific to nearly nonexistent. To provide some perspective on the range of requirements, following is a review of data retention directives in several countries.
Australia’s Data Retention Requirements
The primary driver of data retention policies in Australia is the Telecommunications (Interception and Access) Amendment (Data Retention) Act 2015. According to this law, Australian service providers must retain metadata for at least two years.
To adhere to Australia’s data retention law, the following metadata must be stored:
- Identification information about the account holder, such as name, address, and billing details
- Information about the source and destination of a communication
- Date, time, and duration of a communication
- Recipient of communications
- Type of communication—voice, SMA, or email
- Type of relevant service used for communication—ADSL, WiFi, VoIP, or cable
- Location of equipment or line used at both ends of a communication, including cell towers and WiFi hotspots
China’s Data Retention Requirements
In China, the principal legislation relating to data protection and retention is the Cybersecurity Law of the People’s Republic of China (also known as the Cybersecurity Law or CSL). The CSL does not have requirements for data retention beyond requiring network operators only to collect personal data necessary for the provisioning of their customers.
EU’s Data Retention Requirements
The EU General Data Protection Regulation or GDPR is far-reaching legislation that significantly impacts data retention worldwide as it applies to any transaction made in the EU. Rather than dictating a minimum period to retain data, GDPR focused on the destruction of personal data. According to the GDPR, personal data may only be kept in a form that permits identification of the individual for no longer than is necessary for the purposes for which it was processed.
There are considerations in the GDPR for organizations that want to retain personal data. Anonymized data can be retained indefinitely. In addition, data retention can be extended in cases where the data is retained for archiving purposes in the public interest, scientific or historical research purposes, or statistical purposes.
The GDPR also requires organizations to develop a data retention policy. This must include details about processes and how personal data is managed.
Russia’s Data Retention Requirements
Russia’s Federal Law of 27 July 2006 No. 152-FZ on Personal Data (also known as the Law on Personal Data) does not specify data retention periods. Like the GDPR, the Law on Personal Data’s direction is that personal data should not be kept/processed for a longer period than is necessary for the purpose for which it is processed.
Switzerland’s Data Retention Requirements
Several codes and ordinances define Switzerland’s data retention requirements. These include:
- Code of Obligations
- Ordinance on Commercial Bookkeeping and Retention
- Data Protection Act
- Value-Added Tax Act
- Criminal Code
According to Swiss data retention regulations, business data must be retained for 10 years after the end of the financial year. This applies to dissolved companies.
There is an exception that requires records related to immovable assets with VAT implications to be retained for at least 20 years. Switzerland’s data retention rules must be adhered to by a number of organizations, including:
- Commercial companies, such as general partnerships, limited partnerships, and limited companies (e.g., AG, GmbH)
- Businesses run on a commercial basis with turnover greater than CHF 100,000
- All businesses dealing in cash transactions, bill brokerage, securities, stock market trades, and collections, including factors, agents, brokers, trustees, administrators, message transfer agents, and insurance companies
Mobile phone operators in Switzerland must retain certain data for six months. According to Swiss data retention rules, the following data must be saved:
- Phone numbers of incoming and outgoing calls
- Subscriber Identity Module (SIM), International Mobile Subscribers Identity (IMSI), and International Mobile Equipment Identity (IMEI-numbers)
- Location and the electrical boresight of the antenna of the mobile phone with which they connected
- Date, time, and duration of the connection
Service providers must comply with the Swiss government’s data retention requirements for email. According to the regulations, the following must be retained for six months:
- Type of connections used for email, such as telephone, xDSL, Cable, or permanent line
- Login data, address information of the origin, name, address, and occupation of the user (as much as is known by the service provider)
- Duration of the connection from beginning to end
- Time of the transmission or reception of an email
- Header information
- IP addresses of the sending and receiving email application
Data Retention Requirements in the United States
There is no single data retention legislation in the United States; rather, requirements can be found across a collection of federal and state laws. At the federal level, the Federal Trade Commission Act includes requirements for data retention as it pertains to data privacy.
Other legislation in the United States with data retention requirements includes:
- Fair Labor Standards Act
- Bank Secrecy Act
- Health Insurance Portability and Accountability Act (HIPAA)
- Federal Information Security Management Act (FISMA)
The Electronic Communication Transactional Records Act regulates data preservation for service providers, directing them to retain all records for 90 days and be able to make them available “upon the request of a governmental entity.”
Data Retention Regulations
In addition to government requirements for data retention, there are a number of industry-specific regulations, including the following.
Federal Information Security Management Act (FISMA)
Contractors and federal agencies are subject to FISMA, which requires data retention for a minimum of three years.
National Energy Commission (NERC)
Bulk power system owners, operators, and users through approved regional delegation agreements must adhere to data retention requirements set forth by the NERC Rules of Procedure. These entities must retain data that proves their compliance with NERC Reliability Standards for a compliance period that ranges from three to six months.
The International Regulatory Framework for Banks (Basel III)
According to Basel III data retention requirements, banks must archive three to seven years of data history.
Sarbanes-Oxley Act (SOX)
Relevant auditing and review documents must be retained for seven years after the conclusion of an audit or review of the financial statements to comply with the SOX’s data retention requirements.
Health Insurance Portability and Accountability Act (HIPAA)
Health plans, health care clearinghouses, and health care providers who electronically transmit any health information are subject to HIPAA’s data retention requirements. They must retain health information for at least six years from the date it was created.
National Industrial Security Program Operating Manual (NISPOM)
All government contractors, who create or work with classified information, are subject to the data retention requirements set forth in the NISPOM. According to the NISPOM, classified material received or generated under a contract should be retained for two years unless otherwise directed.
Payment Card Industry Data Security Standard (PCI-DSS)
Any organization that accepts credit card payments is subject to PCI-DSS and its variable data retention requirements. Policy can be set by the organization, but it must submit required data for annual audits. Some guidelines are provided with regard to email archiving.
Creating a Data Retention Policy
A data retention policy, also known as a records retention policy, is a set of guidelines used by organizations that detail protocols for how data should be archived and how long data should be kept. Policies are developed in accordance with internal, legal, and regulatory requirements. Direction on how records should be formatted and what devices or systems should be used for storage is also included in data retention policies.
When creating a data retention policy, finding and classifying all data is of the utmost importance.
Then, the data should be reviewed to determine what requirements apply to what data. Finally, five phases of the data lifecycle should be considered.
When creating a data retention policy, all data produced and collected in fulfilling business activities should be considered. This includes:
- Email and other electronic communications
- “Office” files, such as documents, presentations, spreadsheets
- Databases
- Customer, partner, and supplier records
- Employee documentation
- Transactional data
- Contracts and billing information
- Invoices and receipts
- Accounting, banking, finance, and tax documents and data
Why Data Retention Policies are Important
A data retention policy is important, because it provides protocols for how to handle data that requires storage. Directives detailed in the policy ensure that all requirements are met as related to:
- Where data is stored
- What storage is used
- How long data is stored
- What happens when data is no longer needed
- Enabling compliance with internal, legal, and government rules
What is a Data Retention Period?
A data retention period is the amount of time that data must be stored according to internal and external requirements. Time periods vary by organization and industry, but generally range from three to ten years. Once its objective has been fulfilled, the data should be archived, anonymized, or destroyed.
Data Retention Policies and Compliance
In addition to an organization’s data retention policies, government and industry compliance requirements must be met. Policies must be comprehensive and include detailed explanations about:
- what data is being collected
- why it is being collected
- where it is stored
- how it is protected
- how the retention period is managed
Personal information should only be saved if it is necessary as it has inherent risk. If personal information is retained, special care should be taken to ensure compliance with the stringent rules that govern it, especially the EU’s General Data Protection Regulation (GDPR) and state privacy laws across the United States like the California Consumer Privacy Act (CCPA).
Data Retention Policy Benefits
The benefits of well-crafted data retention policies cannot be overstated. They include:
- Enabling compliance with internal, legal, and regulatory requirements
- Reducing storage costs by deleting documents that are no longer needed or moving files that are not accessed as frequently to a lower-level storage tier
- Organizing documents so they can quickly and easily be searched and accessed when necessary
- Setting dates for data disposal to ensure that unnecessary data is not saved
- Eliminating manual retention processes by automating data retention processes
- Expediting data backup and recovery with better-organized and streamlined data
Data Retention Best Practices
Any data retention program must start with data classification. Codified in a data retention policy, classification should include specific direction for managing the lifecycle of data, including:
- What type of data it is—public, internal-only, confidential, restricted.
- How long it should be retained.
- Where the data should be stored and with what level of security.
- Which organization representatives can move, modify, or delete it.
Additional best practices:
- Get all stakeholders on board—executives, legal, human resources, finance, IT/security, compliance and records managers, and end-users.
- Create and implement document lifecycle management with protocols related to:
- Data retention
- eDiscovery
- Compliance
- Data destruction or archiving
- Access controls and monitoring
- Start with the most sensitive information, then add the rest based on priority.
- Avoid overcomplicating protocols and processes.
- Retain only data that is required.
- Do not be afraid to delete obsolete data.
Data Retention Challenges
- Data categorization: People struggle to consistently, effectively, efficiently tag data, often relying on time-consuming manual processes.
- Data retention timestamping: Too often, data is moved to storage without information about when it should be destroyed or archived.
- Risks related to sensitive data: Incredible volumes of information are moved to storage and left there indefinitely. This data increases an organization’s threat landscape, exposing organizations to the untold risk associated with sensitive data.
- High costs: Without proper systems for categorizing data to determine what actually needs to be stored, the volume of data in storage skyrockets leading to growing expenses for space and management.
- Compliance: Data retention to meet compliance requirements is fraught with challenges, primarily because of the number of regulations, each with different rules for what data must be retained and for how long, especially as related to sensitive data.
Data Retention Success
Leveraging the right tools to govern and secure data is the key to success. Done correctly, data retention programs provide the processes and policies necessary to control risks and automate compliance while reducing the cost of data storage. By employing content lifecycle management solutions and other data retention best practices, organizations are able to gain control of their data for more effective and secure operations.
Egnyte has experts ready to answer your questions. For more than a decade, Egnyte has helped more than 16,000 customers with millions of customers worldwide.
